From patchwork Wed Feb 14 01:55:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitrii Ovchinnikov X-Patchwork-Id: 46251 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:c493:b0:19e:cdac:8cce with SMTP id eo19csp110178pzb; Tue, 13 Feb 2024 17:55:54 -0800 (PST) X-Forwarded-Encrypted: i=2; AJvYcCUxYvISX7Qdl+s9yfr9uAP3xy7VX4A1gSykkXMiZBp4I+C87QkUIDSthLW6nAXUdHxI7HOQob5SM2lEMjU+jZtahWZzROxi/p6qHA== X-Google-Smtp-Source: AGHT+IGhu8sXfBtCNbzagke5t86ysGKMWjhLqdGZqGRlx1GzbJha6MB3pzolKPrt9AlvDvOer44r X-Received: by 2002:aa7:c1d4:0:b0:560:6469:a190 with SMTP id d20-20020aa7c1d4000000b005606469a190mr847068edp.30.1707875753998; Tue, 13 Feb 2024 17:55:53 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1707875753; cv=none; d=google.com; s=arc-20160816; b=cgYnww/b2kjIn4Ot4jUtMTX9yWLBZoomprA/xCMab7NcIIW7gV3oObIh0pDoTqnCrC QhlDG0WSu/HgXQDhyOsJkc61HA8AhbB7VlmkSdNkketU+0EczmB2NONGSjea+Kn5hjUQ apJY/N0CfNYFZxYFz1qPWmep6Ga+lsuCs+n6ViiYBSIH+pGDfHUHn1yRbk4zS844WOcx oO5VO3FsjdusS7Zc/ymp4VKiYxTEGDLB+jAKb6z6QiQxzLLT8mwnBH2HxG7d0WG4wZ2l 1gaWYgIqzcJLmteufhzq5u3thV/IhzvB2cfhSKwTayI0MWs04Rhpq2lc0EKP8LQErmQt HT6g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=OJ4aYCW149NZ1OToVtvxE1CxAfmB7Q4XR37l9mwdjSA=; fh=u6hfVS/n0PXHA+Ge6nQ9v2mu9lbBt9hKc+k0wbYDFlQ=; b=FqNaX0E9AJXa3k+2aBgF21TcBakHLnsmGbPYdumMsBwglw371Tgdh/uFe7jPk3RPpP hE9Bt8TYPHw76dVCH1PHXFPHpZt/O3ucNmgzbVW/vb5hshLxoXeBeycaRL9grHhA/JAH UUy/0Bd5jcTHijf0YlzPzharITy+wSwfqUtTRCrP6Dju5jPUSihqxx9CSc8q4Ae0ynU7 8+ZlwrUy4yRhe6JuNz34JvHNaA01mNQmfH8HnxSkjeEbMtz38E+pajlyWXPaWlxH1ysx YzXVDHKMf46YZuRiEHCKepwWRf4VOYhOS1BYbB1m6f7KRtq6V+Pod6GSbwqgNmHJoA1O vsXA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=ab7vhaYN; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com X-Forwarded-Encrypted: i=1; AJvYcCUjvo/V8MKglOXRUR1cQUFo+LRXc2PVwL3gOcmYwDM+QAAZxWCY5Wb0TgNJwWvEgwO1cP7doxHCl9VC7ytTehudEVBIg06jF5aMv7pLr141Va9FrDzNlPkzphydL2825UItw7WI2ouxwGkvujZ2oI+PC4z7gMMncPbn23hQyzeYdcpYi+ldgReOtVfJsr06sbA1Fw48GI+/+Uh0+3CNtb1jTXuAJFSQ21/n9SKJ8habfKC63qzV0jrVpO5nXNpm/xldCZraz0knYiFdIVUcv9mfmcapvIGyjRxb5qq5REAFSxz4szrTd6KvPGr58dkj/p0nkAIig/UB/H0PXgmu3YJuVC9vcw6LromQOxE6cjm/AObdBvJ3FxNXIXoHVXHv+CpcaqoHbSGLGygK6we7mMqCOaokrZOckDCJkKqGEx0s+d4XZmTAZoZGcSYkUEu7Nw3HMHbyEIDS+aCLS4lNBzkT+XrR8TL8W4zqopTHa71l/9taMX8002Ta9pwv/A9YUtUm8/hq8+coZM6BOMLGbVRdC9XIIqg5jBZ0r2tuwQOvLtJUE0KAHJ9fnGSF1UjRW4T1RWybIdo6XbQ7HFVeM2+OwvzkOOtVL3/tm8UmDJq7IgnGcHkPyZ5G42K6sNwSjEl1XHiGEjndPO0soZsK5Y7Fo0hEbxH1xLCNym6tI1gkBCUN3AqY8/ZyLv2xSou35FBAeaDzYQSelKr1129PISji88uj2aSncsmzFNZfW28hF8kWjrCqltscq+lJ+mhDcL72VL59xepc+8PrWuiXq+oYSbDT2P6cq1UV0bBw3avIwx+NFuSwOHg7AvD4M/sGCLkOe792d9zCdxroM181zA1kZuvv4RNdNbQwaHWJ9VtfqFKkgY9q/RKEdoYI+IDGS1b5lAWV7eRiFyB5pn9d2u9NtMUPOaSqok/VQDaWB4Erpzzj5SVQbMO0feZRuGbNcIT3BL wgp2/qi4TQWtLTXxBbIBExgaQ6k9Ok91nnOxfxEpVqhwBNDtap1MGsEzVKC7hm5OnMW/mG2oGa/y2IMqd2epuX1p+kwqpXizGxLyV3Gg2fumhxnqXF2ckCwu4F86CF0sKQE4/0qY6A2iTkIeN2t4uJ+tKcY/Ko9xfsg3ofDEqVCnGb8cpwSHnKvbgcJlN1NtzBkg+XI+Kcmossy5j8OXcVTC/bVlpiP5yq4Y7vJWuXx15CSy3u+WwxL52HxMdsklTRMenu68GlL/EnCm/Vb+deAmwDtEo3xigy2NDGu9j1qPLMT2/SHUia5ltGzU+DaO6HbOm0wPrFWs3bXg2ke8VCj0u/5IO+Dg0vbhleQ2NpJCPxf9bdgG9PCVoSgyhIQzA15QQP31Y5psFquNCoJlh7VgnuZXB8jd1lpS7W7ftTvi84QGhwYIscxHsaoXMcE1xYuv0Kc6MIVnVdEayLmUCQo4jgSZsNyXqXbLsmd5J+Lpav9aHemWU2voJP1fOf0+h6o5eS3/mkMayegBtQwc0PIiRjhmfw7vkc0IVvFQKmcRJpTH/oCQOgWOuM6g4w86337Ko995y8ijbi2hpvs3SnejyytWkLV+iOgCJIqU58qkV+gzIOo82mLz2lUqk/tPawW20D Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id bd22-20020a056402207600b00560b6e6491csi4194594edb.623.2024.02.13.17.55.53; Tue, 13 Feb 2024 17:55:53 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=ab7vhaYN; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 84FF968D1C1; Wed, 14 Feb 2024 03:55:41 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f49.google.com (mail-ed1-f49.google.com [209.85.208.49]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 345DE68D185 for ; Wed, 14 Feb 2024 03:55:33 +0200 (EET) Received: by mail-ed1-f49.google.com with SMTP id 4fb4d7f45d1cf-562131bb958so811331a12.2 for ; Tue, 13 Feb 2024 17:55:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1707875732; x=1708480532; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QghbqhtXGzvl+yBqD2t3F9pcZyNcGp0335Vl5XsPHQQ=; b=ab7vhaYNd7JXdz8haNX0gCfmOCqm3xXjY6IybqWfiB9uX2Rij2PdTML9NSlCq5qtzX WBVjj6aoqNh9L6nDmVxAPO8trvsA6KXthA1UzVyos0IbmCR16Ia3ccsKDeaOTNZWGvtg tB9GtoW5I2VbKi2sJCJ+y+7NXKk5w0MPCidd0D6om3hPy8nYpLLZHMpKyJeFR6xzuAj1 wCwx6t9/GmNG7gMJTPcyJfkV5pq+q7NeQiwQiKhnYcNJAcLbZ5P2ciqP9uFnVkSTAol8 9m5CEWn4Rx+njSr5VapvBklyZ+T8Pg9MbTMv0pG6tkGwBplfmyhSa8ZK78Lbb//1tmm9 0dXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707875732; x=1708480532; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QghbqhtXGzvl+yBqD2t3F9pcZyNcGp0335Vl5XsPHQQ=; b=mURTCyPUjrLBsGF7EaQWDk9iQoB4m9/qu+F1A3x7os8vRaU9GC7K9Aprt9+lyIo7IM cFxte+fPIBRehMFyOgP0kCzkCHFPWIrTaTgVReK2Twy2sqZjecQZXR2Xu02DZx2aEQqr KzpnoWM8VVJOqVVLKRxQvgks3zQ1bVmPZjFb9JnhD88TYxD8orPTgS6BCAzrpgVIsHbB qwBhUQjRuEZU6iUQm/Lqmi6AJD9TegzBtXMTD7/QYN/yPUEefFtiuWWNKT4A/ODZvmLd pKybdjhqHw1TdPKFqJrbaVB6Rwh1rc7rBWp7CG8ZwMb16Kfu9DkSdUZzLcR6b+fJ/WBv Q7Lg== X-Gm-Message-State: AOJu0YweadVM7nzAuBmc1qKrEwp8vVcswFXx+7vvO1u/tG1eQnH/MV3p uASH0PFVwNaiVNer1cjDyCHYSSPzZWWmpWuN/qpWR009jJbXOsibDEZaeIqD X-Received: by 2002:a17:906:4715:b0:a3c:ef3f:a7c4 with SMTP id y21-20020a170906471500b00a3cef3fa7c4mr703261ejq.26.1707875731366; Tue, 13 Feb 2024 17:55:31 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCVr9TBMVFBSfZTlm9iz+J4K2703tVfYLMMAjlMQGdnGqjBKPIafr6RgzsrXbXPg7X89xb91pWpS7dlzqUkKLG6E513qU08zbqe0aV768Q== Received: from dovchinn.amd.com (cable-178-148-16-149.dynamic.sbb.rs. [178.148.16.149]) by smtp.gmail.com with ESMTPSA id vi16-20020a170907d41000b00a3ce31d3ffdsm1663230ejc.93.2024.02.13.17.55.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 17:55:30 -0800 (PST) From: Dmitrii Ovchinnikov To: ffmpeg-devel@ffmpeg.org Date: Wed, 14 Feb 2024 02:55:08 +0100 Message-Id: <20240214015515.1027-2-ovchinnikov.dmitrii@gmail.com> X-Mailer: git-send-email 2.38.1.windows.1 In-Reply-To: <20240214015515.1027-1-ovchinnikov.dmitrii@gmail.com> References: <20240214015515.1027-1-ovchinnikov.dmitrii@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/9] libavcodec: add amfdec. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Evgeny Pavlov , Dmitrii Ovchinnikov Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: fnquiuxlW1bX From: Evgeny Pavlov Added AMF based h264, hevc, av1 decoders. Co-authored-by: Dmitrii Ovchinnikov --- libavcodec/Makefile | 4 +- libavcodec/allcodecs.c | 3 + libavcodec/amfdec.c | 667 ++++++++++++++++++++++++++++++++++++++++ libavcodec/amfdec.h | 75 +++++ libavcodec/h264_slice.c | 3 + libavcodec/h264dec.c | 3 + libavcodec/hwconfig.h | 2 + 7 files changed, 755 insertions(+), 2 deletions(-) create mode 100644 libavcodec/amfdec.c create mode 100644 libavcodec/amfdec.h diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 470d7cb9b1..c2e4715f4b 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -69,7 +69,7 @@ include $(SRC_PATH)/libavcodec/x86/vvc/Makefile OBJS-$(CONFIG_AANDCTTABLES) += aandcttab.o OBJS-$(CONFIG_AC3DSP) += ac3dsp.o ac3.o ac3tab.o OBJS-$(CONFIG_ADTS_HEADER) += adts_header.o mpeg4audio_sample_rates.o -OBJS-$(CONFIG_AMF) += amfenc.o +OBJS-$(CONFIG_AMF) += amfenc.o amfdec.o OBJS-$(CONFIG_AUDIO_FRAME_QUEUE) += audio_frame_queue.o OBJS-$(CONFIG_ATSC_A53) += atsc_a53.o OBJS-$(CONFIG_AUDIODSP) += audiodsp.o @@ -1265,7 +1265,7 @@ SKIPHEADERS += %_tablegen.h \ vulkan_video_codec_av1std.h \ $(ARCH)/vpx_arith.h \ -SKIPHEADERS-$(CONFIG_AMF) += amfenc.h +SKIPHEADERS-$(CONFIG_AMF) += amfenc.h amfdec.h SKIPHEADERS-$(CONFIG_D3D11VA) += d3d11va.h dxva2_internal.h SKIPHEADERS-$(CONFIG_D3D12VA) += d3d12va_decode.h SKIPHEADERS-$(CONFIG_DXVA2) += dxva2.h dxva2_internal.h diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index ef8c3a6d7d..c344c70e00 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -851,10 +851,12 @@ extern const FFCodec ff_av1_nvenc_encoder; extern const FFCodec ff_av1_qsv_decoder; extern const FFCodec ff_av1_qsv_encoder; extern const FFCodec ff_av1_amf_encoder; +extern const FFCodec ff_av1_amf_decoder; extern const FFCodec ff_av1_vaapi_encoder; extern const FFCodec ff_libopenh264_encoder; extern const FFCodec ff_libopenh264_decoder; extern const FFCodec ff_h264_amf_encoder; +extern const FFCodec ff_h264_amf_decoder; extern const FFCodec ff_h264_cuvid_decoder; extern const FFCodec ff_h264_mf_encoder; extern const FFCodec ff_h264_nvenc_encoder; @@ -864,6 +866,7 @@ extern const FFCodec ff_h264_v4l2m2m_encoder; extern const FFCodec ff_h264_vaapi_encoder; extern const FFCodec ff_h264_videotoolbox_encoder; extern const FFCodec ff_hevc_amf_encoder; +extern const FFCodec ff_hevc_amf_decoder; extern const FFCodec ff_hevc_cuvid_decoder; extern const FFCodec ff_hevc_mediacodec_decoder; extern const FFCodec ff_hevc_mediacodec_encoder; diff --git a/libavcodec/amfdec.c b/libavcodec/amfdec.c new file mode 100644 index 0000000000..9d618ff442 --- /dev/null +++ b/libavcodec/amfdec.c @@ -0,0 +1,667 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include +#include +#include +#include "libavutil/hwcontext_amf.h" +#include "amfdec.h" +#include "codec_internal.h" +#include "hwconfig.h" +#include "libavutil/imgutils.h" +#include "libavutil/time.h" +#include "decode.h" +#include "libavutil/mastering_display_metadata.h" + +#if CONFIG_D3D11VA +#include "libavutil/hwcontext_d3d11va.h" +#endif +#if CONFIG_DXVA2 +#define COBJMACROS +#include "libavutil/hwcontext_dxva2.h" +#endif + +#ifdef _WIN32 +#include "compat/w32dlfcn.h" +#else +#include +#endif + +#define propNotFound 0 + +const enum AVPixelFormat amf_dec_pix_fmts[] = { + AV_PIX_FMT_YUV420P, + AV_PIX_FMT_NV12, + AV_PIX_FMT_BGRA, + AV_PIX_FMT_ARGB, + AV_PIX_FMT_RGBA, + AV_PIX_FMT_GRAY8, + AV_PIX_FMT_BGR0, + AV_PIX_FMT_YUYV422, + AV_PIX_FMT_P010, + AV_PIX_FMT_P012, + AV_PIX_FMT_YUV420P10, + AV_PIX_FMT_YUV420P12, + AV_PIX_FMT_YUV420P16, +#if CONFIG_D3D11VA + AV_PIX_FMT_D3D11, +#endif +#if CONFIG_DXVA2 + AV_PIX_FMT_DXVA2_VLD, +#endif + AV_PIX_FMT_AMF, + AV_PIX_FMT_NONE +}; + +static const AVCodecHWConfigInternal *const amf_hw_configs[] = { + &(const AVCodecHWConfigInternal) { + .public = { + .pix_fmt = AV_PIX_FMT_AMF, + .methods = AV_CODEC_HW_CONFIG_METHOD_HW_FRAMES_CTX | + AV_CODEC_HW_CONFIG_METHOD_HW_DEVICE_CTX, + .device_type = AV_HWDEVICE_TYPE_AMF, + }, + .hwaccel = NULL, + }, + NULL +}; + +static void amf_free_amfsurface(void *opaque, uint8_t *data) +{ + AMFSurface *surface = (AMFSurface*)(data); + surface->pVtbl->Release(surface); +} + +static int amf_init_decoder(AVCodecContext *avctx) +{ + enum AMF_SURFACE_FORMAT output_format = AMF_SURFACE_UNKNOWN; + AvAmfDecoderContext *ctx = avctx->priv_data; + AVAMFDeviceContextInternal * internal = (AVAMFDeviceContextInternal *)ctx->amf_device_ctx_internal->data; + const wchar_t *codec_id = NULL; + AMF_RESULT res; + AMFBuffer *buffer; + amf_int64 color_profile; + int pool_size = 35; + + if (avctx->pix_fmt == AV_PIX_FMT_AMF){ + if (avctx->hw_frames_ctx) { + AVHWFramesContext *hwframes_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; + output_format = av_amf_av_to_amf_format(hwframes_ctx->sw_format); + } else + output_format = av_amf_av_to_amf_format(avctx->sw_pix_fmt); + } else + output_format = av_amf_av_to_amf_format(avctx->pix_fmt); + + if (output_format == AMF_SURFACE_UNKNOWN) + output_format = AMF_SURFACE_NV12; + + ctx->drained = 0; + + switch (avctx->codec->id) { + case AV_CODEC_ID_H264: + codec_id = AMFVideoDecoderUVD_H264_AVC; + break; + case AV_CODEC_ID_HEVC: { + if (output_format == AMF_SURFACE_P010) + codec_id = AMFVideoDecoderHW_H265_MAIN10; + else + codec_id = AMFVideoDecoderHW_H265_HEVC; + } break; + case AV_CODEC_ID_AV1: + if (output_format == AMF_SURFACE_P012) + codec_id = AMFVideoDecoderHW_AV1_12BIT; + else + codec_id = AMFVideoDecoderHW_AV1; + break; + default: + break; + } + AMF_RETURN_IF_FALSE(ctx, codec_id != NULL, AVERROR(EINVAL), "Codec %d is not supported\n", avctx->codec->id); + + res = internal->factory->pVtbl->CreateComponent(internal->factory, internal->context, codec_id, &ctx->decoder); + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_ENCODER_NOT_FOUND, "CreateComponent(%ls) failed with error %d\n", codec_id, res); + + // Color Metadata + /// Color Range (Support for older Drivers) + if (avctx->color_range == AVCOL_RANGE_JPEG) { + AMF_ASSIGN_PROPERTY_BOOL(res, ctx->decoder, AMF_VIDEO_DECODER_FULL_RANGE_COLOR, 1); + } else if (avctx->color_range != AVCOL_RANGE_UNSPECIFIED) { + AMF_ASSIGN_PROPERTY_BOOL(res, ctx->decoder, AMF_VIDEO_DECODER_FULL_RANGE_COLOR, 0); + } + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN; + switch (avctx->colorspace) { + case AVCOL_SPC_SMPTE170M: + if (avctx->color_range == AVCOL_RANGE_JPEG) { + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_601; + } else { + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_601; + } + break; + case AVCOL_SPC_BT709: + if (avctx->color_range == AVCOL_RANGE_JPEG) { + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_709; + } else { + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_709; + } + break; + case AVCOL_SPC_BT2020_NCL: + case AVCOL_SPC_BT2020_CL: + if (avctx->color_range == AVCOL_RANGE_JPEG) { + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_2020; + } else { + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_2020; + } + break; + } + if (color_profile != AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_COLOR_PROFILE, color_profile); + if (avctx->color_trc != AVCOL_TRC_UNSPECIFIED) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_COLOR_TRANSFER_CHARACTERISTIC, (amf_int64)avctx->color_trc); + + if (avctx->color_primaries != AVCOL_PRI_UNSPECIFIED) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_COLOR_PRIMARIES, (amf_int64)avctx->color_primaries); + + if (ctx->timestamp_mode != -1) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_TIMESTAMP_MODE, ctx->timestamp_mode); + if (ctx->decoder_mode != -1) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_REORDER_MODE, ctx->decoder_mode); + if (ctx->dpb_size != -1) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_DPB_SIZE, ctx->dpb_size); + if (ctx->lowlatency != -1) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_LOW_LATENCY, ctx->lowlatency); + if (ctx->smart_access_video != -1) { + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_ENABLE_SMART_ACCESS_VIDEO, ctx->smart_access_video != 0); + if (res != AMF_OK) { + av_log(avctx, AV_LOG_ERROR, "The Smart Access Video is not supported by AMF decoder.\n"); + return AVERROR(EINVAL); + } else { + av_log(avctx, AV_LOG_INFO, "The Smart Access Video (%d) is set.\n", ctx->smart_access_video); + // Set low latency mode if Smart Access Video is enabled + if (ctx->smart_access_video != 0) { + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_LOW_LATENCY, true); + av_log(avctx, AV_LOG_INFO, "The Smart Access Video set low latency mode for decoder.\n"); + } + } + } + if (ctx->skip_transfer_sav != -1) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_SKIP_TRANSFER_SMART_ACCESS_VIDEO, ctx->skip_transfer_sav); + + if (avctx->extradata_size) { + res = internal->context->pVtbl->AllocBuffer(internal->context, AMF_MEMORY_HOST, avctx->extradata_size, &buffer); + if (res == AMF_OK) { + memcpy(buffer->pVtbl->GetNative(buffer), avctx->extradata, avctx->extradata_size); + AMF_ASSIGN_PROPERTY_INTERFACE(res,ctx->decoder, AMF_VIDEO_DECODER_EXTRADATA, buffer); + buffer->pVtbl->Release(buffer); + buffer = NULL; + } + } + if (ctx->surface_pool_size == -1) { + ctx->surface_pool_size = pool_size; + if (avctx->extra_hw_frames > 0) + ctx->surface_pool_size += avctx->extra_hw_frames; + if (avctx->active_thread_type & FF_THREAD_FRAME) + ctx->surface_pool_size += avctx->thread_count; + } + + //at the moment, there is such a restriction in AMF. + //when it is possible, I will remove this code + if (ctx->surface_pool_size > 100) + ctx->surface_pool_size = 100; + + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_SURFACE_POOL_SIZE, ctx->surface_pool_size); + res = ctx->decoder->pVtbl->Init(ctx->decoder, output_format, avctx->width, avctx->height); + return 0; +} + +static int amf_init_decoder_context(AVCodecContext *avctx) +{ + AvAmfDecoderContext *ctx = avctx->priv_data; + int ret; + + if (avctx->hw_frames_ctx) { + AVHWFramesContext *frames_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; + ret = av_amf_context_derive((AVAMFDeviceContextInternal *)ctx->amf_device_ctx_internal->data, frames_ctx->device_ctx, NULL, 0); + if (ret < 0) + return ret; + ctx->hw_frames_ctx = av_buffer_ref(avctx->hw_frames_ctx); + if (!ctx->hw_frames_ctx) + return AVERROR(ENOMEM); + } + else if (avctx->hw_device_ctx) { + AVHWDeviceContext *device_ctx = (AVHWDeviceContext*)avctx->hw_device_ctx->data; + ret = av_amf_context_derive((AVAMFDeviceContextInternal *)ctx->amf_device_ctx_internal->data, device_ctx, NULL, 0); + if (ret < 0) + return ret; + ctx->hw_device_ctx = av_buffer_ref(avctx->hw_device_ctx); + if (!ctx->hw_device_ctx) + return AVERROR(ENOMEM); + } else { + ret = av_amf_context_init((AVAMFDeviceContextInternal *)ctx->amf_device_ctx_internal->data, avctx); + if (ret != 0) { + return ret; + } + } + + return ret; +} + +static int amf_decode_close(AVCodecContext *avctx) +{ + AvAmfDecoderContext *ctx = avctx->priv_data; + + if (ctx->decoder) { + ctx->decoder->pVtbl->Terminate(ctx->decoder); + ctx->decoder->pVtbl->Release(ctx->decoder); + ctx->decoder = NULL; + } + + av_buffer_unref(&ctx->amf_device_ctx_internal); + av_buffer_unref(&ctx->hw_device_ctx); + av_buffer_unref(&ctx->hw_frames_ctx); + av_buffer_unref(&ctx->amf_device_ctx); + + return 0; + +} + +static int amf_decode_init(AVCodecContext *avctx) +{ + AvAmfDecoderContext *ctx = avctx->priv_data; + int ret; + enum AVPixelFormat pix_fmts[3] = { + AV_PIX_FMT_AMF, + avctx->pix_fmt, + AV_PIX_FMT_NONE }; + + ret = ff_get_format(avctx, pix_fmts); + if (ret < 0) { + avctx->pix_fmt = AV_PIX_FMT_NONE; + } + + if (avctx->hw_frames_ctx){ + AVHWFramesContext *frames_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; + if (frames_ctx->device_ctx->type == AV_HWDEVICE_TYPE_AMF) { + AVAMFDeviceContext * amf_ctx = frames_ctx->device_ctx->hwctx; + ctx->amf_device_ctx_internal = av_buffer_ref(amf_ctx->internal); + } + } + else if (avctx->hw_device_ctx && !avctx->hw_frames_ctx && ret == AV_PIX_FMT_AMF) { + AVHWDeviceContext *hwdev_ctx; + AVHWFramesContext *hwframes_ctx; + hwdev_ctx = (AVHWDeviceContext*)avctx->hw_device_ctx->data; + if (hwdev_ctx->type == AV_HWDEVICE_TYPE_AMF) + { + AVAMFDeviceContext * amf_ctx = hwdev_ctx->hwctx; + ctx->amf_device_ctx_internal = av_buffer_ref(amf_ctx->internal); + } + + avctx->hw_frames_ctx = av_hwframe_ctx_alloc(avctx->hw_device_ctx); + + if (!avctx->hw_frames_ctx) { + av_log(avctx, AV_LOG_ERROR, "av_hwframe_ctx_alloc failed\n"); + return AVERROR(ENOMEM); + } + + hwframes_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; + hwframes_ctx->width = FFALIGN(avctx->coded_width, 32); + hwframes_ctx->height = FFALIGN(avctx->coded_height, 32); + hwframes_ctx->format = AV_PIX_FMT_AMF; + hwframes_ctx->sw_format = avctx->sw_pix_fmt == AV_PIX_FMT_YUV420P10 ? AV_PIX_FMT_P010 : AV_PIX_FMT_NV12; + hwframes_ctx->initial_pool_size = ctx->surface_pool_size + 8; + avctx->pix_fmt = AV_PIX_FMT_AMF; + + ret = av_hwframe_ctx_init(avctx->hw_frames_ctx); + + if (ret < 0) { + av_log(NULL, AV_LOG_ERROR, "Error initializing a AMF frame pool\n"); + av_buffer_unref(&avctx->hw_frames_ctx); + return ret; + } + } else { + AVAMFDeviceContextInternal *wrapped = av_mallocz(sizeof(*wrapped)); + ctx->amf_device_ctx_internal = av_buffer_create((uint8_t *)wrapped, sizeof(*wrapped), + av_amf_context_internal_free, NULL, 0); + if ((ret = av_amf_context_internal_create((AVAMFDeviceContextInternal *)ctx->amf_device_ctx_internal->data, avctx, "", NULL, 0)) != 0) { + amf_decode_close(avctx); + return ret; + } + if ((ret = amf_init_decoder_context(avctx)) != 0) { + return ret; + } + } + if ((ret = amf_init_decoder(avctx)) == 0) { + return 0; + } + amf_decode_close(avctx); + return ret; +} + +static AMF_RESULT amf_get_property_buffer(AMFData *object, const wchar_t *name, AMFBuffer **val) +{ + AMF_RESULT res; + AMFVariantStruct var; + res = AMFVariantInit(&var); + if (res == AMF_OK) { + res = object->pVtbl->GetProperty(object, name, &var); + if (res == AMF_OK) { + if (var.type == AMF_VARIANT_INTERFACE) { + AMFGuid guid_AMFBuffer = IID_AMFBuffer(); + AMFInterface *amf_interface = AMFVariantInterface(&var); + res = amf_interface->pVtbl->QueryInterface(amf_interface, &guid_AMFBuffer, (void**)val); + } else { + res = AMF_INVALID_DATA_TYPE; + } + } + AMFVariantClear(&var); + } + return res; +} + +static int amf_amfsurface_to_avframe(AVCodecContext *avctx, AMFSurface* surface, AVFrame *frame) +{ + AMFVariantStruct var = {0}; + AMFPlane *plane; + int i; + int ret; + + if (avctx->hw_frames_ctx) { + AVHWFramesContext *hwframes_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; + if (hwframes_ctx->format == AV_PIX_FMT_AMF) { + ret = ff_get_buffer(avctx, frame, AV_GET_BUFFER_FLAG_REF); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Get hw frame failed.\n"); + return ret; + } + //we need to release surface with frame to return it to decoder + frame->buf[1] = av_buffer_create((uint8_t *)surface, sizeof(AMFSurface), + amf_free_amfsurface, (void*)avctx, + AV_BUFFER_FLAG_READONLY); + frame->data[3] = (uint8_t *)surface; + } else { + av_log(avctx, AV_LOG_ERROR, "Unknown format for hwframes_ctx\n"); + return AVERROR(ENOMEM); + } + } else { + ret = surface->pVtbl->Convert(surface, AMF_MEMORY_HOST); + AMF_RETURN_IF_FALSE(avctx, ret == AMF_OK, AVERROR_UNKNOWN, "Convert(amf::AMF_MEMORY_HOST) failed with error %d\n", ret); + + for (i = 0; i < surface->pVtbl->GetPlanesCount(surface); i++) { + plane = surface->pVtbl->GetPlaneAt(surface, i); + frame->data[i] = plane->pVtbl->GetNative(plane); + frame->linesize[i] = plane->pVtbl->GetHPitch(plane); + } + + frame->buf[0] = av_buffer_create((uint8_t *)surface, sizeof(AMFSurface), + amf_free_amfsurface, (void*)avctx, + AV_BUFFER_FLAG_READONLY); + frame->format = av_amf_to_av_format(surface->pVtbl->GetFormat(surface)); + } + + frame->width = avctx->width; + frame->height = avctx->height; + + frame->pts = surface->pVtbl->GetPts(surface); + + surface->pVtbl->GetProperty(surface, L"FFMPEG:dts", &var); + frame->pkt_dts = var.int64Value; + + frame->duration = surface->pVtbl->GetDuration(surface); + if (frame->duration < 0) + frame->duration = 0; + + frame->color_range = avctx->color_range; + frame->colorspace = avctx->colorspace; + frame->color_trc = avctx->color_trc; + frame->color_primaries = avctx->color_primaries; + + if (frame->color_trc == AVCOL_TRC_SMPTE2084) { + AMFBuffer * hdrmeta_buffer = NULL; + ret = amf_get_property_buffer((AMFData *)surface, AMF_VIDEO_DECODER_HDR_METADATA, &hdrmeta_buffer); + if (hdrmeta_buffer != NULL) { + AMFHDRMetadata * hdrmeta = (AMFHDRMetadata*)hdrmeta_buffer->pVtbl->GetNative(hdrmeta_buffer); + if (ret != AMF_OK) + return ret; + if (hdrmeta != NULL) { + AVMasteringDisplayMetadata *mastering = av_mastering_display_metadata_create_side_data(frame); + const int chroma_den = 50000; + const int luma_den = 10000; + + if (!mastering) + return AVERROR(ENOMEM); + + mastering->display_primaries[0][0] = av_make_q(hdrmeta->redPrimary[0], chroma_den); + mastering->display_primaries[0][1] = av_make_q(hdrmeta->redPrimary[1], chroma_den); + + mastering->display_primaries[1][0] = av_make_q(hdrmeta->greenPrimary[0], chroma_den); + mastering->display_primaries[1][1] = av_make_q(hdrmeta->greenPrimary[1], chroma_den); + + mastering->display_primaries[2][0] = av_make_q(hdrmeta->bluePrimary[0], chroma_den); + mastering->display_primaries[2][1] = av_make_q(hdrmeta->bluePrimary[1], chroma_den); + + mastering->white_point[0] = av_make_q(hdrmeta->whitePoint[0], chroma_den); + mastering->white_point[1] = av_make_q(hdrmeta->whitePoint[1], chroma_den); + + mastering->max_luminance = av_make_q(hdrmeta->maxMasteringLuminance, luma_den); + mastering->min_luminance = av_make_q(hdrmeta->maxMasteringLuminance, luma_den); + + mastering->has_luminance = 1; + mastering->has_primaries = 1; + if (hdrmeta->maxContentLightLevel) { + AVContentLightMetadata *light = av_content_light_metadata_create_side_data(frame); + + if (!light) + return AVERROR(ENOMEM); + + light->MaxCLL = hdrmeta->maxContentLightLevel; + light->MaxFALL = hdrmeta->maxFrameAverageLightLevel; + } + } + } + } + return 0; +} + +static AMF_RESULT amf_receive_frame(AVCodecContext *avctx, AVFrame *frame) +{ + AvAmfDecoderContext *ctx = avctx->priv_data; + AMF_RESULT ret = AMF_OK; + AMFSurface *surface = NULL; + AMFData *data_out = NULL; + + ret = ctx->decoder->pVtbl->QueryOutput(ctx->decoder, &data_out); + if (ret != AMF_OK && ret != AMF_REPEAT) { + return ret; + } + if (data_out == NULL) { + return AMF_FAIL; + } + + if (data_out) { + AMFGuid guid = IID_AMFSurface(); + data_out->pVtbl->QueryInterface(data_out, &guid, (void**)&surface); // query for buffer interface + data_out->pVtbl->Release(data_out); + data_out = NULL; + } + + ret = amf_amfsurface_to_avframe(avctx, surface, frame); + AMF_GOTO_FAIL_IF_FALSE(avctx, ret >= 0, AMF_FAIL, "Failed to convert AMFSurface to AVFrame = %d\n", ret); + return AMF_OK; +fail: + + if (surface) { + surface->pVtbl->Release(surface); + surface = NULL; + } + return ret; +} + +static AMF_RESULT amf_update_buffer_properties(AVCodecContext *avctx, AMFBuffer* buffer, const AVPacket* pkt) +{ + AvAmfDecoderContext *ctx = avctx->priv_data; + AVAMFDeviceContextInternal * internal = (AVAMFDeviceContextInternal * )ctx->amf_device_ctx_internal->data; + AMFContext *ctxt = internal->context; + + AMF_RESULT res; + + AMF_RETURN_IF_FALSE(ctxt, buffer != NULL, AMF_INVALID_ARG, "update_buffer_properties() - buffer not passed in"); + AMF_RETURN_IF_FALSE(ctxt, pkt != NULL, AMF_INVALID_ARG, "update_buffer_properties() - packet not passed in"); + buffer->pVtbl->SetPts(buffer, pkt->pts); + buffer->pVtbl->SetDuration(buffer, pkt->duration); + AMF_ASSIGN_PROPERTY_INT64(res, buffer, L"FFMPEG:dts", pkt->dts); + if (res != AMF_OK) + av_log(avctx, AV_LOG_VERBOSE, "Failed to assign dts value."); + return AMF_OK; +} + +static AMF_RESULT amf_buffer_from_packet(AVCodecContext *avctx, const AVPacket* pkt, AMFBuffer** buffer) +{ + AvAmfDecoderContext *ctx = avctx->priv_data; + AVAMFDeviceContextInternal * internal = (AVAMFDeviceContextInternal * )ctx->amf_device_ctx_internal->data; + AMFContext *ctxt = internal->context; + void *mem; + AMF_RESULT err; + AMFBuffer *buf = NULL; + + AMF_RETURN_IF_FALSE(ctxt, pkt != NULL, AMF_INVALID_ARG, "amf_buffer_from_packet() - packet not passed in"); + AMF_RETURN_IF_FALSE(ctxt, buffer != NULL, AMF_INVALID_ARG, "amf_buffer_from_packet() - buffer pointer not passed in"); + + err = ctxt->pVtbl->AllocBuffer(ctxt, AMF_MEMORY_HOST, pkt->size + AV_INPUT_BUFFER_PADDING_SIZE, buffer); + AMF_RETURN_IF_FALSE(ctxt, err == AMF_OK, err, "amf_buffer_from_packet() - failed"); + buf = *buffer; + err = buf->pVtbl->SetSize(buf, pkt->size); + AMF_RETURN_IF_FALSE(ctxt, err == AMF_OK, err, "amf_buffer_from_packet() - SetSize failed"); + // get the memory location and check the buffer was indeed allocated + mem = buf->pVtbl->GetNative(buf); + AMF_RETURN_IF_FALSE(ctxt, mem != NULL, AMF_INVALID_POINTER, "amf_buffer_from_packet() - GetNative failed"); + + // copy the packet memory and clear data padding + memcpy(mem, pkt->data, pkt->size); + memset((amf_int8*)(mem)+pkt->size, 0, AV_INPUT_BUFFER_PADDING_SIZE); + + return amf_update_buffer_properties(avctx, buf, pkt); +} + +static int amf_decode_frame(AVCodecContext *avctx, AVFrame *data, + int *got_frame, AVPacket *avpkt) +{ + AvAmfDecoderContext *ctx = avctx->priv_data; + AVFrame *frame = data; + AMFBuffer *buf; + AMF_RESULT res; + + if (!ctx->decoder) + return AVERROR(EINVAL); + + if (!avpkt->size && ctx->drained == 0) { + ctx->decoder->pVtbl->Drain(ctx->decoder); + ctx->drained = 1; + } + if (avpkt->size > 0) { + res = amf_buffer_from_packet(avctx, avpkt, &buf); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, 0, "Cannot convert AVPacket to AMFbuffer"); + res = ctx->decoder->pVtbl->SubmitInput(ctx->decoder, (AMFData*) buf); + // FIXME: check other return values + if (res == AMF_OK || res == AMF_NEED_MORE_INPUT) + { + *got_frame = 0; + } else { + av_log(avctx, AV_LOG_VERBOSE, "SubmitInput() returned %d\n", res); + } + + buf->pVtbl->Release(buf); + buf = NULL; + if (res == AMF_INPUT_FULL) { // handle full queue + *got_frame = 0; + } + } + + res = amf_receive_frame(avctx, frame); + if (res == AMF_OK) { + AMF_RETURN_IF_FALSE(avctx, !*got_frame, avpkt->size, "frame already got"); + *got_frame = 1; + } else if (res != AMF_EOF && res != AMF_FAIL) { + av_log(avctx, AV_LOG_ERROR, "Unkown result from QueryOutput %d\n", res); + } + + return avpkt->size; +} + +static void amf_decode_flush(AVCodecContext *avctx) +{ + AvAmfDecoderContext *ctx = avctx->priv_data; + ctx->decoder->pVtbl->Flush(ctx->decoder); +} + +#define OFFSET(x) offsetof(AvAmfDecoderContext, x) +#define VD AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_DECODING_PARAM + +static const AVOption options[] = { + // Decoder mode + { "decoder_mode", "Decoder mode", OFFSET(decoder_mode), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, AMF_VIDEO_DECODER_MODE_LOW_LATENCY, VD, "decoder_mode" }, + { "regular", "DPB delay is based on number of reference frames + 1", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_DECODER_MODE_REGULAR }, 0, 0, VD, "decoder_mode" }, + { "compliant", "DPB delay is based on profile - up to 16", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_DECODER_MODE_COMPLIANT }, 0, 0, VD, "decoder_mode" }, + { "low_latency", "DPB delay is 0", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_DECODER_MODE_LOW_LATENCY }, 0, 0, VD, "decoder_mode" }, + + // Timestamp mode + { "timestamp_mode", "Timestamp mode", OFFSET(timestamp_mode), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, AMF_TS_DECODE, VD, "timestamp_mode" }, + { "presentation", "Preserve timestamps from input to output", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_TS_PRESENTATION }, 0, 0, VD, "timestamp_mode" }, + { "sort", "Resort PTS list", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_TS_SORT }, 0, 0, VD, "timestamp_mode" }, + { "decode", "Decode order", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_TS_DECODE }, 0, 0, VD, "timestamp_mode" }, + + // Reference frame management + { "surface_pool_size", "Number of surfaces in the decode pool", OFFSET(surface_pool_size), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, INT_MAX, VD, NULL }, + { "dpb_size", "Minimum number of surfaces for reordering", OFFSET(dpb_size), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 32, VD, NULL }, + + { "lowlatency", "Low latency", OFFSET(lowlatency), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 1, VD, NULL }, + { "smart_access_video", "Smart Access Video", OFFSET(smart_access_video), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 1, VD, NULL }, + { "skip_transfer_sav", "Skip transfer on another GPU when SAV enabled", OFFSET(skip_transfer_sav), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 1, VD, NULL }, + + { NULL } +}; + +static const AVClass amf_decode_class = { + .class_name = "amf", + .item_name = av_default_item_name, + .option = options, + .version = LIBAVUTIL_VERSION_INT, +}; + + +#define DEFINE_AMF_DECODER(x, X, bsf_name) \ +const FFCodec ff_##x##_amf_decoder = { \ + .p.name = #x "_amf", \ + CODEC_LONG_NAME(#X " AMD AMF video decoder"), \ + .priv_data_size = sizeof(AvAmfDecoderContext), \ + .p.type = AVMEDIA_TYPE_VIDEO, \ + .p.id = AV_CODEC_ID_##X, \ + .init = amf_decode_init, \ + FF_CODEC_DECODE_CB(amf_decode_frame), \ + .flush = amf_decode_flush, \ + .close = amf_decode_close, \ + .bsfs = bsf_name, \ + .p.capabilities = AV_CODEC_CAP_HARDWARE | AV_CODEC_CAP_DELAY | AV_CODEC_CAP_AVOID_PROBING, \ + .p.priv_class = &amf_decode_class, \ + .p.pix_fmts = amf_dec_pix_fmts, \ + .hw_configs = amf_hw_configs, \ + .p.wrapper_name = "amf", \ + .caps_internal = FF_CODEC_CAP_NOT_INIT_THREADSAFE, \ +}; \ + +DEFINE_AMF_DECODER(h264, H264, "h264_mp4toannexb") +DEFINE_AMF_DECODER(hevc, HEVC, NULL) +DEFINE_AMF_DECODER(av1, AV1, NULL) diff --git a/libavcodec/amfdec.h b/libavcodec/amfdec.h new file mode 100644 index 0000000000..4c45d2426b --- /dev/null +++ b/libavcodec/amfdec.h @@ -0,0 +1,75 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_AMFDEC_H +#define AVCODEC_AMFDEC_H + +#include +#include +#include +#include +#include +#include + +#include "avcodec.h" +#include "libavformat/avformat.h" +#include "libavutil/fifo.h" +#include "libavutil/frame.h" +#include "libavutil/opt.h" + +/** +* AMF decoder context +*/ + +typedef struct AvAmfDecoderContext { + AVClass *avclass; + + AVBufferRef *amf_device_ctx_internal; + AVBufferRef *amf_device_ctx; + + //decoder + AMFComponent *decoder; ///< AMF decoder object + AMF_SURFACE_FORMAT format; ///< AMF surface format + + AVBufferRef *hw_device_ctx; ///< pointer to HW accelerator (decoder) + AVBufferRef *hw_frames_ctx; ///< pointer to HW accelerator (frame allocator) + + AVBufferRef *hw_device_ref; + AVBufferRef *hw_frames_ref; + + // shift dts back by max_b_frames in timing + AVFifoBuffer *timestamp_list; + int64_t dts_delay; + + amf_uint64 version; ///< version of AMF runtime + // common encoder option options + + int log_to_dbg; + // Static options, have to be set before Init() call + int decoder_mode; + int timestamp_mode; + int surface_pool_size; + int dpb_size; + int lowlatency; + int smart_access_video; + int skip_transfer_sav; + int drained; + +} AvAmfDecoderContext; + +#endif // AVCODEC_AMFDEC_H \ No newline at end of file diff --git a/libavcodec/h264_slice.c b/libavcodec/h264_slice.c index 8464a0b34c..d11821194f 100644 --- a/libavcodec/h264_slice.c +++ b/libavcodec/h264_slice.c @@ -864,6 +864,9 @@ static enum AVPixelFormat get_pixel_format(H264Context *h, int force_callback) #if CONFIG_H264_NVDEC_HWACCEL *fmt++ = AV_PIX_FMT_CUDA; #endif +#if CONFIG_H264_AMFDEC_HWACCEL + *fmt++ = AV_PIX_FMT_AMF; +#endif #if CONFIG_H264_VIDEOTOOLBOX_HWACCEL if (h->avctx->colorspace != AVCOL_SPC_RGB) *fmt++ = AV_PIX_FMT_VIDEOTOOLBOX; diff --git a/libavcodec/h264dec.c b/libavcodec/h264dec.c index 9f5893c512..7a2c9eecef 100644 --- a/libavcodec/h264dec.c +++ b/libavcodec/h264dec.c @@ -1137,6 +1137,9 @@ const FFCodec ff_h264_decoder = { #if CONFIG_H264_NVDEC_HWACCEL HWACCEL_NVDEC(h264), #endif +#if CONFIG_H264_AMFDEC_HWACCEL + HWACCEL_AMFDEC(h264), +#endif #if CONFIG_H264_VAAPI_HWACCEL HWACCEL_VAAPI(h264), #endif diff --git a/libavcodec/hwconfig.h b/libavcodec/hwconfig.h index ee29ca631d..556f724895 100644 --- a/libavcodec/hwconfig.h +++ b/libavcodec/hwconfig.h @@ -67,6 +67,8 @@ void ff_hwaccel_uninit(AVCodecContext *avctx); HW_CONFIG_HWACCEL(1, 1, 0, D3D11, D3D11VA, ff_ ## codec ## _d3d11va2_hwaccel) #define HWACCEL_NVDEC(codec) \ HW_CONFIG_HWACCEL(1, 1, 0, CUDA, CUDA, ff_ ## codec ## _nvdec_hwaccel) +#define HWACCEL_AMFDEC(codec) \ + HW_CONFIG_HWACCEL(1, 1, 0, AMF, AMF, ff_ ## codec ## _amfdec_hwaccel) #define HWACCEL_VAAPI(codec) \ HW_CONFIG_HWACCEL(1, 1, 1, VAAPI, VAAPI, ff_ ## codec ## _vaapi_hwaccel) #define HWACCEL_VDPAU(codec) \