From patchwork Mon Oct 21 23:16:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitrii Ovchinnikov X-Patchwork-Id: 52441 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:143:b0:48e:c0f8:d0de with SMTP id h3csp3099763vqi; Mon, 21 Oct 2024 21:04:09 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCVtQe1exEoYRC1RRpl3ijSvoLDYC4UIEnMuIsYJ5XyXr0COGh7juDUyPOOk6r6zqNzfH2idOdPR3qFOfJjQgeu6@gmail.com X-Google-Smtp-Source: AGHT+IEnWdaEW6HZUbAfd619xoDtqIYSZ4+Tc0xX/2DvvqSdLJhQ10+7JIlju5shf58A3j/EBlg3 X-Received: by 2002:a05:6402:254c:b0:5ca:e44:6945 with SMTP id 4fb4d7f45d1cf-5ca0e446a42mr11619120a12.15.1729569849112; Mon, 21 Oct 2024 21:04:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1729569849; cv=none; d=google.com; s=arc-20240605; b=L4iIYGnxPSgVYDth8v4djl5s3acXFvQLT9y3OrFpZIznXskQ6+hCKRIFLUhlYNMjlr tdGPzFvk+Y8SdDmaJgDfx8cFqq2M/0AHGOf5olrLZJ4tS5d8TKz0gl7eL+q8MBD7ENVd hOyqko3JASWZGOy/zxNIjXD8dZSXEWoqaWlr2rKzHcC41kTrVsqvniJ3cT5wUZDNvDnj TPTPCIH7VYTDsyob2coyHE8s1VthuzjVcrrmBqziQ0FyAI+sqWuCBur0OVCjizFe7H0Y g+ycoX8vLjKBaQKWVxcBDo5N+zMewLQ1Z0I1NTFqeIcBCFIfQiybhd9Nmc1yRzNAZFJJ qpkA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=a3vcIom/WK3Vp1Vo8kSCxjeyOyrJ0CTLC8r2KgccRc0=; fh=GFMLWjWsOMgBmoPKQLjFbTnXINkN4pkSjsZqwDcYLuI=; b=Y1MHOTgTgMa1ZvpSDhFdbfPQEfSgT483stwo/sODAFv9T2uki60lCDBLmxxLPEjqKK qPUUILgnfSYrwCl6XHTaRD/dczTJkeXCOi29ljO9185aY6K3pw3K/CBdm73pA00Tvv0z sh4g5mDkWGDwZIqvZ9p5slgdc+0pxYdXyJMi0K9GLs/j/qOAmjR0ucPwxCQ0Y8YfNu8q xaG1hnMk+GB82NGU4oo4fbffc+tigNgJMjomkgor5Zh1sLBdugmURmZL/uNt/1YM3lE+ dVOx/4lBXAcEa1UQe3IsXvnMR6PyZHO4MgRXKmVsdxMbvnG+FC/U0X3mZIb57ywTAejm 7cfw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=UKVCX2YI; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 4fb4d7f45d1cf-5cb66c7db34si3386768a12.460.2024.10.21.21.04.08; Mon, 21 Oct 2024 21:04:09 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=UKVCX2YI; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 19DA668DE56; Tue, 22 Oct 2024 02:17:13 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f52.google.com (mail-wr1-f52.google.com [209.85.221.52]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E0A1A68DE44 for ; Tue, 22 Oct 2024 02:17:05 +0300 (EEST) Received: by mail-wr1-f52.google.com with SMTP id ffacd0b85a97d-37d4ac91d97so4879114f8f.2 for ; Mon, 21 Oct 2024 16:17:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729552625; x=1730157425; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JQYBkXKxb8IaDya3yDyAKBypcqwKGQsuomC1g0wZdXM=; b=UKVCX2YIGTZSl3wjdrv3q63FmIRWO4qO6zz8pbO0s586rmQE97Y5MDfSyxfAyegabk +Hm/yCcoy6ZRO7PLymleWszDyq9i+iPPxlz6zinR4qn+V60jgm/rzg0SVO4D572I/Gxl O6o5orZKmergEKr7DVeJfWhTncRiLo1PIBN8jOwYvl2HcTMhMcDpnIr1MjSJxQdbqxKt RUp2sKqHpUqofTnaJp5cPNqEfUGwhHm6R7yTg3aGWiZixKib2CAmAGLXMzatfjxgqjDM Jlag0zN9O1fYLCll3qEM/x+5EqEtjIAU44nPMwl7YmbgYbkDNTiHHwqtpCjPnnocabqD l2DA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729552625; x=1730157425; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JQYBkXKxb8IaDya3yDyAKBypcqwKGQsuomC1g0wZdXM=; b=BZ8or3Kl+arqSw/ixyduDJEP9l5cStRERLLZrCb7kjXMR0s/7h6a2TWwbdeDUMLXmg nuFIRem9aTE8v3MgAyR2C7x05/h8/ms7JROESlw4FqtHL3mH339o32w34uI3GJ5WOpA5 7F4vMt/MaMMSN+Ig2Z1POPqRissP+LEhBPCMWemuxR7UZ1PDpQ3Z5Z7DVfUnlb3soRsu y2jZo7bsgawF/BaDaI5TpCArwDpweqS84e5AzFQVM4iE93qxICaTRyCdpxJO+6cwKHHc dKVm/RRrFQqjluHxYjhhXQC46+RORwjWSI1o7jBchCDXHjDpSvtA/zL9nsjgfff+xhyo EEqw== X-Gm-Message-State: AOJu0YxBuQo5uPJCYNI/CHNFSXNllZrynLZBPH5YbOL/UO1CzxZyExM0 Ydbz7LtlVPnMWUlU3Rg8g3qAZhoy9tosijui7dAyF/R8fWONDDzx00Cxsg== X-Received: by 2002:adf:8b96:0:b0:37d:4376:6e1d with SMTP id ffacd0b85a97d-37eab755c62mr11331131f8f.41.1729552624201; Mon, 21 Oct 2024 16:17:04 -0700 (PDT) Received: from dovchinn.amd.com (cable-178-148-16-149.dynamic.sbb.rs. [178.148.16.149]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-37ee0a57e0fsm5293937f8f.47.2024.10.21.16.17.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Oct 2024 16:17:02 -0700 (PDT) From: Dmitrii Ovchinnikov To: ffmpeg-devel@ffmpeg.org Date: Tue, 22 Oct 2024 01:16:33 +0200 Message-Id: <20241021231636.1199-2-ovchinnikov.dmitrii@gmail.com> X-Mailer: git-send-email 2.38.1.windows.1 In-Reply-To: <20241021231636.1199-1-ovchinnikov.dmitrii@gmail.com> References: <20241021231636.1199-1-ovchinnikov.dmitrii@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/5] avcodec: add amfdec. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Evgeny Pavlov , Dmitrii Ovchinnikov Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 9XDPYnbO/3It From: Evgeny Pavlov Added AMF based h264, hevc, av1 decoders. Co-authored-by: Dmitrii Ovchinnikov v2: added encoder reinitialisation v3: use AMF_SURFACE_UNKNOWN to int decoder(ctx->output_format before) --- configure | 3 + libavcodec/Makefile | 7 +- libavcodec/allcodecs.c | 3 + libavcodec/amfdec.c | 624 +++++++++++++++++++++++++++++++++++++++++ libavcodec/amfdec.h | 61 ++++ 5 files changed, 696 insertions(+), 2 deletions(-) create mode 100644 libavcodec/amfdec.c create mode 100644 libavcodec/amfdec.h diff --git a/configure b/configure index 9f508a2527..3c92fb6a04 100755 --- a/configure +++ b/configure @@ -3343,6 +3343,7 @@ amrnb_mediacodec_decoder_select="amr_parser" amrwb_mediacodec_decoder_deps="mediacodec" amrwb_mediacodec_decoder_select="amr_parser" av1_amf_encoder_deps="amf" +av1_amf_decoder_deps="amf" av1_cuvid_decoder_deps="cuvid CUVIDAV1PICPARAMS" av1_mediacodec_decoder_deps="mediacodec" av1_mediacodec_encoder_deps="mediacodec" @@ -3358,6 +3359,7 @@ av1_vaapi_encoder_select="cbs_av1 vaapi_encode" h263_v4l2m2m_decoder_deps="v4l2_m2m h263_v4l2_m2m" h263_v4l2m2m_encoder_deps="v4l2_m2m h263_v4l2_m2m" h264_amf_encoder_deps="amf" +h264_amf_decoder_deps="amf" h264_cuvid_decoder_deps="cuvid" h264_cuvid_decoder_select="h264_mp4toannexb_bsf" h264_mediacodec_decoder_deps="mediacodec" @@ -3379,6 +3381,7 @@ h264_v4l2m2m_decoder_deps="v4l2_m2m h264_v4l2_m2m" h264_v4l2m2m_decoder_select="h264_mp4toannexb_bsf" h264_v4l2m2m_encoder_deps="v4l2_m2m h264_v4l2_m2m" hevc_amf_encoder_deps="amf" +hevc_amf_decoder_deps="amf" hevc_cuvid_decoder_deps="cuvid" hevc_cuvid_decoder_select="hevc_mp4toannexb_bsf" hevc_d3d12va_encoder_select="cbs_h265 d3d12va_encode" diff --git a/libavcodec/Makefile b/libavcodec/Makefile index dd5d0de898..a651e366f5 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -75,7 +75,7 @@ include $(SRC_PATH)/libavcodec/vulkan/Makefile OBJS-$(CONFIG_AANDCTTABLES) += aandcttab.o OBJS-$(CONFIG_AC3DSP) += ac3dsp.o ac3.o ac3tab.o OBJS-$(CONFIG_ADTS_HEADER) += adts_header.o mpeg4audio_sample_rates.o -OBJS-$(CONFIG_AMF) += amfenc.o +OBJS-$(CONFIG_AMF) += amfenc.o amfdec.o OBJS-$(CONFIG_AUDIO_FRAME_QUEUE) += audio_frame_queue.o OBJS-$(CONFIG_ATSC_A53) += atsc_a53.o OBJS-$(CONFIG_AUDIODSP) += audiodsp.o @@ -172,6 +172,7 @@ OBJS-$(CONFIG_TEXTUREDSPENC) += texturedspenc.o OBJS-$(CONFIG_TPELDSP) += tpeldsp.o OBJS-$(CONFIG_VAAPI_ENCODE) += vaapi_encode.o hw_base_encode.o OBJS-$(CONFIG_AV1_AMF_ENCODER) += amfenc_av1.o +OBJS-$(CONFIG_AV1_AMF_DECODER) += amfdec.o OBJS-$(CONFIG_VC1DSP) += vc1dsp.o OBJS-$(CONFIG_VIDEODSP) += videodsp.o OBJS-$(CONFIG_VP3DSP) += vp3dsp.o @@ -417,6 +418,7 @@ OBJS-$(CONFIG_H264_DECODER) += h264dec.o h264_cabac.o h264_cavlc.o \ h264_refs.o \ h264_slice.o h264data.o h274.o OBJS-$(CONFIG_H264_AMF_ENCODER) += amfenc_h264.o +OBJS-$(CONFIG_H264_AMF_DECODER) += amfdec.o OBJS-$(CONFIG_H264_CUVID_DECODER) += cuviddec.o OBJS-$(CONFIG_H264_MEDIACODEC_DECODER) += mediacodecdec.o OBJS-$(CONFIG_H264_MEDIACODEC_ENCODER) += mediacodecenc.o @@ -443,6 +445,7 @@ OBJS-$(CONFIG_HDR_DECODER) += hdrdec.o OBJS-$(CONFIG_HDR_ENCODER) += hdrenc.o OBJS-$(CONFIG_HEVC_DECODER) += aom_film_grain.o h274.o container_fifo.o OBJS-$(CONFIG_HEVC_AMF_ENCODER) += amfenc_hevc.o +OBJS-$(CONFIG_HEVC_AMF_DECODER) += amfdec.o OBJS-$(CONFIG_HEVC_CUVID_DECODER) += cuviddec.o OBJS-$(CONFIG_HEVC_D3D12VA_ENCODER) += d3d12va_encode_hevc.o h265_profile_level.o \ h2645data.o @@ -1278,7 +1281,7 @@ SKIPHEADERS += %_tablegen.h \ bitstream_template.h \ $(ARCH)/vpx_arith.h \ -SKIPHEADERS-$(CONFIG_AMF) += amfenc.h +SKIPHEADERS-$(CONFIG_AMF) += amfenc.h amfdec.h SKIPHEADERS-$(CONFIG_D3D11VA) += d3d11va.h dxva2_internal.h SKIPHEADERS-$(CONFIG_D3D12VA) += d3d12va_decode.h d3d12va_encode.h SKIPHEADERS-$(CONFIG_DXVA2) += dxva2.h dxva2_internal.h diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index c7e5f9910c..8266b56d22 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -840,11 +840,13 @@ extern const FFCodec ff_av1_nvenc_encoder; extern const FFCodec ff_av1_qsv_decoder; extern const FFCodec ff_av1_qsv_encoder; extern const FFCodec ff_av1_amf_encoder; +extern const FFCodec ff_av1_amf_decoder; extern const FFCodec ff_av1_mf_encoder; extern const FFCodec ff_av1_vaapi_encoder; extern const FFCodec ff_libopenh264_encoder; extern const FFCodec ff_libopenh264_decoder; extern const FFCodec ff_h264_amf_encoder; +extern const FFCodec ff_h264_amf_decoder; extern const FFCodec ff_h264_cuvid_decoder; extern const FFCodec ff_h264_mf_encoder; extern const FFCodec ff_h264_nvenc_encoder; @@ -855,6 +857,7 @@ extern const FFCodec ff_h264_vaapi_encoder; extern const FFCodec ff_h264_videotoolbox_encoder; extern const FFCodec ff_h264_vulkan_encoder; extern const FFCodec ff_hevc_amf_encoder; +extern const FFCodec ff_hevc_amf_decoder; extern const FFCodec ff_hevc_cuvid_decoder; extern const FFCodec ff_hevc_d3d12va_encoder; extern const FFCodec ff_hevc_mediacodec_decoder; diff --git a/libavcodec/amfdec.c b/libavcodec/amfdec.c new file mode 100644 index 0000000000..9bbb03f98f --- /dev/null +++ b/libavcodec/amfdec.c @@ -0,0 +1,624 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/hwcontext_amf.h" +#include "libavutil/hwcontext_amf_internal.h" +#include "amfdec.h" +#include "codec_internal.h" +#include "hwconfig.h" +#include "libavutil/imgutils.h" +#include "libavutil/mem.h" +#include "libavutil/time.h" +#include "decode.h" +#include "libavutil/mastering_display_metadata.h" + +#if CONFIG_D3D11VA +#include "libavutil/hwcontext_d3d11va.h" +#endif +#if CONFIG_DXVA2 +#define COBJMACROS +#include "libavutil/hwcontext_dxva2.h" +#endif + +#ifdef _WIN32 +#include "compat/w32dlfcn.h" +#else +#include +#endif +//will be in public headers soon +#define AMF_VIDEO_DECODER_OUTPUT_FORMAT L"OutputDecodeFormat" + +const enum AVPixelFormat amf_dec_pix_fmts[] = { + AV_PIX_FMT_NV12, + AV_PIX_FMT_P010, + AV_PIX_FMT_P012, + AV_PIX_FMT_AMF_SURFACE, + AV_PIX_FMT_NONE +}; + +static const AVCodecHWConfigInternal *const amf_hw_configs[] = { + &(const AVCodecHWConfigInternal) { + .public = { + .pix_fmt = AV_PIX_FMT_AMF_SURFACE, + .methods = AV_CODEC_HW_CONFIG_METHOD_HW_FRAMES_CTX | + AV_CODEC_HW_CONFIG_METHOD_HW_DEVICE_CTX, + .device_type = AV_HWDEVICE_TYPE_AMF, + }, + .hwaccel = NULL, + }, + NULL +}; + +static void amf_free_amfsurface(void *opaque, uint8_t *data) +{ + AMFSurface *surface = (AMFSurface*)(data); + surface->pVtbl->Release(surface); +} + +static int amf_init_decoder(AVCodecContext *avctx) +{ + AMFDecoderContext *ctx = avctx->priv_data; + AVHWDeviceContext *hw_device_ctx = (AVHWDeviceContext*)ctx->device_ctx_ref->data; + AVAMFDeviceContext *amf_device_ctx = (AVAMFDeviceContext *)hw_device_ctx->hwctx; + const wchar_t *codec_id = NULL; + AMF_RESULT res; + AMFBuffer *buffer; + amf_int64 color_profile; + int pool_size = 36; + + ctx->drain = 0; + ctx->resolution_changed = 0; + + switch (avctx->codec->id) { + case AV_CODEC_ID_H264: + codec_id = AMFVideoDecoderUVD_H264_AVC; + break; + case AV_CODEC_ID_HEVC: { + codec_id = AMFVideoDecoderHW_H265_HEVC; + } break; + case AV_CODEC_ID_AV1: + codec_id = AMFVideoDecoderHW_AV1; + break; + default: + break; + } + AMF_RETURN_IF_FALSE(ctx, codec_id != NULL, AVERROR(EINVAL), "Codec %d is not supported\n", avctx->codec->id); + + res = amf_device_ctx->factory->pVtbl->CreateComponent(amf_device_ctx->factory, amf_device_ctx->context, codec_id, &ctx->decoder); + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_ENCODER_NOT_FOUND, "CreateComponent(%ls) failed with error %d\n", codec_id, res); + + // Color Metadata + /// Color Range (Support for older Drivers) + if (avctx->color_range == AVCOL_RANGE_JPEG) { + AMF_ASSIGN_PROPERTY_BOOL(res, ctx->decoder, AMF_VIDEO_DECODER_FULL_RANGE_COLOR, 1); + } else if (avctx->color_range != AVCOL_RANGE_UNSPECIFIED) { + AMF_ASSIGN_PROPERTY_BOOL(res, ctx->decoder, AMF_VIDEO_DECODER_FULL_RANGE_COLOR, 0); + } + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN; + switch (avctx->colorspace) { + case AVCOL_SPC_SMPTE170M: + if (avctx->color_range == AVCOL_RANGE_JPEG) { + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_601; + } else { + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_601; + } + break; + case AVCOL_SPC_BT709: + if (avctx->color_range == AVCOL_RANGE_JPEG) { + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_709; + } else { + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_709; + } + break; + case AVCOL_SPC_BT2020_NCL: + case AVCOL_SPC_BT2020_CL: + if (avctx->color_range == AVCOL_RANGE_JPEG) { + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_2020; + } else { + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_2020; + } + break; + } + if (color_profile != AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_COLOR_PROFILE, color_profile); + if (avctx->color_trc != AVCOL_TRC_UNSPECIFIED) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_COLOR_TRANSFER_CHARACTERISTIC, (amf_int64)avctx->color_trc); + + if (avctx->color_primaries != AVCOL_PRI_UNSPECIFIED) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_COLOR_PRIMARIES, (amf_int64)avctx->color_primaries); + + if (ctx->timestamp_mode != -1) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_TIMESTAMP_MODE, ctx->timestamp_mode); + if (ctx->decoder_mode != -1) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_REORDER_MODE, ctx->decoder_mode); + if (ctx->dpb_size != -1) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_DPB_SIZE, ctx->dpb_size); + if (ctx->lowlatency != -1) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_LOW_LATENCY, ctx->lowlatency); + if (ctx->smart_access_video != -1) { + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_ENABLE_SMART_ACCESS_VIDEO, ctx->smart_access_video != 0); + if (res != AMF_OK) { + av_log(avctx, AV_LOG_ERROR, "The Smart Access Video is not supported by AMF decoder.\n"); + return AVERROR(EINVAL); + } else { + av_log(avctx, AV_LOG_INFO, "The Smart Access Video (%d) is set.\n", ctx->smart_access_video); + // Set low latency mode if Smart Access Video is enabled + if (ctx->smart_access_video != 0) { + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_LOW_LATENCY, true); + av_log(avctx, AV_LOG_INFO, "The Smart Access Video set low latency mode for decoder.\n"); + } + } + } + if (ctx->skip_transfer_sav != -1) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_SKIP_TRANSFER_SMART_ACCESS_VIDEO, ctx->skip_transfer_sav); + + if (avctx->extradata_size) { + res = amf_device_ctx->context->pVtbl->AllocBuffer(amf_device_ctx->context, AMF_MEMORY_HOST, avctx->extradata_size, &buffer); + if (res == AMF_OK) { + memcpy(buffer->pVtbl->GetNative(buffer), avctx->extradata, avctx->extradata_size); + AMF_ASSIGN_PROPERTY_INTERFACE(res,ctx->decoder, AMF_VIDEO_DECODER_EXTRADATA, buffer); + buffer->pVtbl->Release(buffer); + buffer = NULL; + } + } + if (ctx->surface_pool_size == -1) { + ctx->surface_pool_size = pool_size; + if (avctx->extra_hw_frames > 0) + ctx->surface_pool_size += avctx->extra_hw_frames; + if (avctx->active_thread_type & FF_THREAD_FRAME) + ctx->surface_pool_size += avctx->thread_count; + } + + //at the moment, there is such a restriction in AMF. + //when it is possible, I will remove this code + if (ctx->surface_pool_size > 100) + ctx->surface_pool_size = 100; + + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_SURFACE_POOL_SIZE, ctx->surface_pool_size); + res = ctx->decoder->pVtbl->Init(ctx->decoder, AMF_SURFACE_UNKNOWN, avctx->width, avctx->height); + + return 0; +} + +static int amf_decode_close(AVCodecContext *avctx) +{ + AMFDecoderContext *ctx = avctx->priv_data; + + if (ctx->decoder) { + ctx->decoder->pVtbl->Terminate(ctx->decoder); + ctx->decoder->pVtbl->Release(ctx->decoder); + ctx->decoder = NULL; + } + + av_buffer_unref(&ctx->device_ctx_ref); + av_packet_unref(ctx->in_pkt); + + return 0; +} + +static int amf_decode_init(AVCodecContext *avctx) +{ + AMFDecoderContext *ctx = avctx->priv_data; + int ret; + ctx->in_pkt = av_packet_alloc(); + if (!ctx->in_pkt) + return AVERROR(ENOMEM); + + if (avctx->hw_device_ctx) { + AVHWDeviceContext *hwdev_ctx; + AVHWFramesContext *hwframes_ctx; + hwdev_ctx = (AVHWDeviceContext*)avctx->hw_device_ctx->data; + if (hwdev_ctx->type == AV_HWDEVICE_TYPE_AMF) + { + ctx->device_ctx_ref = av_buffer_ref(avctx->hw_device_ctx); + avctx->hw_frames_ctx = av_hwframe_ctx_alloc(avctx->hw_device_ctx); + + if (!avctx->hw_frames_ctx) { + av_log(avctx, AV_LOG_ERROR, "av_hwframe_ctx_alloc failed\n"); + return AVERROR(ENOMEM); + } + + hwframes_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; + hwframes_ctx->width = FFALIGN(avctx->coded_width, 32); + hwframes_ctx->height = FFALIGN(avctx->coded_height, 32); + hwframes_ctx->format = AV_PIX_FMT_AMF_SURFACE; + hwframes_ctx->sw_format = AV_PIX_FMT_NV12; + hwframes_ctx->initial_pool_size = ctx->surface_pool_size + 8; + + ret = av_hwframe_ctx_init(avctx->hw_frames_ctx); + if (ret < 0) { + av_log(NULL, AV_LOG_ERROR, "Error initializing a AMF frame pool\n"); + av_buffer_unref(&avctx->hw_frames_ctx); + return ret; + } + } else { + ret = av_hwdevice_ctx_create_derived(&ctx->device_ctx_ref, AV_HWDEVICE_TYPE_AMF, avctx->hw_device_ctx, 0); + AMF_RETURN_IF_FALSE(avctx, ret == 0, ret, "Failed to create derived AMF device context: %s\n", av_err2str(ret)); + } + } else { + ret = av_hwdevice_ctx_create(&ctx->device_ctx_ref, AV_HWDEVICE_TYPE_AMF, NULL, NULL, 0); + AMF_RETURN_IF_FALSE(avctx, ret == 0, ret, "Failed to create hardware device context (AMF) : %s\n", av_err2str(ret)); + } + if ((ret = amf_init_decoder(avctx)) == 0) { + AMFVariantStruct format_var = {0}; + ret = ctx->decoder->pVtbl->GetProperty(ctx->decoder, AMF_VIDEO_DECODER_OUTPUT_FORMAT, &format_var); + if (ret != AMF_OK) { + amf_decode_close(avctx); + return AVERROR(EINVAL); + } + + if(avctx->hw_frames_ctx) + ((AVHWFramesContext*)avctx->hw_frames_ctx->data)->sw_format = av_amf_to_av_format(format_var.int64Value); + else + avctx->pix_fmt = av_amf_to_av_format(format_var.int64Value); + + return 0; + } + amf_decode_close(avctx); + return ret; +} + +static AMF_RESULT amf_get_property_buffer(AMFData *object, const wchar_t *name, AMFBuffer **val) +{ + AMF_RESULT res; + AMFVariantStruct var; + res = AMFVariantInit(&var); + if (res == AMF_OK) { + res = object->pVtbl->GetProperty(object, name, &var); + if (res == AMF_OK) { + if (var.type == AMF_VARIANT_INTERFACE) { + AMFGuid guid_AMFBuffer = IID_AMFBuffer(); + AMFInterface *amf_interface = AMFVariantInterface(&var); + res = amf_interface->pVtbl->QueryInterface(amf_interface, &guid_AMFBuffer, (void**)val); + } else { + res = AMF_INVALID_DATA_TYPE; + } + } + AMFVariantClear(&var); + } + return res; +} + +static int amf_amfsurface_to_avframe(AVCodecContext *avctx, AMFSurface* surface, AVFrame *frame) +{ + AMFVariantStruct var = {0}; + AMFPlane *plane; + int i; + int ret; + + if (avctx->hw_device_ctx && ((AVHWDeviceContext*)avctx->hw_device_ctx->data)->type == AV_HWDEVICE_TYPE_AMF) { + ret = ff_get_buffer(avctx, frame, AV_GET_BUFFER_FLAG_REF); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Get hw frame failed.\n"); + return ret; + } + frame->buf[0] = av_buffer_create((uint8_t *)surface, sizeof(AMFSurface), + amf_free_amfsurface, (void*)avctx, + AV_BUFFER_FLAG_READONLY); + frame->data[0] = (uint8_t *)surface; + frame->format = AV_PIX_FMT_AMF_SURFACE; + frame->hw_frames_ctx = av_buffer_ref(avctx->hw_frames_ctx); + + } else { + ret = surface->pVtbl->Convert(surface, AMF_MEMORY_HOST); + AMF_RETURN_IF_FALSE(avctx, ret == AMF_OK, AVERROR_UNKNOWN, "Convert(amf::AMF_MEMORY_HOST) failed with error %d\n", ret); + + for (i = 0; i < surface->pVtbl->GetPlanesCount(surface); i++) { + plane = surface->pVtbl->GetPlaneAt(surface, i); + frame->data[i] = plane->pVtbl->GetNative(plane); + frame->linesize[i] = plane->pVtbl->GetHPitch(plane); + } + + frame->buf[0] = av_buffer_create((uint8_t *)surface, sizeof(AMFSurface), + amf_free_amfsurface, (void*)avctx, + AV_BUFFER_FLAG_READONLY); + frame->format = av_amf_to_av_format(surface->pVtbl->GetFormat(surface)); + } + + frame->width = avctx->width; + frame->height = avctx->height; + + frame->pts = surface->pVtbl->GetPts(surface); + + surface->pVtbl->GetProperty(surface, L"FFMPEG:dts", &var); + frame->pkt_dts = var.int64Value; + + frame->duration = surface->pVtbl->GetDuration(surface); + if (frame->duration < 0) + frame->duration = 0; + + frame->color_range = avctx->color_range; + frame->colorspace = avctx->colorspace; + frame->color_trc = avctx->color_trc; + frame->color_primaries = avctx->color_primaries; + + if (frame->color_trc == AVCOL_TRC_SMPTE2084) { + AMFBuffer * hdrmeta_buffer = NULL; + ret = amf_get_property_buffer((AMFData *)surface, AMF_VIDEO_DECODER_HDR_METADATA, &hdrmeta_buffer); + if (hdrmeta_buffer != NULL) { + AMFHDRMetadata * hdrmeta = (AMFHDRMetadata*)hdrmeta_buffer->pVtbl->GetNative(hdrmeta_buffer); + if (ret != AMF_OK) + return ret; + if (hdrmeta != NULL) { + AVMasteringDisplayMetadata *mastering = av_mastering_display_metadata_create_side_data(frame); + const int chroma_den = 50000; + const int luma_den = 10000; + + if (!mastering) + return AVERROR(ENOMEM); + + mastering->display_primaries[0][0] = av_make_q(hdrmeta->redPrimary[0], chroma_den); + mastering->display_primaries[0][1] = av_make_q(hdrmeta->redPrimary[1], chroma_den); + + mastering->display_primaries[1][0] = av_make_q(hdrmeta->greenPrimary[0], chroma_den); + mastering->display_primaries[1][1] = av_make_q(hdrmeta->greenPrimary[1], chroma_den); + + mastering->display_primaries[2][0] = av_make_q(hdrmeta->bluePrimary[0], chroma_den); + mastering->display_primaries[2][1] = av_make_q(hdrmeta->bluePrimary[1], chroma_den); + + mastering->white_point[0] = av_make_q(hdrmeta->whitePoint[0], chroma_den); + mastering->white_point[1] = av_make_q(hdrmeta->whitePoint[1], chroma_den); + + mastering->max_luminance = av_make_q(hdrmeta->maxMasteringLuminance, luma_den); + mastering->min_luminance = av_make_q(hdrmeta->maxMasteringLuminance, luma_den); + + mastering->has_luminance = 1; + mastering->has_primaries = 1; + if (hdrmeta->maxContentLightLevel) { + AVContentLightMetadata *light = av_content_light_metadata_create_side_data(frame); + + if (!light) + return AVERROR(ENOMEM); + + light->MaxCLL = hdrmeta->maxContentLightLevel; + light->MaxFALL = hdrmeta->maxFrameAverageLightLevel; + } + } + } + } + return 0; +} + +static AMF_RESULT amf_receive_frame(AVCodecContext *avctx, AVFrame *frame) +{ + AMFDecoderContext *ctx = avctx->priv_data; + AMF_RESULT ret = AMF_OK; + AMFSurface *surface = NULL; + AMFData *data_out = NULL; + + ret = ctx->decoder->pVtbl->QueryOutput(ctx->decoder, &data_out); + if (ret != AMF_OK && ret != AMF_REPEAT) { + return ret; + } + if (data_out == NULL) { + return AMF_REPEAT; + } + + if (data_out) { + AMFGuid guid = IID_AMFSurface(); + data_out->pVtbl->QueryInterface(data_out, &guid, (void**)&surface); // query for buffer interface + data_out->pVtbl->Release(data_out); + data_out = NULL; + } + + ret = amf_amfsurface_to_avframe(avctx, surface, frame); + AMF_GOTO_FAIL_IF_FALSE(avctx, ret >= 0, AMF_FAIL, "Failed to convert AMFSurface to AVFrame = %d\n", ret); + return AMF_OK; +fail: + + if (surface) { + surface->pVtbl->Release(surface); + surface = NULL; + } + return ret; +} + +static AMF_RESULT amf_update_buffer_properties(AVCodecContext *avctx, AMFBuffer* buffer, const AVPacket* pkt) +{ + AMF_RESULT res; + + AMF_RETURN_IF_FALSE(avctx, buffer != NULL, AMF_INVALID_ARG, "update_buffer_properties() - buffer not passed in"); + AMF_RETURN_IF_FALSE(avctx, pkt != NULL, AMF_INVALID_ARG, "update_buffer_properties() - packet not passed in"); + buffer->pVtbl->SetPts(buffer, pkt->pts); + buffer->pVtbl->SetDuration(buffer, pkt->duration); + AMF_ASSIGN_PROPERTY_INT64(res, buffer, L"FFMPEG:dts", pkt->dts); + if (res != AMF_OK) + av_log(avctx, AV_LOG_VERBOSE, "Failed to assign dts value."); + return AMF_OK; +} + +static AMF_RESULT amf_buffer_from_packet(AVCodecContext *avctx, const AVPacket* pkt, AMFBuffer** buffer) +{ + AMFDecoderContext *ctx = avctx->priv_data; + AVHWDeviceContext *hw_device_ctx = (AVHWDeviceContext*)ctx->device_ctx_ref->data; + AVAMFDeviceContext *amf_device_ctx = (AVAMFDeviceContext *)hw_device_ctx->hwctx; + AMFContext *ctxt = amf_device_ctx->context; + void *mem; + AMF_RESULT err; + AMFBuffer *buf = NULL; + + AMF_RETURN_IF_FALSE(ctxt, pkt != NULL, AMF_INVALID_ARG, "amf_buffer_from_packet() - packet not passed in"); + AMF_RETURN_IF_FALSE(ctxt, buffer != NULL, AMF_INVALID_ARG, "amf_buffer_from_packet() - buffer pointer not passed in"); + + err = ctxt->pVtbl->AllocBuffer(ctxt, AMF_MEMORY_HOST, pkt->size + AV_INPUT_BUFFER_PADDING_SIZE, buffer); + AMF_RETURN_IF_FALSE(ctxt, err == AMF_OK, err, "amf_buffer_from_packet() - failed"); + buf = *buffer; + err = buf->pVtbl->SetSize(buf, pkt->size); + AMF_RETURN_IF_FALSE(ctxt, err == AMF_OK, err, "amf_buffer_from_packet() - SetSize failed"); + // get the memory location and check the buffer was indeed allocated + mem = buf->pVtbl->GetNative(buf); + AMF_RETURN_IF_FALSE(ctxt, mem != NULL, AMF_INVALID_POINTER, "amf_buffer_from_packet() - GetNative failed"); + + // copy the packet memory and clear data padding + memcpy(mem, pkt->data, pkt->size); + memset((amf_int8*)(mem)+pkt->size, 0, AV_INPUT_BUFFER_PADDING_SIZE); + + return amf_update_buffer_properties(avctx, buf, pkt); +} + +static int amf_decode_frame(AVCodecContext *avctx, struct AVFrame *frame) +{ + AMFDecoderContext *ctx = avctx->priv_data; + AMFBuffer *buf; + AMF_RESULT res; + int got_frame = 0; + AVPacket *avpkt = ctx->in_pkt; + + if (!ctx->decoder) + return AVERROR(EINVAL); + + // get packet if needed + if(!ctx->drain){ + if(ctx->resolution_changed) + ctx->resolution_changed = 0; + else{ + av_packet_unref(avpkt); + int ret; + ret = ff_decode_get_packet(avctx, avpkt); + if (ret < 0 && ret != AVERROR_EOF) + return ret; + if (ret == AVERROR_EOF) { + //nothing to consume, start external drain + ctx->decoder->pVtbl->Drain(ctx->decoder); + ctx->drain = 1; + } + } + } + + if (!ctx->drain) { + // submit frame + res = amf_buffer_from_packet(avctx, avpkt, &buf); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, 0, "Cannot convert AVPacket to AMFbuffer"); + res = ctx->decoder->pVtbl->SubmitInput(ctx->decoder, (AMFData*) buf); + buf->pVtbl->Release(buf); + + if(res == AMF_DECODER_NO_FREE_SURFACES) { + // input is not consumed, need to QueryOutput and submit again + av_log(avctx, AV_LOG_VERBOSE, "SubmitInput() returned NO_FREE_SURFACES\n"); + } else if (res == AMF_RESOLUTION_CHANGED) { + //input is not consumed, start internal drain + ctx->decoder->pVtbl->Drain(ctx->decoder); + ctx->drain = 1; + // process resolution_changed when internal drain is complete + ctx->resolution_changed = 1; + res = AMF_OK; + } else if (res != AMF_OK && res != AMF_NEED_MORE_INPUT && res != AMF_REPEAT) { + av_log(avctx, AV_LOG_ERROR, "SubmitInput() returned error %d\n", res); + return AVERROR(EINVAL); + } + + } + + res = amf_receive_frame(avctx, frame); + if (res == AMF_OK) + got_frame = 1; + else if (res == AMF_REPEAT) + // decoder has no output yet + res = AMF_OK; + else if (res == AMF_EOF) { + // drain is complete + ctx->drain = 0; + if(ctx->resolution_changed){ + // re-initialze decoder + AMFVariantStruct size_var = {0}; + res = ctx->decoder->pVtbl->GetProperty(ctx->decoder, AMF_VIDEO_DECODER_CURRENT_SIZE, &size_var); + if (res != AMF_OK) { + return AVERROR(EINVAL); + } + + avctx->width = size_var.sizeValue.width; + avctx->height = size_var.sizeValue.height; + avctx->coded_width = size_var.sizeValue.width; + avctx->coded_height = size_var.sizeValue.height; + res = ctx->decoder->pVtbl->ReInit(ctx->decoder, avctx->width, avctx->height); + if (res != AMF_OK) { + av_log(avctx, AV_LOG_ERROR, "ReInit() returned %d\n", res); + return AVERROR(EINVAL); + } + + }else + return AVERROR_EOF; + } else { + av_log(avctx, AV_LOG_ERROR, "Unkown result from QueryOutput %d\n", res); + } + return got_frame ? 0 : AVERROR(EAGAIN); +} + +static void amf_decode_flush(AVCodecContext *avctx) +{ + AMFDecoderContext *ctx = avctx->priv_data; + ctx->decoder->pVtbl->Flush(ctx->decoder); +} + +#define OFFSET(x) offsetof(AMFDecoderContext, x) +#define VD AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_DECODING_PARAM + +static const AVOption options[] = { + // Decoder mode + { "decoder_mode", "Decoder mode", OFFSET(decoder_mode), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, AMF_VIDEO_DECODER_MODE_LOW_LATENCY, VD, "decoder_mode" }, + { "regular", "DPB delay is based on number of reference frames + 1", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_DECODER_MODE_REGULAR }, 0, 0, VD, "decoder_mode" }, + { "compliant", "DPB delay is based on profile - up to 16", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_DECODER_MODE_COMPLIANT }, 0, 0, VD, "decoder_mode" }, + { "low_latency", "DPB delay is 0", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_DECODER_MODE_LOW_LATENCY }, 0, 0, VD, "decoder_mode" }, + + // Timestamp mode + { "timestamp_mode", "Timestamp mode", OFFSET(timestamp_mode), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, AMF_TS_DECODE, VD, "timestamp_mode" }, + { "presentation", "Preserve timestamps from input to output", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_TS_PRESENTATION }, 0, 0, VD, "timestamp_mode" }, + { "sort", "Resort PTS list", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_TS_SORT }, 0, 0, VD, "timestamp_mode" }, + { "decode", "Decode order", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_TS_DECODE }, 0, 0, VD, "timestamp_mode" }, + + // Reference frame management + { "surface_pool_size", "Number of surfaces in the decode pool", OFFSET(surface_pool_size), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, INT_MAX, VD, NULL }, + { "dpb_size", "Minimum number of surfaces for reordering", OFFSET(dpb_size), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 32, VD, NULL }, + + { "lowlatency", "Low latency", OFFSET(lowlatency), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 1, VD, NULL }, + { "smart_access_video", "Smart Access Video", OFFSET(smart_access_video), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 1, VD, NULL }, + { "skip_transfer_sav", "Skip transfer on another GPU when SAV enabled", OFFSET(skip_transfer_sav), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 1, VD, NULL }, + + { NULL } +}; + +static const AVClass amf_decode_class = { + .class_name = "amf", + .item_name = av_default_item_name, + .option = options, + .version = LIBAVUTIL_VERSION_INT, +}; + +#define DEFINE_AMF_DECODER(x, X, bsf_name) \ +const FFCodec ff_##x##_amf_decoder = { \ + .p.name = #x "_amf", \ + CODEC_LONG_NAME(#X " AMD AMF video decoder"), \ + .priv_data_size = sizeof(AMFDecoderContext), \ + .p.type = AVMEDIA_TYPE_VIDEO, \ + .p.id = AV_CODEC_ID_##X, \ + .init = amf_decode_init, \ + FF_CODEC_RECEIVE_FRAME_CB(amf_decode_frame), \ + .flush = amf_decode_flush, \ + .close = amf_decode_close, \ + .bsfs = bsf_name, \ + .p.capabilities = AV_CODEC_CAP_HARDWARE | AV_CODEC_CAP_DELAY | AV_CODEC_CAP_AVOID_PROBING, \ + .p.priv_class = &amf_decode_class, \ + .p.pix_fmts = amf_dec_pix_fmts, \ + .hw_configs = amf_hw_configs, \ + .p.wrapper_name = "amf", \ + .caps_internal = FF_CODEC_CAP_NOT_INIT_THREADSAFE, \ +}; \ + +DEFINE_AMF_DECODER(h264, H264, "h264_mp4toannexb") +DEFINE_AMF_DECODER(hevc, HEVC, NULL) +DEFINE_AMF_DECODER(av1, AV1, NULL) diff --git a/libavcodec/amfdec.h b/libavcodec/amfdec.h new file mode 100644 index 0000000000..82b65c324d --- /dev/null +++ b/libavcodec/amfdec.h @@ -0,0 +1,61 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_AMFDEC_H +#define AVCODEC_AMFDEC_H + +#include +#include +#include +#include +#include +#include + +#include "avcodec.h" +#include "libavformat/avformat.h" +#include "libavutil/fifo.h" +#include "libavutil/frame.h" +#include "libavutil/opt.h" +#include "libavutil/hwcontext_amf.h" +/** +* AMF decoder context +*/ +typedef struct AMFDecoderContext { + AVClass *avclass; + AVBufferRef *device_ctx_ref; + + //decoder + AMFComponent *decoder; ///< AMF decoder object + AMF_SURFACE_FORMAT format; ///< AMF surface format + + // common decoder options + int decoder_mode; + int timestamp_mode; + int surface_pool_size; + int dpb_size; + int lowlatency; + int smart_access_video; + int skip_transfer_sav; + int drain; + int resolution_changed; + AVPacket* in_pkt; + enum AMF_SURFACE_FORMAT output_format; + +} AMFDecoderContext; + +#endif // AVCODEC_AMFDEC_H \ No newline at end of file From patchwork Mon Oct 21 23:16:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitrii Ovchinnikov X-Patchwork-Id: 52443 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:143:b0:48e:c0f8:d0de with SMTP id h3csp3203993vqi; Tue, 22 Oct 2024 01:44:06 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCWWAKPCBN+y3kykewdN2iUxZX4e8/s6/pZPyNwj0d3qMJTX4dXO9+2Xcw/36fndq7zCD4hsjvCrEenbSHLdXVFd@gmail.com X-Google-Smtp-Source: AGHT+IFYNoX/UvXJy3k98MHfvnPpGf95/ScnPg3ajPfItJTnxGT6EBtGCMx+FjsP8PW3eRdwTv6j X-Received: by 2002:a05:651c:2226:b0:2fc:9759:4e20 with SMTP id 38308e7fff4ca-2fc975952e4mr3478701fa.12.1729586646199; Tue, 22 Oct 2024 01:44:06 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1729586646; cv=none; d=google.com; s=arc-20240605; b=c8W091U+ZOApqa9BLycC8YU7ZOm8YEKicWhcLd/dieLaX6ln6Mrx7FG5AwoCdcQVZG CpEH/Ji0hs5DqR+IlbmBJFmmjqwker6+RNEY6jdt8e1EQoIQaRwYdQL2KYJAWRm04PtS JkSenyxuQBSkwhkS57nsLqEd9yDCaf2T0gfkKHCNtQewCwF4uwsB+hUSLKmrHp3jmziF oWqi48fg5D9Cea7Vfj1Zspxv5CBEwFAsQiT9LbyvxWLdwAFnrvJ6V4dUGtMInf1SLIqW fCm2zdMZ5VRQG42mGSHK3hOdBOezY5YJ1I/Ch5PLlof2LTY2ADxkw0tyHdPs5xnET7Nh PG6A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=WZ7NQmHlQdH25OF1mIBR03j9SasDgd6LSyr82KJfXu4=; fh=l8BBgSaHK1qPkRTUMT3LasR23D+FgYCLMIPW5YpagSs=; b=cGc3Lz+4LkpxdhFKUmEsY511NbtHx+cFjVAXVQjC7MkR7ENARfFyQ1iHyToAvVl129 D7YKNvwi05dL2xmv3LP8ocEnn9VrHsQW9hCRkpk7UWdoBc1Zeuq2YQ4vL2Rc8sDmxZ7x p/CopfX2OQAs1xwf8+lGJBU3r1hZLiUGh5qIEqaryCOEoU54vR/UhLBsL7Rdb1F6O0Lt AcKE/KOHNx8fSmglU9lxRFOBD0YnIpsS4b6/Ls11k/NpuSq2viRiZpmDMIaIVzD8+wXc m63SBvkzB9T4xicBlwgKTDzjR43Hh2nRp+kQl9BGq0GAS4KeUUgvcXDyVddiamCzoWmM 9wSA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b="g21h/BjR"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 38308e7fff4ca-2fb9addfc15si16126141fa.162.2024.10.22.01.44.05; Tue, 22 Oct 2024 01:44:06 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b="g21h/BjR"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5A16C68DE80; Tue, 22 Oct 2024 02:17:23 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f43.google.com (mail-wr1-f43.google.com [209.85.221.43]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5D76E68DE60 for ; Tue, 22 Oct 2024 02:17:15 +0300 (EEST) Received: by mail-wr1-f43.google.com with SMTP id ffacd0b85a97d-37d4b0943c7so3653597f8f.1 for ; Mon, 21 Oct 2024 16:17:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729552634; x=1730157434; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=dSE379ECvEj54r/VmY/eB49U9kQ0s4xhBNaKilF7NfA=; b=g21h/BjRM3bcpGYt9f51TjRrxuZIPPpCDB4yz6+wcU5awh8Y9pxkukbj4C67p7EZgU DqtJXkyu19RpxYyyQ7aBR44n94Bx7YF3/RZ40PyxaWTDXa0PJoaMoXAQF4ORTrsCXwbS Y5gWHdqly4+JHPk67P1uOnjahiQI5oxdiJIM2DRJ3yF95sbdO6emsZCZ1zml6fSi28yp 3ApRutfEUHOPW10Zn2jKWme2mFi5/j27XAab57tbV3RKscjWWQWxAWbZplTcE0ATAJ99 kw/Y1KU9v2YAtW6gkugQyXPz3tRWcB9+ldASwY1Sek7Kl0eAnemwlKHdKso17oGjytv8 iKvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729552634; x=1730157434; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dSE379ECvEj54r/VmY/eB49U9kQ0s4xhBNaKilF7NfA=; b=ohySm437I80Xlfw3U//o1hwDu4OktJRTyAAUCAQmKaa+9uJPd3IMBCqauZjauca1B4 Pqm2tR8gOxc92cp52Of+FoJlRcHntVd04EluLBtk3Ax5/BZ4+3CAcIEV7xS6GS1mE54l KvDN3PqjMvcX0LyR9hiYRGrPGR/B2mnd2p0ZubDoxTGKgbX0t6XmM3Fc/dW9mcrJdp46 h40HlUFu/V8JNAI7THYMAl3Xadq8w/T0bh4+Kp9wg862Ux1TBua2IUxo2wJzz1IvOQvJ XiQ94306h2XmKedsRCZTsnJbM8lhdg37zxcjdxRicx0Ft14kNQB22mhIpPCekEBJ3pwN jo1Q== X-Gm-Message-State: AOJu0Yy8jAcBpUiToIvmTs7dPo1MsOzjjscQVhYDcciaxnB69W63vWrV jb9MD1H2KDrWmQ5rqko4YukzOrKwgbIWESZy+if6POva60q9x7NjFvprnw== X-Received: by 2002:a05:6000:104a:b0:37d:890c:63e4 with SMTP id ffacd0b85a97d-37ebd3aa743mr7640285f8f.55.1729552634041; Mon, 21 Oct 2024 16:17:14 -0700 (PDT) Received: from dovchinn.amd.com (cable-178-148-16-149.dynamic.sbb.rs. [178.148.16.149]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-37ee0a57e0fsm5293937f8f.47.2024.10.21.16.17.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Oct 2024 16:17:12 -0700 (PDT) From: Dmitrii Ovchinnikov To: ffmpeg-devel@ffmpeg.org Date: Tue, 22 Oct 2024 01:16:35 +0200 Message-Id: <20241021231636.1199-4-ovchinnikov.dmitrii@gmail.com> X-Mailer: git-send-email 2.38.1.windows.1 In-Reply-To: <20241021231636.1199-1-ovchinnikov.dmitrii@gmail.com> References: <20241021231636.1199-1-ovchinnikov.dmitrii@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 4/5] doc/filters: Add documentation for AMF filters X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Evgeny Pavlov Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 9mWYZghjgxU5 From: Evgeny Pavlov Signed-off-by: Evgeny Pavlov --- doc/filters.texi | 238 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 238 insertions(+) diff --git a/doc/filters.texi b/doc/filters.texi index 428986a1e9..7d75ebfa3e 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -22827,6 +22827,76 @@ input upscaled using bicubic upscaling with proper scale factor. To get full functionality (such as async execution), please use the @ref{dnn_processing} filter. +@anchor{sr_amf} +@section sr_amf + +Upscale (size increasing) for the input video using AMD Advanced Media Framework library for hardware acceleration. +Use advanced algorithms for upscaling with higher output quality. +Setting the output width and height works in the same way as for the @ref{scale} filter. + +The filter accepts the following options: +@table @option +@item w +@item h +Set the output video dimension expression. Default value is the input dimension. + +Allows for the same expressions as the @ref{scale} filter. + +@item algorithm +Sets the algorithm used for scaling: + +@table @var +@item bilinear +Bilinear + +@item bicubic +Bicubic + +@item sr1-0 +Video SR1.0 +This is a default value + +@item point +Point + +@item sr1-1 +Video SR1.1 + +@end table + +@item sharpness +Control hq scaler sharpening. The value is a float in the range of [0.0, 2.0] + +@item format +Controls the output pixel format. By default, or if none is specified, the input +pixel format is used. + +@item keep-ratio +Force the scaler to keep the aspect ratio of the input image when the output size has a different aspect ratio. +Default value is false. + +@item fill +Specifies whether the output image outside the region of interest, +which does not fill the entire output surface should be filled with a solid color. + +@end table + +@subsection Examples + +@itemize +@item +Scale input to 720p, keeping aspect ratio and ensuring the output is yuv420p. +@example +sr_amf=-2:720:format=yuv420p +@end example + +@item +Upscale to 4K with algorithm video SR1.1. +@example +sr_amf=4096:2160:algorithm=sr1-1 +@end example +@end itemize + @section ssim Obtain the SSIM (Structural SImilarity Metric) between two input videos. @@ -25565,6 +25635,174 @@ Example: ffmpeg -i ref.mpg -vf vmafmotion -f null - @end example +@anchor{vpp_amf} +@section vpp_amf + +Scale (resize) and convert colorspace, transfer characteristics or color primaries for the input video, using AMD Advanced Media Framework library for hardware acceleration. +Setting the output width and height works in the same way as for the @ref{scale} filter. + +The filter accepts the following options: +@table @option +@item w +@item h +Set the output video dimension expression. Default value is the input dimension. + +Allows for the same expressions as the @ref{scale} filter. + +@item scale_type +Sets the algorithm used for scaling: + +@table @var +@item bilinear +Bilinear + +This is the default. + +@item bicubic +Bicubic + +@end table + +@item format +Controls the output pixel format. By default, or if none is specified, the input +pixel format is used. + + +@item force_original_aspect_ratio +@item force_divisible_by +Work the same as the identical @ref{scale} filter options. + +@anchor{color_profile} +@item color_profile +Specify all color properties at once. + +The accepted values are: +@table @samp +@item bt601 +BT.601 + +@item bt709 +BT.709 + +@item bt2020 +BT.2020 + +@end table + +@anchor{trc} +@item trc +Specify output transfer characteristics. + +The accepted values are: +@table @samp +@item bt709 +BT.709 + +@item gamma22 +Constant gamma of 2.2 + +@item gamma28 +Constant gamma of 2.8 + +@item smpte170m +SMPTE-170M + +@item smpte240m +SMPTE-240M + +@item linear +Linear + +@item log +LOG + +@item log-sqrt +LOG_SQRT + +@item iec61966-2-4 +iec61966-2-4 + +@item bt1361-ecg +BT1361_ECG + +@item iec61966-2-1 +iec61966-2-1 + +@item bt2020-10 +BT.2020 for 10-bits content + +@item bt2020-12 +BT.2020 for 12-bits content + +@item smpte2084 +SMPTE2084 + +@item smpte428 +SMPTE428 + +@item arib-std-b67 +ARIB_STD_B67 + +@end table + +@anchor{primaries} +@item primaries +Specify output color primaries. + +The accepted values are: +@table @samp +@item bt709 +BT.709 + +@item bt470m +BT.470M + +@item bt470bg +BT.470BG or BT.601-6 625 + +@item smpte170m +SMPTE-170M or BT.601-6 525 + +@item smpte240m +SMPTE-240M + +@item film +film + +@item bt2020 +BT.2020 + +@item smpte428 +SMPTE-428 + +@item smpte431 +SMPTE-431 + +@item smpte432 +SMPTE-432 + +@item jedec-p22 +JEDEC P22 phosphors + +@end table +@end table + +@subsection Examples + +@itemize +@item +Scale input to 720p, keeping aspect ratio and ensuring the output is yuv420p. +@example +vpp_amf=-2:720:format=yuv420p +@end example + +@item +Upscale to 4K and change color profile to bt2020. +@example +vpp_amf=4096:2160:color_profile=bt2020 +@end example +@end itemize + @anchor{vstack} @section vstack Stack input videos vertically. From patchwork Mon Oct 21 23:16:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitrii Ovchinnikov X-Patchwork-Id: 52440 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:143:b0:48e:c0f8:d0de with SMTP id h3csp3099758vqi; Mon, 21 Oct 2024 21:04:08 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCXE2ZzFFDqxXqbRYrefZ5EledCVgo85B63Dp24GWUgmS0ELQcVv+LibPhJ+IgaFDiP0dHhREuWOJcN4AqQofq10@gmail.com X-Google-Smtp-Source: AGHT+IGP1065jkrvKNbwMl8MCGiMm56gNeBCZbnKh1r8nAXYfJnCCWSJdLU6PaCu3ovNstg5psRN X-Received: by 2002:a17:907:3f8a:b0:a9a:3ca0:d55a with SMTP id a640c23a62f3a-a9aad371832mr84438766b.57.1729569847821; Mon, 21 Oct 2024 21:04:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1729569847; cv=none; d=google.com; s=arc-20240605; b=D5Grk3NTyRQnY9SkN1N1khZvpHC6AbRXPHLhGqKy+yh0z4ZnVSBr0J0GDO4goQN3p/ t4scGYi3TXecVPdpzRvPob4k/PvENDb9ULbIqkWqLcfb/gJjB+Iwe88Pa9MoEMwvNw3k KaK+7tSi6e+NHCP2tvWTM8hxcllWaFThaTKhF0qxaNVyuRDRgui27gu39phrMDarSvGX 2+y0wIx4zsj89SjNnG4E0s9dqLUMhZdiAIQbtCBNs7EK2fG7YaFFIBPWHfe+I34v3M2w VBjgNytWOlV/eB4qJV7cv8SXmjHbzPXhuCDxWPbbSauYezPaJy1Xnw7SZHZmIJL6+/+t wHyw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=IyDMElgLishgBszxxUwISgLlu2e9pgI+0AztO7ag868=; fh=GFMLWjWsOMgBmoPKQLjFbTnXINkN4pkSjsZqwDcYLuI=; b=LuR78/aEtYMekdUrTwGpY2/U1xW9Mzx0gG06ujtqEta16uS5r6Gl/QYPl4sjKn/5CI +02Tx3sVHdghEjbV/euwb7Rf6Cy0AgnRL4tz12S1qzovFhPHXRffrTvsJkDQGWrovQgA lCg68ewwt3oFRENaHFwi437vrO7IqykA5M4hmViUBBeDguWdVCD1UlPwO82zTen4mcu0 0kX4baiE/YNkSUNiZ6BGurFYxp1UAeZhb/eGrHANB62R0rQ4dKqAJNHEEb1a9xs59xq5 cJrqxPVRm8bJxxo6zUxT5fvb5h2XTL2ZWEvStZ3Q3Gaf9mxfRFh5FXAU1CHpRzK17Rqx T9vg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=AQtOlOvm; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a9a9134d26asi359178966b.555.2024.10.21.21.04.07; Mon, 21 Oct 2024 21:04:07 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=AQtOlOvm; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 35F7D68DE77; Tue, 22 Oct 2024 02:17:22 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f47.google.com (mail-wm1-f47.google.com [209.85.128.47]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3AFF468DE6D for ; Tue, 22 Oct 2024 02:17:20 +0300 (EEST) Received: by mail-wm1-f47.google.com with SMTP id 5b1f17b1804b1-4314c4cb752so53152445e9.2 for ; Mon, 21 Oct 2024 16:17:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729552639; x=1730157439; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=gGZF5mBnmBGjcWa165OFpRiwvBiEeemsKDDZbpLRRnQ=; b=AQtOlOvm2kZEugZ/CNIxM2sxByNQv3YEGSVbBsJCSeJzXVQgEffYqDeMKFyO57iwmE JD9KPQtv3asop0yJWePlppsw4B52884Hyt4n0VodntYLtOHHTp4UEsClysfk0i1cExH8 Ih0eVnxs1H7NnBII9Rh04qwYAKqxahDaYzBgVr/thfUH1GNZYR/mkwBzHEJkKUUZNCQY 7Qu9+dCMqtDXFLeKml8goMrtxuKfDW80QLQYeqdFvHLt1KT7ezMt7hm4LBIWhcYuEaiO mGrqTaBxP9xyl9FSMtGobS5B/r020yWujxGqlnYDCJURMNgoniBzApqfp5eQho/BbK7M lPSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729552639; x=1730157439; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gGZF5mBnmBGjcWa165OFpRiwvBiEeemsKDDZbpLRRnQ=; b=MH5i6sIu2tQKqcMbRfY0Zqcz9OaLZ9QZL0Kj05uwdqrfeOMxvbl4Ryuj4hGTEgm2LH E/dKHYzpJJLAg/NcCL434afH2MTK9PKizXHQ60EI9Sf+sTL2cmz1X0egVODHE4rm5Zop IhlMk0s+s227BOZKaiK94rtpYAfkQRsWoDntbsBYTA3KnRslCa7RKvmU7L9K0T5SBZTo ztETgrMW8x1H29dGQqdWdb4gBJGEWn2FqNbEwAbsXWkJMUThlwlI/fQ0rUuHx5Wjxca2 FPIYI9nds7sTjLKt9pFGKCqvziznKyincC/irWOmchx8DTCACGamJ+4uMJM0X0MVxY5m PNIA== X-Gm-Message-State: AOJu0Yzgp1zMEUV3M5hh6l68pb2WKbE1A6FtOp00U2NHtoIdYmhCXibU 8I01zeo3cBdvV5xDscuicq01LGDcEVraaChlBFZooO8c7Ci4E1DWUrImcA== X-Received: by 2002:a05:600c:45d2:b0:42c:acb0:dda5 with SMTP id 5b1f17b1804b1-4317ca980a6mr5133195e9.1.1729552638530; Mon, 21 Oct 2024 16:17:18 -0700 (PDT) Received: from dovchinn.amd.com (cable-178-148-16-149.dynamic.sbb.rs. [178.148.16.149]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-37ee0a57e0fsm5293937f8f.47.2024.10.21.16.17.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Oct 2024 16:17:16 -0700 (PDT) From: Dmitrii Ovchinnikov To: ffmpeg-devel@ffmpeg.org Date: Tue, 22 Oct 2024 01:16:36 +0200 Message-Id: <20241021231636.1199-5-ovchinnikov.dmitrii@gmail.com> X-Mailer: git-send-email 2.38.1.windows.1 In-Reply-To: <20241021231636.1199-1-ovchinnikov.dmitrii@gmail.com> References: <20241021231636.1199-1-ovchinnikov.dmitrii@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 5/5] avcodec/amfenc: redesign to use hwcontext_amf. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Evgeny Pavlov , Dmitrii Ovchinnikov Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: osrVBf4jkhLG Co-authored-by: Evgeny Pavlov v3: cleanup code --- libavcodec/amfenc.c | 520 +++++++++------------------------------ libavcodec/amfenc.h | 29 +-- libavcodec/amfenc_av1.c | 8 +- libavcodec/amfenc_h264.c | 6 +- libavcodec/amfenc_hevc.c | 6 +- 5 files changed, 132 insertions(+), 437 deletions(-) diff --git a/libavcodec/amfenc.c b/libavcodec/amfenc.c index 225fb9df27..8632cb53c1 100644 --- a/libavcodec/amfenc.c +++ b/libavcodec/amfenc.c @@ -22,6 +22,8 @@ #include "libavutil/avassert.h" #include "libavutil/imgutils.h" #include "libavutil/hwcontext.h" +#include "libavutil/hwcontext_amf.h" +#include "libavutil/hwcontext_amf_internal.h" #if CONFIG_D3D11VA #include "libavutil/hwcontext_d3d11va.h" #endif @@ -112,288 +114,18 @@ const enum AVPixelFormat ff_amf_pix_fmts[] = { AV_PIX_FMT_DXVA2_VLD, #endif AV_PIX_FMT_P010, + AV_PIX_FMT_AMF_SURFACE, AV_PIX_FMT_NONE }; -typedef struct FormatMap { - enum AVPixelFormat av_format; - enum AMF_SURFACE_FORMAT amf_format; -} FormatMap; - -static const FormatMap format_map[] = -{ - { AV_PIX_FMT_NONE, AMF_SURFACE_UNKNOWN }, - { AV_PIX_FMT_NV12, AMF_SURFACE_NV12 }, - { AV_PIX_FMT_P010, AMF_SURFACE_P010 }, - { AV_PIX_FMT_BGR0, AMF_SURFACE_BGRA }, - { AV_PIX_FMT_RGB0, AMF_SURFACE_RGBA }, - { AV_PIX_FMT_GRAY8, AMF_SURFACE_GRAY8 }, - { AV_PIX_FMT_YUV420P, AMF_SURFACE_YUV420P }, - { AV_PIX_FMT_YUYV422, AMF_SURFACE_YUY2 }, -}; - -static enum AMF_SURFACE_FORMAT amf_av_to_amf_format(enum AVPixelFormat fmt) -{ - int i; - for (i = 0; i < amf_countof(format_map); i++) { - if (format_map[i].av_format == fmt) { - return format_map[i].amf_format; - } - } - return AMF_SURFACE_UNKNOWN; -} - -static void AMF_CDECL_CALL AMFTraceWriter_Write(AMFTraceWriter *pThis, - const wchar_t *scope, const wchar_t *message) -{ - AmfTraceWriter *tracer = (AmfTraceWriter*)pThis; - av_log(tracer->avctx, AV_LOG_DEBUG, "%ls: %ls", scope, message); // \n is provided from AMF -} - -static void AMF_CDECL_CALL AMFTraceWriter_Flush(AMFTraceWriter *pThis) -{ -} - -static AMFTraceWriterVtbl tracer_vtbl = -{ - .Write = AMFTraceWriter_Write, - .Flush = AMFTraceWriter_Flush, -}; - -static int amf_load_library(AVCodecContext *avctx) -{ - AmfContext *ctx = avctx->priv_data; - AMFInit_Fn init_fun; - AMFQueryVersion_Fn version_fun; - AMF_RESULT res; - - ctx->delayed_frame = av_frame_alloc(); - if (!ctx->delayed_frame) { - return AVERROR(ENOMEM); - } - // hardcoded to current HW queue size - will auto-realloc if too small - ctx->timestamp_list = av_fifo_alloc2(avctx->max_b_frames + 16, sizeof(int64_t), - AV_FIFO_FLAG_AUTO_GROW); - if (!ctx->timestamp_list) { - return AVERROR(ENOMEM); - } - ctx->dts_delay = 0; - - - ctx->library = dlopen(AMF_DLL_NAMEA, RTLD_NOW | RTLD_LOCAL); - AMF_RETURN_IF_FALSE(ctx, ctx->library != NULL, - AVERROR_UNKNOWN, "DLL %s failed to open\n", AMF_DLL_NAMEA); - - init_fun = (AMFInit_Fn)dlsym(ctx->library, AMF_INIT_FUNCTION_NAME); - AMF_RETURN_IF_FALSE(ctx, init_fun != NULL, AVERROR_UNKNOWN, "DLL %s failed to find function %s\n", AMF_DLL_NAMEA, AMF_INIT_FUNCTION_NAME); - - version_fun = (AMFQueryVersion_Fn)dlsym(ctx->library, AMF_QUERY_VERSION_FUNCTION_NAME); - AMF_RETURN_IF_FALSE(ctx, version_fun != NULL, AVERROR_UNKNOWN, "DLL %s failed to find function %s\n", AMF_DLL_NAMEA, AMF_QUERY_VERSION_FUNCTION_NAME); - - res = version_fun(&ctx->version); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "%s failed with error %d\n", AMF_QUERY_VERSION_FUNCTION_NAME, res); - res = init_fun(AMF_FULL_VERSION, &ctx->factory); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "%s failed with error %d\n", AMF_INIT_FUNCTION_NAME, res); - res = ctx->factory->pVtbl->GetTrace(ctx->factory, &ctx->trace); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "GetTrace() failed with error %d\n", res); - res = ctx->factory->pVtbl->GetDebug(ctx->factory, &ctx->debug); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "GetDebug() failed with error %d\n", res); - return 0; -} - -#if CONFIG_D3D11VA -static int amf_init_from_d3d11_device(AVCodecContext *avctx, AVD3D11VADeviceContext *hwctx) -{ - AmfContext *ctx = avctx->priv_data; - AMF_RESULT res; - - res = ctx->context->pVtbl->InitDX11(ctx->context, hwctx->device, AMF_DX11_1); - if (res != AMF_OK) { - if (res == AMF_NOT_SUPPORTED) - av_log(avctx, AV_LOG_ERROR, "AMF via D3D11 is not supported on the given device.\n"); - else - av_log(avctx, AV_LOG_ERROR, "AMF failed to initialise on the given D3D11 device: %d.\n", res); - return AVERROR(ENODEV); - } - - return 0; -} -#endif - -#if CONFIG_DXVA2 -static int amf_init_from_dxva2_device(AVCodecContext *avctx, AVDXVA2DeviceContext *hwctx) -{ - AmfContext *ctx = avctx->priv_data; - HANDLE device_handle; - IDirect3DDevice9 *device; - HRESULT hr; - AMF_RESULT res; - int ret; - - hr = IDirect3DDeviceManager9_OpenDeviceHandle(hwctx->devmgr, &device_handle); - if (FAILED(hr)) { - av_log(avctx, AV_LOG_ERROR, "Failed to open device handle for Direct3D9 device: %lx.\n", (unsigned long)hr); - return AVERROR_EXTERNAL; - } - - hr = IDirect3DDeviceManager9_LockDevice(hwctx->devmgr, device_handle, &device, FALSE); - if (SUCCEEDED(hr)) { - IDirect3DDeviceManager9_UnlockDevice(hwctx->devmgr, device_handle, FALSE); - ret = 0; - } else { - av_log(avctx, AV_LOG_ERROR, "Failed to lock device handle for Direct3D9 device: %lx.\n", (unsigned long)hr); - ret = AVERROR_EXTERNAL; - } - - IDirect3DDeviceManager9_CloseDeviceHandle(hwctx->devmgr, device_handle); - - if (ret < 0) - return ret; - - res = ctx->context->pVtbl->InitDX9(ctx->context, device); - - IDirect3DDevice9_Release(device); - - if (res != AMF_OK) { - if (res == AMF_NOT_SUPPORTED) - av_log(avctx, AV_LOG_ERROR, "AMF via D3D9 is not supported on the given device.\n"); - else - av_log(avctx, AV_LOG_ERROR, "AMF failed to initialise on given D3D9 device: %d.\n", res); - return AVERROR(ENODEV); - } - - return 0; -} -#endif - -static int amf_init_context(AVCodecContext *avctx) -{ - AmfContext *ctx = avctx->priv_data; - AMFContext1 *context1 = NULL; - AMF_RESULT res; - av_unused int ret; - - ctx->hwsurfaces_in_queue = 0; - ctx->hwsurfaces_in_queue_max = 16; - - // configure AMF logger - // the return of these functions indicates old state and do not affect behaviour - ctx->trace->pVtbl->EnableWriter(ctx->trace, AMF_TRACE_WRITER_DEBUG_OUTPUT, ctx->log_to_dbg != 0 ); - if (ctx->log_to_dbg) - ctx->trace->pVtbl->SetWriterLevel(ctx->trace, AMF_TRACE_WRITER_DEBUG_OUTPUT, AMF_TRACE_TRACE); - ctx->trace->pVtbl->EnableWriter(ctx->trace, AMF_TRACE_WRITER_CONSOLE, 0); - ctx->trace->pVtbl->SetGlobalLevel(ctx->trace, AMF_TRACE_TRACE); - - // connect AMF logger to av_log - ctx->tracer.vtbl = &tracer_vtbl; - ctx->tracer.avctx = avctx; - ctx->trace->pVtbl->RegisterWriter(ctx->trace, FFMPEG_AMF_WRITER_ID,(AMFTraceWriter*)&ctx->tracer, 1); - ctx->trace->pVtbl->SetWriterLevel(ctx->trace, FFMPEG_AMF_WRITER_ID, AMF_TRACE_TRACE); - - res = ctx->factory->pVtbl->CreateContext(ctx->factory, &ctx->context); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "CreateContext() failed with error %d\n", res); - - // If a device was passed to the encoder, try to initialise from that. - if (avctx->hw_frames_ctx) { - AVHWFramesContext *frames_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; - - if (amf_av_to_amf_format(frames_ctx->sw_format) == AMF_SURFACE_UNKNOWN) { - av_log(avctx, AV_LOG_ERROR, "Format of input frames context (%s) is not supported by AMF.\n", - av_get_pix_fmt_name(frames_ctx->sw_format)); - return AVERROR(EINVAL); - } - - switch (frames_ctx->device_ctx->type) { -#if CONFIG_D3D11VA - case AV_HWDEVICE_TYPE_D3D11VA: - ret = amf_init_from_d3d11_device(avctx, frames_ctx->device_ctx->hwctx); - if (ret < 0) - return ret; - break; -#endif -#if CONFIG_DXVA2 - case AV_HWDEVICE_TYPE_DXVA2: - ret = amf_init_from_dxva2_device(avctx, frames_ctx->device_ctx->hwctx); - if (ret < 0) - return ret; - break; -#endif - default: - av_log(avctx, AV_LOG_ERROR, "AMF initialisation from a %s frames context is not supported.\n", - av_hwdevice_get_type_name(frames_ctx->device_ctx->type)); - return AVERROR(ENOSYS); - } - - ctx->hw_frames_ctx = av_buffer_ref(avctx->hw_frames_ctx); - if (!ctx->hw_frames_ctx) - return AVERROR(ENOMEM); - - if (frames_ctx->initial_pool_size > 0) - ctx->hwsurfaces_in_queue_max = frames_ctx->initial_pool_size - 1; - - } else if (avctx->hw_device_ctx) { - AVHWDeviceContext *device_ctx = (AVHWDeviceContext*)avctx->hw_device_ctx->data; - - switch (device_ctx->type) { -#if CONFIG_D3D11VA - case AV_HWDEVICE_TYPE_D3D11VA: - ret = amf_init_from_d3d11_device(avctx, device_ctx->hwctx); - if (ret < 0) - return ret; - break; -#endif -#if CONFIG_DXVA2 - case AV_HWDEVICE_TYPE_DXVA2: - ret = amf_init_from_dxva2_device(avctx, device_ctx->hwctx); - if (ret < 0) - return ret; - break; -#endif - default: - av_log(avctx, AV_LOG_ERROR, "AMF initialisation from a %s device is not supported.\n", - av_hwdevice_get_type_name(device_ctx->type)); - return AVERROR(ENOSYS); - } - - ctx->hw_device_ctx = av_buffer_ref(avctx->hw_device_ctx); - if (!ctx->hw_device_ctx) - return AVERROR(ENOMEM); - - } else { - res = ctx->context->pVtbl->InitDX11(ctx->context, NULL, AMF_DX11_1); - if (res == AMF_OK) { - av_log(avctx, AV_LOG_VERBOSE, "AMF initialisation succeeded via D3D11.\n"); - } else { - res = ctx->context->pVtbl->InitDX9(ctx->context, NULL); - if (res == AMF_OK) { - av_log(avctx, AV_LOG_VERBOSE, "AMF initialisation succeeded via D3D9.\n"); - } else { - AMFGuid guid = IID_AMFContext1(); - res = ctx->context->pVtbl->QueryInterface(ctx->context, &guid, (void**)&context1); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "CreateContext1() failed with error %d\n", res); - - res = context1->pVtbl->InitVulkan(context1, NULL); - context1->pVtbl->Release(context1); - if (res != AMF_OK) { - if (res == AMF_NOT_SUPPORTED) - av_log(avctx, AV_LOG_ERROR, "AMF via Vulkan is not supported on the given device.\n"); - else - av_log(avctx, AV_LOG_ERROR, "AMF failed to initialise on the given Vulkan device: %d.\n", res); - return AVERROR(ENOSYS); - } - av_log(avctx, AV_LOG_VERBOSE, "AMF initialisation succeeded via Vulkan.\n"); - } - } - } - return 0; -} - static int amf_init_encoder(AVCodecContext *avctx) { - AmfContext *ctx = avctx->priv_data; - const wchar_t *codec_id = NULL; - AMF_RESULT res; - enum AVPixelFormat pix_fmt; + AMFEncoderContext *ctx = avctx->priv_data; + const wchar_t *codec_id = NULL; + AMF_RESULT res; + enum AVPixelFormat pix_fmt; + AVHWDeviceContext *hw_device_ctx = (AVHWDeviceContext*)ctx->device_ctx_ref->data; + AVAMFDeviceContext *amf_device_ctx = (AVAMFDeviceContext *)hw_device_ctx->hwctx; switch (avctx->codec->id) { case AV_CODEC_ID_H264: @@ -410,20 +142,20 @@ static int amf_init_encoder(AVCodecContext *avctx) } AMF_RETURN_IF_FALSE(ctx, codec_id != NULL, AVERROR(EINVAL), "Codec %d is not supported\n", avctx->codec->id); - if (ctx->hw_frames_ctx) - pix_fmt = ((AVHWFramesContext*)ctx->hw_frames_ctx->data)->sw_format; + if (avctx->hw_frames_ctx) + pix_fmt = ((AVHWFramesContext*)avctx->hw_frames_ctx->data)->sw_format; else pix_fmt = avctx->pix_fmt; if (pix_fmt == AV_PIX_FMT_P010) { - AMF_RETURN_IF_FALSE(ctx, ctx->version >= AMF_MAKE_FULL_VERSION(1, 4, 32, 0), AVERROR_UNKNOWN, "10-bit encoder is not supported by AMD GPU drivers versions lower than 23.30.\n"); + AMF_RETURN_IF_FALSE(ctx, amf_device_ctx->version >= AMF_MAKE_FULL_VERSION(1, 4, 32, 0), AVERROR_UNKNOWN, "10-bit encoder is not supported by AMD GPU drivers versions lower than 23.30.\n"); } - ctx->format = amf_av_to_amf_format(pix_fmt); + ctx->format = av_av_to_amf_format(pix_fmt); AMF_RETURN_IF_FALSE(ctx, ctx->format != AMF_SURFACE_UNKNOWN, AVERROR(EINVAL), "Format %s is not supported\n", av_get_pix_fmt_name(pix_fmt)); - res = ctx->factory->pVtbl->CreateComponent(ctx->factory, ctx->context, codec_id, &ctx->encoder); + res = amf_device_ctx->factory->pVtbl->CreateComponent(amf_device_ctx->factory, amf_device_ctx->context, codec_id, &ctx->encoder); AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_ENCODER_NOT_FOUND, "CreateComponent(%ls) failed with error %d\n", codec_id, res); return 0; @@ -431,12 +163,7 @@ static int amf_init_encoder(AVCodecContext *avctx) int av_cold ff_amf_encode_close(AVCodecContext *avctx) { - AmfContext *ctx = avctx->priv_data; - - if (ctx->delayed_surface) { - ctx->delayed_surface->pVtbl->Release(ctx->delayed_surface); - ctx->delayed_surface = NULL; - } + AMFEncoderContext *ctx = avctx->priv_data; if (ctx->encoder) { ctx->encoder->pVtbl->Terminate(ctx->encoder); @@ -444,27 +171,7 @@ int av_cold ff_amf_encode_close(AVCodecContext *avctx) ctx->encoder = NULL; } - if (ctx->context) { - ctx->context->pVtbl->Terminate(ctx->context); - ctx->context->pVtbl->Release(ctx->context); - ctx->context = NULL; - } - av_buffer_unref(&ctx->hw_device_ctx); - av_buffer_unref(&ctx->hw_frames_ctx); - - if (ctx->trace) { - ctx->trace->pVtbl->UnregisterWriter(ctx->trace, FFMPEG_AMF_WRITER_ID); - } - if (ctx->library) { - dlclose(ctx->library); - ctx->library = NULL; - } - ctx->trace = NULL; - ctx->debug = NULL; - ctx->factory = NULL; - ctx->version = 0; - ctx->delayed_drain = 0; - av_frame_free(&ctx->delayed_frame); + av_buffer_unref(&ctx->device_ctx_ref); av_fifo_freep2(&ctx->timestamp_list); return 0; @@ -474,8 +181,8 @@ static int amf_copy_surface(AVCodecContext *avctx, const AVFrame *frame, AMFSurface* surface) { AMFPlane *plane; - uint8_t *dst_data[4]; - int dst_linesize[4]; + uint8_t *dst_data[4] = {0}; + int dst_linesize[4] = {0}; int planes; int i; @@ -496,7 +203,7 @@ static int amf_copy_surface(AVCodecContext *avctx, const AVFrame *frame, static int amf_copy_buffer(AVCodecContext *avctx, AVPacket *pkt, AMFBuffer *buffer) { - AmfContext *ctx = avctx->priv_data; + AMFEncoderContext *ctx = avctx->priv_data; int ret; AMFVariantStruct var = {0}; int64_t timestamp = AV_NOPTS_VALUE; @@ -558,14 +265,51 @@ static int amf_copy_buffer(AVCodecContext *avctx, AVPacket *pkt, AMFBuffer *buff int ff_amf_encode_init(AVCodecContext *avctx) { int ret; + AMFEncoderContext *ctx = avctx->priv_data; - if ((ret = amf_load_library(avctx)) == 0) { - if ((ret = amf_init_context(avctx)) == 0) { - if ((ret = amf_init_encoder(avctx)) == 0) { - return 0; + // hardcoded to current HW queue size - will auto-realloc if too small + ctx->timestamp_list = av_fifo_alloc2(avctx->max_b_frames + 16, sizeof(int64_t), + AV_FIFO_FLAG_AUTO_GROW); + if (!ctx->timestamp_list) { + return AVERROR(ENOMEM); + } + ctx->dts_delay = 0; + + ctx->hwsurfaces_in_queue = 0; + ctx->hwsurfaces_in_queue_max = 16; + + AVHWDeviceContext *hwdev_ctx = NULL; + if (avctx->hw_device_ctx) { + hwdev_ctx = (AVHWDeviceContext*)avctx->hw_device_ctx->data; + if (hwdev_ctx->type == AV_HWDEVICE_TYPE_AMF) + { + ctx->device_ctx_ref = av_buffer_ref(avctx->hw_device_ctx); + } + else { + ret = av_hwdevice_ctx_create_derived(&ctx->device_ctx_ref, AV_HWDEVICE_TYPE_AMF, avctx->hw_device_ctx, 0); + AMF_RETURN_IF_FALSE(avctx, ret == 0, ret, "Failed to create derived AMF device context: %s\n", av_err2str(ret)); + } + } else if (avctx->hw_frames_ctx) { + AVHWFramesContext *frames_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; + if (frames_ctx->device_ref ) { + if (frames_ctx->format == AV_PIX_FMT_AMF_SURFACE) { + ctx->device_ctx_ref = av_buffer_ref(frames_ctx->device_ref); + } + else { + ret = av_hwdevice_ctx_create_derived(&ctx->device_ctx_ref, AV_HWDEVICE_TYPE_AMF, frames_ctx->device_ref, 0); + AMF_RETURN_IF_FALSE(avctx, ret == 0, ret, "Failed to create derived AMF device context: %s\n", av_err2str(ret)); } } } + else { + ret = av_hwdevice_ctx_create(&ctx->device_ctx_ref, AV_HWDEVICE_TYPE_AMF, NULL, NULL, 0); + AMF_RETURN_IF_FALSE(avctx, ret == 0, ret, "Failed to create hardware device context (AMF) : %s\n", av_err2str(ret)); + } + + if ((ret = amf_init_encoder(avctx)) == 0) { + return 0; + } + ff_amf_encode_close(avctx); return ret; } @@ -642,31 +386,27 @@ static void amf_release_buffer_with_frame_ref(AMFBuffer *frame_ref_storage_buffe int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) { - AmfContext *ctx = avctx->priv_data; + AMFEncoderContext *ctx = avctx->priv_data; + AVHWDeviceContext *hw_device_ctx = (AVHWDeviceContext*)ctx->device_ctx_ref->data; + AVAMFDeviceContext *amf_device_ctx = (AVAMFDeviceContext *)hw_device_ctx->hwctx; AMFSurface *surface; AMF_RESULT res; int ret; AMF_RESULT res_query; AMFData *data = NULL; - AVFrame *frame = ctx->delayed_frame; + AVFrame *frame = av_frame_alloc(); int block_and_wait; - int query_output_data_flag = 0; - AMF_RESULT res_resubmit; if (!ctx->encoder) return AVERROR(EINVAL); - if (!frame->buf[0]) { - ret = ff_encode_get_frame(avctx, frame); - if (ret < 0 && ret != AVERROR_EOF) - return ret; - } + ret = ff_encode_get_frame(avctx, frame); + if (ret < 0 && ret != AVERROR_EOF) + return ret; if (!frame->buf[0]) { // submit drain if (!ctx->eof) { // submit drain one time only - if (ctx->delayed_surface != NULL) { - ctx->delayed_drain = 1; // input queue is full: resubmit Drain() in ff_amf_receive_packet - } else if(!ctx->delayed_drain) { + if(!ctx->delayed_drain) { res = ctx->encoder->pVtbl->Drain(ctx->encoder); if (res == AMF_INPUT_FULL) { ctx->delayed_drain = 1; // input queue is full: resubmit Drain() in ff_amf_receive_packet @@ -678,7 +418,7 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) } } } - } else if (!ctx->delayed_surface) { // submit frame + } else { // submit frame int hw_surface = 0; // prepare surface from frame @@ -690,12 +430,12 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) ID3D11Texture2D *texture = (ID3D11Texture2D*)frame->data[0]; // actual texture int index = (intptr_t)frame->data[1]; // index is a slice in texture array is - set to tell AMF which slice to use - av_assert0(frame->hw_frames_ctx && ctx->hw_frames_ctx && - frame->hw_frames_ctx->data == ctx->hw_frames_ctx->data); + av_assert0(frame->hw_frames_ctx && avctx->hw_frames_ctx && + frame->hw_frames_ctx->data == avctx->hw_frames_ctx->data); texture->lpVtbl->SetPrivateData(texture, &AMFTextureArrayIndexGUID, sizeof(index), &index); - res = ctx->context->pVtbl->CreateSurfaceFromDX11Native(ctx->context, texture, &surface, NULL); // wrap to AMF surface + res = amf_device_ctx->context->pVtbl->CreateSurfaceFromDX11Native(amf_device_ctx->context, texture, &surface, NULL); // wrap to AMF surface AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR(ENOMEM), "CreateSurfaceFromDX11Native() failed with error %d\n", res); hw_surface = 1; @@ -707,16 +447,23 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) { IDirect3DSurface9 *texture = (IDirect3DSurface9 *)frame->data[3]; // actual texture - res = ctx->context->pVtbl->CreateSurfaceFromDX9Native(ctx->context, texture, &surface, NULL); // wrap to AMF surface + res = amf_device_ctx->context->pVtbl->CreateSurfaceFromDX9Native(amf_device_ctx->context, texture, &surface, NULL); // wrap to AMF surface AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR(ENOMEM), "CreateSurfaceFromDX9Native() failed with error %d\n", res); hw_surface = 1; } break; #endif + case AV_PIX_FMT_AMF_SURFACE: + { + surface = (AMFSurface*)frame->data[0]; + surface->pVtbl->Acquire(surface); + hw_surface = 1; + } + break; default: { - res = ctx->context->pVtbl->AllocSurface(ctx->context, AMF_MEMORY_HOST, ctx->format, avctx->width, avctx->height, &surface); + res = amf_device_ctx->context->pVtbl->AllocSurface(amf_device_ctx->context, AMF_MEMORY_HOST, ctx->format, avctx->width, avctx->height, &surface); AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR(ENOMEM), "AllocSurface() failed with error %d\n", res); amf_copy_surface(avctx, frame, surface); } @@ -729,7 +476,7 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) // input HW surfaces can be vertically aligned by 16; tell AMF the real size surface->pVtbl->SetCrop(surface, 0, 0, frame->width, frame->height); - frame_ref_storage_buffer = amf_create_buffer_with_frame_ref(frame, ctx->context); + frame_ref_storage_buffer = amf_create_buffer_with_frame_ref(frame, amf_device_ctx->context); AMF_RETURN_IF_FALSE(ctx, frame_ref_storage_buffer != NULL, AVERROR(ENOMEM), "create_buffer_with_frame_ref() returned NULL\n"); res = amf_set_property_buffer(surface, L"av_frame_ref", frame_ref_storage_buffer); @@ -741,7 +488,7 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) // HDR10 metadata if (frame->color_trc == AVCOL_TRC_SMPTE2084) { AMFBuffer * hdrmeta_buffer = NULL; - res = ctx->context->pVtbl->AllocBuffer(ctx->context, AMF_MEMORY_HOST, sizeof(AMFHDRMetadata), &hdrmeta_buffer); + res = amf_device_ctx->context->pVtbl->AllocBuffer(amf_device_ctx->context, AMF_MEMORY_HOST, sizeof(AMFHDRMetadata), &hdrmeta_buffer); if (res == AMF_OK) { AMFHDRMetadata * hdrmeta = (AMFHDRMetadata*)hdrmeta_buffer->pVtbl->GetNative(hdrmeta_buffer); if (amf_save_hdr_metadata(avctx, frame, hdrmeta) == 0) { @@ -818,7 +565,7 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) res = ctx->encoder->pVtbl->SubmitInput(ctx->encoder, (AMFData*)surface); if (res == AMF_INPUT_FULL) { // handle full queue //store surface for later submission - ctx->delayed_surface = surface; + av_usleep(1000); // wait and poll again } else { int64_t pts = frame->pts; surface->pVtbl->Release(surface); @@ -835,75 +582,42 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) do { block_and_wait = 0; // poll data - if (!avpkt->data && !avpkt->buf) { - res_query = ctx->encoder->pVtbl->QueryOutput(ctx->encoder, &data); - if (data) { - // copy data to packet - AMFBuffer *buffer; - AMFGuid guid = IID_AMFBuffer(); - query_output_data_flag = 1; - data->pVtbl->QueryInterface(data, &guid, (void**)&buffer); // query for buffer interface - ret = amf_copy_buffer(avctx, avpkt, buffer); - - buffer->pVtbl->Release(buffer); - - if (data->pVtbl->HasProperty(data, L"av_frame_ref")) { - AMFBuffer* frame_ref_storage_buffer; - res = amf_get_property_buffer(data, L"av_frame_ref", &frame_ref_storage_buffer); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "GetProperty failed for \"av_frame_ref\" with error %d\n", res); - amf_release_buffer_with_frame_ref(frame_ref_storage_buffer); - ctx->hwsurfaces_in_queue--; - } - - data->pVtbl->Release(data); - AMF_RETURN_IF_FALSE(ctx, ret >= 0, ret, "amf_copy_buffer() failed with error %d\n", ret); + res_query = ctx->encoder->pVtbl->QueryOutput(ctx->encoder, &data); + if (data) { + // copy data to packet + AMFBuffer *buffer; + AMFGuid guid = IID_AMFBuffer(); + data->pVtbl->QueryInterface(data, &guid, (void**)&buffer); // query for buffer interface + ret = amf_copy_buffer(avctx, avpkt, buffer); + + buffer->pVtbl->Release(buffer); + + if (data->pVtbl->HasProperty(data, L"av_frame_ref")) { + AMFBuffer* frame_ref_storage_buffer; + res = amf_get_property_buffer(data, L"av_frame_ref", &frame_ref_storage_buffer); + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "GetProperty failed for \"av_frame_ref\" with error %d\n", res); + amf_release_buffer_with_frame_ref(frame_ref_storage_buffer); + ctx->hwsurfaces_in_queue--; } - } - res_resubmit = AMF_OK; - if (ctx->delayed_surface != NULL) { // try to resubmit frame - if (ctx->delayed_surface->pVtbl->HasProperty(ctx->delayed_surface, L"av_frame_hdrmeta")) { - AMFBuffer * hdrmeta_buffer = NULL; - res = amf_get_property_buffer((AMFData *)ctx->delayed_surface, L"av_frame_hdrmeta", &hdrmeta_buffer); - AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "GetProperty failed for \"av_frame_hdrmeta\" with error %d\n", res); - switch (avctx->codec->id) { - case AV_CODEC_ID_H264: - AMF_ASSIGN_PROPERTY_INTERFACE(res, ctx->encoder, AMF_VIDEO_ENCODER_INPUT_HDR_METADATA, hdrmeta_buffer); break; - case AV_CODEC_ID_HEVC: - AMF_ASSIGN_PROPERTY_INTERFACE(res, ctx->encoder, AMF_VIDEO_ENCODER_HEVC_INPUT_HDR_METADATA, hdrmeta_buffer); break; - case AV_CODEC_ID_AV1: - AMF_ASSIGN_PROPERTY_INTERFACE(res, ctx->encoder, AMF_VIDEO_ENCODER_AV1_INPUT_HDR_METADATA, hdrmeta_buffer); break; - } - hdrmeta_buffer->pVtbl->Release(hdrmeta_buffer); - } - res_resubmit = ctx->encoder->pVtbl->SubmitInput(ctx->encoder, (AMFData*)ctx->delayed_surface); - if (res_resubmit != AMF_INPUT_FULL) { - int64_t pts = ctx->delayed_surface->pVtbl->GetPts(ctx->delayed_surface); - ctx->delayed_surface->pVtbl->Release(ctx->delayed_surface); - ctx->delayed_surface = NULL; - av_frame_unref(ctx->delayed_frame); - AMF_RETURN_IF_FALSE(ctx, res_resubmit == AMF_OK, AVERROR_UNKNOWN, "Repeated SubmitInput() failed with error %d\n", res_resubmit); - - ret = av_fifo_write(ctx->timestamp_list, &pts, 1); - if (ret < 0) - return ret; - } - } else if (ctx->delayed_drain) { // try to resubmit drain - res = ctx->encoder->pVtbl->Drain(ctx->encoder); - if (res != AMF_INPUT_FULL) { - ctx->delayed_drain = 0; - ctx->eof = 1; // drain started - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "Repeated Drain() failed with error %d\n", res); - } else { - av_log(avctx, AV_LOG_WARNING, "Data acquired but delayed drain submission got AMF_INPUT_FULL- should not happen\n"); - } - } - if (query_output_data_flag == 0) { - if (res_resubmit == AMF_INPUT_FULL || ctx->delayed_drain || (ctx->eof && res_query != AMF_EOF) || (ctx->hwsurfaces_in_queue >= ctx->hwsurfaces_in_queue_max)) { - block_and_wait = 1; - av_usleep(1000); + data->pVtbl->Release(data); + + AMF_RETURN_IF_FALSE(ctx, ret >= 0, ret, "amf_copy_buffer() failed with error %d\n", ret); + + if (ctx->delayed_drain) { // try to resubmit drain + res = ctx->encoder->pVtbl->Drain(ctx->encoder); + if (res != AMF_INPUT_FULL) { + ctx->delayed_drain = 0; + ctx->eof = 1; // drain started + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "Repeated Drain() failed with error %d\n", res); + } else { + av_log(avctx, AV_LOG_WARNING, "Data acquired but delayed drain submission got AMF_INPUT_FULL- should not happen\n"); + } } + } else if (ctx->delayed_drain || (ctx->eof && res_query != AMF_EOF) || (ctx->hwsurfaces_in_queue >= ctx->hwsurfaces_in_queue_max)) { + block_and_wait = 1; + av_usleep(1000); // wait and poll again } } while (block_and_wait); @@ -961,5 +675,7 @@ const AVCodecHWConfigInternal *const ff_amfenc_hw_configs[] = { HW_CONFIG_ENCODER_FRAMES(DXVA2_VLD, DXVA2), HW_CONFIG_ENCODER_DEVICE(NONE, DXVA2), #endif + HW_CONFIG_ENCODER_FRAMES(AMF_SURFACE, AMF), + HW_CONFIG_ENCODER_DEVICE(NONE, AMF), NULL, }; diff --git a/libavcodec/amfenc.h b/libavcodec/amfenc.h index 0f2abcbd82..5063d6c0d4 100644 --- a/libavcodec/amfenc.h +++ b/libavcodec/amfenc.h @@ -33,46 +33,25 @@ #define MAX_LOOKAHEAD_DEPTH 41 -/** -* AMF trace writer callback class -* Used to capture all AMF logging -*/ - -typedef struct AmfTraceWriter { - AMFTraceWriterVtbl *vtbl; - AVCodecContext *avctx; -} AmfTraceWriter; - /** * AMF encoder context */ -typedef struct AmfContext { +typedef struct AMFEncoderContext { AVClass *avclass; // access to AMF runtime - amf_handle library; ///< handle to DLL library - AMFFactory *factory; ///< pointer to AMF factory - AMFDebug *debug; ///< pointer to AMF debug interface - AMFTrace *trace; ///< pointer to AMF trace interface - - amf_uint64 version; ///< version of AMF runtime - AmfTraceWriter tracer; ///< AMF writer registered with AMF - AMFContext *context; ///< AMF context + AVBufferRef *device_ctx_ref; + //encoder AMFComponent *encoder; ///< AMF encoder object amf_bool eof; ///< flag indicating EOF happened AMF_SURFACE_FORMAT format; ///< AMF surface format - AVBufferRef *hw_device_ctx; ///< pointer to HW accelerator (decoder) - AVBufferRef *hw_frames_ctx; ///< pointer to HW accelerator (frame allocator) - int hwsurfaces_in_queue; int hwsurfaces_in_queue_max; // helpers to handle async calls int delayed_drain; - AMFSurface *delayed_surface; - AVFrame *delayed_frame; // shift dts back by max_b_frames in timing AVFifo *timestamp_list; @@ -151,7 +130,7 @@ typedef struct AmfContext { int pa_adaptive_mini_gop; -} AmfContext; +} AMFEncoderContext; extern const AVCodecHWConfigInternal *const ff_amfenc_hw_configs[]; diff --git a/libavcodec/amfenc_av1.c b/libavcodec/amfenc_av1.c index b40d54f70c..fad4733a40 100644 --- a/libavcodec/amfenc_av1.c +++ b/libavcodec/amfenc_av1.c @@ -26,7 +26,7 @@ #define AMF_VIDEO_ENCODER_AV1_CAP_WIDTH_ALIGNMENT_FACTOR_LOCAL L"Av1WidthAlignmentFactor" // amf_int64; default = 1 #define AMF_VIDEO_ENCODER_AV1_CAP_HEIGHT_ALIGNMENT_FACTOR_LOCAL L"Av1HeightAlignmentFactor" // amf_int64; default = 1 -#define OFFSET(x) offsetof(AmfContext, x) +#define OFFSET(x) offsetof(AMFEncoderContext, x) #define VE AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM static const AVOption options[] = { @@ -178,11 +178,11 @@ static av_cold int amf_encode_init_av1(AVCodecContext* avctx) { int ret = 0; AMF_RESULT res = AMF_OK; - AmfContext* ctx = avctx->priv_data; + AMFEncoderContext *ctx = avctx->priv_data; AMFVariantStruct var = { 0 }; amf_int64 profile = 0; amf_int64 profile_level = 0; - AMFBuffer* buffer; + AMFBuffer *buffer; AMFGuid guid; AMFRate framerate; AMFSize framesize = AMFConstructSize(avctx->width, avctx->height); @@ -623,7 +623,7 @@ const FFCodec ff_av1_amf_encoder = { .init = amf_encode_init_av1, FF_CODEC_RECEIVE_PACKET_CB(ff_amf_receive_packet), .close = ff_amf_encode_close, - .priv_data_size = sizeof(AmfContext), + .priv_data_size = sizeof(AMFEncoderContext), .p.priv_class = &av1_amf_class, .defaults = defaults, .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_HARDWARE | diff --git a/libavcodec/amfenc_h264.c b/libavcodec/amfenc_h264.c index 959be9eab6..f82edd3708 100644 --- a/libavcodec/amfenc_h264.c +++ b/libavcodec/amfenc_h264.c @@ -24,7 +24,7 @@ #include "codec_internal.h" #include -#define OFFSET(x) offsetof(AmfContext, x) +#define OFFSET(x) offsetof(AMFEncoderContext, x) #define VE AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM static const AVOption options[] = { @@ -193,7 +193,7 @@ static av_cold int amf_encode_init_h264(AVCodecContext *avctx) { int ret = 0; AMF_RESULT res = AMF_OK; - AmfContext *ctx = avctx->priv_data; + AMFEncoderContext *ctx = avctx->priv_data; AMFVariantStruct var = { 0 }; amf_int64 profile = 0; amf_int64 profile_level = 0; @@ -595,7 +595,7 @@ const FFCodec ff_h264_amf_encoder = { .init = amf_encode_init_h264, FF_CODEC_RECEIVE_PACKET_CB(ff_amf_receive_packet), .close = ff_amf_encode_close, - .priv_data_size = sizeof(AmfContext), + .priv_data_size = sizeof(AMFEncoderContext), .p.priv_class = &h264_amf_class, .defaults = defaults, .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_HARDWARE | diff --git a/libavcodec/amfenc_hevc.c b/libavcodec/amfenc_hevc.c index f9f6f8adb3..77b331c400 100644 --- a/libavcodec/amfenc_hevc.c +++ b/libavcodec/amfenc_hevc.c @@ -23,7 +23,7 @@ #include "codec_internal.h" #include -#define OFFSET(x) offsetof(AmfContext, x) +#define OFFSET(x) offsetof(AMFEncoderContext, x) #define VE AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM static const AVOption options[] = { @@ -158,7 +158,7 @@ static av_cold int amf_encode_init_hevc(AVCodecContext *avctx) { int ret = 0; AMF_RESULT res = AMF_OK; - AmfContext *ctx = avctx->priv_data; + AMFEncoderContext *ctx = avctx->priv_data; AMFVariantStruct var = {0}; amf_int64 profile = 0; amf_int64 profile_level = 0; @@ -530,7 +530,7 @@ const FFCodec ff_hevc_amf_encoder = { .init = amf_encode_init_hevc, FF_CODEC_RECEIVE_PACKET_CB(ff_amf_receive_packet), .close = ff_amf_encode_close, - .priv_data_size = sizeof(AmfContext), + .priv_data_size = sizeof(AMFEncoderContext), .p.priv_class = &hevc_amf_class, .defaults = defaults, .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_HARDWARE |