From patchwork Thu Oct 15 00:16:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitrii Ovchinnikov X-Patchwork-Id: 22961 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id E7A7E44A83F for ; Thu, 15 Oct 2020 03:22:21 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id CB42C68BB03; Thu, 15 Oct 2020 03:22:21 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lj1-f177.google.com (mail-lj1-f177.google.com [209.85.208.177]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9941368BAEB for ; Thu, 15 Oct 2020 03:22:15 +0300 (EEST) Received: by mail-lj1-f177.google.com with SMTP id m20so1310212ljj.5 for ; Wed, 14 Oct 2020 17:22:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=RFCN3+CyEuWVJ7nk1t1w/mh0VoNDBUWHDBJZVigIynw=; b=KDS+Xt9eyz9NOY0YEnjCrOali98iyXUqdjrNyajq1vGjSFfsDS+VEdJARIfIG2Y55Y JrtW+2QcMi64rzRscRaR8AiPxsxxoPiRBRB4kY7gVO90Q2cwm15Acpayocfk+A0yO6RC EGjvOHPXAy0TVRmMhli13CIFpZ8p0Ck8lg7bIfvBZIQeVJ8QQTdaJtXyHwajkJ6vdjYy v9ujhdg47tD1eEVPqb3IkmMUb83lQ42RB+3Un9jvMCI0lC6f6tqWx2RcAzivu5vVCZGt Yu6zefwL3sEdC8GikpnC/L6lp7FoWHOLDiYIW76qRtpORpbbDEtu7oA90/5oPFuipBu3 of/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=RFCN3+CyEuWVJ7nk1t1w/mh0VoNDBUWHDBJZVigIynw=; b=UkXmIMBnckl/+s8CejVGh9t/q0/DuQ4lBYG/QYykjwoH0Rx1pzrG84lzmn4OqeTGN9 JzzDh4uTgHmEXmJ2NgMw+TVZ7GzaFN7DAW7lf08WA4QTiJmqlCQFMJJB02ufQZ9fm4xr tZyv3s6nHdjLefvAw4bO0tPhuP1hDOpW9Tz/rsUiWJ8XBjoe0G5WH2CH3/X3QWG6UW08 jr5YueThHtRIs/6dN9IKQg7SbuCgqhkW06nUANgoYL5TN18rFwxRWRug9mmOPFApQFC/ 0rYEWCOfJgF/TrBv9ke0N3SDmI19/qqSR90wrXJ7uu8p0mk0qWLBkns8hVMJVk6kT4rQ rl0g== X-Gm-Message-State: AOAM531d+V/fvgyqr44E3pU+Kg6+K25CBs1dernMq99Gon1twQ422/zU wfEMBZ1hPZ88TZH//3LHixF2muOe1i085Q== X-Google-Smtp-Source: ABdhPJzZxpZyACWjSUVzVyl8ojM01RET2bCXqYf2SvNHsyAxk1uFFDHomdml93kMOM6r3fcCMVf13g== X-Received: by 2002:a2e:834f:: with SMTP id l15mr224007ljh.112.1602720998766; Wed, 14 Oct 2020 17:16:38 -0700 (PDT) Received: from localhost.localdomain (95-55-4-47.dynamic.avangarddsl.ru. [95.55.4.47]) by smtp.googlemail.com with ESMTPSA id q20sm333291lfp.43.2020.10.14.17.16.37 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 14 Oct 2020 17:16:38 -0700 (PDT) From: OvchinnikovDmitrii To: ffmpeg-devel@ffmpeg.org Date: Thu, 15 Oct 2020 03:16:21 +0300 Message-Id: <20201015001625.32832-1-ovchinnikov.dmitrii@gmail.com> X-Mailer: git-send-email 2.19.1.windows.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/5] lavc/amfenc: HWConfig for amf encoder. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: OvchinnikovDmitrii Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/amfenc.c | 12 ++++++++++++ libavcodec/amfenc.h | 4 +++- libavcodec/amfenc_h264.c | 1 + libavcodec/amfenc_hevc.c | 1 + 4 files changed, 17 insertions(+), 1 deletion(-) diff --git a/libavcodec/amfenc.c b/libavcodec/amfenc.c index da0652943d..33f97dd582 100644 --- a/libavcodec/amfenc.c +++ b/libavcodec/amfenc.c @@ -782,3 +782,15 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) } return ret; } + +const AVCodecHWConfigInternal *ff_amfenc_hw_configs[] = { +#if CONFIG_D3D11VA + HW_CONFIG_ENCODER_FRAMES(D3D11, D3D11VA), + HW_CONFIG_ENCODER_DEVICE(NONE, D3D11VA), +#endif +#if CONFIG_DXVA2 + HW_CONFIG_ENCODER_FRAMES(DXVA2_VLD, DXVA2), + HW_CONFIG_ENCODER_DEVICE(NONE, DXVA2), +#endif + NULL, +}; \ No newline at end of file diff --git a/libavcodec/amfenc.h b/libavcodec/amfenc.h index 80658c6b2a..06994da543 100644 --- a/libavcodec/amfenc.h +++ b/libavcodec/amfenc.h @@ -27,7 +27,7 @@ #include "libavutil/fifo.h" #include "avcodec.h" - +#include "hwconfig.h" /** * AMF trace writer callback class @@ -117,6 +117,8 @@ typedef struct AmfContext { int tier; } AmfContext; +extern const AVCodecHWConfigInternal *ff_amfenc_hw_configs[]; + /** * Common encoder initization function */ diff --git a/libavcodec/amfenc_h264.c b/libavcodec/amfenc_h264.c index 7a8029f3b7..622ee85946 100644 --- a/libavcodec/amfenc_h264.c +++ b/libavcodec/amfenc_h264.c @@ -392,4 +392,5 @@ AVCodec ff_h264_amf_encoder = { .caps_internal = FF_CODEC_CAP_INIT_CLEANUP, .pix_fmts = ff_amf_pix_fmts, .wrapper_name = "amf", + .hw_configs = ff_amfenc_hw_configs, }; diff --git a/libavcodec/amfenc_hevc.c b/libavcodec/amfenc_hevc.c index fa12a3a3f6..bb224c5fec 100644 --- a/libavcodec/amfenc_hevc.c +++ b/libavcodec/amfenc_hevc.c @@ -322,4 +322,5 @@ AVCodec ff_hevc_amf_encoder = { .caps_internal = FF_CODEC_CAP_INIT_CLEANUP, .pix_fmts = ff_amf_pix_fmts, .wrapper_name = "amf", + .hw_configs = ff_amfenc_hw_configs, }; From patchwork Thu Oct 15 00:16:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitrii Ovchinnikov X-Patchwork-Id: 22958 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 8704B44928A for ; Thu, 15 Oct 2020 03:16:46 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5FDC868B9F9; Thu, 15 Oct 2020 03:16:46 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f53.google.com (mail-lf1-f53.google.com [209.85.167.53]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B1ADB68B95D for ; Thu, 15 Oct 2020 03:16:40 +0300 (EEST) Received: by mail-lf1-f53.google.com with SMTP id v6so1417829lfa.13 for ; Wed, 14 Oct 2020 17:16:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=y/rgrBbqXk8yjk5r2fKnyoPjchX2/2Il3nYNTHWAFBE=; b=W1OSWCtLJx9qyJNvOSjiYOaRwtU1B/w3+I/LzleZCtXGYfkPYWTZfFXX7RjPRnhSRp zm+NgcAjSTQq+EWwZworourLnUGgFZ8MxnQX6rERHR2vtUx4XcaSyFoeD5WQyxVHx01v 3vE/wWr1uXFs6fdDMSyK0AJX29JlZAEpwoZ9GeqTHgd2B8fQbGDeyiyxI7retnr+f3Od CykZN/xe3A0eHSwIWI0Ijr0G4InRRk5ezBjd+lNWFNMIm24B+HWbfbKUQF+lb7z7VkQf JH6mGjm69WVwA2a7XxhDNdEMotIDyf9fNmzZGccUHnxQ71ykNWl+zojUeCg9eZHVJroZ a7+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=y/rgrBbqXk8yjk5r2fKnyoPjchX2/2Il3nYNTHWAFBE=; b=uNGctK1mJtML8XmxQ2/TNcSpQTPk+khkOXV0ahR0llAHSykDlMdGe1kOohJ6BGH0Pk By0wUdB6N0eeH2bhstKYFj74TpUzMy8wvmDe43PYcqFMPUlikp6+fAemKHgxaay3mN9I wm+T4WY8VuV9hsh4cI8evXlx9ZoNeTDS5LupeWe0Dli05C1t4wCEX2zJKU2TxdS+crou JjpNSNyl0Obi79OM1PLML8xE5W34L42e3lQ9x7qQwWZdaScwLcY52xFVrJZe5u8WhvAq SvzqNy7DiI3LULIE/QjutKkqjYSoARSdq4CPKtkN6NpuwBt5/n/wxjPecyQKvXMUSqYu 5J4g== X-Gm-Message-State: AOAM531SQBGmX+C7yy4+uH90yfSQmEp7jiPLJGyunTZYZkmCl6GZCGlT pmNpq8Re9Zkh8xis7pImr45hXR8+hEAtHQ== X-Google-Smtp-Source: ABdhPJyjM9FLNXDofuWDBQQKqGKd68dmaNYGXEEBUN8hF2aiXrTx5ZWNFCBRFi0Iv5zMD0O0QV+djA== X-Received: by 2002:a19:22d8:: with SMTP id i207mr176857lfi.85.1602720999529; Wed, 14 Oct 2020 17:16:39 -0700 (PDT) Received: from localhost.localdomain (95-55-4-47.dynamic.avangarddsl.ru. [95.55.4.47]) by smtp.googlemail.com with ESMTPSA id q20sm333291lfp.43.2020.10.14.17.16.38 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 14 Oct 2020 17:16:39 -0700 (PDT) From: OvchinnikovDmitrii To: ffmpeg-devel@ffmpeg.org Date: Thu, 15 Oct 2020 03:16:22 +0300 Message-Id: <20201015001625.32832-2-ovchinnikov.dmitrii@gmail.com> X-Mailer: git-send-email 2.19.1.windows.1 In-Reply-To: <20201015001625.32832-1-ovchinnikov.dmitrii@gmail.com> References: <20201015001625.32832-1-ovchinnikov.dmitrii@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/5] lavu/hwcontext_amf: HWContext for AMF based components X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Alexander Kravchenko Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Alexander Kravchenko --- libavutil/Makefile | 2 + libavutil/hwcontext.c | 4 + libavutil/hwcontext.h | 1 + libavutil/hwcontext_amf.c | 284 +++++++++++++++++++++++++++++++++ libavutil/hwcontext_amf.h | 54 +++++++ libavutil/hwcontext_internal.h | 1 + 6 files changed, 346 insertions(+) create mode 100644 libavutil/hwcontext_amf.c create mode 100644 libavutil/hwcontext_amf.h diff --git a/libavutil/Makefile b/libavutil/Makefile index 9b08372eb2..4160f5d972 100644 --- a/libavutil/Makefile +++ b/libavutil/Makefile @@ -172,6 +172,7 @@ OBJS = adler32.o \ video_enc_params.o \ +OBJS-$(CONFIG_AMF) += hwcontext_amf.o OBJS-$(CONFIG_CUDA) += hwcontext_cuda.o OBJS-$(CONFIG_D3D11VA) += hwcontext_d3d11va.o OBJS-$(CONFIG_DXVA2) += hwcontext_dxva2.o @@ -190,6 +191,7 @@ OBJS += $(COMPAT_OBJS:%=../compat/%) # Windows resource file SLIBOBJS-$(HAVE_GNU_WINDRES) += avutilres.o +SKIPHEADERS-$(CONFIG_AMF) += hwcontext_amf.h SKIPHEADERS-$(HAVE_CUDA_H) += hwcontext_cuda.h SKIPHEADERS-$(CONFIG_CUDA) += hwcontext_cuda_internal.h \ cuda_check.h diff --git a/libavutil/hwcontext.c b/libavutil/hwcontext.c index d13d0f7c9b..ff140f04cb 100644 --- a/libavutil/hwcontext.c +++ b/libavutil/hwcontext.c @@ -29,6 +29,9 @@ #include "pixfmt.h" static const HWContextType * const hw_table[] = { +#if CONFIG_AMF + &ff_hwcontext_type_amf, +#endif #if CONFIG_CUDA &ff_hwcontext_type_cuda, #endif @@ -66,6 +69,7 @@ static const HWContextType * const hw_table[] = { }; static const char *const hw_type_names[] = { + [AV_HWDEVICE_TYPE_AMF] = "amf", [AV_HWDEVICE_TYPE_CUDA] = "cuda", [AV_HWDEVICE_TYPE_DRM] = "drm", [AV_HWDEVICE_TYPE_DXVA2] = "dxva2", diff --git a/libavutil/hwcontext.h b/libavutil/hwcontext.h index 04d19d89c2..9e1e76dee6 100644 --- a/libavutil/hwcontext.h +++ b/libavutil/hwcontext.h @@ -37,6 +37,7 @@ enum AVHWDeviceType { AV_HWDEVICE_TYPE_OPENCL, AV_HWDEVICE_TYPE_MEDIACODEC, AV_HWDEVICE_TYPE_VULKAN, + AV_HWDEVICE_TYPE_AMF, }; typedef struct AVHWDeviceInternal AVHWDeviceInternal; diff --git a/libavutil/hwcontext_amf.c b/libavutil/hwcontext_amf.c new file mode 100644 index 0000000000..b70ee90d40 --- /dev/null +++ b/libavutil/hwcontext_amf.c @@ -0,0 +1,284 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include + +#include "config.h" + +#include "avassert.h" +#include "avstring.h" +#include "common.h" +#include "hwcontext.h" +#include "hwcontext_internal.h" +#include "hwcontext_amf.h" + +#if CONFIG_D3D11VA +#include "libavutil/hwcontext_d3d11va.h" +#endif + +#if CONFIG_DXVA2 +#define COBJMACROS +#include "libavutil/hwcontext_dxva2.h" +#endif + +#ifdef _WIN32 +#include "compat/w32dlfcn.h" +#else +#include +#endif + +/** +* Error handling helper +*/ +#define AMFAV_RETURN_IF_FALSE(avctx, exp, ret_value, /*message,*/ ...) \ + if (!(exp)) { \ + av_log(avctx, AV_LOG_ERROR, __VA_ARGS__); \ + return ret_value; \ + } + +static const AVClass amflib_class = { + .class_name = "amf", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, +}; + +typedef struct AMFLibraryContext { + const AVClass *avclass; +} AMFLibraryContext; + +static AMFLibraryContext amflib_context = +{ + .avclass = &amflib_class, +}; + +typedef struct AmfTraceWriter { + const AMFTraceWriterVtbl *vtbl; + void *avcl; +} AmfTraceWriter; + +static void AMF_CDECL_CALL AMFTraceWriter_Write(AMFTraceWriter *pThis, + const wchar_t *scope, const wchar_t *message) +{ + AmfTraceWriter *tracer = (AmfTraceWriter*)pThis; + av_log(tracer->avcl, AV_LOG_DEBUG, "%ls: %ls", scope, message); +} + +static void AMF_CDECL_CALL AMFTraceWriter_Flush(AMFTraceWriter *pThis) +{ +} + +static const AMFTraceWriterVtbl tracer_vtbl = +{ + .Write = AMFTraceWriter_Write, + .Flush = AMFTraceWriter_Flush, +}; + +static const AmfTraceWriter amf_trace_writer = +{ + .vtbl = &tracer_vtbl, + .avcl = &amflib_context, +}; +#define AMFAV_WRITER_ID L"avlog" + +typedef struct AMFDeviceContextPrivate { + amf_handle library; + AMFDebug *debug; + AMFTrace *trace; + AmfTraceWriter tracer; +} AMFDeviceContextPrivate; + +static void amf_device_free(AVHWDeviceContext *ctx) +{ + AVAMFDeviceContext *amf_ctx = ctx->hwctx; + AMFDeviceContextPrivate *priv = ctx->internal->priv; + if (amf_ctx->context) { + amf_ctx->context->pVtbl->Terminate(amf_ctx->context); + amf_ctx->context->pVtbl->Release(amf_ctx->context); + amf_ctx->context = NULL; + } + if(priv->library) { + dlclose(priv->library); + } +} + +static int amf_init_device_ctx_object(AVHWDeviceContext *ctx) +{ + AVAMFDeviceContext *hwctx = ctx->hwctx; + AMFDeviceContextPrivate *priv = ctx->internal->priv; + AMF_RESULT res; + AMFInit_Fn init_fun; + + ctx->free = amf_device_free; + + priv->library = dlopen(AMF_DLL_NAMEA, RTLD_NOW | RTLD_LOCAL); + AMFAV_RETURN_IF_FALSE(ctx, priv->library != NULL, AVERROR_UNKNOWN, "DLL %s failed to open\n", AMF_DLL_NAMEA); + + init_fun = (AMFInit_Fn)dlsym(priv->library, AMF_INIT_FUNCTION_NAME); + AMFAV_RETURN_IF_FALSE(ctx, init_fun != NULL, AVERROR_UNKNOWN, "DLL %s failed to find function %s\n", AMF_DLL_NAMEA, AMF_INIT_FUNCTION_NAME); + + res = init_fun(AMF_FULL_VERSION, &hwctx->factory); + AMFAV_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "%s failed with error %d\n", AMF_INIT_FUNCTION_NAME, res); + + res = hwctx->factory->pVtbl->GetTrace(hwctx->factory, &priv->trace); + AMFAV_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "GetTrace() failed with error %d\n", res); + res = hwctx->factory->pVtbl->GetDebug(hwctx->factory, &priv->debug); + AMFAV_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "GetDebug() failed with error %d\n", res); + + priv->trace->pVtbl->EnableWriter(priv->trace, AMF_TRACE_WRITER_CONSOLE, 0); + priv->trace->pVtbl->SetGlobalLevel(priv->trace, AMF_TRACE_TRACE); + + // connect AMF logger to av_log + priv->trace->pVtbl->RegisterWriter(priv->trace, AMFAV_WRITER_ID, (AMFTraceWriter*)&amf_trace_writer, 1); + priv->trace->pVtbl->SetWriterLevel(priv->trace, AMFAV_WRITER_ID, AMF_TRACE_TRACE); + + res = hwctx->factory->pVtbl->CreateContext(hwctx->factory, &hwctx->context); + AMFAV_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "CreateContext() failed with error %d\n", res); + return 0; +} + +static int amf_device_create(AVHWDeviceContext *ctx, const char *device, + AVDictionary *opts, int flags) +{ + AVAMFDeviceContext *amf_ctx = ctx->hwctx; + AMFContext1 *context1 = NULL; + AMF_RESULT res; + int err; + + err = amf_init_device_ctx_object(ctx); + if(err < 0) + return err; + + res = amf_ctx->context->pVtbl->InitDX11(amf_ctx->context, NULL, AMF_DX11_1); + if (res == AMF_OK) { + av_log(ctx, AV_LOG_VERBOSE, "AMF initialisation succeeded via D3D11.\n"); + } else { + res = amf_ctx->context->pVtbl->InitDX9(amf_ctx->context, NULL); + if (res == AMF_OK) { + av_log(ctx, AV_LOG_VERBOSE, "AMF initialisation succeeded via D3D9.\n"); + } else { + AMFGuid guid = IID_AMFContext1(); + res = amf_ctx->context->pVtbl->QueryInterface(amf_ctx->context, &guid, (void**)&context1); + AMFAV_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "CreateContext1() failed with error %d\n", res); + + res = context1->pVtbl->InitVulkan(context1, NULL); + context1->pVtbl->Release(context1); + if (res != AMF_OK) { + if (res == AMF_NOT_SUPPORTED) + av_log(ctx, AV_LOG_ERROR, "AMF via Vulkan is not supported on the given device.\n"); + else + av_log(ctx, AV_LOG_ERROR, "AMF failed to initialise on the given Vulkan device: %d.\n", res); + return AVERROR(ENOSYS); + } + av_log(ctx, AV_LOG_VERBOSE, "AMF initialisation succeeded via Vulkan.\n"); + } + } + return 0; +} + +static int amf_device_derive(AVHWDeviceContext *dst_ctx, + AVHWDeviceContext *src_ctx, + int flags) +{ + AVAMFDeviceContext *amf_ctx = dst_ctx->hwctx; + AMF_RESULT res; + int err; + + err = amf_init_device_ctx_object(dst_ctx); + if(err < 0) + return err; + + switch (src_ctx->type) { + +#if CONFIG_D3D11VA + case AV_HWDEVICE_TYPE_DXVA2: + { + AVDXVA2DeviceContext *dxva2_ctx = src_ctx->hwctx; + HANDLE device_handle; + IDirect3DDevice9 *device; + HRESULT hr; + AMF_RESULT res; + int ret; + + hr = IDirect3DDeviceManager9_OpenDeviceHandle(dxva2_ctx->devmgr, &device_handle); + if (FAILED(hr)) { + av_log(dst_ctx, AV_LOG_ERROR, "Failed to open device handle for Direct3D9 device: %lx.\n", (unsigned long)hr); + return AVERROR_EXTERNAL; + } + + hr = IDirect3DDeviceManager9_LockDevice(dxva2_ctx->devmgr, device_handle, &device, FALSE); + if (SUCCEEDED(hr)) { + IDirect3DDeviceManager9_UnlockDevice(dxva2_ctx->devmgr, device_handle, FALSE); + ret = 0; + } else { + av_log(dst_ctx, AV_LOG_ERROR, "Failed to lock device handle for Direct3D9 device: %lx.\n", (unsigned long)hr); + ret = AVERROR_EXTERNAL; + } + + IDirect3DDeviceManager9_CloseDeviceHandle(dxva2_ctx->devmgr, device_handle); + + if (ret < 0) + return ret; + + res = amf_ctx->context->pVtbl->InitDX9(amf_ctx->context, device); + + IDirect3DDevice9_Release(device); + + if (res != AMF_OK) { + if (res == AMF_NOT_SUPPORTED) + av_log(dst_ctx, AV_LOG_ERROR, "AMF via D3D9 is not supported on the given device.\n"); + else + av_log(dst_ctx, AV_LOG_ERROR, "AMF failed to initialise on given D3D9 device: %d.\n", res); + return AVERROR(ENODEV); + } + } + break; +#endif + +#if CONFIG_D3D11VA + case AV_HWDEVICE_TYPE_D3D11VA: + { + AVD3D11VADeviceContext *d3d11_ctx = src_ctx->hwctx; + res = amf_ctx->context->pVtbl->InitDX11(amf_ctx->context, d3d11_ctx->device, AMF_DX11_1); + if (res != AMF_OK) { + if (res == AMF_NOT_SUPPORTED) + av_log(dst_ctx, AV_LOG_ERROR, "AMF via D3D11 is not supported on the given device.\n"); + else + av_log(dst_ctx, AV_LOG_ERROR, "AMF failed to initialise on the given D3D11 device: %d.\n", res); + return AVERROR(ENODEV); + } + } + break; +#endif + default: + av_log(dst_ctx, AV_LOG_ERROR, "AMF initialisation from a %s device is not supported.\n", + av_hwdevice_get_type_name(src_ctx->type)); + return AVERROR(ENOSYS); + } + return 0; +} + +const HWContextType ff_hwcontext_type_amf = { + .type = AV_HWDEVICE_TYPE_AMF, + .name = "AMF", + + .device_hwctx_size = sizeof(AVAMFDeviceContext), + .device_priv_size = sizeof(AMFDeviceContextPrivate), + + .device_create = &amf_device_create, + .device_derive = &amf_device_derive, +}; diff --git a/libavutil/hwcontext_amf.h b/libavutil/hwcontext_amf.h new file mode 100644 index 0000000000..ae2a37f425 --- /dev/null +++ b/libavutil/hwcontext_amf.h @@ -0,0 +1,54 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + + +#ifndef AVUTIL_HWCONTEXT_AMF_H +#define AVUTIL_HWCONTEXT_AMF_H + +/** + * @file + * API-specific header for AV_HWDEVICE_TYPE_AMF. + * + */ + +#include "frame.h" +#include "AMF/core/Context.h" +#include "AMF/core/Factory.h" + + +/** + * This struct is allocated as AVHWDeviceContext.hwctx + */ +typedef struct AVAMFDeviceContext { + /** + * Context used for: + * texture and buffers allocation. + * Access to device objects (DX9, DX11, OpenCL, OpenGL) which are being used in the context + */ + AMFContext *context; + + /** + * Factory used for: + * AMF component creation such as encoder, decoder, converter... + * Access AMF Library settings such as trace/debug/cache + */ + AMFFactory *factory; +} AVAMFDeviceContext; + + +#endif /* AVUTIL_HWCONTEXT_AMF_H */ diff --git a/libavutil/hwcontext_internal.h b/libavutil/hwcontext_internal.h index e6266494ac..7d9c6f0a68 100644 --- a/libavutil/hwcontext_internal.h +++ b/libavutil/hwcontext_internal.h @@ -163,6 +163,7 @@ int ff_hwframe_map_create(AVBufferRef *hwframe_ref, */ int ff_hwframe_map_replace(AVFrame *dst, const AVFrame *src); +extern const HWContextType ff_hwcontext_type_amf; extern const HWContextType ff_hwcontext_type_cuda; extern const HWContextType ff_hwcontext_type_d3d11va; extern const HWContextType ff_hwcontext_type_drm; From patchwork Thu Oct 15 00:16:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitrii Ovchinnikov X-Patchwork-Id: 22962 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 0EB41449F2C for ; Thu, 15 Oct 2020 03:42:56 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E1A2668B9F1; Thu, 15 Oct 2020 03:42:55 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f68.google.com (mail-lf1-f68.google.com [209.85.167.68]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 21FD368A94D for ; Thu, 15 Oct 2020 03:42:49 +0300 (EEST) Received: by mail-lf1-f68.google.com with SMTP id z2so1557549lfr.1 for ; Wed, 14 Oct 2020 17:42:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SGNkpsYxvaQu2E3yih1iXMvqKaK9p1l+d+OJpmiA2lk=; b=QB3aYcutOGJ6N7Uz3LXYcFc2ugdgJr4HuIrvHd+oM2XZdT5j/v3zI5W9i48tXgPtXZ yNe5E1nlBCJN/j8PDS4kNUYxKnp23QUFitOAm6tJWN+SrFG61gjUyq5GwPWMn/W2hgWC tAJURVcIlcGCo+2+b1IYsW+UFgoFuXyL8aRXHlTT0k7mf1O/u5yGLj5cnYGa7TesfapH dGBw9EIInQ0t2JZ9bB292RquSaBABT1m73ceTzLE8qw8EoSZmGgcPubusja+jMrt4g6p QqEgsXLTlsmPJpkTDMn87jOesMUJCTt9g4lhk5pcGm+NeWrGr09dZB7lpzh70xQZl//l RDDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SGNkpsYxvaQu2E3yih1iXMvqKaK9p1l+d+OJpmiA2lk=; b=mCkrWOnqzOL/M8CziQwFlW9D3KuDZaTAMSAl5qF6vkXDNUxl/y9rM1L2LEDfV7E5V9 S8GVZQ6vpfH6XD3NcuZjtKYwZZ+EKVJKkgtGmansKaQw7z+2/tLlr1kMpRI6VezNp3gB wG0NMq2xRK2CrbW/xFYClHjA/lX8oZ9svHvn+vYSk+otXUB0hzAeSJdHrIANRUTJ+v7a +0lRjRdycgk1mKVHR4rfhzRL8RZDP8bhegX/0aWWY97i9blHGz2fehWiOCWpY8s0NdPj OcFrZG07Yjg+z4dEF5EuB7XVDZvQfWILiWVFUpaYfBZ/4IqoX/iFLiu60wDTQe5+uEk+ CCKg== X-Gm-Message-State: AOAM533iZlCk08NBGsMcc24hcR9QfF/XDE7JP9Wjt7afrKIOjKA8GNu3 0VLwtBNP0pVp5aUQYWuiNE559admSuTXvg== X-Google-Smtp-Source: ABdhPJxfRR9cudYWUhGCK9Z0usWL+Sk1jCuiXS/qgIe0eaj2TiKTePGhmSsWWcU10aVYjOxGat3wSg== X-Received: by 2002:a2e:8e32:: with SMTP id r18mr235289ljk.171.1602721000536; Wed, 14 Oct 2020 17:16:40 -0700 (PDT) Received: from localhost.localdomain (95-55-4-47.dynamic.avangarddsl.ru. [95.55.4.47]) by smtp.googlemail.com with ESMTPSA id q20sm333291lfp.43.2020.10.14.17.16.39 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 14 Oct 2020 17:16:39 -0700 (PDT) From: OvchinnikovDmitrii To: ffmpeg-devel@ffmpeg.org Date: Thu, 15 Oct 2020 03:16:23 +0300 Message-Id: <20201015001625.32832-3-ovchinnikov.dmitrii@gmail.com> X-Mailer: git-send-email 2.19.1.windows.1 In-Reply-To: <20201015001625.32832-1-ovchinnikov.dmitrii@gmail.com> References: <20201015001625.32832-1-ovchinnikov.dmitrii@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 3/5] lavc/amfenc: redesign to use hwcontext_amf. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Alexander Kravchenko Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Alexander Kravchenko --- libavcodec/amfenc.c | 266 +++++--------------------------------------- libavcodec/amfenc.h | 25 +---- 2 files changed, 32 insertions(+), 259 deletions(-) diff --git a/libavcodec/amfenc.c b/libavcodec/amfenc.c index 33f97dd582..767eab3d56 100644 --- a/libavcodec/amfenc.c +++ b/libavcodec/amfenc.c @@ -21,13 +21,6 @@ #include "libavutil/avassert.h" #include "libavutil/imgutils.h" #include "libavutil/hwcontext.h" -#if CONFIG_D3D11VA -#include "libavutil/hwcontext_d3d11va.h" -#endif -#if CONFIG_DXVA2 -#define COBJMACROS -#include "libavutil/hwcontext_dxva2.h" -#endif #include "libavutil/mem.h" #include "libavutil/pixdesc.h" #include "libavutil/time.h" @@ -36,18 +29,14 @@ #include "encode.h" #include "internal.h" -#if CONFIG_D3D11VA -#include +#if CONFIG_DXVA2 +#include #endif -#ifdef _WIN32 -#include "compat/w32dlfcn.h" -#else -#include +#if CONFIG_D3D11VA +#include #endif -#define FFMPEG_AMF_WRITER_ID L"ffmpeg_amf" - #define PTS_PROP L"PtsProp" const enum AVPixelFormat ff_amf_pix_fmts[] = { @@ -89,29 +78,11 @@ static enum AMF_SURFACE_FORMAT amf_av_to_amf_format(enum AVPixelFormat fmt) return AMF_SURFACE_UNKNOWN; } -static void AMF_CDECL_CALL AMFTraceWriter_Write(AMFTraceWriter *pThis, - const wchar_t *scope, const wchar_t *message) -{ - AmfTraceWriter *tracer = (AmfTraceWriter*)pThis; - av_log(tracer->avctx, AV_LOG_DEBUG, "%ls: %ls", scope, message); // \n is provided from AMF -} - -static void AMF_CDECL_CALL AMFTraceWriter_Flush(AMFTraceWriter *pThis) -{ -} - -static AMFTraceWriterVtbl tracer_vtbl = -{ - .Write = AMFTraceWriter_Write, - .Flush = AMFTraceWriter_Flush, -}; - -static int amf_load_library(AVCodecContext *avctx) +static int amf_init_context(AVCodecContext *avctx) { - AmfContext *ctx = avctx->priv_data; - AMFInit_Fn init_fun; - AMFQueryVersion_Fn version_fun; - AMF_RESULT res; + AmfContext *ctx = avctx->priv_data; + AVAMFDeviceContext *amf_ctx; + int ret; ctx->delayed_frame = av_frame_alloc(); if (!ctx->delayed_frame) { @@ -124,120 +95,9 @@ static int amf_load_library(AVCodecContext *avctx) } ctx->dts_delay = 0; - - ctx->library = dlopen(AMF_DLL_NAMEA, RTLD_NOW | RTLD_LOCAL); - AMF_RETURN_IF_FALSE(ctx, ctx->library != NULL, - AVERROR_UNKNOWN, "DLL %s failed to open\n", AMF_DLL_NAMEA); - - init_fun = (AMFInit_Fn)dlsym(ctx->library, AMF_INIT_FUNCTION_NAME); - AMF_RETURN_IF_FALSE(ctx, init_fun != NULL, AVERROR_UNKNOWN, "DLL %s failed to find function %s\n", AMF_DLL_NAMEA, AMF_INIT_FUNCTION_NAME); - - version_fun = (AMFQueryVersion_Fn)dlsym(ctx->library, AMF_QUERY_VERSION_FUNCTION_NAME); - AMF_RETURN_IF_FALSE(ctx, version_fun != NULL, AVERROR_UNKNOWN, "DLL %s failed to find function %s\n", AMF_DLL_NAMEA, AMF_QUERY_VERSION_FUNCTION_NAME); - - res = version_fun(&ctx->version); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "%s failed with error %d\n", AMF_QUERY_VERSION_FUNCTION_NAME, res); - res = init_fun(AMF_FULL_VERSION, &ctx->factory); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "%s failed with error %d\n", AMF_INIT_FUNCTION_NAME, res); - res = ctx->factory->pVtbl->GetTrace(ctx->factory, &ctx->trace); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "GetTrace() failed with error %d\n", res); - res = ctx->factory->pVtbl->GetDebug(ctx->factory, &ctx->debug); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "GetDebug() failed with error %d\n", res); - return 0; -} - -#if CONFIG_D3D11VA -static int amf_init_from_d3d11_device(AVCodecContext *avctx, AVD3D11VADeviceContext *hwctx) -{ - AmfContext *ctx = avctx->priv_data; - AMF_RESULT res; - - res = ctx->context->pVtbl->InitDX11(ctx->context, hwctx->device, AMF_DX11_1); - if (res != AMF_OK) { - if (res == AMF_NOT_SUPPORTED) - av_log(avctx, AV_LOG_ERROR, "AMF via D3D11 is not supported on the given device.\n"); - else - av_log(avctx, AV_LOG_ERROR, "AMF failed to initialise on the given D3D11 device: %d.\n", res); - return AVERROR(ENODEV); - } - - return 0; -} -#endif - -#if CONFIG_DXVA2 -static int amf_init_from_dxva2_device(AVCodecContext *avctx, AVDXVA2DeviceContext *hwctx) -{ - AmfContext *ctx = avctx->priv_data; - HANDLE device_handle; - IDirect3DDevice9 *device; - HRESULT hr; - AMF_RESULT res; - int ret; - - hr = IDirect3DDeviceManager9_OpenDeviceHandle(hwctx->devmgr, &device_handle); - if (FAILED(hr)) { - av_log(avctx, AV_LOG_ERROR, "Failed to open device handle for Direct3D9 device: %lx.\n", (unsigned long)hr); - return AVERROR_EXTERNAL; - } - - hr = IDirect3DDeviceManager9_LockDevice(hwctx->devmgr, device_handle, &device, FALSE); - if (SUCCEEDED(hr)) { - IDirect3DDeviceManager9_UnlockDevice(hwctx->devmgr, device_handle, FALSE); - ret = 0; - } else { - av_log(avctx, AV_LOG_ERROR, "Failed to lock device handle for Direct3D9 device: %lx.\n", (unsigned long)hr); - ret = AVERROR_EXTERNAL; - } - - IDirect3DDeviceManager9_CloseDeviceHandle(hwctx->devmgr, device_handle); - - if (ret < 0) - return ret; - - res = ctx->context->pVtbl->InitDX9(ctx->context, device); - - IDirect3DDevice9_Release(device); - - if (res != AMF_OK) { - if (res == AMF_NOT_SUPPORTED) - av_log(avctx, AV_LOG_ERROR, "AMF via D3D9 is not supported on the given device.\n"); - else - av_log(avctx, AV_LOG_ERROR, "AMF failed to initialise on given D3D9 device: %d.\n", res); - return AVERROR(ENODEV); - } - - return 0; -} -#endif - -static int amf_init_context(AVCodecContext *avctx) -{ - AmfContext *ctx = avctx->priv_data; - AMFContext1 *context1 = NULL; - AMF_RESULT res; - av_unused int ret; - ctx->hwsurfaces_in_queue = 0; ctx->hwsurfaces_in_queue_max = 16; - // configure AMF logger - // the return of these functions indicates old state and do not affect behaviour - ctx->trace->pVtbl->EnableWriter(ctx->trace, AMF_TRACE_WRITER_DEBUG_OUTPUT, ctx->log_to_dbg != 0 ); - if (ctx->log_to_dbg) - ctx->trace->pVtbl->SetWriterLevel(ctx->trace, AMF_TRACE_WRITER_DEBUG_OUTPUT, AMF_TRACE_TRACE); - ctx->trace->pVtbl->EnableWriter(ctx->trace, AMF_TRACE_WRITER_CONSOLE, 0); - ctx->trace->pVtbl->SetGlobalLevel(ctx->trace, AMF_TRACE_TRACE); - - // connect AMF logger to av_log - ctx->tracer.vtbl = &tracer_vtbl; - ctx->tracer.avctx = avctx; - ctx->trace->pVtbl->RegisterWriter(ctx->trace, FFMPEG_AMF_WRITER_ID,(AMFTraceWriter*)&ctx->tracer, 1); - ctx->trace->pVtbl->SetWriterLevel(ctx->trace, FFMPEG_AMF_WRITER_ID, AMF_TRACE_TRACE); - - res = ctx->factory->pVtbl->CreateContext(ctx->factory, &ctx->context); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "CreateContext() failed with error %d\n", res); - // If a device was passed to the encoder, try to initialise from that. if (avctx->hw_frames_ctx) { AVHWFramesContext *frames_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; @@ -248,27 +108,9 @@ static int amf_init_context(AVCodecContext *avctx) return AVERROR(EINVAL); } - switch (frames_ctx->device_ctx->type) { -#if CONFIG_D3D11VA - case AV_HWDEVICE_TYPE_D3D11VA: - ret = amf_init_from_d3d11_device(avctx, frames_ctx->device_ctx->hwctx); - if (ret < 0) - return ret; - break; -#endif -#if CONFIG_DXVA2 - case AV_HWDEVICE_TYPE_DXVA2: - ret = amf_init_from_dxva2_device(avctx, frames_ctx->device_ctx->hwctx); - if (ret < 0) - return ret; - break; -#endif - default: - av_log(avctx, AV_LOG_ERROR, "AMF initialisation from a %s frames context is not supported.\n", - av_hwdevice_get_type_name(frames_ctx->device_ctx->type)); - return AVERROR(ENOSYS); - } - + ret = av_hwdevice_ctx_create_derived(&ctx->amf_device_ctx, AV_HWDEVICE_TYPE_AMF, frames_ctx->device_ref, 0); + if (ret < 0) + return ret; ctx->hw_frames_ctx = av_buffer_ref(avctx->hw_frames_ctx); if (!ctx->hw_frames_ctx) return AVERROR(ENOMEM); @@ -277,59 +119,17 @@ static int amf_init_context(AVCodecContext *avctx) ctx->hwsurfaces_in_queue_max = frames_ctx->initial_pool_size - 1; } else if (avctx->hw_device_ctx) { - AVHWDeviceContext *device_ctx = (AVHWDeviceContext*)avctx->hw_device_ctx->data; - - switch (device_ctx->type) { -#if CONFIG_D3D11VA - case AV_HWDEVICE_TYPE_D3D11VA: - ret = amf_init_from_d3d11_device(avctx, device_ctx->hwctx); - if (ret < 0) - return ret; - break; -#endif -#if CONFIG_DXVA2 - case AV_HWDEVICE_TYPE_DXVA2: - ret = amf_init_from_dxva2_device(avctx, device_ctx->hwctx); - if (ret < 0) - return ret; - break; -#endif - default: - av_log(avctx, AV_LOG_ERROR, "AMF initialisation from a %s device is not supported.\n", - av_hwdevice_get_type_name(device_ctx->type)); - return AVERROR(ENOSYS); - } - - ctx->hw_device_ctx = av_buffer_ref(avctx->hw_device_ctx); - if (!ctx->hw_device_ctx) - return AVERROR(ENOMEM); - + ret = av_hwdevice_ctx_create_derived(&ctx->amf_device_ctx, AV_HWDEVICE_TYPE_AMF, avctx->hw_device_ctx, 0); + if (ret < 0) + return ret; } else { - res = ctx->context->pVtbl->InitDX11(ctx->context, NULL, AMF_DX11_1); - if (res == AMF_OK) { - av_log(avctx, AV_LOG_VERBOSE, "AMF initialisation succeeded via D3D11.\n"); - } else { - res = ctx->context->pVtbl->InitDX9(ctx->context, NULL); - if (res == AMF_OK) { - av_log(avctx, AV_LOG_VERBOSE, "AMF initialisation succeeded via D3D9.\n"); - } else { - AMFGuid guid = IID_AMFContext1(); - res = ctx->context->pVtbl->QueryInterface(ctx->context, &guid, (void**)&context1); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "CreateContext1() failed with error %d\n", res); - - res = context1->pVtbl->InitVulkan(context1, NULL); - context1->pVtbl->Release(context1); - if (res != AMF_OK) { - if (res == AMF_NOT_SUPPORTED) - av_log(avctx, AV_LOG_ERROR, "AMF via Vulkan is not supported on the given device.\n"); - else - av_log(avctx, AV_LOG_ERROR, "AMF failed to initialise on the given Vulkan device: %d.\n", res); - return AVERROR(ENOSYS); - } - av_log(avctx, AV_LOG_VERBOSE, "AMF initialisation succeeded via Vulkan.\n"); - } - } + ret = av_hwdevice_ctx_create(&ctx->amf_device_ctx, AV_HWDEVICE_TYPE_AMF, NULL, NULL, 0); + if (ret < 0) + return ret; } + amf_ctx = ((AVHWDeviceContext*)ctx->amf_device_ctx->data)->hwctx; + ctx->context = amf_ctx->context; + ctx->factory = amf_ctx->factory; return 0; } @@ -382,29 +182,17 @@ int av_cold ff_amf_encode_close(AVCodecContext *avctx) ctx->encoder = NULL; } - if (ctx->context) { - ctx->context->pVtbl->Terminate(ctx->context); - ctx->context->pVtbl->Release(ctx->context); - ctx->context = NULL; - } av_buffer_unref(&ctx->hw_device_ctx); av_buffer_unref(&ctx->hw_frames_ctx); - if (ctx->trace) { - ctx->trace->pVtbl->UnregisterWriter(ctx->trace, FFMPEG_AMF_WRITER_ID); - } - if (ctx->library) { - dlclose(ctx->library); - ctx->library = NULL; - } - ctx->trace = NULL; - ctx->debug = NULL; ctx->factory = NULL; - ctx->version = 0; + ctx->context = NULL; ctx->delayed_drain = 0; av_frame_free(&ctx->delayed_frame); av_fifo_freep(&ctx->timestamp_list); + av_buffer_unref(&ctx->amf_device_ctx); + return 0; } @@ -508,11 +296,9 @@ int ff_amf_encode_init(AVCodecContext *avctx) { int ret; - if ((ret = amf_load_library(avctx)) == 0) { - if ((ret = amf_init_context(avctx)) == 0) { - if ((ret = amf_init_encoder(avctx)) == 0) { - return 0; - } + if ((ret = amf_init_context(avctx)) == 0) { + if ((ret = amf_init_encoder(avctx)) == 0) { + return 0; } } ff_amf_encode_close(avctx); diff --git a/libavcodec/amfenc.h b/libavcodec/amfenc.h index 06994da543..96d1942a37 100644 --- a/libavcodec/amfenc.h +++ b/libavcodec/amfenc.h @@ -25,35 +25,22 @@ #include #include "libavutil/fifo.h" +#include "libavutil/hwcontext_amf.h" #include "avcodec.h" #include "hwconfig.h" -/** -* AMF trace writer callback class -* Used to capture all AMF logging -*/ - -typedef struct AmfTraceWriter { - AMFTraceWriterVtbl *vtbl; - AVCodecContext *avctx; -} AmfTraceWriter; - /** * AMF encoder context */ typedef struct AmfContext { AVClass *avclass; - // access to AMF runtime - amf_handle library; ///< handle to DLL library - AMFFactory *factory; ///< pointer to AMF factory - AMFDebug *debug; ///< pointer to AMF debug interface - AMFTrace *trace; ///< pointer to AMF trace interface - - amf_uint64 version; ///< version of AMF runtime - AmfTraceWriter tracer; ///< AMF writer registered with AMF - AMFContext *context; ///< AMF context + + AMFContext *context; + AMFFactory *factory; + AVBufferRef *amf_device_ctx; + //encoder AMFComponent *encoder; ///< AMF encoder object amf_bool eof; ///< flag indicating EOF happened From patchwork Thu Oct 15 00:16:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitrii Ovchinnikov X-Patchwork-Id: 22960 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id EC5B744A83F for ; Thu, 15 Oct 2020 03:22:13 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D235468B9F9; Thu, 15 Oct 2020 03:22:13 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f66.google.com (mail-lf1-f66.google.com [209.85.167.66]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id BAF9A68B941 for ; Thu, 15 Oct 2020 03:22:06 +0300 (EEST) Received: by mail-lf1-f66.google.com with SMTP id l28so1450891lfp.10 for ; Wed, 14 Oct 2020 17:22:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=k2juirwkQREguJJBaw33lMzdT9waSXVEFOp91m1k7i8=; b=BFN0AeTKwFizLPKPEzRSsAYpYC+PkSr5sh+xuh1jxEGNLB3XvGd/vVoVx+AF1tzvBc Ec9mLX8iGM5VwuTyw4uLJTnGJL22WeA7AFn3r9JxTvKTppjpO9uR7nahWYlm7mxSFZSO xyZrXz5T+M/tBynVXXW00g3ZMSR3nwBXuUMIKnhKd3aIr/wBx/9oNQJrJUynDtEIgBpv tywMLxl8xbPD91h4jDHUYpgGu8d23YRNFAYX4JYp3HClS9iZzC7RZEFM1bUnVEghHs9s HR5tiv37Mn92mQasf1T2lN0jeH2nHESswCZioP7sNF9ZV/mS6r/rm9hTPIbiOnOvWXST gxuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=k2juirwkQREguJJBaw33lMzdT9waSXVEFOp91m1k7i8=; b=sDpuLSG7SzpCLc7J5MBbQYFHTUwkLFPxoKHlM9wbg8v3IpIKjtodG1tCI6J6n9ss1o 6KrrFCnvGR+2LWmaWikzUROl0CXPy9byaTweqMO9oCHg57oTxEV+x70i9eYr4Ac5Xk68 HcUrZ139y7qgZ4mvSnF9uEr8sMxcB7yz6GQErN3HCSzZswjPE1EqoRxd6QfdV9CTnCUs 7X6t0e2GxBi7BhWbVX/sqHUJ1Yh7fTGl5ykP2G1BAyoKcwcXCD8+dyiEyjlgrJYpjQbq 1QhKSt/warBY0wYPPE1W1GEAZ4JJ1UGkWYqb1QI7BUuMDtcPRSoLQhifZmkieCZWx0Ws GLiA== X-Gm-Message-State: AOAM530RsbVlB0C3deDaLfRikMRU4z/9jCKYY1s8lVURlCeCRCW356vx 7H4ZnuK3G2/LjblF3nMN/Ahh7wzbk6gLHA== X-Google-Smtp-Source: ABdhPJxB1/MDo7d+lSi+MWAFeownt7EIGIt7MzkwXX4wceaTtA/uYNlVYV8GCqlfL6sMkTY2S6TSwQ== X-Received: by 2002:a05:651c:543:: with SMTP id q3mr231333ljp.6.1602721001496; Wed, 14 Oct 2020 17:16:41 -0700 (PDT) Received: from localhost.localdomain (95-55-4-47.dynamic.avangarddsl.ru. [95.55.4.47]) by smtp.googlemail.com with ESMTPSA id q20sm333291lfp.43.2020.10.14.17.16.40 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 14 Oct 2020 17:16:40 -0700 (PDT) From: OvchinnikovDmitrii To: ffmpeg-devel@ffmpeg.org Date: Thu, 15 Oct 2020 03:16:24 +0300 Message-Id: <20201015001625.32832-4-ovchinnikov.dmitrii@gmail.com> X-Mailer: git-send-email 2.19.1.windows.1 In-Reply-To: <20201015001625.32832-1-ovchinnikov.dmitrii@gmail.com> References: <20201015001625.32832-1-ovchinnikov.dmitrii@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 4/5] lavu/hwcontext_amf: Engine selection support for AMF context X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: OvchinnikovDmitrii Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/amfenc.c | 9 +++- libavcodec/amfenc.h | 1 + libavcodec/amfenc_h264.c | 8 +++ libavcodec/amfenc_hevc.c | 8 +++ libavutil/hwcontext_amf.c | 104 ++++++++++++++++++++++++++++++-------- libavutil/hwcontext_amf.h | 7 +++ 6 files changed, 116 insertions(+), 21 deletions(-) diff --git a/libavcodec/amfenc.c b/libavcodec/amfenc.c index 767eab3d56..b7eb74811b 100644 --- a/libavcodec/amfenc.c +++ b/libavcodec/amfenc.c @@ -82,6 +82,7 @@ static int amf_init_context(AVCodecContext *avctx) { AmfContext *ctx = avctx->priv_data; AVAMFDeviceContext *amf_ctx; + AVDictionary *dict = NULL; int ret; ctx->delayed_frame = av_frame_alloc(); @@ -123,10 +124,16 @@ static int amf_init_context(AVCodecContext *avctx) if (ret < 0) return ret; } else { - ret = av_hwdevice_ctx_create(&ctx->amf_device_ctx, AV_HWDEVICE_TYPE_AMF, NULL, NULL, 0); +#if defined(_WIN32) + if (ctx->engine) { + ret = av_dict_set_int(&dict, "engine", ctx->engine, 0); if (ret < 0) return ret; } +#endif + ret = av_hwdevice_ctx_create(&ctx->amf_device_ctx, AV_HWDEVICE_TYPE_AMF, NULL, dict, 0);if (ret < 0) + return ret; + } amf_ctx = ((AVHWDeviceContext*)ctx->amf_device_ctx->data)->hwctx; ctx->context = amf_ctx->context; ctx->factory = amf_ctx->factory; diff --git a/libavcodec/amfenc.h b/libavcodec/amfenc.h index 96d1942a37..f4897c9b2a 100644 --- a/libavcodec/amfenc.h +++ b/libavcodec/amfenc.h @@ -73,6 +73,7 @@ typedef struct AmfContext { int quality; int b_frame_delta_qp; int ref_b_frame_delta_qp; + int engine; // Dynamic options, can be set after Init() call diff --git a/libavcodec/amfenc_h264.c b/libavcodec/amfenc_h264.c index 622ee85946..9fb42323d8 100644 --- a/libavcodec/amfenc_h264.c +++ b/libavcodec/amfenc_h264.c @@ -71,6 +71,14 @@ static const AVOption options[] = { { "balanced", "Balanced", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_ENCODER_QUALITY_PRESET_BALANCED }, 0, 0, VE, "quality" }, { "quality", "Prefer Quality", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_ENCODER_QUALITY_PRESET_QUALITY }, 0, 0, VE, "quality" }, +#if defined(_WIN32) +//preffered engine + { "engine", "Preffered engine", OFFSET(engine), AV_OPT_TYPE_INT, { .i64 = AMF_VIDEO_ENCODER_ENGINE_DEFAULT }, AMF_VIDEO_ENCODER_ENGINE_DEFAULT, AMF_VIDEO_ENCODER_ENGINE_VULKAN, VE, "engine" }, + { "dxva2", "DirectX Video Acceleration 2", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_ENCODER_ENGINE_DXVA2 }, 0, 0, VE, "engine" }, + { "d3d11", "Direct2D11", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_ENCODER_ENGINE_D3D11 }, 0, 0, VE, "engine" }, + { "vulkan", "Vulkan", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_ENCODER_ENGINE_VULKAN }, 0, 0, VE, "engine" }, +#endif + // Dynamic /// Rate Control Method { "rc", "Rate Control Method", OFFSET(rate_control_mode), AV_OPT_TYPE_INT, { .i64 = AMF_VIDEO_ENCODER_RATE_CONTROL_METHOD_UNKNOWN }, AMF_VIDEO_ENCODER_RATE_CONTROL_METHOD_UNKNOWN, AMF_VIDEO_ENCODER_RATE_CONTROL_METHOD_LATENCY_CONSTRAINED_VBR, VE, "rc" }, diff --git a/libavcodec/amfenc_hevc.c b/libavcodec/amfenc_hevc.c index bb224c5fec..fc2c40a6ed 100644 --- a/libavcodec/amfenc_hevc.c +++ b/libavcodec/amfenc_hevc.c @@ -58,6 +58,14 @@ static const AVOption options[] = { { "speed", "", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_ENCODER_HEVC_QUALITY_PRESET_SPEED }, 0, 0, VE, "quality" }, { "quality", "", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_ENCODER_HEVC_QUALITY_PRESET_QUALITY }, 0, 0, VE, "quality" }, +#if defined(_WIN32) +//preffered engine + { "engine", "Preffered engine", OFFSET(engine), AV_OPT_TYPE_INT, { .i64 = AMF_VIDEO_ENCODER_ENGINE_DEFAULT }, AMF_VIDEO_ENCODER_ENGINE_DEFAULT, AMF_VIDEO_ENCODER_ENGINE_VULKAN, VE, "engine" }, + { "dxva2", "DirectX Video Acceleration 2", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_ENCODER_ENGINE_DXVA2 }, 0, 0, VE, "engine" }, + { "d3d11", "Direct2D11", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_ENCODER_ENGINE_D3D11 }, 0, 0, VE, "engine" }, + { "vulkan", "Vulkan", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_ENCODER_ENGINE_VULKAN }, 0, 0, VE, "engine" }, +#endif + { "rc", "Set the rate control mode", OFFSET(rate_control_mode), AV_OPT_TYPE_INT, { .i64 = AMF_VIDEO_ENCODER_HEVC_RATE_CONTROL_METHOD_UNKNOWN }, AMF_VIDEO_ENCODER_HEVC_RATE_CONTROL_METHOD_UNKNOWN, AMF_VIDEO_ENCODER_HEVC_RATE_CONTROL_METHOD_CBR, VE, "rc" }, { "cqp", "Constant Quantization Parameter", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_ENCODER_HEVC_RATE_CONTROL_METHOD_CONSTANT_QP }, 0, 0, VE, "rc" }, { "cbr", "Constant Bitrate", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_ENCODER_HEVC_RATE_CONTROL_METHOD_CBR }, 0, 0, VE, "rc" }, diff --git a/libavutil/hwcontext_amf.c b/libavutil/hwcontext_amf.c index b70ee90d40..faae57dda3 100644 --- a/libavutil/hwcontext_amf.c +++ b/libavutil/hwcontext_amf.c @@ -41,6 +41,7 @@ #else #include #endif +#include "libavutil/opt.h" /** * Error handling helper @@ -151,6 +152,51 @@ static int amf_init_device_ctx_object(AVHWDeviceContext *ctx) return 0; } +static AMF_RESULT amf_context_init_d3d11(AVHWDeviceContext * avctx) +{ + AMF_RESULT res; + AVAMFDeviceContext * ctx = avctx->hwctx; + res = ctx->context->pVtbl->InitDX11(ctx->context, NULL, AMF_DX11_1); + if (res == AMF_OK){ + av_log(avctx, AV_LOG_VERBOSE, "AMF initialisation successed via D3D11.\n"); + } + return res; +} + +static AMF_RESULT amf_context_init_dxva2(AVHWDeviceContext * avctx) +{ + AMF_RESULT res; + AVAMFDeviceContext * ctx = avctx->hwctx; + res = ctx->context->pVtbl->InitDX9(ctx->context, NULL); + if (res == AMF_OK){ + av_log(avctx, AV_LOG_VERBOSE, "AMF initialisation successed via DXVA2.\n"); + } + return res; +} + +static AMF_RESULT amf_context_init_vulkan(AVHWDeviceContext * avctx) +{ + AMF_RESULT res; + AVAMFDeviceContext * ctx = avctx->hwctx; + AMFContext1 *context1 = NULL; + AMFGuid guid = IID_AMFContext1(); + + res = ctx->context->pVtbl->QueryInterface(ctx->context, &guid, (void**)&context1); + AMFAV_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "CreateContext1() fialed with error %d\n", res); + + res = context1->pVtbl->InitVulkan(context1, NULL); + context1->pVtbl->Release(context1); + if (res != AMF_OK){ + if (res == AMF_NOT_SUPPORTED) + av_log(avctx, AV_LOG_VERBOSE, "AMF via vulkan is not supported on the given device.\n"); + else + av_log(avctx, AV_LOG_VERBOSE, "AMF failed to initialise on the given vulkan device. %d.\n", res); + return AMF_FAIL; + } + av_log(avctx, AV_LOG_VERBOSE, "AMF initialisation successed via vulkan.\n"); + return res; +} + static int amf_device_create(AVHWDeviceContext *ctx, const char *device, AVDictionary *opts, int flags) { @@ -158,35 +204,53 @@ static int amf_device_create(AVHWDeviceContext *ctx, const char *device, AMFContext1 *context1 = NULL; AMF_RESULT res; int err; + AVDictionaryEntry *e; + int preffered_engine = AMF_VIDEO_ENCODER_ENGINE_DEFAULT; err = amf_init_device_ctx_object(ctx); if(err < 0) return err; - res = amf_ctx->context->pVtbl->InitDX11(amf_ctx->context, NULL, AMF_DX11_1); - if (res == AMF_OK) { - av_log(ctx, AV_LOG_VERBOSE, "AMF initialisation succeeded via D3D11.\n"); - } else { - res = amf_ctx->context->pVtbl->InitDX9(amf_ctx->context, NULL); - if (res == AMF_OK) { - av_log(ctx, AV_LOG_VERBOSE, "AMF initialisation succeeded via D3D9.\n"); - } else { - AMFGuid guid = IID_AMFContext1(); - res = amf_ctx->context->pVtbl->QueryInterface(amf_ctx->context, &guid, (void**)&context1); - AMFAV_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "CreateContext1() failed with error %d\n", res); - - res = context1->pVtbl->InitVulkan(context1, NULL); - context1->pVtbl->Release(context1); + e = av_dict_get(opts, "engine", NULL, 0); + if (!!e) { + preffered_engine = atoi(e->value); + } + + res = AMF_FAIL; + switch(preffered_engine) { + case AMF_VIDEO_ENCODER_ENGINE_D3D11: + res = amf_context_init_d3d11(ctx); + break; + case AMF_VIDEO_ENCODER_ENGINE_DXVA2: + res = amf_context_init_dxva2(ctx); + break; + case AMF_VIDEO_ENCODER_ENGINE_VULKAN: + res = amf_context_init_vulkan(ctx); + break; + default: + break; + } + if (res != AMF_OK) { + if (preffered_engine != AMF_VIDEO_ENCODER_ENGINE_DEFAULT) + av_log(ctx, AV_LOG_ERROR, "AMF failed to initialise via preffered engine.\n"); + + if (preffered_engine != AMF_VIDEO_ENCODER_ENGINE_D3D11)//already tried in switch case + res = amf_context_init_d3d11(ctx); + + if (res != AMF_OK) { + if (preffered_engine != AMF_VIDEO_ENCODER_ENGINE_DXVA2) + res = amf_context_init_dxva2(ctx); + if (res != AMF_OK) { - if (res == AMF_NOT_SUPPORTED) - av_log(ctx, AV_LOG_ERROR, "AMF via Vulkan is not supported on the given device.\n"); - else - av_log(ctx, AV_LOG_ERROR, "AMF failed to initialise on the given Vulkan device: %d.\n", res); - return AVERROR(ENOSYS); + if (preffered_engine != AMF_VIDEO_ENCODER_ENGINE_VULKAN) + res = amf_context_init_vulkan(ctx); } - av_log(ctx, AV_LOG_VERBOSE, "AMF initialisation succeeded via Vulkan.\n"); } } + + if (res != AMF_OK) { + return AVERROR(ENOSYS); + } return 0; } diff --git a/libavutil/hwcontext_amf.h b/libavutil/hwcontext_amf.h index ae2a37f425..82e2ce7abb 100644 --- a/libavutil/hwcontext_amf.h +++ b/libavutil/hwcontext_amf.h @@ -30,6 +30,13 @@ #include "AMF/core/Context.h" #include "AMF/core/Factory.h" +enum AMF_VIDEO_ENCODER_PREFFERED_ENGINE +{ + AMF_VIDEO_ENCODER_ENGINE_DEFAULT = 0, + AMF_VIDEO_ENCODER_ENGINE_DXVA2, + AMF_VIDEO_ENCODER_ENGINE_D3D11, + AMF_VIDEO_ENCODER_ENGINE_VULKAN +}; /** * This struct is allocated as AVHWDeviceContext.hwctx From patchwork Thu Oct 15 00:16:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitrii Ovchinnikov X-Patchwork-Id: 22959 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 940A744928A for ; Thu, 15 Oct 2020 03:16:49 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 79BB168B9FA; Thu, 15 Oct 2020 03:16:49 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f52.google.com (mail-lf1-f52.google.com [209.85.167.52]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 989D968B83F for ; Thu, 15 Oct 2020 03:16:43 +0300 (EEST) Received: by mail-lf1-f52.google.com with SMTP id a9so1464689lfc.7 for ; Wed, 14 Oct 2020 17:16:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=up9zLwomOVC283GF3trWEj/k1MYbujHMYNjdwFsaGH4=; b=Y+HY0ufGKgYhmxIScftQfK8PU9utx/1ZSqtYTzQ/d/Y88kedOrmn6aLS6F/IfTZXWO w2WmAzSrFwBHvk6Ag0Prz1P/wReJvF/MGCHtNkaEjxKtXmc00AtHXdcWg91AmtxkEP32 5+S7dQh2ZGCLLGo+lQ+cCSRav6DUJXbHXxp8Pr1H5TIAYsD6ARjZc/jsiN0PJJA0Fycf zA/frK9PHym9EBXGOzENN2iM+9nU8VSw8spgv6hY8Y2c20VftlTQGzrtgVHOY7q7YtvF NcgTHTeOD7fqZF/bRyWt9GdHIl7QEpaTQnFEEaCKae2Xx9bGHAMTBOxY7yHtBMi25wZ6 TERg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=up9zLwomOVC283GF3trWEj/k1MYbujHMYNjdwFsaGH4=; b=CADoXX/YXA6/3xL4YyqsCu8Fg3y3P/l+wnUmp0xkVydJ/dabgckg9OdZ9aWJIIgpWX 6Ac1zbSqiLJyt2O85cGiFoGCzhyEL9CrkpQ5D4DCRO4QO+oqtXAgOjmcgxjwSlE9446e JiLEN2gXbrBCdYSpzoX2yBlC/a+Lzmu3FiPDOWm9crWRLyAd5jjQk4vbfiR8jnYs3NGc RtuI8vUOd05HV/4Njyjcho7+Z2GIefD6/eFAPoerkt4hH6j5oET77UWEWVhKUkFuSVOT 0K0j8FcdZyOmY1cI3hzrqWGSjJPdj8qZOfsV6mAbzcWWKM0OxL85WZR0vSMGW04zD9E0 mpMg== X-Gm-Message-State: AOAM533CbNXx4hxmFYRYS/tAn8wEqjVF6R9F8J5pGE/aJ7ifkdIF8ibV JBqGAHSuPMLP79czZRZ4J2yKZiax3duBdA== X-Google-Smtp-Source: ABdhPJw9Re2wEeBQ48ZeUScYaVeKxFUBDnroZA9au6PETOWiZ2jfTNrmYQ5FIYLo3JQGfy4oXtj4rA== X-Received: by 2002:a05:6512:209:: with SMTP id a9mr149495lfo.309.1602721002391; Wed, 14 Oct 2020 17:16:42 -0700 (PDT) Received: from localhost.localdomain (95-55-4-47.dynamic.avangarddsl.ru. [95.55.4.47]) by smtp.googlemail.com with ESMTPSA id q20sm333291lfp.43.2020.10.14.17.16.41 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 14 Oct 2020 17:16:41 -0700 (PDT) From: OvchinnikovDmitrii To: ffmpeg-devel@ffmpeg.org Date: Thu, 15 Oct 2020 03:16:25 +0300 Message-Id: <20201015001625.32832-5-ovchinnikov.dmitrii@gmail.com> X-Mailer: git-send-email 2.19.1.windows.1 In-Reply-To: <20201015001625.32832-1-ovchinnikov.dmitrii@gmail.com> References: <20201015001625.32832-1-ovchinnikov.dmitrii@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 5/5] lavf/vf_scale_amf: AMF scaler/format converter filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Alexander Kravchenko Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Alexander Kravchenko --- configure | 1 + libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/vf_scale_amf.c | 623 +++++++++++++++++++++++++++++++++++++ 4 files changed, 626 insertions(+) create mode 100644 libavfilter/vf_scale_amf.c diff --git a/configure b/configure index 75f0a0fcaa..0f87067aa4 100755 --- a/configure +++ b/configure @@ -3595,6 +3595,7 @@ rubberband_filter_deps="librubberband" sab_filter_deps="gpl swscale" scale2ref_filter_deps="swscale" scale_filter_deps="swscale" +scale_amf_filter_deps="amf" scale_qsv_filter_deps="libmfx" scdet_filter_select="scene_sad" select_filter_select="scene_sad" diff --git a/libavfilter/Makefile b/libavfilter/Makefile index e6d3c283da..c761a5bfff 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -372,6 +372,7 @@ OBJS-$(CONFIG_ROBERTS_OPENCL_FILTER) += vf_convolution_opencl.o opencl.o OBJS-$(CONFIG_ROTATE_FILTER) += vf_rotate.o OBJS-$(CONFIG_SAB_FILTER) += vf_sab.o OBJS-$(CONFIG_SCALE_FILTER) += vf_scale.o scale_eval.o +OBJS-$(CONFIG_SCALE_AMF_FILTER) += vf_scale_amf.o scale_eval.o OBJS-$(CONFIG_SCALE_CUDA_FILTER) += vf_scale_cuda.o vf_scale_cuda.ptx.o scale_eval.o OBJS-$(CONFIG_SCALE_NPP_FILTER) += vf_scale_npp.o scale_eval.o OBJS-$(CONFIG_SCALE_QSV_FILTER) += vf_scale_qsv.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index fa91e608e4..ff40aad9f3 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -354,6 +354,7 @@ extern AVFilter ff_vf_roberts_opencl; extern AVFilter ff_vf_rotate; extern AVFilter ff_vf_sab; extern AVFilter ff_vf_scale; +extern AVFilter ff_vf_scale_amf; extern AVFilter ff_vf_scale_cuda; extern AVFilter ff_vf_scale_npp; extern AVFilter ff_vf_scale_qsv; diff --git a/libavfilter/vf_scale_amf.c b/libavfilter/vf_scale_amf.c new file mode 100644 index 0000000000..de38cb1a12 --- /dev/null +++ b/libavfilter/vf_scale_amf.c @@ -0,0 +1,623 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * scale video filter - AMF + */ + +#include +#include + +#include "libavutil/avassert.h" +#include "libavutil/imgutils.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "libavutil/time.h" + +#include "libavutil/hwcontext.h" +#include "libavutil/hwcontext_amf.h" + +#include "AMF/components/VideoConverter.h" + +#include "avfilter.h" +#include "formats.h" +#include "internal.h" +#include "video.h" +#include "scale_eval.h" + +#if CONFIG_DXVA2 +#include +#endif + +#if CONFIG_D3D11VA +#include +#endif + +#define AMFAV_RETURN_IF_FALSE(avctx, exp, ret_value, /*message,*/ ...) \ + if (!(exp)) { \ + av_log(avctx, AV_LOG_ERROR, __VA_ARGS__); \ + return ret_value; \ + } + +#define AMFAV_GOTO_FAIL_IF_FALSE(avctx, exp, ret_value, /*message,*/ ...) \ + if (!(exp)) { \ + av_log(avctx, AV_LOG_ERROR, __VA_ARGS__); \ + ret = ret_value; \ + goto fail; \ + } + +typedef struct FormatMap { + enum AVPixelFormat av_format; + enum AMF_SURFACE_FORMAT amf_format; +} FormatMap; + +static const FormatMap format_map[] = +{ + { AV_PIX_FMT_NV12, AMF_SURFACE_NV12 }, + + { AV_PIX_FMT_BGR0, AMF_SURFACE_BGRA }, + { AV_PIX_FMT_BGRA, AMF_SURFACE_BGRA }, + + { AV_PIX_FMT_RGB0, AMF_SURFACE_RGBA }, + { AV_PIX_FMT_RGBA, AMF_SURFACE_RGBA }, + + { AV_PIX_FMT_0RGB, AMF_SURFACE_ARGB }, + { AV_PIX_FMT_ARGB, AMF_SURFACE_ARGB }, + + { AV_PIX_FMT_GRAY8, AMF_SURFACE_GRAY8 }, + { AV_PIX_FMT_YUV420P, AMF_SURFACE_YUV420P }, + { AV_PIX_FMT_YUYV422, AMF_SURFACE_YUY2 }, +}; + +static enum AMF_SURFACE_FORMAT amf_av_to_amf_format(enum AVPixelFormat fmt) +{ + int i; + for (i = 0; i < amf_countof(format_map); i) { + if (format_map[i].av_format == fmt) { + return format_map[i].amf_format; + } + } + return AMF_SURFACE_UNKNOWN; +} + +typedef struct AMFScaleContext { + const AVClass *class; + + int width, height; + enum AVPixelFormat format; + int scale_type; + + char *w_expr; + char *h_expr; + char *format_str; + + AMFComponent *converter; + AVBufferRef *amf_device_ref; + + AVBufferRef *hwframes_in_ref; + AVBufferRef *hwframes_out_ref; + AVBufferRef *hwdevice_ref; + + AMFContext *context; + AMFFactory *factory; + +} AMFScaleContext; + + +static int amf_copy_surface(AVFilterContext *avctx, const AVFrame *frame, + AMFSurface* surface) +{ + AMFPlane *plane; + uint8_t *dst_data[4]; + int dst_linesize[4]; + int planes; + int i; + + planes = surface->pVtbl->GetPlanesCount(surface); + av_assert0(planes < FF_ARRAY_ELEMS(dst_data)); + + for (i = 0; i < planes; i++) { + plane = surface->pVtbl->GetPlaneAt(surface, i++); + dst_data[i] = plane->pVtbl->GetNative(plane); + dst_linesize[i] = plane->pVtbl->GetHPitch(plane); + } + av_image_copy(dst_data, dst_linesize, + (const uint8_t**)frame->data, frame->linesize, frame->format, + frame->width, frame->height); + + return 0; +} + +static void amf_free_amfsurface(void *opaque, uint8_t *data) +{ + AMFSurface *surface = (AMFSurface*)(opaque); + surface->pVtbl->Release(surface); +} + +static AVFrame *amf_amfsurface_to_avframe(AVFilterContext *avctx, AMFSurface* pSurface) +{ + AVFrame *frame = av_frame_alloc(); + + if (!frame) + return NULL; + + switch (pSurface->pVtbl->GetMemoryType(pSurface)) + { +#if CONFIG_D3D11VA + case AMF_MEMORY_DX11: + { + AMFPlane *plane0 = pSurface->pVtbl->GetPlaneAt(pSurface, 0); + frame->data[0] = plane0->pVtbl->GetNative(plane0); + frame->data[1] = (uint8_t*)(intptr_t)0; + + frame->buf[0] = av_buffer_create(NULL, + 0, + amf_free_amfsurface, + pSurface, + AV_BUFFER_FLAG_READONLY); + pSurface->pVtbl->Acquire(pSurface); + } + break; +#endif +#if CONFIG_DXVA2 + case AMF_MEMORY_DX9: + { + AMFPlane *plane0 = pSurface->pVtbl->GetPlaneAt(pSurface, 0); + frame->data[3] = plane0->pVtbl->GetNative(plane0); + + frame->buf[0] = av_buffer_create(NULL, + 0, + amf_free_amfsurface, + pSurface, + AV_BUFFER_FLAG_READONLY); + pSurface->pVtbl->Acquire(pSurface); + } + break; +#endif + default: + { + av_assert0(0);//should not happen + } + } + + return frame; +} + +static int amf_avframe_to_amfsurface(AVFilterContext *avctx, const AVFrame *frame, AMFSurface** ppSurface) +{ + AMFScaleContext *ctx = avctx->priv; + AMFSurface *surface; + AMF_RESULT res; + int hw_surface = 0; + + switch (frame->format) { +#if CONFIG_D3D11VA + case AV_PIX_FMT_D3D11: + { + static const GUID AMFTextureArrayIndexGUID = { 0x28115527, 0xe7c3, 0x4b66, { 0x99, 0xd3, 0x4f, 0x2a, 0xe6, 0xb4, 0x7f, 0xaf } }; + ID3D11Texture2D *texture = (ID3D11Texture2D*)frame->data[0]; // actual texture + int index = (intptr_t)frame->data[1]; // index is a slice in texture array is - set to tell AMF which slice to use + texture->lpVtbl->SetPrivateData(texture, &AMFTextureArrayIndexGUID, sizeof(index), &index); + + res = ctx->context->pVtbl->CreateSurfaceFromDX11Native(ctx->context, texture, &surface, NULL); // wrap to AMF surface + AMFAV_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR(ENOMEM), "CreateSurfaceFromDX11Native() failed with error %d\n", res); + hw_surface = 1; + } + break; +#endif +#if CONFIG_DXVA2 + case AV_PIX_FMT_DXVA2_VLD: + { + IDirect3DSurface9 *texture = (IDirect3DSurface9 *)frame->data[3]; // actual texture + + res = ctx->context->pVtbl->CreateSurfaceFromDX9Native(ctx->context, texture, &surface, NULL); // wrap to AMF surface + AMFAV_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR(ENOMEM), "CreateSurfaceFromDX9Native() failed with error %d\n", res); + hw_surface = 1; + } + break; +#endif + default: + { + AMF_SURFACE_FORMAT amf_fmt = amf_av_to_amf_format(frame->format); + res = ctx->context->pVtbl->AllocSurface(ctx->context, AMF_MEMORY_HOST, amf_fmt, frame->width, frame->height, &surface); + AMFAV_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR(ENOMEM), "AllocSurface() failed with error %d\n", res); + amf_copy_surface(avctx, frame, surface); + } + break; + } + + if (hw_surface) { + // input HW surfaces can be vertically aligned by 16; tell AMF the real size + surface->pVtbl->SetCrop(surface, 0, 0, frame->width, frame->height); + } + + surface->pVtbl->SetPts(surface, frame->pts); + *ppSurface = surface; + return 0; +} + +static int amf_scale_init(AVFilterContext *avctx) +{ + AMFScaleContext *ctx = avctx->priv; + + if (!strcmp(ctx->format_str, "same")) { + ctx->format = AV_PIX_FMT_NONE; + } else { + ctx->format = av_get_pix_fmt(ctx->format_str); + if (ctx->format == AV_PIX_FMT_NONE) { + av_log(avctx, AV_LOG_ERROR, "Unrecognized pixel format: %s\n", ctx->format_str); + return AVERROR(EINVAL); + } + } + + return 0; +} + +static void amf_scale_uninit(AVFilterContext *avctx) +{ + AMFScaleContext *ctx = avctx->priv; + + if (ctx->converter) { + ctx->converter->pVtbl->Terminate(ctx->converter); + ctx->converter->pVtbl->Release(ctx->converter); + ctx->converter = NULL; + } + + av_buffer_unref(&ctx->amf_device_ref); + av_buffer_unref(&ctx->hwdevice_ref); + av_buffer_unref(&ctx->hwframes_in_ref); + av_buffer_unref(&ctx->hwframes_out_ref); +} + +static int amf_scale_query_formats(AVFilterContext *avctx) +{ + const enum AVPixelFormat *output_pix_fmts; + AVFilterFormats *input_formats; + int err; + int i; + static const enum AVPixelFormat input_pix_fmts[] = { + AV_PIX_FMT_NV12, + AV_PIX_FMT_0RGB, + AV_PIX_FMT_BGR0, + AV_PIX_FMT_RGB0, + AV_PIX_FMT_GRAY8, + AV_PIX_FMT_YUV420P, + AV_PIX_FMT_YUYV422, + AV_PIX_FMT_NONE, + }; + static const enum AVPixelFormat output_pix_fmts_default[] = { + AV_PIX_FMT_D3D11, + AV_PIX_FMT_DXVA2_VLD, + AV_PIX_FMT_NONE, + }; + output_pix_fmts = output_pix_fmts_default; + + //in case if hw_device_ctx is set to DXVA2 we change order of pixel formats to set DXVA2 be choosen by default + //The order is ignored if hw_frames_ctx is not NULL on the config_output stage + if (avctx->hw_device_ctx) { + AVHWDeviceContext *device_ctx = (AVHWDeviceContext*)avctx->hw_device_ctx->data; + + switch (device_ctx->type) { + #if CONFIG_D3D11VA + case AV_HWDEVICE_TYPE_D3D11VA: + { + static const enum AVPixelFormat output_pix_fmts_d3d11[] = { + AV_PIX_FMT_D3D11, + AV_PIX_FMT_NONE, + }; + output_pix_fmts = output_pix_fmts_d3d11; + } + break; + #endif + #if CONFIG_DXVA2 + case AV_HWDEVICE_TYPE_DXVA2: + { + static const enum AVPixelFormat output_pix_fmts_dxva2[] = { + AV_PIX_FMT_DXVA2_VLD, + AV_PIX_FMT_NONE, + }; + output_pix_fmts = output_pix_fmts_dxva2; + } + break; + #endif + default: + { + av_log(avctx, AV_LOG_ERROR, "Unsupported device : %s\n", av_hwdevice_get_type_name(device_ctx->type)); + return AVERROR(EINVAL); + } + break; + } + } + + input_formats = ff_make_format_list(output_pix_fmts); + if (!input_formats) { + return AVERROR(ENOMEM); + } + + for (i = 0; input_pix_fmts[i] != AV_PIX_FMT_NONE; i++) { + err = ff_add_format(&input_formats, input_pix_fmts[i]); + if (err < 0) + return err; + } + + if ((err = ff_formats_ref(input_formats, &avctx->inputs[0]->outcfg.formats)) < 0 || + (err = ff_formats_ref(ff_make_format_list(output_pix_fmts), + &avctx->outputs[0]->incfg.formats)) < 0) + return err; + + return 0; +} + +static int amf_scale_config_output(AVFilterLink *outlink) +{ + AVFilterContext *avctx = outlink->src; + AVFilterLink *inlink = avctx->inputs[0]; + AMFScaleContext *ctx = avctx->priv; + AVAMFDeviceContext *amf_ctx; + AVHWFramesContext *hwframes_out; + enum AVPixelFormat pix_fmt_in; + AMFSize out_size; + int err; + AMF_RESULT res; + + if ((err = ff_scale_eval_dimensions(avctx, + ctx->w_expr, ctx->h_expr, + inlink, outlink, + &ctx->width, &ctx->height)) < 0) + return err; + + av_buffer_unref(&ctx->amf_device_ref); + av_buffer_unref(&ctx->hwframes_in_ref); + av_buffer_unref(&ctx->hwframes_out_ref); + + if (inlink->hw_frames_ctx) { + AVHWFramesContext *frames_ctx = (AVHWFramesContext*)inlink->hw_frames_ctx->data; + + if (amf_av_to_amf_format(frames_ctx->sw_format) == AMF_SURFACE_UNKNOWN) { + av_log(avctx, AV_LOG_ERROR, "Format of input frames context (%s) is not supported by AMF.\n", + av_get_pix_fmt_name(frames_ctx->sw_format)); + return AVERROR(EINVAL); + } + + err = av_hwdevice_ctx_create_derived(&ctx->amf_device_ref, AV_HWDEVICE_TYPE_AMF, frames_ctx->device_ref, 0); + if (err < 0) + return err; + + ctx->hwframes_in_ref = av_buffer_ref(inlink->hw_frames_ctx); + if (!ctx->hwframes_in_ref) + return AVERROR(ENOMEM); + + ctx->hwframes_out_ref = av_hwframe_ctx_alloc(frames_ctx->device_ref); + if (!ctx->hwframes_out_ref) + return AVERROR(ENOMEM); + + hwframes_out = (AVHWFramesContext*)ctx->hwframes_out_ref->data; + hwframes_out->format = outlink->format; + hwframes_out->sw_format = frames_ctx->sw_format; + pix_fmt_in = frames_ctx->sw_format; + + } else if (avctx->hw_device_ctx) { + err = av_hwdevice_ctx_create_derived(&ctx->amf_device_ref, AV_HWDEVICE_TYPE_AMF, avctx->hw_device_ctx, 0); + if (err < 0) + return err; + + ctx->hwdevice_ref = av_buffer_ref(avctx->hw_device_ctx); + if (!ctx->hwdevice_ref) + return AVERROR(ENOMEM); + + ctx->hwframes_out_ref = av_hwframe_ctx_alloc(ctx->hwdevice_ref); + if (!ctx->hwframes_out_ref) + return AVERROR(ENOMEM); + + hwframes_out = (AVHWFramesContext*)ctx->hwframes_out_ref->data; + hwframes_out->format = outlink->format; + hwframes_out->sw_format = inlink->format; + pix_fmt_in = inlink->format; + + } else { + av_log(ctx, AV_LOG_ERROR, "A hardware device reference is required to init hwcontext_amf.\n"); + return AVERROR(EINVAL); + } + + if (ctx->format != AV_PIX_FMT_NONE) { + hwframes_out->sw_format = ctx->format; + } + + outlink->w = ctx->width; + outlink->h = ctx->height; + + hwframes_out->width = outlink->w; + hwframes_out->height = outlink->h; + + err = av_hwframe_ctx_init(ctx->hwframes_out_ref); + if (err < 0) + return err; + + outlink->hw_frames_ctx = av_buffer_ref(ctx->hwframes_out_ref); + if (!outlink->hw_frames_ctx) { + return AVERROR(ENOMEM); + } + + amf_ctx = ((AVHWDeviceContext*)ctx->amf_device_ref->data)->hwctx; + ctx->context = amf_ctx->context; + ctx->factory = amf_ctx->factory; + + res = ctx->factory->pVtbl->CreateComponent(ctx->factory, ctx->context, AMFVideoConverter, &ctx->converter); + AMFAV_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_FILTER_NOT_FOUND, "CreateComponent(%ls) failed with error %d\n", AMFVideoConverter, res); + + AMF_ASSIGN_PROPERTY_INT64(res, ctx->converter, AMF_VIDEO_CONVERTER_OUTPUT_FORMAT, (amf_int32)amf_av_to_amf_format(hwframes_out->sw_format)); + AMFAV_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "AMFConverter-SetProperty() failed with error %d\n", res); + + out_size.width = outlink->w; + out_size.height = outlink->h; + AMF_ASSIGN_PROPERTY_SIZE(res, ctx->converter, AMF_VIDEO_CONVERTER_OUTPUT_SIZE, out_size); + AMFAV_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "AMFConverter-SetProperty() failed with error %d\n", res); + + AMF_ASSIGN_PROPERTY_INT64(res, ctx->converter, AMF_VIDEO_CONVERTER_SCALE, (amf_int32)ctx->scale_type); + AMFAV_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "AMFConverter-SetProperty() failed with error %d\n", res); + + + res = ctx->converter->pVtbl->Init(ctx->converter, amf_av_to_amf_format(pix_fmt_in), inlink->w, inlink->h); + AMFAV_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "AMFConverter-Init() failed with error %d\n", res); + + return 0; +} + +static int amf_scale_filter_frame(AVFilterLink *inlink, AVFrame *in) +{ + AVFilterContext *avctx = inlink->dst; + AMFScaleContext *ctx = avctx->priv; + AVFilterLink *outlink = avctx->outputs[0]; + AMF_RESULT res; + AMFSurface *surface_in; + AMFSurface *surface_out; + AMFData *data_out; + enum AMF_VIDEO_CONVERTER_COLOR_PROFILE_ENUM amf_color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN; + + AVFrame *out = NULL; + int ret = 0; + + if (!ctx->converter) + return AVERROR(EINVAL); + + ret = amf_avframe_to_amfsurface(avctx, in, &surface_in); + if (ret < 0) + goto fail; + + res = ctx->converter->pVtbl->SubmitInput(ctx->converter, (AMFData*)surface_in); + AMFAV_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "SubmitInput() failed with error %d\n", res); + + res = ctx->converter->pVtbl->QueryOutput(ctx->converter, &data_out); + AMFAV_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "QueryOutput() failed with error %d\n", res); + + if (data_out) { + AMFGuid guid = IID_AMFSurface(); + data_out->pVtbl->QueryInterface(data_out, &guid, (void**)&surface_out); // query for buffer interface + data_out->pVtbl->Release(data_out); + } + + out = amf_amfsurface_to_avframe(avctx, surface_out); + + ret = av_frame_copy_props(out, in); + if (ret < 0) + goto fail; + + switch(in->colorspace) { + case AVCOL_SPC_BT470BG: + case AVCOL_SPC_SMPTE170M: + case AVCOL_SPC_SMPTE240M: + amf_color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_601; + break; + case AVCOL_SPC_BT709: + amf_color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_709; + break; + case AVCOL_SPC_BT2020_NCL: + case AVCOL_SPC_BT2020_CL: + amf_color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_2020; + break; + case AVCOL_SPC_RGB: + amf_color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_JPEG; + break; + default: + amf_color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN; + break; + } + + if (amf_color_profile != AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN) { + AMF_ASSIGN_PROPERTY_INT64(res, ctx->converter, AMF_VIDEO_CONVERTER_COLOR_PROFILE, amf_color_profile); + } + + out->format = outlink->format; + out->width = outlink->w; + out->height = outlink->h; + + out->hw_frames_ctx = av_buffer_ref(ctx->hwframes_out_ref); + if (!out->hw_frames_ctx) { + ret = AVERROR(ENOMEM); + goto fail; + } + + surface_in->pVtbl->Release(surface_in); + surface_out->pVtbl->Release(surface_out); + + if (inlink->sample_aspect_ratio.num) { + outlink->sample_aspect_ratio = av_mul_q((AVRational){outlink->h * inlink->w, outlink->w * inlink->h}, inlink->sample_aspect_ratio); + } else + outlink->sample_aspect_ratio = inlink->sample_aspect_ratio; + + av_frame_free(&in); + return ff_filter_frame(outlink, out); +fail: + av_frame_free(&in); + av_frame_free(&out); + return ret; +} + +#define OFFSET(x) offsetof(AMFScaleContext, x) +#define FLAGS AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_FILTERING_PARAM +static const AVOption scale_amf_options[] = { + { "w", "Output video width", OFFSET(w_expr), AV_OPT_TYPE_STRING, { .str = "iw" }, .flags = FLAGS }, + { "h", "Output video height", OFFSET(h_expr), AV_OPT_TYPE_STRING, { .str = "ih" }, .flags = FLAGS }, + { "format", "Output pixel format", OFFSET(format_str), AV_OPT_TYPE_STRING, { .str = "same" }, .flags = FLAGS }, + + { "scale_type", "Scale type", OFFSET(scale_type), AV_OPT_TYPE_INT, { .i64 = AMF_VIDEO_CONVERTER_SCALE_BILINEAR }, + AMF_VIDEO_CONVERTER_SCALE_BILINEAR, AMF_VIDEO_CONVERTER_SCALE_BICUBIC, FLAGS, "scale_type" }, + { "bilinear", "Bilinear", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_CONVERTER_SCALE_BILINEAR }, 0, 0, FLAGS, "scale_type" }, + { "bicubic", "Bicubic", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_CONVERTER_SCALE_BICUBIC }, 0, 0, FLAGS, "scale_type" }, + + { NULL }, +}; + + +AVFILTER_DEFINE_CLASS(scale_amf); + +static const AVFilterPad amf_scale_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .filter_frame = amf_scale_filter_frame, + }, + { NULL } +}; + +static const AVFilterPad amf_scale_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = amf_scale_config_output, + }, + { NULL } +}; + +AVFilter ff_vf_scale_amf = { + .name = "scale_amf", + .description = NULL_IF_CONFIG_SMALL("AMF video scaling and format conversion"), + + .init = amf_scale_init, + .uninit = amf_scale_uninit, + .query_formats = amf_scale_query_formats, + + .priv_size = sizeof(AMFScaleContext), + .priv_class = &scale_amf_class, + + .inputs = amf_scale_inputs, + .outputs = amf_scale_outputs, + + .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, +}; \ No newline at end of file