From patchwork Wed May 1 18:38:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitrii Ovchinnikov X-Patchwork-Id: 48444 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:1203:b0:1a9:af23:56c1 with SMTP id v3csp1103618pzf; Wed, 1 May 2024 11:38:29 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCXchsdoQxhv48aPStUhoqr9v6CzYuxrhrozgqzfI9hKY2bj3qVfKyLEgHiozhUxWqTzvyGBi0/I3KVsgDmo6SDumorR7C5XxQcEvA== X-Google-Smtp-Source: AGHT+IEDaUb+emEyVKWwBVK/Fc4iSHfe/R8NplX4xbGNKQ3F/SSpccpTlOmKP9Z4AynR2F+kgiiP X-Received: by 2002:a05:6512:114a:b0:51b:d382:9fe6 with SMTP id m10-20020a056512114a00b0051bd3829fe6mr2237530lfg.47.1714588709558; Wed, 01 May 2024 11:38:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1714588709; cv=none; d=google.com; s=arc-20160816; b=Ma7jc+ez0gb+5Xc7v/H7EwRVAj6dGTjB1jCLctszR7owvAg3J+cfxBaYgcUz+OFBYZ X2oEFRAHNNeigBqpARA0BV8nec5P4EFiR4U6syDNI76fGK1F6t6k99arkJwkiisSf/jV eAfbDlAXlco9rHgE+146oKGpHlmBfd1y+ddyzN+zBSHWXaLTkl7DgVbdfZWlBi7Dm678 2NEejAjuqcyNZiem5ysHugG0tJyRo0OFaNw6Y58Th3i7yrItbs3RanPIvURjfX/De4Gv xkrbhM4c64GDnO4H8OHKjeezISkbicEecbDeVkER1VlvNNbE1Z+elNAxGMU7hf3RXIgA yeGQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=jcvGxf1cnLnZOiyw8+rUV1ksc+xT/gNfHx7gAb2Zu9o=; fh=dTRBPGPJ3wU94AH5tguagErHz3vxaVKbj09dNGTx2d0=; b=ZQGH9j7E8i5MGg/YMS/8z519EpOUORKWx5y7OwVP7ru2Ek/UGZWtzwnB6ad6F5FiLl nY5gJbeubgRtGUiy9dDZSTAkgQn32tUBb2UjuHL8DZNab3AcvW+xVQjjMmakNdwsdHnD 5v/r7dTj5CiAsy637Mc6dc/wzdAn+al0+QABFMvy+JGpMqbennB40ftGx1hLWtKrU6n/ NZ0TL5RyIckyLE2QIupHq9z9l+z97KcL7sYQyy0Wvlv/80H8u79RdVgN7kp/nPMTFdMg QC+f8DaZGSl1pA80Hn+nKfqXyjubqfqk7wn8jBCWMs8viUcZamDv53rS0ZCkRkpQf203 1sSQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=HktiWz4k; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id du15-20020a17090772cf00b00a5900eb018bsi4081844ejc.969.2024.05.01.11.38.28; Wed, 01 May 2024 11:38:29 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=HktiWz4k; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A672C68D67E; Wed, 1 May 2024 21:38:23 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f50.google.com (mail-ej1-f50.google.com [209.85.218.50]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 431B568D5FE for ; Wed, 1 May 2024 21:38:16 +0300 (EEST) Received: by mail-ej1-f50.google.com with SMTP id a640c23a62f3a-a58e7628aeaso584031966b.2 for ; Wed, 01 May 2024 11:38:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1714588695; x=1715193495; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=S5n+CPpvRX74Ete1ChFGEfvbI3uSvEjPGTHDXe0Epgk=; b=HktiWz4kDaq3Q08CjDnbLF17BQXg3LqD/CldIbjzeyCpvtVcKFboZUXHeed+Nfn69F AQFEHsSahp+7oz1lZrso9Qv0KNy3fVZJgDhuHmhNyGOHrGbno0buf1Dj1tKFwb+E9chI V2PTNe/sVfKCo5/fRns3re3FEf0pYpPf3udq5xQUhKwg9OTPy0EiA7yjjaTqApw7dK4g k/gG4YF3wF0Xt5LpJJeMRhKZwnHjGBts0tPRk92ZuGlhk6iGggFwZzMPH14kuj8x4o/g prvSLBqjQ9dnU0KUTgGnooVgtAVYgg54IWca4GBG6eD2q/OYFwuHrWEoCsYSJxbbWjUv QVyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714588695; x=1715193495; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=S5n+CPpvRX74Ete1ChFGEfvbI3uSvEjPGTHDXe0Epgk=; b=MksCboSYxH8di0CtNzPY4/6TmRBmsUGzIflmr7+Fn+qvkIEfVerfl9obKOO4T9dOMj SBieNU0S8UrwSC/1SDeOfAvc3PpFCOyqpP169mXbRQeajK5gROiSTjx1JMB+pvVaIUhm Esg4A9yWx1tbuUNtoLsRIGTXjm8sbTgSMOqLhsljqh59eRFLf36q8Hoz3IJ4s0ziuLUI GXy3a6mKoAtzek/D8n0469tZ6hZftIXEYZzJN+bGRYGvikw8+cynUiy3/a726uAASfjD Ql5SE9bS2nvTV2jzi9qxFO7Dzx7Qz2cg4ujekWj/QKJh2d/oD0/gzYYrxHSk2ytxa8Vs 5OKg== X-Gm-Message-State: AOJu0YzoMVrzQBD+DomuG2BghjfbitjWiMcHOff4NPpB+L83aryfKtC5 US65dl33JvyZm6JfZZxph5N+ZqeZjV7OIg8ObxBR07qaMHt9QlFuk9PYbrc5 X-Received: by 2002:a17:907:9850:b0:a56:2556:8e3 with SMTP id jj16-20020a170907985000b00a56255608e3mr2176343ejc.73.1714588694809; Wed, 01 May 2024 11:38:14 -0700 (PDT) Received: from dovchinn.amd.com (cable-178-148-16-149.dynamic.sbb.rs. [178.148.16.149]) by smtp.gmail.com with ESMTPSA id kh20-20020a170906f81400b00a58bf434876sm7367337ejb.58.2024.05.01.11.38.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 May 2024 11:38:14 -0700 (PDT) From: Dmitrii Ovchinnikov To: ffmpeg-devel@ffmpeg.org Date: Wed, 1 May 2024 20:38:00 +0200 Message-Id: <20240501183809.1060-1-ovchinnikov.dmitrii@gmail.com> X-Mailer: git-send-email 2.38.1.windows.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 01/10, v2] avutil: add hwcontext_amf. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Dmitrii Ovchinnikov Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: jNVLw0vbVwDL Adds hwcontext_amf, which allows to use shared AMF context for the encoder, decoder and AMF-based filters, without copy to the host memory. It will also allow you to use some optimizations in the interaction of components (for example, SAV) and make a more manageable and optimal setup for using GPU devices with AMF in the case of a fully AMF pipeline. It will be a significant performance uplift when full AMF pipeline with filters is used. We also plan to add Compression artefact removal filter in near feature. v2: cleanup header files, fixed naming and formats --- libavutil/Makefile | 4 + libavutil/hwcontext.c | 4 + libavutil/hwcontext.h | 1 + libavutil/hwcontext_amf.c | 588 +++++++++++++++++++++++++++++ libavutil/hwcontext_amf.h | 41 ++ libavutil/hwcontext_amf_internal.h | 77 ++++ libavutil/hwcontext_internal.h | 1 + libavutil/pixdesc.c | 4 + libavutil/pixfmt.h | 5 + 9 files changed, 725 insertions(+) create mode 100644 libavutil/hwcontext_amf.c create mode 100644 libavutil/hwcontext_amf.h create mode 100644 libavutil/hwcontext_amf_internal.h diff --git a/libavutil/Makefile b/libavutil/Makefile index 6e6fa8d800..13c318560d 100644 --- a/libavutil/Makefile +++ b/libavutil/Makefile @@ -45,6 +45,7 @@ HEADERS = adler32.h \ hwcontext_d3d12va.h \ hwcontext_drm.h \ hwcontext_dxva2.h \ + hwcontext_amf.h \ hwcontext_qsv.h \ hwcontext_mediacodec.h \ hwcontext_opencl.h \ @@ -196,6 +197,7 @@ OBJS-$(CONFIG_CUDA) += hwcontext_cuda.o OBJS-$(CONFIG_D3D11VA) += hwcontext_d3d11va.o OBJS-$(CONFIG_D3D12VA) += hwcontext_d3d12va.o OBJS-$(CONFIG_DXVA2) += hwcontext_dxva2.o +OBJS-$(CONFIG_AMF) += hwcontext_amf.o OBJS-$(CONFIG_LIBDRM) += hwcontext_drm.o OBJS-$(CONFIG_MACOS_KPERF) += macos_kperf.o OBJS-$(CONFIG_MEDIACODEC) += hwcontext_mediacodec.o @@ -220,6 +222,8 @@ SKIPHEADERS-$(CONFIG_CUDA) += hwcontext_cuda_internal.h \ SKIPHEADERS-$(CONFIG_D3D11VA) += hwcontext_d3d11va.h SKIPHEADERS-$(CONFIG_D3D12VA) += hwcontext_d3d12va.h SKIPHEADERS-$(CONFIG_DXVA2) += hwcontext_dxva2.h +SKIPHEADERS-$(CONFIG_AMF) += hwcontext_amf.h \ + hwcontext_amf_internal SKIPHEADERS-$(CONFIG_QSV) += hwcontext_qsv.h SKIPHEADERS-$(CONFIG_OPENCL) += hwcontext_opencl.h SKIPHEADERS-$(CONFIG_VAAPI) += hwcontext_vaapi.h diff --git a/libavutil/hwcontext.c b/libavutil/hwcontext.c index fa99a0d8a4..f06d49c45c 100644 --- a/libavutil/hwcontext.c +++ b/libavutil/hwcontext.c @@ -65,6 +65,9 @@ static const HWContextType * const hw_table[] = { #endif #if CONFIG_VULKAN &ff_hwcontext_type_vulkan, +#endif +#if CONFIG_AMF + &ff_hwcontext_type_amf, #endif NULL, }; @@ -82,6 +85,7 @@ static const char *const hw_type_names[] = { [AV_HWDEVICE_TYPE_VIDEOTOOLBOX] = "videotoolbox", [AV_HWDEVICE_TYPE_MEDIACODEC] = "mediacodec", [AV_HWDEVICE_TYPE_VULKAN] = "vulkan", + [AV_HWDEVICE_TYPE_AMF] = "amf", }; typedef struct FFHWDeviceContext { diff --git a/libavutil/hwcontext.h b/libavutil/hwcontext.h index bac30debae..96042ba197 100644 --- a/libavutil/hwcontext.h +++ b/libavutil/hwcontext.h @@ -38,6 +38,7 @@ enum AVHWDeviceType { AV_HWDEVICE_TYPE_MEDIACODEC, AV_HWDEVICE_TYPE_VULKAN, AV_HWDEVICE_TYPE_D3D12VA, + AV_HWDEVICE_TYPE_AMF, }; /** diff --git a/libavutil/hwcontext_amf.c b/libavutil/hwcontext_amf.c new file mode 100644 index 0000000000..8edfb20fbb --- /dev/null +++ b/libavutil/hwcontext_amf.c @@ -0,0 +1,588 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "buffer.h" +#include "common.h" +#include "hwcontext.h" +#include "hwcontext_amf.h" +#include "hwcontext_internal.h" +#include "hwcontext_amf_internal.h" +#if CONFIG_VULKAN +#include "hwcontext_vulkan.h" +#endif +#if CONFIG_D3D11VA +#include "libavutil/hwcontext_d3d11va.h" +#endif +#if CONFIG_DXVA2 +#define COBJMACROS +#include "libavutil/hwcontext_dxva2.h" +#endif +#include "mem.h" +#include "pixdesc.h" +#include "pixfmt.h" +#include "imgutils.h" +#include "libavutil/avassert.h" +#include +#ifdef _WIN32 +#include "compat/w32dlfcn.h" +#else +#include +#endif +#define FFMPEG_AMF_WRITER_ID L"ffmpeg_amf" +int av_amf_load_library(AVAMFDeviceContextInternal* internal, void* avcl); + +typedef struct AMFFramesContext { + AMFSurface * surfaces; + int nb_surfaces; +} AMFFramesContext; + +typedef struct AVAMFFormatMap { + enum AVPixelFormat av_format; + enum AMF_SURFACE_FORMAT amf_format; +} FormatMap; + + +static const AVClass amflib_class = { + .class_name = "amf", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, +}; + +typedef struct AMFLibraryContext { + const AVClass *avclass; +} AMFLibraryContext; + +static AMFLibraryContext amflib_context = +{ + .avclass = &amflib_class, +}; + + +static void AMF_CDECL_CALL AMFTraceWriter_Write(AMFTraceWriter *pThis, + const wchar_t *scope, const wchar_t *message) +{ + AmfTraceWriter *tracer = (AmfTraceWriter*)pThis; + av_log(tracer->avcl, AV_LOG_DEBUG, "%ls: %ls", scope, message); // \n is provided from AMF +} + +static void AMF_CDECL_CALL AMFTraceWriter_Flush(AMFTraceWriter *pThis) +{ +} + +static AMFTraceWriterVtbl tracer_vtbl = +{ + .Write = AMFTraceWriter_Write, + .Flush = AMFTraceWriter_Flush, +}; + +AmfTraceWriter av_amf_trace_writer = +{ + .vtbl = &tracer_vtbl, + .avcl = &amflib_context, + .avctx = NULL +}; + +const FormatMap format_map[] = +{ + { AV_PIX_FMT_NONE, AMF_SURFACE_UNKNOWN }, + { AV_PIX_FMT_NV12, AMF_SURFACE_NV12 }, + { AV_PIX_FMT_BGR0, AMF_SURFACE_BGRA }, + { AV_PIX_FMT_RGB0, AMF_SURFACE_RGBA }, + { AV_PIX_FMT_BGRA, AMF_SURFACE_BGRA }, + { AV_PIX_FMT_ARGB, AMF_SURFACE_ARGB }, + { AV_PIX_FMT_RGBA, AMF_SURFACE_RGBA }, + { AV_PIX_FMT_GRAY8, AMF_SURFACE_GRAY8 }, + { AV_PIX_FMT_YUV420P, AMF_SURFACE_YUV420P }, + { AV_PIX_FMT_YUYV422, AMF_SURFACE_YUY2 }, + { AV_PIX_FMT_P010, AMF_SURFACE_P010 }, +}; + +enum AMF_SURFACE_FORMAT av_amf_av_to_amf_format(enum AVPixelFormat fmt) +{ + int i; + for (i = 0; i < amf_countof(format_map); i++) { + if (format_map[i].av_format == fmt) { + return format_map[i].amf_format; + } + } + return AMF_SURFACE_UNKNOWN; +} + +enum AVPixelFormat av_amf_to_av_format(enum AMF_SURFACE_FORMAT fmt) +{ + int i; + for (i = 0; i < amf_countof(format_map); i++) { + if (format_map[i].amf_format == fmt) { + return format_map[i].av_format; + } + } + return AMF_SURFACE_UNKNOWN; +} + +static const enum AVPixelFormat supported_formats[] = { + AV_PIX_FMT_NV12, + AV_PIX_FMT_YUV420P, + AV_PIX_FMT_BGRA, + AV_PIX_FMT_P010, + AV_PIX_FMT_YUV420P10, +#if CONFIG_D3D11VA + AV_PIX_FMT_D3D11, +#endif +#if CONFIG_DXVA2 + AV_PIX_FMT_DXVA2_VLD, +#endif + AV_PIX_FMT_AMF_SURFACE +}; + +static int amf_frames_get_constraints(AVHWDeviceContext *ctx, + const void *hwconfig, + AVHWFramesConstraints *constraints) +{ + int i; + + constraints->valid_sw_formats = av_malloc_array(FF_ARRAY_ELEMS(supported_formats) + 1, + sizeof(*constraints->valid_sw_formats)); + if (!constraints->valid_sw_formats) + return AVERROR(ENOMEM); + + for (i = 0; i < FF_ARRAY_ELEMS(supported_formats); i++) + constraints->valid_sw_formats[i] = supported_formats[i]; + constraints->valid_sw_formats[FF_ARRAY_ELEMS(supported_formats)] = AV_PIX_FMT_NONE; + + constraints->valid_hw_formats = av_malloc_array(2, sizeof(*constraints->valid_hw_formats)); + if (!constraints->valid_hw_formats) + return AVERROR(ENOMEM); + + constraints->valid_hw_formats[0] = AV_PIX_FMT_AMF_SURFACE; + constraints->valid_hw_formats[1] = AV_PIX_FMT_NONE; + + return 0; +} + +static void amf_dummy_free(void *opaque, uint8_t *data) +{ + +} + +static AVBufferRef *amf_pool_alloc(void *opaque, size_t size) +{ + AVHWFramesContext *hwfc = (AVHWFramesContext *)opaque; + AVBufferRef *buf; + + buf = av_buffer_create(NULL, NULL, amf_dummy_free, hwfc, AV_BUFFER_FLAG_READONLY); + if (!buf) { + av_log(hwfc, AV_LOG_ERROR, "Failed to create buffer for AMF context.\n"); + return NULL; + } + return buf; +} + +static int amf_frames_init(AVHWFramesContext *ctx) +{ + int i; + + for (i = 0; i < FF_ARRAY_ELEMS(supported_formats); i++) { + if (ctx->sw_format == supported_formats[i]) + break; + } + if (i == FF_ARRAY_ELEMS(supported_formats)) { + av_log(ctx, AV_LOG_ERROR, "Pixel format '%s' is not supported\n", + av_get_pix_fmt_name(ctx->sw_format)); + return AVERROR(ENOSYS); + } + + ffhwframesctx(ctx)->pool_internal = + av_buffer_pool_init2(sizeof(AMFSurface), ctx, + amf_pool_alloc, NULL); + + + return 0; +} + +static int amf_get_buffer(AVHWFramesContext *ctx, AVFrame *frame) +{ + frame->buf[0] = av_buffer_pool_get(ctx->pool); + if (!frame->buf[0]) + return AVERROR(ENOMEM); + + frame->data[0] = frame->buf[0]->data; + frame->format = AV_PIX_FMT_AMF_SURFACE; + frame->width = ctx->width; + frame->height = ctx->height; + return 0; +} + +static int amf_transfer_get_formats(AVHWFramesContext *ctx, + enum AVHWFrameTransferDirection dir, + enum AVPixelFormat **formats) +{ + enum AVPixelFormat *fmts; + + fmts = av_malloc_array(2, sizeof(*fmts)); + if (!fmts) + return AVERROR(ENOMEM); + + fmts[0] = ctx->sw_format; + fmts[1] = AV_PIX_FMT_NONE; + + *formats = fmts; + + return 0; +} + +static int amf_transfer_data_to(AVHWFramesContext *ctx, AVFrame *dst, + const AVFrame *src) +{ + AMFSurface* surface = (AMFSurface*)dst->data[0]; + AMFPlane *plane; + uint8_t *dst_data[4]; + int dst_linesize[4]; + int planes; + int i; + int w = FFMIN(dst->width, src->width); + int h = FFMIN(dst->height, src->height); + + planes = (int)surface->pVtbl->GetPlanesCount(surface); + av_assert0(planes < FF_ARRAY_ELEMS(dst_data)); + + for (i = 0; i < planes; i++) { + plane = surface->pVtbl->GetPlaneAt(surface, i); + dst_data[i] = plane->pVtbl->GetNative(plane); + dst_linesize[i] = plane->pVtbl->GetHPitch(plane); + } + av_image_copy(dst_data, dst_linesize, + (const uint8_t**)src->data, src->linesize, src->format, + w, h); + + return 0; +} +static int amf_transfer_data_from(AVHWFramesContext *ctx, AVFrame *dst, + const AVFrame *src) +{ + AMFSurface* surface = (AMFSurface*)src->data[0]; + AMFPlane *plane; + uint8_t *src_data[4]; + int src_linesize[4]; + int planes; + int i; + int w = FFMIN(dst->width, src->width); + int h = FFMIN(dst->height, src->height); + int ret; + + ret = surface->pVtbl->Convert(surface, AMF_MEMORY_HOST); + AMF_RETURN_IF_FALSE(ctx, ret == AMF_OK, AVERROR_UNKNOWN, "Convert(amf::AMF_MEMORY_HOST) failed with error %d\n", AVERROR_UNKNOWN); + + planes = (int)surface->pVtbl->GetPlanesCount(surface); + av_assert0(planes < FF_ARRAY_ELEMS(src_data)); + + for (i = 0; i < planes; i++) { + plane = surface->pVtbl->GetPlaneAt(surface, i); + src_data[i] = plane->pVtbl->GetNative(plane); + src_linesize[i] = plane->pVtbl->GetHPitch(plane); + } + av_image_copy(dst->data, dst->linesize, + (const uint8_t **)src_data, src_linesize, dst->format, + w, h); + surface->pVtbl->Release(surface); + return 0; +} + + +static void amf_device_uninit(AVHWDeviceContext *device_ctx) +{ + AVAMFDeviceContext *amf_ctx = device_ctx->hwctx; + av_buffer_unref(&amf_ctx->internal); +} + +static int amf_device_init(AVHWDeviceContext *ctx) +{ + AVAMFDeviceContext *amf_ctx = ctx->hwctx; + return av_amf_context_init((AVAMFDeviceContextInternal * )amf_ctx->internal->data, ctx); +} + +static int amf_device_create(AVHWDeviceContext *device_ctx, + const char *device, + AVDictionary *opts, int flags) +{ + AVAMFDeviceContext *ctx = device_ctx->hwctx; + AVAMFDeviceContextInternal *wrapped = av_mallocz(sizeof(*wrapped)); + ctx->internal = av_buffer_create((uint8_t *)wrapped, sizeof(*wrapped), + av_amf_context_internal_free, NULL, 0); + AVAMFDeviceContextInternal * internal = (AVAMFDeviceContextInternal * )ctx->internal->data; + int ret; + if ((ret = av_amf_load_library(internal, device_ctx)) == 0) { + if ((ret = av_amf_create_context(internal, device_ctx, "", opts, flags)) == 0){ + return 0; + } + } + amf_device_uninit(device_ctx); + return ret; +} + +static int amf_device_derive(AVHWDeviceContext *device_ctx, + AVHWDeviceContext *child_device_ctx, AVDictionary *opts, + int flags) +{ + AVAMFDeviceContext *ctx = device_ctx->hwctx; + AVAMFDeviceContextInternal * internal = (AVAMFDeviceContextInternal * )ctx->internal->data; + int ret; + + ret = amf_device_create(device_ctx, "", opts, flags); + if(ret < 0) + return ret; + + return av_amf_context_derive(internal, child_device_ctx, opts, flags); +} + +#if CONFIG_DXVA2 +static int amf_init_from_dxva2_device(AVAMFDeviceContextInternal * internal, AVDXVA2DeviceContext *hwctx) +{ + IDirect3DDevice9 *device; + HANDLE device_handle; + HRESULT hr; + AMF_RESULT res; + int ret; + + hr = IDirect3DDeviceManager9_OpenDeviceHandle(hwctx->devmgr, &device_handle); + if (FAILED(hr)) { + av_log(hwctx, AV_LOG_ERROR, "Failed to open device handle for Direct3D9 device: %lx.\n", (unsigned long)hr); + return AVERROR_EXTERNAL; + } + + hr = IDirect3DDeviceManager9_LockDevice(hwctx->devmgr, device_handle, &device, FALSE); + if (SUCCEEDED(hr)) { + IDirect3DDeviceManager9_UnlockDevice(hwctx->devmgr, device_handle, FALSE); + ret = 0; + } else { + av_log(hwctx, AV_LOG_ERROR, "Failed to lock device handle for Direct3D9 device: %lx.\n", (unsigned long)hr); + ret = AVERROR_EXTERNAL; + } + + + IDirect3DDeviceManager9_CloseDeviceHandle(hwctx->devmgr, device_handle); + + if (ret < 0) + return ret; + + res = internal->context->pVtbl->InitDX9(internal->context, device); + + IDirect3DDevice9_Release(device); + + if (res != AMF_OK) { + if (res == AMF_NOT_SUPPORTED) + av_log(hwctx, AV_LOG_ERROR, "AMF via D3D9 is not supported on the given device.\n"); + else + av_log(hwctx, AV_LOG_ERROR, "AMF failed to initialise on given D3D9 device: %d.\n", res); + return AVERROR(ENODEV); + } + internal->mem_type = AMF_MEMORY_DX9; + return 0; +} +#endif + +#if CONFIG_D3D11VA +static int amf_init_from_d3d11_device(AVAMFDeviceContextInternal* internal, AVD3D11VADeviceContext *hwctx) +{ + AMF_RESULT res; + res = internal->context->pVtbl->InitDX11(internal->context, hwctx->device, AMF_DX11_1); + if (res != AMF_OK) { + if (res == AMF_NOT_SUPPORTED) + av_log(hwctx, AV_LOG_ERROR, "AMF via D3D11 is not supported on the given device.\n"); + else + av_log(hwctx, AV_LOG_ERROR, "AMF failed to initialise on the given D3D11 device: %d.\n", res); + return AVERROR(ENODEV); + } + internal->mem_type = AMF_MEMORY_DX11; + return 0; +} +#endif + +int av_amf_context_init(AVAMFDeviceContextInternal* internal, void* avcl) +{ + AMFContext1 *context1 = NULL; + AMF_RESULT res; + + res = internal->context->pVtbl->InitDX11(internal->context, NULL, AMF_DX11_1); + if (res == AMF_OK) { + internal->mem_type = AMF_MEMORY_DX11; + av_log(avcl, AV_LOG_VERBOSE, "AMF initialisation succeeded via D3D11.\n"); + } else { + res = internal->context->pVtbl->InitDX9(internal->context, NULL); + if (res == AMF_OK) { + internal->mem_type = AMF_MEMORY_DX9; + av_log(avcl, AV_LOG_VERBOSE, "AMF initialisation succeeded via D3D9.\n"); + } else { + AMFGuid guid = IID_AMFContext1(); + res = internal->context->pVtbl->QueryInterface(internal->context, &guid, (void**)&context1); + AMF_RETURN_IF_FALSE(avcl, res == AMF_OK, AVERROR_UNKNOWN, "CreateContext1() failed with error %d\n", res); + + res = context1->pVtbl->InitVulkan(context1, NULL); + context1->pVtbl->Release(context1); + if (res != AMF_OK) { + if (res == AMF_NOT_SUPPORTED) + av_log(avcl, AV_LOG_ERROR, "AMF via Vulkan is not supported on the given device.\n"); + else + av_log(avcl, AV_LOG_ERROR, "AMF failed to initialise on the given Vulkan device: %d.\n", res); + return AVERROR(ENOSYS); + } + internal->mem_type = AMF_MEMORY_VULKAN; + av_log(avcl, AV_LOG_VERBOSE, "AMF initialisation succeeded via Vulkan.\n"); + } + } + return 0; +} +int av_amf_load_library(AVAMFDeviceContextInternal* internal, void* avcl) +{ + AMFInit_Fn init_fun; + AMFQueryVersion_Fn version_fun; + AMF_RESULT res; + + internal->library = dlopen(AMF_DLL_NAMEA, RTLD_NOW | RTLD_LOCAL); + AMF_RETURN_IF_FALSE(avcl, internal->library != NULL, + AVERROR_UNKNOWN, "DLL %s failed to open\n", AMF_DLL_NAMEA); + + init_fun = (AMFInit_Fn)dlsym(internal->library, AMF_INIT_FUNCTION_NAME); + AMF_RETURN_IF_FALSE(avcl, init_fun != NULL, AVERROR_UNKNOWN, "DLL %s failed to find function %s\n", AMF_DLL_NAMEA, AMF_INIT_FUNCTION_NAME); + + version_fun = (AMFQueryVersion_Fn)dlsym(internal->library, AMF_QUERY_VERSION_FUNCTION_NAME); + AMF_RETURN_IF_FALSE(avcl, version_fun != NULL, AVERROR_UNKNOWN, "DLL %s failed to find function %s\n", AMF_DLL_NAMEA, AMF_QUERY_VERSION_FUNCTION_NAME); + + res = version_fun(&internal->version); + AMF_RETURN_IF_FALSE(avcl, res == AMF_OK, AVERROR_UNKNOWN, "%s failed with error %d\n", AMF_QUERY_VERSION_FUNCTION_NAME, res); + res = init_fun(AMF_FULL_VERSION, &internal->factory); + AMF_RETURN_IF_FALSE(avcl, res == AMF_OK, AVERROR_UNKNOWN, "%s failed with error %d\n", AMF_INIT_FUNCTION_NAME, res); + res = internal->factory->pVtbl->GetTrace(internal->factory, &internal->trace); + AMF_RETURN_IF_FALSE(avcl, res == AMF_OK, AVERROR_UNKNOWN, "GetTrace() failed with error %d\n", res); + res = internal->factory->pVtbl->GetDebug(internal->factory, &internal->debug); + AMF_RETURN_IF_FALSE(avcl, res == AMF_OK, AVERROR_UNKNOWN, "GetDebug() failed with error %d\n", res); + return 0; +} + +int av_amf_create_context( AVAMFDeviceContextInternal * internal, + void* avcl, + const char *device, + AVDictionary *opts, int flags) +{ + AMF_RESULT res; + + internal->trace->pVtbl->EnableWriter(internal->trace, AMF_TRACE_WRITER_CONSOLE, 0); + internal->trace->pVtbl->SetGlobalLevel(internal->trace, AMF_TRACE_TRACE); + + // connect AMF logger to av_log + av_amf_trace_writer.avctx = avcl; + internal->trace->pVtbl->RegisterWriter(internal->trace, FFMPEG_AMF_WRITER_ID, (AMFTraceWriter*)&av_amf_trace_writer, 1); + internal->trace->pVtbl->SetWriterLevel(internal->trace, FFMPEG_AMF_WRITER_ID, AMF_TRACE_TRACE); + + res = internal->factory->pVtbl->CreateContext(internal->factory, &internal->context); + AMF_RETURN_IF_FALSE(avcl, res == AMF_OK, AVERROR_UNKNOWN, "CreateContext() failed with error %d\n", res); + + return 0; +} + +int av_amf_context_internal_create(AVAMFDeviceContextInternal * internal, + void* avcl, + const char *device, + AVDictionary *opts, int flags) +{ + int ret; + if ((ret = av_amf_load_library(internal, avcl)) == 0) { + if ((ret = av_amf_create_context(internal, avcl, "", opts, flags)) == 0){ + return 0; + } + } + av_amf_context_internal_free(0, (uint8_t *)internal); + return ret; +} + +void av_amf_context_internal_free(void *opaque, uint8_t *data) +{ + AVAMFDeviceContextInternal *amf_ctx = (AVAMFDeviceContextInternal *)data; + if (amf_ctx->context) { + amf_ctx->context->pVtbl->Terminate(amf_ctx->context); + amf_ctx->context->pVtbl->Release(amf_ctx->context); + amf_ctx->context = NULL; + } + + if (amf_ctx->trace) { + amf_ctx->trace->pVtbl->UnregisterWriter(amf_ctx->trace, FFMPEG_AMF_WRITER_ID); + } + + if(amf_ctx->library) { + dlclose(amf_ctx->library); + amf_ctx->library = NULL; + } + + amf_ctx->debug = NULL; + amf_ctx->version = 0; + av_free(amf_ctx); +} + +int av_amf_context_derive(AVAMFDeviceContextInternal * internal, + AVHWDeviceContext *child_device_ctx, AVDictionary *opts, + int flags) +{ + + switch (child_device_ctx->type) { + +#if CONFIG_DXVA2 + case AV_HWDEVICE_TYPE_DXVA2: + { + AVDXVA2DeviceContext *child_device_hwctx = child_device_ctx->hwctx; + return amf_init_from_dxva2_device(internal, child_device_hwctx); + } + break; +#endif + +#if CONFIG_D3D11VA + case AV_HWDEVICE_TYPE_D3D11VA: + { + AVD3D11VADeviceContext *child_device_hwctx = child_device_ctx->hwctx; + return amf_init_from_d3d11_device(internal, child_device_hwctx); + } + break; +#endif + default: + { + av_log(child_device_ctx, AV_LOG_ERROR, "AMF initialisation from a %s device is not supported.\n", + av_hwdevice_get_type_name(child_device_ctx->type)); + return AVERROR(ENOSYS); + } + } + return 0; +} + +const HWContextType ff_hwcontext_type_amf = { + .type = AV_HWDEVICE_TYPE_AMF, + .name = "AMF", + + .device_hwctx_size = sizeof(AVAMFDeviceContext), + .frames_hwctx_size = sizeof(AMFFramesContext), + + .device_create = amf_device_create, + .device_derive = amf_device_derive, + .device_init = amf_device_init, + .device_uninit = amf_device_uninit, + .frames_get_constraints = amf_frames_get_constraints, + .frames_init = amf_frames_init, + .frames_get_buffer = amf_get_buffer, + .transfer_get_formats = amf_transfer_get_formats, + .transfer_data_to = amf_transfer_data_to, + .transfer_data_from = amf_transfer_data_from, + + .pix_fmts = (const enum AVPixelFormat[]){ AV_PIX_FMT_AMF_SURFACE, AV_PIX_FMT_NONE }, +}; diff --git a/libavutil/hwcontext_amf.h b/libavutil/hwcontext_amf.h new file mode 100644 index 0000000000..36f13890a0 --- /dev/null +++ b/libavutil/hwcontext_amf.h @@ -0,0 +1,41 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + + +#ifndef AVUTIL_HWCONTEXT_AMF_H +#define AVUTIL_HWCONTEXT_AMF_H + +#include "pixfmt.h" + +#include "libavformat/avformat.h" +#include "libavutil/hwcontext.h" + +typedef struct AVAMFDeviceContextInternal AVAMFDeviceContextInternal; + +/** + * This struct is allocated as AVHWDeviceContext.hwctx + */ + +typedef struct AVAMFDeviceContext { + AVBufferRef *internal; +} AVAMFDeviceContext; + +enum AMF_SURFACE_FORMAT av_amf_av_to_amf_format(enum AVPixelFormat fmt); +enum AVPixelFormat av_amf_to_av_format(enum AMF_SURFACE_FORMAT fmt); + +#endif /* AVUTIL_HWCONTEXT_AMF_H */ diff --git a/libavutil/hwcontext_amf_internal.h b/libavutil/hwcontext_amf_internal.h new file mode 100644 index 0000000000..023dc8b4bb --- /dev/null +++ b/libavutil/hwcontext_amf_internal.h @@ -0,0 +1,77 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + + +#ifndef AVUTIL_HWCONTEXT_AMF_INTERNAL_H +#define AVUTIL_HWCONTEXT_AMF_INTERNAL_H +#include +#include +#include + +typedef struct AVAMFDeviceContextInternal { + amf_handle library; ///< handle to DLL library + AMFFactory *factory; ///< pointer to AMF factory + AMFDebug *debug; ///< pointer to AMF debug interface + AMFTrace *trace; ///< pointer to AMF trace interface + + amf_uint64 version; ///< version of AMF runtime + AMFContext *context; ///< AMF context + AMF_MEMORY_TYPE mem_type; +} AVAMFDeviceContextInternal; + +typedef struct AmfTraceWriter { + AMFTraceWriterVtbl *vtbl; + void *avctx; + void *avcl; +} AmfTraceWriter; + +extern AmfTraceWriter av_amf_trace_writer; + +int av_amf_context_init(AVAMFDeviceContextInternal* internal, void* avcl); +int av_amf_create_context( AVAMFDeviceContextInternal * internal, + void* avcl, + const char *device, + AVDictionary *opts, int flags); +int av_amf_context_internal_create(AVAMFDeviceContextInternal * internal, + void* avcl, + const char *device, + AVDictionary *opts, int flags); +void av_amf_context_internal_free(void *opaque, uint8_t *data); +int av_amf_context_derive(AVAMFDeviceContextInternal * internal, + AVHWDeviceContext *child_device_ctx, AVDictionary *opts, + int flags); + + +/** +* Error handling helper +*/ +#define AMF_RETURN_IF_FALSE(avctx, exp, ret_value, /*message,*/ ...) \ + if (!(exp)) { \ + av_log(avctx, AV_LOG_ERROR, __VA_ARGS__); \ + return ret_value; \ + } + +#define AMF_GOTO_FAIL_IF_FALSE(avctx, exp, ret_value, /*message,*/ ...) \ + if (!(exp)) { \ + av_log(avctx, AV_LOG_ERROR, __VA_ARGS__); \ + ret = ret_value; \ + goto fail; \ + } + +#define AMF_TIME_BASE_Q (AVRational){1, AMF_SECOND} +#endif /* AVUTIL_HWCONTEXT_AMF_INTERNAL_H */ \ No newline at end of file diff --git a/libavutil/hwcontext_internal.h b/libavutil/hwcontext_internal.h index e32b786238..db23579c9e 100644 --- a/libavutil/hwcontext_internal.h +++ b/libavutil/hwcontext_internal.h @@ -163,5 +163,6 @@ extern const HWContextType ff_hwcontext_type_vdpau; extern const HWContextType ff_hwcontext_type_videotoolbox; extern const HWContextType ff_hwcontext_type_mediacodec; extern const HWContextType ff_hwcontext_type_vulkan; +extern const HWContextType ff_hwcontext_type_amf; #endif /* AVUTIL_HWCONTEXT_INTERNAL_H */ diff --git a/libavutil/pixdesc.c b/libavutil/pixdesc.c index 1c0bcf2232..5438c97a8f 100644 --- a/libavutil/pixdesc.c +++ b/libavutil/pixdesc.c @@ -2119,6 +2119,10 @@ static const AVPixFmtDescriptor av_pix_fmt_descriptors[AV_PIX_FMT_NB] = { .name = "cuda", .flags = AV_PIX_FMT_FLAG_HWACCEL, }, + [AV_PIX_FMT_AMF_SURFACE] = { + .name = "amf", + .flags = AV_PIX_FMT_FLAG_HWACCEL, + }, [AV_PIX_FMT_AYUV64LE] = { .name = "ayuv64le", .nb_components = 4, diff --git a/libavutil/pixfmt.h b/libavutil/pixfmt.h index a7f50e1690..29f67e707f 100644 --- a/libavutil/pixfmt.h +++ b/libavutil/pixfmt.h @@ -439,6 +439,11 @@ enum AVPixelFormat { */ AV_PIX_FMT_D3D12, + /** + * HW acceleration through AMF. data[0] contain AMFSurface pointer + */ + AV_PIX_FMT_AMF_SURFACE, + AV_PIX_FMT_NB ///< number of pixel formats, DO NOT USE THIS if you want to link with shared libav* because the number of formats might differ between versions }; From patchwork Wed May 1 18:38:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitrii Ovchinnikov X-Patchwork-Id: 48446 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:1203:b0:1a9:af23:56c1 with SMTP id v3csp1103783pzf; Wed, 1 May 2024 11:38:42 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCXSoeXwxDCTMCuCLFufoFGth33yp8SD7gnUsVqMkGTmB7nYWcrNoBc7dHirEHzpR3parNevRxlvGoRonF1rY0foOvSRiKynGTjcvg== X-Google-Smtp-Source: AGHT+IFH0JxYbYwteIY/9zcbW8kP4Nu244ctvswK0PZric89QhU6nKQI2PYz+HfxTdzmI1UpoLkf X-Received: by 2002:a2e:780b:0:b0:2e1:ad93:b64d with SMTP id t11-20020a2e780b000000b002e1ad93b64dmr749075ljc.46.1714588721864; Wed, 01 May 2024 11:38:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1714588721; cv=none; d=google.com; s=arc-20160816; b=EHVA0GSq+yw1iZIwr3GlQ+SbskXoGegmfl51OHrJa7OVvf7hO5SqlyzFGSg5ZMky0X xbuyVg7U+z8AD7Mp1xHgAQ1ZIpjvAN605v0t3EMAf2xI4VbZf3zHaJ7cK58imqTO4w/y Puue8yG70kXSobMoSmlQWgpeJL5/XEiSLO6s9utz9FhrIPD8VreXlMD/00TCe4nNu24E 51G9wOZTu0B25gOque9DpmiMIhm8KceLF+JVmbKEW9TXpxcwCxaIjBNefLpVh9DdvPDt tf40InM8f8bCvzmML5PVWmpETCxbusWkUE/2EEkWFbVo3ZzW4+6svAk0112Hl8SuuLnc 6wcA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=sge17DWqO+/YC6ucl9wXRZIPi7NYe1j47DQVIOINvxY=; fh=GFMLWjWsOMgBmoPKQLjFbTnXINkN4pkSjsZqwDcYLuI=; b=zqOoRk4b16ToZGxdJdyN2t02pplTCq/O7h1cuTHqIkznxHeucpXkASxoxQpjPXx8os xXL2vTvQ8R5SWRU/87oibxYqpv4n3UTzPMwJdB7LX7Z5Qifny17ffpYTJrSohEXJqoFN XkxkSq9hpKqHrYvqcrmUIf+ZG8QQnWSYK4dtKm3c6TzyfhTOSR0uSamWZWMKFSzF4UyL hHKXtIgD/1ROj0GU5xy7E8tst/BXVgiX33XBd9QZaQ1ZJDBjhRDfuFxOIjRvQlXKhGcS Ai6CEEybdb7A5QgKnqr1qqpm3uXnRlYhnry8AlnoDXsWszjC6Jc0ssHNbwZJKGMdEEa8 ZduQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=ijDbvVOE; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id u3-20020a17090626c300b00a556d10a90asi16169016ejc.343.2024.05.01.11.38.41; Wed, 01 May 2024 11:38:41 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=ijDbvVOE; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0849968D6F8; Wed, 1 May 2024 21:38:25 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f42.google.com (mail-lf1-f42.google.com [209.85.167.42]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id F00A468D647 for ; Wed, 1 May 2024 21:38:17 +0300 (EEST) Received: by mail-lf1-f42.google.com with SMTP id 2adb3069b0e04-51abd9fcbf6so12059509e87.1 for ; Wed, 01 May 2024 11:38:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1714588697; x=1715193497; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nnOH8MvJDb3g+ksKK30mT2hdHERUY57EIzLESxpIysE=; b=ijDbvVOEn4tLtq3CBgWQ6/aOBiJnUHG0DmZ81H/pc0PIO3jExolWRBmy6m4ANiYh4V P+b57NCQhWrUXsad0YBU3XOTvu+AtVy/Kt4XC86E1yAOsYrCKlZxvFKNUecDxzyIF8S8 3LmMooJcoPDYd+l1mAEE4tLDkMG5/+P9YOWPrPE/D8ScUA3mrPhEY8NoDfEdVvAXhoDL rREjl+a+QyCWfUrWMMZ/ZtoYbv1zTaxv+DSCL7sxFytkNI7RR4W+zHHHs0SXXKqAZSkO 7DP1qIWXeXKM60jM1xDJBLJBIK/hQz18gGLygKOEg7f+LYlOZGoWaaIHL4FOKFBbOMxy ZaTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714588697; x=1715193497; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nnOH8MvJDb3g+ksKK30mT2hdHERUY57EIzLESxpIysE=; b=Sac+oDhSTGzFb9EGLNKlVgiyGfOtGak0e7hb/LMP4zxUUCue9O8gTHo/Zy8dO977dU 6jEjhguxiRuPCJDA45rhPqpXuNUoBqwDVHZoWu9zR5lKS8qCKGwu75x9RI+/fqW43Lx9 GBaBd6ZefWg4WCdNNtE58aTxTO3oYt3fixkFNoS9pELb4OxMbfkKj3R+5gl5FZPvGEk5 3lQJWcPPNJ7DhqJqMxTLuOnSVJcSb3+/mpOER89CO5O3y94XuhzXLVG9IiHD0N8PIyb2 EHdDIXvZPT7DYq45AWAoetEMny3/rudt+llQGJWeOAwtqxok52sjCqF+Tiz4nu1AFfVC IKug== X-Gm-Message-State: AOJu0Yxw57qjvLABKBowqvGWD11UzJ5ia24bN0DXzP5YBdXJzYXMdKRL SEfCj6muTaDsmYbYTGSYVMQ2ReTXygVLlq7876oZHb6lLk61ZwirDP+2IDI5 X-Received: by 2002:ac2:5445:0:b0:51b:4961:91b8 with SMTP id d5-20020ac25445000000b0051b496191b8mr2617227lfn.46.1714588696092; Wed, 01 May 2024 11:38:16 -0700 (PDT) Received: from dovchinn.amd.com (cable-178-148-16-149.dynamic.sbb.rs. [178.148.16.149]) by smtp.gmail.com with ESMTPSA id kh20-20020a170906f81400b00a58bf434876sm7367337ejb.58.2024.05.01.11.38.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 May 2024 11:38:15 -0700 (PDT) From: Dmitrii Ovchinnikov To: ffmpeg-devel@ffmpeg.org Date: Wed, 1 May 2024 20:38:01 +0200 Message-Id: <20240501183809.1060-2-ovchinnikov.dmitrii@gmail.com> X-Mailer: git-send-email 2.38.1.windows.1 In-Reply-To: <20240501183809.1060-1-ovchinnikov.dmitrii@gmail.com> References: <20240501183809.1060-1-ovchinnikov.dmitrii@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 02/10, v2] avcodec: add amfdec. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Evgeny Pavlov , Dmitrii Ovchinnikov Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: w22fl3Ii3DIU From: Evgeny Pavlov Added AMF based h264, hevc, av1 decoders. Co-authored-by: Dmitrii Ovchinnikov v2: added encoder reinitialisation --- libavcodec/Makefile | 7 +- libavcodec/allcodecs.c | 3 + libavcodec/amfdec.c | 719 +++++++++++++++++++++++++++++++++++++++++ libavcodec/amfdec.h | 72 +++++ 4 files changed, 799 insertions(+), 2 deletions(-) create mode 100644 libavcodec/amfdec.c create mode 100644 libavcodec/amfdec.h diff --git a/libavcodec/Makefile b/libavcodec/Makefile index cff6347bdb..bba5191a70 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -70,7 +70,7 @@ include $(SRC_PATH)/libavcodec/vvc/Makefile OBJS-$(CONFIG_AANDCTTABLES) += aandcttab.o OBJS-$(CONFIG_AC3DSP) += ac3dsp.o ac3.o ac3tab.o OBJS-$(CONFIG_ADTS_HEADER) += adts_header.o mpeg4audio_sample_rates.o -OBJS-$(CONFIG_AMF) += amfenc.o +OBJS-$(CONFIG_AMF) += amfenc.o amfdec.o OBJS-$(CONFIG_AUDIO_FRAME_QUEUE) += audio_frame_queue.o OBJS-$(CONFIG_ATSC_A53) += atsc_a53.o OBJS-$(CONFIG_AUDIODSP) += audiodsp.o @@ -167,6 +167,7 @@ OBJS-$(CONFIG_TEXTUREDSPENC) += texturedspenc.o OBJS-$(CONFIG_TPELDSP) += tpeldsp.o OBJS-$(CONFIG_VAAPI_ENCODE) += vaapi_encode.o OBJS-$(CONFIG_AV1_AMF_ENCODER) += amfenc_av1.o +OBJS-$(CONFIG_AV1_AMF_DECODER) += amfdec.o OBJS-$(CONFIG_VC1DSP) += vc1dsp.o OBJS-$(CONFIG_VIDEODSP) += videodsp.o OBJS-$(CONFIG_VP3DSP) += vp3dsp.o @@ -411,6 +412,7 @@ OBJS-$(CONFIG_H264_DECODER) += h264dec.o h264_cabac.o h264_cavlc.o \ h264_refs.o \ h264_slice.o h264data.o h274.o OBJS-$(CONFIG_H264_AMF_ENCODER) += amfenc_h264.o +OBJS-$(CONFIG_H264_AMF_DECODER) += amfdec.o OBJS-$(CONFIG_H264_CUVID_DECODER) += cuviddec.o OBJS-$(CONFIG_H264_MEDIACODEC_DECODER) += mediacodecdec.o OBJS-$(CONFIG_H264_MEDIACODEC_ENCODER) += mediacodecenc.o @@ -437,6 +439,7 @@ OBJS-$(CONFIG_HEVC_DECODER) += hevcdec.o hevc_mvs.o \ hevcdsp.o hevc_filter.o hevc_data.o \ h274.o aom_film_grain.o OBJS-$(CONFIG_HEVC_AMF_ENCODER) += amfenc_hevc.o +OBJS-$(CONFIG_HEVC_AMF_DECODER) += amfdec.o OBJS-$(CONFIG_HEVC_CUVID_DECODER) += cuviddec.o OBJS-$(CONFIG_HEVC_MEDIACODEC_DECODER) += mediacodecdec.o OBJS-$(CONFIG_HEVC_MEDIACODEC_ENCODER) += mediacodecenc.o @@ -1265,7 +1268,7 @@ SKIPHEADERS += %_tablegen.h \ bitstream_template.h \ $(ARCH)/vpx_arith.h \ -SKIPHEADERS-$(CONFIG_AMF) += amfenc.h +SKIPHEADERS-$(CONFIG_AMF) += amfenc.h amfdec.h SKIPHEADERS-$(CONFIG_D3D11VA) += d3d11va.h dxva2_internal.h SKIPHEADERS-$(CONFIG_D3D12VA) += d3d12va_decode.h SKIPHEADERS-$(CONFIG_DXVA2) += dxva2.h dxva2_internal.h diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index f4705651fb..58a49b5825 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -843,10 +843,12 @@ extern const FFCodec ff_av1_nvenc_encoder; extern const FFCodec ff_av1_qsv_decoder; extern const FFCodec ff_av1_qsv_encoder; extern const FFCodec ff_av1_amf_encoder; +extern const FFCodec ff_av1_amf_decoder; extern const FFCodec ff_av1_vaapi_encoder; extern const FFCodec ff_libopenh264_encoder; extern const FFCodec ff_libopenh264_decoder; extern const FFCodec ff_h264_amf_encoder; +extern const FFCodec ff_h264_amf_decoder; extern const FFCodec ff_h264_cuvid_decoder; extern const FFCodec ff_h264_mf_encoder; extern const FFCodec ff_h264_nvenc_encoder; @@ -856,6 +858,7 @@ extern const FFCodec ff_h264_v4l2m2m_encoder; extern const FFCodec ff_h264_vaapi_encoder; extern const FFCodec ff_h264_videotoolbox_encoder; extern const FFCodec ff_hevc_amf_encoder; +extern const FFCodec ff_hevc_amf_decoder; extern const FFCodec ff_hevc_cuvid_decoder; extern const FFCodec ff_hevc_mediacodec_decoder; extern const FFCodec ff_hevc_mediacodec_encoder; diff --git a/libavcodec/amfdec.c b/libavcodec/amfdec.c new file mode 100644 index 0000000000..c41ad3c2db --- /dev/null +++ b/libavcodec/amfdec.c @@ -0,0 +1,719 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include +#include +#include +#include "libavutil/hwcontext_amf.h" +#include "libavutil/hwcontext_amf_internal.h" +#include "amfdec.h" +#include "codec_internal.h" +#include "hwconfig.h" +#include "libavutil/imgutils.h" +#include "libavutil/time.h" +#include "decode.h" +#include "libavutil/mastering_display_metadata.h" + +#if CONFIG_D3D11VA +#include "libavutil/hwcontext_d3d11va.h" +#endif +#if CONFIG_DXVA2 +#define COBJMACROS +#include "libavutil/hwcontext_dxva2.h" +#endif + +#ifdef _WIN32 +#include "compat/w32dlfcn.h" +#else +#include +#endif + +#define propNotFound 0 + +const enum AVPixelFormat amf_dec_pix_fmts[] = { + AV_PIX_FMT_YUV420P, + AV_PIX_FMT_NV12, + AV_PIX_FMT_BGRA, + AV_PIX_FMT_ARGB, + AV_PIX_FMT_RGBA, + AV_PIX_FMT_GRAY8, + AV_PIX_FMT_BGR0, + AV_PIX_FMT_YUYV422, + AV_PIX_FMT_P010, + AV_PIX_FMT_P012, + AV_PIX_FMT_YUV420P10, + AV_PIX_FMT_YUV420P12, + AV_PIX_FMT_YUV420P16, +#if CONFIG_D3D11VA + AV_PIX_FMT_D3D11, +#endif +#if CONFIG_DXVA2 + AV_PIX_FMT_DXVA2_VLD, +#endif + AV_PIX_FMT_AMF_SURFACE, + AV_PIX_FMT_NONE +}; + +static const AVCodecHWConfigInternal *const amf_hw_configs[] = { + &(const AVCodecHWConfigInternal) { + .public = { + .pix_fmt = AV_PIX_FMT_AMF_SURFACE, + .methods = AV_CODEC_HW_CONFIG_METHOD_HW_FRAMES_CTX | + AV_CODEC_HW_CONFIG_METHOD_HW_DEVICE_CTX, + .device_type = AV_HWDEVICE_TYPE_AMF, + }, + .hwaccel = NULL, + }, + NULL +}; + +static void amf_free_amfsurface(void *opaque, uint8_t *data) +{ + AMFSurface *surface = (AMFSurface*)(data); + surface->pVtbl->Release(surface); +} + +static int amf_init_decoder(AVCodecContext *avctx) +{ + AvAmfDecoderContext *ctx = avctx->priv_data; + AVAMFDeviceContextInternal * internal = (AVAMFDeviceContextInternal *)ctx->amf_device_ctx_internal->data; + const wchar_t *codec_id = NULL; + AMF_RESULT res; + AMFBuffer *buffer; + amf_int64 color_profile; + int pool_size = 36; + + ctx->output_format = AMF_SURFACE_UNKNOWN; + if (avctx->pix_fmt == AV_PIX_FMT_AMF_SURFACE){ + if (avctx->hw_frames_ctx) { + AVHWFramesContext *hwframes_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; + ctx->output_format = av_amf_av_to_amf_format(hwframes_ctx->sw_format); + } else + ctx->output_format = av_amf_av_to_amf_format(avctx->sw_pix_fmt); + } else + ctx->output_format = av_amf_av_to_amf_format(avctx->pix_fmt); + + if (ctx->output_format == AMF_SURFACE_UNKNOWN) + ctx->output_format = AMF_SURFACE_NV12; + + ctx->drain = 0; + ctx->resolution_changed = 0; + + switch (avctx->codec->id) { + case AV_CODEC_ID_H264: + codec_id = AMFVideoDecoderUVD_H264_AVC; + break; + case AV_CODEC_ID_HEVC: { + if (ctx->output_format == AMF_SURFACE_P010) + codec_id = AMFVideoDecoderHW_H265_MAIN10; + else + codec_id = AMFVideoDecoderHW_H265_HEVC; + } break; + case AV_CODEC_ID_AV1: + if (ctx->output_format == AMF_SURFACE_P012) + codec_id = AMFVideoDecoderHW_AV1_12BIT; + else + codec_id = AMFVideoDecoderHW_AV1; + break; + default: + break; + } + AMF_RETURN_IF_FALSE(ctx, codec_id != NULL, AVERROR(EINVAL), "Codec %d is not supported\n", avctx->codec->id); + + res = internal->factory->pVtbl->CreateComponent(internal->factory, internal->context, codec_id, &ctx->decoder); + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_ENCODER_NOT_FOUND, "CreateComponent(%ls) failed with error %d\n", codec_id, res); + + // Color Metadata + /// Color Range (Support for older Drivers) + if (avctx->color_range == AVCOL_RANGE_JPEG) { + AMF_ASSIGN_PROPERTY_BOOL(res, ctx->decoder, AMF_VIDEO_DECODER_FULL_RANGE_COLOR, 1); + } else if (avctx->color_range != AVCOL_RANGE_UNSPECIFIED) { + AMF_ASSIGN_PROPERTY_BOOL(res, ctx->decoder, AMF_VIDEO_DECODER_FULL_RANGE_COLOR, 0); + } + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN; + switch (avctx->colorspace) { + case AVCOL_SPC_SMPTE170M: + if (avctx->color_range == AVCOL_RANGE_JPEG) { + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_601; + } else { + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_601; + } + break; + case AVCOL_SPC_BT709: + if (avctx->color_range == AVCOL_RANGE_JPEG) { + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_709; + } else { + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_709; + } + break; + case AVCOL_SPC_BT2020_NCL: + case AVCOL_SPC_BT2020_CL: + if (avctx->color_range == AVCOL_RANGE_JPEG) { + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_2020; + } else { + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_2020; + } + break; + } + if (color_profile != AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_COLOR_PROFILE, color_profile); + if (avctx->color_trc != AVCOL_TRC_UNSPECIFIED) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_COLOR_TRANSFER_CHARACTERISTIC, (amf_int64)avctx->color_trc); + + if (avctx->color_primaries != AVCOL_PRI_UNSPECIFIED) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_COLOR_PRIMARIES, (amf_int64)avctx->color_primaries); + + if (ctx->timestamp_mode != -1) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_TIMESTAMP_MODE, ctx->timestamp_mode); + if (ctx->decoder_mode != -1) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_REORDER_MODE, ctx->decoder_mode); + if (ctx->dpb_size != -1) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_DPB_SIZE, ctx->dpb_size); + if (ctx->lowlatency != -1) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_LOW_LATENCY, ctx->lowlatency); + if (ctx->smart_access_video != -1) { + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_ENABLE_SMART_ACCESS_VIDEO, ctx->smart_access_video != 0); + if (res != AMF_OK) { + av_log(avctx, AV_LOG_ERROR, "The Smart Access Video is not supported by AMF decoder.\n"); + return AVERROR(EINVAL); + } else { + av_log(avctx, AV_LOG_INFO, "The Smart Access Video (%d) is set.\n", ctx->smart_access_video); + // Set low latency mode if Smart Access Video is enabled + if (ctx->smart_access_video != 0) { + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_LOW_LATENCY, true); + av_log(avctx, AV_LOG_INFO, "The Smart Access Video set low latency mode for decoder.\n"); + } + } + } + if (ctx->skip_transfer_sav != -1) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_SKIP_TRANSFER_SMART_ACCESS_VIDEO, ctx->skip_transfer_sav); + + if (avctx->extradata_size) { + res = internal->context->pVtbl->AllocBuffer(internal->context, AMF_MEMORY_HOST, avctx->extradata_size, &buffer); + if (res == AMF_OK) { + memcpy(buffer->pVtbl->GetNative(buffer), avctx->extradata, avctx->extradata_size); + AMF_ASSIGN_PROPERTY_INTERFACE(res,ctx->decoder, AMF_VIDEO_DECODER_EXTRADATA, buffer); + buffer->pVtbl->Release(buffer); + buffer = NULL; + } + } + if (ctx->surface_pool_size == -1) { + ctx->surface_pool_size = pool_size; + if (avctx->extra_hw_frames > 0) + ctx->surface_pool_size += avctx->extra_hw_frames; + if (avctx->active_thread_type & FF_THREAD_FRAME) + ctx->surface_pool_size += avctx->thread_count; + } + + //at the moment, there is such a restriction in AMF. + //when it is possible, I will remove this code + if (ctx->surface_pool_size > 100) + ctx->surface_pool_size = 100; + + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_SURFACE_POOL_SIZE, ctx->surface_pool_size); + res = ctx->decoder->pVtbl->Init(ctx->decoder, ctx->output_format, avctx->width, avctx->height); + + return 0; +} + +static int amf_init_decoder_context(AVCodecContext *avctx) +{ + AvAmfDecoderContext *ctx = avctx->priv_data; + int ret; + + if (avctx->hw_frames_ctx) { + AVHWFramesContext *frames_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; + ret = av_amf_context_derive((AVAMFDeviceContextInternal *)ctx->amf_device_ctx_internal->data, frames_ctx->device_ctx, NULL, 0); + if (ret < 0) + return ret; + ctx->hw_frames_ctx = av_buffer_ref(avctx->hw_frames_ctx); + if (!ctx->hw_frames_ctx) + return AVERROR(ENOMEM); + } + else if (avctx->hw_device_ctx) { + AVHWDeviceContext *device_ctx = (AVHWDeviceContext*)avctx->hw_device_ctx->data; + ret = av_amf_context_derive((AVAMFDeviceContextInternal *)ctx->amf_device_ctx_internal->data, device_ctx, NULL, 0); + if (ret < 0) + return ret; + ctx->hw_device_ctx = av_buffer_ref(avctx->hw_device_ctx); + if (!ctx->hw_device_ctx) + return AVERROR(ENOMEM); + } else { + ret = av_amf_context_init((AVAMFDeviceContextInternal *)ctx->amf_device_ctx_internal->data, avctx); + if (ret != 0) { + return ret; + } + } + + return ret; +} + +static int amf_decode_close(AVCodecContext *avctx) +{ + AvAmfDecoderContext *ctx = avctx->priv_data; + + if (ctx->decoder) { + ctx->decoder->pVtbl->Terminate(ctx->decoder); + ctx->decoder->pVtbl->Release(ctx->decoder); + ctx->decoder = NULL; + } + + av_buffer_unref(&ctx->amf_device_ctx_internal); + av_buffer_unref(&ctx->hw_device_ctx); + av_buffer_unref(&ctx->hw_frames_ctx); + av_buffer_unref(&ctx->amf_device_ctx); + + av_packet_unref(ctx->in_pkt); + + return 0; + +} + +static int amf_decode_init(AVCodecContext *avctx) +{ + AvAmfDecoderContext *ctx = avctx->priv_data; + int ret; + enum AVPixelFormat pix_fmts[3] = { + AV_PIX_FMT_AMF_SURFACE, + avctx->pix_fmt, + AV_PIX_FMT_NONE }; + + ret = ff_get_format(avctx, pix_fmts); + if (ret < 0) { + avctx->pix_fmt = AV_PIX_FMT_NONE; + } + ctx->in_pkt = av_packet_alloc(); + if (!ctx->in_pkt) + return AVERROR(ENOMEM); + + if (avctx->hw_frames_ctx){ + AVHWFramesContext *frames_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; + if (frames_ctx->device_ctx->type == AV_HWDEVICE_TYPE_AMF) { + AVAMFDeviceContext * amf_ctx = frames_ctx->device_ctx->hwctx; + ctx->amf_device_ctx_internal = av_buffer_ref(amf_ctx->internal); + } + } + else if (avctx->hw_device_ctx && !avctx->hw_frames_ctx && ret == AV_PIX_FMT_AMF_SURFACE) { + AVHWDeviceContext *hwdev_ctx; + AVHWFramesContext *hwframes_ctx; + hwdev_ctx = (AVHWDeviceContext*)avctx->hw_device_ctx->data; + if (hwdev_ctx->type == AV_HWDEVICE_TYPE_AMF) + { + AVAMFDeviceContext * amf_ctx = hwdev_ctx->hwctx; + ctx->amf_device_ctx_internal = av_buffer_ref(amf_ctx->internal); + } + + avctx->hw_frames_ctx = av_hwframe_ctx_alloc(avctx->hw_device_ctx); + + if (!avctx->hw_frames_ctx) { + av_log(avctx, AV_LOG_ERROR, "av_hwframe_ctx_alloc failed\n"); + return AVERROR(ENOMEM); + } + + hwframes_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; + hwframes_ctx->width = FFALIGN(avctx->coded_width, 32); + hwframes_ctx->height = FFALIGN(avctx->coded_height, 32); + hwframes_ctx->format = AV_PIX_FMT_AMF_SURFACE; + hwframes_ctx->sw_format = avctx->sw_pix_fmt == AV_PIX_FMT_YUV420P10 ? AV_PIX_FMT_P010 : AV_PIX_FMT_NV12; + hwframes_ctx->initial_pool_size = ctx->surface_pool_size + 8; + avctx->pix_fmt = AV_PIX_FMT_AMF_SURFACE; + + ret = av_hwframe_ctx_init(avctx->hw_frames_ctx); + + if (ret < 0) { + av_log(NULL, AV_LOG_ERROR, "Error initializing a AMF frame pool\n"); + av_buffer_unref(&avctx->hw_frames_ctx); + return ret; + } + } else { + AVAMFDeviceContextInternal *wrapped = av_mallocz(sizeof(*wrapped)); + ctx->amf_device_ctx_internal = av_buffer_create((uint8_t *)wrapped, sizeof(*wrapped), + av_amf_context_internal_free, NULL, 0); + if ((ret = av_amf_context_internal_create((AVAMFDeviceContextInternal *)ctx->amf_device_ctx_internal->data, avctx, "", NULL, 0)) != 0) { + amf_decode_close(avctx); + return ret; + } + if ((ret = amf_init_decoder_context(avctx)) != 0) { + return ret; + } + } + if ((ret = amf_init_decoder(avctx)) == 0) { + return 0; + } + amf_decode_close(avctx); + return ret; +} + +static AMF_RESULT amf_get_property_buffer(AMFData *object, const wchar_t *name, AMFBuffer **val) +{ + AMF_RESULT res; + AMFVariantStruct var; + res = AMFVariantInit(&var); + if (res == AMF_OK) { + res = object->pVtbl->GetProperty(object, name, &var); + if (res == AMF_OK) { + if (var.type == AMF_VARIANT_INTERFACE) { + AMFGuid guid_AMFBuffer = IID_AMFBuffer(); + AMFInterface *amf_interface = AMFVariantInterface(&var); + res = amf_interface->pVtbl->QueryInterface(amf_interface, &guid_AMFBuffer, (void**)val); + } else { + res = AMF_INVALID_DATA_TYPE; + } + } + AMFVariantClear(&var); + } + return res; +} + +static int amf_amfsurface_to_avframe(AVCodecContext *avctx, AMFSurface* surface, AVFrame *frame) +{ + AMFVariantStruct var = {0}; + AMFPlane *plane; + int i; + int ret; + + if (avctx->hw_frames_ctx) { + AVHWFramesContext *hwframes_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; + if (hwframes_ctx->format == AV_PIX_FMT_AMF_SURFACE) { + ret = ff_get_buffer(avctx, frame, AV_GET_BUFFER_FLAG_REF); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Get hw frame failed.\n"); + return ret; + } + //we need to release surface with frame to return it to decoder + frame->buf[1] = av_buffer_create((uint8_t *)surface, sizeof(AMFSurface), + amf_free_amfsurface, (void*)avctx, + AV_BUFFER_FLAG_READONLY); + frame->data[0] = (uint8_t *)surface; + } else { + av_log(avctx, AV_LOG_ERROR, "Unknown format for hwframes_ctx\n"); + return AVERROR(ENOMEM); + } + } else { + ret = surface->pVtbl->Convert(surface, AMF_MEMORY_HOST); + AMF_RETURN_IF_FALSE(avctx, ret == AMF_OK, AVERROR_UNKNOWN, "Convert(amf::AMF_MEMORY_HOST) failed with error %d\n", ret); + + for (i = 0; i < surface->pVtbl->GetPlanesCount(surface); i++) { + plane = surface->pVtbl->GetPlaneAt(surface, i); + frame->data[i] = plane->pVtbl->GetNative(plane); + frame->linesize[i] = plane->pVtbl->GetHPitch(plane); + } + + frame->buf[0] = av_buffer_create((uint8_t *)surface, sizeof(AMFSurface), + amf_free_amfsurface, (void*)avctx, + AV_BUFFER_FLAG_READONLY); + frame->format = av_amf_to_av_format(surface->pVtbl->GetFormat(surface)); + } + + frame->width = avctx->width; + frame->height = avctx->height; + + frame->pts = surface->pVtbl->GetPts(surface); + + surface->pVtbl->GetProperty(surface, L"FFMPEG:dts", &var); + frame->pkt_dts = var.int64Value; + + frame->duration = surface->pVtbl->GetDuration(surface); + if (frame->duration < 0) + frame->duration = 0; + + frame->color_range = avctx->color_range; + frame->colorspace = avctx->colorspace; + frame->color_trc = avctx->color_trc; + frame->color_primaries = avctx->color_primaries; + + if (frame->color_trc == AVCOL_TRC_SMPTE2084) { + AMFBuffer * hdrmeta_buffer = NULL; + ret = amf_get_property_buffer((AMFData *)surface, AMF_VIDEO_DECODER_HDR_METADATA, &hdrmeta_buffer); + if (hdrmeta_buffer != NULL) { + AMFHDRMetadata * hdrmeta = (AMFHDRMetadata*)hdrmeta_buffer->pVtbl->GetNative(hdrmeta_buffer); + if (ret != AMF_OK) + return ret; + if (hdrmeta != NULL) { + AVMasteringDisplayMetadata *mastering = av_mastering_display_metadata_create_side_data(frame); + const int chroma_den = 50000; + const int luma_den = 10000; + + if (!mastering) + return AVERROR(ENOMEM); + + mastering->display_primaries[0][0] = av_make_q(hdrmeta->redPrimary[0], chroma_den); + mastering->display_primaries[0][1] = av_make_q(hdrmeta->redPrimary[1], chroma_den); + + mastering->display_primaries[1][0] = av_make_q(hdrmeta->greenPrimary[0], chroma_den); + mastering->display_primaries[1][1] = av_make_q(hdrmeta->greenPrimary[1], chroma_den); + + mastering->display_primaries[2][0] = av_make_q(hdrmeta->bluePrimary[0], chroma_den); + mastering->display_primaries[2][1] = av_make_q(hdrmeta->bluePrimary[1], chroma_den); + + mastering->white_point[0] = av_make_q(hdrmeta->whitePoint[0], chroma_den); + mastering->white_point[1] = av_make_q(hdrmeta->whitePoint[1], chroma_den); + + mastering->max_luminance = av_make_q(hdrmeta->maxMasteringLuminance, luma_den); + mastering->min_luminance = av_make_q(hdrmeta->maxMasteringLuminance, luma_den); + + mastering->has_luminance = 1; + mastering->has_primaries = 1; + if (hdrmeta->maxContentLightLevel) { + AVContentLightMetadata *light = av_content_light_metadata_create_side_data(frame); + + if (!light) + return AVERROR(ENOMEM); + + light->MaxCLL = hdrmeta->maxContentLightLevel; + light->MaxFALL = hdrmeta->maxFrameAverageLightLevel; + } + } + } + } + return 0; +} + +static AMF_RESULT amf_receive_frame(AVCodecContext *avctx, AVFrame *frame) +{ + AvAmfDecoderContext *ctx = avctx->priv_data; + AMF_RESULT ret = AMF_OK; + AMFSurface *surface = NULL; + AMFData *data_out = NULL; + + ret = ctx->decoder->pVtbl->QueryOutput(ctx->decoder, &data_out); + + if (ret != AMF_OK && ret != AMF_REPEAT) { + return ret; + } + if (data_out == NULL) { + return AMF_REPEAT; + } + + if (data_out) { + AMFGuid guid = IID_AMFSurface(); + data_out->pVtbl->QueryInterface(data_out, &guid, (void**)&surface); // query for buffer interface + data_out->pVtbl->Release(data_out); + data_out = NULL; + } + + ret = amf_amfsurface_to_avframe(avctx, surface, frame); + AMF_GOTO_FAIL_IF_FALSE(avctx, ret >= 0, AMF_FAIL, "Failed to convert AMFSurface to AVFrame = %d\n", ret); + return AMF_OK; +fail: + + if (surface) { + surface->pVtbl->Release(surface); + surface = NULL; + } + return ret; +} + +static AMF_RESULT amf_update_buffer_properties(AVCodecContext *avctx, AMFBuffer* buffer, const AVPacket* pkt) +{ + AvAmfDecoderContext *ctx = avctx->priv_data; + AVAMFDeviceContextInternal * internal = (AVAMFDeviceContextInternal * )ctx->amf_device_ctx_internal->data; + AMFContext *ctxt = internal->context; + + AMF_RESULT res; + + AMF_RETURN_IF_FALSE(ctxt, buffer != NULL, AMF_INVALID_ARG, "update_buffer_properties() - buffer not passed in"); + AMF_RETURN_IF_FALSE(ctxt, pkt != NULL, AMF_INVALID_ARG, "update_buffer_properties() - packet not passed in"); + buffer->pVtbl->SetPts(buffer, pkt->pts); + buffer->pVtbl->SetDuration(buffer, pkt->duration); + AMF_ASSIGN_PROPERTY_INT64(res, buffer, L"FFMPEG:dts", pkt->dts); + if (res != AMF_OK) + av_log(avctx, AV_LOG_VERBOSE, "Failed to assign dts value."); + return AMF_OK; +} + +static AMF_RESULT amf_buffer_from_packet(AVCodecContext *avctx, const AVPacket* pkt, AMFBuffer** buffer) +{ + AvAmfDecoderContext *ctx = avctx->priv_data; + AVAMFDeviceContextInternal * internal = (AVAMFDeviceContextInternal * )ctx->amf_device_ctx_internal->data; + AMFContext *ctxt = internal->context; + void *mem; + AMF_RESULT err; + AMFBuffer *buf = NULL; + + AMF_RETURN_IF_FALSE(ctxt, pkt != NULL, AMF_INVALID_ARG, "amf_buffer_from_packet() - packet not passed in"); + AMF_RETURN_IF_FALSE(ctxt, buffer != NULL, AMF_INVALID_ARG, "amf_buffer_from_packet() - buffer pointer not passed in"); + + err = ctxt->pVtbl->AllocBuffer(ctxt, AMF_MEMORY_HOST, pkt->size + AV_INPUT_BUFFER_PADDING_SIZE, buffer); + AMF_RETURN_IF_FALSE(ctxt, err == AMF_OK, err, "amf_buffer_from_packet() - failed"); + buf = *buffer; + err = buf->pVtbl->SetSize(buf, pkt->size); + AMF_RETURN_IF_FALSE(ctxt, err == AMF_OK, err, "amf_buffer_from_packet() - SetSize failed"); + // get the memory location and check the buffer was indeed allocated + mem = buf->pVtbl->GetNative(buf); + AMF_RETURN_IF_FALSE(ctxt, mem != NULL, AMF_INVALID_POINTER, "amf_buffer_from_packet() - GetNative failed"); + + // copy the packet memory and clear data padding + memcpy(mem, pkt->data, pkt->size); + memset((amf_int8*)(mem)+pkt->size, 0, AV_INPUT_BUFFER_PADDING_SIZE); + + return amf_update_buffer_properties(avctx, buf, pkt); +} + +static int amf_decode_frame(AVCodecContext *avctx, struct AVFrame *frame) +{ + AvAmfDecoderContext *ctx = avctx->priv_data; + AMFBuffer *buf; + AMF_RESULT res; + int got_frame = 0; + AVPacket *avpkt = ctx->in_pkt; + + if (!ctx->decoder) + return AVERROR(EINVAL); + // get packet if needed + if(!ctx->drain){ + if(ctx->resolution_changed) + ctx->resolution_changed = 0; + else{ + av_packet_unref(avpkt); + int ret; + ret = ff_decode_get_packet(avctx, avpkt); + if (ret < 0 && ret != AVERROR_EOF) + return ret; + if (ret == AVERROR_EOF) { + //nothing to consume, start external drain + ctx->decoder->pVtbl->Drain(ctx->decoder); + ctx->drain = 1; + } + } + } + + if (!ctx->drain) { + // submit frame + res = amf_buffer_from_packet(avctx, avpkt, &buf); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, 0, "Cannot convert AVPacket to AMFbuffer"); + res = ctx->decoder->pVtbl->SubmitInput(ctx->decoder, (AMFData*) buf); + buf->pVtbl->Release(buf); + + if(res == AMF_DECODER_NO_FREE_SURFACES) { + // input is not consumed, need to QueryOutput and submit again + av_log(avctx, AV_LOG_VERBOSE, "SubmitInput() returned NO_FREE_SURFACES\n"); + } else if (res == AMF_RESOLUTION_CHANGED) { + //input is not consumed, start internal drain + ctx->decoder->pVtbl->Drain(ctx->decoder); + ctx->drain = 1; + // process resolution_changed when internal drain is complete + ctx->resolution_changed = 1; + res = AMF_OK; + } else if (res != AMF_OK && res != AMF_NEED_MORE_INPUT && res != AMF_REPEAT) { + av_log(avctx, AV_LOG_ERROR, "SubmitInput() returned error %d\n", res); + return AVERROR(EINVAL); + } + + } + + res = amf_receive_frame(avctx, frame); + if (res == AMF_OK) + got_frame = 1; + else if (res == AMF_REPEAT) + // decoder has no output yet + res = AMF_OK; + else if (res == AMF_EOF) { + // drain is complete + ctx->drain = 0; + if(ctx->resolution_changed){ + // re-initialze decoder + AMFVariantStruct size_var = {0}; + res = ctx->decoder->pVtbl->GetProperty(ctx->decoder, AMF_VIDEO_DECODER_CURRENT_SIZE, &size_var); + if (res != AMF_OK) { + return AVERROR(EINVAL); + } + + avctx->width = size_var.sizeValue.width; + avctx->height = size_var.sizeValue.height; + avctx->coded_width = size_var.sizeValue.width; + avctx->coded_height = size_var.sizeValue.height; + res = ctx->decoder->pVtbl->ReInit(ctx->decoder, avctx->width, avctx->height); + if (res != AMF_OK) { + av_log(avctx, AV_LOG_ERROR, "ReInit() returned %d\n", res); + return AVERROR(EINVAL); + } + + }else + return AVERROR_EOF; + } else { + av_log(avctx, AV_LOG_ERROR, "Unkown result from QueryOutput %d\n", res); + } + return got_frame ? 0 : AVERROR(EAGAIN); +} + +static void amf_decode_flush(AVCodecContext *avctx) +{ + AvAmfDecoderContext *ctx = avctx->priv_data; + ctx->decoder->pVtbl->Flush(ctx->decoder); +} + +#define OFFSET(x) offsetof(AvAmfDecoderContext, x) +#define VD AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_DECODING_PARAM + +static const AVOption options[] = { + // Decoder mode + { "decoder_mode", "Decoder mode", OFFSET(decoder_mode), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, AMF_VIDEO_DECODER_MODE_LOW_LATENCY, VD, "decoder_mode" }, + { "regular", "DPB delay is based on number of reference frames + 1", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_DECODER_MODE_REGULAR }, 0, 0, VD, "decoder_mode" }, + { "compliant", "DPB delay is based on profile - up to 16", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_DECODER_MODE_COMPLIANT }, 0, 0, VD, "decoder_mode" }, + { "low_latency", "DPB delay is 0", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_DECODER_MODE_LOW_LATENCY }, 0, 0, VD, "decoder_mode" }, + + // Timestamp mode + { "timestamp_mode", "Timestamp mode", OFFSET(timestamp_mode), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, AMF_TS_DECODE, VD, "timestamp_mode" }, + { "presentation", "Preserve timestamps from input to output", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_TS_PRESENTATION }, 0, 0, VD, "timestamp_mode" }, + { "sort", "Resort PTS list", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_TS_SORT }, 0, 0, VD, "timestamp_mode" }, + { "decode", "Decode order", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_TS_DECODE }, 0, 0, VD, "timestamp_mode" }, + + // Reference frame management + { "surface_pool_size", "Number of surfaces in the decode pool", OFFSET(surface_pool_size), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, INT_MAX, VD, NULL }, + { "dpb_size", "Minimum number of surfaces for reordering", OFFSET(dpb_size), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 32, VD, NULL }, + + { "lowlatency", "Low latency", OFFSET(lowlatency), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 1, VD, NULL }, + { "smart_access_video", "Smart Access Video", OFFSET(smart_access_video), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 1, VD, NULL }, + { "skip_transfer_sav", "Skip transfer on another GPU when SAV enabled", OFFSET(skip_transfer_sav), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 1, VD, NULL }, + + { NULL } +}; + +static const AVClass amf_decode_class = { + .class_name = "amf", + .item_name = av_default_item_name, + .option = options, + .version = LIBAVUTIL_VERSION_INT, +}; + +// FF_CODEC_DECODE_CB(amf_decode_frame), + +#define DEFINE_AMF_DECODER(x, X, bsf_name) \ +const FFCodec ff_##x##_amf_decoder = { \ + .p.name = #x "_amf", \ + CODEC_LONG_NAME(#X " AMD AMF video decoder"), \ + .priv_data_size = sizeof(AvAmfDecoderContext), \ + .p.type = AVMEDIA_TYPE_VIDEO, \ + .p.id = AV_CODEC_ID_##X, \ + .init = amf_decode_init, \ + FF_CODEC_RECEIVE_FRAME_CB(amf_decode_frame), \ + .flush = amf_decode_flush, \ + .close = amf_decode_close, \ + .bsfs = bsf_name, \ + .p.capabilities = AV_CODEC_CAP_HARDWARE | AV_CODEC_CAP_DELAY | AV_CODEC_CAP_AVOID_PROBING, \ + .p.priv_class = &amf_decode_class, \ + .p.pix_fmts = amf_dec_pix_fmts, \ + .hw_configs = amf_hw_configs, \ + .p.wrapper_name = "amf", \ + .caps_internal = FF_CODEC_CAP_NOT_INIT_THREADSAFE, \ +}; \ + +DEFINE_AMF_DECODER(h264, H264, "h264_mp4toannexb") +DEFINE_AMF_DECODER(hevc, HEVC, NULL) +DEFINE_AMF_DECODER(av1, AV1, NULL) diff --git a/libavcodec/amfdec.h b/libavcodec/amfdec.h new file mode 100644 index 0000000000..542e613ce1 --- /dev/null +++ b/libavcodec/amfdec.h @@ -0,0 +1,72 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_AMFDEC_H +#define AVCODEC_AMFDEC_H + +#include +#include +#include +#include +#include +#include + +#include "avcodec.h" +#include "libavformat/avformat.h" +#include "libavutil/fifo.h" +#include "libavutil/frame.h" +#include "libavutil/opt.h" + +/** +* AMF decoder context +*/ + +typedef struct AvAmfDecoderContext { + AVClass *avclass; + + AVBufferRef *amf_device_ctx_internal; + AVBufferRef *amf_device_ctx; + + //decoder + AMFComponent *decoder; ///< AMF decoder object + AMF_SURFACE_FORMAT format; ///< AMF surface format + + AVBufferRef *hw_device_ctx; ///< pointer to HW accelerator (decoder) + AVBufferRef *hw_frames_ctx; ///< pointer to HW accelerator (frame allocator) + + AVBufferRef *hw_device_ref; + AVBufferRef *hw_frames_ref; + + // common decoder options + int log_to_dbg; + // Static options, have to be set before Init() call + int decoder_mode; + int timestamp_mode; + int surface_pool_size; + int dpb_size; + int lowlatency; + int smart_access_video; + int skip_transfer_sav; + int drain; + int resolution_changed; + AVPacket* in_pkt; + enum AMF_SURFACE_FORMAT output_format; + +} AvAmfDecoderContext; + +#endif // AVCODEC_AMFDEC_H \ No newline at end of file From patchwork Wed May 1 18:38:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitrii Ovchinnikov X-Patchwork-Id: 48448 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:1203:b0:1a9:af23:56c1 with SMTP id v3csp1103870pzf; Wed, 1 May 2024 11:38:52 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCWekqW52PCQWsew5xC9nDiGFqIdEg1+tMknJcAnBP0WBoUZ9cKpkGa+cbqwHdSgq+IxJPK12M0OJPuqdpOxEKrqPaoqrs20Ebpi2Q== X-Google-Smtp-Source: AGHT+IGGJU/e9R1H3Oo/DVOoc0VjJOiaCm+6uJpPxiEgklTCfiaU5GjUZ3hmNjDUIKLQxM4V9+AL X-Received: by 2002:a17:906:3715:b0:a55:75f6:ce0f with SMTP id d21-20020a170906371500b00a5575f6ce0fmr2211048ejc.13.1714588732135; Wed, 01 May 2024 11:38:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1714588732; cv=none; d=google.com; s=arc-20160816; b=LFL/belTzta+tzmE9mqLb4JnK8yJJy+k/p5VBDtJTOMiOrRUGEBLILf+u5Iv8Jfsrn RnCn8WNjCFIqOjZqFb5sbsNFypzXlUATIPNFjpbgHdnZnXMFcnRXrFYhv5bDbVyDNBGe 2M/BqYYrjHroP4T1qvEsLpyWPhpJLvEXgMBUZXWNWpPiw+NIG7M2WVm8GyOw0/+KnJK4 Z/kpsQ806I72CcLK/xF3SAYRrY1gt9lrERAQX1TCmZINBNofQ1tOOyQTmT/bqA7lYizu j3csv4WAYwLb3hit6um5TQtzuvhT4DfpDEPzwTeqZvR2TVyS+HujQeEHeSr57EgrPSi6 GtGg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=APN9sFaDhW43Fp7fIpxVFbuaw8ub7NwXNvkUrkvrl4o=; fh=zB0rU6xSa7LGWTgt/l7Iil94icazssUcZE74d2AuqUk=; b=SYQlyD+sL8zG1reOgRAVaNYanG/UZRf3Auzj9UMgUZIIK0s/PCq/rzZ2ET/eFi4L56 insfx1zxwOXpD8L/6o696Lgwye6vlKXla4Wg6mGrl5LEO59DlPf4Rc2zeWrQG+CDts06 DHA/P56KBdBxLhzIdnLpWlL/9hur2mnF3TGa6lHz85C9dGSSD6ir1uYyhWCsRyih5Q2o nVqhAdqtiT5TQ2Zo38J8uKncslv+UDsttNnMAiyMaQvVExEiQyiutI5eBAh6lz/VKBFL QOmOXffZWUWrG6gXUZ3iN577aYJYdfbzLWBEBDKcLMZCG/XEPzl/B/MvsNkntGaU31ms pw3Q==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=kn8wkytW; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id r18-20020a170906705200b00a4e41d4080dsi16286452ejj.1021.2024.05.01.11.38.51; Wed, 01 May 2024 11:38:52 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=kn8wkytW; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 477DA68D6FE; Wed, 1 May 2024 21:38:26 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f49.google.com (mail-ed1-f49.google.com [209.85.208.49]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 558EA68D67E for ; Wed, 1 May 2024 21:38:18 +0300 (EEST) Received: by mail-ed1-f49.google.com with SMTP id 4fb4d7f45d1cf-56e6a1edecfso10584110a12.1 for ; Wed, 01 May 2024 11:38:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1714588697; x=1715193497; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Js7Un6oSH+9nlhzp6wDmLR5gSKboZ/0QhnUekG6JT8o=; b=kn8wkytWf2wvc5BXvGSGEX9hlYI8OuNxlpv2sf7VPePnOZNyadBg3Mhqq2traCl0xd CRYNDg0xJ82/s+DdEeSt+aTjCvGFyLEL4joWMkLBvOUvP5i83IbpJkj9Uo8Y6QW36diL 43Qm6lSLIsPLdv54nYi4sW1k7gFYGcVQH3VNE/cgrFwUyBRhUADX8AwdTtmhNSkJdxjn rMPo9TFD63BHAUEFiuBcmGFXRuCIRyTwH4EQo6kHkdfFRhUUhQgai4q6diYOzEIjTqj3 b+0ntpEEg7QH5n8y5ji48okkkPKCnILkXuQfn1pNj1dIh5R7RsrmhIQrvPEpkXzLOShA JAvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714588697; x=1715193497; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Js7Un6oSH+9nlhzp6wDmLR5gSKboZ/0QhnUekG6JT8o=; b=drK29xrCXkt9DUTaWACxtKlqObQAqs89lQTt1euHbd/4+Rm6t+lLnSMvlBM0ZhVLLk EBR9Q2VQuvdaIpPzZHjDsyk0XLWHlruS+ShJYkUR71ez6akaF5o+JdeawyVSUD1SVsMP LdhLPPMKkV8gDnPWphPjCwut4jQwYKg8lZ4EJLBBLzClTEarN2TczMIZqaGk0R6ZlOse a+KAL6lWQHNKptgSEG1m0xnYgJ1mvhHjSFxF3ZBBnkTUC+zZRAugeOcU2eTS4rnby5ww i5L83lK0hMAoRtuS2DNzNCw6a0AWIn8Un0ds2VdsoWozE4eyPsqr+fxQuxw7Zr5WehQw zPfA== X-Gm-Message-State: AOJu0YyIIUqTIQG+x9oevSOUAMfyqKeRBiE3hCdTf7O0XzuhWZ8niamU GwNB380s9OCVPQX2V/O8ZUGyomWPc7buV+x0/3T2n1VyHBkCneWU0Znm1NFb X-Received: by 2002:a17:906:c2d9:b0:a55:6355:9e1a with SMTP id ch25-20020a170906c2d900b00a5563559e1amr1940842ejb.1.1714588697044; Wed, 01 May 2024 11:38:17 -0700 (PDT) Received: from dovchinn.amd.com (cable-178-148-16-149.dynamic.sbb.rs. [178.148.16.149]) by smtp.gmail.com with ESMTPSA id kh20-20020a170906f81400b00a58bf434876sm7367337ejb.58.2024.05.01.11.38.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 May 2024 11:38:16 -0700 (PDT) From: Dmitrii Ovchinnikov To: ffmpeg-devel@ffmpeg.org Date: Wed, 1 May 2024 20:38:02 +0200 Message-Id: <20240501183809.1060-3-ovchinnikov.dmitrii@gmail.com> X-Mailer: git-send-email 2.38.1.windows.1 In-Reply-To: <20240501183809.1060-1-ovchinnikov.dmitrii@gmail.com> References: <20240501183809.1060-1-ovchinnikov.dmitrii@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 03/10, v2] avcodec/amfenc: Fixes the color information in the output. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Evgeny Pavlov , Michael Fabian 'Xaymar' Dirks , Araz Iusubov Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: n7p0LssgDjin From: Michael Fabian 'Xaymar' Dirks added 10 bit support for amf hevc. before: command - ffmpeg.exe -hide_banner -y -hwaccel d3d11va -hwaccel_output_format d3d11 -i test_10bit_file.mkv -an -c:v h264_amf res.dx11_hw_h264.mkv output - Format of input frames context (p010le) is not supported by AMF. command - ffmpeg.exe -hide_banner -y -hwaccel d3d11va -hwaccel_output_format d3d11 -i test_10bit_file -an -c:v hevc_amf res.dx11_hw_hevc.mkv output - Format of input frames context (p010le) is not supported by AMF. after: command - ffmpeg.exe -hide_banner -y -hwaccel d3d11va -hwaccel_output_format d3d11 -i test_10bit_file -an -c:v h264_amf res.dx11_hw_h264.mkv output - 10-bit input video is not supported by AMF H264 encoder command - ffmpeg.exe -hide_banner -y -hwaccel d3d11va -hwaccel_output_format d3d11 -i test_10bit_file -an -c:v hevc_amf res.dx11_hw_hevc.mkv output - 10bit file v2 - lost line returned in ff_amf_pix_fmts v3 - fixes after review v4 - extract duplicated code, fix incorrect processing of 10-bit input for h264 v5 - non-functional changes after review Co-authored-by: Evgeny Pavlov Co-authored-by: Araz Iusubov --- libavcodec/amfenc.c | 37 +++++++++++++++++++++++++++++++++++++ libavcodec/amfenc.h | 3 +++ libavcodec/amfenc_h264.c | 24 ++++++++++++++++++++---- libavcodec/amfenc_hevc.c | 26 +++++++++++++++++++++++++- 4 files changed, 85 insertions(+), 5 deletions(-) diff --git a/libavcodec/amfenc.c b/libavcodec/amfenc.c index 061859f85c..0bd15dd812 100644 --- a/libavcodec/amfenc.c +++ b/libavcodec/amfenc.c @@ -60,6 +60,7 @@ const enum AVPixelFormat ff_amf_pix_fmts[] = { #if CONFIG_DXVA2 AV_PIX_FMT_DXVA2_VLD, #endif + AV_PIX_FMT_P010, AV_PIX_FMT_NONE }; @@ -72,6 +73,7 @@ static const FormatMap format_map[] = { { AV_PIX_FMT_NONE, AMF_SURFACE_UNKNOWN }, { AV_PIX_FMT_NV12, AMF_SURFACE_NV12 }, + { AV_PIX_FMT_P010, AMF_SURFACE_P010 }, { AV_PIX_FMT_BGR0, AMF_SURFACE_BGRA }, { AV_PIX_FMT_RGB0, AMF_SURFACE_RGBA }, { AV_PIX_FMT_GRAY8, AMF_SURFACE_GRAY8 }, @@ -785,6 +787,41 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) return ret; } +int ff_amf_get_color_profile(AVCodecContext *avctx) +{ + amf_int64 color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN; + if (avctx->color_range == AVCOL_RANGE_JPEG) { + /// Color Space for Full (JPEG) Range + switch (avctx->colorspace) { + case AVCOL_SPC_SMPTE170M: + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_601; + break; + case AVCOL_SPC_BT709: + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_709; + break; + case AVCOL_SPC_BT2020_NCL: + case AVCOL_SPC_BT2020_CL: + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_2020; + break; + } + } else { + /// Color Space for Limited (MPEG) range + switch (avctx->colorspace) { + case AVCOL_SPC_SMPTE170M: + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_601; + break; + case AVCOL_SPC_BT709: + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_709; + break; + case AVCOL_SPC_BT2020_NCL: + case AVCOL_SPC_BT2020_CL: + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_2020; + break; + } + } + return color_profile; +} + const AVCodecHWConfigInternal *const ff_amfenc_hw_configs[] = { #if CONFIG_D3D11VA HW_CONFIG_ENCODER_FRAMES(D3D11, D3D11VA), diff --git a/libavcodec/amfenc.h b/libavcodec/amfenc.h index 2dbd378ef8..62736ef579 100644 --- a/libavcodec/amfenc.h +++ b/libavcodec/amfenc.h @@ -21,6 +21,7 @@ #include +#include #include #include #include @@ -170,6 +171,8 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt); */ extern const enum AVPixelFormat ff_amf_pix_fmts[]; +int ff_amf_get_color_profile(AVCodecContext *avctx); + /** * Error handling helper */ diff --git a/libavcodec/amfenc_h264.c b/libavcodec/amfenc_h264.c index abfac2a90f..ad5fcc9ecb 100644 --- a/libavcodec/amfenc_h264.c +++ b/libavcodec/amfenc_h264.c @@ -199,6 +199,8 @@ static av_cold int amf_encode_init_h264(AVCodecContext *avctx) AMFRate framerate; AMFSize framesize = AMFConstructSize(avctx->width, avctx->height); int deblocking_filter = (avctx->flags & AV_CODEC_FLAG_LOOP_FILTER) ? 1 : 0; + amf_int64 color_profile; + enum AVPixelFormat pix_fmt; if (avctx->framerate.num > 0 && avctx->framerate.den > 0) { framerate = AMFConstructRate(avctx->framerate.num, avctx->framerate.den); @@ -262,10 +264,24 @@ FF_ENABLE_DEPRECATION_WARNINGS AMF_ASSIGN_PROPERTY_RATIO(res, ctx->encoder, AMF_VIDEO_ENCODER_ASPECT_RATIO, ratio); } - /// Color Range (Partial/TV/MPEG or Full/PC/JPEG) - if (avctx->color_range == AVCOL_RANGE_JPEG) { - AMF_ASSIGN_PROPERTY_BOOL(res, ctx->encoder, AMF_VIDEO_ENCODER_FULL_RANGE_COLOR, 1); - } + color_profile = ff_amf_get_color_profile(avctx); + AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_OUTPUT_COLOR_PROFILE, color_profile); + + /// Color Range (Support for older Drivers) + AMF_ASSIGN_PROPERTY_BOOL(res, ctx->encoder, AMF_VIDEO_ENCODER_FULL_RANGE_COLOR, !!(avctx->color_range == AVCOL_RANGE_JPEG)); + + /// Color Depth + pix_fmt = avctx->hw_frames_ctx ? ((AVHWFramesContext*)avctx->hw_frames_ctx->data)->sw_format + : avctx->pix_fmt; + + // 10 bit input video is not supported by AMF H264 encoder + AMF_RETURN_IF_FALSE(ctx, pix_fmt != AV_PIX_FMT_P010, AVERROR_INVALIDDATA, "10-bit input video is not supported by AMF H264 encoder\n"); + + AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_COLOR_BIT_DEPTH, AMF_COLOR_BIT_DEPTH_8); + /// Color Transfer Characteristics (AMF matches ISO/IEC) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_OUTPUT_TRANSFER_CHARACTERISTIC, (amf_int64)avctx->color_trc); + /// Color Primaries (AMF matches ISO/IEC) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_OUTPUT_COLOR_PRIMARIES, (amf_int64)avctx->color_primaries); // autodetect rate control method if (ctx->rate_control_mode == AMF_VIDEO_ENCODER_RATE_CONTROL_METHOD_UNKNOWN) { diff --git a/libavcodec/amfenc_hevc.c b/libavcodec/amfenc_hevc.c index 0a74ccd6c4..a89a3cf20c 100644 --- a/libavcodec/amfenc_hevc.c +++ b/libavcodec/amfenc_hevc.c @@ -34,8 +34,9 @@ static const AVOption options[] = { { "high_quality", "high quality trancoding", 0, AV_OPT_TYPE_CONST, {.i64 = AMF_VIDEO_ENCODER_HEVC_USAGE_HIGH_QUALITY }, 0, 0, VE, .unit = "usage" }, { "lowlatency_high_quality","low latency yet high quality trancoding", 0, AV_OPT_TYPE_CONST, {.i64 = AMF_VIDEO_ENCODER_HEVC_USAGE_LOW_LATENCY_HIGH_QUALITY }, 0, 0, VE, .unit = "usage" }, - { "profile", "Set the profile (default main)", OFFSET(profile), AV_OPT_TYPE_INT,{ .i64 = AMF_VIDEO_ENCODER_HEVC_PROFILE_MAIN }, AMF_VIDEO_ENCODER_HEVC_PROFILE_MAIN, AMF_VIDEO_ENCODER_HEVC_PROFILE_MAIN, VE, .unit = "profile" }, + { "profile", "Set the profile (default main)", OFFSET(profile), AV_OPT_TYPE_INT,{ .i64 = AMF_VIDEO_ENCODER_HEVC_PROFILE_MAIN }, AMF_VIDEO_ENCODER_HEVC_PROFILE_MAIN, AMF_VIDEO_ENCODER_HEVC_PROFILE_MAIN_10, VE, .unit = "profile" }, { "main", "", 0, AV_OPT_TYPE_CONST,{ .i64 = AMF_VIDEO_ENCODER_HEVC_PROFILE_MAIN }, 0, 0, VE, .unit = "profile" }, + { "main10", "", 0, AV_OPT_TYPE_CONST,{ .i64 = AMF_VIDEO_ENCODER_HEVC_PROFILE_MAIN_10 }, 0, 0, VE, .unit = "profile" }, { "profile_tier", "Set the profile tier (default main)", OFFSET(tier), AV_OPT_TYPE_INT,{ .i64 = AMF_VIDEO_ENCODER_HEVC_TIER_MAIN }, AMF_VIDEO_ENCODER_HEVC_TIER_MAIN, AMF_VIDEO_ENCODER_HEVC_TIER_HIGH, VE, .unit = "tier" }, { "main", "", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_ENCODER_HEVC_TIER_MAIN }, 0, 0, VE, .unit = "tier" }, @@ -160,6 +161,9 @@ static av_cold int amf_encode_init_hevc(AVCodecContext *avctx) AMFRate framerate; AMFSize framesize = AMFConstructSize(avctx->width, avctx->height); int deblocking_filter = (avctx->flags & AV_CODEC_FLAG_LOOP_FILTER) ? 1 : 0; + amf_int64 color_depth; + amf_int64 color_profile; + enum AVPixelFormat pix_fmt; if (avctx->framerate.num > 0 && avctx->framerate.den > 0) { framerate = AMFConstructRate(avctx->framerate.num, avctx->framerate.den); @@ -187,6 +191,9 @@ FF_ENABLE_DEPRECATION_WARNINGS case AV_PROFILE_HEVC_MAIN: profile = AMF_VIDEO_ENCODER_HEVC_PROFILE_MAIN; break; + case AV_PROFILE_HEVC_MAIN_10: + profile = AMF_VIDEO_ENCODER_HEVC_PROFILE_MAIN_10; + break; default: break; } @@ -215,6 +222,23 @@ FF_ENABLE_DEPRECATION_WARNINGS AMF_ASSIGN_PROPERTY_RATIO(res, ctx->encoder, AMF_VIDEO_ENCODER_HEVC_ASPECT_RATIO, ratio); } + color_profile = ff_amf_get_color_profile(avctx); + AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_HEVC_OUTPUT_COLOR_PROFILE, color_profile); + /// Color Range (Support for older Drivers) + AMF_ASSIGN_PROPERTY_BOOL(res, ctx->encoder, AMF_VIDEO_ENCODER_HEVC_NOMINAL_RANGE, !!(avctx->color_range == AVCOL_RANGE_JPEG)); + /// Color Depth + color_depth = AMF_COLOR_BIT_DEPTH_8; + pix_fmt = avctx->hw_frames_ctx ? ((AVHWFramesContext*)avctx->hw_frames_ctx->data)->sw_format + : avctx->pix_fmt; + if (pix_fmt == AV_PIX_FMT_P010) { + color_depth = AMF_COLOR_BIT_DEPTH_10; + } + AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_HEVC_COLOR_BIT_DEPTH, color_depth); + /// Color Transfer Characteristics (AMF matches ISO/IEC) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_HEVC_OUTPUT_TRANSFER_CHARACTERISTIC, (amf_int64)avctx->color_trc); + /// Color Primaries (AMF matches ISO/IEC) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_HEVC_OUTPUT_COLOR_PRIMARIES, (amf_int64)avctx->color_primaries); + // Picture control properties AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_HEVC_NUM_GOPS_PER_IDR, ctx->gops_per_idr); AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_HEVC_GOP_SIZE, avctx->gop_size); From patchwork Wed May 1 18:38:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitrii Ovchinnikov X-Patchwork-Id: 48449 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:1203:b0:1a9:af23:56c1 with SMTP id v3csp1103959pzf; Wed, 1 May 2024 11:39:02 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCVyB4UujekUYsEGiZyyquzXJXv4wtYvbbl+VMI3dyZ2o2P+TsB4dNPLZQkvlXgOWAMugO4QFXgq51YtAGcMgx6VxDjIXPqU3QTdzg== X-Google-Smtp-Source: AGHT+IGhA5e4oeakST2ZpUZ2l/R9paxH3aX3/5TSRpz91Rxs9NDr2+6qAtL6R3mGSaeKociYJ3tQ X-Received: by 2002:a17:906:4712:b0:a58:ebdd:8aa4 with SMTP id y18-20020a170906471200b00a58ebdd8aa4mr323230ejq.20.1714588741772; Wed, 01 May 2024 11:39:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1714588741; cv=none; d=google.com; s=arc-20160816; b=ZzfXpHMG0f/pW5p8WftOum1KGKu50VMlDWmI4oBo1//Ou1Y6hLZwri+0Gz0KOvZ2GC NT4Q+6Ouqkg7DXLKKUuesaIYJHHPe3r1lQYjuUQkd0ihdDX6tEdhhtKC71KHhxLzB5Nm Kl4IcXjKrTH3Wra4Dz/EOtpNc7wNDavZP+MXoSPl6JSQMcf6W4qCX4q+NreGb1E+5Mfn /OYMtWUgUlUWEyzSa3pbGAf0zv/XvPH9WNkzQbokH8SaCtyUOhpl/AB9IVfsnrzfBKYC 2D26VZc8icpD8Hzn956RxUfc/7FKIjf/rACRntEp6bRySyARhUH6Ql+6uWZ5MyZQjJCS 9v0g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=y+A/NS4FgHHaJcqLidHtzs8QtkRkU9zyYH0MG9eYEcg=; fh=s4NEqlgyeQvuJ9M2FER/qIrd96lZ1v/frktaqQpFGmg=; b=jDyvpFnFhNCg0UkCtm/e215rhcLHX2aMZWZy2hPt2MBf3yU2DcJTa1xmYdoDpoYsRM n4TUGETpUv7qVvJpzXERdCJrAOVtRe4xrgcch1cGHpa5Ol4sgQdNzPeLVu53VYX+SWgq +8vmu0XQIX2hAor0KdMTe/QfCiIR8/BNh93n3vHjoGxIjk9uEoS0Nv+/3GGCwaDYhlvs 5R5HH5KwnjKOM+Vc64/L9yZSzttEIN/qJdAh4iNLMOBvOtvggnPYF4BbZC9+Yo0snjN/ lBlxex85yfxhNSzg7E46OJ6Hcn7uRRcHwPsH3mHjSxVQmJQZKDvOPmbU2SomGx+cjEJl BNew==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=P8XmVl83; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id dm1-20020a05640222c100b0056dc864382asi16833725edb.158.2024.05.01.11.39.01; Wed, 01 May 2024 11:39:01 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=P8XmVl83; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 95DFD68D6F3; Wed, 1 May 2024 21:38:27 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f52.google.com (mail-ej1-f52.google.com [209.85.218.52]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3717A68D6F3 for ; Wed, 1 May 2024 21:38:19 +0300 (EEST) Received: by mail-ej1-f52.google.com with SMTP id a640c23a62f3a-a5966f5a76bso15581866b.1 for ; Wed, 01 May 2024 11:38:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1714588698; x=1715193498; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2NMJSLFysE9TxkZix8jJsJb4xxeIhpp9vnuBQqDGZbw=; b=P8XmVl83VtAblGSByTKPgyHpNcAKbA0dkjkD/Jo/UAQJUbyIo+O4RNysSXOCk8LGyh wejzfnMcGs1LqNCO4G2F3yN+ICoqHAKuLMRAQgenpN+R0QP5olglzVW7WVC54l5VW9tB t6Tyv8oxNaafhQNBmd0R/gjmZRpfQEgTVyuNY6x7fl/VZChd3Mo7FsqMglfR/5CZt6qU uVlG/bCgAtqHKh7SiOCzpBjU178eZzgLd8/tmvP1JGVE0Z5v4CI09LSr5vvlIdYh/F2X DjYi9sHbRVuaK51PuVf0wafwLfJPC1RktYD/GfV5StSNV8LYNyQ/GB5utTthAcu/eRRR /o4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714588698; x=1715193498; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2NMJSLFysE9TxkZix8jJsJb4xxeIhpp9vnuBQqDGZbw=; b=VTcmhsQqvdc/RPkyDGiPh9nMgX4eYBi6PRPc15u76KxUUcYfrGiyXHzd1tRWGknkJc 7lQIYik57YzgeSvIZUi+1iPSlNdCjoL+4z6klodDHYgq1Yk9u0zB4DXQCDf6Pq8KmES/ Ul9sagHZhs8BBiDa2kTwtogPaPJc/p8ZC36HoBt6zY45nR+jpkKm6+6FmVeR437L1bWH GBT1hYVsI1bl6vRW4x88hHkGH3efGWPBFjJLBWwNONv4xOOKQTAdQwlQlY7axGK6Bbo7 AqqjsBeeTcB3KCUplYe9eDAwDsnmAYXROPifcgTEl0PN92bAgzxrqnd6M4fiYwOXprgn DXSw== X-Gm-Message-State: AOJu0Yxqeqpy3374owJ7gcI56a9soejTwXy4eDnbud4s/xw/BKmTlsoE OimXdcc8GJz9Wf5DcARRfxda9faYsHAELoOJ6bH7aG6s7iM9fUk7F4ObP73+ X-Received: by 2002:a17:906:f1c6:b0:a58:c239:48ae with SMTP id gx6-20020a170906f1c600b00a58c23948aemr302888ejb.12.1714588698145; Wed, 01 May 2024 11:38:18 -0700 (PDT) Received: from dovchinn.amd.com (cable-178-148-16-149.dynamic.sbb.rs. [178.148.16.149]) by smtp.gmail.com with ESMTPSA id kh20-20020a170906f81400b00a58bf434876sm7367337ejb.58.2024.05.01.11.38.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 May 2024 11:38:17 -0700 (PDT) From: Dmitrii Ovchinnikov To: ffmpeg-devel@ffmpeg.org Date: Wed, 1 May 2024 20:38:03 +0200 Message-Id: <20240501183809.1060-4-ovchinnikov.dmitrii@gmail.com> X-Mailer: git-send-email 2.38.1.windows.1 In-Reply-To: <20240501183809.1060-1-ovchinnikov.dmitrii@gmail.com> References: <20240501183809.1060-1-ovchinnikov.dmitrii@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 04/10, v2] avcodec/amfenc: HDR metadata. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: nyanmisaka Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: koHKmqBoZQBM From: nyanmisaka v2: fixes for indentation --- libavcodec/amfenc.c | 83 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 83 insertions(+) diff --git a/libavcodec/amfenc.c b/libavcodec/amfenc.c index 0bd15dd812..068bb53002 100644 --- a/libavcodec/amfenc.c +++ b/libavcodec/amfenc.c @@ -36,6 +36,57 @@ #include "amfenc.h" #include "encode.h" #include "internal.h" +#include "libavutil/mastering_display_metadata.h" + +static int amf_save_hdr_metadata(AVCodecContext *avctx, const AVFrame *frame, AMFHDRMetadata *hdrmeta) +{ + AVFrameSideData *sd_display; + AVFrameSideData *sd_light; + AVMasteringDisplayMetadata *display_meta; + AVContentLightMetadata *light_meta; + + sd_display = av_frame_get_side_data(frame, AV_FRAME_DATA_MASTERING_DISPLAY_METADATA); + if (sd_display) { + display_meta = (AVMasteringDisplayMetadata *)sd_display->data; + if (display_meta->has_luminance) { + const unsigned int luma_den = 10000; + hdrmeta->maxMasteringLuminance = + (amf_uint32)(luma_den * av_q2d(display_meta->max_luminance)); + hdrmeta->minMasteringLuminance = + FFMIN((amf_uint32)(luma_den * av_q2d(display_meta->min_luminance)), hdrmeta->maxMasteringLuminance); + } + if (display_meta->has_primaries) { + const unsigned int chroma_den = 50000; + hdrmeta->redPrimary[0] = + FFMIN((amf_uint16)(chroma_den * av_q2d(display_meta->display_primaries[0][0])), chroma_den); + hdrmeta->redPrimary[1] = + FFMIN((amf_uint16)(chroma_den * av_q2d(display_meta->display_primaries[0][1])), chroma_den); + hdrmeta->greenPrimary[0] = + FFMIN((amf_uint16)(chroma_den * av_q2d(display_meta->display_primaries[1][0])), chroma_den); + hdrmeta->greenPrimary[1] = + FFMIN((amf_uint16)(chroma_den * av_q2d(display_meta->display_primaries[1][1])), chroma_den); + hdrmeta->bluePrimary[0] = + FFMIN((amf_uint16)(chroma_den * av_q2d(display_meta->display_primaries[2][0])), chroma_den); + hdrmeta->bluePrimary[1] = + FFMIN((amf_uint16)(chroma_den * av_q2d(display_meta->display_primaries[2][1])), chroma_den); + hdrmeta->whitePoint[0] = + FFMIN((amf_uint16)(chroma_den * av_q2d(display_meta->white_point[0])), chroma_den); + hdrmeta->whitePoint[1] = + FFMIN((amf_uint16)(chroma_den * av_q2d(display_meta->white_point[1])), chroma_den); + } + + sd_light = av_frame_get_side_data(frame, AV_FRAME_DATA_CONTENT_LIGHT_LEVEL); + if (sd_light) { + light_meta = (AVContentLightMetadata *)sd_light->data; + if (light_meta) { + hdrmeta->maxContentLightLevel = (amf_uint16)light_meta->MaxCLL; + hdrmeta->maxFrameAverageLightLevel = (amf_uint16)light_meta->MaxFALL; + } + } + return 0; + } + return 1; +} #if CONFIG_D3D11VA #include @@ -683,6 +734,26 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) frame_ref_storage_buffer->pVtbl->Release(frame_ref_storage_buffer); } + // HDR10 metadata + if (frame->color_trc == AVCOL_TRC_SMPTE2084) { + AMFBuffer * hdrmeta_buffer = NULL; + res = ctx->context->pVtbl->AllocBuffer(ctx->context, AMF_MEMORY_HOST, sizeof(AMFHDRMetadata), &hdrmeta_buffer); + if (res == AMF_OK) { + AMFHDRMetadata * hdrmeta = (AMFHDRMetadata*)hdrmeta_buffer->pVtbl->GetNative(hdrmeta_buffer); + if (amf_save_hdr_metadata(avctx, frame, hdrmeta) == 0) { + switch (avctx->codec->id) { + case AV_CODEC_ID_H264: + AMF_ASSIGN_PROPERTY_INTERFACE(res, ctx->encoder, AMF_VIDEO_ENCODER_INPUT_HDR_METADATA, hdrmeta_buffer); break; + case AV_CODEC_ID_HEVC: + AMF_ASSIGN_PROPERTY_INTERFACE(res, ctx->encoder, AMF_VIDEO_ENCODER_HEVC_INPUT_HDR_METADATA, hdrmeta_buffer); break; + } + res = amf_set_property_buffer(surface, L"av_frame_hdrmeta", hdrmeta_buffer); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "SetProperty failed for \"av_frame_hdrmeta\" with error %d\n", res); + } + hdrmeta_buffer->pVtbl->Release(hdrmeta_buffer); + } + } + surface->pVtbl->SetPts(surface, frame->pts); AMF_ASSIGN_PROPERTY_INT64(res, surface, PTS_PROP, frame->pts); @@ -746,6 +817,18 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) } res_resubmit = AMF_OK; if (ctx->delayed_surface != NULL) { // try to resubmit frame + if (ctx->delayed_surface->pVtbl->HasProperty(ctx->delayed_surface, L"av_frame_hdrmeta")) { + AMFBuffer * hdrmeta_buffer = NULL; + res = amf_get_property_buffer((AMFData *)ctx->delayed_surface, L"av_frame_hdrmeta", &hdrmeta_buffer); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "GetProperty failed for \"av_frame_hdrmeta\" with error %d\n", res); + switch (avctx->codec->id) { + case AV_CODEC_ID_H264: + AMF_ASSIGN_PROPERTY_INTERFACE(res, ctx->encoder, AMF_VIDEO_ENCODER_INPUT_HDR_METADATA, hdrmeta_buffer); break; + case AV_CODEC_ID_HEVC: + AMF_ASSIGN_PROPERTY_INTERFACE(res, ctx->encoder, AMF_VIDEO_ENCODER_HEVC_INPUT_HDR_METADATA, hdrmeta_buffer); break; + } + hdrmeta_buffer->pVtbl->Release(hdrmeta_buffer); + } res_resubmit = ctx->encoder->pVtbl->SubmitInput(ctx->encoder, (AMFData*)ctx->delayed_surface); if (res_resubmit != AMF_INPUT_FULL) { int64_t pts = ctx->delayed_surface->pVtbl->GetPts(ctx->delayed_surface); From patchwork Wed May 1 18:38:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitrii Ovchinnikov X-Patchwork-Id: 48450 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:1203:b0:1a9:af23:56c1 with SMTP id v3csp1104102pzf; Wed, 1 May 2024 11:39:15 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCWUOi9Bd8Gwt58y0GtdXW4HFjORIDVdKHLWExpjLO9qngbANsFovd6mDvLzpjJoljkEcfA5U5Ihe6HhYqgCaNJMEbfRYjCIqfK3OQ== X-Google-Smtp-Source: AGHT+IH+j2EB4cbMrR0D2BZLVBmDpFWIYLHxNwh+gDMuKNaxEDY4V0TfgGzX7maM4Yp6QU2aFHFG X-Received: by 2002:a2e:a684:0:b0:2de:e0dd:e789 with SMTP id q4-20020a2ea684000000b002dee0dde789mr2224348lje.21.1714588752556; Wed, 01 May 2024 11:39:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1714588752; cv=none; d=google.com; s=arc-20160816; b=yobXcN4EKa9Mctxo47zrvEJ49R65koZnqb99adZ1TgZoqKuL6DZ95rB+JuodrPbcVN BWOLZCQJF0Yo5GwMYGXbKDPCTTCiFbOKNJ2pXk6HMaZ/bP1Z0h/uQNjmGp9S4UEHKFNg TbzmIqhadpFDY/2rRosY+C1yEN3/lNAIe0oGEK7hZQHV/szYov/7H9WobdBqbQKYLVUC lx8c7LVZ4W0t85mdm5MupgUmTUTkAnUo35XktU9CHmDT5phfeCUzR91F5vAPqoY3gkCi kCIjT5T2pz7VvHXhwf0WJlQ81L0lM56soM0BSUrp+CWz/6LVQhUmjUergK/8ZeohQMG4 6MJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=NpHMqHn07asFapk+9vqptMEkdOF0DnNMqvQS8Jqia6c=; fh=ZIZ7Ej7aoSMNmZSSnSJ7NXY4c5NrV8up1kK8HgJcvhA=; b=gGnylWEp3zXsd6FjUE06qC1EYoVS7pfbp5MYYJr/fMLZ5+q+rUk16jZVWsPOla0zjI YRr9Ik3uy+BnRD0cWgVEk2Mn6AyFNtFismhSJmbVz/7ojUcQy29XrAXPO9FN7syyI7ov FjNG9UdnWy82m7ah5QykH2Xmr9lk9I9MvPBhouBAvJuGc+GtXGgC7Yg0mv04/MaGVqxF rT2v8tbbgKataY2A1MHOmmLx8EQ9N2hr8GgfQmdHs0fcEnMZD1MwmzolcJ5axkzRQJWX mKJo9QAej/dxWXhg+Pl+b9IXPbSgsig+lArLJEiD2hW22OjtaS6nK31G3onAF89niO6K SMjA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=ZEu9XXJF; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id q21-20020a2e8755000000b002d6cf1f86b1si8811499ljj.67.2024.05.01.11.39.11; Wed, 01 May 2024 11:39:12 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=ZEu9XXJF; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0861C68D70F; Wed, 1 May 2024 21:38:29 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f50.google.com (mail-ej1-f50.google.com [209.85.218.50]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4A8CE68D6F9 for ; Wed, 1 May 2024 21:38:20 +0300 (EEST) Received: by mail-ej1-f50.google.com with SMTP id a640c23a62f3a-a5883518135so793675466b.3 for ; Wed, 01 May 2024 11:38:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1714588699; x=1715193499; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7VOLezHTsoTsOS0oo+4eyxvfiUlPvhAxWMTXRubE+g8=; b=ZEu9XXJFoiMrQKlaAjv5MyJVOYgguhyS2+JNgsEkNy9DEI+D4ocRLs9w5Cejq5uJ96 RbtYqFXIcWaVfljfelqLofP+qQRbnzJD2JzPM20dRgU5WvS8L9KaMOuEcjCcEQQrnArG bhxobOUxvVADCKYZN8xTgjb+kKzCP1+X23aLhr9LM0d7ltMmLq1ETp0zIRA/3czDCPRp AmAHek0S1fM64dBXnWhqAi1WDu/FnWAscWsXppL4p5wnV4VHWkL0lk5yvHGN6XG0EjO4 l6vvYWshQxdmosoWOBHSFwwmMczGgnuW5Pl9Vi3vre0xEDKuVWudmnaVaUa/rQtcpEr0 zZug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714588699; x=1715193499; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7VOLezHTsoTsOS0oo+4eyxvfiUlPvhAxWMTXRubE+g8=; b=p+BcM7GWltrVEs4wSfq7i0H/OH55r8vgaK687DVkzxfwyDo+ccrrLHEn+A366IKkXR c+Az/kSkxg2Knm3doP25OA1k/5yzzBsgrhJr98hP4ThtM8weL8DIbsSHzjObIbAXzR9d TVhKGRJxvd1TQqW2tAdqWu53zTfbI6I83IhxQn5GYuVhL2Asmr0yydVTpneJRD+p03Au gDuiMu9e/fdX6gaDbiHUa716kwOhQ+LG05V0GM7Voe0zFPPpLXlxZIDmXKvLRJ7iZal1 JtGA2OXglDBFz2ELon2vN6PKUHWCLeuPmD1KOwvYFbH7m1GCe0DxyciKryyQMXb+7P9A +rOw== X-Gm-Message-State: AOJu0YwxT6MZfxH2gie3RdCUpSFo1g6ZCRblfl/1d3W3YZPFfvuz6GtN FsymmLWiqFrp+DuEPLD4SBVxGKtpPtusGxUJz4C6+MnqtbJYzklT+D11YpYD X-Received: by 2002:a17:906:f353:b0:a52:3f01:e11d with SMTP id hg19-20020a170906f35300b00a523f01e11dmr2373592ejb.34.1714588699349; Wed, 01 May 2024 11:38:19 -0700 (PDT) Received: from dovchinn.amd.com (cable-178-148-16-149.dynamic.sbb.rs. [178.148.16.149]) by smtp.gmail.com with ESMTPSA id kh20-20020a170906f81400b00a58bf434876sm7367337ejb.58.2024.05.01.11.38.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 May 2024 11:38:18 -0700 (PDT) From: Dmitrii Ovchinnikov To: ffmpeg-devel@ffmpeg.org Date: Wed, 1 May 2024 20:38:04 +0200 Message-Id: <20240501183809.1060-5-ovchinnikov.dmitrii@gmail.com> X-Mailer: git-send-email 2.38.1.windows.1 In-Reply-To: <20240501183809.1060-1-ovchinnikov.dmitrii@gmail.com> References: <20240501183809.1060-1-ovchinnikov.dmitrii@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 05/10, v2] avcodec/amfenc: add 10 bit encoding in av1_amf X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Evgeny Pavlov , Araz Iusubov Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: WnQz99jKGin7 From: Evgeny Pavlov v2: refactored after review Signed-off-by: Evgeny Pavlov Co-authored-by: Araz Iusubov --- libavcodec/amfenc.c | 2 ++ libavcodec/amfenc_av1.c | 22 ++++++++++++++++++++++ 2 files changed, 24 insertions(+) diff --git a/libavcodec/amfenc.c b/libavcodec/amfenc.c index 068bb53002..49dd91c4e0 100644 --- a/libavcodec/amfenc.c +++ b/libavcodec/amfenc.c @@ -746,6 +746,8 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) AMF_ASSIGN_PROPERTY_INTERFACE(res, ctx->encoder, AMF_VIDEO_ENCODER_INPUT_HDR_METADATA, hdrmeta_buffer); break; case AV_CODEC_ID_HEVC: AMF_ASSIGN_PROPERTY_INTERFACE(res, ctx->encoder, AMF_VIDEO_ENCODER_HEVC_INPUT_HDR_METADATA, hdrmeta_buffer); break; + case AV_CODEC_ID_AV1: + AMF_ASSIGN_PROPERTY_INTERFACE(res, ctx->encoder, AMF_VIDEO_ENCODER_AV1_INPUT_HDR_METADATA, hdrmeta_buffer); break; } res = amf_set_property_buffer(surface, L"av_frame_hdrmeta", hdrmeta_buffer); AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "SetProperty failed for \"av_frame_hdrmeta\" with error %d\n", res); diff --git a/libavcodec/amfenc_av1.c b/libavcodec/amfenc_av1.c index 9f18aac648..cc48e93fcb 100644 --- a/libavcodec/amfenc_av1.c +++ b/libavcodec/amfenc_av1.c @@ -165,6 +165,9 @@ static av_cold int amf_encode_init_av1(AVCodecContext* avctx) AMFGuid guid; AMFRate framerate; AMFSize framesize = AMFConstructSize(avctx->width, avctx->height); + amf_int64 color_depth; + amf_int64 color_profile; + enum AVPixelFormat pix_fmt; @@ -203,6 +206,25 @@ FF_ENABLE_DEPRECATION_WARNINGS } AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_AV1_PROFILE, profile); + /// Color profile + color_profile = ff_amf_get_color_profile(avctx); + AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_AV1_OUTPUT_COLOR_PROFILE, color_profile); + + /// Color Depth + pix_fmt = avctx->hw_frames_ctx ? ((AVHWFramesContext*)avctx->hw_frames_ctx->data)->sw_format + : avctx->pix_fmt; + color_depth = AMF_COLOR_BIT_DEPTH_8; + if (pix_fmt == AV_PIX_FMT_P010) { + color_depth = AMF_COLOR_BIT_DEPTH_10; + } + + AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_AV1_COLOR_BIT_DEPTH, color_depth); + AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_AV1_OUTPUT_COLOR_PROFILE, color_profile); + /// Color Transfer Characteristics (AMF matches ISO/IEC) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_AV1_OUTPUT_TRANSFER_CHARACTERISTIC, (amf_int64)avctx->color_trc); + /// Color Primaries (AMF matches ISO/IEC) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_AV1_OUTPUT_COLOR_PRIMARIES, (amf_int64)avctx->color_primaries); + profile_level = avctx->level; if (profile_level == AV_LEVEL_UNKNOWN) { profile_level = ctx->level; From patchwork Wed May 1 18:38:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitrii Ovchinnikov X-Patchwork-Id: 48453 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:1203:b0:1a9:af23:56c1 with SMTP id v3csp1110965pzf; Wed, 1 May 2024 11:49:35 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCXqNaNRlOU/9kqJzY697XBh6jYAV5H3UCiV3DuxbNsrwPfL9IIrJG6r8Y29c42VtokHX1vzRe1hUWmBxlyC1zGA9P8KSJyNhNFa4Q== X-Google-Smtp-Source: AGHT+IHhhvAyahl3ag23UxdM8ZPEqKhreEBRwDj2xqRT9/UJzqib/4kuEE7FuLZvzeW6X1vIA6za X-Received: by 2002:a2e:914b:0:b0:2de:9c32:b98b with SMTP id q11-20020a2e914b000000b002de9c32b98bmr2310693ljg.2.1714589373391; Wed, 01 May 2024 11:49:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1714589373; cv=none; d=google.com; s=arc-20160816; b=wLVvgWbe7zZt0MWEEe3RJnarZQBoP3i1yAt9ApaOL2E5f1ckf4Q96uz6S0FJQJGEPX mqtc2UpRMk36omgO4UKCewjgwiR3aW/me8mBTaTa2Y+044WgI448NUWiNLFVN29yGlUK 7XuPRIzxViaJCww9k4OEi1LgsbyU0qIX3LExYOZ9JIjnfqL/dPgS4uXhcqGpZT+wtzJD NPFdR1Q48qhhRngaO8Ze63M/HDgmbqsSReu5IiPqPj4WPGTc6euzOXvy9pvedCu0vmLT bTACneN7mDvj1VM9oMnCBBaETaB4VFXMTBn25Nl842dbUk/lVjZl1LUrt++hPV7uf5CB om6g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=6Zc52CFUA3Loa6TedBqh6ZVjLpJylzJIasNquqC8OjU=; fh=OWLPUf18s9dlUQuxhl6xLimy4NybloePU6lVbhxT1Ck=; b=MrClPdDordZUpfS2FZXKJPI2bxtLGGd7kdkHCaGm8+LvtZ2HKYLkPnPVI1uNLZu1h5 A8JDg9XUTEO0tkGfeyVEutkj/rnyCWgi8OHxt3wQ/J8fToHJoxfHlgV9ugWjudWsQFLz k/3EoEyuHolt8NBlQYBnwhd/sWgQ04v0y20e4jXT58W7clwXoHyAdiLhBTFuX6U8uV9O Udj7X6l4nrww/B6aw0wQWANaEXJKyHhEP0w5CdNvvIqO9wLWGVYRluNJ/hBeoao9j50S iYb74nXpI5l/zo4zSFP7lb+ioW9NkUUFaIzy+frmOKCaKXT6225FY/gef/CJPlgNrDIg MtzA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=mjNxvT6U; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id e12-20020a2e930c000000b002d889c5de23si9095937ljh.100.2024.05.01.11.49.32; Wed, 01 May 2024 11:49:33 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=mjNxvT6U; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4649968D6FF; Wed, 1 May 2024 21:38:31 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f45.google.com (mail-ej1-f45.google.com [209.85.218.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 40C0668D6FD for ; Wed, 1 May 2024 21:38:21 +0300 (EEST) Received: by mail-ej1-f45.google.com with SMTP id a640c23a62f3a-a524ecaf215so882038166b.2 for ; Wed, 01 May 2024 11:38:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1714588700; x=1715193500; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=oGEnjpZqbfU16Ro3j/O+gYRfqptfl2dXlCAtKMWZnik=; b=mjNxvT6UdUWLWGGD9Mzo09/+r0rxwdHGZCoUyTHKrKAFNFpMjKqkBtVMdH/O89jCzd fQJKiyzZbdAiZJJ2fHwv1LMnr9xQCWNssujZb4Aizc19HNWtabXCbMwQDA21IvhFcPc2 43zRTsQQ1y2mUqaMGT/KcEq+LckaPGI7yPARmXMvZKqituEtBwASQBj0T2RJ1U7Zmewy QVQXjCdTIPpzueLvhwoqe6f98k3lEdXc4u9zYL/zr0HarNTUccqRGCWIoQkcIg/OONV/ jzCUsk8usV9wZ6Qavj2hW3SY71LIJVOL4LZIwA3DurrrbxkiCv/JeMjTLTxIr6DOUFN7 pB3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714588700; x=1715193500; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oGEnjpZqbfU16Ro3j/O+gYRfqptfl2dXlCAtKMWZnik=; b=Jn32hnlgiTYmM0d3BXePNGuLhdDuf+QcUP6uM9qNQelFJMoE5wGJy/ald+5/2xCZRq uhqM4uF+/P1RmH1620Zutd13YXJgAYGnNBnu90Gc+Q0gjHx/0Snjq20sL6qw9a5KtJTr 7iBowexAcaxqQoYZog7yAzj7tngKs3fkNOZK6cKyaKRraPh2XnHJ4cza3Pmv5mFFNdfp 2L7jxlo7ZUBSmJqg6IGU/gDvU6DIE+9zEuQ0PtbRS1QAy0T17OwAz4vhc7qeRrFK/zik sITZixvYZwbh6dSmWSVnhxjhTSoTvZ4gQA4Hv7BuDaWeWv1Uw69LPn2DHgMgE5ZUf83K jLPA== X-Gm-Message-State: AOJu0Yw1qeVhPvdwYgiFlciyykTM8PKaPV7YQ30Kl0Dw5vtX/B09ysiZ vGRD+YxERaNem3U4dCrSoHppYadu4tiDZyWP4gni97xfsxNrjl0G3iHfpWjS X-Received: by 2002:a17:907:1047:b0:a58:d757:6cfe with SMTP id oy7-20020a170907104700b00a58d7576cfemr2466112ejb.6.1714588700198; Wed, 01 May 2024 11:38:20 -0700 (PDT) Received: from dovchinn.amd.com (cable-178-148-16-149.dynamic.sbb.rs. [178.148.16.149]) by smtp.gmail.com with ESMTPSA id kh20-20020a170906f81400b00a58bf434876sm7367337ejb.58.2024.05.01.11.38.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 May 2024 11:38:19 -0700 (PDT) From: Dmitrii Ovchinnikov To: ffmpeg-devel@ffmpeg.org Date: Wed, 1 May 2024 20:38:05 +0200 Message-Id: <20240501183809.1060-6-ovchinnikov.dmitrii@gmail.com> X-Mailer: git-send-email 2.38.1.windows.1 In-Reply-To: <20240501183809.1060-1-ovchinnikov.dmitrii@gmail.com> References: <20240501183809.1060-1-ovchinnikov.dmitrii@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 06/10, v2] avcodec/amfenc: GPU driver version check X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Araz Iusubov Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: ouxstu8pLqnB From: Araz Iusubov Implemented gpu driver check. 10-bit patch works incorrectly on driver version lower than 23.30. Signed-off-by: Araz Iusubov --- libavcodec/amfenc.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/libavcodec/amfenc.c b/libavcodec/amfenc.c index 49dd91c4e0..510050e282 100644 --- a/libavcodec/amfenc.c +++ b/libavcodec/amfenc.c @@ -558,6 +558,10 @@ int ff_amf_encode_init(AVCodecContext *avctx) if ((ret = amf_load_library(avctx)) == 0) { if ((ret = amf_init_context(avctx)) == 0) { if ((ret = amf_init_encoder(avctx)) == 0) { + if (avctx->pix_fmt == AV_PIX_FMT_P010) { + AmfContext *ctx = avctx->priv_data; + AMF_RETURN_IF_FALSE(ctx, ctx->version >= AMF_MAKE_FULL_VERSION(1, 4, 32, 0), AVERROR_UNKNOWN, "10-bit encoder is not supported by AMD GPU drivers versions lower than 23.30.\n"); + } return 0; } } From patchwork Wed May 1 18:38:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitrii Ovchinnikov X-Patchwork-Id: 48452 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:1203:b0:1a9:af23:56c1 with SMTP id v3csp1110923pzf; Wed, 1 May 2024 11:49:31 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCUghoTZm8GFKnDrK0jFz78j+DKtpm6m8C8N8K9zkEy6XjoIg9KN5pfXiuGN489wK8XVRnKPDqHEDW9RoFlAU1mU3YjiPra/YGHWdA== X-Google-Smtp-Source: AGHT+IH0oLVRq6OH6pybJn5kHml02A1Bcp2iIwl3E5ICv8eI1hkgZ9VqjfS/uI8v+FhPbyu+V1IA X-Received: by 2002:a50:ab18:0:b0:572:a7ec:1a5f with SMTP id s24-20020a50ab18000000b00572a7ec1a5fmr2373434edc.24.1714589370762; Wed, 01 May 2024 11:49:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1714589370; cv=none; d=google.com; s=arc-20160816; b=p2LrJtzXpqXkn/E3b0Pb0zafkV/QZ3x8CzTjDlQuqtkoi1UpfC8gLq3pln6szMOSvD bO1aXDdkjlBVA41a8c7Qpu0IjhOdeQaIWFSxgZP++5eU2zSnw16jhq8W1T/TWmRm4uTm L3+W4jce+Qkk68xqVrPWnSB01V5fyoTf4s8Hc8S9b+VzpmkOj3FxI0gu6rH3bEq/bwK3 Q1Ji9T19PYjRkgtdGCoXczWiKBcHjvzNiQMtom5ATBQcN73Q+CJ2ov2Py7vwS/m0xVDU iSu3ger6AFWYhcHHpbgr3HCvfvASk1FgUcn1ZduL8MZD01k5CewnTIZCgrp5mQ+N3mdc hsiA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=p21poJytvM81HDQoP5+GKU3+bmO1tGkmfc/sQ5F5oT4=; fh=l8BBgSaHK1qPkRTUMT3LasR23D+FgYCLMIPW5YpagSs=; b=wNRwKF6ytgjGR58TMZVLYCmf+xa9iRuheUpvNo3xyY4x8eTKJw7GgWtt1IyRWneyzF rgNazaIJH65Zn1fj8FtuCLCZFcB9ZAnQa8Fx2tZwQENXWJH5mwaJR5W0XR7fdcqmlRIQ q3IZOrwon+1yllqUNnVr3gcTdZUyOjkY2EQh30R6F7bJ19IOuwkNyGvX5eQxf8rwFW7I WLnYqsllfZjflQq/3uKM+JmlEIgBgK12vMCnBuYFMpRyUryi6gq0AWT/J0aR6CNGuuiH lfBytUZ81W3ZbxbEKLPtM/huiG7b0+GHYquUf3sRCIdd0LUV8Ork/Kj8bzw9sr/j+6hC SeYg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=i409Nx4B; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id bo5-20020a0564020b2500b00572b35dbff4si344730edb.260.2024.05.01.11.49.30; Wed, 01 May 2024 11:49:30 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=i409Nx4B; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id AE12468D710; Wed, 1 May 2024 21:38:33 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f52.google.com (mail-ej1-f52.google.com [209.85.218.52]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 16F7968D701 for ; Wed, 1 May 2024 21:38:22 +0300 (EEST) Received: by mail-ej1-f52.google.com with SMTP id a640c23a62f3a-a58d0aea14cso713365266b.2 for ; Wed, 01 May 2024 11:38:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1714588701; x=1715193501; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FB+xkIZAK0Ta8G0mP47ML47wYzZvqx3U1zO2LtbScX8=; b=i409Nx4B9KXBQQjph0PL2npNh6Z46nBjUo77XJ3Hz7Wr1cVn2dCakQ7dDXnTkpdykm Kw23R7tyN6bjYfJsYZJpi7OAUlk7PrxqHYHzhFNZ0dF9YCIGmCN5cXTa/LAxj8eEAyTI WumGAoMYkOzZaLKdzUkeKE4PmhQ8inPJh0TmttSEzy+SSbP6m/V7bkrQL0a7NvT9Va9o fTx14ckk1o01X/f3FvH7A/WOvPsfUMn3zJaM5HcpF8xrmqObz8On5LeB0SHgTSLsJhEm n8V1qe27AOmo5hp+8gozei57Esj3Xvp2s5Zr41xvPEmiomckqdfbxlHXKjNZ0b1TQeqT igmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714588701; x=1715193501; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FB+xkIZAK0Ta8G0mP47ML47wYzZvqx3U1zO2LtbScX8=; b=beemssnxYLn5CDKDzzhg0tcrpUVApcQipR7oufWrSBtmNC8e9haeFRnNNnBkhY1nq/ AcMBRBLIi812kDKSI93B8CjUlIZEnMp8jIehST7boHBxK2Di1inGCGGT7KO+u6s6FsCO LY8uj3xOfkRi9oxREdMyqLfMGkSxUWaDyUthU2FACxReAbgZ/4GACzKxFTJOtfQZ41kT iKN6wIz6fdZlSax7RZv2ZKxjJA59k6+SBqZuMiCWv4AklT7LYdSp6Bv2YYGtehnSr9oo qg23BBSKgexR3Sf58e11X+Ua8vE7seFlIj3q+2vYS2kgiPBcGXs2cCWXloDKEvwnEHo4 A6EQ== X-Gm-Message-State: AOJu0Yz/ZP4lTqhGDnxiQffRRQVvRz9WP37NOKysajZYiKZ9ISX6gm0P umtw++DQSMPVEicdl3YhxRaRgNFkektEiGYqCwrAZnLXcvFBlWmG8WFtxj+t X-Received: by 2002:a17:906:3c15:b0:a46:3ce4:5acb with SMTP id h21-20020a1709063c1500b00a463ce45acbmr2418028ejg.75.1714588701038; Wed, 01 May 2024 11:38:21 -0700 (PDT) Received: from dovchinn.amd.com (cable-178-148-16-149.dynamic.sbb.rs. [178.148.16.149]) by smtp.gmail.com with ESMTPSA id kh20-20020a170906f81400b00a58bf434876sm7367337ejb.58.2024.05.01.11.38.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 May 2024 11:38:20 -0700 (PDT) From: Dmitrii Ovchinnikov To: ffmpeg-devel@ffmpeg.org Date: Wed, 1 May 2024 20:38:06 +0200 Message-Id: <20240501183809.1060-7-ovchinnikov.dmitrii@gmail.com> X-Mailer: git-send-email 2.38.1.windows.1 In-Reply-To: <20240501183809.1060-1-ovchinnikov.dmitrii@gmail.com> References: <20240501183809.1060-1-ovchinnikov.dmitrii@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 07/10, v2] avcodec/amfenc: add smart access video option X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Evgeny Pavlov Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 9kciAxxojiGF From: Evgeny Pavlov This commit adds option for enabling SmartAccess Video (SAV) in AMF encoders. SAV is an AMD hardware-specific feature which enables the parallelization of encode and decode streams across multiple Video Codec Engine (VCN) hardware instances. Signed-off-by: Evgeny Pavlov --- libavcodec/amfenc.h | 1 + libavcodec/amfenc_av1.c | 18 ++++++++++++++++++ libavcodec/amfenc_h264.c | 18 ++++++++++++++++++ libavcodec/amfenc_hevc.c | 18 ++++++++++++++++++ 4 files changed, 55 insertions(+) diff --git a/libavcodec/amfenc.h b/libavcodec/amfenc.h index 62736ef579..1bda0136bd 100644 --- a/libavcodec/amfenc.h +++ b/libavcodec/amfenc.h @@ -90,6 +90,7 @@ typedef struct AmfContext { int quality; int b_frame_delta_qp; int ref_b_frame_delta_qp; + int smart_access_video; // Dynamic options, can be set after Init() call diff --git a/libavcodec/amfenc_av1.c b/libavcodec/amfenc_av1.c index cc48e93fcb..7d37a242fc 100644 --- a/libavcodec/amfenc_av1.c +++ b/libavcodec/amfenc_av1.c @@ -104,6 +104,8 @@ static const AVOption options[] = { { "log_to_dbg", "Enable AMF logging to debug output", OFFSET(log_to_dbg), AV_OPT_TYPE_BOOL,{.i64 = 0 }, 0, 1, VE }, + { "smart_access_video", "Enable Smart Access Video", OFFSET(smart_access_video), AV_OPT_TYPE_BOOL, {.i64 = -1 }, -1, 1, VE}, + //Pre Analysis options { "preanalysis", "Enable preanalysis", OFFSET(preanalysis), AV_OPT_TYPE_BOOL, {.i64 = -1 }, -1, 1, VE }, @@ -265,6 +267,22 @@ FF_ENABLE_DEPRECATION_WARNINGS } } + if (ctx->smart_access_video != -1) { + AMF_ASSIGN_PROPERTY_BOOL(res, ctx->encoder, AMF_VIDEO_ENCODER_AV1_ENABLE_SMART_ACCESS_VIDEO, ctx->smart_access_video != 0); + if (res != AMF_OK) { + av_log(avctx, AV_LOG_ERROR, "The Smart Access Video is not supported by AMF.\n"); + if (ctx->smart_access_video != 0) + return AVERROR(ENOSYS); + } else { + av_log(avctx, AV_LOG_INFO, "The Smart Access Video (%d) is set.\n", ctx->smart_access_video); + // Set low latency mode if Smart Access Video is enabled + if (ctx->smart_access_video != 0) { + AMF_ASSIGN_PROPERTY_BOOL(res, ctx->encoder, AMF_VIDEO_ENCODER_AV1_ENCODING_LATENCY_MODE, AMF_VIDEO_ENCODER_AV1_ENCODING_LATENCY_MODE_LOWEST_LATENCY); + av_log(avctx, AV_LOG_INFO, "The Smart Access Video set low latency mode.\n"); + } + } + } + // Pre-Pass, Pre-Analysis, Two-Pass if (ctx->rate_control_mode == AMF_VIDEO_ENCODER_AV1_RATE_CONTROL_METHOD_CONSTANT_QP) { AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_AV1_RATE_CONTROL_PREENCODE, 0); diff --git a/libavcodec/amfenc_h264.c b/libavcodec/amfenc_h264.c index ad5fcc9ecb..a26a6dbef8 100644 --- a/libavcodec/amfenc_h264.c +++ b/libavcodec/amfenc_h264.c @@ -136,6 +136,8 @@ static const AVOption options[] = { { "log_to_dbg", "Enable AMF logging to debug output", OFFSET(log_to_dbg) , AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, VE }, + { "smart_access_video", "Enable Smart Access Video", OFFSET(smart_access_video), AV_OPT_TYPE_BOOL, {.i64 = -1 }, -1, 1, VE}, + //Pre Analysis options { "preanalysis", "Enable preanalysis", OFFSET(preanalysis), AV_OPT_TYPE_BOOL, {.i64 = -1 }, -1, 1, VE }, @@ -369,6 +371,22 @@ FF_ENABLE_DEPRECATION_WARNINGS av_log(ctx, AV_LOG_WARNING, "rate control mode is PEAK_CONSTRAINED_VBR but rc_max_rate is not set\n"); } + if (ctx->smart_access_video != -1) { + AMF_ASSIGN_PROPERTY_BOOL(res, ctx->encoder, AMF_VIDEO_ENCODER_ENABLE_SMART_ACCESS_VIDEO, ctx->smart_access_video != 0); + if (res != AMF_OK) { + av_log(avctx, AV_LOG_ERROR, "The Smart Access Video is not supported by AMF.\n"); + if (ctx->smart_access_video != 0) + return AVERROR(ENOSYS); + } else { + av_log(avctx, AV_LOG_INFO, "The Smart Access Video (%d) is set.\n", ctx->smart_access_video); + // Set low latency mode if Smart Access Video is enabled + if (ctx->smart_access_video != 0) { + AMF_ASSIGN_PROPERTY_BOOL(res, ctx->encoder, AMF_VIDEO_ENCODER_LOWLATENCY_MODE, true); + av_log(avctx, AV_LOG_INFO, "The Smart Access Video set low latency mode.\n"); + } + } + } + if (ctx->preanalysis != -1) { AMF_ASSIGN_PROPERTY_BOOL(res, ctx->encoder, AMF_VIDEO_ENCODER_PRE_ANALYSIS_ENABLE, !!((ctx->preanalysis == 0) ? false : true)); } diff --git a/libavcodec/amfenc_hevc.c b/libavcodec/amfenc_hevc.c index a89a3cf20c..8c26956513 100644 --- a/libavcodec/amfenc_hevc.c +++ b/libavcodec/amfenc_hevc.c @@ -100,6 +100,8 @@ static const AVOption options[] = { { "log_to_dbg", "Enable AMF logging to debug output", OFFSET(log_to_dbg), AV_OPT_TYPE_BOOL,{ .i64 = 0 }, 0, 1, VE }, + { "smart_access_video", "Enable Smart Access Video", OFFSET(smart_access_video), AV_OPT_TYPE_BOOL, {.i64 = -1 }, -1, 1, VE}, + //Pre Analysis options { "preanalysis", "Enable preanalysis", OFFSET(preanalysis), AV_OPT_TYPE_BOOL, {.i64 = -1 }, -1, 1, VE }, @@ -265,6 +267,22 @@ FF_ENABLE_DEPRECATION_WARNINGS } } + if (ctx->smart_access_video != -1) { + AMF_ASSIGN_PROPERTY_BOOL(res, ctx->encoder, AMF_VIDEO_ENCODER_HEVC_ENABLE_SMART_ACCESS_VIDEO, ctx->smart_access_video != 0); + if (res != AMF_OK) { + av_log(avctx, AV_LOG_ERROR, "The Smart Access Video is not supported by AMF.\n"); + if (ctx->smart_access_video != 0) + return AVERROR(ENOSYS); + } else { + av_log(avctx, AV_LOG_INFO, "The Smart Access Video (%d) is set.\n", ctx->smart_access_video); + // Set low latency mode if Smart Access Video is enabled + if (ctx->smart_access_video != 0) { + AMF_ASSIGN_PROPERTY_BOOL(res, ctx->encoder, AMF_VIDEO_ENCODER_HEVC_LOWLATENCY_MODE, true); + av_log(avctx, AV_LOG_INFO, "The Smart Access Video set low latency mode.\n"); + } + } + } + // Pre-Pass, Pre-Analysis, Two-Pass if (ctx->rate_control_mode == AMF_VIDEO_ENCODER_HEVC_RATE_CONTROL_METHOD_CONSTANT_QP) { AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_HEVC_PREENCODE_ENABLE, 0); From patchwork Wed May 1 18:38:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitrii Ovchinnikov X-Patchwork-Id: 48445 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:1203:b0:1a9:af23:56c1 with SMTP id v3csp1104381pzf; Wed, 1 May 2024 11:39:41 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCWtcBdPbnFNfwMYJFgPIj7asDzPMlKDPJSoD8E4Qh5gHZNPH5G8ixR09zVlKqUXV2LjRBKgYPj6GpoeZNzXSfUGGjdaoh97ZSsOnA== X-Google-Smtp-Source: AGHT+IFfoRuLjhI0O24wFYL8E+V5YG3k5mrUBXScQ+5bCYWMIkgpsS90kbtCrpCn6pj5nEEpfdd4 X-Received: by 2002:a17:907:26cb:b0:a58:deb8:8514 with SMTP id bp11-20020a17090726cb00b00a58deb88514mr2524487ejc.1.1714588780525; Wed, 01 May 2024 11:39:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1714588780; cv=none; d=google.com; s=arc-20160816; b=o5VNRg2UGlEIuK103AUEp2c6e5AUVuEip2+XYQoUoZ1D/MvWsZgE/zQ5HC9b/Kpckz hofJEbL/7bXtbewewBwVWsM3au2IP1mfwQGkYRLnbNL/LHXY8HdGo8Ia/HIv72bk+vwH CsX7RTuWeGzOltyyhCf8MbrxfcTxBM6aHu5jvl3a2qCgywWh0/K/lx955tsludsz0tPD SjY1HNlsutwY/rDOBDcj4tFNiAGXuwklwFjdLkUR1K9hZ3QZVwPsACj7VkSZ8nS6HCWY TrxWYbziN9EQ9YzSEIJsaE7Tg+wJuCAemgOJ5Ef9H1/2EQg/iZJkUHbdAU3wLwdwZ7JG eNqQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=UudYIWFjetCoaMdO43L5jgYtL6wgZgBPFazdjjGBZNw=; fh=GFMLWjWsOMgBmoPKQLjFbTnXINkN4pkSjsZqwDcYLuI=; b=C4DnVNv2RjPvqWUdi3IOtzW+V9N1MMgXEZNLEBANlCw9gFHpfDEkeH2D0V5tsX9qZJ QSulgfZ8o4cbbsV6gnYjT16KSmXRZgYmMsCZyXjQO5abqyhdrgEFyzUKhn48PdlPHYWK i4bgNUCpek0lSc41Qg8/TQOAb563YAXgB3FAORA/J9MHUt6iKImod34oDP/WSZL/6+mt KojfK5xsN28TDuWfS92LFWdTBQzplIHDx+H+LPa6AgxxvuMJGXBo6jyidVhB/I1p/yww rYmqXUFlMM30L1vkpxSiFnk3v/B8w1g3j+jsN07hsy+8fHeFsHgX/hKrq8x2Jni0ij9F VycA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=iTwMJiSg; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id lt7-20020a170906fa8700b00a52363224c7si15958305ejb.811.2024.05.01.11.39.40; Wed, 01 May 2024 11:39:40 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=iTwMJiSg; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5D77E68D725; Wed, 1 May 2024 21:38:35 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f52.google.com (mail-lf1-f52.google.com [209.85.167.52]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E6CF268D70D for ; Wed, 1 May 2024 21:38:23 +0300 (EEST) Received: by mail-lf1-f52.google.com with SMTP id 2adb3069b0e04-51b09c3a111so9907478e87.1 for ; Wed, 01 May 2024 11:38:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1714588702; x=1715193502; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0fWVFkNRr3EVb7MeflnJziTaDXv2yz3oerls4tOm9xY=; b=iTwMJiSg8QQ0P+LfAQOL6mH+gcX+xHYe/axp3O8+CASGy/s2KoOpxiHNPk/Ve2v8kA udRLhwZBS47W8JCxdDBIRjGJ37GZrcHaFlatbqn6JWjaiXB0PgLUTH9qhBWQYxnlG54+ jP52dButA6Pmcpm5XAVbpc9YIwq0ZvMj8ogP7fX82YQPLxgNUXPgBBKkejQwIoMLqaYp cpLuE9EoxWiVaN2R9pOUW2Nt3BHnlB2IOxVbCRasKZqLUmDCkOosOREbScyU6KtTdesq 88AsdOHjHMnTubSkphGVCAQAllXXUOgW/gY6+KeRLaieoQDL2PLZFAO3xt+xPKPjHfkn qXfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714588702; x=1715193502; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0fWVFkNRr3EVb7MeflnJziTaDXv2yz3oerls4tOm9xY=; b=goP5PEfNtrMU3EFcC5oGevj6caybf81XkCcc2hcSsehWPgI7wjEQJFXVWo4HKMIbZe lqeFaJC/l88oUr6W+V4blDFNhwUlpf2HsEhNEqMJzmtRi/qqMarzXOe7LzjvuWpG8xjD xp9kXqK4jBPt3uiZB28Ri+1WGKBOjkAsSmVHe3oZoFtlfD687TpiXD+rYUBq8Go/8xvt 9bah87LgWSrmo8EPDNVEDP9KD7F5Go03m50lJVYEIZDLHHS42jH26EYYHUu0AmirzZK/ cfppsNUFlWugq1aUrEaDHfNO/N8W4pM/IsQ/rkmpNjYgkw4vBoYsFIK6+iTkXYPDwPRg cPLQ== X-Gm-Message-State: AOJu0YzIjbDuIlQ81KTEyDUoanZfFfpGDZGzReYpmVGwU5dqBES+oPym +D/I8SUoekoRTqUqi9GcPjrwFL2BVnKVlD9UUFzaRz4iV5zw/S6//Je6vVPk X-Received: by 2002:a05:6512:3f21:b0:51d:8159:598 with SMTP id y33-20020a0565123f2100b0051d81590598mr1968714lfa.19.1714588702128; Wed, 01 May 2024 11:38:22 -0700 (PDT) Received: from dovchinn.amd.com (cable-178-148-16-149.dynamic.sbb.rs. [178.148.16.149]) by smtp.gmail.com with ESMTPSA id kh20-20020a170906f81400b00a58bf434876sm7367337ejb.58.2024.05.01.11.38.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 May 2024 11:38:21 -0700 (PDT) From: Dmitrii Ovchinnikov To: ffmpeg-devel@ffmpeg.org Date: Wed, 1 May 2024 20:38:07 +0200 Message-Id: <20240501183809.1060-8-ovchinnikov.dmitrii@gmail.com> X-Mailer: git-send-email 2.38.1.windows.1 In-Reply-To: <20240501183809.1060-1-ovchinnikov.dmitrii@gmail.com> References: <20240501183809.1060-1-ovchinnikov.dmitrii@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 08/10, v2] avcodec/amfenc: redesign to use hwcontext_amf. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Evgeny Pavlov , Dmitrii Ovchinnikov Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 59kzs31PseUV Co-authored-by: Evgeny Pavlov --- libavcodec/amfenc.c | 568 +++++++++++++------------------------------- libavcodec/amfenc.h | 23 +- 2 files changed, 163 insertions(+), 428 deletions(-) diff --git a/libavcodec/amfenc.c b/libavcodec/amfenc.c index 510050e282..781254327a 100644 --- a/libavcodec/amfenc.c +++ b/libavcodec/amfenc.c @@ -29,6 +29,8 @@ #define COBJMACROS #include "libavutil/hwcontext_dxva2.h" #endif +#include "libavutil/hwcontext_amf.h" +#include "libavutil/hwcontext_amf_internal.h" #include "libavutil/mem.h" #include "libavutil/pixdesc.h" #include "libavutil/time.h" @@ -38,6 +40,18 @@ #include "internal.h" #include "libavutil/mastering_display_metadata.h" +#if CONFIG_D3D11VA +#include +#endif + +#ifdef _WIN32 +#include "compat/w32dlfcn.h" +#else +#include +#endif + +#define PTS_PROP L"PtsProp" + static int amf_save_hdr_metadata(AVCodecContext *avctx, const AVFrame *frame, AMFHDRMetadata *hdrmeta) { AVFrameSideData *sd_display; @@ -88,20 +102,6 @@ static int amf_save_hdr_metadata(AVCodecContext *avctx, const AVFrame *frame, AM return 1; } -#if CONFIG_D3D11VA -#include -#endif - -#ifdef _WIN32 -#include "compat/w32dlfcn.h" -#else -#include -#endif - -#define FFMPEG_AMF_WRITER_ID L"ffmpeg_amf" - -#define PTS_PROP L"PtsProp" - const enum AVPixelFormat ff_amf_pix_fmts[] = { AV_PIX_FMT_NV12, AV_PIX_FMT_YUV420P, @@ -111,289 +111,18 @@ const enum AVPixelFormat ff_amf_pix_fmts[] = { #if CONFIG_DXVA2 AV_PIX_FMT_DXVA2_VLD, #endif + AV_PIX_FMT_AMF_SURFACE, AV_PIX_FMT_P010, AV_PIX_FMT_NONE }; -typedef struct FormatMap { - enum AVPixelFormat av_format; - enum AMF_SURFACE_FORMAT amf_format; -} FormatMap; - -static const FormatMap format_map[] = -{ - { AV_PIX_FMT_NONE, AMF_SURFACE_UNKNOWN }, - { AV_PIX_FMT_NV12, AMF_SURFACE_NV12 }, - { AV_PIX_FMT_P010, AMF_SURFACE_P010 }, - { AV_PIX_FMT_BGR0, AMF_SURFACE_BGRA }, - { AV_PIX_FMT_RGB0, AMF_SURFACE_RGBA }, - { AV_PIX_FMT_GRAY8, AMF_SURFACE_GRAY8 }, - { AV_PIX_FMT_YUV420P, AMF_SURFACE_YUV420P }, - { AV_PIX_FMT_YUYV422, AMF_SURFACE_YUY2 }, -}; - -static enum AMF_SURFACE_FORMAT amf_av_to_amf_format(enum AVPixelFormat fmt) -{ - int i; - for (i = 0; i < amf_countof(format_map); i++) { - if (format_map[i].av_format == fmt) { - return format_map[i].amf_format; - } - } - return AMF_SURFACE_UNKNOWN; -} - -static void AMF_CDECL_CALL AMFTraceWriter_Write(AMFTraceWriter *pThis, - const wchar_t *scope, const wchar_t *message) -{ - AmfTraceWriter *tracer = (AmfTraceWriter*)pThis; - av_log(tracer->avctx, AV_LOG_DEBUG, "%ls: %ls", scope, message); // \n is provided from AMF -} - -static void AMF_CDECL_CALL AMFTraceWriter_Flush(AMFTraceWriter *pThis) -{ -} - -static AMFTraceWriterVtbl tracer_vtbl = -{ - .Write = AMFTraceWriter_Write, - .Flush = AMFTraceWriter_Flush, -}; - -static int amf_load_library(AVCodecContext *avctx) -{ - AmfContext *ctx = avctx->priv_data; - AMFInit_Fn init_fun; - AMFQueryVersion_Fn version_fun; - AMF_RESULT res; - - ctx->delayed_frame = av_frame_alloc(); - if (!ctx->delayed_frame) { - return AVERROR(ENOMEM); - } - // hardcoded to current HW queue size - will auto-realloc if too small - ctx->timestamp_list = av_fifo_alloc2(avctx->max_b_frames + 16, sizeof(int64_t), - AV_FIFO_FLAG_AUTO_GROW); - if (!ctx->timestamp_list) { - return AVERROR(ENOMEM); - } - ctx->dts_delay = 0; - - - ctx->library = dlopen(AMF_DLL_NAMEA, RTLD_NOW | RTLD_LOCAL); - AMF_RETURN_IF_FALSE(ctx, ctx->library != NULL, - AVERROR_UNKNOWN, "DLL %s failed to open\n", AMF_DLL_NAMEA); - - init_fun = (AMFInit_Fn)dlsym(ctx->library, AMF_INIT_FUNCTION_NAME); - AMF_RETURN_IF_FALSE(ctx, init_fun != NULL, AVERROR_UNKNOWN, "DLL %s failed to find function %s\n", AMF_DLL_NAMEA, AMF_INIT_FUNCTION_NAME); - - version_fun = (AMFQueryVersion_Fn)dlsym(ctx->library, AMF_QUERY_VERSION_FUNCTION_NAME); - AMF_RETURN_IF_FALSE(ctx, version_fun != NULL, AVERROR_UNKNOWN, "DLL %s failed to find function %s\n", AMF_DLL_NAMEA, AMF_QUERY_VERSION_FUNCTION_NAME); - - res = version_fun(&ctx->version); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "%s failed with error %d\n", AMF_QUERY_VERSION_FUNCTION_NAME, res); - res = init_fun(AMF_FULL_VERSION, &ctx->factory); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "%s failed with error %d\n", AMF_INIT_FUNCTION_NAME, res); - res = ctx->factory->pVtbl->GetTrace(ctx->factory, &ctx->trace); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "GetTrace() failed with error %d\n", res); - res = ctx->factory->pVtbl->GetDebug(ctx->factory, &ctx->debug); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "GetDebug() failed with error %d\n", res); - return 0; -} - -#if CONFIG_D3D11VA -static int amf_init_from_d3d11_device(AVCodecContext *avctx, AVD3D11VADeviceContext *hwctx) -{ - AmfContext *ctx = avctx->priv_data; - AMF_RESULT res; - - res = ctx->context->pVtbl->InitDX11(ctx->context, hwctx->device, AMF_DX11_1); - if (res != AMF_OK) { - if (res == AMF_NOT_SUPPORTED) - av_log(avctx, AV_LOG_ERROR, "AMF via D3D11 is not supported on the given device.\n"); - else - av_log(avctx, AV_LOG_ERROR, "AMF failed to initialise on the given D3D11 device: %d.\n", res); - return AVERROR(ENODEV); - } - - return 0; -} -#endif - -#if CONFIG_DXVA2 -static int amf_init_from_dxva2_device(AVCodecContext *avctx, AVDXVA2DeviceContext *hwctx) -{ - AmfContext *ctx = avctx->priv_data; - HANDLE device_handle; - IDirect3DDevice9 *device; - HRESULT hr; - AMF_RESULT res; - int ret; - - hr = IDirect3DDeviceManager9_OpenDeviceHandle(hwctx->devmgr, &device_handle); - if (FAILED(hr)) { - av_log(avctx, AV_LOG_ERROR, "Failed to open device handle for Direct3D9 device: %lx.\n", (unsigned long)hr); - return AVERROR_EXTERNAL; - } - - hr = IDirect3DDeviceManager9_LockDevice(hwctx->devmgr, device_handle, &device, FALSE); - if (SUCCEEDED(hr)) { - IDirect3DDeviceManager9_UnlockDevice(hwctx->devmgr, device_handle, FALSE); - ret = 0; - } else { - av_log(avctx, AV_LOG_ERROR, "Failed to lock device handle for Direct3D9 device: %lx.\n", (unsigned long)hr); - ret = AVERROR_EXTERNAL; - } - - IDirect3DDeviceManager9_CloseDeviceHandle(hwctx->devmgr, device_handle); - - if (ret < 0) - return ret; - - res = ctx->context->pVtbl->InitDX9(ctx->context, device); - - IDirect3DDevice9_Release(device); - - if (res != AMF_OK) { - if (res == AMF_NOT_SUPPORTED) - av_log(avctx, AV_LOG_ERROR, "AMF via D3D9 is not supported on the given device.\n"); - else - av_log(avctx, AV_LOG_ERROR, "AMF failed to initialise on given D3D9 device: %d.\n", res); - return AVERROR(ENODEV); - } - - return 0; -} -#endif - -static int amf_init_context(AVCodecContext *avctx) -{ - AmfContext *ctx = avctx->priv_data; - AMFContext1 *context1 = NULL; - AMF_RESULT res; - av_unused int ret; - - ctx->hwsurfaces_in_queue = 0; - ctx->hwsurfaces_in_queue_max = 16; - - // configure AMF logger - // the return of these functions indicates old state and do not affect behaviour - ctx->trace->pVtbl->EnableWriter(ctx->trace, AMF_TRACE_WRITER_DEBUG_OUTPUT, ctx->log_to_dbg != 0 ); - if (ctx->log_to_dbg) - ctx->trace->pVtbl->SetWriterLevel(ctx->trace, AMF_TRACE_WRITER_DEBUG_OUTPUT, AMF_TRACE_TRACE); - ctx->trace->pVtbl->EnableWriter(ctx->trace, AMF_TRACE_WRITER_CONSOLE, 0); - ctx->trace->pVtbl->SetGlobalLevel(ctx->trace, AMF_TRACE_TRACE); - - // connect AMF logger to av_log - ctx->tracer.vtbl = &tracer_vtbl; - ctx->tracer.avctx = avctx; - ctx->trace->pVtbl->RegisterWriter(ctx->trace, FFMPEG_AMF_WRITER_ID,(AMFTraceWriter*)&ctx->tracer, 1); - ctx->trace->pVtbl->SetWriterLevel(ctx->trace, FFMPEG_AMF_WRITER_ID, AMF_TRACE_TRACE); - - res = ctx->factory->pVtbl->CreateContext(ctx->factory, &ctx->context); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "CreateContext() failed with error %d\n", res); - - // If a device was passed to the encoder, try to initialise from that. - if (avctx->hw_frames_ctx) { - AVHWFramesContext *frames_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; - - if (amf_av_to_amf_format(frames_ctx->sw_format) == AMF_SURFACE_UNKNOWN) { - av_log(avctx, AV_LOG_ERROR, "Format of input frames context (%s) is not supported by AMF.\n", - av_get_pix_fmt_name(frames_ctx->sw_format)); - return AVERROR(EINVAL); - } - - switch (frames_ctx->device_ctx->type) { -#if CONFIG_D3D11VA - case AV_HWDEVICE_TYPE_D3D11VA: - ret = amf_init_from_d3d11_device(avctx, frames_ctx->device_ctx->hwctx); - if (ret < 0) - return ret; - break; -#endif -#if CONFIG_DXVA2 - case AV_HWDEVICE_TYPE_DXVA2: - ret = amf_init_from_dxva2_device(avctx, frames_ctx->device_ctx->hwctx); - if (ret < 0) - return ret; - break; -#endif - default: - av_log(avctx, AV_LOG_ERROR, "AMF initialisation from a %s frames context is not supported.\n", - av_hwdevice_get_type_name(frames_ctx->device_ctx->type)); - return AVERROR(ENOSYS); - } - - ctx->hw_frames_ctx = av_buffer_ref(avctx->hw_frames_ctx); - if (!ctx->hw_frames_ctx) - return AVERROR(ENOMEM); - - if (frames_ctx->initial_pool_size > 0) - ctx->hwsurfaces_in_queue_max = frames_ctx->initial_pool_size - 1; - - } else if (avctx->hw_device_ctx) { - AVHWDeviceContext *device_ctx = (AVHWDeviceContext*)avctx->hw_device_ctx->data; - - switch (device_ctx->type) { -#if CONFIG_D3D11VA - case AV_HWDEVICE_TYPE_D3D11VA: - ret = amf_init_from_d3d11_device(avctx, device_ctx->hwctx); - if (ret < 0) - return ret; - break; -#endif -#if CONFIG_DXVA2 - case AV_HWDEVICE_TYPE_DXVA2: - ret = amf_init_from_dxva2_device(avctx, device_ctx->hwctx); - if (ret < 0) - return ret; - break; -#endif - default: - av_log(avctx, AV_LOG_ERROR, "AMF initialisation from a %s device is not supported.\n", - av_hwdevice_get_type_name(device_ctx->type)); - return AVERROR(ENOSYS); - } - - ctx->hw_device_ctx = av_buffer_ref(avctx->hw_device_ctx); - if (!ctx->hw_device_ctx) - return AVERROR(ENOMEM); - - } else { - res = ctx->context->pVtbl->InitDX11(ctx->context, NULL, AMF_DX11_1); - if (res == AMF_OK) { - av_log(avctx, AV_LOG_VERBOSE, "AMF initialisation succeeded via D3D11.\n"); - } else { - res = ctx->context->pVtbl->InitDX9(ctx->context, NULL); - if (res == AMF_OK) { - av_log(avctx, AV_LOG_VERBOSE, "AMF initialisation succeeded via D3D9.\n"); - } else { - AMFGuid guid = IID_AMFContext1(); - res = ctx->context->pVtbl->QueryInterface(ctx->context, &guid, (void**)&context1); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "CreateContext1() failed with error %d\n", res); - - res = context1->pVtbl->InitVulkan(context1, NULL); - context1->pVtbl->Release(context1); - if (res != AMF_OK) { - if (res == AMF_NOT_SUPPORTED) - av_log(avctx, AV_LOG_ERROR, "AMF via Vulkan is not supported on the given device.\n"); - else - av_log(avctx, AV_LOG_ERROR, "AMF failed to initialise on the given Vulkan device: %d.\n", res); - return AVERROR(ENOSYS); - } - av_log(avctx, AV_LOG_VERBOSE, "AMF initialisation succeeded via Vulkan.\n"); - } - } - } - return 0; -} - static int amf_init_encoder(AVCodecContext *avctx) { AmfContext *ctx = avctx->priv_data; const wchar_t *codec_id = NULL; AMF_RESULT res; enum AVPixelFormat pix_fmt; + AVAMFDeviceContextInternal* internal = (AVAMFDeviceContextInternal *)ctx->amf_device_ctx_internal->data; switch (avctx->codec->id) { case AV_CODEC_ID_H264: @@ -415,13 +144,16 @@ static int amf_init_encoder(AVCodecContext *avctx) else pix_fmt = avctx->pix_fmt; - ctx->format = amf_av_to_amf_format(pix_fmt); + if (avctx->pix_fmt != AV_PIX_FMT_AMF_SURFACE) + ctx->format = av_amf_av_to_amf_format(pix_fmt); + else + ctx->format = av_amf_av_to_amf_format(avctx->sw_pix_fmt); + AMF_RETURN_IF_FALSE(ctx, ctx->format != AMF_SURFACE_UNKNOWN, AVERROR(EINVAL), - "Format %s is not supported\n", av_get_pix_fmt_name(pix_fmt)); + "Format %s is not supported\n", av_get_pix_fmt_name(pix_fmt)); - res = ctx->factory->pVtbl->CreateComponent(ctx->factory, ctx->context, codec_id, &ctx->encoder); + res = internal->factory->pVtbl->CreateComponent(internal->factory, internal->context, codec_id, &ctx->encoder); AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_ENCODER_NOT_FOUND, "CreateComponent(%ls) failed with error %d\n", codec_id, res); - return 0; } @@ -429,49 +161,61 @@ int av_cold ff_amf_encode_close(AVCodecContext *avctx) { AmfContext *ctx = avctx->priv_data; - if (ctx->delayed_surface) { - ctx->delayed_surface->pVtbl->Release(ctx->delayed_surface); - ctx->delayed_surface = NULL; - } - if (ctx->encoder) { ctx->encoder->pVtbl->Terminate(ctx->encoder); ctx->encoder->pVtbl->Release(ctx->encoder); ctx->encoder = NULL; } - if (ctx->context) { - ctx->context->pVtbl->Terminate(ctx->context); - ctx->context->pVtbl->Release(ctx->context); - ctx->context = NULL; - } av_buffer_unref(&ctx->hw_device_ctx); av_buffer_unref(&ctx->hw_frames_ctx); + av_buffer_unref(&ctx->amf_device_ctx_internal); + av_fifo_freep2(&ctx->timestamp_list); + + return 0; +} - if (ctx->trace) { - ctx->trace->pVtbl->UnregisterWriter(ctx->trace, FFMPEG_AMF_WRITER_ID); +static int amf_init_encoder_context(AVCodecContext *avctx) +{ + AmfContext *ctx = avctx->priv_data; + AMFContext1 *context1 = NULL; + int ret; + + if (avctx->hw_frames_ctx) { + AVHWFramesContext *frames_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; + ret = av_amf_context_derive((AVAMFDeviceContextInternal *)ctx->amf_device_ctx_internal->data, frames_ctx->device_ctx, NULL, 0); + if (ret < 0) + return ret; + ctx->hw_frames_ctx = av_buffer_ref(avctx->hw_frames_ctx); + if (!ctx->hw_frames_ctx) + return AVERROR(ENOMEM); } - if (ctx->library) { - dlclose(ctx->library); - ctx->library = NULL; + else if (avctx->hw_device_ctx) { + AVHWDeviceContext *device_ctx = (AVHWDeviceContext*)avctx->hw_device_ctx->data; + ret = av_amf_context_derive((AVAMFDeviceContextInternal *)ctx->amf_device_ctx_internal->data, device_ctx, NULL, 0); + if (ret < 0) + return ret; + ctx->hw_device_ctx = av_buffer_ref(avctx->hw_device_ctx); + if (!ctx->hw_device_ctx) + return AVERROR(ENOMEM); + + } else { + ret = av_amf_context_init((AVAMFDeviceContextInternal *)ctx->amf_device_ctx_internal->data, avctx); + if (ret != 0) { + return ret; + } } - ctx->trace = NULL; - ctx->debug = NULL; - ctx->factory = NULL; - ctx->version = 0; - ctx->delayed_drain = 0; - av_frame_free(&ctx->delayed_frame); - av_fifo_freep2(&ctx->timestamp_list); - return 0; + + return ret; } static int amf_copy_surface(AVCodecContext *avctx, const AVFrame *frame, AMFSurface* surface) { AMFPlane *plane; - uint8_t *dst_data[4]; - int dst_linesize[4]; + uint8_t *dst_data[4] = {0}; + int dst_linesize[4] = {0}; int planes; int i; @@ -555,17 +299,56 @@ int ff_amf_encode_init(AVCodecContext *avctx) { int ret; - if ((ret = amf_load_library(avctx)) == 0) { - if ((ret = amf_init_context(avctx)) == 0) { - if ((ret = amf_init_encoder(avctx)) == 0) { - if (avctx->pix_fmt == AV_PIX_FMT_P010) { - AmfContext *ctx = avctx->priv_data; - AMF_RETURN_IF_FALSE(ctx, ctx->version >= AMF_MAKE_FULL_VERSION(1, 4, 32, 0), AVERROR_UNKNOWN, "10-bit encoder is not supported by AMD GPU drivers versions lower than 23.30.\n"); - } - return 0; - } + AmfContext *ctx = avctx->priv_data; + AVHWDeviceContext *hwdev_ctx = NULL; + if (avctx->hw_device_ctx) { + hwdev_ctx = (AVHWDeviceContext*)avctx->hw_device_ctx->data; + } else if (avctx->hw_frames_ctx) { + AVHWFramesContext *frames_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; + hwdev_ctx = (AVHWDeviceContext*)frames_ctx->device_ctx; + } + if (av_amf_trace_writer.avctx == NULL) + av_amf_trace_writer.avctx = avctx; + // hardcoded to current HW queue size - will auto-realloc if too small + ctx->timestamp_list = av_fifo_alloc2(avctx->max_b_frames + 16, sizeof(int64_t), + AV_FIFO_FLAG_AUTO_GROW); + if (!ctx->timestamp_list) { + return AVERROR(ENOMEM); + } + ctx->dts_delay = 0; + + ctx->hwsurfaces_in_queue = 0; + ctx->hwsurfaces_in_queue_max = 16; + + if (avctx->hw_frames_ctx && hwdev_ctx && hwdev_ctx->type == AV_HWDEVICE_TYPE_AMF) { + AVHWFramesContext *frames_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; + AVAMFDeviceContext * amf_ctx = hwdev_ctx->hwctx; + ctx->amf_device_ctx_internal = av_buffer_ref(amf_ctx->internal); + } + else if (avctx->hw_device_ctx && hwdev_ctx && hwdev_ctx->type == AV_HWDEVICE_TYPE_AMF) { + AVAMFDeviceContext * amf_ctx = hwdev_ctx->hwctx; + ctx->amf_device_ctx_internal = av_buffer_ref(amf_ctx->internal); + } else { + AVAMFDeviceContextInternal *wrapped = av_mallocz(sizeof(*wrapped)); + ctx->amf_device_ctx_internal = av_buffer_create((uint8_t *)wrapped, sizeof(*wrapped), + av_amf_context_internal_free, NULL, 0); + if ((ret = av_amf_context_internal_create((AVAMFDeviceContextInternal *)ctx->amf_device_ctx_internal->data, avctx, "", NULL, 0)) != 0) { + ff_amf_encode_close(avctx); + return ret; + } + if ((ret = amf_init_encoder_context(avctx)) != 0) { + ff_amf_encode_close(avctx); + return ret; + } + } + if ((ret = amf_init_encoder(avctx)) == 0) { + if (avctx->pix_fmt == AV_PIX_FMT_P010) { + AVAMFDeviceContextInternal * internal = (AVAMFDeviceContextInternal *)ctx->amf_device_ctx_internal->data; + AMF_RETURN_IF_FALSE(ctx, internal->version >= AMF_MAKE_FULL_VERSION(1, 4, 32, 0), AVERROR_UNKNOWN, "10-bit encoder is not supported by AMD GPU drivers versions lower than 23.30.\n"); } + return 0; } + ff_amf_encode_close(avctx); return ret; } @@ -643,30 +426,27 @@ static void amf_release_buffer_with_frame_ref(AMFBuffer *frame_ref_storage_buffe int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) { AmfContext *ctx = avctx->priv_data; + AVAMFDeviceContextInternal * internal = (AVAMFDeviceContextInternal *)ctx->amf_device_ctx_internal->data; AMFSurface *surface; AMF_RESULT res; int ret; AMF_RESULT res_query; AMFData *data = NULL; - AVFrame *frame = ctx->delayed_frame; + AVFrame *frame = av_frame_alloc(); int block_and_wait; int query_output_data_flag = 0; - AMF_RESULT res_resubmit; + int count = 0; if (!ctx->encoder) return AVERROR(EINVAL); - if (!frame->buf[0]) { - ret = ff_encode_get_frame(avctx, frame); - if (ret < 0 && ret != AVERROR_EOF) - return ret; - } + ret = ff_encode_get_frame(avctx, frame); + if (ret < 0 && ret != AVERROR_EOF) + return ret; if (!frame->buf[0]) { // submit drain if (!ctx->eof) { // submit drain one time only - if (ctx->delayed_surface != NULL) { - ctx->delayed_drain = 1; // input queue is full: resubmit Drain() in ff_amf_receive_packet - } else if(!ctx->delayed_drain) { + if(!ctx->delayed_drain) { res = ctx->encoder->pVtbl->Drain(ctx->encoder); if (res == AMF_INPUT_FULL) { ctx->delayed_drain = 1; // input queue is full: resubmit Drain() in ff_amf_receive_packet @@ -678,7 +458,7 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) } } } - } else if (!ctx->delayed_surface) { // submit frame + } else { // submit frame int hw_surface = 0; // prepare surface from frame @@ -695,7 +475,7 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) texture->lpVtbl->SetPrivateData(texture, &AMFTextureArrayIndexGUID, sizeof(index), &index); - res = ctx->context->pVtbl->CreateSurfaceFromDX11Native(ctx->context, texture, &surface, NULL); // wrap to AMF surface + res = internal->context->pVtbl->CreateSurfaceFromDX11Native(internal->context, texture, &surface, NULL); // wrap to AMF surface AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR(ENOMEM), "CreateSurfaceFromDX11Native() failed with error %d\n", res); hw_surface = 1; @@ -707,16 +487,23 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) { IDirect3DSurface9 *texture = (IDirect3DSurface9 *)frame->data[3]; // actual texture - res = ctx->context->pVtbl->CreateSurfaceFromDX9Native(ctx->context, texture, &surface, NULL); // wrap to AMF surface + res = internal->context->pVtbl->CreateSurfaceFromDX9Native(internal->context, texture, &surface, NULL); // wrap to AMF surface AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR(ENOMEM), "CreateSurfaceFromDX9Native() failed with error %d\n", res); hw_surface = 1; } break; #endif + case AV_PIX_FMT_AMF_SURFACE: + { + surface = (AMFSurface*)frame->data[0]; + surface->pVtbl->Acquire(surface); + hw_surface = 1; + } + break; default: { - res = ctx->context->pVtbl->AllocSurface(ctx->context, AMF_MEMORY_HOST, ctx->format, avctx->width, avctx->height, &surface); + res = internal->context->pVtbl->AllocSurface(internal->context, AMF_MEMORY_HOST, ctx->format, avctx->width, avctx->height, &surface); AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR(ENOMEM), "AllocSurface() failed with error %d\n", res); amf_copy_surface(avctx, frame, surface); } @@ -729,7 +516,7 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) // input HW surfaces can be vertically aligned by 16; tell AMF the real size surface->pVtbl->SetCrop(surface, 0, 0, frame->width, frame->height); - frame_ref_storage_buffer = amf_create_buffer_with_frame_ref(frame, ctx->context); + frame_ref_storage_buffer = amf_create_buffer_with_frame_ref(frame, internal->context); AMF_RETURN_IF_FALSE(ctx, frame_ref_storage_buffer != NULL, AVERROR(ENOMEM), "create_buffer_with_frame_ref() returned NULL\n"); res = amf_set_property_buffer(surface, L"av_frame_ref", frame_ref_storage_buffer); @@ -741,7 +528,7 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) // HDR10 metadata if (frame->color_trc == AVCOL_TRC_SMPTE2084) { AMFBuffer * hdrmeta_buffer = NULL; - res = ctx->context->pVtbl->AllocBuffer(ctx->context, AMF_MEMORY_HOST, sizeof(AMFHDRMetadata), &hdrmeta_buffer); + res = internal->context->pVtbl->AllocBuffer(internal->context, AMF_MEMORY_HOST, sizeof(AMFHDRMetadata), &hdrmeta_buffer); if (res == AMF_OK) { AMFHDRMetadata * hdrmeta = (AMFHDRMetadata*)hdrmeta_buffer->pVtbl->GetNative(hdrmeta_buffer); if (amf_save_hdr_metadata(avctx, frame, hdrmeta) == 0) { @@ -778,13 +565,11 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) // submit surface res = ctx->encoder->pVtbl->SubmitInput(ctx->encoder, (AMFData*)surface); if (res == AMF_INPUT_FULL) { // handle full queue - //store surface for later submission - ctx->delayed_surface = surface; + av_usleep(1000); // wait and poll again } else { int64_t pts = frame->pts; surface->pVtbl->Release(surface); AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "SubmitInput() failed with error %d\n", res); - av_frame_unref(frame); ret = av_fifo_write(ctx->timestamp_list, &pts, 1); if (ret < 0) @@ -796,73 +581,40 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) do { block_and_wait = 0; // poll data - if (!avpkt->data && !avpkt->buf) { - res_query = ctx->encoder->pVtbl->QueryOutput(ctx->encoder, &data); - if (data) { - // copy data to packet - AMFBuffer *buffer; - AMFGuid guid = IID_AMFBuffer(); - query_output_data_flag = 1; - data->pVtbl->QueryInterface(data, &guid, (void**)&buffer); // query for buffer interface - ret = amf_copy_buffer(avctx, avpkt, buffer); - - buffer->pVtbl->Release(buffer); - - if (data->pVtbl->HasProperty(data, L"av_frame_ref")) { - AMFBuffer* frame_ref_storage_buffer; - res = amf_get_property_buffer(data, L"av_frame_ref", &frame_ref_storage_buffer); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "GetProperty failed for \"av_frame_ref\" with error %d\n", res); - amf_release_buffer_with_frame_ref(frame_ref_storage_buffer); - ctx->hwsurfaces_in_queue--; - } + res_query = ctx->encoder->pVtbl->QueryOutput(ctx->encoder, &data); + if (data) { + // copy data to packet + AMFBuffer* buffer; + AMFGuid guid = IID_AMFBuffer(); + data->pVtbl->QueryInterface(data, &guid, (void**)&buffer); // query for buffer interface + ret = amf_copy_buffer(avctx, avpkt, buffer); + + buffer->pVtbl->Release(buffer); + + if (data->pVtbl->HasProperty(data, L"av_frame_ref")) { + AMFBuffer *frame_ref_storage_buffer; + res = amf_get_property_buffer(data, L"av_frame_ref", &frame_ref_storage_buffer); + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "GetProperty failed for \"av_frame_ref\" with error %d\n", res); + amf_release_buffer_with_frame_ref(frame_ref_storage_buffer); + ctx->hwsurfaces_in_queue--; + } + data->pVtbl->Release(data); - data->pVtbl->Release(data); + AMF_RETURN_IF_FALSE(ctx, ret >= 0, ret, "amf_copy_buffer() failed with error %d\n", ret); - AMF_RETURN_IF_FALSE(ctx, ret >= 0, ret, "amf_copy_buffer() failed with error %d\n", ret); - } - } - res_resubmit = AMF_OK; - if (ctx->delayed_surface != NULL) { // try to resubmit frame - if (ctx->delayed_surface->pVtbl->HasProperty(ctx->delayed_surface, L"av_frame_hdrmeta")) { - AMFBuffer * hdrmeta_buffer = NULL; - res = amf_get_property_buffer((AMFData *)ctx->delayed_surface, L"av_frame_hdrmeta", &hdrmeta_buffer); - AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "GetProperty failed for \"av_frame_hdrmeta\" with error %d\n", res); - switch (avctx->codec->id) { - case AV_CODEC_ID_H264: - AMF_ASSIGN_PROPERTY_INTERFACE(res, ctx->encoder, AMF_VIDEO_ENCODER_INPUT_HDR_METADATA, hdrmeta_buffer); break; - case AV_CODEC_ID_HEVC: - AMF_ASSIGN_PROPERTY_INTERFACE(res, ctx->encoder, AMF_VIDEO_ENCODER_HEVC_INPUT_HDR_METADATA, hdrmeta_buffer); break; + if (ctx->delayed_drain) { // try to resubmit drain + res = ctx->encoder->pVtbl->Drain(ctx->encoder); + if (res != AMF_INPUT_FULL) { + ctx->delayed_drain = 0; + ctx->eof = 1; // drain started + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "Repeated Drain() failed with error %d\n", res); + } else { + av_log(avctx, AV_LOG_WARNING, "Data acquired but delayed drain submission got AMF_INPUT_FULL- should not happen\n"); } - hdrmeta_buffer->pVtbl->Release(hdrmeta_buffer); - } - res_resubmit = ctx->encoder->pVtbl->SubmitInput(ctx->encoder, (AMFData*)ctx->delayed_surface); - if (res_resubmit != AMF_INPUT_FULL) { - int64_t pts = ctx->delayed_surface->pVtbl->GetPts(ctx->delayed_surface); - ctx->delayed_surface->pVtbl->Release(ctx->delayed_surface); - ctx->delayed_surface = NULL; - av_frame_unref(ctx->delayed_frame); - AMF_RETURN_IF_FALSE(ctx, res_resubmit == AMF_OK, AVERROR_UNKNOWN, "Repeated SubmitInput() failed with error %d\n", res_resubmit); - - ret = av_fifo_write(ctx->timestamp_list, &pts, 1); - if (ret < 0) - return ret; - } - } else if (ctx->delayed_drain) { // try to resubmit drain - res = ctx->encoder->pVtbl->Drain(ctx->encoder); - if (res != AMF_INPUT_FULL) { - ctx->delayed_drain = 0; - ctx->eof = 1; // drain started - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "Repeated Drain() failed with error %d\n", res); - } else { - av_log(avctx, AV_LOG_WARNING, "Data acquired but delayed drain submission got AMF_INPUT_FULL- should not happen\n"); - } - } - - if (query_output_data_flag == 0) { - if (res_resubmit == AMF_INPUT_FULL || ctx->delayed_drain || (ctx->eof && res_query != AMF_EOF) || (ctx->hwsurfaces_in_queue >= ctx->hwsurfaces_in_queue_max)) { - block_and_wait = 1; - av_usleep(1000); } + } else if (ctx->delayed_drain || (ctx->eof && res_query != AMF_EOF) || (ctx->hwsurfaces_in_queue >= ctx->hwsurfaces_in_queue_max)) { + block_and_wait = 1; + av_usleep(1000); // wait and poll again } } while (block_and_wait); @@ -920,5 +672,7 @@ const AVCodecHWConfigInternal *const ff_amfenc_hw_configs[] = { HW_CONFIG_ENCODER_FRAMES(DXVA2_VLD, DXVA2), HW_CONFIG_ENCODER_DEVICE(NONE, DXVA2), #endif + HW_CONFIG_ENCODER_FRAMES(AMF_SURFACE, AMF), + HW_CONFIG_ENCODER_DEVICE(NONE, AMF), NULL, }; diff --git a/libavcodec/amfenc.h b/libavcodec/amfenc.h index 1bda0136bd..1f680beb83 100644 --- a/libavcodec/amfenc.h +++ b/libavcodec/amfenc.h @@ -25,7 +25,7 @@ #include #include #include - +#include "libavutil/hwcontext_amf.h" #include "libavutil/fifo.h" #include "avcodec.h" @@ -33,16 +33,6 @@ #define MAX_LOOKAHEAD_DEPTH 41 -/** -* AMF trace writer callback class -* Used to capture all AMF logging -*/ - -typedef struct AmfTraceWriter { - AMFTraceWriterVtbl *vtbl; - AVCodecContext *avctx; -} AmfTraceWriter; - /** * AMF encoder context */ @@ -50,14 +40,7 @@ typedef struct AmfTraceWriter { typedef struct AmfContext { AVClass *avclass; // access to AMF runtime - amf_handle library; ///< handle to DLL library - AMFFactory *factory; ///< pointer to AMF factory - AMFDebug *debug; ///< pointer to AMF debug interface - AMFTrace *trace; ///< pointer to AMF trace interface - - amf_uint64 version; ///< version of AMF runtime - AmfTraceWriter tracer; ///< AMF writer registered with AMF - AMFContext *context; ///< AMF context + AVBufferRef *amf_device_ctx_internal; //encoder AMFComponent *encoder; ///< AMF encoder object amf_bool eof; ///< flag indicating EOF happened @@ -71,8 +54,6 @@ typedef struct AmfContext { // helpers to handle async calls int delayed_drain; - AMFSurface *delayed_surface; - AVFrame *delayed_frame; // shift dts back by max_b_frames in timing AVFifo *timestamp_list; From patchwork Wed May 1 18:38:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitrii Ovchinnikov X-Patchwork-Id: 48447 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:1203:b0:1a9:af23:56c1 with SMTP id v3csp1104519pzf; Wed, 1 May 2024 11:39:53 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCUh17KxhCSnqVg4/b2rAP38RjSjsb6Px9uc3si0vFHwX4XiCIZtouNDOD6AHKQ6gM2T99g51S7qfstpUnJNbtpivAtamaIh6iIeuA== X-Google-Smtp-Source: AGHT+IGTdrF6r05SR2+Y4DyiZYWPSRJPXAhWLmC9N/bstYuJx+A5Ev8ITrotEkxOMjsVav3AEjJC X-Received: by 2002:a2e:9303:0:b0:2e0:ffea:4282 with SMTP id e3-20020a2e9303000000b002e0ffea4282mr2489294ljh.38.1714588792807; Wed, 01 May 2024 11:39:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1714588792; cv=none; d=google.com; s=arc-20160816; b=fVndyOQ14u6mbImDtQmqFzI5nFSFMTG/77pBA8VcJPkL2yoxZxS7F7mI+AXfYJ/KJ5 h0rdIJOkm6SX1RFW6b3QUGRsQ32SwDwBnYZMwkahfVtGzZOQKIWbKCu16KfwxV3vqnAa 66wO2p1Yze0fRffAgTGm2BFahLiqT2nboGB8UKlk9aKhEJX7qGjeFCpO9vxSENepxhx+ YTohy/fksMM4Y+Myy5t3zyLXki4DWC6cBxyZ+Bhxqz9TbFihAFiQBvDxMjqEsYUcSZOT 3Z89sIg4tPHX1LvFfDxPic7UVK18/rs2LIHKZHZ37N5xE80ZLjH9AVWtnWTj1RGmNr/L 0Glg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=Q5O+dQF7MzXtADCRoAi84xBiWdgZkiakybcgx1eM8t4=; fh=l8BBgSaHK1qPkRTUMT3LasR23D+FgYCLMIPW5YpagSs=; b=YZSlC9rqvuOylswItDxBnf5tmhgTZHe5XisEX2rwuo3ZksjQL3GSSTZeMHQRi5EX7i qViMvLm3QtejVrsS8fuopVvird0KkmMv6wUY++/v/IpNN7R/ysHTFCdXQhmDnjzp4nAd 9+AT8LJgPvutYcJ1jnOkUEEgQFn+ba3U/q2VIZ+BQTkB+MKD8FtPsYVbkYusZU5LUvgn cxhjYgMY+x/nCUKb/MrtcMEZd+bWEWzLNKSSNyyFufJHEkw/fZLWV0jv9NFW0Tas7ISc IiJJkjZwHCmjaESCyYB0u6bN4slZfeHIca5Qew43S6B9Dp3N5hoRAWwQKtINvyHRP0lP ok0g==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=KpJsb4XI; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id e4-20020a2e8ec4000000b002dab24ea81esi8935699ljl.401.2024.05.01.11.39.51; Wed, 01 May 2024 11:39:52 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=KpJsb4XI; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D383768D72A; Wed, 1 May 2024 21:38:36 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f49.google.com (mail-lf1-f49.google.com [209.85.167.49]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CCB1068D707 for ; Wed, 1 May 2024 21:38:24 +0300 (EEST) Received: by mail-lf1-f49.google.com with SMTP id 2adb3069b0e04-51ca95db667so7171741e87.0 for ; Wed, 01 May 2024 11:38:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1714588703; x=1715193503; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=au0kyeWopyOgxWdmkRTZj0wa4IJANaw3Osr8Ol5hOag=; b=KpJsb4XILQylmZXxZPH/RfitGL2Y4I81whGbhQtKw4kyOSRxHpILMDFS4CM6NoBEVr DxOo07AMRrl9suIIsBVeC9fjhhNczyRTu6+fRy+w1p6LbilkkoTu16jvaluG/qaNrHtd Z2hzMuVhOp5t3ZoWYDPAwYjWyWVlVW1m8gY9kAABOPVPR7TOBawtN9XB6u7Iof88vKBO 8NG4x5BedGIri4UqOjxkxL/Fi5Z6IuHwEJ2ZA8QE9sYEUL/Bs9Wb1Ihqnm0c1KbOl3kw JH9pZSuKwSbGRH+VmJYhvE4+oOa8bIG+pTdO8xtBrd0KAUVzGOUX2H8y6cUKQ0KRxM9s 8LNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714588703; x=1715193503; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=au0kyeWopyOgxWdmkRTZj0wa4IJANaw3Osr8Ol5hOag=; b=CsYwExwaTf/y1O13KRjo40hRV2knZSQjWcd0BScpjLrdS4U9sVKlh0pDPtsrNtBIyX yKBljFXSuKP/MEET484tDCB4V1UY1VwD7AR7/zIZb9pU9qBCS7KFaUfR6wwb6VOh7cBC 5sXuEM+Gqb44RF8fBGE6R1MuLqHffQ9Sp5xItkDukm9ztnWacZ68L4pwVAJbr7ZpJ1qv 9o2HNUio7Ac/0D+CFHhQbWjrsqdl8knEFnXIX4XDfavrbDycI1WuF989Q3FThg2oL3+J vD3MQOPbJQNNtJyX9QenYSqTaBo8lmsVhBvTy+CLG3IREUpt8SVMT71eC67+3Idnf/dv HluA== X-Gm-Message-State: AOJu0YzmNydg7+VWnalHFX0pnr79q3qXBDTzQwC/z+Cl83VaX2xwEKep PpbF51gT5hw4gljnwqCkPDYYqKK6gFSP6Pzzftb52rlszRUm0MM5EH3CYWEP X-Received: by 2002:ac2:48a2:0:b0:51b:1e76:4ea9 with SMTP id u2-20020ac248a2000000b0051b1e764ea9mr2433059lfg.4.1714588703211; Wed, 01 May 2024 11:38:23 -0700 (PDT) Received: from dovchinn.amd.com (cable-178-148-16-149.dynamic.sbb.rs. [178.148.16.149]) by smtp.gmail.com with ESMTPSA id kh20-20020a170906f81400b00a58bf434876sm7367337ejb.58.2024.05.01.11.38.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 May 2024 11:38:22 -0700 (PDT) From: Dmitrii Ovchinnikov To: ffmpeg-devel@ffmpeg.org Date: Wed, 1 May 2024 20:38:08 +0200 Message-Id: <20240501183809.1060-9-ovchinnikov.dmitrii@gmail.com> X-Mailer: git-send-email 2.38.1.windows.1 In-Reply-To: <20240501183809.1060-1-ovchinnikov.dmitrii@gmail.com> References: <20240501183809.1060-1-ovchinnikov.dmitrii@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 09/10, v2] avfilter/scale_amf: Add AMF VPP & super resolution filters X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Evgeny Pavlov Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 5mkABoo3p+jY From: Evgeny Pavlov This commit adds two AMF filters: vpp_amf & sr_amf. Both filters are using AMF hardware acceleration. vpp_amf supports simple scaling algorithms & color conversion. sr_amf supports advanced scaling algorithms such as FSR & can be used for upscaling only. --- configure | 1 + libavfilter/Makefile | 2 + libavfilter/allfilters.c | 2 + libavfilter/vf_amf_common.c | 516 ++++++++++++++++++++++++++++++++++++ libavfilter/vf_amf_common.h | 71 +++++ libavfilter/vf_sr_amf.c | 192 ++++++++++++++ libavfilter/vf_vpp_amf.c | 267 +++++++++++++++++++ 7 files changed, 1051 insertions(+) create mode 100644 libavfilter/vf_amf_common.c create mode 100644 libavfilter/vf_amf_common.h create mode 100644 libavfilter/vf_sr_amf.c create mode 100644 libavfilter/vf_vpp_amf.c diff --git a/configure b/configure index 8101b4fce6..34313172e1 100755 --- a/configure +++ b/configure @@ -3902,6 +3902,7 @@ rubberband_filter_deps="librubberband" sab_filter_deps="gpl swscale" scale2ref_filter_deps="swscale" scale_filter_deps="swscale" +scale_amf_filter_deps="amf" scale_qsv_filter_deps="libmfx" scale_qsv_filter_select="qsvvpp" scdet_filter_select="scene_sad" diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 8571e9e2af..f903e8031e 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -499,6 +499,7 @@ OBJS-$(CONFIG_SITI_FILTER) += vf_siti.o OBJS-$(CONFIG_SPLIT_FILTER) += split.o OBJS-$(CONFIG_SPP_FILTER) += vf_spp.o qp_table.o OBJS-$(CONFIG_SR_FILTER) += vf_sr.o +OBJS-$(CONFIG_SR_AMF_FILTER) += vf_sr_amf.o scale_eval.o vf_amf_common.o OBJS-$(CONFIG_SSIM_FILTER) += vf_ssim.o framesync.o OBJS-$(CONFIG_SSIM360_FILTER) += vf_ssim360.o framesync.o OBJS-$(CONFIG_STEREO3D_FILTER) += vf_stereo3d.o @@ -552,6 +553,7 @@ OBJS-$(CONFIG_VIDSTABTRANSFORM_FILTER) += vidstabutils.o vf_vidstabtransfo OBJS-$(CONFIG_VIF_FILTER) += vf_vif.o framesync.o OBJS-$(CONFIG_VIGNETTE_FILTER) += vf_vignette.o OBJS-$(CONFIG_VMAFMOTION_FILTER) += vf_vmafmotion.o framesync.o +OBJS-$(CONFIG_VPP_AMF_FILTER) += vf_vpp_amf.o scale_eval.o vf_amf_common.o OBJS-$(CONFIG_VPP_QSV_FILTER) += vf_vpp_qsv.o OBJS-$(CONFIG_VSTACK_FILTER) += vf_stack.o framesync.o OBJS-$(CONFIG_W3FDIF_FILTER) += vf_w3fdif.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index c532682fc2..2f40fb8f6f 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -430,6 +430,8 @@ extern const AVFilter ff_vf_roberts_opencl; extern const AVFilter ff_vf_rotate; extern const AVFilter ff_vf_sab; extern const AVFilter ff_vf_scale; +extern const AVFilter ff_vf_vpp_amf; +extern const AVFilter ff_vf_sr_amf; extern const AVFilter ff_vf_scale_cuda; extern const AVFilter ff_vf_scale_npp; extern const AVFilter ff_vf_scale_qsv; diff --git a/libavfilter/vf_amf_common.c b/libavfilter/vf_amf_common.c new file mode 100644 index 0000000000..144b9cf604 --- /dev/null +++ b/libavfilter/vf_amf_common.c @@ -0,0 +1,516 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "vf_amf_common.h" + +#include "libavutil/avassert.h" +#include "avfilter.h" +#include "internal.h" +#include "formats.h" +#include "libavutil/imgutils.h" + +#include "libavutil/hwcontext_amf.h" +#include "libavutil/hwcontext_amf_internal.h" +#include "AMF/components/ColorSpace.h" +#include "scale_eval.h" + +#if CONFIG_DXVA2 +#include +#endif + +#if CONFIG_D3D11VA +#include +#endif + +int amf_filter_init(AVFilterContext *avctx) +{ + AMFFilterContext *ctx = avctx->priv; + + if (!strcmp(ctx->format_str, "same")) { + ctx->format = AV_PIX_FMT_NONE; + } else { + ctx->format = av_get_pix_fmt(ctx->format_str); + if (ctx->format == AV_PIX_FMT_NONE) { + av_log(avctx, AV_LOG_ERROR, "Unrecognized pixel format: %s\n", ctx->format_str); + return AVERROR(EINVAL); + } + } + + return 0; +} + +void amf_filter_uninit(AVFilterContext *avctx) +{ + AMFFilterContext *ctx = avctx->priv; + + if (ctx->component) { + ctx->component->pVtbl->Terminate(ctx->component); + ctx->component->pVtbl->Release(ctx->component); + ctx->component = NULL; + } + + av_buffer_unref(&ctx->amf_device_ref); + av_buffer_unref(&ctx->hwdevice_ref); + av_buffer_unref(&ctx->hwframes_in_ref); + av_buffer_unref(&ctx->hwframes_out_ref); +} + +int amf_filter_filter_frame(AVFilterLink *inlink, AVFrame *in) +{ + AVFilterContext *avctx = inlink->dst; + AMFFilterContext *ctx = avctx->priv; + AVFilterLink *outlink = avctx->outputs[0]; + AMF_RESULT res; + AMFSurface *surface_in; + AMFSurface *surface_out; + AMFData *data_out = NULL; + enum AVColorSpace out_colorspace; + enum AVColorRange out_color_range; + + AVFrame *out = NULL; + int ret = 0; + + if (!ctx->component) + return AVERROR(EINVAL); + + ret = amf_avframe_to_amfsurface(avctx, in, &surface_in); + if (ret < 0) + goto fail; + + res = ctx->component->pVtbl->SubmitInput(ctx->component, (AMFData*)surface_in); + AMF_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "SubmitInput() failed with error %d\n", res); + res = ctx->component->pVtbl->QueryOutput(ctx->component, &data_out); + AMF_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "QueryOutput() failed with error %d\n", res); + + if (data_out) { + AMFGuid guid = IID_AMFSurface(); + data_out->pVtbl->QueryInterface(data_out, &guid, (void**)&surface_out); // query for buffer interface + data_out->pVtbl->Release(data_out); + } + + out = amf_amfsurface_to_avframe(avctx, surface_out); + + ret = av_frame_copy_props(out, in); + av_frame_unref(in); + + out_colorspace = AVCOL_SPC_UNSPECIFIED; + + if (ctx->color_profile != AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN) { + switch(ctx->color_profile) { + case AMF_VIDEO_CONVERTER_COLOR_PROFILE_601: + out_colorspace = AVCOL_SPC_SMPTE170M; + break; + case AMF_VIDEO_CONVERTER_COLOR_PROFILE_709: + out_colorspace = AVCOL_SPC_BT709; + break; + case AMF_VIDEO_CONVERTER_COLOR_PROFILE_2020: + out_colorspace = AVCOL_SPC_BT2020_NCL; + break; + case AMF_VIDEO_CONVERTER_COLOR_PROFILE_JPEG: + out_colorspace = AVCOL_SPC_RGB; + break; + default: + out_colorspace = AVCOL_SPC_UNSPECIFIED; + break; + } + out->colorspace = out_colorspace; + } + + out_color_range = AVCOL_RANGE_UNSPECIFIED; + if (ctx->color_range == AMF_COLOR_RANGE_FULL) + out_color_range = AVCOL_RANGE_JPEG; + else if (ctx->color_range == AMF_COLOR_RANGE_STUDIO) + out_color_range = AVCOL_RANGE_MPEG; + + if (ctx->color_range != AMF_COLOR_RANGE_UNDEFINED) + out->color_range = out_color_range; + + if (ctx->primaries != AMF_COLOR_PRIMARIES_UNDEFINED) + out->color_primaries = ctx->primaries; + + if (ctx->trc != AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED) + out->color_trc = ctx->trc; + + + if (ret < 0) + goto fail; + + out->format = outlink->format; + out->width = outlink->w; + out->height = outlink->h; + + out->hw_frames_ctx = av_buffer_ref(ctx->hwframes_out_ref); + if (!out->hw_frames_ctx) { + ret = AVERROR(ENOMEM); + goto fail; + } + + if (inlink->sample_aspect_ratio.num) { + outlink->sample_aspect_ratio = av_mul_q((AVRational){outlink->h * inlink->w, outlink->w * inlink->h}, inlink->sample_aspect_ratio); + } else + outlink->sample_aspect_ratio = inlink->sample_aspect_ratio; + + av_frame_free(&in); + return ff_filter_frame(outlink, out); +fail: + av_frame_free(&in); + av_frame_free(&out); + return ret; +} + + + +int amf_setup_input_output_formats(AVFilterContext *avctx, + const enum AVPixelFormat *input_pix_fmts, + const enum AVPixelFormat *output_pix_fmts) +{ + int err; + int i; + AVFilterFormats *input_formats; + + //in case if hw_device_ctx is set to DXVA2 we change order of pixel formats to set DXVA2 be choosen by default + //The order is ignored if hw_frames_ctx is not NULL on the config_output stage + if (avctx->hw_device_ctx) { + AVHWDeviceContext *device_ctx = (AVHWDeviceContext*)avctx->hw_device_ctx->data; + + switch (device_ctx->type) { + #if CONFIG_D3D11VA + case AV_HWDEVICE_TYPE_D3D11VA: + { + static const enum AVPixelFormat output_pix_fmts_d3d11[] = { + AV_PIX_FMT_D3D11, + AV_PIX_FMT_NONE, + }; + output_pix_fmts = output_pix_fmts_d3d11; + } + break; + #endif + #if CONFIG_DXVA2 + case AV_HWDEVICE_TYPE_DXVA2: + { + static const enum AVPixelFormat output_pix_fmts_dxva2[] = { + AV_PIX_FMT_DXVA2_VLD, + AV_PIX_FMT_NONE, + }; + output_pix_fmts = output_pix_fmts_dxva2; + } + break; + #endif + default: + { + av_log(avctx, AV_LOG_ERROR, "Unsupported device : %s\n", av_hwdevice_get_type_name(device_ctx->type)); + return AVERROR(EINVAL); + } + break; + } + } + + input_formats = ff_make_format_list(output_pix_fmts); + if (!input_formats) { + return AVERROR(ENOMEM); + } + + for (i = 0; input_pix_fmts[i] != AV_PIX_FMT_NONE; i++) { + err = ff_add_format(&input_formats, input_pix_fmts[i]); + if (err < 0) + return err; + } + + if ((err = ff_formats_ref(input_formats, &avctx->inputs[0]->outcfg.formats)) < 0 || + (err = ff_formats_ref(ff_make_format_list(output_pix_fmts), + &avctx->outputs[0]->incfg.formats)) < 0) + return err; + return 0; +} + +int amf_copy_surface(AVFilterContext *avctx, const AVFrame *frame, + AMFSurface* surface) +{ + AMFPlane *plane; + uint8_t *dst_data[4]; + int dst_linesize[4]; + int planes; + int i; + + planes = surface->pVtbl->GetPlanesCount(surface); + av_assert0(planes < FF_ARRAY_ELEMS(dst_data)); + + for (i = 0; i < planes; i++) { + plane = surface->pVtbl->GetPlaneAt(surface, i); + dst_data[i] = plane->pVtbl->GetNative(plane); + dst_linesize[i] = plane->pVtbl->GetHPitch(plane); + } + av_image_copy(dst_data, dst_linesize, + (const uint8_t**)frame->data, frame->linesize, frame->format, + frame->width, frame->height); + + return 0; +} + +int amf_init_filter_config(AVFilterLink *outlink, enum AVPixelFormat *in_format) +{ + AVFilterContext *avctx = outlink->src; + AVFilterLink *inlink = avctx->inputs[0]; + AMFFilterContext *ctx = avctx->priv; + AVHWFramesContext *hwframes_out; + int err; + AMF_RESULT res; + + if ((err = ff_scale_eval_dimensions(avctx, + ctx->w_expr, ctx->h_expr, + inlink, outlink, + &ctx->width, &ctx->height)) < 0) + return err; + + ff_scale_adjust_dimensions(inlink, &ctx->width, &ctx->height, + ctx->force_original_aspect_ratio, ctx->force_divisible_by); + + av_buffer_unref(&ctx->amf_device_ref); + av_buffer_unref(&ctx->hwframes_in_ref); + av_buffer_unref(&ctx->hwframes_out_ref); + + if (inlink->hw_frames_ctx) { + AVHWFramesContext *frames_ctx = (AVHWFramesContext*)inlink->hw_frames_ctx->data; + if (frames_ctx->device_ctx->type == AV_HWDEVICE_TYPE_AMF) { + AVAMFDeviceContext * amf_ctx = frames_ctx->device_ctx->hwctx; + ctx->amf_device_ctx_internal = av_buffer_ref(amf_ctx->internal); + } + if (av_amf_av_to_amf_format(frames_ctx->sw_format) == AMF_SURFACE_UNKNOWN) { + av_log(avctx, AV_LOG_ERROR, "Format of input frames context (%s) is not supported by AMF.\n", + av_get_pix_fmt_name(frames_ctx->sw_format)); + return AVERROR(EINVAL); + } + + err = av_hwdevice_ctx_create_derived(&ctx->amf_device_ref, AV_HWDEVICE_TYPE_AMF, frames_ctx->device_ref, 0); + if (err < 0) + return err; + + ctx->hwframes_in_ref = av_buffer_ref(inlink->hw_frames_ctx); + if (!ctx->hwframes_in_ref) + return AVERROR(ENOMEM); + + ctx->hwframes_out_ref = av_hwframe_ctx_alloc(frames_ctx->device_ref); + if (!ctx->hwframes_out_ref) + return AVERROR(ENOMEM); + + hwframes_out = (AVHWFramesContext*)ctx->hwframes_out_ref->data; + hwframes_out->format = outlink->format; + hwframes_out->sw_format = frames_ctx->sw_format; + } else if (avctx->hw_device_ctx) { + AVHWDeviceContext *hwdev_ctx; + err = av_hwdevice_ctx_create_derived(&ctx->amf_device_ref, AV_HWDEVICE_TYPE_AMF, avctx->hw_device_ctx, 0); + if (err < 0) + return err; + hwdev_ctx = (AVHWDeviceContext*)avctx->hw_device_ctx->data; + if (hwdev_ctx->type == AV_HWDEVICE_TYPE_AMF) + { + AVAMFDeviceContext * amf_ctx = hwdev_ctx->hwctx; + ctx->amf_device_ctx_internal = av_buffer_ref(amf_ctx->internal); + } + ctx->hwdevice_ref = av_buffer_ref(avctx->hw_device_ctx); + if (!ctx->hwdevice_ref) + return AVERROR(ENOMEM); + + ctx->hwframes_out_ref = av_hwframe_ctx_alloc(ctx->hwdevice_ref); + if (!ctx->hwframes_out_ref) + return AVERROR(ENOMEM); + + hwframes_out = (AVHWFramesContext*)ctx->hwframes_out_ref->data; + hwframes_out->format = AV_PIX_FMT_AMF_SURFACE; + hwframes_out->sw_format = outlink->format; + } else { + AVAMFDeviceContextInternal *wrapped = av_mallocz(sizeof(*wrapped)); + ctx->amf_device_ctx_internal = av_buffer_create((uint8_t *)wrapped, sizeof(*wrapped), + av_amf_context_internal_free, NULL, 0); + if ((res == av_amf_context_internal_create((AVAMFDeviceContextInternal *)ctx->amf_device_ctx_internal->data, avctx, "", NULL, 0)) != 0) { + return res; + } + ctx->hwframes_out_ref = av_hwframe_ctx_alloc(ctx->amf_device_ref); + if (!ctx->hwframes_out_ref) + return AVERROR(ENOMEM); + + hwframes_out = (AVHWFramesContext*)ctx->hwframes_out_ref->data; + hwframes_out->format = outlink->format; + hwframes_out->sw_format = inlink->format; + } + + if (ctx->format != AV_PIX_FMT_NONE) { + hwframes_out->sw_format = ctx->format; + } + + if (inlink->format == AV_PIX_FMT_AMF_SURFACE) { + if (!inlink->hw_frames_ctx || !inlink->hw_frames_ctx->data) + return AVERROR(EINVAL); + else + *in_format = ((AVHWFramesContext*)inlink->hw_frames_ctx->data)->sw_format; + } else + *in_format = inlink->format; + + outlink->w = ctx->width; + outlink->h = ctx->height; + + hwframes_out->width = outlink->w; + hwframes_out->height = outlink->h; + + err = av_hwframe_ctx_init(ctx->hwframes_out_ref); + if (err < 0) + return err; + + outlink->hw_frames_ctx = av_buffer_ref(ctx->hwframes_out_ref); + if (!outlink->hw_frames_ctx) { + return AVERROR(ENOMEM); + } + return 0; +} + +void amf_free_amfsurface(void *opaque, uint8_t *data) +{ + AMFSurface *surface = (AMFSurface*)data; + surface->pVtbl->Release(surface); +} + +AVFrame *amf_amfsurface_to_avframe(AVFilterContext *avctx, AMFSurface* pSurface) +{ + AVFrame *frame = av_frame_alloc(); + AMFFilterContext *ctx = avctx->priv; + + if (!frame) + return NULL; + + if (ctx->hwframes_out_ref) { + AVHWFramesContext *hwframes_out = (AVHWFramesContext *)ctx->hwframes_out_ref->data; + if (hwframes_out->format == AV_PIX_FMT_AMF_SURFACE) { + int ret = av_hwframe_get_buffer(ctx->hwframes_out_ref, frame, 0); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Get hw frame failed.\n"); + av_frame_free(&frame); + return NULL; + } + frame->data[3] = (uint8_t *)pSurface; + frame->buf[1] = av_buffer_create((uint8_t *)pSurface, sizeof(AMFSurface), + amf_free_amfsurface, + (void*)avctx, + AV_BUFFER_FLAG_READONLY); + } else { // FIXME: add processing of other hw formats + av_log(ctx, AV_LOG_ERROR, "Unknown pixel format\n"); + return NULL; + } + } else { + + switch (pSurface->pVtbl->GetMemoryType(pSurface)) + { + #if CONFIG_D3D11VA + case AMF_MEMORY_DX11: + { + AMFPlane *plane0 = pSurface->pVtbl->GetPlaneAt(pSurface, 0); + frame->data[0] = plane0->pVtbl->GetNative(plane0); + frame->data[1] = (uint8_t*)(intptr_t)0; + + frame->buf[0] = av_buffer_create(NULL, + 0, + amf_free_amfsurface, + pSurface, + AV_BUFFER_FLAG_READONLY); + } + break; + #endif + #if CONFIG_DXVA2 + case AMF_MEMORY_DX9: + { + AMFPlane *plane0 = pSurface->pVtbl->GetPlaneAt(pSurface, 0); + frame->data[3] = plane0->pVtbl->GetNative(plane0); + + frame->buf[0] = av_buffer_create(NULL, + 0, + amf_free_amfsurface, + pSurface, + AV_BUFFER_FLAG_READONLY); + } + break; + #endif + default: + { + av_log(avctx, AV_LOG_ERROR, "Unsupported memory type : %d\n", pSurface->pVtbl->GetMemoryType(pSurface)); + return NULL; + } + } + } + + return frame; +} + +int amf_avframe_to_amfsurface(AVFilterContext *avctx, const AVFrame *frame, AMFSurface** ppSurface) +{ + AMFFilterContext *ctx = avctx->priv; + AVAMFDeviceContextInternal* internal = (AVAMFDeviceContextInternal *)ctx->amf_device_ctx_internal->data; + AMFSurface *surface; + AMF_RESULT res; + int hw_surface = 0; + + switch (frame->format) { +#if CONFIG_D3D11VA + case AV_PIX_FMT_D3D11: + { + static const GUID AMFTextureArrayIndexGUID = { 0x28115527, 0xe7c3, 0x4b66, { 0x99, 0xd3, 0x4f, 0x2a, 0xe6, 0xb4, 0x7f, 0xaf } }; + ID3D11Texture2D *texture = (ID3D11Texture2D*)frame->data[0]; // actual texture + int index = (intptr_t)frame->data[1]; // index is a slice in texture array is - set to tell AMF which slice to use + texture->lpVtbl->SetPrivateData(texture, &AMFTextureArrayIndexGUID, sizeof(index), &index); + + res = internal->context->pVtbl->CreateSurfaceFromDX11Native(internal->context, texture, &surface, NULL); // wrap to AMF surface + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR(ENOMEM), "CreateSurfaceFromDX11Native() failed with error %d\n", res); + hw_surface = 1; + } + break; +#endif + case AV_PIX_FMT_AMF_SURFACE: + { + surface = (AMFSurface*)frame->data[3]; // actual surface + hw_surface = 1; + } + break; + +#if CONFIG_DXVA2 + case AV_PIX_FMT_DXVA2_VLD: + { + IDirect3DSurface9 *texture = (IDirect3DSurface9 *)frame->data[3]; // actual texture + + res = internal->context->pVtbl->CreateSurfaceFromDX9Native(internal->context, texture, &surface, NULL); // wrap to AMF surface + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR(ENOMEM), "CreateSurfaceFromDX9Native() failed with error %d\n", res); + hw_surface = 1; + } + break; +#endif + default: + { + AMF_SURFACE_FORMAT amf_fmt = av_amf_av_to_amf_format(frame->format); + res = internal->context->pVtbl->AllocSurface(internal->context, AMF_MEMORY_HOST, amf_fmt, frame->width, frame->height, &surface); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR(ENOMEM), "AllocSurface() failed with error %d\n", res); + amf_copy_surface(avctx, frame, surface); + } + break; + } + + if (hw_surface) { + // input HW surfaces can be vertically aligned by 16; tell AMF the real size + surface->pVtbl->SetCrop(surface, 0, 0, frame->width, frame->height); + } + + surface->pVtbl->SetPts(surface, frame->pts); + *ppSurface = surface; + return 0; +} diff --git a/libavfilter/vf_amf_common.h b/libavfilter/vf_amf_common.h new file mode 100644 index 0000000000..d8aa76adaf --- /dev/null +++ b/libavfilter/vf_amf_common.h @@ -0,0 +1,71 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVFILTER_AMF_COMMON_H +#define AVFILTER_AMF_COMMON_H + +#include "avfilter.h" + +#include "AMF/core/Surface.h" +#include "AMF/components/Component.h" + +typedef struct AMFFilterContext { + const AVClass *class; + + int width, height; + enum AVPixelFormat format; + int scale_type; + int color_profile; + int color_range; + int primaries; + int trc; + int fill; + int fill_color; + int keep_ratio; + + // HQScaler properties + int algorithm; + float sharpness; + + char *w_expr; + char *h_expr; + char *format_str; + int force_original_aspect_ratio; + int force_divisible_by; + + AMFComponent *component; + AVBufferRef *amf_device_ref; + + AVBufferRef *hwframes_in_ref; + AVBufferRef *hwframes_out_ref; + AVBufferRef *hwdevice_ref; + + AVBufferRef *amf_device_ctx_internal; +} AMFFilterContext; + +int amf_filter_init(AVFilterContext *avctx); +void amf_filter_uninit(AVFilterContext *avctx); +int amf_init_filter_config(AVFilterLink *outlink, enum AVPixelFormat *in_format); +int amf_copy_surface(AVFilterContext *avctx, const AVFrame *frame, AMFSurface* surface); +void amf_free_amfsurface(void *opaque, uint8_t *data); +AVFrame *amf_amfsurface_to_avframe(AVFilterContext *avctx, AMFSurface* pSurface); +int amf_avframe_to_amfsurface(AVFilterContext *avctx, const AVFrame *frame, AMFSurface** ppSurface); +int amf_setup_input_output_formats(AVFilterContext *avctx, const enum AVPixelFormat *input_pix_fmts, const enum AVPixelFormat *output_pix_fmts); +int amf_filter_filter_frame(AVFilterLink *inlink, AVFrame *in); + +#endif /* AVFILTER_AMF_COMMON_H */ diff --git a/libavfilter/vf_sr_amf.c b/libavfilter/vf_sr_amf.c new file mode 100644 index 0000000000..fc44f7afbb --- /dev/null +++ b/libavfilter/vf_sr_amf.c @@ -0,0 +1,192 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * Super resolution video filter with AMF hardware acceleration + */ + +#include +#include + +#include "libavutil/avassert.h" +#include "libavutil/imgutils.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "libavutil/time.h" + +#include "libavutil/hwcontext.h" +#include "libavutil/hwcontext_amf.h" +#include "libavutil/hwcontext_amf_internal.h" + +#include "AMF/components/HQScaler.h" +#include "AMF/components/ColorSpace.h" +#include "vf_amf_common.h" + +#include "avfilter.h" +#include "internal.h" +#include "formats.h" +#include "video.h" + +#if CONFIG_DXVA2 +#include +#endif + +#if CONFIG_D3D11VA +#include +#endif + + +static int amf_filter_query_formats(AVFilterContext *avctx) +{ + const enum AVPixelFormat *output_pix_fmts; + static const enum AVPixelFormat input_pix_fmts[] = { + AV_PIX_FMT_NV12, + AV_PIX_FMT_P010, + AV_PIX_FMT_BGRA, + AV_PIX_FMT_RGBA, + AV_PIX_FMT_AMF_SURFACE, + AV_PIX_FMT_RGBAF16, + AV_PIX_FMT_NONE, + }; + static const enum AVPixelFormat output_pix_fmts_default[] = { + AV_PIX_FMT_NV12, + AV_PIX_FMT_P010, + AV_PIX_FMT_BGRA, + AV_PIX_FMT_RGBA, + AV_PIX_FMT_AMF_SURFACE, + AV_PIX_FMT_D3D11, + AV_PIX_FMT_DXVA2_VLD, + AV_PIX_FMT_RGBAF16, + AV_PIX_FMT_NONE, + }; + output_pix_fmts = output_pix_fmts_default; + + return amf_setup_input_output_formats(avctx, input_pix_fmts, output_pix_fmts); +} + +static int amf_filter_config_output(AVFilterLink *outlink) +{ + AVFilterContext *avctx = outlink->src; + AVFilterLink *inlink = avctx->inputs[0]; + AMFFilterContext *ctx = avctx->priv; + AVAMFDeviceContextInternal * internal = NULL; + AMFSize out_size; + int err; + AMF_RESULT res; + enum AVPixelFormat in_format; + + err = amf_init_filter_config(outlink, &in_format); + if (err < 0) + return err; + + // HQ scaler should be used for upscaling only + if (inlink->w > outlink->w || inlink->h > outlink->h) { + av_log(avctx, AV_LOG_ERROR, "AMF HQ scaler should be used for upscaling only.\n"); + return AVERROR_UNKNOWN; + } + // FIXME: add checks whether we have HW context + + internal = (AVAMFDeviceContextInternal * )ctx->amf_device_ctx_internal->data; + res = internal->factory->pVtbl->CreateComponent(internal->factory, internal->context, AMFHQScaler, &ctx->component); + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_FILTER_NOT_FOUND, "CreateComponent(%ls) failed with error %d\n", AMFHQScaler, res); + + out_size.width = outlink->w; + out_size.height = outlink->h; + AMF_ASSIGN_PROPERTY_SIZE(res, ctx->component, AMF_HQ_SCALER_OUTPUT_SIZE, out_size); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "AMFHQScaler-SetProperty() failed with error %d\n", res); + + if (ctx->algorithm != -1) { + AMF_ASSIGN_PROPERTY_INT64(res, ctx->component, AMF_HQ_SCALER_ALGORITHM, ctx->algorithm); + } + if (ctx->sharpness != -1) { + AMF_ASSIGN_PROPERTY_DOUBLE(res, ctx->component, AMF_HQ_SCALER_SHARPNESS, ctx->sharpness); + } + AMF_ASSIGN_PROPERTY_BOOL(res, ctx->component, AMF_HQ_SCALER_FILL, ctx->fill); + AMF_ASSIGN_PROPERTY_BOOL(res, ctx->component, AMF_HQ_SCALER_KEEP_ASPECT_RATIO, ctx->keep_ratio); + // Setup default options to skip color conversion + ctx->color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN; + ctx->color_range = AMF_COLOR_RANGE_UNDEFINED; + ctx->primaries = AMF_COLOR_PRIMARIES_UNDEFINED; + ctx->trc = AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED; + + res = ctx->component->pVtbl->Init(ctx->component, av_amf_av_to_amf_format(in_format), inlink->w, inlink->h); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "AMFHQScaler-Init() failed with error %d\n", res); + + return 0; +} + +#define OFFSET(x) offsetof(AMFFilterContext, x) +#define FLAGS AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_FILTERING_PARAM +static const AVOption sr_amf_options[] = { + { "w", "Output video width", OFFSET(w_expr), AV_OPT_TYPE_STRING, { .str = "iw" }, .flags = FLAGS }, + { "h", "Output video height", OFFSET(h_expr), AV_OPT_TYPE_STRING, { .str = "ih" }, .flags = FLAGS }, + + { "format", "Output pixel format", OFFSET(format_str), AV_OPT_TYPE_STRING, { .str = "same" }, .flags = FLAGS }, + { "sharpness", "Sharpness", OFFSET(sharpness), AV_OPT_TYPE_FLOAT, { .dbl = -1 }, -1, 2., FLAGS, "sharpness" }, + { "keep-ratio", "Keep aspect ratio", OFFSET(keep_ratio), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, FLAGS, "keep_ration" }, + { "fill", "Fill", OFFSET(fill), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, FLAGS, "fill" }, + + { "algorithm", "Scaling algorithm", OFFSET(algorithm), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, AMF_HQ_SCALER_ALGORITHM_VIDEOSR1_1, FLAGS, "algorithm" }, + { "bilinear", "Bilinear", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_HQ_SCALER_ALGORITHM_BILINEAR }, 0, 0, FLAGS, "algorithm" }, + { "bicubic", "Bicubic", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_HQ_SCALER_ALGORITHM_BICUBIC }, 0, 0, FLAGS, "algorithm" }, + { "sr1-0", "Video SR1.0", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_HQ_SCALER_ALGORITHM_VIDEOSR1_0 }, 0, 0, FLAGS, "algorithm" }, + { "point", "Point", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_HQ_SCALER_ALGORITHM_POINT }, 0, 0, FLAGS, "algorithm" }, + { "sr1-1", "Video SR1.1", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_HQ_SCALER_ALGORITHM_VIDEOSR1_1 }, 0, 0, FLAGS, "algorithm" }, + + { NULL }, +}; + + +AVFILTER_DEFINE_CLASS(sr_amf); + +static const AVFilterPad amf_filter_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .filter_frame = amf_filter_filter_frame, + } +}; + +static const AVFilterPad amf_filter_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = amf_filter_config_output, + } +}; + +AVFilter ff_vf_sr_amf = { + .name = "sr_amf", + .description = NULL_IF_CONFIG_SMALL("AMF HQ video upscaling"), + + .init = amf_filter_init, + .uninit = amf_filter_uninit, + FILTER_QUERY_FUNC(&amf_filter_query_formats), + + .priv_size = sizeof(AMFFilterContext), + .priv_class = &sr_amf_class, + + FILTER_INPUTS(amf_filter_inputs), + FILTER_OUTPUTS(amf_filter_outputs), + + FILTER_SINGLE_PIXFMT(AV_PIX_FMT_AMF_SURFACE), + + .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + .flags = AVFILTER_FLAG_HWDEVICE, +}; \ No newline at end of file diff --git a/libavfilter/vf_vpp_amf.c b/libavfilter/vf_vpp_amf.c new file mode 100644 index 0000000000..18c7fb82bd --- /dev/null +++ b/libavfilter/vf_vpp_amf.c @@ -0,0 +1,267 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * VPP video filter with AMF hardware acceleration + */ + +#include +#include + +#include "libavutil/avassert.h" +#include "libavutil/imgutils.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "libavutil/time.h" + +#include "libavutil/hwcontext.h" +#include "libavutil/hwcontext_amf.h" +#include "libavutil/hwcontext_amf_internal.h" + +#include "AMF/components/VideoConverter.h" +#include "vf_amf_common.h" + +#include "avfilter.h" +#include "formats.h" +#include "video.h" +#include "scale_eval.h" +#include "internal.h" + +#if CONFIG_DXVA2 +#include +#endif + +#if CONFIG_D3D11VA +#include +#endif + +static int amf_filter_query_formats(AVFilterContext *avctx) +{ + const enum AVPixelFormat *output_pix_fmts; + static const enum AVPixelFormat input_pix_fmts[] = { + AV_PIX_FMT_NV12, + AV_PIX_FMT_P010, + AV_PIX_FMT_0RGB, + AV_PIX_FMT_BGR0, + AV_PIX_FMT_BGRA, + AV_PIX_FMT_RGB0, + AV_PIX_FMT_RGBA, + AV_PIX_FMT_GRAY8, + AV_PIX_FMT_YUV420P, + AV_PIX_FMT_YUV420P10, + AV_PIX_FMT_YUYV422, + AV_PIX_FMT_AMF_SURFACE, + AV_PIX_FMT_NONE, + }; + static const enum AVPixelFormat output_pix_fmts_default[] = { + AV_PIX_FMT_AMF_SURFACE, + AV_PIX_FMT_D3D11, + AV_PIX_FMT_DXVA2_VLD, + AV_PIX_FMT_NV12, + AV_PIX_FMT_BGRA, + AV_PIX_FMT_YUV420P, + AV_PIX_FMT_NONE, + }; + output_pix_fmts = output_pix_fmts_default; + + return amf_setup_input_output_formats(avctx, input_pix_fmts, output_pix_fmts); +} + +static int amf_filter_config_output(AVFilterLink *outlink) +{ + AVFilterContext *avctx = outlink->src; + AVFilterLink *inlink = avctx->inputs[0]; + AMFFilterContext *ctx = avctx->priv; + AVAMFDeviceContextInternal * internal = NULL; + AVHWFramesContext *hwframes_out = NULL; + AMFSize out_size; + int err; + AMF_RESULT res; + enum AMF_VIDEO_CONVERTER_COLOR_PROFILE_ENUM amf_color_profile; + enum AVPixelFormat in_format; + + err = amf_init_filter_config(outlink, &in_format); + if (err < 0) + return err; + // FIXME: add checks whether we have HW context + hwframes_out = (AVHWFramesContext*)ctx->hwframes_out_ref->data; + + internal = (AVAMFDeviceContextInternal * )ctx->amf_device_ctx_internal->data; + res = internal->factory->pVtbl->CreateComponent(internal->factory, internal->context, AMFVideoConverter, &ctx->component); + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_FILTER_NOT_FOUND, "CreateComponent(%ls) failed with error %d\n", AMFVideoConverter, res); + // FIXME: add checks whether we have HW context + AMF_ASSIGN_PROPERTY_INT64(res, ctx->component, AMF_VIDEO_CONVERTER_OUTPUT_FORMAT, (amf_int32)av_amf_av_to_amf_format(hwframes_out->sw_format)); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "AMFConverter-SetProperty() failed with error %d\n", res); + + out_size.width = outlink->w; + out_size.height = outlink->h; + AMF_ASSIGN_PROPERTY_SIZE(res, ctx->component, AMF_VIDEO_CONVERTER_OUTPUT_SIZE, out_size); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "AMFConverter-SetProperty() failed with error %d\n", res); + + AMF_ASSIGN_PROPERTY_INT64(res, ctx->component, AMF_VIDEO_CONVERTER_SCALE, (amf_int32)ctx->scale_type); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "AMFConverter-SetProperty() failed with error %d\n", res); + + amf_color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN; + + switch(ctx->color_profile) { + case AMF_VIDEO_CONVERTER_COLOR_PROFILE_601: + if (ctx->color_range == AMF_COLOR_RANGE_FULL) { + amf_color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_601; + } else { + amf_color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_601; + } + break; + case AMF_VIDEO_CONVERTER_COLOR_PROFILE_709: + if (ctx->color_range == AMF_COLOR_RANGE_FULL) { + amf_color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_709; + } else { + amf_color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_709; + } + break; + case AMF_VIDEO_CONVERTER_COLOR_PROFILE_2020: + if (ctx->color_range == AMF_COLOR_RANGE_FULL) { + amf_color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_2020; + } else { + amf_color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_2020; + } + break; + default: + amf_color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN; + break; + } + + if (amf_color_profile != AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN) { + AMF_ASSIGN_PROPERTY_INT64(res, ctx->component, AMF_VIDEO_CONVERTER_COLOR_PROFILE, amf_color_profile); + } + + if (ctx->color_range != AMF_COLOR_RANGE_UNDEFINED) { + AMF_ASSIGN_PROPERTY_INT64(res, ctx->component, AMF_VIDEO_CONVERTER_OUTPUT_COLOR_RANGE, ctx->color_range); + } + + if (ctx->primaries != AMF_COLOR_PRIMARIES_UNDEFINED) { + AMF_ASSIGN_PROPERTY_INT64(res, ctx->component, AMF_VIDEO_CONVERTER_OUTPUT_COLOR_PRIMARIES, ctx->primaries); + } + + if (ctx->trc != AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED) { + AMF_ASSIGN_PROPERTY_INT64(res, ctx->component, AMF_VIDEO_CONVERTER_OUTPUT_TRANSFER_CHARACTERISTIC, ctx->trc); + } + + res = ctx->component->pVtbl->Init(ctx->component, av_amf_av_to_amf_format(in_format), inlink->w, inlink->h); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "AMFConverter-Init() failed with error %d\n", res); + + return 0; +} + +#define OFFSET(x) offsetof(AMFFilterContext, x) +#define FLAGS AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_FILTERING_PARAM +static const AVOption vpp_amf_options[] = { + { "w", "Output video width", OFFSET(w_expr), AV_OPT_TYPE_STRING, { .str = "iw" }, .flags = FLAGS }, + { "h", "Output video height", OFFSET(h_expr), AV_OPT_TYPE_STRING, { .str = "ih" }, .flags = FLAGS }, + { "format", "Output pixel format", OFFSET(format_str), AV_OPT_TYPE_STRING, { .str = "same" }, .flags = FLAGS }, + + { "scale_type", "Scale type", OFFSET(scale_type), AV_OPT_TYPE_INT, { .i64 = AMF_VIDEO_CONVERTER_SCALE_BILINEAR }, AMF_VIDEO_CONVERTER_SCALE_BILINEAR, AMF_VIDEO_CONVERTER_SCALE_BICUBIC, FLAGS, "scale_type" }, + { "bilinear", "Bilinear", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_CONVERTER_SCALE_BILINEAR }, 0, 0, FLAGS, "scale_type" }, + { "bicubic", "Bicubic", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_CONVERTER_SCALE_BICUBIC }, 0, 0, FLAGS, "scale_type" }, + + { "color_profile", "Color profile", OFFSET(color_profile), AV_OPT_TYPE_INT, { .i64 = AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN }, AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN, AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_2020, FLAGS, "color_profile" }, + { "bt601", "BT.601", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_CONVERTER_COLOR_PROFILE_601 }, 0, 0, FLAGS, "color_profile" }, + { "bt709", "BT.709", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_CONVERTER_COLOR_PROFILE_709 }, 0, 0, FLAGS, "color_profile" }, + { "bt2020", "BT.2020", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_CONVERTER_COLOR_PROFILE_2020 }, 0, 0, FLAGS, "color_profile" }, + + { "color_range", "Color range", OFFSET(color_range), AV_OPT_TYPE_INT, { .i64 = AMF_COLOR_RANGE_UNDEFINED }, AMF_COLOR_RANGE_UNDEFINED, AMF_COLOR_RANGE_FULL, FLAGS, "color_range" }, + { "studio", "Studio", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_RANGE_STUDIO }, 0, 0, FLAGS, "color_range" }, + { "full", "Full", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_RANGE_FULL }, 0, 0, FLAGS, "color_range" }, + + { "primaries", "Output color primaries", OFFSET(primaries), AV_OPT_TYPE_INT, { .i64 = AMF_COLOR_PRIMARIES_UNDEFINED }, AMF_COLOR_PRIMARIES_UNDEFINED, AMF_COLOR_PRIMARIES_JEDEC_P22, FLAGS, "primaries" }, + { "bt709", "BT.709", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_PRIMARIES_BT709 }, 0, 0, FLAGS, "primaries" }, + { "bt470m", "BT.470M", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_PRIMARIES_BT470M }, 0, 0, FLAGS, "primaries" }, + { "bt470bg", "BT.470BG", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_PRIMARIES_BT470BG }, 0, 0, FLAGS, "primaries" }, + { "smpte170m", "SMPTE170M", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_PRIMARIES_SMPTE170M }, 0, 0, FLAGS, "primaries" }, + { "smpte240m", "SMPTE240M", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_PRIMARIES_SMPTE240M }, 0, 0, FLAGS, "primaries" }, + { "film", "FILM", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_PRIMARIES_FILM }, 0, 0, FLAGS, "primaries" }, + { "bt2020", "BT2020", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_PRIMARIES_BT2020 }, 0, 0, FLAGS, "primaries" }, + { "smpte428", "SMPTE428", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_PRIMARIES_SMPTE428 }, 0, 0, FLAGS, "primaries" }, + { "smpte431", "SMPTE431", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_PRIMARIES_SMPTE431 }, 0, 0, FLAGS, "primaries" }, + { "smpte432", "SMPTE432", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_PRIMARIES_SMPTE432 }, 0, 0, FLAGS, "primaries" }, + { "jedec-p22", "JEDEC_P22", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_PRIMARIES_JEDEC_P22 }, 0, 0, FLAGS, "primaries" }, + + { "trc", "Output transfer characteristics", OFFSET(trc), AV_OPT_TYPE_INT, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED }, AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED, AMF_COLOR_TRANSFER_CHARACTERISTIC_ARIB_STD_B67, FLAGS, "trc" }, + { "bt709", "BT.709", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_BT709 }, 0, 0, FLAGS, "trc" }, + { "gamma22", "GAMMA22", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_GAMMA22 }, 0, 0, FLAGS, "trc" }, + { "gamma28", "GAMMA28", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_GAMMA28 }, 0, 0, FLAGS, "trc" }, + { "smpte170m", "SMPTE170M", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_SMPTE170M }, 0, 0, FLAGS, "trc" }, + { "smpte240m", "SMPTE240M", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_SMPTE240M }, 0, 0, FLAGS, "trc" }, + { "linear", "Linear", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_LINEAR }, 0, 0, FLAGS, "trc" }, + { "log", "LOG", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_LOG }, 0, 0, FLAGS, "trc" }, + { "log-sqrt", "LOG_SQRT", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_LOG_SQRT }, 0, 0, FLAGS, "trc" }, + { "iec61966-2-4", "IEC61966_2_4", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_IEC61966_2_4 }, 0, 0, FLAGS, "trc" }, + { "bt1361-ecg", "BT1361_ECG", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_BT1361_ECG }, 0, 0, FLAGS, "trc" }, + { "iec61966-2-1", "IEC61966_2_1", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_IEC61966_2_1 }, 0, 0, FLAGS, "trc" }, + { "bt2020-10", "BT.2020_10", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_BT2020_10 }, 0, 0, FLAGS, "trc" }, + { "bt2020-12", "BT.2020-12", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_BT2020_12 }, 0, 0, FLAGS, "trc" }, + { "smpte2084", "SMPTE2084", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_SMPTE2084 }, 0, 0, FLAGS, "trc" }, + { "smpte428", "SMPTE428", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_SMPTE428 }, 0, 0, FLAGS, "trc" }, + { "arib-std-b67", "ARIB_STD_B67", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_ARIB_STD_B67 }, 0, 0, FLAGS, "trc" }, + + { "force_original_aspect_ratio", "decrease or increase w/h if necessary to keep the original AR", OFFSET(force_original_aspect_ratio), AV_OPT_TYPE_INT, { .i64 = 0}, 0, 2, FLAGS, "force_oar" }, + { "disable", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = 0 }, 0, 0, FLAGS, "force_oar" }, + { "decrease", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = 1 }, 0, 0, FLAGS, "force_oar" }, + { "increase", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = 2 }, 0, 0, FLAGS, "force_oar" }, + { "force_divisible_by", "enforce that the output resolution is divisible by a defined integer when force_original_aspect_ratio is used", OFFSET(force_divisible_by), AV_OPT_TYPE_INT, { .i64 = 1}, 1, 256, FLAGS }, + + { NULL }, +}; + + +AVFILTER_DEFINE_CLASS(vpp_amf); + +static const AVFilterPad amf_filter_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .filter_frame = amf_filter_filter_frame, + } +}; + +static const AVFilterPad amf_filter_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = amf_filter_config_output, + } +}; + +AVFilter ff_vf_vpp_amf = { + .name = "vpp_amf", + .description = NULL_IF_CONFIG_SMALL("AMF video scaling and format conversion"), + + .init = amf_filter_init, + .uninit = amf_filter_uninit, + FILTER_QUERY_FUNC(&amf_filter_query_formats), + + .priv_size = sizeof(AMFFilterContext), + .priv_class = &vpp_amf_class, + + FILTER_INPUTS(amf_filter_inputs), + FILTER_OUTPUTS(amf_filter_outputs), + FILTER_SINGLE_PIXFMT(AV_PIX_FMT_AMF_SURFACE), + + .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + .flags = AVFILTER_FLAG_HWDEVICE, +}; From patchwork Wed May 1 18:38:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitrii Ovchinnikov X-Patchwork-Id: 48451 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:1203:b0:1a9:af23:56c1 with SMTP id v3csp1104598pzf; Wed, 1 May 2024 11:40:02 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCUvz5h7YxHVXeSLBZxlrNzTFnFyz6lhJEL30wcokK8/dxJ0TUoEywwQObLr9NMva7j39YrjwGRl0qvRXFTsdUAZslhlOP8JrNRcgA== X-Google-Smtp-Source: AGHT+IH5VXNaoWtZk9z45s9/ZGEPtuQloH4TilbO20GXbYhCV35Uo+WEdHNieliJNU+qKe3qosn6 X-Received: by 2002:a17:906:2a05:b0:a58:7dab:28a2 with SMTP id j5-20020a1709062a0500b00a587dab28a2mr2368231eje.48.1714588802532; Wed, 01 May 2024 11:40:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1714588802; cv=none; d=google.com; s=arc-20160816; b=Let/rvmIXhSOGmBK+9gD5II+nMKdV0EJl6T7MB0TUR++hkhHOiFRIBhJ4VzXoFeLGc uCpobArw4atM7rxfE3Puvl9McjYLZa+tpQbR0e6pPBZz+LrHXRj6X9b1qqtCEn5fEDq4 dU5h8Q38dw3RF4zTV9045A79ckBAA/L/iha0+BEa7O1PpVDHhqlQPI2NvRWUxYSaJ1Xy cRe+sdsVKQklZW+DrXWVerIOVeYNdY+MrprQ0gnJ61V5MTWW470fUxh8Th5xNFpKQjbD BXhUOmAuzRI6SWkUDYX/TJOfYy2SsM22PuhtRfAeit4aVyv0LDUrRNcD7S9aM0rqJRHe ceJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=NMRXhnsvrzV66mwuvn+yjKr6tiCVecGx9fVamg6iGnQ=; fh=l8BBgSaHK1qPkRTUMT3LasR23D+FgYCLMIPW5YpagSs=; b=sEP780mOAc5yFJKN7XnO6WZ7LhTo7mzK7BmQLD60v0bAU3VmaVMauIJmpJ+2zIv9gx 8cdtlvuJPrO7wvF3U3T3O56Lrz1VwdRUbmqK5QJUbE3KqAY2+8UJLPhNyDaBp/YmBSK+ gGfoFthAViqKYRDUqJNrRubSBGT8rvBRI7ydtU2BbIEQdxOty2oYEJR7Qd2wyME08yLT g9+5MV17ppDGXzTIaPTtQN6wlUxR+t+1/YxIGd9f4GBRGInlZUnuS+SSRmEE5WDWtX8F FdG13J1AvW34ksL/fgP8OvF0F1+WOjwyivtmk9YVxjf4Re3/oqCtVMfzRHdZPtjRf0J9 2YNg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=jjnQigAj; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id o22-20020a17090611d600b00a58ea693f5dsi5356893eja.408.2024.05.01.11.40.02; Wed, 01 May 2024 11:40:02 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=jjnQigAj; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4E2EE68D6F0; Wed, 1 May 2024 21:38:38 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f53.google.com (mail-ej1-f53.google.com [209.85.218.53]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 547BD68D714 for ; Wed, 1 May 2024 21:38:25 +0300 (EEST) Received: by mail-ej1-f53.google.com with SMTP id a640c23a62f3a-a51a7d4466bso768442666b.2 for ; Wed, 01 May 2024 11:38:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1714588704; x=1715193504; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hcDXzaZhX62X2LsEXIJuSyR8Pku9XshTxW4OT1N4G9A=; b=jjnQigAjMf6L3Vo5bB2BbkAAbHLcWJ79AWt/iT3+dEhAT0Y4Vzoq74j1CL1z+H5maj eOu/seRKu7Mz/hBdm7rKVAApQ4kotixhDlzDwJeW82QTqmLst5dvgP/i1ti+VOyOKO6y e753F8vEbtlleqTZvoyUJItfUU1wela6HyLUuBjqO5rrhJePTuY/uzTGufqnYSsbLsio t3eKDAS0cHao9oP/4lTDxwZ21KzEgDuq6VbEnty1L2zsTBqi2z5N3GhFBnFAx+c2KvEl aufHclfkxATPbR1tinnP8vUOZtBfvWuqt9xHyzYys4EYYOqg4w05dVyg4742ni3egvqT fxwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714588704; x=1715193504; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hcDXzaZhX62X2LsEXIJuSyR8Pku9XshTxW4OT1N4G9A=; b=ka4g+YGst42Hbnyb78TUhZIbs8lmTOx8RWaGie1wplCVtLAwwtbsGwOPbGEW651puV c9o83AVietDtIKc+ujWahUw25Y4rob4c99qNaEQeTQ2tKeYPiEht5Vms1iWncJXTxtA1 FRvxddeiF3S8wyXkullxZvhm+IJUzuaSse2zmYetOf1aQU3MJm4XsnPxC27RKVdkCswD ls2NLKRFN0Lkp7kihh1JpOzXUOmdh42EXMQQuRKnPKxV7OUjatbTflnxaWMjxQwZ9Aoe XWuYVfSh7wp/26HsXsvH2PERcIu4iLDFRUN5n9IFK2eEkP6sxMLzC6rYJafgqXbF966a okww== X-Gm-Message-State: AOJu0Ywav32SlYzZATnl8a4SCJ8bNpjSwZprm4V2nUulqg5zGyZRdhbI 3bXf184spAh0wr1FHLOwJLwm2G2g574G+JI40aMIiJ+ZcBxiss58S/hoVuCd X-Received: by 2002:a17:906:3fc7:b0:a55:aeb0:cb22 with SMTP id k7-20020a1709063fc700b00a55aeb0cb22mr2096225ejj.9.1714588704233; Wed, 01 May 2024 11:38:24 -0700 (PDT) Received: from dovchinn.amd.com (cable-178-148-16-149.dynamic.sbb.rs. [178.148.16.149]) by smtp.gmail.com with ESMTPSA id kh20-20020a170906f81400b00a58bf434876sm7367337ejb.58.2024.05.01.11.38.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 May 2024 11:38:23 -0700 (PDT) From: Dmitrii Ovchinnikov To: ffmpeg-devel@ffmpeg.org Date: Wed, 1 May 2024 20:38:09 +0200 Message-Id: <20240501183809.1060-10-ovchinnikov.dmitrii@gmail.com> X-Mailer: git-send-email 2.38.1.windows.1 In-Reply-To: <20240501183809.1060-1-ovchinnikov.dmitrii@gmail.com> References: <20240501183809.1060-1-ovchinnikov.dmitrii@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 10/10, v2] doc/filters: Add documentation for AMF filters X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Evgeny Pavlov Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: EwJ+VFQ48MBN From: Evgeny Pavlov Signed-off-by: Evgeny Pavlov --- doc/filters.texi | 238 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 238 insertions(+) diff --git a/doc/filters.texi b/doc/filters.texi index fc813f12c1..81dad32ce9 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -22822,6 +22822,76 @@ input upscaled using bicubic upscaling with proper scale factor. To get full functionality (such as async execution), please use the @ref{dnn_processing} filter. +@anchor{sr_amf} +@section sr_amf + +Upscale (size increasing) for the input video using AMD Advanced Media Framework library for hardware acceleration. +Use advanced algorithms for upscaling with higher output quality. +Setting the output width and height works in the same way as for the @ref{scale} filter. + +The filter accepts the following options: +@table @option +@item w +@item h +Set the output video dimension expression. Default value is the input dimension. + +Allows for the same expressions as the @ref{scale} filter. + +@item algorithm +Sets the algorithm used for scaling: + +@table @var +@item bilinear +Bilinear + +@item bicubic +Bicubic + +@item sr1-0 +Video SR1.0 +This is a default value + +@item point +Point + +@item sr1-1 +Video SR1.1 + +@end table + +@item sharpness +Control hq scaler sharpening. The value is a float in the range of [0.0, 2.0] + +@item format +Controls the output pixel format. By default, or if none is specified, the input +pixel format is used. + +@item keep-ratio +Force the scaler to keep the aspect ratio of the input image when the output size has a different aspect ratio. +Default value is false. + +@item fill +Specifies whether the output image outside the region of interest, +which does not fill the entire output surface should be filled with a solid color. + +@end table + +@subsection Examples + +@itemize +@item +Scale input to 720p, keeping aspect ratio and ensuring the output is yuv420p. +@example +sr_amf=-2:720:format=yuv420p +@end example + +@item +Upscale to 4K with algorithm video SR1.1. +@example +sr_amf=4096:2160:algorithm=sr1-1 +@end example +@end itemize + @section ssim Obtain the SSIM (Structural SImilarity Metric) between two input videos. @@ -25559,6 +25629,174 @@ Example: ffmpeg -i ref.mpg -vf vmafmotion -f null - @end example +@anchor{vpp_amf} +@section vpp_amf + +Scale (resize) and convert colorspace, transfer characteristics or color primaries for the input video, using AMD Advanced Media Framework library for hardware acceleration. +Setting the output width and height works in the same way as for the @ref{scale} filter. + +The filter accepts the following options: +@table @option +@item w +@item h +Set the output video dimension expression. Default value is the input dimension. + +Allows for the same expressions as the @ref{scale} filter. + +@item scale_type +Sets the algorithm used for scaling: + +@table @var +@item bilinear +Bilinear + +This is the default. + +@item bicubic +Bicubic + +@end table + +@item format +Controls the output pixel format. By default, or if none is specified, the input +pixel format is used. + + +@item force_original_aspect_ratio +@item force_divisible_by +Work the same as the identical @ref{scale} filter options. + +@anchor{color_profile} +@item color_profile +Specify all color properties at once. + +The accepted values are: +@table @samp +@item bt601 +BT.601 + +@item bt709 +BT.709 + +@item bt2020 +BT.2020 + +@end table + +@anchor{trc} +@item trc +Specify output transfer characteristics. + +The accepted values are: +@table @samp +@item bt709 +BT.709 + +@item gamma22 +Constant gamma of 2.2 + +@item gamma28 +Constant gamma of 2.8 + +@item smpte170m +SMPTE-170M + +@item smpte240m +SMPTE-240M + +@item linear +Linear + +@item log +LOG + +@item log-sqrt +LOG_SQRT + +@item iec61966-2-4 +iec61966-2-4 + +@item bt1361-ecg +BT1361_ECG + +@item iec61966-2-1 +iec61966-2-1 + +@item bt2020-10 +BT.2020 for 10-bits content + +@item bt2020-12 +BT.2020 for 12-bits content + +@item smpte2084 +SMPTE2084 + +@item smpte428 +SMPTE428 + +@item arib-std-b67 +ARIB_STD_B67 + +@end table + +@anchor{primaries} +@item primaries +Specify output color primaries. + +The accepted values are: +@table @samp +@item bt709 +BT.709 + +@item bt470m +BT.470M + +@item bt470bg +BT.470BG or BT.601-6 625 + +@item smpte170m +SMPTE-170M or BT.601-6 525 + +@item smpte240m +SMPTE-240M + +@item film +film + +@item bt2020 +BT.2020 + +@item smpte428 +SMPTE-428 + +@item smpte431 +SMPTE-431 + +@item smpte432 +SMPTE-432 + +@item jedec-p22 +JEDEC P22 phosphors + +@end table +@end table + +@subsection Examples + +@itemize +@item +Scale input to 720p, keeping aspect ratio and ensuring the output is yuv420p. +@example +vpp_amf=-2:720:format=yuv420p +@end example + +@item +Upscale to 4K and change color profile to bt2020. +@example +vpp_amf=4096:2160:color_profile=bt2020 +@end example +@end itemize + @anchor{vstack} @section vstack Stack input videos vertically.