From patchwork Wed Feb 14 01:55:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitrii Ovchinnikov X-Patchwork-Id: 46250 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:c493:b0:19e:cdac:8cce with SMTP id eo19csp110126pzb; Tue, 13 Feb 2024 17:55:43 -0800 (PST) X-Forwarded-Encrypted: i=2; AJvYcCWkHGRJvDOfV0HmkTSbMgqoVo/OyGn097PN1PH1nKPt8Ilt1/IF507MTMM3rarh6VQI7fkKgedsdAgN2rggN1ol73Lpgj+cAp7r5Q== X-Google-Smtp-Source: AGHT+IHcQxY81ScvTx+sYpE8sk0slom5a+xkOhdEMR1pPUouTniOQ+o0PCz+4+MH0ojHyCoBdxrC X-Received: by 2002:a05:651c:170c:b0:2d0:b3c3:d8d8 with SMTP id be12-20020a05651c170c00b002d0b3c3d8d8mr838239ljb.43.1707875742788; Tue, 13 Feb 2024 17:55:42 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1707875742; cv=none; d=google.com; s=arc-20160816; b=TDdZvmvOkGT9QwIl0ey7av/B+/Po4SjJjWXbPc3pw6sUUMn6cYugiLxZiflMTOjJuY w/FV0IZVUGnZgNeK56BPC65iTxGTRSldI4RXnTum+Nes94cLhFgtxsJAyF3RLs8Eg8vL CzB0C7E4xaGOY2x4jFvIpopcyV6IuhunLbgemoIwqMTo07UzWKWlDlWWJuA7LjZykqIH gXY9Ug0lFHG1xmIiSVGqVvHaTzbjCgH9susEnV2POTQqWnGWzIXyT1+5Ufe0HUhS26Iq XnHy+QIVBbkVnaBPZqIFEOvkdSstiXfZ16UmYexOE0t1ZWjeTQvYDJ3T4CNHIyh14uVx G3dg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=UoMHZZTS/zHgC+vcAWP+lIsP8NwenwJ6znmhy1IqEck=; fh=KjpWoOWEUNjfW6NaDUgHhoIxGngXHNPS0MKMzFleHno=; b=dmo8kRyYY1DlEHZ2rH+6QI1A9A3jNrzT6LtcLMfdUpabYic66/o0JA1JtSc3MNuqIc Z4tg5os3qcAipZrjWnYj0voT91LK/KWGATkP509kMy28EY07WJLlkbz6wO0vv/eh4Euu mQ97ZBrKlyZsAp15C2aHYeYnI0R1QrlXc3A3CCcmPp3t3QJTJD4hZQyY1A6pcQjDhNNp lGeXTBu+TOaPJwlq9swcv39Y3nxZpveeHJkyZBu6dYi47fwtDs2CDoBq2rGVVTShY8NI SkPLRaB44t5r3YELhHod96vNvVG90cPy6TYpKyfrO/uZ6IGD8VUKX8g0zlSm0LhZuRph OYhg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=mXTuGXYy; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com X-Forwarded-Encrypted: i=1; AJvYcCXV+M3n7O8GltOiayK4gKXD8xZJn8zxOjx5UJ9SVQV2RtF8hfxyqKkF+9OFN9jBglyyQ8JNMmGg05LLMiXyidj9tCas7N3HTh7DMGnu32eReJt18NaMtq5cPLCH7gxWcneBx+I+APGMik6/uoDFJnXcWfZuBf7PY7j1yEWpUaKFk6wG0GuUjnQ8qoLC+j7e3jX56y64Zl/ZToyWJ+s4wx8aNhzPimQ/DCgaDBwRPlTiV4mxZcRon/Zq96591DEisJTyTuhMowdFcZrXVOrLIBv2/PqvG+RDDK8l9ZAlJ+3XJsnB7RB9tQ+Rt8gorkktUVwyc5uaa+rupvWYQUYuK2pu+Evb/UmEMryl73/Q+TD/arLn/rmGmvcqp46pENd4BNh+5Q25z0+3WAzyzJyvWGbWkp+kRQZWG9dzz4JAjizijNfamaoxpapCq48jQ+IWTs4tU6GTMF7UzsM9lARmhycMHaSz8pjhn4+g7UUM8sFWtpZQoUB6RaW5aa/PrNxNlz2WJdgxAJwW7BSDZ8tjRKan2seopkRusWqSXmKEo/l1S/ZEyh2+GFTsaNzlvddsoT2MdXInTIx37nw+BKv++JBhYCKP3D9VCsO3X5vrvKGUSycnzuM8Iu7/fm5nZkiGxYblwt7ekt2DxUXUBVwiHzKlqaZQfd4g/BqsPvd+NWSa14YVp0TDvcnK6EYLQttkuLWgUYmTRRg4vb80elj/x4hTrkVdJ0zemW/HsLOnnsdusJZ1PQQiZqsp3UWf8UmTCJXVqUZFsTdfINZYj9tXt4J+pgtzaqHXEvw5tx4wTdh9dS9fA58HpqT8G6d2/qurLBZ9Np+e+MLj1jQ7LecBJmkFZn+Zvx8p5UQtse+cXOuYTLLWU7qjVlfifVba8zunUQ7t7hhUZgwRgkBxrRlJpjM9zISYYwJO3p9MmRjt7U5ys6IrO3JDXuiRY+NWygryBH6QK1 CJNcN3fVcacmL44APRbx6lkOKTOsM/cLimi/cz4/J3QMWycL4+3zzXdNkEBrAKNJkNJiX5SdMZoobjWJ6O9suoNina1brxJybRQD9gu2rMxF9h9nBhLAX3QEADkSQjMuzGPh/P1Cp9meU5yX1B72aegfwLKANW+DUkocdpCsBZMeBC0qckdLkYEEM1htt6xZMcByAIAy5QDPcH6eCX1eMlEw2VVYaw1WRX7mwzjJJ9/HEMD4P7FDlYNUWR7wEz5djt5LUGEy3WlKUdEqnY1Qa2XaJaG/TL0BOk+L6Uyh8GrGoPPFWtpVk4Z0KyNOGHkIddLH7iCvwGJ/ne41FYHEZkKoUa3vFyfgkpqe26bkb5/M3CfJ4EpeZYm/Dkft//LpqfZ5hP1viuRws2f58LRGQ28YGaUNLI/7JMQZZbeCoh25npV+xvP01G34DZhqupayYNuM/RSvUJDQzhhEs0f3KdOvgmynnSAyvnlTtiUGPUwDCoXbc0DtuRgYpr5j+DVaR8+KqIXcxAZRrMFk4dGQaHYTnv6zTLKC+qsqnLxqAsVZJ7wmMI816CoFiRDm7ouv0kwBhdn13OmmRdNfGAv4Ke2e/tt1ijm14ojYt/sKOaAOwSdevlLzqh5vy1cc0= Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id s16-20020aa7c550000000b0055eef1ceb3dsi4303191edr.197.2024.02.13.17.55.42; Tue, 13 Feb 2024 17:55:42 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=mXTuGXYy; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6C27968D185; Wed, 14 Feb 2024 03:55:38 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f54.google.com (mail-ej1-f54.google.com [209.85.218.54]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7925368D098 for ; Wed, 14 Feb 2024 03:55:31 +0200 (EET) Received: by mail-ej1-f54.google.com with SMTP id a640c23a62f3a-a29c4bbb2f4so658603066b.1 for ; Tue, 13 Feb 2024 17:55:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1707875730; x=1708480530; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=Ho4/GocOSY0UQlB9iSFl9hxAaJ//24POI+zE7nLkxts=; b=mXTuGXYyWjkZydyczTfKMEev/61rJqPcnDMFC//Obv4/lxIVn+Qgj/oVvB1qBd79aS 1bZYsgRWaKPsub2+LdlCy2/I7bJsOVMoyJOTQnSuOC3f0uepivU8R6c3bQxjsWFdkBAg e+6kaK3nYJK0UP3aBHkOOLWgsKBUnsW0fTbGuQpqQXd8ukfSXAAxUF+Xds5Swzdun8OU KUSoBqZDo8a4+isfkYT74wgqxC3otxAR1YpuqfxhaQmOw89arGgH8yAMr6grvd53FxMP hEY5ZQPk/MdMffFMKl+6vCuPa7R8BHXbATK749dkbY+uE1EoksPiEJKjxsgm9mGZwdjC SGgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707875730; x=1708480530; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Ho4/GocOSY0UQlB9iSFl9hxAaJ//24POI+zE7nLkxts=; b=c3bpHDYAheQ2urPbojYX9BBTEFweiRJJULIax09+X+7iiOdbkRBQla3L/fTpqHGbUg KuUrasUvUAj68Pg5M+jpWuUZnfSSaUZNIdvWurHZOHb43rgvwBjtZ+GewogyxGld8R81 BFsIxsRl9Gv/cC7UX0DlV0DwvYlGrDIQtphWYbiVzuVgpqzeboHohxoBqKFG/ux4t3f7 QnblwIppFb3Kt3BHN4ceEuqf+1fJFNM+zSmpIMbP5PEgMaJKr2PE5iN8WEkooL2sjYMf XbaJryTfxl4GPTgCOQrpc3gP+nCnhgK021x0tA1Nisjue6jnbreKxIFyVZxh0bNjeIIJ XYNw== X-Gm-Message-State: AOJu0Yxy3nCjaC4QuAk3vLPEaCIif2S+Z2l3N3dcaoZZ6FG7/II9zN5K 30JWsgfxrF/7qT0hzXo0X5BXLyceVUNcBvGfswkATUMll7ahFdhtq/xF14Gd X-Received: by 2002:a17:906:6d0b:b0:a38:a2f:c131 with SMTP id m11-20020a1709066d0b00b00a380a2fc131mr702615ejr.43.1707875729869; Tue, 13 Feb 2024 17:55:29 -0800 (PST) Received: from dovchinn.amd.com (cable-178-148-16-149.dynamic.sbb.rs. [178.148.16.149]) by smtp.gmail.com with ESMTPSA id vi16-20020a170907d41000b00a3ce31d3ffdsm1663230ejc.93.2024.02.13.17.55.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 17:55:29 -0800 (PST) From: Dmitrii Ovchinnikov To: ffmpeg-devel@ffmpeg.org Date: Wed, 14 Feb 2024 02:55:07 +0100 Message-Id: <20240214015515.1027-1-ovchinnikov.dmitrii@gmail.com> X-Mailer: git-send-email 2.38.1.windows.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/9] libavutil: add hwcontext_amf. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Dmitrii Ovchinnikov Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: bqpCuy4o04qN Adds hwcontext_amf, which allows to use shared AMF context for the encoder, decoder and AMF-based filters, without copy to the host memory. It will also allow you to use some optimizations in the interaction of components (for example, SAV) and make a more manageable and optimal setup for using GPU devices with AMF in the case of a fully AMF pipeline. It will be a significant performance uplift when full AMF pipeline with filters is used. We also plan to add Compression artefact removal filter in near feature. --- libavutil/Makefile | 3 + libavutil/hwcontext.c | 4 + libavutil/hwcontext.h | 1 + libavutil/hwcontext_amf.c | 580 +++++++++++++++++++++++++++++++++ libavutil/hwcontext_amf.h | 105 ++++++ libavutil/hwcontext_internal.h | 1 + libavutil/pixdesc.c | 4 + libavutil/pixfmt.h | 4 + 8 files changed, 702 insertions(+) create mode 100644 libavutil/hwcontext_amf.c create mode 100644 libavutil/hwcontext_amf.h diff --git a/libavutil/Makefile b/libavutil/Makefile index e7709b97d0..785af8e926 100644 --- a/libavutil/Makefile +++ b/libavutil/Makefile @@ -45,6 +45,7 @@ HEADERS = adler32.h \ hwcontext_d3d12va.h \ hwcontext_drm.h \ hwcontext_dxva2.h \ + hwcontext_amf.h \ hwcontext_qsv.h \ hwcontext_mediacodec.h \ hwcontext_opencl.h \ @@ -195,6 +196,7 @@ OBJS-$(CONFIG_CUDA) += hwcontext_cuda.o OBJS-$(CONFIG_D3D11VA) += hwcontext_d3d11va.o OBJS-$(CONFIG_D3D12VA) += hwcontext_d3d12va.o OBJS-$(CONFIG_DXVA2) += hwcontext_dxva2.o +OBJS-$(CONFIG_AMF) += hwcontext_amf.o OBJS-$(CONFIG_LIBDRM) += hwcontext_drm.o OBJS-$(CONFIG_MACOS_KPERF) += macos_kperf.o OBJS-$(CONFIG_MEDIACODEC) += hwcontext_mediacodec.o @@ -219,6 +221,7 @@ SKIPHEADERS-$(CONFIG_CUDA) += hwcontext_cuda_internal.h \ SKIPHEADERS-$(CONFIG_D3D11VA) += hwcontext_d3d11va.h SKIPHEADERS-$(CONFIG_D3D12VA) += hwcontext_d3d12va.h SKIPHEADERS-$(CONFIG_DXVA2) += hwcontext_dxva2.h +SKIPHEADERS-$(CONFIG_AMF) += hwcontext_amf.h SKIPHEADERS-$(CONFIG_QSV) += hwcontext_qsv.h SKIPHEADERS-$(CONFIG_OPENCL) += hwcontext_opencl.h SKIPHEADERS-$(CONFIG_VAAPI) += hwcontext_vaapi.h diff --git a/libavutil/hwcontext.c b/libavutil/hwcontext.c index e23bad230f..8c53fc1862 100644 --- a/libavutil/hwcontext.c +++ b/libavutil/hwcontext.c @@ -65,6 +65,9 @@ static const HWContextType * const hw_table[] = { #endif #if CONFIG_VULKAN &ff_hwcontext_type_vulkan, +#endif +#if CONFIG_AMF + &ff_hwcontext_type_amf, #endif NULL, }; @@ -82,6 +85,7 @@ static const char *const hw_type_names[] = { [AV_HWDEVICE_TYPE_VIDEOTOOLBOX] = "videotoolbox", [AV_HWDEVICE_TYPE_MEDIACODEC] = "mediacodec", [AV_HWDEVICE_TYPE_VULKAN] = "vulkan", + [AV_HWDEVICE_TYPE_AMF] = "amf", }; enum AVHWDeviceType av_hwdevice_find_type_by_name(const char *name) diff --git a/libavutil/hwcontext.h b/libavutil/hwcontext.h index 2b33721a97..53292ea67f 100644 --- a/libavutil/hwcontext.h +++ b/libavutil/hwcontext.h @@ -38,6 +38,7 @@ enum AVHWDeviceType { AV_HWDEVICE_TYPE_MEDIACODEC, AV_HWDEVICE_TYPE_VULKAN, AV_HWDEVICE_TYPE_D3D12VA, + AV_HWDEVICE_TYPE_AMF, }; typedef struct AVHWDeviceInternal AVHWDeviceInternal; diff --git a/libavutil/hwcontext_amf.c b/libavutil/hwcontext_amf.c new file mode 100644 index 0000000000..6f43780dc5 --- /dev/null +++ b/libavutil/hwcontext_amf.c @@ -0,0 +1,580 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "buffer.h" +#include "common.h" +#include "hwcontext.h" +#include "hwcontext_amf.h" +#include "hwcontext_internal.h" +#if CONFIG_VULKAN +#include "hwcontext_vulkan.h" +#endif +#if CONFIG_D3D11VA +#include "libavutil/hwcontext_d3d11va.h" +#endif +#if CONFIG_DXVA2 +#define COBJMACROS +#include "libavutil/hwcontext_dxva2.h" +#endif +#include "mem.h" +#include "pixdesc.h" +#include "pixfmt.h" +#include "imgutils.h" +#include "libavutil/avassert.h" +#include +#ifdef _WIN32 +#include "compat/w32dlfcn.h" +#else +#include +#endif + +static const AVClass amflib_class = { + .class_name = "amf", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, +}; + +typedef struct AMFLibraryContext { + const AVClass *avclass; +} AMFLibraryContext; + +static AMFLibraryContext amflib_context = +{ + .avclass = &amflib_class, +}; + + +static void AMF_CDECL_CALL AMFTraceWriter_Write(AMFTraceWriter *pThis, + const wchar_t *scope, const wchar_t *message) +{ + AmfTraceWriter *tracer = (AmfTraceWriter*)pThis; + av_log(tracer->avcl, AV_LOG_DEBUG, "%ls: %ls", scope, message); // \n is provided from AMF +} + +static void AMF_CDECL_CALL AMFTraceWriter_Flush(AMFTraceWriter *pThis) +{ +} + +static AMFTraceWriterVtbl tracer_vtbl = +{ + .Write = AMFTraceWriter_Write, + .Flush = AMFTraceWriter_Flush, +}; + +AmfTraceWriter av_amf_trace_writer = +{ + .vtbl = &tracer_vtbl, + .avcl = &amflib_context, + .avctx = NULL +}; + +const FormatMap format_map[] = +{ + { AV_PIX_FMT_NONE, AMF_SURFACE_UNKNOWN }, + { AV_PIX_FMT_NV12, AMF_SURFACE_NV12 }, + { AV_PIX_FMT_BGR0, AMF_SURFACE_BGRA }, + { AV_PIX_FMT_RGB0, AMF_SURFACE_RGBA }, + { AV_PIX_FMT_BGRA, AMF_SURFACE_BGRA }, + { AV_PIX_FMT_ARGB, AMF_SURFACE_ARGB }, + { AV_PIX_FMT_RGBA, AMF_SURFACE_RGBA }, + { AV_PIX_FMT_GRAY8, AMF_SURFACE_GRAY8 }, + { AV_PIX_FMT_YUV420P, AMF_SURFACE_YUV420P }, + { AV_PIX_FMT_YUYV422, AMF_SURFACE_YUY2 }, + { AV_PIX_FMT_P010, AMF_SURFACE_P010 }, + { AV_PIX_FMT_YUV420P10, AMF_SURFACE_P010 }, + { AV_PIX_FMT_YUV420P12, AMF_SURFACE_P012 }, + { AV_PIX_FMT_YUV420P12, AMF_SURFACE_P012 }, + { AV_PIX_FMT_YUV420P16, AMF_SURFACE_P016 }, + { AV_PIX_FMT_YUV422P10LE, AMF_SURFACE_Y210 }, + { AV_PIX_FMT_YUV444P10LE, AMF_SURFACE_Y416 }, +}; + +enum AMF_SURFACE_FORMAT av_amf_av_to_amf_format(enum AVPixelFormat fmt) +{ + int i; + for (i = 0; i < amf_countof(format_map); i++) { + if (format_map[i].av_format == fmt) { + return format_map[i].amf_format; + } + } + return AMF_SURFACE_UNKNOWN; +} + +enum AVPixelFormat av_amf_to_av_format(enum AMF_SURFACE_FORMAT fmt) +{ + int i; + for (i = 0; i < amf_countof(format_map); i++) { + if (format_map[i].amf_format == fmt) { + return format_map[i].av_format; + } + } + return AMF_SURFACE_UNKNOWN; +} + +static const enum AVPixelFormat supported_formats[] = { + AV_PIX_FMT_NV12, + AV_PIX_FMT_YUV420P, + AV_PIX_FMT_BGRA, + AV_PIX_FMT_P010, + AV_PIX_FMT_YUV420P10, +#if CONFIG_D3D11VA + AV_PIX_FMT_D3D11, +#endif +#if CONFIG_DXVA2 + AV_PIX_FMT_DXVA2_VLD, +#endif + AV_PIX_FMT_AMF +}; + +static int amf_frames_get_constraints(AVHWDeviceContext *ctx, + const void *hwconfig, + AVHWFramesConstraints *constraints) +{ + int i; + + constraints->valid_sw_formats = av_malloc_array(FF_ARRAY_ELEMS(supported_formats) + 1, + sizeof(*constraints->valid_sw_formats)); + if (!constraints->valid_sw_formats) + return AVERROR(ENOMEM); + + for (i = 0; i < FF_ARRAY_ELEMS(supported_formats); i++) + constraints->valid_sw_formats[i] = supported_formats[i]; + constraints->valid_sw_formats[FF_ARRAY_ELEMS(supported_formats)] = AV_PIX_FMT_NONE; + + constraints->valid_hw_formats = av_malloc_array(2, sizeof(*constraints->valid_hw_formats)); + if (!constraints->valid_hw_formats) + return AVERROR(ENOMEM); + + constraints->valid_hw_formats[0] = AV_PIX_FMT_AMF; + constraints->valid_hw_formats[1] = AV_PIX_FMT_NONE; + + return 0; +} + +static void amf_dummy_free(void *opaque, uint8_t *data) +{ + +} + +static AVBufferRef *amf_pool_alloc(void *opaque, size_t size) +{ + AVHWFramesContext *hwfc = (AVHWFramesContext *)opaque; + AVBufferRef *buf; + + buf = av_buffer_create(NULL, NULL, amf_dummy_free, hwfc, AV_BUFFER_FLAG_READONLY); + if (!buf) { + av_log(hwfc, AV_LOG_ERROR, "Failed to create buffer for AMF context.\n"); + return NULL; + } + return buf; +} + +static int amf_frames_init(AVHWFramesContext *ctx) +{ + int i; + + for (i = 0; i < FF_ARRAY_ELEMS(supported_formats); i++) { + if (ctx->sw_format == supported_formats[i]) + break; + } + if (i == FF_ARRAY_ELEMS(supported_formats)) { + av_log(ctx, AV_LOG_ERROR, "Pixel format '%s' is not supported\n", + av_get_pix_fmt_name(ctx->sw_format)); + return AVERROR(ENOSYS); + } + + ctx->internal->pool_internal = + av_buffer_pool_init2(sizeof(AMFSurface), ctx, + &amf_pool_alloc, NULL); + + + return 0; +} + +static int amf_get_buffer(AVHWFramesContext *ctx, AVFrame *frame) +{ + frame->buf[0] = av_buffer_pool_get(ctx->pool); + if (!frame->buf[0]) + return AVERROR(ENOMEM); + + frame->data[3] = frame->buf[0]->data; + frame->format = AV_PIX_FMT_AMF; + frame->width = ctx->width; + frame->height = ctx->height; + return 0; +} + +static int amf_transfer_get_formats(AVHWFramesContext *ctx, + enum AVHWFrameTransferDirection dir, + enum AVPixelFormat **formats) +{ + enum AVPixelFormat *fmts; + + fmts = av_malloc_array(2, sizeof(*fmts)); + if (!fmts) + return AVERROR(ENOMEM); + + fmts[0] = ctx->sw_format; + fmts[1] = AV_PIX_FMT_NONE; + + *formats = fmts; + + return 0; +} + +static int amf_transfer_data_to(AVHWFramesContext *ctx, AVFrame *dst, + const AVFrame *src) +{ + AMFSurface* surface = (AMFSurface*)dst->data[3]; + AMFPlane *plane; + uint8_t *dst_data[4]; + int dst_linesize[4]; + int planes; + int i; + int w = FFMIN(dst->width, src->width); + int h = FFMIN(dst->height, src->height); + + planes = (int)surface->pVtbl->GetPlanesCount(surface); + av_assert0(planes < FF_ARRAY_ELEMS(dst_data)); + + for (i = 0; i < planes; i++) { + plane = surface->pVtbl->GetPlaneAt(surface, i); + dst_data[i] = plane->pVtbl->GetNative(plane); + dst_linesize[i] = plane->pVtbl->GetHPitch(plane); + } + av_image_copy(dst_data, dst_linesize, + (const uint8_t**)src->data, src->linesize, src->format, + w, h); + + return 0; +} +static int amf_transfer_data_from(AVHWFramesContext *ctx, AVFrame *dst, + const AVFrame *src) +{ + AMFSurface* surface = (AMFSurface*)src->data[3]; + AMFPlane *plane; + uint8_t *src_data[4]; + int src_linesize[4]; + int planes; + int i; + int w = FFMIN(dst->width, src->width); + int h = FFMIN(dst->height, src->height); + int ret; + + ret = surface->pVtbl->Convert(surface, AMF_MEMORY_HOST); + AMF_RETURN_IF_FALSE(ctx, ret == AMF_OK, AVERROR_UNKNOWN, "Convert(amf::AMF_MEMORY_HOST) failed with error %d\n", AVERROR_UNKNOWN); + + planes = (int)surface->pVtbl->GetPlanesCount(surface); + av_assert0(planes < FF_ARRAY_ELEMS(src_data)); + + for (i = 0; i < planes; i++) { + plane = surface->pVtbl->GetPlaneAt(surface, i); + src_data[i] = plane->pVtbl->GetNative(plane); + src_linesize[i] = plane->pVtbl->GetHPitch(plane); + } + av_image_copy(dst->data, dst->linesize, + (const uint8_t **)src_data, src_linesize, dst->format, + w, h); + surface->pVtbl->Release(surface); + return 0; +} + + +static void amf_device_uninit(AVHWDeviceContext *device_ctx) +{ + AVAMFDeviceContext *amf_ctx = device_ctx->hwctx; + av_buffer_unref(&amf_ctx->internal); +} + +static int amf_device_init(AVHWDeviceContext *ctx) +{ + AVAMFDeviceContext *amf_ctx = ctx->hwctx; + return av_amf_context_init((AVAMFDeviceContextInternal * )amf_ctx->internal->data, ctx); +} + +static int amf_device_create(AVHWDeviceContext *device_ctx, + const char *device, + AVDictionary *opts, int flags) +{ + AVAMFDeviceContext *ctx = device_ctx->hwctx; + AVAMFDeviceContextInternal *wrapped = av_mallocz(sizeof(*wrapped)); + ctx->internal = av_buffer_create((uint8_t *)wrapped, sizeof(*wrapped), + av_amf_context_internal_free, NULL, 0); + AVAMFDeviceContextInternal * internal = (AVAMFDeviceContextInternal * )ctx->internal->data; + int ret; + if ((ret = av_amf_load_library(internal, device_ctx)) == 0) { + if ((ret = av_amf_create_context(internal, device_ctx, "", opts, flags)) == 0){ + return 0; + } + } + amf_device_uninit(device_ctx); + return ret; +} + +static int amf_device_derive(AVHWDeviceContext *device_ctx, + AVHWDeviceContext *child_device_ctx, AVDictionary *opts, + int flags) +{ + AVAMFDeviceContext *ctx = device_ctx->hwctx; + AVAMFDeviceContextInternal * internal = (AVAMFDeviceContextInternal * )ctx->internal->data; + int ret; + + ret = amf_device_create(device_ctx, "", opts, flags); + if(ret < 0) + return ret; + + return av_amf_context_derive(internal, child_device_ctx, opts, flags); +} + +#if CONFIG_DXVA2 +static int amf_init_from_dxva2_device(AVAMFDeviceContextInternal * internal, AVDXVA2DeviceContext *hwctx) +{ + IDirect3DDevice9 *device; + HANDLE device_handle; + HRESULT hr; + AMF_RESULT res; + int ret; + + hr = IDirect3DDeviceManager9_OpenDeviceHandle(hwctx->devmgr, &device_handle); + if (FAILED(hr)) { + av_log(hwctx, AV_LOG_ERROR, "Failed to open device handle for Direct3D9 device: %lx.\n", (unsigned long)hr); + return AVERROR_EXTERNAL; + } + + hr = IDirect3DDeviceManager9_LockDevice(hwctx->devmgr, device_handle, &device, FALSE); + if (SUCCEEDED(hr)) { + IDirect3DDeviceManager9_UnlockDevice(hwctx->devmgr, device_handle, FALSE); + ret = 0; + } else { + av_log(hwctx, AV_LOG_ERROR, "Failed to lock device handle for Direct3D9 device: %lx.\n", (unsigned long)hr); + ret = AVERROR_EXTERNAL; + } + + + IDirect3DDeviceManager9_CloseDeviceHandle(hwctx->devmgr, device_handle); + + if (ret < 0) + return ret; + + res = internal->context->pVtbl->InitDX9(internal->context, device); + + IDirect3DDevice9_Release(device); + + if (res != AMF_OK) { + if (res == AMF_NOT_SUPPORTED) + av_log(hwctx, AV_LOG_ERROR, "AMF via D3D9 is not supported on the given device.\n"); + else + av_log(hwctx, AV_LOG_ERROR, "AMF failed to initialise on given D3D9 device: %d.\n", res); + return AVERROR(ENODEV); + } + internal->mem_type = AMF_MEMORY_DX9; + return 0; +} +#endif + +#if CONFIG_D3D11VA +static int amf_init_from_d3d11_device(AVAMFDeviceContextInternal* internal, AVD3D11VADeviceContext *hwctx) +{ + AMF_RESULT res; + res = internal->context->pVtbl->InitDX11(internal->context, hwctx->device, AMF_DX11_1); + if (res != AMF_OK) { + if (res == AMF_NOT_SUPPORTED) + av_log(hwctx, AV_LOG_ERROR, "AMF via D3D11 is not supported on the given device.\n"); + else + av_log(hwctx, AV_LOG_ERROR, "AMF failed to initialise on the given D3D11 device: %d.\n", res); + return AVERROR(ENODEV); + } + internal->mem_type = AMF_MEMORY_DX11; + return 0; +} +#endif + +int av_amf_context_init(AVAMFDeviceContextInternal* internal, void* avcl) +{ + AMFContext1 *context1 = NULL; + AMF_RESULT res; + + res = internal->context->pVtbl->InitDX11(internal->context, NULL, AMF_DX11_1); + if (res == AMF_OK) { + internal->mem_type = AMF_MEMORY_DX11; + av_log(avcl, AV_LOG_VERBOSE, "AMF initialisation succeeded via D3D11.\n"); + } else { + res = internal->context->pVtbl->InitDX9(internal->context, NULL); + if (res == AMF_OK) { + internal->mem_type = AMF_MEMORY_DX9; + av_log(avcl, AV_LOG_VERBOSE, "AMF initialisation succeeded via D3D9.\n"); + } else { + AMFGuid guid = IID_AMFContext1(); + res = internal->context->pVtbl->QueryInterface(internal->context, &guid, (void**)&context1); + AMF_RETURN_IF_FALSE(avcl, res == AMF_OK, AVERROR_UNKNOWN, "CreateContext1() failed with error %d\n", res); + + res = context1->pVtbl->InitVulkan(context1, NULL); + context1->pVtbl->Release(context1); + if (res != AMF_OK) { + if (res == AMF_NOT_SUPPORTED) + av_log(avcl, AV_LOG_ERROR, "AMF via Vulkan is not supported on the given device.\n"); + else + av_log(avcl, AV_LOG_ERROR, "AMF failed to initialise on the given Vulkan device: %d.\n", res); + return AVERROR(ENOSYS); + } + internal->mem_type = AMF_MEMORY_VULKAN; + av_log(avcl, AV_LOG_VERBOSE, "AMF initialisation succeeded via Vulkan.\n"); + } + } + return 0; +} +int av_amf_load_library(AVAMFDeviceContextInternal* internal, void* avcl) +{ + AMFInit_Fn init_fun; + AMFQueryVersion_Fn version_fun; + AMF_RESULT res; + + internal->library = dlopen(AMF_DLL_NAMEA, RTLD_NOW | RTLD_LOCAL); + AMF_RETURN_IF_FALSE(avcl, internal->library != NULL, + AVERROR_UNKNOWN, "DLL %s failed to open\n", AMF_DLL_NAMEA); + + init_fun = (AMFInit_Fn)dlsym(internal->library, AMF_INIT_FUNCTION_NAME); + AMF_RETURN_IF_FALSE(avcl, init_fun != NULL, AVERROR_UNKNOWN, "DLL %s failed to find function %s\n", AMF_DLL_NAMEA, AMF_INIT_FUNCTION_NAME); + + version_fun = (AMFQueryVersion_Fn)dlsym(internal->library, AMF_QUERY_VERSION_FUNCTION_NAME); + AMF_RETURN_IF_FALSE(avcl, version_fun != NULL, AVERROR_UNKNOWN, "DLL %s failed to find function %s\n", AMF_DLL_NAMEA, AMF_QUERY_VERSION_FUNCTION_NAME); + + res = version_fun(&internal->version); + AMF_RETURN_IF_FALSE(avcl, res == AMF_OK, AVERROR_UNKNOWN, "%s failed with error %d\n", AMF_QUERY_VERSION_FUNCTION_NAME, res); + res = init_fun(AMF_FULL_VERSION, &internal->factory); + AMF_RETURN_IF_FALSE(avcl, res == AMF_OK, AVERROR_UNKNOWN, "%s failed with error %d\n", AMF_INIT_FUNCTION_NAME, res); + res = internal->factory->pVtbl->GetTrace(internal->factory, &internal->trace); + AMF_RETURN_IF_FALSE(avcl, res == AMF_OK, AVERROR_UNKNOWN, "GetTrace() failed with error %d\n", res); + res = internal->factory->pVtbl->GetDebug(internal->factory, &internal->debug); + AMF_RETURN_IF_FALSE(avcl, res == AMF_OK, AVERROR_UNKNOWN, "GetDebug() failed with error %d\n", res); + return 0; +} + +int av_amf_create_context( AVAMFDeviceContextInternal * internal, + void* avcl, + const char *device, + AVDictionary *opts, int flags) +{ + AMF_RESULT res; + + internal->trace->pVtbl->EnableWriter(internal->trace, AMF_TRACE_WRITER_CONSOLE, 0); + internal->trace->pVtbl->SetGlobalLevel(internal->trace, AMF_TRACE_TRACE); + + // connect AMF logger to av_log + av_amf_trace_writer.avctx = avcl; + internal->trace->pVtbl->RegisterWriter(internal->trace, FFMPEG_AMF_WRITER_ID, (AMFTraceWriter*)&av_amf_trace_writer, 1); + internal->trace->pVtbl->SetWriterLevel(internal->trace, FFMPEG_AMF_WRITER_ID, AMF_TRACE_TRACE); + + res = internal->factory->pVtbl->CreateContext(internal->factory, &internal->context); + AMF_RETURN_IF_FALSE(avcl, res == AMF_OK, AVERROR_UNKNOWN, "CreateContext() failed with error %d\n", res); + + return 0; +} + +int av_amf_context_internal_create(AVAMFDeviceContextInternal * internal, + void* avcl, + const char *device, + AVDictionary *opts, int flags) +{ + int ret; + if ((ret = av_amf_load_library(internal, avcl)) == 0) { + if ((ret = av_amf_create_context(internal, avcl, "", opts, flags)) == 0){ + return 0; + } + } + av_amf_context_internal_free(0, (uint8_t *)internal); + return ret; +} + +void av_amf_context_internal_free(void *opaque, uint8_t *data) +{ + AVAMFDeviceContextInternal *amf_ctx = (AVAMFDeviceContextInternal *)data; + if (amf_ctx->context) { + amf_ctx->context->pVtbl->Terminate(amf_ctx->context); + amf_ctx->context->pVtbl->Release(amf_ctx->context); + amf_ctx->context = NULL; + } + + if (amf_ctx->trace) { + amf_ctx->trace->pVtbl->UnregisterWriter(amf_ctx->trace, FFMPEG_AMF_WRITER_ID); + } + + if(amf_ctx->library) { + dlclose(amf_ctx->library); + amf_ctx->library = NULL; + } + + amf_ctx->debug = NULL; + amf_ctx->version = 0; + av_free(amf_ctx); +} + +int av_amf_context_derive(AVAMFDeviceContextInternal * internal, + AVHWDeviceContext *child_device_ctx, AVDictionary *opts, + int flags) +{ + + switch (child_device_ctx->type) { + +#if CONFIG_DXVA2 + case AV_HWDEVICE_TYPE_DXVA2: + { + AVDXVA2DeviceContext *child_device_hwctx = child_device_ctx->hwctx; + return amf_init_from_dxva2_device(internal, child_device_hwctx); + } + break; +#endif + +#if CONFIG_D3D11VA + case AV_HWDEVICE_TYPE_D3D11VA: + { + AVD3D11VADeviceContext *child_device_hwctx = child_device_ctx->hwctx; + return amf_init_from_d3d11_device(internal, child_device_hwctx); + } + break; +#endif + default: + { + av_log(child_device_ctx, AV_LOG_ERROR, "AMF initialisation from a %s device is not supported.\n", + av_hwdevice_get_type_name(child_device_ctx->type)); + return AVERROR(ENOSYS); + } + } + return 0; +} + +const HWContextType ff_hwcontext_type_amf = { + .type = AV_HWDEVICE_TYPE_AMF, + .name = "AMF", + + .device_hwctx_size = sizeof(AVAMFDeviceContext), + .frames_priv_size = sizeof(AMFFramesContext), + + .device_create = amf_device_create, + .device_derive = amf_device_derive, + .device_init = amf_device_init, + .device_uninit = amf_device_uninit, + .frames_get_constraints = amf_frames_get_constraints, + .frames_init = amf_frames_init, + .frames_get_buffer = amf_get_buffer, + .transfer_get_formats = amf_transfer_get_formats, + .transfer_data_to = amf_transfer_data_to, + .transfer_data_from = amf_transfer_data_from, + + .pix_fmts = (const enum AVPixelFormat[]){ AV_PIX_FMT_AMF, AV_PIX_FMT_NONE }, +}; diff --git a/libavutil/hwcontext_amf.h b/libavutil/hwcontext_amf.h new file mode 100644 index 0000000000..0161b9a29c --- /dev/null +++ b/libavutil/hwcontext_amf.h @@ -0,0 +1,105 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + + +#ifndef AVUTIL_HWCONTEXT_AMF_H +#define AVUTIL_HWCONTEXT_AMF_H +#include +#include +#include +#include "pixfmt.h" + +#include "libavformat/avformat.h" +#include "libavutil/hwcontext.h" + +#define FFMPEG_AMF_WRITER_ID L"ffmpeg_amf" + +typedef struct AmfTraceWriter { + AMFTraceWriterVtbl *vtbl; + void *avctx; + void *avcl; +} AmfTraceWriter; + +typedef struct AVAMFDeviceContextInternal { + amf_handle library; ///< handle to DLL library + AMFFactory *factory; ///< pointer to AMF factory + AMFDebug *debug; ///< pointer to AMF debug interface + AMFTrace *trace; ///< pointer to AMF trace interface + + amf_uint64 version; ///< version of AMF runtime + AMFContext *context; ///< AMF context + AMF_MEMORY_TYPE mem_type; +} AVAMFDeviceContextInternal; + +/** + * This struct is allocated as AVHWDeviceContext.hwctx + */ + +typedef struct AVAMFDeviceContext { + AVBufferRef *internal; +} AVAMFDeviceContext; + +typedef struct AMFFramesContext { + AMFSurface * surfaces; + int nb_surfaces; +} AMFFramesContext; + +/** +* Error handling helper +*/ +#define AMF_RETURN_IF_FALSE(avctx, exp, ret_value, /*message,*/ ...) \ + if (!(exp)) { \ + av_log(avctx, AV_LOG_ERROR, __VA_ARGS__); \ + return ret_value; \ + } + +#define AMF_GOTO_FAIL_IF_FALSE(avctx, exp, ret_value, /*message,*/ ...) \ + if (!(exp)) { \ + av_log(avctx, AV_LOG_ERROR, __VA_ARGS__); \ + ret = ret_value; \ + goto fail; \ + } + +#define AMF_TIME_BASE_Q (AVRational){1, AMF_SECOND} + +typedef struct FormatMap { + enum AVPixelFormat av_format; + enum AMF_SURFACE_FORMAT amf_format; +} FormatMap; + +extern const FormatMap format_map[]; +enum AMF_SURFACE_FORMAT av_amf_av_to_amf_format(enum AVPixelFormat fmt); +enum AVPixelFormat av_amf_to_av_format(enum AMF_SURFACE_FORMAT fmt); +extern AmfTraceWriter av_amf_trace_writer; + +int av_amf_context_init(AVAMFDeviceContextInternal* internal, void* avcl); +int av_amf_load_library(AVAMFDeviceContextInternal* internal, void* avcl); +int av_amf_create_context( AVAMFDeviceContextInternal * internal, + void* avcl, + const char *device, + AVDictionary *opts, int flags); +int av_amf_context_internal_create(AVAMFDeviceContextInternal * internal, + void* avcl, + const char *device, + AVDictionary *opts, int flags); +void av_amf_context_internal_free(void *opaque, uint8_t *data); +int av_amf_context_derive(AVAMFDeviceContextInternal * internal, + AVHWDeviceContext *child_device_ctx, AVDictionary *opts, + int flags); + +#endif /* AVUTIL_HWCONTEXT_AMF_H */ diff --git a/libavutil/hwcontext_internal.h b/libavutil/hwcontext_internal.h index 4df516ee6a..48d2dc012c 100644 --- a/libavutil/hwcontext_internal.h +++ b/libavutil/hwcontext_internal.h @@ -175,5 +175,6 @@ extern const HWContextType ff_hwcontext_type_vdpau; extern const HWContextType ff_hwcontext_type_videotoolbox; extern const HWContextType ff_hwcontext_type_mediacodec; extern const HWContextType ff_hwcontext_type_vulkan; +extern const HWContextType ff_hwcontext_type_amf; #endif /* AVUTIL_HWCONTEXT_INTERNAL_H */ diff --git a/libavutil/pixdesc.c b/libavutil/pixdesc.c index f6d4d01460..ebc79b0c74 100644 --- a/libavutil/pixdesc.c +++ b/libavutil/pixdesc.c @@ -2125,6 +2125,10 @@ static const AVPixFmtDescriptor av_pix_fmt_descriptors[AV_PIX_FMT_NB] = { .name = "cuda", .flags = AV_PIX_FMT_FLAG_HWACCEL, }, + [AV_PIX_FMT_AMF] = { + .name = "amf", + .flags = AV_PIX_FMT_FLAG_HWACCEL, + }, [AV_PIX_FMT_AYUV64LE] = { .name = "ayuv64le", .nb_components = 4, diff --git a/libavutil/pixfmt.h b/libavutil/pixfmt.h index 9c87571f49..06459be62c 100644 --- a/libavutil/pixfmt.h +++ b/libavutil/pixfmt.h @@ -251,6 +251,10 @@ enum AVPixelFormat { * exactly as for system memory frames. */ AV_PIX_FMT_CUDA, + /** + * HW acceleration through AMF. data[3] contain AMFSurface pointer + */ + AV_PIX_FMT_AMF, AV_PIX_FMT_0RGB, ///< packed RGB 8:8:8, 32bpp, XRGBXRGB... X=unused/undefined AV_PIX_FMT_RGB0, ///< packed RGB 8:8:8, 32bpp, RGBXRGBX... X=unused/undefined From patchwork Wed Feb 14 01:55:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitrii Ovchinnikov X-Patchwork-Id: 46251 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:c493:b0:19e:cdac:8cce with SMTP id eo19csp110178pzb; Tue, 13 Feb 2024 17:55:54 -0800 (PST) X-Forwarded-Encrypted: i=2; AJvYcCUxYvISX7Qdl+s9yfr9uAP3xy7VX4A1gSykkXMiZBp4I+C87QkUIDSthLW6nAXUdHxI7HOQob5SM2lEMjU+jZtahWZzROxi/p6qHA== X-Google-Smtp-Source: AGHT+IGhu8sXfBtCNbzagke5t86ysGKMWjhLqdGZqGRlx1GzbJha6MB3pzolKPrt9AlvDvOer44r X-Received: by 2002:aa7:c1d4:0:b0:560:6469:a190 with SMTP id d20-20020aa7c1d4000000b005606469a190mr847068edp.30.1707875753998; Tue, 13 Feb 2024 17:55:53 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1707875753; cv=none; d=google.com; s=arc-20160816; b=cgYnww/b2kjIn4Ot4jUtMTX9yWLBZoomprA/xCMab7NcIIW7gV3oObIh0pDoTqnCrC QhlDG0WSu/HgXQDhyOsJkc61HA8AhbB7VlmkSdNkketU+0EczmB2NONGSjea+Kn5hjUQ apJY/N0CfNYFZxYFz1qPWmep6Ga+lsuCs+n6ViiYBSIH+pGDfHUHn1yRbk4zS844WOcx oO5VO3FsjdusS7Zc/ymp4VKiYxTEGDLB+jAKb6z6QiQxzLLT8mwnBH2HxG7d0WG4wZ2l 1gaWYgIqzcJLmteufhzq5u3thV/IhzvB2cfhSKwTayI0MWs04Rhpq2lc0EKP8LQErmQt HT6g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=OJ4aYCW149NZ1OToVtvxE1CxAfmB7Q4XR37l9mwdjSA=; fh=u6hfVS/n0PXHA+Ge6nQ9v2mu9lbBt9hKc+k0wbYDFlQ=; b=FqNaX0E9AJXa3k+2aBgF21TcBakHLnsmGbPYdumMsBwglw371Tgdh/uFe7jPk3RPpP hE9Bt8TYPHw76dVCH1PHXFPHpZt/O3ucNmgzbVW/vb5hshLxoXeBeycaRL9grHhA/JAH UUy/0Bd5jcTHijf0YlzPzharITy+wSwfqUtTRCrP6Dju5jPUSihqxx9CSc8q4Ae0ynU7 8+ZlwrUy4yRhe6JuNz34JvHNaA01mNQmfH8HnxSkjeEbMtz38E+pajlyWXPaWlxH1ysx YzXVDHKMf46YZuRiEHCKepwWRf4VOYhOS1BYbB1m6f7KRtq6V+Pod6GSbwqgNmHJoA1O vsXA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=ab7vhaYN; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com X-Forwarded-Encrypted: i=1; AJvYcCUjvo/V8MKglOXRUR1cQUFo+LRXc2PVwL3gOcmYwDM+QAAZxWCY5Wb0TgNJwWvEgwO1cP7doxHCl9VC7ytTehudEVBIg06jF5aMv7pLr141Va9FrDzNlPkzphydL2825UItw7WI2ouxwGkvujZ2oI+PC4z7gMMncPbn23hQyzeYdcpYi+ldgReOtVfJsr06sbA1Fw48GI+/+Uh0+3CNtb1jTXuAJFSQ21/n9SKJ8habfKC63qzV0jrVpO5nXNpm/xldCZraz0knYiFdIVUcv9mfmcapvIGyjRxb5qq5REAFSxz4szrTd6KvPGr58dkj/p0nkAIig/UB/H0PXgmu3YJuVC9vcw6LromQOxE6cjm/AObdBvJ3FxNXIXoHVXHv+CpcaqoHbSGLGygK6we7mMqCOaokrZOckDCJkKqGEx0s+d4XZmTAZoZGcSYkUEu7Nw3HMHbyEIDS+aCLS4lNBzkT+XrR8TL8W4zqopTHa71l/9taMX8002Ta9pwv/A9YUtUm8/hq8+coZM6BOMLGbVRdC9XIIqg5jBZ0r2tuwQOvLtJUE0KAHJ9fnGSF1UjRW4T1RWybIdo6XbQ7HFVeM2+OwvzkOOtVL3/tm8UmDJq7IgnGcHkPyZ5G42K6sNwSjEl1XHiGEjndPO0soZsK5Y7Fo0hEbxH1xLCNym6tI1gkBCUN3AqY8/ZyLv2xSou35FBAeaDzYQSelKr1129PISji88uj2aSncsmzFNZfW28hF8kWjrCqltscq+lJ+mhDcL72VL59xepc+8PrWuiXq+oYSbDT2P6cq1UV0bBw3avIwx+NFuSwOHg7AvD4M/sGCLkOe792d9zCdxroM181zA1kZuvv4RNdNbQwaHWJ9VtfqFKkgY9q/RKEdoYI+IDGS1b5lAWV7eRiFyB5pn9d2u9NtMUPOaSqok/VQDaWB4Erpzzj5SVQbMO0feZRuGbNcIT3BL wgp2/qi4TQWtLTXxBbIBExgaQ6k9Ok91nnOxfxEpVqhwBNDtap1MGsEzVKC7hm5OnMW/mG2oGa/y2IMqd2epuX1p+kwqpXizGxLyV3Gg2fumhxnqXF2ckCwu4F86CF0sKQE4/0qY6A2iTkIeN2t4uJ+tKcY/Ko9xfsg3ofDEqVCnGb8cpwSHnKvbgcJlN1NtzBkg+XI+Kcmossy5j8OXcVTC/bVlpiP5yq4Y7vJWuXx15CSy3u+WwxL52HxMdsklTRMenu68GlL/EnCm/Vb+deAmwDtEo3xigy2NDGu9j1qPLMT2/SHUia5ltGzU+DaO6HbOm0wPrFWs3bXg2ke8VCj0u/5IO+Dg0vbhleQ2NpJCPxf9bdgG9PCVoSgyhIQzA15QQP31Y5psFquNCoJlh7VgnuZXB8jd1lpS7W7ftTvi84QGhwYIscxHsaoXMcE1xYuv0Kc6MIVnVdEayLmUCQo4jgSZsNyXqXbLsmd5J+Lpav9aHemWU2voJP1fOf0+h6o5eS3/mkMayegBtQwc0PIiRjhmfw7vkc0IVvFQKmcRJpTH/oCQOgWOuM6g4w86337Ko995y8ijbi2hpvs3SnejyytWkLV+iOgCJIqU58qkV+gzIOo82mLz2lUqk/tPawW20D Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id bd22-20020a056402207600b00560b6e6491csi4194594edb.623.2024.02.13.17.55.53; Tue, 13 Feb 2024 17:55:53 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=ab7vhaYN; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 84FF968D1C1; Wed, 14 Feb 2024 03:55:41 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f49.google.com (mail-ed1-f49.google.com [209.85.208.49]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 345DE68D185 for ; Wed, 14 Feb 2024 03:55:33 +0200 (EET) Received: by mail-ed1-f49.google.com with SMTP id 4fb4d7f45d1cf-562131bb958so811331a12.2 for ; Tue, 13 Feb 2024 17:55:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1707875732; x=1708480532; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QghbqhtXGzvl+yBqD2t3F9pcZyNcGp0335Vl5XsPHQQ=; b=ab7vhaYNd7JXdz8haNX0gCfmOCqm3xXjY6IybqWfiB9uX2Rij2PdTML9NSlCq5qtzX WBVjj6aoqNh9L6nDmVxAPO8trvsA6KXthA1UzVyos0IbmCR16Ia3ccsKDeaOTNZWGvtg tB9GtoW5I2VbKi2sJCJ+y+7NXKk5w0MPCidd0D6om3hPy8nYpLLZHMpKyJeFR6xzuAj1 wCwx6t9/GmNG7gMJTPcyJfkV5pq+q7NeQiwQiKhnYcNJAcLbZ5P2ciqP9uFnVkSTAol8 9m5CEWn4Rx+njSr5VapvBklyZ+T8Pg9MbTMv0pG6tkGwBplfmyhSa8ZK78Lbb//1tmm9 0dXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707875732; x=1708480532; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QghbqhtXGzvl+yBqD2t3F9pcZyNcGp0335Vl5XsPHQQ=; b=mURTCyPUjrLBsGF7EaQWDk9iQoB4m9/qu+F1A3x7os8vRaU9GC7K9Aprt9+lyIo7IM cFxte+fPIBRehMFyOgP0kCzkCHFPWIrTaTgVReK2Twy2sqZjecQZXR2Xu02DZx2aEQqr KzpnoWM8VVJOqVVLKRxQvgks3zQ1bVmPZjFb9JnhD88TYxD8orPTgS6BCAzrpgVIsHbB qwBhUQjRuEZU6iUQm/Lqmi6AJD9TegzBtXMTD7/QYN/yPUEefFtiuWWNKT4A/ODZvmLd pKybdjhqHw1TdPKFqJrbaVB6Rwh1rc7rBWp7CG8ZwMb16Kfu9DkSdUZzLcR6b+fJ/WBv Q7Lg== X-Gm-Message-State: AOJu0YweadVM7nzAuBmc1qKrEwp8vVcswFXx+7vvO1u/tG1eQnH/MV3p uASH0PFVwNaiVNer1cjDyCHYSSPzZWWmpWuN/qpWR009jJbXOsibDEZaeIqD X-Received: by 2002:a17:906:4715:b0:a3c:ef3f:a7c4 with SMTP id y21-20020a170906471500b00a3cef3fa7c4mr703261ejq.26.1707875731366; Tue, 13 Feb 2024 17:55:31 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCVr9TBMVFBSfZTlm9iz+J4K2703tVfYLMMAjlMQGdnGqjBKPIafr6RgzsrXbXPg7X89xb91pWpS7dlzqUkKLG6E513qU08zbqe0aV768Q== Received: from dovchinn.amd.com (cable-178-148-16-149.dynamic.sbb.rs. [178.148.16.149]) by smtp.gmail.com with ESMTPSA id vi16-20020a170907d41000b00a3ce31d3ffdsm1663230ejc.93.2024.02.13.17.55.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 17:55:30 -0800 (PST) From: Dmitrii Ovchinnikov To: ffmpeg-devel@ffmpeg.org Date: Wed, 14 Feb 2024 02:55:08 +0100 Message-Id: <20240214015515.1027-2-ovchinnikov.dmitrii@gmail.com> X-Mailer: git-send-email 2.38.1.windows.1 In-Reply-To: <20240214015515.1027-1-ovchinnikov.dmitrii@gmail.com> References: <20240214015515.1027-1-ovchinnikov.dmitrii@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/9] libavcodec: add amfdec. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Evgeny Pavlov , Dmitrii Ovchinnikov Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: fnquiuxlW1bX From: Evgeny Pavlov Added AMF based h264, hevc, av1 decoders. Co-authored-by: Dmitrii Ovchinnikov --- libavcodec/Makefile | 4 +- libavcodec/allcodecs.c | 3 + libavcodec/amfdec.c | 667 ++++++++++++++++++++++++++++++++++++++++ libavcodec/amfdec.h | 75 +++++ libavcodec/h264_slice.c | 3 + libavcodec/h264dec.c | 3 + libavcodec/hwconfig.h | 2 + 7 files changed, 755 insertions(+), 2 deletions(-) create mode 100644 libavcodec/amfdec.c create mode 100644 libavcodec/amfdec.h diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 470d7cb9b1..c2e4715f4b 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -69,7 +69,7 @@ include $(SRC_PATH)/libavcodec/x86/vvc/Makefile OBJS-$(CONFIG_AANDCTTABLES) += aandcttab.o OBJS-$(CONFIG_AC3DSP) += ac3dsp.o ac3.o ac3tab.o OBJS-$(CONFIG_ADTS_HEADER) += adts_header.o mpeg4audio_sample_rates.o -OBJS-$(CONFIG_AMF) += amfenc.o +OBJS-$(CONFIG_AMF) += amfenc.o amfdec.o OBJS-$(CONFIG_AUDIO_FRAME_QUEUE) += audio_frame_queue.o OBJS-$(CONFIG_ATSC_A53) += atsc_a53.o OBJS-$(CONFIG_AUDIODSP) += audiodsp.o @@ -1265,7 +1265,7 @@ SKIPHEADERS += %_tablegen.h \ vulkan_video_codec_av1std.h \ $(ARCH)/vpx_arith.h \ -SKIPHEADERS-$(CONFIG_AMF) += amfenc.h +SKIPHEADERS-$(CONFIG_AMF) += amfenc.h amfdec.h SKIPHEADERS-$(CONFIG_D3D11VA) += d3d11va.h dxva2_internal.h SKIPHEADERS-$(CONFIG_D3D12VA) += d3d12va_decode.h SKIPHEADERS-$(CONFIG_DXVA2) += dxva2.h dxva2_internal.h diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index ef8c3a6d7d..c344c70e00 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -851,10 +851,12 @@ extern const FFCodec ff_av1_nvenc_encoder; extern const FFCodec ff_av1_qsv_decoder; extern const FFCodec ff_av1_qsv_encoder; extern const FFCodec ff_av1_amf_encoder; +extern const FFCodec ff_av1_amf_decoder; extern const FFCodec ff_av1_vaapi_encoder; extern const FFCodec ff_libopenh264_encoder; extern const FFCodec ff_libopenh264_decoder; extern const FFCodec ff_h264_amf_encoder; +extern const FFCodec ff_h264_amf_decoder; extern const FFCodec ff_h264_cuvid_decoder; extern const FFCodec ff_h264_mf_encoder; extern const FFCodec ff_h264_nvenc_encoder; @@ -864,6 +866,7 @@ extern const FFCodec ff_h264_v4l2m2m_encoder; extern const FFCodec ff_h264_vaapi_encoder; extern const FFCodec ff_h264_videotoolbox_encoder; extern const FFCodec ff_hevc_amf_encoder; +extern const FFCodec ff_hevc_amf_decoder; extern const FFCodec ff_hevc_cuvid_decoder; extern const FFCodec ff_hevc_mediacodec_decoder; extern const FFCodec ff_hevc_mediacodec_encoder; diff --git a/libavcodec/amfdec.c b/libavcodec/amfdec.c new file mode 100644 index 0000000000..9d618ff442 --- /dev/null +++ b/libavcodec/amfdec.c @@ -0,0 +1,667 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include +#include +#include +#include "libavutil/hwcontext_amf.h" +#include "amfdec.h" +#include "codec_internal.h" +#include "hwconfig.h" +#include "libavutil/imgutils.h" +#include "libavutil/time.h" +#include "decode.h" +#include "libavutil/mastering_display_metadata.h" + +#if CONFIG_D3D11VA +#include "libavutil/hwcontext_d3d11va.h" +#endif +#if CONFIG_DXVA2 +#define COBJMACROS +#include "libavutil/hwcontext_dxva2.h" +#endif + +#ifdef _WIN32 +#include "compat/w32dlfcn.h" +#else +#include +#endif + +#define propNotFound 0 + +const enum AVPixelFormat amf_dec_pix_fmts[] = { + AV_PIX_FMT_YUV420P, + AV_PIX_FMT_NV12, + AV_PIX_FMT_BGRA, + AV_PIX_FMT_ARGB, + AV_PIX_FMT_RGBA, + AV_PIX_FMT_GRAY8, + AV_PIX_FMT_BGR0, + AV_PIX_FMT_YUYV422, + AV_PIX_FMT_P010, + AV_PIX_FMT_P012, + AV_PIX_FMT_YUV420P10, + AV_PIX_FMT_YUV420P12, + AV_PIX_FMT_YUV420P16, +#if CONFIG_D3D11VA + AV_PIX_FMT_D3D11, +#endif +#if CONFIG_DXVA2 + AV_PIX_FMT_DXVA2_VLD, +#endif + AV_PIX_FMT_AMF, + AV_PIX_FMT_NONE +}; + +static const AVCodecHWConfigInternal *const amf_hw_configs[] = { + &(const AVCodecHWConfigInternal) { + .public = { + .pix_fmt = AV_PIX_FMT_AMF, + .methods = AV_CODEC_HW_CONFIG_METHOD_HW_FRAMES_CTX | + AV_CODEC_HW_CONFIG_METHOD_HW_DEVICE_CTX, + .device_type = AV_HWDEVICE_TYPE_AMF, + }, + .hwaccel = NULL, + }, + NULL +}; + +static void amf_free_amfsurface(void *opaque, uint8_t *data) +{ + AMFSurface *surface = (AMFSurface*)(data); + surface->pVtbl->Release(surface); +} + +static int amf_init_decoder(AVCodecContext *avctx) +{ + enum AMF_SURFACE_FORMAT output_format = AMF_SURFACE_UNKNOWN; + AvAmfDecoderContext *ctx = avctx->priv_data; + AVAMFDeviceContextInternal * internal = (AVAMFDeviceContextInternal *)ctx->amf_device_ctx_internal->data; + const wchar_t *codec_id = NULL; + AMF_RESULT res; + AMFBuffer *buffer; + amf_int64 color_profile; + int pool_size = 35; + + if (avctx->pix_fmt == AV_PIX_FMT_AMF){ + if (avctx->hw_frames_ctx) { + AVHWFramesContext *hwframes_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; + output_format = av_amf_av_to_amf_format(hwframes_ctx->sw_format); + } else + output_format = av_amf_av_to_amf_format(avctx->sw_pix_fmt); + } else + output_format = av_amf_av_to_amf_format(avctx->pix_fmt); + + if (output_format == AMF_SURFACE_UNKNOWN) + output_format = AMF_SURFACE_NV12; + + ctx->drained = 0; + + switch (avctx->codec->id) { + case AV_CODEC_ID_H264: + codec_id = AMFVideoDecoderUVD_H264_AVC; + break; + case AV_CODEC_ID_HEVC: { + if (output_format == AMF_SURFACE_P010) + codec_id = AMFVideoDecoderHW_H265_MAIN10; + else + codec_id = AMFVideoDecoderHW_H265_HEVC; + } break; + case AV_CODEC_ID_AV1: + if (output_format == AMF_SURFACE_P012) + codec_id = AMFVideoDecoderHW_AV1_12BIT; + else + codec_id = AMFVideoDecoderHW_AV1; + break; + default: + break; + } + AMF_RETURN_IF_FALSE(ctx, codec_id != NULL, AVERROR(EINVAL), "Codec %d is not supported\n", avctx->codec->id); + + res = internal->factory->pVtbl->CreateComponent(internal->factory, internal->context, codec_id, &ctx->decoder); + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_ENCODER_NOT_FOUND, "CreateComponent(%ls) failed with error %d\n", codec_id, res); + + // Color Metadata + /// Color Range (Support for older Drivers) + if (avctx->color_range == AVCOL_RANGE_JPEG) { + AMF_ASSIGN_PROPERTY_BOOL(res, ctx->decoder, AMF_VIDEO_DECODER_FULL_RANGE_COLOR, 1); + } else if (avctx->color_range != AVCOL_RANGE_UNSPECIFIED) { + AMF_ASSIGN_PROPERTY_BOOL(res, ctx->decoder, AMF_VIDEO_DECODER_FULL_RANGE_COLOR, 0); + } + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN; + switch (avctx->colorspace) { + case AVCOL_SPC_SMPTE170M: + if (avctx->color_range == AVCOL_RANGE_JPEG) { + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_601; + } else { + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_601; + } + break; + case AVCOL_SPC_BT709: + if (avctx->color_range == AVCOL_RANGE_JPEG) { + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_709; + } else { + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_709; + } + break; + case AVCOL_SPC_BT2020_NCL: + case AVCOL_SPC_BT2020_CL: + if (avctx->color_range == AVCOL_RANGE_JPEG) { + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_2020; + } else { + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_2020; + } + break; + } + if (color_profile != AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_COLOR_PROFILE, color_profile); + if (avctx->color_trc != AVCOL_TRC_UNSPECIFIED) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_COLOR_TRANSFER_CHARACTERISTIC, (amf_int64)avctx->color_trc); + + if (avctx->color_primaries != AVCOL_PRI_UNSPECIFIED) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_COLOR_PRIMARIES, (amf_int64)avctx->color_primaries); + + if (ctx->timestamp_mode != -1) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_TIMESTAMP_MODE, ctx->timestamp_mode); + if (ctx->decoder_mode != -1) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_REORDER_MODE, ctx->decoder_mode); + if (ctx->dpb_size != -1) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_DPB_SIZE, ctx->dpb_size); + if (ctx->lowlatency != -1) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_LOW_LATENCY, ctx->lowlatency); + if (ctx->smart_access_video != -1) { + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_ENABLE_SMART_ACCESS_VIDEO, ctx->smart_access_video != 0); + if (res != AMF_OK) { + av_log(avctx, AV_LOG_ERROR, "The Smart Access Video is not supported by AMF decoder.\n"); + return AVERROR(EINVAL); + } else { + av_log(avctx, AV_LOG_INFO, "The Smart Access Video (%d) is set.\n", ctx->smart_access_video); + // Set low latency mode if Smart Access Video is enabled + if (ctx->smart_access_video != 0) { + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_LOW_LATENCY, true); + av_log(avctx, AV_LOG_INFO, "The Smart Access Video set low latency mode for decoder.\n"); + } + } + } + if (ctx->skip_transfer_sav != -1) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_SKIP_TRANSFER_SMART_ACCESS_VIDEO, ctx->skip_transfer_sav); + + if (avctx->extradata_size) { + res = internal->context->pVtbl->AllocBuffer(internal->context, AMF_MEMORY_HOST, avctx->extradata_size, &buffer); + if (res == AMF_OK) { + memcpy(buffer->pVtbl->GetNative(buffer), avctx->extradata, avctx->extradata_size); + AMF_ASSIGN_PROPERTY_INTERFACE(res,ctx->decoder, AMF_VIDEO_DECODER_EXTRADATA, buffer); + buffer->pVtbl->Release(buffer); + buffer = NULL; + } + } + if (ctx->surface_pool_size == -1) { + ctx->surface_pool_size = pool_size; + if (avctx->extra_hw_frames > 0) + ctx->surface_pool_size += avctx->extra_hw_frames; + if (avctx->active_thread_type & FF_THREAD_FRAME) + ctx->surface_pool_size += avctx->thread_count; + } + + //at the moment, there is such a restriction in AMF. + //when it is possible, I will remove this code + if (ctx->surface_pool_size > 100) + ctx->surface_pool_size = 100; + + AMF_ASSIGN_PROPERTY_INT64(res, ctx->decoder, AMF_VIDEO_DECODER_SURFACE_POOL_SIZE, ctx->surface_pool_size); + res = ctx->decoder->pVtbl->Init(ctx->decoder, output_format, avctx->width, avctx->height); + return 0; +} + +static int amf_init_decoder_context(AVCodecContext *avctx) +{ + AvAmfDecoderContext *ctx = avctx->priv_data; + int ret; + + if (avctx->hw_frames_ctx) { + AVHWFramesContext *frames_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; + ret = av_amf_context_derive((AVAMFDeviceContextInternal *)ctx->amf_device_ctx_internal->data, frames_ctx->device_ctx, NULL, 0); + if (ret < 0) + return ret; + ctx->hw_frames_ctx = av_buffer_ref(avctx->hw_frames_ctx); + if (!ctx->hw_frames_ctx) + return AVERROR(ENOMEM); + } + else if (avctx->hw_device_ctx) { + AVHWDeviceContext *device_ctx = (AVHWDeviceContext*)avctx->hw_device_ctx->data; + ret = av_amf_context_derive((AVAMFDeviceContextInternal *)ctx->amf_device_ctx_internal->data, device_ctx, NULL, 0); + if (ret < 0) + return ret; + ctx->hw_device_ctx = av_buffer_ref(avctx->hw_device_ctx); + if (!ctx->hw_device_ctx) + return AVERROR(ENOMEM); + } else { + ret = av_amf_context_init((AVAMFDeviceContextInternal *)ctx->amf_device_ctx_internal->data, avctx); + if (ret != 0) { + return ret; + } + } + + return ret; +} + +static int amf_decode_close(AVCodecContext *avctx) +{ + AvAmfDecoderContext *ctx = avctx->priv_data; + + if (ctx->decoder) { + ctx->decoder->pVtbl->Terminate(ctx->decoder); + ctx->decoder->pVtbl->Release(ctx->decoder); + ctx->decoder = NULL; + } + + av_buffer_unref(&ctx->amf_device_ctx_internal); + av_buffer_unref(&ctx->hw_device_ctx); + av_buffer_unref(&ctx->hw_frames_ctx); + av_buffer_unref(&ctx->amf_device_ctx); + + return 0; + +} + +static int amf_decode_init(AVCodecContext *avctx) +{ + AvAmfDecoderContext *ctx = avctx->priv_data; + int ret; + enum AVPixelFormat pix_fmts[3] = { + AV_PIX_FMT_AMF, + avctx->pix_fmt, + AV_PIX_FMT_NONE }; + + ret = ff_get_format(avctx, pix_fmts); + if (ret < 0) { + avctx->pix_fmt = AV_PIX_FMT_NONE; + } + + if (avctx->hw_frames_ctx){ + AVHWFramesContext *frames_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; + if (frames_ctx->device_ctx->type == AV_HWDEVICE_TYPE_AMF) { + AVAMFDeviceContext * amf_ctx = frames_ctx->device_ctx->hwctx; + ctx->amf_device_ctx_internal = av_buffer_ref(amf_ctx->internal); + } + } + else if (avctx->hw_device_ctx && !avctx->hw_frames_ctx && ret == AV_PIX_FMT_AMF) { + AVHWDeviceContext *hwdev_ctx; + AVHWFramesContext *hwframes_ctx; + hwdev_ctx = (AVHWDeviceContext*)avctx->hw_device_ctx->data; + if (hwdev_ctx->type == AV_HWDEVICE_TYPE_AMF) + { + AVAMFDeviceContext * amf_ctx = hwdev_ctx->hwctx; + ctx->amf_device_ctx_internal = av_buffer_ref(amf_ctx->internal); + } + + avctx->hw_frames_ctx = av_hwframe_ctx_alloc(avctx->hw_device_ctx); + + if (!avctx->hw_frames_ctx) { + av_log(avctx, AV_LOG_ERROR, "av_hwframe_ctx_alloc failed\n"); + return AVERROR(ENOMEM); + } + + hwframes_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; + hwframes_ctx->width = FFALIGN(avctx->coded_width, 32); + hwframes_ctx->height = FFALIGN(avctx->coded_height, 32); + hwframes_ctx->format = AV_PIX_FMT_AMF; + hwframes_ctx->sw_format = avctx->sw_pix_fmt == AV_PIX_FMT_YUV420P10 ? AV_PIX_FMT_P010 : AV_PIX_FMT_NV12; + hwframes_ctx->initial_pool_size = ctx->surface_pool_size + 8; + avctx->pix_fmt = AV_PIX_FMT_AMF; + + ret = av_hwframe_ctx_init(avctx->hw_frames_ctx); + + if (ret < 0) { + av_log(NULL, AV_LOG_ERROR, "Error initializing a AMF frame pool\n"); + av_buffer_unref(&avctx->hw_frames_ctx); + return ret; + } + } else { + AVAMFDeviceContextInternal *wrapped = av_mallocz(sizeof(*wrapped)); + ctx->amf_device_ctx_internal = av_buffer_create((uint8_t *)wrapped, sizeof(*wrapped), + av_amf_context_internal_free, NULL, 0); + if ((ret = av_amf_context_internal_create((AVAMFDeviceContextInternal *)ctx->amf_device_ctx_internal->data, avctx, "", NULL, 0)) != 0) { + amf_decode_close(avctx); + return ret; + } + if ((ret = amf_init_decoder_context(avctx)) != 0) { + return ret; + } + } + if ((ret = amf_init_decoder(avctx)) == 0) { + return 0; + } + amf_decode_close(avctx); + return ret; +} + +static AMF_RESULT amf_get_property_buffer(AMFData *object, const wchar_t *name, AMFBuffer **val) +{ + AMF_RESULT res; + AMFVariantStruct var; + res = AMFVariantInit(&var); + if (res == AMF_OK) { + res = object->pVtbl->GetProperty(object, name, &var); + if (res == AMF_OK) { + if (var.type == AMF_VARIANT_INTERFACE) { + AMFGuid guid_AMFBuffer = IID_AMFBuffer(); + AMFInterface *amf_interface = AMFVariantInterface(&var); + res = amf_interface->pVtbl->QueryInterface(amf_interface, &guid_AMFBuffer, (void**)val); + } else { + res = AMF_INVALID_DATA_TYPE; + } + } + AMFVariantClear(&var); + } + return res; +} + +static int amf_amfsurface_to_avframe(AVCodecContext *avctx, AMFSurface* surface, AVFrame *frame) +{ + AMFVariantStruct var = {0}; + AMFPlane *plane; + int i; + int ret; + + if (avctx->hw_frames_ctx) { + AVHWFramesContext *hwframes_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; + if (hwframes_ctx->format == AV_PIX_FMT_AMF) { + ret = ff_get_buffer(avctx, frame, AV_GET_BUFFER_FLAG_REF); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Get hw frame failed.\n"); + return ret; + } + //we need to release surface with frame to return it to decoder + frame->buf[1] = av_buffer_create((uint8_t *)surface, sizeof(AMFSurface), + amf_free_amfsurface, (void*)avctx, + AV_BUFFER_FLAG_READONLY); + frame->data[3] = (uint8_t *)surface; + } else { + av_log(avctx, AV_LOG_ERROR, "Unknown format for hwframes_ctx\n"); + return AVERROR(ENOMEM); + } + } else { + ret = surface->pVtbl->Convert(surface, AMF_MEMORY_HOST); + AMF_RETURN_IF_FALSE(avctx, ret == AMF_OK, AVERROR_UNKNOWN, "Convert(amf::AMF_MEMORY_HOST) failed with error %d\n", ret); + + for (i = 0; i < surface->pVtbl->GetPlanesCount(surface); i++) { + plane = surface->pVtbl->GetPlaneAt(surface, i); + frame->data[i] = plane->pVtbl->GetNative(plane); + frame->linesize[i] = plane->pVtbl->GetHPitch(plane); + } + + frame->buf[0] = av_buffer_create((uint8_t *)surface, sizeof(AMFSurface), + amf_free_amfsurface, (void*)avctx, + AV_BUFFER_FLAG_READONLY); + frame->format = av_amf_to_av_format(surface->pVtbl->GetFormat(surface)); + } + + frame->width = avctx->width; + frame->height = avctx->height; + + frame->pts = surface->pVtbl->GetPts(surface); + + surface->pVtbl->GetProperty(surface, L"FFMPEG:dts", &var); + frame->pkt_dts = var.int64Value; + + frame->duration = surface->pVtbl->GetDuration(surface); + if (frame->duration < 0) + frame->duration = 0; + + frame->color_range = avctx->color_range; + frame->colorspace = avctx->colorspace; + frame->color_trc = avctx->color_trc; + frame->color_primaries = avctx->color_primaries; + + if (frame->color_trc == AVCOL_TRC_SMPTE2084) { + AMFBuffer * hdrmeta_buffer = NULL; + ret = amf_get_property_buffer((AMFData *)surface, AMF_VIDEO_DECODER_HDR_METADATA, &hdrmeta_buffer); + if (hdrmeta_buffer != NULL) { + AMFHDRMetadata * hdrmeta = (AMFHDRMetadata*)hdrmeta_buffer->pVtbl->GetNative(hdrmeta_buffer); + if (ret != AMF_OK) + return ret; + if (hdrmeta != NULL) { + AVMasteringDisplayMetadata *mastering = av_mastering_display_metadata_create_side_data(frame); + const int chroma_den = 50000; + const int luma_den = 10000; + + if (!mastering) + return AVERROR(ENOMEM); + + mastering->display_primaries[0][0] = av_make_q(hdrmeta->redPrimary[0], chroma_den); + mastering->display_primaries[0][1] = av_make_q(hdrmeta->redPrimary[1], chroma_den); + + mastering->display_primaries[1][0] = av_make_q(hdrmeta->greenPrimary[0], chroma_den); + mastering->display_primaries[1][1] = av_make_q(hdrmeta->greenPrimary[1], chroma_den); + + mastering->display_primaries[2][0] = av_make_q(hdrmeta->bluePrimary[0], chroma_den); + mastering->display_primaries[2][1] = av_make_q(hdrmeta->bluePrimary[1], chroma_den); + + mastering->white_point[0] = av_make_q(hdrmeta->whitePoint[0], chroma_den); + mastering->white_point[1] = av_make_q(hdrmeta->whitePoint[1], chroma_den); + + mastering->max_luminance = av_make_q(hdrmeta->maxMasteringLuminance, luma_den); + mastering->min_luminance = av_make_q(hdrmeta->maxMasteringLuminance, luma_den); + + mastering->has_luminance = 1; + mastering->has_primaries = 1; + if (hdrmeta->maxContentLightLevel) { + AVContentLightMetadata *light = av_content_light_metadata_create_side_data(frame); + + if (!light) + return AVERROR(ENOMEM); + + light->MaxCLL = hdrmeta->maxContentLightLevel; + light->MaxFALL = hdrmeta->maxFrameAverageLightLevel; + } + } + } + } + return 0; +} + +static AMF_RESULT amf_receive_frame(AVCodecContext *avctx, AVFrame *frame) +{ + AvAmfDecoderContext *ctx = avctx->priv_data; + AMF_RESULT ret = AMF_OK; + AMFSurface *surface = NULL; + AMFData *data_out = NULL; + + ret = ctx->decoder->pVtbl->QueryOutput(ctx->decoder, &data_out); + if (ret != AMF_OK && ret != AMF_REPEAT) { + return ret; + } + if (data_out == NULL) { + return AMF_FAIL; + } + + if (data_out) { + AMFGuid guid = IID_AMFSurface(); + data_out->pVtbl->QueryInterface(data_out, &guid, (void**)&surface); // query for buffer interface + data_out->pVtbl->Release(data_out); + data_out = NULL; + } + + ret = amf_amfsurface_to_avframe(avctx, surface, frame); + AMF_GOTO_FAIL_IF_FALSE(avctx, ret >= 0, AMF_FAIL, "Failed to convert AMFSurface to AVFrame = %d\n", ret); + return AMF_OK; +fail: + + if (surface) { + surface->pVtbl->Release(surface); + surface = NULL; + } + return ret; +} + +static AMF_RESULT amf_update_buffer_properties(AVCodecContext *avctx, AMFBuffer* buffer, const AVPacket* pkt) +{ + AvAmfDecoderContext *ctx = avctx->priv_data; + AVAMFDeviceContextInternal * internal = (AVAMFDeviceContextInternal * )ctx->amf_device_ctx_internal->data; + AMFContext *ctxt = internal->context; + + AMF_RESULT res; + + AMF_RETURN_IF_FALSE(ctxt, buffer != NULL, AMF_INVALID_ARG, "update_buffer_properties() - buffer not passed in"); + AMF_RETURN_IF_FALSE(ctxt, pkt != NULL, AMF_INVALID_ARG, "update_buffer_properties() - packet not passed in"); + buffer->pVtbl->SetPts(buffer, pkt->pts); + buffer->pVtbl->SetDuration(buffer, pkt->duration); + AMF_ASSIGN_PROPERTY_INT64(res, buffer, L"FFMPEG:dts", pkt->dts); + if (res != AMF_OK) + av_log(avctx, AV_LOG_VERBOSE, "Failed to assign dts value."); + return AMF_OK; +} + +static AMF_RESULT amf_buffer_from_packet(AVCodecContext *avctx, const AVPacket* pkt, AMFBuffer** buffer) +{ + AvAmfDecoderContext *ctx = avctx->priv_data; + AVAMFDeviceContextInternal * internal = (AVAMFDeviceContextInternal * )ctx->amf_device_ctx_internal->data; + AMFContext *ctxt = internal->context; + void *mem; + AMF_RESULT err; + AMFBuffer *buf = NULL; + + AMF_RETURN_IF_FALSE(ctxt, pkt != NULL, AMF_INVALID_ARG, "amf_buffer_from_packet() - packet not passed in"); + AMF_RETURN_IF_FALSE(ctxt, buffer != NULL, AMF_INVALID_ARG, "amf_buffer_from_packet() - buffer pointer not passed in"); + + err = ctxt->pVtbl->AllocBuffer(ctxt, AMF_MEMORY_HOST, pkt->size + AV_INPUT_BUFFER_PADDING_SIZE, buffer); + AMF_RETURN_IF_FALSE(ctxt, err == AMF_OK, err, "amf_buffer_from_packet() - failed"); + buf = *buffer; + err = buf->pVtbl->SetSize(buf, pkt->size); + AMF_RETURN_IF_FALSE(ctxt, err == AMF_OK, err, "amf_buffer_from_packet() - SetSize failed"); + // get the memory location and check the buffer was indeed allocated + mem = buf->pVtbl->GetNative(buf); + AMF_RETURN_IF_FALSE(ctxt, mem != NULL, AMF_INVALID_POINTER, "amf_buffer_from_packet() - GetNative failed"); + + // copy the packet memory and clear data padding + memcpy(mem, pkt->data, pkt->size); + memset((amf_int8*)(mem)+pkt->size, 0, AV_INPUT_BUFFER_PADDING_SIZE); + + return amf_update_buffer_properties(avctx, buf, pkt); +} + +static int amf_decode_frame(AVCodecContext *avctx, AVFrame *data, + int *got_frame, AVPacket *avpkt) +{ + AvAmfDecoderContext *ctx = avctx->priv_data; + AVFrame *frame = data; + AMFBuffer *buf; + AMF_RESULT res; + + if (!ctx->decoder) + return AVERROR(EINVAL); + + if (!avpkt->size && ctx->drained == 0) { + ctx->decoder->pVtbl->Drain(ctx->decoder); + ctx->drained = 1; + } + if (avpkt->size > 0) { + res = amf_buffer_from_packet(avctx, avpkt, &buf); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, 0, "Cannot convert AVPacket to AMFbuffer"); + res = ctx->decoder->pVtbl->SubmitInput(ctx->decoder, (AMFData*) buf); + // FIXME: check other return values + if (res == AMF_OK || res == AMF_NEED_MORE_INPUT) + { + *got_frame = 0; + } else { + av_log(avctx, AV_LOG_VERBOSE, "SubmitInput() returned %d\n", res); + } + + buf->pVtbl->Release(buf); + buf = NULL; + if (res == AMF_INPUT_FULL) { // handle full queue + *got_frame = 0; + } + } + + res = amf_receive_frame(avctx, frame); + if (res == AMF_OK) { + AMF_RETURN_IF_FALSE(avctx, !*got_frame, avpkt->size, "frame already got"); + *got_frame = 1; + } else if (res != AMF_EOF && res != AMF_FAIL) { + av_log(avctx, AV_LOG_ERROR, "Unkown result from QueryOutput %d\n", res); + } + + return avpkt->size; +} + +static void amf_decode_flush(AVCodecContext *avctx) +{ + AvAmfDecoderContext *ctx = avctx->priv_data; + ctx->decoder->pVtbl->Flush(ctx->decoder); +} + +#define OFFSET(x) offsetof(AvAmfDecoderContext, x) +#define VD AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_DECODING_PARAM + +static const AVOption options[] = { + // Decoder mode + { "decoder_mode", "Decoder mode", OFFSET(decoder_mode), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, AMF_VIDEO_DECODER_MODE_LOW_LATENCY, VD, "decoder_mode" }, + { "regular", "DPB delay is based on number of reference frames + 1", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_DECODER_MODE_REGULAR }, 0, 0, VD, "decoder_mode" }, + { "compliant", "DPB delay is based on profile - up to 16", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_DECODER_MODE_COMPLIANT }, 0, 0, VD, "decoder_mode" }, + { "low_latency", "DPB delay is 0", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_DECODER_MODE_LOW_LATENCY }, 0, 0, VD, "decoder_mode" }, + + // Timestamp mode + { "timestamp_mode", "Timestamp mode", OFFSET(timestamp_mode), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, AMF_TS_DECODE, VD, "timestamp_mode" }, + { "presentation", "Preserve timestamps from input to output", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_TS_PRESENTATION }, 0, 0, VD, "timestamp_mode" }, + { "sort", "Resort PTS list", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_TS_SORT }, 0, 0, VD, "timestamp_mode" }, + { "decode", "Decode order", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_TS_DECODE }, 0, 0, VD, "timestamp_mode" }, + + // Reference frame management + { "surface_pool_size", "Number of surfaces in the decode pool", OFFSET(surface_pool_size), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, INT_MAX, VD, NULL }, + { "dpb_size", "Minimum number of surfaces for reordering", OFFSET(dpb_size), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 32, VD, NULL }, + + { "lowlatency", "Low latency", OFFSET(lowlatency), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 1, VD, NULL }, + { "smart_access_video", "Smart Access Video", OFFSET(smart_access_video), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 1, VD, NULL }, + { "skip_transfer_sav", "Skip transfer on another GPU when SAV enabled", OFFSET(skip_transfer_sav), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 1, VD, NULL }, + + { NULL } +}; + +static const AVClass amf_decode_class = { + .class_name = "amf", + .item_name = av_default_item_name, + .option = options, + .version = LIBAVUTIL_VERSION_INT, +}; + + +#define DEFINE_AMF_DECODER(x, X, bsf_name) \ +const FFCodec ff_##x##_amf_decoder = { \ + .p.name = #x "_amf", \ + CODEC_LONG_NAME(#X " AMD AMF video decoder"), \ + .priv_data_size = sizeof(AvAmfDecoderContext), \ + .p.type = AVMEDIA_TYPE_VIDEO, \ + .p.id = AV_CODEC_ID_##X, \ + .init = amf_decode_init, \ + FF_CODEC_DECODE_CB(amf_decode_frame), \ + .flush = amf_decode_flush, \ + .close = amf_decode_close, \ + .bsfs = bsf_name, \ + .p.capabilities = AV_CODEC_CAP_HARDWARE | AV_CODEC_CAP_DELAY | AV_CODEC_CAP_AVOID_PROBING, \ + .p.priv_class = &amf_decode_class, \ + .p.pix_fmts = amf_dec_pix_fmts, \ + .hw_configs = amf_hw_configs, \ + .p.wrapper_name = "amf", \ + .caps_internal = FF_CODEC_CAP_NOT_INIT_THREADSAFE, \ +}; \ + +DEFINE_AMF_DECODER(h264, H264, "h264_mp4toannexb") +DEFINE_AMF_DECODER(hevc, HEVC, NULL) +DEFINE_AMF_DECODER(av1, AV1, NULL) diff --git a/libavcodec/amfdec.h b/libavcodec/amfdec.h new file mode 100644 index 0000000000..4c45d2426b --- /dev/null +++ b/libavcodec/amfdec.h @@ -0,0 +1,75 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_AMFDEC_H +#define AVCODEC_AMFDEC_H + +#include +#include +#include +#include +#include +#include + +#include "avcodec.h" +#include "libavformat/avformat.h" +#include "libavutil/fifo.h" +#include "libavutil/frame.h" +#include "libavutil/opt.h" + +/** +* AMF decoder context +*/ + +typedef struct AvAmfDecoderContext { + AVClass *avclass; + + AVBufferRef *amf_device_ctx_internal; + AVBufferRef *amf_device_ctx; + + //decoder + AMFComponent *decoder; ///< AMF decoder object + AMF_SURFACE_FORMAT format; ///< AMF surface format + + AVBufferRef *hw_device_ctx; ///< pointer to HW accelerator (decoder) + AVBufferRef *hw_frames_ctx; ///< pointer to HW accelerator (frame allocator) + + AVBufferRef *hw_device_ref; + AVBufferRef *hw_frames_ref; + + // shift dts back by max_b_frames in timing + AVFifoBuffer *timestamp_list; + int64_t dts_delay; + + amf_uint64 version; ///< version of AMF runtime + // common encoder option options + + int log_to_dbg; + // Static options, have to be set before Init() call + int decoder_mode; + int timestamp_mode; + int surface_pool_size; + int dpb_size; + int lowlatency; + int smart_access_video; + int skip_transfer_sav; + int drained; + +} AvAmfDecoderContext; + +#endif // AVCODEC_AMFDEC_H \ No newline at end of file diff --git a/libavcodec/h264_slice.c b/libavcodec/h264_slice.c index 8464a0b34c..d11821194f 100644 --- a/libavcodec/h264_slice.c +++ b/libavcodec/h264_slice.c @@ -864,6 +864,9 @@ static enum AVPixelFormat get_pixel_format(H264Context *h, int force_callback) #if CONFIG_H264_NVDEC_HWACCEL *fmt++ = AV_PIX_FMT_CUDA; #endif +#if CONFIG_H264_AMFDEC_HWACCEL + *fmt++ = AV_PIX_FMT_AMF; +#endif #if CONFIG_H264_VIDEOTOOLBOX_HWACCEL if (h->avctx->colorspace != AVCOL_SPC_RGB) *fmt++ = AV_PIX_FMT_VIDEOTOOLBOX; diff --git a/libavcodec/h264dec.c b/libavcodec/h264dec.c index 9f5893c512..7a2c9eecef 100644 --- a/libavcodec/h264dec.c +++ b/libavcodec/h264dec.c @@ -1137,6 +1137,9 @@ const FFCodec ff_h264_decoder = { #if CONFIG_H264_NVDEC_HWACCEL HWACCEL_NVDEC(h264), #endif +#if CONFIG_H264_AMFDEC_HWACCEL + HWACCEL_AMFDEC(h264), +#endif #if CONFIG_H264_VAAPI_HWACCEL HWACCEL_VAAPI(h264), #endif diff --git a/libavcodec/hwconfig.h b/libavcodec/hwconfig.h index ee29ca631d..556f724895 100644 --- a/libavcodec/hwconfig.h +++ b/libavcodec/hwconfig.h @@ -67,6 +67,8 @@ void ff_hwaccel_uninit(AVCodecContext *avctx); HW_CONFIG_HWACCEL(1, 1, 0, D3D11, D3D11VA, ff_ ## codec ## _d3d11va2_hwaccel) #define HWACCEL_NVDEC(codec) \ HW_CONFIG_HWACCEL(1, 1, 0, CUDA, CUDA, ff_ ## codec ## _nvdec_hwaccel) +#define HWACCEL_AMFDEC(codec) \ + HW_CONFIG_HWACCEL(1, 1, 0, AMF, AMF, ff_ ## codec ## _amfdec_hwaccel) #define HWACCEL_VAAPI(codec) \ HW_CONFIG_HWACCEL(1, 1, 1, VAAPI, VAAPI, ff_ ## codec ## _vaapi_hwaccel) #define HWACCEL_VDPAU(codec) \ From patchwork Wed Feb 14 01:55:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitrii Ovchinnikov X-Patchwork-Id: 46252 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:c493:b0:19e:cdac:8cce with SMTP id eo19csp110239pzb; Tue, 13 Feb 2024 17:56:03 -0800 (PST) X-Forwarded-Encrypted: i=2; AJvYcCUPQboeY29N+GLHefEbB4zxar9iKJ6tL50tDoxg5xE06zKbGL+skm3HAUwAPMJqjYb8zuERSUceZX+XHcyC4vJaEmJ2v7/hU3oJGg== X-Google-Smtp-Source: AGHT+IEhfsekt+le829yeQIi5vG47xsS90mRszk6uuHVa80Dt2ThErGOfEIzS2tjTaiN5poen+YH X-Received: by 2002:a05:6512:33d6:b0:511:87eb:8dc4 with SMTP id d22-20020a05651233d600b0051187eb8dc4mr1032152lfg.32.1707875763346; Tue, 13 Feb 2024 17:56:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1707875763; cv=none; d=google.com; s=arc-20160816; b=BJ7CXslsHTYmSH1BvRPV5pM3Ws7GJstrKJNDMSIiVpz53B+TkAfEV0LgS2HFfpWq3P WcGCpSYByXJ0+WdHH06VJ49YH87rbN6bfAs3AAwjjJYjYhQS/NXTabHm5kXRA/NxEoo8 2vS+xAJhYpExHPnavoSyS3iedmdrN0LrjekKWh5zlec+bvRNbu5Ps/l+4lxIplhV8CNi aYrdHvQTgn4uuNZRmVgD4Fda616gjmthKpycsdfV8QcpFfIO4mCuelBVb/2P/x7deh0G LCqffuAkrwL063iOJwBrMO71SeQqPD3DI9WNwt3cqvL3k7gNq9ZZpCGrs4tlXVS9KOng vVOg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=6ursnJeFdZfNyeLO834TrRx/dsNXnZUzsLHSCn5EkqQ=; fh=+QfgPMSZ/aLeE/otCFKFTXJWcH4xdx+WMSntfrSBEIY=; b=Q/ZMC0j6ewV5GQu5JUlpzA4ArVlhjpr9kKlFy6Q1Dh/THpUPmwOhZ3juGaEILXf7w1 D1Q6V+fzyCFJmwmqeteMkmQ/vwUY55lipQBuPa8VKvpjQP3f5K/VgVWe+Kv+OLMUp+a5 Ro3Tp7UKGwURTJREKvHdx0wbp+Ge4sJqAujeu0/Div9CzbRicO0NaEEk2NeE17I+ztTs x9DGtInZ3fjaG6unsdwVQ2xf8rmOd4utNF8uGwBLLsObXUX+aOGdsXrYViEXDVL/zne3 SZq5N9vZcqYOL/j6kaxljlZjJ2jlai3bhiwnMaAuQpImPJcnwHfe3AN+sD7P7btMQo+v HyiA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=l7eFQ+K+; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com X-Forwarded-Encrypted: i=1; AJvYcCU+RIOBZudjgl5AIzFTYJN3ia+VwFs7AOdrLSzq9DsMjdeB0HevRrV5rwcBlhp5NnvVdDx2d3WVDVRE0+JUfOn4+LZI7rWDlbwFNgmrmWF5ljkTNQQFd3Nb35y/+zxqV4VsMYGCDtVfTY0rWr7qjgJlroiomj35pqQ/eynL7C+2JTx9qC2VE9QKE/abzWogS8vFjVq6TwVZHjViEqX40sR6E7LgAWmIhFWFAqrFxmTVFVPNRXWKaOXqXNLTBBGpCEZmsCRCXAOfaPMh/SJctNG0n3ijWuYf8u/LA6ELP6kQXItKhNN+VHEd+7fiyeaM2cgtMR+Rek+6VDbqJ59KK/vCM5zKm2c6EsBINI8/8dz/NE8lbFzeGbr7rRzQisTt1bs9aBSpHfAcxRkLhPpkI8S1LflXYoRjnPpFSK/VstjsbGk7T/eyt+n9NDuProU8vYVvt8MuSRe/R3nnz3io9TEz+a68bbqpgPnFSJQv0pXuVmFTcxtxmnpIFxIGZXf9+1zXuXZwCo4DZn9AT8mYDNPQw5wx4trD0RjuRV8opjh4D4szIditYyFAIBohSYiqA65uyNJE5PDsF9r23uuenFHmZY8bV5ObVriclydXPjsE6HAdzmXV5FRO0oZsvzKRMQOKFsgciHx0o4BvitMkmu7vm8kVOEOGt7pNBDTXYgxk1w1QE5J2feLwLz4P7XsJgcGNNi7rnqhMwD+mNp3PJRMdIxpEpBozg7BW7phDDkqI20ZeDbjDHs57o90WE2WWfn85JHYja+shtkjADNOTH9unr9q+w690w+9BesnaThHrQ+62Q5KRGcvhJm5G+sAs1By6ZYwrJmFb+5813jxdNK/4NjxZSNuBwOK/DmDg4sb3E/yHF5a6rzULEEvjkzoopck4a9aXgBWiVYdPmXs1u5qQOc6G62Y2i2KilOmsDA2Qd2PL8hPYwW3YFuRVIBj1s3PS1i aw5QdbnpDh8iJ3O0ayRQkH+n4D6KExtfUm9W3wfSSpOSgaNQXIOV8qY53oZRrxU/aBV34ALJFtX3iC9RaGcL3xol04/p92x5h7EvDoxdagKlq9Ee8oXur2Xx07DSZ4RDf0Y6b55wapTKInvmFcj3xivB304BvY6+eH2Cffma9Aq6ov4UohDzf8ByhJwHvBpNr9CMAvepVhAA85VSErM/yonYwLZk23l1wHsRxDLFVF6wz+/NufCVMguD29yjHC9f9QQgPqgu0wq6XClQobEP1zJ4P4arY9Xymoq6EieVno7eBQX7gBDf4o4wvZNFQMnSVe3PMkP+ozKS/o1CVwK05n8D29+4isPruEheWCof5BwWN4D+ttkpgMlcwDL6KF5Wr/maBVpZTLJmFrvftU3R5LJ31PU05wICWX0jFCVLB+qxhrhgBm9HssyE8dAxOwnzTJmpESKi5lGMXz9yV4muWXO/jUGk3V/+PDz3pKTtLakGvq7NatdeKHEpdtlhU5Dv9FuhhUIQ3mOYpbiFEVuBtcL9qskWlIvHfFm3Ypd7z/XLCuXn6uzEDAqVoWbL840O4/SOzEHrJNcRN+bCKeLnwAC4PA/UKIzTHg Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id jp13-20020a170906f74d00b00a3d32c7598asi363739ejb.939.2024.02.13.17.56.02; Tue, 13 Feb 2024 17:56:03 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=l7eFQ+K+; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9B67468D174; Wed, 14 Feb 2024 03:55:42 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f52.google.com (mail-ej1-f52.google.com [209.85.218.52]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CA3E968D19E for ; Wed, 14 Feb 2024 03:55:33 +0200 (EET) Received: by mail-ej1-f52.google.com with SMTP id a640c23a62f3a-a2f79e79f0cso759682866b.2 for ; Tue, 13 Feb 2024 17:55:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1707875732; x=1708480532; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9paxm12JA2SgOK9UmotyTCwm2cT9xR4KVDTJMvndGEY=; b=l7eFQ+K+A6Qi/YqiJ8TroNq4EA8pv9nSP2fwLV3aYDyXzlFrXyDHhXgDrzRL19e1Dz FH1O7KR6tt7RShcMv0y4zH/A/d91Ylghrx0YT2gZ9D3XhESQZHlE01H8dAbRm63nibcc 7vcFCIiqm9TUggQGtxWxA4S7Br4yRXC90DZp3fis4FDQ58o3FsnK70lQ7WerqBXQdqv9 MNUEur3XF7SCYrKqak///dwPcin3D7pPl5zluE8/P2le445Hh9MeqrVV7yr6aSz1f/DD FP1oOYPb6mZkpzpsEdXZUICjmZssHhl6Wvn/VtgxiFtdMqkw4jiEwTe6FhzQz/3S7Fq6 3IVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707875732; x=1708480532; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9paxm12JA2SgOK9UmotyTCwm2cT9xR4KVDTJMvndGEY=; b=d4PEdoqgY6MYQG+wGfR+DhJs8zLo0JNLAb1n6kzdhAeGGpLtezWDWDLcZy9BwiGU/w 7zdlsWUawXACTalIvjlV2VUqdILugLJeLJ3/MFY/ITUhGsbiMGHatdsLMun/5ddbMRiy EnbulktjIonEeKE2r1EzzWQz5LxY3eh5EFjf82IY7usvkf+SXMzmShNfgT4xbUzS/X5L U2Qp+EeW8tl7CW6k74ig46BNkOH61aeKDzPYl7JTjQq6nMRPASs2s/j9pZKrk1FdU0fN izyYm8dHAQQBCluUBPvrNA5FyK6fwauESoBlDH4sFPb28vo9OHsDaEltyqBC0oK9rqv0 IQ4g== X-Gm-Message-State: AOJu0Yx0EJzKBfGwdygPCKD7U/3tTb5f84PCdhM+ujkK/iIQugcPQnSv i1AKIOSMdKb8dE26o0VG2rNLds9i4l99v7bZYUW/BXeAsbElALOyQylFUe0FoDM= X-Received: by 2002:a17:906:81cc:b0:a3c:c7dd:b10e with SMTP id e12-20020a17090681cc00b00a3cc7ddb10emr685470ejx.22.1707875732572; Tue, 13 Feb 2024 17:55:32 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCVjUjUcVrZUMwW9YSmEia9SfE3ar77YAL39Qc80wsBhp0mLFcbYcbLT7UHieqvv6BTnOYZEjsIqF89Ezk+oBsBXywtArQ== Received: from dovchinn.amd.com (cable-178-148-16-149.dynamic.sbb.rs. [178.148.16.149]) by smtp.gmail.com with ESMTPSA id vi16-20020a170907d41000b00a3ce31d3ffdsm1663230ejc.93.2024.02.13.17.55.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 17:55:31 -0800 (PST) From: Dmitrii Ovchinnikov To: ffmpeg-devel@ffmpeg.org Date: Wed, 14 Feb 2024 02:55:09 +0100 Message-Id: <20240214015515.1027-3-ovchinnikov.dmitrii@gmail.com> X-Mailer: git-send-email 2.38.1.windows.1 In-Reply-To: <20240214015515.1027-1-ovchinnikov.dmitrii@gmail.com> References: <20240214015515.1027-1-ovchinnikov.dmitrii@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 3/9] avcodec/amfenc: Fixes the color information in the output. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Evgeny Pavlov , Michael Fabian 'Xaymar' Dirks Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: TmsM386MAZ4R From: Michael Fabian 'Xaymar' Dirks added 10 bit support for amf hevc. before: command - ffmpeg.exe -hide_banner -y -hwaccel d3d11va -hwaccel_output_format d3d11 -i test_10bit_file.mkv -an -c:v h264_amf res.dx11_hw_h264.mkv output - Format of input frames context (p010le) is not supported by AMF. command - ffmpeg.exe -hide_banner -y -hwaccel d3d11va -hwaccel_output_format d3d11 -i test_10bit_file -an -c:v hevc_amf res.dx11_hw_hevc.mkv output - Format of input frames context (p010le) is not supported by AMF. after: command - ffmpeg.exe -hide_banner -y -hwaccel d3d11va -hwaccel_output_format d3d11 -i test_10bit_file -an -c:v h264_amf res.dx11_hw_h264.mkv output - 10-bit input video is not supported by AMF H264 encoder command - ffmpeg.exe -hide_banner -y -hwaccel d3d11va -hwaccel_output_format d3d11 -i test_10bit_file -an -c:v hevc_amf res.dx11_hw_hevc.mkv output - 10bit file v2 - lost line returned in ff_amf_pix_fmts v3 - fixes after review v4 - extract duplicated code, fix incorrect processing of 10-bit input for h264 v5 - non-functional changes after review Co-authored-by: Evgeny Pavlov --- libavcodec/amfenc.c | 37 +++++++++++++++++++++++++++++++++++++ libavcodec/amfenc.h | 3 +++ libavcodec/amfenc_h264.c | 24 ++++++++++++++++++++---- libavcodec/amfenc_hevc.c | 26 +++++++++++++++++++++++++- 4 files changed, 85 insertions(+), 5 deletions(-) diff --git a/libavcodec/amfenc.c b/libavcodec/amfenc.c index 061859f85c..0bd15dd812 100644 --- a/libavcodec/amfenc.c +++ b/libavcodec/amfenc.c @@ -60,6 +60,7 @@ const enum AVPixelFormat ff_amf_pix_fmts[] = { #if CONFIG_DXVA2 AV_PIX_FMT_DXVA2_VLD, #endif + AV_PIX_FMT_P010, AV_PIX_FMT_NONE }; @@ -72,6 +73,7 @@ static const FormatMap format_map[] = { { AV_PIX_FMT_NONE, AMF_SURFACE_UNKNOWN }, { AV_PIX_FMT_NV12, AMF_SURFACE_NV12 }, + { AV_PIX_FMT_P010, AMF_SURFACE_P010 }, { AV_PIX_FMT_BGR0, AMF_SURFACE_BGRA }, { AV_PIX_FMT_RGB0, AMF_SURFACE_RGBA }, { AV_PIX_FMT_GRAY8, AMF_SURFACE_GRAY8 }, @@ -785,6 +787,41 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) return ret; } +int ff_amf_get_color_profile(AVCodecContext *avctx) +{ + amf_int64 color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN; + if (avctx->color_range == AVCOL_RANGE_JPEG) { + /// Color Space for Full (JPEG) Range + switch (avctx->colorspace) { + case AVCOL_SPC_SMPTE170M: + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_601; + break; + case AVCOL_SPC_BT709: + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_709; + break; + case AVCOL_SPC_BT2020_NCL: + case AVCOL_SPC_BT2020_CL: + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_2020; + break; + } + } else { + /// Color Space for Limited (MPEG) range + switch (avctx->colorspace) { + case AVCOL_SPC_SMPTE170M: + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_601; + break; + case AVCOL_SPC_BT709: + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_709; + break; + case AVCOL_SPC_BT2020_NCL: + case AVCOL_SPC_BT2020_CL: + color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_2020; + break; + } + } + return color_profile; +} + const AVCodecHWConfigInternal *const ff_amfenc_hw_configs[] = { #if CONFIG_D3D11VA HW_CONFIG_ENCODER_FRAMES(D3D11, D3D11VA), diff --git a/libavcodec/amfenc.h b/libavcodec/amfenc.h index 2dbd378ef8..62736ef579 100644 --- a/libavcodec/amfenc.h +++ b/libavcodec/amfenc.h @@ -21,6 +21,7 @@ #include +#include #include #include #include @@ -170,6 +171,8 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt); */ extern const enum AVPixelFormat ff_amf_pix_fmts[]; +int ff_amf_get_color_profile(AVCodecContext *avctx); + /** * Error handling helper */ diff --git a/libavcodec/amfenc_h264.c b/libavcodec/amfenc_h264.c index bd544d12df..f785e091c9 100644 --- a/libavcodec/amfenc_h264.c +++ b/libavcodec/amfenc_h264.c @@ -199,6 +199,8 @@ static av_cold int amf_encode_init_h264(AVCodecContext *avctx) AMFRate framerate; AMFSize framesize = AMFConstructSize(avctx->width, avctx->height); int deblocking_filter = (avctx->flags & AV_CODEC_FLAG_LOOP_FILTER) ? 1 : 0; + amf_int64 color_profile; + enum AVPixelFormat pix_fmt; if (avctx->framerate.num > 0 && avctx->framerate.den > 0) { framerate = AMFConstructRate(avctx->framerate.num, avctx->framerate.den); @@ -262,10 +264,24 @@ FF_ENABLE_DEPRECATION_WARNINGS AMF_ASSIGN_PROPERTY_RATIO(res, ctx->encoder, AMF_VIDEO_ENCODER_ASPECT_RATIO, ratio); } - /// Color Range (Partial/TV/MPEG or Full/PC/JPEG) - if (avctx->color_range == AVCOL_RANGE_JPEG) { - AMF_ASSIGN_PROPERTY_BOOL(res, ctx->encoder, AMF_VIDEO_ENCODER_FULL_RANGE_COLOR, 1); - } + color_profile = ff_amf_get_color_profile(avctx); + AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_OUTPUT_COLOR_PROFILE, color_profile); + + /// Color Range (Support for older Drivers) + AMF_ASSIGN_PROPERTY_BOOL(res, ctx->encoder, AMF_VIDEO_ENCODER_FULL_RANGE_COLOR, !!(avctx->color_range == AVCOL_RANGE_JPEG)); + + /// Color Depth + pix_fmt = avctx->hw_frames_ctx ? ((AVHWFramesContext*)avctx->hw_frames_ctx->data)->sw_format + : avctx->pix_fmt; + + // 10 bit input video is not supported by AMF H264 encoder + AMF_RETURN_IF_FALSE(ctx, pix_fmt != AV_PIX_FMT_P010, AVERROR_INVALIDDATA, "10-bit input video is not supported by AMF H264 encoder\n"); + + AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_COLOR_BIT_DEPTH, AMF_COLOR_BIT_DEPTH_8); + /// Color Transfer Characteristics (AMF matches ISO/IEC) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_OUTPUT_TRANSFER_CHARACTERISTIC, (amf_int64)avctx->color_trc); + /// Color Primaries (AMF matches ISO/IEC) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_OUTPUT_COLOR_PRIMARIES, (amf_int64)avctx->color_primaries); // autodetect rate control method if (ctx->rate_control_mode == AMF_VIDEO_ENCODER_RATE_CONTROL_METHOD_UNKNOWN) { diff --git a/libavcodec/amfenc_hevc.c b/libavcodec/amfenc_hevc.c index 352564a301..8c6401c646 100644 --- a/libavcodec/amfenc_hevc.c +++ b/libavcodec/amfenc_hevc.c @@ -34,8 +34,9 @@ static const AVOption options[] = { { "high_quality", "high quality trancoding", 0, AV_OPT_TYPE_CONST, {.i64 = AMF_VIDEO_ENCODER_HEVC_USAGE_HIGH_QUALITY }, 0, 0, VE, "usage" }, { "lowlatency_high_quality","low latency yet high quality trancoding", 0, AV_OPT_TYPE_CONST, {.i64 = AMF_VIDEO_ENCODER_HEVC_USAGE_LOW_LATENCY_HIGH_QUALITY }, 0, 0, VE, "usage" }, - { "profile", "Set the profile (default main)", OFFSET(profile), AV_OPT_TYPE_INT,{ .i64 = AMF_VIDEO_ENCODER_HEVC_PROFILE_MAIN }, AMF_VIDEO_ENCODER_HEVC_PROFILE_MAIN, AMF_VIDEO_ENCODER_HEVC_PROFILE_MAIN, VE, "profile" }, + { "profile", "Set the profile (default main)", OFFSET(profile), AV_OPT_TYPE_INT,{ .i64 = AMF_VIDEO_ENCODER_HEVC_PROFILE_MAIN }, AMF_VIDEO_ENCODER_HEVC_PROFILE_MAIN, AMF_VIDEO_ENCODER_HEVC_PROFILE_MAIN_10, VE, "profile" }, { "main", "", 0, AV_OPT_TYPE_CONST,{ .i64 = AMF_VIDEO_ENCODER_HEVC_PROFILE_MAIN }, 0, 0, VE, "profile" }, + { "main10", "", 0, AV_OPT_TYPE_CONST,{ .i64 = AMF_VIDEO_ENCODER_HEVC_PROFILE_MAIN_10 }, 0, 0, VE, "profile" }, { "profile_tier", "Set the profile tier (default main)", OFFSET(tier), AV_OPT_TYPE_INT,{ .i64 = AMF_VIDEO_ENCODER_HEVC_TIER_MAIN }, AMF_VIDEO_ENCODER_HEVC_TIER_MAIN, AMF_VIDEO_ENCODER_HEVC_TIER_HIGH, VE, "tier" }, { "main", "", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_ENCODER_HEVC_TIER_MAIN }, 0, 0, VE, "tier" }, @@ -160,6 +161,9 @@ static av_cold int amf_encode_init_hevc(AVCodecContext *avctx) AMFRate framerate; AMFSize framesize = AMFConstructSize(avctx->width, avctx->height); int deblocking_filter = (avctx->flags & AV_CODEC_FLAG_LOOP_FILTER) ? 1 : 0; + amf_int64 color_depth; + amf_int64 color_profile; + enum AVPixelFormat pix_fmt; if (avctx->framerate.num > 0 && avctx->framerate.den > 0) { framerate = AMFConstructRate(avctx->framerate.num, avctx->framerate.den); @@ -187,6 +191,9 @@ FF_ENABLE_DEPRECATION_WARNINGS case AV_PROFILE_HEVC_MAIN: profile = AMF_VIDEO_ENCODER_HEVC_PROFILE_MAIN; break; + case AV_PROFILE_HEVC_MAIN_10: + profile = AMF_VIDEO_ENCODER_HEVC_PROFILE_MAIN_10; + break; default: break; } @@ -215,6 +222,23 @@ FF_ENABLE_DEPRECATION_WARNINGS AMF_ASSIGN_PROPERTY_RATIO(res, ctx->encoder, AMF_VIDEO_ENCODER_HEVC_ASPECT_RATIO, ratio); } + color_profile = ff_amf_get_color_profile(avctx); + AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_HEVC_OUTPUT_COLOR_PROFILE, color_profile); + /// Color Range (Support for older Drivers) + AMF_ASSIGN_PROPERTY_BOOL(res, ctx->encoder, AMF_VIDEO_ENCODER_HEVC_NOMINAL_RANGE, !!(avctx->color_range == AVCOL_RANGE_JPEG)); + /// Color Depth + color_depth = AMF_COLOR_BIT_DEPTH_8; + pix_fmt = avctx->hw_frames_ctx ? ((AVHWFramesContext*)avctx->hw_frames_ctx->data)->sw_format + : avctx->pix_fmt; + if (pix_fmt == AV_PIX_FMT_P010) { + color_depth = AMF_COLOR_BIT_DEPTH_10; + } + AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_HEVC_COLOR_BIT_DEPTH, color_depth); + /// Color Transfer Characteristics (AMF matches ISO/IEC) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_HEVC_OUTPUT_TRANSFER_CHARACTERISTIC, (amf_int64)avctx->color_trc); + /// Color Primaries (AMF matches ISO/IEC) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_HEVC_OUTPUT_COLOR_PRIMARIES, (amf_int64)avctx->color_primaries); + // Picture control properties AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_HEVC_NUM_GOPS_PER_IDR, ctx->gops_per_idr); AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_HEVC_GOP_SIZE, avctx->gop_size); From patchwork Wed Feb 14 01:55:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitrii Ovchinnikov X-Patchwork-Id: 46253 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:c493:b0:19e:cdac:8cce with SMTP id eo19csp110267pzb; Tue, 13 Feb 2024 17:56:12 -0800 (PST) X-Forwarded-Encrypted: i=2; AJvYcCVdG0/9DJvi+dZ9gMmv7ylVLLZx9gJwAuWG63Rr+NU5izc8bGnux/cIGcmut1rqiAXhA6Q2fzTXTkgl+C3NGkcVuEgGkH66Mc0/1w== X-Google-Smtp-Source: AGHT+IEIIRuIizfgvc36p5ay+QM0FRYoBvmysl31/zwq5d+2ZhGWzesVIjCWwk6CJu0+qMWulhTi X-Received: by 2002:a17:906:f897:b0:a3d:16dd:307c with SMTP id lg23-20020a170906f89700b00a3d16dd307cmr733461ejb.5.1707875772244; Tue, 13 Feb 2024 17:56:12 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1707875772; cv=none; d=google.com; s=arc-20160816; b=flROo3rKShUn+O7xtip44WfII9uAmExCTIRD55c2+Zh2tlQI9x1+J4/HmQ8OT3l6Lv XwzaMlf+gd8RMbcjYYHUMJQ9mOPSsB13PdKN0+Ry0U1ah8XXRb8W2vSe3CpPkRJW/PPZ Zy/L1xjwOsa2lSiGq4ec+Eq5XMIKwkPczCVXuJpLIKEbz0mEknJ4w51MptTZJMM7wngG UwFDYq3t+8USVA1LaTRiX1bmu30g6NEfDZ7g6ejU9fyBHErTNU+FsxYD3Z96KdFPEzNP otrlqbwssp7WYFB6KNrubdF6usIWU0pcMXyyuVAZHu1Sg2Hg/lvKkAUKoWNKF4fLwtb6 A6aQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=y+A/NS4FgHHaJcqLidHtzs8QtkRkU9zyYH0MG9eYEcg=; fh=q7beNF6UnhnwP9VCUwtirh3iuWQNFcao1uoSTkO1cBE=; b=C6cl2ae97KaW0A8/5iji0t4h20GK7gXNWDVWiuWBsDIBtyVlaDYz5s7erydovZcYbz jCxaj9cTjGj67bQv/tmg7yi+ftMxvWf9veDJA6wwp/k0Pr6bwhnaNbYG9WPrlIAfHoXl FAu4TpGEogTk07BNBFaXWlTs8eFtIsKRZdVZUGbXRslMWiYT7QzoJwpagKRj7J6pP5+u 5uZsY9Qt5aXc7I+mdwLLtaquP8WB8OtcJe5N+XSkd4GVlVkVtjUz7+8c0+pzKzSAQHGH vkzDErNRDyMbLj7LHtkj5TJ99UK8h4MRxddWXC20dVG188mx+r4vU3FyiWGfmmqCYyQU BhXg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=Movz0Xoc; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com X-Forwarded-Encrypted: i=1; AJvYcCVV9ITD72XgU3nvzKf/k8LtK7bIQsF71QOKTeDsJAPK7Rkj7GDZ25XWk9QoJywqCK+YQep86XDg58UHcvr/JbnfE3KTb9XVtubG2Yq39vvaMGW/TdDA8DDgMpW268A3EHCVClePAZCMDiRXmdQHD0DgFBkO7shUi9XqpV13N0KcAzmMdkyMi0D8uI0m5yzOASsQPkq6BBOSWAh3ATz3eGiQmwC2G1NPjO7/l3dGgyq3Aer6zoRdno9zJDlbZPorbAaJLJ3BKXN5N+bt6585/jpl/ydGME5m3bhboqoRuKiPS2rbC+BeKElfqPrKzITSF31ns1V+R/ozzilmTrfWQSR5nbsD6VG1FLT2ausZPYKIq/eXThBZjCCEkpGjyp9o0VAfxFnk4BJMJxzNBTdL8bvNRs9iQU889xtzG+udnWkKfqGKAJZ0a796FyvwtpiE/pUl5lpAnqNx+elODQsr3XUg4D5cORuXzNLJyjYXHps+2gcqGki9snNJv4cPJ6aK6+nqo4zqffPXCjDuAwyRHUNIYgBscdECusAcRzAGIOSEjaq/8yL24yrYwkHCRg2G/gC+8WYBbJi8bARFgFXNTniT5MqAPZHaUV4g7JBzgcs04B2bKag/Oqo6rKRgPpME0yNTsecl4JM98SoSaN7MEwgDkQnYcFpQfcbj1LvrRLYUu+ZRnBCL9UXIrv0eEoqYClWmcE7WrYMxX3UcuMZ4WGBC+iFu/VDOkWXHht/1PuZxIIWqpLPzp0bE2Z9FFoPSpHyhLRQdlqdT1Sm7MJCwGVe2Vz3cXYB0y/mauuaWB0o0C+21va1eFeoceHzPFx6R51/ci08bV1Y5M2NZ1vik13btG1WprRwco4wCNa+lX1+5Y0N99latPRn5iY2CeR9iieByVE9G5ESP5qHuWvSCpSRhrXM+qpKYShKI4bozIugRCOxoWKNc3qt9rtHzGOqQivPCib u7Jv1UeqBjXwWbQK5HNnOnasyyRqwCylfSS3Vzt4r7hWcfS+XoGgSJL5JFLRCszgGWtRgdMHBsxv+B+nrGwrnNFqBNAJ2h0OZrz35emJGheBptiUqvB4Hfsi4wj2m3ZRhpLxUFVs4ZxxiQ2Hv4hxG9FhEMx43IbTVT7tjl0JMgrrnwju4eUlXVRPh7sVZA+ONHqTYtJpwoMDklHNpaynceiJmq4rQV3+3d2ZUisH7uXrTghzW5KKGC2PppbxT47YhRZd1DhOcwydmJIpgc6DwLRGUcEW+X9q3tEK4wU6B2UyRGsssi8lK4Lvj1mvyNiexj8HD+RuDkvn8gcJlGf5ySd2iHzsg5S464vinhlXxuspqNwjZ3EdYmjj9HH4AfLJEFLVNpobS4Cgj5yUOshyxwTYA20di9PArEUVkKy9qNPb+CM6qw3YiwhxyBojTyFkvakpR/s6YNaUN1dow3BwjLYO86tc4KLHqbbYYGLtS+0SBc54LSk25Dxhb9X0e0FknL72fNiP1UVtugOmpAaY5tPsPAk0R/Y+N2tLpeuc6xaj+A+AbUuY6GJ8+wC75BhQjiHB2Z7SlX22iFnNyEniTVUscqcgR4eu8bELil5MVzl4iQsFtuPeS+EmLIgyF8 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id gy25-20020a170906f25900b00a3496faabbcsi1719199ejb.544.2024.02.13.17.56.11; Tue, 13 Feb 2024 17:56:12 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=Movz0Xoc; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A3C4F68D1D1; Wed, 14 Feb 2024 03:55:43 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f49.google.com (mail-ej1-f49.google.com [209.85.218.49]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CAAF468D098 for ; Wed, 14 Feb 2024 03:55:34 +0200 (EET) Received: by mail-ej1-f49.google.com with SMTP id a640c23a62f3a-a3ce44c5ac0so248082766b.1 for ; Tue, 13 Feb 2024 17:55:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1707875733; x=1708480533; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2NMJSLFysE9TxkZix8jJsJb4xxeIhpp9vnuBQqDGZbw=; b=Movz0XocUNbVCQil2Bpr6iC0r8d3JS1aYqUbA6U4HOpmTPLuyDc5yB1YAYbK8loJJ7 nmPrgNRfZx7EslDDAf89xrsyKP9Ah3CSVuJ6PzK7bA3Qx3K3boAjKIa2inIfgT4Iiz5J CNqGXLzgt9hXR76y9GsZ+rYH9YIfYuYEREasG9EInoiCJKNE2avzXGmrKEq2vkVpf75Z mS3Lw3/8cp0A0B63Gm0B3MkHgPWwDoDyVpHYVtwwXuyjAYy47xuLKOUCxDt/vNsyBnFz OmvuPPYxVImZWCtYVMcrVAFymkx6KmPbnN/sVTU3ArgfGCLJziQxgMApuakPNiwJeuE9 xqhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707875733; x=1708480533; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2NMJSLFysE9TxkZix8jJsJb4xxeIhpp9vnuBQqDGZbw=; b=iZ+dFWwxhDzCoWgU9869M/nFo0iwwP9J8dF9Z94LXr2VMZVJRHkZw3lt0e1lahEn4w gjIht4G47GxEGc2uap0pRNJg5nHJTaSBqPzhux5iX8/bYUny+M4mxoixTQKxv4eYBERB jR5iT/h8MkzYW21kj3R00GCFoljd7jAre3hw7mUaGlBy8crOlMVUrJEH1Iowmo2ZI1X/ abTRQTfPyDc8LNNpq6N449y4s0nA8aejXF+N3mJ0kRwTC19wofc+j0qwUfrmxNJ3BAOR ffJK/CIzqsRszJmm/PjCtVucJa23PXaGP1190/LrhtszC1Mn7pjNvOlS158k18vyJlkI HyBw== X-Gm-Message-State: AOJu0YwKz3V8WrVyuBW/qez9JwKj3zX21546GByv5iaSqFv4V0QQDU/B NFs5nFTbTAThW9QjDdqMtnqIhtLKtUuDab/DeHrq6VmpmSykHLxEff/a1wW9zxY= X-Received: by 2002:a17:906:4a53:b0:a3d:20c9:6ba1 with SMTP id a19-20020a1709064a5300b00a3d20c96ba1mr740159ejv.39.1707875733655; Tue, 13 Feb 2024 17:55:33 -0800 (PST) Received: from dovchinn.amd.com (cable-178-148-16-149.dynamic.sbb.rs. [178.148.16.149]) by smtp.gmail.com with ESMTPSA id vi16-20020a170907d41000b00a3ce31d3ffdsm1663230ejc.93.2024.02.13.17.55.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 17:55:33 -0800 (PST) From: Dmitrii Ovchinnikov To: ffmpeg-devel@ffmpeg.org Date: Wed, 14 Feb 2024 02:55:10 +0100 Message-Id: <20240214015515.1027-4-ovchinnikov.dmitrii@gmail.com> X-Mailer: git-send-email 2.38.1.windows.1 In-Reply-To: <20240214015515.1027-1-ovchinnikov.dmitrii@gmail.com> References: <20240214015515.1027-1-ovchinnikov.dmitrii@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 4/9] avcodec/amfenc: HDR metadata. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: nyanmisaka Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: FJ5pt1NarS4x From: nyanmisaka v2: fixes for indentation --- libavcodec/amfenc.c | 83 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 83 insertions(+) diff --git a/libavcodec/amfenc.c b/libavcodec/amfenc.c index 0bd15dd812..068bb53002 100644 --- a/libavcodec/amfenc.c +++ b/libavcodec/amfenc.c @@ -36,6 +36,57 @@ #include "amfenc.h" #include "encode.h" #include "internal.h" +#include "libavutil/mastering_display_metadata.h" + +static int amf_save_hdr_metadata(AVCodecContext *avctx, const AVFrame *frame, AMFHDRMetadata *hdrmeta) +{ + AVFrameSideData *sd_display; + AVFrameSideData *sd_light; + AVMasteringDisplayMetadata *display_meta; + AVContentLightMetadata *light_meta; + + sd_display = av_frame_get_side_data(frame, AV_FRAME_DATA_MASTERING_DISPLAY_METADATA); + if (sd_display) { + display_meta = (AVMasteringDisplayMetadata *)sd_display->data; + if (display_meta->has_luminance) { + const unsigned int luma_den = 10000; + hdrmeta->maxMasteringLuminance = + (amf_uint32)(luma_den * av_q2d(display_meta->max_luminance)); + hdrmeta->minMasteringLuminance = + FFMIN((amf_uint32)(luma_den * av_q2d(display_meta->min_luminance)), hdrmeta->maxMasteringLuminance); + } + if (display_meta->has_primaries) { + const unsigned int chroma_den = 50000; + hdrmeta->redPrimary[0] = + FFMIN((amf_uint16)(chroma_den * av_q2d(display_meta->display_primaries[0][0])), chroma_den); + hdrmeta->redPrimary[1] = + FFMIN((amf_uint16)(chroma_den * av_q2d(display_meta->display_primaries[0][1])), chroma_den); + hdrmeta->greenPrimary[0] = + FFMIN((amf_uint16)(chroma_den * av_q2d(display_meta->display_primaries[1][0])), chroma_den); + hdrmeta->greenPrimary[1] = + FFMIN((amf_uint16)(chroma_den * av_q2d(display_meta->display_primaries[1][1])), chroma_den); + hdrmeta->bluePrimary[0] = + FFMIN((amf_uint16)(chroma_den * av_q2d(display_meta->display_primaries[2][0])), chroma_den); + hdrmeta->bluePrimary[1] = + FFMIN((amf_uint16)(chroma_den * av_q2d(display_meta->display_primaries[2][1])), chroma_den); + hdrmeta->whitePoint[0] = + FFMIN((amf_uint16)(chroma_den * av_q2d(display_meta->white_point[0])), chroma_den); + hdrmeta->whitePoint[1] = + FFMIN((amf_uint16)(chroma_den * av_q2d(display_meta->white_point[1])), chroma_den); + } + + sd_light = av_frame_get_side_data(frame, AV_FRAME_DATA_CONTENT_LIGHT_LEVEL); + if (sd_light) { + light_meta = (AVContentLightMetadata *)sd_light->data; + if (light_meta) { + hdrmeta->maxContentLightLevel = (amf_uint16)light_meta->MaxCLL; + hdrmeta->maxFrameAverageLightLevel = (amf_uint16)light_meta->MaxFALL; + } + } + return 0; + } + return 1; +} #if CONFIG_D3D11VA #include @@ -683,6 +734,26 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) frame_ref_storage_buffer->pVtbl->Release(frame_ref_storage_buffer); } + // HDR10 metadata + if (frame->color_trc == AVCOL_TRC_SMPTE2084) { + AMFBuffer * hdrmeta_buffer = NULL; + res = ctx->context->pVtbl->AllocBuffer(ctx->context, AMF_MEMORY_HOST, sizeof(AMFHDRMetadata), &hdrmeta_buffer); + if (res == AMF_OK) { + AMFHDRMetadata * hdrmeta = (AMFHDRMetadata*)hdrmeta_buffer->pVtbl->GetNative(hdrmeta_buffer); + if (amf_save_hdr_metadata(avctx, frame, hdrmeta) == 0) { + switch (avctx->codec->id) { + case AV_CODEC_ID_H264: + AMF_ASSIGN_PROPERTY_INTERFACE(res, ctx->encoder, AMF_VIDEO_ENCODER_INPUT_HDR_METADATA, hdrmeta_buffer); break; + case AV_CODEC_ID_HEVC: + AMF_ASSIGN_PROPERTY_INTERFACE(res, ctx->encoder, AMF_VIDEO_ENCODER_HEVC_INPUT_HDR_METADATA, hdrmeta_buffer); break; + } + res = amf_set_property_buffer(surface, L"av_frame_hdrmeta", hdrmeta_buffer); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "SetProperty failed for \"av_frame_hdrmeta\" with error %d\n", res); + } + hdrmeta_buffer->pVtbl->Release(hdrmeta_buffer); + } + } + surface->pVtbl->SetPts(surface, frame->pts); AMF_ASSIGN_PROPERTY_INT64(res, surface, PTS_PROP, frame->pts); @@ -746,6 +817,18 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) } res_resubmit = AMF_OK; if (ctx->delayed_surface != NULL) { // try to resubmit frame + if (ctx->delayed_surface->pVtbl->HasProperty(ctx->delayed_surface, L"av_frame_hdrmeta")) { + AMFBuffer * hdrmeta_buffer = NULL; + res = amf_get_property_buffer((AMFData *)ctx->delayed_surface, L"av_frame_hdrmeta", &hdrmeta_buffer); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "GetProperty failed for \"av_frame_hdrmeta\" with error %d\n", res); + switch (avctx->codec->id) { + case AV_CODEC_ID_H264: + AMF_ASSIGN_PROPERTY_INTERFACE(res, ctx->encoder, AMF_VIDEO_ENCODER_INPUT_HDR_METADATA, hdrmeta_buffer); break; + case AV_CODEC_ID_HEVC: + AMF_ASSIGN_PROPERTY_INTERFACE(res, ctx->encoder, AMF_VIDEO_ENCODER_HEVC_INPUT_HDR_METADATA, hdrmeta_buffer); break; + } + hdrmeta_buffer->pVtbl->Release(hdrmeta_buffer); + } res_resubmit = ctx->encoder->pVtbl->SubmitInput(ctx->encoder, (AMFData*)ctx->delayed_surface); if (res_resubmit != AMF_INPUT_FULL) { int64_t pts = ctx->delayed_surface->pVtbl->GetPts(ctx->delayed_surface); From patchwork Wed Feb 14 01:55:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitrii Ovchinnikov X-Patchwork-Id: 46254 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:c493:b0:19e:cdac:8cce with SMTP id eo19csp110310pzb; Tue, 13 Feb 2024 17:56:21 -0800 (PST) X-Forwarded-Encrypted: i=2; AJvYcCXGvimQiOUe0UQzy8GZxaRd/RdV08Doi35SvOKgRgNH4oqCDzFM0+iiV+t2Ma5EUiTsvwcD7uR24nbGL0zsgZn1taEtwnti5h927Q== X-Google-Smtp-Source: AGHT+IGklCTMMvFq7NEjs24eihmp6i3YTiYL9yO2N9JqPUAfe18Kv0oKbffsMOI66H5E83DVMN8B X-Received: by 2002:a17:906:1690:b0:a3d:4:8511 with SMTP id s16-20020a170906169000b00a3d00048511mr586843ejd.69.1707875780880; Tue, 13 Feb 2024 17:56:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1707875780; cv=none; d=google.com; s=arc-20160816; b=gfW8VGpmuK+0KnvkVzwicnl5fKVIlvStV97/FZPd+m/qFKVMxWFboRz0tk5xZilVuy xtjJ3A1EzMj47MMipT84mtB9fdsPIRRDlxhSF5nTs37TOrrG5RplpYautAzbpDP0P9et uQJa5ADe5vSaE0/GD2NZzG7LdUzQ7xAevIX3zAqJFS/4Ma+NH8XpJVhMtq3JpEKbfr63 on0XEm9u50e5gNiXilo34iVElIRXBZ9td+rzTGMEGzM1adJJ+dpYifkTO4jYOj4h9Rb9 XCpNU/srXtNzNuPCtQS9kKSYvjYoMVfecqFkVHEO+6uyYEXaEDQLKddRwg2W/Hhgsmu8 aArA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=Ok0JJTHY48jkD/S61nWlJFBZLNTcsW0JpXtPcxY3QDI=; fh=nBs/+bobZlVv35OJM1dLkFxZjBTd18L2aUuJZac7xjk=; b=Ok3OZZWj5jenBd94Jh+cPL6+xtZZ/Ekw+hZbxHDchPghxV1nYTxpdpBl0WfKC2l7B2 BZYwEqBJvyEWq2sCKV+ZYPIlmBrPdym7RhL2EADNFugWr5TH7x6UWbJUJcz/kpASlxgf DHNlOR8M2Cd0s87if3nrEnNq3tQk1bf8wj5p7pcr7dC6Vq3L8o8n+uxUV48xdyRNI29t cM8f8Gw/ev5tATypu2tWWnT20UYMvZRGtFejCjUtCJCfLrzZGm1hq9GaEvdONK052UXp et5YNWTlXl49ZNe/n59lBHMA0o1nlgibaRHsNuqFoUXzbotQCzx9DnfXAIggDxoQCDGc /FOA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=K8iwt68b; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com X-Forwarded-Encrypted: i=1; AJvYcCVHtJIakappcK4PAtkwjS62EEPC/faIxwzO4aqSX6UvCu/6uBPuK7FNbp2Fzot7bh5IXsVfsHZRvYoQLo9pdoRxUOxUfg4IUCVnZOIBtomp0ozoCIt8sQFp3ndANZ80LZzoSpBHW/IVFgMEYER8gCO1+GEOOJuEel3AKsVCreqqLZrBZ3mMYy5ai46Rnc/TFbvuqrNEawhwB//Ji5TXMoLTeQaUcnMNFlOhGdiiDbXM84E2ZXepF66X6MmbXx4NQ13OzOUBjjWtrNM9sI0rb+lCssZwbghrP36TWFlx/V0Xgf9s3C+EooWumaIWH/MuAfb9QsSqIncqMVInmnpXMCnM8b3xeKhTLJsxBtLkLJoiiEqCoXoMbK3uka9LMKRZiE6bxvg1fUaKlHbVbihPgs7pp/MTgnMpfAjoAJu/48uX5yX6cF8rCy1Pg/Z9Tj/Zs4GQ6llkLpM0FPamJTnLRFtpH9yBa6UgnF3DnIoLXLNIetqEmodFB8cOSrO+IOJO/1PdWeYyveo0hShvHYNvd6iL6AVpqJBsagi7H4jqQZ4qPZWr3f91doy5WbzgED72GKYXXjBwV4oATg7mBaLZWL/fBYqXCrzpfZQrJuNf/6+YLd+g3zT23chgXtLwLIrpUenNAmfRATFz5KFiVKYnSK+nAmpzeDHSYBDMKpva9Y40aFX4oE5KdsoMI1Uc9JbBUzfoMKUN3MAj2IQEavQwUgJqKH8EShIZRrCRKgrFnVAx66H/Hdql70im198pSRImTGgaaFHqlYnOosUEE0u6fW5kLk6WB63VIRojgFKMNBAcZ8E9QwrzLIW+UGV9u50UNgVL6QyR9CshezSzzM+tlPXwLdRrEU05qr9/PVVg9W+/5nWsp/jksVSHvQxLgPGR2VeI25fBURhepPrLwOHUfAP5//jQdqYtvPH/bHdtuzbf7+ssCKG6zsLGWzsDYDUIIb0mwA ZXrENZxlcPeRSJURl7RdQ3rg6y8szmjDQ/DQcglPJeOzBMC5KLbwrPH1anwg1yYZxveZAzwNXXlKyi5e7REOsfNUsQDcssowl7nTJAp/Teb4QPAZ4SmvPBBQjsr1Sm4ilPd8DGEQST1C2YZtAPhSiEmkcTEyc0joVBUSnYo/m69wgn3b19412i58x3dQI4C9mNirhioyVU/+P/B+M/szWoZrJSXpgH6Ulwmy8/47xNSY31jy2+6mvrtVDvVep+csmwcSMr+cIM5yUzCJRrwtCCESzrHE+yO61kX13/3oFUvMogDu0i+NgkfR735zkkC/SQ5o9RbYg+O1lUeWB06iKT2niWgD+NwRwfp5N7J0upGpVLdMOlRQTLLwGl2PKM3LagvFpO+JSSxcESR+HIbXOQ7ehdsiXPtLxNsdjQ5R8gapuvsdgBT/aqrYxy9m4m2gBw1kDx16O3fj93ClB/MR1bpxPUCQMLVQDCwQvqmTE/v/OM7EUQiQJgubE3yENr8uHMa/cv9LAW0PhL+hMnKMs3aOXUc9b/29pFGopwcS+Z/J36SdsSqJQY+UEis6fWuoYUk8QwvvtNZXCMhGHrGiUNPzi4lJZdWksZPYb3hJ6a/bA2iHogdSt0UgtXUnOlsuOSnum+ Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id dc6-20020a170906c7c600b00a3c5e1188c5si1779892ejb.486.2024.02.13.17.56.20; Tue, 13 Feb 2024 17:56:20 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=K8iwt68b; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id BE53168D1B9; Wed, 14 Feb 2024 03:55:45 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f42.google.com (mail-ej1-f42.google.com [209.85.218.42]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D706F68D16C for ; Wed, 14 Feb 2024 03:55:35 +0200 (EET) Received: by mail-ej1-f42.google.com with SMTP id a640c23a62f3a-a3832ef7726so596510166b.0 for ; Tue, 13 Feb 2024 17:55:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1707875735; x=1708480535; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AcMOmLMKzscjDGeJvZ0PfrBVtpBiKsUbNlGdEdzAOgg=; b=K8iwt68bKoz7k9dbtuo7UTB5K9owDEKiGzx0FZ6aqw8Us6G8fQI7JKWaQjaguermXt aZg5xpFj4kDNLS2Cb7bRap5yIH0YISsR8sMnYcskNnwvZNiWTUoGiqd+sa0ZT5w3IUEH CT81jZ3CJ5T6pSzJmi9Rdd78jtJ9cSSxO7KVKt3255W+pRHtdmP2YcIC/6if2FSLi5YQ daEOCehzlJihEAfRtoAYsCQtjKWf6qyjrSBU5AG9ruKyhR3U85mBvfbE5Yur/5YUEt0d s1Y8A8tzMqqSXfnVRofB9OV5oKCHBU69p0aI1ai4wbS50nP+hEZT3Jm5S+i13CEjDfEO d7Kw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707875735; x=1708480535; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AcMOmLMKzscjDGeJvZ0PfrBVtpBiKsUbNlGdEdzAOgg=; b=haDc8qLKyhe+ydjbL0gijM0WnrkentoFZWncoZgP0fBbvxuUpnBiP8PoFV2VR5JuQd wDB4l77gzy+jtgdU4qU7iWNoVbtQdgPCBv2FoO5yx3XI6wtwn7Xye88alwgPER3WWDAi 8/II6mIZiqwf9Pk+pYy222wiWuSE5CbQDN1P4NajgrhOeh6eo7aPTQw8PTOzshBd1vqG 12m2uvIpNrgCw0H8JfeCrjRDEtC3aNEiTobkJyrANa24ozsiP1IXpg1+w/kVhJaJ9PhQ BCgB+Eg8hKOFHtCVcgJrJDF6gYJUbeUjZ3cOaq07nIrLRjicG5zap4fwo8BK//YeqTdC Gqkw== X-Gm-Message-State: AOJu0Yw7jfFuyQF8K00i9akjROtU4IDMgiocYkGBRD+nabYjDtF2KeTz iD4wdgwowl5CuuCtdIQBwFuVotRTrSSarIl3xcqabw0P7Qwo1JP5otqeXU31kfc= X-Received: by 2002:a17:906:f6c3:b0:a3c:f71e:65dd with SMTP id jo3-20020a170906f6c300b00a3cf71e65ddmr658232ejb.59.1707875734679; Tue, 13 Feb 2024 17:55:34 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCVLXzihxX9APZVeBtZtEEjJD20IygVsQhWxLuWNSDegxN94SE8/SSpZCeKB+OnqQCMe5dcwIipuTdiN3ghg7YzHKC9Cs7IvYsHjWaLJoA== Received: from dovchinn.amd.com (cable-178-148-16-149.dynamic.sbb.rs. [178.148.16.149]) by smtp.gmail.com with ESMTPSA id vi16-20020a170907d41000b00a3ce31d3ffdsm1663230ejc.93.2024.02.13.17.55.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 17:55:34 -0800 (PST) From: Dmitrii Ovchinnikov To: ffmpeg-devel@ffmpeg.org Date: Wed, 14 Feb 2024 02:55:11 +0100 Message-Id: <20240214015515.1027-5-ovchinnikov.dmitrii@gmail.com> X-Mailer: git-send-email 2.38.1.windows.1 In-Reply-To: <20240214015515.1027-1-ovchinnikov.dmitrii@gmail.com> References: <20240214015515.1027-1-ovchinnikov.dmitrii@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 5/9] avcodec/amfenc: add 10 bit encoding in av1_amf X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Evgeny Pavlov , Dmitrii Ovchinnikov Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: iurYnmpydqIQ From: Evgeny Pavlov v2: refactored after review Signed-off-by: Evgeny Pavlov Co-authored-by: Dmitrii Ovchinnikov --- libavcodec/amfenc.c | 2 ++ libavcodec/amfenc_av1.c | 22 ++++++++++++++++++++++ 2 files changed, 24 insertions(+) diff --git a/libavcodec/amfenc.c b/libavcodec/amfenc.c index 068bb53002..f1b76bd6aa 100644 --- a/libavcodec/amfenc.c +++ b/libavcodec/amfenc.c @@ -826,6 +826,8 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) AMF_ASSIGN_PROPERTY_INTERFACE(res, ctx->encoder, AMF_VIDEO_ENCODER_INPUT_HDR_METADATA, hdrmeta_buffer); break; case AV_CODEC_ID_HEVC: AMF_ASSIGN_PROPERTY_INTERFACE(res, ctx->encoder, AMF_VIDEO_ENCODER_HEVC_INPUT_HDR_METADATA, hdrmeta_buffer); break; + case AV_CODEC_ID_AV1: + AMF_ASSIGN_PROPERTY_INTERFACE(res, ctx->encoder, AMF_VIDEO_ENCODER_AV1_INPUT_HDR_METADATA, hdrmeta_buffer); break; } hdrmeta_buffer->pVtbl->Release(hdrmeta_buffer); } diff --git a/libavcodec/amfenc_av1.c b/libavcodec/amfenc_av1.c index 3f164ccc59..9b32616aa8 100644 --- a/libavcodec/amfenc_av1.c +++ b/libavcodec/amfenc_av1.c @@ -165,6 +165,9 @@ static av_cold int amf_encode_init_av1(AVCodecContext* avctx) AMFGuid guid; AMFRate framerate; AMFSize framesize = AMFConstructSize(avctx->width, avctx->height); + amf_int64 color_depth; + amf_int64 color_profile; + enum AVPixelFormat pix_fmt; @@ -203,6 +206,25 @@ FF_ENABLE_DEPRECATION_WARNINGS } AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_AV1_PROFILE, profile); + /// Color profile + color_profile = ff_amf_get_color_profile(avctx); + AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_AV1_OUTPUT_COLOR_PROFILE, color_profile); + + /// Color Depth + pix_fmt = avctx->hw_frames_ctx ? ((AVHWFramesContext*)avctx->hw_frames_ctx->data)->sw_format + : avctx->pix_fmt; + color_depth = AMF_COLOR_BIT_DEPTH_8; + if (pix_fmt == AV_PIX_FMT_P010) { + color_depth = AMF_COLOR_BIT_DEPTH_10; + } + + AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_AV1_COLOR_BIT_DEPTH, color_depth); + AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_AV1_OUTPUT_COLOR_PROFILE, color_profile); + /// Color Transfer Characteristics (AMF matches ISO/IEC) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_AV1_OUTPUT_TRANSFER_CHARACTERISTIC, (amf_int64)avctx->color_trc); + /// Color Primaries (AMF matches ISO/IEC) + AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_AV1_OUTPUT_COLOR_PRIMARIES, (amf_int64)avctx->color_primaries); + profile_level = avctx->level; if (profile_level == AV_LEVEL_UNKNOWN) { profile_level = ctx->level; From patchwork Wed Feb 14 01:55:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitrii Ovchinnikov X-Patchwork-Id: 46255 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:c493:b0:19e:cdac:8cce with SMTP id eo19csp110347pzb; Tue, 13 Feb 2024 17:56:29 -0800 (PST) X-Forwarded-Encrypted: i=2; AJvYcCUE8U6xkvFxQqHZmqghJgqpykke9kGBfCztbilMktJ4kta63iAgVK9ls59v7Q5QM/3bMz4WcG0+GlKy3DynIt/8/CdoGxGOAaNmbQ== X-Google-Smtp-Source: AGHT+IHQFYjNineiSOOYjR8aTNkxpGr6WKfmAku8jNbKsVO/4K7+yHQUAj/oF3SBcojSSeY1G0/b X-Received: by 2002:a17:906:b48:b0:a3c:a3eb:6109 with SMTP id v8-20020a1709060b4800b00a3ca3eb6109mr768862ejg.14.1707875789312; Tue, 13 Feb 2024 17:56:29 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1707875789; cv=none; d=google.com; s=arc-20160816; b=ERZ0I7A3ChIfR2ZVoUisdvPAUvjDscEmuA8tzwruSs4TFmpJhw36AvOCySls1+Xhpg 3izZ83X/OVaa5KYwK7BjhAWd2VGqY7uCpG5VoXGiDyoQX6za3ilGmlswjqGIOC9AcSEw P4j7XlBANFgdltwI8GK3QG9/TYWT+oB5UZ9FSTFLIVEsLhFhr5bWfKLVy0RAIBP5Sjq5 8wVpccxTqx/iEdMh24XoVrxOIXUWKLhONOl1VdYbymAhy3xN+n5qwtWRiNDM07GKC7Cg dbvNUOQMJv2XISnCSwZuePaenTIXOJyBxddqgf74xZqO0/qCmKw6lKWsAPNr46dgv87e xWDQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=l3Af3ezltufxaW16d2i9TmdUcC+vMh2SRwm2VSaZdzU=; fh=rbgWa6GtZdCI0QIN02wUAp1qOBvbMteUM3x4/JsdIjA=; b=xKXskdfOhm3t1i2cEKoAWWgSufgV/W9DL0ewsxjtWftNKrU6LwfS4kYS6ATT72rL89 DxDPoK3r87Yx8ueh/i82/v57RT5csxyiENxcz2i/JBTls4zpps+Di3O1awI1wNhJ8mgR KdIzudUiAITwK8vLXk+Ji8EcNgwLGHjyBkR9Wqw9j6cB994TP6z92FlSxGfKLlOMUfSR 0bNEPH6/PhqsxK4UvWSPJBSkQnyEaKnIWaXidA6iPVH/h/wvxribNkteQefZtXpRMcPa gFXGz8g0c1bUiD5uoqg2dnwY3foILH0Ac+RXB4+pU7wfmMyX1d7s8bVz6AHRv8NnWKSb WIEw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=e7ckCv8z; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com X-Forwarded-Encrypted: i=1; AJvYcCUUQ/23LFKdCyBO5ZQCHDF2ONwEUeaqMe/vWYXhgnpp1oMGHtWvuEdLGT8p+ZBX9C7aHrbQvgFt495/ZIYEX659hs4fRQZrF4NUVfLPEoqEvvJMF7q9DGlmjihywvzoTLt0cu0mx5UH6apbaXPCNyHzuKQiTYLUOtSZJPBkCXTnQ1CuN+20sDU0EXCOaH8Uv7LmIyHOSro3MsQG2XV9lfF5vzqWMxHi+NoAA7F+DiCu90ed357WhIHnnkjH/Y1Xs6XGqs/EoWg4ypgC1WQjEn4ZlAL0zepo8gbOyZ0dG+H7lutTmjeKbiMYd/WAY5fKSuS+UkVEU6BSzeoV+Maq+jCCAjdprjP5D9ykS+jQg0qttD3JpTFrgZWDB9OX/MX8jEq1IFn+SePyNc1vT47wKHpqg0H3xQl8fuZCq86Rw27tx+9lF4F+7eDaokqz1RRktu/TOw2bki37CZ0wTBirc2chZUZJRm40v+1IYdmJihWTirOzwLmfUVqrDMCVA2LelYmNI9GmXsUV1Fu+qbixCZDfS9TJBsl/PGx/BbzujzaW3g959vh1HWzh9QIRTOAaqyZBazcMT3PUCXH3kTf/1yMKixyb/w8OLPcstb7QsfBmwJvyQFwyQNItA465Ipfajr6BgEQYbl79UTlryiI9ntc/2gBAP7o4J45lDUg+7RcQkjYk8F7zAopBvkBSKDRuXU8C+nmTCyVeA6TaOcURHp8PY5bI3RCUfV4G4g3Uq8kwgfQz4zkR6bJQPsqTwZXXGnVXlCX0Ykv2ZYdOPz/SMpu9etf+SdpDEvcUEqKlV3N/czsCmhR+J8veXUlpFzIo/TfcG0QvrZ/J4D8AnVzHtFMR7eLf0RRy3+5HkJMH4+1PybPMRjjou/Yhr6uKOoJYLlC8luKWqns0zpVYS8rqLBACSREOc4EWoRodSjmsRvunu7kH9EBPfgIAQcl+SY9YBbbooO bVwqW6mO1A0m734ksCLDmUQL+BWr8DlhZW/EZVtRyoS4dsLoA0Vss27WLUzrQi2Pg3qq1vWRIr/lN5ey3FCg+bQCG7XuKb8WB61nhIegr3gwwcTIY6n8uHoTD4VEcDmNMJHf8VkBAGE7jeNNOuiYmax/BdIRXfaVPyBw+A/RVuLPiFINKWDjqNpRwQgLxRLQukqBkQHnbOhiN7ry2GH/YyfNAcwsyvwexj5uMeUD1lsTg6uYfOsJ4LAujrMiLRKU3gOLCsVGWP+AsQW2fd/z935BxPxCXLB9vYwOyBlyb5F5NKF9I63gC/71QdmMjncyqlneFnTmlYfPA4W0kfORZuBA9gfmeh3+kuMoo8qVzBiEdc95YqOy/MFuvRw+sodgYFGkKDmdGP1C9QtnW+FnC2v8dNn7PchOZBwCHzjw84FKZhzrsuibw7IBbtMYC4Np5HeI0+ahfQlNpB4pZrzCSRBGffDoKOFPpYZPAt+uwtp5ylxECZXQv3mQvcmMA4MxzGpzzxlDY6+jXFRwH4meSf7guint2prA1XEobpNhTozy4F7zY4EtOpKQYnl7eY7FO69DJ7QAU0L1B8eXGFncQQ1F5ODsT2C6lUIA== Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id f13-20020a1709067f8d00b00a3d1de3e2d3si696392ejr.589.2024.02.13.17.56.28; Tue, 13 Feb 2024 17:56:29 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=e7ckCv8z; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B15A868D1E7; Wed, 14 Feb 2024 03:55:46 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f52.google.com (mail-lf1-f52.google.com [209.85.167.52]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4142368D174 for ; Wed, 14 Feb 2024 03:55:37 +0200 (EET) Received: by mail-lf1-f52.google.com with SMTP id 2adb3069b0e04-51183b02564so2044508e87.0 for ; Tue, 13 Feb 2024 17:55:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1707875736; x=1708480536; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=cPfA1ZQAzIpIR+NGiJp9+kOvr5Z6l8DJQTAmmdn4Up8=; b=e7ckCv8zW1fBnMEzTz1XLh+UaAI60pF+wPSZZGAQAmERpJ/JpWQh3JDjstgz8zywf6 wg4ZF7U6qQMSCR0DQOgbdcLBuFcQ0Bvprq7iQxL6ipOok2LMbkNhqvPfdh84Cm8P7jvS qiiQ53AU47pU9U2d3rtXaayZCzxiF5aspKsHqVHYI88vVtknB/en371Yx8gjCRpnjXIt OzT6x3l8JHXuzMS1rZsAub3hA05jHSmLiUO7NVQ6/7UdOVURTu+k6yyybLyOwTgpRqz0 mx+koEcgaxuxKMgGy7YEGX4jpKib0AASS5KqrBqqasnr7Sm+EqQU9w5PTgJLUyUbYrwi 3t0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707875736; x=1708480536; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cPfA1ZQAzIpIR+NGiJp9+kOvr5Z6l8DJQTAmmdn4Up8=; b=s18nWXwLOP7k3GRVA5WQBwFkJkadtp4AorspItmJ+i1ZIPd2QFdabI2oLrl4TLxk8n HIaNEdfOoumqje8haO8E1PT0KB8Ujci0b6YtQLHuKzYBtp8B1EVV5ivnyIW6oERWjkTF 4o+317Yn9YvFWaM3ZeK79ri/K7ppr5S25ov+1ET6pSYiG21RbtDhhArzdzy1tTju2El2 JcdgjeDywNRtE7xydNft2cKe/K645IpJ79d0GLkCC0oHxKVoWMHn/rnYaATyZeQuTPaD ATwVgn0qLRr5iqKhlI7BrQd2lTWcCggo4PjJG0lu0dk6yrwOHsswgGnbP3ZFSPZq9N17 Gqbw== X-Gm-Message-State: AOJu0Yw/15vBCOMK23TUvYJogkeAb/HtgkR8H24/ioAEeCV6xRTjzPZE Dvx+929pvXOzfe+ruAm0cOXYt1i5z3DjnWBRVPMRdJ/vr8kpIENmulCihUGYp74= X-Received: by 2002:ac2:530e:0:b0:511:706d:72f2 with SMTP id c14-20020ac2530e000000b00511706d72f2mr722668lfh.68.1707875735746; Tue, 13 Feb 2024 17:55:35 -0800 (PST) Received: from dovchinn.amd.com (cable-178-148-16-149.dynamic.sbb.rs. [178.148.16.149]) by smtp.gmail.com with ESMTPSA id vi16-20020a170907d41000b00a3ce31d3ffdsm1663230ejc.93.2024.02.13.17.55.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 17:55:35 -0800 (PST) From: Dmitrii Ovchinnikov To: ffmpeg-devel@ffmpeg.org Date: Wed, 14 Feb 2024 02:55:12 +0100 Message-Id: <20240214015515.1027-6-ovchinnikov.dmitrii@gmail.com> X-Mailer: git-send-email 2.38.1.windows.1 In-Reply-To: <20240214015515.1027-1-ovchinnikov.dmitrii@gmail.com> References: <20240214015515.1027-1-ovchinnikov.dmitrii@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 6/9] avcodec/amfenc: add smart access video option X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Evgeny Pavlov Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: tisVOP0B48yM From: Evgeny Pavlov This commit adds option for enabling SmartAccess Video (SAV) in AMF encoders. SAV is an AMD hardware-specific feature which enables the parallelization of encode and decode streams across multiple Video Codec Engine (VCN) hardware instances. Signed-off-by: Evgeny Pavlov --- libavcodec/amfenc.h | 1 + libavcodec/amfenc_av1.c | 18 ++++++++++++++++++ libavcodec/amfenc_h264.c | 18 ++++++++++++++++++ libavcodec/amfenc_hevc.c | 18 ++++++++++++++++++ 4 files changed, 55 insertions(+) diff --git a/libavcodec/amfenc.h b/libavcodec/amfenc.h index 62736ef579..1bda0136bd 100644 --- a/libavcodec/amfenc.h +++ b/libavcodec/amfenc.h @@ -90,6 +90,7 @@ typedef struct AmfContext { int quality; int b_frame_delta_qp; int ref_b_frame_delta_qp; + int smart_access_video; // Dynamic options, can be set after Init() call diff --git a/libavcodec/amfenc_av1.c b/libavcodec/amfenc_av1.c index 9b32616aa8..109533fb2d 100644 --- a/libavcodec/amfenc_av1.c +++ b/libavcodec/amfenc_av1.c @@ -104,6 +104,8 @@ static const AVOption options[] = { { "log_to_dbg", "Enable AMF logging to debug output", OFFSET(log_to_dbg), AV_OPT_TYPE_BOOL,{.i64 = 0 }, 0, 1, VE }, + { "smart_access_video", "Enable Smart Access Video", OFFSET(smart_access_video), AV_OPT_TYPE_BOOL, {.i64 = -1 }, -1, 1, VE}, + //Pre Analysis options { "preanalysis", "Enable preanalysis", OFFSET(preanalysis), AV_OPT_TYPE_BOOL, {.i64 = -1 }, -1, 1, VE }, @@ -265,6 +267,22 @@ FF_ENABLE_DEPRECATION_WARNINGS } } + if (ctx->smart_access_video != -1) { + AMF_ASSIGN_PROPERTY_BOOL(res, ctx->encoder, AMF_VIDEO_ENCODER_AV1_ENABLE_SMART_ACCESS_VIDEO, ctx->smart_access_video != 0); + if (res != AMF_OK) { + av_log(avctx, AV_LOG_ERROR, "The Smart Access Video is not supported by AMF.\n"); + if (ctx->smart_access_video != 0) + return AVERROR(ENOSYS); + } else { + av_log(avctx, AV_LOG_INFO, "The Smart Access Video (%d) is set.\n", ctx->smart_access_video); + // Set low latency mode if Smart Access Video is enabled + if (ctx->smart_access_video != 0) { + AMF_ASSIGN_PROPERTY_BOOL(res, ctx->encoder, AMF_VIDEO_ENCODER_AV1_ENCODING_LATENCY_MODE, AMF_VIDEO_ENCODER_AV1_ENCODING_LATENCY_MODE_LOWEST_LATENCY); + av_log(avctx, AV_LOG_INFO, "The Smart Access Video set low latency mode.\n"); + } + } + } + // Pre-Pass, Pre-Analysis, Two-Pass if (ctx->rate_control_mode == AMF_VIDEO_ENCODER_AV1_RATE_CONTROL_METHOD_CONSTANT_QP) { AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_AV1_RATE_CONTROL_PREENCODE, 0); diff --git a/libavcodec/amfenc_h264.c b/libavcodec/amfenc_h264.c index f785e091c9..f83a9fcaa7 100644 --- a/libavcodec/amfenc_h264.c +++ b/libavcodec/amfenc_h264.c @@ -136,6 +136,8 @@ static const AVOption options[] = { { "log_to_dbg", "Enable AMF logging to debug output", OFFSET(log_to_dbg) , AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, VE }, + { "smart_access_video", "Enable Smart Access Video", OFFSET(smart_access_video), AV_OPT_TYPE_BOOL, {.i64 = -1 }, -1, 1, VE}, + //Pre Analysis options { "preanalysis", "Enable preanalysis", OFFSET(preanalysis), AV_OPT_TYPE_BOOL, {.i64 = -1 }, -1, 1, VE }, @@ -369,6 +371,22 @@ FF_ENABLE_DEPRECATION_WARNINGS av_log(ctx, AV_LOG_WARNING, "rate control mode is PEAK_CONSTRAINED_VBR but rc_max_rate is not set\n"); } + if (ctx->smart_access_video != -1) { + AMF_ASSIGN_PROPERTY_BOOL(res, ctx->encoder, AMF_VIDEO_ENCODER_ENABLE_SMART_ACCESS_VIDEO, ctx->smart_access_video != 0); + if (res != AMF_OK) { + av_log(avctx, AV_LOG_ERROR, "The Smart Access Video is not supported by AMF.\n"); + if (ctx->smart_access_video != 0) + return AVERROR(ENOSYS); + } else { + av_log(avctx, AV_LOG_INFO, "The Smart Access Video (%d) is set.\n", ctx->smart_access_video); + // Set low latency mode if Smart Access Video is enabled + if (ctx->smart_access_video != 0) { + AMF_ASSIGN_PROPERTY_BOOL(res, ctx->encoder, AMF_VIDEO_ENCODER_LOWLATENCY_MODE, true); + av_log(avctx, AV_LOG_INFO, "The Smart Access Video set low latency mode.\n"); + } + } + } + if (ctx->preanalysis != -1) { AMF_ASSIGN_PROPERTY_BOOL(res, ctx->encoder, AMF_VIDEO_ENCODER_PRE_ANALYSIS_ENABLE, !!((ctx->preanalysis == 0) ? false : true)); } diff --git a/libavcodec/amfenc_hevc.c b/libavcodec/amfenc_hevc.c index 8c6401c646..f3caa7cb29 100644 --- a/libavcodec/amfenc_hevc.c +++ b/libavcodec/amfenc_hevc.c @@ -100,6 +100,8 @@ static const AVOption options[] = { { "log_to_dbg", "Enable AMF logging to debug output", OFFSET(log_to_dbg), AV_OPT_TYPE_BOOL,{ .i64 = 0 }, 0, 1, VE }, + { "smart_access_video", "Enable Smart Access Video", OFFSET(smart_access_video), AV_OPT_TYPE_BOOL, {.i64 = -1 }, -1, 1, VE}, + //Pre Analysis options { "preanalysis", "Enable preanalysis", OFFSET(preanalysis), AV_OPT_TYPE_BOOL, {.i64 = -1 }, -1, 1, VE }, @@ -265,6 +267,22 @@ FF_ENABLE_DEPRECATION_WARNINGS } } + if (ctx->smart_access_video != -1) { + AMF_ASSIGN_PROPERTY_BOOL(res, ctx->encoder, AMF_VIDEO_ENCODER_HEVC_ENABLE_SMART_ACCESS_VIDEO, ctx->smart_access_video != 0); + if (res != AMF_OK) { + av_log(avctx, AV_LOG_ERROR, "The Smart Access Video is not supported by AMF.\n"); + if (ctx->smart_access_video != 0) + return AVERROR(ENOSYS); + } else { + av_log(avctx, AV_LOG_INFO, "The Smart Access Video (%d) is set.\n", ctx->smart_access_video); + // Set low latency mode if Smart Access Video is enabled + if (ctx->smart_access_video != 0) { + AMF_ASSIGN_PROPERTY_BOOL(res, ctx->encoder, AMF_VIDEO_ENCODER_HEVC_LOWLATENCY_MODE, true); + av_log(avctx, AV_LOG_INFO, "The Smart Access Video set low latency mode.\n"); + } + } + } + // Pre-Pass, Pre-Analysis, Two-Pass if (ctx->rate_control_mode == AMF_VIDEO_ENCODER_HEVC_RATE_CONTROL_METHOD_CONSTANT_QP) { AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, AMF_VIDEO_ENCODER_HEVC_PREENCODE_ENABLE, 0); From patchwork Wed Feb 14 01:55:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitrii Ovchinnikov X-Patchwork-Id: 46256 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:c493:b0:19e:cdac:8cce with SMTP id eo19csp110401pzb; Tue, 13 Feb 2024 17:56:38 -0800 (PST) X-Forwarded-Encrypted: i=2; AJvYcCX0AhmWPOTgHcxTQm0wTJqkR7tAo7XIRkSfXw3JvZzMjWhCICWzPChvPuZMtwfa2YVja4RZjQnA2vRmee1o0S8HBOnheXwR3RX46Q== X-Google-Smtp-Source: AGHT+IHcnlsA16Vz7lOgpcNelZjBysnWhZ7/4j4R6xS1a24Csi2WUUNk4ttlyp8DKkrw5IAR881z X-Received: by 2002:a2e:a68e:0:b0:2d0:dfa7:fa1a with SMTP id q14-20020a2ea68e000000b002d0dfa7fa1amr786725lje.0.1707875797776; Tue, 13 Feb 2024 17:56:37 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1707875797; cv=none; d=google.com; s=arc-20160816; b=k2/n8c5iNioieKkbGuNks7SvKdr1UT7+KKV2tg/ZA7QAEQYC6i74WO/OYwNkRbNrEt ybwqcRZCU0Hf90lj46f7eBBgCIeaFcpIfbHqp6r/UlTxGNbVMW8yIOHdsxKVLn4JaSQb bT7saXM9J6+Z2HKOFh7p2mRTxs0Ls2bexI+8jLe4HeCmbJKAMVk/4WxHs+ZVnfzlP9fx RaQyE8+VjkeMW2G5xh4By950jsEzwaKll5Nr1z01q2phWrIxWYGLKY3Rmr4OZ6qnftw5 V1Ec+eMiAcqPsFq1C8u56oKZ6CzcW+qXLJY1YB1AbG3Gu5GoQ75g1tS47yhmAFm6X/Wc iFBw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=buzGBDLBR6pczOA1rC7gOi9OMuOcKuBzQ8nI3fKo50M=; fh=GuxXn2KGWHQ3b+4lveRrpJyIqLe8Vx6h/aX0stozvw8=; b=RtdoPBWDIVcKxfrOwW8CGpe6V4xKlBxV87l2tdm7SRfeOgCnnzScEYCS8TpKs/ux56 apj1yS9r86iB97RHR+P1rp5Oj7bAL4XygiBDVNoawAuPvmvqzrN4E+wcBlJCLGd9E06p 3cIFDqdHCBKOGWICbUnkyJPQkrPQEN2iXZNFhTug1bxws69QLRzGhUcozUNWd5cE3eyg u7KCYWE3UZ0hivgFaNKsGyq8rKus7bt7KRrtfogmT25hnfniCJmPOVpSw1ecTz6WQoRf O+qp3Ihx4s/LbGq7rubeAVN/UlA9zCKXmUMuHwt0yNsiQBgmD4KFzJv8N9UprahwX9kj Z13g==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=fjXJxZI8; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com X-Forwarded-Encrypted: i=1; AJvYcCX+aW/oMx8Cswhh0wTmN9qFBrWisj7h4Pe0dYJr6/gejruDdI+Ye0LPBtWmsHKLrx/nsgeC1LCcyB7wr7P1l8Onq9ilRwyV+L+8kENV+Uh1vacy8s12P4H/mEF5IYwMLPyVtcIJMFrv+gVch53P+UOYYFgPqB/IquxSE8u9moFQm27xhsrbOCAItTAwFbtYAZrmxdoltzvv/gM+GdQMy49mZZfiAOq1jVb+EnHS33jBeqVVBNBAYpguI5jB5UEU+bSdVPLUl04yurS+1q9H+NZAttKJIt9I5biw4wX9LBD55T9YnlpsgWHMJsUN5ZvsxPhguBkIPfMizO9qHxcdM/eNEKU99PgWkZmd8nJVcbLjJMmiosmT3mg2Lin+k3Rde9IS7XJ5+CKpqWQE/XH36Lx+ocHTgx940I6uEXRtX/V2FxsvIYQg3lHIwPeSPWo8i8I2y4WvZjOJsRPaszioqzBZg6ozSlngK6Tiqxr8D3kD4Pahtvtxf7OyqDBdIHqCckjXm3/8qx7geq9bsOec7XaEjRlcp6l7gVYB9Dzs68uxmCY+R9O2ooPTmIP+QXedzdhHneVke0iR5tNlj7rHXT7T4HU9PqXESrrn5JEUOgd6yfAx90AhKuNhjzcVUdyDBWpQJ5ou2MeayunV0AORu8dsbEN13YgpHpyS4b1gp6tQhahxbrk+t/eiedEM7DeBICpUthBYymKHPLy83aimud+x/0TjfgZ3ecc/V8ZkjRmefdutA4FJO2rV2WPmuejAyhlMoaysFSdQi2c8iZCYMQtnDjRSRWK0vXmODNq7GNe1MM/8zsCAZ8evcTtCatjal/2S7xZyGwznlfJ8ARTOrH+IpS0/nNuZijjVVU05LLaF0Bmu587jC0dCU8IHilqo9FR0Us+uv09N2Ee0weApKXxDI3ChpFzBW70FEdohzopzahfvsyMKDRkZDolShTH52dxp4x rc4CUbrleSs9/DyCCJOpRR0fHSk+7BxBRpxpWvutWhGEWSN+vCSWGYLwW2JAjMc/SyLcdHh8nddiNmLKzoKGMvTB97OhQD+iLrsFibcxIi4J1P/h3TrxxgoqMn9UMLJ+hhAPvf/mGtKYvZPZGHQJmJkj6iZJ9hbmgsMg6kDov6n7oRUsOCa2E83okEJbzbxEn7N2jUQp13POelcnI0i734hvu9AtLYuVM48tV82uH2hjlDQ/Sx8xXnSTOZfE+ATEP1Ud7puCQcW00eMADmjlAhgFo21D4e7IlDI8VZKb4OHxCTPwHTSS/lsvUWz+anSlmyNCwd+ULNYZ4MWngAy8O/XFezKOnYeU0PIjw8VgJGQ7ZNy12bZlxvKjj2jRBp/B4fbxNu9BLp3J4+vT6oNamaOakpv4oOnhGTs7moYWWPbXyVbVkhtwQ+Tym5YKCgeOXdhQi7T6aqjyCCqLzswy1+EtvFgd9Hpe/4Q5ZoLdGtK+BFSr1kXJK9dxybQh7rK0SnCGZ4q4T4baF8AbDB+kemmELZmyCBUlqzUkr4NLjzZTSD/jBqHFOe0pZahUHVT3ZGLmn6BcoVfFfa2NtutkvCJXRESNiojb1UspghNCdXBKz++3OWFGz2B4ig/yfp Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id y15-20020aa7cccf000000b005621aeec21asi429388edt.527.2024.02.13.17.56.37; Tue, 13 Feb 2024 17:56:37 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=fjXJxZI8; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A710068D1E5; Wed, 14 Feb 2024 03:55:47 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f43.google.com (mail-ej1-f43.google.com [209.85.218.43]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 20B4868D1C3 for ; Wed, 14 Feb 2024 03:55:38 +0200 (EET) Received: by mail-ej1-f43.google.com with SMTP id a640c23a62f3a-a26f73732c5so704687966b.3 for ; Tue, 13 Feb 2024 17:55:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1707875737; x=1708480537; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FHI0xs6M4feCybq8tD/CYzMneSStTfJRyXdOXi1j1Kw=; b=fjXJxZI8LhDbcF17lN4QTlIlJ2QJlFdPY/ACaSwk20r4e7IlgxeEZ8mRQcU2SG86iF uAsDzkCWYneHcgGVxCxGe32tOPYZlzb5G3H0Fe05tl/WHilXaZRQxbWuNu6JaG0tyMw4 NEy9AV8JR5ySvq6aNgQJQ4hKRP+iLQbsET2nvr+HMcHfys4HsHcUtEKmX6oriAUiyuOZ e/XhB7kJlLPnkWAqvOqSWjKvLYIq24+BrqIyNWINIH0g/PrUrs/ewo2BkxnZ08uR5Q12 OrvDnM4kWUEUYZy8bzB5JQaqIMFVEw6+Uk0v5Pj6hrSvqNGsdvRWPd3q3/fEEdk4iYgu 8Y7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707875737; x=1708480537; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FHI0xs6M4feCybq8tD/CYzMneSStTfJRyXdOXi1j1Kw=; b=bvNczhNRuvUQxa+0T5aI29BBbbmAOerKZZ88OJuPzFS9miOcjugPIhtY68FYFVyXID Ykg4mpItYei7G6zU3fU44StZhqc37B5ioLCQYCi8AcwON4WiskAnE7CSzJ5TvC0PBOoq ZzuCHdcLeZsXvCOCgNOaHasPW0BKwOAZUSs0GpaLzIWiIOx5l+HcZ52sZMmAAnZYzM7F TXNiVgpU/el594ZHPQcU4fSmdSyBbSG+sWCmtdGHtgoSmXIRIUTks7H+I/CMjiEgt5Vt Lb53Myk7X7CHLO9Be7sTUk4AEDhmhtvVy4iSK+ZfyzkSEc/el7TQqQf2NddYiFPFyE9h SwLw== X-Gm-Message-State: AOJu0Yz8YanWEHHJ/vt8BZ9+b6O/zAggkKPsbkL5/LRicJ+PY/oCxhI0 yCE9qa2PPKx1cLLA5l6QSPFEK8DTZzKsS9Xq336LipTP/lQuw5WQUJ9vqVMVkxM= X-Received: by 2002:a17:906:459:b0:a3c:eb19:8e9d with SMTP id e25-20020a170906045900b00a3ceb198e9dmr654503eja.51.1707875736990; Tue, 13 Feb 2024 17:55:36 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCVLKZWryzT/7IhnzW18GRADrhmQ5GKkFfJ7jevCEytLp3OHWELll+ZJAIxpzWpZeokR9mbyRtpMp0WJ1Tgmqw0R3FUOKw== Received: from dovchinn.amd.com (cable-178-148-16-149.dynamic.sbb.rs. [178.148.16.149]) by smtp.gmail.com with ESMTPSA id vi16-20020a170907d41000b00a3ce31d3ffdsm1663230ejc.93.2024.02.13.17.55.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 17:55:36 -0800 (PST) From: Dmitrii Ovchinnikov To: ffmpeg-devel@ffmpeg.org Date: Wed, 14 Feb 2024 02:55:13 +0100 Message-Id: <20240214015515.1027-7-ovchinnikov.dmitrii@gmail.com> X-Mailer: git-send-email 2.38.1.windows.1 In-Reply-To: <20240214015515.1027-1-ovchinnikov.dmitrii@gmail.com> References: <20240214015515.1027-1-ovchinnikov.dmitrii@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 7/9] libavcodec/amfenc: redesign to use hwcontext_amf. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Evgeny Pavlov , Dmitrii Ovchinnikov Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 38gnvRsKRw6v Co-authored-by: Evgeny Pavlov --- libavcodec/amfenc.c | 560 ++++++++++++-------------------------------- libavcodec/amfenc.h | 23 +- 2 files changed, 158 insertions(+), 425 deletions(-) diff --git a/libavcodec/amfenc.c b/libavcodec/amfenc.c index f1b76bd6aa..e6e5302cca 100644 --- a/libavcodec/amfenc.c +++ b/libavcodec/amfenc.c @@ -29,6 +29,7 @@ #define COBJMACROS #include "libavutil/hwcontext_dxva2.h" #endif +#include "libavutil/hwcontext_amf.h" #include "libavutil/mem.h" #include "libavutil/pixdesc.h" #include "libavutil/time.h" @@ -38,6 +39,18 @@ #include "internal.h" #include "libavutil/mastering_display_metadata.h" +#if CONFIG_D3D11VA +#include +#endif + +#ifdef _WIN32 +#include "compat/w32dlfcn.h" +#else +#include +#endif + +#define PTS_PROP L"PtsProp" + static int amf_save_hdr_metadata(AVCodecContext *avctx, const AVFrame *frame, AMFHDRMetadata *hdrmeta) { AVFrameSideData *sd_display; @@ -88,20 +101,6 @@ static int amf_save_hdr_metadata(AVCodecContext *avctx, const AVFrame *frame, AM return 1; } -#if CONFIG_D3D11VA -#include -#endif - -#ifdef _WIN32 -#include "compat/w32dlfcn.h" -#else -#include -#endif - -#define FFMPEG_AMF_WRITER_ID L"ffmpeg_amf" - -#define PTS_PROP L"PtsProp" - const enum AVPixelFormat ff_amf_pix_fmts[] = { AV_PIX_FMT_NV12, AV_PIX_FMT_YUV420P, @@ -111,289 +110,18 @@ const enum AVPixelFormat ff_amf_pix_fmts[] = { #if CONFIG_DXVA2 AV_PIX_FMT_DXVA2_VLD, #endif + AV_PIX_FMT_AMF, AV_PIX_FMT_P010, AV_PIX_FMT_NONE }; -typedef struct FormatMap { - enum AVPixelFormat av_format; - enum AMF_SURFACE_FORMAT amf_format; -} FormatMap; - -static const FormatMap format_map[] = -{ - { AV_PIX_FMT_NONE, AMF_SURFACE_UNKNOWN }, - { AV_PIX_FMT_NV12, AMF_SURFACE_NV12 }, - { AV_PIX_FMT_P010, AMF_SURFACE_P010 }, - { AV_PIX_FMT_BGR0, AMF_SURFACE_BGRA }, - { AV_PIX_FMT_RGB0, AMF_SURFACE_RGBA }, - { AV_PIX_FMT_GRAY8, AMF_SURFACE_GRAY8 }, - { AV_PIX_FMT_YUV420P, AMF_SURFACE_YUV420P }, - { AV_PIX_FMT_YUYV422, AMF_SURFACE_YUY2 }, -}; - -static enum AMF_SURFACE_FORMAT amf_av_to_amf_format(enum AVPixelFormat fmt) -{ - int i; - for (i = 0; i < amf_countof(format_map); i++) { - if (format_map[i].av_format == fmt) { - return format_map[i].amf_format; - } - } - return AMF_SURFACE_UNKNOWN; -} - -static void AMF_CDECL_CALL AMFTraceWriter_Write(AMFTraceWriter *pThis, - const wchar_t *scope, const wchar_t *message) -{ - AmfTraceWriter *tracer = (AmfTraceWriter*)pThis; - av_log(tracer->avctx, AV_LOG_DEBUG, "%ls: %ls", scope, message); // \n is provided from AMF -} - -static void AMF_CDECL_CALL AMFTraceWriter_Flush(AMFTraceWriter *pThis) -{ -} - -static AMFTraceWriterVtbl tracer_vtbl = -{ - .Write = AMFTraceWriter_Write, - .Flush = AMFTraceWriter_Flush, -}; - -static int amf_load_library(AVCodecContext *avctx) -{ - AmfContext *ctx = avctx->priv_data; - AMFInit_Fn init_fun; - AMFQueryVersion_Fn version_fun; - AMF_RESULT res; - - ctx->delayed_frame = av_frame_alloc(); - if (!ctx->delayed_frame) { - return AVERROR(ENOMEM); - } - // hardcoded to current HW queue size - will auto-realloc if too small - ctx->timestamp_list = av_fifo_alloc2(avctx->max_b_frames + 16, sizeof(int64_t), - AV_FIFO_FLAG_AUTO_GROW); - if (!ctx->timestamp_list) { - return AVERROR(ENOMEM); - } - ctx->dts_delay = 0; - - - ctx->library = dlopen(AMF_DLL_NAMEA, RTLD_NOW | RTLD_LOCAL); - AMF_RETURN_IF_FALSE(ctx, ctx->library != NULL, - AVERROR_UNKNOWN, "DLL %s failed to open\n", AMF_DLL_NAMEA); - - init_fun = (AMFInit_Fn)dlsym(ctx->library, AMF_INIT_FUNCTION_NAME); - AMF_RETURN_IF_FALSE(ctx, init_fun != NULL, AVERROR_UNKNOWN, "DLL %s failed to find function %s\n", AMF_DLL_NAMEA, AMF_INIT_FUNCTION_NAME); - - version_fun = (AMFQueryVersion_Fn)dlsym(ctx->library, AMF_QUERY_VERSION_FUNCTION_NAME); - AMF_RETURN_IF_FALSE(ctx, version_fun != NULL, AVERROR_UNKNOWN, "DLL %s failed to find function %s\n", AMF_DLL_NAMEA, AMF_QUERY_VERSION_FUNCTION_NAME); - - res = version_fun(&ctx->version); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "%s failed with error %d\n", AMF_QUERY_VERSION_FUNCTION_NAME, res); - res = init_fun(AMF_FULL_VERSION, &ctx->factory); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "%s failed with error %d\n", AMF_INIT_FUNCTION_NAME, res); - res = ctx->factory->pVtbl->GetTrace(ctx->factory, &ctx->trace); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "GetTrace() failed with error %d\n", res); - res = ctx->factory->pVtbl->GetDebug(ctx->factory, &ctx->debug); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "GetDebug() failed with error %d\n", res); - return 0; -} - -#if CONFIG_D3D11VA -static int amf_init_from_d3d11_device(AVCodecContext *avctx, AVD3D11VADeviceContext *hwctx) -{ - AmfContext *ctx = avctx->priv_data; - AMF_RESULT res; - - res = ctx->context->pVtbl->InitDX11(ctx->context, hwctx->device, AMF_DX11_1); - if (res != AMF_OK) { - if (res == AMF_NOT_SUPPORTED) - av_log(avctx, AV_LOG_ERROR, "AMF via D3D11 is not supported on the given device.\n"); - else - av_log(avctx, AV_LOG_ERROR, "AMF failed to initialise on the given D3D11 device: %d.\n", res); - return AVERROR(ENODEV); - } - - return 0; -} -#endif - -#if CONFIG_DXVA2 -static int amf_init_from_dxva2_device(AVCodecContext *avctx, AVDXVA2DeviceContext *hwctx) -{ - AmfContext *ctx = avctx->priv_data; - HANDLE device_handle; - IDirect3DDevice9 *device; - HRESULT hr; - AMF_RESULT res; - int ret; - - hr = IDirect3DDeviceManager9_OpenDeviceHandle(hwctx->devmgr, &device_handle); - if (FAILED(hr)) { - av_log(avctx, AV_LOG_ERROR, "Failed to open device handle for Direct3D9 device: %lx.\n", (unsigned long)hr); - return AVERROR_EXTERNAL; - } - - hr = IDirect3DDeviceManager9_LockDevice(hwctx->devmgr, device_handle, &device, FALSE); - if (SUCCEEDED(hr)) { - IDirect3DDeviceManager9_UnlockDevice(hwctx->devmgr, device_handle, FALSE); - ret = 0; - } else { - av_log(avctx, AV_LOG_ERROR, "Failed to lock device handle for Direct3D9 device: %lx.\n", (unsigned long)hr); - ret = AVERROR_EXTERNAL; - } - - IDirect3DDeviceManager9_CloseDeviceHandle(hwctx->devmgr, device_handle); - - if (ret < 0) - return ret; - - res = ctx->context->pVtbl->InitDX9(ctx->context, device); - - IDirect3DDevice9_Release(device); - - if (res != AMF_OK) { - if (res == AMF_NOT_SUPPORTED) - av_log(avctx, AV_LOG_ERROR, "AMF via D3D9 is not supported on the given device.\n"); - else - av_log(avctx, AV_LOG_ERROR, "AMF failed to initialise on given D3D9 device: %d.\n", res); - return AVERROR(ENODEV); - } - - return 0; -} -#endif - -static int amf_init_context(AVCodecContext *avctx) -{ - AmfContext *ctx = avctx->priv_data; - AMFContext1 *context1 = NULL; - AMF_RESULT res; - av_unused int ret; - - ctx->hwsurfaces_in_queue = 0; - ctx->hwsurfaces_in_queue_max = 16; - - // configure AMF logger - // the return of these functions indicates old state and do not affect behaviour - ctx->trace->pVtbl->EnableWriter(ctx->trace, AMF_TRACE_WRITER_DEBUG_OUTPUT, ctx->log_to_dbg != 0 ); - if (ctx->log_to_dbg) - ctx->trace->pVtbl->SetWriterLevel(ctx->trace, AMF_TRACE_WRITER_DEBUG_OUTPUT, AMF_TRACE_TRACE); - ctx->trace->pVtbl->EnableWriter(ctx->trace, AMF_TRACE_WRITER_CONSOLE, 0); - ctx->trace->pVtbl->SetGlobalLevel(ctx->trace, AMF_TRACE_TRACE); - - // connect AMF logger to av_log - ctx->tracer.vtbl = &tracer_vtbl; - ctx->tracer.avctx = avctx; - ctx->trace->pVtbl->RegisterWriter(ctx->trace, FFMPEG_AMF_WRITER_ID,(AMFTraceWriter*)&ctx->tracer, 1); - ctx->trace->pVtbl->SetWriterLevel(ctx->trace, FFMPEG_AMF_WRITER_ID, AMF_TRACE_TRACE); - - res = ctx->factory->pVtbl->CreateContext(ctx->factory, &ctx->context); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "CreateContext() failed with error %d\n", res); - - // If a device was passed to the encoder, try to initialise from that. - if (avctx->hw_frames_ctx) { - AVHWFramesContext *frames_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; - - if (amf_av_to_amf_format(frames_ctx->sw_format) == AMF_SURFACE_UNKNOWN) { - av_log(avctx, AV_LOG_ERROR, "Format of input frames context (%s) is not supported by AMF.\n", - av_get_pix_fmt_name(frames_ctx->sw_format)); - return AVERROR(EINVAL); - } - - switch (frames_ctx->device_ctx->type) { -#if CONFIG_D3D11VA - case AV_HWDEVICE_TYPE_D3D11VA: - ret = amf_init_from_d3d11_device(avctx, frames_ctx->device_ctx->hwctx); - if (ret < 0) - return ret; - break; -#endif -#if CONFIG_DXVA2 - case AV_HWDEVICE_TYPE_DXVA2: - ret = amf_init_from_dxva2_device(avctx, frames_ctx->device_ctx->hwctx); - if (ret < 0) - return ret; - break; -#endif - default: - av_log(avctx, AV_LOG_ERROR, "AMF initialisation from a %s frames context is not supported.\n", - av_hwdevice_get_type_name(frames_ctx->device_ctx->type)); - return AVERROR(ENOSYS); - } - - ctx->hw_frames_ctx = av_buffer_ref(avctx->hw_frames_ctx); - if (!ctx->hw_frames_ctx) - return AVERROR(ENOMEM); - - if (frames_ctx->initial_pool_size > 0) - ctx->hwsurfaces_in_queue_max = frames_ctx->initial_pool_size - 1; - - } else if (avctx->hw_device_ctx) { - AVHWDeviceContext *device_ctx = (AVHWDeviceContext*)avctx->hw_device_ctx->data; - - switch (device_ctx->type) { -#if CONFIG_D3D11VA - case AV_HWDEVICE_TYPE_D3D11VA: - ret = amf_init_from_d3d11_device(avctx, device_ctx->hwctx); - if (ret < 0) - return ret; - break; -#endif -#if CONFIG_DXVA2 - case AV_HWDEVICE_TYPE_DXVA2: - ret = amf_init_from_dxva2_device(avctx, device_ctx->hwctx); - if (ret < 0) - return ret; - break; -#endif - default: - av_log(avctx, AV_LOG_ERROR, "AMF initialisation from a %s device is not supported.\n", - av_hwdevice_get_type_name(device_ctx->type)); - return AVERROR(ENOSYS); - } - - ctx->hw_device_ctx = av_buffer_ref(avctx->hw_device_ctx); - if (!ctx->hw_device_ctx) - return AVERROR(ENOMEM); - - } else { - res = ctx->context->pVtbl->InitDX11(ctx->context, NULL, AMF_DX11_1); - if (res == AMF_OK) { - av_log(avctx, AV_LOG_VERBOSE, "AMF initialisation succeeded via D3D11.\n"); - } else { - res = ctx->context->pVtbl->InitDX9(ctx->context, NULL); - if (res == AMF_OK) { - av_log(avctx, AV_LOG_VERBOSE, "AMF initialisation succeeded via D3D9.\n"); - } else { - AMFGuid guid = IID_AMFContext1(); - res = ctx->context->pVtbl->QueryInterface(ctx->context, &guid, (void**)&context1); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "CreateContext1() failed with error %d\n", res); - - res = context1->pVtbl->InitVulkan(context1, NULL); - context1->pVtbl->Release(context1); - if (res != AMF_OK) { - if (res == AMF_NOT_SUPPORTED) - av_log(avctx, AV_LOG_ERROR, "AMF via Vulkan is not supported on the given device.\n"); - else - av_log(avctx, AV_LOG_ERROR, "AMF failed to initialise on the given Vulkan device: %d.\n", res); - return AVERROR(ENOSYS); - } - av_log(avctx, AV_LOG_VERBOSE, "AMF initialisation succeeded via Vulkan.\n"); - } - } - } - return 0; -} - static int amf_init_encoder(AVCodecContext *avctx) { AmfContext *ctx = avctx->priv_data; const wchar_t *codec_id = NULL; AMF_RESULT res; enum AVPixelFormat pix_fmt; + AVAMFDeviceContextInternal* internal = (AVAMFDeviceContextInternal *)ctx->amf_device_ctx_internal->data; switch (avctx->codec->id) { case AV_CODEC_ID_H264: @@ -415,13 +143,16 @@ static int amf_init_encoder(AVCodecContext *avctx) else pix_fmt = avctx->pix_fmt; - ctx->format = amf_av_to_amf_format(pix_fmt); + if (avctx->pix_fmt != AV_PIX_FMT_AMF) + ctx->format = av_amf_av_to_amf_format(pix_fmt); + else + ctx->format = av_amf_av_to_amf_format(avctx->sw_pix_fmt); + AMF_RETURN_IF_FALSE(ctx, ctx->format != AMF_SURFACE_UNKNOWN, AVERROR(EINVAL), - "Format %s is not supported\n", av_get_pix_fmt_name(pix_fmt)); + "Format %s is not supported\n", av_get_pix_fmt_name(pix_fmt)); - res = ctx->factory->pVtbl->CreateComponent(ctx->factory, ctx->context, codec_id, &ctx->encoder); + res = internal->factory->pVtbl->CreateComponent(internal->factory, internal->context, codec_id, &ctx->encoder); AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_ENCODER_NOT_FOUND, "CreateComponent(%ls) failed with error %d\n", codec_id, res); - return 0; } @@ -429,49 +160,61 @@ int av_cold ff_amf_encode_close(AVCodecContext *avctx) { AmfContext *ctx = avctx->priv_data; - if (ctx->delayed_surface) { - ctx->delayed_surface->pVtbl->Release(ctx->delayed_surface); - ctx->delayed_surface = NULL; - } - if (ctx->encoder) { ctx->encoder->pVtbl->Terminate(ctx->encoder); ctx->encoder->pVtbl->Release(ctx->encoder); ctx->encoder = NULL; } - if (ctx->context) { - ctx->context->pVtbl->Terminate(ctx->context); - ctx->context->pVtbl->Release(ctx->context); - ctx->context = NULL; - } av_buffer_unref(&ctx->hw_device_ctx); av_buffer_unref(&ctx->hw_frames_ctx); + av_buffer_unref(&ctx->amf_device_ctx_internal); + av_fifo_freep2(&ctx->timestamp_list); + + return 0; +} - if (ctx->trace) { - ctx->trace->pVtbl->UnregisterWriter(ctx->trace, FFMPEG_AMF_WRITER_ID); +static int amf_init_encoder_context(AVCodecContext *avctx) +{ + AmfContext *ctx = avctx->priv_data; + AMFContext1 *context1 = NULL; + int ret; + + if (avctx->hw_frames_ctx) { + AVHWFramesContext *frames_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; + ret = av_amf_context_derive((AVAMFDeviceContextInternal *)ctx->amf_device_ctx_internal->data, frames_ctx->device_ctx, NULL, 0); + if (ret < 0) + return ret; + ctx->hw_frames_ctx = av_buffer_ref(avctx->hw_frames_ctx); + if (!ctx->hw_frames_ctx) + return AVERROR(ENOMEM); } - if (ctx->library) { - dlclose(ctx->library); - ctx->library = NULL; + else if (avctx->hw_device_ctx) { + AVHWDeviceContext *device_ctx = (AVHWDeviceContext*)avctx->hw_device_ctx->data; + ret = av_amf_context_derive((AVAMFDeviceContextInternal *)ctx->amf_device_ctx_internal->data, device_ctx, NULL, 0); + if (ret < 0) + return ret; + ctx->hw_device_ctx = av_buffer_ref(avctx->hw_device_ctx); + if (!ctx->hw_device_ctx) + return AVERROR(ENOMEM); + + } else { + ret = av_amf_context_init((AVAMFDeviceContextInternal *)ctx->amf_device_ctx_internal->data, avctx); + if (ret != 0) { + return ret; + } } - ctx->trace = NULL; - ctx->debug = NULL; - ctx->factory = NULL; - ctx->version = 0; - ctx->delayed_drain = 0; - av_frame_free(&ctx->delayed_frame); - av_fifo_freep2(&ctx->timestamp_list); - return 0; + + return ret; } static int amf_copy_surface(AVCodecContext *avctx, const AVFrame *frame, AMFSurface* surface) { AMFPlane *plane; - uint8_t *dst_data[4]; - int dst_linesize[4]; + uint8_t *dst_data[4] = {0}; + int dst_linesize[4] = {0}; int planes; int i; @@ -555,13 +298,52 @@ int ff_amf_encode_init(AVCodecContext *avctx) { int ret; - if ((ret = amf_load_library(avctx)) == 0) { - if ((ret = amf_init_context(avctx)) == 0) { - if ((ret = amf_init_encoder(avctx)) == 0) { - return 0; - } + AmfContext *ctx = avctx->priv_data; + AVHWDeviceContext *hwdev_ctx = NULL; + if (avctx->hw_device_ctx) { + hwdev_ctx = (AVHWDeviceContext*)avctx->hw_device_ctx->data; + } else if (avctx->hw_frames_ctx) { + AVHWFramesContext *frames_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; + hwdev_ctx = (AVHWDeviceContext*)frames_ctx->device_ctx; + } + if (av_amf_trace_writer.avctx == NULL) + av_amf_trace_writer.avctx = avctx; + // hardcoded to current HW queue size - will auto-realloc if too small + ctx->timestamp_list = av_fifo_alloc2(avctx->max_b_frames + 16, sizeof(int64_t), + AV_FIFO_FLAG_AUTO_GROW); + if (!ctx->timestamp_list) { + return AVERROR(ENOMEM); + } + ctx->dts_delay = 0; + + ctx->hwsurfaces_in_queue = 0; + ctx->hwsurfaces_in_queue_max = 16; + + if (avctx->hw_frames_ctx && hwdev_ctx && hwdev_ctx->type == AV_HWDEVICE_TYPE_AMF) { + AVHWFramesContext *frames_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; + AVAMFDeviceContext * amf_ctx = hwdev_ctx->hwctx; + ctx->amf_device_ctx_internal = av_buffer_ref(amf_ctx->internal); + } + else if (avctx->hw_device_ctx && hwdev_ctx && hwdev_ctx->type == AV_HWDEVICE_TYPE_AMF) { + AVAMFDeviceContext * amf_ctx = hwdev_ctx->hwctx; + ctx->amf_device_ctx_internal = av_buffer_ref(amf_ctx->internal); + } else { + AVAMFDeviceContextInternal *wrapped = av_mallocz(sizeof(*wrapped)); + ctx->amf_device_ctx_internal = av_buffer_create((uint8_t *)wrapped, sizeof(*wrapped), + av_amf_context_internal_free, NULL, 0); + if ((ret = av_amf_context_internal_create((AVAMFDeviceContextInternal *)ctx->amf_device_ctx_internal->data, avctx, "", NULL, 0)) != 0) { + ff_amf_encode_close(avctx); + return ret; } + if ((ret = amf_init_encoder_context(avctx)) != 0) { + ff_amf_encode_close(avctx); + return ret; + } + } + if ((ret = amf_init_encoder(avctx)) == 0) { + return 0; } + ff_amf_encode_close(avctx); return ret; } @@ -639,30 +421,28 @@ static void amf_release_buffer_with_frame_ref(AMFBuffer *frame_ref_storage_buffe int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) { AmfContext *ctx = avctx->priv_data; + AVAMFDeviceContextInternal * internal = (AVAMFDeviceContextInternal *)ctx->amf_device_ctx_internal->data; AMFSurface *surface; AMF_RESULT res; int ret; AMF_RESULT res_query; AMFData *data = NULL; - AVFrame *frame = ctx->delayed_frame; + AVFrame *frame = av_frame_alloc(); int block_and_wait; int query_output_data_flag = 0; AMF_RESULT res_resubmit; + int count = 0; if (!ctx->encoder) return AVERROR(EINVAL); - if (!frame->buf[0]) { - ret = ff_encode_get_frame(avctx, frame); - if (ret < 0 && ret != AVERROR_EOF) - return ret; - } + ret = ff_encode_get_frame(avctx, frame); + if (ret < 0 && ret != AVERROR_EOF) + return ret; if (!frame->buf[0]) { // submit drain if (!ctx->eof) { // submit drain one time only - if (ctx->delayed_surface != NULL) { - ctx->delayed_drain = 1; // input queue is full: resubmit Drain() in ff_amf_receive_packet - } else if(!ctx->delayed_drain) { + if(!ctx->delayed_drain) { res = ctx->encoder->pVtbl->Drain(ctx->encoder); if (res == AMF_INPUT_FULL) { ctx->delayed_drain = 1; // input queue is full: resubmit Drain() in ff_amf_receive_packet @@ -674,7 +454,7 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) } } } - } else if (!ctx->delayed_surface) { // submit frame + } else { // submit frame int hw_surface = 0; // prepare surface from frame @@ -691,7 +471,7 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) texture->lpVtbl->SetPrivateData(texture, &AMFTextureArrayIndexGUID, sizeof(index), &index); - res = ctx->context->pVtbl->CreateSurfaceFromDX11Native(ctx->context, texture, &surface, NULL); // wrap to AMF surface + res = internal->context->pVtbl->CreateSurfaceFromDX11Native(internal->context, texture, &surface, NULL); // wrap to AMF surface AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR(ENOMEM), "CreateSurfaceFromDX11Native() failed with error %d\n", res); hw_surface = 1; @@ -703,16 +483,23 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) { IDirect3DSurface9 *texture = (IDirect3DSurface9 *)frame->data[3]; // actual texture - res = ctx->context->pVtbl->CreateSurfaceFromDX9Native(ctx->context, texture, &surface, NULL); // wrap to AMF surface + res = internal->context->pVtbl->CreateSurfaceFromDX9Native(internal->context, texture, &surface, NULL); // wrap to AMF surface AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR(ENOMEM), "CreateSurfaceFromDX9Native() failed with error %d\n", res); hw_surface = 1; } break; #endif + case AV_PIX_FMT_AMF: + { + surface = (AMFSurface*)frame->data[3]; + surface->pVtbl->Acquire(surface); + hw_surface = 1; + } + break; default: { - res = ctx->context->pVtbl->AllocSurface(ctx->context, AMF_MEMORY_HOST, ctx->format, avctx->width, avctx->height, &surface); + res = internal->context->pVtbl->AllocSurface(internal->context, AMF_MEMORY_HOST, ctx->format, avctx->width, avctx->height, &surface); AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR(ENOMEM), "AllocSurface() failed with error %d\n", res); amf_copy_surface(avctx, frame, surface); } @@ -725,7 +512,7 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) // input HW surfaces can be vertically aligned by 16; tell AMF the real size surface->pVtbl->SetCrop(surface, 0, 0, frame->width, frame->height); - frame_ref_storage_buffer = amf_create_buffer_with_frame_ref(frame, ctx->context); + frame_ref_storage_buffer = amf_create_buffer_with_frame_ref(frame, internal->context); AMF_RETURN_IF_FALSE(ctx, frame_ref_storage_buffer != NULL, AVERROR(ENOMEM), "create_buffer_with_frame_ref() returned NULL\n"); res = amf_set_property_buffer(surface, L"av_frame_ref", frame_ref_storage_buffer); @@ -737,7 +524,7 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) // HDR10 metadata if (frame->color_trc == AVCOL_TRC_SMPTE2084) { AMFBuffer * hdrmeta_buffer = NULL; - res = ctx->context->pVtbl->AllocBuffer(ctx->context, AMF_MEMORY_HOST, sizeof(AMFHDRMetadata), &hdrmeta_buffer); + res = internal->context->pVtbl->AllocBuffer(internal->context, AMF_MEMORY_HOST, sizeof(AMFHDRMetadata), &hdrmeta_buffer); if (res == AMF_OK) { AMFHDRMetadata * hdrmeta = (AMFHDRMetadata*)hdrmeta_buffer->pVtbl->GetNative(hdrmeta_buffer); if (amf_save_hdr_metadata(avctx, frame, hdrmeta) == 0) { @@ -772,13 +559,11 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) // submit surface res = ctx->encoder->pVtbl->SubmitInput(ctx->encoder, (AMFData*)surface); if (res == AMF_INPUT_FULL) { // handle full queue - //store surface for later submission - ctx->delayed_surface = surface; + av_usleep(1000); // wait and poll again } else { int64_t pts = frame->pts; surface->pVtbl->Release(surface); AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "SubmitInput() failed with error %d\n", res); - av_frame_unref(frame); ret = av_fifo_write(ctx->timestamp_list, &pts, 1); if (ret < 0) @@ -790,75 +575,40 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) do { block_and_wait = 0; // poll data - if (!avpkt->data && !avpkt->buf) { - res_query = ctx->encoder->pVtbl->QueryOutput(ctx->encoder, &data); - if (data) { - // copy data to packet - AMFBuffer *buffer; - AMFGuid guid = IID_AMFBuffer(); - query_output_data_flag = 1; - data->pVtbl->QueryInterface(data, &guid, (void**)&buffer); // query for buffer interface - ret = amf_copy_buffer(avctx, avpkt, buffer); - - buffer->pVtbl->Release(buffer); - - if (data->pVtbl->HasProperty(data, L"av_frame_ref")) { - AMFBuffer* frame_ref_storage_buffer; - res = amf_get_property_buffer(data, L"av_frame_ref", &frame_ref_storage_buffer); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "GetProperty failed for \"av_frame_ref\" with error %d\n", res); - amf_release_buffer_with_frame_ref(frame_ref_storage_buffer); - ctx->hwsurfaces_in_queue--; - } + res_query = ctx->encoder->pVtbl->QueryOutput(ctx->encoder, &data); + if (data) { + // copy data to packet + AMFBuffer* buffer; + AMFGuid guid = IID_AMFBuffer(); + data->pVtbl->QueryInterface(data, &guid, (void**)&buffer); // query for buffer interface + ret = amf_copy_buffer(avctx, avpkt, buffer); + + buffer->pVtbl->Release(buffer); + + if (data->pVtbl->HasProperty(data, L"av_frame_ref")) { + AMFBuffer *frame_ref_storage_buffer; + res = amf_get_property_buffer(data, L"av_frame_ref", &frame_ref_storage_buffer); + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "GetProperty failed for \"av_frame_ref\" with error %d\n", res); + amf_release_buffer_with_frame_ref(frame_ref_storage_buffer); + ctx->hwsurfaces_in_queue--; + } + data->pVtbl->Release(data); - data->pVtbl->Release(data); + AMF_RETURN_IF_FALSE(ctx, ret >= 0, ret, "amf_copy_buffer() failed with error %d\n", ret); - AMF_RETURN_IF_FALSE(ctx, ret >= 0, ret, "amf_copy_buffer() failed with error %d\n", ret); - } - } - res_resubmit = AMF_OK; - if (ctx->delayed_surface != NULL) { // try to resubmit frame - if (ctx->delayed_surface->pVtbl->HasProperty(ctx->delayed_surface, L"av_frame_hdrmeta")) { - AMFBuffer * hdrmeta_buffer = NULL; - res = amf_get_property_buffer((AMFData *)ctx->delayed_surface, L"av_frame_hdrmeta", &hdrmeta_buffer); - AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "GetProperty failed for \"av_frame_hdrmeta\" with error %d\n", res); - switch (avctx->codec->id) { - case AV_CODEC_ID_H264: - AMF_ASSIGN_PROPERTY_INTERFACE(res, ctx->encoder, AMF_VIDEO_ENCODER_INPUT_HDR_METADATA, hdrmeta_buffer); break; - case AV_CODEC_ID_HEVC: - AMF_ASSIGN_PROPERTY_INTERFACE(res, ctx->encoder, AMF_VIDEO_ENCODER_HEVC_INPUT_HDR_METADATA, hdrmeta_buffer); break; - case AV_CODEC_ID_AV1: - AMF_ASSIGN_PROPERTY_INTERFACE(res, ctx->encoder, AMF_VIDEO_ENCODER_AV1_INPUT_HDR_METADATA, hdrmeta_buffer); break; + if (ctx->delayed_drain) { // try to resubmit drain + res = ctx->encoder->pVtbl->Drain(ctx->encoder); + if (res != AMF_INPUT_FULL) { + ctx->delayed_drain = 0; + ctx->eof = 1; // drain started + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "Repeated Drain() failed with error %d\n", res); + } else { + av_log(avctx, AV_LOG_WARNING, "Data acquired but delayed drain submission got AMF_INPUT_FULL- should not happen\n"); } - hdrmeta_buffer->pVtbl->Release(hdrmeta_buffer); - } - res_resubmit = ctx->encoder->pVtbl->SubmitInput(ctx->encoder, (AMFData*)ctx->delayed_surface); - if (res_resubmit != AMF_INPUT_FULL) { - int64_t pts = ctx->delayed_surface->pVtbl->GetPts(ctx->delayed_surface); - ctx->delayed_surface->pVtbl->Release(ctx->delayed_surface); - ctx->delayed_surface = NULL; - av_frame_unref(ctx->delayed_frame); - AMF_RETURN_IF_FALSE(ctx, res_resubmit == AMF_OK, AVERROR_UNKNOWN, "Repeated SubmitInput() failed with error %d\n", res_resubmit); - - ret = av_fifo_write(ctx->timestamp_list, &pts, 1); - if (ret < 0) - return ret; - } - } else if (ctx->delayed_drain) { // try to resubmit drain - res = ctx->encoder->pVtbl->Drain(ctx->encoder); - if (res != AMF_INPUT_FULL) { - ctx->delayed_drain = 0; - ctx->eof = 1; // drain started - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "Repeated Drain() failed with error %d\n", res); - } else { - av_log(avctx, AV_LOG_WARNING, "Data acquired but delayed drain submission got AMF_INPUT_FULL- should not happen\n"); - } - } - - if (query_output_data_flag == 0) { - if (res_resubmit == AMF_INPUT_FULL || ctx->delayed_drain || (ctx->eof && res_query != AMF_EOF) || (ctx->hwsurfaces_in_queue >= ctx->hwsurfaces_in_queue_max)) { - block_and_wait = 1; - av_usleep(1000); } + } else if (ctx->delayed_drain || (ctx->eof && res_query != AMF_EOF) || (ctx->hwsurfaces_in_queue >= ctx->hwsurfaces_in_queue_max)) { + block_and_wait = 1; + av_usleep(1000); // wait and poll again } } while (block_and_wait); @@ -916,5 +666,7 @@ const AVCodecHWConfigInternal *const ff_amfenc_hw_configs[] = { HW_CONFIG_ENCODER_FRAMES(DXVA2_VLD, DXVA2), HW_CONFIG_ENCODER_DEVICE(NONE, DXVA2), #endif + HW_CONFIG_ENCODER_FRAMES(AMF, AMF), + HW_CONFIG_ENCODER_DEVICE(NONE, AMF), NULL, }; diff --git a/libavcodec/amfenc.h b/libavcodec/amfenc.h index 1bda0136bd..1f680beb83 100644 --- a/libavcodec/amfenc.h +++ b/libavcodec/amfenc.h @@ -25,7 +25,7 @@ #include #include #include - +#include "libavutil/hwcontext_amf.h" #include "libavutil/fifo.h" #include "avcodec.h" @@ -33,16 +33,6 @@ #define MAX_LOOKAHEAD_DEPTH 41 -/** -* AMF trace writer callback class -* Used to capture all AMF logging -*/ - -typedef struct AmfTraceWriter { - AMFTraceWriterVtbl *vtbl; - AVCodecContext *avctx; -} AmfTraceWriter; - /** * AMF encoder context */ @@ -50,14 +40,7 @@ typedef struct AmfTraceWriter { typedef struct AmfContext { AVClass *avclass; // access to AMF runtime - amf_handle library; ///< handle to DLL library - AMFFactory *factory; ///< pointer to AMF factory - AMFDebug *debug; ///< pointer to AMF debug interface - AMFTrace *trace; ///< pointer to AMF trace interface - - amf_uint64 version; ///< version of AMF runtime - AmfTraceWriter tracer; ///< AMF writer registered with AMF - AMFContext *context; ///< AMF context + AVBufferRef *amf_device_ctx_internal; //encoder AMFComponent *encoder; ///< AMF encoder object amf_bool eof; ///< flag indicating EOF happened @@ -71,8 +54,6 @@ typedef struct AmfContext { // helpers to handle async calls int delayed_drain; - AMFSurface *delayed_surface; - AVFrame *delayed_frame; // shift dts back by max_b_frames in timing AVFifo *timestamp_list; From patchwork Wed Feb 14 01:55:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitrii Ovchinnikov X-Patchwork-Id: 46257 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:c493:b0:19e:cdac:8cce with SMTP id eo19csp110457pzb; Tue, 13 Feb 2024 17:56:47 -0800 (PST) X-Forwarded-Encrypted: i=2; AJvYcCUPtomInCwl23LaTuewlPzdmRK/mhyAf1VndMZFL7Fdcj2bx1AM8aajdAjpq99VRhWdzkJxDGASimYKlEJuvWVjgSdayURrKLxfhg== X-Google-Smtp-Source: AGHT+IELGK3ySydzAHWuRpWmBQLRLencKEuhLdYA2HtLeEM+sr28TfXI0rTV2JNzAiJwlf88WsH1 X-Received: by 2002:ac2:484d:0:b0:511:8e33:639 with SMTP id 13-20020ac2484d000000b005118e330639mr941787lfy.8.1707875807281; Tue, 13 Feb 2024 17:56:47 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1707875807; cv=none; d=google.com; s=arc-20160816; b=BIcmqonMmswGunccI86wbZm0zaBU/TwSBAxRCd9lt/m0gTWaSLMNcDYlu3Rdolx845 TB9B8cSGFeolaCWCygWg9gAEy9jxZ6qlJsxw7HLgJZEBEAkkg/fJktLa3nZekJBGoNJG XqMkestc6L+dsRYNIDXuSzFWWh1yeA3VVHPm+vr3azUYgI29LUhR38UmN0C45J0JkjRd ZYECesOF8rXa1ig1prDnfr7R9TIFONe2YSMRC7Hd27sVUsDkbEXedBgeK25/QJpD0Up2 HnpYOvHbSDXcAcR5Eef5GdFxigZuyjbVpmStm+smtqWWG83M1HslbVgvC4Om214DAJ3s I+0A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=xZvbkbwBLwQ4KZyKRMUQOivCHbqFe6zZzUdNg+1E8Aw=; fh=OfVi/NCo4jLKMfiRbxC0nolFwFFXgvfkucaawu3GlOQ=; b=Kd10Qmuyz3Nm/Ne7wwXai7CmbpZeWHK7b+BIxc4TOJYR1Mg1o0oiRp0h9cSxPL+TDE LSw8nxJP8HfW2ZYGlletnw0qcNDUMMGfAxRkowG790O/NilXnxpCogE2NaNhOc3ySTjW Tx9+odaCx5a+VGf15CIcPuuB0UyBGNH8o6QXiwGsHFKuEz8jk95v9+EVu4yMsDTyDCK3 WvlWoUmlP4oBFakddwRJPooorIcSgcap2NMSGKOCma0AwTqTkPybAyJv4s431R8zUFBv ShctuBg3B0CL14jtgIS5jQ5SIQMe5gReUGme19CDM4lNv0MapUWfQ/U2KKmmSLVptKFe Qkqw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=iu4ZX0wi; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com X-Forwarded-Encrypted: i=1; AJvYcCUsY8yml09mgKFG1uZ/IctJNCPgxDu/PLTysPR6X/dw6jBaFCPPICHJM4mXIQ+4ANXZ7tf1w89pAjhilM/JEK90SQPtl3ED+mf8MD1nhIP97cqxwWzP6NSftle5jhXgr99lPvAUfvMSpKXScfnVzxsDAyk8t8Y7sokED/32gKaJgalwEUX//0Xp7Mlgkvz1a5Ov6iWvxn6jvKhDEDbcwdioYWEGM8qggCYVrwWs1++eiSqaAbQT7YOHDX/8FaHRBrn2LULfkwMWykuRrpMdATF2TsY2hCQtRodaktH1itf11zj0LLIGmvWI9RVLBl1Y+e/OGH03wxRkvDIUFJkTx6Oq9AcXTJU7jfhlgBU+5mcjEMf44+ibiKNPuLr2qxSTtsit2ckxq9HascoMEXmzh33miaDaeAnoJBguXodlr5+dgM7CkieUFcfQB5THcb/xwvPsfO1PWRLe110RLH28iV55LkkR3RlnGtoauQUJZzBTsyzfrLqoV7th7jFxhX7xRZ69OsBP3IZG0yVcKs+aX7leX4TjO3pjXvs9/6Dc8t9F2NIlvyTuD9M9ZGwWGmrNSIF9rpyYotZrUyPiAZao5ENl4LpG03fN1+57KNi3G8PX97YMSpOEepzI0VvddlqZXIBttwNWhG93E+N6rE/KraHgA5UL29ZcicVlZ+tsxtiC3ha+8ka1zeZYsHI2jdwVcv2KUt0cUfgoBJzTknFXMFPlWPFPyJ/Z/hhYrxO0p/81G8Ecj1io+BQc3lB6fWM9V77gf5nU1rFTDcCkmCnDok89ThKAo+3khJ5eRKanw89gJgCfFZHcOT7OtQeMj6bmJsbBoQo0oXv2hD+N3wb/3D+UGb2GyHyBn1Vb6WK4aHwqDKU8uyOhIJaZIgVlGgG4augSzdhK2tNVvShTwgH+E5x7t50XQl63JOHloh2bBBdgqpUOYGr9ro9rPi0oXLPfK8IoG6 5A+/GE8buDsQcGrRM07ZuL7AytcWd1t/HOU53QcpwOdZWSCT7YUoy3epngzteDc/2SsJcAJO7id/OYT6CUTNoYcb81jbs/M2Twmt2lrTAcWvj3BKeN66+HaGZnWEREIr7KZCn6WZg5xs/cQda0WEZZP3D71LywSpoqgcmIfBEx5rbLmzkayFVcc53+yaneeH5VpVYJVlue43OJm/EWW8XaPhx/vp0uNBwDGviwdVOf8yi7YJ9V8Y9BSHGwHYw0xO43VhNV55pOLBHc+oALQY+jmaZq+Nkv47DU1uFnIyoCdxCF4ku17rek3uD6jRM7bQRerig6dUgoMkSZtxwIyRCeCL4ySAm1KX78mHaipDDtmI1Y22ZcxgDReyScGxZFYG2tKZTcbbeBI6cl+tqr0KuZEbF/OVGkofvNa69wAvSXJ0s/3zo+140keMnIrXOgIUPQYXvJCiZRt9Q/TexsfXn44e4ROqmNO9TrwIbkigv2owdU/Gox/YkkL1LQETH0t5rkQsdqQ13jDV2MuPrT9mVcUh6XUTMAktorwiEJEBEQ9MPoWlTVvdyaQliie0YO6FTHMPD3ndz7IrRYTai5sKDNoU/YZAc8OcWdDpErIV4fKF+COXIIM+3uv7HKhI9ACjUg1S5z Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id y12-20020a17090668cc00b00a3d47b7b91csi104560ejr.511.2024.02.13.17.56.46; Tue, 13 Feb 2024 17:56:47 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=iu4ZX0wi; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D6FD868D1EC; Wed, 14 Feb 2024 03:55:48 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f49.google.com (mail-lf1-f49.google.com [209.85.167.49]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D061068D1CB for ; Wed, 14 Feb 2024 03:55:39 +0200 (EET) Received: by mail-lf1-f49.google.com with SMTP id 2adb3069b0e04-511972043c3so1933103e87.1 for ; Tue, 13 Feb 2024 17:55:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1707875738; x=1708480538; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=KBRdw+LwhVL/bT7XYd7Yhjl5w/jM5TqC9x5HFPTZB94=; b=iu4ZX0wigxd39hZFvdrnmU2L7YO5A4X4p4dlUibfazrXgKxS3QQzy3Vj4tF/tm6kYO 0AhJfATZTuN1PjKbp4UsVAI588/d8M5Hw00dHQvUY8IGVjgVEz2TpJOWZEuGnrem7rB7 tLoSZAtxb6wPMcDYNmlWmONEOeTYNzNXViUqMxgYWJzuLCS7UHGeUvIOYT8vlP4lJJEX Pe6NhfYTgoJVxxOuysRZcgB7Xxgo2EAt2JNHOs+u9QniT0s61wkKo9+sWVrynLYJ8ERA BRJxLIjn0LR5ezCsKLUDfJOZsW5wEjW85i7Tc5MLFCXDQCwuaGU2E0yAUJkolXNy+iJL ZArw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707875738; x=1708480538; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KBRdw+LwhVL/bT7XYd7Yhjl5w/jM5TqC9x5HFPTZB94=; b=vg7tqdp+m3x3mSu6DS9hTUHyrRJ+o6IL0kvTQnViK5I94TXuxM8ccbUtEfHZ9DEHSx qLidkWMiGFHkEEOC67lWEwBhiIgbXZLZ8LNRvTxVS7bUwtwjbn/0CzZnotYM8MrGPjym Ri/qDwVr88KyowqIOnMG0ywN5EVe2KgfSbwdTLHiipIfVxChXbBo0LmSSqhDefrXcu/x oynxNGE0WqScgqCvJUI2QLoNLfJcO2iVo0Ssdx8YlnToUNvNj5iW2J6CR1RkBwDCDffl oKtGk9L9wy7BkghnDebwQagd+DK7Z4KOymbwO0x/mOUjc5G+mVwQmoGSV/YJcd5NIBM5 F/Zw== X-Gm-Message-State: AOJu0YyLFSKmGS+Xn3wTWLsGWFBs0wGjz7pw788KoGkwOSs8EfHhlWJR zsMzZrA6P9j8mWClusEnuDMVMhaH9TDjEtJcKUKXAKfMEbU79lAc0GHSuG8NZX0= X-Received: by 2002:ac2:44b9:0:b0:511:87c0:a327 with SMTP id c25-20020ac244b9000000b0051187c0a327mr843519lfm.68.1707875738218; Tue, 13 Feb 2024 17:55:38 -0800 (PST) Received: from dovchinn.amd.com (cable-178-148-16-149.dynamic.sbb.rs. [178.148.16.149]) by smtp.gmail.com with ESMTPSA id vi16-20020a170907d41000b00a3ce31d3ffdsm1663230ejc.93.2024.02.13.17.55.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 17:55:37 -0800 (PST) From: Dmitrii Ovchinnikov To: ffmpeg-devel@ffmpeg.org Date: Wed, 14 Feb 2024 02:55:14 +0100 Message-Id: <20240214015515.1027-8-ovchinnikov.dmitrii@gmail.com> X-Mailer: git-send-email 2.38.1.windows.1 In-Reply-To: <20240214015515.1027-1-ovchinnikov.dmitrii@gmail.com> References: <20240214015515.1027-1-ovchinnikov.dmitrii@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 8/9] avfilter/scale_amf: Add AMF HW scaler & color converter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Evgeny Pavlov Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 13DyxOj95+5H From: Evgeny Pavlov This commit adds two AMF filters: scale_amf & scale_amf_hq. Both filters are using AMF hardware acceleration. scale_amf supports simple scaling algorithms & color conversion. scale_amf_hq supports advanced scaling algorithms & might be used for upscaling only. Signed-off-by: Evgeny Pavlov --- configure | 1 + libavfilter/Makefile | 2 + libavfilter/allfilters.c | 2 + libavfilter/vf_scale_amf.c | 266 +++++++++++++++ libavfilter/vf_scale_amf_common.c | 515 ++++++++++++++++++++++++++++++ libavfilter/vf_scale_amf_common.h | 71 ++++ libavfilter/vf_scale_amf_hq.c | 191 +++++++++++ 7 files changed, 1048 insertions(+) create mode 100644 libavfilter/vf_scale_amf.c create mode 100644 libavfilter/vf_scale_amf_common.c create mode 100644 libavfilter/vf_scale_amf_common.h create mode 100644 libavfilter/vf_scale_amf_hq.c diff --git a/configure b/configure index f72533b7d2..3d1c44d7ae 100755 --- a/configure +++ b/configure @@ -3826,6 +3826,7 @@ rubberband_filter_deps="librubberband" sab_filter_deps="gpl swscale" scale2ref_filter_deps="swscale" scale_filter_deps="swscale" +scale_amf_filter_deps="amf" scale_qsv_filter_deps="libmfx" scale_qsv_filter_select="qsvvpp" scdet_filter_select="scene_sad" diff --git a/libavfilter/Makefile b/libavfilter/Makefile index f6c1d641d6..a87e519b85 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -456,6 +456,8 @@ OBJS-$(CONFIG_ROBERTS_OPENCL_FILTER) += vf_convolution_opencl.o opencl.o OBJS-$(CONFIG_ROTATE_FILTER) += vf_rotate.o OBJS-$(CONFIG_SAB_FILTER) += vf_sab.o OBJS-$(CONFIG_SCALE_FILTER) += vf_scale.o scale_eval.o +OBJS-$(CONFIG_SCALE_AMF_FILTER) += vf_scale_amf.o scale_eval.o vf_scale_amf_common.o +OBJS-$(CONFIG_SCALE_AMF_FILTER) += vf_scale_amf_hq.o scale_eval.o vf_scale_amf_common.o OBJS-$(CONFIG_SCALE_CUDA_FILTER) += vf_scale_cuda.o scale_eval.o \ vf_scale_cuda.ptx.o cuda/load_helper.o OBJS-$(CONFIG_SCALE_NPP_FILTER) += vf_scale_npp.o scale_eval.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 149bf50997..299cfb148b 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -430,6 +430,8 @@ extern const AVFilter ff_vf_roberts_opencl; extern const AVFilter ff_vf_rotate; extern const AVFilter ff_vf_sab; extern const AVFilter ff_vf_scale; +extern const AVFilter ff_vf_scale_amf; +extern const AVFilter ff_vf_scale_amf_hq; extern const AVFilter ff_vf_scale_cuda; extern const AVFilter ff_vf_scale_npp; extern const AVFilter ff_vf_scale_qsv; diff --git a/libavfilter/vf_scale_amf.c b/libavfilter/vf_scale_amf.c new file mode 100644 index 0000000000..6d2c211d83 --- /dev/null +++ b/libavfilter/vf_scale_amf.c @@ -0,0 +1,266 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * scale video filter - AMF + */ + +#include +#include + +#include "libavutil/avassert.h" +#include "libavutil/imgutils.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "libavutil/time.h" + +#include "libavutil/hwcontext.h" +#include "libavutil/hwcontext_amf.h" + +#include "AMF/components/VideoConverter.h" +#include "vf_scale_amf_common.h" + +#include "avfilter.h" +#include "formats.h" +#include "video.h" +#include "scale_eval.h" +#include "internal.h" + +#if CONFIG_DXVA2 +#include +#endif + +#if CONFIG_D3D11VA +#include +#endif + +static int amf_scale_query_formats(AVFilterContext *avctx) +{ + const enum AVPixelFormat *output_pix_fmts; + static const enum AVPixelFormat input_pix_fmts[] = { + AV_PIX_FMT_NV12, + AV_PIX_FMT_P010, + AV_PIX_FMT_0RGB, + AV_PIX_FMT_BGR0, + AV_PIX_FMT_BGRA, + AV_PIX_FMT_RGB0, + AV_PIX_FMT_RGBA, + AV_PIX_FMT_GRAY8, + AV_PIX_FMT_YUV420P, + AV_PIX_FMT_YUV420P10, + AV_PIX_FMT_YUYV422, + AV_PIX_FMT_AMF, + AV_PIX_FMT_NONE, + }; + static const enum AVPixelFormat output_pix_fmts_default[] = { + AV_PIX_FMT_AMF, + AV_PIX_FMT_D3D11, + AV_PIX_FMT_DXVA2_VLD, + AV_PIX_FMT_NV12, + AV_PIX_FMT_BGRA, + AV_PIX_FMT_YUV420P, + AV_PIX_FMT_NONE, + }; + output_pix_fmts = output_pix_fmts_default; + + return amf_setup_input_output_formats(avctx, input_pix_fmts, output_pix_fmts); +} + +static int amf_scale_config_output(AVFilterLink *outlink) +{ + AVFilterContext *avctx = outlink->src; + AVFilterLink *inlink = avctx->inputs[0]; + AMFScaleContext *ctx = avctx->priv; + AVAMFDeviceContextInternal * internal = NULL; + AVHWFramesContext *hwframes_out = NULL; + AMFSize out_size; + int err; + AMF_RESULT res; + enum AMF_VIDEO_CONVERTER_COLOR_PROFILE_ENUM amf_color_profile; + enum AVPixelFormat in_format; + + err = amf_init_scale_config(outlink, &in_format); + if (err < 0) + return err; + // FIXME: add checks whether we have HW context + hwframes_out = (AVHWFramesContext*)ctx->hwframes_out_ref->data; + + internal = (AVAMFDeviceContextInternal * )ctx->amf_device_ctx_internal->data; + res = internal->factory->pVtbl->CreateComponent(internal->factory, internal->context, AMFVideoConverter, &ctx->scaler); + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_FILTER_NOT_FOUND, "CreateComponent(%ls) failed with error %d\n", AMFVideoConverter, res); + // FIXME: add checks whether we have HW context + AMF_ASSIGN_PROPERTY_INT64(res, ctx->scaler, AMF_VIDEO_CONVERTER_OUTPUT_FORMAT, (amf_int32)av_amf_av_to_amf_format(hwframes_out->sw_format)); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "AMFConverter-SetProperty() failed with error %d\n", res); + + out_size.width = outlink->w; + out_size.height = outlink->h; + AMF_ASSIGN_PROPERTY_SIZE(res, ctx->scaler, AMF_VIDEO_CONVERTER_OUTPUT_SIZE, out_size); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "AMFConverter-SetProperty() failed with error %d\n", res); + + AMF_ASSIGN_PROPERTY_INT64(res, ctx->scaler, AMF_VIDEO_CONVERTER_SCALE, (amf_int32)ctx->scale_type); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "AMFConverter-SetProperty() failed with error %d\n", res); + + amf_color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN; + + switch(ctx->color_profile) { + case AMF_VIDEO_CONVERTER_COLOR_PROFILE_601: + if (ctx->color_range == AMF_COLOR_RANGE_FULL) { + amf_color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_601; + } else { + amf_color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_601; + } + break; + case AMF_VIDEO_CONVERTER_COLOR_PROFILE_709: + if (ctx->color_range == AMF_COLOR_RANGE_FULL) { + amf_color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_709; + } else { + amf_color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_709; + } + break; + case AMF_VIDEO_CONVERTER_COLOR_PROFILE_2020: + if (ctx->color_range == AMF_COLOR_RANGE_FULL) { + amf_color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_2020; + } else { + amf_color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_2020; + } + break; + default: + amf_color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN; + break; + } + + if (amf_color_profile != AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN) { + AMF_ASSIGN_PROPERTY_INT64(res, ctx->scaler, AMF_VIDEO_CONVERTER_COLOR_PROFILE, amf_color_profile); + } + + if (ctx->color_range != AMF_COLOR_RANGE_UNDEFINED) { + AMF_ASSIGN_PROPERTY_INT64(res, ctx->scaler, AMF_VIDEO_CONVERTER_OUTPUT_COLOR_RANGE, ctx->color_range); + } + + if (ctx->primaries != AMF_COLOR_PRIMARIES_UNDEFINED) { + AMF_ASSIGN_PROPERTY_INT64(res, ctx->scaler, AMF_VIDEO_CONVERTER_OUTPUT_COLOR_PRIMARIES, ctx->primaries); + } + + if (ctx->trc != AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED) { + AMF_ASSIGN_PROPERTY_INT64(res, ctx->scaler, AMF_VIDEO_CONVERTER_OUTPUT_TRANSFER_CHARACTERISTIC, ctx->trc); + } + + res = ctx->scaler->pVtbl->Init(ctx->scaler, av_amf_av_to_amf_format(in_format), inlink->w, inlink->h); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "AMFConverter-Init() failed with error %d\n", res); + + return 0; +} + +#define OFFSET(x) offsetof(AMFScaleContext, x) +#define FLAGS AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_FILTERING_PARAM +static const AVOption scale_amf_options[] = { + { "w", "Output video width", OFFSET(w_expr), AV_OPT_TYPE_STRING, { .str = "iw" }, .flags = FLAGS }, + { "h", "Output video height", OFFSET(h_expr), AV_OPT_TYPE_STRING, { .str = "ih" }, .flags = FLAGS }, + { "format", "Output pixel format", OFFSET(format_str), AV_OPT_TYPE_STRING, { .str = "same" }, .flags = FLAGS }, + + { "scale_type", "Scale type", OFFSET(scale_type), AV_OPT_TYPE_INT, { .i64 = AMF_VIDEO_CONVERTER_SCALE_BILINEAR }, AMF_VIDEO_CONVERTER_SCALE_BILINEAR, AMF_VIDEO_CONVERTER_SCALE_BICUBIC, FLAGS, "scale_type" }, + { "bilinear", "Bilinear", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_CONVERTER_SCALE_BILINEAR }, 0, 0, FLAGS, "scale_type" }, + { "bicubic", "Bicubic", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_CONVERTER_SCALE_BICUBIC }, 0, 0, FLAGS, "scale_type" }, + + { "color_profile", "Color profile", OFFSET(color_profile), AV_OPT_TYPE_INT, { .i64 = AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN }, AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN, AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_2020, FLAGS, "color_profile" }, + { "bt601", "BT.601", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_CONVERTER_COLOR_PROFILE_601 }, 0, 0, FLAGS, "color_profile" }, + { "bt709", "BT.709", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_CONVERTER_COLOR_PROFILE_709 }, 0, 0, FLAGS, "color_profile" }, + { "bt2020", "BT.2020", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_CONVERTER_COLOR_PROFILE_2020 }, 0, 0, FLAGS, "color_profile" }, + + { "color_range", "Color range", OFFSET(color_range), AV_OPT_TYPE_INT, { .i64 = AMF_COLOR_RANGE_UNDEFINED }, AMF_COLOR_RANGE_UNDEFINED, AMF_COLOR_RANGE_FULL, FLAGS, "color_range" }, + { "studio", "Studio", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_RANGE_STUDIO }, 0, 0, FLAGS, "color_range" }, + { "full", "Full", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_RANGE_FULL }, 0, 0, FLAGS, "color_range" }, + + { "primaries", "Output color primaries", OFFSET(primaries), AV_OPT_TYPE_INT, { .i64 = AMF_COLOR_PRIMARIES_UNDEFINED }, AMF_COLOR_PRIMARIES_UNDEFINED, AMF_COLOR_PRIMARIES_JEDEC_P22, FLAGS, "primaries" }, + { "bt709", "BT.709", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_PRIMARIES_BT709 }, 0, 0, FLAGS, "primaries" }, + { "bt470m", "BT.470M", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_PRIMARIES_BT470M }, 0, 0, FLAGS, "primaries" }, + { "bt470bg", "BT.470BG", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_PRIMARIES_BT470BG }, 0, 0, FLAGS, "primaries" }, + { "smpte170m", "SMPTE170M", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_PRIMARIES_SMPTE170M }, 0, 0, FLAGS, "primaries" }, + { "smpte240m", "SMPTE240M", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_PRIMARIES_SMPTE240M }, 0, 0, FLAGS, "primaries" }, + { "film", "FILM", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_PRIMARIES_FILM }, 0, 0, FLAGS, "primaries" }, + { "bt2020", "BT2020", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_PRIMARIES_BT2020 }, 0, 0, FLAGS, "primaries" }, + { "smpte428", "SMPTE428", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_PRIMARIES_SMPTE428 }, 0, 0, FLAGS, "primaries" }, + { "smpte431", "SMPTE431", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_PRIMARIES_SMPTE431 }, 0, 0, FLAGS, "primaries" }, + { "smpte432", "SMPTE432", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_PRIMARIES_SMPTE432 }, 0, 0, FLAGS, "primaries" }, + { "jedec-p22", "JEDEC_P22", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_PRIMARIES_JEDEC_P22 }, 0, 0, FLAGS, "primaries" }, + + { "trc", "Output transfer characteristics", OFFSET(trc), AV_OPT_TYPE_INT, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED }, AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED, AMF_COLOR_TRANSFER_CHARACTERISTIC_ARIB_STD_B67, FLAGS, "trc" }, + { "bt709", "BT.709", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_BT709 }, 0, 0, FLAGS, "trc" }, + { "gamma22", "GAMMA22", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_GAMMA22 }, 0, 0, FLAGS, "trc" }, + { "gamma28", "GAMMA28", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_GAMMA28 }, 0, 0, FLAGS, "trc" }, + { "smpte170m", "SMPTE170M", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_SMPTE170M }, 0, 0, FLAGS, "trc" }, + { "smpte240m", "SMPTE240M", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_SMPTE240M }, 0, 0, FLAGS, "trc" }, + { "linear", "Linear", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_LINEAR }, 0, 0, FLAGS, "trc" }, + { "log", "LOG", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_LOG }, 0, 0, FLAGS, "trc" }, + { "log-sqrt", "LOG_SQRT", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_LOG_SQRT }, 0, 0, FLAGS, "trc" }, + { "iec61966-2-4", "IEC61966_2_4", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_IEC61966_2_4 }, 0, 0, FLAGS, "trc" }, + { "bt1361-ecg", "BT1361_ECG", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_BT1361_ECG }, 0, 0, FLAGS, "trc" }, + { "iec61966-2-1", "IEC61966_2_1", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_IEC61966_2_1 }, 0, 0, FLAGS, "trc" }, + { "bt2020-10", "BT.2020_10", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_BT2020_10 }, 0, 0, FLAGS, "trc" }, + { "bt2020-12", "BT.2020-12", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_BT2020_12 }, 0, 0, FLAGS, "trc" }, + { "smpte2084", "SMPTE2084", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_SMPTE2084 }, 0, 0, FLAGS, "trc" }, + { "smpte428", "SMPTE428", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_SMPTE428 }, 0, 0, FLAGS, "trc" }, + { "arib-std-b67", "ARIB_STD_B67", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_ARIB_STD_B67 }, 0, 0, FLAGS, "trc" }, + + { "force_original_aspect_ratio", "decrease or increase w/h if necessary to keep the original AR", OFFSET(force_original_aspect_ratio), AV_OPT_TYPE_INT, { .i64 = 0}, 0, 2, FLAGS, "force_oar" }, + { "disable", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = 0 }, 0, 0, FLAGS, "force_oar" }, + { "decrease", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = 1 }, 0, 0, FLAGS, "force_oar" }, + { "increase", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = 2 }, 0, 0, FLAGS, "force_oar" }, + { "force_divisible_by", "enforce that the output resolution is divisible by a defined integer when force_original_aspect_ratio is used", OFFSET(force_divisible_by), AV_OPT_TYPE_INT, { .i64 = 1}, 1, 256, FLAGS }, + + { NULL }, +}; + + +AVFILTER_DEFINE_CLASS(scale_amf); + +static const AVFilterPad amf_scale_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .filter_frame = amf_scale_filter_frame, + } +}; + +static const AVFilterPad amf_scale_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = amf_scale_config_output, + } +}; + +AVFilter ff_vf_scale_amf = { + .name = "scale_amf", + .description = NULL_IF_CONFIG_SMALL("AMF video scaling and format conversion"), + + .init = amf_scale_init, + .uninit = amf_scale_uninit, + FILTER_QUERY_FUNC(&amf_scale_query_formats), + + .priv_size = sizeof(AMFScaleContext), + .priv_class = &scale_amf_class, + + FILTER_INPUTS(amf_scale_inputs), + FILTER_OUTPUTS(amf_scale_outputs), + FILTER_SINGLE_PIXFMT(AV_PIX_FMT_AMF), + + .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + .flags = AVFILTER_FLAG_HWDEVICE, +}; diff --git a/libavfilter/vf_scale_amf_common.c b/libavfilter/vf_scale_amf_common.c new file mode 100644 index 0000000000..5c26ebd2c9 --- /dev/null +++ b/libavfilter/vf_scale_amf_common.c @@ -0,0 +1,515 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "vf_scale_amf_common.h" + +#include "libavutil/avassert.h" +#include "avfilter.h" +#include "internal.h" +#include "formats.h" +#include "libavutil/imgutils.h" + +#include "libavutil/hwcontext_amf.h" +#include "AMF/components/ColorSpace.h" +#include "scale_eval.h" + +#if CONFIG_DXVA2 +#include +#endif + +#if CONFIG_D3D11VA +#include +#endif + +int amf_scale_init(AVFilterContext *avctx) +{ + AMFScaleContext *ctx = avctx->priv; + + if (!strcmp(ctx->format_str, "same")) { + ctx->format = AV_PIX_FMT_NONE; + } else { + ctx->format = av_get_pix_fmt(ctx->format_str); + if (ctx->format == AV_PIX_FMT_NONE) { + av_log(avctx, AV_LOG_ERROR, "Unrecognized pixel format: %s\n", ctx->format_str); + return AVERROR(EINVAL); + } + } + + return 0; +} + +void amf_scale_uninit(AVFilterContext *avctx) +{ + AMFScaleContext *ctx = avctx->priv; + + if (ctx->scaler) { + ctx->scaler->pVtbl->Terminate(ctx->scaler); + ctx->scaler->pVtbl->Release(ctx->scaler); + ctx->scaler = NULL; + } + + av_buffer_unref(&ctx->amf_device_ref); + av_buffer_unref(&ctx->hwdevice_ref); + av_buffer_unref(&ctx->hwframes_in_ref); + av_buffer_unref(&ctx->hwframes_out_ref); +} + +int amf_scale_filter_frame(AVFilterLink *inlink, AVFrame *in) +{ + AVFilterContext *avctx = inlink->dst; + AMFScaleContext *ctx = avctx->priv; + AVFilterLink *outlink = avctx->outputs[0]; + AMF_RESULT res; + AMFSurface *surface_in; + AMFSurface *surface_out; + AMFData *data_out = NULL; + enum AVColorSpace out_colorspace; + enum AVColorRange out_color_range; + + AVFrame *out = NULL; + int ret = 0; + + if (!ctx->scaler) + return AVERROR(EINVAL); + + ret = amf_avframe_to_amfsurface(avctx, in, &surface_in); + if (ret < 0) + goto fail; + + res = ctx->scaler->pVtbl->SubmitInput(ctx->scaler, (AMFData*)surface_in); + AMF_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "SubmitInput() failed with error %d\n", res); + res = ctx->scaler->pVtbl->QueryOutput(ctx->scaler, &data_out); + AMF_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "QueryOutput() failed with error %d\n", res); + + if (data_out) { + AMFGuid guid = IID_AMFSurface(); + data_out->pVtbl->QueryInterface(data_out, &guid, (void**)&surface_out); // query for buffer interface + data_out->pVtbl->Release(data_out); + } + + out = amf_amfsurface_to_avframe(avctx, surface_out); + + ret = av_frame_copy_props(out, in); + av_frame_unref(in); + + out_colorspace = AVCOL_SPC_UNSPECIFIED; + + if (ctx->color_profile != AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN) { + switch(ctx->color_profile) { + case AMF_VIDEO_CONVERTER_COLOR_PROFILE_601: + out_colorspace = AVCOL_SPC_SMPTE170M; + break; + case AMF_VIDEO_CONVERTER_COLOR_PROFILE_709: + out_colorspace = AVCOL_SPC_BT709; + break; + case AMF_VIDEO_CONVERTER_COLOR_PROFILE_2020: + out_colorspace = AVCOL_SPC_BT2020_NCL; + break; + case AMF_VIDEO_CONVERTER_COLOR_PROFILE_JPEG: + out_colorspace = AVCOL_SPC_RGB; + break; + default: + out_colorspace = AVCOL_SPC_UNSPECIFIED; + break; + } + out->colorspace = out_colorspace; + } + + out_color_range = AVCOL_RANGE_UNSPECIFIED; + if (ctx->color_range == AMF_COLOR_RANGE_FULL) + out_color_range = AVCOL_RANGE_JPEG; + else if (ctx->color_range == AMF_COLOR_RANGE_STUDIO) + out_color_range = AVCOL_RANGE_MPEG; + + if (ctx->color_range != AMF_COLOR_RANGE_UNDEFINED) + out->color_range = out_color_range; + + if (ctx->primaries != AMF_COLOR_PRIMARIES_UNDEFINED) + out->color_primaries = ctx->primaries; + + if (ctx->trc != AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED) + out->color_trc = ctx->trc; + + + if (ret < 0) + goto fail; + + out->format = outlink->format; + out->width = outlink->w; + out->height = outlink->h; + + out->hw_frames_ctx = av_buffer_ref(ctx->hwframes_out_ref); + if (!out->hw_frames_ctx) { + ret = AVERROR(ENOMEM); + goto fail; + } + + if (inlink->sample_aspect_ratio.num) { + outlink->sample_aspect_ratio = av_mul_q((AVRational){outlink->h * inlink->w, outlink->w * inlink->h}, inlink->sample_aspect_ratio); + } else + outlink->sample_aspect_ratio = inlink->sample_aspect_ratio; + + av_frame_free(&in); + return ff_filter_frame(outlink, out); +fail: + av_frame_free(&in); + av_frame_free(&out); + return ret; +} + + + +int amf_setup_input_output_formats(AVFilterContext *avctx, + const enum AVPixelFormat *input_pix_fmts, + const enum AVPixelFormat *output_pix_fmts) +{ + int err; + int i; + AVFilterFormats *input_formats; + + //in case if hw_device_ctx is set to DXVA2 we change order of pixel formats to set DXVA2 be choosen by default + //The order is ignored if hw_frames_ctx is not NULL on the config_output stage + if (avctx->hw_device_ctx) { + AVHWDeviceContext *device_ctx = (AVHWDeviceContext*)avctx->hw_device_ctx->data; + + switch (device_ctx->type) { + #if CONFIG_D3D11VA + case AV_HWDEVICE_TYPE_D3D11VA: + { + static const enum AVPixelFormat output_pix_fmts_d3d11[] = { + AV_PIX_FMT_D3D11, + AV_PIX_FMT_NONE, + }; + output_pix_fmts = output_pix_fmts_d3d11; + } + break; + #endif + #if CONFIG_DXVA2 + case AV_HWDEVICE_TYPE_DXVA2: + { + static const enum AVPixelFormat output_pix_fmts_dxva2[] = { + AV_PIX_FMT_DXVA2_VLD, + AV_PIX_FMT_NONE, + }; + output_pix_fmts = output_pix_fmts_dxva2; + } + break; + #endif + default: + { + av_log(avctx, AV_LOG_ERROR, "Unsupported device : %s\n", av_hwdevice_get_type_name(device_ctx->type)); + return AVERROR(EINVAL); + } + break; + } + } + + input_formats = ff_make_format_list(output_pix_fmts); + if (!input_formats) { + return AVERROR(ENOMEM); + } + + for (i = 0; input_pix_fmts[i] != AV_PIX_FMT_NONE; i++) { + err = ff_add_format(&input_formats, input_pix_fmts[i]); + if (err < 0) + return err; + } + + if ((err = ff_formats_ref(input_formats, &avctx->inputs[0]->outcfg.formats)) < 0 || + (err = ff_formats_ref(ff_make_format_list(output_pix_fmts), + &avctx->outputs[0]->incfg.formats)) < 0) + return err; + return 0; +} + +int amf_copy_surface(AVFilterContext *avctx, const AVFrame *frame, + AMFSurface* surface) +{ + AMFPlane *plane; + uint8_t *dst_data[4]; + int dst_linesize[4]; + int planes; + int i; + + planes = surface->pVtbl->GetPlanesCount(surface); + av_assert0(planes < FF_ARRAY_ELEMS(dst_data)); + + for (i = 0; i < planes; i++) { + plane = surface->pVtbl->GetPlaneAt(surface, i); + dst_data[i] = plane->pVtbl->GetNative(plane); + dst_linesize[i] = plane->pVtbl->GetHPitch(plane); + } + av_image_copy(dst_data, dst_linesize, + (const uint8_t**)frame->data, frame->linesize, frame->format, + frame->width, frame->height); + + return 0; +} + +int amf_init_scale_config(AVFilterLink *outlink, enum AVPixelFormat *in_format) +{ + AVFilterContext *avctx = outlink->src; + AVFilterLink *inlink = avctx->inputs[0]; + AMFScaleContext *ctx = avctx->priv; + AVHWFramesContext *hwframes_out; + int err; + AMF_RESULT res; + + if ((err = ff_scale_eval_dimensions(avctx, + ctx->w_expr, ctx->h_expr, + inlink, outlink, + &ctx->width, &ctx->height)) < 0) + return err; + + ff_scale_adjust_dimensions(inlink, &ctx->width, &ctx->height, + ctx->force_original_aspect_ratio, ctx->force_divisible_by); + + av_buffer_unref(&ctx->amf_device_ref); + av_buffer_unref(&ctx->hwframes_in_ref); + av_buffer_unref(&ctx->hwframes_out_ref); + + if (inlink->hw_frames_ctx) { + AVHWFramesContext *frames_ctx = (AVHWFramesContext*)inlink->hw_frames_ctx->data; + if (frames_ctx->device_ctx->type == AV_HWDEVICE_TYPE_AMF) { + AVAMFDeviceContext * amf_ctx = frames_ctx->device_ctx->hwctx; + ctx->amf_device_ctx_internal = av_buffer_ref(amf_ctx->internal); + } + if (av_amf_av_to_amf_format(frames_ctx->sw_format) == AMF_SURFACE_UNKNOWN) { + av_log(avctx, AV_LOG_ERROR, "Format of input frames context (%s) is not supported by AMF.\n", + av_get_pix_fmt_name(frames_ctx->sw_format)); + return AVERROR(EINVAL); + } + + err = av_hwdevice_ctx_create_derived(&ctx->amf_device_ref, AV_HWDEVICE_TYPE_AMF, frames_ctx->device_ref, 0); + if (err < 0) + return err; + + ctx->hwframes_in_ref = av_buffer_ref(inlink->hw_frames_ctx); + if (!ctx->hwframes_in_ref) + return AVERROR(ENOMEM); + + ctx->hwframes_out_ref = av_hwframe_ctx_alloc(frames_ctx->device_ref); + if (!ctx->hwframes_out_ref) + return AVERROR(ENOMEM); + + hwframes_out = (AVHWFramesContext*)ctx->hwframes_out_ref->data; + hwframes_out->format = outlink->format; + hwframes_out->sw_format = frames_ctx->sw_format; + } else if (avctx->hw_device_ctx) { + AVHWDeviceContext *hwdev_ctx; + err = av_hwdevice_ctx_create_derived(&ctx->amf_device_ref, AV_HWDEVICE_TYPE_AMF, avctx->hw_device_ctx, 0); + if (err < 0) + return err; + hwdev_ctx = (AVHWDeviceContext*)avctx->hw_device_ctx->data; + if (hwdev_ctx->type == AV_HWDEVICE_TYPE_AMF) + { + AVAMFDeviceContext * amf_ctx = hwdev_ctx->hwctx; + ctx->amf_device_ctx_internal = av_buffer_ref(amf_ctx->internal); + } + ctx->hwdevice_ref = av_buffer_ref(avctx->hw_device_ctx); + if (!ctx->hwdevice_ref) + return AVERROR(ENOMEM); + + ctx->hwframes_out_ref = av_hwframe_ctx_alloc(ctx->hwdevice_ref); + if (!ctx->hwframes_out_ref) + return AVERROR(ENOMEM); + + hwframes_out = (AVHWFramesContext*)ctx->hwframes_out_ref->data; + hwframes_out->format = AV_PIX_FMT_AMF; + hwframes_out->sw_format = outlink->format; + } else { + AVAMFDeviceContextInternal *wrapped = av_mallocz(sizeof(*wrapped)); + ctx->amf_device_ctx_internal = av_buffer_create((uint8_t *)wrapped, sizeof(*wrapped), + av_amf_context_internal_free, NULL, 0); + if ((res == av_amf_context_internal_create((AVAMFDeviceContextInternal *)ctx->amf_device_ctx_internal->data, avctx, "", NULL, 0)) != 0) { + return res; + } + ctx->hwframes_out_ref = av_hwframe_ctx_alloc(ctx->amf_device_ref); + if (!ctx->hwframes_out_ref) + return AVERROR(ENOMEM); + + hwframes_out = (AVHWFramesContext*)ctx->hwframes_out_ref->data; + hwframes_out->format = outlink->format; + hwframes_out->sw_format = inlink->format; + } + + if (ctx->format != AV_PIX_FMT_NONE) { + hwframes_out->sw_format = ctx->format; + } + + if (inlink->format == AV_PIX_FMT_AMF) { + if (!inlink->hw_frames_ctx || !inlink->hw_frames_ctx->data) + return AVERROR(EINVAL); + else + *in_format = ((AVHWFramesContext*)inlink->hw_frames_ctx->data)->sw_format; + } else + *in_format = inlink->format; + + outlink->w = ctx->width; + outlink->h = ctx->height; + + hwframes_out->width = outlink->w; + hwframes_out->height = outlink->h; + + err = av_hwframe_ctx_init(ctx->hwframes_out_ref); + if (err < 0) + return err; + + outlink->hw_frames_ctx = av_buffer_ref(ctx->hwframes_out_ref); + if (!outlink->hw_frames_ctx) { + return AVERROR(ENOMEM); + } + return 0; +} + +void amf_free_amfsurface(void *opaque, uint8_t *data) +{ + AMFSurface *surface = (AMFSurface*)data; + surface->pVtbl->Release(surface); +} + +AVFrame *amf_amfsurface_to_avframe(AVFilterContext *avctx, AMFSurface* pSurface) +{ + AVFrame *frame = av_frame_alloc(); + AMFScaleContext *ctx = avctx->priv; + + if (!frame) + return NULL; + + if (ctx->hwframes_out_ref) { + AVHWFramesContext *hwframes_out = (AVHWFramesContext *)ctx->hwframes_out_ref->data; + if (hwframes_out->format == AV_PIX_FMT_AMF) { + int ret = av_hwframe_get_buffer(ctx->hwframes_out_ref, frame, 0); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Get hw frame failed.\n"); + av_frame_free(&frame); + return NULL; + } + frame->data[3] = (uint8_t *)pSurface; + frame->buf[1] = av_buffer_create((uint8_t *)pSurface, sizeof(AMFSurface), + amf_free_amfsurface, + (void*)avctx, + AV_BUFFER_FLAG_READONLY); + } else { // FIXME: add processing of other hw formats + av_log(ctx, AV_LOG_ERROR, "Unknown pixel format\n"); + return NULL; + } + } else { + + switch (pSurface->pVtbl->GetMemoryType(pSurface)) + { + #if CONFIG_D3D11VA + case AMF_MEMORY_DX11: + { + AMFPlane *plane0 = pSurface->pVtbl->GetPlaneAt(pSurface, 0); + frame->data[0] = plane0->pVtbl->GetNative(plane0); + frame->data[1] = (uint8_t*)(intptr_t)0; + + frame->buf[0] = av_buffer_create(NULL, + 0, + amf_free_amfsurface, + pSurface, + AV_BUFFER_FLAG_READONLY); + } + break; + #endif + #if CONFIG_DXVA2 + case AMF_MEMORY_DX9: + { + AMFPlane *plane0 = pSurface->pVtbl->GetPlaneAt(pSurface, 0); + frame->data[3] = plane0->pVtbl->GetNative(plane0); + + frame->buf[0] = av_buffer_create(NULL, + 0, + amf_free_amfsurface, + pSurface, + AV_BUFFER_FLAG_READONLY); + } + break; + #endif + default: + { + av_log(avctx, AV_LOG_ERROR, "Unsupported memory type : %d\n", pSurface->pVtbl->GetMemoryType(pSurface)); + return NULL; + } + } + } + + return frame; +} + +int amf_avframe_to_amfsurface(AVFilterContext *avctx, const AVFrame *frame, AMFSurface** ppSurface) +{ + AMFScaleContext *ctx = avctx->priv; + AVAMFDeviceContextInternal* internal = (AVAMFDeviceContextInternal *)ctx->amf_device_ctx_internal->data; + AMFSurface *surface; + AMF_RESULT res; + int hw_surface = 0; + + switch (frame->format) { +#if CONFIG_D3D11VA + case AV_PIX_FMT_D3D11: + { + static const GUID AMFTextureArrayIndexGUID = { 0x28115527, 0xe7c3, 0x4b66, { 0x99, 0xd3, 0x4f, 0x2a, 0xe6, 0xb4, 0x7f, 0xaf } }; + ID3D11Texture2D *texture = (ID3D11Texture2D*)frame->data[0]; // actual texture + int index = (intptr_t)frame->data[1]; // index is a slice in texture array is - set to tell AMF which slice to use + texture->lpVtbl->SetPrivateData(texture, &AMFTextureArrayIndexGUID, sizeof(index), &index); + + res = internal->context->pVtbl->CreateSurfaceFromDX11Native(internal->context, texture, &surface, NULL); // wrap to AMF surface + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR(ENOMEM), "CreateSurfaceFromDX11Native() failed with error %d\n", res); + hw_surface = 1; + } + break; +#endif + case AV_PIX_FMT_AMF: + { + surface = (AMFSurface*)frame->data[3]; // actual surface + hw_surface = 1; + } + break; + +#if CONFIG_DXVA2 + case AV_PIX_FMT_DXVA2_VLD: + { + IDirect3DSurface9 *texture = (IDirect3DSurface9 *)frame->data[3]; // actual texture + + res = internal->context->pVtbl->CreateSurfaceFromDX9Native(internal->context, texture, &surface, NULL); // wrap to AMF surface + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR(ENOMEM), "CreateSurfaceFromDX9Native() failed with error %d\n", res); + hw_surface = 1; + } + break; +#endif + default: + { + AMF_SURFACE_FORMAT amf_fmt = av_amf_av_to_amf_format(frame->format); + res = internal->context->pVtbl->AllocSurface(internal->context, AMF_MEMORY_HOST, amf_fmt, frame->width, frame->height, &surface); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR(ENOMEM), "AllocSurface() failed with error %d\n", res); + amf_copy_surface(avctx, frame, surface); + } + break; + } + + if (hw_surface) { + // input HW surfaces can be vertically aligned by 16; tell AMF the real size + surface->pVtbl->SetCrop(surface, 0, 0, frame->width, frame->height); + } + + surface->pVtbl->SetPts(surface, frame->pts); + *ppSurface = surface; + return 0; +} diff --git a/libavfilter/vf_scale_amf_common.h b/libavfilter/vf_scale_amf_common.h new file mode 100644 index 0000000000..a43c7602cb --- /dev/null +++ b/libavfilter/vf_scale_amf_common.h @@ -0,0 +1,71 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVFILTER_AMF_COMMON_H +#define AVFILTER_AMF_COMMON_H + +#include "avfilter.h" + +#include "AMF/core/Surface.h" +#include "AMF/components/Component.h" + +typedef struct AMFScaleContext { + const AVClass *class; + + int width, height; + enum AVPixelFormat format; + int scale_type; + int color_profile; + int color_range; + int primaries; + int trc; + int fill; + int fill_color; + int keep_ratio; + + // HQScaler properties + int algorithm; + float sharpness; + + char *w_expr; + char *h_expr; + char *format_str; + int force_original_aspect_ratio; + int force_divisible_by; + + AMFComponent *scaler; + AVBufferRef *amf_device_ref; + + AVBufferRef *hwframes_in_ref; + AVBufferRef *hwframes_out_ref; + AVBufferRef *hwdevice_ref; + + AVBufferRef *amf_device_ctx_internal; +} AMFScaleContext; + +int amf_scale_init(AVFilterContext *avctx); +void amf_scale_uninit(AVFilterContext *avctx); +int amf_init_scale_config(AVFilterLink *outlink, enum AVPixelFormat *in_format); +int amf_copy_surface(AVFilterContext *avctx, const AVFrame *frame, AMFSurface* surface); +void amf_free_amfsurface(void *opaque, uint8_t *data); +AVFrame *amf_amfsurface_to_avframe(AVFilterContext *avctx, AMFSurface* pSurface); +int amf_avframe_to_amfsurface(AVFilterContext *avctx, const AVFrame *frame, AMFSurface** ppSurface); +int amf_setup_input_output_formats(AVFilterContext *avctx, const enum AVPixelFormat *input_pix_fmts, const enum AVPixelFormat *output_pix_fmts); +int amf_scale_filter_frame(AVFilterLink *inlink, AVFrame *in); + +#endif /* AVFILTER_AMF_COMMON_H */ diff --git a/libavfilter/vf_scale_amf_hq.c b/libavfilter/vf_scale_amf_hq.c new file mode 100644 index 0000000000..63c9bd8301 --- /dev/null +++ b/libavfilter/vf_scale_amf_hq.c @@ -0,0 +1,191 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * scale video filter - AMF + */ + +#include +#include + +#include "libavutil/avassert.h" +#include "libavutil/imgutils.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "libavutil/time.h" + +#include "libavutil/hwcontext.h" +#include "libavutil/hwcontext_amf.h" + +#include "AMF/components/HQScaler.h" +#include "AMF/components/ColorSpace.h" +#include "vf_scale_amf_common.h" + +#include "avfilter.h" +#include "internal.h" +#include "formats.h" +#include "video.h" + +#if CONFIG_DXVA2 +#include +#endif + +#if CONFIG_D3D11VA +#include +#endif + + +static int amf_scale_query_formats(AVFilterContext *avctx) +{ + const enum AVPixelFormat *output_pix_fmts; + static const enum AVPixelFormat input_pix_fmts[] = { + AV_PIX_FMT_NV12, + AV_PIX_FMT_P010, + AV_PIX_FMT_BGRA, + AV_PIX_FMT_RGBA, + AV_PIX_FMT_AMF, + AV_PIX_FMT_RGBAF16, + AV_PIX_FMT_NONE, + }; + static const enum AVPixelFormat output_pix_fmts_default[] = { + AV_PIX_FMT_NV12, + AV_PIX_FMT_P010, + AV_PIX_FMT_BGRA, + AV_PIX_FMT_RGBA, + AV_PIX_FMT_AMF, + AV_PIX_FMT_D3D11, + AV_PIX_FMT_DXVA2_VLD, + AV_PIX_FMT_RGBAF16, + AV_PIX_FMT_NONE, + }; + output_pix_fmts = output_pix_fmts_default; + + return amf_setup_input_output_formats(avctx, input_pix_fmts, output_pix_fmts); +} + +static int amf_scale_config_output(AVFilterLink *outlink) +{ + AVFilterContext *avctx = outlink->src; + AVFilterLink *inlink = avctx->inputs[0]; + AMFScaleContext *ctx = avctx->priv; + AVAMFDeviceContextInternal * internal = NULL; + AMFSize out_size; + int err; + AMF_RESULT res; + enum AVPixelFormat in_format; + + err = amf_init_scale_config(outlink, &in_format); + if (err < 0) + return err; + + // HQ scaler should be used for upscaling only + if (inlink->w > outlink->w || inlink->h > outlink->h) { + av_log(avctx, AV_LOG_ERROR, "AMF HQ scaler should be used for upscaling only.\n"); + return AVERROR_UNKNOWN; + } + // FIXME: add checks whether we have HW context + + internal = (AVAMFDeviceContextInternal * )ctx->amf_device_ctx_internal->data; + res = internal->factory->pVtbl->CreateComponent(internal->factory, internal->context, AMFHQScaler, &ctx->scaler); + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_FILTER_NOT_FOUND, "CreateComponent(%ls) failed with error %d\n", AMFHQScaler, res); + + out_size.width = outlink->w; + out_size.height = outlink->h; + AMF_ASSIGN_PROPERTY_SIZE(res, ctx->scaler, AMF_HQ_SCALER_OUTPUT_SIZE, out_size); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "AMFHQScaler-SetProperty() failed with error %d\n", res); + + if (ctx->algorithm != -1) { + AMF_ASSIGN_PROPERTY_INT64(res, ctx->scaler, AMF_HQ_SCALER_ALGORITHM, ctx->algorithm); + } + if (ctx->sharpness != -1) { + AMF_ASSIGN_PROPERTY_DOUBLE(res, ctx->scaler, AMF_HQ_SCALER_SHARPNESS, ctx->sharpness); + } + AMF_ASSIGN_PROPERTY_BOOL(res, ctx->scaler, AMF_HQ_SCALER_FILL, ctx->fill); + AMF_ASSIGN_PROPERTY_BOOL(res, ctx->scaler, AMF_HQ_SCALER_KEEP_ASPECT_RATIO, ctx->keep_ratio); + // Setup default options to skip color conversion + ctx->color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN; + ctx->color_range = AMF_COLOR_RANGE_UNDEFINED; + ctx->primaries = AMF_COLOR_PRIMARIES_UNDEFINED; + ctx->trc = AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED; + + res = ctx->scaler->pVtbl->Init(ctx->scaler, av_amf_av_to_amf_format(in_format), inlink->w, inlink->h); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "AMFHQScaler-Init() failed with error %d\n", res); + + return 0; +} + +#define OFFSET(x) offsetof(AMFScaleContext, x) +#define FLAGS AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_FILTERING_PARAM +static const AVOption scale_amf_hq_options[] = { + { "w", "Output video width", OFFSET(w_expr), AV_OPT_TYPE_STRING, { .str = "iw" }, .flags = FLAGS }, + { "h", "Output video height", OFFSET(h_expr), AV_OPT_TYPE_STRING, { .str = "ih" }, .flags = FLAGS }, + + { "format", "Output pixel format", OFFSET(format_str), AV_OPT_TYPE_STRING, { .str = "same" }, .flags = FLAGS }, + { "sharpness", "Sharpness", OFFSET(sharpness), AV_OPT_TYPE_FLOAT, { .dbl = -1 }, -1, 2., FLAGS, "sharpness" }, + { "keep-ratio", "Keep aspect ratio", OFFSET(keep_ratio), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, FLAGS, "keep_ration" }, + { "fill", "Fill", OFFSET(fill), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, FLAGS, "fill" }, + + { "algorithm", "Scaling algorithm", OFFSET(algorithm), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, AMF_HQ_SCALER_ALGORITHM_VIDEOSR1_1, FLAGS, "algorithm" }, + { "bilinear", "Bilinear", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_HQ_SCALER_ALGORITHM_BILINEAR }, 0, 0, FLAGS, "algorithm" }, + { "bicubic", "Bicubic", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_HQ_SCALER_ALGORITHM_BICUBIC }, 0, 0, FLAGS, "algorithm" }, + { "sr1-0", "Video SR1.0", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_HQ_SCALER_ALGORITHM_VIDEOSR1_0 }, 0, 0, FLAGS, "algorithm" }, + { "point", "Point", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_HQ_SCALER_ALGORITHM_POINT }, 0, 0, FLAGS, "algorithm" }, + { "sr1-1", "Video SR1.1", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_HQ_SCALER_ALGORITHM_VIDEOSR1_1 }, 0, 0, FLAGS, "algorithm" }, + + { NULL }, +}; + + +AVFILTER_DEFINE_CLASS(scale_amf_hq); + +static const AVFilterPad amf_scale_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .filter_frame = amf_scale_filter_frame, + } +}; + +static const AVFilterPad amf_scale_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = amf_scale_config_output, + } +}; + +AVFilter ff_vf_scale_amf_hq = { + .name = "scale_amf_hq", + .description = NULL_IF_CONFIG_SMALL("AMF HQ video upscaling"), + + .init = amf_scale_init, + .uninit = amf_scale_uninit, + FILTER_QUERY_FUNC(&amf_scale_query_formats), + + .priv_size = sizeof(AMFScaleContext), + .priv_class = &scale_amf_hq_class, + + FILTER_INPUTS(amf_scale_inputs), + FILTER_OUTPUTS(amf_scale_outputs), + + FILTER_SINGLE_PIXFMT(AV_PIX_FMT_AMF), + + .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + .flags = AVFILTER_FLAG_HWDEVICE, +}; \ No newline at end of file From patchwork Wed Feb 14 01:55:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitrii Ovchinnikov X-Patchwork-Id: 46258 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:c493:b0:19e:cdac:8cce with SMTP id eo19csp110500pzb; Tue, 13 Feb 2024 17:56:56 -0800 (PST) X-Forwarded-Encrypted: i=2; AJvYcCWORpkoWdg6ZVBmbrrtcZAkGIDFwONCwczdVqcqqzq5zumPJ2U+r68AcoQVJzw69aYia36h04aMltYaTxpSCGm54u2dvRBFRCJlYw== X-Google-Smtp-Source: AGHT+IGSA92ABM8zJ9u6riEpcmI01AhBWdFtZ4le5uJd5nt4UXuMJodetitBkwpTolPs1mwQ3gDp X-Received: by 2002:a17:906:cd09:b0:a3d:4e12:c0c1 with SMTP id oz9-20020a170906cd0900b00a3d4e12c0c1mr18947ejb.39.1707875816196; Tue, 13 Feb 2024 17:56:56 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1707875816; cv=none; d=google.com; s=arc-20160816; b=F2Tr9djZ64X88N0qlU2MFbmiY9v42wF4PTUeGfLWBIAKGD4b39uZ6riZ30XDZKRzW9 js2ek9Mi5q1Nd4fO0bJtR54/VefGMx525BvoeDpeEg2VkMS/4YexognVAH35LNo6nGGK Iq4LVrKcmTo5CF9ZuMABTWzzDNfSdmCtlkZK8ua3w9vhOwavlefDzH1CH7ag0b2swrFW 1vM5Tzck7/8MkowXOE2khVFp6ajzk4lcSx12presr4AVv7pl0uycsP7lXmNN9BILd/+d zo7hNK+lE0R2GpjPBGv5vSs/BWJ6JrIrLklY9yjsjecPIEXA0mfsf7/feC0ZmLtkuBNI tNWA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=1unzYQt3ITLjW8ITXcKV83jbJMmI8jBndUizbRIgpuo=; fh=Nj3Hej1N6VJPtAbDZNT48UshbQgWoWJaV7ZadruWd4c=; b=tSSm6FN/UIxWmemOgRYApifdFCtYq8vS7EyLCQIQUn/CuL+YTCF1m1Pb+5G2q/GvxG mwAL7/xMyQfjLa/+VDLTlNK47iGT9xgatvTrxJxWACzgHEYHzcty87rOtJoPHvADaKsx cwyCYwSBSoQBPMVWtY1d58pwYQ+y7Uq4S+S+Pr/w0usx4XSeBxVMTpYZsqJBkU+9ak7B 7mW5gPDlbvyKssLGMHxweEWHoIS61RH80yT6+QGdjboxEdie4XXBq9dKroHaUpzoBWCD /nN4R86hZJPy4B8Aho050ZHE2ESbuTHcPtpTKA8yxkTr3gM5PuR01ktSMRJoD1zKKpD9 uJVA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=DPGzLD+y; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com X-Forwarded-Encrypted: i=1; AJvYcCX1ZdhxRRx7uSSPQ1OD2jNTRT4M2IfivkF0irbES+lYuH71iaxNF0STRLNJnddmy39O0hRjdbgqLGbBybv1TvAEAtUiR8w3yDjB860Y788e96sWpsKO5u3/BUNyvWYQd5Kb5eC0U8Os9sT0Of8/OqM4jdaUD5FuRYyhRghDqo4NogD33s95YhuhDT77RshW+ya3dV5ffXiB1E94qVuU2ziQL3k2fRTH3trwTOi9cSBIsjF2kp0EuBxZ/TXnQy+ewwx7DiuIL4tUBtVFj3WCtm6jkzwd/l8rQ4jXplZQkwZ2g12PmsNbpCjg8I0bqH6ne2V043WbhrPEKdQ/8WYXYGWbA5ZI8L5wsugNtz2z+HiY30Mkkz+lLz9N+01ZlrHzDBbNICKLmFcICH4mub/4C6OhvIOIc/7qG8/uN2ZrY7WsOBfO+7YUvGsiOuSsP1CFRbPDBRgRdD+InKJC/SlrxdHdb5hBxUJkderBqIYUEm9KDDv693Qj9+sYweyf8eOk3NJgmTG8YfWTXL2bz/5NTcLTx21cifOvpop2nf4P5Bb+1BcpvwBwMjP4LkHXXj3s9BIVsywdA6uzyykak+WwjwreZ0anqHkV3CbYZwQ9psyUNx33r/DApq2lm3MN2owiC/p8n/bEH2NoDf/qbeu8SyxUOeiYULL9XzFLAlbUnHHrstvVwzXuIk+DZiSfj12BX85Ea470Z5qZMWClqK5spiOi88XJy0BEoCv5ReA+Hbh9qeXIIbHkT0pyS9KKX1z0D/cBjPWcZBZ3tklXx/OHBWo7qtkgFfpZKw7YnYV0617Ohw0jQEBWjBuPJ6ECvd+Q3pvCzXH6v1lPBjkSHJC+gQeUVGqvq1HFqQaSMtoeVlUcZ0GZmikco+udVnsxedQ13TFfqWmRIB8bGXw6mH5kXL6ToJzKbdPwc4QKGWldbb6IPd3qCGBiokcIu8Hb+LUfqrRsfF 7LRtz8ySWB085o+S9+LnSetmU69KNkcYWVWaOIvCWRIYoh9qRWeXDBl0zeThZ5OJNlLGNWDJxyQwnEd5CBONbZsKsEwbl5xbeCwCwzAi/CNPLV4n/jV5IkUiwv0RY5AwFcohPf4QW7v1eX3stITlrE3+9PYKtaxEzy8+0nRNkwSiLpHRiHb7yiAQabRz280LgkaD/NpSXaRbjxU9z5xW8Zr2TC+q6tGZJBtJuPz7wx3wAGjkduZx+rOG8KH6aiJLXN5VEk9kj1hgWTXdWUibDSNxZKx5TzOH1EgEjlMCm3qTUaIiU2plMom4ZUgCCAHLtF1Tiy19LlGJWxqC+gkYJjM60oL4pPd+/TN5pVf/Oz2FjrSwGNkTdqzMo+y9n5Mofuv93y441/rAac7IMWc4qpPnIjkrpXt2B779XYtwtaMmuWkRwUCHP8VnTxbW6b4M9bKlv+DRySyHZOhtQCqay82cb3nX9lXXDLiHLyO5UtjntxDvI3++iRmYAJl6jVDOIe3wIU5B9MaUVYz+1aBIzgqOuIRxigr7HD8rHyosKAz/e0ddyv2rzBst5K8KPJw82eHSFJo9SibINx2bqO6OEv1grcJFgtLXOQ Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id gb37-20020a170907962500b00a3cecdeb74dsi1518069ejc.586.2024.02.13.17.56.55; Tue, 13 Feb 2024 17:56:56 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=DPGzLD+y; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id DA75468D1F8; Wed, 14 Feb 2024 03:55:49 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f54.google.com (mail-ej1-f54.google.com [209.85.218.54]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D4D6F68D1DF for ; Wed, 14 Feb 2024 03:55:40 +0200 (EET) Received: by mail-ej1-f54.google.com with SMTP id a640c23a62f3a-a2a17f3217aso639032966b.2 for ; Tue, 13 Feb 2024 17:55:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1707875740; x=1708480540; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=oeufYrTfoEOZuy/qPEO5tLldeonCuvJLaU/9VhC42d4=; b=DPGzLD+yY0AISwE8gLO+SdNKhzvVZljrtkOP5qUn9ARYU2aguIeu3EGw1PNIvHlmAg z+mJoJGK+z5rkahlZMLv1I6BqQEDB2gA0qas6sJaEKX4PVCJoJThZDAXOy1dwAqOHM0Y zsVCfLQwi+ku5pbP3dZCocN3aAeLQP6E5A5/jVgVwxnekTs9nCFDArwXSxhyUK3C0gT+ q0YhCAE2xy8SJcj4U9GHZv3l+Zpb7PuM3Xc98p/4xb6irVOVrZjDJ6c9WOlBQBOqMjNy htZNQ3DEIaoR2HPwnhBLFH0fKWoAlMVEfNH8sff+KRcF9YyLnAGkLh6xajXQSq/F/VEP p6lg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707875740; x=1708480540; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oeufYrTfoEOZuy/qPEO5tLldeonCuvJLaU/9VhC42d4=; b=gQ8RNCxfUN3DYZSLr7TQlT5jIegLc2UBd5hRj5yslZ6pltybo/pdJtrgekVBpX4YAG nd5TWAmLUIdSHTz6w9D4HsSbExtsYY/1Qighrn/eqCQbYUIdadTJSw4fWWTqVYqLsOTd +wqmQa1+cFVSd/BG8lEOUCZYtRc42DSHA7eQaSPUh0z/0UOWeQ2TlsuGZqFHbF4Usnlq PpGat/rv22S7owAOcp/BrZ6EZ3jHnMMl4bDTgTowKSFGhPVW79vk0wxSGP/L2idMpo2x 508RKVTBB553EEU996BiEYnNzUMzzJwG+h3ZrPPRLCo+UsMLD2INEehq3YqcAwTLKwBe 4yOQ== X-Gm-Message-State: AOJu0Ywdrzw0K7LCe76+n+AR8UizHOZ81/Ksg91qtowUl21Akmhg/I7E JE5IVSrD4xrXYRjCb1PK7sKP50x3FQVBXbWDVGTzfiXTHIrGhnJzGKFBcZaTT88= X-Received: by 2002:a17:906:378a:b0:a3b:e975:c530 with SMTP id n10-20020a170906378a00b00a3be975c530mr673391ejc.51.1707875739602; Tue, 13 Feb 2024 17:55:39 -0800 (PST) Received: from dovchinn.amd.com (cable-178-148-16-149.dynamic.sbb.rs. [178.148.16.149]) by smtp.gmail.com with ESMTPSA id vi16-20020a170907d41000b00a3ce31d3ffdsm1663230ejc.93.2024.02.13.17.55.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 17:55:38 -0800 (PST) From: Dmitrii Ovchinnikov To: ffmpeg-devel@ffmpeg.org Date: Wed, 14 Feb 2024 02:55:15 +0100 Message-Id: <20240214015515.1027-9-ovchinnikov.dmitrii@gmail.com> X-Mailer: git-send-email 2.38.1.windows.1 In-Reply-To: <20240214015515.1027-1-ovchinnikov.dmitrii@gmail.com> References: <20240214015515.1027-1-ovchinnikov.dmitrii@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 9/9] doc/filters: Add documentation for AMF filters X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Evgeny Pavlov Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 93eKw9Hj8YH6 From: Evgeny Pavlov Signed-off-by: Evgeny Pavlov --- doc/filters.texi | 238 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 238 insertions(+) diff --git a/doc/filters.texi b/doc/filters.texi index e0436a5755..4c5b9c1f63 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -21330,6 +21330,244 @@ If the specified expression is not valid, it is kept at its current value. @end table +@anchor{scale_amf} +@section scale_amf + +Scale (resize) and convert colorspace, transfer characteristics or color primaries for the input video, using AMD Advanced Media Framework library for hardware acceleration. +Setting the output width and height works in the same way as for the @ref{scale} filter. + +The filter accepts the following options: +@table @option +@item w +@item h +Set the output video dimension expression. Default value is the input dimension. + +Allows for the same expressions as the @ref{scale} filter. + +@item scale_type +Sets the algorithm used for scaling: + +@table @var +@item bilinear +Bilinear + +This is the default. + +@item bicubic +Bicubic + +@end table + +@item format +Controls the output pixel format. By default, or if none is specified, the input +pixel format is used. + + +@item force_original_aspect_ratio +@item force_divisible_by +Work the same as the identical @ref{scale} filter options. + +@anchor{color_profile} +@item color_profile +Specify all color properties at once. + +The accepted values are: +@table @samp +@item bt601 +BT.601 + +@item bt709 +BT.709 + +@item bt2020 +BT.2020 + +@end table + +@anchor{trc} +@item trc +Specify output transfer characteristics. + +The accepted values are: +@table @samp +@item bt709 +BT.709 + +@item gamma22 +Constant gamma of 2.2 + +@item gamma28 +Constant gamma of 2.8 + +@item smpte170m +SMPTE-170M + +@item smpte240m +SMPTE-240M + +@item linear +Linear + +@item log +LOG + +@item log-sqrt +LOG_SQRT + +@item iec61966-2-4 +iec61966-2-4 + +@item bt1361-ecg +BT1361_ECG + +@item iec61966-2-1 +iec61966-2-1 + +@item bt2020-10 +BT.2020 for 10-bits content + +@item bt2020-12 +BT.2020 for 12-bits content + +@item smpte2084 +SMPTE2084 + +@item smpte428 +SMPTE428 + +@item arib-std-b67 +ARIB_STD_B67 + +@end table + +@anchor{primaries} +@item primaries +Specify output color primaries. + +The accepted values are: +@table @samp +@item bt709 +BT.709 + +@item bt470m +BT.470M + +@item bt470bg +BT.470BG or BT.601-6 625 + +@item smpte170m +SMPTE-170M or BT.601-6 525 + +@item smpte240m +SMPTE-240M + +@item film +film + +@item bt2020 +BT.2020 + +@item smpte428 +SMPTE-428 + +@item smpte431 +SMPTE-431 + +@item smpte432 +SMPTE-432 + +@item jedec-p22 +JEDEC P22 phosphors + +@end table +@end table + +@subsection Examples + +@itemize +@item +Scale input to 720p, keeping aspect ratio and ensuring the output is yuv420p. +@example +scale_amf=-2:720:format=yuv420p +@end example + +@item +Upscale to 4K and change color profile to bt2020. +@example +scale_amf=4096:2160:color_profile=bt2020 +@end example +@end itemize + +@anchor{scale_amf_hq} +@section scale_amf_hq + +Upscale (size increasing) for the input video using AMD Advanced Media Framework library for hardware acceleration. +Use advanced algorithms for upscaling with higher output quality. +Setting the output width and height works in the same way as for the @ref{scale} filter. + +The filter accepts the following options: +@table @option +@item w +@item h +Set the output video dimension expression. Default value is the input dimension. + +Allows for the same expressions as the @ref{scale} filter. + +@item algorithm +Sets the algorithm used for scaling: + +@table @var +@item bilinear +Bilinear + +@item bicubic +Bicubic + +@item sr1-0 +Video SR1.0 +This is a default value + +@item point +Point + +@item sr1-1 +Video SR1.1 + +@end table + +@item sharpness +Control hq scaler sharpening. The value is a float in the range of [0.0, 2.0] + +@item format +Controls the output pixel format. By default, or if none is specified, the input +pixel format is used. + +@item keep-ratio +Force the scaler to keep the aspect ratio of the input image when the output size has a different aspect ratio. +Default value is false. + +@item fill +Specifies whether the output image outside the region of interest, +which does not fill the entire output surface should be filled with a solid color. + +@end table + +@subsection Examples + +@itemize +@item +Scale input to 720p, keeping aspect ratio and ensuring the output is yuv420p. +@example +scale_amf_hq=-2:720:format=yuv420p +@end example + +@item +Upscale to 4K with algorithm video SR1.1. +@example +scale_amf_hq=4096:2160:algorithm=sr1-1 +@end example +@end itemize + @anchor{scale_cuda} @section scale_cuda