From patchwork Mon Oct 21 23:16:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitrii Ovchinnikov X-Patchwork-Id: 52481 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:1c41:b0:48e:c0f8:d0de with SMTP id im1csp670598vqb; Thu, 24 Oct 2024 14:24:05 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCW3VBt8HU+3lmOUxUZBS7UPfbxxgeQ4huOPtuGxdrBzfCDQgJ8EM4k99ZN9JRyQ6r2R8352gls6heBvBH/Bhzt0@gmail.com X-Google-Smtp-Source: AGHT+IFRou1++y+/wjzmV/HQpBT/+/wGxKOoMhDh7wAMne2hMkPHiCyy4aZs8bm4jHC8vdUBnTTL X-Received: by 2002:a05:6512:6cd:b0:539:fb6f:cb8d with SMTP id 2adb3069b0e04-53b23e06b25mr2483836e87.27.1729805045378; Thu, 24 Oct 2024 14:24:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1729805045; cv=none; d=google.com; s=arc-20240605; b=BSDlY6SjXJtVnbtRvtwV6q/IQ7g/OZIWMgRPSauwGhnPrylzHdvVWSqALaGnql3jKQ ZRisPzt6iqNxyt/Nv/KJ6KSQcDfQ2hPaz4vmozThY/iKhw5jpkkQEzkfzn+KvaA3EdDW iSuI19Y6RMbpZgxIeMgK9O72HDVDuq2XuRItu29wMuK8BOXQ0oork7MXDcJS3oQJBrJV T8wSYQp3xIxHFVpECY0Ddav/Od6EBnwWwHNmFkAM7hPon1doQFxlaC2ztIxoGWGOUBzt +PlmJaWcOZwnwBigou4vz1o+dAmn8SxZg3iy5XQIeYcd3tfJGwjRXsSM4L2WGDyRMXTn gftA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=6riHX9B24ykWQL45peQJbthHYaifyh+wDKSR3RG9i5U=; fh=dTRBPGPJ3wU94AH5tguagErHz3vxaVKbj09dNGTx2d0=; b=WoWYhcG+aDbZGMhc4MMjzi439fEmFRWWuYjMyaa1zrkCRhu/ztwKfNai8I6zWioyfr NJYQHnmNKFW0pCD/YHZtz65wQRTC+azDl8aQgNIZ6eDGyQ76Ff7YtvGwNuLt72kXeXB2 N7JQ6VuA+8gTY0XkRdAv5wuX8MjQVNjldVrnPZZjXSsoF1BCDhK2XZyVIj1ffOHqOqs/ zWLinoeyPz/vPZ7NGO+0iehCVM3NZHNU4iTq+Koy6FHE67ObAsSZTPaEPZ87I+VtWx+f /zTOQx8KZMC5ZrOXRljT4fCI1c2M+Zwf6bhpsFvHKfQ2EO6YETrRWTUc0rTeL/0icaqC 8Msg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=SCsyHnee; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 2adb3069b0e04-53a223f0492si3636388e87.171.2024.10.24.14.24.04; Thu, 24 Oct 2024 14:24:05 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=SCsyHnee; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4B96A68DDFC; Tue, 22 Oct 2024 02:17:06 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f51.google.com (mail-wr1-f51.google.com [209.85.221.51]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 29CC168D663 for ; Tue, 22 Oct 2024 02:16:59 +0300 (EEST) Received: by mail-wr1-f51.google.com with SMTP id ffacd0b85a97d-37d5689eea8so3341795f8f.1 for ; Mon, 21 Oct 2024 16:16:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729552618; x=1730157418; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=Ikce9rc/CAzD3hd6fA5KKQrdbzPeTU28RfHDCi6i5N0=; b=SCsyHneei+hYeZsXRDv8UIftf6CAsZpUhQ0l++9RrU76IJfk6GxK5cclV7KYTpFX0F TITLVz9IdC2mJWB4Sp7QJvMZltQPPtheSfvlTHzKsap3vLR+3xcfpIC2dcxiy+exleWH 9r7GIztuzxcMTcFJQDIIHENJ1eixTwllCNBpFZojSrARH+h2M+BWRbEz6ZMVB7w0yYbH 2pb7HQn+RWH9O6QGG08FmL50PqK/igAc0rLcYlk0r3Kw/QMwtB+lQMTJ7FksdQPNvi9D e1srC7PmJC7FLT5NDnb1ssHEJdmvmD88j1r22aAj8PgMXlVhxGKecfhG9i2gLZK+oyQL /H7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729552618; x=1730157418; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Ikce9rc/CAzD3hd6fA5KKQrdbzPeTU28RfHDCi6i5N0=; b=NQW5RTGLj88w2gvEfIHHsD2Bo94cQZ3qQtcsP/+nL88+tXuW7bBgOQGWIWFIrgtZfb hE8lVzijJyMLrhax6/airr4rZxOrrMncaJBVLfdiwBTKFyw8UU7FpeZrbMcvCci0ih3b ijlH36Pw3i3JiMkdy1vm+uAcRf5RHIPfAsy/jtSuCbwD3xop6QF93G3/jrknZ4hDrj0a r84oH+SkaK7c6dlTQEvZ0wsfKLKlh9geHH0dJRdUQ0TfHm9So92k/KzPxDDzHMjx32Dc BLonjGNSAQuyAuS9msf74NyLtz1mQfxbPd2x1x5xucHAjO9IuKLDyE2E43F7OuFCdHlC Z4Vg== X-Gm-Message-State: AOJu0YyA9gxKrLofu1HFEgTezEKBQSgHWYIy99udlDhCaAqc/0Y5IOUY OOj0R2O43Sr8YndwitRgbRn6ZSiwAxpP3FiQPfJxsCZcyaIIHrBugEsARA== X-Received: by 2002:a05:6000:100d:b0:37d:4f1b:359 with SMTP id ffacd0b85a97d-37eb48a0f28mr8078688f8f.53.1729552617415; Mon, 21 Oct 2024 16:16:57 -0700 (PDT) Received: from dovchinn.amd.com (cable-178-148-16-149.dynamic.sbb.rs. [178.148.16.149]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-37ee0a57e0fsm5293937f8f.47.2024.10.21.16.16.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Oct 2024 16:16:55 -0700 (PDT) From: Dmitrii Ovchinnikov To: ffmpeg-devel@ffmpeg.org Date: Tue, 22 Oct 2024 01:16:32 +0200 Message-Id: <20241021231636.1199-1-ovchinnikov.dmitrii@gmail.com> X-Mailer: git-send-email 2.38.1.windows.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/5] avutil: add hwcontext_amf. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Dmitrii Ovchinnikov Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Euqo2AtjTim8 Adds hwcontext_amf, which allows to use shared AMF context for the encoder, decoder and AMF-based filters, without copy to the host memory. It will also allow you to use some optimisations in the interaction of components (for example, SAV) and make a more manageable and optimal setup for using GPU devices with AMF in the case of a fully AMF pipeline. It will be a significant performance uplift when full AMF pipeline with filters is used. We also plan to add Compression artefact removal filter in near feature. v2: cleanup header files v3: an unnecessary class has been removed. --- libavutil/Makefile | 4 + libavutil/hwcontext.c | 4 + libavutil/hwcontext.h | 1 + libavutil/hwcontext_amf.c | 546 +++++++++++++++++++++++++++++ libavutil/hwcontext_amf.h | 45 +++ libavutil/hwcontext_amf_internal.h | 44 +++ libavutil/hwcontext_internal.h | 1 + libavutil/pixdesc.c | 4 + libavutil/pixfmt.h | 5 + 9 files changed, 654 insertions(+) create mode 100644 libavutil/hwcontext_amf.c create mode 100644 libavutil/hwcontext_amf.h create mode 100644 libavutil/hwcontext_amf_internal.h diff --git a/libavutil/Makefile b/libavutil/Makefile index 847878d7a4..2992b53277 100644 --- a/libavutil/Makefile +++ b/libavutil/Makefile @@ -46,6 +46,7 @@ HEADERS = adler32.h \ hwcontext_d3d12va.h \ hwcontext_drm.h \ hwcontext_dxva2.h \ + hwcontext_amf.h \ hwcontext_qsv.h \ hwcontext_mediacodec.h \ hwcontext_opencl.h \ @@ -196,6 +197,7 @@ OBJS-$(CONFIG_CUDA) += hwcontext_cuda.o OBJS-$(CONFIG_D3D11VA) += hwcontext_d3d11va.o OBJS-$(CONFIG_D3D12VA) += hwcontext_d3d12va.o OBJS-$(CONFIG_DXVA2) += hwcontext_dxva2.o +OBJS-$(CONFIG_AMF) += hwcontext_amf.o OBJS-$(CONFIG_LIBDRM) += hwcontext_drm.o OBJS-$(CONFIG_MACOS_KPERF) += macos_kperf.o OBJS-$(CONFIG_MEDIACODEC) += hwcontext_mediacodec.o @@ -220,6 +222,8 @@ SKIPHEADERS-$(CONFIG_CUDA) += hwcontext_cuda_internal.h \ SKIPHEADERS-$(CONFIG_D3D11VA) += hwcontext_d3d11va.h SKIPHEADERS-$(CONFIG_D3D12VA) += hwcontext_d3d12va.h SKIPHEADERS-$(CONFIG_DXVA2) += hwcontext_dxva2.h +SKIPHEADERS-$(CONFIG_AMF) += hwcontext_amf.h \ + hwcontext_amf_internal SKIPHEADERS-$(CONFIG_QSV) += hwcontext_qsv.h SKIPHEADERS-$(CONFIG_OPENCL) += hwcontext_opencl.h SKIPHEADERS-$(CONFIG_VAAPI) += hwcontext_vaapi.h diff --git a/libavutil/hwcontext.c b/libavutil/hwcontext.c index fa99a0d8a4..f06d49c45c 100644 --- a/libavutil/hwcontext.c +++ b/libavutil/hwcontext.c @@ -65,6 +65,9 @@ static const HWContextType * const hw_table[] = { #endif #if CONFIG_VULKAN &ff_hwcontext_type_vulkan, +#endif +#if CONFIG_AMF + &ff_hwcontext_type_amf, #endif NULL, }; @@ -82,6 +85,7 @@ static const char *const hw_type_names[] = { [AV_HWDEVICE_TYPE_VIDEOTOOLBOX] = "videotoolbox", [AV_HWDEVICE_TYPE_MEDIACODEC] = "mediacodec", [AV_HWDEVICE_TYPE_VULKAN] = "vulkan", + [AV_HWDEVICE_TYPE_AMF] = "amf", }; typedef struct FFHWDeviceContext { diff --git a/libavutil/hwcontext.h b/libavutil/hwcontext.h index bac30debae..96042ba197 100644 --- a/libavutil/hwcontext.h +++ b/libavutil/hwcontext.h @@ -38,6 +38,7 @@ enum AVHWDeviceType { AV_HWDEVICE_TYPE_MEDIACODEC, AV_HWDEVICE_TYPE_VULKAN, AV_HWDEVICE_TYPE_D3D12VA, + AV_HWDEVICE_TYPE_AMF, }; /** diff --git a/libavutil/hwcontext_amf.c b/libavutil/hwcontext_amf.c new file mode 100644 index 0000000000..2491e968ae --- /dev/null +++ b/libavutil/hwcontext_amf.c @@ -0,0 +1,546 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "buffer.h" +#include "common.h" +#include "hwcontext.h" +#include "hwcontext_amf.h" +#include "hwcontext_internal.h" +#include "hwcontext_amf_internal.h" +#if CONFIG_VULKAN +#include "hwcontext_vulkan.h" +#endif +#if CONFIG_D3D11VA +#include "libavutil/hwcontext_d3d11va.h" +#endif +#if CONFIG_DXVA2 +#define COBJMACROS +#include "libavutil/hwcontext_dxva2.h" +#endif +#include "mem.h" +#include "pixdesc.h" +#include "pixfmt.h" +#include "imgutils.h" +#include "libavutil/avassert.h" +#include +#include +#ifdef _WIN32 +#include "compat/w32dlfcn.h" +#else +#include +#endif +#define FFMPEG_AMF_WRITER_ID L"ffmpeg_amf" + + +typedef struct AmfTraceWriter { + AMFTraceWriterVtbl *vtblp; + void *avctx; + AMFTraceWriterVtbl vtbl; +} AmfTraceWriter; + +static void AMF_CDECL_CALL AMFTraceWriter_Write(AMFTraceWriter *pThis, + const wchar_t *scope, const wchar_t *message) +{ + AmfTraceWriter *tracer = (AmfTraceWriter*)pThis; + av_log(tracer->avctx, AV_LOG_DEBUG, "%ls: %ls", scope, message); // \n is provided from AMF +} + +static void AMF_CDECL_CALL AMFTraceWriter_Flush(AMFTraceWriter *pThis) +{ +} + +static AmfTraceWriter * amf_writer_alloc(void *avctx) +{ + AmfTraceWriter * writer = av_mallocz(sizeof(AmfTraceWriter)); + if (!writer) + return NULL; + + writer->vtblp = &writer->vtbl; + writer->vtblp->Write = AMFTraceWriter_Write; + writer->vtblp->Flush = AMFTraceWriter_Flush; + writer->avctx = avctx; + + return writer; +} + +static void amf_writer_free(void *opaque) +{ + AmfTraceWriter *writer = (AmfTraceWriter *)opaque; + av_freep(&writer); +} + +/** + * We still need AVHWFramesContext to utilize our hardware memory + * otherwise, we will receive the error "HW format requires hw_frames_ctx to be non-NULL". + * (libavfilter\buffersrc.c function query_formats) +*/ +typedef struct { + void *dummy; +} AMFFramesContext; + +typedef struct AVAMFFormatMap { + enum AVPixelFormat av_format; + enum AMF_SURFACE_FORMAT amf_format; +} FormatMap; + +const FormatMap format_map[] = +{ + { AV_PIX_FMT_NONE, AMF_SURFACE_UNKNOWN }, + { AV_PIX_FMT_NV12, AMF_SURFACE_NV12 }, + { AV_PIX_FMT_BGR0, AMF_SURFACE_BGRA }, + { AV_PIX_FMT_RGB0, AMF_SURFACE_RGBA }, + { AV_PIX_FMT_BGRA, AMF_SURFACE_BGRA }, + { AV_PIX_FMT_ARGB, AMF_SURFACE_ARGB }, + { AV_PIX_FMT_RGBA, AMF_SURFACE_RGBA }, + { AV_PIX_FMT_GRAY8, AMF_SURFACE_GRAY8 }, + { AV_PIX_FMT_YUV420P, AMF_SURFACE_YUV420P }, + { AV_PIX_FMT_YUYV422, AMF_SURFACE_YUY2 }, + { AV_PIX_FMT_P010, AMF_SURFACE_P010 }, +}; + +enum AMF_SURFACE_FORMAT av_av_to_amf_format(enum AVPixelFormat fmt) +{ + int i; + for (i = 0; i < amf_countof(format_map); i++) { + if (format_map[i].av_format == fmt) { + return format_map[i].amf_format; + } + } + return AMF_SURFACE_UNKNOWN; +} + +enum AVPixelFormat av_amf_to_av_format(enum AMF_SURFACE_FORMAT fmt) +{ + int i; + for (i = 0; i < amf_countof(format_map); i++) { + if (format_map[i].amf_format == fmt) { + return format_map[i].av_format; + } + } + return AMF_SURFACE_UNKNOWN; +} + +static const enum AVPixelFormat supported_formats[] = { + AV_PIX_FMT_NV12, + AV_PIX_FMT_YUV420P, + AV_PIX_FMT_BGRA, + AV_PIX_FMT_P010, + AV_PIX_FMT_YUV420P10, +#if CONFIG_D3D11VA + AV_PIX_FMT_D3D11, +#endif +#if CONFIG_DXVA2 + AV_PIX_FMT_DXVA2_VLD, +#endif +}; + +static int amf_frames_get_constraints(AVHWDeviceContext *ctx, + const void *hwconfig, + AVHWFramesConstraints *constraints) +{ + int i; + + constraints->valid_sw_formats = av_malloc_array(FF_ARRAY_ELEMS(supported_formats) + 1, + sizeof(*constraints->valid_sw_formats)); + if (!constraints->valid_sw_formats) + return AVERROR(ENOMEM); + + for (i = 0; i < FF_ARRAY_ELEMS(supported_formats); i++) + constraints->valid_sw_formats[i] = supported_formats[i]; + constraints->valid_sw_formats[FF_ARRAY_ELEMS(supported_formats)] = AV_PIX_FMT_NONE; + + constraints->valid_hw_formats = av_malloc_array(2, sizeof(*constraints->valid_hw_formats)); + if (!constraints->valid_hw_formats) + return AVERROR(ENOMEM); + + constraints->valid_hw_formats[0] = AV_PIX_FMT_AMF_SURFACE; + constraints->valid_hw_formats[1] = AV_PIX_FMT_NONE; + + return 0; +} + +static void amf_dummy_free(void *opaque, uint8_t *data) +{ + +} + +static AVBufferRef *amf_pool_alloc(void *opaque, size_t size) +{ + AVHWFramesContext *hwfc = (AVHWFramesContext *)opaque; + AVBufferRef *buf; + + buf = av_buffer_create(NULL, NULL, amf_dummy_free, hwfc, AV_BUFFER_FLAG_READONLY); + if (!buf) { + av_log(hwfc, AV_LOG_ERROR, "Failed to create buffer for AMF context.\n"); + return NULL; + } + return buf; +} + +static int amf_frames_init(AVHWFramesContext *ctx) +{ + int i; + + for (i = 0; i < FF_ARRAY_ELEMS(supported_formats); i++) { + if (ctx->sw_format == supported_formats[i]) + break; + } + if (i == FF_ARRAY_ELEMS(supported_formats)) { + av_log(ctx, AV_LOG_ERROR, "Pixel format '%s' is not supported\n", + av_get_pix_fmt_name(ctx->sw_format)); + return AVERROR(ENOSYS); + } + + ffhwframesctx(ctx)->pool_internal = + av_buffer_pool_init2(sizeof(AMFSurface), ctx, + &amf_pool_alloc, NULL); + + return 0; +} + + +static int amf_get_buffer(AVHWFramesContext *ctx, AVFrame *frame) +{ + frame->buf[0] = av_buffer_pool_get(ctx->pool); + if (!frame->buf[0]) + return AVERROR(ENOMEM); + + frame->data[0] = frame->buf[0]->data; + frame->format = AV_PIX_FMT_AMF_SURFACE; + frame->width = ctx->width; + frame->height = ctx->height; + return 0; +} + +static int amf_transfer_get_formats(AVHWFramesContext *ctx, + enum AVHWFrameTransferDirection dir, + enum AVPixelFormat **formats) +{ + enum AVPixelFormat *fmts; + + fmts = av_malloc_array(2, sizeof(*fmts)); + if (!fmts) + return AVERROR(ENOMEM); + + fmts[0] = ctx->sw_format; + fmts[1] = AV_PIX_FMT_NONE; + + *formats = fmts; + + return 0; +} + +static int amf_transfer_data_to(AVHWFramesContext *ctx, AVFrame *dst, + const AVFrame *src) +{ + AMFSurface* surface = (AMFSurface*)dst->data[0]; + AMFPlane *plane; + uint8_t *dst_data[4]; + int dst_linesize[4]; + int planes; + int i; + int res; + int w = FFMIN(dst->width, src->width); + int h = FFMIN(dst->height, src->height); + + if (!surface) { + AVHWDeviceContext *hwdev_ctx = ctx->device_ctx; + AVAMFDeviceContext *amf_device_ctx = (AVAMFDeviceContext *)hwdev_ctx->hwctx; + AMF_SURFACE_FORMAT format = av_av_to_amf_format(ctx->sw_format); + res = amf_device_ctx->context->pVtbl->AllocSurface(amf_device_ctx->context, AMF_MEMORY_HOST, format, dst->width, dst->height, &surface); + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR(ENOMEM), "AllocSurface() failed with error %d\n", res); + dst->data[0] = (uint8_t *)surface; + } + + planes = (int)surface->pVtbl->GetPlanesCount(surface); + av_assert0(planes < FF_ARRAY_ELEMS(dst_data)); + + for (i = 0; i < planes; i++) { + plane = surface->pVtbl->GetPlaneAt(surface, i); + dst_data[i] = plane->pVtbl->GetNative(plane); + dst_linesize[i] = plane->pVtbl->GetHPitch(plane); + } + av_image_copy2(dst_data, dst_linesize, + src->data, src->linesize, src->format, + w, h); + + return 0; +} + +static int amf_transfer_data_from(AVHWFramesContext *ctx, AVFrame *dst, + const AVFrame *src) +{ + AMFSurface* surface = (AMFSurface*)src->data[0]; + AMFPlane *plane; + uint8_t *src_data[4]; + int src_linesize[4]; + int planes; + int i; + int w = FFMIN(dst->width, src->width); + int h = FFMIN(dst->height, src->height); + int ret; + + ret = surface->pVtbl->Convert(surface, AMF_MEMORY_HOST); + AMF_RETURN_IF_FALSE(ctx, ret == AMF_OK, AVERROR_UNKNOWN, "Convert(amf::AMF_MEMORY_HOST) failed with error %d\n", AVERROR_UNKNOWN); + + planes = (int)surface->pVtbl->GetPlanesCount(surface); + av_assert0(planes < FF_ARRAY_ELEMS(src_data)); + + for (i = 0; i < planes; i++) { + plane = surface->pVtbl->GetPlaneAt(surface, i); + src_data[i] = plane->pVtbl->GetNative(plane); + src_linesize[i] = plane->pVtbl->GetHPitch(plane); + } + av_image_copy2(dst->data, dst->linesize, + src_data, src_linesize, dst->format, + w, h); + return 0; +} + + +static void amf_device_uninit(AVHWDeviceContext *device_ctx) +{ + AVAMFDeviceContext *amf_ctx = device_ctx->hwctx; + AMF_RESULT res; + AMFTrace *trace; + + if (amf_ctx->context) { + amf_ctx->context->pVtbl->Terminate(amf_ctx->context); + amf_ctx->context->pVtbl->Release(amf_ctx->context); + amf_ctx->context = NULL; + } + + res = amf_ctx->factory->pVtbl->GetTrace(amf_ctx->factory, &trace); + if (res == AMF_OK) { + trace->pVtbl->UnregisterWriter(trace, FFMPEG_AMF_WRITER_ID); + } + + if(amf_ctx->library) { + dlclose(amf_ctx->library); + amf_ctx->library = NULL; + } + if (amf_ctx->trace_writer) { + amf_writer_free(amf_ctx->trace_writer); + } + + amf_ctx->version = 0; +} + +static int amf_device_init(AVHWDeviceContext *ctx) +{ + AVAMFDeviceContext *amf_ctx = ctx->hwctx; + AMFContext1 *context1 = NULL; + AMF_RESULT res; + + res = amf_ctx->context->pVtbl->InitDX11(amf_ctx->context, NULL, AMF_DX11_1); + if (res == AMF_OK) { + av_log(ctx, AV_LOG_VERBOSE, "AMF initialisation succeeded via D3D11.\n"); + } else { + res = amf_ctx->context->pVtbl->InitDX9(amf_ctx->context, NULL); + if (res == AMF_OK) { + av_log(ctx, AV_LOG_VERBOSE, "AMF initialisation succeeded via D3D9.\n"); + } else { + AMFGuid guid = IID_AMFContext1(); + res = amf_ctx->context->pVtbl->QueryInterface(amf_ctx->context, &guid, (void**)&context1); + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "CreateContext1() failed with error %d\n", res); + + res = context1->pVtbl->InitVulkan(context1, NULL); + context1->pVtbl->Release(context1); + if (res != AMF_OK) { + if (res == AMF_NOT_SUPPORTED) + av_log(ctx, AV_LOG_ERROR, "AMF via Vulkan is not supported on the given device.\n"); + else + av_log(ctx, AV_LOG_ERROR, "AMF failed to initialise on the given Vulkan device: %d.\n", res); + return AVERROR(ENOSYS); + } + av_log(ctx, AV_LOG_VERBOSE, "AMF initialisation succeeded via Vulkan.\n"); + } + } + return 0; +} + +static int amf_load_library(AVAMFDeviceContext* amf_ctx, void* avcl) +{ + AMFInit_Fn init_fun; + AMFQueryVersion_Fn version_fun; + AMF_RESULT res; + + amf_ctx->library = dlopen(AMF_DLL_NAMEA, RTLD_NOW | RTLD_LOCAL); + AMF_RETURN_IF_FALSE(avcl, amf_ctx->library != NULL, + AVERROR_UNKNOWN, "DLL %s failed to open\n", AMF_DLL_NAMEA); + + init_fun = (AMFInit_Fn)dlsym(amf_ctx->library, AMF_INIT_FUNCTION_NAME); + AMF_RETURN_IF_FALSE(avcl, init_fun != NULL, AVERROR_UNKNOWN, "DLL %s failed to find function %s\n", AMF_DLL_NAMEA, AMF_INIT_FUNCTION_NAME); + + version_fun = (AMFQueryVersion_Fn)dlsym(amf_ctx->library, AMF_QUERY_VERSION_FUNCTION_NAME); + AMF_RETURN_IF_FALSE(avcl, version_fun != NULL, AVERROR_UNKNOWN, "DLL %s failed to find function %s\n", AMF_DLL_NAMEA, AMF_QUERY_VERSION_FUNCTION_NAME); + + res = version_fun(&amf_ctx->version); + AMF_RETURN_IF_FALSE(avcl, res == AMF_OK, AVERROR_UNKNOWN, "%s failed with error %d\n", AMF_QUERY_VERSION_FUNCTION_NAME, res); + res = init_fun(AMF_FULL_VERSION, &amf_ctx->factory); + AMF_RETURN_IF_FALSE(avcl, res == AMF_OK, AVERROR_UNKNOWN, "%s failed with error %d\n", AMF_INIT_FUNCTION_NAME, res); + return 0; +} + +static int amf_device_create(AVHWDeviceContext *device_ctx, + const char *device, + AVDictionary *opts, int flags) +{ + AVAMFDeviceContext *ctx = device_ctx->hwctx; + AMFTrace *trace; + int ret; + if ((ret = amf_load_library(ctx, device_ctx)) == 0) { + ret = ctx->factory->pVtbl->GetTrace(ctx->factory, &trace); + if (ret == AMF_OK) { + trace->pVtbl->EnableWriter(trace, AMF_TRACE_WRITER_CONSOLE, 0); + trace->pVtbl->SetGlobalLevel(trace, AMF_TRACE_TRACE); + + // connect AMF logger to av_log + ctx->trace_writer = amf_writer_alloc(device_ctx); + trace->pVtbl->RegisterWriter(trace, FFMPEG_AMF_WRITER_ID, (AMFTraceWriter*)ctx->trace_writer, 1); + trace->pVtbl->SetWriterLevel(trace, FFMPEG_AMF_WRITER_ID, AMF_TRACE_TRACE); + } + + + ret = ctx->factory->pVtbl->CreateContext(ctx->factory, &ctx->context); + if (ret == AMF_OK) + return 0; + av_log(device_ctx, AV_LOG_ERROR, "CreateContext() failed with error %d.\n", ret); + } + amf_device_uninit(device_ctx); + return ret; +} + +#if CONFIG_DXVA2 +static int amf_init_from_dxva2_device(AVAMFDeviceContext * amf_ctx, AVDXVA2DeviceContext *hwctx) +{ + IDirect3DDevice9 *device; + HANDLE device_handle; + HRESULT hr; + AMF_RESULT res; + int ret; + + hr = IDirect3DDeviceManager9_OpenDeviceHandle(hwctx->devmgr, &device_handle); + if (FAILED(hr)) { + av_log(hwctx, AV_LOG_ERROR, "Failed to open device handle for Direct3D9 device: %lx.\n", (unsigned long)hr); + return AVERROR_EXTERNAL; + } + + hr = IDirect3DDeviceManager9_LockDevice(hwctx->devmgr, device_handle, &device, FALSE); + if (SUCCEEDED(hr)) { + IDirect3DDeviceManager9_UnlockDevice(hwctx->devmgr, device_handle, FALSE); + ret = 0; + } else { + av_log(hwctx, AV_LOG_ERROR, "Failed to lock device handle for Direct3D9 device: %lx.\n", (unsigned long)hr); + ret = AVERROR_EXTERNAL; + } + + + IDirect3DDeviceManager9_CloseDeviceHandle(hwctx->devmgr, device_handle); + + if (ret < 0) + return ret; + + res = amf_ctx->context->pVtbl->InitDX9(amf_ctx->context, device); + + IDirect3DDevice9_Release(device); + + if (res != AMF_OK) { + if (res == AMF_NOT_SUPPORTED) + av_log(hwctx, AV_LOG_ERROR, "AMF via D3D9 is not supported on the given device.\n"); + else + av_log(hwctx, AV_LOG_ERROR, "AMF failed to initialise on given D3D9 device: %d.\n", res); + return AVERROR(ENODEV); + } + return 0; +} +#endif + +#if CONFIG_D3D11VA +static int amf_init_from_d3d11_device(AVAMFDeviceContext* amf_ctx, AVD3D11VADeviceContext *hwctx) +{ + AMF_RESULT res; + res = amf_ctx->context->pVtbl->InitDX11(amf_ctx->context, hwctx->device, AMF_DX11_1); + if (res != AMF_OK) { + if (res == AMF_NOT_SUPPORTED) + av_log(hwctx, AV_LOG_ERROR, "AMF via D3D11 is not supported on the given device.\n"); + else + av_log(hwctx, AV_LOG_ERROR, "AMF failed to initialise on the given D3D11 device: %d.\n", res); + return AVERROR(ENODEV); + } + return 0; +} +#endif + +static int amf_device_derive(AVHWDeviceContext *device_ctx, + AVHWDeviceContext *child_device_ctx, AVDictionary *opts, + int flags) +{ + AVAMFDeviceContext *amf_ctx = device_ctx->hwctx; + int ret; + + ret = amf_device_create(device_ctx, "", opts, flags); + if(ret < 0) + return ret; + + switch (child_device_ctx->type) { + +#if CONFIG_DXVA2 + case AV_HWDEVICE_TYPE_DXVA2: { + AVDXVA2DeviceContext *child_device_hwctx = child_device_ctx->hwctx; + return amf_init_from_dxva2_device(amf_ctx, child_device_hwctx); + } + break; +#endif + +#if CONFIG_D3D11VA + case AV_HWDEVICE_TYPE_D3D11VA: { + AVD3D11VADeviceContext *child_device_hwctx = child_device_ctx->hwctx; + return amf_init_from_d3d11_device(amf_ctx, child_device_hwctx); + } + break; +#endif + default: { + av_log(child_device_ctx, AV_LOG_ERROR, "AMF initialisation from a %s device is not supported.\n", + av_hwdevice_get_type_name(child_device_ctx->type)); + return AVERROR(ENOSYS); + } + } + return 0; +} + +const HWContextType ff_hwcontext_type_amf = { + .type = AV_HWDEVICE_TYPE_AMF, + .name = "AMF", + + .device_hwctx_size = sizeof(AVAMFDeviceContext), + .frames_hwctx_size = sizeof(AMFFramesContext), + + .device_create = amf_device_create, + .device_derive = amf_device_derive, + .device_init = amf_device_init, + .device_uninit = amf_device_uninit, + .frames_get_constraints = amf_frames_get_constraints, + .frames_init = amf_frames_init, + .frames_get_buffer = amf_get_buffer, + .transfer_get_formats = amf_transfer_get_formats, + .transfer_data_to = amf_transfer_data_to, + .transfer_data_from = amf_transfer_data_from, + + .pix_fmts = (const enum AVPixelFormat[]){ AV_PIX_FMT_AMF_SURFACE, AV_PIX_FMT_NONE }, +}; diff --git a/libavutil/hwcontext_amf.h b/libavutil/hwcontext_amf.h new file mode 100644 index 0000000000..5ad2237f75 --- /dev/null +++ b/libavutil/hwcontext_amf.h @@ -0,0 +1,45 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + + +#ifndef AVUTIL_HWCONTEXT_AMF_H +#define AVUTIL_HWCONTEXT_AMF_H + +#include "pixfmt.h" +#include "hwcontext.h" +#include +#include +#include +#include + +/** + * This struct is allocated as AVHWDeviceContext.hwctx + */ +typedef struct AVAMFDeviceContext { + void * library; + AMFFactory *factory; + void *trace_writer; + + int64_t version; ///< version of AMF runtime + AMFContext *context; +} AVAMFDeviceContext; + +enum AMF_SURFACE_FORMAT av_av_to_amf_format(enum AVPixelFormat fmt); +enum AVPixelFormat av_amf_to_av_format(enum AMF_SURFACE_FORMAT fmt); + +#endif /* AVUTIL_HWCONTEXT_AMF_H */ diff --git a/libavutil/hwcontext_amf_internal.h b/libavutil/hwcontext_amf_internal.h new file mode 100644 index 0000000000..b991f357a6 --- /dev/null +++ b/libavutil/hwcontext_amf_internal.h @@ -0,0 +1,44 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + + +#ifndef AVUTIL_HWCONTEXT_AMF_INTERNAL_H +#define AVUTIL_HWCONTEXT_AMF_INTERNAL_H +#include +#include + +/** +* Error handling helper +*/ +#define AMF_RETURN_IF_FALSE(avctx, exp, ret_value, /*message,*/ ...) \ + if (!(exp)) { \ + av_log(avctx, AV_LOG_ERROR, __VA_ARGS__); \ + return ret_value; \ + } + +#define AMF_GOTO_FAIL_IF_FALSE(avctx, exp, ret_value, /*message,*/ ...) \ + if (!(exp)) { \ + av_log(avctx, AV_LOG_ERROR, __VA_ARGS__); \ + ret = ret_value; \ + goto fail; \ + } + +#define AMF_TIME_BASE_Q (AVRational){1, AMF_SECOND} + + +#endif /* AVUTIL_HWCONTEXT_AMF_INTERNAL_H */ \ No newline at end of file diff --git a/libavutil/hwcontext_internal.h b/libavutil/hwcontext_internal.h index e32b786238..db23579c9e 100644 --- a/libavutil/hwcontext_internal.h +++ b/libavutil/hwcontext_internal.h @@ -163,5 +163,6 @@ extern const HWContextType ff_hwcontext_type_vdpau; extern const HWContextType ff_hwcontext_type_videotoolbox; extern const HWContextType ff_hwcontext_type_mediacodec; extern const HWContextType ff_hwcontext_type_vulkan; +extern const HWContextType ff_hwcontext_type_amf; #endif /* AVUTIL_HWCONTEXT_INTERNAL_H */ diff --git a/libavutil/pixdesc.c b/libavutil/pixdesc.c index e15105e9f4..df781c73d0 100644 --- a/libavutil/pixdesc.c +++ b/libavutil/pixdesc.c @@ -2133,6 +2133,10 @@ static const AVPixFmtDescriptor av_pix_fmt_descriptors[AV_PIX_FMT_NB] = { .name = "cuda", .flags = AV_PIX_FMT_FLAG_HWACCEL, }, + [AV_PIX_FMT_AMF_SURFACE] = { + .name = "amf", + .flags = AV_PIX_FMT_FLAG_HWACCEL, + }, [AV_PIX_FMT_VYU444] = { .name = "vyu444", .nb_components = 3, diff --git a/libavutil/pixfmt.h b/libavutil/pixfmt.h index 0dc4abc972..4cababfc3f 100644 --- a/libavutil/pixfmt.h +++ b/libavutil/pixfmt.h @@ -457,6 +457,11 @@ enum AVPixelFormat { AV_PIX_FMT_RGB96BE, ///< packed RGBA 32:32:32, 96bpp, RGBRGB..., big-endian AV_PIX_FMT_RGB96LE, ///< packed RGBA 32:32:32, 96bpp, RGBRGB..., little-endian + /** + * HW acceleration through AMF. data[0] contain AMFSurface pointer + */ + AV_PIX_FMT_AMF_SURFACE, + AV_PIX_FMT_NB ///< number of pixel formats, DO NOT USE THIS if you want to link with shared libav* because the number of formats might differ between versions };