From patchwork Fri Mar 17 12:42:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 40709 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:d046:b0:cd:afd7:272c with SMTP id hv6csp526859pzb; Fri, 17 Mar 2023 05:43:00 -0700 (PDT) X-Google-Smtp-Source: AK7set+6OJVqi2Zx9FuNc6XOi5f8+yxsVwh2t3aBuTea93VJd6nhqzxwm9LOfLvX9bUKmbVJGe5x X-Received: by 2002:a17:906:99cd:b0:930:3907:5238 with SMTP id s13-20020a17090699cd00b0093039075238mr6195425ejn.26.1679056980350; Fri, 17 Mar 2023 05:43:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1679056980; cv=none; d=google.com; s=arc-20160816; b=mr0+FlVoWF2b9eqfZZAjXBBMjtRwTJkizoO94xy8RllFhqyVwao66eMNqWOvHZLzgc YqMDVJp07AKcYzVk1srlE3DxhPfMiFHBpIcJOwThF5UnrivK91co3T3N7eD60B1BGfG2 fydoqhLVo2/vn5b/Mt1PXscbguJ1Ol0cCAzaP5KthVsYOO3UrhTanZgxc0vIWkVoM0bk qfqL757Be9MvEU4jBqfeBFwyRFoCIRsaLUT+JsrlzlahmQNepWcmn0nVWH+M6WXCSFNL Kb9pvq09VXcV7a2G103JoZZQXyVGlmlsWpRx81qJVkUqptTASsGREmysKzEty3LiZODF 8paA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :delivered-to; bh=bbyL3ZLeNDlp3MLxbfJTudMPQ/EhpE3URz058g3eL7A=; b=cjqSpz3cPM+Qpa95t731RWpsTPSVnPEEINIWYn2qCIQMyby/ngr/5KP2rJLtYp4FZu +eY4tWLJB2CfneojgW2C7myE2fzmU3ZZZ66K/PbT14N59aeeSyUO/8Sm7KNGkXLVcLM5 W+GfTOyOHEfxaN0cTH11s8gzqKWtq/6SI9BBgDeKZGSE/YG1lfQipGRPphfeRUYoWZxz +Fe+4/SCW70wcQq3LWwHTYWLOqyIPAzEo9uFxhc5HUR2342nAOVQ821I2rRE3V9DEi9g fLJYWyWNMYwtUiR2jo//p+QpeDGVd2t0xU70NmUoHA2pP13haQ67RG0jaHcPVUMCSoYA o45A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id uz14-20020a170907118e00b0092b93aded0csi2260756ejb.149.2023.03.17.05.42.59; Fri, 17 Mar 2023 05:43:00 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 69D6068C1D5; Fri, 17 Mar 2023 14:42:47 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail0.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1606168C1BF for ; Fri, 17 Mar 2023 14:42:40 +0200 (EET) Received: from localhost (localhost [IPv6:::1]) by mail0.khirnov.net (Postfix) with ESMTP id D081B240178 for ; Fri, 17 Mar 2023 13:42:39 +0100 (CET) Received: from mail0.khirnov.net ([IPv6:::1]) by localhost (mail0.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id Y2wIB0odD5wn for ; Fri, 17 Mar 2023 13:42:38 +0100 (CET) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail0.khirnov.net (Postfix) with ESMTPS id 7975D2404EA for ; Fri, 17 Mar 2023 13:42:38 +0100 (CET) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:::1]) by libav.khirnov.net (Postfix) with ESMTP id 2771B3A00B6 for ; Fri, 17 Mar 2023 13:42:32 +0100 (CET) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Fri, 17 Mar 2023 13:42:29 +0100 Message-Id: <20230317124230.8835-1-anton@khirnov.net> X-Mailer: git-send-email 2.39.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/2] lavfi: add a flag for filters able to work with hw_device_ctx X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: zbX1Mznkckmt This way the caller can set it just on the filters that can make use of it. --- doc/APIchanges | 3 +++ libavfilter/avfilter.h | 9 +++++++++ libavfilter/stack_internal.c | 4 ++-- libavfilter/version.h | 2 +- libavfilter/vf_avgblur_opencl.c | 1 + libavfilter/vf_avgblur_vulkan.c | 1 + libavfilter/vf_blend_vulkan.c | 1 + libavfilter/vf_chromaber_vulkan.c | 1 + libavfilter/vf_colorkey_opencl.c | 3 ++- libavfilter/vf_convolution_opencl.c | 4 ++++ libavfilter/vf_deshake_opencl.c | 3 ++- libavfilter/vf_flip_vulkan.c | 1 + libavfilter/vf_gblur_vulkan.c | 1 + libavfilter/vf_libplacebo.c | 1 + libavfilter/vf_neighbor_opencl.c | 1 + libavfilter/vf_nlmeans_opencl.c | 1 + libavfilter/vf_overlay_opencl.c | 1 + libavfilter/vf_overlay_qsv.c | 1 + libavfilter/vf_overlay_vulkan.c | 1 + libavfilter/vf_pad_opencl.c | 3 ++- libavfilter/vf_program_opencl.c | 4 +++- libavfilter/vf_remap_opencl.c | 1 + libavfilter/vf_scale_vulkan.c | 1 + libavfilter/vf_stack_qsv.c | 6 +++--- libavfilter/vf_stack_vaapi.c | 6 +++--- libavfilter/vf_tonemap_opencl.c | 1 + libavfilter/vf_transpose_opencl.c | 1 + libavfilter/vf_transpose_vulkan.c | 1 + libavfilter/vf_unsharp_opencl.c | 1 + libavfilter/vf_vpp_qsv.c | 1 + libavfilter/vf_xfade_opencl.c | 1 + libavfilter/vsrc_ddagrab.c | 1 + 32 files changed, 55 insertions(+), 13 deletions(-) diff --git a/doc/APIchanges b/doc/APIchanges index 14737223cb0..eeb2f1c49c4 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -2,6 +2,9 @@ The last version increases of all libraries were on 2023-02-09 API changes, most recent first: +2023-03-xx - xxxxxxxxxx - lavfi 9.5.100 - avfilter.h + Add AVFILTER_FLAG_HWDEVICE. + 2023-03-02 - xxxxxxxxxx - lavc 60.6.100 - avcodec.h Add FF_PROFILE_EAC3_DDP_ATMOS, FF_PROFILE_TRUEHD_ATMOS, FF_PROFILE_DTS_HD_MA_X and FF_PROFILE_DTS_HD_MA_X_IMAX. diff --git a/libavfilter/avfilter.h b/libavfilter/avfilter.h index d0d45844681..d69381aed4f 100644 --- a/libavfilter/avfilter.h +++ b/libavfilter/avfilter.h @@ -131,6 +131,11 @@ enum AVMediaType avfilter_pad_get_type(const AVFilterPad *pads, int pad_idx); * received by the filter on one of its inputs. */ #define AVFILTER_FLAG_METADATA_ONLY (1 << 3) + +/** + * The filter can create hardware frames using AVFilterContext.hw_device_ctx. + */ +#define AVFILTER_FLAG_HWDEVICE (1 << 4) /** * Some filters support a generic "enable" expression option that can be used * to enable or disable a filter in the timeline. Filters supporting this @@ -444,6 +449,10 @@ struct AVFilterContext { * in particular, a filter which consumes or processes hardware frames will * instead use the hw_frames_ctx field in AVFilterLink to carry the * hardware context information. + * + * May be set by the caller on filters flagged with AVFILTER_FLAG_HWDEVICE + * before initializing the filter with avfilter_init_str() or + * avfilter_init_dict(). */ AVBufferRef *hw_device_ctx; diff --git a/libavfilter/stack_internal.c b/libavfilter/stack_internal.c index 0697ebdd12c..1ee20d66cf2 100644 --- a/libavfilter/stack_internal.c +++ b/libavfilter/stack_internal.c @@ -333,7 +333,7 @@ static const AVFilterPad stack_outputs[] = { { NULL } \ } -#define DEFINE_STACK_FILTER(category, api, capi) \ +#define DEFINE_STACK_FILTER(category, api, capi, filter_flags) \ static const AVClass category##_##api##_class = { \ .class_name = #category "_" #api, \ .item_name = av_default_item_name, \ @@ -351,5 +351,5 @@ static const AVFilterPad stack_outputs[] = { FILTER_QUERY_FUNC(api##_stack_query_formats), \ FILTER_OUTPUTS(stack_outputs), \ .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, \ - .flags = AVFILTER_FLAG_DYNAMIC_INPUTS, \ + .flags = AVFILTER_FLAG_DYNAMIC_INPUTS | filter_flags, \ } diff --git a/libavfilter/version.h b/libavfilter/version.h index 4d8f28e5127..523a7fe0a6e 100644 --- a/libavfilter/version.h +++ b/libavfilter/version.h @@ -31,7 +31,7 @@ #include "version_major.h" -#define LIBAVFILTER_VERSION_MINOR 4 +#define LIBAVFILTER_VERSION_MINOR 5 #define LIBAVFILTER_VERSION_MICRO 100 diff --git a/libavfilter/vf_avgblur_opencl.c b/libavfilter/vf_avgblur_opencl.c index 7c0578b694b..68f3a632492 100644 --- a/libavfilter/vf_avgblur_opencl.c +++ b/libavfilter/vf_avgblur_opencl.c @@ -390,6 +390,7 @@ const AVFilter ff_vf_boxblur_opencl = { FILTER_OUTPUTS(avgblur_opencl_outputs), FILTER_SINGLE_PIXFMT(AV_PIX_FMT_OPENCL), .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + .flags = AVFILTER_FLAG_HWDEVICE, }; #endif /* CONFIG_BOXBLUR_OPENCL_FILTER */ diff --git a/libavfilter/vf_avgblur_vulkan.c b/libavfilter/vf_avgblur_vulkan.c index d118ce802ca..6a54d158ce1 100644 --- a/libavfilter/vf_avgblur_vulkan.c +++ b/libavfilter/vf_avgblur_vulkan.c @@ -418,4 +418,5 @@ const AVFilter ff_vf_avgblur_vulkan = { FILTER_SINGLE_PIXFMT(AV_PIX_FMT_VULKAN), .priv_class = &avgblur_vulkan_class, .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + .flags = AVFILTER_FLAG_HWDEVICE, }; diff --git a/libavfilter/vf_blend_vulkan.c b/libavfilter/vf_blend_vulkan.c index fcc21cbc8d5..4cee688a22f 100644 --- a/libavfilter/vf_blend_vulkan.c +++ b/libavfilter/vf_blend_vulkan.c @@ -500,5 +500,6 @@ const AVFilter ff_vf_blend_vulkan = { FILTER_SINGLE_PIXFMT(AV_PIX_FMT_VULKAN), .priv_class = &blend_vulkan_class, .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + .flags = AVFILTER_FLAG_HWDEVICE, .process_command = &process_command, }; diff --git a/libavfilter/vf_chromaber_vulkan.c b/libavfilter/vf_chromaber_vulkan.c index b9423e417e3..62b99cc4d91 100644 --- a/libavfilter/vf_chromaber_vulkan.c +++ b/libavfilter/vf_chromaber_vulkan.c @@ -349,4 +349,5 @@ const AVFilter ff_vf_chromaber_vulkan = { FILTER_SINGLE_PIXFMT(AV_PIX_FMT_VULKAN), .priv_class = &chromaber_vulkan_class, .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + .flags = AVFILTER_FLAG_HWDEVICE, }; diff --git a/libavfilter/vf_colorkey_opencl.c b/libavfilter/vf_colorkey_opencl.c index 84be1999a89..2b019b290c1 100644 --- a/libavfilter/vf_colorkey_opencl.c +++ b/libavfilter/vf_colorkey_opencl.c @@ -238,5 +238,6 @@ const AVFilter ff_vf_colorkey_opencl = { FILTER_INPUTS(colorkey_opencl_inputs), FILTER_OUTPUTS(colorkey_opencl_outputs), FILTER_SINGLE_PIXFMT(AV_PIX_FMT_OPENCL), - .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE + .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + .flags = AVFILTER_FLAG_HWDEVICE, }; diff --git a/libavfilter/vf_convolution_opencl.c b/libavfilter/vf_convolution_opencl.c index de3d38b553b..bf721a7416b 100644 --- a/libavfilter/vf_convolution_opencl.c +++ b/libavfilter/vf_convolution_opencl.c @@ -373,6 +373,7 @@ const AVFilter ff_vf_convolution_opencl = { FILTER_OUTPUTS(convolution_opencl_outputs), FILTER_SINGLE_PIXFMT(AV_PIX_FMT_OPENCL), .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + .flags = AVFILTER_FLAG_HWDEVICE, }; #endif /* CONFIG_CONVOLUTION_OPENCL_FILTER */ @@ -399,6 +400,7 @@ const AVFilter ff_vf_sobel_opencl = { FILTER_OUTPUTS(convolution_opencl_outputs), FILTER_SINGLE_PIXFMT(AV_PIX_FMT_OPENCL), .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + .flags = AVFILTER_FLAG_HWDEVICE, }; #endif /* CONFIG_SOBEL_OPENCL_FILTER */ @@ -425,6 +427,7 @@ const AVFilter ff_vf_prewitt_opencl = { FILTER_OUTPUTS(convolution_opencl_outputs), FILTER_SINGLE_PIXFMT(AV_PIX_FMT_OPENCL), .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + .flags = AVFILTER_FLAG_HWDEVICE, }; #endif /* CONFIG_PREWITT_OPENCL_FILTER */ @@ -451,6 +454,7 @@ const AVFilter ff_vf_roberts_opencl = { FILTER_OUTPUTS(convolution_opencl_outputs), FILTER_SINGLE_PIXFMT(AV_PIX_FMT_OPENCL), .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + .flags = AVFILTER_FLAG_HWDEVICE, }; #endif /* CONFIG_ROBERTS_OPENCL_FILTER */ diff --git a/libavfilter/vf_deshake_opencl.c b/libavfilter/vf_deshake_opencl.c index d488da7fa0c..e670a4cc234 100644 --- a/libavfilter/vf_deshake_opencl.c +++ b/libavfilter/vf_deshake_opencl.c @@ -2169,5 +2169,6 @@ const AVFilter ff_vf_deshake_opencl = { FILTER_INPUTS(deshake_opencl_inputs), FILTER_OUTPUTS(deshake_opencl_outputs), FILTER_SINGLE_PIXFMT(AV_PIX_FMT_OPENCL), - .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE + .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + .flags = AVFILTER_FLAG_HWDEVICE, }; diff --git a/libavfilter/vf_flip_vulkan.c b/libavfilter/vf_flip_vulkan.c index 0223786ef1f..6868e39ee63 100644 --- a/libavfilter/vf_flip_vulkan.c +++ b/libavfilter/vf_flip_vulkan.c @@ -366,4 +366,5 @@ const AVFilter ff_vf_flip_vulkan = { FILTER_SINGLE_PIXFMT(AV_PIX_FMT_VULKAN), .priv_class = &flip_vulkan_class, .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + .flags = AVFILTER_FLAG_HWDEVICE, }; diff --git a/libavfilter/vf_gblur_vulkan.c b/libavfilter/vf_gblur_vulkan.c index d61f3c778c0..e6ffc8c0735 100644 --- a/libavfilter/vf_gblur_vulkan.c +++ b/libavfilter/vf_gblur_vulkan.c @@ -503,4 +503,5 @@ const AVFilter ff_vf_gblur_vulkan = { FILTER_SINGLE_PIXFMT(AV_PIX_FMT_VULKAN), .priv_class = &gblur_vulkan_class, .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + .flags = AVFILTER_FLAG_HWDEVICE, }; diff --git a/libavfilter/vf_libplacebo.c b/libavfilter/vf_libplacebo.c index 757dc519e9a..ba852de08d6 100644 --- a/libavfilter/vf_libplacebo.c +++ b/libavfilter/vf_libplacebo.c @@ -893,4 +893,5 @@ const AVFilter ff_vf_libplacebo = { FILTER_QUERY_FUNC(libplacebo_query_format), .priv_class = &libplacebo_class, .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + .flags = AVFILTER_FLAG_HWDEVICE, }; diff --git a/libavfilter/vf_neighbor_opencl.c b/libavfilter/vf_neighbor_opencl.c index 3b83cee3da6..d2d93cd2405 100644 --- a/libavfilter/vf_neighbor_opencl.c +++ b/libavfilter/vf_neighbor_opencl.c @@ -312,6 +312,7 @@ const AVFilter ff_vf_dilation_opencl = { FILTER_OUTPUTS(neighbor_opencl_outputs), FILTER_SINGLE_PIXFMT(AV_PIX_FMT_OPENCL), .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + .flags = AVFILTER_FLAG_HWDEVICE, }; #endif /* CONFIG_DILATION_OPENCL_FILTER */ diff --git a/libavfilter/vf_nlmeans_opencl.c b/libavfilter/vf_nlmeans_opencl.c index c0ac8bf95dd..ca3ec45d7a3 100644 --- a/libavfilter/vf_nlmeans_opencl.c +++ b/libavfilter/vf_nlmeans_opencl.c @@ -438,4 +438,5 @@ const AVFilter ff_vf_nlmeans_opencl = { FILTER_OUTPUTS(nlmeans_opencl_outputs), FILTER_SINGLE_PIXFMT(AV_PIX_FMT_OPENCL), .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + .flags = AVFILTER_FLAG_HWDEVICE, }; diff --git a/libavfilter/vf_overlay_opencl.c b/libavfilter/vf_overlay_opencl.c index 1e3ad903e11..38a3fc8795a 100644 --- a/libavfilter/vf_overlay_opencl.c +++ b/libavfilter/vf_overlay_opencl.c @@ -322,4 +322,5 @@ const AVFilter ff_vf_overlay_opencl = { FILTER_OUTPUTS(overlay_opencl_outputs), FILTER_SINGLE_PIXFMT(AV_PIX_FMT_OPENCL), .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + .flags = AVFILTER_FLAG_HWDEVICE, }; diff --git a/libavfilter/vf_overlay_qsv.c b/libavfilter/vf_overlay_qsv.c index 5bec7dd414c..edc7f79c865 100644 --- a/libavfilter/vf_overlay_qsv.c +++ b/libavfilter/vf_overlay_qsv.c @@ -429,4 +429,5 @@ const AVFilter ff_vf_overlay_qsv = { FILTER_QUERY_FUNC(overlay_qsv_query_formats), .priv_class = &overlay_qsv_class, .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + .flags = AVFILTER_FLAG_HWDEVICE, }; diff --git a/libavfilter/vf_overlay_vulkan.c b/libavfilter/vf_overlay_vulkan.c index e87ee830007..6db7baddfdd 100644 --- a/libavfilter/vf_overlay_vulkan.c +++ b/libavfilter/vf_overlay_vulkan.c @@ -494,4 +494,5 @@ const AVFilter ff_vf_overlay_vulkan = { FILTER_SINGLE_PIXFMT(AV_PIX_FMT_VULKAN), .priv_class = &overlay_vulkan_class, .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + .flags = AVFILTER_FLAG_HWDEVICE, }; diff --git a/libavfilter/vf_pad_opencl.c b/libavfilter/vf_pad_opencl.c index 728e850f720..d6b71765eeb 100644 --- a/libavfilter/vf_pad_opencl.c +++ b/libavfilter/vf_pad_opencl.c @@ -391,5 +391,6 @@ const AVFilter ff_vf_pad_opencl = { FILTER_INPUTS(pad_opencl_inputs), FILTER_OUTPUTS(pad_opencl_outputs), FILTER_SINGLE_PIXFMT(AV_PIX_FMT_OPENCL), - .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE + .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + .flags = AVFILTER_FLAG_HWDEVICE, }; diff --git a/libavfilter/vf_program_opencl.c b/libavfilter/vf_program_opencl.c index 55c1e6547c4..8a4881b38e4 100644 --- a/libavfilter/vf_program_opencl.c +++ b/libavfilter/vf_program_opencl.c @@ -362,7 +362,8 @@ const AVFilter ff_vf_program_opencl = { .description = NULL_IF_CONFIG_SMALL("Filter video using an OpenCL program"), .priv_size = sizeof(ProgramOpenCLContext), .priv_class = &program_opencl_class, - .flags = AVFILTER_FLAG_DYNAMIC_INPUTS, + .flags = AVFILTER_FLAG_DYNAMIC_INPUTS | + AVFILTER_FLAG_HWDEVICE, .preinit = &program_opencl_framesync_preinit, .init = &program_opencl_init, .uninit = &program_opencl_uninit, @@ -421,6 +422,7 @@ const AVFilter ff_vsrc_openclsrc = { FILTER_OUTPUTS(openclsrc_outputs), FILTER_SINGLE_PIXFMT(AV_PIX_FMT_OPENCL), .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + .flags = AVFILTER_FLAG_HWDEVICE, }; #endif diff --git a/libavfilter/vf_remap_opencl.c b/libavfilter/vf_remap_opencl.c index f3f84bde64a..eeb1eb5d69d 100644 --- a/libavfilter/vf_remap_opencl.c +++ b/libavfilter/vf_remap_opencl.c @@ -351,4 +351,5 @@ const AVFilter ff_vf_remap_opencl = { FILTER_SINGLE_PIXFMT(AV_PIX_FMT_OPENCL), .priv_class = &remap_opencl_class, .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + .flags = AVFILTER_FLAG_HWDEVICE, }; diff --git a/libavfilter/vf_scale_vulkan.c b/libavfilter/vf_scale_vulkan.c index c1404208962..3b09f0dcc12 100644 --- a/libavfilter/vf_scale_vulkan.c +++ b/libavfilter/vf_scale_vulkan.c @@ -531,4 +531,5 @@ const AVFilter ff_vf_scale_vulkan = { FILTER_SINGLE_PIXFMT(AV_PIX_FMT_VULKAN), .priv_class = &scale_vulkan_class, .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + .flags = AVFILTER_FLAG_HWDEVICE, }; diff --git a/libavfilter/vf_stack_qsv.c b/libavfilter/vf_stack_qsv.c index 9eb0748bd61..3e6aefe44bf 100644 --- a/libavfilter/vf_stack_qsv.c +++ b/libavfilter/vf_stack_qsv.c @@ -235,20 +235,20 @@ static int qsv_stack_query_formats(AVFilterContext *ctx) #if CONFIG_HSTACK_QSV_FILTER DEFINE_HSTACK_OPTIONS(qsv); -DEFINE_STACK_FILTER(hstack, qsv, "Quick Sync Video"); +DEFINE_STACK_FILTER(hstack, qsv, "Quick Sync Video", AVFILTER_FLAG_HWDEVICE); #endif #if CONFIG_VSTACK_QSV_FILTER DEFINE_VSTACK_OPTIONS(qsv); -DEFINE_STACK_FILTER(vstack, qsv, "Quick Sync Video"); +DEFINE_STACK_FILTER(vstack, qsv, "Quick Sync Video", AVFILTER_FLAG_HWDEVICE); #endif #if CONFIG_XSTACK_QSV_FILTER DEFINE_XSTACK_OPTIONS(qsv); -DEFINE_STACK_FILTER(xstack, qsv, "Quick Sync Video"); +DEFINE_STACK_FILTER(xstack, qsv, "Quick Sync Video", AVFILTER_FLAG_HWDEVICE); #endif diff --git a/libavfilter/vf_stack_vaapi.c b/libavfilter/vf_stack_vaapi.c index 26dbe3f7aa7..8e9471e6d7b 100644 --- a/libavfilter/vf_stack_vaapi.c +++ b/libavfilter/vf_stack_vaapi.c @@ -234,20 +234,20 @@ static int vaapi_stack_query_formats(AVFilterContext *avctx) #if CONFIG_HSTACK_VAAPI_FILTER DEFINE_HSTACK_OPTIONS(vaapi); -DEFINE_STACK_FILTER(hstack, vaapi, "VA-API"); +DEFINE_STACK_FILTER(hstack, vaapi, "VA-API", 0); #endif #if CONFIG_VSTACK_VAAPI_FILTER DEFINE_VSTACK_OPTIONS(vaapi); -DEFINE_STACK_FILTER(vstack, vaapi, "VA-API"); +DEFINE_STACK_FILTER(vstack, vaapi, "VA-API", 0); #endif #if CONFIG_XSTACK_VAAPI_FILTER DEFINE_XSTACK_OPTIONS(vaapi); -DEFINE_STACK_FILTER(xstack, vaapi, "VA-API"); +DEFINE_STACK_FILTER(xstack, vaapi, "VA-API", 0); #endif diff --git a/libavfilter/vf_tonemap_opencl.c b/libavfilter/vf_tonemap_opencl.c index f6ebb694a81..883eb043427 100644 --- a/libavfilter/vf_tonemap_opencl.c +++ b/libavfilter/vf_tonemap_opencl.c @@ -547,4 +547,5 @@ const AVFilter ff_vf_tonemap_opencl = { FILTER_OUTPUTS(tonemap_opencl_outputs), FILTER_SINGLE_PIXFMT(AV_PIX_FMT_OPENCL), .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + .flags = AVFILTER_FLAG_HWDEVICE, }; diff --git a/libavfilter/vf_transpose_opencl.c b/libavfilter/vf_transpose_opencl.c index c6e8bd2f531..56d34d193bc 100644 --- a/libavfilter/vf_transpose_opencl.c +++ b/libavfilter/vf_transpose_opencl.c @@ -281,4 +281,5 @@ const AVFilter ff_vf_transpose_opencl = { FILTER_OUTPUTS(transpose_opencl_outputs), FILTER_SINGLE_PIXFMT(AV_PIX_FMT_OPENCL), .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + .flags = AVFILTER_FLAG_HWDEVICE, }; diff --git a/libavfilter/vf_transpose_vulkan.c b/libavfilter/vf_transpose_vulkan.c index 30d052e08cc..3b2ce4fb693 100644 --- a/libavfilter/vf_transpose_vulkan.c +++ b/libavfilter/vf_transpose_vulkan.c @@ -343,4 +343,5 @@ const AVFilter ff_vf_transpose_vulkan = { FILTER_SINGLE_PIXFMT(AV_PIX_FMT_VULKAN), .priv_class = &transpose_vulkan_class, .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + .flags = AVFILTER_FLAG_HWDEVICE, }; diff --git a/libavfilter/vf_unsharp_opencl.c b/libavfilter/vf_unsharp_opencl.c index 53e310de099..2c3ac14050d 100644 --- a/libavfilter/vf_unsharp_opencl.c +++ b/libavfilter/vf_unsharp_opencl.c @@ -407,4 +407,5 @@ const AVFilter ff_vf_unsharp_opencl = { FILTER_OUTPUTS(unsharp_opencl_outputs), FILTER_SINGLE_PIXFMT(AV_PIX_FMT_OPENCL), .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + .flags = AVFILTER_FLAG_HWDEVICE, }; diff --git a/libavfilter/vf_vpp_qsv.c b/libavfilter/vf_vpp_qsv.c index ac77f7bb4ba..814a4322ae7 100644 --- a/libavfilter/vf_vpp_qsv.c +++ b/libavfilter/vf_vpp_qsv.c @@ -648,6 +648,7 @@ const AVFilter ff_vf_##sn##_qsv = { \ fmts, \ .activate = activate, \ .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, \ + .flags = AVFILTER_FLAG_HWDEVICE, \ }; #if CONFIG_VPP_QSV_FILTER diff --git a/libavfilter/vf_xfade_opencl.c b/libavfilter/vf_xfade_opencl.c index cca4980e012..415cf7ac35d 100644 --- a/libavfilter/vf_xfade_opencl.c +++ b/libavfilter/vf_xfade_opencl.c @@ -433,4 +433,5 @@ const AVFilter ff_vf_xfade_opencl = { FILTER_OUTPUTS(xfade_opencl_outputs), FILTER_SINGLE_PIXFMT(AV_PIX_FMT_OPENCL), .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + .flags = AVFILTER_FLAG_HWDEVICE, }; diff --git a/libavfilter/vsrc_ddagrab.c b/libavfilter/vsrc_ddagrab.c index 00c72187eaa..042842104f0 100644 --- a/libavfilter/vsrc_ddagrab.c +++ b/libavfilter/vsrc_ddagrab.c @@ -1056,4 +1056,5 @@ const AVFilter ff_vsrc_ddagrab = { FILTER_OUTPUTS(ddagrab_outputs), FILTER_SINGLE_PIXFMT(AV_PIX_FMT_D3D11), .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + .flags = AVFILTER_FLAG_HWDEVICE, }; From patchwork Fri Mar 17 12:42:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 40708 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:d046:b0:cd:afd7:272c with SMTP id hv6csp526771pzb; Fri, 17 Mar 2023 05:42:49 -0700 (PDT) X-Google-Smtp-Source: AK7set/ITBVrp255C+DSB+pRbyDCksQ7/8fvURiCWhSQrxbb9B7rZrhXax4MyQHbCqdOX+ca9/Cy X-Received: by 2002:a17:906:e093:b0:8b1:7e88:c20f with SMTP id gh19-20020a170906e09300b008b17e88c20fmr16142867ejb.15.1679056969797; Fri, 17 Mar 2023 05:42:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1679056969; cv=none; d=google.com; s=arc-20160816; b=qXviBC3ygwBfa57QJXTpDy4jNfNR4ruk3wdC4JFH6fpjC6NX7dSFEYAB0As60qavWv caVFmXJBOp8XjC1JLv+VfPeaRFjQJMNQaO76ce4oW6bU1idtoP8EFHgIMqQsoQUBi6eX Zf5E5ShcSIkxvfKd+uJdrlGxoPhUv2G7a/wka3NshxONsxbWAI5VzmalQaMSvM3OxE3S mWY8++6UVqPA4kpRvrMKcdQQKBtPQE9dblwbfG8hbGp/6HrfonAstXElJ/+jAGGS8fdC Bg4nJnCA4WN9qdFVT5sHGkJ4n8bUWbhtTRoz3fbGMMHQvIXULXNimrspEskIlycoogc/ IHmw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:delivered-to; bh=qDnNNUJHLttah874WSpa9IP+p/4GYSX2E1O4qkkhS+U=; b=opyh1j3SKvrPHcqrnylk8aV0GCYvrAtgL+k2eKGP+f22LA0av8icLiHeeliHN+QO9K U6DimA9QF8EKMyGOiCLQjHn401QBfBPuRUvWJaP//vBUXiRMD89A+H4fFu07xT1RANMS Pb5+cr+WOIGkoYhm7IkyQZ0QwNvfLbXXPqv/0AMEL8oZ1T2/DJk0usxXpfHpaRsXUZ20 Dhg0dgC9h/FovoF+V0zU40HrFH64gTp+3kofCTFsoqk3alrL4rheGhCxN1y2uPdawXnB y7v5soIWD5BRn9ltDv4zos6LAkys2QOamSv7v2zw6SnBtm/gyHReyJ3ESGb5wSqpq1Ab scvA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a3-20020a1709062b0300b0093105432da9si2147122ejg.151.2023.03.17.05.42.49; Fri, 17 Mar 2023 05:42:49 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4426068C1CD; Fri, 17 Mar 2023 14:42:46 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail0.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id DFD3268C1AD for ; Fri, 17 Mar 2023 14:42:39 +0200 (EET) Received: from localhost (localhost [IPv6:::1]) by mail0.khirnov.net (Postfix) with ESMTP id 915A62404EE for ; Fri, 17 Mar 2023 13:42:39 +0100 (CET) Received: from mail0.khirnov.net ([IPv6:::1]) by localhost (mail0.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id N6BTuE2MQF-i for ; Fri, 17 Mar 2023 13:42:38 +0100 (CET) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail0.khirnov.net (Postfix) with ESMTPS id 777CA240178 for ; Fri, 17 Mar 2023 13:42:38 +0100 (CET) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:::1]) by libav.khirnov.net (Postfix) with ESMTP id 355633A039B for ; Fri, 17 Mar 2023 13:42:32 +0100 (CET) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Fri, 17 Mar 2023 13:42:30 +0100 Message-Id: <20230317124230.8835-2-anton@khirnov.net> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20230317124230.8835-1-anton@khirnov.net> References: <20230317124230.8835-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/2] fftools/ffmpeg: supply hw_device_ctx to filters before initializing them X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: x7t8WkEYvTIZ This is more correct, but was not possible before the recently-added filtergraph parsing API. Also, only pass hw devices to filters that are flagged as capable of using them. --- fftools/ffmpeg.h | 7 ++++++- fftools/ffmpeg_filter.c | 24 ++++++++++++++++++------ fftools/ffmpeg_hw.c | 21 +++++---------------- 3 files changed, 29 insertions(+), 23 deletions(-) diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index 4d4433f5bad..2020770312f 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -802,7 +802,12 @@ void hw_device_free_all(void); int hw_device_setup_for_decode(InputStream *ist); int hw_device_setup_for_encode(OutputStream *ost); -int hw_device_setup_for_filter(FilterGraph *fg); +/** + * Get a hardware device to be used with this filtergraph. + * The returned reference is owned by the callee, the caller + * must ref it explicitly for long-term use. + */ +AVBufferRef *hw_device_for_filter(FilterGraph *fg); int hwaccel_decode_init(AVCodecContext *avctx); diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c index 3504a3cc0ae..8f924a74538 100644 --- a/fftools/ffmpeg_filter.c +++ b/fftools/ffmpeg_filter.c @@ -440,7 +440,8 @@ static int graph_opts_apply(AVFilterGraphSegment *seg) } static int graph_parse(AVFilterGraph *graph, const char *desc, - AVFilterInOut **inputs, AVFilterInOut **outputs) + AVFilterInOut **inputs, AVFilterInOut **outputs, + AVBufferRef *hw_device) { AVFilterGraphSegment *seg; int ret; @@ -453,6 +454,18 @@ static int graph_parse(AVFilterGraph *graph, const char *desc, if (ret < 0) goto fail; + if (hw_device) { + for (int i = 0; i < graph->nb_filters; i++) { + AVFilterContext *f = graph->filters[i]; + + if (!(f->filter->flags & AVFILTER_FLAG_HWDEVICE)) + continue; + f->hw_device_ctx = av_buffer_ref(hw_device); + if (!f->hw_device_ctx) + return AVERROR(ENOMEM); + } + } + ret = graph_opts_apply(seg); if (ret < 0) goto fail; @@ -477,7 +490,7 @@ int init_complex_filtergraph(FilterGraph *fg) return AVERROR(ENOMEM); graph->nb_threads = 1; - ret = graph_parse(graph, fg->graph_desc, &inputs, &outputs); + ret = graph_parse(graph, fg->graph_desc, &inputs, &outputs, NULL); if (ret < 0) goto fail; @@ -1111,6 +1124,7 @@ static int graph_is_meta(AVFilterGraph *graph) int configure_filtergraph(FilterGraph *fg) { + AVBufferRef *hw_device; AVFilterInOut *inputs, *outputs, *cur; int ret, i, simple = filtergraph_is_simple(fg); const char *graph_desc = simple ? fg->outputs[0]->ost->avfilter : @@ -1154,11 +1168,9 @@ int configure_filtergraph(FilterGraph *fg) fg->graph->nb_threads = filter_complex_nbthreads; } - if ((ret = graph_parse(fg->graph, graph_desc, &inputs, &outputs)) < 0) - goto fail; + hw_device = hw_device_for_filter(fg); - ret = hw_device_setup_for_filter(fg); - if (ret < 0) + if ((ret = graph_parse(fg->graph, graph_desc, &inputs, &outputs, hw_device)) < 0) goto fail; if (simple && (!inputs || inputs->next || !outputs || outputs->next)) { diff --git a/fftools/ffmpeg_hw.c b/fftools/ffmpeg_hw.c index 88fa7824701..a3351d98b59 100644 --- a/fftools/ffmpeg_hw.c +++ b/fftools/ffmpeg_hw.c @@ -548,17 +548,14 @@ int hwaccel_decode_init(AVCodecContext *avctx) return 0; } -int hw_device_setup_for_filter(FilterGraph *fg) +AVBufferRef *hw_device_for_filter(FilterGraph *fg) { - HWDevice *dev; - int i; - // Pick the last hardware device if the user doesn't pick the device for // filters explicitly with the filter_hw_device option. if (filter_hw_device) - dev = filter_hw_device; + return filter_hw_device->device_ref; else if (nb_hw_devices > 0) { - dev = hw_devices[nb_hw_devices - 1]; + HWDevice *dev = hw_devices[nb_hw_devices - 1]; if (nb_hw_devices > 1) av_log(NULL, AV_LOG_WARNING, "There are %d hardware devices. device " @@ -567,17 +564,9 @@ int hw_device_setup_for_filter(FilterGraph *fg) "%s is not usable for filters.\n", nb_hw_devices, dev->name, av_hwdevice_get_type_name(dev->type), dev->name); - } else - dev = NULL; - if (dev) { - for (i = 0; i < fg->graph->nb_filters; i++) { - fg->graph->filters[i]->hw_device_ctx = - av_buffer_ref(dev->device_ref); - if (!fg->graph->filters[i]->hw_device_ctx) - return AVERROR(ENOMEM); - } + return dev->device_ref; } - return 0; + return NULL; }