| Message ID | 20230425072620.512-5-tong1.wu@intel.com |
|---|---|
| State | New |
| Headers | show |
| Series | [FFmpeg-devel,v5,1/5] avutil/hwcontext: add a function to get the AVHWDeviceType | expand |
| Context | Check | Description |
|---|---|---|
| yinshiyou/make_loongarch64 | success | Make finished |
| yinshiyou/make_fate_loongarch64 | success | Make fate finished |
| andriy/make_x86 | success | Make finished |
| andriy/make_fate_x86 | success | Make fate finished |
On 25 Apr 2023, at 9:26, Tong Wu wrote: > When two formats lists cannot be merged, a scale filter is > auto-inserted. However, when it comes to hardware map, we have to > manually add a hwmap filter to do the conversion. This patch introduces > an auto hwmap filter to do the hwmap conversion automatically. > Thanks for trying to improve this! I've recently done quite a bit of experimentation with hardware filters and at least for the Cuda - Vulkan - Cuda case, hwmap was useless, and I was told I need to use hwupload instead, so I wonder what cases this would help with? I just fear that, especially given the bad error messages hwmap gives, this will just implicitly insert it because it seemingly works but then just fail to actually do the job and give an absolutely indescriptive error to the user for a filter they did not even insert themselves. > Signed-off-by: Tong Wu <tong1.wu@intel.com> > --- > libavfilter/avfiltergraph.c | 3 ++- > libavfilter/formats.c | 4 ++++ > 2 files changed, 6 insertions(+), 1 deletion(-) > > diff --git a/libavfilter/avfiltergraph.c b/libavfilter/avfiltergraph.c > index 8af0467bc5..b2b627ad6a 100644 > --- a/libavfilter/avfiltergraph.c > +++ b/libavfilter/avfiltergraph.c > @@ -402,7 +402,8 @@ static int insert_auto_filter(AVFilterContext **convert, AVFilterGraph *graph, > AVFilterContext *ctx; > AVFilterLink *inlink, *outlink; > char inst_name[30]; > - const char *opts = FF_FIELD_AT(char *, neg->conversion_filters[conv_step].conversion_opts_offset, *graph); > + const char *opts = neg->conversion_filters[conv_step].conversion_opts_offset == 0 ? NULL : > + FF_FIELD_AT(char *, neg->conversion_filters[conv_step].conversion_opts_offset, *graph); > const char *name = neg->conversion_filters[conv_step].conversion_filter; > > if (!(filter = avfilter_get_by_name(name))) { > diff --git a/libavfilter/formats.c b/libavfilter/formats.c > index c8e20e5b20..fee10fa0ee 100644 > --- a/libavfilter/formats.c > +++ b/libavfilter/formats.c > @@ -331,6 +331,10 @@ static const AVFilterFormatsFilter filters_video[] = { > .conversion_filter = "scale", > .conversion_opts_offset = offsetof(AVFilterGraph, scale_sws_opts), > }, > + { > + .conversion_filter = "hwmap", > + .conversion_opts_offset = 0, > + } > }; > > static const AVFilterFormatsFilter filters_audio[] = { > -- > 2.39.1.windows.1 > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>On 25 Apr 2023, at 9:26, Tong Wu wrote: > >> When two formats lists cannot be merged, a scale filter is >> auto-inserted. However, when it comes to hardware map, we have to >> manually add a hwmap filter to do the conversion. This patch introduces >> an auto hwmap filter to do the hwmap conversion automatically. >> > >Thanks for trying to improve this! > >I've recently done quite a bit of experimentation with hardware >filters and at least for the Cuda - Vulkan - Cuda case, hwmap >was useless, and I was told I need to use hwupload instead, so I wonder >what cases this would help with? Thanks for the reply! Actually we are using hwmap filter to do vaapi<->qsv, d3d11<->qsv conversions. For the use case I have replied in another mail. I'll just copy here again. For example, ffmpeg -init_hw_device d3d11va=d3d11 -init_hw_device qsv=qsv@d3d11 - hwaccel d3d11va -hwaccel_output_format d3d11 -i input.mp4 -vf "hwmap=derive_device=qsv:extra_hw_frames=16,format=qsv" -c:v h264_qsv output.mp4 Now we don't have to explicitly specify hwmap. The auto filter mechanism will try to insert a hwmap filter automatically. The command line will be much simpler. ffmpeg -init_hw_device d3d11va=d3d11 -init_hw_device qsv=qsv@d3d11 - hwaccel d3d11va -hwaccel_output_format d3d11 -i input.mp4 -c:v h264_qsv output.mp4 > >I just fear that, especially given the bad error messages hwmap gives, >this will just implicitly insert it because it seemingly works but then >just fail to actually do the job and give an absolutely indescriptive >error to the user for a filter they did not even insert themselves. Yes I agree with that. However, we already have an auto scale filter in the framework. It will always be inserted when two filters cannot be linked. In my opinion, I'm just kinda appending another filter. It should be no harmful to the original framework. The only way you can get to this auto hwmap filter is that you already fail to insert scale filter and the program will return error. That's the current situation. With this patch set, those failed cmdlines could still fail but some cmdlines like above could pass. It just give it another chance to run and simplify the cmdlines. Regards, Tong > >> Signed-off-by: Tong Wu <tong1.wu@intel.com> >> --- >> libavfilter/avfiltergraph.c | 3 ++- >> libavfilter/formats.c | 4 ++++ >> 2 files changed, 6 insertions(+), 1 deletion(-) >> >> diff --git a/libavfilter/avfiltergraph.c b/libavfilter/avfiltergraph.c >> index 8af0467bc5..b2b627ad6a 100644 >> --- a/libavfilter/avfiltergraph.c >> +++ b/libavfilter/avfiltergraph.c >> @@ -402,7 +402,8 @@ static int insert_auto_filter(AVFilterContext >**convert, AVFilterGraph *graph, >> AVFilterContext *ctx; >> AVFilterLink *inlink, *outlink; >> char inst_name[30]; >> - const char *opts = FF_FIELD_AT(char *, neg- >>conversion_filters[conv_step].conversion_opts_offset, *graph); >> + const char *opts = neg- >>conversion_filters[conv_step].conversion_opts_offset == 0 ? NULL : >> + FF_FIELD_AT(char *, neg- >>conversion_filters[conv_step].conversion_opts_offset, *graph); >> const char *name = neg->conversion_filters[conv_step].conversion_filter; >> >> if (!(filter = avfilter_get_by_name(name))) { >> diff --git a/libavfilter/formats.c b/libavfilter/formats.c >> index c8e20e5b20..fee10fa0ee 100644 >> --- a/libavfilter/formats.c >> +++ b/libavfilter/formats.c >> @@ -331,6 +331,10 @@ static const AVFilterFormatsFilter filters_video[] = { >> .conversion_filter = "scale", >> .conversion_opts_offset = offsetof(AVFilterGraph, scale_sws_opts), >> }, >> + { >> + .conversion_filter = "hwmap", >> + .conversion_opts_offset = 0, >> + } >> }; >> >> static const AVFilterFormatsFilter filters_audio[] = { >> -- >> 2.39.1.windows.1 >> >> _______________________________________________ >> ffmpeg-devel mailing list >> ffmpeg-devel@ffmpeg.org >> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel >> >> To unsubscribe, visit link above, or email >> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >_______________________________________________ >ffmpeg-devel mailing list >ffmpeg-devel@ffmpeg.org >https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > >To unsubscribe, visit link above, or email >ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
diff --git a/libavfilter/avfiltergraph.c b/libavfilter/avfiltergraph.c index 8af0467bc5..b2b627ad6a 100644 --- a/libavfilter/avfiltergraph.c +++ b/libavfilter/avfiltergraph.c @@ -402,7 +402,8 @@ static int insert_auto_filter(AVFilterContext **convert, AVFilterGraph *graph, AVFilterContext *ctx; AVFilterLink *inlink, *outlink; char inst_name[30]; - const char *opts = FF_FIELD_AT(char *, neg->conversion_filters[conv_step].conversion_opts_offset, *graph); + const char *opts = neg->conversion_filters[conv_step].conversion_opts_offset == 0 ? NULL : + FF_FIELD_AT(char *, neg->conversion_filters[conv_step].conversion_opts_offset, *graph); const char *name = neg->conversion_filters[conv_step].conversion_filter; if (!(filter = avfilter_get_by_name(name))) { diff --git a/libavfilter/formats.c b/libavfilter/formats.c index c8e20e5b20..fee10fa0ee 100644 --- a/libavfilter/formats.c +++ b/libavfilter/formats.c @@ -331,6 +331,10 @@ static const AVFilterFormatsFilter filters_video[] = { .conversion_filter = "scale", .conversion_opts_offset = offsetof(AVFilterGraph, scale_sws_opts), }, + { + .conversion_filter = "hwmap", + .conversion_opts_offset = 0, + } }; static const AVFilterFormatsFilter filters_audio[] = {
When two formats lists cannot be merged, a scale filter is auto-inserted. However, when it comes to hardware map, we have to manually add a hwmap filter to do the conversion. This patch introduces an auto hwmap filter to do the hwmap conversion automatically. Signed-off-by: Tong Wu <tong1.wu@intel.com> --- libavfilter/avfiltergraph.c | 3 ++- libavfilter/formats.c | 4 ++++ 2 files changed, 6 insertions(+), 1 deletion(-)