| Message ID | 20210805081910.9386-1-haihao.xiang@intel.com |
|---|---|
| Headers | show |
| Series | clean-up QSV filters | expand |
> -----Original Message----- > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > Haihao Xiang > Sent: Thursday, 5 August 2021 10:19 > To: ffmpeg-devel@ffmpeg.org > Cc: Haihao Xiang <haihao.xiang@intel.com> > Subject: [FFmpeg-devel] [PATCH v5 00/20] clean-up QSV filters > > This patchset clean up scale_qsv and deinterlace_qsv filters, and take the > two filters as the special cases of vpp_qsv, so vf_scale_qsv.c and > vf_deinterlace_qsv.c can be deleted from FFmpeg. In addition, a few small > features are added in this patchset. > --- > v5: > * Rebased this patchset against the latest master branch and fixed conflicts > > Haihao Xiang (20): > lavfi/qsv: use QSVVPPContext as base context in > vf_vpp_qsv/vf_overlay_qsv > lavfi/scale_qsv: simplify scale_qsv filter > lavfi/scale_qsv: don't need variables for constants in FFmpeg > lavfi/vpp_qsv: add "a", "dar" and "sar" variables > lavfi/vpp_qsv: handle NULL pointer when evaluating an expression > lavfi/vpp_qsv: allow special values for the output dimensions > lavfi/vpp_qsv: factorize extra MFX configuration > lavfi/vpp_qsv: allow user to set scale_mode with constant > lavfi/vpp_qsv: add vpp_preinit callback > lavfi/scale_qsv: re-use VPPContext for scale_qsv filter > lavfi/vpp_qsv: factor common QSV filter definition > lavfi/scale_qsv: add new options for scale_qsv filter > lavfi/scale_qsv: add more input / output pixel formats > lavfi/qsvvpp: avoid overriding the returned value > lavfi/qsvvpp: set PTS for output frame > lavfi/vpp_qsv: check output format string against NULL pointer > lavfi/deinterlace_qsv: simplify deinterlace_qsv filter > lavfi/deinterlace_qsv: re-use VPPContext for deinterlace_qsv filter > lavfi/deinterlace_qsv: add async_depth option > lavfi/deinterlace_qsv: add more input / output pixel formats > > libavfilter/Makefile | 4 +- > libavfilter/qsvvpp.c | 57 ++- > libavfilter/qsvvpp.h | 11 +- > libavfilter/vf_deinterlace_qsv.c | 611 --------------------------- > libavfilter/vf_overlay_qsv.c | 11 +- > libavfilter/vf_scale_qsv.c | 685 ------------------------------- > libavfilter/vf_vpp_qsv.c | 473 +++++++++++++-------- > 7 files changed, 347 insertions(+), 1505 deletions(-) delete mode 100644 > libavfilter/vf_deinterlace_qsv.c delete mode 100644 > libavfilter/vf_scale_qsv.c > > -- > 2.17.1 Hi Hihao, The general idea of this patch makes sense to me. Currently there are implementation differences between these filters, and there are cases where vpp_qsv doesn't work and I need to use scale_qsv instead. I have never analyzed the actual reason, but this should be done before replacing scale_qsv with an aliased vpp_qsv. I'll try to dig out an example.. softworkz
On Thu, 2021-08-05 at 16:21 +0000, Soft Works wrote: > > -----Original Message----- > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > > Haihao Xiang > > Sent: Thursday, 5 August 2021 10:19 > > To: ffmpeg-devel@ffmpeg.org > > Cc: Haihao Xiang <haihao.xiang@intel.com> > > Subject: [FFmpeg-devel] [PATCH v5 00/20] clean-up QSV filters > > > > This patchset clean up scale_qsv and deinterlace_qsv filters, and take the > > two filters as the special cases of vpp_qsv, so vf_scale_qsv.c and > > vf_deinterlace_qsv.c can be deleted from FFmpeg. In addition, a few small > > features are added in this patchset. > > --- > > v5: > > * Rebased this patchset against the latest master branch and fixed conflicts > > > > Haihao Xiang (20): > > lavfi/qsv: use QSVVPPContext as base context in > > vf_vpp_qsv/vf_overlay_qsv > > lavfi/scale_qsv: simplify scale_qsv filter > > lavfi/scale_qsv: don't need variables for constants in FFmpeg > > lavfi/vpp_qsv: add "a", "dar" and "sar" variables > > lavfi/vpp_qsv: handle NULL pointer when evaluating an expression > > lavfi/vpp_qsv: allow special values for the output dimensions > > lavfi/vpp_qsv: factorize extra MFX configuration > > lavfi/vpp_qsv: allow user to set scale_mode with constant > > lavfi/vpp_qsv: add vpp_preinit callback > > lavfi/scale_qsv: re-use VPPContext for scale_qsv filter > > lavfi/vpp_qsv: factor common QSV filter definition > > lavfi/scale_qsv: add new options for scale_qsv filter > > lavfi/scale_qsv: add more input / output pixel formats > > lavfi/qsvvpp: avoid overriding the returned value > > lavfi/qsvvpp: set PTS for output frame > > lavfi/vpp_qsv: check output format string against NULL pointer > > lavfi/deinterlace_qsv: simplify deinterlace_qsv filter > > lavfi/deinterlace_qsv: re-use VPPContext for deinterlace_qsv filter > > lavfi/deinterlace_qsv: add async_depth option > > lavfi/deinterlace_qsv: add more input / output pixel formats > > > > libavfilter/Makefile | 4 +- > > libavfilter/qsvvpp.c | 57 ++- > > libavfilter/qsvvpp.h | 11 +- > > libavfilter/vf_deinterlace_qsv.c | 611 --------------------------- > > libavfilter/vf_overlay_qsv.c | 11 +- > > libavfilter/vf_scale_qsv.c | 685 ------------------------------- > > libavfilter/vf_vpp_qsv.c | 473 +++++++++++++-------- > > 7 files changed, 347 insertions(+), 1505 deletions(-) delete mode 100644 > > libavfilter/vf_deinterlace_qsv.c delete mode 100644 > > libavfilter/vf_scale_qsv.c > > > > -- > > 2.17.1 > > Hi Hihao, > > The general idea of this patch makes sense to me. > > Currently there are implementation differences between these filters, > and there are cases where vpp_qsv doesn't work and I need to use > scale_qsv instead. > > I have never analyzed the actual reason, but this should be done > before replacing scale_qsv with an aliased vpp_qsv. > > I'll try to dig out an example.. Hi Softworkz, Could you provide the cases when you have time ? I may look into the issues. Thanks Haihao
> -----Original Message----- > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > Xiang, Haihao > Sent: Monday, 30 August 2021 06:20 > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH v5 00/20] clean-up QSV filters > > On Thu, 2021-08-05 at 16:21 +0000, Soft Works wrote: > > > -----Original Message----- > > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > > > Haihao Xiang > > > Sent: Thursday, 5 August 2021 10:19 > > > To: ffmpeg-devel@ffmpeg.org > > > Cc: Haihao Xiang <haihao.xiang@intel.com> > > > Subject: [FFmpeg-devel] [PATCH v5 00/20] clean-up QSV filters > > > > > > This patchset clean up scale_qsv and deinterlace_qsv filters, and > take the > > > two filters as the special cases of vpp_qsv, so vf_scale_qsv.c > and > > > vf_deinterlace_qsv.c can be deleted from FFmpeg. In addition, a > few small > > > features are added in this patchset. > > > --- > > > v5: > > > * Rebased this patchset against the latest master branch and > fixed conflicts > > > > > > Haihao Xiang (20): > > > lavfi/qsv: use QSVVPPContext as base context in > > > vf_vpp_qsv/vf_overlay_qsv > > > lavfi/scale_qsv: simplify scale_qsv filter > > > lavfi/scale_qsv: don't need variables for constants in FFmpeg > > > lavfi/vpp_qsv: add "a", "dar" and "sar" variables > > > lavfi/vpp_qsv: handle NULL pointer when evaluating an > expression > > > lavfi/vpp_qsv: allow special values for the output dimensions > > > lavfi/vpp_qsv: factorize extra MFX configuration > > > lavfi/vpp_qsv: allow user to set scale_mode with constant > > > lavfi/vpp_qsv: add vpp_preinit callback > > > lavfi/scale_qsv: re-use VPPContext for scale_qsv filter > > > lavfi/vpp_qsv: factor common QSV filter definition > > > lavfi/scale_qsv: add new options for scale_qsv filter > > > lavfi/scale_qsv: add more input / output pixel formats > > > lavfi/qsvvpp: avoid overriding the returned value > > > lavfi/qsvvpp: set PTS for output frame > > > lavfi/vpp_qsv: check output format string against NULL pointer > > > lavfi/deinterlace_qsv: simplify deinterlace_qsv filter > > > lavfi/deinterlace_qsv: re-use VPPContext for deinterlace_qsv > filter > > > lavfi/deinterlace_qsv: add async_depth option > > > lavfi/deinterlace_qsv: add more input / output pixel formats > > > > > > libavfilter/Makefile | 4 +- > > > libavfilter/qsvvpp.c | 57 ++- > > > libavfilter/qsvvpp.h | 11 +- > > > libavfilter/vf_deinterlace_qsv.c | 611 ------------------------- > -- > > > libavfilter/vf_overlay_qsv.c | 11 +- > > > libavfilter/vf_scale_qsv.c | 685 ------------------------- > ------ > > > libavfilter/vf_vpp_qsv.c | 473 +++++++++++++-------- > > > 7 files changed, 347 insertions(+), 1505 deletions(-) delete > mode 100644 > > > libavfilter/vf_deinterlace_qsv.c delete mode 100644 > > > libavfilter/vf_scale_qsv.c > > > > > > -- > > > 2.17.1 > > > > Hi Hihao, > > > > The general idea of this patch makes sense to me. > > > > Currently there are implementation differences between these > filters, > > and there are cases where vpp_qsv doesn't work and I need to use > > scale_qsv instead. > > > > I have never analyzed the actual reason, but this should be done > > before replacing scale_qsv with an aliased vpp_qsv. > > > > I'll try to dig out an example.. > > > Hi Softworkz, > > Could you provide the cases when you have time ? I may look into the > issues. > > Thanks > Haihao Hi Haihao, IIRC think it can be reproduced easily by using the filters without parameters, where scale_qsv works and vpp_qsv doesn't work. If it doesn't reproduce, try this in combination with hwupload, e.g. - "hwupload@f1=extra_hw_frames=32,vpp_qsv" vs. - "hwupload@f1=extra_hw_frames=32,scale_qsv" Please let me know whether you can reproduce. Kind regards, softworkz
On Mon, 2021-08-30 at 05:52 +0000, Soft Works wrote: > > -----Original Message----- > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > > Xiang, Haihao > > Sent: Monday, 30 August 2021 06:20 > > To: ffmpeg-devel@ffmpeg.org > > Subject: Re: [FFmpeg-devel] [PATCH v5 00/20] clean-up QSV filters > > > > On Thu, 2021-08-05 at 16:21 +0000, Soft Works wrote: > > > > -----Original Message----- > > > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > > > > Haihao Xiang > > > > Sent: Thursday, 5 August 2021 10:19 > > > > To: ffmpeg-devel@ffmpeg.org > > > > Cc: Haihao Xiang <haihao.xiang@intel.com> > > > > Subject: [FFmpeg-devel] [PATCH v5 00/20] clean-up QSV filters > > > > > > > > This patchset clean up scale_qsv and deinterlace_qsv filters, and > > > > take the > > > > two filters as the special cases of vpp_qsv, so vf_scale_qsv.c > > > > and > > > > vf_deinterlace_qsv.c can be deleted from FFmpeg. In addition, a > > > > few small > > > > features are added in this patchset. > > > > --- > > > > v5: > > > > * Rebased this patchset against the latest master branch and > > > > fixed conflicts > > > > > > > > Haihao Xiang (20): > > > > lavfi/qsv: use QSVVPPContext as base context in > > > > vf_vpp_qsv/vf_overlay_qsv > > > > lavfi/scale_qsv: simplify scale_qsv filter > > > > lavfi/scale_qsv: don't need variables for constants in FFmpeg > > > > lavfi/vpp_qsv: add "a", "dar" and "sar" variables > > > > lavfi/vpp_qsv: handle NULL pointer when evaluating an > > > > expression > > > > lavfi/vpp_qsv: allow special values for the output dimensions > > > > lavfi/vpp_qsv: factorize extra MFX configuration > > > > lavfi/vpp_qsv: allow user to set scale_mode with constant > > > > lavfi/vpp_qsv: add vpp_preinit callback > > > > lavfi/scale_qsv: re-use VPPContext for scale_qsv filter > > > > lavfi/vpp_qsv: factor common QSV filter definition > > > > lavfi/scale_qsv: add new options for scale_qsv filter > > > > lavfi/scale_qsv: add more input / output pixel formats > > > > lavfi/qsvvpp: avoid overriding the returned value > > > > lavfi/qsvvpp: set PTS for output frame > > > > lavfi/vpp_qsv: check output format string against NULL pointer > > > > lavfi/deinterlace_qsv: simplify deinterlace_qsv filter > > > > lavfi/deinterlace_qsv: re-use VPPContext for deinterlace_qsv > > > > filter > > > > lavfi/deinterlace_qsv: add async_depth option > > > > lavfi/deinterlace_qsv: add more input / output pixel formats > > > > > > > > libavfilter/Makefile | 4 +- > > > > libavfilter/qsvvpp.c | 57 ++- > > > > libavfilter/qsvvpp.h | 11 +- > > > > libavfilter/vf_deinterlace_qsv.c | 611 ------------------------- > > > > -- > > > > libavfilter/vf_overlay_qsv.c | 11 +- > > > > libavfilter/vf_scale_qsv.c | 685 ------------------------- > > > > ------ > > > > libavfilter/vf_vpp_qsv.c | 473 +++++++++++++-------- > > > > 7 files changed, 347 insertions(+), 1505 deletions(-) delete > > > > mode 100644 > > > > libavfilter/vf_deinterlace_qsv.c delete mode 100644 > > > > libavfilter/vf_scale_qsv.c > > > > > > > > -- > > > > 2.17.1 > > > > > > Hi Hihao, > > > > > > The general idea of this patch makes sense to me. > > > > > > Currently there are implementation differences between these > > > > filters, > > > and there are cases where vpp_qsv doesn't work and I need to use > > > scale_qsv instead. > > > > > > I have never analyzed the actual reason, but this should be done > > > before replacing scale_qsv with an aliased vpp_qsv. > > > > > > I'll try to dig out an example.. > > > > > > Hi Softworkz, > > > > Could you provide the cases when you have time ? I may look into the > > issues. > > > > Thanks > > Haihao > > > Hi Haihao, > > IIRC think it can be reproduced easily by using the filters without > parameters, > where scale_qsv works and vpp_qsv doesn't work. scale_qsv only supports video memory input and output however vpp_qsv may support both video memory and system memory, there is a case where scale_qsv doesn't work but vpp_qsv may work. $> ffmpeg -c:v h264_qsv -i input.mp4 -vf scale_qsv -f null - Impossible to convert between the formats supported by the filter 'graph 0 input from stream 0:0' and the filter 'auto_scale_0' Error reinitializing filters! Failed to inject frame into filter network: Function not implemented Error while processing the decoded data for stream #0:0 Conversion failed! After applying this patchset, scale_qsv may work with both video memory and system memory. In addition, vpp_qsv may support crop however scale_qsv doesn't support crop. After applying this patchset, scale_qsv may support crop too. > > If it doesn't reproduce, try this in combination with hwupload, e.g. > > - "hwupload@f1=extra_hw_frames=32,vpp_qsv" > vs. > - "hwupload@f1=extra_hw_frames=32,scale_qsv" > The above two combinations work for me Thanks Haihao
On Mon, 2021-08-30 at 08:04 +0000, Xiang, Haihao wrote: > On Mon, 2021-08-30 at 05:52 +0000, Soft Works wrote: > > > -----Original Message----- > > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > > > Xiang, Haihao > > > Sent: Monday, 30 August 2021 06:20 > > > To: ffmpeg-devel@ffmpeg.org > > > Subject: Re: [FFmpeg-devel] [PATCH v5 00/20] clean-up QSV filters > > > > > > On Thu, 2021-08-05 at 16:21 +0000, Soft Works wrote: > > > > > -----Original Message----- > > > > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > > > > > Haihao Xiang > > > > > Sent: Thursday, 5 August 2021 10:19 > > > > > To: ffmpeg-devel@ffmpeg.org > > > > > Cc: Haihao Xiang <haihao.xiang@intel.com> > > > > > Subject: [FFmpeg-devel] [PATCH v5 00/20] clean-up QSV filters > > > > > > > > > > This patchset clean up scale_qsv and deinterlace_qsv filters, and > > > > > > take the > > > > > two filters as the special cases of vpp_qsv, so vf_scale_qsv.c > > > > > > and > > > > > vf_deinterlace_qsv.c can be deleted from FFmpeg. In addition, a > > > > > > few small > > > > > features are added in this patchset. > > > > > --- > > > > > v5: > > > > > * Rebased this patchset against the latest master branch and > > > > > > fixed conflicts > > > > > > > > > > Haihao Xiang (20): > > > > > lavfi/qsv: use QSVVPPContext as base context in > > > > > vf_vpp_qsv/vf_overlay_qsv > > > > > lavfi/scale_qsv: simplify scale_qsv filter > > > > > lavfi/scale_qsv: don't need variables for constants in FFmpeg > > > > > lavfi/vpp_qsv: add "a", "dar" and "sar" variables > > > > > lavfi/vpp_qsv: handle NULL pointer when evaluating an > > > > > > expression > > > > > lavfi/vpp_qsv: allow special values for the output dimensions > > > > > lavfi/vpp_qsv: factorize extra MFX configuration > > > > > lavfi/vpp_qsv: allow user to set scale_mode with constant > > > > > lavfi/vpp_qsv: add vpp_preinit callback > > > > > lavfi/scale_qsv: re-use VPPContext for scale_qsv filter > > > > > lavfi/vpp_qsv: factor common QSV filter definition > > > > > lavfi/scale_qsv: add new options for scale_qsv filter > > > > > lavfi/scale_qsv: add more input / output pixel formats > > > > > lavfi/qsvvpp: avoid overriding the returned value > > > > > lavfi/qsvvpp: set PTS for output frame > > > > > lavfi/vpp_qsv: check output format string against NULL pointer > > > > > lavfi/deinterlace_qsv: simplify deinterlace_qsv filter > > > > > lavfi/deinterlace_qsv: re-use VPPContext for deinterlace_qsv > > > > > > filter > > > > > lavfi/deinterlace_qsv: add async_depth option > > > > > lavfi/deinterlace_qsv: add more input / output pixel formats > > > > > > > > > > libavfilter/Makefile | 4 +- > > > > > libavfilter/qsvvpp.c | 57 ++- > > > > > libavfilter/qsvvpp.h | 11 +- > > > > > libavfilter/vf_deinterlace_qsv.c | 611 ------------------------- > > > > > > -- > > > > > libavfilter/vf_overlay_qsv.c | 11 +- > > > > > libavfilter/vf_scale_qsv.c | 685 ------------------------- > > > > > > ------ > > > > > libavfilter/vf_vpp_qsv.c | 473 +++++++++++++-------- > > > > > 7 files changed, 347 insertions(+), 1505 deletions(-) delete > > > > > > mode 100644 > > > > > libavfilter/vf_deinterlace_qsv.c delete mode 100644 > > > > > libavfilter/vf_scale_qsv.c > > > > > > > > > > -- > > > > > 2.17.1 > > > > > > > > Hi Hihao, > > > > > > > > The general idea of this patch makes sense to me. > > > > > > > > Currently there are implementation differences between these > > > > > > filters, > > > > and there are cases where vpp_qsv doesn't work and I need to use > > > > scale_qsv instead. > > > > > > > > I have never analyzed the actual reason, but this should be done > > > > before replacing scale_qsv with an aliased vpp_qsv. > > > > > > > > I'll try to dig out an example.. > > > > > > > > > Hi Softworkz, > > > > > > Could you provide the cases when you have time ? I may look into the > > > issues. > > > > > > Thanks > > > Haihao > > > > > > Hi Haihao, > > > > IIRC think it can be reproduced easily by using the filters without > > parameters, > > where scale_qsv works and vpp_qsv doesn't work. > > scale_qsv only supports video memory input and output however vpp_qsv may > support both video memory and system memory, there is a case where scale_qsv > doesn't work but vpp_qsv may work. > > $> ffmpeg -c:v h264_qsv -i input.mp4 -vf scale_qsv -f null - > > Impossible to convert between the formats supported by the filter 'graph 0 > input > from stream 0:0' and the filter 'auto_scale_0' > Error reinitializing filters! > Failed to inject frame into filter network: Function not implemented > Error while processing the decoded data for stream #0:0 > Conversion failed! > > After applying this patchset, scale_qsv may work with both video memory and > system memory. > > In addition, vpp_qsv may support crop however scale_qsv doesn't support crop. > After applying this patchset, scale_qsv may support crop too. > > > > > If it doesn't reproduce, try this in combination with hwupload, e.g. > > > > - "hwupload@f1=extra_hw_frames=32,vpp_qsv" > > vs. > > - "hwupload@f1=extra_hw_frames=32,scale_qsv" > > > > The above two combinations work for me Hi Softworkz, I failed to reproduce your issue, could you provide more info if possible? Thanks Haihao
On Thu, 2021-09-09 at 14:56 +0000, Xiang, Haihao wrote: > On Mon, 2021-08-30 at 08:04 +0000, Xiang, Haihao wrote: > > On Mon, 2021-08-30 at 05:52 +0000, Soft Works wrote: > > > > -----Original Message----- > > > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > > > > Xiang, Haihao > > > > Sent: Monday, 30 August 2021 06:20 > > > > To: ffmpeg-devel@ffmpeg.org > > > > Subject: Re: [FFmpeg-devel] [PATCH v5 00/20] clean-up QSV filters > > > > > > > > On Thu, 2021-08-05 at 16:21 +0000, Soft Works wrote: > > > > > > -----Original Message----- > > > > > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > > > > > > Haihao Xiang > > > > > > Sent: Thursday, 5 August 2021 10:19 > > > > > > To: ffmpeg-devel@ffmpeg.org > > > > > > Cc: Haihao Xiang <haihao.xiang@intel.com> > > > > > > Subject: [FFmpeg-devel] [PATCH v5 00/20] clean-up QSV filters > > > > > > > > > > > > This patchset clean up scale_qsv and deinterlace_qsv filters, and > > > > > > > > take the > > > > > > two filters as the special cases of vpp_qsv, so vf_scale_qsv.c > > > > > > > > and > > > > > > vf_deinterlace_qsv.c can be deleted from FFmpeg. In addition, a > > > > > > > > few small > > > > > > features are added in this patchset. > > > > > > --- > > > > > > v5: > > > > > > * Rebased this patchset against the latest master branch and > > > > > > > > fixed conflicts > > > > > > > > > > > > Haihao Xiang (20): > > > > > > lavfi/qsv: use QSVVPPContext as base context in > > > > > > vf_vpp_qsv/vf_overlay_qsv > > > > > > lavfi/scale_qsv: simplify scale_qsv filter > > > > > > lavfi/scale_qsv: don't need variables for constants in FFmpeg > > > > > > lavfi/vpp_qsv: add "a", "dar" and "sar" variables > > > > > > lavfi/vpp_qsv: handle NULL pointer when evaluating an > > > > > > > > expression > > > > > > lavfi/vpp_qsv: allow special values for the output dimensions > > > > > > lavfi/vpp_qsv: factorize extra MFX configuration > > > > > > lavfi/vpp_qsv: allow user to set scale_mode with constant > > > > > > lavfi/vpp_qsv: add vpp_preinit callback > > > > > > lavfi/scale_qsv: re-use VPPContext for scale_qsv filter > > > > > > lavfi/vpp_qsv: factor common QSV filter definition > > > > > > lavfi/scale_qsv: add new options for scale_qsv filter > > > > > > lavfi/scale_qsv: add more input / output pixel formats > > > > > > lavfi/qsvvpp: avoid overriding the returned value > > > > > > lavfi/qsvvpp: set PTS for output frame > > > > > > lavfi/vpp_qsv: check output format string against NULL pointer > > > > > > lavfi/deinterlace_qsv: simplify deinterlace_qsv filter > > > > > > lavfi/deinterlace_qsv: re-use VPPContext for deinterlace_qsv > > > > > > > > filter > > > > > > lavfi/deinterlace_qsv: add async_depth option > > > > > > lavfi/deinterlace_qsv: add more input / output pixel formats > > > > > > > > > > > > libavfilter/Makefile | 4 +- > > > > > > libavfilter/qsvvpp.c | 57 ++- > > > > > > libavfilter/qsvvpp.h | 11 +- > > > > > > libavfilter/vf_deinterlace_qsv.c | 611 ------------------------- > > > > > > > > -- > > > > > > libavfilter/vf_overlay_qsv.c | 11 +- > > > > > > libavfilter/vf_scale_qsv.c | 685 ------------------------- > > > > > > > > ------ > > > > > > libavfilter/vf_vpp_qsv.c | 473 +++++++++++++-------- > > > > > > 7 files changed, 347 insertions(+), 1505 deletions(-) delete > > > > > > > > mode 100644 > > > > > > libavfilter/vf_deinterlace_qsv.c delete mode 100644 > > > > > > libavfilter/vf_scale_qsv.c > > > > > > > > > > > > -- > > > > > > 2.17.1 > > > > > > > > > > Hi Hihao, > > > > > > > > > > The general idea of this patch makes sense to me. > > > > > > > > > > Currently there are implementation differences between these > > > > > > > > filters, > > > > > and there are cases where vpp_qsv doesn't work and I need to use > > > > > scale_qsv instead. > > > > > > > > > > I have never analyzed the actual reason, but this should be done > > > > > before replacing scale_qsv with an aliased vpp_qsv. > > > > > > > > > > I'll try to dig out an example.. > > > > > > > > > > > > Hi Softworkz, > > > > > > > > Could you provide the cases when you have time ? I may look into the > > > > issues. > > > > > > > > Thanks > > > > Haihao > > > > > > > > > Hi Haihao, > > > > > > IIRC think it can be reproduced easily by using the filters without > > > parameters, > > > where scale_qsv works and vpp_qsv doesn't work. > > > > scale_qsv only supports video memory input and output however vpp_qsv may > > support both video memory and system memory, there is a case where scale_qsv > > doesn't work but vpp_qsv may work. > > > > $> ffmpeg -c:v h264_qsv -i input.mp4 -vf scale_qsv -f null - > > > > Impossible to convert between the formats supported by the filter 'graph 0 > > input > > from stream 0:0' and the filter 'auto_scale_0' > > Error reinitializing filters! > > Failed to inject frame into filter network: Function not implemented > > Error while processing the decoded data for stream #0:0 > > Conversion failed! > > > > After applying this patchset, scale_qsv may work with both video memory and > > system memory. > > > > In addition, vpp_qsv may support crop however scale_qsv doesn't support > > crop. > > After applying this patchset, scale_qsv may support crop too. > > > > > > > > If it doesn't reproduce, try this in combination with hwupload, e.g. > > > > > > - "hwupload@f1=extra_hw_frames=32,vpp_qsv" > > > vs. > > > - "hwupload@f1=extra_hw_frames=32,scale_qsv" > > > > > > > The above two combinations work for me > > Hi Softworkz, > > I failed to reproduce your issue, could you provide more info if possible? Hi Softworkz, I didn't reproduce your issue, maybe the issue has been fixed on master. How about to merge this patchset first? I will rebase the patchset if no objection. BTW we didn't see regression in our internal testing. Thanks Haihao
> -----Original Message----- > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > Xiang, Haihao > Sent: Saturday, September 18, 2021 3:27 AM > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH v5 00/20] clean-up QSV filters > > On Thu, 2021-09-09 at 14:56 +0000, Xiang, Haihao wrote: > > On Mon, 2021-08-30 at 08:04 +0000, Xiang, Haihao wrote: > > > On Mon, 2021-08-30 at 05:52 +0000, Soft Works wrote: [..] > > > > > > > > > > > > Hi Hihao, > > > > > > > > > > > > The general idea of this patch makes sense to me. > > > > > > > > > > > > Currently there are implementation differences between > these > > > > > > > > > > filters, > > > > > > and there are cases where vpp_qsv doesn't work and I need > to use > > > > > > scale_qsv instead. > > > > > > > > > > > > I have never analyzed the actual reason, but this should be > done > > > > > > before replacing scale_qsv with an aliased vpp_qsv. > > > > > > > > > > > > I'll try to dig out an example.. > > > > > > > > > > > > > > > Hi Softworkz, > > > > > > > > > > Could you provide the cases when you have time ? I may look > into the > > > > > issues. > > > > > > > > > > Thanks > > > > > Haihao > > > > > > > > > > > > Hi Haihao, > > > > > > > > IIRC think it can be reproduced easily by using the filters > without > > > > parameters, > > > > where scale_qsv works and vpp_qsv doesn't work. > > > > > > scale_qsv only supports video memory input and output however > vpp_qsv may > > > support both video memory and system memory, there is a case > where scale_qsv > > > doesn't work but vpp_qsv may work. > > > > > > $> ffmpeg -c:v h264_qsv -i input.mp4 -vf scale_qsv -f null - > > > > > > Impossible to convert between the formats supported by the filter > 'graph 0 > > > input > > > from stream 0:0' and the filter 'auto_scale_0' > > > Error reinitializing filters! > > > Failed to inject frame into filter network: Function not > implemented > > > Error while processing the decoded data for stream #0:0 > > > Conversion failed! > > > > > > After applying this patchset, scale_qsv may work with both video > memory and > > > system memory. > > > > > > In addition, vpp_qsv may support crop however scale_qsv doesn't > support > > > crop. > > > After applying this patchset, scale_qsv may support crop too. > > > > > > > > > > > If it doesn't reproduce, try this in combination with hwupload, > e.g. > > > > > > > > - "hwupload@f1=extra_hw_frames=32,vpp_qsv" > > > > vs. > > > > - "hwupload@f1=extra_hw_frames=32,scale_qsv" > > > > > > > > > > The above two combinations work for me > > > > Hi Softworkz, > > > > I failed to reproduce your issue, could you provide more info if > possible? > > Hi Softworkz, > > I didn't reproduce your issue, maybe the issue has been fixed on > master. How > about to merge this patchset first? I will rebase the patchset if no > objection. > BTW we didn't see regression in our internal testing. Hi Haihao, (sending a CC to the ML allowing others to follow) I found an example for the cases I was referring to. An example command line is this: ffmpeg.exe -y -loglevel verbose -init_hw_device "qsv=qd1:hw_any,child_device=1,child_device_type=d3d11va" -hwaccel qsv -c:v h264_qsv -canvas_size:s:0 "1920:1080" -i red2.mkv -filter_complex "[0:0]vpp_qsv[vid];[0:3]format=nv12,hwupload@f1=extra_hw_frames=16,vpp_qsv=w=400:h=200[subs];[vid][subs]overlay_qsv[out]" -map [out] -c:v h264_qsv -b:v 5M output.mp4 It contains two uses of vpp_qsv: one in “passthrough mode” and the other one with scaling. Only when replacing BOTH usages of vpp_qsv with scale_qsv, the command will work. (test file on request) Kind regards, softworkz
On Sun, 2021-10-10 at 22:37 +0000, Soft Works wrote: > > -----Original Message----- > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > > Xiang, Haihao > > Sent: Saturday, September 18, 2021 3:27 AM > > To: ffmpeg-devel@ffmpeg.org > > Subject: Re: [FFmpeg-devel] [PATCH v5 00/20] clean-up QSV filters > > > > On Thu, 2021-09-09 at 14:56 +0000, Xiang, Haihao wrote: > > > On Mon, 2021-08-30 at 08:04 +0000, Xiang, Haihao wrote: > > > > On Mon, 2021-08-30 at 05:52 +0000, Soft Works wrote: > > > [..] > > > > > > > > > > > > > > > Hi Hihao, > > > > > > > > > > > > > > The general idea of this patch makes sense to me. > > > > > > > > > > > > > > Currently there are implementation differences between > > > > these > > > > > > > > > > > > filters, > > > > > > > and there are cases where vpp_qsv doesn't work and I need > > > > to use > > > > > > > scale_qsv instead. > > > > > > > > > > > > > > I have never analyzed the actual reason, but this should be > > > > done > > > > > > > before replacing scale_qsv with an aliased vpp_qsv. > > > > > > > > > > > > > > I'll try to dig out an example.. > > > > > > > > > > > > > > > > > > Hi Softworkz, > > > > > > > > > > > > Could you provide the cases when you have time ? I may look > > > > into the > > > > > > issues. > > > > > > > > > > > > Thanks > > > > > > Haihao > > > > > > > > > > > > > > > Hi Haihao, > > > > > > > > > > IIRC think it can be reproduced easily by using the filters > > > > without > > > > > parameters, > > > > > where scale_qsv works and vpp_qsv doesn't work. > > > > > > > > scale_qsv only supports video memory input and output however > > > > vpp_qsv may > > > > support both video memory and system memory, there is a case > > > > where scale_qsv > > > > doesn't work but vpp_qsv may work. > > > > > > > > $> ffmpeg -c:v h264_qsv -i input.mp4 -vf scale_qsv -f null - > > > > > > > > Impossible to convert between the formats supported by the filter > > > > 'graph 0 > > > > input > > > > from stream 0:0' and the filter 'auto_scale_0' > > > > Error reinitializing filters! > > > > Failed to inject frame into filter network: Function not > > > > implemented > > > > Error while processing the decoded data for stream #0:0 > > > > Conversion failed! > > > > > > > > After applying this patchset, scale_qsv may work with both video > > > > memory and > > > > system memory. > > > > > > > > In addition, vpp_qsv may support crop however scale_qsv doesn't > > > > support > > > > crop. > > > > After applying this patchset, scale_qsv may support crop too. > > > > > > > > > > > > > > If it doesn't reproduce, try this in combination with hwupload, > > > > e.g. > > > > > > > > > > - "hwupload@f1=extra_hw_frames=32,vpp_qsv" > > > > > vs. > > > > > - "hwupload@f1=extra_hw_frames=32,scale_qsv" > > > > > > > > > > > > > The above two combinations work for me > > > > > > Hi Softworkz, > > > > > > I failed to reproduce your issue, could you provide more info if > > > > possible? > > > > Hi Softworkz, > > > > I didn't reproduce your issue, maybe the issue has been fixed on > > master. How > > about to merge this patchset first? I will rebase the patchset if no > > objection. > > BTW we didn't see regression in our internal testing. > > > Hi Haihao, > (sending a CC to the ML allowing others to follow) > > I found an example for the cases I was referring to. > > An example command line is this: > > ffmpeg.exe -y -loglevel verbose -init_hw_device > "qsv=qd1:hw_any,child_device=1,child_device_type=d3d11va" -hwaccel qsv -c:v > h264_qsv -canvas_size:s:0 "1920:1080" -i red2.mkv -filter_complex > "[0:0]vpp_qsv[vid];[0:3]format=nv12,hwupload@f1=extra_hw_frames=16,vpp_qsv=w=4 > 00:h=200[subs];[vid][subs]overlay_qsv[out]" -map [out] -c:v h264_qsv -b:v 5M > output.mp4 > > It contains two uses of vpp_qsv: one in “passthrough mode” and > the other one with scaling. > Only when replacing BOTH usages of vpp_qsv with scale_qsv, > the command will work. > > (test file on request) Thanks for the info, I'll look into it. BRs Haihao
This patchset clean up scale_qsv and deinterlace_qsv filters, and take the two filters as the special cases of vpp_qsv, so vf_scale_qsv.c and vf_deinterlace_qsv.c can be deleted from FFmpeg. In addition, a few small features are added in this patchset. --- v5: * Rebased this patchset against the latest master branch and fixed conflicts Haihao Xiang (20): lavfi/qsv: use QSVVPPContext as base context in vf_vpp_qsv/vf_overlay_qsv lavfi/scale_qsv: simplify scale_qsv filter lavfi/scale_qsv: don't need variables for constants in FFmpeg lavfi/vpp_qsv: add "a", "dar" and "sar" variables lavfi/vpp_qsv: handle NULL pointer when evaluating an expression lavfi/vpp_qsv: allow special values for the output dimensions lavfi/vpp_qsv: factorize extra MFX configuration lavfi/vpp_qsv: allow user to set scale_mode with constant lavfi/vpp_qsv: add vpp_preinit callback lavfi/scale_qsv: re-use VPPContext for scale_qsv filter lavfi/vpp_qsv: factor common QSV filter definition lavfi/scale_qsv: add new options for scale_qsv filter lavfi/scale_qsv: add more input / output pixel formats lavfi/qsvvpp: avoid overriding the returned value lavfi/qsvvpp: set PTS for output frame lavfi/vpp_qsv: check output format string against NULL pointer lavfi/deinterlace_qsv: simplify deinterlace_qsv filter lavfi/deinterlace_qsv: re-use VPPContext for deinterlace_qsv filter lavfi/deinterlace_qsv: add async_depth option lavfi/deinterlace_qsv: add more input / output pixel formats libavfilter/Makefile | 4 +- libavfilter/qsvvpp.c | 57 ++- libavfilter/qsvvpp.h | 11 +- libavfilter/vf_deinterlace_qsv.c | 611 --------------------------- libavfilter/vf_overlay_qsv.c | 11 +- libavfilter/vf_scale_qsv.c | 685 ------------------------------- libavfilter/vf_vpp_qsv.c | 473 +++++++++++++-------- 7 files changed, 347 insertions(+), 1505 deletions(-) delete mode 100644 libavfilter/vf_deinterlace_qsv.c delete mode 100644 libavfilter/vf_scale_qsv.c