[FFmpeg-devel,v2,1/2] configure: Change the configure check for tonemap_vaapi

"VAProcFilterParameterBufferHDRToneMapping" was defined in libva 2.4.1, which will lead to
build failure for the filter tonemap_vaapi for libva 2.3.0 with current check. This patch
is to fix this build error.

Signed-off-by: Xinpeng Sun <xinpeng.sun@intel.com>
---
 configure | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

> -----Original Message-----
> From: Sun, Xinpeng <xinpeng.sun@intel.com>
> Sent: Monday, December 30, 2019 3:33 PM
> To: ffmpeg-devel@ffmpeg.org
> Cc: Sun, Xinpeng <xinpeng.sun@intel.com>
> Subject: [PATCH v2 1/2] configure: Change the configure check for
> tonemap_vaapi
> 
> "VAProcFilterParameterBufferHDRToneMapping" was defined in libva 2.4.1,
> which will lead to build failure for the filter tonemap_vaapi for libva 2.3.0 with
> current check. This patch is to fix this build error.
> 
> Signed-off-by: Xinpeng Sun <xinpeng.sun@intel.com>
> ---
>  configure | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/configure b/configure
> index 43dc409fe6..01b4acd77e 100755
> --- a/configure
> +++ b/configure
> @@ -3579,7 +3579,7 @@ tinterlace_filter_deps="gpl"
>  tinterlace_merge_test_deps="tinterlace_filter"
>  tinterlace_pad_test_deps="tinterlace_filter"
>  tonemap_filter_deps="const_nan"
> -tonemap_vaapi_filter_deps="vaapi
> VAProcPipelineParameterBuffer_output_hdr_metadata"
> +tonemap_vaapi_filter_deps="vaapi
> VAProcFilterParameterBufferHDRToneMapping"
>  tonemap_opencl_filter_deps="opencl const_nan"
>  transpose_opencl_filter_deps="opencl"
>  transpose_vaapi_filter_deps="vaapi VAProcPipelineCaps_rotation_flags"
> @@ -6584,7 +6584,7 @@ if enabled vaapi; then
> 
>      check_type "va/va.h va/va_dec_hevc.h" "VAPictureParameterBufferHEVC"
>      check_struct "va/va.h" "VADecPictureParameterBufferVP9" bit_depth
> -    check_struct "va/va.h va/va_vpp.h" "VAProcPipelineParameterBuffer"
> output_hdr_metadata
> +    check_type   "va/va.h va/va_vpp.h"
> "VAProcFilterParameterBufferHDRToneMapping"
>      check_struct "va/va.h va/va_vpp.h" "VAProcPipelineCaps" rotation_flags
>      check_type "va/va.h va/va_enc_hevc.h"
> "VAEncPictureParameterBufferHEVC"
>      check_type "va/va.h va/va_enc_jpeg.h" "VAEncPictureParameterBufferJPEG"
> --
> 2.17.1

Ping.

Thanks,
Xinpeng

On 30/12/2019 07:32, Xinpeng Sun wrote:
> "VAProcFilterParameterBufferHDRToneMapping" was defined in libva 2.4.1, which will lead to
> build failure for the filter tonemap_vaapi for libva 2.3.0 with current check. This patch
> is to fix this build error.
> 
> Signed-off-by: Xinpeng Sun <xinpeng.sun@intel.com>
> ---
>  configure | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/configure b/configure
> index 43dc409fe6..01b4acd77e 100755
> --- a/configure
> +++ b/configure
> @@ -3579,7 +3579,7 @@ tinterlace_filter_deps="gpl"
>  tinterlace_merge_test_deps="tinterlace_filter"
>  tinterlace_pad_test_deps="tinterlace_filter"
>  tonemap_filter_deps="const_nan"
> -tonemap_vaapi_filter_deps="vaapi VAProcPipelineParameterBuffer_output_hdr_metadata"
> +tonemap_vaapi_filter_deps="vaapi VAProcFilterParameterBufferHDRToneMapping"
>  tonemap_opencl_filter_deps="opencl const_nan"
>  transpose_opencl_filter_deps="opencl"
>  transpose_vaapi_filter_deps="vaapi VAProcPipelineCaps_rotation_flags"
> @@ -6584,7 +6584,7 @@ if enabled vaapi; then
>  
>      check_type "va/va.h va/va_dec_hevc.h" "VAPictureParameterBufferHEVC"
>      check_struct "va/va.h" "VADecPictureParameterBufferVP9" bit_depth
> -    check_struct "va/va.h va/va_vpp.h" "VAProcPipelineParameterBuffer" output_hdr_metadata
> +    check_type   "va/va.h va/va_vpp.h" "VAProcFilterParameterBufferHDRToneMapping"
>      check_struct "va/va.h va/va_vpp.h" "VAProcPipelineCaps" rotation_flags
>      check_type "va/va.h va/va_enc_hevc.h" "VAEncPictureParameterBufferHEVC"
>      check_type "va/va.h va/va_enc_jpeg.h" "VAEncPictureParameterBufferJPEG"
> 

Thanks, applied.

Relatedly: can you explain how to get this filter to work?

So far the only non-identity transformation I have managed to achieve is to make the output entirely black (which happens if the set the output colour matrix to anything other than the same as the input).

E.g. given an input which is:

    Stream #0:0[0x101]: Video: hevc (Main 10) ([36][0][0][0] / 0x0024), yuv420p10le(tv, bt2020nc/bt2020/smpte2084), 3840x2160 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 25 tbc
[Parsed_showinfo_0 @ 0x5609cc1a6d00]   side data - mastering display: has_primaries:1 has_luminance:1 r(0.6800,0.3200) g(0.2650,0.6900) b(0.1500 0.0600) wp(0.3127, 0.3290) min_luminance=0.050000, max_luminance=1200.000000

then doing:

-vf tonemap_vaapi=format=nv12

gives identical output to:

-vf scale_vaapi=format=nv12

while:

-vf tonemap_vaapi=format=nv12:t=bt709:p=bt709:m=bt709

gives output which is entirely black.

(Hardware is Icelake 1065G7.)

- Mark

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Mark
> Thompson
> Sent: Tuesday, January 7, 2020 8:07 AM
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v2 1/2] configure: Change the configure
> check for tonemap_vaapi
> 
> On 30/12/2019 07:32, Xinpeng Sun wrote:
> > "VAProcFilterParameterBufferHDRToneMapping" was defined in libva
> > 2.4.1, which will lead to build failure for the filter tonemap_vaapi
> > for libva 2.3.0 with current check. This patch is to fix this build error.
> >
> > Signed-off-by: Xinpeng Sun <xinpeng.sun@intel.com>
> > ---
> >  configure | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git a/configure b/configure
> > index 43dc409fe6..01b4acd77e 100755
> > --- a/configure
> > +++ b/configure
> > @@ -3579,7 +3579,7 @@ tinterlace_filter_deps="gpl"
> >  tinterlace_merge_test_deps="tinterlace_filter"
> >  tinterlace_pad_test_deps="tinterlace_filter"
> >  tonemap_filter_deps="const_nan"
> > -tonemap_vaapi_filter_deps="vaapi
> VAProcPipelineParameterBuffer_output_hdr_metadata"
> > +tonemap_vaapi_filter_deps="vaapi
> VAProcFilterParameterBufferHDRToneMapping"
> >  tonemap_opencl_filter_deps="opencl const_nan"
> >  transpose_opencl_filter_deps="opencl"
> >  transpose_vaapi_filter_deps="vaapi VAProcPipelineCaps_rotation_flags"
> > @@ -6584,7 +6584,7 @@ if enabled vaapi; then
> >
> >      check_type "va/va.h va/va_dec_hevc.h" "VAPictureParameterBufferHEVC"
> >      check_struct "va/va.h" "VADecPictureParameterBufferVP9" bit_depth
> > -    check_struct "va/va.h va/va_vpp.h" "VAProcPipelineParameterBuffer"
> output_hdr_metadata
> > +    check_type   "va/va.h va/va_vpp.h"
> "VAProcFilterParameterBufferHDRToneMapping"
> >      check_struct "va/va.h va/va_vpp.h" "VAProcPipelineCaps" rotation_flags
> >      check_type "va/va.h va/va_enc_hevc.h"
> "VAEncPictureParameterBufferHEVC"
> >      check_type "va/va.h va/va_enc_jpeg.h"
> "VAEncPictureParameterBufferJPEG"
> >
> 
> Thanks, applied.
> 
> Relatedly: can you explain how to get this filter to work?
> 
> So far the only non-identity transformation I have managed to achieve is to
> make the output entirely black (which happens if the set the output colour
> matrix to anything other than the same as the input).
> 
> E.g. given an input which is:
> 
>     Stream #0:0[0x101]: Video: hevc (Main 10) ([36][0][0][0] / 0x0024),
> yuv420p10le(tv, bt2020nc/bt2020/smpte2084), 3840x2160 [SAR 1:1 DAR 16:9],
> 25 fps, 25 tbr, 90k tbn, 25 tbc
> [Parsed_showinfo_0 @ 0x5609cc1a6d00]   side data - mastering display:
> has_primaries:1 has_luminance:1 r(0.6800,0.3200) g(0.2650,0.6900) b(0.1500
> 0.0600) wp(0.3127, 0.3290) min_luminance=0.050000,
> max_luminance=1200.000000
> 
> then doing:
> 
> -vf tonemap_vaapi=format=nv12
> 
> gives identical output to:
> 
> -vf scale_vaapi=format=nv12
> 
> while:
> 
> -vf tonemap_vaapi=format=nv12:t=bt709:p=bt709:m=bt709
> 
> gives output which is entirely black.
> 
> (Hardware is Icelake 1065G7.)
> 
> - Mark

Hi Mark,

For your first test(tonemap_vaapi has the same output with scale_vaapi), I am working with others to investigate the root-cause in iHD driver.

And for your second test(the output is black), scale_vaapi also has similar issue when performing bt2020 to bt709/bt601 on the color matrix for 4K resolution. 
It’s a driver issue and I think they may share the same root-cause.
The issue link is:  https://github.com/intel/media-driver/issues/760
Same issue from gstreamer-vaapi side: https://github.com/intel/media-driver/issues/763

I'm so sorry for forgetting to test the bt709-color-matrix for output when developing the patch.

Regards,
Xinpeng
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org
> with subject "unsubscribe".

Hi Mark,

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Sun,
> Xinpeng
> Sent: Wednesday, January 8, 2020 8:02 PM
> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH v2 1/2] configure: Change the configure
> check for tonemap_vaapi
> 
> 
> > -----Original Message-----
> > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Mark
> > Thompson
> > Sent: Tuesday, January 7, 2020 8:07 AM
> > To: ffmpeg-devel@ffmpeg.org
> > Subject: Re: [FFmpeg-devel] [PATCH v2 1/2] configure: Change the
> > configure check for tonemap_vaapi
> >
> > On 30/12/2019 07:32, Xinpeng Sun wrote:
> > > "VAProcFilterParameterBufferHDRToneMapping" was defined in libva
> > > 2.4.1, which will lead to build failure for the filter tonemap_vaapi
> > > for libva 2.3.0 with current check. This patch is to fix this build error.
> > >
> > > Signed-off-by: Xinpeng Sun <xinpeng.sun@intel.com>
> > > ---
> > >  configure | 4 ++--
> > >  1 file changed, 2 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/configure b/configure
> > > index 43dc409fe6..01b4acd77e 100755
> > > --- a/configure
> > > +++ b/configure
> > > @@ -3579,7 +3579,7 @@ tinterlace_filter_deps="gpl"
> > >  tinterlace_merge_test_deps="tinterlace_filter"
> > >  tinterlace_pad_test_deps="tinterlace_filter"
> > >  tonemap_filter_deps="const_nan"
> > > -tonemap_vaapi_filter_deps="vaapi
> > VAProcPipelineParameterBuffer_output_hdr_metadata"
> > > +tonemap_vaapi_filter_deps="vaapi
> > VAProcFilterParameterBufferHDRToneMapping"
> > >  tonemap_opencl_filter_deps="opencl const_nan"
> > >  transpose_opencl_filter_deps="opencl"
> > >  transpose_vaapi_filter_deps="vaapi VAProcPipelineCaps_rotation_flags"
> > > @@ -6584,7 +6584,7 @@ if enabled vaapi; then
> > >
> > >      check_type "va/va.h va/va_dec_hevc.h"
> "VAPictureParameterBufferHEVC"
> > >      check_struct "va/va.h" "VADecPictureParameterBufferVP9" bit_depth
> > > -    check_struct "va/va.h va/va_vpp.h" "VAProcPipelineParameterBuffer"
> > output_hdr_metadata
> > > +    check_type   "va/va.h va/va_vpp.h"
> > "VAProcFilterParameterBufferHDRToneMapping"
> > >      check_struct "va/va.h va/va_vpp.h" "VAProcPipelineCaps" rotation_flags
> > >      check_type "va/va.h va/va_enc_hevc.h"
> > "VAEncPictureParameterBufferHEVC"
> > >      check_type "va/va.h va/va_enc_jpeg.h"
> > "VAEncPictureParameterBufferJPEG"
> > >
> >
> > Thanks, applied.
> >
> > Relatedly: can you explain how to get this filter to work?
> >
> > So far the only non-identity transformation I have managed to achieve
> > is to make the output entirely black (which happens if the set the
> > output colour matrix to anything other than the same as the input).
> >
> > E.g. given an input which is:
> >
> >     Stream #0:0[0x101]: Video: hevc (Main 10) ([36][0][0][0] /
> > 0x0024), yuv420p10le(tv, bt2020nc/bt2020/smpte2084), 3840x2160 [SAR
> > 1:1 DAR 16:9],
> > 25 fps, 25 tbr, 90k tbn, 25 tbc
> > [Parsed_showinfo_0 @ 0x5609cc1a6d00]   side data - mastering display:
> > has_primaries:1 has_luminance:1 r(0.6800,0.3200) g(0.2650,0.6900)
> > b(0.1500
> > 0.0600) wp(0.3127, 0.3290) min_luminance=0.050000,
> > max_luminance=1200.000000
> >
> > then doing:
> >
> > -vf tonemap_vaapi=format=nv12
> >
> > gives identical output to:
> >
> > -vf scale_vaapi=format=nv12
> >
> > while:
> >
> > -vf tonemap_vaapi=format=nv12:t=bt709:p=bt709:m=bt709
> >
> > gives output which is entirely black.
> >
> > (Hardware is Icelake 1065G7.)
> >
> > - Mark
> 
> Hi Mark,
> 
> For your first test(tonemap_vaapi has the same output with scale_vaapi), I am
> working with others to investigate the root-cause in iHD driver.
> 

iHD driver didn't add VAProcColorStandardExplicit into the query list before, so ff_vaapi_vpp_init_params cannot 
query it from driver and set the color standard to VAProcColorStandardExplicit. Only when the color standard is 
VAProcColorStandardExplicit, the transfer characteristic can be set to SMPTE 2084 in the driver. If the transfer of 
input is not SMPTE 2084, driver will process the input in the same way as scale_vaapi. I sent a patch to iHD driver 
to fix this query issue and it has been merged as the commit 73bcc3415e32fa0697e384aa1f301a8b56327bc1.

Another thing needs to be fixed in tonemap_vaapi is to pass the filter parameters to VAPocPipelineParameterBuffer.
I have sent a patch to cover this: 
https://patchwork.ffmpeg.org/project/ffmpeg/patch/20200117035650.16416-1-xinpeng.sun@intel.com/

Could you help to review and test it? BTW, this filter has been supported on CFL/KBL/ICL/TGL/DG1.

> And for your second test(the output is black), scale_vaapi also has similar issue
> when performing bt2020 to bt709/bt601 on the color matrix for 4K resolution.
> It’s a driver issue and I think they may share the same root-cause.
> The issue link is:  https://github.com/intel/media-driver/issues/760
> Same issue from gstreamer-vaapi side: https://github.com/intel/media-
> driver/issues/763
> 

After the above changes in tonemap_vaapi and iHD dirver, the tonemap_vaapi filter won't have this issue for 4K 
resolution, but scale_vaapi still has this issue.

> I'm so sorry for forgetting to test the bt709-color-matrix for output when
> developing the patch.
> 
> Regards,
> Xinpeng
> > _______________________________________________
> > ffmpeg-devel mailing list
> > ffmpeg-devel@ffmpeg.org
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >
> > To unsubscribe, visit link above, or email
> > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org
> with subject "unsubscribe".