diff mbox

[FFmpeg-devel] vdpau: do not use buggy HEVC support by default

Message ID 20170701094038.23587-1-nfxjfg@googlemail.com
State Accepted
Commit 64ecb78b7179cab2dbdf835463104679dbb7c895
Headers show

Commit Message

wm4 July 1, 2017, 9:40 a.m. UTC
NVIDIA broke its own API when using VDPAU decoding. If you retrieve the
decoded YUV data, or if you map the surfaces with GL interop, the result
are interlacing artifacts. The only way to get non-broken data is by
using the vdpau video mixer to convert it to RGB. There is no way to
block the non-working operations in a reasonable way (a VdpVideoSurface
has to support all operations).

NVIDIA refuses to fix this issue (it "fixed" it by making it work with
the video mixer, but the rest is still broken). There is no sign of that
changing.

Do not use HEVC by default with the generic hwaccle API. Detect whether
it's the NVIDIA native implementation, and exit with an error. (The same
thing work with the MESA implementation.)

As an escape hatch and to allow applications to use the decoder if they
really want to (perhaps because they make sure to explicitly use the
video mixer), reuse AV_HWACCEL_FLAG_ALLOW_PROFILE_MISMATCH to disable
this check.

Once NVIDIA fixes the bug, working driver versions could be detected,
and it could be allowed again.
---
 libavcodec/vdpau.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

Comments

Philip Langdale July 1, 2017, 2:10 p.m. UTC | #1
On Sat,  1 Jul 2017 11:40:38 +0200
wm4 <nfxjfg@googlemail.com> wrote:

> NVIDIA broke its own API when using VDPAU decoding. If you retrieve
> the decoded YUV data, or if you map the surfaces with GL interop, the
> result are interlacing artifacts. The only way to get non-broken data
> is by using the vdpau video mixer to convert it to RGB. There is no
> way to block the non-working operations in a reasonable way (a
> VdpVideoSurface has to support all operations).
> 
> NVIDIA refuses to fix this issue (it "fixed" it by making it work with
> the video mixer, but the rest is still broken). There is no sign of
> that changing.
> 
> Do not use HEVC by default with the generic hwaccle API. Detect
> whether it's the NVIDIA native implementation, and exit with an
> error. (The same thing work with the MESA implementation.)
> 
> As an escape hatch and to allow applications to use the decoder if
> they really want to (perhaps because they make sure to explicitly use
> the video mixer), reuse AV_HWACCEL_FLAG_ALLOW_PROFILE_MISMATCH to
> disable this check.
> 
> Once NVIDIA fixes the bug, working driver versions could be detected,
> and it could be allowed again.
> ---
>  libavcodec/vdpau.c | 19 +++++++++++++++++++
>  1 file changed, 19 insertions(+)
> 
> diff --git a/libavcodec/vdpau.c b/libavcodec/vdpau.c
> index 9c7804a287..42ebddbeee 100644
> --- a/libavcodec/vdpau.c
> +++ b/libavcodec/vdpau.c
> @@ -127,6 +127,8 @@ int ff_vdpau_common_init(AVCodecContext *avctx,
> VdpDecoderProfile profile, VdpVideoSurfaceQueryCapabilities
> *surface_query_caps; VdpDecoderQueryCapabilities *decoder_query_caps;
>      VdpDecoderCreate *create;
> +    VdpGetInformationString *info;
> +    const char *info_string;
>      void *func;
>      VdpStatus status;
>      VdpBool supported;
> @@ -209,6 +211,23 @@ int ff_vdpau_common_init(AVCodecContext *avctx,
> VdpDecoderProfile profile, return AVERROR(ENOTSUP);
>  
>      status = vdctx->get_proc_address(vdctx->device,
> +
> VDP_FUNC_ID_GET_INFORMATION_STRING,
> +                                     &func);
> +    if (status != VDP_STATUS_OK)
> +        return vdpau_error(status);
> +    else
> +        info = func;
> +
> +    status = info(&info_string);
> +    if (status != VDP_STATUS_OK)
> +        return vdpau_error(status);
> +    if (avctx->codec_id == AV_CODEC_ID_HEVC && strncmp(info_string,
> "NVIDIA ", 7) == 0 &&
> +        !(avctx->hwaccel_flags &
> AV_HWACCEL_FLAG_ALLOW_PROFILE_MISMATCH)) {
> +        av_log(avctx, AV_LOG_VERBOSE, "HEVC with NVIDIA VDPAU
> drivers is buggy, skipping.\n");
> +        return AVERROR(ENOTSUP);
> +    }
> +
> +    status = vdctx->get_proc_address(vdctx->device,
>                                       VDP_FUNC_ID_VIDEO_SURFACE_QUERY_CAPABILITIES,
>                                       &func);
>      if (status != VDP_STATUS_OK)

Go for it.

--phil
wm4 July 3, 2017, 10:59 a.m. UTC | #2
On Sat, 1 Jul 2017 07:10:47 -0700
Philip Langdale <philipl@overt.org> wrote:

> On Sat,  1 Jul 2017 11:40:38 +0200
> wm4 <nfxjfg@googlemail.com> wrote:
> 
> > NVIDIA broke its own API when using VDPAU decoding. If you retrieve
> > the decoded YUV data, or if you map the surfaces with GL interop, the
> > result are interlacing artifacts. The only way to get non-broken data
> > is by using the vdpau video mixer to convert it to RGB. There is no
> > way to block the non-working operations in a reasonable way (a
> > VdpVideoSurface has to support all operations).
> > 
> > NVIDIA refuses to fix this issue (it "fixed" it by making it work with
> > the video mixer, but the rest is still broken). There is no sign of
> > that changing.
> > 
> > Do not use HEVC by default with the generic hwaccle API. Detect
> > whether it's the NVIDIA native implementation, and exit with an
> > error. (The same thing work with the MESA implementation.)
> > 
> > As an escape hatch and to allow applications to use the decoder if
> > they really want to (perhaps because they make sure to explicitly use
> > the video mixer), reuse AV_HWACCEL_FLAG_ALLOW_PROFILE_MISMATCH to
> > disable this check.
> > 
> > Once NVIDIA fixes the bug, working driver versions could be detected,
> > and it could be allowed again.
> > ---
> >  libavcodec/vdpau.c | 19 +++++++++++++++++++
> >  1 file changed, 19 insertions(+)
> > 
> > diff --git a/libavcodec/vdpau.c b/libavcodec/vdpau.c
> > index 9c7804a287..42ebddbeee 100644
> > --- a/libavcodec/vdpau.c
> > +++ b/libavcodec/vdpau.c
> > @@ -127,6 +127,8 @@ int ff_vdpau_common_init(AVCodecContext *avctx,
> > VdpDecoderProfile profile, VdpVideoSurfaceQueryCapabilities
> > *surface_query_caps; VdpDecoderQueryCapabilities *decoder_query_caps;
> >      VdpDecoderCreate *create;
> > +    VdpGetInformationString *info;
> > +    const char *info_string;
> >      void *func;
> >      VdpStatus status;
> >      VdpBool supported;
> > @@ -209,6 +211,23 @@ int ff_vdpau_common_init(AVCodecContext *avctx,
> > VdpDecoderProfile profile, return AVERROR(ENOTSUP);
> >  
> >      status = vdctx->get_proc_address(vdctx->device,
> > +
> > VDP_FUNC_ID_GET_INFORMATION_STRING,
> > +                                     &func);
> > +    if (status != VDP_STATUS_OK)
> > +        return vdpau_error(status);
> > +    else
> > +        info = func;
> > +
> > +    status = info(&info_string);
> > +    if (status != VDP_STATUS_OK)
> > +        return vdpau_error(status);
> > +    if (avctx->codec_id == AV_CODEC_ID_HEVC && strncmp(info_string,
> > "NVIDIA ", 7) == 0 &&
> > +        !(avctx->hwaccel_flags &
> > AV_HWACCEL_FLAG_ALLOW_PROFILE_MISMATCH)) {
> > +        av_log(avctx, AV_LOG_VERBOSE, "HEVC with NVIDIA VDPAU
> > drivers is buggy, skipping.\n");
> > +        return AVERROR(ENOTSUP);
> > +    }
> > +
> > +    status = vdctx->get_proc_address(vdctx->device,
> >                                       VDP_FUNC_ID_VIDEO_SURFACE_QUERY_CAPABILITIES,
> >                                       &func);
> >      if (status != VDP_STATUS_OK)  
> 
> Go for it.

And pushed.
Yogender Gupta July 14, 2017, 8:19 a.m. UTC | #3
Please help to understand what is broken with a command line, so that we can address this issue.

Thanks,
Yogender

-----Original Message-----
From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On Behalf Of wm4

Sent: Monday, July 03, 2017 4:29 PM
To: ffmpeg-devel@ffmpeg.org
Subject: Re: [FFmpeg-devel] [PATCH] vdpau: do not use buggy HEVC support by default

On Sat, 1 Jul 2017 07:10:47 -0700
Philip Langdale <philipl@overt.org> wrote:

> On Sat,  1 Jul 2017 11:40:38 +0200

> wm4 <nfxjfg@googlemail.com> wrote:

> 

> > NVIDIA broke its own API when using VDPAU decoding. If you retrieve 

> > the decoded YUV data, or if you map the surfaces with GL interop, 

> > the result are interlacing artifacts. The only way to get non-broken 

> > data is by using the vdpau video mixer to convert it to RGB. There 

> > is no way to block the non-working operations in a reasonable way (a 

> > VdpVideoSurface has to support all operations).

> > 

> > NVIDIA refuses to fix this issue (it "fixed" it by making it work 

> > with the video mixer, but the rest is still broken). There is no 

> > sign of that changing.

> > 

> > Do not use HEVC by default with the generic hwaccle API. Detect 

> > whether it's the NVIDIA native implementation, and exit with an 

> > error. (The same thing work with the MESA implementation.)

> > 

> > As an escape hatch and to allow applications to use the decoder if 

> > they really want to (perhaps because they make sure to explicitly 

> > use the video mixer), reuse AV_HWACCEL_FLAG_ALLOW_PROFILE_MISMATCH 

> > to disable this check.

> > 

> > Once NVIDIA fixes the bug, working driver versions could be 

> > detected, and it could be allowed again.

> > ---

> >  libavcodec/vdpau.c | 19 +++++++++++++++++++

> >  1 file changed, 19 insertions(+)

> > 

> > diff --git a/libavcodec/vdpau.c b/libavcodec/vdpau.c index 

> > 9c7804a287..42ebddbeee 100644

> > --- a/libavcodec/vdpau.c

> > +++ b/libavcodec/vdpau.c

> > @@ -127,6 +127,8 @@ int ff_vdpau_common_init(AVCodecContext *avctx, 

> > VdpDecoderProfile profile, VdpVideoSurfaceQueryCapabilities 

> > *surface_query_caps; VdpDecoderQueryCapabilities *decoder_query_caps;

> >      VdpDecoderCreate *create;

> > +    VdpGetInformationString *info;

> > +    const char *info_string;

> >      void *func;

> >      VdpStatus status;

> >      VdpBool supported;

> > @@ -209,6 +211,23 @@ int ff_vdpau_common_init(AVCodecContext *avctx, 

> > VdpDecoderProfile profile, return AVERROR(ENOTSUP);

> >  

> >      status = vdctx->get_proc_address(vdctx->device,

> > +

> > VDP_FUNC_ID_GET_INFORMATION_STRING,

> > +                                     &func);

> > +    if (status != VDP_STATUS_OK)

> > +        return vdpau_error(status);

> > +    else

> > +        info = func;

> > +

> > +    status = info(&info_string);

> > +    if (status != VDP_STATUS_OK)

> > +        return vdpau_error(status);

> > +    if (avctx->codec_id == AV_CODEC_ID_HEVC && strncmp(info_string,

> > "NVIDIA ", 7) == 0 &&

> > +        !(avctx->hwaccel_flags &

> > AV_HWACCEL_FLAG_ALLOW_PROFILE_MISMATCH)) {

> > +        av_log(avctx, AV_LOG_VERBOSE, "HEVC with NVIDIA VDPAU

> > drivers is buggy, skipping.\n");

> > +        return AVERROR(ENOTSUP);

> > +    }

> > +

> > +    status = vdctx->get_proc_address(vdctx->device,

> >                                       VDP_FUNC_ID_VIDEO_SURFACE_QUERY_CAPABILITIES,

> >                                       &func);

> >      if (status != VDP_STATUS_OK)

> 

> Go for it.


And pushed.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

-----------------------------------------------------------------------------------
This email message is for the sole use of the intended recipient(s) and may contain
confidential information.  Any unauthorized review, use, disclosure or distribution
is prohibited.  If you are not the intended recipient, please contact the sender by
reply email and destroy all copies of the original message.
-----------------------------------------------------------------------------------
wm4 July 14, 2017, 9:26 a.m. UTC | #4
On Fri, 14 Jul 2017 08:19:25 +0000
Yogender Gupta <ygupta@nvidia.com> wrote:

> Please help to understand what is broken with a command line, so that we can address this issue.

With ffmpeg CLI, you can try something like:

  ffmpeg -hwaccel vdpau -i some_8bit_hevc_file.mkv -frames 10 -pix_fmt yuv420p out.y4m

You will see quite obvious artifacts, that look like interlacing gone
wrong.

GL_NV_vdpau_interop has a similar bug. You can test that with e.g.:

  mpv --hwdec=vdpau some_8bit_hevc_file.mkv

(other software which uses this extension to map VdpVideoSurface as GL
textures should be affected too.)
Philip Langdale July 14, 2017, 9:33 a.m. UTC | #5
On 2017-07-14 08:19, Yogender Gupta wrote:
> Please help to understand what is broken with a command line, so that
> we can address this issue.

This is the issue that the hevc low level decoder outputs frames instead
of fields - which all the other decoders do.

Originally, the vdpau support didn't account for this at all - so it 
would
take the memory allocated for the progressive frame and then treat it as
two fields and return those two fields up (because vdpau assumes output
frames are made up of two fields too) which lead to garbage output, as 
it
would either return the top and bottom half in each 'field' if using the
opengl interop, or it would interleave the two 'fields' when returning a
frame using the read-back API or the vdpau mixer.

I pointed this out to the vdpau maintainer (back when he still worked 
for
nvidia) and he fixed the vdpau mixer case - but read-back and opengl 
remained
broken. He then left nvidia and no work has happened ever since - that 
was
almost two years ago.

--phil
ManojGuptaBonda July 14, 2017, 9:55 a.m. UTC | #6
Hi,

Thanks i'll look into the issue and come up with an ETA  ASAP. 
Going on I will be the maintainer of VDPAU. Please direct any VDPAU issues to me.

Thanks,
ManojGupta.

-----Original Message-----
From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On Behalf Of Philip Langdale

Sent: Friday, July 14, 2017 3:04 PM
To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
Cc: Yogender Gupta <ygupta@nvidia.com>
Subject: Re: [FFmpeg-devel] [PATCH] vdpau: do not use buggy HEVC support by default

On 2017-07-14 08:19, Yogender Gupta wrote:
> Please help to understand what is broken with a command line, so that 

> we can address this issue.


This is the issue that the hevc low level decoder outputs frames instead of fields - which all the other decoders do.

Originally, the vdpau support didn't account for this at all - so it would take the memory allocated for the progressive frame and then treat it as two fields and return those two fields up (because vdpau assumes output frames are made up of two fields too) which lead to garbage output, as it would either return the top and bottom half in each 'field' if using the opengl interop, or it would interleave the two 'fields' when returning a frame using the read-back API or the vdpau mixer.

I pointed this out to the vdpau maintainer (back when he still worked for
nvidia) and he fixed the vdpau mixer case - but read-back and opengl remained broken. He then left nvidia and no work has happened ever since - that was almost two years ago.

--phil
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

-----------------------------------------------------------------------------------
This email message is for the sole use of the intended recipient(s) and may contain
confidential information.  Any unauthorized review, use, disclosure or distribution
is prohibited.  If you are not the intended recipient, please contact the sender by
reply email and destroy all copies of the original message.
-----------------------------------------------------------------------------------
wm4 July 14, 2017, 10:24 a.m. UTC | #7
On Fri, 14 Jul 2017 09:55:58 +0000
Manoj Bonda <mbonda@nvidia.com> wrote:

> Hi,
> 
> Thanks i'll look into the issue and come up with an ETA  ASAP. 
> Going on I will be the maintainer of VDPAU. Please direct any VDPAU issues to me.

That's great to hear. Is there a bug tracker for this? Other than
fixing the bug that prompted my patch, on top of my wish list would be:

 - 10 bit surface support
 - mapping GL surfaces as progressive frames instead of fields
 - getting rid of display preemption
ManojGuptaBonda July 14, 2017, 10:36 a.m. UTC | #8
>That's great to hear. Is there a bug tracker for this?

Please find the Bug ID 200328360 for above.

> - 10 bit surface support

 - mapping GL surfaces as progressive frames instead of fields
 - getting rid of display preemption

Will add these to feature request List and will take them up along with other tasks based on the priority 

Thanks,
ManojGupta.


-----Original Message-----
From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On Behalf Of wm4

Sent: Friday, July 14, 2017 3:54 PM
To: ffmpeg-devel@ffmpeg.org
Subject: Re: [FFmpeg-devel] [PATCH] vdpau: do not use buggy HEVC support by default

On Fri, 14 Jul 2017 09:55:58 +0000
Manoj Bonda <mbonda@nvidia.com> wrote:

> Hi,

> 

> Thanks i'll look into the issue and come up with an ETA  ASAP. 

> Going on I will be the maintainer of VDPAU. Please direct any VDPAU issues to me.


That's great to hear. Is there a bug tracker for this? Other than fixing the bug that prompted my patch, on top of my wish list would be:

 - 10 bit surface support
 - mapping GL surfaces as progressive frames instead of fields
 - getting rid of display preemption
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

-----------------------------------------------------------------------------------
This email message is for the sole use of the intended recipient(s) and may contain
confidential information.  Any unauthorized review, use, disclosure or distribution
is prohibited.  If you are not the intended recipient, please contact the sender by
reply email and destroy all copies of the original message.
-----------------------------------------------------------------------------------
ManojGuptaBonda Aug. 8, 2017, 5:15 a.m. UTC | #9
Hi , 

HEVC issue for read-back API has been fixed and will be part of the upcoming drivers. 
Please help us understand the issue with the open gl interop. 
As per our understanding we are mapping the video surface to gl using the gl-interop
and the app(mpv) will be doing the merging/de-interlacing part.

As per our understanding, in mpv we see merging/de-interlacing is being done using shader at

Call stack:
               gl_sc_generate() at utils.c:1,162 0x565daa            
               finish_pass_direct() at video.c:1,115 0x569cb2    
               reinterleave_vdpau() at video.c:3,031 0x57277a 
               pass_upload_image() at video.c:3,079 0x572b6b                
               pass_render_frame() at video.c:2,506 0x570162 
               gl_video_render_frame() at video.c:2,877 0x571ce9         
               draw_frame() at vo_opengl.c:133 0x57d920         
               render_frame() at vo.c:817 0x579113      
               vo_thread() at vo.c:916 0x579610

we are not able to get how ffmpeg is using the vdpau-opengl interop. 
Please suggest us how to repro vdpau-opengl  interop issue with ffmpeg. 

Please help us understand what needs to be done from our side for resolving this issue.
Please do let us know if any our understandings are wrong.

Thanks,
ManojGupta

-----------------------------------------------------------------------------------
This email message is for the sole use of the intended recipient(s) and may contain
confidential information.  Any unauthorized review, use, disclosure or distribution
is prohibited.  If you are not the intended recipient, please contact the sender by
reply email and destroy all copies of the original message.
-----------------------------------------------------------------------------------
Philip Langdale Aug. 9, 2017, 2:13 a.m. UTC | #10
On Tue, 8 Aug 2017 05:15:52 +0000
Manoj Bonda <mbonda@nvidia.com> wrote:

> Hi , 
> 
> HEVC issue for read-back API has been fixed and will be part of the
> upcoming drivers. Please help us understand the issue with the open
> gl interop. As per our understanding we are mapping the video surface
> to gl using the gl-interop and the app(mpv) will be doing the
> merging/de-interlacing part.
> 
> As per our understanding, in mpv we see merging/de-interlacing is
> being done using shader at
> 
> Call stack:
>                gl_sc_generate() at utils.c:1,162 0x565daa            
>                finish_pass_direct() at video.c:1,115 0x569cb2    
>                reinterleave_vdpau() at video.c:3,031 0x57277a 
>                pass_upload_image() at video.c:3,079
> 0x572b6b pass_render_frame() at video.c:2,506 0x570162 
>                gl_video_render_frame() at video.c:2,877
> 0x571ce9 draw_frame() at vo_opengl.c:133 0x57d920         
>                render_frame() at vo.c:817 0x579113      
>                vo_thread() at vo.c:916 0x579610
> 
> we are not able to get how ffmpeg is using the vdpau-opengl interop. 
> Please suggest us how to repro vdpau-opengl  interop issue with
> ffmpeg. 
> 
> Please help us understand what needs to be done from our side for
> resolving this issue. Please do let us know if any our understandings
> are wrong.

It's the same issue. The GL interop API returns video data as fields, so
the field splitting for hevc is done incorrectly, just as it was
incorrect for the read-back API. Presumably, depending on how the driver
code is written, you may have needed to fix only a single spot, but I'm
guessing you probably need to fix the GL case separately, given that
fixing the vdpau mixer didn't fix either of the other two paths.

There is no repro with ffmpeg as it does not use the GL interop.

The mpv code does field merging, but that is what any client using
vdpau gl interop would do. You could reproduce this in any client.

And just because we're discussing this, we'd all love it if there was a
new mode where the interop surfaces were frame, rather than field based.

Thanks,

--phil
wm4 Aug. 18, 2017, 10:49 a.m. UTC | #11
On Tue, 8 Aug 2017 05:15:52 +0000
Manoj Bonda <mbonda@nvidia.com> wrote:

> Hi , 
> 
> HEVC issue for read-back API has been fixed and will be part of the upcoming drivers. 
> Please help us understand the issue with the open gl interop. 

That's great to hear. (Sorry, saw this only now...)

> As per our understanding we are mapping the video surface to gl using the gl-interop
> and the app(mpv) will be doing the merging/de-interlacing part.
> 
> As per our understanding, in mpv we see merging/de-interlacing is being done using shader at
> 
> Call stack:
>                gl_sc_generate() at utils.c:1,162 0x565daa            
>                finish_pass_direct() at video.c:1,115 0x569cb2    
>                reinterleave_vdpau() at video.c:3,031 0x57277a 
>                pass_upload_image() at video.c:3,079 0x572b6b                
>                pass_render_frame() at video.c:2,506 0x570162 
>                gl_video_render_frame() at video.c:2,877 0x571ce9         
>                draw_frame() at vo_opengl.c:133 0x57d920         
>                render_frame() at vo.c:817 0x579113      
>                vo_thread() at vo.c:916 0x579610

Yes, the API requires this. But with HEVC the driver returned pretty
broken textures.

Here's the extension definition:

https://www.khronos.org/registry/OpenGL/extensions/NV/NV_vdpau_interop.txt

It contains this as part of a table:

      VDP_CHROMA_TYPE_420  4                0      w   x h/2  R8        Top-field luma
                                            1      w   x h/2  R8        Bottom-field luma
As you can see, the API _always_ returns video as separate fields. The
top field consists of all even lines, while the bottom field of all odd
lines (or maybe it was the other way around).

But with HEVC, the top field literally returned the top half of the
video frame, and bottom field the second half.

But ideally, nvidia would provide a new GL interop extension, which
does not require this interlacing nonsense. (It could support high bit
depth surfaces too.)

> we are not able to get how ffmpeg is using the vdpau-opengl interop. 
> Please suggest us how to repro vdpau-opengl  interop issue with ffmpeg. 

FFmpeg doesn't use vdpau-opengl interop, so strictly speaking this
discussion is offtopic here. What we saw for readback was pretty
similar to what we saw on the textures, though.
ManojGuptaBonda Oct. 24, 2018, 9:35 a.m. UTC | #12
Hi ,

The bug reported earlier that caused VDPAU-decoded H.265/HEVC content to have interlaced artifacts, when
shared with OpenGL through the GL_NV_vdpau_interop extension is fixed in 410.66 driver release.

As requested in this thread, NVIDIA is now proposing a new interop
extension, where the interop surfaces are frame or field based.
The corresponding VDPAU API change is sent out to VDPAU mailing list for review.

Detailed VDPAU API proposal is available at -
https://lists.freedesktop.org/archives/vdpau/2018-October/000415.html

The proposed new VDPAU OpenGL interop spec (NV_vdpau_inteop2.txt) is available at 
https://github.com/KhronosGroup/OpenGL-Registry/pull/223/files 
for reference.

Please review both the proposals and provide your feedback.

Thanks,
ManojGupta.


-----Original Message-----
From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of wm4

Sent: Friday, August 18, 2017 4:20 PM
To: ffmpeg-devel@ffmpeg.org
Subject: Re: [FFmpeg-devel] [PATCH] vdpau: do not use buggy HEVC support by default

On Tue, 8 Aug 2017 05:15:52 +0000
Manoj Bonda <mbonda@nvidia.com> wrote:

> Hi ,

> 

> HEVC issue for read-back API has been fixed and will be part of the upcoming drivers. 

> Please help us understand the issue with the open gl interop. 


That's great to hear. (Sorry, saw this only now...)

> As per our understanding we are mapping the video surface to gl using 

> the gl-interop and the app(mpv) will be doing the merging/de-interlacing part.

> 

> As per our understanding, in mpv we see merging/de-interlacing is 

> being done using shader at

> 

> Call stack:

>                gl_sc_generate() at utils.c:1,162 0x565daa            

>                finish_pass_direct() at video.c:1,115 0x569cb2    

>                reinterleave_vdpau() at video.c:3,031 0x57277a 

>                pass_upload_image() at video.c:3,079 0x572b6b                

>                pass_render_frame() at video.c:2,506 0x570162 

>                gl_video_render_frame() at video.c:2,877 0x571ce9         

>                draw_frame() at vo_opengl.c:133 0x57d920         

>                render_frame() at vo.c:817 0x579113      

>                vo_thread() at vo.c:916 0x579610


Yes, the API requires this. But with HEVC the driver returned pretty broken textures.

Here's the extension definition:

https://www.khronos.org/registry/OpenGL/extensions/NV/NV_vdpau_interop.txt

It contains this as part of a table:

      VDP_CHROMA_TYPE_420  4                0      w   x h/2  R8        Top-field luma
                                            1      w   x h/2  R8        Bottom-field luma
As you can see, the API _always_ returns video as separate fields. The top field consists of all even lines, while the bottom field of all odd lines (or maybe it was the other way around).

But with HEVC, the top field literally returned the top half of the video frame, and bottom field the second half.

But ideally, nvidia would provide a new GL interop extension, which does not require this interlacing nonsense. (It could support high bit depth surfaces too.)

> we are not able to get how ffmpeg is using the vdpau-opengl interop. 

> Please suggest us how to repro vdpau-opengl  interop issue with ffmpeg. 


FFmpeg doesn't use vdpau-opengl interop, so strictly speaking this discussion is offtopic here. What we saw for readback was pretty similar to what we saw on the textures, though.

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
diff mbox

Patch

diff --git a/libavcodec/vdpau.c b/libavcodec/vdpau.c
index 9c7804a287..42ebddbeee 100644
--- a/libavcodec/vdpau.c
+++ b/libavcodec/vdpau.c
@@ -127,6 +127,8 @@  int ff_vdpau_common_init(AVCodecContext *avctx, VdpDecoderProfile profile,
     VdpVideoSurfaceQueryCapabilities *surface_query_caps;
     VdpDecoderQueryCapabilities *decoder_query_caps;
     VdpDecoderCreate *create;
+    VdpGetInformationString *info;
+    const char *info_string;
     void *func;
     VdpStatus status;
     VdpBool supported;
@@ -209,6 +211,23 @@  int ff_vdpau_common_init(AVCodecContext *avctx, VdpDecoderProfile profile,
         return AVERROR(ENOTSUP);
 
     status = vdctx->get_proc_address(vdctx->device,
+                                     VDP_FUNC_ID_GET_INFORMATION_STRING,
+                                     &func);
+    if (status != VDP_STATUS_OK)
+        return vdpau_error(status);
+    else
+        info = func;
+
+    status = info(&info_string);
+    if (status != VDP_STATUS_OK)
+        return vdpau_error(status);
+    if (avctx->codec_id == AV_CODEC_ID_HEVC && strncmp(info_string, "NVIDIA ", 7) == 0 &&
+        !(avctx->hwaccel_flags & AV_HWACCEL_FLAG_ALLOW_PROFILE_MISMATCH)) {
+        av_log(avctx, AV_LOG_VERBOSE, "HEVC with NVIDIA VDPAU drivers is buggy, skipping.\n");
+        return AVERROR(ENOTSUP);
+    }
+
+    status = vdctx->get_proc_address(vdctx->device,
                                      VDP_FUNC_ID_VIDEO_SURFACE_QUERY_CAPABILITIES,
                                      &func);
     if (status != VDP_STATUS_OK)