diff mbox series

[FFmpeg-devel,2/2] lavfi/vf_vpp_qsv: fix the time_base for outlink

Message ID 20210609041830.1302044-2-haihao.xiang@intel.com
State Superseded
Headers show
Series [FFmpeg-devel,1/2] lavc/qsvdec: fix pts
Related show

Checks

Context Check Description
andriy/x86_make success Make finished
andriy/x86_make_fate success Make fate finished
andriy/PPC64_make success Make finished
andriy/PPC64_make_fate success Make fate finished

Commit Message

Xiang, Haihao June 9, 2021, 4:18 a.m. UTC
Since commit 89ffcd1, the pts on output pad is in the time base of the
input link, not the time base of the output link when EOF is reached, so
a filter after vpp_qsv might output some unexpected frames. In order to
avoid this issue, use the same time base for input and ouput links

The issue can be triggered with the command below:
$> ffmpeg -hwaccel qsv -c:v hevc_qsv -i input.h265 -vf
"vpp_qsv=w=1920:h=1080,fps=fps=60" -f null -

[out_0_0 @ 0x55eb017b5060] 100 buffers queued in out_0_0, something may
be wrong.
[out_0_0 @ 0x55eb017b5060] 1000 buffers queued in out_0_0, something may
be wrong.
[out_0_0 @ 0x55eb017b5060] 10000 buffers queued in out_0_0, something
may be wrong.
---
 libavfilter/vf_vpp_qsv.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Comments

Rogozhkin, Dmitry V June 9, 2021, 4:03 p.m. UTC | #1
On Wed, 2021-06-09 at 12:18 +0800, Haihao Xiang wrote:
> Since commit 89ffcd1, the pts on output pad is in the time base of
> the
> input link, not the time base of the output link when EOF is reached,
> so
> a filter after vpp_qsv might output some unexpected frames. In order
> to
> avoid this issue, use the same time base for input and ouput links

I believe this is partially misleading description since it is not
clearly describing application behavior. Impression might be that below
console error messages and some unexpected frames is the only
consequence. But it is not. In fact, application hangs in the very end
of the processing. Secondly, it is not clear what unexpected frames
results to? will there be more frames than should be in the final
bitstream or less?

Can you, please, polish commit message clarifying how exactly
application could misbehave? Pay attention on the hang, please.

Also, please, refer https://trac.ffmpeg.org/ticket/9286 which I
submitted for this issue. I verified that these 2 patches fix the
issue, but I am not sure which patch exactly fixed app hang.

> 
> The issue can be triggered with the command below:
> $> ffmpeg -hwaccel qsv -c:v hevc_qsv -i input.h265 -vf
> "vpp_qsv=w=1920:h=1080,fps=fps=60" -f null -
> 
> [out_0_0 @ 0x55eb017b5060] 100 buffers queued in out_0_0, something
> may
> be wrong.
> [out_0_0 @ 0x55eb017b5060] 1000 buffers queued in out_0_0, something
> may
> be wrong.
> [out_0_0 @ 0x55eb017b5060] 10000 buffers queued in out_0_0, something
> may be wrong.
> ---
>  libavfilter/vf_vpp_qsv.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/libavfilter/vf_vpp_qsv.c b/libavfilter/vf_vpp_qsv.c
> index b9ab5c6490..74d1d51e7c 100644
> --- a/libavfilter/vf_vpp_qsv.c
> +++ b/libavfilter/vf_vpp_qsv.c
> @@ -303,7 +303,7 @@ static int config_output(AVFilterLink *outlink)
>      outlink->w          = vpp->out_width;
>      outlink->h          = vpp->out_height;
>      outlink->frame_rate = vpp->framerate;
> -    outlink->time_base  = av_inv_q(vpp->framerate);
> +    outlink->time_base  = inlink->time_base;
>  
>      param.filter_frame  = NULL;
>      param.num_ext_buf   = 0;
Xiang, Haihao June 10, 2021, 1:31 a.m. UTC | #2
On Wed, 2021-06-09 at 16:03 +0000, Rogozhkin, Dmitry V wrote:
> On Wed, 2021-06-09 at 12:18 +0800, Haihao Xiang wrote:
> > Since commit 89ffcd1, the pts on output pad is in the time base of
> > the
> > input link, not the time base of the output link when EOF is reached,
> > so
> > a filter after vpp_qsv might output some unexpected frames. In order
> > to
> > avoid this issue, use the same time base for input and ouput links
> 
> I believe this is partially misleading description since it is not
> clearly describing application behavior. Impression might be that below
> console error messages and some unexpected frames is the only
> consequence. But it is not. In fact, application hangs in the very end
> of the processing. Secondly, it is not clear what unexpected frames
> results to? will there be more frames than should be in the final
> bitstream or less?


Actually there is not a hang, but you have to wait for a long time to output the
frames. e.g. for a clip with 100 frames in my testing, there will be 4119956
frames to output.

[out_0_0 @ 0x55933fe3cd80] 100 buffers queued in out_0_0, something may be
wrong.
[out_0_0 @ 0x55933fe3cd80] 1000 buffers queued in out_0_0, something may be
wrong.
[out_0_0 @ 0x55933fe3cd80] 10000 buffers queued in out_0_0, something may be
wrong.
[out_0_0 @ 0x55933fe3cd80] 100000 buffers queued in out_0_0, something may be
wrong.
[out_0_0 @ 0x55933fe3cd80] 1000000 buffers queued in out_0_0, something may be
wrong.
frame=4119956 fps=509098 q=-0.0 Lsize=N/A time=38:08:51.96 bitrate=N/A
speed=1.7e+04x     
video:1770294kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB
muxing overhead: unknow



> 
> Can you, please, polish commit message clarifying how exactly
> application could misbehave? Pay attention on the hang, please.
> 
> Also, please, refer https://trac.ffmpeg.org/ticket/9286 which I
> submitted for this issue. I verified that these 2 patches fix the
> issue, but I am not sure which patch exactly fixed app hang.
> 
> > 
> > The issue can be triggered with the command below:
> > $> ffmpeg -hwaccel qsv -c:v hevc_qsv -i input.h265 -vf
> > "vpp_qsv=w=1920:h=1080,fps=fps=60" -f null -
> > 
> > [out_0_0 @ 0x55eb017b5060] 100 buffers queued in out_0_0, something
> > may
> > be wrong.
> > [out_0_0 @ 0x55eb017b5060] 1000 buffers queued in out_0_0, something
> > may
> > be wrong.
> > [out_0_0 @ 0x55eb017b5060] 10000 buffers queued in out_0_0, something
> > may be wrong.
> > ---
> >  libavfilter/vf_vpp_qsv.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/libavfilter/vf_vpp_qsv.c b/libavfilter/vf_vpp_qsv.c
> > index b9ab5c6490..74d1d51e7c 100644
> > --- a/libavfilter/vf_vpp_qsv.c
> > +++ b/libavfilter/vf_vpp_qsv.c
> > @@ -303,7 +303,7 @@ static int config_output(AVFilterLink *outlink)
> >      outlink->w          = vpp->out_width;
> >      outlink->h          = vpp->out_height;
> >      outlink->frame_rate = vpp->framerate;
> > -    outlink->time_base  = av_inv_q(vpp->framerate);
> > +    outlink->time_base  = inlink->time_base;
> >  
> >      param.filter_frame  = NULL;
> >      param.num_ext_buf   = 0;
diff mbox series

Patch

diff --git a/libavfilter/vf_vpp_qsv.c b/libavfilter/vf_vpp_qsv.c
index b9ab5c6490..74d1d51e7c 100644
--- a/libavfilter/vf_vpp_qsv.c
+++ b/libavfilter/vf_vpp_qsv.c
@@ -303,7 +303,7 @@  static int config_output(AVFilterLink *outlink)
     outlink->w          = vpp->out_width;
     outlink->h          = vpp->out_height;
     outlink->frame_rate = vpp->framerate;
-    outlink->time_base  = av_inv_q(vpp->framerate);
+    outlink->time_base  = inlink->time_base;
 
     param.filter_frame  = NULL;
     param.num_ext_buf   = 0;