| Message ID | 20211210022152.3782708-1-wenbin.chen@intel.com |
|---|---|
| State | New |
| Headers | show |
| Series | [FFmpeg-devel] libavfilter/vf_overlay_qsv: Use format of first input to set output format for overlay_qsv | expand |
| Context | Check | Description |
|---|---|---|
| andriy/make_x86 | success | Make finished |
| andriy/make_fate_x86 | success | Make fate finished |
| andriy/make_ppc | success | Make finished |
| andriy/make_fate_ppc | success | Make fate finished |
> -----Original Message----- > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Wenbin Chen > Sent: Friday, December 10, 2021 3:22 AM > To: ffmpeg-devel@ffmpeg.org > Cc: Wenbin Chen <wenbin.chen@intel.com> > Subject: [FFmpeg-devel] [PATCH] libavfilter/vf_overlay_qsv: Use format of > first input to set output format for overlay_qsv > > overlay_qsv hard coded to use nv12 as output format. Now use the format > of the first input to set output format. Now the following command > works: > > ffmpeg -hwaccel qsv -c:v hevc_qsv -i input_p010.265 -hwaccel qsv -c:v > hevc_qsv \ > -i input2_p010.265 -filter_complex "[0:v][1:v]overlay_qsv=x=0:y=0:alpha=255, > \ > hwdownload,format=p010le" -f rawvideo -y output_p010.yuv > > Signed-off-by: Wenbin Chen <wenbin.chen@intel.com> > --- > libavfilter/vf_overlay_qsv.c | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/libavfilter/vf_overlay_qsv.c b/libavfilter/vf_overlay_qsv.c > index 7e76b39aa9..d947a1faa1 100644 > --- a/libavfilter/vf_overlay_qsv.c > +++ b/libavfilter/vf_overlay_qsv.c > @@ -276,6 +276,7 @@ static int config_output(AVFilterLink *outlink) > int ret; > > av_log(ctx, AV_LOG_DEBUG, "Output is of %s.\n", > av_get_pix_fmt_name(outlink->format)); > + vpp->qsv_param.out_sw_format = in0->format; > if ((in0->format == AV_PIX_FMT_QSV && in1->format != AV_PIX_FMT_QSV) || > (in0->format != AV_PIX_FMT_QSV && in1->format == AV_PIX_FMT_QSV)) { > av_log(ctx, AV_LOG_ERROR, "Mixing hardware and software pixel > formats is not supported.\n"); > @@ -288,6 +289,7 @@ static int config_output(AVFilterLink *outlink) > av_log(ctx, AV_LOG_ERROR, "Inputs with different underlying QSV > devices are forbidden.\n"); > return AVERROR(EINVAL); > } > + vpp->qsv_param.out_sw_format = hw_frame0->sw_format; > } > > outlink->w = vpp->var_values[VAR_MW]; > -- Isn't this a bit too optimistic? The doc says: The only supported combinations of input and output color formats are: RGB to RGB, NV12 to NV12, RGB and NV12 to NV12, for per pixel alpha blending use case. https://github.com/Intel-Media-SDK/MediaSDK/blob/master/doc/mediasdk-man.md#mfxextvppcomposite Kind regards, softworkz
> -----Original Message----- > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Wenbin Chen > Sent: Friday, December 10, 2021 3:22 AM > To: ffmpeg-devel@ffmpeg.org > Cc: Wenbin Chen <wenbin.chen@intel.com> > Subject: [FFmpeg-devel] [PATCH] libavfilter/vf_overlay_qsv: Use format of > first input to set output format for overlay_qsv > > overlay_qsv hard coded to use nv12 as output format. Now use the format > of the first input to set output format. Now the following command > works: > > ffmpeg -hwaccel qsv -c:v hevc_qsv -i input_p010.265 -hwaccel qsv -c:v > hevc_qsv \ > -i input2_p010.265 -filter_complex "[0:v][1:v]overlay_qsv=x=0:y=0:alpha=255, > \ > hwdownload,format=p010le" -f rawvideo -y output_p010.yuv > > Signed-off-by: Wenbin Chen <wenbin.chen@intel.com> > --- > libavfilter/vf_overlay_qsv.c | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/libavfilter/vf_overlay_qsv.c b/libavfilter/vf_overlay_qsv.c > index 7e76b39aa9..d947a1faa1 100644 > --- a/libavfilter/vf_overlay_qsv.c > +++ b/libavfilter/vf_overlay_qsv.c > @@ -276,6 +276,7 @@ static int config_output(AVFilterLink *outlink) > int ret; > > av_log(ctx, AV_LOG_DEBUG, "Output is of %s.\n", > av_get_pix_fmt_name(outlink->format)); > + vpp->qsv_param.out_sw_format = in0->format; > if ((in0->format == AV_PIX_FMT_QSV && in1->format != AV_PIX_FMT_QSV) || > (in0->format != AV_PIX_FMT_QSV && in1->format == AV_PIX_FMT_QSV)) { > av_log(ctx, AV_LOG_ERROR, "Mixing hardware and software pixel > formats is not supported.\n"); > @@ -288,6 +289,7 @@ static int config_output(AVFilterLink *outlink) > av_log(ctx, AV_LOG_ERROR, "Inputs with different underlying QSV > devices are forbidden.\n"); > return AVERROR(EINVAL); > } > + vpp->qsv_param.out_sw_format = hw_frame0->sw_format; > } > > outlink->w = vpp->var_values[VAR_MW]; > -- When you're already looking at that area: Do you have any idea why it sets the output time_base to the reciprocal input0 framerate? I think it should rather set it to the same as the input0 time_base. https://github.com/FFmpeg/FFmpeg/blob/4f44a218e53cd92e64ba10a935bc1e7583c3e218/libavfilter/vf_overlay_qsv.c#L293-L296 What do you think? (also @Haihao) Thanks, softworkz
> > -----Original Message----- > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > Wenbin Chen > > Sent: Friday, December 10, 2021 3:22 AM > > To: ffmpeg-devel@ffmpeg.org > > Cc: Wenbin Chen <wenbin.chen@intel.com> > > Subject: [FFmpeg-devel] [PATCH] libavfilter/vf_overlay_qsv: Use format of > > first input to set output format for overlay_qsv > > > > overlay_qsv hard coded to use nv12 as output format. Now use the format > > of the first input to set output format. Now the following command > > works: > > > > ffmpeg -hwaccel qsv -c:v hevc_qsv -i input_p010.265 -hwaccel qsv -c:v > > hevc_qsv \ > > -i input2_p010.265 -filter_complex > "[0:v][1:v]overlay_qsv=x=0:y=0:alpha=255, > > \ > > hwdownload,format=p010le" -f rawvideo -y output_p010.yuv > > > > Signed-off-by: Wenbin Chen <wenbin.chen@intel.com> > > --- > > libavfilter/vf_overlay_qsv.c | 2 ++ > > 1 file changed, 2 insertions(+) > > > > diff --git a/libavfilter/vf_overlay_qsv.c b/libavfilter/vf_overlay_qsv.c > > index 7e76b39aa9..d947a1faa1 100644 > > --- a/libavfilter/vf_overlay_qsv.c > > +++ b/libavfilter/vf_overlay_qsv.c > > @@ -276,6 +276,7 @@ static int config_output(AVFilterLink *outlink) > > int ret; > > > > av_log(ctx, AV_LOG_DEBUG, "Output is of %s.\n", > > av_get_pix_fmt_name(outlink->format)); > > + vpp->qsv_param.out_sw_format = in0->format; > > if ((in0->format == AV_PIX_FMT_QSV && in1->format != > AV_PIX_FMT_QSV) || > > (in0->format != AV_PIX_FMT_QSV && in1->format == > AV_PIX_FMT_QSV)) { > > av_log(ctx, AV_LOG_ERROR, "Mixing hardware and software pixel > > formats is not supported.\n"); > > @@ -288,6 +289,7 @@ static int config_output(AVFilterLink *outlink) > > av_log(ctx, AV_LOG_ERROR, "Inputs with different underlying QSV > > devices are forbidden.\n"); > > return AVERROR(EINVAL); > > } > > + vpp->qsv_param.out_sw_format = hw_frame0->sw_format; > > } > > > > outlink->w = vpp->var_values[VAR_MW]; > > -- > > Isn't this a bit too optimistic? > > The doc says: > > The only supported combinations of input and output color formats are: > > RGB to RGB, > NV12 to NV12, > RGB and NV12 to NV12, for per pixel alpha blending use case. > > https://github.com/Intel-Media-SDK/MediaSDK/blob/master/doc/mediasdk- > man.md#mfxextvppcomposite > > Kind regards, > softworkz Thanks for your feedback. We are checking this. > > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> > -----Original Message----- > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > Wenbin Chen > > Sent: Friday, December 10, 2021 3:22 AM > > To: ffmpeg-devel@ffmpeg.org > > Cc: Wenbin Chen <wenbin.chen@intel.com> > > Subject: [FFmpeg-devel] [PATCH] libavfilter/vf_overlay_qsv: Use format of > > first input to set output format for overlay_qsv > > > > overlay_qsv hard coded to use nv12 as output format. Now use the format > > of the first input to set output format. Now the following command > > works: > > > > ffmpeg -hwaccel qsv -c:v hevc_qsv -i input_p010.265 -hwaccel qsv -c:v > > hevc_qsv \ > > -i input2_p010.265 -filter_complex > "[0:v][1:v]overlay_qsv=x=0:y=0:alpha=255, > > \ > > hwdownload,format=p010le" -f rawvideo -y output_p010.yuv > > > > Signed-off-by: Wenbin Chen <wenbin.chen@intel.com> > > --- > > libavfilter/vf_overlay_qsv.c | 2 ++ > > 1 file changed, 2 insertions(+) > > > > diff --git a/libavfilter/vf_overlay_qsv.c b/libavfilter/vf_overlay_qsv.c > > index 7e76b39aa9..d947a1faa1 100644 > > --- a/libavfilter/vf_overlay_qsv.c > > +++ b/libavfilter/vf_overlay_qsv.c > > @@ -276,6 +276,7 @@ static int config_output(AVFilterLink *outlink) > > int ret; > > > > av_log(ctx, AV_LOG_DEBUG, "Output is of %s.\n", > > av_get_pix_fmt_name(outlink->format)); > > + vpp->qsv_param.out_sw_format = in0->format; > > if ((in0->format == AV_PIX_FMT_QSV && in1->format != > AV_PIX_FMT_QSV) || > > (in0->format != AV_PIX_FMT_QSV && in1->format == > AV_PIX_FMT_QSV)) { > > av_log(ctx, AV_LOG_ERROR, "Mixing hardware and software pixel > > formats is not supported.\n"); > > @@ -288,6 +289,7 @@ static int config_output(AVFilterLink *outlink) > > av_log(ctx, AV_LOG_ERROR, "Inputs with different underlying QSV > > devices are forbidden.\n"); > > return AVERROR(EINVAL); > > } > > + vpp->qsv_param.out_sw_format = hw_frame0->sw_format; > > } > > > > outlink->w = vpp->var_values[VAR_MW]; > > -- > > When you're already looking at that area: > > Do you have any idea why it sets the output time_base to the reciprocal > input0 framerate? > I think it should rather set it to the same as the input0 time_base. > > https://github.com/FFmpeg/FFmpeg/blob/4f44a218e53cd92e64ba10a935bc > 1e7583c3e218/libavfilter/vf_overlay_qsv.c#L293-L296 > > What do you think? (also @Haihao) > > Thanks, > softworkz I have no idea why it is set to the reciprocal input0 framerate. I agree with you. I think it should be set to in0->time_base. I tried and changing to "in0->time_base" also works fine. > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
On Mon, 2021-12-13 at 09:40 +0000, Chen, Wenbin wrote: > > > -----Original Message----- > > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > > > > Wenbin Chen > > > Sent: Friday, December 10, 2021 3:22 AM > > > To: ffmpeg-devel@ffmpeg.org > > > Cc: Wenbin Chen <wenbin.chen@intel.com> > > > Subject: [FFmpeg-devel] [PATCH] libavfilter/vf_overlay_qsv: Use format of > > > first input to set output format for overlay_qsv > > > > > > overlay_qsv hard coded to use nv12 as output format. Now use the format > > > of the first input to set output format. Now the following command > > > works: > > > > > > ffmpeg -hwaccel qsv -c:v hevc_qsv -i input_p010.265 -hwaccel qsv -c:v > > > hevc_qsv \ > > > -i input2_p010.265 -filter_complex > > > > "[0:v][1:v]overlay_qsv=x=0:y=0:alpha=255, > > > \ > > > hwdownload,format=p010le" -f rawvideo -y output_p010.yuv > > > > > > Signed-off-by: Wenbin Chen <wenbin.chen@intel.com> > > > --- > > > libavfilter/vf_overlay_qsv.c | 2 ++ > > > 1 file changed, 2 insertions(+) > > > > > > diff --git a/libavfilter/vf_overlay_qsv.c b/libavfilter/vf_overlay_qsv.c > > > index 7e76b39aa9..d947a1faa1 100644 > > > --- a/libavfilter/vf_overlay_qsv.c > > > +++ b/libavfilter/vf_overlay_qsv.c > > > @@ -276,6 +276,7 @@ static int config_output(AVFilterLink *outlink) > > > int ret; > > > > > > av_log(ctx, AV_LOG_DEBUG, "Output is of %s.\n", > > > av_get_pix_fmt_name(outlink->format)); > > > + vpp->qsv_param.out_sw_format = in0->format; > > > if ((in0->format == AV_PIX_FMT_QSV && in1->format != > > > > AV_PIX_FMT_QSV) || > > > (in0->format != AV_PIX_FMT_QSV && in1->format == > > > > AV_PIX_FMT_QSV)) { > > > av_log(ctx, AV_LOG_ERROR, "Mixing hardware and software pixel > > > formats is not supported.\n"); > > > @@ -288,6 +289,7 @@ static int config_output(AVFilterLink *outlink) > > > av_log(ctx, AV_LOG_ERROR, "Inputs with different underlying > > > QSV > > > devices are forbidden.\n"); > > > return AVERROR(EINVAL); > > > } > > > + vpp->qsv_param.out_sw_format = hw_frame0->sw_format; > > > } > > > > > > outlink->w = vpp->var_values[VAR_MW]; > > > -- > > > > When you're already looking at that area: > > > > Do you have any idea why it sets the output time_base to the reciprocal > > input0 framerate? > > I think it should rather set it to the same as the input0 time_base. > > > > https://github.com/FFmpeg/FFmpeg/blob/4f44a218e53cd92e64ba10a935bc > > 1e7583c3e218/libavfilter/vf_overlay_qsv.c#L293-L296 > > > > What do you think? (also @Haihao) > > > > Thanks, > > softworkz > > I have no idea why it is set to the reciprocal input0 framerate. > I agree with you. I think it should be set to in0->time_base. > I tried and changing to "in0->time_base" also works fine. Maybe the code was copied from other files. As soon as the PTS of the frames passing through the output link uses the output time_base as unit, I think this code may work too except the precision of the output time_base. The PTS of the output frame from overlay_qsv uses the output time_base as unit: https://github.com/FFmpeg/FFmpeg/blob/master/libavfilter/vf_overlay_qsv.c#L314-L315 I may submit a patch to set the output time_base to the input0 time_base if you are thinking this is better. Thanks Haihao
diff --git a/libavfilter/vf_overlay_qsv.c b/libavfilter/vf_overlay_qsv.c index 7e76b39aa9..d947a1faa1 100644 --- a/libavfilter/vf_overlay_qsv.c +++ b/libavfilter/vf_overlay_qsv.c @@ -276,6 +276,7 @@ static int config_output(AVFilterLink *outlink) int ret; av_log(ctx, AV_LOG_DEBUG, "Output is of %s.\n", av_get_pix_fmt_name(outlink->format)); + vpp->qsv_param.out_sw_format = in0->format; if ((in0->format == AV_PIX_FMT_QSV && in1->format != AV_PIX_FMT_QSV) || (in0->format != AV_PIX_FMT_QSV && in1->format == AV_PIX_FMT_QSV)) { av_log(ctx, AV_LOG_ERROR, "Mixing hardware and software pixel formats is not supported.\n"); @@ -288,6 +289,7 @@ static int config_output(AVFilterLink *outlink) av_log(ctx, AV_LOG_ERROR, "Inputs with different underlying QSV devices are forbidden.\n"); return AVERROR(EINVAL); } + vpp->qsv_param.out_sw_format = hw_frame0->sw_format; } outlink->w = vpp->var_values[VAR_MW];
overlay_qsv hard coded to use nv12 as output format. Now use the format of the first input to set output format. Now the following command works: ffmpeg -hwaccel qsv -c:v hevc_qsv -i input_p010.265 -hwaccel qsv -c:v hevc_qsv \ -i input2_p010.265 -filter_complex "[0:v][1:v]overlay_qsv=x=0:y=0:alpha=255, \ hwdownload,format=p010le" -f rawvideo -y output_p010.yuv Signed-off-by: Wenbin Chen <wenbin.chen@intel.com> --- libavfilter/vf_overlay_qsv.c | 2 ++ 1 file changed, 2 insertions(+)