Message ID | 20200724023909.1156568-1-haihao.xiang@intel.com |
---|---|
State | Accepted |
Commit | fa3da243e6ba1567aadfa198bafd32c320e01205 |
Headers | show |
Series | [FFmpeg-devel] hwcontext_vaapi: remove duplicate formats from sw_format list | expand |
Context | Check | Description |
---|---|---|
andriy/default | pending | |
andriy/make | success | Make finished |
andriy/make_fate | success | Make fate finished |
On 24/07/2020 03:39, Haihao Xiang wrote: > hwcontext_vaapi maps different VA fourcc to the same pix_fmt for U/V > plane swap cases, however duplicate formats are not expected in sw_format > list when merging formats. > > For example: > ffmpeg -loglevel debug -init_hw_device vaapi -filter_hw_device vaapi0 \ > -f lavfi -i smptebars -vf \ > "hwupload=derive_device=vaapi,scale_vaapi,hwdownload,format=yuv420p" \ > -vframes 1 -f null - > > Without this fix, an auto scaler is required for the above command > Duplicate formats in ff_merge_formats detected > [auto_scaler_0 @ 0x560df58f4550] Setting 'flags' to value 'bicubic' > [auto_scaler_0 @ 0x560df58f4550] w:iw h:ih flags:'bicubic' interl:0 > [Parsed_hwupload_0 @ 0x560df58f0ec0] auto-inserting filter > 'auto_scaler_0' between the filter 'graph 0 input from stream 0:0' and > the filter 'Parsed_hwupload_0' > > Signed-off-by: Haihao Xiang <haihao.xiang@intel.com> > --- > libavutil/hwcontext_vaapi.c | 30 +++++++++++++++++++++++++----- > 1 file changed, 25 insertions(+), 5 deletions(-) > > diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c > index b31cf95850..fb9be19647 100644 > --- a/libavutil/hwcontext_vaapi.c > +++ b/libavutil/hwcontext_vaapi.c > @@ -268,14 +268,24 @@ static int vaapi_frames_get_constraints(AVHWDeviceContext *hwdev, > } > > for (i = j = 0; i < attr_count; i++) { > + int k; > + > if (attr_list[i].type != VASurfaceAttribPixelFormat) > continue; > fourcc = attr_list[i].value.value.i; > pix_fmt = vaapi_pix_fmt_from_fourcc(fourcc); > - if (pix_fmt != AV_PIX_FMT_NONE) > + > + if (pix_fmt == AV_PIX_FMT_NONE) > + continue; > + > + for (k = 0; k < j; k++) { > + if (constraints->valid_sw_formats[k] == pix_fmt) > + break; > + } > + > + if (k == j) > constraints->valid_sw_formats[j++] = pix_fmt; > } > - av_assert0(j == pix_fmt_count); > constraints->valid_sw_formats[j] = AV_PIX_FMT_NONE; > } > } else { > @@ -287,9 +297,19 @@ static int vaapi_frames_get_constraints(AVHWDeviceContext *hwdev, > err = AVERROR(ENOMEM); > goto fail; > } > - for (i = 0; i < ctx->nb_formats; i++) > - constraints->valid_sw_formats[i] = ctx->formats[i].pix_fmt; > - constraints->valid_sw_formats[i] = AV_PIX_FMT_NONE; > + for (i = j = 0; i < ctx->nb_formats; i++) { > + int k; > + > + for (k = 0; k < j; k++) { > + if (constraints->valid_sw_formats[k] == ctx->formats[i].pix_fmt) > + break; > + } > + > + if (k == j) > + constraints->valid_sw_formats[j++] = ctx->formats[i].pix_fmt; > + } > + > + constraints->valid_sw_formats[j] = AV_PIX_FMT_NONE; > } > > constraints->valid_hw_formats = av_malloc_array(2, sizeof(pix_fmt)); > Not a very fun case. Looks like a good way to handle it, though, so applied. Thanks, - Mark
diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c index b31cf95850..fb9be19647 100644 --- a/libavutil/hwcontext_vaapi.c +++ b/libavutil/hwcontext_vaapi.c @@ -268,14 +268,24 @@ static int vaapi_frames_get_constraints(AVHWDeviceContext *hwdev, } for (i = j = 0; i < attr_count; i++) { + int k; + if (attr_list[i].type != VASurfaceAttribPixelFormat) continue; fourcc = attr_list[i].value.value.i; pix_fmt = vaapi_pix_fmt_from_fourcc(fourcc); - if (pix_fmt != AV_PIX_FMT_NONE) + + if (pix_fmt == AV_PIX_FMT_NONE) + continue; + + for (k = 0; k < j; k++) { + if (constraints->valid_sw_formats[k] == pix_fmt) + break; + } + + if (k == j) constraints->valid_sw_formats[j++] = pix_fmt; } - av_assert0(j == pix_fmt_count); constraints->valid_sw_formats[j] = AV_PIX_FMT_NONE; } } else { @@ -287,9 +297,19 @@ static int vaapi_frames_get_constraints(AVHWDeviceContext *hwdev, err = AVERROR(ENOMEM); goto fail; } - for (i = 0; i < ctx->nb_formats; i++) - constraints->valid_sw_formats[i] = ctx->formats[i].pix_fmt; - constraints->valid_sw_formats[i] = AV_PIX_FMT_NONE; + for (i = j = 0; i < ctx->nb_formats; i++) { + int k; + + for (k = 0; k < j; k++) { + if (constraints->valid_sw_formats[k] == ctx->formats[i].pix_fmt) + break; + } + + if (k == j) + constraints->valid_sw_formats[j++] = ctx->formats[i].pix_fmt; + } + + constraints->valid_sw_formats[j] = AV_PIX_FMT_NONE; } constraints->valid_hw_formats = av_malloc_array(2, sizeof(pix_fmt));
hwcontext_vaapi maps different VA fourcc to the same pix_fmt for U/V plane swap cases, however duplicate formats are not expected in sw_format list when merging formats. For example: ffmpeg -loglevel debug -init_hw_device vaapi -filter_hw_device vaapi0 \ -f lavfi -i smptebars -vf \ "hwupload=derive_device=vaapi,scale_vaapi,hwdownload,format=yuv420p" \ -vframes 1 -f null - Without this fix, an auto scaler is required for the above command Duplicate formats in ff_merge_formats detected [auto_scaler_0 @ 0x560df58f4550] Setting 'flags' to value 'bicubic' [auto_scaler_0 @ 0x560df58f4550] w:iw h:ih flags:'bicubic' interl:0 [Parsed_hwupload_0 @ 0x560df58f0ec0] auto-inserting filter 'auto_scaler_0' between the filter 'graph 0 input from stream 0:0' and the filter 'Parsed_hwupload_0' Signed-off-by: Haihao Xiang <haihao.xiang@intel.com> --- libavutil/hwcontext_vaapi.c | 30 +++++++++++++++++++++++++----- 1 file changed, 25 insertions(+), 5 deletions(-)