| Message ID | 20221118153422.67632-1-ruijing.dong@amd.com |
|---|---|
| State | New |
| Headers | show |
| Series | [FFmpeg-devel] avcodec/av1_vaapi: add direct film grain mode | expand |
| Context | Check | Description |
|---|---|---|
| yinshiyou/make_loongarch64 | success | Make finished |
| yinshiyou/make_fate_loongarch64 | success | Make fate finished |
| andriy/make_x86 | success | Make finished |
| andriy/make_fate_x86 | success | Make fate finished |
Nov 21, 2022, 20:50 by ruijing.dong@amd.com: > Add flag AV_VAAPI_DRIVER_QUIRK_DIRCT_FILM_GRAIN_ATTRIBUTES to > specify a direct film grain mode for AMD av1 decoder. > > issue: > By using AMD av1 decoder via VAAPI, when used with film > grain content, the output displays black screen with > incorrect frame order. > > The issue being discussed in here: > https://gitlab.freedesktop.org/mesa/mesa/-/issues/6903#note_1613807 > > Signed-off-by: Ruijing Dong <ruijing.dong@amd.com> > Signed-off-by: nyanmisaka <nst799610810@gmail.com> > --- > > update: using driver_quirks to identify AMD driver instead of > extra AV_HWACCEL_FLAG. > > libavcodec/avcodec.h | 1 - > libavcodec/vaapi_av1.c | 6 ++++-- > libavutil/hwcontext_vaapi.c | 5 +++++ > libavutil/hwcontext_vaapi.h | 6 ++++++ > 4 files changed, 15 insertions(+), 3 deletions(-) > I think this looks fine. Just needs a micro bump and APIchanges, but whoever commits it can apply that. jkqxz should take a look at it too.
On Mon, 2022-11-21 at 14:50 -0500, Ruijing Dong wrote: > Add flag AV_VAAPI_DRIVER_QUIRK_DIRCT_FILM_GRAIN_ATTRIBUTES to > specify a direct film grain mode for AMD av1 decoder. > > issue: > By using AMD av1 decoder via VAAPI, when used with film > grain content, the output displays black screen with > incorrect frame order. > > The issue being discussed in here: > https://gitlab.freedesktop.org/mesa/mesa/-/issues/6903#note_1613807 > > Signed-off-by: Ruijing Dong <ruijing.dong@amd.com> > Signed-off-by: nyanmisaka <nst799610810@gmail.com> > --- > > update: using driver_quirks to identify AMD driver instead of > extra AV_HWACCEL_FLAG. > > libavcodec/avcodec.h | 1 - > libavcodec/vaapi_av1.c | 6 ++++-- > libavutil/hwcontext_vaapi.c | 5 +++++ > libavutil/hwcontext_vaapi.h | 6 ++++++ > 4 files changed, 15 insertions(+), 3 deletions(-) > > diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h > index 3edd8e2636..51adb8a1dc 100644 > --- a/libavcodec/avcodec.h > +++ b/libavcodec/avcodec.h > @@ -2252,7 +2252,6 @@ typedef struct AVHWAccel { > * while indicating success. > */ > #define AV_HWACCEL_FLAG_ALLOW_PROFILE_MISMATCH (1 << 2) > - This change is not needed Thanks Haihao > /** > * @} > */ > diff --git a/libavcodec/vaapi_av1.c b/libavcodec/vaapi_av1.c > index d0339b2705..1eb6b87264 100644 > --- a/libavcodec/vaapi_av1.c > +++ b/libavcodec/vaapi_av1.c > @@ -127,6 +127,7 @@ static int vaapi_av1_start_frame(AVCodecContext *avctx, > int8_t bit_depth_idx; > int err = 0; > int apply_grain = !(avctx->export_side_data & > AV_CODEC_EXPORT_DATA_FILM_GRAIN) && film_grain->apply_grain; > + int direct_film_grain = ctx->base.hwctx->driver_quirks & > AV_VAAPI_DRIVER_QUIRK_DIRECT_FILM_GRAIN_ATTRIBUTES; > uint8_t remap_lr_type[4] = {AV1_RESTORE_NONE, AV1_RESTORE_SWITCHABLE, > AV1_RESTORE_WIENER, AV1_RESTORE_SGRPROJ}; > uint8_t segmentation_feature_signed[AV1_SEG_LVL_MAX] = {1, 1, 1, 1, 1, 0, > 0, 0}; > uint8_t segmentation_feature_max[AV1_SEG_LVL_MAX] = {255, > AV1_MAX_LOOP_FILTER, > @@ -136,7 +137,7 @@ static int vaapi_av1_start_frame(AVCodecContext *avctx, > if (bit_depth_idx < 0) > goto fail; > > - if (apply_grain) { > + if (apply_grain && !direct_film_grain) { > if (ctx->tmp_frame->buf[0]) > ff_thread_release_buffer(avctx, ctx->tmp_frame); > err = ff_thread_get_buffer(avctx, ctx->tmp_frame, > AV_GET_BUFFER_FLAG_REF); > @@ -375,6 +376,7 @@ static int vaapi_av1_end_frame(AVCodecContext *avctx) > VAAPIAV1DecContext *ctx = avctx->internal->hwaccel_priv_data; > > int apply_grain = !(avctx->export_side_data & > AV_CODEC_EXPORT_DATA_FILM_GRAIN) && film_grain->apply_grain; > + int direct_film_grain = ctx->base.hwctx->driver_quirks & > AV_VAAPI_DRIVER_QUIRK_DIRECT_FILM_GRAIN_ATTRIBUTES; > int ret; > ret = ff_vaapi_decode_issue(avctx, pic); > if (ret < 0) > @@ -385,7 +387,7 @@ static int vaapi_av1_end_frame(AVCodecContext *avctx) > if (ctx->ref_tab[i].frame->buf[0]) > ff_thread_release_buffer(avctx, ctx->ref_tab[i].frame); > > - if (apply_grain) { > + if (apply_grain && !direct_film_grain) { > ret = av_frame_ref(ctx->ref_tab[i].frame, ctx->tmp_frame); > if (ret < 0) > return ret; > diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c > index 4bcde74f62..dced6e9401 100644 > --- a/libavutil/hwcontext_vaapi.c > +++ b/libavutil/hwcontext_vaapi.c > @@ -371,6 +371,11 @@ static const struct { > "Splitted-Desktop Systems VDPAU backend for VA-API", > AV_VAAPI_DRIVER_QUIRK_SURFACE_ATTRIBUTES, > }, > + { > + "AMD Radeon", > + "AMD Radeon", > + AV_VAAPI_DRIVER_QUIRK_DIRECT_FILM_GRAIN_ATTRIBUTES, > + } > }; > > static int vaapi_device_init(AVHWDeviceContext *hwdev) > diff --git a/libavutil/hwcontext_vaapi.h b/libavutil/hwcontext_vaapi.h > index 0b2e071cb3..b93d676423 100644 > --- a/libavutil/hwcontext_vaapi.h > +++ b/libavutil/hwcontext_vaapi.h > @@ -58,6 +58,12 @@ enum { > * and the results of the vaQuerySurfaceAttributes() call will be faked. > */ > AV_VAAPI_DRIVER_QUIRK_SURFACE_ATTRIBUTES = (1 << 3), > + > + /** > + * The identified driver which suppports film grain direct output mode. > + * and this is to avoid av1 decoding film grain corruption. > + */ > + AV_VAAPI_DRIVER_QUIRK_DIRECT_FILM_GRAIN_ATTRIBUTES = (1 << 4), > }; > > /** > -- > 2.25.1 > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index 3edd8e2636..9420e7820d 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -2253,6 +2253,23 @@ typedef struct AVHWAccel { */ #define AV_HWACCEL_FLAG_ALLOW_PROFILE_MISMATCH (1 << 2) +/** + * The film grain synthesis could be seperate from decoding process. + * The downstream device would apply the film grain parameters seperately. + * The desired film grain parameters can be found in SEI section in H264 + * or H265 bitstreams. + * + * In AV1, film grain is mandatorily specified, av1 decoders like AMD + * av1 decoder process film grain content internally, and the output + * includes applied film grain. For the purpose of supporting these av1 + * decoders, this flag needs to be utilized. + * + * @warning If the stream has no film grain content, this flag will + * be ignored in the supported av1 decoders. It is advised + * that this flag should only be used in av1 decoders + * that support it. + */ +#define AV_HWACCEL_FLAG_DIRECT_FILM_GRAIN (1 << 3) /** * @} */ diff --git a/libavcodec/vaapi_av1.c b/libavcodec/vaapi_av1.c index d0339b2705..6db910f2bf 100644 --- a/libavcodec/vaapi_av1.c +++ b/libavcodec/vaapi_av1.c @@ -127,6 +127,7 @@ static int vaapi_av1_start_frame(AVCodecContext *avctx, int8_t bit_depth_idx; int err = 0; int apply_grain = !(avctx->export_side_data & AV_CODEC_EXPORT_DATA_FILM_GRAIN) && film_grain->apply_grain; + int direct_film_grain = avctx->hwaccel_flags & AV_HWACCEL_FLAG_DIRECT_FILM_GRAIN; uint8_t remap_lr_type[4] = {AV1_RESTORE_NONE, AV1_RESTORE_SWITCHABLE, AV1_RESTORE_WIENER, AV1_RESTORE_SGRPROJ}; uint8_t segmentation_feature_signed[AV1_SEG_LVL_MAX] = {1, 1, 1, 1, 1, 0, 0, 0}; uint8_t segmentation_feature_max[AV1_SEG_LVL_MAX] = {255, AV1_MAX_LOOP_FILTER, @@ -136,7 +137,7 @@ static int vaapi_av1_start_frame(AVCodecContext *avctx, if (bit_depth_idx < 0) goto fail; - if (apply_grain) { + if (apply_grain && !direct_film_grain) { if (ctx->tmp_frame->buf[0]) ff_thread_release_buffer(avctx, ctx->tmp_frame); err = ff_thread_get_buffer(avctx, ctx->tmp_frame, AV_GET_BUFFER_FLAG_REF); @@ -375,6 +376,7 @@ static int vaapi_av1_end_frame(AVCodecContext *avctx) VAAPIAV1DecContext *ctx = avctx->internal->hwaccel_priv_data; int apply_grain = !(avctx->export_side_data & AV_CODEC_EXPORT_DATA_FILM_GRAIN) && film_grain->apply_grain; + int direct_film_grain = avctx->hwaccel_flags & AV_HWACCEL_FLAG_DIRECT_FILM_GRAIN; int ret; ret = ff_vaapi_decode_issue(avctx, pic); if (ret < 0) @@ -385,7 +387,7 @@ static int vaapi_av1_end_frame(AVCodecContext *avctx) if (ctx->ref_tab[i].frame->buf[0]) ff_thread_release_buffer(avctx, ctx->ref_tab[i].frame); - if (apply_grain) { + if (apply_grain && !direct_film_grain) { ret = av_frame_ref(ctx->ref_tab[i].frame, ctx->tmp_frame); if (ret < 0) return ret;
Adding direct film grain mode for av1 decoder, which outputs alongside film grain. AV_HWACCEL_FLAG_DIRECT_FILM_GRAIN is the new flag introduced to enable this path. issue: By using AMD av1 decoder via VAAPI, when used with film grain content, the output displays black screen with incorrect frame order. The issue being discussed in here: https://gitlab.freedesktop.org/mesa/mesa/-/issues/6903#note_1613807 example: This flag can be used in ffmpeg command: ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -hwaccel_flags 8 -i input_with_film_gram.obu output_with_film_grain.yuv Signed-off-by: Ruijing Dong <ruijing.dong@amd.com> --- libavcodec/avcodec.h | 17 +++++++++++++++++ libavcodec/vaapi_av1.c | 6 ++++-- 2 files changed, 21 insertions(+), 2 deletions(-)