diff mbox series

[FFmpeg-devel] avcodec/cuviddec: fix AV1 decoding error

Message ID tencent_6168FE7DD2ACDE3899C251B6B46B7E00850A@qq.com
State New
Headers show
Series [FFmpeg-devel] avcodec/cuviddec: fix AV1 decoding error | expand

Checks

Context Check Description
yinshiyou/make_loongarch64 success Make finished
yinshiyou/make_fate_loongarch64 success Make fate finished
andriy/make_x86 success Make finished
andriy/make_fate_x86 success Make fate finished
andriy/make_armv7_RPi4 success Make finished
andriy/make_fate_armv7_RPi4 success Make fate finished

Commit Message

Zhao Zhili June 24, 2022, 4:04 a.m. UTC
From: Zhao Zhili <zhilizhao@tencent.com>

cuvidParseVideoData only supports pure OBUs, it report unknown
error with AV1CodecConfigurationRecord. Check whether extradata
is AV1CodecConfigurationRecord and skip the first 4 bytes to fix
the issue.

The bug is revealed in ffmpeg cmd since 45e3b6a68 and ffd1316e.
---
 libavcodec/cuviddec.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

Comments

James Almer June 24, 2022, 12:06 p.m. UTC | #1
On 6/24/2022 1:04 AM, Zhao Zhili wrote:
> From: Zhao Zhili <zhilizhao@tencent.com>
> 
> cuvidParseVideoData only supports pure OBUs, it report unknown
> error with AV1CodecConfigurationRecord. Check whether extradata
> is AV1CodecConfigurationRecord and skip the first 4 bytes to fix
> the issue.
> 
> The bug is revealed in ffmpeg cmd since 45e3b6a68 and ffd1316e.
> ---
>   libavcodec/cuviddec.c | 10 ++++++++++
>   1 file changed, 10 insertions(+)
> 
> diff --git a/libavcodec/cuviddec.c b/libavcodec/cuviddec.c
> index cb3cda7e24..fc9c0515fe 100644
> --- a/libavcodec/cuviddec.c
> +++ b/libavcodec/cuviddec.c
> @@ -953,6 +953,16 @@ static av_cold int cuvid_decode_init(AVCodecContext *avctx)
>           extradata_size = avctx->extradata_size;
>       }
>   
> +    // Check first bit to determine whether it's AV1CodecConfigurationRecord.
> +    // Skip first 4 bytes of AV1CodecConfigurationRecord to keep configOBUs
> +    // only, otherwise cuvidParseVideoData report unknown error.
> +    if (avctx->codec->id == AV_CODEC_ID_AV1 &&
> +            extradata_size > 4 &&
> +            extradata && extradata[0] & 0x80) {

IMO the check for extradata here is unnecessary as the memcpy 
dereferencing it below will crash anyway if it were NULL.

If anything, a check like (avctx->extradata_size || !avctx->extradata) 
should be added to avcodec_open2() if there isn't one already to ensure 
the user set up valid extradata when opening a decoder.

> +        extradata += 4;
> +        extradata_size -= 4;
> +    }
> +
>       ctx->cuparse_ext = av_mallocz(sizeof(*ctx->cuparse_ext)
>               + FFMAX(extradata_size - (int)sizeof(ctx->cuparse_ext->raw_seqhdr_data), 0));
>       if (!ctx->cuparse_ext) {

LGTM either way.
diff mbox series

Patch

diff --git a/libavcodec/cuviddec.c b/libavcodec/cuviddec.c
index cb3cda7e24..fc9c0515fe 100644
--- a/libavcodec/cuviddec.c
+++ b/libavcodec/cuviddec.c
@@ -953,6 +953,16 @@  static av_cold int cuvid_decode_init(AVCodecContext *avctx)
         extradata_size = avctx->extradata_size;
     }
 
+    // Check first bit to determine whether it's AV1CodecConfigurationRecord.
+    // Skip first 4 bytes of AV1CodecConfigurationRecord to keep configOBUs
+    // only, otherwise cuvidParseVideoData report unknown error.
+    if (avctx->codec->id == AV_CODEC_ID_AV1 &&
+            extradata_size > 4 &&
+            extradata && extradata[0] & 0x80) {
+        extradata += 4;
+        extradata_size -= 4;
+    }
+
     ctx->cuparse_ext = av_mallocz(sizeof(*ctx->cuparse_ext)
             + FFMAX(extradata_size - (int)sizeof(ctx->cuparse_ext->raw_seqhdr_data), 0));
     if (!ctx->cuparse_ext) {