diff mbox series

[FFmpeg-devel] avcodec/cuviddec: fix AV1 decoding error

Message ID tencent_51F5A7EB2A8253051C618B8D660A50BC6606@qq.com
State New
Headers show
Series [FFmpeg-devel] avcodec/cuviddec: fix AV1 decoding error | expand

Checks

Context Check Description
yinshiyou/make_fate_loongarch64 success Make fate finished
yinshiyou/make_loongarch64 warning New warnings during build
andriy/make_x86 success Make finished
andriy/make_fate_x86 success Make fate finished
andriy/make_armv7_RPi4 success Make finished
andriy/make_fate_armv7_RPi4 success Make fate finished

Commit Message

zhilizhao(赵志立) June 23, 2022, 9:10 a.m. UTC
From: Zhao Zhili <zhilizhao@tencent.com>

Regression from 45e3b6a68. cuvidParseVideoData report unknown
error with AV1CodecConfigurationRecord.
---
 libavcodec/cuviddec.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

Comments

Timo Rothenpieler June 23, 2022, 2:14 p.m. UTC | #1
On 23/06/2022 11:10, Zhao Zhili wrote:
> From: Zhao Zhili <zhilizhao@tencent.com>
> 
> Regression from 45e3b6a68. cuvidParseVideoData report unknown
> error with AV1CodecConfigurationRecord.
> ---
>   libavcodec/cuviddec.c | 11 +++++++++++
>   1 file changed, 11 insertions(+)
> 
> diff --git a/libavcodec/cuviddec.c b/libavcodec/cuviddec.c
> index cb3cda7e24..584fff40ed 100644
> --- a/libavcodec/cuviddec.c
> +++ b/libavcodec/cuviddec.c
> @@ -953,6 +953,17 @@ static av_cold int cuvid_decode_init(AVCodecContext *avctx)
>           extradata_size = avctx->extradata_size;
>       }
>   
> +#if CONFIG_AV1_CUVID_DECODER && defined(CUVID_HAS_AV1_SUPPORT)
> +    if (avctx->codec->id == AV_CODEC_ID_AV1 &&
> +            extradata_size > 4 &&
> +            extradata && extradata[0] & 0x80) {
> +        // skip AV1CodecConfigurationRecord to keep configOBUs only,
> +        // otherwise cuvidParseVideoData report unknown error.
> +        extradata += 4;
> +        extradata_size -= 4;
> +    }
> +#endif

Checking the defines shouldn't be necessary. The codec ID should never 
be AV1 if it's not enabled, and the code does not contain anything AV1 
specific.

>       ctx->cuparse_ext = av_mallocz(sizeof(*ctx->cuparse_ext)
>               + FFMAX(extradata_size - (int)sizeof(ctx->cuparse_ext->raw_seqhdr_data), 0));
>       if (!ctx->cuparse_ext) {

I'm not quite sure I understand what causes this.
Something about the mp4 demuxer changed, yes.
But what about other containers? Why wasn't this an issue before?

The code that was removed in 45e3b6a68 seems to just have always skipped 
the first 4 byte unconditionally.

A little more info in the commit message would be nice here. Specially a 
sentence or two about the "& 80" part.
Code LGTM otherwise.
zhilizhao(赵志立) June 24, 2022, 2:52 a.m. UTC | #2
> On Jun 23, 2022, at 10:14 PM, Timo Rothenpieler <timo@rothenpieler.org> wrote:
> 
> On 23/06/2022 11:10, Zhao Zhili wrote:
>> From: Zhao Zhili <zhilizhao@tencent.com>
>> Regression from 45e3b6a68. cuvidParseVideoData report unknown
>> error with AV1CodecConfigurationRecord.
>> ---
>>  libavcodec/cuviddec.c | 11 +++++++++++
>>  1 file changed, 11 insertions(+)
>> diff --git a/libavcodec/cuviddec.c b/libavcodec/cuviddec.c
>> index cb3cda7e24..584fff40ed 100644
>> --- a/libavcodec/cuviddec.c
>> +++ b/libavcodec/cuviddec.c
>> @@ -953,6 +953,17 @@ static av_cold int cuvid_decode_init(AVCodecContext *avctx)
>>          extradata_size = avctx->extradata_size;
>>      }
>>  +#if CONFIG_AV1_CUVID_DECODER && defined(CUVID_HAS_AV1_SUPPORT)
>> +    if (avctx->codec->id == AV_CODEC_ID_AV1 &&
>> +            extradata_size > 4 &&
>> +            extradata && extradata[0] & 0x80) {
>> +        // skip AV1CodecConfigurationRecord to keep configOBUs only,
>> +        // otherwise cuvidParseVideoData report unknown error.
>> +        extradata += 4;
>> +        extradata_size -= 4;
>> +    }
>> +#endif
> 
> Checking the defines shouldn't be necessary. The codec ID should never be AV1 if it's not enabled, and the code does not contain anything AV1 specific.

I have thought about this. On one hand, the conditional compile is
unnecessary. On the other hand, it can remove dead code explicitly
when the condition is false, instead of depending on compiler.

I’m OK with remove the condition to keep code simple.

> 
>>      ctx->cuparse_ext = av_mallocz(sizeof(*ctx->cuparse_ext)
>>              + FFMAX(extradata_size - (int)sizeof(ctx->cuparse_ext->raw_seqhdr_data), 0));
>>      if (!ctx->cuparse_ext) {
> 
> I'm not quite sure I understand what causes this.
> Something about the mp4 demuxer changed, yes.
> But what about other containers? Why wasn't this an issue before?

It’s a regression from the point of fftools. If we take libavcodec
as standalone lib, the issue exist before 45e3b6a68.

> 
> The code that was removed in 45e3b6a68 seems to just have always skipped the first 4 byte unconditionally.

In the context of mp4 demuxer, it’s guaranteed the extradata is
AV1CodecConfigurationRecord. There is no guarantee that the
extradata is AV1CodecConfigurationRecord in the context of
decoder. So that’s why the check `extradata[0] & 0x80` exist.

AV1CodecConfigurationRecord is used by mp4 and mkv both. And
there is a similar patch in mkv: ffd1316

    avformat/matroskadec: remove special handling of av1c extradata

    Libavcodec can now handle the standard AV1CodecConfigurationRecord
    extradata as-is.

> 
> A little more info in the commit message would be nice here. Specially a sentence or two about the "& 80" part.

Will add comments about the “& 80” part.

Thanks for the review!

> Code LGTM otherwise.
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
zhilizhao(赵志立) June 24, 2022, 4:09 a.m. UTC | #3
> On Jun 24, 2022, at 10:52 AM, zhilizhao(赵志立) <quinkblack@foxmail.com> wrote:
> 
> 
> 
>> On Jun 23, 2022, at 10:14 PM, Timo Rothenpieler <timo@rothenpieler.org> wrote:
>> 
>> On 23/06/2022 11:10, Zhao Zhili wrote:
>>> From: Zhao Zhili <zhilizhao@tencent.com>
>>> Regression from 45e3b6a68. cuvidParseVideoData report unknown
>>> error with AV1CodecConfigurationRecord.
>>> ---
>>> libavcodec/cuviddec.c | 11 +++++++++++
>>> 1 file changed, 11 insertions(+)
>>> diff --git a/libavcodec/cuviddec.c b/libavcodec/cuviddec.c
>>> index cb3cda7e24..584fff40ed 100644
>>> --- a/libavcodec/cuviddec.c
>>> +++ b/libavcodec/cuviddec.c
>>> @@ -953,6 +953,17 @@ static av_cold int cuvid_decode_init(AVCodecContext *avctx)
>>>         extradata_size = avctx->extradata_size;
>>>     }
>>> +#if CONFIG_AV1_CUVID_DECODER && defined(CUVID_HAS_AV1_SUPPORT)
>>> +    if (avctx->codec->id == AV_CODEC_ID_AV1 &&
>>> +            extradata_size > 4 &&
>>> +            extradata && extradata[0] & 0x80) {
>>> +        // skip AV1CodecConfigurationRecord to keep configOBUs only,
>>> +        // otherwise cuvidParseVideoData report unknown error.
>>> +        extradata += 4;
>>> +        extradata_size -= 4;
>>> +    }
>>> +#endif
>> 
...
> 
>> 
>> A little more info in the commit message would be nice here. Specially a sentence or two about the "& 80" part.
> 
> Will add comments about the “& 80” part.

v2 (Sorry I forgot to add v2 in patch subject):

http://ffmpeg.org/pipermail/ffmpeg-devel/2022-June/297983.html

> 
> Thanks for the review!
> 
>> Code LGTM otherwise.
>> _______________________________________________
>> ffmpeg-devel mailing list
>> ffmpeg-devel@ffmpeg.org
>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>> 
>> To unsubscribe, visit link above, or email
>> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> 
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
diff mbox series

Patch

diff --git a/libavcodec/cuviddec.c b/libavcodec/cuviddec.c
index cb3cda7e24..584fff40ed 100644
--- a/libavcodec/cuviddec.c
+++ b/libavcodec/cuviddec.c
@@ -953,6 +953,17 @@  static av_cold int cuvid_decode_init(AVCodecContext *avctx)
         extradata_size = avctx->extradata_size;
     }
 
+#if CONFIG_AV1_CUVID_DECODER && defined(CUVID_HAS_AV1_SUPPORT)
+    if (avctx->codec->id == AV_CODEC_ID_AV1 &&
+            extradata_size > 4 &&
+            extradata && extradata[0] & 0x80) {
+        // skip AV1CodecConfigurationRecord to keep configOBUs only,
+        // otherwise cuvidParseVideoData report unknown error.
+        extradata += 4;
+        extradata_size -= 4;
+    }
+#endif
+
     ctx->cuparse_ext = av_mallocz(sizeof(*ctx->cuparse_ext)
             + FFMAX(extradata_size - (int)sizeof(ctx->cuparse_ext->raw_seqhdr_data), 0));
     if (!ctx->cuparse_ext) {