Message ID | tencent_51F5A7EB2A8253051C618B8D660A50BC6606@qq.com |
---|---|
State | New |
Headers | show |
Series | [FFmpeg-devel] avcodec/cuviddec: fix AV1 decoding error | expand |
Context | Check | Description |
---|---|---|
yinshiyou/make_fate_loongarch64 | success | Make fate finished |
yinshiyou/make_loongarch64 | warning | New warnings during build |
andriy/make_x86 | success | Make finished |
andriy/make_fate_x86 | success | Make fate finished |
andriy/make_armv7_RPi4 | success | Make finished |
andriy/make_fate_armv7_RPi4 | success | Make fate finished |
On 23/06/2022 11:10, Zhao Zhili wrote: > From: Zhao Zhili <zhilizhao@tencent.com> > > Regression from 45e3b6a68. cuvidParseVideoData report unknown > error with AV1CodecConfigurationRecord. > --- > libavcodec/cuviddec.c | 11 +++++++++++ > 1 file changed, 11 insertions(+) > > diff --git a/libavcodec/cuviddec.c b/libavcodec/cuviddec.c > index cb3cda7e24..584fff40ed 100644 > --- a/libavcodec/cuviddec.c > +++ b/libavcodec/cuviddec.c > @@ -953,6 +953,17 @@ static av_cold int cuvid_decode_init(AVCodecContext *avctx) > extradata_size = avctx->extradata_size; > } > > +#if CONFIG_AV1_CUVID_DECODER && defined(CUVID_HAS_AV1_SUPPORT) > + if (avctx->codec->id == AV_CODEC_ID_AV1 && > + extradata_size > 4 && > + extradata && extradata[0] & 0x80) { > + // skip AV1CodecConfigurationRecord to keep configOBUs only, > + // otherwise cuvidParseVideoData report unknown error. > + extradata += 4; > + extradata_size -= 4; > + } > +#endif Checking the defines shouldn't be necessary. The codec ID should never be AV1 if it's not enabled, and the code does not contain anything AV1 specific. > ctx->cuparse_ext = av_mallocz(sizeof(*ctx->cuparse_ext) > + FFMAX(extradata_size - (int)sizeof(ctx->cuparse_ext->raw_seqhdr_data), 0)); > if (!ctx->cuparse_ext) { I'm not quite sure I understand what causes this. Something about the mp4 demuxer changed, yes. But what about other containers? Why wasn't this an issue before? The code that was removed in 45e3b6a68 seems to just have always skipped the first 4 byte unconditionally. A little more info in the commit message would be nice here. Specially a sentence or two about the "& 80" part. Code LGTM otherwise.
> On Jun 23, 2022, at 10:14 PM, Timo Rothenpieler <timo@rothenpieler.org> wrote: > > On 23/06/2022 11:10, Zhao Zhili wrote: >> From: Zhao Zhili <zhilizhao@tencent.com> >> Regression from 45e3b6a68. cuvidParseVideoData report unknown >> error with AV1CodecConfigurationRecord. >> --- >> libavcodec/cuviddec.c | 11 +++++++++++ >> 1 file changed, 11 insertions(+) >> diff --git a/libavcodec/cuviddec.c b/libavcodec/cuviddec.c >> index cb3cda7e24..584fff40ed 100644 >> --- a/libavcodec/cuviddec.c >> +++ b/libavcodec/cuviddec.c >> @@ -953,6 +953,17 @@ static av_cold int cuvid_decode_init(AVCodecContext *avctx) >> extradata_size = avctx->extradata_size; >> } >> +#if CONFIG_AV1_CUVID_DECODER && defined(CUVID_HAS_AV1_SUPPORT) >> + if (avctx->codec->id == AV_CODEC_ID_AV1 && >> + extradata_size > 4 && >> + extradata && extradata[0] & 0x80) { >> + // skip AV1CodecConfigurationRecord to keep configOBUs only, >> + // otherwise cuvidParseVideoData report unknown error. >> + extradata += 4; >> + extradata_size -= 4; >> + } >> +#endif > > Checking the defines shouldn't be necessary. The codec ID should never be AV1 if it's not enabled, and the code does not contain anything AV1 specific. I have thought about this. On one hand, the conditional compile is unnecessary. On the other hand, it can remove dead code explicitly when the condition is false, instead of depending on compiler. I’m OK with remove the condition to keep code simple. > >> ctx->cuparse_ext = av_mallocz(sizeof(*ctx->cuparse_ext) >> + FFMAX(extradata_size - (int)sizeof(ctx->cuparse_ext->raw_seqhdr_data), 0)); >> if (!ctx->cuparse_ext) { > > I'm not quite sure I understand what causes this. > Something about the mp4 demuxer changed, yes. > But what about other containers? Why wasn't this an issue before? It’s a regression from the point of fftools. If we take libavcodec as standalone lib, the issue exist before 45e3b6a68. > > The code that was removed in 45e3b6a68 seems to just have always skipped the first 4 byte unconditionally. In the context of mp4 demuxer, it’s guaranteed the extradata is AV1CodecConfigurationRecord. There is no guarantee that the extradata is AV1CodecConfigurationRecord in the context of decoder. So that’s why the check `extradata[0] & 0x80` exist. AV1CodecConfigurationRecord is used by mp4 and mkv both. And there is a similar patch in mkv: ffd1316 avformat/matroskadec: remove special handling of av1c extradata Libavcodec can now handle the standard AV1CodecConfigurationRecord extradata as-is. > > A little more info in the commit message would be nice here. Specially a sentence or two about the "& 80" part. Will add comments about the “& 80” part. Thanks for the review! > Code LGTM otherwise. > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> On Jun 24, 2022, at 10:52 AM, zhilizhao(赵志立) <quinkblack@foxmail.com> wrote: > > > >> On Jun 23, 2022, at 10:14 PM, Timo Rothenpieler <timo@rothenpieler.org> wrote: >> >> On 23/06/2022 11:10, Zhao Zhili wrote: >>> From: Zhao Zhili <zhilizhao@tencent.com> >>> Regression from 45e3b6a68. cuvidParseVideoData report unknown >>> error with AV1CodecConfigurationRecord. >>> --- >>> libavcodec/cuviddec.c | 11 +++++++++++ >>> 1 file changed, 11 insertions(+) >>> diff --git a/libavcodec/cuviddec.c b/libavcodec/cuviddec.c >>> index cb3cda7e24..584fff40ed 100644 >>> --- a/libavcodec/cuviddec.c >>> +++ b/libavcodec/cuviddec.c >>> @@ -953,6 +953,17 @@ static av_cold int cuvid_decode_init(AVCodecContext *avctx) >>> extradata_size = avctx->extradata_size; >>> } >>> +#if CONFIG_AV1_CUVID_DECODER && defined(CUVID_HAS_AV1_SUPPORT) >>> + if (avctx->codec->id == AV_CODEC_ID_AV1 && >>> + extradata_size > 4 && >>> + extradata && extradata[0] & 0x80) { >>> + // skip AV1CodecConfigurationRecord to keep configOBUs only, >>> + // otherwise cuvidParseVideoData report unknown error. >>> + extradata += 4; >>> + extradata_size -= 4; >>> + } >>> +#endif >> ... > >> >> A little more info in the commit message would be nice here. Specially a sentence or two about the "& 80" part. > > Will add comments about the “& 80” part. v2 (Sorry I forgot to add v2 in patch subject): http://ffmpeg.org/pipermail/ffmpeg-devel/2022-June/297983.html > > Thanks for the review! > >> Code LGTM otherwise. >> _______________________________________________ >> ffmpeg-devel mailing list >> ffmpeg-devel@ffmpeg.org >> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel >> >> To unsubscribe, visit link above, or email >> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
diff --git a/libavcodec/cuviddec.c b/libavcodec/cuviddec.c index cb3cda7e24..584fff40ed 100644 --- a/libavcodec/cuviddec.c +++ b/libavcodec/cuviddec.c @@ -953,6 +953,17 @@ static av_cold int cuvid_decode_init(AVCodecContext *avctx) extradata_size = avctx->extradata_size; } +#if CONFIG_AV1_CUVID_DECODER && defined(CUVID_HAS_AV1_SUPPORT) + if (avctx->codec->id == AV_CODEC_ID_AV1 && + extradata_size > 4 && + extradata && extradata[0] & 0x80) { + // skip AV1CodecConfigurationRecord to keep configOBUs only, + // otherwise cuvidParseVideoData report unknown error. + extradata += 4; + extradata_size -= 4; + } +#endif + ctx->cuparse_ext = av_mallocz(sizeof(*ctx->cuparse_ext) + FFMAX(extradata_size - (int)sizeof(ctx->cuparse_ext->raw_seqhdr_data), 0)); if (!ctx->cuparse_ext) {
From: Zhao Zhili <zhilizhao@tencent.com> Regression from 45e3b6a68. cuvidParseVideoData report unknown error with AV1CodecConfigurationRecord. --- libavcodec/cuviddec.c | 11 +++++++++++ 1 file changed, 11 insertions(+)