[FFmpeg-devel] avcodec/cbs_av1: don't abort when splitting Temporal Units containing OBUs with no obu_size field

The ISOBMFF and Matroska specs allow the last OBU in a Sample/Block to have
obu_has_size_field equal to 0.

Signed-off-by: James Almer <jamrial@gmail.com>
---
See https://0x0.st/sUsU.mp4

It apparently can't be decoded with either aom or dav1d atm, but the latter
should be able to soon.
You can try remuxing it using the av1_metadata bsf after this patch while at it,
since the code to write obu_size fields unconditionally is still in place, and
that should make it decodable with either library.

For some reason cbs_av1_read_unit() is already able to handle OBUs with no
obu_size field. It was only cbs_av1_split_fragment() bailing out with them.

 libavcodec/cbs_av1.c | 16 ++++++----------
 1 file changed, 6 insertions(+), 10 deletions(-)

On 11/15/2018 1:27 AM, James Almer wrote:
> The ISOBMFF and Matroska specs allow the last OBU in a Sample/Block to have
> obu_has_size_field equal to 0.
> 
> Signed-off-by: James Almer <jamrial@gmail.com>
> ---
> See https://0x0.st/sUsU.mp4
> 
> It apparently can't be decoded with either aom or dav1d atm, but the latter
> should be able to soon.

libdav1d can decode it now.

On 15/11/18 04:27, James Almer wrote:
> The ISOBMFF and Matroska specs allow the last OBU in a Sample/Block to have
> obu_has_size_field equal to 0.
> 
> Signed-off-by: James Almer <jamrial@gmail.com>
> ---
> See https://0x0.st/sUsU.mp4
> 
> It apparently can't be decoded with either aom or dav1d atm, but the latter
> should be able to soon.
> You can try remuxing it using the av1_metadata bsf after this patch while at it,
> since the code to write obu_size fields unconditionally is still in place, and
> that should make it decodable with either library.

I wonder whether there should be an option somewhere for omitting the last size field when writing?  It would save a few bytes on every TU when using any container which already has sizes.

> For some reason cbs_av1_read_unit() is already able to handle OBUs with no
> obu_size field. It was only cbs_av1_split_fragment() bailing out with them.

That came from the annex B support, before we decided that AVPackets would never contain annex B.

>  libavcodec/cbs_av1.c | 16 ++++++----------
>  1 file changed, 6 insertions(+), 10 deletions(-)
> 
> diff --git a/libavcodec/cbs_av1.c b/libavcodec/cbs_av1.c
> index ed5211be19..e02bc7027a 100644
> --- a/libavcodec/cbs_av1.c
> +++ b/libavcodec/cbs_av1.c
> @@ -788,13 +788,6 @@ static int cbs_av1_split_fragment(CodedBitstreamContext *ctx,
>          if (err < 0)
>              goto fail;
>  
> -        if (!header.obu_has_size_field) {
> -            av_log(ctx->log_ctx, AV_LOG_ERROR, "Invalid OBU for raw "
> -                   "stream: size field must be present.\n");
> -            err = AVERROR_INVALIDDATA;
> -            goto fail;
> -        }
> -
>          if (get_bits_left(&gbc) < 8) {
>              av_log(ctx->log_ctx, AV_LOG_ERROR, "Invalid OBU: fragment "
>                     "too short (%zu bytes).\n", size);
> @@ -802,9 +795,12 @@ static int cbs_av1_split_fragment(CodedBitstreamContext *ctx,
>              goto fail;
>          }
>  
> -        err = cbs_av1_read_leb128(ctx, &gbc, "obu_size", &obu_size);
> -        if (err < 0)
> -            goto fail;
> +        if (header.obu_has_size_field) {
> +            err = cbs_av1_read_leb128(ctx, &gbc, "obu_size", &obu_size);
> +            if (err < 0)
> +                goto fail;
> +        } else
> +            obu_size = size - 1 - header.obu_extension_flag;
>  
>          pos = get_bits_count(&gbc);
>          av_assert0(pos % 8 == 0 && pos / 8 <= size);
> 

LGTM.

Thanks,

- Mark

On 11/18/2018 3:41 PM, Mark Thompson wrote:
> On 15/11/18 04:27, James Almer wrote:
>> The ISOBMFF and Matroska specs allow the last OBU in a Sample/Block to have
>> obu_has_size_field equal to 0.
>>
>> Signed-off-by: James Almer <jamrial@gmail.com>
>> ---
>> See https://0x0.st/sUsU.mp4
>>
>> It apparently can't be decoded with either aom or dav1d atm, but the latter
>> should be able to soon.
>> You can try remuxing it using the av1_metadata bsf after this patch while at it,
>> since the code to write obu_size fields unconditionally is still in place, and
>> that should make it decodable with either library.
> 
> I wonder whether there should be an option somewhere for omitting the last size field when writing?  It would save a few bytes on every TU when using any container which already has sizes.

Yeah, that'd be neat as an option in av1_metadata, with choices of
bypass and remove/add if present or not in last obu, much like the one
for TDs.

Only thing i worry about is how as i mentioned libaom doesn't seem to be
able to handle OBUs with no size field. That means any such generated
stream would fail to decode on pretty much everything, at least until
dav1d gets widespread adoption, or aom implements support for the feature.

> 
>> For some reason cbs_av1_read_unit() is already able to handle OBUs with no
>> obu_size field. It was only cbs_av1_split_fragment() bailing out with them.
> 
> That came from the annex B support, before we decided that AVPackets would never contain annex B.
> 
>>  libavcodec/cbs_av1.c | 16 ++++++----------
>>  1 file changed, 6 insertions(+), 10 deletions(-)
>>
>> diff --git a/libavcodec/cbs_av1.c b/libavcodec/cbs_av1.c
>> index ed5211be19..e02bc7027a 100644
>> --- a/libavcodec/cbs_av1.c
>> +++ b/libavcodec/cbs_av1.c
>> @@ -788,13 +788,6 @@ static int cbs_av1_split_fragment(CodedBitstreamContext *ctx,
>>          if (err < 0)
>>              goto fail;
>>  
>> -        if (!header.obu_has_size_field) {
>> -            av_log(ctx->log_ctx, AV_LOG_ERROR, "Invalid OBU for raw "
>> -                   "stream: size field must be present.\n");
>> -            err = AVERROR_INVALIDDATA;
>> -            goto fail;
>> -        }
>> -
>>          if (get_bits_left(&gbc) < 8) {
>>              av_log(ctx->log_ctx, AV_LOG_ERROR, "Invalid OBU: fragment "
>>                     "too short (%zu bytes).\n", size);
>> @@ -802,9 +795,12 @@ static int cbs_av1_split_fragment(CodedBitstreamContext *ctx,
>>              goto fail;
>>          }
>>  
>> -        err = cbs_av1_read_leb128(ctx, &gbc, "obu_size", &obu_size);
>> -        if (err < 0)
>> -            goto fail;
>> +        if (header.obu_has_size_field) {
>> +            err = cbs_av1_read_leb128(ctx, &gbc, "obu_size", &obu_size);
>> +            if (err < 0)
>> +                goto fail;
>> +        } else
>> +            obu_size = size - 1 - header.obu_extension_flag;
>>  
>>          pos = get_bits_count(&gbc);
>>          av_assert0(pos % 8 == 0 && pos / 8 <= size);
>>
> 
> LGTM.
> 
> Thanks,
> 
> - Mark

Pushed. Thanks!