| Message ID | 20210109073421.23721-11-nuomi2021@gmail.com |
|---|---|
| State | Superseded |
| Headers | show |
| Series | add vvc raw demuxer, muxer, parser, metadata bsf, vvdec decoder | expand |
| Context | Check | Description |
|---|---|---|
| andriy/x86_make | success | Make finished |
| andriy/x86_make_fate | success | Make fate finished |
| andriy/PPC64_make | success | Make finished |
| andriy/PPC64_make_fate | success | Make fate finished |
On 1/9/2021 4:34 AM, Nuo Mi wrote: > use following command to test: > ffmpeg -i in.bin -c:v copy -bsf vvc_metadata -f vvc out.bin > > 79.11%(197/249) can bit match with original clips > 6.83%(17/249) are not bit match, the original clips has redundant emulation prevent bytes > other all failed since vvdec failed to decode the first frame You don't (or shouldn't) need a decoder in order to do bitstream filtering, since it works on packets (codec copy) and not frames. Since the decoder you submitted is, if committed, going to be external, it will not be present in most libavcodec builds out there. This means that you should ensure parsing of such files can succeed without a decoder, so bsfs can run without it. If not possible already in the state you wrote it in patch 8/11, it can be achieved by temporarily setting some values in the parser until an internal decoder is implemented. For AV1 i had to set avctx dimensions and pixel format, both things i removed once the internal decoder (as barebones as it is) was introduced, since it's its job. I assume h266 will require the same. > --- > configure | 1 + > libavcodec/Makefile | 1 + > libavcodec/bitstream_filters.c | 1 + > libavcodec/vvc_metadata_bsf.c | 243 +++++++++++++++++++++++++++++++++ > 4 files changed, 246 insertions(+) > create mode 100644 libavcodec/vvc_metadata_bsf.c > > diff --git a/configure b/configure > index 5ff743d9c2..b41f2af151 100755 > --- a/configure > +++ b/configure > @@ -3184,6 +3184,7 @@ mjpeg2jpeg_bsf_select="jpegtables" > mpeg2_metadata_bsf_select="cbs_mpeg2" > trace_headers_bsf_select="cbs" > vp9_metadata_bsf_select="cbs_vp9" > +vvc_metadata_bsf_select="cbs_h266" For the sake of consistency, this bsf should be called h266_metadata and not vcc_metadata, seeing the other two are h264 and h265 respectively. It's part of what i suggested, but guess it got mixed in between the rest.
Hi James, On Sat, Jan 9, 2021 at 10:53 PM James Almer <jamrial@gmail.com> wrote: > On 1/9/2021 4:34 AM, Nuo Mi wrote: > > use following command to test: > > ffmpeg -i in.bin -c:v copy -bsf vvc_metadata -f vvc out.bin > > > > 79.11%(197/249) can bit match with original clips > > 6.83%(17/249) are not bit match, the original clips has redundant > emulation prevent bytes > > other all failed since vvdec failed to decode the first frame > > You don't (or shouldn't) need a decoder in order to do bitstream > filtering, since it works on packets (codec copy) and not frames. > > Since the decoder you submitted is, if committed, going to be external, > it will not be present in most libavcodec builds out there. This means > that you should ensure parsing of such files can succeed without a > decoder, so bsfs can run without it. If not possible already in the > state you wrote it in patch 8/11, it can be achieved by temporarily > setting some values in the parser until an internal decoder is implemented. > For AV1 i had to set avctx dimensions and pixel format, both things i > removed once the internal decoder (as barebones as it is) was > introduced, since it's its job. I assume h266 will require the same. Thanks for suggestion. I will try it. I tried to disable hevc native decoder, and use hevc bsf only. It got the same error as vvc. Do we need fix this too? > > --- > > configure | 1 + > > libavcodec/Makefile | 1 + > > libavcodec/bitstream_filters.c | 1 + > > libavcodec/vvc_metadata_bsf.c | 243 +++++++++++++++++++++++++++++++++ > > 4 files changed, 246 insertions(+) > > create mode 100644 libavcodec/vvc_metadata_bsf.c > > > > diff --git a/configure b/configure > > index 5ff743d9c2..b41f2af151 100755 > > --- a/configure > > +++ b/configure > > @@ -3184,6 +3184,7 @@ mjpeg2jpeg_bsf_select="jpegtables" > > mpeg2_metadata_bsf_select="cbs_mpeg2" > > trace_headers_bsf_select="cbs" > > vp9_metadata_bsf_select="cbs_vp9" > > +vvc_metadata_bsf_select="cbs_h266" > > For the sake of consistency, this bsf should be called h266_metadata and > not vcc_metadata, seeing the other two are h264 and h265 respectively. > It's part of what i suggested, but guess it got mixed in between the rest. > This is a funny part. Seems the filter name is hevc_metadata. But the class name is h265_metadata_bsf... > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
On Sat, Jan 9, 2021 at 11:11 PM Nuo Mi <nuomi2021@gmail.com> wrote: > > Hi James, > On Sat, Jan 9, 2021 at 10:53 PM James Almer <jamrial@gmail.com> wrote: > >> On 1/9/2021 4:34 AM, Nuo Mi wrote: >> > use following command to test: >> > ffmpeg -i in.bin -c:v copy -bsf vvc_metadata -f vvc out.bin >> > >> > 79.11%(197/249) can bit match with original clips >> > 6.83%(17/249) are not bit match, the original clips has redundant >> emulation prevent bytes >> > other all failed since vvdec failed to decode the first frame >> >> You don't (or shouldn't) need a decoder in order to do bitstream >> filtering, since it works on packets (codec copy) and not frames. >> >> Since the decoder you submitted is, if committed, going to be external, >> it will not be present in most libavcodec builds out there. This means >> that you should ensure parsing of such files can succeed without a >> decoder, so bsfs can run without it. If not possible already in the >> state you wrote it in patch 8/11, it can be achieved by temporarily >> setting some values in the parser until an internal decoder is >> implemented. >> For AV1 i had to set avctx dimensions and pixel format, both things i >> removed once the internal decoder (as barebones as it is) was >> introduced, since it's its job. I assume h266 will require the same. > > Thanks for suggestion. I will try it. > I tried to disable hevc native decoder, and use hevc bsf only. It got the > same error as vvc. > Do we need fix this too? > We have set dimensions and format code in 8/11, but it still need decoder. I will dig it more. > > > > >> > --- >> > configure | 1 + >> > libavcodec/Makefile | 1 + >> > libavcodec/bitstream_filters.c | 1 + >> > libavcodec/vvc_metadata_bsf.c | 243 +++++++++++++++++++++++++++++++++ >> > 4 files changed, 246 insertions(+) >> > create mode 100644 libavcodec/vvc_metadata_bsf.c >> > >> > diff --git a/configure b/configure >> > index 5ff743d9c2..b41f2af151 100755 >> > --- a/configure >> > +++ b/configure >> > @@ -3184,6 +3184,7 @@ mjpeg2jpeg_bsf_select="jpegtables" >> > mpeg2_metadata_bsf_select="cbs_mpeg2" >> > trace_headers_bsf_select="cbs" >> > vp9_metadata_bsf_select="cbs_vp9" >> > +vvc_metadata_bsf_select="cbs_h266" >> >> For the sake of consistency, this bsf should be called h266_metadata and >> not vcc_metadata, seeing the other two are h264 and h265 respectively. >> It's part of what i suggested, but guess it got mixed in between the rest. >> > This is a funny part. Seems the filter name is hevc_metadata. But the > class name is h265_metadata_bsf... > >> _______________________________________________ >> ffmpeg-devel mailing list >> ffmpeg-devel@ffmpeg.org >> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel >> >> To unsubscribe, visit link above, or email >> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". > >
On 1/9/2021 12:11 PM, Nuo Mi wrote: > Hi James, > On Sat, Jan 9, 2021 at 10:53 PM James Almer <jamrial@gmail.com> wrote: > >> On 1/9/2021 4:34 AM, Nuo Mi wrote: >>> use following command to test: >>> ffmpeg -i in.bin -c:v copy -bsf vvc_metadata -f vvc out.bin >>> >>> 79.11%(197/249) can bit match with original clips >>> 6.83%(17/249) are not bit match, the original clips has redundant >> emulation prevent bytes >>> other all failed since vvdec failed to decode the first frame >> >> You don't (or shouldn't) need a decoder in order to do bitstream >> filtering, since it works on packets (codec copy) and not frames. >> >> Since the decoder you submitted is, if committed, going to be external, >> it will not be present in most libavcodec builds out there. This means >> that you should ensure parsing of such files can succeed without a >> decoder, so bsfs can run without it. If not possible already in the >> state you wrote it in patch 8/11, it can be achieved by temporarily >> setting some values in the parser until an internal decoder is implemented. >> For AV1 i had to set avctx dimensions and pixel format, both things i >> removed once the internal decoder (as barebones as it is) was >> introduced, since it's its job. I assume h266 will require the same. > > Thanks for suggestion. I will try it. > I tried to disable hevc native decoder, and use hevc bsf only. It got the > same error as vvc. > Do we need fix this too? For now, the objective is not requiring the presence of an external decoder to ensure an unrelated internal module works. Fixing this same issue when it happens after disabling an internal decoder could be looked into, too, but that's unrelated. > > > > >>> --- >>> configure | 1 + >>> libavcodec/Makefile | 1 + >>> libavcodec/bitstream_filters.c | 1 + >>> libavcodec/vvc_metadata_bsf.c | 243 +++++++++++++++++++++++++++++++++ >>> 4 files changed, 246 insertions(+) >>> create mode 100644 libavcodec/vvc_metadata_bsf.c >>> >>> diff --git a/configure b/configure >>> index 5ff743d9c2..b41f2af151 100755 >>> --- a/configure >>> +++ b/configure >>> @@ -3184,6 +3184,7 @@ mjpeg2jpeg_bsf_select="jpegtables" >>> mpeg2_metadata_bsf_select="cbs_mpeg2" >>> trace_headers_bsf_select="cbs" >>> vp9_metadata_bsf_select="cbs_vp9" >>> +vvc_metadata_bsf_select="cbs_h266" >> >> For the sake of consistency, this bsf should be called h266_metadata and >> not vcc_metadata, seeing the other two are h264 and h265 respectively. >> It's part of what i suggested, but guess it got mixed in between the rest. >> > This is a funny part. Seems the filter name is hevc_metadata. But the class > name is h265_metadata_bsf... Well, that's surprising. Maybe follow suit and call the file h266_metadata_bsf.c, too, but otherwise guess it's fine as is, and the bsf can be named vvc_metadata. > >> _______________________________________________ >> ffmpeg-devel mailing list >> ffmpeg-devel@ffmpeg.org >> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel >> >> To unsubscribe, visit link above, or email >> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >
On 1/9/2021 12:14 PM, Nuo Mi wrote: > On Sat, Jan 9, 2021 at 11:11 PM Nuo Mi <nuomi2021@gmail.com> wrote: > >> >> Hi James, >> On Sat, Jan 9, 2021 at 10:53 PM James Almer <jamrial@gmail.com> wrote: >> >>> On 1/9/2021 4:34 AM, Nuo Mi wrote: >>>> use following command to test: >>>> ffmpeg -i in.bin -c:v copy -bsf vvc_metadata -f vvc out.bin >>>> >>>> 79.11%(197/249) can bit match with original clips >>>> 6.83%(17/249) are not bit match, the original clips has redundant >>> emulation prevent bytes >>>> other all failed since vvdec failed to decode the first frame >>> >>> You don't (or shouldn't) need a decoder in order to do bitstream >>> filtering, since it works on packets (codec copy) and not frames. >>> >>> Since the decoder you submitted is, if committed, going to be external, >>> it will not be present in most libavcodec builds out there. This means >>> that you should ensure parsing of such files can succeed without a >>> decoder, so bsfs can run without it. If not possible already in the >>> state you wrote it in patch 8/11, it can be achieved by temporarily >>> setting some values in the parser until an internal decoder is >>> implemented. >>> For AV1 i had to set avctx dimensions and pixel format, both things i >>> removed once the internal decoder (as barebones as it is) was >>> introduced, since it's its job. I assume h266 will require the same. >> >> Thanks for suggestion. I will try it. >> I tried to disable hevc native decoder, and use hevc bsf only. It got the >> same error as vvc. >> Do we need fix this too? >> > We have set dimensions and format code in 8/11, but it still need decoder. > I will dig it more. You're setting them in AVCodecParserContext, which is standard for parsers. What you need to do until an internal decoder is introduced is setting the same values in avctx (You're currently setting profile and level there). For dimensions, compare the current values in avctx with the ones you read from the bitstream, and use ff_set_dimensions() to set them if they don't match. That should hopefully be enough. > >> >> >> >> >>>> --- >>>> configure | 1 + >>>> libavcodec/Makefile | 1 + >>>> libavcodec/bitstream_filters.c | 1 + >>>> libavcodec/vvc_metadata_bsf.c | 243 +++++++++++++++++++++++++++++++++ >>>> 4 files changed, 246 insertions(+) >>>> create mode 100644 libavcodec/vvc_metadata_bsf.c >>>> >>>> diff --git a/configure b/configure >>>> index 5ff743d9c2..b41f2af151 100755 >>>> --- a/configure >>>> +++ b/configure >>>> @@ -3184,6 +3184,7 @@ mjpeg2jpeg_bsf_select="jpegtables" >>>> mpeg2_metadata_bsf_select="cbs_mpeg2" >>>> trace_headers_bsf_select="cbs" >>>> vp9_metadata_bsf_select="cbs_vp9" >>>> +vvc_metadata_bsf_select="cbs_h266" >>> >>> For the sake of consistency, this bsf should be called h266_metadata and >>> not vcc_metadata, seeing the other two are h264 and h265 respectively. >>> It's part of what i suggested, but guess it got mixed in between the rest. >>> >> This is a funny part. Seems the filter name is hevc_metadata. But the >> class name is h265_metadata_bsf... >> >>> _______________________________________________ >>> ffmpeg-devel mailing list >>> ffmpeg-devel@ffmpeg.org >>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel >>> >>> To unsubscribe, visit link above, or email >>> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >> >> > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >
On Sat, Jan 9, 2021 at 11:53 PM James Almer <jamrial@gmail.com> wrote: > On 1/9/2021 12:11 PM, Nuo Mi wrote: > > Hi James, > > On Sat, Jan 9, 2021 at 10:53 PM James Almer <jamrial@gmail.com> wrote: > > > >> On 1/9/2021 4:34 AM, Nuo Mi wrote: > >>> use following command to test: > >>> ffmpeg -i in.bin -c:v copy -bsf vvc_metadata -f vvc out.bin > >>> > >>> 79.11%(197/249) can bit match with original clips > >>> 6.83%(17/249) are not bit match, the original clips has redundant > >> emulation prevent bytes > >>> other all failed since vvdec failed to decode the first frame > >> > >> You don't (or shouldn't) need a decoder in order to do bitstream > >> filtering, since it works on packets (codec copy) and not frames. > >> > >> Since the decoder you submitted is, if committed, going to be external, > >> it will not be present in most libavcodec builds out there. This means > >> that you should ensure parsing of such files can succeed without a > >> decoder, so bsfs can run without it. If not possible already in the > >> state you wrote it in patch 8/11, it can be achieved by temporarily > >> setting some values in the parser until an internal decoder is > implemented. > >> For AV1 i had to set avctx dimensions and pixel format, both things i > >> removed once the internal decoder (as barebones as it is) was > >> introduced, since it's its job. I assume h266 will require the same. > > > > Thanks for suggestion. I will try it. > > I tried to disable hevc native decoder, and use hevc bsf only. It got the > > same error as vvc. > > Do we need fix this too? > > For now, the objective is not requiring the presence of an external > decoder to ensure an unrelated internal module works. > > Fixing this same issue when it happens after disabling an internal > decoder could be looked into, too, but that's unrelated thanks for the suggestion, I have sent the v3 for the parser. We can get almost 100% match with VTM11 streams > > > > > > > > > > >>> --- > >>> configure | 1 + > >>> libavcodec/Makefile | 1 + > >>> libavcodec/bitstream_filters.c | 1 + > >>> libavcodec/vvc_metadata_bsf.c | 243 > +++++++++++++++++++++++++++++++++ > >>> 4 files changed, 246 insertions(+) > >>> create mode 100644 libavcodec/vvc_metadata_bsf.c > >>> > >>> diff --git a/configure b/configure > >>> index 5ff743d9c2..b41f2af151 100755 > >>> --- a/configure > >>> +++ b/configure > >>> @@ -3184,6 +3184,7 @@ mjpeg2jpeg_bsf_select="jpegtables" > >>> mpeg2_metadata_bsf_select="cbs_mpeg2" > >>> trace_headers_bsf_select="cbs" > >>> vp9_metadata_bsf_select="cbs_vp9" > >>> +vvc_metadata_bsf_select="cbs_h266" > >> > >> For the sake of consistency, this bsf should be called h266_metadata and > >> not vcc_metadata, seeing the other two are h264 and h265 respectively. > >> It's part of what i suggested, but guess it got mixed in between the > rest. > >> > > This is a funny part. Seems the filter name is hevc_metadata. But the > class > > name is h265_metadata_bsf... > > Well, that's surprising. Maybe follow suit and call the file > h266_metadata_bsf.c, too, but otherwise guess it's fine as is, and the > bsf can be named vvc_metadata. > > > > >> _______________________________________________ > >> ffmpeg-devel mailing list > >> ffmpeg-devel@ffmpeg.org > >> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > >> > >> To unsubscribe, visit link above, or email > >> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". > > _______________________________________________ > > ffmpeg-devel mailing list > > ffmpeg-devel@ffmpeg.org > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > > > To unsubscribe, visit link above, or email > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". > > > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
diff --git a/configure b/configure index 5ff743d9c2..b41f2af151 100755 --- a/configure +++ b/configure @@ -3184,6 +3184,7 @@ mjpeg2jpeg_bsf_select="jpegtables" mpeg2_metadata_bsf_select="cbs_mpeg2" trace_headers_bsf_select="cbs" vp9_metadata_bsf_select="cbs_vp9" +vvc_metadata_bsf_select="cbs_h266" # external libraries aac_at_decoder_deps="audiotoolbox" diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 30e9a92e9f..d4cd64e43c 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1166,6 +1166,7 @@ OBJS-$(CONFIG_VP9_METADATA_BSF) += vp9_metadata_bsf.o OBJS-$(CONFIG_VP9_RAW_REORDER_BSF) += vp9_raw_reorder_bsf.o OBJS-$(CONFIG_VP9_SUPERFRAME_BSF) += vp9_superframe_bsf.o OBJS-$(CONFIG_VP9_SUPERFRAME_SPLIT_BSF) += vp9_superframe_split_bsf.o +OBJS-$(CONFIG_VVC_METADATA_BSF) += vvc_metadata_bsf.o # thread libraries OBJS-$(HAVE_LIBC_MSVCRT) += file_open.o diff --git a/libavcodec/bitstream_filters.c b/libavcodec/bitstream_filters.c index b26d6a910e..001a7bb3a4 100644 --- a/libavcodec/bitstream_filters.c +++ b/libavcodec/bitstream_filters.c @@ -60,6 +60,7 @@ extern const AVBitStreamFilter ff_vp9_metadata_bsf; extern const AVBitStreamFilter ff_vp9_raw_reorder_bsf; extern const AVBitStreamFilter ff_vp9_superframe_bsf; extern const AVBitStreamFilter ff_vp9_superframe_split_bsf; +extern const AVBitStreamFilter ff_vvc_metadata_bsf; #include "libavcodec/bsf_list.c" diff --git a/libavcodec/vvc_metadata_bsf.c b/libavcodec/vvc_metadata_bsf.c new file mode 100644 index 0000000000..a7deced47c --- /dev/null +++ b/libavcodec/vvc_metadata_bsf.c @@ -0,0 +1,243 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/common.h" +#include "libavutil/opt.h" + +#include "bsf.h" +#include "bsf_internal.h" +#include "cbs.h" +#include "cbs_h266.h" +#include "vvc.h" + +enum { + PASS, + INSERT, + REMOVE, +}; + +typedef struct VVCMetadataContext { + const AVClass *class; + + CodedBitstreamContext *input; + CodedBitstreamContext *output; + CodedBitstreamFragment access_unit; + + H266RawAUD aud_nal; + + int aud; + +} VVCMetadataContext; + +static int vvc_metadata_update_side_data(AVBSFContext *bsf, AVPacket *pkt) +{ + VVCMetadataContext *ctx = bsf->priv_data; + CodedBitstreamFragment *au = &ctx->access_unit; + uint8_t *side_data; + int side_data_size; + int err; + + side_data = av_packet_get_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA, + &side_data_size); + if (!side_data_size) + return 0; + + err = ff_cbs_read(ctx->input, au, side_data, side_data_size); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to read extradata from packet side data.\n"); + return err; + } + + err = ff_cbs_write_fragment_data(ctx->output, au); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to write extradata into packet side data.\n"); + return err; + } + + side_data = av_packet_new_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA, au->data_size); + if (!side_data) + return AVERROR(ENOMEM); + memcpy(side_data, au->data, au->data_size); + + ff_cbs_fragment_reset(au); + + return 0; +} + +static int vvc_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) +{ + VVCMetadataContext *ctx = bsf->priv_data; + CodedBitstreamFragment *au = &ctx->access_unit; + int err, i; + + err = ff_bsf_get_packet_ref(bsf, pkt); + if (err < 0) + return err; + + err = vvc_metadata_update_side_data(bsf, pkt); + if (err < 0) + goto fail; + + err = ff_cbs_read_packet(ctx->input, au, pkt); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to read packet.\n"); + goto fail; + } + + if (au->nb_units == 0) { + av_log(bsf, AV_LOG_ERROR, "No NAL units in packet.\n"); + err = AVERROR_INVALIDDATA; + goto fail; + } + + // If an AUD is present, it must be the first NAL unit. + if (au->units[0].type == VVC_AUD_NUT) { + if (ctx->aud == REMOVE) + ff_cbs_delete_unit(au, 0); + } else { + if (ctx->aud == INSERT) { + H266RawAUD *aud = &ctx->aud_nal; + int pic_type = 0, temporal_id = 8, layer_id = 0; + CodedBitstreamH266Context *h266 = ctx->input->priv_data; + for (i = 0; i < au->nb_units; i++) { + const H266RawNALUnitHeader *nal = au->units[i].content; + if (!nal) + continue; + if (nal->nuh_temporal_id_plus1 < temporal_id + 1) + temporal_id = nal->nuh_temporal_id_plus1 - 1; + + if (au->units[i].type <= VVC_RSV_IRAP_11) { + const H266RawSlice *slice = au->units[i].content; + layer_id = nal->nuh_layer_id; + if (slice->header.sh_slice_type == VVC_SLICE_TYPE_B && + pic_type < 2) + pic_type = 2; + if (slice->header.sh_slice_type == VVC_SLICE_TYPE_P && + pic_type < 1) + pic_type = 1; + } + } + + aud->nal_unit_header = (H266RawNALUnitHeader) { + .nal_unit_type = VVC_AUD_NUT, + .nuh_layer_id = layer_id, + .nuh_temporal_id_plus1 = temporal_id + 1, + }; + aud->aud_pic_type = pic_type; + aud->aud_irap_or_gdr_flag = h266->active_ph->ph_gdr_or_irap_pic_flag; + + err = ff_cbs_insert_unit_content(au, 0, VVC_AUD_NUT, aud, NULL); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to insert AUD.\n"); + goto fail; + } + } + } + + err = ff_cbs_write_packet(ctx->output, pkt, au); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to write packet.\n"); + goto fail; + } + +fail: + ff_cbs_fragment_reset(au); + + if (err < 0) + av_packet_unref(pkt); + + return err; +} + +static int vvc_metadata_init(AVBSFContext *bsf) +{ + VVCMetadataContext *ctx = bsf->priv_data; + CodedBitstreamFragment *au = &ctx->access_unit; + int err; + + err = ff_cbs_init(&ctx->input, AV_CODEC_ID_VVC, bsf); + if (err < 0) + return err; + err = ff_cbs_init(&ctx->output, AV_CODEC_ID_VVC, bsf); + if (err < 0) + return err; + + if (bsf->par_in->extradata) { + err = ff_cbs_read_extradata(ctx->input, au, bsf->par_in); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to read extradata.\n"); + goto fail; + } + + err = ff_cbs_write_extradata(ctx->output, bsf->par_out, au); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to write extradata.\n"); + goto fail; + } + } + +fail: + ff_cbs_fragment_reset(au); + return err; +} + +static void vvc_metadata_close(AVBSFContext *bsf) +{ + VVCMetadataContext *ctx = bsf->priv_data; + + ff_cbs_fragment_free(&ctx->access_unit); + ff_cbs_close(&ctx->input); + ff_cbs_close(&ctx->output); +} + +#define OFFSET(x) offsetof(VVCMetadataContext, x) +#define FLAGS (AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_BSF_PARAM) +static const AVOption vvc_metadata_options[] = { + { "aud", "Access Unit Delimiter NAL units", + OFFSET(aud), AV_OPT_TYPE_INT, + { .i64 = PASS }, PASS, REMOVE, FLAGS, "aud" }, + { "pass", NULL, 0, AV_OPT_TYPE_CONST, + { .i64 = PASS }, .flags = FLAGS, .unit = "aud" }, + { "insert", NULL, 0, AV_OPT_TYPE_CONST, + { .i64 = INSERT }, .flags = FLAGS, .unit = "aud" }, + { "remove", NULL, 0, AV_OPT_TYPE_CONST, + { .i64 = REMOVE }, .flags = FLAGS, .unit = "aud" }, + + { NULL } +}; + +static const AVClass vvc_metadata_class = { + .class_name = "vvc_metadata_bsf", + .item_name = av_default_item_name, + .option = vvc_metadata_options, + .version = LIBAVUTIL_VERSION_INT, +}; + +static const enum AVCodecID vvc_metadata_codec_ids[] = { + AV_CODEC_ID_VVC, AV_CODEC_ID_NONE, +}; + +const AVBitStreamFilter ff_vvc_metadata_bsf = { + .name = "vvc_metadata", + .priv_data_size = sizeof(VVCMetadataContext), + .priv_class = &vvc_metadata_class, + .init = &vvc_metadata_init, + .close = &vvc_metadata_close, + .filter = &vvc_metadata_filter, + .codec_ids = vvc_metadata_codec_ids, +};