| Message ID | 20180709182654.9996-6-jamrial@gmail.com |
|---|---|
| State | New |
| Headers | show |
On Mon, Jul 09, 2018 at 03:26:54PM -0300, James Almer wrote: > Signed-off-by: James Almer <jamrial@gmail.com> > --- > ff_av1_filter_obus() could eventually be replaced by an autoinserted > filter_units bsf, assuming it doesn't slow down the muxing process > too much (CBS is fast reading packets, but not so much assembling and > writing packets). > ff_isom_write_av1c() however can't be replaced given filter_units > doesn't handle extradata (either codecpar or packet side data). > [...] > diff --git a/libavformat/av1.h b/libavformat/av1.h > new file mode 100644 > index 0000000000..733034c12d > --- /dev/null > +++ b/libavformat/av1.h > @@ -0,0 +1,70 @@ > +/* > + * AV1 helper functions for muxers > + * > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +#ifndef AVFORMAT_AV1_H > +#define AVFORMAT_AV1_H > + > +#include <stdint.h> > + > +#include "avio.h" > + > +/** > + * Filter out AV1 OBUs not meant to be present in ISOBMFF sample data and write > + * the resulting bitstream to the provided AVIOContext. > + * > + * @param pb pointer to the AVIOContext where the filtered bitstream shall be > + * written > + * @param buf input data buffer > + * @param size size of the input data buffer > + * > + * @return the amount of bytes written in case of success, a negative AVERROR > + * code in case of failure > + */ > +int ff_av1_filter_obus(AVIOContext *pb, const uint8_t *buf, int size); > + > +/** > + * Filter out AV1 OBUs not meant to be present in ISOBMFF sample data and write > + * the resulting bitstream to a newly allocated data buffer. > + * > + * @param pb pointer to the AVIOContext where the filtered bitstream shall be > + * written > + * @param buf input data buffer > + * @param out pointer to pointer that will hold the allocated data buffer > + * @param size size of the input data buffer. The size of the resulting output > + data buffer will be written here > + * > + * @return the amount of bytes written in case of success, a negative AVERROR > + * code in case of failure this leaves it unspecified what happens to out/size in case of errors are they 0/null are they undefined, left as before ? > + */ > +int ff_av1_filter_obus_buf(const uint8_t *buf, uint8_t **out, int *size); > + > +/** > + * Writes AV1 extradata (Sequence Header and Metadata OBUs) to the provided > + * AVIOContext. > + * > + * @param pb pointer to the AVIOContext where the hvcC shall be written > + * @param buf input data buffer > + * @param size size of the input data buffer very minor nitpick but you could add "in bytes" > + * > + * @return 0 in case of success, a negative AVERROR code in case of failure if >= 0 is defined as success then its possible to use this in the future for some additional information without the need to review all callers i guess most of this doesnt matter much as its not public API ... [...]
On 7/9/2018 6:08 PM, Michael Niedermayer wrote: > On Mon, Jul 09, 2018 at 03:26:54PM -0300, James Almer wrote: >> Signed-off-by: James Almer <jamrial@gmail.com> >> --- >> ff_av1_filter_obus() could eventually be replaced by an autoinserted >> filter_units bsf, assuming it doesn't slow down the muxing process >> too much (CBS is fast reading packets, but not so much assembling and >> writing packets). >> ff_isom_write_av1c() however can't be replaced given filter_units >> doesn't handle extradata (either codecpar or packet side data). >> > [...] >> diff --git a/libavformat/av1.h b/libavformat/av1.h >> new file mode 100644 >> index 0000000000..733034c12d >> --- /dev/null >> +++ b/libavformat/av1.h >> @@ -0,0 +1,70 @@ >> +/* >> + * AV1 helper functions for muxers >> + * >> + * This file is part of FFmpeg. >> + * >> + * FFmpeg is free software; you can redistribute it and/or >> + * modify it under the terms of the GNU Lesser General Public >> + * License as published by the Free Software Foundation; either >> + * version 2.1 of the License, or (at your option) any later version. >> + * >> + * FFmpeg is distributed in the hope that it will be useful, >> + * but WITHOUT ANY WARRANTY; without even the implied warranty of >> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU >> + * Lesser General Public License for more details. >> + * >> + * You should have received a copy of the GNU Lesser General Public >> + * License along with FFmpeg; if not, write to the Free Software >> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA >> + */ >> + >> +#ifndef AVFORMAT_AV1_H >> +#define AVFORMAT_AV1_H >> + >> +#include <stdint.h> >> + >> +#include "avio.h" >> + >> +/** >> + * Filter out AV1 OBUs not meant to be present in ISOBMFF sample data and write >> + * the resulting bitstream to the provided AVIOContext. >> + * >> + * @param pb pointer to the AVIOContext where the filtered bitstream shall be >> + * written >> + * @param buf input data buffer >> + * @param size size of the input data buffer >> + * >> + * @return the amount of bytes written in case of success, a negative AVERROR >> + * code in case of failure >> + */ >> +int ff_av1_filter_obus(AVIOContext *pb, const uint8_t *buf, int size); >> + >> +/** >> + * Filter out AV1 OBUs not meant to be present in ISOBMFF sample data and write >> + * the resulting bitstream to a newly allocated data buffer. >> + * >> + * @param pb pointer to the AVIOContext where the filtered bitstream shall be >> + * written >> + * @param buf input data buffer > >> + * @param out pointer to pointer that will hold the allocated data buffer >> + * @param size size of the input data buffer. The size of the resulting output >> + data buffer will be written here >> + * >> + * @return the amount of bytes written in case of success, a negative AVERROR >> + * code in case of failure > > this leaves it unspecified what happens to out/size in case of errors > are they 0/null are they undefined, left as before ? Depending on when in the function an error happens, i will either be untouched or *out will be set to NULL. I can make it so the latter always happens in case of failure and document it as such if that's preferred, but the current callers pass a pointer to a pointer set to NULL, so that's already guaranteed. This is based on ff_avc_parse_nal_units_buf() and ff_hevc_annexb2mp4_buf(), for that matter, so those could also be fixed. > > >> + */ >> +int ff_av1_filter_obus_buf(const uint8_t *buf, uint8_t **out, int *size); >> + >> +/** >> + * Writes AV1 extradata (Sequence Header and Metadata OBUs) to the provided >> + * AVIOContext. >> + * >> + * @param pb pointer to the AVIOContext where the hvcC shall be written >> + * @param buf input data buffer > >> + * @param size size of the input data buffer > > very minor nitpick but you could add "in bytes" Sure. > > >> + * >> + * @return 0 in case of success, a negative AVERROR code in case of failure > > if >= 0 is defined as success then its possible to use this in the future > for some additional information without the need to review all callers I can make this change, but as you well mention below it doesn't really matter since it's internal, so anything defined now can be changed later. > > i guess most of this doesnt matter much as its not public API ... > > > > [...] > > > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel >
Hi James, On Mon, Jul 9, 2018 at 11:26 AM, James Almer <jamrial@gmail.com> wrote: [...] > @@ -5438,7 +5461,7 @@ int ff_mov_write_packet(AVFormatContext *s, > AVPacket *pkt) > av_log(s, AV_LOG_WARNING, "pts has no value\n"); > pkt->pts = pkt->dts; > } > - if (pkt->dts != pkt->pts) > + if (pkt->dts != pkt->pts && par->codec_id != AV_CODEC_ID_AV1) > trk->flags |= MOV_TRACK_CTTS; > trk->cluster[trk->entry].cts = pkt->pts - pkt->dts; > trk->cluster[trk->entry].flags = 0; > [...] Interesting, is there a spec document that would explicitly mention that AV1 forbids "ctts" atom ? Thanks!
On Tue, Jul 10, 2018 at 12:29 AM Baptiste Coudurier <baptiste.coudurier@gmail.com> wrote: > > Hi James, > > On Mon, Jul 9, 2018 at 11:26 AM, James Almer <jamrial@gmail.com> wrote: > > [...] > > > > @@ -5438,7 +5461,7 @@ int ff_mov_write_packet(AVFormatContext *s, > > AVPacket *pkt) > > av_log(s, AV_LOG_WARNING, "pts has no value\n"); > > pkt->pts = pkt->dts; > > } > > - if (pkt->dts != pkt->pts) > > + if (pkt->dts != pkt->pts && par->codec_id != AV_CODEC_ID_AV1) > > trk->flags |= MOV_TRACK_CTTS; > > trk->cluster[trk->entry].cts = pkt->pts - pkt->dts; > > trk->cluster[trk->entry].flags = 0; > > > > [...] > > Interesting, is there a spec document that would explicitly mention that > AV1 forbids "ctts" atom ? > The draft spec does: https://aomediacodec.github.io/av1-isobmff/#sampleformat Unlike many video standards, AV1 does not distinguish the display order from the decoding order, but achieves similar effects by grouping multiple frames within a sample. Therefore, composition offsets are not used. In tracks using the AV1SampleEntry, the ctts box and composition offsets in movie fragments SHALL NOT be used. - Hendrik
On 7/9/2018 7:23 PM, Baptiste Coudurier wrote: > Hi James, > > On Mon, Jul 9, 2018 at 11:26 AM, James Almer <jamrial@gmail.com> wrote: > > [...] > > >> @@ -5438,7 +5461,7 @@ int ff_mov_write_packet(AVFormatContext *s, >> AVPacket *pkt) >> av_log(s, AV_LOG_WARNING, "pts has no value\n"); >> pkt->pts = pkt->dts; >> } >> - if (pkt->dts != pkt->pts) >> + if (pkt->dts != pkt->pts && par->codec_id != AV_CODEC_ID_AV1) >> trk->flags |= MOV_TRACK_CTTS; >> trk->cluster[trk->entry].cts = pkt->pts - pkt->dts; >> trk->cluster[trk->entry].flags = 0; >> > > [...] > > Interesting, is there a spec document that would explicitly mention that > AV1 forbids "ctts" atom ? > > Thanks! Yes, https://aomediacodec.github.io/av1-isobmff/#sampleformat "Unlike many video standards, AV1 does not distinguish the display order from the decoding order, but achieves similar effects by grouping multiple frames within a sample. Therefore, composition offsets are not used. In tracks using the AV1SampleEntry, the ctts box and composition offsets in movie fragments SHALL NOT be used. Similarly, the is_leading flag, if used, SHALL be set to 0 or 2."
On Mon, Jul 09, 2018 at 03:26:54PM -0300, James Almer wrote: > Signed-off-by: James Almer <jamrial@gmail.com> > --- > ff_av1_filter_obus() could eventually be replaced by an autoinserted > filter_units bsf, assuming it doesn't slow down the muxing process > too much (CBS is fast reading packets, but not so much assembling and > writing packets). > ff_isom_write_av1c() however can't be replaced given filter_units > doesn't handle extradata (either codecpar or packet side data). [...] > +#endif /* AVFORMAT_AV1_H */ > diff --git a/libavformat/movenc.c b/libavformat/movenc.c > index fe0a244a8f..784df6d08d 100644 > --- a/libavformat/movenc.c > +++ b/libavformat/movenc.c > @@ -30,6 +30,7 @@ > #include "riff.h" > #include "avio.h" > #include "isom.h" > +#include "av1.h" > #include "avc.h" > #include "libavcodec/ac3_parser_internal.h" > #include "libavcodec/dnxhddata.h" > @@ -1163,6 +1164,19 @@ static int mov_write_d263_tag(AVIOContext *pb) > return 0xf; > } > > +static int mov_write_av1c_tag(AVIOContext *pb, MOVTrack *track) > +{ > + int64_t pos = avio_tell(pb); > + > + avio_wb32(pb, 0); > + ffio_wfourcc(pb, "av1C"); > + avio_w8(pb, 0); /* version */ > + avio_wb24(pb, 0); /* flags */ > + avio_w8(pb, 0); /* reserved (3), initial_presentation_delay_present (1), initial_presentation_delay_minus_one/reserved (4) */ > + ff_isom_write_av1c(pb, track->vos_data, track->vos_len); > + return update_size(pb, pos); > +} > + > static int mov_write_avcc_tag(AVIOContext *pb, MOVTrack *track) > { > int64_t pos = avio_tell(pb); > @@ -2009,6 +2023,8 @@ static int mov_write_video_tag(AVIOContext *pb, MOVMuxContext *mov, MOVTrack *tr > mov_write_uuid_tag_ipod(pb); > } else if (track->par->codec_id == AV_CODEC_ID_VP9) { > mov_write_vpcc_tag(mov->fc, pb, track); > + } else if (track->par->codec_id == AV_CODEC_ID_AV1) { > + mov_write_av1c_tag(pb, track); > } else if (track->par->codec_id == AV_CODEC_ID_VC1 && track->vos_len > 0) > mov_write_dvc1_tag(pb, track); > else if (track->par->codec_id == AV_CODEC_ID_VP6F || > @@ -5319,6 +5335,13 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) > } else { > size = ff_hevc_annexb2mp4(pb, pkt->data, pkt->size, 0, NULL); > } > + } else if (par->codec_id == AV_CODEC_ID_AV1) { > + if (trk->hint_track >= 0 && trk->hint_track < mov->nb_streams) { > + ff_av1_filter_obus_buf(pkt->data, &reformatted_data, &size); > + avio_write(pb, reformatted_data, size); > + } else { > + size = ff_av1_filter_obus(pb, pkt->data, pkt->size); > + } > #if CONFIG_AC3_PARSER > } else if (par->codec_id == AV_CODEC_ID_EAC3) { > size = handle_eac3(mov, pkt, trk); > @@ -5438,7 +5461,7 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) > av_log(s, AV_LOG_WARNING, "pts has no value\n"); > pkt->pts = pkt->dts; > } > - if (pkt->dts != pkt->pts) > + if (pkt->dts != pkt->pts && par->codec_id != AV_CODEC_ID_AV1) When is dts != pts && par->codec_id == AV_CODEC_ID_AV1 ? Iam asking because if it never is then this check is not needed and if it is sometimes true then i suspect the pts values will be lost that is the demuxer input would differ from the muxer output, which doesnt seem right Maybe iam missing something [...]
On 7/12/2018 8:26 AM, Michael Niedermayer wrote: > On Mon, Jul 09, 2018 at 03:26:54PM -0300, James Almer wrote: >> Signed-off-by: James Almer <jamrial@gmail.com> >> --- >> ff_av1_filter_obus() could eventually be replaced by an autoinserted >> filter_units bsf, assuming it doesn't slow down the muxing process >> too much (CBS is fast reading packets, but not so much assembling and >> writing packets). >> ff_isom_write_av1c() however can't be replaced given filter_units >> doesn't handle extradata (either codecpar or packet side data). > [...] >> +#endif /* AVFORMAT_AV1_H */ >> diff --git a/libavformat/movenc.c b/libavformat/movenc.c >> index fe0a244a8f..784df6d08d 100644 >> --- a/libavformat/movenc.c >> +++ b/libavformat/movenc.c >> @@ -30,6 +30,7 @@ >> #include "riff.h" >> #include "avio.h" >> #include "isom.h" >> +#include "av1.h" >> #include "avc.h" >> #include "libavcodec/ac3_parser_internal.h" >> #include "libavcodec/dnxhddata.h" >> @@ -1163,6 +1164,19 @@ static int mov_write_d263_tag(AVIOContext *pb) >> return 0xf; >> } >> >> +static int mov_write_av1c_tag(AVIOContext *pb, MOVTrack *track) >> +{ >> + int64_t pos = avio_tell(pb); >> + >> + avio_wb32(pb, 0); >> + ffio_wfourcc(pb, "av1C"); >> + avio_w8(pb, 0); /* version */ >> + avio_wb24(pb, 0); /* flags */ >> + avio_w8(pb, 0); /* reserved (3), initial_presentation_delay_present (1), initial_presentation_delay_minus_one/reserved (4) */ >> + ff_isom_write_av1c(pb, track->vos_data, track->vos_len); >> + return update_size(pb, pos); >> +} >> + >> static int mov_write_avcc_tag(AVIOContext *pb, MOVTrack *track) >> { >> int64_t pos = avio_tell(pb); >> @@ -2009,6 +2023,8 @@ static int mov_write_video_tag(AVIOContext *pb, MOVMuxContext *mov, MOVTrack *tr >> mov_write_uuid_tag_ipod(pb); >> } else if (track->par->codec_id == AV_CODEC_ID_VP9) { >> mov_write_vpcc_tag(mov->fc, pb, track); >> + } else if (track->par->codec_id == AV_CODEC_ID_AV1) { >> + mov_write_av1c_tag(pb, track); >> } else if (track->par->codec_id == AV_CODEC_ID_VC1 && track->vos_len > 0) >> mov_write_dvc1_tag(pb, track); >> else if (track->par->codec_id == AV_CODEC_ID_VP6F || >> @@ -5319,6 +5335,13 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) >> } else { >> size = ff_hevc_annexb2mp4(pb, pkt->data, pkt->size, 0, NULL); >> } >> + } else if (par->codec_id == AV_CODEC_ID_AV1) { >> + if (trk->hint_track >= 0 && trk->hint_track < mov->nb_streams) { >> + ff_av1_filter_obus_buf(pkt->data, &reformatted_data, &size); >> + avio_write(pb, reformatted_data, size); >> + } else { >> + size = ff_av1_filter_obus(pb, pkt->data, pkt->size); >> + } >> #if CONFIG_AC3_PARSER >> } else if (par->codec_id == AV_CODEC_ID_EAC3) { >> size = handle_eac3(mov, pkt, trk); > >> @@ -5438,7 +5461,7 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) >> av_log(s, AV_LOG_WARNING, "pts has no value\n"); >> pkt->pts = pkt->dts; >> } >> - if (pkt->dts != pkt->pts) >> + if (pkt->dts != pkt->pts && par->codec_id != AV_CODEC_ID_AV1) > > When is dts != pts && par->codec_id == AV_CODEC_ID_AV1 ? > > Iam asking because if it never is then this check is not needed > and if it is sometimes true then i suspect the pts values will be lost > that is the demuxer input would differ from the muxer output, which doesnt > seem right > Maybe iam missing something It should never be. I just added this check for extra paranoid precaution since the spec forbids the ctts box. I can remove it.
On Thu, Jul 12, 2018 at 10:59:44AM -0300, James Almer wrote: > On 7/12/2018 8:26 AM, Michael Niedermayer wrote: > > On Mon, Jul 09, 2018 at 03:26:54PM -0300, James Almer wrote: > >> Signed-off-by: James Almer <jamrial@gmail.com> > >> --- > >> ff_av1_filter_obus() could eventually be replaced by an autoinserted > >> filter_units bsf, assuming it doesn't slow down the muxing process > >> too much (CBS is fast reading packets, but not so much assembling and > >> writing packets). > >> ff_isom_write_av1c() however can't be replaced given filter_units > >> doesn't handle extradata (either codecpar or packet side data). > > [...] > >> +#endif /* AVFORMAT_AV1_H */ > >> diff --git a/libavformat/movenc.c b/libavformat/movenc.c > >> index fe0a244a8f..784df6d08d 100644 > >> --- a/libavformat/movenc.c > >> +++ b/libavformat/movenc.c > >> @@ -30,6 +30,7 @@ > >> #include "riff.h" > >> #include "avio.h" > >> #include "isom.h" > >> +#include "av1.h" > >> #include "avc.h" > >> #include "libavcodec/ac3_parser_internal.h" > >> #include "libavcodec/dnxhddata.h" > >> @@ -1163,6 +1164,19 @@ static int mov_write_d263_tag(AVIOContext *pb) > >> return 0xf; > >> } > >> > >> +static int mov_write_av1c_tag(AVIOContext *pb, MOVTrack *track) > >> +{ > >> + int64_t pos = avio_tell(pb); > >> + > >> + avio_wb32(pb, 0); > >> + ffio_wfourcc(pb, "av1C"); > >> + avio_w8(pb, 0); /* version */ > >> + avio_wb24(pb, 0); /* flags */ > >> + avio_w8(pb, 0); /* reserved (3), initial_presentation_delay_present (1), initial_presentation_delay_minus_one/reserved (4) */ > >> + ff_isom_write_av1c(pb, track->vos_data, track->vos_len); > >> + return update_size(pb, pos); > >> +} > >> + > >> static int mov_write_avcc_tag(AVIOContext *pb, MOVTrack *track) > >> { > >> int64_t pos = avio_tell(pb); > >> @@ -2009,6 +2023,8 @@ static int mov_write_video_tag(AVIOContext *pb, MOVMuxContext *mov, MOVTrack *tr > >> mov_write_uuid_tag_ipod(pb); > >> } else if (track->par->codec_id == AV_CODEC_ID_VP9) { > >> mov_write_vpcc_tag(mov->fc, pb, track); > >> + } else if (track->par->codec_id == AV_CODEC_ID_AV1) { > >> + mov_write_av1c_tag(pb, track); > >> } else if (track->par->codec_id == AV_CODEC_ID_VC1 && track->vos_len > 0) > >> mov_write_dvc1_tag(pb, track); > >> else if (track->par->codec_id == AV_CODEC_ID_VP6F || > >> @@ -5319,6 +5335,13 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) > >> } else { > >> size = ff_hevc_annexb2mp4(pb, pkt->data, pkt->size, 0, NULL); > >> } > >> + } else if (par->codec_id == AV_CODEC_ID_AV1) { > >> + if (trk->hint_track >= 0 && trk->hint_track < mov->nb_streams) { > >> + ff_av1_filter_obus_buf(pkt->data, &reformatted_data, &size); > >> + avio_write(pb, reformatted_data, size); > >> + } else { > >> + size = ff_av1_filter_obus(pb, pkt->data, pkt->size); > >> + } > >> #if CONFIG_AC3_PARSER > >> } else if (par->codec_id == AV_CODEC_ID_EAC3) { > >> size = handle_eac3(mov, pkt, trk); > > > >> @@ -5438,7 +5461,7 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) > >> av_log(s, AV_LOG_WARNING, "pts has no value\n"); > >> pkt->pts = pkt->dts; > >> } > >> - if (pkt->dts != pkt->pts) > >> + if (pkt->dts != pkt->pts && par->codec_id != AV_CODEC_ID_AV1) > > > > When is dts != pts && par->codec_id == AV_CODEC_ID_AV1 ? > > > > Iam asking because if it never is then this check is not needed > > and if it is sometimes true then i suspect the pts values will be lost > > that is the demuxer input would differ from the muxer output, which doesnt > > seem right > > Maybe iam missing something > > It should never be. I just added this check for extra paranoid > precaution since the spec forbids the ctts box. I can remove it. I dont think this is the ideal behavior then some random thoughts: For example if pts!= dts, ommiting the ctts complies to that one rule but why did they mismatch and is teh result actually a well playable file. The user also wont know about any of this happening as there will be no warning i think. And if pts != dts, which should be use dts or pts or neither ? also iam not sure if the muxer is the best place for this. > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
On 7/12/2018 9:29 PM, Michael Niedermayer wrote: > On Thu, Jul 12, 2018 at 10:59:44AM -0300, James Almer wrote: >> On 7/12/2018 8:26 AM, Michael Niedermayer wrote: >>> On Mon, Jul 09, 2018 at 03:26:54PM -0300, James Almer wrote: >>>> Signed-off-by: James Almer <jamrial@gmail.com> >>>> --- >>>> ff_av1_filter_obus() could eventually be replaced by an autoinserted >>>> filter_units bsf, assuming it doesn't slow down the muxing process >>>> too much (CBS is fast reading packets, but not so much assembling and >>>> writing packets). >>>> ff_isom_write_av1c() however can't be replaced given filter_units >>>> doesn't handle extradata (either codecpar or packet side data). >>> [...] >>>> +#endif /* AVFORMAT_AV1_H */ >>>> diff --git a/libavformat/movenc.c b/libavformat/movenc.c >>>> index fe0a244a8f..784df6d08d 100644 >>>> --- a/libavformat/movenc.c >>>> +++ b/libavformat/movenc.c >>>> @@ -30,6 +30,7 @@ >>>> #include "riff.h" >>>> #include "avio.h" >>>> #include "isom.h" >>>> +#include "av1.h" >>>> #include "avc.h" >>>> #include "libavcodec/ac3_parser_internal.h" >>>> #include "libavcodec/dnxhddata.h" >>>> @@ -1163,6 +1164,19 @@ static int mov_write_d263_tag(AVIOContext *pb) >>>> return 0xf; >>>> } >>>> >>>> +static int mov_write_av1c_tag(AVIOContext *pb, MOVTrack *track) >>>> +{ >>>> + int64_t pos = avio_tell(pb); >>>> + >>>> + avio_wb32(pb, 0); >>>> + ffio_wfourcc(pb, "av1C"); >>>> + avio_w8(pb, 0); /* version */ >>>> + avio_wb24(pb, 0); /* flags */ >>>> + avio_w8(pb, 0); /* reserved (3), initial_presentation_delay_present (1), initial_presentation_delay_minus_one/reserved (4) */ >>>> + ff_isom_write_av1c(pb, track->vos_data, track->vos_len); >>>> + return update_size(pb, pos); >>>> +} >>>> + >>>> static int mov_write_avcc_tag(AVIOContext *pb, MOVTrack *track) >>>> { >>>> int64_t pos = avio_tell(pb); >>>> @@ -2009,6 +2023,8 @@ static int mov_write_video_tag(AVIOContext *pb, MOVMuxContext *mov, MOVTrack *tr >>>> mov_write_uuid_tag_ipod(pb); >>>> } else if (track->par->codec_id == AV_CODEC_ID_VP9) { >>>> mov_write_vpcc_tag(mov->fc, pb, track); >>>> + } else if (track->par->codec_id == AV_CODEC_ID_AV1) { >>>> + mov_write_av1c_tag(pb, track); >>>> } else if (track->par->codec_id == AV_CODEC_ID_VC1 && track->vos_len > 0) >>>> mov_write_dvc1_tag(pb, track); >>>> else if (track->par->codec_id == AV_CODEC_ID_VP6F || >>>> @@ -5319,6 +5335,13 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) >>>> } else { >>>> size = ff_hevc_annexb2mp4(pb, pkt->data, pkt->size, 0, NULL); >>>> } >>>> + } else if (par->codec_id == AV_CODEC_ID_AV1) { >>>> + if (trk->hint_track >= 0 && trk->hint_track < mov->nb_streams) { >>>> + ff_av1_filter_obus_buf(pkt->data, &reformatted_data, &size); >>>> + avio_write(pb, reformatted_data, size); >>>> + } else { >>>> + size = ff_av1_filter_obus(pb, pkt->data, pkt->size); >>>> + } >>>> #if CONFIG_AC3_PARSER >>>> } else if (par->codec_id == AV_CODEC_ID_EAC3) { >>>> size = handle_eac3(mov, pkt, trk); >>> >>>> @@ -5438,7 +5461,7 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) >>>> av_log(s, AV_LOG_WARNING, "pts has no value\n"); >>>> pkt->pts = pkt->dts; >>>> } >>>> - if (pkt->dts != pkt->pts) >>>> + if (pkt->dts != pkt->pts && par->codec_id != AV_CODEC_ID_AV1) >>> >>> When is dts != pts && par->codec_id == AV_CODEC_ID_AV1 ? >>> >>> Iam asking because if it never is then this check is not needed >>> and if it is sometimes true then i suspect the pts values will be lost >>> that is the demuxer input would differ from the muxer output, which doesnt >>> seem right >>> Maybe iam missing something >> >> It should never be. I just added this check for extra paranoid >> precaution since the spec forbids the ctts box. I can remove it. > > I dont think this is the ideal behavior then > > some random thoughts: > > For example if pts!= dts, ommiting the ctts complies to that one rule > but why did they mismatch and is teh result actually a well playable > file. > The user also wont know about any of this happening as there will be > no warning i think. > And if pts != dts, which should be use dts or pts or neither ? > also iam not sure if the muxer is the best place for this. A file reporting such timestamps would be invalid and something that should be handled at the demuxer level, yes. This check can be removed as it was a bogus addition to blindly enforce a spec rule.
diff --git a/libavformat/Makefile b/libavformat/Makefile index 8fb075f06f..f2f3aabdc2 100644 --- a/libavformat/Makefile +++ b/libavformat/Makefile @@ -302,7 +302,7 @@ OBJS-$(CONFIG_MM_DEMUXER) += mm.o OBJS-$(CONFIG_MMF_DEMUXER) += mmf.o OBJS-$(CONFIG_MMF_MUXER) += mmf.o rawenc.o OBJS-$(CONFIG_MOV_DEMUXER) += mov.o mov_chan.o mov_esds.o replaygain.o -OBJS-$(CONFIG_MOV_MUXER) += movenc.o avc.o hevc.o vpcc.o \ +OBJS-$(CONFIG_MOV_MUXER) += movenc.o av1.o avc.o hevc.o vpcc.o \ movenchint.o mov_chan.o rtp.o \ movenccenc.o rawutils.o OBJS-$(CONFIG_MP2_MUXER) += rawenc.o diff --git a/libavformat/av1.c b/libavformat/av1.c new file mode 100644 index 0000000000..3b200176a5 --- /dev/null +++ b/libavformat/av1.c @@ -0,0 +1,107 @@ +/* + * AV1 helper functions for muxers + * Copyright (c) 2018 James Almer <jamrial@gmail.com> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/mem.h" +#include "libavcodec/av1.h" +#include "libavcodec/av1_parse.h" +#include "av1.h" +#include "avio.h" + +int ff_av1_filter_obus(AVIOContext *pb, const uint8_t *buf, int size) +{ + const uint8_t *end = buf + size; + int64_t obu_size; + int start_pos, type, temporal_id, spatial_id; + + size = 0; + while (buf < end) { + int ret = parse_obu_header(buf, end - buf, &obu_size, &start_pos, + &type, &temporal_id, &spatial_id); + if (ret < 0) + return ret; + + obu_size += start_pos; + if (obu_size > INT_MAX) + return AVERROR_INVALIDDATA; + + switch (type) { + case AV1_OBU_TEMPORAL_DELIMITER: + case AV1_OBU_REDUNDANT_FRAME_HEADER: + case AV1_OBU_PADDING: + break; + default: + avio_write(pb, buf, obu_size); + size += obu_size; + break; + } + buf += obu_size; + } + + return size; +} + +int ff_av1_filter_obus_buf(const uint8_t *buf, uint8_t **out, int *size) +{ + AVIOContext *pb; + int ret; + + ret = avio_open_dyn_buf(&pb); + if (ret < 0) + return ret; + + ret = ff_av1_filter_obus(pb, buf, *size); + *size = avio_close_dyn_buf(pb, out); + + if (ret < 0) + av_freep(out); + + return ret; +} + +int ff_isom_write_av1c(AVIOContext *pb, const uint8_t *buf, int size) +{ + int64_t obu_size; + int start_pos, type, temporal_id, spatial_id; + + while (size > 0) { + int ret = parse_obu_header(buf, size, &obu_size, &start_pos, + &type, &temporal_id, &spatial_id); + if (ret < 0) + return ret; + + obu_size += start_pos; + if (obu_size > INT_MAX) + return AVERROR_INVALIDDATA; + + switch (type) { + case AV1_OBU_SEQUENCE_HEADER: + case AV1_OBU_METADATA: + avio_write(pb, buf, obu_size); + size -= obu_size; + break; + default: + break; + } + buf += obu_size; + } + + return 0; +} diff --git a/libavformat/av1.h b/libavformat/av1.h new file mode 100644 index 0000000000..733034c12d --- /dev/null +++ b/libavformat/av1.h @@ -0,0 +1,70 @@ +/* + * AV1 helper functions for muxers + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVFORMAT_AV1_H +#define AVFORMAT_AV1_H + +#include <stdint.h> + +#include "avio.h" + +/** + * Filter out AV1 OBUs not meant to be present in ISOBMFF sample data and write + * the resulting bitstream to the provided AVIOContext. + * + * @param pb pointer to the AVIOContext where the filtered bitstream shall be + * written + * @param buf input data buffer + * @param size size of the input data buffer + * + * @return the amount of bytes written in case of success, a negative AVERROR + * code in case of failure + */ +int ff_av1_filter_obus(AVIOContext *pb, const uint8_t *buf, int size); + +/** + * Filter out AV1 OBUs not meant to be present in ISOBMFF sample data and write + * the resulting bitstream to a newly allocated data buffer. + * + * @param pb pointer to the AVIOContext where the filtered bitstream shall be + * written + * @param buf input data buffer + * @param out pointer to pointer that will hold the allocated data buffer + * @param size size of the input data buffer. The size of the resulting output + data buffer will be written here + * + * @return the amount of bytes written in case of success, a negative AVERROR + * code in case of failure + */ +int ff_av1_filter_obus_buf(const uint8_t *buf, uint8_t **out, int *size); + +/** + * Writes AV1 extradata (Sequence Header and Metadata OBUs) to the provided + * AVIOContext. + * + * @param pb pointer to the AVIOContext where the hvcC shall be written + * @param buf input data buffer + * @param size size of the input data buffer + * + * @return 0 in case of success, a negative AVERROR code in case of failure + */ +int ff_isom_write_av1c(AVIOContext *pb, const uint8_t *buf, int size); + +#endif /* AVFORMAT_AV1_H */ diff --git a/libavformat/movenc.c b/libavformat/movenc.c index fe0a244a8f..784df6d08d 100644 --- a/libavformat/movenc.c +++ b/libavformat/movenc.c @@ -30,6 +30,7 @@ #include "riff.h" #include "avio.h" #include "isom.h" +#include "av1.h" #include "avc.h" #include "libavcodec/ac3_parser_internal.h" #include "libavcodec/dnxhddata.h" @@ -1163,6 +1164,19 @@ static int mov_write_d263_tag(AVIOContext *pb) return 0xf; } +static int mov_write_av1c_tag(AVIOContext *pb, MOVTrack *track) +{ + int64_t pos = avio_tell(pb); + + avio_wb32(pb, 0); + ffio_wfourcc(pb, "av1C"); + avio_w8(pb, 0); /* version */ + avio_wb24(pb, 0); /* flags */ + avio_w8(pb, 0); /* reserved (3), initial_presentation_delay_present (1), initial_presentation_delay_minus_one/reserved (4) */ + ff_isom_write_av1c(pb, track->vos_data, track->vos_len); + return update_size(pb, pos); +} + static int mov_write_avcc_tag(AVIOContext *pb, MOVTrack *track) { int64_t pos = avio_tell(pb); @@ -2009,6 +2023,8 @@ static int mov_write_video_tag(AVIOContext *pb, MOVMuxContext *mov, MOVTrack *tr mov_write_uuid_tag_ipod(pb); } else if (track->par->codec_id == AV_CODEC_ID_VP9) { mov_write_vpcc_tag(mov->fc, pb, track); + } else if (track->par->codec_id == AV_CODEC_ID_AV1) { + mov_write_av1c_tag(pb, track); } else if (track->par->codec_id == AV_CODEC_ID_VC1 && track->vos_len > 0) mov_write_dvc1_tag(pb, track); else if (track->par->codec_id == AV_CODEC_ID_VP6F || @@ -5319,6 +5335,13 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) } else { size = ff_hevc_annexb2mp4(pb, pkt->data, pkt->size, 0, NULL); } + } else if (par->codec_id == AV_CODEC_ID_AV1) { + if (trk->hint_track >= 0 && trk->hint_track < mov->nb_streams) { + ff_av1_filter_obus_buf(pkt->data, &reformatted_data, &size); + avio_write(pb, reformatted_data, size); + } else { + size = ff_av1_filter_obus(pb, pkt->data, pkt->size); + } #if CONFIG_AC3_PARSER } else if (par->codec_id == AV_CODEC_ID_EAC3) { size = handle_eac3(mov, pkt, trk); @@ -5438,7 +5461,7 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) av_log(s, AV_LOG_WARNING, "pts has no value\n"); pkt->pts = pkt->dts; } - if (pkt->dts != pkt->pts) + if (pkt->dts != pkt->pts && par->codec_id != AV_CODEC_ID_AV1) trk->flags |= MOV_TRACK_CTTS; trk->cluster[trk->entry].cts = pkt->pts - pkt->dts; trk->cluster[trk->entry].flags = 0; @@ -5512,6 +5535,7 @@ static int mov_write_single_packet(AVFormatContext *s, AVPacket *pkt) if (trk->par->codec_id == AV_CODEC_ID_MP4ALS || trk->par->codec_id == AV_CODEC_ID_AAC || + trk->par->codec_id == AV_CODEC_ID_AV1 || trk->par->codec_id == AV_CODEC_ID_FLAC) { int side_size = 0; uint8_t *side = av_packet_get_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA, &side_size); @@ -6209,15 +6233,12 @@ static int mov_init(AVFormatContext *s) pix_fmt == AV_PIX_FMT_MONOWHITE || pix_fmt == AV_PIX_FMT_MONOBLACK; } - if (track->par->codec_id == AV_CODEC_ID_VP9) { + if (track->par->codec_id == AV_CODEC_ID_VP9 || + track->par->codec_id == AV_CODEC_ID_AV1) { if (track->mode != MODE_MP4) { - av_log(s, AV_LOG_ERROR, "VP9 only supported in MP4.\n"); + av_log(s, AV_LOG_ERROR, "%s only supported in MP4.\n", avcodec_get_name(track->par->codec_id)); return AVERROR(EINVAL); } - } else if (track->par->codec_id == AV_CODEC_ID_AV1) { - /* spec is not finished, so forbid for now */ - av_log(s, AV_LOG_ERROR, "AV1 muxing is currently not supported.\n"); - return AVERROR_PATCHWELCOME; } else if (track->par->codec_id == AV_CODEC_ID_VP8) { /* altref frames handling is not defined in the spec as of version v1.0, * so just forbid muxing VP8 streams altogether until a new version does */ @@ -6732,6 +6753,7 @@ const AVCodecTag codec_mp4_tags[] = { { AV_CODEC_ID_DIRAC , MKTAG('d', 'r', 'a', 'c') }, { AV_CODEC_ID_TSCC2 , MKTAG('m', 'p', '4', 'v') }, { AV_CODEC_ID_VP9 , MKTAG('v', 'p', '0', '9') }, + { AV_CODEC_ID_AV1 , MKTAG('a', 'v', '0', '1') }, { AV_CODEC_ID_AAC , MKTAG('m', 'p', '4', 'a') }, { AV_CODEC_ID_MP4ALS , MKTAG('m', 'p', '4', 'a') }, { AV_CODEC_ID_MP3 , MKTAG('m', 'p', '4', 'a') },
Signed-off-by: James Almer <jamrial@gmail.com> --- ff_av1_filter_obus() could eventually be replaced by an autoinserted filter_units bsf, assuming it doesn't slow down the muxing process too much (CBS is fast reading packets, but not so much assembling and writing packets). ff_isom_write_av1c() however can't be replaced given filter_units doesn't handle extradata (either codecpar or packet side data). libavformat/Makefile | 2 +- libavformat/av1.c | 107 +++++++++++++++++++++++++++++++++++++++++++ libavformat/av1.h | 70 ++++++++++++++++++++++++++++ libavformat/movenc.c | 36 ++++++++++++--- 4 files changed, 207 insertions(+), 8 deletions(-) create mode 100644 libavformat/av1.c create mode 100644 libavformat/av1.h