| Message ID | 51a0b43f07f546bab99887b46ca037be@scisys.com |
|---|---|
| State | Superseded |
| Headers | show |
Is there still something wrong with this patch? Cheers, Jörg > -----Ursprüngliche Nachricht----- > Von: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> Im Auftrag von Jörg > Beckmann > Gesendet: Montag, 9. Dezember 2019 16:05 > An: ffmpeg-devel@ffmpeg.org > Betreff: [FFmpeg-devel] [PATCH v5] avformat/mov: Memory optimization with > QuickTime/MP4 > > Invents a new option "discard_fragments" for the MP4/Quicktime/MOV decoder. > > If the option is not set, nothing changes at all. If it is set, old fragments are > discarded as far as possible on each call to switch_root. For pure audio streams, > the memory usage is now constant. For video streams, the memory usage is > reduced. It's tested with audio streams received from a professional DAB+ > receiver and with video streams created on my own with "ffmpeg -i <video>.m4v - > c:a:0 copy -c:v copy -c:s copy -f ismv -movflags \ frag_keyframe -movflags > faststart tcp://localhost:1234?listen" and "ffmpeg -i tcp://localhost:1234 -c:a copy - > c:v copy -c:s copy -y <output>". > > Signed-off-by: Jörg Beckmann <joerg.beckmann@scisys.com> > --- > libavformat/isom.h | 1 + > libavformat/mov.c | 49 > ++++++++++++++++++++++++++++++++++++++++++++++++- > 2 files changed, 49 insertions(+), 1 deletion(-) > > diff --git a/libavformat/isom.h b/libavformat/isom.h index 4943b80ccf..9b4753f4d7 > 100644 > --- a/libavformat/isom.h > +++ b/libavformat/isom.h > @@ -268,6 +268,7 @@ typedef struct MOVContext { > int advanced_editlist; > int ignore_chapters; > int seek_individually; > + int discard_fragments; > int64_t next_root_atom; ///< offset of the next root atom > int export_all; > int export_xmp; > diff --git a/libavformat/mov.c b/libavformat/mov.c index 7553a7fdfc..deb3ff5508 > 100644 > --- a/libavformat/mov.c > +++ b/libavformat/mov.c > @@ -7698,8 +7698,11 @@ static int should_retry(AVIOContext *pb, int error_code) > { > > static int mov_switch_root(AVFormatContext *s, int64_t target, int index) { > - int ret; > + int ret, i; > MOVContext *mov = s->priv_data; > + AVStream *st = NULL; > + MOVStreamContext *sc; > + MOVFragment *frag; > > if (index >= 0 && index < mov->frag_index.nb_items) > target = mov->frag_index.item[index].moof_offset; > @@ -7721,6 +7724,43 @@ static int mov_switch_root(AVFormatContext *s, int64_t > target, int index) > > mov->found_mdat = 0; > > + if (mov->discard_fragments) { > + frag = &mov->fragment; > + > + for (i = 0; i < mov->fc->nb_streams; i++) { > + if (mov->fc->streams[i]->id == frag->track_id) { > + st = mov->fc->streams[i]; > + break; > + } > + } > + > + av_assert0(st); > + > + sc = st->priv_data; > + > + switch (st->codecpar->codec_type) { > + case AVMEDIA_TYPE_AUDIO: > + case AVMEDIA_TYPE_SUBTITLE: > + /* Freeing VIDEO tables leads to corrupted video when writing to eg. > MKV */ > + av_freep(&st->index_entries); > + st->nb_index_entries = 0; > + st->index_entries_allocated_size = 0; > + > + sc->current_index = 0; > + sc->current_sample = 0; > + > + av_freep(&sc->ctts_data); > + sc->ctts_allocated_size = 0; > + sc->ctts_count = 0; > + break; > + } > + > + av_free(mov->frag_index.item->stream_info); > + av_freep(&mov->frag_index.item); > + mov->frag_index.allocated_size = 0; > + mov->frag_index.nb_items = 0; > + } > + > ret = mov_read_default(mov, s->pb, (MOVAtom){ AV_RL32("root"), INT64_MAX > }); > if (ret < 0) > return ret; > @@ -7975,6 +8015,9 @@ static int mov_read_seek(AVFormatContext *s, int > stream_index, int64_t sample_ti > int sample; > int i; > > + if (mc->discard_fragments) // Seeking is not possible if fragments are > discarded. > + return AVERROR(ENOTSUP); > + > if (stream_index >= s->nb_streams) > return AVERROR_INVALIDDATA; > > @@ -8063,6 +8106,10 @@ static const AVOption mov_options[] = { > { "decryption_key", "The media decryption key (hex)", > OFFSET(decryption_key), AV_OPT_TYPE_BINARY, .flags = > AV_OPT_FLAG_DECODING_PARAM }, > { "enable_drefs", "Enable external track support.", OFFSET(enable_drefs), > AV_OPT_TYPE_BOOL, > {.i64 = 0}, 0, 1, FLAGS }, > + {"discard_fragments", > + "Discard fragments after they have been read to support live streams.", > + OFFSET(discard_fragments), AV_OPT_TYPE_BOOL, { .i64 = 0 }, > + 0, 1, FLAGS }, > > { NULL }, > }; > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with > subject "unsubscribe".
Am Mo., 9. Dez. 2019 um 16:05 Uhr schrieb Jörg Beckmann <Joerg.Beckmann@scisys.com>: > > Invents a new option "discard_fragments" for the MP4/Quicktime/MOV decoder. > > If the option is not set, nothing changes at all. If it is set, old fragments > are discarded as far as possible on each call to switch_root. > For pure audio streams, the memory usage is now constant. Is it possible to detect this case? If possible, the new option could have an "auto" setting. Carl Eugen
> -----Ursprüngliche Nachricht----- > Von: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> Im Auftrag von Carl > Eugen Hoyos > Gesendet: Montag, 16. Dezember 2019 11:50 > An: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org> > Betreff: [SCISYS Possible Spam] Re: [FFmpeg-devel] [PATCH v5] avformat/mov: > Memory optimization with QuickTime/MP4 > > Am Mo., 9. Dez. 2019 um 16:05 Uhr schrieb Jörg Beckmann > <Joerg.Beckmann@scisys.com>: > > > > Invents a new option "discard_fragments" for the MP4/Quicktime/MOV decoder. > > > > If the option is not set, nothing changes at all. If it is set, old > > fragments are discarded as far as possible on each call to switch_root. > > > For pure audio streams, the memory usage is now constant. > > Is it possible to detect this case? > If possible, the new option could have an "auto" setting. I'm not sure whether it really works with all possible stream types. It works with all types I tried. Therefore I think the default behavior should be the old one and no "auto" mode. The main use case is 24x7 recording. In most other cases the memory usage is not important on modern computers. I'm even not sure whether there are many Quicktime/MP4 life streams at all. I did not find any but my audio streams. If you know additional ones, I would like to try them with the option. Maybe there are additional tables that could be freed also. > Carl Eugen Jörg > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with > subject "unsubscribe".
Hi, is there something I can do to get this patch applied? Cheers, Jörg > -----Ursprüngliche Nachricht----- > Von: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> Im Auftrag von Jörg > Beckmann > Gesendet: Montag, 9. Dezember 2019 16:05 > An: ffmpeg-devel@ffmpeg.org > Betreff: [FFmpeg-devel] [PATCH v5] avformat/mov: Memory optimization with > QuickTime/MP4 > > Invents a new option "discard_fragments" for the MP4/Quicktime/MOV decoder. > > If the option is not set, nothing changes at all. If it is set, old fragments are > discarded as far as possible on each call to switch_root. For pure audio streams, > the memory usage is now constant. For video streams, the memory usage is > reduced. It's tested with audio streams received from a professional DAB+ > receiver and with video streams created on my own with "ffmpeg -i <video>.m4v - > c:a:0 copy -c:v copy -c:s copy -f ismv -movflags \ frag_keyframe -movflags > faststart tcp://localhost:1234?listen" and "ffmpeg -i tcp://localhost:1234 -c:a copy - > c:v copy -c:s copy -y <output>". > > Signed-off-by: Jörg Beckmann <joerg.beckmann@scisys.com> > --- > libavformat/isom.h | 1 + > libavformat/mov.c | 49 > ++++++++++++++++++++++++++++++++++++++++++++++++- > 2 files changed, 49 insertions(+), 1 deletion(-) > > diff --git a/libavformat/isom.h b/libavformat/isom.h index 4943b80ccf..9b4753f4d7 > 100644 > --- a/libavformat/isom.h > +++ b/libavformat/isom.h > @@ -268,6 +268,7 @@ typedef struct MOVContext { > int advanced_editlist; > int ignore_chapters; > int seek_individually; > + int discard_fragments; > int64_t next_root_atom; ///< offset of the next root atom > int export_all; > int export_xmp; > diff --git a/libavformat/mov.c b/libavformat/mov.c index 7553a7fdfc..deb3ff5508 > 100644 > --- a/libavformat/mov.c > +++ b/libavformat/mov.c > @@ -7698,8 +7698,11 @@ static int should_retry(AVIOContext *pb, int error_code) > { > > static int mov_switch_root(AVFormatContext *s, int64_t target, int index) { > - int ret; > + int ret, i; > MOVContext *mov = s->priv_data; > + AVStream *st = NULL; > + MOVStreamContext *sc; > + MOVFragment *frag; > > if (index >= 0 && index < mov->frag_index.nb_items) > target = mov->frag_index.item[index].moof_offset; > @@ -7721,6 +7724,43 @@ static int mov_switch_root(AVFormatContext *s, int64_t > target, int index) > > mov->found_mdat = 0; > > + if (mov->discard_fragments) { > + frag = &mov->fragment; > + > + for (i = 0; i < mov->fc->nb_streams; i++) { > + if (mov->fc->streams[i]->id == frag->track_id) { > + st = mov->fc->streams[i]; > + break; > + } > + } > + > + av_assert0(st); > + > + sc = st->priv_data; > + > + switch (st->codecpar->codec_type) { > + case AVMEDIA_TYPE_AUDIO: > + case AVMEDIA_TYPE_SUBTITLE: > + /* Freeing VIDEO tables leads to corrupted video when writing to eg. > MKV */ > + av_freep(&st->index_entries); > + st->nb_index_entries = 0; > + st->index_entries_allocated_size = 0; > + > + sc->current_index = 0; > + sc->current_sample = 0; > + > + av_freep(&sc->ctts_data); > + sc->ctts_allocated_size = 0; > + sc->ctts_count = 0; > + break; > + } > + > + av_free(mov->frag_index.item->stream_info); > + av_freep(&mov->frag_index.item); > + mov->frag_index.allocated_size = 0; > + mov->frag_index.nb_items = 0; > + } > + > ret = mov_read_default(mov, s->pb, (MOVAtom){ AV_RL32("root"), INT64_MAX > }); > if (ret < 0) > return ret; > @@ -7975,6 +8015,9 @@ static int mov_read_seek(AVFormatContext *s, int > stream_index, int64_t sample_ti > int sample; > int i; > > + if (mc->discard_fragments) // Seeking is not possible if fragments are > discarded. > + return AVERROR(ENOTSUP); > + > if (stream_index >= s->nb_streams) > return AVERROR_INVALIDDATA; > > @@ -8063,6 +8106,10 @@ static const AVOption mov_options[] = { > { "decryption_key", "The media decryption key (hex)", > OFFSET(decryption_key), AV_OPT_TYPE_BINARY, .flags = > AV_OPT_FLAG_DECODING_PARAM }, > { "enable_drefs", "Enable external track support.", OFFSET(enable_drefs), > AV_OPT_TYPE_BOOL, > {.i64 = 0}, 0, 1, FLAGS }, > + {"discard_fragments", > + "Discard fragments after they have been read to support live streams.", > + OFFSET(discard_fragments), AV_OPT_TYPE_BOOL, { .i64 = 0 }, > + 0, 1, FLAGS }, > > { NULL }, > }; > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with > subject "unsubscribe".
Am Mo., 16. Dez. 2019 um 12:13 Uhr schrieb Jörg Beckmann <Joerg.Beckmann@scisys.com>: > > > > -----Ursprüngliche Nachricht----- > > Von: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> Im Auftrag von Carl > > Eugen Hoyos > > Gesendet: Montag, 16. Dezember 2019 11:50 > > An: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org> > > Betreff: [SCISYS Possible Spam] Re: [FFmpeg-devel] [PATCH v5] avformat/mov: > > Memory optimization with QuickTime/MP4 > > > > Am Mo., 9. Dez. 2019 um 16:05 Uhr schrieb Jörg Beckmann > > <Joerg.Beckmann@scisys.com>: > > > > > > Invents a new option "discard_fragments" for the MP4/Quicktime/MOV decoder. > > > > > > If the option is not set, nothing changes at all. If it is set, old > > > fragments are discarded as far as possible on each call to switch_root. > > > > > For pure audio streams, the memory usage is now constant. > > > > Is it possible to detect this case? > > If possible, the new option could have an "auto" setting. > > I'm not sure whether it really works with all possible stream types. Why is it not possible to detect that the stream only contains audio? Carl Eugen
> -----Ursprüngliche Nachricht----- > Von: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> Im Auftrag von Carl > Eugen Hoyos > Gesendet: Montag, 6. Januar 2020 15:24 > An: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org> > Betreff: [SCISYS Possible Spam] Re: [FFmpeg-devel] [SCISYS Possible Spam] > Re: [PATCH v5] avformat/mov: Memory optimization with QuickTime/MP4 > > Am Mo., 16. Dez. 2019 um 12:13 Uhr schrieb Jörg Beckmann > <Joerg.Beckmann@scisys.com>: > > > > > > > -----Ursprüngliche Nachricht----- > > > Von: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> Im Auftrag von > > > Carl Eugen Hoyos > > > Gesendet: Montag, 16. Dezember 2019 11:50 > > > An: FFmpeg development discussions and patches > > > <ffmpeg-devel@ffmpeg.org> > > > Betreff: [SCISYS Possible Spam] Re: [FFmpeg-devel] [PATCH v5] > avformat/mov: > > > Memory optimization with QuickTime/MP4 > > > > > > Am Mo., 9. Dez. 2019 um 16:05 Uhr schrieb Jörg Beckmann > > > <Joerg.Beckmann@scisys.com>: > > > > > > > > Invents a new option "discard_fragments" for the MP4/Quicktime/MOV > decoder. > > > > > > > > If the option is not set, nothing changes at all. If it is set, > > > > old fragments are discarded as far as possible on each call to switch_root. > > > > > > > For pure audio streams, the memory usage is now constant. > > > > > > Is it possible to detect this case? > > > If possible, the new option could have an "auto" setting. > > > > I'm not sure whether it really works with all possible stream types. > > Why is it not possible to detect that the stream only contains audio? Oh sorry, I misunderstood your question. Of course it is possible to detect that it contains only audio. I thought you suggested to detect that is fragmented. I'll take a look into the sources to find an appropriate location for the check and send the modified patch again. But it will take a few days. Do you think, "auto" should be the default value? > > Carl Eugen Jörg
Am Mo., 6. Jan. 2020 um 15:37 Uhr schrieb Jörg Beckmann <Joerg.Beckmann@scisys.com>: > > > -----Ursprüngliche Nachricht----- > > Von: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> Im Auftrag von Carl > > Eugen Hoyos > > Gesendet: Montag, 6. Januar 2020 15:24 > > An: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org> > > Betreff: [SCISYS Possible Spam] Re: [FFmpeg-devel] [SCISYS Possible Spam] > > Re: [PATCH v5] avformat/mov: Memory optimization with QuickTime/MP4 > > > > Am Mo., 16. Dez. 2019 um 12:13 Uhr schrieb Jörg Beckmann > > <Joerg.Beckmann@scisys.com>: > > > > > > > > > > -----Ursprüngliche Nachricht----- > > > > Von: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> Im Auftrag von > > > > Carl Eugen Hoyos > > > > Gesendet: Montag, 16. Dezember 2019 11:50 > > > > An: FFmpeg development discussions and patches > > > > <ffmpeg-devel@ffmpeg.org> > > > > Betreff: [SCISYS Possible Spam] Re: [FFmpeg-devel] [PATCH v5] > > avformat/mov: > > > > Memory optimization with QuickTime/MP4 > > > > > > > > Am Mo., 9. Dez. 2019 um 16:05 Uhr schrieb Jörg Beckmann > > > > <Joerg.Beckmann@scisys.com>: > > > > > > > > > > Invents a new option "discard_fragments" for the MP4/Quicktime/MOV > > decoder. > > > > > > > > > > If the option is not set, nothing changes at all. If it is set, > > > > > old fragments are discarded as far as possible on each call to switch_root. > > > > > > > > > For pure audio streams, the memory usage is now constant. > > > > > > > > Is it possible to detect this case? > > > > If possible, the new option could have an "auto" setting. > > > > > > I'm not sure whether it really works with all possible stream types. > > > > Why is it not possible to detect that the stream only contains audio? > > Oh sorry, I misunderstood your question. Of course it is possible to detect > that it contains only audio. I thought you suggested to detect that is fragmented. > I'll take a look into the sources to find an appropriate location for the check and > send the modified patch again. But it will take a few days. > > Do you think, "auto" should be the default value? If your patch means that receiving audio streams allocates less memory and the audio-only case can be detected, yes. Carl Eugen
diff --git a/libavformat/isom.h b/libavformat/isom.h index 4943b80ccf..9b4753f4d7 100644 --- a/libavformat/isom.h +++ b/libavformat/isom.h @@ -268,6 +268,7 @@ typedef struct MOVContext { int advanced_editlist; int ignore_chapters; int seek_individually; + int discard_fragments; int64_t next_root_atom; ///< offset of the next root atom int export_all; int export_xmp; diff --git a/libavformat/mov.c b/libavformat/mov.c index 7553a7fdfc..deb3ff5508 100644 --- a/libavformat/mov.c +++ b/libavformat/mov.c @@ -7698,8 +7698,11 @@ static int should_retry(AVIOContext *pb, int error_code) { static int mov_switch_root(AVFormatContext *s, int64_t target, int index) { - int ret; + int ret, i; MOVContext *mov = s->priv_data; + AVStream *st = NULL; + MOVStreamContext *sc; + MOVFragment *frag; if (index >= 0 && index < mov->frag_index.nb_items) target = mov->frag_index.item[index].moof_offset; @@ -7721,6 +7724,43 @@ static int mov_switch_root(AVFormatContext *s, int64_t target, int index) mov->found_mdat = 0; + if (mov->discard_fragments) { + frag = &mov->fragment; + + for (i = 0; i < mov->fc->nb_streams; i++) { + if (mov->fc->streams[i]->id == frag->track_id) { + st = mov->fc->streams[i]; + break; + } + } + + av_assert0(st); + + sc = st->priv_data; + + switch (st->codecpar->codec_type) { + case AVMEDIA_TYPE_AUDIO: + case AVMEDIA_TYPE_SUBTITLE: + /* Freeing VIDEO tables leads to corrupted video when writing to eg. MKV */ + av_freep(&st->index_entries); + st->nb_index_entries = 0; + st->index_entries_allocated_size = 0; + + sc->current_index = 0; + sc->current_sample = 0; + + av_freep(&sc->ctts_data); + sc->ctts_allocated_size = 0; + sc->ctts_count = 0; + break; + } + + av_free(mov->frag_index.item->stream_info); + av_freep(&mov->frag_index.item); + mov->frag_index.allocated_size = 0; + mov->frag_index.nb_items = 0; + } + ret = mov_read_default(mov, s->pb, (MOVAtom){ AV_RL32("root"), INT64_MAX }); if (ret < 0) return ret; @@ -7975,6 +8015,9 @@ static int mov_read_seek(AVFormatContext *s, int stream_index, int64_t sample_ti int sample; int i; + if (mc->discard_fragments) // Seeking is not possible if fragments are discarded. + return AVERROR(ENOTSUP); + if (stream_index >= s->nb_streams) return AVERROR_INVALIDDATA; @@ -8063,6 +8106,10 @@ static const AVOption mov_options[] = { { "decryption_key", "The media decryption key (hex)", OFFSET(decryption_key), AV_OPT_TYPE_BINARY, .flags = AV_OPT_FLAG_DECODING_PARAM }, { "enable_drefs", "Enable external track support.", OFFSET(enable_drefs), AV_OPT_TYPE_BOOL, {.i64 = 0}, 0, 1, FLAGS }, + {"discard_fragments", + "Discard fragments after they have been read to support live streams.", + OFFSET(discard_fragments), AV_OPT_TYPE_BOOL, { .i64 = 0 }, + 0, 1, FLAGS }, { NULL }, };
Invents a new option "discard_fragments" for the MP4/Quicktime/MOV decoder. If the option is not set, nothing changes at all. If it is set, old fragments are discarded as far as possible on each call to switch_root. For pure audio streams, the memory usage is now constant. For video streams, the memory usage is reduced. It's tested with audio streams received from a professional DAB+ receiver and with video streams created on my own with "ffmpeg -i <video>.m4v -c:a:0 copy -c:v copy -c:s copy -f ismv -movflags \ frag_keyframe -movflags faststart tcp://localhost:1234?listen" and "ffmpeg -i tcp://localhost:1234 -c:a copy -c:v copy -c:s copy -y <output>". Signed-off-by: Jörg Beckmann <joerg.beckmann@scisys.com> --- libavformat/isom.h | 1 + libavformat/mov.c | 49 ++++++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 49 insertions(+), 1 deletion(-)