From patchwork Mon Dec  2 09:05:33 2019
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Patchwork-Submitter: =?utf-8?q?J=C3=B6rg_Beckmann?=
 <Joerg.Beckmann@scisys.com>
X-Patchwork-Id: 16529
Return-Path: <ffmpeg-devel-bounces@ffmpeg.org>
X-Original-To: patchwork@ffaux-bg.ffmpeg.org
Delivered-To: patchwork@ffaux-bg.ffmpeg.org
Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100])
	by ffaux.localdomain (Postfix) with ESMTP id B1F8B448E28
	for <patchwork@ffaux-bg.ffmpeg.org>;
	Mon,  2 Dec 2019 11:05:41 +0200 (EET)
Received: from [127.0.1.1] (localhost [127.0.0.1])
	by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8B42268ADCD;
	Mon,  2 Dec 2019 11:05:41 +0200 (EET)
X-Original-To: ffmpeg-devel@ffmpeg.org
Delivered-To: ffmpeg-devel@ffmpeg.org
Received: from mail01.scisys.de (mail01.scisys.de [194.231.52.239])
	by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 71B7568AC85
	for <ffmpeg-devel@ffmpeg.org>; Mon,  2 Dec 2019 11:05:35 +0200 (EET)
Received: from mail01.scisys.de (localhost.localdomain [127.0.0.1])
	by localhost (Email Security Appliance) with SMTP id
	913B73891C5_DE4D3DEB
	for <ffmpeg-devel@ffmpeg.org>; Mon,  2 Dec 2019 09:05:34 +0000 (GMT)
Received: from CITBO-EX01.vcs.de (citbo-vlm-int.vcs.de [172.17.2.200])
	by mail01.scisys.de (Sophos Email Appliance) with ESMTP id
	3E0AE387600_DE4D3DEF
	for <ffmpeg-devel@ffmpeg.org>; Mon,  2 Dec 2019 09:05:34 +0000 (GMT)
Received: from CITBO-EX12.vcs.de (10.148.80.20) by CITBO-EX01.vcs.de
	(172.16.16.21) with Microsoft SMTP Server (TLS) id 14.3.468.0;
	Mon, 2 Dec 2019 10:05:34 +0100
Received: from CITBO-EX12.vcs.de (10.148.64.22) by CITBO-EX12.vcs.de
	(10.148.64.22) with Microsoft SMTP Server (version=TLS1_2,
	cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.1531.3;
	Mon, 2 Dec 2019 10:05:33 +0100
Received: from CITBO-EX12.vcs.de ([10.148.64.22]) by CITBO-EX12.vcs.de
	([10.148.64.22]) with mapi id 15.01.1531.003;
	Mon, 2 Dec 2019 10:05:33 +0100
From: =?iso-8859-1?Q?J=F6rg_Beckmann?= <Joerg.Beckmann@scisys.com>
To: "ffmpeg-devel@ffmpeg.org" <ffmpeg-devel@ffmpeg.org>
Thread-Topic: [PATCH] Patch for memory optimization with QuickTime/MP4
Thread-Index: AdWo7RNXG0ouYV7VQ6ipI/WfziiDpQ==
Date: Mon, 2 Dec 2019 09:05:33 +0000
Message-ID: <ade3661cdf374486b5e3049010c929a3@scisys.com>
Accept-Language: en-GB, en-US
Content-Language: de-DE
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [172.16.166.10]
MIME-Version: 1.0
X-SASI-RCODE: 200
X-Content-Filtered-By: Mailman/MimeDel 2.1.20
Subject: [FFmpeg-devel] [PATCH] Patch for memory optimization with
	QuickTime/MP4
X-BeenThere: ffmpeg-devel@ffmpeg.org
X-Mailman-Version: 2.1.20
Precedence: list
List-Id: FFmpeg development discussions and patches <ffmpeg-devel.ffmpeg.org>
List-Unsubscribe: <https://ffmpeg.org/mailman/options/ffmpeg-devel>,
	<mailto:ffmpeg-devel-request@ffmpeg.org?subject=unsubscribe>
List-Archive: <https://ffmpeg.org/pipermail/ffmpeg-devel>
List-Post: <mailto:ffmpeg-devel@ffmpeg.org>
List-Help: <mailto:ffmpeg-devel-request@ffmpeg.org?subject=help>
List-Subscribe: <https://ffmpeg.org/mailman/listinfo/ffmpeg-devel>,
	<mailto:ffmpeg-devel-request@ffmpeg.org?subject=subscribe>
Reply-To: FFmpeg development discussions and patches
	<ffmpeg-devel@ffmpeg.org>
Errors-To: ffmpeg-devel-bounces@ffmpeg.org
Sender: "ffmpeg-devel" <ffmpeg-devel-bounces@ffmpeg.org>

Hi,

this patch invents a new option "discard_fragments" for the MP4/Quicktime/MOV decoder. If the option is not set, nothing changes at all. If it is set, old fragments are discarded as far as possible on each call to switch_root. For pure audio streams, the memory usage is now constant. For video streams, the memory usage is reduced. I've tested it with audio streams received from a professional DAB+ receiver and with video streams created on my own with "ffmpeg -i <video>.m4v -c:a:0 copy -c:v copy -c:s copy -f ismv -movflags frag_keyframe -movflags faststart tcp://localhost:1234?listen" and "ffmpeg -i tcp://localhost:1234 -c:a copy -c:v copy -c:s copy -y <output>". Does someone know sources for MP4 video streams? I could not find any with more than a few minutes. That's not sufficient to test memory usage.

Cheers,
Jörg
---
libavformat/isom.h |  1 +
libavformat/mov.c  | 52 +++++++++++++++++++++++++++++++++++++++++++++-
2 files changed, 52 insertions(+), 1 deletion(-)

+    if (mov->discard_fragments) {
+        frag = &mov->fragment;
+
+        for (i = 0; i < mov->fc->nb_streams; i++) {
+            if (mov->fc->streams[i]->id == frag->track_id) {
+                st = mov->fc->streams[i];
+                break;
+            }
+        }
+
+        if (!st) {
+            av_log(mov->fc, AV_LOG_WARNING, "could not find corresponding track id %u\n", frag->track_id);
+        } else {
+            sc = st->priv_data;
+
+            switch (st->codecpar->codec_type) {
+                case AVMEDIA_TYPE_AUDIO:
+                case AVMEDIA_TYPE_SUBTITLE:
+                    /* Freeing VIDEO tables leads to corrupted video when writing to eg. MKV */
+                    av_freep(&st->index_entries);
+                    st->nb_index_entries = 0;
+                    st->index_entries_allocated_size = 0;
+
+                    sc->current_index = 0;
+                    sc->current_sample = 0;
+
+                    av_freep(&sc->ctts_data);
+                    sc->ctts_data = NULL;
+                    sc->ctts_allocated_size = 0;
+                    sc->ctts_count = 0;
+                    break;
+            }
+
+            av_free(mov->frag_index.item->stream_info);
+            av_freep(&mov->frag_index.item);
+            mov->frag_index.allocated_size = 0;
+            mov->frag_index.nb_items = 0;
+        }
+    }
+
     ret = mov_read_default(mov, s->pb, (MOVAtom){ AV_RL32("root"), INT64_MAX });
     if (ret < 0)
         return ret;
@@ -7975,6 +8018,9 @@ static int mov_read_seek(AVFormatContext *s, int stream_index, int64_t sample_ti
     int sample;
     int i;
+    if (mc->discard_fragments)  // Seeking is not possible if fragments are discarded.
+        return AVERROR(ENOTSUP);
+
     if (stream_index >= s->nb_streams)
         return AVERROR_INVALIDDATA;
@@ -8063,6 +8109,10 @@ static const AVOption mov_options[] = {
     { "decryption_key", "The media decryption key (hex)", OFFSET(decryption_key), AV_OPT_TYPE_BINARY, .flags = AV_OPT_FLAG_DECODING_PARAM },
     { "enable_drefs", "Enable external track support.", OFFSET(enable_drefs), AV_OPT_TYPE_BOOL,
         {.i64 = 0}, 0, 1, FLAGS },
+    {"discard_fragments",
+            "Discards fragments after they have been read to support live streams.",
+            OFFSET(discard_fragments), AV_OPT_TYPE_BOOL, { .i64 = 0 },
+            0, 1, FLAGS },
     { NULL },
};

diff --git a/libavformat/isom.h b/libavformat/isom.h
index 4943b80ccf..9b4753f4d7 100644
--- a/libavformat/isom.h
+++ b/libavformat/isom.h
@@ -268,6 +268,7 @@ typedef struct MOVContext {
     int advanced_editlist;
     int ignore_chapters;
     int seek_individually;
+    int discard_fragments;
     int64_t next_root_atom; ///< offset of the next root atom
     int export_all;
     int export_xmp;
diff --git a/libavformat/mov.c b/libavformat/mov.c
index 7553a7fdfc..3a5cfa23f8 100644
--- a/libavformat/mov.c
+++ b/libavformat/mov.c
@@ -7698,8 +7698,11 @@ static int should_retry(AVIOContext *pb, int error_code) {
 static int mov_switch_root(AVFormatContext *s, int64_t target, int index)
{
-    int ret;
+    int ret, i;
     MOVContext *mov = s->priv_data;
+    AVStream *st = NULL;
+    MOVStreamContext *sc;
+    MOVFragment *frag;
     if (index >= 0 && index < mov->frag_index.nb_items)
         target = mov->frag_index.item[index].moof_offset;
@@ -7721,6 +7724,46 @@ static int mov_switch_root(AVFormatContext *s, int64_t target, int index)
     mov->found_mdat = 0;