[FFmpeg-devel] qt-faststart: Add mode for probing faststart-ness

Submitted by Tomas Härdin on July 27, 2018, 2:06 p.m.

Details

Message ID 1532700416.22213.1.camel@acc.umu.se
State New
Headers show

Commit Message

Tomas Härdin July 27, 2018, 2:06 p.m.
Hi

This is something the PeerTube guys need, and this seemed a
straightforward enough approach. Feedback appreciated

/Tomas

Comments

Gyan July 27, 2018, 2:24 p.m.
On 27-07-2018 07:36 PM, Tomas Härdin wrote:

> This is something the PeerTube guys need, and this seemed a
> straightforward enough approach. Feedback appreciated

Not a patch review, just a note that one can do this with ffmpeg/ffprobe 
and basic *nix tools.

To check for moof,

      ffmpeg -i in.mp4 -v 56 2>&1 | grep -e type:'moof'

To see if moov precedes mdat

      ffmpeg -i in.mp4 -v 56 2>&1 | grep -e "type:'moov'" -e 
"type:'mdat'" | head -1 | grep moov


Gyan
Tomas Härdin July 27, 2018, 10:03 p.m.
fre 2018-07-27 klockan 19:54 +0530 skrev Gyan Doshi:
> 
> On 27-07-2018 07:36 PM, Tomas Härdin wrote:
> 
> > This is something the PeerTube guys need, and this seemed a
> > straightforward enough approach. Feedback appreciated
> 
> Not a patch review, just a note that one can do this with
> ffmpeg/ffprobe 
> and basic *nix tools.
> 
> To check for moof,
> 
>       ffmpeg -i in.mp4 -v 56 2>&1 | grep -e type:'moof'
> 
> To see if moov precedes mdat
> 
>       ffmpeg -i in.mp4 -v 56 2>&1 | grep -e "type:'moov'" -e 
> "type:'mdat'" | head -1 | grep moov

woot! I didn't know the mov demuxer dumped such things. It is quite
slow however, since it will scan through every leaf atom in the file.
For example, running time ffmpeg -i input.mov -v 56 2>&1 | wc on a 1.5 
GiB MP4 on an SSD takes:

real    0m4,740s
user    0m6,708s
sys     0m2,356s

While using the patched qt-faststart:

real    0m0,002s
user    0m0,000s
sys     0m0,000s

/Tomas
Gyan July 28, 2018, 5:01 a.m.
On 28-07-2018 03:33 AM, Tomas Härdin wrote:
> 
> woot! I didn't know the mov demuxer dumped such things. It is quite
> slow however, since it will scan through every leaf atom in the file.
> For example, running time ffmpeg -i input.mov -v 56 2>&1 | wc on a 1.5
> GiB MP4 on an SSD takes:

Use the subfile protocol to forgo parsing the whole file, e.g.

     ffmpeg -i subfile,,start,0,end,10000,,:in.mp4 -v 56 2>&1 | grep -e 
"type:'moov'" -e "type:'mdat'" | head -1 | grep moov

This assumes that at least one of the two targeted boxes moov/mdat start 
within the first 10000 bytes.

Gyan
Tomas Härdin July 28, 2018, 8:56 a.m.
lör 2018-07-28 klockan 10:31 +0530 skrev Gyan Doshi:
> 
> On 28-07-2018 03:33 AM, Tomas Härdin wrote:
> > 
> > woot! I didn't know the mov demuxer dumped such things. It is quite
> > slow however, since it will scan through every leaf atom in the
> > file.
> > For example, running time ffmpeg -i input.mov -v 56 2>&1 | wc on a
> > 1.5
> > GiB MP4 on an SSD takes:
> 
> Use the subfile protocol to forgo parsing the whole file, e.g.
> 
>      ffmpeg -i subfile,,start,0,end,10000,,:in.mp4 -v 56 2>&1 | grep
> -e 
> "type:'moov'" -e "type:'mdat'" | head -1 | grep moov
> 
> This assumes that at least one of the two targeted boxes moov/mdat
> start 
> within the first 10000 bytes.

Neat. This is similar to the first suggestion I had to the peertube
devs, dd:ing off the first MiB or so then probing that. We'll see what
they say, and what Baptiste thinks I guess

/Tomas
Baptiste Coudurier July 31, 2018, 9:46 p.m.
Hey Tomas,

On Sat, Jul 28, 2018 at 1:56 AM, Tomas Härdin <tjoppen@acc.umu.se> wrote:

> lör 2018-07-28 klockan 10:31 +0530 skrev Gyan Doshi:
> >
> > On 28-07-2018 03:33 AM, Tomas Härdin wrote:
> > >
> > > woot! I didn't know the mov demuxer dumped such things. It is quite
> > > slow however, since it will scan through every leaf atom in the
> > > file.
> > > For example, running time ffmpeg -i input.mov -v 56 2>&1 | wc on a
> > > 1.5
> > > GiB MP4 on an SSD takes:
> >
> > Use the subfile protocol to forgo parsing the whole file, e.g.
> >
> >      ffmpeg -i subfile,,start,0,end,10000,,:in.mp4 -v 56 2>&1 | grep
> > -e
> > "type:'moov'" -e "type:'mdat'" | head -1 | grep moov
> >
> > This assumes that at least one of the two targeted boxes moov/mdat
> > start
> > within the first 10000 bytes.
>
> Neat. This is similar to the first suggestion I had to the peertube
> devs, dd:ing off the first MiB or so then probing that. We'll see what
> they say, and what Baptiste thinks I guess


Interesting, I can see the interest of checking this. Could ffprobe do that
somehow ?
Would be an interesting information to report.
Not sure it belongs in qt-faststart, and not sure checking fragmented files
would be in
the scope of qt-faststart. What do you think ?
Tomas Härdin Aug. 1, 2018, 6:33 p.m.
tis 2018-07-31 klockan 14:46 -0700 skrev Baptiste Coudurier:
> Hey Tomas,
> 
> > On Sat, Jul 28, 2018 at 1:56 AM, Tomas Härdin <tjoppen@acc.umu.se> wrote:
> 
> > lör 2018-07-28 klockan 10:31 +0530 skrev Gyan Doshi:
> > > 
> > > On 28-07-2018 03:33 AM, Tomas Härdin wrote:
> > > > 
> > > > woot! I didn't know the mov demuxer dumped such things. It is quite
> > > > slow however, since it will scan through every leaf atom in the
> > > > file.
> > > > For example, running time ffmpeg -i input.mov -v 56 2>&1 | wc on a
> > > > 1.5
> > > > GiB MP4 on an SSD takes:
> > > 
> > > Use the subfile protocol to forgo parsing the whole file, e.g.
> > > 
> > >      ffmpeg -i subfile,,start,0,end,10000,,:in.mp4 -v 56 2>&1 | grep
> > > -e
> > > "type:'moov'" -e "type:'mdat'" | head -1 | grep moov
> > > 
> > > This assumes that at least one of the two targeted boxes moov/mdat
> > > start
> > > within the first 10000 bytes.
> > 
> > Neat. This is similar to the first suggestion I had to the peertube
> > devs, dd:ing off the first MiB or so then probing that. We'll see what
> > they say, and what Baptiste thinks I guess
> 
> 
> Interesting, I can see the interest of checking this. Could ffprobe do that
> somehow ?
> Would be an interesting information to report.
> Not sure it belongs in qt-faststart, and not sure checking fragmented files
> would be in
> the scope of qt-faststart. What do you think ?

My main concern with adding this in ffprobe is the amount of plumbing
needed. Perhaps the demuxer could set a metadata field?

I judged separating fragmented from non-fragmented streamable files as
valuable for this patch, partly because I had to do something to make
qt-faststart not complain about fragmented files. But that also allows
it to abort early. Not that parsing top-level atoms only is
particularly hard work.. Do we know of any players that *don't* support
fragmented MP4?

For now PeerTube seem to be going with something similar to Gyan's
suggestion, so they don't have to keep more C code than necessary
around: https://gist.github.com/rigelk/95e41dacb66df652039e59350a9eb7b4

I think we might want users that actually need this before pushing.
Else it's just technical debt. Good exercise for me to get a bit better
grasp on FATE however

/Tomas

Patch hide | download patch | download mbox

From 8d5627c02cc2da8708b5342b58d2a6eaebf189c5 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Tomas=20H=C3=A4rdin?= <tjoppen@acc.umu.se>
Date: Fri, 27 Jul 2018 13:54:52 +0200
Subject: [PATCH] qt-faststart: Add mode for probing faststart-ness

Activated by giving the program only a single filename.
There are three cases, with corresponding exit codes:

 0: File has moov before mdat, is streamable (not fragmented)
 2: File has moov after mdat, not streamable
 3: File is fragmented (has moof atoms, is streamable in some players)

Fragmented files get a separate exit code since they may not be
supported in all players.
---
 Changelog            |  1 +
 tests/fate/mov.mak   | 40 +++++++++++++++++++++++++++
 tools/qt-faststart.c | 78 ++++++++++++++++++++++++++++++++++++++++++++++++----
 3 files changed, 113 insertions(+), 6 deletions(-)

diff --git a/Changelog b/Changelog
index 807a05dec9..973b1f9fdd 100644
--- a/Changelog
+++ b/Changelog
@@ -16,6 +16,7 @@  version <next>:
 - ATRAC9 decoder
 - lensfun wrapper filter
 - colorconstancy filter
+- added ability to probe faststart-ness with tools/qt-faststart
 
 
 version 4.0:
diff --git a/tests/fate/mov.mak b/tests/fate/mov.mak
index 6f0e28d21e..404e6c662e 100644
--- a/tests/fate/mov.mak
+++ b/tests/fate/mov.mak
@@ -29,6 +29,37 @@  FATE_MOV_FFPROBE = fate-mov-neg-firstpts-discard \
                    fate-mov-guess-delay-3 \
 
 FATE_MOV_FASTSTART = fate-mov-faststart-4gb-overflow \
+                     fate-mov-faststart-probe-streamable \
+                     fate-mov-faststart-probe-fragmented \
+                     fate-mov-faststart-probe-not-streamable \
+
+FATE_MOV_FASTSTART_SAMPLES_STREAMABLE = \
+    440hz-10ms.m4a \
+    displaymatrix.mov \
+    fcp_export8-236.mov \
+    white_zombie_scrunch-part.mov \
+
+FATE_MOV_FASTSTART_SAMPLES_FRAGMENTED = \
+    buck480p30_na.mp4 \
+    frag_overlap.mp4 \
+
+FATE_MOV_FASTSTART_SAMPLES_NOT_STREAMABLE = \
+    aac-2048-priming.mov \
+    elst_ends_betn_b_and_i.mp4 \
+    fake-gp-media-with-real-gpmf.mp4 \
+    invalid_elst_entry_count.mov \
+    mov-1elist-1ctts.mov \
+    mov-1elist-ends-last-bframe.mov \
+    mov-1elist-noctts.mov \
+    mov-2elist-elist1-ends-bframe.mov \
+    mov-3elist-1ctts.mov \
+    mov-3elist-encrypted.mov \
+    mov-3elist.mov \
+    mov-elist-starts-ctts-2ndsample.mov \
+    mov_ibi_elst_starts_b.mov \
+    moviedispmat.mp4 \
+    mp4-init-nonkeyframe.mp4 \
+    spherical.mov \
 
 FATE_SAMPLES_AVCONV += $(FATE_MOV)
 FATE_SAMPLES_FFPROBE += $(FATE_MOV_FFPROBE)
@@ -116,3 +147,12 @@  fate-mov-guess-delay-3: CMD = run ffprobe$(PROGSSUF)$(EXESUF) -show_entries stre
 fate-mov-faststart-4gb-overflow: CMD = run tools/qt-faststart$(EXESUF) $(TARGET_SAMPLES)/mov/faststart-4gb-overflow.mov faststart-4gb-overflow-output.mov > /dev/null ; md5sum faststart-4gb-overflow-output.mov | cut -d " " -f1 ; rm faststart-4gb-overflow-output.mov
 fate-mov-faststart-4gb-overflow: CMP = oneline
 fate-mov-faststart-4gb-overflow: REF = bc875921f151871e787c4b4023269b29
+
+fate-mov-faststart-probe-streamable: CMD = for f in $(FATE_MOV_FASTSTART_SAMPLES_STREAMABLE); do run tools/qt-faststart$(EXESUF) $(TARGET_SAMPLES)/mov/$$f || exit 1; done
+fate-mov-faststart-probe-streamable: CMP = null
+
+fate-mov-faststart-probe-fragmented: CMD = for f in $(FATE_MOV_FASTSTART_SAMPLES_FRAGMENTED); do run tools/qt-faststart$(EXESUF) $(TARGET_SAMPLES)/mov/$$f; [ $$? -eq 3 ] || exit 1; done
+fate-mov-faststart-probe-fragmented: CMP = null
+
+fate-mov-faststart-probe-not-streamable: CMD = for f in $(FATE_MOV_FASTSTART_SAMPLES_NOT_STREAMABLE); do run tools/qt-faststart$(EXESUF) $(TARGET_SAMPLES)/mov/$$f; [ $$? -eq 2 ] || exit 1; done
+fate-mov-faststart-probe-not-streamable: CMP = null
diff --git a/tools/qt-faststart.c b/tools/qt-faststart.c
index 5e88c38e6b..54f44a78f3 100644
--- a/tools/qt-faststart.c
+++ b/tools/qt-faststart.c
@@ -6,6 +6,8 @@ 
  *
  * This utility rearranges a Quicktime file such that the moov atom
  * is in front of the data, thus facilitating network streaming.
+ * Alternatively, it can also be used to probe a file for whether it is
+ * streamable or not, and if it's fragmented.
  *
  * To compile this program, start from the base directory from which you
  * are building FFmpeg and type:
@@ -15,6 +17,8 @@ 
  * guaranteed, particularly on 64-bit platforms.
  * Invoke the program with:
  *  qt-faststart <infile.mov> <outfile.mov>
+ * To probe a file for faststart-ness, invoke it with:
+ *  qt-faststart <infile.mov>
  *
  * Notes: Quicktime files can come in many configurations of top-level
  * atoms. This utility stipulates that the very last atom in the file needs
@@ -88,6 +92,9 @@ 
 #define PICT_ATOM QT_ATOM('P', 'I', 'C', 'T')
 #define FTYP_ATOM QT_ATOM('f', 't', 'y', 'p')
 #define UUID_ATOM QT_ATOM('u', 'u', 'i', 'd')
+#define SIDX_ATOM QT_ATOM('s', 'i', 'd', 'x')
+#define STYP_ATOM QT_ATOM('s', 't', 'y', 'p')
+#define MOOF_ATOM QT_ATOM('m', 'o', 'o', 'f')
 
 #define CMOV_ATOM QT_ATOM('c', 'm', 'o', 'v')
 #define TRAK_ATOM QT_ATOM('t', 'r', 'a', 'k')
@@ -443,14 +450,29 @@  int main(int argc, char *argv[])
     int64_t start_offset = 0;
     unsigned char *copy_buffer = NULL;
     int bytes_to_copy;
-
-    if (argc != 3) {
-        printf("Usage: qt-faststart <infile.mov> <outfile.mov>\n"
-               "Note: alternatively you can use -movflags +faststart in ffmpeg\n");
+    int probe_only = argc == 2;
+    int is_fragmented = 0;
+    uint64_t first_mdat = 0;
+    uint64_t first_moov = 0;
+
+    if (argc != 2 && argc != 3) {
+        printf("Usage: qt-faststart <infile.mov> [<outfile.mov>]\n"
+               "Note: alternatively you can use -movflags +faststart in ffmpeg\n"
+               "\n"
+               "If both input and output filenames are given then these exit codes may be returned:\n"
+               " 0: Input file exists and is valid. If the file had to be faststarted then outfile.mov was written. Else nothing was written\n"
+               " 1: There was some kind of error, for example invalid input file, not enough memory or not enough disk space\n"
+               "\n"
+               "If no output filename is given then infile.mov is probed for faststart-ness and the following exit codes may be returned:\n"
+               " 0: File has moov before mdat, is streamable (not fragmented)\n"
+               " 1: There was some kind of error\n"
+               " 2: File has moov after mdat, not streamable\n"
+               " 3: File is fragmented (has moof atoms, is streamable in some players)\n"
+               );
         return 0;
     }
 
-    if (!strcmp(argv[1], argv[2])) {
+    if (!probe_only && !strcmp(argv[1], argv[2])) {
         fprintf(stderr, "input and output files need to be different\n");
         return 1;
     }
@@ -515,6 +537,19 @@  int main(int argc, char *argv[])
                (atom_type >>  0) & 255,
                atom_offset,
                atom_size);
+
+        if (atom_type == MOOV_ATOM && first_moov == 0) {
+            first_moov = atom_offset;
+        } else if (atom_type == MDAT_ATOM && first_mdat == 0) {
+            first_mdat = atom_offset;
+        } else if (atom_type == MOOF_ATOM) {
+            is_fragmented = 1;
+            if (probe_only) {
+                //we've determined that the file is fragmented - stop
+                break;
+            }
+        }
+
         if ((atom_type != FREE_ATOM) &&
             (atom_type != JUNK_ATOM) &&
             (atom_type != MDAT_ATOM) &&
@@ -524,7 +559,10 @@  int main(int argc, char *argv[])
             (atom_type != WIDE_ATOM) &&
             (atom_type != PICT_ATOM) &&
             (atom_type != UUID_ATOM) &&
-            (atom_type != FTYP_ATOM)) {
+            (atom_type != FTYP_ATOM) &&
+            (atom_type != SIDX_ATOM) &&
+            (atom_type != STYP_ATOM) &&
+            (atom_type != MOOF_ATOM)) {
             fprintf(stderr, "encountered non-QT top-level atom (is this a QuickTime file?)\n");
             break;
         }
@@ -537,6 +575,34 @@  int main(int argc, char *argv[])
             break;
     }
 
+    if (probe_only) {
+        free(ftyp_atom);
+        fclose(infile);
+
+        /* try to be a bit more helpful when probing */
+        if (is_fragmented) {
+            if (first_moov == 0) {
+                printf("fragmented file with no moov before moof, probably invalid\n");
+                return 1;
+            } else {
+                printf("file is fragmented, streamable in some players\n");
+                return 3;
+            }
+        } else if (first_moov == 0 || first_mdat == 0) {
+            printf("file seems to be missing moov and/or mdat\n");
+            return 1;
+        } else if (first_moov > first_mdat) {
+            printf("file has moov after mdat, not streamable\n");
+            return 2;
+        } else {
+            printf("file has moov before mdat, streamable\n");
+            return 0;
+        }
+    }
+    /* non-probe mode beyond this point */
+
+    /* this is possibly stricter than necessary,
+     * but it does ensure fragmented files aren't messed with */
     if (atom_type != MOOV_ATOM) {
         printf("last atom in file was not a moov atom\n");
         free(ftyp_atom);
-- 
2.11.0