diff mbox series

[FFmpeg-devel,2/2] fftools/ffmpeg: add an option for writing pre-muxing stats

Message ID 20230207093306.11959-2-anton@khirnov.net
State New
Headers show
Series [FFmpeg-devel,1/2] fftools/ffmpeg: store output packet timebases in the packet | expand

Checks

Context Check Description
yinshiyou/make_loongarch64 success Make finished
yinshiyou/make_fate_loongarch64 success Make fate finished
andriy/make_x86 success Make finished
andriy/make_fate_x86 success Make fate finished

Commit Message

Anton Khirnov Feb. 7, 2023, 9:33 a.m. UTC
Analogous to -enc_stats*, but happens right before muxing. Useful
because bitstream filters and the sync queue can modify packets after
encoding and before muxing. Also has access to the muxing timebase.
---
 Changelog                 |  3 ++-
 doc/ffmpeg.texi           | 13 +++++++++----
 fftools/ffmpeg.c          | 14 ++++++++------
 fftools/ffmpeg.h          |  8 ++++++++
 fftools/ffmpeg_mux.c      | 10 +++++++++-
 fftools/ffmpeg_mux.h      |  2 ++
 fftools/ffmpeg_mux_init.c | 23 ++++++++++++++++++-----
 fftools/ffmpeg_opt.c      |  4 ++++
 8 files changed, 60 insertions(+), 17 deletions(-)

Comments

Anton Khirnov Feb. 7, 2023, 9:41 a.m. UTC | #1
Also, I'm wondering if we shouldn't rename the options to
 -stats_{mux,enc_pre,enc_post}[_fmt] for consistency.
They were added just recently, so it should be fine if done before the
release.

Opinions?
Thilo Borgmann Feb. 7, 2023, 10:50 a.m. UTC | #2
Am 07.02.23 um 10:41 schrieb Anton Khirnov:
> Also, I'm wondering if we shouldn't rename the options to
>   -stats_{mux,enc_pre,enc_post}[_fmt] for consistency.
> They were added just recently, so it should be fine if done before the
> release.
> 
> Opinions?

I'd like that. Can't comment on the code itself, though.

-Thilo
Anton Khirnov Feb. 8, 2023, 1:53 p.m. UTC | #3
Quoting Tim Harris (2023-02-07 18:19:48)
> > > Am 07.02.23 um 10:41 schrieb Anton Khirnov:
> > > Also, I'm wondering if we shouldn't rename the options to
> > >   -stats_{mux,enc_pre,enc_post}[_fmt] for consistency.
> > > They were added just recently, so it should be fine if done before the
> > > release.
> > >
> > > Opinions?
> >
> > I'd like that. Can't comment on the code itself, though.
> >
> 
> -stats_mux_pre (in case we want to report stats post muxing in the future?)

Muxers are fundamentally consume-only, I don't see what stats could
possibly be printed after muxing, even conceptually.
Devin Heitmueller Feb. 8, 2023, 3:59 p.m. UTC | #4
On Wed, Feb 8, 2023 at 8:54 AM Anton Khirnov <anton@khirnov.net> wrote:
> Muxers are fundamentally consume-only, I don't see what stats could
> possibly be printed after muxing, even conceptually.

I can offer some examples, in particular because I generate lots of
such statistics after muxing in my private tree.

- Decklink output statistics - late arrival and drop counts for audio
and video packets, pipeline latency (time between submission to the
hardware and the callback indicating frame was played out), FIFO
levels within the module, video frames actually played out in last
second, audio samples played out in last second.

- SRT output - various SRT related statistics (lost packets,
retransmits, connection RTT, etc)

- HLS output - counters for various HTTP recoverable events (HTTP
redirects, resends, etc), packets written

Devin
Anton Khirnov Feb. 8, 2023, 4:10 p.m. UTC | #5
Quoting Devin Heitmueller (2023-02-08 16:59:43)
> On Wed, Feb 8, 2023 at 8:54 AM Anton Khirnov <anton@khirnov.net> wrote:
> > Muxers are fundamentally consume-only, I don't see what stats could
> > possibly be printed after muxing, even conceptually.
> 
> I can offer some examples, in particular because I generate lots of
> such statistics after muxing in my private tree.
> 
> - Decklink output statistics - late arrival and drop counts for audio
> and video packets, pipeline latency (time between submission to the
> hardware and the callback indicating frame was played out), FIFO
> levels within the module, video frames actually played out in last
> second, audio samples played out in last second.
> 
> - SRT output - various SRT related statistics (lost packets,
> retransmits, connection RTT, etc)
> 
> - HLS output - counters for various HTTP recoverable events (HTTP
> redirects, resends, etc), packets written

Hmm, I have not considered it but you're right that this information
could be exported via AVOptions and printed by ffmpeg CLI.

I'll send a patch with the renames then.
diff mbox series

Patch

diff --git a/Changelog b/Changelog
index df9cd69da2..3738df2827 100644
--- a/Changelog
+++ b/Changelog
@@ -32,7 +32,8 @@  version <next>:
 - WADY DPCM decoder and demuxer
 - CBD2 DPCM decoder
 - ssim360 video filter
-- ffmpeg CLI new options: -enc_stats_pre[_fmt], -enc_stats_post[_fmt]
+- ffmpeg CLI new options: -enc_stats_pre[_fmt], -enc_stats_post[_fmt],
+  -mux_stats[_fmt]
 - hstack_vaapi, vstack_vaapi and xstack_vaapi filters
 - XMD ADPCM decoder and demuxer
 - media100 to mjpegb bsf
diff --git a/doc/ffmpeg.texi b/doc/ffmpeg.texi
index 592c4b4393..aa7e1cc1e8 100644
--- a/doc/ffmpeg.texi
+++ b/doc/ffmpeg.texi
@@ -2063,14 +2063,17 @@  or invalid output files.
 
 @item -enc_stats_pre[:@var{stream_specifier}] @var{path} (@emph{output,per-stream})
 @item -enc_stats_post[:@var{stream_specifier}] @var{path} (@emph{output,per-stream})
+@item -mux_stats[:@var{stream_specifier}] @var{path} (@emph{output,per-stream})
 Write per-frame encoding information about the matching streams into the file
 given by @var{path}.
 
 @option{-enc_stats_pre} writes information about raw video or audio frames right
 before they are sent for encoding, while @option{-enc_stats_post} writes
-information about encoded packets as they are received from the encoder. Every
-frame or packet produces one line in the specified file. The format of this line
-is controlled by @option{-enc_stats_pre_fmt} / @option{-enc_stats_post_fmt}.
+information about encoded packets as they are received from the encoder.
+@option{-mux_stats} writes information about packets just as they are about to
+be sent to the muxer. Every frame or packet produces one line in the specified
+file. The format of this line is controlled by @option{-enc_stats_pre_fmt} /
+@option{-enc_stats_post_fmt} / @option{-mux_stats_fmt}.
 
 When stats for multiple streams are written into a single file, the lines
 corresponding to different streams will be interleaved. The precise order of
@@ -2079,8 +2082,9 @@  different invocations of the program, even with the same options.
 
 @item -enc_stats_pre_fmt[:@var{stream_specifier}] @var{format_spec} (@emph{output,per-stream})
 @item -enc_stats_post_fmt[:@var{stream_specifier}] @var{format_spec} (@emph{output,per-stream})
+@item -mux_stats_fmt[:@var{stream_specifier}] @var{format_spec} (@emph{output,per-stream})
 Specify the format for the lines written with @option{-enc_stats_pre} /
-@option{-enc_stats_post}.
+@option{-enc_stats_post} / @option{-mux_stats}.
 
 @var{format_spec} is a string that may contain directives of the form
 @var{@{fmt@}}. @var{format_spec} is backslash-escaped --- use \@{, \@}, and \\
@@ -2097,6 +2101,7 @@  Index of the output stream in the file.
 @item n
 Frame number. Pre-encoding: number of frames sent to the encoder so far.
 Post-encoding: number of packets received from the encoder so far.
+Muxing: number of packets submitted to the muxer for this stream so far.
 
 @item ni
 Input frame number. Index of the input frame (i.e. output by a decoder) that
diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c
index 32e0c3febd..aac393c714 100644
--- a/fftools/ffmpeg.c
+++ b/fftools/ffmpeg.c
@@ -808,8 +808,9 @@  static void update_video_stats(OutputStream *ost, const AVPacket *pkt, int write
     fprintf(vstats_file, "type= %c\n", av_get_picture_type_char(ost->pict_type));
 }
 
-static void enc_stats_write(OutputStream *ost, EncStats *es,
-                            const AVFrame *frame, const AVPacket *pkt)
+void enc_stats_write(OutputStream *ost, EncStats *es,
+                     const AVFrame *frame, const AVPacket *pkt,
+                     uint64_t frame_num)
 {
     AVIOContext *io = es->io;
     AVRational   tb = frame ? frame->time_base : pkt->time_base;
@@ -840,12 +841,12 @@  static void enc_stats_write(OutputStream *ost, EncStats *es,
         case ENC_STATS_PTS_TIME:        avio_printf(io, "%g",       pts * av_q2d(tb));              continue;
         case ENC_STATS_PTS_TIME_IN:     avio_printf(io, "%g",       ptsi == INT64_MAX ?
                                                                     INFINITY : ptsi * av_q2d(tbi)); continue;
+        case ENC_STATS_FRAME_NUM:       avio_printf(io, "%"PRIu64,  frame_num);                     continue;
         case ENC_STATS_FRAME_NUM_IN:    avio_printf(io, "%"PRIu64,  fd ? fd->idx : -1);             continue;
         }
 
         if (frame) {
             switch (c->type) {
-            case ENC_STATS_FRAME_NUM:   avio_printf(io, "%"PRIu64,  ost->frames_encoded);           continue;
             case ENC_STATS_SAMPLE_NUM:  avio_printf(io, "%"PRIu64,  ost->samples_encoded);          continue;
             case ENC_STATS_NB_SAMPLES:  avio_printf(io, "%d",       frame->nb_samples);             continue;
             default: av_assert0(0);
@@ -855,7 +856,6 @@  static void enc_stats_write(OutputStream *ost, EncStats *es,
             case ENC_STATS_DTS:         avio_printf(io, "%"PRId64,  pkt->dts);                      continue;
             case ENC_STATS_DTS_TIME:    avio_printf(io, "%g",       pkt->dts * av_q2d(tb));         continue;
             case ENC_STATS_PKT_SIZE:    avio_printf(io, "%d",       pkt->size);                     continue;
-            case ENC_STATS_FRAME_NUM:   avio_printf(io, "%"PRIu64,  ost->packets_encoded);          continue;
             case ENC_STATS_BITRATE: {
                 double duration = FFMAX(pkt->duration, 1) * av_q2d(tb);
                 avio_printf(io, "%g",  8.0 * pkt->size / duration);
@@ -884,7 +884,8 @@  static int encode_frame(OutputFile *of, OutputStream *ost, AVFrame *frame)
 
     if (frame) {
         if (ost->enc_stats_pre.io)
-            enc_stats_write(ost, &ost->enc_stats_pre, frame, NULL);
+            enc_stats_write(ost, &ost->enc_stats_pre, frame, NULL,
+                            ost->frames_encoded);
 
         ost->frames_encoded++;
         ost->samples_encoded += frame->nb_samples;
@@ -932,7 +933,8 @@  static int encode_frame(OutputFile *of, OutputStream *ost, AVFrame *frame)
         if (enc->codec_type == AVMEDIA_TYPE_VIDEO)
             update_video_stats(ost, pkt, !!vstats_filename);
         if (ost->enc_stats_post.io)
-            enc_stats_write(ost, &ost->enc_stats_post, NULL, pkt);
+            enc_stats_write(ost, &ost->enc_stats_post, NULL, pkt,
+                            ost->packets_encoded);
 
         if (debug_ts) {
             av_log(ost, AV_LOG_INFO, "encoder -> type:%s "
diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h
index 933312dba7..f1412f6446 100644
--- a/fftools/ffmpeg.h
+++ b/fftools/ffmpeg.h
@@ -258,10 +258,14 @@  typedef struct OptionsContext {
     int        nb_enc_stats_pre;
     SpecifierOpt *enc_stats_post;
     int        nb_enc_stats_post;
+    SpecifierOpt *mux_stats;
+    int        nb_mux_stats;
     SpecifierOpt *enc_stats_pre_fmt;
     int        nb_enc_stats_pre_fmt;
     SpecifierOpt *enc_stats_post_fmt;
     int        nb_enc_stats_post_fmt;
+    SpecifierOpt *mux_stats_fmt;
+    int        nb_mux_stats_fmt;
 } OptionsContext;
 
 typedef struct InputFilter {
@@ -789,6 +793,10 @@  int ifilter_parameters_from_frame(InputFilter *ifilter, const AVFrame *frame);
 
 int ffmpeg_parse_options(int argc, char **argv);
 
+void enc_stats_write(OutputStream *ost, EncStats *es,
+                     const AVFrame *frame, const AVPacket *pkt,
+                     uint64_t frame_num);
+
 HWDevice *hw_device_get_by_name(const char *name);
 int hw_device_init_from_string(const char *arg, HWDevice **dev);
 void hw_device_free_all(void);
diff --git a/fftools/ffmpeg_mux.c b/fftools/ffmpeg_mux.c
index 30764e22d1..dffc1410c8 100644
--- a/fftools/ffmpeg_mux.c
+++ b/fftools/ffmpeg_mux.c
@@ -64,6 +64,7 @@  static int write_packet(Muxer *mux, OutputStream *ost, AVPacket *pkt)
     AVFormatContext *s = mux->fc;
     AVStream *st = ost->st;
     int64_t fs;
+    uint64_t frame_num;
     int ret;
 
     fs = filesize(s->pb);
@@ -128,7 +129,7 @@  static int write_packet(Muxer *mux, OutputStream *ost, AVPacket *pkt)
     ms->last_mux_dts = pkt->dts;
 
     ost->data_size_mux += pkt->size;
-    atomic_fetch_add(&ost->packets_written, 1);
+    frame_num = atomic_fetch_add(&ost->packets_written, 1);
 
     pkt->stream_index = ost->index;
 
@@ -143,6 +144,9 @@  static int write_packet(Muxer *mux, OutputStream *ost, AVPacket *pkt)
               );
     }
 
+    if (ms->stats.io)
+        enc_stats_write(ost, &ms->stats, NULL, pkt, frame_num);
+
     ret = av_interleaved_write_frame(s, pkt);
     if (ret < 0) {
         print_error("av_interleaved_write_frame()", ret);
@@ -688,6 +692,10 @@  static void ost_free(OutputStream **post)
         av_freep(&ost->enc_stats_post.components[i].str);
     av_freep(&ost->enc_stats_post.components);
 
+    for (int i = 0; i < ms->stats.nb_components; i++)
+        av_freep(&ms->stats.components[i].str);
+    av_freep(&ms->stats.components);
+
     av_freep(post);
 }
 
diff --git a/fftools/ffmpeg_mux.h b/fftools/ffmpeg_mux.h
index 1487d86ae1..c76dc2e524 100644
--- a/fftools/ffmpeg_mux.h
+++ b/fftools/ffmpeg_mux.h
@@ -45,6 +45,8 @@  typedef struct MuxStream {
 
     AVBSFContext *bsf_ctx;
 
+    EncStats stats;
+
     int64_t max_frames;
 
     /*
diff --git a/fftools/ffmpeg_mux_init.c b/fftools/ffmpeg_mux_init.c
index 834cdbcc9f..f4ef83f6af 100644
--- a/fftools/ffmpeg_mux_init.c
+++ b/fftools/ffmpeg_mux_init.c
@@ -57,8 +57,10 @@  static const char *const opt_name_disposition[]               = {"disposition",
 static const char *const opt_name_enc_time_bases[]            = {"enc_time_base", NULL};
 static const char *const opt_name_enc_stats_pre[]             = {"enc_stats_pre", NULL};
 static const char *const opt_name_enc_stats_post[]            = {"enc_stats_post", NULL};
+static const char *const opt_name_mux_stats[]                 = {"mux_stats", NULL};
 static const char *const opt_name_enc_stats_pre_fmt[]         = {"enc_stats_pre_fmt", NULL};
 static const char *const opt_name_enc_stats_post_fmt[]        = {"enc_stats_post_fmt", NULL};
+static const char *const opt_name_mux_stats_fmt[]             = {"mux_stats_fmt", NULL};
 static const char *const opt_name_filters[]                   = {"filter", "af", "vf", NULL};
 static const char *const opt_name_filter_scripts[]            = {"filter_script", NULL};
 static const char *const opt_name_fix_sub_duration_heartbeat[] = {"fix_sub_duration_heartbeat", NULL};
@@ -262,7 +264,7 @@  static int unescape(char **pdst, size_t *dst_len,
     return 0;
 }
 
-static int enc_stats_init(OutputStream *ost, int pre,
+static int enc_stats_init(OutputStream *ost, EncStats *es, int pre,
                           const char *path, const char *fmt_spec)
 {
     static const struct {
@@ -290,7 +292,6 @@  static int enc_stats_init(OutputStream *ost, int pre,
         { ENC_STATS_BITRATE,        "br",       0, 1            },
         { ENC_STATS_AVG_BITRATE,    "abr",      0, 1            },
     };
-    EncStats *es = pre ? &ost->enc_stats_pre : &ost->enc_stats_post;
     const char *next = fmt_spec;
 
     int ret;
@@ -479,7 +480,7 @@  static OutputStream *new_output_stream(Muxer *mux, const OptionsContext *o,
         AVCodecContext *enc = ost->enc_ctx;
         AVIOContext *s = NULL;
         char *buf = NULL, *arg = NULL, *preset = NULL;
-        const char *enc_stats_pre = NULL, *enc_stats_post = NULL;
+        const char *enc_stats_pre = NULL, *enc_stats_post = NULL, *mux_stats = NULL;
 
         ost->encoder_opts = filter_codec_opts(o->g->codec_opts, enc->codec_id,
                                               oc, st, enc->codec);
@@ -518,7 +519,7 @@  static OutputStream *new_output_stream(Muxer *mux, const OptionsContext *o,
 
             MATCH_PER_STREAM_OPT(enc_stats_pre_fmt, str, format, oc, st);
 
-            ret = enc_stats_init(ost, 1, enc_stats_pre, format);
+            ret = enc_stats_init(ost, &ost->enc_stats_pre, 1, enc_stats_pre, format);
             if (ret < 0)
                 exit_program(1);
         }
@@ -530,7 +531,19 @@  static OutputStream *new_output_stream(Muxer *mux, const OptionsContext *o,
 
             MATCH_PER_STREAM_OPT(enc_stats_post_fmt, str, format, oc, st);
 
-            ret = enc_stats_init(ost, 0, enc_stats_post, format);
+            ret = enc_stats_init(ost, &ost->enc_stats_post, 0, enc_stats_post, format);
+            if (ret < 0)
+                exit_program(1);
+        }
+
+        MATCH_PER_STREAM_OPT(mux_stats, str, mux_stats, oc, st);
+        if (mux_stats &&
+            (type == AVMEDIA_TYPE_VIDEO || type == AVMEDIA_TYPE_AUDIO)) {
+            const char *format = "{fidx} {sidx} {n} {t}";
+
+            MATCH_PER_STREAM_OPT(mux_stats_fmt, str, format, oc, st);
+
+            ret = enc_stats_init(ost, &ms->stats, 0, mux_stats, format);
             if (ret < 0)
                 exit_program(1);
         }
diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c
index 204be38c94..93fd9509ce 100644
--- a/fftools/ffmpeg_opt.c
+++ b/fftools/ffmpeg_opt.c
@@ -1548,10 +1548,14 @@  const OptionDef options[] = {
         "write encoding stats before encoding" },
     { "enc_stats_post",     HAS_ARG | OPT_SPEC | OPT_EXPERT | OPT_OUTPUT | OPT_STRING, { .off = OFFSET(enc_stats_post)     },
         "write encoding stats after encoding" },
+    { "mux_stats",          HAS_ARG | OPT_SPEC | OPT_EXPERT | OPT_OUTPUT | OPT_STRING, { .off = OFFSET(mux_stats)          },
+        "write packets stats before muxing" },
     { "enc_stats_pre_fmt",  HAS_ARG | OPT_SPEC | OPT_EXPERT | OPT_OUTPUT | OPT_STRING, { .off = OFFSET(enc_stats_pre_fmt)  },
         "format of the stats written with -enc_stats_pre" },
     { "enc_stats_post_fmt", HAS_ARG | OPT_SPEC | OPT_EXPERT | OPT_OUTPUT | OPT_STRING, { .off = OFFSET(enc_stats_post_fmt) },
         "format of the stats written with -enc_stats_post" },
+    { "mux_stats_fmt",      HAS_ARG | OPT_SPEC | OPT_EXPERT | OPT_OUTPUT | OPT_STRING, { .off = OFFSET(mux_stats_fmt)      },
+        "format of the stats written with -mux_stats" },
 
     /* video options */
     { "vframes",      OPT_VIDEO | HAS_ARG  | OPT_PERFILE | OPT_OUTPUT,           { .func_arg = opt_video_frames },