From patchwork Tue Feb 7 09:33:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 40314 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:5494:b0:bf:7b3a:fd32 with SMTP id i20csp4130109pzk; Tue, 7 Feb 2023 01:33:44 -0800 (PST) X-Google-Smtp-Source: AK7set8LZdaBkhU4fvoH/3OjbsDmFmUXH+AwVY/RlVWzTelMgv3ju2TcfaE2T79YVEyft1zNWmmU X-Received: by 2002:a50:d74c:0:b0:4aa:a4e9:fa28 with SMTP id i12-20020a50d74c000000b004aaa4e9fa28mr2790944edj.34.1675762424047; Tue, 07 Feb 2023 01:33:44 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675762424; cv=none; d=google.com; s=arc-20160816; b=kYuLqEs0mwXYmoGZbG1YjWuCA6D9GDlaay0x9PeAzWEgGogoPKUiKqhoUdAwsUPxTE mb+x6+sFM+Fkr1ar2QA0fV0VK7ijXHmoalI/HvC8x1Q44Yf/JrOG1ZfbhvtHkHdPz77F Ne+gJEL9iMQmTsJdiv6PTx9I6wiRmTjtcnBhkUKp9LoHx68eoZzrOAd5Fqo4azlWkTja AuQiQ4zKtpF9zzrVgXN/5Dp7t4+8RELKKN/+ypp3LyA4NQJqOBg0q4n10ELgs6nBqo3b moygNM3fOpdvAdeBfBvovYHGAy3OITyvL2cHrOHsN5P1ZGJkckClWlfXMI1zkfoM0Vnq McxQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:delivered-to; bh=QWbYtbKCNT2olug4YDFevwuMGzivDFPTc5CW75/5ZIU=; b=DyPSjdQIYhA0GT6cv/E7v6M1SKisxORUZAEEd4LEri0Uct7Jiiv+mWi4QTAwfsXh+T S/MhZgiLIQzCoF0jBxL2gnjuZyHJgCaU4huymIgFSqw9drReMw02OcnGSQ5m2gIrpVKu LV4YVUhT91Idr9ybk27IHg0+z8hFzuxMIfZLckGpM8KB/QTiHhYY6zEoc0xzfZeb9hDJ oIgiA+C+b+yRKwjof/qRiMaCOPy7yf3+KCNSQOTysqod5TY4P5JkXhBam6AB6wedl4Ht AvjFbElWIOp33ierWdB5kl9i3ujl0EfphXXJSYif0au5eaEULyWwaOfIwLYndwhPkG/G EoVQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id p19-20020a056402045300b004aad854e24csi407605edw.24.2023.02.07.01.33.43; Tue, 07 Feb 2023 01:33:44 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3E81868BEFA; Tue, 7 Feb 2023 11:33:30 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail0.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0154D68BECA for ; Tue, 7 Feb 2023 11:33:21 +0200 (EET) Received: from localhost (localhost [IPv6:::1]) by mail0.khirnov.net (Postfix) with ESMTP id 335252404EC for ; Tue, 7 Feb 2023 10:33:21 +0100 (CET) Received: from mail0.khirnov.net ([IPv6:::1]) by localhost (mail0.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id PEz02aJQm3FH for ; Tue, 7 Feb 2023 10:33:20 +0100 (CET) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail0.khirnov.net (Postfix) with ESMTPS id 0517F2404EE for ; Tue, 7 Feb 2023 10:33:20 +0100 (CET) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:::1]) by libav.khirnov.net (Postfix) with ESMTP id BE8193A0D6F for ; Tue, 7 Feb 2023 10:33:19 +0100 (CET) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Tue, 7 Feb 2023 10:33:06 +0100 Message-Id: <20230207093306.11959-2-anton@khirnov.net> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20230207093306.11959-1-anton@khirnov.net> References: <20230207093306.11959-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/2] fftools/ffmpeg: add an option for writing pre-muxing stats X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: gv4dBlEoclYr Analogous to -enc_stats*, but happens right before muxing. Useful because bitstream filters and the sync queue can modify packets after encoding and before muxing. Also has access to the muxing timebase. --- Changelog | 3 ++- doc/ffmpeg.texi | 13 +++++++++---- fftools/ffmpeg.c | 14 ++++++++------ fftools/ffmpeg.h | 8 ++++++++ fftools/ffmpeg_mux.c | 10 +++++++++- fftools/ffmpeg_mux.h | 2 ++ fftools/ffmpeg_mux_init.c | 23 ++++++++++++++++++----- fftools/ffmpeg_opt.c | 4 ++++ 8 files changed, 60 insertions(+), 17 deletions(-) diff --git a/Changelog b/Changelog index df9cd69da2..3738df2827 100644 --- a/Changelog +++ b/Changelog @@ -32,7 +32,8 @@ version : - WADY DPCM decoder and demuxer - CBD2 DPCM decoder - ssim360 video filter -- ffmpeg CLI new options: -enc_stats_pre[_fmt], -enc_stats_post[_fmt] +- ffmpeg CLI new options: -enc_stats_pre[_fmt], -enc_stats_post[_fmt], + -mux_stats[_fmt] - hstack_vaapi, vstack_vaapi and xstack_vaapi filters - XMD ADPCM decoder and demuxer - media100 to mjpegb bsf diff --git a/doc/ffmpeg.texi b/doc/ffmpeg.texi index 592c4b4393..aa7e1cc1e8 100644 --- a/doc/ffmpeg.texi +++ b/doc/ffmpeg.texi @@ -2063,14 +2063,17 @@ or invalid output files. @item -enc_stats_pre[:@var{stream_specifier}] @var{path} (@emph{output,per-stream}) @item -enc_stats_post[:@var{stream_specifier}] @var{path} (@emph{output,per-stream}) +@item -mux_stats[:@var{stream_specifier}] @var{path} (@emph{output,per-stream}) Write per-frame encoding information about the matching streams into the file given by @var{path}. @option{-enc_stats_pre} writes information about raw video or audio frames right before they are sent for encoding, while @option{-enc_stats_post} writes -information about encoded packets as they are received from the encoder. Every -frame or packet produces one line in the specified file. The format of this line -is controlled by @option{-enc_stats_pre_fmt} / @option{-enc_stats_post_fmt}. +information about encoded packets as they are received from the encoder. +@option{-mux_stats} writes information about packets just as they are about to +be sent to the muxer. Every frame or packet produces one line in the specified +file. The format of this line is controlled by @option{-enc_stats_pre_fmt} / +@option{-enc_stats_post_fmt} / @option{-mux_stats_fmt}. When stats for multiple streams are written into a single file, the lines corresponding to different streams will be interleaved. The precise order of @@ -2079,8 +2082,9 @@ different invocations of the program, even with the same options. @item -enc_stats_pre_fmt[:@var{stream_specifier}] @var{format_spec} (@emph{output,per-stream}) @item -enc_stats_post_fmt[:@var{stream_specifier}] @var{format_spec} (@emph{output,per-stream}) +@item -mux_stats_fmt[:@var{stream_specifier}] @var{format_spec} (@emph{output,per-stream}) Specify the format for the lines written with @option{-enc_stats_pre} / -@option{-enc_stats_post}. +@option{-enc_stats_post} / @option{-mux_stats}. @var{format_spec} is a string that may contain directives of the form @var{@{fmt@}}. @var{format_spec} is backslash-escaped --- use \@{, \@}, and \\ @@ -2097,6 +2101,7 @@ Index of the output stream in the file. @item n Frame number. Pre-encoding: number of frames sent to the encoder so far. Post-encoding: number of packets received from the encoder so far. +Muxing: number of packets submitted to the muxer for this stream so far. @item ni Input frame number. Index of the input frame (i.e. output by a decoder) that diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index 32e0c3febd..aac393c714 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -808,8 +808,9 @@ static void update_video_stats(OutputStream *ost, const AVPacket *pkt, int write fprintf(vstats_file, "type= %c\n", av_get_picture_type_char(ost->pict_type)); } -static void enc_stats_write(OutputStream *ost, EncStats *es, - const AVFrame *frame, const AVPacket *pkt) +void enc_stats_write(OutputStream *ost, EncStats *es, + const AVFrame *frame, const AVPacket *pkt, + uint64_t frame_num) { AVIOContext *io = es->io; AVRational tb = frame ? frame->time_base : pkt->time_base; @@ -840,12 +841,12 @@ static void enc_stats_write(OutputStream *ost, EncStats *es, case ENC_STATS_PTS_TIME: avio_printf(io, "%g", pts * av_q2d(tb)); continue; case ENC_STATS_PTS_TIME_IN: avio_printf(io, "%g", ptsi == INT64_MAX ? INFINITY : ptsi * av_q2d(tbi)); continue; + case ENC_STATS_FRAME_NUM: avio_printf(io, "%"PRIu64, frame_num); continue; case ENC_STATS_FRAME_NUM_IN: avio_printf(io, "%"PRIu64, fd ? fd->idx : -1); continue; } if (frame) { switch (c->type) { - case ENC_STATS_FRAME_NUM: avio_printf(io, "%"PRIu64, ost->frames_encoded); continue; case ENC_STATS_SAMPLE_NUM: avio_printf(io, "%"PRIu64, ost->samples_encoded); continue; case ENC_STATS_NB_SAMPLES: avio_printf(io, "%d", frame->nb_samples); continue; default: av_assert0(0); @@ -855,7 +856,6 @@ static void enc_stats_write(OutputStream *ost, EncStats *es, case ENC_STATS_DTS: avio_printf(io, "%"PRId64, pkt->dts); continue; case ENC_STATS_DTS_TIME: avio_printf(io, "%g", pkt->dts * av_q2d(tb)); continue; case ENC_STATS_PKT_SIZE: avio_printf(io, "%d", pkt->size); continue; - case ENC_STATS_FRAME_NUM: avio_printf(io, "%"PRIu64, ost->packets_encoded); continue; case ENC_STATS_BITRATE: { double duration = FFMAX(pkt->duration, 1) * av_q2d(tb); avio_printf(io, "%g", 8.0 * pkt->size / duration); @@ -884,7 +884,8 @@ static int encode_frame(OutputFile *of, OutputStream *ost, AVFrame *frame) if (frame) { if (ost->enc_stats_pre.io) - enc_stats_write(ost, &ost->enc_stats_pre, frame, NULL); + enc_stats_write(ost, &ost->enc_stats_pre, frame, NULL, + ost->frames_encoded); ost->frames_encoded++; ost->samples_encoded += frame->nb_samples; @@ -932,7 +933,8 @@ static int encode_frame(OutputFile *of, OutputStream *ost, AVFrame *frame) if (enc->codec_type == AVMEDIA_TYPE_VIDEO) update_video_stats(ost, pkt, !!vstats_filename); if (ost->enc_stats_post.io) - enc_stats_write(ost, &ost->enc_stats_post, NULL, pkt); + enc_stats_write(ost, &ost->enc_stats_post, NULL, pkt, + ost->packets_encoded); if (debug_ts) { av_log(ost, AV_LOG_INFO, "encoder -> type:%s " diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index 933312dba7..f1412f6446 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -258,10 +258,14 @@ typedef struct OptionsContext { int nb_enc_stats_pre; SpecifierOpt *enc_stats_post; int nb_enc_stats_post; + SpecifierOpt *mux_stats; + int nb_mux_stats; SpecifierOpt *enc_stats_pre_fmt; int nb_enc_stats_pre_fmt; SpecifierOpt *enc_stats_post_fmt; int nb_enc_stats_post_fmt; + SpecifierOpt *mux_stats_fmt; + int nb_mux_stats_fmt; } OptionsContext; typedef struct InputFilter { @@ -789,6 +793,10 @@ int ifilter_parameters_from_frame(InputFilter *ifilter, const AVFrame *frame); int ffmpeg_parse_options(int argc, char **argv); +void enc_stats_write(OutputStream *ost, EncStats *es, + const AVFrame *frame, const AVPacket *pkt, + uint64_t frame_num); + HWDevice *hw_device_get_by_name(const char *name); int hw_device_init_from_string(const char *arg, HWDevice **dev); void hw_device_free_all(void); diff --git a/fftools/ffmpeg_mux.c b/fftools/ffmpeg_mux.c index 30764e22d1..dffc1410c8 100644 --- a/fftools/ffmpeg_mux.c +++ b/fftools/ffmpeg_mux.c @@ -64,6 +64,7 @@ static int write_packet(Muxer *mux, OutputStream *ost, AVPacket *pkt) AVFormatContext *s = mux->fc; AVStream *st = ost->st; int64_t fs; + uint64_t frame_num; int ret; fs = filesize(s->pb); @@ -128,7 +129,7 @@ static int write_packet(Muxer *mux, OutputStream *ost, AVPacket *pkt) ms->last_mux_dts = pkt->dts; ost->data_size_mux += pkt->size; - atomic_fetch_add(&ost->packets_written, 1); + frame_num = atomic_fetch_add(&ost->packets_written, 1); pkt->stream_index = ost->index; @@ -143,6 +144,9 @@ static int write_packet(Muxer *mux, OutputStream *ost, AVPacket *pkt) ); } + if (ms->stats.io) + enc_stats_write(ost, &ms->stats, NULL, pkt, frame_num); + ret = av_interleaved_write_frame(s, pkt); if (ret < 0) { print_error("av_interleaved_write_frame()", ret); @@ -688,6 +692,10 @@ static void ost_free(OutputStream **post) av_freep(&ost->enc_stats_post.components[i].str); av_freep(&ost->enc_stats_post.components); + for (int i = 0; i < ms->stats.nb_components; i++) + av_freep(&ms->stats.components[i].str); + av_freep(&ms->stats.components); + av_freep(post); } diff --git a/fftools/ffmpeg_mux.h b/fftools/ffmpeg_mux.h index 1487d86ae1..c76dc2e524 100644 --- a/fftools/ffmpeg_mux.h +++ b/fftools/ffmpeg_mux.h @@ -45,6 +45,8 @@ typedef struct MuxStream { AVBSFContext *bsf_ctx; + EncStats stats; + int64_t max_frames; /* diff --git a/fftools/ffmpeg_mux_init.c b/fftools/ffmpeg_mux_init.c index 834cdbcc9f..f4ef83f6af 100644 --- a/fftools/ffmpeg_mux_init.c +++ b/fftools/ffmpeg_mux_init.c @@ -57,8 +57,10 @@ static const char *const opt_name_disposition[] = {"disposition", static const char *const opt_name_enc_time_bases[] = {"enc_time_base", NULL}; static const char *const opt_name_enc_stats_pre[] = {"enc_stats_pre", NULL}; static const char *const opt_name_enc_stats_post[] = {"enc_stats_post", NULL}; +static const char *const opt_name_mux_stats[] = {"mux_stats", NULL}; static const char *const opt_name_enc_stats_pre_fmt[] = {"enc_stats_pre_fmt", NULL}; static const char *const opt_name_enc_stats_post_fmt[] = {"enc_stats_post_fmt", NULL}; +static const char *const opt_name_mux_stats_fmt[] = {"mux_stats_fmt", NULL}; static const char *const opt_name_filters[] = {"filter", "af", "vf", NULL}; static const char *const opt_name_filter_scripts[] = {"filter_script", NULL}; static const char *const opt_name_fix_sub_duration_heartbeat[] = {"fix_sub_duration_heartbeat", NULL}; @@ -262,7 +264,7 @@ static int unescape(char **pdst, size_t *dst_len, return 0; } -static int enc_stats_init(OutputStream *ost, int pre, +static int enc_stats_init(OutputStream *ost, EncStats *es, int pre, const char *path, const char *fmt_spec) { static const struct { @@ -290,7 +292,6 @@ static int enc_stats_init(OutputStream *ost, int pre, { ENC_STATS_BITRATE, "br", 0, 1 }, { ENC_STATS_AVG_BITRATE, "abr", 0, 1 }, }; - EncStats *es = pre ? &ost->enc_stats_pre : &ost->enc_stats_post; const char *next = fmt_spec; int ret; @@ -479,7 +480,7 @@ static OutputStream *new_output_stream(Muxer *mux, const OptionsContext *o, AVCodecContext *enc = ost->enc_ctx; AVIOContext *s = NULL; char *buf = NULL, *arg = NULL, *preset = NULL; - const char *enc_stats_pre = NULL, *enc_stats_post = NULL; + const char *enc_stats_pre = NULL, *enc_stats_post = NULL, *mux_stats = NULL; ost->encoder_opts = filter_codec_opts(o->g->codec_opts, enc->codec_id, oc, st, enc->codec); @@ -518,7 +519,7 @@ static OutputStream *new_output_stream(Muxer *mux, const OptionsContext *o, MATCH_PER_STREAM_OPT(enc_stats_pre_fmt, str, format, oc, st); - ret = enc_stats_init(ost, 1, enc_stats_pre, format); + ret = enc_stats_init(ost, &ost->enc_stats_pre, 1, enc_stats_pre, format); if (ret < 0) exit_program(1); } @@ -530,7 +531,19 @@ static OutputStream *new_output_stream(Muxer *mux, const OptionsContext *o, MATCH_PER_STREAM_OPT(enc_stats_post_fmt, str, format, oc, st); - ret = enc_stats_init(ost, 0, enc_stats_post, format); + ret = enc_stats_init(ost, &ost->enc_stats_post, 0, enc_stats_post, format); + if (ret < 0) + exit_program(1); + } + + MATCH_PER_STREAM_OPT(mux_stats, str, mux_stats, oc, st); + if (mux_stats && + (type == AVMEDIA_TYPE_VIDEO || type == AVMEDIA_TYPE_AUDIO)) { + const char *format = "{fidx} {sidx} {n} {t}"; + + MATCH_PER_STREAM_OPT(mux_stats_fmt, str, format, oc, st); + + ret = enc_stats_init(ost, &ms->stats, 0, mux_stats, format); if (ret < 0) exit_program(1); } diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c index 204be38c94..93fd9509ce 100644 --- a/fftools/ffmpeg_opt.c +++ b/fftools/ffmpeg_opt.c @@ -1548,10 +1548,14 @@ const OptionDef options[] = { "write encoding stats before encoding" }, { "enc_stats_post", HAS_ARG | OPT_SPEC | OPT_EXPERT | OPT_OUTPUT | OPT_STRING, { .off = OFFSET(enc_stats_post) }, "write encoding stats after encoding" }, + { "mux_stats", HAS_ARG | OPT_SPEC | OPT_EXPERT | OPT_OUTPUT | OPT_STRING, { .off = OFFSET(mux_stats) }, + "write packets stats before muxing" }, { "enc_stats_pre_fmt", HAS_ARG | OPT_SPEC | OPT_EXPERT | OPT_OUTPUT | OPT_STRING, { .off = OFFSET(enc_stats_pre_fmt) }, "format of the stats written with -enc_stats_pre" }, { "enc_stats_post_fmt", HAS_ARG | OPT_SPEC | OPT_EXPERT | OPT_OUTPUT | OPT_STRING, { .off = OFFSET(enc_stats_post_fmt) }, "format of the stats written with -enc_stats_post" }, + { "mux_stats_fmt", HAS_ARG | OPT_SPEC | OPT_EXPERT | OPT_OUTPUT | OPT_STRING, { .off = OFFSET(mux_stats_fmt) }, + "format of the stats written with -mux_stats" }, /* video options */ { "vframes", OPT_VIDEO | HAS_ARG | OPT_PERFILE | OPT_OUTPUT, { .func_arg = opt_video_frames },