From patchwork Tue May 23 13:58:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 41804 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:c51c:b0:10c:5e6f:955f with SMTP id gm28csp601612pzb; Tue, 23 May 2023 07:01:04 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6jY4k3+wA9On4U7IPxptQX7DnackEdzbn5e4TDXHxsTo1S6LiR6SAuVBnwBtyx/8azAA7S X-Received: by 2002:a17:907:948f:b0:96b:48d2:77ef with SMTP id dm15-20020a170907948f00b0096b48d277efmr11892359ejc.6.1684850464090; Tue, 23 May 2023 07:01:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684850464; cv=none; d=google.com; s=arc-20160816; b=QjLFAVhw+Di4KStULqjRv/ihTxhr4YOyXU2svTNUGxuvzCTS7o3TbVrpOQHRuepzts 7zVBrqzNRBs+4ICbYMMJuVuYlz5/3lWYUbbmHM2Rev2ybco3ISfaz/K/OH16+TFV1hKr yY0QFbpxta6j+9sGKgMqxLXqgphzIRXlucPiCDNR0MkLFXzOTup+g0ifzsKVvA6ZSTgE yUxY6tU99lwZrF3RdrhM174CbOwKnffPTMURPKIGSZWuAed+nB/5738sa5KXTRNCl/mu QufRqWo3ndeTyg0du85NGN3PfPWbztK22fCnf1K6otQUhOqEPY8CwPtcT2kS41OswBQv A3zg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:delivered-to; bh=DikChbcKPDP23do9SxqXgBcZoIO8B/yyBR7Mo/tpm/o=; b=JeH5eccBfNe0cvDZ3vrmrt+AALvAYxzCmx9nkGrqGcC1NyNQFCXlUrNZxwu+RaFiZy 12F9gqr8CnLR15op+1ZR0d5COQyNdrsqjgvbfaDkwBfUSMTx7ccZYxdh7DbJ2tVdoF5i lQRLxsBdcd4i9EwUNgyNZc6dtQwKsWmcpY0dhFtCEhRX/h1iRwe16YYnjsouG5tm9F7V h80pmfCB3kLD8TiIq066onBOATTeBcigPnPxDu91i0YVDTrPUZs3RCJyZNsBSNSzmT0B TqAcRrlWbICyGn2zVEGHR4xXJ3904Cphehhc/KtjL+6MrOp4OwNw2dw2/XLY0bmA/IHn NImA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id qo5-20020a170907874500b0096621c999c5si6197792ejc.288.2023.05.23.07.01.03; Tue, 23 May 2023 07:01:04 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id BC35F68C16B; Tue, 23 May 2023 16:59:15 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail0.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 10DC768C105 for ; Tue, 23 May 2023 16:59:03 +0300 (EEST) Received: from localhost (localhost [IPv6:::1]) by mail0.khirnov.net (Postfix) with ESMTP id A08412405F9 for ; Tue, 23 May 2023 15:59:00 +0200 (CEST) Received: from mail0.khirnov.net ([IPv6:::1]) by localhost (mail0.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id tcTX738sRjfO for ; Tue, 23 May 2023 15:58:59 +0200 (CEST) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail0.khirnov.net (Postfix) with ESMTPS id A02B32406CB for ; Tue, 23 May 2023 15:58:48 +0200 (CEST) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:::1]) by libav.khirnov.net (Postfix) with ESMTP id 9296C3A031B for ; Tue, 23 May 2023 15:58:48 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Tue, 23 May 2023 15:58:38 +0200 Message-Id: <20230523135842.20388-11-anton@khirnov.net> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230523135842.20388-1-anton@khirnov.net> References: <20230523135842.20388-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 11/15] fftools/ffmpeg_dec: move timestamp estimation state to Decoder X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 3d6BwZLUxNI4 It is purely internal to decoding. --- fftools/ffmpeg.h | 10 ----- fftools/ffmpeg_dec.c | 87 +++++++++++++++++++++++++----------------- fftools/ffmpeg_demux.c | 5 --- 3 files changed, 52 insertions(+), 50 deletions(-) diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index b377871980..d9cac95710 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -352,16 +352,6 @@ typedef struct InputStream { AVRational framerate_guessed; - // pts/estimated duration of the last decoded frame - // * in decoder timebase for video, - // * in last_frame_tb (may change during decoding) for audio - int64_t last_frame_pts; - int64_t last_frame_duration_est; - AVRational last_frame_tb; - int last_frame_sample_rate; - - int64_t filter_in_rescale_delta_last; - int64_t nb_samples; /* number of samples in the last decoded audio frame before looping */ AVDictionary *decoder_opts; diff --git a/fftools/ffmpeg_dec.c b/fftools/ffmpeg_dec.c index 23c50cc272..c952473c16 100644 --- a/fftools/ffmpeg_dec.c +++ b/fftools/ffmpeg_dec.c @@ -34,6 +34,15 @@ struct Decoder { AVFrame *frame; AVPacket *pkt; + + // pts/estimated duration of the last decoded frame + // * in decoder timebase for video, + // * in last_frame_tb (may change during decoding) for audio + int64_t last_frame_pts; + int64_t last_frame_duration_est; + AVRational last_frame_tb; + int64_t last_filter_in_rescale_delta; + int last_frame_sample_rate; }; void dec_free(Decoder **pdec) @@ -67,6 +76,9 @@ static int dec_alloc(Decoder **pdec) if (!dec->pkt) goto fail; + dec->last_filter_in_rescale_delta = AV_NOPTS_VALUE; + dec->last_frame_pts = AV_NOPTS_VALUE; + dec->last_frame_tb = (AVRational){ 1, 1 }; *pdec = dec; @@ -94,21 +106,22 @@ static int send_frame_to_filters(InputStream *ist, AVFrame *decoded_frame) return ret; } -static AVRational audio_samplerate_update(InputStream *ist, const AVFrame *frame) +static AVRational audio_samplerate_update(void *logctx, Decoder *d, + const AVFrame *frame) { - const int prev = ist->last_frame_tb.den; + const int prev = d->last_frame_tb.den; const int sr = frame->sample_rate; AVRational tb_new; int64_t gcd; - if (frame->sample_rate == ist->last_frame_sample_rate) + if (frame->sample_rate == d->last_frame_sample_rate) goto finish; gcd = av_gcd(prev, sr); if (prev / gcd >= INT_MAX / sr) { - av_log(ist, AV_LOG_WARNING, + av_log(logctx, AV_LOG_WARNING, "Audio timestamps cannot be represented exactly after " "sample rate change: %d -> %d\n", prev, sr); @@ -123,20 +136,20 @@ static AVRational audio_samplerate_update(InputStream *ist, const AVFrame *frame !(frame->time_base.den % tb_new.den)) tb_new = frame->time_base; - if (ist->last_frame_pts != AV_NOPTS_VALUE) - ist->last_frame_pts = av_rescale_q(ist->last_frame_pts, - ist->last_frame_tb, tb_new); - ist->last_frame_duration_est = av_rescale_q(ist->last_frame_duration_est, - ist->last_frame_tb, tb_new); + if (d->last_frame_pts != AV_NOPTS_VALUE) + d->last_frame_pts = av_rescale_q(d->last_frame_pts, + d->last_frame_tb, tb_new); + d->last_frame_duration_est = av_rescale_q(d->last_frame_duration_est, + d->last_frame_tb, tb_new); - ist->last_frame_tb = tb_new; - ist->last_frame_sample_rate = frame->sample_rate; + d->last_frame_tb = tb_new; + d->last_frame_sample_rate = frame->sample_rate; finish: - return ist->last_frame_tb; + return d->last_frame_tb; } -static void audio_ts_process(InputStream *ist, AVFrame *frame) +static void audio_ts_process(void *logctx, Decoder *d, AVFrame *frame) { AVRational tb_filter = (AVRational){1, frame->sample_rate}; AVRational tb; @@ -145,27 +158,27 @@ static void audio_ts_process(InputStream *ist, AVFrame *frame) // on samplerate change, choose a new internal timebase for timestamp // generation that can represent timestamps from all the samplerates // seen so far - tb = audio_samplerate_update(ist, frame); - pts_pred = ist->last_frame_pts == AV_NOPTS_VALUE ? 0 : - ist->last_frame_pts + ist->last_frame_duration_est; + tb = audio_samplerate_update(logctx, d, frame); + pts_pred = d->last_frame_pts == AV_NOPTS_VALUE ? 0 : + d->last_frame_pts + d->last_frame_duration_est; if (frame->pts == AV_NOPTS_VALUE) { frame->pts = pts_pred; frame->time_base = tb; - } else if (ist->last_frame_pts != AV_NOPTS_VALUE && + } else if (d->last_frame_pts != AV_NOPTS_VALUE && frame->pts > av_rescale_q_rnd(pts_pred, tb, frame->time_base, AV_ROUND_UP)) { // there was a gap in timestamps, reset conversion state - ist->filter_in_rescale_delta_last = AV_NOPTS_VALUE; + d->last_filter_in_rescale_delta = AV_NOPTS_VALUE; } frame->pts = av_rescale_delta(frame->time_base, frame->pts, tb, frame->nb_samples, - &ist->filter_in_rescale_delta_last, tb); + &d->last_filter_in_rescale_delta, tb); - ist->last_frame_pts = frame->pts; - ist->last_frame_duration_est = av_rescale_q(frame->nb_samples, - tb_filter, tb); + d->last_frame_pts = frame->pts; + d->last_frame_duration_est = av_rescale_q(frame->nb_samples, + tb_filter, tb); // finally convert to filtering timebase frame->pts = av_rescale_q(frame->pts, tb, tb_filter); @@ -175,6 +188,7 @@ static void audio_ts_process(InputStream *ist, AVFrame *frame) static int64_t video_duration_estimate(const InputStream *ist, const AVFrame *frame) { + const Decoder *d = ist->decoder; const InputFile *ifile = input_files[ist->file_index]; int64_t codec_duration = 0; @@ -202,9 +216,9 @@ static int64_t video_duration_estimate(const InputStream *ist, const AVFrame *fr // when timestamps are available, repeat last frame's actual duration // (i.e. pts difference between this and last frame) - if (frame->pts != AV_NOPTS_VALUE && ist->last_frame_pts != AV_NOPTS_VALUE && - frame->pts > ist->last_frame_pts) - return frame->pts - ist->last_frame_pts; + if (frame->pts != AV_NOPTS_VALUE && d->last_frame_pts != AV_NOPTS_VALUE && + frame->pts > d->last_frame_pts) + return frame->pts - d->last_frame_pts; // try frame/codec duration if (frame->duration > 0) @@ -221,11 +235,13 @@ static int64_t video_duration_estimate(const InputStream *ist, const AVFrame *fr } // last resort is last frame's estimated duration, and 1 - return FFMAX(ist->last_frame_duration_est, 1); + return FFMAX(d->last_frame_duration_est, 1); } static int video_frame_process(InputStream *ist, AVFrame *frame) { + Decoder *d = ist->decoder; + // The following line may be required in some cases where there is no parser // or the parser does not has_b_frames correctly if (ist->par->video_delay < ist->dec_ctx->has_b_frames) { @@ -273,13 +289,13 @@ static int video_frame_process(InputStream *ist, AVFrame *frame) // no timestamp available - extrapolate from previous frame duration if (frame->pts == AV_NOPTS_VALUE) - frame->pts = ist->last_frame_pts == AV_NOPTS_VALUE ? 0 : - ist->last_frame_pts + ist->last_frame_duration_est; + frame->pts = d->last_frame_pts == AV_NOPTS_VALUE ? 0 : + d->last_frame_pts + d->last_frame_duration_est; // update timestamp history - ist->last_frame_duration_est = video_duration_estimate(ist, frame); - ist->last_frame_pts = frame->pts; - ist->last_frame_tb = frame->time_base; + d->last_frame_duration_est = video_duration_estimate(ist, frame); + d->last_frame_pts = frame->pts; + d->last_frame_tb = frame->time_base; if (debug_ts) { av_log(ist, AV_LOG_INFO, @@ -404,12 +420,13 @@ static int transcode_subtitles(InputStream *ist, const AVPacket *pkt) static int send_filter_eof(InputStream *ist) { + Decoder *d = ist->decoder; int i, ret; for (i = 0; i < ist->nb_filters; i++) { - int64_t end_pts = ist->last_frame_pts == AV_NOPTS_VALUE ? AV_NOPTS_VALUE : - ist->last_frame_pts + ist->last_frame_duration_est; - ret = ifilter_send_eof(ist->filters[i], end_pts, ist->last_frame_tb); + int64_t end_pts = d->last_frame_pts == AV_NOPTS_VALUE ? AV_NOPTS_VALUE : + d->last_frame_pts + d->last_frame_duration_est; + ret = ifilter_send_eof(ist->filters[i], end_pts, d->last_frame_tb); if (ret < 0) return ret; } @@ -506,7 +523,7 @@ int dec_packet(InputStream *ist, const AVPacket *pkt, int no_eof) ist->samples_decoded += frame->nb_samples; ist->nb_samples = frame->nb_samples; - audio_ts_process(ist, frame); + audio_ts_process(ist, ist->decoder, frame); } else { ret = video_frame_process(ist, frame); if (ret < 0) { diff --git a/fftools/ffmpeg_demux.c b/fftools/ffmpeg_demux.c index c65c72f556..e02bdc3b96 100644 --- a/fftools/ffmpeg_demux.c +++ b/fftools/ffmpeg_demux.c @@ -1181,11 +1181,6 @@ static void add_input_streams(const OptionsContext *o, Demuxer *d) exit_program(1); } - ist->filter_in_rescale_delta_last = AV_NOPTS_VALUE; - - ist->last_frame_pts = AV_NOPTS_VALUE; - ist->last_frame_tb = (AVRational){ 1, 1 }; - ist->dec_ctx = avcodec_alloc_context3(ist->dec); if (!ist->dec_ctx) report_and_exit(AVERROR(ENOMEM));