From patchwork Sun Aug 27 10:50:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 43360 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:7194:b0:149:dfde:5c0a with SMTP id s20csp802300pzb; Sun, 27 Aug 2023 03:51:37 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGgugLnG1p/pjM9F2tG2AwrA2ZoL0CAOzaHOdj4p7FsEc5scqWhnO0NETkCWvGhTNRvvgNY X-Received: by 2002:a17:906:10db:b0:9a2:28dc:4166 with SMTP id v27-20020a17090610db00b009a228dc4166mr6450679ejv.75.1693133496963; Sun, 27 Aug 2023 03:51:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1693133496; cv=none; d=google.com; s=arc-20160816; b=KDGPJixsgXd5oiJUo4ssqhjtpY0nKjA+mRAf3rVXCGnzdhxIFYWKPJOyKhKtlibqvz LXJxRehHNRRyUNEoBUSbKU9GSkJMmSbGMFXqbAF5pW0Y1TvbTJuHIcGgq+GbKX7ZAOhy HhqA+0rNi3lTYWLSbTmQibm/eGaZHpRAHsBP6RWuDDE/7njLPO56ihWPTuhVwFiBdCLI /7Na92JvihrmaRIMHLtjIofKUs9C5c9fK9Mqk42BZvMMFd4QQ5/0z0ebV2pq96MTh51A IT9pGxwqCkX/n1GFZYVsOmnM0hl0UgMZ36fyYxFH+ApngSj2HjGQPek7iS2TVDameVY7 eXhQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:delivered-to; bh=woljRFhILcVQPjMx9a4g51EOBlmbC5eBsuS0zVnmg1I=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=A7NzfXAiR66D3+7RWyTsSiWmxRk4TLwl8/Sz8m7xLJJTAdLwl4aFUyAiv+hIiF2juv pLjK2HrK4YzKV724U+zYdQGX0FoWoZc3CD9ilpFoRoIRqYz/ZHqybGL0g95yaV1BeQ/F wjMLfAO1Ni1LJE2wQa3tQQpiXDFd5WsE2xLnTXyptPuFKmMoe9lZGQDjYtPRjznB/cId iAivlKt6bhPH2crGi7Y5xBgYsPb7eNs0V1zcCAjINBBYM7GoPeuFMUBVJuWYqcMJajUF EHEwgLVQt3RNIYqbRxPBVDqHT4XbEzqf75db1zjlEACu8Af8MB7eDTnNWe+f+n6ttewf p4/w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id cd11-20020a170906b34b00b0099bd0f172b0si3122767ejb.1036.2023.08.27.03.51.33; Sun, 27 Aug 2023 03:51:36 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E56A168C5C7; Sun, 27 Aug 2023 13:51:28 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail0.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A5CF468AFA8 for ; Sun, 27 Aug 2023 13:51:21 +0300 (EEST) Received: from localhost (localhost [IPv6:::1]) by mail0.khirnov.net (Postfix) with ESMTP id 0C10424018E for ; Sun, 27 Aug 2023 12:51:21 +0200 (CEST) Received: from mail0.khirnov.net ([IPv6:::1]) by localhost (mail0.khirnov.net [IPv6:::1]) (amavis, port 10024) with ESMTP id tf5bItyl-HJa for ; Sun, 27 Aug 2023 12:51:18 +0200 (CEST) Received: from mail1.khirnov.net (mail1.khirnov.net [IPv6:2a00:c500:561:206::5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mail1.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail0.khirnov.net (Postfix) with ESMTPS id B60E2240177 for ; Sun, 27 Aug 2023 12:51:18 +0200 (CEST) Received: from localhost (mail1.khirnov.net [IPv6:::1]) by mail1.khirnov.net (Postfix) with ESMTP id DDB84410E for ; Sun, 27 Aug 2023 12:53:22 +0200 (CEST) Received: from mail1.khirnov.net ([IPv6:::1]) by localhost (mail1.khirnov.net [IPv6:::1]) (amavis, port 10024) with ESMTP id 3d4OrcXwGbYi for ; Sun, 27 Aug 2023 12:53:18 +0200 (CEST) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail1.khirnov.net (Postfix) with ESMTPS id 322C0120B for ; Sun, 27 Aug 2023 12:53:17 +0200 (CEST) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:::1]) by libav.khirnov.net (Postfix) with ESMTP id 5F6DD3A022D for ; Sun, 27 Aug 2023 12:51:13 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Sun, 27 Aug 2023 12:50:32 +0200 Message-Id: <20230827105032.22186-1-anton@khirnov.net> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230826202027.GU7802@pb2> References: <20230826202027.GU7802@pb2> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 04/18 v2] fftools/ffmpeg_mux: stop rescaling timestamps in of_streamcopy() X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: JkpQSYBTFpcE This function converts packet timestamps from the input stream timebase to OutputStream.mux_timebase, which may or may not be equal to the actual output AVStream timebase (and even when it is, this may not always be the optimal choice due to bitstream filtering). Just keep the timestamps in input stream timebase, they will be rescaled as needed before bitstream filtering and/or sending the packet to the muxer. Move the av_rescale_delta() call for audio (needed to preserve accuracy with coarse muxer timebases) to write_packet. Drop now-unused OutputStream.mux_timebase. --- fftools/ffmpeg.h | 2 -- fftools/ffmpeg_enc.c | 2 -- fftools/ffmpeg_mux.c | 44 ++++++++++++++++++++------------------- fftools/ffmpeg_mux.h | 2 +- fftools/ffmpeg_mux_init.c | 2 -- 5 files changed, 24 insertions(+), 28 deletions(-) diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index d53181e427..ef5bb13908 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -523,8 +523,6 @@ typedef struct OutputStream { /* dts of the last packet sent to the muxing queue, in AV_TIME_BASE_Q */ int64_t last_mux_dts; - // the timebase of the packets sent to the muxer - AVRational mux_timebase; AVRational enc_timebase; Encoder *enc; diff --git a/fftools/ffmpeg_enc.c b/fftools/ffmpeg_enc.c index 96424272bf..3c9fdd3237 100644 --- a/fftools/ffmpeg_enc.c +++ b/fftools/ffmpeg_enc.c @@ -463,8 +463,6 @@ int enc_open(OutputStream *ost, AVFrame *frame) if (ost->st->time_base.num <= 0 || ost->st->time_base.den <= 0) ost->st->time_base = av_add_q(ost->enc_ctx->time_base, (AVRational){0, 1}); - ost->mux_timebase = enc_ctx->time_base; - ret = of_stream_init(of, ost); if (ret < 0) return ret; diff --git a/fftools/ffmpeg_mux.c b/fftools/ffmpeg_mux.c index ab9bb398c1..54410aac5c 100644 --- a/fftools/ffmpeg_mux.c +++ b/fftools/ffmpeg_mux.c @@ -76,7 +76,23 @@ static int write_packet(Muxer *mux, OutputStream *ost, AVPacket *pkt) if (ost->type == AVMEDIA_TYPE_VIDEO && ost->vsync_method == VSYNC_DROP) pkt->pts = pkt->dts = AV_NOPTS_VALUE; - av_packet_rescale_ts(pkt, pkt->time_base, ost->st->time_base); + // rescale timestamps to the stream timebase + if (ost->type == AVMEDIA_TYPE_AUDIO && !ost->enc) { + // use av_rescale_delta() for streamcopying audio, to preserve + // accuracy with coarse input timebases + int duration = av_get_audio_frame_duration2(ost->st->codecpar, pkt->size); + + if (!duration) + duration = ost->st->codecpar->frame_size; + + pkt->dts = av_rescale_delta(pkt->time_base, pkt->dts, + (AVRational){1, ost->st->codecpar->sample_rate}, duration, + &ms->ts_rescale_delta_last, ost->st->time_base); + pkt->pts = pkt->dts; + + pkt->duration = av_rescale_q(pkt->duration, pkt->time_base, ost->st->time_base); + } else + av_packet_rescale_ts(pkt, pkt->time_base, ost->st->time_base); pkt->time_base = ost->st->time_base; if (!(s->oformat->flags & AVFMT_NOTIMESTAMPS)) { @@ -392,7 +408,7 @@ int of_streamcopy(OutputStream *ost, const AVPacket *pkt, int64_t dts) OutputFile *of = output_files[ost->file_index]; MuxStream *ms = ms_from_ost(ost); int64_t start_time = (of->start_time == AV_NOPTS_VALUE) ? 0 : of->start_time; - int64_t ost_tb_start_time = av_rescale_q(start_time, AV_TIME_BASE_Q, ost->mux_timebase); + int64_t ts_offset; AVPacket *opkt = ms->pkt; int ret; @@ -425,27 +441,17 @@ int of_streamcopy(OutputStream *ost, const AVPacket *pkt, int64_t dts) if (ret < 0) return ret; - opkt->time_base = ost->mux_timebase; + ts_offset = av_rescale_q(start_time, AV_TIME_BASE_Q, opkt->time_base); if (pkt->pts != AV_NOPTS_VALUE) - opkt->pts = av_rescale_q(pkt->pts, pkt->time_base, opkt->time_base) - ost_tb_start_time; + opkt->pts -= ts_offset; if (pkt->dts == AV_NOPTS_VALUE) { opkt->dts = av_rescale_q(dts, AV_TIME_BASE_Q, opkt->time_base); } else if (ost->st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO) { - int duration = av_get_audio_frame_duration2(ost->par_in, pkt->size); - if(!duration) - duration = ost->par_in->frame_size; - opkt->dts = av_rescale_delta(pkt->time_base, pkt->dts, - (AVRational){1, ost->par_in->sample_rate}, duration, - &ms->ts_rescale_delta_last, opkt->time_base); - /* dts will be set immediately afterwards to what pts is now */ - opkt->pts = opkt->dts - ost_tb_start_time; - } else - opkt->dts = av_rescale_q(pkt->dts, pkt->time_base, opkt->time_base); - opkt->dts -= ost_tb_start_time; - - opkt->duration = av_rescale_q(pkt->duration, pkt->time_base, opkt->time_base); + opkt->pts = opkt->dts - ts_offset; + } + opkt->dts -= ts_offset; { int ret = trigger_fix_sub_duration_heartbeat(ost, pkt); @@ -511,10 +517,6 @@ static int thread_start(Muxer *mux) MuxStream *ms = ms_from_ost(ost); AVPacket *pkt; - /* try to improve muxing time_base (only possible if nothing has been written yet) */ - if (!av_fifo_can_read(ms->muxing_queue)) - ost->mux_timebase = ost->st->time_base; - while (av_fifo_read(ms->muxing_queue, &pkt, 1) >= 0) { ret = thread_submit_packet(mux, ost, pkt); if (pkt) { diff --git a/fftools/ffmpeg_mux.h b/fftools/ffmpeg_mux.h index 7f34b86548..a2bb4dfc7d 100644 --- a/fftools/ffmpeg_mux.h +++ b/fftools/ffmpeg_mux.h @@ -75,7 +75,7 @@ typedef struct MuxStream { int64_t stream_duration; AVRational stream_duration_tb; - // audio streamcopy - state for av_rescale_delta() + // state for av_rescale_delta() call for audio in write_packet() int64_t ts_rescale_delta_last; // combined size of all the packets sent to the muxer diff --git a/fftools/ffmpeg_mux_init.c b/fftools/ffmpeg_mux_init.c index 0289cdabad..cf4cd2d5b7 100644 --- a/fftools/ffmpeg_mux_init.c +++ b/fftools/ffmpeg_mux_init.c @@ -1090,8 +1090,6 @@ static int streamcopy_init(const Muxer *mux, OutputStream *ost) } } - ost->mux_timebase = ist->st->time_base; - fail: avcodec_free_context(&codec_ctx); av_dict_free(&codec_opts);