From patchwork Thu Jun 16 19:55:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 36264 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:1a22:b0:84:42e0:ad30 with SMTP id cj34csp1127875pzb; Thu, 16 Jun 2022 13:04:48 -0700 (PDT) X-Google-Smtp-Source: AGRyM1vC3PzP14FTao7ASPmzvZSHXvDcWrE8fnb9lcFuY+z//TH8Dbuig/+/Hq3CPouMNCks1be7 X-Received: by 2002:a05:6402:4003:b0:42d:fb4d:dada with SMTP id d3-20020a056402400300b0042dfb4ddadamr8528225eda.183.1655409888119; Thu, 16 Jun 2022 13:04:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1655409888; cv=none; d=google.com; s=arc-20160816; b=yJDoiHdCjqAGn0w5grli30hz40XlnbMSC7zte/PTr1nHLWcOdCOWi/s0ud6xOsm/8f BAFCjT9/qguwnVy/0YSN7ipsAC6IeSyNnrhefJDCR7Nm9vSQqas6f3CHeuc3NSlM23Wh qxgj/LkbKIidZfldu6oeX6zEcnyXEdveKLE2ySF6mouKIOtLnJ1MPw2DKEDVOUAwQgRj 5grQZcw8b8toSFmSwYGB3NHR8Ghb0JLGwW5QFAX9pGB7PR81hSnqJy3OyaNEEe8A/nhw BtzYiieigYDV5my4/sJG1Uig1vDITl90IiE26RYUIPfW2xj266ukujyMIfaPxTnDWe4r Qmvg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:delivered-to; bh=+wOVISu4SE4M7JH/fYlo0t5pmEtpIBADpW6mrY3yz7I=; b=kX9kGa07+0YSwyKu5KjXkbaU2fJMNDCiJ1bOEx3ZERgU3NE0OR+UxHEyfTqlM9aNoS EyKy07nkQdN0gMCo5JQSeU/uk9RNhH4U0+3tA3S8RXXxKkRGS19h6QxSIbDkyFoBQaMt b6VBUhAl47yKHZjnH4iqhbEA0+GLMLGCzSjiHb2NgVy8yWcanUsaHuBdvoLV9tyC+BiA QyghLro0YZtqqkJyC2HMqbFJDiBoaSRzSETw1EugQu30a1AtTIo0OVi7CsASfaf9Di1r NTgyVknrbTzjroucwofYlbo0/xRC7z02kPljJGv0CsK51zVJaywDmFf//6vYv3VGSGsG ut8Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id y3-20020aa7d503000000b0042dfcc43f17si2921492edq.274.2022.06.16.13.04.47; Thu, 16 Jun 2022 13:04:48 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E34C168B874; Thu, 16 Jun 2022 23:03:47 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail0.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 73EAE68B7F5 for ; Thu, 16 Jun 2022 23:03:42 +0300 (EEST) Received: from localhost (localhost [IPv6:::1]) by mail0.khirnov.net (Postfix) with ESMTP id 59C4624056A for ; Thu, 16 Jun 2022 22:03:40 +0200 (CEST) Received: from mail0.khirnov.net ([IPv6:::1]) by localhost (mail0.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id 0KWw1uCCLgp2 for ; Thu, 16 Jun 2022 22:03:39 +0200 (CEST) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail0.khirnov.net (Postfix) with ESMTPS id 28A84240592 for ; Thu, 16 Jun 2022 22:03:34 +0200 (CEST) Received: by libav.khirnov.net (Postfix, from userid 1000) id E44513A20FA; Thu, 16 Jun 2022 22:03:30 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Thu, 16 Jun 2022 21:55:23 +0200 Message-Id: <20220616195534.5278-24-anton@khirnov.net> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220616195534.5278-1-anton@khirnov.net> References: <20220616195534.5278-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 24/35] fftools/ffmpeg: use the sync queues to handle -frames X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: HHhme2BMC2RA Same issues apply to it as to -shortest. Changes the results of the following tests: - matroska-flac-extradata-update The test reencodes two input FLAC streams into three output FLAC streams. The last output stream is limited to 8 frames. The current code results in the first two output streams having 12 frames, after this commit all three streams have 8 frames and are the same length. This new result is better, since it is predictable. - mkv-1242 The test streamcopies one video and one audio stream, video is limited to 11 frames. The new result shortens the audio stream so that it is not longer than the video. --- fftools/ffmpeg.c | 6 ------ fftools/ffmpeg_mux.c | 10 +--------- fftools/ffmpeg_opt.c | 18 ++++++++++++++--- fftools/sync_queue.c | 20 +++++++++++++++++++ fftools/sync_queue.h | 7 +++++++ tests/ref/fate/matroska-flac-extradata-update | 16 +++++---------- tests/ref/fate/mkv-1242 | 3 --- 7 files changed, 48 insertions(+), 32 deletions(-) diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index 1a14637ece..a471fa0f8f 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -3467,12 +3467,6 @@ static int need_output(void) if (ost->finished || of_finished(of)) continue; - if (ost->frame_number >= ost->max_frames) { - int j; - for (j = 0; j < of->ctx->nb_streams; j++) - close_output_stream(output_streams[of->ost_index + j]); - continue; - } return 1; } diff --git a/fftools/ffmpeg_mux.c b/fftools/ffmpeg_mux.c index 641bdb98b0..56444770bf 100644 --- a/fftools/ffmpeg_mux.c +++ b/fftools/ffmpeg_mux.c @@ -237,19 +237,11 @@ void of_submit_packet(OutputFile *of, AVPacket *pkt, OutputStream *ost) if (pkt) { /* - * Audio encoders may split the packets -- #frames in != #packets out. - * But there is no reordering, so we can limit the number of output packets - * by simply dropping them here. * Counting encoded video frames needs to be done separately because of * reordering, see do_video_out(). */ - if (!(st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO && ost->encoding_needed)) { - if (ost->frame_number >= ost->max_frames) { - av_packet_unref(pkt); - return; - } + if (!(st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO && ost->encoding_needed)) ost->frame_number++; - } } if (of->mux->header_written) { diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c index 2cd4d42f2a..22eb558ff3 100644 --- a/fftools/ffmpeg_opt.c +++ b/fftools/ffmpeg_opt.c @@ -2331,6 +2331,7 @@ static int init_complex_filters(void) static int setup_sync_queues(OutputFile *of, AVFormatContext *oc, int64_t buf_size_us) { int nb_av_enc = 0, nb_interleaved = 0; + int limit_frames = 0, limit_frames_av_enc = 0; #define IS_AV_ENC(ost, type) \ (ost->encoding_needed && (type == AVMEDIA_TYPE_VIDEO || type == AVMEDIA_TYPE_AUDIO)) @@ -2345,14 +2346,19 @@ static int setup_sync_queues(OutputFile *of, AVFormatContext *oc, int64_t buf_si nb_interleaved += IS_INTERLEAVED(type); nb_av_enc += IS_AV_ENC(ost, type); + + limit_frames |= ost->max_frames < INT64_MAX; + limit_frames_av_enc |= (ost->max_frames < INT64_MAX) && IS_AV_ENC(ost, type); } - if (!(nb_interleaved > 1 && of->shortest)) + if (!((nb_interleaved > 1 && of->shortest) || + (nb_interleaved > 0 && limit_frames))) return 0; - /* if we have more than one encoded audio/video streams, then we + /* if we have more than one encoded audio/video streams, or at least + * one encoded audio/video stream is frame-limited, then we * synchronize them before encoding */ - if (nb_av_enc > 1) { + if ((of->shortest && nb_av_enc > 1) || limit_frames_av_enc) { of->sq_encode = sq_alloc(SYNC_QUEUE_FRAMES, buf_size_us); if (!of->sq_encode) return AVERROR(ENOMEM); @@ -2371,6 +2377,9 @@ static int setup_sync_queues(OutputFile *of, AVFormatContext *oc, int64_t buf_si ost->sq_frame = av_frame_alloc(); if (!ost->sq_frame) return AVERROR(ENOMEM); + + if (ost->max_frames != INT64_MAX) + sq_limit_frames(of->sq_encode, ost->sq_idx_encode, ost->max_frames); } } @@ -2391,6 +2400,9 @@ static int setup_sync_queues(OutputFile *of, AVFormatContext *oc, int64_t buf_si ost->sq_idx_mux = sq_add_stream(of->sq_mux); if (ost->sq_idx_mux < 0) return ost->sq_idx_mux; + + if (ost->max_frames != INT64_MAX) + sq_limit_frames(of->sq_mux, ost->sq_idx_mux, ost->max_frames); } } diff --git a/fftools/sync_queue.c b/fftools/sync_queue.c index 49f0600b6c..08c3cf4f2a 100644 --- a/fftools/sync_queue.c +++ b/fftools/sync_queue.c @@ -36,6 +36,9 @@ typedef struct SyncQueueStream { int64_t head_ts; /* no more frames will be sent for this stream */ int finished; + + uint64_t frames_sent; + uint64_t frames_max; } SyncQueueStream; struct SyncQueue { @@ -264,6 +267,10 @@ int sq_send(SyncQueue *sq, unsigned int stream_idx, SyncQueueFrame frame) stream_update_ts(sq, stream_idx, ts); + st->frames_sent++; + if (st->frames_sent >= st->frames_max) + finish_stream(sq, stream_idx); + return 0; } @@ -362,6 +369,7 @@ int sq_add_stream(SyncQueue *sq) * streams forever; cf. overflow_heartbeat() */ st->tb = (AVRational){ 1, 1 }; st->head_ts = AV_NOPTS_VALUE; + st->frames_max = UINT64_MAX; return sq->nb_streams++; } @@ -381,6 +389,18 @@ void sq_set_tb(SyncQueue *sq, unsigned int stream_idx, AVRational tb) st->tb = tb; } +void sq_limit_frames(SyncQueue *sq, unsigned int stream_idx, uint64_t frames) +{ + SyncQueueStream *st; + + av_assert0(stream_idx < sq->nb_streams); + st = &sq->streams[stream_idx]; + + st->frames_max = frames; + if (st->frames_sent >= st->frames_max) + finish_stream(sq, stream_idx); +} + SyncQueue *sq_alloc(enum SyncQueueType type, int64_t buf_size_us) { SyncQueue *sq = av_mallocz(sizeof(*sq)); diff --git a/fftools/sync_queue.h b/fftools/sync_queue.h index e08780b7bf..7beddf8c4a 100644 --- a/fftools/sync_queue.h +++ b/fftools/sync_queue.h @@ -63,6 +63,13 @@ int sq_add_stream(SyncQueue *sq); */ void sq_set_tb(SyncQueue *sq, unsigned int stream_idx, AVRational tb); +/** + * Limit the number of output frames for stream with index stream_idx + * to max_frames. + */ +void sq_limit_frames(SyncQueue *sq, unsigned int stream_idx, + uint64_t max_frames); + /** * Submit a frame for the stream with index stream_idx. * diff --git a/tests/ref/fate/matroska-flac-extradata-update b/tests/ref/fate/matroska-flac-extradata-update index b0276f734d..84dcc9c1d0 100644 --- a/tests/ref/fate/matroska-flac-extradata-update +++ b/tests/ref/fate/matroska-flac-extradata-update @@ -1,8 +1,8 @@ -56ff5763fd81ad3bc02c22402cd685e2 *tests/data/fate/matroska-flac-extradata-update.matroska -2008 tests/data/fate/matroska-flac-extradata-update.matroska -#extradata 0: 34, 0x7acb09e7 -#extradata 1: 34, 0x7acb09e7 -#extradata 2: 34, 0x443402dd +8ec02dffd603f44e08b2ae3b81a0d5a0 *tests/data/fate/matroska-flac-extradata-update.matroska +1816 tests/data/fate/matroska-flac-extradata-update.matroska +#extradata 0: 34, 0x93650c81 +#extradata 1: 34, 0x93650c81 +#extradata 2: 34, 0x93650c81 #tb 0: 1/1000 #media_type 0: audio #codec_id 0: flac @@ -42,9 +42,3 @@ 0, 672, 672, 96, 26, 0x50dd042e 1, 672, 672, 96, 26, 0x50dd042e 2, 672, 672, 96, 26, 0x50dd042e -0, 768, 768, 96, 26, 0x53de0499 -1, 768, 768, 96, 26, 0x53de0499 -0, 864, 864, 96, 26, 0x53df04b4 -1, 864, 864, 96, 26, 0x53df04b4 -0, 960, 960, 42, 26, 0x5740044b -1, 960, 960, 42, 26, 0x5740044b diff --git a/tests/ref/fate/mkv-1242 b/tests/ref/fate/mkv-1242 index e025701093..1d1a227832 100644 --- a/tests/ref/fate/mkv-1242 +++ b/tests/ref/fate/mkv-1242 @@ -42,6 +42,3 @@ 1, 383, 383, 21, 325, 0xcd7a9fd6 1, 404, 404, 22, 359, 0x6edeb91c 1, 426, 426, 21, 333, 0xb8999fb7 -1, 447, 447, 21, 317, 0xf2589e1a -1, 468, 468, 21, 319, 0x82ed9572 -1, 489, 489, 22, 473, 0xea54e696