From patchwork Mon Apr 4 11:30:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 35185 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:c05:b0:7a:e998:b410 with SMTP id bw5csp1914357pzb; Mon, 4 Apr 2022 04:37:49 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxoc6tn0hXcWjKZHjjcfN8qTkIKP5Kj59KWta12fNLUiW8Ysv1SVvCCCt+JOxRWIkhvyjhp X-Received: by 2002:aa7:c3d6:0:b0:419:2370:4856 with SMTP id l22-20020aa7c3d6000000b0041923704856mr32725308edr.180.1649072269328; Mon, 04 Apr 2022 04:37:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1649072269; cv=none; d=google.com; s=arc-20160816; b=n9UvOuHYDd7hvu2SnndKpTK3uu5+wvQOU3NqW/xkVoTOhRL63GLq9YM5+pA08IspVk 33vWab1v/wyRk8iEDSyDm3kqjtCBQgmJdfX6rELniWB2Q4dP3ZXAWiqxMOZGYU7luzgc ljTjhKzCr4XF/95ByGVGA4FzvVrUavF1XBOuhJtqqwzzu9DEbwMWxybvzUhtN574Ve+9 DwWpQeaJIZQeHqIKebLdamQ+86KQaB3BhXoRwIm28nPYIdVAPBT3W59OyMlmNGeaWKuv yrVaJ+XtlfC6/wB0bnpEEVsfpk5xZoqZabD+OBCqnWHSQfAbpTDMsNm5r1dmKnc6EIgI 7xkg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:delivered-to; bh=Hr56+eXWqrRq3+qeYhUWa/AhQ0rTWr3g06PqZQ20tzo=; b=AN1uA/ioT3uOswgVU/1ukdBSpaLcnGCFPtZQDKTovzDdkaDuIxAFh+qwe9tOfNJUtE 17tzCD0jGX6JjOHvNTWfpU9bKv9cgRGsdb9855TPyeqzVOa4Pg/rZ63b6wXPuP0D23UV EsBwmfQpyKNQ1gQibQAl5QQOLO0I+FcQDpjCcC9UNRr+jw8d/pbKY9Z69dV9VFTYXEQ6 mp/FfIhLO2qN3kAuHYcdVy6FiDYnyq8SSW8ZcOOc8gKi87F0Jl0CztHQu4rw+CVeZWKC nwrmanwr18lUlBSJlSxTwPq4WAhVMksbxJQivYgbav7jAPoTzDiYG8SNmfPA0dDeEApZ xZkQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id u26-20020a17090657da00b006df7b230bfcsi6132879ejr.182.2022.04.04.04.37.49; Mon, 04 Apr 2022 04:37:49 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A623468B2B5; Mon, 4 Apr 2022 14:37:43 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail0.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5145768B29B for ; Mon, 4 Apr 2022 14:37:42 +0300 (EEST) Received: from localhost (localhost [IPv6:::1]) by mail0.khirnov.net (Postfix) with ESMTP id 0572F240593 for ; Mon, 4 Apr 2022 13:37:42 +0200 (CEST) Received: from mail0.khirnov.net ([IPv6:::1]) by localhost (mail0.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id JIpiHCnkpGnz for ; Mon, 4 Apr 2022 13:37:41 +0200 (CEST) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail0.khirnov.net (Postfix) with ESMTPS id 33FDF2400F5 for ; Mon, 4 Apr 2022 13:37:41 +0200 (CEST) Received: by libav.khirnov.net (Postfix, from userid 1000) id 245C53A0D98; Mon, 4 Apr 2022 13:32:12 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Mon, 4 Apr 2022 13:30:23 +0200 Message-Id: <20220404113037.13070-36-anton@khirnov.net> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220404113037.13070-1-anton@khirnov.net> References: <20220404113037.13070-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 35/49] fftools/ffmpeg: use the sync queues to handle -frames X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 3ydhTen6SHJh Same issues apply to it as to -shortest. Changes the results of the following tests: - matroska-flac-extradata-update The test reencodes two input FLAC streams into three output FLAC streams. The last output stream is limited to 8 frames. The current code results in the first two output streams having 12 frames, after this commit all three streams have 8 frames and are the same length. This new result is better, since it is predictable. - mkv-1242 The test streamcopies one video and one audio stream, video is limited to 11 frames. The new result shortens the audio stream so that it is not longer than the video. --- fftools/ffmpeg.c | 6 ------ fftools/ffmpeg_mux.c | 10 +--------- fftools/ffmpeg_opt.c | 18 +++++++++++++++--- fftools/sync_queue.c | 13 +++++++++++++ fftools/sync_queue.h | 7 +++++++ tests/ref/fate/matroska-flac-extradata-update | 16 +++++----------- tests/ref/fate/mkv-1242 | 3 --- 7 files changed, 41 insertions(+), 32 deletions(-) diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index 8674f52047..e46554ddf4 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -3412,12 +3412,6 @@ static int need_output(void) if (ost->finished || of_finished(of)) continue; - if (ost->frame_number >= ost->max_frames) { - int j; - for (j = 0; j < of->ctx->nb_streams; j++) - close_output_stream(output_streams[of->ost_index + j]); - continue; - } return 1; } diff --git a/fftools/ffmpeg_mux.c b/fftools/ffmpeg_mux.c index 69f01448b8..11d5a0a885 100644 --- a/fftools/ffmpeg_mux.c +++ b/fftools/ffmpeg_mux.c @@ -273,19 +273,11 @@ void of_submit_packet(OutputFile *of, AVPacket *pkt, OutputStream *ost, int eof) if (!eof) { /* - * Audio encoders may split the packets -- #frames in != #packets out. - * But there is no reordering, so we can limit the number of output packets - * by simply dropping them here. * Counting encoded video frames needs to be done separately because of * reordering, see do_video_out(). */ - if (!(st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO && ost->encoding_needed)) { - if (ost->frame_number >= ost->max_frames) { - av_packet_unref(pkt); - return; - } + if (!(st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO && ost->encoding_needed)) ost->frame_number++; - } } if (ost->sq_idx_mux >= 0) { diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c index 8884a5f9ed..ac0b4fc7d8 100644 --- a/fftools/ffmpeg_opt.c +++ b/fftools/ffmpeg_opt.c @@ -2304,6 +2304,7 @@ static int init_complex_filters(void) static int setup_sync_queues(OutputFile *of, AVFormatContext *oc) { int nb_av_enc = 0, nb_interleaved = 0; + int limit_frames = 0, limit_frames_av_enc = 0; #define IS_AV_ENC(ost, type) \ (ost->encoding_needed && (type == AVMEDIA_TYPE_VIDEO || type == AVMEDIA_TYPE_AUDIO)) @@ -2318,14 +2319,19 @@ static int setup_sync_queues(OutputFile *of, AVFormatContext *oc) nb_interleaved += IS_INTERLEAVED(type); nb_av_enc += IS_AV_ENC(ost, type); + + limit_frames |= ost->max_frames < INT64_MAX; + limit_frames_av_enc |= (ost->max_frames < INT64_MAX) && IS_AV_ENC(ost, type); } - if (!(nb_interleaved > 1 && of->shortest)) + if (!((nb_interleaved > 1 && of->shortest) || + (nb_interleaved > 0 && limit_frames))) return 0; - /* if we have more than one encoded audio/video streams, then we + /* if we have more than one encoded audio/video streams, or at least + * one encoded audio/video stream is frame-limited, then we * synchronize them before encoding */ - if (nb_av_enc > 1) { + if ((of->shortest && nb_av_enc > 1) || limit_frames_av_enc) { of->sq_encode = sq_alloc(SYNC_QUEUE_FRAMES); if (!of->sq_encode) return AVERROR(ENOMEM); @@ -2344,6 +2350,9 @@ static int setup_sync_queues(OutputFile *of, AVFormatContext *oc) ost->sq_frame = av_frame_alloc(); if (!ost->sq_frame) return AVERROR(ENOMEM); + + if (ost->max_frames != INT64_MAX) + sq_limit_frames(of->sq_encode, ost->sq_idx_encode, ost->max_frames); } } @@ -2364,6 +2373,9 @@ static int setup_sync_queues(OutputFile *of, AVFormatContext *oc) ost->sq_idx_mux = sq_add_stream(of->sq_mux); if (ost->sq_idx_mux < 0) return ost->sq_idx_mux; + + if (ost->max_frames != INT64_MAX) + sq_limit_frames(of->sq_mux, ost->sq_idx_mux, ost->max_frames); } } diff --git a/fftools/sync_queue.c b/fftools/sync_queue.c index cdf6a32cae..8bfdcbab4d 100644 --- a/fftools/sync_queue.c +++ b/fftools/sync_queue.c @@ -32,6 +32,9 @@ typedef struct SyncQueueStream { AVRational tb; int64_t head_ts; int finished; + + uint64_t frames_sent; + uint64_t frames_max; } SyncQueueStream; struct SyncQueue { @@ -205,6 +208,9 @@ int sq_send(SyncQueue *sq, unsigned int stream_idx, SyncQueueFrame frame) } } + st->frames_sent++; + if (st->frames_sent >= st->frames_max) + finish_stream(sq, stream_idx); return 0; } @@ -297,6 +303,7 @@ int sq_add_stream(SyncQueue *sq) return AVERROR(ENOMEM); st->head_ts = AV_NOPTS_VALUE; + st->frames_max = UINT64_MAX; return sq->nb_streams++; } @@ -307,6 +314,12 @@ void sq_set_tb(SyncQueue *sq, unsigned int stream_idx, AVRational tb) sq->streams[stream_idx].tb = tb; } +void sq_limit_frames(SyncQueue *sq, unsigned int stream_idx, uint64_t frames) +{ + av_assert0(stream_idx < sq->nb_streams); + sq->streams[stream_idx].frames_max = frames; +} + SyncQueue *sq_alloc(enum SyncQueueType type) { SyncQueue *sq = av_mallocz(sizeof(*sq)); diff --git a/fftools/sync_queue.h b/fftools/sync_queue.h index 4a16ae3e62..9afd067c25 100644 --- a/fftools/sync_queue.h +++ b/fftools/sync_queue.h @@ -56,6 +56,13 @@ int sq_add_stream(SyncQueue *sq); */ void sq_set_tb(SyncQueue *sq, unsigned int stream_idx, AVRational tb); +/** + * Limit the number of output frames for stream with index stream_idx + * to max_frames. + */ +void sq_limit_frames(SyncQueue *sq, unsigned int stream_idx, + uint64_t max_frames); + /** * Submit a frame for the stream with index stream_idx. * diff --git a/tests/ref/fate/matroska-flac-extradata-update b/tests/ref/fate/matroska-flac-extradata-update index b0276f734d..84dcc9c1d0 100644 --- a/tests/ref/fate/matroska-flac-extradata-update +++ b/tests/ref/fate/matroska-flac-extradata-update @@ -1,8 +1,8 @@ -56ff5763fd81ad3bc02c22402cd685e2 *tests/data/fate/matroska-flac-extradata-update.matroska -2008 tests/data/fate/matroska-flac-extradata-update.matroska -#extradata 0: 34, 0x7acb09e7 -#extradata 1: 34, 0x7acb09e7 -#extradata 2: 34, 0x443402dd +8ec02dffd603f44e08b2ae3b81a0d5a0 *tests/data/fate/matroska-flac-extradata-update.matroska +1816 tests/data/fate/matroska-flac-extradata-update.matroska +#extradata 0: 34, 0x93650c81 +#extradata 1: 34, 0x93650c81 +#extradata 2: 34, 0x93650c81 #tb 0: 1/1000 #media_type 0: audio #codec_id 0: flac @@ -42,9 +42,3 @@ 0, 672, 672, 96, 26, 0x50dd042e 1, 672, 672, 96, 26, 0x50dd042e 2, 672, 672, 96, 26, 0x50dd042e -0, 768, 768, 96, 26, 0x53de0499 -1, 768, 768, 96, 26, 0x53de0499 -0, 864, 864, 96, 26, 0x53df04b4 -1, 864, 864, 96, 26, 0x53df04b4 -0, 960, 960, 42, 26, 0x5740044b -1, 960, 960, 42, 26, 0x5740044b diff --git a/tests/ref/fate/mkv-1242 b/tests/ref/fate/mkv-1242 index e025701093..1d1a227832 100644 --- a/tests/ref/fate/mkv-1242 +++ b/tests/ref/fate/mkv-1242 @@ -42,6 +42,3 @@ 1, 383, 383, 21, 325, 0xcd7a9fd6 1, 404, 404, 22, 359, 0x6edeb91c 1, 426, 426, 21, 333, 0xb8999fb7 -1, 447, 447, 21, 317, 0xf2589e1a -1, 468, 468, 21, 319, 0x82ed9572 -1, 489, 489, 22, 473, 0xea54e696