From patchwork Fri Jul 7 09:48:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 42482 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3b1e:b0:12b:9ae3:586d with SMTP id c30csp7162138pzh; Fri, 7 Jul 2023 02:50:24 -0700 (PDT) X-Google-Smtp-Source: APBJJlHpC9TvVMGsWJCK8r3DseRg85YlvXD8rX02+vRYHesY/0y0sY+gh4Zfgm4l8H8FkHcnNzGo X-Received: by 2002:aa7:c75a:0:b0:51d:9521:8084 with SMTP id c26-20020aa7c75a000000b0051d95218084mr3581346eds.2.1688723423936; Fri, 07 Jul 2023 02:50:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1688723423; cv=none; d=google.com; s=arc-20160816; b=dV7RiIwWyEdvAGL40D5fSRAqSwW5ovmPV1em0iGfypMYqQQHo8FrNefe39/KbEzB0B CRNjiDRQuALJafi8xLfFH297HRh0n/57G+jNKJtWUq28OoBLz/sOhPd2jQwCh+Zzl/MM rRMI2K/nbIlP+AGkGMwMY/BjDL83U+IyXbn52UZ0LguUHlsvCQJzGwNifw4VwxinNtNY d7o97E4havvv6vaqvv6vK9QHZ/aKMdAeLngX9ySFQRjEavlTFTH5RzxDcIfospKAjXeA SM1LEWFhkE2njD+Pr2vp6dPJ3LKZB3QWPZllI+CGV6tePQsuv31Mqxl6B4/G1csYUwxB ljNw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:delivered-to; bh=5412fJS6ykHGOee9p3BLmv1iEnaDVQbHqfu6JBmLqg8=; fh=8ZF2QtiKOjBtyn9wuO2cMeZaLsE4aq02Bq77gGK61E8=; b=ng57KtnW8rdbPf4oN4D3n0676Ebx+fl+LcjMguOtJnbF84sOEhRQ/dK0nN8ypPHdXO THr5ALWDKO+PJFSEnYg7Ntp4YRlWTYAZfKZcCeFEiJYIjT74vmZw9Bf3enx12ihOXhSr Cd6wQKg0FZ7W4QgYEGdNEL06GpcsOWIJqcWM6Snlb5gxJ9vpz4qyhvUiX1C35MqGFoZ7 rCGIxGtDvJMYnM0JEKyYtUxHEdTnhtD+W3pUpmQzO6QCmlpBUnUItkTMaRKEy03AGcBl 7lOPoW+6qAPnagiGHxaFXeSPfUTZEMyQYMmH9EWkN1uD4NmZXSs1nE3cxZyHyglruHE4 pE5w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id w5-20020aa7d285000000b0051e23edcddcsi853019edq.122.2023.07.07.02.50.23; Fri, 07 Jul 2023 02:50:23 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0E00568C7A9; Fri, 7 Jul 2023 12:49:17 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail0.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9039D68C7BE for ; Fri, 7 Jul 2023 12:49:08 +0300 (EEST) Received: from localhost (localhost [IPv6:::1]) by mail0.khirnov.net (Postfix) with ESMTP id 21A042406CF for ; Fri, 7 Jul 2023 11:49:07 +0200 (CEST) Received: from mail0.khirnov.net ([IPv6:::1]) by localhost (mail0.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id x1rXoAtVDUAH for ; Fri, 7 Jul 2023 11:49:04 +0200 (CEST) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail0.khirnov.net (Postfix) with ESMTPS id EDF712406CB for ; Fri, 7 Jul 2023 11:49:00 +0200 (CEST) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:::1]) by libav.khirnov.net (Postfix) with ESMTP id E9C7B3A1829 for ; Fri, 7 Jul 2023 11:48:54 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Fri, 7 Jul 2023 11:48:41 +0200 Message-Id: <20230707094847.25324-16-anton@khirnov.net> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230707094847.25324-1-anton@khirnov.net> References: <20230707094847.25324-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 16/22] fftools/ffmpeg: rework initializing encoders with no frames X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: e0EIC5niE3Mw When no frames were passed from a filtergraph to an encoder, but the filtergraph is configured (i.e. has output parameters), encoder flush code will use those parameters to initialize the encoder in a last-ditch effort to produce some useful output. Rework this process so that it is triggered by the filtergraph, which now sends a dummy frame with parameters, but no data, to the encoder, rather than the encoder reaching backwards into the filter. This approach is more in line with the natural data flow from filters to encoders and will allow to reduce encoder-filter interactions in following commits. This code is tested by fate-adpcm-ima-cunning-trunc-t2-track1, which (as confirmed by Zane) is supposed to produce empty output. --- fftools/ffmpeg_enc.c | 22 ++--------------- fftools/ffmpeg_filter.c | 54 ++++++++++++++++++++++++++++++++++++++--- 2 files changed, 52 insertions(+), 24 deletions(-) diff --git a/fftools/ffmpeg_enc.c b/fftools/ffmpeg_enc.c index 4029501313..dd056e42f5 100644 --- a/fftools/ffmpeg_enc.c +++ b/fftools/ffmpeg_enc.c @@ -1142,26 +1142,8 @@ void enc_flush(void) AVCodecContext *enc = ost->enc_ctx; OutputFile *of = output_files[ost->file_index]; - if (!enc) - continue; - - // Try to enable encoding with no input frames. - // Maybe we should just let encoding fail instead. - if (!e->opened) { - FilterGraph *fg = ost->filter->graph; - - av_log(ost, AV_LOG_WARNING, - "Finishing stream without any data written to it.\n"); - - if (!fg->graph) - continue; - - ret = enc_open(ost, NULL); - if (ret < 0) - exit_program(1); - } - - if (enc->codec_type != AVMEDIA_TYPE_VIDEO && enc->codec_type != AVMEDIA_TYPE_AUDIO) + if (!enc || !e->opened || + (enc->codec_type != AVMEDIA_TYPE_VIDEO && enc->codec_type != AVMEDIA_TYPE_AUDIO)) continue; ret = submit_encode_frame(of, ost, NULL); diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c index 67a5f48245..f8e64ce6cc 100644 --- a/fftools/ffmpeg_filter.c +++ b/fftools/ffmpeg_filter.c @@ -143,11 +143,17 @@ typedef struct OutputFilterPriv { int sample_rate; AVChannelLayout ch_layout; + AVRational time_base; + AVRational sample_aspect_ratio; + // those are only set if no format is specified and the encoder gives us multiple options // They point directly to the relevant lists of the encoder. const int *formats; const AVChannelLayout *ch_layouts; const int *sample_rates; + + // set to 1 after at least one frame passed through this output + int got_frame; } OutputFilterPriv; static OutputFilterPriv *ofp_from_ofilter(OutputFilter *ofilter) @@ -1576,6 +1582,9 @@ static int configure_filtergraph(FilterGraph *fg) ofp->width = av_buffersink_get_w(sink); ofp->height = av_buffersink_get_h(sink); + ofp->time_base = av_buffersink_get_time_base(sink); + ofp->sample_aspect_ratio = av_buffersink_get_sample_aspect_ratio(sink); + ofp->sample_rate = av_buffersink_get_sample_rate(sink); av_channel_layout_uninit(&ofp->ch_layout); ret = av_buffersink_get_ch_layout(sink, &ofp->ch_layout); @@ -1688,6 +1697,7 @@ int reap_filters(int flush) { /* Reap all buffers present in the buffer sinks */ for (OutputStream *ost = ost_iter(NULL); ost; ost = ost_iter(ost)) { + OutputFilterPriv *ofp; FilterGraphPriv *fgp; AVFrame *filtered_frame; AVFilterContext *filter; @@ -1697,6 +1707,7 @@ int reap_filters(int flush) continue; filter = ost->filter->filter; fgp = fgp_from_fg(ost->filter->graph); + ofp = ofp_from_ofilter(ost->filter); filtered_frame = fgp->frame; @@ -1749,6 +1760,7 @@ int reap_filters(int flush) enc_frame(ost, filtered_frame); av_frame_unref(filtered_frame); + ofp->got_frame = 1; } } @@ -1961,6 +1973,7 @@ int ifilter_send_frame(InputFilter *ifilter, AVFrame *frame, int keep_reference) int fg_transcode_step(FilterGraph *graph, InputStream **best_ist) { + FilterGraphPriv *fgp = fgp_from_fg(graph); int i, ret; int nb_requests, nb_requests_max = 0; InputStream *ist; @@ -1988,10 +2001,43 @@ int fg_transcode_step(FilterGraph *graph, InputStream **best_ist) return reap_filters(0); if (ret == AVERROR_EOF) { - ret = reap_filters(1); - for (i = 0; i < graph->nb_outputs; i++) - close_output_stream(graph->outputs[i]->ost); - return ret; + reap_filters(1); + for (int i = 0; i < graph->nb_outputs; i++) { + OutputFilter *ofilter = graph->outputs[i]; + OutputFilterPriv *ofp = ofp_from_ofilter(ofilter); + + // we are finished and no frames were ever seen at this output, + // at least initialize the encoder with a dummy frame + if (!ofp->got_frame) { + AVFrame *frame = fgp->frame; + + frame->time_base = ofp->time_base; + frame->format = ofp->format; + + frame->width = ofp->width; + frame->height = ofp->height; + frame->sample_aspect_ratio = ofp->sample_aspect_ratio; + + frame->sample_rate = ofp->sample_rate; + if (ofp->ch_layout.nb_channels) { + ret = av_channel_layout_copy(&frame->ch_layout, &ofp->ch_layout); + if (ret < 0) + return ret; + } + + av_assert0(!frame->buf[0]); + + av_log(ofilter->ost, AV_LOG_WARNING, + "No filtered frames for output stream, trying to " + "initialize anyway.\n"); + + enc_open(ofilter->ost, frame); + av_frame_unref(frame); + } + + close_output_stream(ofilter->ost); + } + return 0; } if (ret != AVERROR(EAGAIN)) return ret;