From patchwork Thu Jul 13 10:55:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 42644 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:6da1:b0:131:a7d0:bc6d with SMTP id gl33csp2822632pzb; Thu, 13 Jul 2023 03:58:53 -0700 (PDT) X-Google-Smtp-Source: APBJJlG6J3I/RVTgKX/ox0TclchVrBs871xGYYRizbfik890vjyNjBZmhLdUerg/U9C0HBQ57ERG X-Received: by 2002:a17:906:4698:b0:991:e695:cb7 with SMTP id a24-20020a170906469800b00991e6950cb7mr910965ejr.68.1689245932950; Thu, 13 Jul 2023 03:58:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1689245932; cv=none; d=google.com; s=arc-20160816; b=xHUl0RosVanzgymyXdDMnen3JTo4cARI+1WD5aLikVebKW8uB9E9aaVNxiQFuczBTm r1bjTes0WPa0lDfOYZuY005h5Lmku3FFwSSq+1tVhamOLlUKooXqZ1eLIc2TNCtEKjsb WUU35ZOclUELWsMcJYOrxrp4rUQ87h9IXZ1IQpzv8sk13Fpasju2y+a3M2M1hoO/BkIX 7+nL5oJh+w/AwXx5iACUE3wgSwSwWJfPRGJRVfRzrCpr0nhh6NxhIDsWaX+kCBJPl65o oVFMHhTEfrEFCRP/srAZBvwXk6lVC606zp27Us96AdUiBnm17OgMd4EXUBotS700n+Sq NuUQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:delivered-to; bh=/1vRDzjfj/xTvgg0t1wX1Z6LLSWRdSR0w2mOARrg+V0=; fh=8ZF2QtiKOjBtyn9wuO2cMeZaLsE4aq02Bq77gGK61E8=; b=N8UBZ+9LrT8fhheTY0B+ut/F6dHqzNp+I/XfieCiJR8uNqByOI0KJs6UxBH/WNaajL PqHXSxrLxoD6DHYTTKv27tKDDu5ZRitAGerxEn9QMy7bvgYKkrVGawFDMjkkmOaWuNFq nNrc6UdLWlzc7bOwK8Tk/1KwWV6VD/FLkdPB0dyWl4HFjNVu6ukAaPU4UB5GrNWXe21L oIYepRhijiCZzo2WYvBqw3CKRVjgBhcQtIducdjzP13ip9IDYRYntHqivOWjXgXJLBb/ 4xI6Db9CLO5nKu2e8yEIXMlUbh6dIlhRvs5PI6PRmRQ26JhxyDkSGgt/tmNd8/EE8OfP 81vQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id l17-20020a1709067d5100b009888ce5d3d4si6584177ejp.436.2023.07.13.03.58.52; Thu, 13 Jul 2023 03:58:52 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9F3EB68C6C6; Thu, 13 Jul 2023 13:57:56 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail0.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9D33168C665 for ; Thu, 13 Jul 2023 13:57:50 +0300 (EEST) Received: from localhost (localhost [IPv6:::1]) by mail0.khirnov.net (Postfix) with ESMTP id 59F152404EA for ; Thu, 13 Jul 2023 12:57:50 +0200 (CEST) Received: from mail0.khirnov.net ([IPv6:::1]) by localhost (mail0.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id 1CQ3za91X5en for ; Thu, 13 Jul 2023 12:57:49 +0200 (CEST) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail0.khirnov.net (Postfix) with ESMTPS id AC554240D03 for ; Thu, 13 Jul 2023 12:57:41 +0200 (CEST) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:::1]) by libav.khirnov.net (Postfix) with ESMTP id 340763A2125 for ; Thu, 13 Jul 2023 12:57:36 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Thu, 13 Jul 2023 12:55:47 +0200 Message-Id: <20230713105553.21052-27-anton@khirnov.net> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230713105553.21052-1-anton@khirnov.net> References: <20230713105553.21052-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 27/33] fftools/ffmpeg: rework initializing encoders with no frames X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: r50C++Q61LmR When no frames were passed from a filtergraph to an encoder, but the filtergraph is configured (i.e. has output parameters), encoder flush code will use those parameters to initialize the encoder in a last-ditch effort to produce some useful output. Rework this process so that it is triggered by the filtergraph, which now sends a dummy frame with parameters, but no data, to the encoder, rather than the encoder reaching backwards into the filter. This approach is more in line with the natural data flow from filters to encoders and will allow to reduce encoder-filter interactions in following commits. This code is tested by fate-adpcm-ima-cunning-trunc-t2-track1, which (as confirmed by Zane) is supposed to produce empty output. --- fftools/ffmpeg_enc.c | 22 ++--------------- fftools/ffmpeg_filter.c | 54 ++++++++++++++++++++++++++++++++++++++--- 2 files changed, 52 insertions(+), 24 deletions(-) diff --git a/fftools/ffmpeg_enc.c b/fftools/ffmpeg_enc.c index b35dbb6290..390f1b3b57 100644 --- a/fftools/ffmpeg_enc.c +++ b/fftools/ffmpeg_enc.c @@ -1142,26 +1142,8 @@ void enc_flush(void) AVCodecContext *enc = ost->enc_ctx; OutputFile *of = output_files[ost->file_index]; - if (!enc) - continue; - - // Try to enable encoding with no input frames. - // Maybe we should just let encoding fail instead. - if (!e->opened) { - FilterGraph *fg = ost->filter->graph; - - av_log(ost, AV_LOG_WARNING, - "Finishing stream without any data written to it.\n"); - - if (!fg->graph) - continue; - - ret = enc_open(ost, NULL); - if (ret < 0) - exit_program(1); - } - - if (enc->codec_type != AVMEDIA_TYPE_VIDEO && enc->codec_type != AVMEDIA_TYPE_AUDIO) + if (!enc || !e->opened || + (enc->codec_type != AVMEDIA_TYPE_VIDEO && enc->codec_type != AVMEDIA_TYPE_AUDIO)) continue; ret = submit_encode_frame(of, ost, NULL); diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c index 67a5f48245..f8e64ce6cc 100644 --- a/fftools/ffmpeg_filter.c +++ b/fftools/ffmpeg_filter.c @@ -143,11 +143,17 @@ typedef struct OutputFilterPriv { int sample_rate; AVChannelLayout ch_layout; + AVRational time_base; + AVRational sample_aspect_ratio; + // those are only set if no format is specified and the encoder gives us multiple options // They point directly to the relevant lists of the encoder. const int *formats; const AVChannelLayout *ch_layouts; const int *sample_rates; + + // set to 1 after at least one frame passed through this output + int got_frame; } OutputFilterPriv; static OutputFilterPriv *ofp_from_ofilter(OutputFilter *ofilter) @@ -1576,6 +1582,9 @@ static int configure_filtergraph(FilterGraph *fg) ofp->width = av_buffersink_get_w(sink); ofp->height = av_buffersink_get_h(sink); + ofp->time_base = av_buffersink_get_time_base(sink); + ofp->sample_aspect_ratio = av_buffersink_get_sample_aspect_ratio(sink); + ofp->sample_rate = av_buffersink_get_sample_rate(sink); av_channel_layout_uninit(&ofp->ch_layout); ret = av_buffersink_get_ch_layout(sink, &ofp->ch_layout); @@ -1688,6 +1697,7 @@ int reap_filters(int flush) { /* Reap all buffers present in the buffer sinks */ for (OutputStream *ost = ost_iter(NULL); ost; ost = ost_iter(ost)) { + OutputFilterPriv *ofp; FilterGraphPriv *fgp; AVFrame *filtered_frame; AVFilterContext *filter; @@ -1697,6 +1707,7 @@ int reap_filters(int flush) continue; filter = ost->filter->filter; fgp = fgp_from_fg(ost->filter->graph); + ofp = ofp_from_ofilter(ost->filter); filtered_frame = fgp->frame; @@ -1749,6 +1760,7 @@ int reap_filters(int flush) enc_frame(ost, filtered_frame); av_frame_unref(filtered_frame); + ofp->got_frame = 1; } } @@ -1961,6 +1973,7 @@ int ifilter_send_frame(InputFilter *ifilter, AVFrame *frame, int keep_reference) int fg_transcode_step(FilterGraph *graph, InputStream **best_ist) { + FilterGraphPriv *fgp = fgp_from_fg(graph); int i, ret; int nb_requests, nb_requests_max = 0; InputStream *ist; @@ -1988,10 +2001,43 @@ int fg_transcode_step(FilterGraph *graph, InputStream **best_ist) return reap_filters(0); if (ret == AVERROR_EOF) { - ret = reap_filters(1); - for (i = 0; i < graph->nb_outputs; i++) - close_output_stream(graph->outputs[i]->ost); - return ret; + reap_filters(1); + for (int i = 0; i < graph->nb_outputs; i++) { + OutputFilter *ofilter = graph->outputs[i]; + OutputFilterPriv *ofp = ofp_from_ofilter(ofilter); + + // we are finished and no frames were ever seen at this output, + // at least initialize the encoder with a dummy frame + if (!ofp->got_frame) { + AVFrame *frame = fgp->frame; + + frame->time_base = ofp->time_base; + frame->format = ofp->format; + + frame->width = ofp->width; + frame->height = ofp->height; + frame->sample_aspect_ratio = ofp->sample_aspect_ratio; + + frame->sample_rate = ofp->sample_rate; + if (ofp->ch_layout.nb_channels) { + ret = av_channel_layout_copy(&frame->ch_layout, &ofp->ch_layout); + if (ret < 0) + return ret; + } + + av_assert0(!frame->buf[0]); + + av_log(ofilter->ost, AV_LOG_WARNING, + "No filtered frames for output stream, trying to " + "initialize anyway.\n"); + + enc_open(ofilter->ost, frame); + av_frame_unref(frame); + } + + close_output_stream(ofilter->ost); + } + return 0; } if (ret != AVERROR(EAGAIN)) return ret;