From patchwork Fri May 5 09:07:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 41485 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:dca6:b0:f3:34fa:f187 with SMTP id ky38csp329783pzb; Fri, 5 May 2023 02:08:23 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7MgBMOz9VxQMFcBXBdSDS2bR8TUoyASMtQ4XqWF29TaUyZ5N21tCJ7EvHBYbDyrZz28Zn+ X-Received: by 2002:a05:6402:414:b0:50b:c0ce:d55 with SMTP id q20-20020a056402041400b0050bc0ce0d55mr765008edv.3.1683277703049; Fri, 05 May 2023 02:08:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683277703; cv=none; d=google.com; s=arc-20160816; b=LlEc83mdR0OAUmnNsI+BSfw0gsB3ufKpH5SML0HtNYYL2tRqUwJV1cuSY6gApExS4X 5QdCXsUoszSK+9KenusFiMFZTgdHh+UzOt2hkmVUme5vNpOaE7CZ/Cw1OcQ5+0UvFIt3 TuXvVMCp6GBcymaW9AUDzrS7lem1YHQbP7x1f7FNWMk9JaeoO1yrum69g1xGEnR56kuH ltT6Hz79H+YTi6OR/xny+srz2NURPVBBMZuzNNnO/dAcV0yR424i8BktqenOnTFZ3dSk F4qW9wKVugRPE1IImzIQR5GMN98YykHDYWglF3JFDMSBI+hwKnvH+kuNUoLZvXW3FIPb np+w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:delivered-to; bh=NfEQHX3eOB+z7LJtsBfDLKuEZLwuHmr2ABg21QvGH68=; b=tEXpblm39PMk0whX7KQAe3eN9YwpPQ4Uzn5YC9idQajzq22YDpi9rKV6fEwlY0s1jj ZNMm5JhRKaFN8nODrJTod1eLySjwpPKnsP5znMwnha9AX9mONPnZ9U7TNYe/N/mPtCc/ w6TbEvms+QPCbx7q3LWYDdt1X2D9uWrMbXVfwyLG00MaHSv7IoR3xPKN3ke/05BRcWiR QRkFb45kbE466kCM3reQB7ZgWzubopC5CqyrUEakjDR9ZedzoMymd/9udi0YiXV1O4SQ HQAM6Wtb8CkAaSFfixctBxFdBFbp4LMmCWIV9KMdvX1NAYZEbAw51AHt6lcHEslB7KVw ke1w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id d23-20020a056402079700b0050bc0a5f539si4547013edy.317.2023.05.05.02.08.22; Fri, 05 May 2023 02:08:23 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D9C0068C12A; Fri, 5 May 2023 12:07:42 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail0.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 8D17768BF31 for ; Fri, 5 May 2023 12:07:32 +0300 (EEST) Received: from localhost (localhost [IPv6:::1]) by mail0.khirnov.net (Postfix) with ESMTP id 4F74C2404F5 for ; Fri, 5 May 2023 11:07:32 +0200 (CEST) Received: from mail0.khirnov.net ([IPv6:::1]) by localhost (mail0.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id brQNRa_0DHSB for ; Fri, 5 May 2023 11:07:31 +0200 (CEST) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail0.khirnov.net (Postfix) with ESMTPS id 067C12404F8 for ; Fri, 5 May 2023 11:07:30 +0200 (CEST) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:::1]) by libav.khirnov.net (Postfix) with ESMTP id C87EC3A11DB for ; Fri, 5 May 2023 11:07:29 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Fri, 5 May 2023 11:07:16 +0200 Message-Id: <20230505090723.24872-4-anton@khirnov.net> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230505090723.24872-1-anton@khirnov.net> References: <20230505090723.24872-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 04/11] fftools/ffmpeg_filter: take fallback parameters from decoder, not demuxer X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: ylo3d2FMHeNx When an input stream terminates and no frames were successfully decoded, filtering code will currently configure the filtergraph using demuxer stream parameters. Use decoder parameters instead, which should be more reliable. Also, initialize them immediately when an input stream is bound to a filtergraph input, so that these parameters are always available (if at all) and filtering code does not need to reach into the decoder at some arbitrary later point. --- fftools/ffmpeg.c | 18 ------------- fftools/ffmpeg.h | 8 +++++- fftools/ffmpeg_demux.c | 7 +++++ fftools/ffmpeg_filter.c | 59 ++++++++++++++++++++++++++++++++++++----- 4 files changed, 67 insertions(+), 25 deletions(-) diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index 983573b991..38569c60a4 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -769,24 +769,6 @@ static void print_report(int is_last_report, int64_t timer_start, int64_t cur_ti first_report = 0; } -int ifilter_parameters_from_codecpar(InputFilter *ifilter, AVCodecParameters *par) -{ - int ret; - - // We never got any input. Set a fake format, which will - // come from libavformat. - ifilter->format = par->format; - ifilter->sample_rate = par->sample_rate; - ifilter->width = par->width; - ifilter->height = par->height; - ifilter->sample_aspect_ratio = par->sample_aspect_ratio; - ret = av_channel_layout_copy(&ifilter->ch_layout, &par->ch_layout); - if (ret < 0) - return ret; - - return 0; -} - static void check_decode_result(InputStream *ist, int *got_output, int ret) { if (*got_output || ret<0) diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index ec4c580b91..525a00bfcb 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -791,7 +791,13 @@ void sub2video_update(InputStream *ist, int64_t heartbeat_pts, AVSubtitle *sub); int ifilter_send_frame(InputFilter *ifilter, AVFrame *frame, int keep_reference); int ifilter_send_eof(InputFilter *ifilter, int64_t pts, AVRational tb); -int ifilter_parameters_from_codecpar(InputFilter *ifilter, AVCodecParameters *par); +/** + * Set up fallback filtering parameters from a decoder context. They will only + * be used if no frames are ever sent on this input, otherwise the actual + * parameters are taken from the frame. + */ +int ifilter_parameters_from_dec(InputFilter *ifilter, const AVCodecContext *dec); + int ifilter_has_all_input_formats(FilterGraph *fg); /** diff --git a/fftools/ffmpeg_demux.c b/fftools/ffmpeg_demux.c index 6727193697..ad283356ba 100644 --- a/fftools/ffmpeg_demux.c +++ b/fftools/ffmpeg_demux.c @@ -586,10 +586,17 @@ void ist_output_add(InputStream *ist, OutputStream *ost) void ist_filter_add(InputStream *ist, InputFilter *ifilter, int is_simple) { + int ret; + ist_use(ist, is_simple ? DECODING_FOR_OST : DECODING_FOR_FILTER); GROW_ARRAY(ist->filters, ist->nb_filters); ist->filters[ist->nb_filters - 1] = ifilter; + + // initialize fallback parameters for filtering + ret = ifilter_parameters_from_dec(ifilter, ist->dec_ctx); + if (ret < 0) + report_and_exit(ret); } static const AVCodec *choose_decoder(const OptionsContext *o, AVFormatContext *s, AVStream *st, diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c index 8c408eb7c4..3fad38f782 100644 --- a/fftools/ffmpeg_filter.c +++ b/fftools/ffmpeg_filter.c @@ -58,6 +58,19 @@ typedef struct InputFilterPriv { AVRational time_base; AVFifo *frame_queue; + + // fallback parameters to use when no input is ever sent + struct { + int format; + + int width; + int height; + AVRational sample_aspect_ratio; + + int sample_rate; + AVChannelLayout ch_layout; + } fallback; + } InputFilterPriv; static InputFilterPriv *ifp_from_ifilter(InputFilter *ifilter) @@ -225,6 +238,8 @@ static InputFilter *ifilter_alloc(FilterGraph *fg) ifilter->graph = fg; ifilter->format = -1; + ifp->fallback.format = -1; + ifp->frame_queue = av_fifo_alloc2(8, sizeof(AVFrame*), AV_FIFO_FLAG_AUTO_GROW); if (!ifp->frame_queue) report_and_exit(AVERROR(ENOMEM)); @@ -260,6 +275,9 @@ void fg_free(FilterGraph **pfg) avsubtitle_free(&sub); av_fifo_freep2(&ist->sub2video.sub_queue); } + + av_channel_layout_uninit(&ifp->fallback.ch_layout); + av_buffer_unref(&ifilter->hw_frames_ctx); av_freep(&ifilter->name); av_freep(&fg->inputs[j]); @@ -1361,6 +1379,29 @@ fail: return ret; } +int ifilter_parameters_from_dec(InputFilter *ifilter, const AVCodecContext *dec) +{ + InputFilterPriv *ifp = ifp_from_ifilter(ifilter); + + if (dec->codec_type == AVMEDIA_TYPE_VIDEO) { + ifp->fallback.format = dec->pix_fmt; + ifp->fallback.width = dec->width; + ifp->fallback.height = dec->height; + ifp->fallback.sample_aspect_ratio = dec->sample_aspect_ratio; + } else { + int ret; + + ifp->fallback.format = dec->sample_fmt; + ifp->fallback.sample_rate = dec->sample_rate; + + ret = av_channel_layout_copy(&ifp->fallback.ch_layout, &dec->ch_layout); + if (ret < 0) + return ret; + } + + return 0; +} + static int ifilter_parameters_from_frame(InputFilter *ifilter, const AVFrame *frame) { AVFrameSideData *sd; @@ -1469,12 +1510,18 @@ int ifilter_send_eof(InputFilter *ifilter, int64_t pts, AVRational tb) if (ret < 0) return ret; } else { - // the filtergraph was never configured - if (ifilter->format < 0) { - ret = ifilter_parameters_from_codecpar(ifilter, ifilter->ist->par); - if (ret < 0) - return ret; - } + // the filtergraph was never configured, use the fallback parameters + ifilter->format = ifp->fallback.format; + ifilter->sample_rate = ifp->fallback.sample_rate; + ifilter->width = ifp->fallback.width; + ifilter->height = ifp->fallback.height; + ifilter->sample_aspect_ratio = ifp->fallback.sample_aspect_ratio; + + ret = av_channel_layout_copy(&ifilter->ch_layout, + &ifp->fallback.ch_layout); + if (ret < 0) + return ret; + if (ifilter->format < 0 && (ifilter->type == AVMEDIA_TYPE_AUDIO || ifilter->type == AVMEDIA_TYPE_VIDEO)) { av_log(NULL, AV_LOG_ERROR, "Cannot determine format of input stream %d:%d after EOF\n", ifilter->ist->file_index, ifilter->ist->st->index); return AVERROR_INVALIDDATA;