From patchwork Wed Mar 6 11:03:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 46839 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:d90e:b0:19e:cdac:8cce with SMTP id jd14csp313362pzb; Wed, 6 Mar 2024 03:06:21 -0800 (PST) X-Forwarded-Encrypted: i=2; AJvYcCVeBSTK7RMa6PpFoOmsI3dG8nP1HqvJGfzzwvYgab1mk8GGhe9wL9TNeWchb483bajf0i2x+zrhcoMikb9qa1FuypHYWzUqIJ8YQQ== X-Google-Smtp-Source: AGHT+IF1lscaOWi2gKWV2RN+4BA/Vsl3WQW4F6cL9HoxX7PLqBcq4DPF4zAeYq8jZKHq6bPApEw9 X-Received: by 2002:a17:906:1192:b0:a45:6ffe:c0e5 with SMTP id n18-20020a170906119200b00a456ffec0e5mr4781084eja.18.1709723180922; Wed, 06 Mar 2024 03:06:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1709723180; cv=none; d=google.com; s=arc-20160816; b=UWH1GhSoYpck8RpokSTfw+TUzn0Xh7srynpNfY7beJICfa4WkD4pl67bQc3VtWJjeO FiBHLANDdPX10AFypHZHp7U99r93gNNT9aGWpiEShjrpQTqnhSLDytmRWin/3GyeAXPF aQM8PPE82Ab9mFqWCSmNzjMTg71yKJVrrmM8sehuSqIz141LhtYYKqnPl8ktE9Koccvi pCegco6PYulz1uGB90vNJDSfzc/zmOYSNlAXZCCSj6dtIYRgZx0F5+88Ms2TNMgx8sWY zgJXRXFJ3ECNJNDpHIPKvP2R9ljVL7GRY1uhPm4ISwp7Wuu+Y5iwiSeKcQebmMJxeP0l WVeA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=4wT+nGC76X9Xd2CnEHzW2sR4FqmA/H5LSOtfi8UVzvk=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=tPbGLspZDtE0SKY9ZL642RxoujtZG0y2ta3oZ6gr5xANI8PhiiS4OjFgVwzJdWabKi aHlxkzxY8DXgJhJoav88o+49VTvDtvaOt7LkWb4fwlkPq9Lr92G5Je7oJWpGyxMmWzVB BlTTjRf3UsRVG1CxCHWvhZJ2NfovngUD8b3jhdnW1d1r40+Vb4hpmVA7Gor8vT2PxKN+ gqUGDuvi/enM7z0SYwk9jH5jEFRVdK/g7cT4/IDyDNS3a/HbEFcdY8tmpUHIcaxeSi5F U0hpzD71ddSt3wmKEuP+DIsNztD2EyVjMqW9650lx5c7cN0tSOzTDSyEZDvqGZAxWtPq bEGg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@khirnov.net header.s=mail header.b="XeMH9/7y"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id y17-20020a1709060a9100b00a45a9d652c9si1373921ejf.467.2024.03.06.03.06.20; Wed, 06 Mar 2024 03:06:20 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@khirnov.net header.s=mail header.b="XeMH9/7y"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id BAE5968CC5D; Wed, 6 Mar 2024 13:05:35 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail1.khirnov.net (quelana.khirnov.net [94.230.150.81]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 76B3D68BC64 for ; Wed, 6 Mar 2024 13:05:30 +0200 (EET) Authentication-Results: mail1.khirnov.net; dkim=pass (2048-bit key; unprotected) header.d=khirnov.net header.i=@khirnov.net header.a=rsa-sha256 header.s=mail header.b=XeMH9/7y; dkim-atps=neutral Received: from localhost (mail1.khirnov.net [IPv6:::1]) by mail1.khirnov.net (Postfix) with ESMTP id 0BD564D4E for ; Wed, 6 Mar 2024 12:05:26 +0100 (CET) Received: from mail1.khirnov.net ([IPv6:::1]) by localhost (mail1.khirnov.net [IPv6:::1]) (amavis, port 10024) with ESMTP id t_0-9utCkxYK for ; Wed, 6 Mar 2024 12:05:25 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=khirnov.net; s=mail; t=1709723123; bh=peeWXb0V8EyGWb5a+N7YQFR/CA5kE26PiwSiOYfwNrc=; h=From:To:Subject:Date:In-Reply-To:References:From; b=XeMH9/7yCjugz2fLaywkxNVqP8uGH3ZgBzB1kgNt824PFlTr0hXiVKlQ9qGj/6Jts xwj6eeZqm1B0eBRMhdfHGMKMUzLippmEJePB1FaefUA22j+8wfH35UJJE1NpEHL8a+ uXbdijmy7dwsiPWWVZbM2mayk7nibhhKcV3cMlfcPe14s4V0eM09UU2inEUl9j2Z1/ CZk4g1s0eWhhCMQUErqxqO9eHafHspmmRHjm0bvqWYTzxrziv9roQ0Ov/V5AZOeRjd GY/2yQFdGPKlVRPJpUJzq7uzvnIaJb+0Xb1JYcjTLXW+FQ+v8cIcGI8ZXYe0HG2bpE nAa641G8u+s1g== Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail1.khirnov.net (Postfix) with ESMTPS id ED3C44D52 for ; Wed, 6 Mar 2024 12:05:22 +0100 (CET) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:::1]) by libav.khirnov.net (Postfix) with ESMTP id DC4EA3A0DA6 for ; Wed, 6 Mar 2024 12:05:22 +0100 (CET) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Wed, 6 Mar 2024 12:03:10 +0100 Message-ID: <20240306110319.17339-9-anton@khirnov.net> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240306110319.17339-1-anton@khirnov.net> References: <20240306110319.17339-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 09/18] fftools/ffmpeg: simplify propagating fallback parameters from decoders to filters X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: CNnKSEFZjtLe Current callstack looks like this: * ifilter_bind_ist() (filter) calls ist_filter_add() (demuxer); * ist_filter_add() opens the decoder, and then calls dec_add_filter() (decoder); * dec_add_filter() calls ifilter_parameters_from_dec() (i.e. back into the filtering code) in order to give post-avcodec_open2() parameters to the filter. This is unnecessarily complicated. Pass the parameters as follows instead: * dec_init() (which opens the decoder) returns post-avcodec_open2() parameters to its caller (i.e. the demuxer) in a parameter-only AVFrame * the demuxer passes these parameters to the filter in InputFilterOptions, together with other filter options --- fftools/ffmpeg.h | 9 ++++--- fftools/ffmpeg_dec.c | 36 ++++++++++++++++++-------- fftools/ffmpeg_demux.c | 25 ++++++++++++++---- fftools/ffmpeg_filter.c | 57 ++++++++++++----------------------------- 4 files changed, 67 insertions(+), 60 deletions(-) diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index d3e03543ac..1a60f035b0 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -259,6 +259,8 @@ typedef struct InputFilterOptions { // a combination of IFILTER_FLAG_* unsigned flags; + + AVFrame *fallback; } InputFilterOptions; typedef struct InputFilter { @@ -735,16 +737,17 @@ AVBufferRef *hw_device_for_filter(void); /** * @param dec_opts Dictionary filled with decoder options. Its ownership * is transferred to the decoder. + * @param param_out If non-NULL, media properties after opening the decoder + * are written here. * * @retval ">=0" non-negative scheduler index on success * @retval "<0" an error code on failure */ int dec_init(Decoder **pdec, Scheduler *sch, - AVDictionary **dec_opts, const DecoderOpts *o); + AVDictionary **dec_opts, const DecoderOpts *o, + AVFrame *param_out); void dec_free(Decoder **pdec); -int dec_add_filter(Decoder *dec, InputFilter *ifilter); - int enc_alloc(Encoder **penc, const AVCodec *codec, Scheduler *sch, unsigned sch_idx); void enc_free(Encoder **penc); diff --git a/fftools/ffmpeg_dec.c b/fftools/ffmpeg_dec.c index af57b2bb9d..7005ada527 100644 --- a/fftools/ffmpeg_dec.c +++ b/fftools/ffmpeg_dec.c @@ -1066,7 +1066,8 @@ static int hw_device_setup_for_decode(DecoderPriv *dp, return 0; } -static int dec_open(DecoderPriv *dp, AVDictionary **dec_opts, const DecoderOpts *o) +static int dec_open(DecoderPriv *dp, AVDictionary **dec_opts, + const DecoderOpts *o, AVFrame *param_out) { const AVCodec *codec = o->codec; int ret; @@ -1140,11 +1141,32 @@ static int dec_open(DecoderPriv *dp, AVDictionary **dec_opts, const DecoderOpts dp->dec.subtitle_header = dp->dec_ctx->subtitle_header; dp->dec.subtitle_header_size = dp->dec_ctx->subtitle_header_size; + if (param_out) { + if (dp->dec_ctx->codec_type == AVMEDIA_TYPE_AUDIO) { + param_out->format = dp->dec_ctx->sample_fmt; + param_out->sample_rate = dp->dec_ctx->sample_rate; + + ret = av_channel_layout_copy(¶m_out->ch_layout, &dp->dec_ctx->ch_layout); + if (ret < 0) + return ret; + } else if (dp->dec_ctx->codec_type == AVMEDIA_TYPE_VIDEO) { + param_out->format = dp->dec_ctx->pix_fmt; + param_out->width = dp->dec_ctx->width; + param_out->height = dp->dec_ctx->height; + param_out->sample_aspect_ratio = dp->dec_ctx->sample_aspect_ratio; + param_out->colorspace = dp->dec_ctx->colorspace; + param_out->color_range = dp->dec_ctx->color_range; + } + + param_out->time_base = dp->dec_ctx->pkt_timebase; + } + return 0; } int dec_init(Decoder **pdec, Scheduler *sch, - AVDictionary **dec_opts, const DecoderOpts *o) + AVDictionary **dec_opts, const DecoderOpts *o, + AVFrame *param_out) { DecoderPriv *dp; int ret; @@ -1155,7 +1177,7 @@ int dec_init(Decoder **pdec, Scheduler *sch, if (ret < 0) return ret; - ret = dec_open(dp, dec_opts, o); + ret = dec_open(dp, dec_opts, o, param_out); if (ret < 0) goto fail; @@ -1166,11 +1188,3 @@ fail: dec_free((Decoder**)&dp); return ret; } - -int dec_add_filter(Decoder *dec, InputFilter *ifilter) -{ - DecoderPriv *dp = dp_from_dec(dec); - - // initialize fallback parameters for filtering - return ifilter_parameters_from_dec(ifilter, dp->dec_ctx); -} diff --git a/fftools/ffmpeg_demux.c b/fftools/ffmpeg_demux.c index ae0f635d8c..032a96567f 100644 --- a/fftools/ffmpeg_demux.c +++ b/fftools/ffmpeg_demux.c @@ -81,6 +81,8 @@ typedef struct DemuxStream { AVDictionary *decoder_opts; DecoderOpts dec_opts; char dec_name[16]; + // decoded media properties, as estimated by opening the decoder + AVFrame *decoded_params; AVBSFContext *bsf; @@ -839,6 +841,8 @@ static void ist_free(InputStream **pist) avcodec_parameters_free(&ist->par); + av_frame_free(&ds->decoded_params); + av_bsf_free(&ds->bsf); av_freep(pist); @@ -935,8 +939,12 @@ static int ist_use(InputStream *ist, int decoding_needed) ds->dec_opts.log_parent = ist; + ds->decoded_params = av_frame_alloc(); + if (!ds->decoded_params) + return AVERROR(ENOMEM); + ret = dec_init(&ist->decoder, d->sch, - &ds->decoder_opts, &ds->dec_opts); + &ds->decoder_opts, &ds->dec_opts, ds->decoded_params); if (ret < 0) return ret; ds->sch_idx_dec = ret; @@ -988,10 +996,6 @@ int ist_filter_add(InputStream *ist, InputFilter *ifilter, int is_simple, ist->filters[ist->nb_filters - 1] = ifilter; - ret = dec_add_filter(ist->decoder, ifilter); - if (ret < 0) - return ret; - if (ist->par->codec_type == AVMEDIA_TYPE_VIDEO) { if (ist->framerate.num > 0 && ist->framerate.den > 0) { opts->framerate = ist->framerate; @@ -1027,6 +1031,17 @@ int ist_filter_add(InputStream *ist, InputFilter *ifilter, int is_simple, ds->have_sub2video = 1; } + ret = av_frame_copy_props(opts->fallback, ds->decoded_params); + if (ret < 0) + return ret; + opts->fallback->format = ds->decoded_params->format; + opts->fallback->width = ds->decoded_params->width; + opts->fallback->height = ds->decoded_params->height; + + ret = av_channel_layout_copy(&opts->fallback->ch_layout, &ds->decoded_params->ch_layout); + if (ret < 0) + return ret; + if (copy_ts) { tsoffset = d->f.start_time == AV_NOPTS_VALUE ? 0 : d->f.start_time; if (!start_at_zero && d->f.ctx->start_time != AV_NOPTS_VALUE) diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c index 7aa9305c88..6438cbd73b 100644 --- a/fftools/ffmpeg_filter.c +++ b/fftools/ffmpeg_filter.c @@ -675,6 +675,10 @@ static int ifilter_bind_ist(InputFilter *ifilter, InputStream *ist) ifp->type_src = ist->st->codecpar->codec_type; + ifp->opts.fallback = av_frame_alloc(); + if (!ifp->opts.fallback) + return AVERROR(ENOMEM); + dec_idx = ist_filter_add(ist, ifilter, filtergraph_is_simple(ifilter->graph), &ifp->opts); if (dec_idx < 0) @@ -863,9 +867,8 @@ static InputFilter *ifilter_alloc(FilterGraph *fg) ifp->index = fg->nb_inputs - 1; ifp->format = -1; - ifp->fallback.format = -1; - ifp->color_space = ifp->fallback.color_space = AVCOL_SPC_UNSPECIFIED; - ifp->color_range = ifp->fallback.color_range = AVCOL_RANGE_UNSPECIFIED; + ifp->color_space = AVCOL_SPC_UNSPECIFIED; + ifp->color_range = AVCOL_RANGE_UNSPECIFIED; ifp->frame_queue = av_fifo_alloc2(8, sizeof(AVFrame*), AV_FIFO_FLAG_AUTO_GROW); if (!ifp->frame_queue) @@ -895,9 +898,8 @@ void fg_free(FilterGraph **pfg) } av_frame_free(&ifp->sub2video.frame); - av_channel_layout_uninit(&ifp->fallback.ch_layout); - av_frame_free(&ifp->frame); + av_frame_free(&ifp->opts.fallback); av_buffer_unref(&ifp->hw_frames_ctx); av_freep(&ifp->linklabel); @@ -1804,33 +1806,6 @@ fail: return ret; } -int ifilter_parameters_from_dec(InputFilter *ifilter, const AVCodecContext *dec) -{ - InputFilterPriv *ifp = ifp_from_ifilter(ifilter); - - ifp->fallback.time_base = dec->pkt_timebase; - - if (dec->codec_type == AVMEDIA_TYPE_VIDEO) { - ifp->fallback.format = dec->pix_fmt; - ifp->fallback.width = dec->width; - ifp->fallback.height = dec->height; - ifp->fallback.sample_aspect_ratio = dec->sample_aspect_ratio; - ifp->fallback.color_space = dec->colorspace; - ifp->fallback.color_range = dec->color_range; - } else if (dec->codec_type == AVMEDIA_TYPE_AUDIO) { - int ret; - - ifp->fallback.format = dec->sample_fmt; - ifp->fallback.sample_rate = dec->sample_rate; - - ret = av_channel_layout_copy(&ifp->fallback.ch_layout, &dec->ch_layout); - if (ret < 0) - return ret; - } - - return 0; -} - static int ifilter_parameters_from_frame(InputFilter *ifilter, const AVFrame *frame) { InputFilterPriv *ifp = ifp_from_ifilter(ifilter); @@ -2527,17 +2502,17 @@ static int send_eof(FilterGraphThread *fgt, InputFilter *ifilter, } else { if (ifp->format < 0) { // the filtergraph was never configured, use the fallback parameters - ifp->format = ifp->fallback.format; - ifp->sample_rate = ifp->fallback.sample_rate; - ifp->width = ifp->fallback.width; - ifp->height = ifp->fallback.height; - ifp->sample_aspect_ratio = ifp->fallback.sample_aspect_ratio; - ifp->color_space = ifp->fallback.color_space; - ifp->color_range = ifp->fallback.color_range; - ifp->time_base = ifp->fallback.time_base; + ifp->format = ifp->opts.fallback->format; + ifp->sample_rate = ifp->opts.fallback->sample_rate; + ifp->width = ifp->opts.fallback->width; + ifp->height = ifp->opts.fallback->height; + ifp->sample_aspect_ratio = ifp->opts.fallback->sample_aspect_ratio; + ifp->color_space = ifp->opts.fallback->colorspace; + ifp->color_range = ifp->opts.fallback->color_range; + ifp->time_base = ifp->opts.fallback->time_base; ret = av_channel_layout_copy(&ifp->ch_layout, - &ifp->fallback.ch_layout); + &ifp->opts.fallback->ch_layout); if (ret < 0) return ret;