From patchwork Fri Apr 5 16:12:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 47828 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:24a8:b0:1a3:b6bb:3029 with SMTP id m40csp1073436pzd; Fri, 5 Apr 2024 09:20:21 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCW8PaobSIjREYablu3ah+WEiHLBxnlctbNcPBYaZbA2Op/ICfEsTmwc/4DtY+8PpKbSOl4AOwC47vk93P1/645lwR5oMfQLNjNogQ== X-Google-Smtp-Source: AGHT+IF/zno9ccQluV+D6ocv6af9jK5gTEmYrvJPx/6yPONlmoufiflNpZ38p2LJx/eA8lvAZrKE X-Received: by 2002:a17:907:7243:b0:a51:827d:c471 with SMTP id ds3-20020a170907724300b00a51827dc471mr1699758ejc.2.1712334021092; Fri, 05 Apr 2024 09:20:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1712334021; cv=none; d=google.com; s=arc-20160816; b=IJiTFhROWHZtE5ygVsKqmc5LTK8lknVBCVtRt1syY8vxlMfX4vTUr2yNLUhfsC8YJv eO1gT0qfjoDtmCXY6Pp2nfrCo2LWMtnYCl/Q3rId/B/7xbRyt7J4LnbR42gj03Tu90SF FIsmDqsw5eXjRTn33ncwV/m9axjPDcDZGOyXISCtEWZ7OlWVKPlNQdizxXTGvawbR1RR ZFueNhsysfsv2YGkiZNITs1ogNvhxotuL9cuzFPvH8oCysM5Qndw7gmvwbhUyrSIQ6hc THTOCLhgeGLOggIxJNtJ9APratYoPJSZQTF9WGRZyTznFLjNL6vqrU2cDG0hLeaASS/F RbGg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=Spb9zHudEfIghjLkY4tOVGNEuc8i7q1qdgg5qPysxGw=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=dnO7hZ52iPvRTtJ1ghygsxLozedFiz3iGeTgVk8s4ILGH3IQUo63YmuSHqTr1maoj6 gBX3Q+FKPIRKTeBJBCizrGU9wOFGOJFR+8HpqcpFY+T11zax6rUgaPC1e8SAjVTIEpPF 9QiyKFuSL36BcNHU71dGmcf+w/JEOzDxfytylx2Yc1x2ROYLz0GbBgtAspHfOJrApzu1 /5y2vI2mtbpzGE/9LiWKrqexs5jJhbuHZZgU87UnrQgmJC/V5cYOOWncZRaBqf51lVmQ vXeNGVH/0eAL58lYPEE2V0+QvzwVJHYyuhq8qkUxK5z1kEVzAkijn4H1FPNKViFtsuCk Tc8A==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@khirnov.net header.s=mail header.b="GtM2/q3y"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id l10-20020a170906078a00b00a4e7817756bsi840510ejc.347.2024.04.05.09.20.20; Fri, 05 Apr 2024 09:20:21 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@khirnov.net header.s=mail header.b="GtM2/q3y"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id BF6EC68D365; Fri, 5 Apr 2024 19:20:17 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail1.khirnov.net (quelana.khirnov.net [94.230.150.81]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1369068D25B for ; Fri, 5 Apr 2024 19:20:11 +0300 (EEST) Authentication-Results: mail1.khirnov.net; dkim=pass (2048-bit key; unprotected) header.d=khirnov.net header.i=@khirnov.net header.a=rsa-sha256 header.s=mail header.b=GtM2/q3y; dkim-atps=neutral Received: from localhost (mail1.khirnov.net [IPv6:::1]) by mail1.khirnov.net (Postfix) with ESMTP id B29CA1C2D for ; Fri, 5 Apr 2024 18:20:10 +0200 (CEST) Received: from mail1.khirnov.net ([IPv6:::1]) by localhost (mail1.khirnov.net [IPv6:::1]) (amavis, port 10024) with ESMTP id RWf6i1Bxfjrb for ; Fri, 5 Apr 2024 18:20:08 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=khirnov.net; s=mail; t=1712334008; bh=9ZSssHZ8o1AwhBx6HHQaynVexzZG855zteIAYtjkN+Q=; h=From:To:Subject:Date:In-Reply-To:References:From; b=GtM2/q3yCGCOW+PtCE5hp62BFJb8ep/NqqVJagQVhrjxT+ad7HR6pWdVNnnR4LZ04 2NL3yf784j+LNUMmVahc34Q5Cf417XWpXbxCdyZqRRqTNpdZnmlfRgiq4cduxbB1K0 0wZRVPrXikYlP94r8RZ+sGUNxwieTk4K28NoKfth/oDrpHQGho226WViLpkhbergS5 QZ4Tdpm3Hus0+zXQEldyw9GE8gdvOWE3RoqfNSpo9IDIRcHLje3xcLEJ5bGkGVHcL0 x9Hws39CDSUsgZ02toBcKPND2eebhNAr9F3asE7uqz2iA9XeYvavnrtojMwQOYzmDC jotSzFmrv4JAw== Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail1.khirnov.net (Postfix) with ESMTPS id 6F42125F for ; Fri, 5 Apr 2024 18:20:08 +0200 (CEST) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:::1]) by libav.khirnov.net (Postfix) with ESMTP id D39E83A197B for ; Fri, 5 Apr 2024 18:12:56 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Fri, 5 Apr 2024 18:12:11 +0200 Message-ID: <20240405161212.26167-30-anton@khirnov.net> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240405161212.26167-1-anton@khirnov.net> References: <20240405161212.26167-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 30/31] fftools/ffmpeg_filter: implement filtergraph chaining X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: D485fs+9/bgr This allows one complex filtergraph's output to be sent as input to another one, which is useful in certain situations (one is described in the docs). Chaining filtergraphs was already effectively possible by using a wrapped_avframe encoder connected to a loopback decoder, but it is ugly, non-obvious and inefficient. --- Changelog | 1 + doc/ffmpeg.texi | 58 ++++++++++++++++++++++++--- fftools/ffmpeg_filter.c | 89 +++++++++++++++++++++++++++++++++++++++-- 3 files changed, 140 insertions(+), 8 deletions(-) diff --git a/Changelog b/Changelog index 18e83b99a1..b7a1af4083 100644 --- a/Changelog +++ b/Changelog @@ -4,6 +4,7 @@ releases are sorted from youngest to oldest. version : - Raw Captions with Time (RCWT) closed caption demuxer - LC3/LC3plus decoding/encoding using external library liblc3 +- ffmpeg CLI filtergraph chaining version 7.0: diff --git a/doc/ffmpeg.texi b/doc/ffmpeg.texi index 801c083705..9bd548ce4e 100644 --- a/doc/ffmpeg.texi +++ b/doc/ffmpeg.texi @@ -2145,14 +2145,62 @@ type -- see the @option{-filter} options. @var{filtergraph} is a description of the filtergraph, as described in the ``Filtergraph syntax'' section of the ffmpeg-filters manual. -Input link labels must refer to either input streams or loopback decoders. For -input streams, use the @code{[file_index:stream_specifier]} syntax (i.e. the -same as @option{-map} uses). If @var{stream_specifier} matches multiple streams, -the first one will be used. +Inputs to a complex filtergraph may come from different source types, +distinguished by the format of the corresponding link label: +@itemize +@item +To connect an input stream, use @code{[file_index:stream_specifier]} (i.e. the +same syntax as @option{-map}). If @var{stream_specifier} matches multiple +streams, the first one will be used. -For decoders, the link label must be [dec:@var{dec_idx}], where @var{dec_idx} is +@item +To connect a loopback decoder use [dec:@var{dec_idx}], where @var{dec_idx} is the index of the loopback decoder to be connected to given input. +@item +To connect an output from another complex filtergraph, use its link label. E.g +the following example: + +@example +ffmpeg -i input.mkv \ + -filter_complex '[0:v]scale=size=hd1080,split=outputs=2[for_enc][orig_scaled]' \ + -c:v libx264 -map '[for_enc]' output.mkv \ + -dec 0:0 \ + -filter_complex '[dec:0][orig_scaled]hstack[stacked]' \ + -map '[stacked]' -c:v ffv1 comparison.mkv +@end example + +reads an input video and +@itemize +@item +(line 2) uses a complex filtergraph with one input and two outputs +to scale the video to 1920x1080 and duplicate the result to both +outputs; + +@item +(line 3) encodes one scaled output with @code{libx264} and writes the result to +@file{output.mkv}; + +@item +(line 4) decodes this encoded stream with a loopback decoder; + +@item +(line 5) places the output of the loopback decoder (i.e. the +@code{libx264}-encoded video) side by side with the scaled original input; + +@item +(line 6) combined video is then losslessly encoded and written into +@file{comparison.mkv}. + +@end itemize + +Note that the two filtergraphs cannot be combined into one, because then there +would be a cycle in the transcoding pipeline (filtergraph output goes to +encoding, from there to decoding, then back to the same graph), and such cycles +are not allowed. + +@end itemize + An unlabeled input will be connected to the first unused input stream of the matching type. diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c index 1e14962f41..f108f8daf9 100644 --- a/fftools/ffmpeg_filter.c +++ b/fftools/ffmpeg_filter.c @@ -902,6 +902,63 @@ int ofilter_bind_ost(OutputFilter *ofilter, OutputStream *ost, return 0; } +static int ofilter_bind_ifilter(OutputFilter *ofilter, InputFilterPriv *ifp, + const OutputFilterOptions *opts) +{ + OutputFilterPriv *ofp = ofp_from_ofilter(ofilter); + + av_assert0(!ofilter->bound); + av_assert0(ofilter->type == ifp->type); + + ofilter->bound = 1; + av_freep(&ofilter->linklabel); + + ofp->name = av_strdup(opts->name); + if (!ofp->name) + return AVERROR(EINVAL); + + av_strlcatf(ofp->log_name, sizeof(ofp->log_name), "->%s", ofp->name); + + return 0; +} + +static int ifilter_bind_fg(InputFilterPriv *ifp, FilterGraph *fg_src, int out_idx) +{ + FilterGraphPriv *fgp = fgp_from_fg(ifp->ifilter.graph); + OutputFilter *ofilter_src = fg_src->outputs[out_idx]; + OutputFilterOptions opts; + char name[32]; + int ret; + + av_assert0(!ifp->bound); + ifp->bound = 1; + + if (ifp->type != ofilter_src->type) { + av_log(fgp, AV_LOG_ERROR, "Tried to connect %s output to %s input\n", + av_get_media_type_string(ofilter_src->type), + av_get_media_type_string(ifp->type)); + return AVERROR(EINVAL); + } + + ifp->type_src = ifp->type; + + memset(&opts, 0, sizeof(opts)); + + snprintf(name, sizeof(name), "fg:%d:%d", fgp->fg.index, ifp->index); + opts.name = name; + + ret = ofilter_bind_ifilter(ofilter_src, ifp, &opts); + if (ret < 0) + return ret; + + ret = sch_connect(fgp->sch, SCH_FILTER_OUT(fg_src->index, out_idx), + SCH_FILTER_IN(fgp->sch_idx, ifp->index)); + if (ret < 0) + return ret; + + return 0; +} + static InputFilter *ifilter_alloc(FilterGraph *fg) { InputFilterPriv *ifp; @@ -1213,12 +1270,38 @@ static int fg_complex_bind_input(FilterGraph *fg, InputFilter *ifilter) ifilter->name); return ret; } else if (ifp->linklabel) { - // bind to an explicitly specified demuxer stream AVFormatContext *s; AVStream *st = NULL; char *p; - int file_idx = strtol(ifp->linklabel, &p, 0); + int file_idx; + // try finding an unbound filtergraph output with this label + for (int i = 0; i < nb_filtergraphs; i++) { + FilterGraph *fg_src = filtergraphs[i]; + + if (fg == fg_src) + continue; + + for (int j = 0; j < fg_src->nb_outputs; j++) { + OutputFilter *ofilter = fg_src->outputs[j]; + + if (!ofilter->bound && ofilter->linklabel && + !strcmp(ofilter->linklabel, ifp->linklabel)) { + av_log(fg, AV_LOG_VERBOSE, + "Binding input with label '%s' to filtergraph output %d:%d\n", + ifp->linklabel, i, j); + + ret = ifilter_bind_fg(ifp, fg_src, j); + if (ret < 0) + av_log(fg, AV_LOG_ERROR, "Error binding filtergraph input %s\n", + ifp->linklabel); + return ret; + } + } + } + + // bind to an explicitly specified demuxer stream + file_idx = strtol(ifp->linklabel, &p, 0); if (file_idx < 0 || file_idx >= nb_input_files) { av_log(fg, AV_LOG_FATAL, "Invalid file index %d in filtergraph description %s.\n", file_idx, fgp->graph_desc); @@ -1274,7 +1357,7 @@ static int fg_complex_bind_input(FilterGraph *fg, InputFilter *ifilter) static int bind_inputs(FilterGraph *fg) { - // bind filtergraph inputs to input streams + // bind filtergraph inputs to input streams or other filtergraphs for (int i = 0; i < fg->nb_inputs; i++) { InputFilterPriv *ifp = ifp_from_ifilter(fg->inputs[i]); int ret;