From patchwork Fri Oct 4 07:46:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 52016 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:938f:0:b0:48e:c0f8:d0de with SMTP id z15csp235427vqg; Fri, 4 Oct 2024 00:54:07 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCWz76CfRwAE3SeiEpuwjBHCGSCFzITUrPU/dG9e5znzGs6g6q7BuQrB6hS0CxA3i0yte6wHVV+qaBgo+395tiOR@gmail.com X-Google-Smtp-Source: AGHT+IFNpIsPkkMKYObmNdsUZmXIoBLcOFgkn59OO7L4A6sa4d9g6G/cjdXOGzbv68fp3khHhRJ/ X-Received: by 2002:a05:651c:19a0:b0:2fa:bc0b:fb86 with SMTP id 38308e7fff4ca-2faf3c133d9mr2683771fa.4.1728028447177; Fri, 04 Oct 2024 00:54:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1728028447; cv=none; d=google.com; s=arc-20240605; b=U3rnolpz7CZWQmxZDNWltEMOeI8PQPlh6KcQrqBCUxCZZiZiPkTRs+YbOatfCIg7jA FdZHCPrPBOvT/WTUi3MMU2ytF2WI9Ci5bdYAwFhFKEwTKJuk9+H+VGtdsEacu29FMHXV A5yyx/LtusF0aiyE6WIxwM54MJNn0lITiEo3JwoYGO4CSNYiKruHLYzoMTqqeqRAB3QI ewiWrZc8AGV53ZQYmgg5Kg9fenaS2lKFqHYYCLFlBdR5J84DggEkC5WNntEORGnvVzDv gZUuEPYqxaKARVZposAp9jyTZTZByzq5w7HQlxIZ8jZ46sL3RNB8ldBnxgvnTRyb5ccM IFEQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=VfZWylS79OAQIXISISEsKzYVrrGCXeG/1RwzlNfMRbA=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=KE9AiNdos7Qf/pTXSTkNQBI0s9xIBez1oItv63xvR33INdd/el5LDqcZtG1Wyw4jki x4ag5OFVg2Yk7VYGQJk/uGZoExpeW+XodfRvTmCgGhGsi5wZCe66w8P8KT5mo1OSCRah ORre3Wulh1uDi3DT7yXLh5DrX0LLh/OFTMxGT+GgpuP91Mw18WaRKdU2eIAkki+UIheL G+FfzQbiiN2R90a+NJe6hG71vM8GrjLU6ISMAQtN3RP8WjOB7sQpdZdXZi47y9/3thcO RRmGM1dM8SSSEd/VR2e7nUj56syYXMcBUE20+NVypimjpb/7dvvrq/d05gtNaSK/7PfA qpPQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@khirnov.net header.s=mail header.b=bfggOCgt; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 2adb3069b0e04-539a8287c32si1014186e87.271.2024.10.04.00.54.06; Fri, 04 Oct 2024 00:54:07 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@khirnov.net header.s=mail header.b=bfggOCgt; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E887068D9E9; Fri, 4 Oct 2024 10:46:26 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail1.khirnov.net (quelana.khirnov.net [94.230.150.81]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id DA5F168D0C2 for ; Fri, 4 Oct 2024 10:46:19 +0300 (EEST) Authentication-Results: mail1.khirnov.net; dkim=pass (2048-bit key; unprotected) header.d=khirnov.net header.i=@khirnov.net header.a=rsa-sha256 header.s=mail header.b=bfggOCgt; dkim-atps=neutral Received: from localhost (mail1.khirnov.net [IPv6:::1]) by mail1.khirnov.net (Postfix) with ESMTP id 7BD49EFF for ; Fri, 4 Oct 2024 09:46:19 +0200 (CEST) Received: from mail1.khirnov.net ([IPv6:::1]) by localhost (mail1.khirnov.net [IPv6:::1]) (amavis, port 10024) with ESMTP id wGBYpvPo3AqD for ; Fri, 4 Oct 2024 09:46:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=khirnov.net; s=mail; t=1728027976; bh=x1+ibfRp6zXihu8EAIXnhEK+BkB8qLdED8Xnt9gF3+U=; h=From:To:Subject:Date:From; b=bfggOCgtYoO+AHKXXoMDjgrEsPmua2KKRW6m9FYr6gYS5+lNLJNdIS5RWHxpPLOYX Nw2KwsjGGuHaHXSUuQ5fbjvxByKT9a/VJMObIZJPI2NzphlzXufZlXysAF5kMTuLOX fSwGb5hwlgEWKYMvFr8oObCeqWSBpAQkJVdFDvOVgRr7lD4h8E/LO7PFy3qV5XvV+B kertjNq/AbiFiS3f6mOzyEqT/JGwl4QuAJ8k+d1IQ1ZpBeWMuIscDG31jdFGVkA/Fh s1lxueHX45fnxkz5RrnVSD83dlh6panjQHIHeTE/2dhAPs7moJ9KM2MYI0qS9gtWxF NKXW/Sjqhao3g== Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail1.khirnov.net (Postfix) with ESMTPS id DCFD84DEE for ; Fri, 4 Oct 2024 09:46:16 +0200 (CEST) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:::1]) by libav.khirnov.net (Postfix) with ESMTP id 979503A0139 for ; Fri, 04 Oct 2024 09:46:16 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Fri, 4 Oct 2024 09:46:10 +0200 Message-ID: <20241004074613.21038-1-anton@khirnov.net> X-Mailer: git-send-email 2.43.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/2] doc/ffmpeg: rewrite the detailed description chapter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: iqYmx8D6sXwz Split it into sections that describe in detail * the components of the transcoding pipeline * the main features it handles, in order of complexity * streamcopy * transcoding * filtering Replace the current confusing/misleading diagrams with new ones that actually reflect the program components and data flow between them. --- doc/ffmpeg.texi | 491 +++++++++++++++++++++++++++++++++++++----------- 1 file changed, 378 insertions(+), 113 deletions(-) diff --git a/doc/ffmpeg.texi b/doc/ffmpeg.texi index de140067ae..e17c17bcd7 100644 --- a/doc/ffmpeg.texi +++ b/doc/ffmpeg.texi @@ -87,140 +87,405 @@ The format option may be needed for raw input files. @chapter Detailed description @c man begin DETAILED DESCRIPTION -The transcoding process in @command{ffmpeg} for each output can be described by -the following diagram: +@command{ffmpeg} builds a transcoding pipeline out of the components listed +below. The program's operation then consists of input data chunks flowing from +the sources down the pipes towards the sinks, while being transformed by the +components they encounter along the way. +The following kinds of components are available: +@itemize +@item +@emph{Demuxers} (short for "demultiplexers") read an input source in order to +extract + +@itemize +@item +global properties such as metadata or chapters; +@item +list of input elementary streams and their properties +@end itemize + +One demuxer instance is created for each @option{-i} option, and sends encoded +@emph{packets} to @emph{decoders} or @emph{muxers}. + +In other literature, demuxers are sometimes called @emph{splitters}, because +their main function is splitting a file into elementary streams (though some +files only contain one elementary stream). + +A schematic representation of a demuxer looks like this: @verbatim - _______ ______________ -| | | | -| input | demuxer | encoded data | decoder -| file | ---------> | packets | -----+ -|_______| |______________| | - v - _________ - | | - | decoded | - | frames | - |_________| - ________ ______________ | -| | | | | -| output | <-------- | encoded data | <----+ -| file | muxer | packets | encoder -|________| |______________| - - +┌──────────┬───────────────────────┐ +│ demuxer │ │ packets for stream 0 +╞══════════╡ elementary stream 0 ├──────────────────────⮞ +│ │ │ +│ global ├───────────────────────┤ +│properties│ │ packets for stream 1 +│ and │ elementary stream 1 ├──────────────────────⮞ +│ metadata │ │ +│ ├───────────────────────┤ +│ │ │ +│ │ ........... │ +│ │ │ +│ ├───────────────────────┤ +│ │ │ packets for stream N +│ │ elementary stream N ├──────────────────────⮞ +│ │ │ +└──────────┴───────────────────────┘ + ⯅ + │ + │ read from file, network stream, + │ grabbing device, etc. + │ @end verbatim -@command{ffmpeg} calls the libavformat library (containing demuxers) to read -input files and get packets containing encoded data from them. When there are -multiple input files, @command{ffmpeg} tries to keep them synchronized by -tracking lowest timestamp on any active input stream. +@item +@emph{Decoders} receive encoded (compressed) @emph{packets} for an audio, video, +or subtitle elementary stream, and decode them into raw @emph{frames} (arrays of +pixels for video, PCM for audio). A decoder is typically associated with (and +receives its input from) an elementary stream in a @emph{demuxer}, but sometimes +may also exist on its own (see @ref{Loopback decoders}). + +A schematic representation of a decoder looks like this: +@verbatim + ┌─────────┐ + packets │ │ raw frames +─────────⮞│ decoder ├────────────⮞ + │ │ + └─────────┘ +@end verbatim + +@item +@emph{Filtergraphs} process and transform raw audio or video @emph{frames}. A +filtergraph consists of one or more individual @emph{filters} linked into a +graph. Filtergraphs come in two flavors - @emph{simple} and @emph{complex}, +configured with the @option{-filter} and @option{-filter_complex} options, +respectively. + +A simple filtergraph is associated with an @emph{output elementary stream}; it +receives the input to be filtered from a @emph{decoder} and sends filtered +output to that output stream's @emph{encoder}. + +A simple video filtergraph that performs deinterlacing (using the @code{yadif} +deinterlacer) followed by resizing (using the @code{scale} filter) can look like +this: +@verbatim + + ┌────────────────────────┐ + │ simple filtergraph │ + frames from ╞════════════════════════╡ frames for + a decoder │ ┌───────┐ ┌───────┐ │ an encoder +────────────⮞├─⮞│ yadif ├─⮞│ scale ├─⮞│────────────⮞ + │ └───────┘ └───────┘ │ + └────────────────────────┘ +@end verbatim + +A complex filtergraph is standalone and not associated with any specific stream. +It may have multiple (or zero) inputs, potentially of different types (audio or +video), each of which receiving data either from a decoder or another complex +filtergraph's outputs. It also has one or more outputs that feed either an +encoder or another complex filtergraph's input. + +The following example diagram represents a complex filtergraph with 3 inputs and +2 outputs (all video): +@verbatim + ┌─────────────────────────────────────────────────┐ + │ complex filtergraph │ + ╞═════════════════════════════════════════════════╡ + frames ├───────┐ ┌─────────┐ ┌─────────┐ ┌────────┤ frames +─────────⮞│input 0├─⮞│ overlay ├─────⮞│ overlay ├─⮞│output 0├────────⮞ + ├───────┘ │ │ │ │ └────────┤ + frames ├───────┐╭⮞│ │ ╭⮞│ │ │ +─────────⮞│input 1├╯ └─────────┘ │ └─────────┘ │ + ├───────┘ │ │ + frames ├───────┐ ┌─────┐ ┌─────┬─╯ ┌────────┤ frames +─────────⮞│input 2├⮞│scale├⮞│split├───────────────⮞│output 1├────────⮞ + ├───────┘ └─────┘ └─────┘ └────────┤ + └─────────────────────────────────────────────────┘ +@end verbatim +Frames from second input are overlaid over those from the first. Frames from the +third input are rescaled, then the duplicated into two identical streams. One of +them is overlaid over the combined first two inputs, with the result exposed as +the filtergraph's first output. The other duplicate ends up being the +filtergraph's second output. + +@item +@emph{Encoders} receive raw audio, video, or subtitle @emph{frames} and encode +them into encoded @emph{packets}. The encoding (compression) process is +typically lossy - it degrades stream quality to make the output smaller; some +encoders are @emph{lossless}, but at the cost of much higher output size. A +video or audio encoder receives its input from some filtergraph's output, +subtitle encoders receive input from a decoder (since subtitle filtering is not +supported yet). Every encoder is associated with some muxer's @emph{output +elementary stream} and sends its output to that muxer. + +A schematic representation of an encoder looks like this: +@verbatim + ┌─────────┐ + raw frames │ │ packets +────────────⮞│ encoder ├─────────⮞ + │ │ + └─────────┘ +@end verbatim + +@item +@emph{Muxers} (short for "multiplexers") receive encoded @emph{packets} for +their elementary streams from encoders (the @emph{transcoding} path) or directly +from demuxers (the @emph{streamcopy} path), interleave them (when there is more +than one elementary stream), and write the resulting bytes into the output file +(or pipe, network stream, etc.). + +A schematic representation of a muxer looks like this: +@verbatim + ┌──────────────────────┬───────────┐ + packets for stream 0 │ │ muxer │ +──────────────────────⮞│ elementary stream 0 ╞═══════════╡ + │ │ │ + ├──────────────────────┤ global │ + packets for stream 1 │ │properties │ +──────────────────────⮞│ elementary stream 1 │ and │ + │ │ metadata │ + ├──────────────────────┤ │ + │ │ │ + │ ........... │ │ + │ │ │ + ├──────────────────────┤ │ + packets for stream N │ │ │ +──────────────────────⮞│ elementary stream N │ │ + │ │ │ + └──────────────────────┴─────┬─────┘ + │ + write to file, network stream, │ + grabbing device, etc. │ + │ + ▼ +@end verbatim + +@end itemize + +@section Streamcopy +The simplest pipeline in @command{ffmpeg} is single-stream +@emph{streamcopy}, that is copying one @emph{input elementary stream}'s packets +without decoding, filtering, or encoding them. As an example, consider an input +file called @file{INPUT.mkv} with 3 elementary streams, from which we take the +second and write it to file @file{OUTPUT.mp4}. A schematic representation of +such a pipeline looks like this: +@verbatim +┌──────────┬─────────────────────┐ +│ demuxer │ │ unused +╞══════════╡ elementary stream 0 ├────────╳ +│ │ │ +│INPUT.mkv ├─────────────────────┤ ┌──────────────────────┬───────────┐ +│ │ │ packets │ │ muxer │ +│ │ elementary stream 1 ├─────────⮞│ elementary stream 0 ╞═══════════╡ +│ │ │ │ │OUTPUT.mp4 │ +│ ├─────────────────────┤ └──────────────────────┴───────────┘ +│ │ │ unused +│ │ elementary stream 2 ├────────╳ +│ │ │ +└──────────┴─────────────────────┘ +@end verbatim + +The above pipeline can be constructed with the following commandline: +@example +ffmpeg -i INPUT.mkv -map 0:1 -c copy OUTPUT.mp4 +@end example + +In this commandline +@itemize + +@item +there is a single input @file{INPUT.mkv}; + +@item +there are no input options for this input; + +@item +there is a single output @file{OUTPUT.mp4}; + +@item +there are two output options for this output: + +@itemize +@item +@code{-map 0:1} selects the input stream to be used - from input with index 0 +(i.e. the first one) the stream with index 1 (i.e. the second one); + +@item +@code{-c copy} selects the @code{copy} encoder, i.e. streamcopy with no decoding +or encoding. +@end itemize + +@end itemize + +Streamcopy is useful for changing the elementary stream count, container format, +or modifying container-level metadata. Since there is no decoding or encoding, +it is very fast and there is no quality loss. However, it might not work in some +cases because of a variety of factors (e.g. certain information required by the +target container is not available in the source). Applying filters is obviously +also impossible, since filters work on decoded frames. + +More complex streamcopy scenarios can be constructed - e.g. combining streams +from two input files into a single output: +@verbatim +┌──────────┬────────────────────┐ ┌────────────────────┬───────────┐ +│ demuxer 0│ │ packets │ │ muxer │ +╞══════════╡elementary stream 0 ├────────⮞│elementary stream 0 ╞═══════════╡ +│INPUT0.mkv│ │ │ │OUTPUT.mp4 │ +└──────────┴────────────────────┘ ├────────────────────┤ │ +┌──────────┬────────────────────┐ │ │ │ +│ demuxer 1│ │ packets │elementary stream 1 │ │ +╞══════════╡elementary stream 0 ├────────⮞│ │ │ +│INPUT1.aac│ │ └────────────────────┴───────────┘ +└──────────┴────────────────────┘ +@end verbatim +that can be built by the commandline +@example +ffmpeg -i INPUT0.mkv -i INPUT1.aac -map 0:0 -map 1:0 -c copy OUTPUT.mp4 +@end example + +The output @option{-map} option is used twice here, creating two streams in the +output file - one fed by the first input and one by the second. The single +instance of the @option{-c} option selects streamcopy for both of those streams. +You could also use multiple instances of this option together with +@ref{Stream specifiers} to apply different values to each stream, as will be +demonstrated in following sections. + +A converse scenario is splitting multiple streams from a single input into +multiple outputs: +@verbatim +┌──────────┬─────────────────────┐ ┌───────────────────┬───────────┐ +│ demuxer │ │ packets │ │ muxer 0 │ +╞══════════╡ elementary stream 0 ├─────────⮞│elementary stream 0╞═══════════╡ +│ │ │ │ │OUTPUT0.mp4│ +│INPUT.mkv ├─────────────────────┤ └───────────────────┴───────────┘ +│ │ │ packets ┌───────────────────┬───────────┐ +│ │ elementary stream 1 ├─────────⮞│ │ muxer 1 │ +│ │ │ │elementary stream 0╞═══════════╡ +└──────────┴─────────────────────┘ │ │OUTPUT1.mp4│ + └───────────────────┴───────────┘ +@end verbatim +built with +@example +ffmpeg -i INPUT.mkv -map 0:0 -c copy OUTPUT0.mp4 -map 0:1 -c copy OUTPUT1.mp4 +@end example +Note how a separate instance of the @option{-c} option is needed for every +output file even though their values are the same. This is because non-global +options (which is most of them) only apply in the context of the file before +which they are placed. + +These examples can of course be further generalized into arbitrary remappings +of any number of inputs into any number of outputs. + +@section Trancoding +@emph{Transcoding} is the process of decoding a stream and then encoding it +again. Since encoding tends to be computationally expensive and in most cases +degrades the stream quality (i.e. it is @emph{lossy}), you should only transcode +when you need to and perform streamcopy otherwise. Typical reasons to transcode +are: + +@itemize +@item +applying filters - e.g. resizing, deinterlacing, or overlaying video; resampling +or mixing audio; + +@item +you want to feed the stream to something that cannot decode the original codec. +@end itemize +Note that @command{ffmpeg} will transcode all audio, video, and subtitle streams +unless you specify @option{-c copy} for them. + +Consider an example pipeline that reads an input file with one audio and one +video stream, transcodes the video and copies the audio into a single output +file. This can be schematically represented as follows +@verbatim +┌──────────┬─────────────────────┐ +│ demuxer │ │ audio packets +╞══════════╡ stream 0 (audio) ├─────────────────────────────────────╮ +│ │ │ │ +│INPUT.mkv ├─────────────────────┤ video ┌─────────┐ raw │ +│ │ │ packets │ video │ video frames │ +│ │ stream 1 (video) ├─────────⮞│ decoder ├──────────────╮ │ +│ │ │ │ │ │ │ +└──────────┴─────────────────────┘ └─────────┘ │ │ + ▼ ▼ + │ │ +┌──────────┬─────────────────────┐ video ┌─────────┐ │ │ +│ muxer │ │ packets │ video │ │ │ +╞══════════╡ stream 0 (video) │⮜─────────┤ encoder ├──────────────╯ │ +│ │ │ │(libx264)│ │ +│OUTPUT.mp4├─────────────────────┤ └─────────┘ │ +│ │ │ │ +│ │ stream 1 (audio) │⮜────────────────────────────────────╯ +│ │ │ +└──────────┴─────────────────────┘ +@end verbatim +and implemented with the following commandline: +@example +ffmpeg -i INPUT.mkv -map 0:v -map 0:a -c:v libx264 -c:a copy OUTPUT.mp4 +@end example +Note how it uses stream specifiers @code{:v} and @code{:a} to select input +streams and apply different values of the @option{-c} option to them; see the +@ref{Stream specifiers} section for more details. -Encoded packets are then passed to the decoder (unless streamcopy is selected -for the stream, see further for a description). The decoder produces -uncompressed frames (raw video/PCM audio/...) which can be processed further by -filtering (see next section). After filtering, the frames are passed to the -encoder, which encodes them and outputs encoded packets. Finally, those are -passed to the muxer, which writes the encoded packets to the output file. @section Filtering -Before encoding, @command{ffmpeg} can process raw audio and video frames using -filters from the libavfilter library. Several chained filters form a filter -graph. @command{ffmpeg} distinguishes between two types of filtergraphs: -simple and complex. + +When transcoding, audio and video streams can be filtered before encoding, with +either a @emph{simple} or @emph{complex} filtergraph. @subsection Simple filtergraphs + Simple filtergraphs are those that have exactly one input and output, both of -the same type. In the above diagram they can be represented by simply inserting -an additional step between decoding and encoding: +the same type (audio or video). They are configured with the per-stream +@option{-filter} option (with @option{-vf} and @option{-af} aliases for +@option{-filter:v} (video) and @option{-filter:a} (audio) respectively). Note +that simple filtergraphs are tied to their output stream, so e.g. if you have +multiple audio streams, @option{-af} will create a separate filtergraph for each +one. +Taking the trancoding example from above, adding filtering (and omitting audio, +for clarity) makes it look like this: @verbatim - _________ ______________ -| | | | -| decoded | | encoded data | -| frames |\ _ | packets | -|_________| \ /||______________| - \ __________ / - simple _\|| | / encoder - filtergraph | filtered |/ - | frames | - |__________| - +┌──────────┬───────────────┐ +│ demuxer │ │ ┌─────────┐ +╞══════════╡ video stream │ packets │ video │ frames +│INPUT.mkv │ ├─────────⮞│ decoder ├─────⮞───╮ +│ │ │ └─────────┘ │ +└──────────┴───────────────┘ │ + ╭───────────⮜───────────╯ + │ ┌────────────────────────┐ + │ │ simple filtergraph │ + │ ╞════════════════════════╡ + │ │ ┌───────┐ ┌───────┐ │ + ╰──⮞├─⮞│ yadif ├─⮞│ scale ├─⮞├╮ + │ └───────┘ └───────┘ ││ + └────────────────────────┘│ + │ + │ +┌──────────┬───────────────┐ video ┌─────────┐ │ +│ muxer │ │ packets │ video │ │ +╞══════════╡ video stream │⮜─────────┤ encoder ├───────⮜───────╯ +│OUTPUT.mp4│ │ │ │ +│ │ │ └─────────┘ +└──────────┴───────────────┘ @end verbatim -Simple filtergraphs are configured with the per-stream @option{-filter} option -(with @option{-vf} and @option{-af} aliases for video and audio respectively). -A simple filtergraph for video can look for example like this: - -@verbatim - _______ _____________ _______ ________ -| | | | | | | | -| input | ---> | deinterlace | ---> | scale | ---> | output | -|_______| |_____________| |_______| |________| - -@end verbatim - -Note that some filters change frame properties but not frame contents. E.g. the -@code{fps} filter in the example above changes number of frames, but does not -touch the frame contents. Another example is the @code{setpts} filter, which -only sets timestamps and otherwise passes the frames unchanged. - @subsection Complex filtergraphs + Complex filtergraphs are those which cannot be described as simply a linear -processing chain applied to one stream. This is the case, for example, when the graph has -more than one input and/or output, or when output stream type is different from -input. They can be represented with the following diagram: - -@verbatim - _________ -| | -| input 0 |\ __________ -|_________| \ | | - \ _________ /| output 0 | - \ | | / |__________| - _________ \| complex | / -| | | |/ -| input 1 |---->| filter |\ -|_________| | | \ __________ - /| graph | \ | | - / | | \| output 1 | - _________ / |_________| |__________| -| | / -| input 2 |/ -|_________| - -@end verbatim - -Complex filtergraphs are configured with the @option{-filter_complex} option. -Note that this option is global, since a complex filtergraph, by its nature, -cannot be unambiguously associated with a single stream or file. - -The @option{-lavfi} option is equivalent to @option{-filter_complex}. +processing chain applied to one stream. This is the case, for example, when the +graph has more than one input and/or output, or when output stream type is +different from input. Complex filtergraphs are configured with the +@option{-filter_complex} option. Note that this option is global, since a +complex filtergraph, by its nature, cannot be unambiguously associated with a +single stream or file. Each instance of @option{-filter_complex} creates a new +complex filtergraph, and there can be any number of them. A trivial example of a complex filtergraph is the @code{overlay} filter, which has two video inputs and one video output, containing one video overlaid on top of the other. Its audio counterpart is the @code{amix} filter. -@section Stream copy -Stream copy is a mode selected by supplying the @code{copy} parameter to the -@option{-codec} option. It makes @command{ffmpeg} omit the decoding and encoding -step for the specified stream, so it does only demuxing and muxing. It is useful -for changing the container format or modifying container-level metadata. The -diagram above will, in this case, simplify to this: - -@verbatim - _______ ______________ ________ -| | | | | | -| input | demuxer | encoded data | muxer | output | -| file | ---------> | packets | -------> | file | -|_______| |______________| |________| - -@end verbatim - -Since there is no decoding or encoding, it is very fast and there is no quality -loss. However, it might not work in some cases because of many factors. Applying -filters is obviously also impossible, since filters work on uncompressed data. - +@anchor{Loopback decoders} @section Loopback decoders While decoders are normally associated with demuxer streams, it is also possible to create "loopback" decoders that decode the output from some encoder and allow From patchwork Fri Oct 4 07:46:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 52017 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:938f:0:b0:48e:c0f8:d0de with SMTP id z15csp232776vqg; Fri, 4 Oct 2024 00:46:29 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCV6/F10Qwfof+MKthXBwbVWnM47G8LFzuZLUSlm0VtFYHrPEhz0NUH9TcTikvpsi+RiK8P17cSuz3ySF3GO7cwn@gmail.com X-Google-Smtp-Source: AGHT+IH7uujzp4lZ/1qn9GuQm/pfqRqguZHuBSvhi4/DtvZnYL5K0yb5+TWMoTRvx+pNiHCUrQoC X-Received: by 2002:a05:6402:40cf:b0:5c3:c42e:d60e with SMTP id 4fb4d7f45d1cf-5c8d2a79c6amr586949a12.0.1728027989419; Fri, 04 Oct 2024 00:46:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1728027989; cv=none; d=google.com; s=arc-20240605; b=kkPnlS8WgclbuckZhbC2U8iQ6Nw5r1kMa9BpNGV08SAMptu6VoV0PFdNWcum4rkBdr cNLzG/Y2ovzxlIMIgopV42LvmXvqGVsh4OZ/EXI/Vz4H+aOGN2O3HnQIIdgwEk0Kg+Ro W3K3QoGL1x1boy5pbqeV917+5IXBSLhl3KRPrRp8Eo3DUmPW3tUdjUmwyFrFuiFW8VWP JjmBCnl95UKgO6Y1W8rJzPCjFPQ2ELq7GG4RmCgwn6vqcs+GgUmxXIj6gPPQLnb+ugrh NoWN9YB0UCplJIjV8IjRD55ihFDhwtlW3BTPGNskcLKHMIztjqJNlz73sWwRLN2Mu5V7 3wCA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=J+Dc7n5IHGTtlp1prVE1ojrbQV/14qIDP+BAHqrZN7I=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=fmCarvABHXm4XkbPtl/5u9v0hc0Jjk2ECZ/Qy6tc7TOhDkMv7UrhuAiGfCw/AJFWjZ gzz42CGGRj5sBpjupqpXkwSOJr98AxQNu4RyGRJZpDNqTiLADjnRqhqdJqBQmPUazgM5 f9tQzDwgZbSMlDcoyJ1JII9Dzq/nAMA0+EqkFqSqfAinrvkI3a6suMgARN6dZl6rh+rs rs7TgxCza08/w1YlRsRnErqTZC36oFRRkyr/KZmwWeDwlCrA4kqvtdhyUS62yKgH2sG7 xTQwAOdFDXfAz67hHywKqvfp7RePiZ5n49zs9uMij0FOA0CURm1I96+c/vf+OFRzFlKt 4nNw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@khirnov.net header.s=mail header.b=M33Plwkt; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 4fb4d7f45d1cf-5c8ca498744si1926308a12.411.2024.10.04.00.46.28; Fri, 04 Oct 2024 00:46:29 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@khirnov.net header.s=mail header.b=M33Plwkt; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C21D968D9BD; Fri, 4 Oct 2024 10:46:25 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail1.khirnov.net (quelana.khirnov.net [94.230.150.81]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E585468D3BB for ; Fri, 4 Oct 2024 10:46:18 +0300 (EEST) Authentication-Results: mail1.khirnov.net; dkim=pass (2048-bit key; unprotected) header.d=khirnov.net header.i=@khirnov.net header.a=rsa-sha256 header.s=mail header.b=M33Plwkt; dkim-atps=neutral Received: from localhost (mail1.khirnov.net [IPv6:::1]) by mail1.khirnov.net (Postfix) with ESMTP id 241A84E14 for ; Fri, 4 Oct 2024 09:46:18 +0200 (CEST) Received: from mail1.khirnov.net ([IPv6:::1]) by localhost (mail1.khirnov.net [IPv6:::1]) (amavis, port 10024) with ESMTP id 6Imny8d9xJGC for ; Fri, 4 Oct 2024 09:46:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=khirnov.net; s=mail; t=1728027976; bh=31Z4i/DClIWZ1arR0qnggGgIN/+cmrpqB+F88xbmbyU=; h=From:To:Subject:Date:In-Reply-To:References:From; b=M33PlwktkgUrWwMu/ax6DM5siWTJ6COiVyMa//Gill61uJRi05Hk1lIgfvFx6F2DM 8MHSOU8iHcnHXeC90J9+U8PHMen6lF7YjM3d9doS8tIWzCxPfzvedNEuVJogRdpOpR 2hPGt3VxLynd5YMSM4VKMJ0qV2o7JD0QXGNKgVZZyGU60JYxGN96RNXu94U91h8tzg upO83ciXu1Tv9AfxLCIskbmDAwnA8cUWpjV2ec9WUqmWZiziaWiX5Q7guLYLcpJyc5 nvnPwd6E7F9EzVULXnAsJb3f1Z9hjVvtSAfLLpqdmr36isFkohru57txlEEAXyS1Mj eKzh96UmfrrbA== Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail1.khirnov.net (Postfix) with ESMTPS id DAEF4EFF for ; Fri, 4 Oct 2024 09:46:16 +0200 (CEST) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:::1]) by libav.khirnov.net (Postfix) with ESMTP id A433D3A0F02 for ; Fri, 04 Oct 2024 09:46:16 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Fri, 4 Oct 2024 09:46:11 +0200 Message-ID: <20241004074613.21038-2-anton@khirnov.net> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241004074613.21038-1-anton@khirnov.net> References: <20241004074613.21038-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/2] doc/ffmpeg.texi: add a diagram for the loopback decoder example X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 2rxQ0EQP3RXs --- doc/ffmpeg.texi | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/doc/ffmpeg.texi b/doc/ffmpeg.texi index e17c17bcd7..1ccc48bbad 100644 --- a/doc/ffmpeg.texi +++ b/doc/ffmpeg.texi @@ -526,6 +526,38 @@ reads an input video and @end itemize +Such a transcoding pipeline can be represented with the following diagram: +@verbatim +┌──────────┬───────────────┐ +│ demuxer │ │ ┌─────────┐ ┌─────────┐ ┌────────────────────┐ +╞══════════╡ video stream │ │ video │ │ video │ │ null muxer │ +│ INPUT │ ├──⮞│ decoder ├──┬────────⮞│ encoder ├─┬─⮞│(discards its input)│ +│ │ │ └─────────┘ │ │(libx264)│ │ └────────────────────┘ +└──────────┴───────────────┘ │ └─────────┘ │ + ╭───────⮜──╯ ┌─────────┐ │ + │ │loopback │ │ + │ ╭─────⮜──────┤ decoder ├────⮜──╯ + │ │ └─────────┘ + │ │ + │ │ + │ │ ┌───────────────────┐ + │ │ │complex filtergraph│ + │ │ ╞═══════════════════╡ + │ │ │ ┌─────────────┐ │ + ╰─╫─⮞├─⮞│ hstack ├─⮞├╮ + ╰─⮞├─⮞│ │ ││ + │ └─────────────┘ ││ + └───────────────────┘│ + │ +┌──────────┬───────────────┐ ┌─────────┐ │ +│ muxer │ │ │ video │ │ +╞══════════╡ video stream │⮜─┤ encoder ├───────⮜──────────╯ +│ OUTPUT │ │ │ (ffv1) │ +│ │ │ └─────────┘ +└──────────┴───────────────┘ +@end verbatim + + @c man end DETAILED DESCRIPTION @anchor{Stream selection}