From patchwork Fri Oct 4 07:46:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 52016 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:938f:0:b0:48e:c0f8:d0de with SMTP id z15csp235427vqg; Fri, 4 Oct 2024 00:54:07 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCWz76CfRwAE3SeiEpuwjBHCGSCFzITUrPU/dG9e5znzGs6g6q7BuQrB6hS0CxA3i0yte6wHVV+qaBgo+395tiOR@gmail.com X-Google-Smtp-Source: AGHT+IFNpIsPkkMKYObmNdsUZmXIoBLcOFgkn59OO7L4A6sa4d9g6G/cjdXOGzbv68fp3khHhRJ/ X-Received: by 2002:a05:651c:19a0:b0:2fa:bc0b:fb86 with SMTP id 38308e7fff4ca-2faf3c133d9mr2683771fa.4.1728028447177; Fri, 04 Oct 2024 00:54:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1728028447; cv=none; d=google.com; s=arc-20240605; b=U3rnolpz7CZWQmxZDNWltEMOeI8PQPlh6KcQrqBCUxCZZiZiPkTRs+YbOatfCIg7jA FdZHCPrPBOvT/WTUi3MMU2ytF2WI9Ci5bdYAwFhFKEwTKJuk9+H+VGtdsEacu29FMHXV A5yyx/LtusF0aiyE6WIxwM54MJNn0lITiEo3JwoYGO4CSNYiKruHLYzoMTqqeqRAB3QI ewiWrZc8AGV53ZQYmgg5Kg9fenaS2lKFqHYYCLFlBdR5J84DggEkC5WNntEORGnvVzDv gZUuEPYqxaKARVZposAp9jyTZTZByzq5w7HQlxIZ8jZ46sL3RNB8ldBnxgvnTRyb5ccM IFEQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=VfZWylS79OAQIXISISEsKzYVrrGCXeG/1RwzlNfMRbA=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=KE9AiNdos7Qf/pTXSTkNQBI0s9xIBez1oItv63xvR33INdd/el5LDqcZtG1Wyw4jki x4ag5OFVg2Yk7VYGQJk/uGZoExpeW+XodfRvTmCgGhGsi5wZCe66w8P8KT5mo1OSCRah ORre3Wulh1uDi3DT7yXLh5DrX0LLh/OFTMxGT+GgpuP91Mw18WaRKdU2eIAkki+UIheL G+FfzQbiiN2R90a+NJe6hG71vM8GrjLU6ISMAQtN3RP8WjOB7sQpdZdXZi47y9/3thcO RRmGM1dM8SSSEd/VR2e7nUj56syYXMcBUE20+NVypimjpb/7dvvrq/d05gtNaSK/7PfA qpPQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@khirnov.net header.s=mail header.b=bfggOCgt; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 2adb3069b0e04-539a8287c32si1014186e87.271.2024.10.04.00.54.06; Fri, 04 Oct 2024 00:54:07 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@khirnov.net header.s=mail header.b=bfggOCgt; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E887068D9E9; Fri, 4 Oct 2024 10:46:26 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail1.khirnov.net (quelana.khirnov.net [94.230.150.81]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id DA5F168D0C2 for ; Fri, 4 Oct 2024 10:46:19 +0300 (EEST) Authentication-Results: mail1.khirnov.net; dkim=pass (2048-bit key; unprotected) header.d=khirnov.net header.i=@khirnov.net header.a=rsa-sha256 header.s=mail header.b=bfggOCgt; dkim-atps=neutral Received: from localhost (mail1.khirnov.net [IPv6:::1]) by mail1.khirnov.net (Postfix) with ESMTP id 7BD49EFF for ; Fri, 4 Oct 2024 09:46:19 +0200 (CEST) Received: from mail1.khirnov.net ([IPv6:::1]) by localhost (mail1.khirnov.net [IPv6:::1]) (amavis, port 10024) with ESMTP id wGBYpvPo3AqD for ; Fri, 4 Oct 2024 09:46:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=khirnov.net; s=mail; t=1728027976; bh=x1+ibfRp6zXihu8EAIXnhEK+BkB8qLdED8Xnt9gF3+U=; h=From:To:Subject:Date:From; b=bfggOCgtYoO+AHKXXoMDjgrEsPmua2KKRW6m9FYr6gYS5+lNLJNdIS5RWHxpPLOYX Nw2KwsjGGuHaHXSUuQ5fbjvxByKT9a/VJMObIZJPI2NzphlzXufZlXysAF5kMTuLOX fSwGb5hwlgEWKYMvFr8oObCeqWSBpAQkJVdFDvOVgRr7lD4h8E/LO7PFy3qV5XvV+B kertjNq/AbiFiS3f6mOzyEqT/JGwl4QuAJ8k+d1IQ1ZpBeWMuIscDG31jdFGVkA/Fh s1lxueHX45fnxkz5RrnVSD83dlh6panjQHIHeTE/2dhAPs7moJ9KM2MYI0qS9gtWxF NKXW/Sjqhao3g== Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail1.khirnov.net (Postfix) with ESMTPS id DCFD84DEE for ; Fri, 4 Oct 2024 09:46:16 +0200 (CEST) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:::1]) by libav.khirnov.net (Postfix) with ESMTP id 979503A0139 for ; Fri, 04 Oct 2024 09:46:16 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Fri, 4 Oct 2024 09:46:10 +0200 Message-ID: <20241004074613.21038-1-anton@khirnov.net> X-Mailer: git-send-email 2.43.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/2] doc/ffmpeg: rewrite the detailed description chapter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: iqYmx8D6sXwz Split it into sections that describe in detail * the components of the transcoding pipeline * the main features it handles, in order of complexity * streamcopy * transcoding * filtering Replace the current confusing/misleading diagrams with new ones that actually reflect the program components and data flow between them. --- doc/ffmpeg.texi | 491 +++++++++++++++++++++++++++++++++++++----------- 1 file changed, 378 insertions(+), 113 deletions(-) diff --git a/doc/ffmpeg.texi b/doc/ffmpeg.texi index de140067ae..e17c17bcd7 100644 --- a/doc/ffmpeg.texi +++ b/doc/ffmpeg.texi @@ -87,140 +87,405 @@ The format option may be needed for raw input files. @chapter Detailed description @c man begin DETAILED DESCRIPTION -The transcoding process in @command{ffmpeg} for each output can be described by -the following diagram: +@command{ffmpeg} builds a transcoding pipeline out of the components listed +below. The program's operation then consists of input data chunks flowing from +the sources down the pipes towards the sinks, while being transformed by the +components they encounter along the way. +The following kinds of components are available: +@itemize +@item +@emph{Demuxers} (short for "demultiplexers") read an input source in order to +extract + +@itemize +@item +global properties such as metadata or chapters; +@item +list of input elementary streams and their properties +@end itemize + +One demuxer instance is created for each @option{-i} option, and sends encoded +@emph{packets} to @emph{decoders} or @emph{muxers}. + +In other literature, demuxers are sometimes called @emph{splitters}, because +their main function is splitting a file into elementary streams (though some +files only contain one elementary stream). + +A schematic representation of a demuxer looks like this: @verbatim - _______ ______________ -| | | | -| input | demuxer | encoded data | decoder -| file | ---------> | packets | -----+ -|_______| |______________| | - v - _________ - | | - | decoded | - | frames | - |_________| - ________ ______________ | -| | | | | -| output | <-------- | encoded data | <----+ -| file | muxer | packets | encoder -|________| |______________| - - +┌──────────┬───────────────────────┐ +│ demuxer │ │ packets for stream 0 +╞══════════╡ elementary stream 0 ├──────────────────────⮞ +│ │ │ +│ global ├───────────────────────┤ +│properties│ │ packets for stream 1 +│ and │ elementary stream 1 ├──────────────────────⮞ +│ metadata │ │ +│ ├───────────────────────┤ +│ │ │ +│ │ ........... │ +│ │ │ +│ ├───────────────────────┤ +│ │ │ packets for stream N +│ │ elementary stream N ├──────────────────────⮞ +│ │ │ +└──────────┴───────────────────────┘ + ⯅ + │ + │ read from file, network stream, + │ grabbing device, etc. + │ @end verbatim -@command{ffmpeg} calls the libavformat library (containing demuxers) to read -input files and get packets containing encoded data from them. When there are -multiple input files, @command{ffmpeg} tries to keep them synchronized by -tracking lowest timestamp on any active input stream. +@item +@emph{Decoders} receive encoded (compressed) @emph{packets} for an audio, video, +or subtitle elementary stream, and decode them into raw @emph{frames} (arrays of +pixels for video, PCM for audio). A decoder is typically associated with (and +receives its input from) an elementary stream in a @emph{demuxer}, but sometimes +may also exist on its own (see @ref{Loopback decoders}). + +A schematic representation of a decoder looks like this: +@verbatim + ┌─────────┐ + packets │ │ raw frames +─────────⮞│ decoder ├────────────⮞ + │ │ + └─────────┘ +@end verbatim + +@item +@emph{Filtergraphs} process and transform raw audio or video @emph{frames}. A +filtergraph consists of one or more individual @emph{filters} linked into a +graph. Filtergraphs come in two flavors - @emph{simple} and @emph{complex}, +configured with the @option{-filter} and @option{-filter_complex} options, +respectively. + +A simple filtergraph is associated with an @emph{output elementary stream}; it +receives the input to be filtered from a @emph{decoder} and sends filtered +output to that output stream's @emph{encoder}. + +A simple video filtergraph that performs deinterlacing (using the @code{yadif} +deinterlacer) followed by resizing (using the @code{scale} filter) can look like +this: +@verbatim + + ┌────────────────────────┐ + │ simple filtergraph │ + frames from ╞════════════════════════╡ frames for + a decoder │ ┌───────┐ ┌───────┐ │ an encoder +────────────⮞├─⮞│ yadif ├─⮞│ scale ├─⮞│────────────⮞ + │ └───────┘ └───────┘ │ + └────────────────────────┘ +@end verbatim + +A complex filtergraph is standalone and not associated with any specific stream. +It may have multiple (or zero) inputs, potentially of different types (audio or +video), each of which receiving data either from a decoder or another complex +filtergraph's outputs. It also has one or more outputs that feed either an +encoder or another complex filtergraph's input. + +The following example diagram represents a complex filtergraph with 3 inputs and +2 outputs (all video): +@verbatim + ┌─────────────────────────────────────────────────┐ + │ complex filtergraph │ + ╞═════════════════════════════════════════════════╡ + frames ├───────┐ ┌─────────┐ ┌─────────┐ ┌────────┤ frames +─────────⮞│input 0├─⮞│ overlay ├─────⮞│ overlay ├─⮞│output 0├────────⮞ + ├───────┘ │ │ │ │ └────────┤ + frames ├───────┐╭⮞│ │ ╭⮞│ │ │ +─────────⮞│input 1├╯ └─────────┘ │ └─────────┘ │ + ├───────┘ │ │ + frames ├───────┐ ┌─────┐ ┌─────┬─╯ ┌────────┤ frames +─────────⮞│input 2├⮞│scale├⮞│split├───────────────⮞│output 1├────────⮞ + ├───────┘ └─────┘ └─────┘ └────────┤ + └─────────────────────────────────────────────────┘ +@end verbatim +Frames from second input are overlaid over those from the first. Frames from the +third input are rescaled, then the duplicated into two identical streams. One of +them is overlaid over the combined first two inputs, with the result exposed as +the filtergraph's first output. The other duplicate ends up being the +filtergraph's second output. + +@item +@emph{Encoders} receive raw audio, video, or subtitle @emph{frames} and encode +them into encoded @emph{packets}. The encoding (compression) process is +typically lossy - it degrades stream quality to make the output smaller; some +encoders are @emph{lossless}, but at the cost of much higher output size. A +video or audio encoder receives its input from some filtergraph's output, +subtitle encoders receive input from a decoder (since subtitle filtering is not +supported yet). Every encoder is associated with some muxer's @emph{output +elementary stream} and sends its output to that muxer. + +A schematic representation of an encoder looks like this: +@verbatim + ┌─────────┐ + raw frames │ │ packets +────────────⮞│ encoder ├─────────⮞ + │ │ + └─────────┘ +@end verbatim + +@item +@emph{Muxers} (short for "multiplexers") receive encoded @emph{packets} for +their elementary streams from encoders (the @emph{transcoding} path) or directly +from demuxers (the @emph{streamcopy} path), interleave them (when there is more +than one elementary stream), and write the resulting bytes into the output file +(or pipe, network stream, etc.). + +A schematic representation of a muxer looks like this: +@verbatim + ┌──────────────────────┬───────────┐ + packets for stream 0 │ │ muxer │ +──────────────────────⮞│ elementary stream 0 ╞═══════════╡ + │ │ │ + ├──────────────────────┤ global │ + packets for stream 1 │ │properties │ +──────────────────────⮞│ elementary stream 1 │ and │ + │ │ metadata │ + ├──────────────────────┤ │ + │ │ │ + │ ........... │ │ + │ │ │ + ├──────────────────────┤ │ + packets for stream N │ │ │ +──────────────────────⮞│ elementary stream N │ │ + │ │ │ + └──────────────────────┴─────┬─────┘ + │ + write to file, network stream, │ + grabbing device, etc. │ + │ + ▼ +@end verbatim + +@end itemize + +@section Streamcopy +The simplest pipeline in @command{ffmpeg} is single-stream +@emph{streamcopy}, that is copying one @emph{input elementary stream}'s packets +without decoding, filtering, or encoding them. As an example, consider an input +file called @file{INPUT.mkv} with 3 elementary streams, from which we take the +second and write it to file @file{OUTPUT.mp4}. A schematic representation of +such a pipeline looks like this: +@verbatim +┌──────────┬─────────────────────┐ +│ demuxer │ │ unused +╞══════════╡ elementary stream 0 ├────────╳ +│ │ │ +│INPUT.mkv ├─────────────────────┤ ┌──────────────────────┬───────────┐ +│ │ │ packets │ │ muxer │ +│ │ elementary stream 1 ├─────────⮞│ elementary stream 0 ╞═══════════╡ +│ │ │ │ │OUTPUT.mp4 │ +│ ├─────────────────────┤ └──────────────────────┴───────────┘ +│ │ │ unused +│ │ elementary stream 2 ├────────╳ +│ │ │ +└──────────┴─────────────────────┘ +@end verbatim + +The above pipeline can be constructed with the following commandline: +@example +ffmpeg -i INPUT.mkv -map 0:1 -c copy OUTPUT.mp4 +@end example + +In this commandline +@itemize + +@item +there is a single input @file{INPUT.mkv}; + +@item +there are no input options for this input; + +@item +there is a single output @file{OUTPUT.mp4}; + +@item +there are two output options for this output: + +@itemize +@item +@code{-map 0:1} selects the input stream to be used - from input with index 0 +(i.e. the first one) the stream with index 1 (i.e. the second one); + +@item +@code{-c copy} selects the @code{copy} encoder, i.e. streamcopy with no decoding +or encoding. +@end itemize + +@end itemize + +Streamcopy is useful for changing the elementary stream count, container format, +or modifying container-level metadata. Since there is no decoding or encoding, +it is very fast and there is no quality loss. However, it might not work in some +cases because of a variety of factors (e.g. certain information required by the +target container is not available in the source). Applying filters is obviously +also impossible, since filters work on decoded frames. + +More complex streamcopy scenarios can be constructed - e.g. combining streams +from two input files into a single output: +@verbatim +┌──────────┬────────────────────┐ ┌────────────────────┬───────────┐ +│ demuxer 0│ │ packets │ │ muxer │ +╞══════════╡elementary stream 0 ├────────⮞│elementary stream 0 ╞═══════════╡ +│INPUT0.mkv│ │ │ │OUTPUT.mp4 │ +└──────────┴────────────────────┘ ├────────────────────┤ │ +┌──────────┬────────────────────┐ │ │ │ +│ demuxer 1│ │ packets │elementary stream 1 │ │ +╞══════════╡elementary stream 0 ├────────⮞│ │ │ +│INPUT1.aac│ │ └────────────────────┴───────────┘ +└──────────┴────────────────────┘ +@end verbatim +that can be built by the commandline +@example +ffmpeg -i INPUT0.mkv -i INPUT1.aac -map 0:0 -map 1:0 -c copy OUTPUT.mp4 +@end example + +The output @option{-map} option is used twice here, creating two streams in the +output file - one fed by the first input and one by the second. The single +instance of the @option{-c} option selects streamcopy for both of those streams. +You could also use multiple instances of this option together with +@ref{Stream specifiers} to apply different values to each stream, as will be +demonstrated in following sections. + +A converse scenario is splitting multiple streams from a single input into +multiple outputs: +@verbatim +┌──────────┬─────────────────────┐ ┌───────────────────┬───────────┐ +│ demuxer │ │ packets │ │ muxer 0 │ +╞══════════╡ elementary stream 0 ├─────────⮞│elementary stream 0╞═══════════╡ +│ │ │ │ │OUTPUT0.mp4│ +│INPUT.mkv ├─────────────────────┤ └───────────────────┴───────────┘ +│ │ │ packets ┌───────────────────┬───────────┐ +│ │ elementary stream 1 ├─────────⮞│ │ muxer 1 │ +│ │ │ │elementary stream 0╞═══════════╡ +└──────────┴─────────────────────┘ │ │OUTPUT1.mp4│ + └───────────────────┴───────────┘ +@end verbatim +built with +@example +ffmpeg -i INPUT.mkv -map 0:0 -c copy OUTPUT0.mp4 -map 0:1 -c copy OUTPUT1.mp4 +@end example +Note how a separate instance of the @option{-c} option is needed for every +output file even though their values are the same. This is because non-global +options (which is most of them) only apply in the context of the file before +which they are placed. + +These examples can of course be further generalized into arbitrary remappings +of any number of inputs into any number of outputs. + +@section Trancoding +@emph{Transcoding} is the process of decoding a stream and then encoding it +again. Since encoding tends to be computationally expensive and in most cases +degrades the stream quality (i.e. it is @emph{lossy}), you should only transcode +when you need to and perform streamcopy otherwise. Typical reasons to transcode +are: + +@itemize +@item +applying filters - e.g. resizing, deinterlacing, or overlaying video; resampling +or mixing audio; + +@item +you want to feed the stream to something that cannot decode the original codec. +@end itemize +Note that @command{ffmpeg} will transcode all audio, video, and subtitle streams +unless you specify @option{-c copy} for them. + +Consider an example pipeline that reads an input file with one audio and one +video stream, transcodes the video and copies the audio into a single output +file. This can be schematically represented as follows +@verbatim +┌──────────┬─────────────────────┐ +│ demuxer │ │ audio packets +╞══════════╡ stream 0 (audio) ├─────────────────────────────────────╮ +│ │ │ │ +│INPUT.mkv ├─────────────────────┤ video ┌─────────┐ raw │ +│ │ │ packets │ video │ video frames │ +│ │ stream 1 (video) ├─────────⮞│ decoder ├──────────────╮ │ +│ │ │ │ │ │ │ +└──────────┴─────────────────────┘ └─────────┘ │ │ + ▼ ▼ + │ │ +┌──────────┬─────────────────────┐ video ┌─────────┐ │ │ +│ muxer │ │ packets │ video │ │ │ +╞══════════╡ stream 0 (video) │⮜─────────┤ encoder ├──────────────╯ │ +│ │ │ │(libx264)│ │ +│OUTPUT.mp4├─────────────────────┤ └─────────┘ │ +│ │ │ │ +│ │ stream 1 (audio) │⮜────────────────────────────────────╯ +│ │ │ +└──────────┴─────────────────────┘ +@end verbatim +and implemented with the following commandline: +@example +ffmpeg -i INPUT.mkv -map 0:v -map 0:a -c:v libx264 -c:a copy OUTPUT.mp4 +@end example +Note how it uses stream specifiers @code{:v} and @code{:a} to select input +streams and apply different values of the @option{-c} option to them; see the +@ref{Stream specifiers} section for more details. -Encoded packets are then passed to the decoder (unless streamcopy is selected -for the stream, see further for a description). The decoder produces -uncompressed frames (raw video/PCM audio/...) which can be processed further by -filtering (see next section). After filtering, the frames are passed to the -encoder, which encodes them and outputs encoded packets. Finally, those are -passed to the muxer, which writes the encoded packets to the output file. @section Filtering -Before encoding, @command{ffmpeg} can process raw audio and video frames using -filters from the libavfilter library. Several chained filters form a filter -graph. @command{ffmpeg} distinguishes between two types of filtergraphs: -simple and complex. + +When transcoding, audio and video streams can be filtered before encoding, with +either a @emph{simple} or @emph{complex} filtergraph. @subsection Simple filtergraphs + Simple filtergraphs are those that have exactly one input and output, both of -the same type. In the above diagram they can be represented by simply inserting -an additional step between decoding and encoding: +the same type (audio or video). They are configured with the per-stream +@option{-filter} option (with @option{-vf} and @option{-af} aliases for +@option{-filter:v} (video) and @option{-filter:a} (audio) respectively). Note +that simple filtergraphs are tied to their output stream, so e.g. if you have +multiple audio streams, @option{-af} will create a separate filtergraph for each +one. +Taking the trancoding example from above, adding filtering (and omitting audio, +for clarity) makes it look like this: @verbatim - _________ ______________ -| | | | -| decoded | | encoded data | -| frames |\ _ | packets | -|_________| \ /||______________| - \ __________ / - simple _\|| | / encoder - filtergraph | filtered |/ - | frames | - |__________| - +┌──────────┬───────────────┐ +│ demuxer │ │ ┌─────────┐ +╞══════════╡ video stream │ packets │ video │ frames +│INPUT.mkv │ ├─────────⮞│ decoder ├─────⮞───╮ +│ │ │ └─────────┘ │ +└──────────┴───────────────┘ │ + ╭───────────⮜───────────╯ + │ ┌────────────────────────┐ + │ │ simple filtergraph │ + │ ╞════════════════════════╡ + │ │ ┌───────┐ ┌───────┐ │ + ╰──⮞├─⮞│ yadif ├─⮞│ scale ├─⮞├╮ + │ └───────┘ └───────┘ ││ + └────────────────────────┘│ + │ + │ +┌──────────┬───────────────┐ video ┌─────────┐ │ +│ muxer │ │ packets │ video │ │ +╞══════════╡ video stream │⮜─────────┤ encoder ├───────⮜───────╯ +│OUTPUT.mp4│ │ │ │ +│ │ │ └─────────┘ +└──────────┴───────────────┘ @end verbatim -Simple filtergraphs are configured with the per-stream @option{-filter} option -(with @option{-vf} and @option{-af} aliases for video and audio respectively). -A simple filtergraph for video can look for example like this: - -@verbatim - _______ _____________ _______ ________ -| | | | | | | | -| input | ---> | deinterlace | ---> | scale | ---> | output | -|_______| |_____________| |_______| |________| - -@end verbatim - -Note that some filters change frame properties but not frame contents. E.g. the -@code{fps} filter in the example above changes number of frames, but does not -touch the frame contents. Another example is the @code{setpts} filter, which -only sets timestamps and otherwise passes the frames unchanged. - @subsection Complex filtergraphs + Complex filtergraphs are those which cannot be described as simply a linear -processing chain applied to one stream. This is the case, for example, when the graph has -more than one input and/or output, or when output stream type is different from -input. They can be represented with the following diagram: - -@verbatim - _________ -| | -| input 0 |\ __________ -|_________| \ | | - \ _________ /| output 0 | - \ | | / |__________| - _________ \| complex | / -| | | |/ -| input 1 |---->| filter |\ -|_________| | | \ __________ - /| graph | \ | | - / | | \| output 1 | - _________ / |_________| |__________| -| | / -| input 2 |/ -|_________| - -@end verbatim - -Complex filtergraphs are configured with the @option{-filter_complex} option. -Note that this option is global, since a complex filtergraph, by its nature, -cannot be unambiguously associated with a single stream or file. - -The @option{-lavfi} option is equivalent to @option{-filter_complex}. +processing chain applied to one stream. This is the case, for example, when the +graph has more than one input and/or output, or when output stream type is +different from input. Complex filtergraphs are configured with the +@option{-filter_complex} option. Note that this option is global, since a +complex filtergraph, by its nature, cannot be unambiguously associated with a +single stream or file. Each instance of @option{-filter_complex} creates a new +complex filtergraph, and there can be any number of them. A trivial example of a complex filtergraph is the @code{overlay} filter, which has two video inputs and one video output, containing one video overlaid on top of the other. Its audio counterpart is the @code{amix} filter. -@section Stream copy -Stream copy is a mode selected by supplying the @code{copy} parameter to the -@option{-codec} option. It makes @command{ffmpeg} omit the decoding and encoding -step for the specified stream, so it does only demuxing and muxing. It is useful -for changing the container format or modifying container-level metadata. The -diagram above will, in this case, simplify to this: - -@verbatim - _______ ______________ ________ -| | | | | | -| input | demuxer | encoded data | muxer | output | -| file | ---------> | packets | -------> | file | -|_______| |______________| |________| - -@end verbatim - -Since there is no decoding or encoding, it is very fast and there is no quality -loss. However, it might not work in some cases because of many factors. Applying -filters is obviously also impossible, since filters work on uncompressed data. - +@anchor{Loopback decoders} @section Loopback decoders While decoders are normally associated with demuxer streams, it is also possible to create "loopback" decoders that decode the output from some encoder and allow