| Message ID | 2bd17759-8900-504f-b9b3-a6ad729d30ac@gmail.com |
|---|---|
| State | Superseded |
| Headers | show |
2018-05-24 15:48 GMT+02:00, Gyan Doshi <gyandoshi@gmail.com>: > Stream Selection chapter rewritten to be more > comprehensive and illustrative. Because of the length of the new description I wonder if the existing documentation should be kept as a "short version" or similar. Carl Eugen
On Thu, May 24, 2018 at 07:18:50PM +0530, Gyan Doshi wrote: > Stream Selection chapter rewritten to be more comprehensive and > illustrative. This is very detailed. I think this could byte us if we want to change some of the logic. Maybe this or some of this should be marked as being a description of the current heuristics and not a description of the command line interface. So we can change it and script authors know they should not write code depending on the fine details ... [...]
On 25-05-2018 06:08 AM, Michael Niedermayer wrote: > This is very detailed. I think this could byte us if we want to change some > of the logic. The current chapter already does. Many askers at SE sites and even on the User ML show confusion over command line operation, including stream selection. Many of these users don't read docs in the first place, but those who do, will have an accurate picture; and for those who don't, there will be a link to point to, without having to regurgitate the same quirks over and over again. Much of the current behavior is not documented anywhere and this is the best place for it. The rewrite turned out longer that I anticipated as I checked the code for gotchas of which there are a few. The burden is on doc maintainers, like me, to track code changes and update. But that shouldn't be a reason to not document the behavior. Regards, Gyan
On 24-05-2018 09:58 PM, Carl Eugen Hoyos wrote: > Because of the length of the new description I wonder if the > existing documentation should be kept as a "short version" > or similar. Parts of the short version which are accurate have been retained; other parts are incomplete or misleading and so have been rewritten, qualified (and illustrated). Regards, Gyan
On Thu, 24 May 2018 19:18:50 +0530 Gyan Doshi <gyandoshi@gmail.com> wrote: > From a7ff32bdac54a1f257482526a507ab7823cafb41 Mon Sep 17 00:00:00 2001 > From: Gyan Doshi <ffmpeg@gyani.pro> > Date: Thu, 24 May 2018 19:11:00 +0530 > Subject: [PATCH] doc/ffmpeg - rewrite Stream Selection chapter Nit: Use : instead of - in the subject. > Flesh out with details and examples to show quirks and limitations. > --- > doc/ffmpeg.texi | 170 ++++++++++++++++++++++++++++++++++++++++++++++++++++---- > 1 file changed, 160 insertions(+), 10 deletions(-) > > diff --git a/doc/ffmpeg.texi b/doc/ffmpeg.texi > index 88dbdeb95a..804bcc73ce 100644 > --- a/doc/ffmpeg.texi > +++ b/doc/ffmpeg.texi > @@ -216,16 +216,166 @@ filters is obviously also impossible, since filters work on uncompressed data. > @chapter Stream selection > @c man begin STREAM SELECTION > > -By default, @command{ffmpeg} includes only one stream of each type (video, audio, subtitle) > -present in the input files and adds them to each output file. It picks the > -"best" of each based upon the following criteria: for video, it is the stream > -with the highest resolution, for audio, it is the stream with the most channels, for > -subtitles, it is the first subtitle stream. In the case where several streams of > -the same type rate equally, the stream with the lowest index is chosen. > - > -You can disable some of those defaults by using the @code{-vn/-an/-sn/-dn} options. For > -full manual control, use the @code{-map} option, which disables the defaults just > -described. Due to the length of the new chapter it could be helpful to summarize the most important aspects in the first paragraph then follow with the more detailed documentation. Or at least adding subsections may make reduce wall of text fatigue: I added some suggestions below. > +The @code{-map} option should be used for manual control of stream selection for each > +output file. When used, only user-mapped streams are included in that output file, > +with one possible exception described below. Please name what is involved in the exception instead of just mentioning that there is one so the connection can be more easily made in the following paragraphs. > +In the absence of any map options for a particular output file, ffmpeg inspects the output > +format to check which type of streams can be included in it, viz. video, audio and/or > +subtitles. For each acceptable stream type, ffmpeg will pick one stream, when available, > +from among all the inputs. It will select that stream based upon the following criteria: Using a list here for the stream selection criteria may make it easier to read and break up the wall of text, but I didn't try. Maybe it would be ugly. > +for video, it is the stream with the highest resolution, for audio, it is the stream with > +the most channels, for subtitles, it is the first subtitle stream found but there's a caveat. > +The output format's default subtitle encoder may be text-based or image-based, and only a > +subtitle stream of the same type can be chosen. Data or attachment streams are not > +automatically selected and can only be included using @code{-map}. In the case where several > +streams of the same type rate equally, the stream with the lowest index is chosen. > + > +The @code{-vn}, @code{-an}, @code{-sn} options can be used to skip automatic stream selection > +for video, audio, and subtitle streams respectively. @subsection Complex filtergraphs > +If there are any complex filtergraph output streams with unlabeled pads, they will be added > +to the first output file. This will lead to a fatal error if the stream type is not supported > +by the output format. In the absence of the map option, the inclusion of these streams leads > +to the automatic stream selection of their types being skipped. If map options are present, > +these filtergraph streams are included in addition to the mapped streams. > + > +Complex filtergraph output streams with labeled pads must be mapped once and exactly once. @subsection Stream handling > +Stream handling is independent of stream selection, with an exception for subtitles described > +below. Stream handling is set via the @code{-codec} option addressed to streams within a > +specific @emph{output} file. In particular, codec options are applied by ffmpeg after the > +stream selection process and thus do not influence the latter. An exception exists for subtitles. > +If a subtitle encoder is specified for an output file, the first subtitle stream found, of type > +text or image, will be included. ffmpeg does not validate if the specified encoder can convert > +the selected stream or if the converted stream is acceptable within the output format. This > +applies generally as well: when the user sets an encoder manually, the stream selection process > +cannot check if the encoded stream can be muxed into the output file. If it cannot, ffmpeg will > +abort and @emph{all} output files will fail to be processed. > + > +@section Examples > + > +The following examples illustrate the behavior, quirks and limitations of ffmpeg's automatic > +stream selection methods. > + > +They assume the following three input files. > + > +@verbatim > + > +input file 'A.avi' > + stream 0: video 640x360 > + stream 1: audio 2 channels > + > +input file 'B.mp4' > + stream 0: video 1920x1080 > + stream 1: audio 2 channels > + stream 2: subtitles (text) > + stream 3: audio 5.1 channels > + stream 4: subtitles (text) > + > +input file 'C.mkv' > + stream 0: video 1280x720 > + stream 1: audio 2 channels > + stream 2: subtitles (image) > +@end verbatim > + > +@* Examples are often * itemized by convention, but the resulting text justification such verbose examples makes them hard to read in HTML format. I would ditch the itemization. The examples that exhibit the behavior of the automatic stream selection are informative, but perhaps too bewildering, technical, or intimidating for users. I don't really have a good recommendation, but adding titles for each example can help users understand what they are getting into, or maybe simplify or prune this section a little. Or maybe something like this is a better fit for the wiki. As it is I don't envision general users reading this section as is.
Hi Lou, Most sectioning and formatting suggestions applied locally. On 26-05-2018 03:50 AM, Lou Logan wrote: > Examples are often * itemized by convention, but the resulting text > justification such verbose examples makes them hard to read in HTML > format. I would ditch the itemization. Part of the ugliness is due to how the code fragments are rendered: with vertical margins, creating uneven line spacing. Do you mind if I reduce or eliminate those margins? Regards, Gyan
On Sat, May 26, 2018, at 12:33 AM, Gyan Doshi wrote: > > Part of the ugliness is due to how the code fragments are rendered: with > vertical margins, creating uneven line spacing. Do you mind if I reduce > or eliminate those margins? Which file sets those margins? The ugly justified text in the HTML doc that I was referring to before appears to be due to "body {text-align: justify;}" in style.min.css. I don't see any value to that at the moment, but maybe I'll take a closer look some other time and we can save this discussion for later since it should be dealt with separately from this patch. So disregarding the "ugly rendering" for now if you provide an updated patch with your local changes I'll take another look on Monday or Tuesday.
diff --git a/doc/ffmpeg.texi b/doc/ffmpeg.texi index 88dbdeb95a..804bcc73ce 100644 --- a/doc/ffmpeg.texi +++ b/doc/ffmpeg.texi @@ -216,16 +216,166 @@ filters is obviously also impossible, since filters work on uncompressed data. @chapter Stream selection @c man begin STREAM SELECTION -By default, @command{ffmpeg} includes only one stream of each type (video, audio, subtitle) -present in the input files and adds them to each output file. It picks the -"best" of each based upon the following criteria: for video, it is the stream -with the highest resolution, for audio, it is the stream with the most channels, for -subtitles, it is the first subtitle stream. In the case where several streams of -the same type rate equally, the stream with the lowest index is chosen. - -You can disable some of those defaults by using the @code{-vn/-an/-sn/-dn} options. For -full manual control, use the @code{-map} option, which disables the defaults just -described. +The @code{-map} option should be used for manual control of stream selection for each +output file. When used, only user-mapped streams are included in that output file, +with one possible exception described below. + +In the absence of any map options for a particular output file, ffmpeg inspects the output +format to check which type of streams can be included in it, viz. video, audio and/or +subtitles. For each acceptable stream type, ffmpeg will pick one stream, when available, +from among all the inputs. It will select that stream based upon the following criteria: +for video, it is the stream with the highest resolution, for audio, it is the stream with +the most channels, for subtitles, it is the first subtitle stream found but there's a caveat. +The output format's default subtitle encoder may be text-based or image-based, and only a +subtitle stream of the same type can be chosen. Data or attachment streams are not +automatically selected and can only be included using @code{-map}. In the case where several +streams of the same type rate equally, the stream with the lowest index is chosen. + +The @code{-vn}, @code{-an}, @code{-sn} options can be used to skip automatic stream selection +for video, audio, and subtitle streams respectively. + +If there are any complex filtergraph output streams with unlabeled pads, they will be added +to the first output file. This will lead to a fatal error if the stream type is not supported +by the output format. In the absence of the map option, the inclusion of these streams leads +to the automatic stream selection of their types being skipped. If map options are present, +these filtergraph streams are included in addition to the mapped streams. + +Complex filtergraph output streams with labeled pads must be mapped once and exactly once. + +Stream handling is independent of stream selection, with an exception for subtitles described +below. Stream handling is set via the @code{-codec} option addressed to streams within a +specific @emph{output} file. In particular, codec options are applied by ffmpeg after the +stream selection process and thus do not influence the latter. An exception exists for subtitles. +If a subtitle encoder is specified for an output file, the first subtitle stream found, of type +text or image, will be included. ffmpeg does not validate if the specified encoder can convert +the selected stream or if the converted stream is acceptable within the output format. This +applies generally as well: when the user sets an encoder manually, the stream selection process +cannot check if the encoded stream can be muxed into the output file. If it cannot, ffmpeg will +abort and @emph{all} output files will fail to be processed. + +@section Examples + +The following examples illustrate the behavior, quirks and limitations of ffmpeg's automatic +stream selection methods. + +They assume the following three input files. + +@verbatim + +input file 'A.avi' + stream 0: video 640x360 + stream 1: audio 2 channels + +input file 'B.mp4' + stream 0: video 1920x1080 + stream 1: audio 2 channels + stream 2: subtitles (text) + stream 3: audio 5.1 channels + stream 4: subtitles (text) + +input file 'C.mkv' + stream 0: video 1280x720 + stream 1: audio 2 channels + stream 2: subtitles (image) +@end verbatim + +@* +@itemize +@item +@example +ffmpeg -i A.avi -i B.mp4 out1.mkv out2.wav -map 1:a -c:a copy out3.mov +@end example +There are three output files specified, and for the first two, no @code{-map} options +are set, so ffmpeg will select streams for these two files automatically. + +@code{out1.mkv} is a Matroska container file and accepts video, audio and subtitle streams, +so ffmpeg will try to select one of each type. +For video, it will select @code{stream 0} from @code{B.mp4}, which has the highest +resolution among all the input video streams. +For audio, it will select @code{stream 3} from @code{B.mp4}, since it has the greatest +number of channels. +For subtitles, it will select @code{stream 2} from @code{B.mp4}, which is the first subtitle +stream from among @code{A.avi} and @code{B.mp4}. + +@code{out2.wav} accepts only audio streams, so only @code{stream 3} from @code{B.mp4} is +selected. + +For @code{out3.mov}, since a @code{-map} option is set, no automatic stream selection will +occur. The @code{-map 1:a} will select all audio streams from the second input @code{B.mp4}. +No other streams will be included in this output file. + +For the first two outputs, all included streams will be transcoded. The encoders chosen will +be the default ones registered by each output format, which may not match the codec of the +selected input streams. For the third output, codec option for audio streams has been set +to @code{copy}, so no decoding-filtering-encoding operations will occur, or @emph{can} occur. +Packets of selected streams shall be conveyed from the input file and muxed within the output +file. + +@item +@example +ffmpeg -i C.mkv out1.mkv -c:s dvdsub -an out2.mkv +@end example +Although @code{out1.mkv} is a Matroska container file which accepts subtitle streams, only a +video and audio stream shall be selected. The subtitle stream of @code{C.mkv} is image-based +and the default subtitle encoder of the Matroska muxer is text-based, so a transcode operation +for the subtitles is expected to fail and hence the stream isn't selected. However, in +@code{out2.mkv}, a subtitle encoder is specified in the command and so, the subtitle stream is +selected, in addition to the video stream. The presence of @code{-an} disables audio stream +selection for @code{out2.mkv}. + +@item +@example +ffmpeg -i A.avi -i C.mkv -i B.mp4 -filter_complex "overlay" out1.mp4 out2.srt +@end example +A filtergraph is setup here using the @code{-filter_complex} option and consists of a single +video filter. The @code{overlay} filter requires exactly two video inputs, but none are +specified, so the first two available video streams are used, those of @code{A.avi} and +@code{C.mkv}. The output pad of the filter has no label and so is sent to the first output file +@code{out1.mp4}. Due to this, automatic selection of the video stream is skipped, which would +have selected the stream in @code{B.mp4}. The audio stream with most channels viz. @code{stream 3} +in @code{B.mp4}, is chosen automatically. No subtitle stream is chosen however, since the MP4 +format has no default subtitle encoder registered, and the user hasn't specified a subtitle encoder. + +The 2nd output file, @code{out2.srt}, only accepts text-based subtitle streams. So, even though +the first subtitle stream available belongs to @code{C.mkv}, it is image-based and hence skipped. +The selected stream, @code{stream 2} in @code{B.mp4}, is the first text-based subtitle stream. + +@item +@example +ffmpeg -i A.avi -i B.mp4 -i C.mkv -filter_complex "[1:v]hue=s=0[outv];overlay;aresample" -map '[outv]' -an out1.mp4 out2.mkv -map '[outv]' -map 1:a:0 out3.mkv +@end example + +This command will fail, as the output pad labelled @code{[outv]} has been mapped twice. +No output files shall be processed. + +@example +ffmpeg -i A.avi -i B.mp4 -i C.mkv -filter_complex "[1:v]hue=s=0[outv];overlay;aresample" -an out1.mp4 out2.mkv -map 1:a:0 out3.mkv +@end example + +This command will also fail as the hue filter output is labelled (@code{[outv]}) and hasn't been mapped anywhere. + +The command should be modified as follows, +@example +ffmpeg -i A.avi -i B.mp4 -i C.mkv -filter_complex "[1:v]hue=s=0,split=2[outv1][outv2];overlay;aresample" -map '[outv1]' -an out1.mp4 out2.mkv -map '[outv2]' -map 1:a:0 out3.mkv +@end example +The video stream from @code{B.mp4} is sent to the hue filter, whose output is cloned once, +and both outputs labelled. Then a copy each is mapped to the first and third output files. +The overlay filter, requiring two video inputs, uses the first two unused video streams. +Those are the streams from @code{A.avi} and @code{C.mkv}. The overlay output isn't labelled, +so it is sent to the first output file @code{out1.mp4}, regardless of the presence of the +@code{-map} option. The aresample filter is sent the first unused audio stream, that of +@code{A.avi}. Since this filter output is also unlabelled, it too is mapped to the first output +file. The presence of @code{-an} only suppresses automatic or manual stream selection of audio +streams, not outputs sent from filtergraphs. Both these mapped streams shall be ordered before +the mapped stream in @code{out1.mp4}. + +The video, audio and subtitle streams mapped to @code{out2.mkv} are entirely determined by +automatic stream selection. + +@code{out3.mkv} consists of the video output from the hue filter and the first audio stream from +@code{B.mp4}. +@end itemize +@* @c man end STREAM SELECTION
Stream Selection chapter rewritten to be more comprehensive and illustrative. From a7ff32bdac54a1f257482526a507ab7823cafb41 Mon Sep 17 00:00:00 2001 From: Gyan Doshi <ffmpeg@gyani.pro> Date: Thu, 24 May 2018 19:11:00 +0530 Subject: [PATCH] doc/ffmpeg - rewrite Stream Selection chapter Flesh out with details and examples to show quirks and limitations. --- doc/ffmpeg.texi | 170 ++++++++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 160 insertions(+), 10 deletions(-)