[FFmpeg-devel,v2] doc/filters: document vf_libplacebo

From: Niklas Haas <git@haasn.dev>

Signed-off-by: Niklas Haas <git@haasn.dev>
---
Changes in v2:
- expand documentation of tone mapping curves
- slight rewording of some sections
- add more examples
---
 doc/filters.texi | 494 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 494 insertions(+)

Applied as 234c824820d4c17612c9745e74ef6c934679d138 (with minor changes)

On Thu, 31 Mar 2022 13:34:30 +0200 Niklas Haas <ffmpeg@haasn.xyz> wrote:
> From: Niklas Haas <git@haasn.dev>
> 
> Signed-off-by: Niklas Haas <git@haasn.dev>
> ---
> Changes in v2:
> - expand documentation of tone mapping curves
> - slight rewording of some sections
> - add more examples
> ---
>  doc/filters.texi | 494 +++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 494 insertions(+)
> 
> diff --git a/doc/filters.texi b/doc/filters.texi
> index 1d56d24819..a6f2f1397e 100644
> --- a/doc/filters.texi
> +++ b/doc/filters.texi
> @@ -14793,6 +14793,500 @@ ffmpeg -i input.mov -vf lensfun=make=Canon:model="Canon EOS 100D":lens_model="Ca
>  
>  @end itemize
>  
> +@section libplacebo
> +
> +Flexible GPU-accelerated processing filter based on libplacebo
> +(@url{https://code.videolan.org/videolan/libplacebo}). Note that this filter
> +currently only accepts Vulkan input frames.
> +
> +@subsection Options
> +
> +The options for this filter are divided into the following sections:
> +
> +@subsubsection Output mode
> +These options control the overall output mode. By default, libplacebo will try
> +to preserve the source colorimetry and size as best as it can, but it will
> +apply any embedded film grain, dolby vision metadata or anamorphic SAR present
> +in source frames.
> +@table @option
> +@item w
> +@item h
> +Set the output video dimension expression. Default value is the input dimension.
> +
> +Allows for the same expressions as the @ref{scale} filter.
> +
> +@item format
> +Set the output format override. If unset (the default), frames will be output
> +in the same format as the respective input frames. Otherwise, format conversion
> +will be performed.
> +
> +@item force_original_aspect_ratio
> +@item force_divisible_by
> +Work the same as the identical @ref{scale} filter options.
> +
> +@item normalize_sar
> +If enabled (the default), output frames will always have a pixel aspect ratio
> +of 1:1. If disabled, any aspect ratio mismatches, including those from e.g.
> +anamorphic video sources, are forwarded to the output pixel aspect ratio.
> +
> +@item pad_crop_ratio
> +Specifies a ratio (between @code{0.0} and @code{1.0}) between padding and
> +cropping when the input aspect ratio does not match the output aspect ratio and
> +@option{normalize_sar} is in effect. The default of @code{0.0} always pads the
> +content with black borders, while a value of @code{1.0} always crops off parts
> +of the content. Intermediate values are possible, leading to a mix of the two
> +approaches.
> +
> +@item colorspace
> +@item color_primaries
> +@item color_trc
> +@item range
> +Configure the colorspace that output frames will be delivered in. The default
> +value of @code{auto} outputs frames in the same format as the input frames,
> +leading to no change. For any other value, conversion will be performed.
> +
> +See the @ref{setparams} filter for a list of possible values.
> +
> +@item apply_filmgrain
> +Apply film grain (e.g. AV1 or H.274) if present in source frames, and strip
> +it from the output. Enabled by default.
> +
> +@item apply_dolbyvision
> +Apply Dolby Vision RPU metadata if present in source frames, and strip it from
> +the output. Enabled by default. Note that Dolby Vision will always output
> +BT.2020+PQ, overriding the usual input frame metadata. These will also be
> +picked as the values of @code{auto} for the respective frame output options.
> +@end table
> +
> +@subsubsection Scaling
> +The options in this section control how libplacebo performs upscaling and (if
> +necessary) downscaling. Note that libplacebo will always internally operate on
> +4:4:4 content, so any sub-sampled chroma formats such as @code{yuv420p} will
> +necessarily be upsampled and downsampled as part of the rendering process. That
> +means scaling might be in effect even if the source and destination resolution
> +are the same.
> +@table @option
> +@item upscaler
> +@item downscaler
> +Configure the filter kernel used for upscaling and downscaling. The respective
> +defaults are @code{spline36} and @code{mitchell}. For a full list of possible
> +values, pass @code{help} to these options. The most important values are:
> +@table @samp
> +
> +@item none
> +Forces the use of built-in GPU texture sampling (typically bilinear). Extremely
> +fast but poor quality, especially when downscaling.
> +
> +@item bilinear
> +Bilinear interpolation. Can generally be done for free on GPUs, except when
> +doing so would lead to aliasing. Fast and low quality.
> +
> +@item nearest
> +Nearest-neighbour interpolation. Sharp but highly aliasing.
> +
> +@item oversample
> +Algorithm that looks visually similar to nearest-neighbour interpolation but
> +tries to preserve pixel aspect ratio. Good for pixel art, since it results in
> +minimal distortion of the artistic appearance.
> +
> +@item lanczos
> +Standard sinc-sinc interpolation kernel.
> +
> +@item spline36
> +Cubic spline approximation of lanczos. No difference in performance, but has
> +very slightly less ringing.
> +
> +@item ewa_lanczos
> +Elliptically weighted average version of lanczos, based on a jinc-sinc kernel.
> +This is also popularly referred to as just "Jinc scaling". Slow but very high
> +quality.
> +
> +@item gaussian
> +Gaussian kernel. Has certain ideal mathematical properties, but subjectively
> +very blurry.
> +
> +@item mitchell
> +Cubic BC spline with parameters recommended by Mitchell and Netravali. Very
> +little ringing.
> +@end table
> +
> +@item lut_entries
> +Configures the size of scaler LUTs, ranging from @code{1} to @code{256}. The
> +default of @code{0} will pick libplacebo's internal default, typically
> +@code{64}.
> +
> +@item antiringing
> +Enables anti-ringing (for non-EWA filters). The value (between @code{0.0} and
> +@code{1.0}) configures the strength of the anti-ringing algorithm. May increase
> +aliasing if set too high. Disabled by default.
> +
> +@item sigmoid
> +Enable sigmoidal compression during upscaling. Reduces ringing slightly.
> +Enabled by default.
> +@end table
> +
> +@subsubsection Debanding
> +Libplacebo comes with a built-in debanding filter that is good at counteracting
> +many common sources of banding and blocking. Turning this on is highly
> +recommended whenever quality is desired.
> +@table @option
> +@item deband
> +Enable (fast) debanding algorithm. Disabled by default.
> +
> +@item deband_iterations
> +Number of deband iterations of the debanding algorithm. Each iteration is
> +performed with progressively increased radius (and diminished threshold).
> +Recommended values are in the range @code{1} to @code{4}. Defaults to @code{1}.
> +
> +@item deband_threshold
> +Debanding filter strength. Higher numbers lead to more aggressive debanding.
> +Defaults to @code{4.0}.
> +
> +@item deband_radius
> +Debanding filter radius. A higher radius is better for slow gradients, while
> +a lower radius is better for steep gradients. Defaults to @code{16.0}.
> +
> +@item deband_grain
> +Amount of extra output grain to add. Helps hide imperfections. Defaults to
> +@code{6.0}.
> +@end table
> +
> +@subsubsection Color adjustment
> +A collection of subjective color controls. Not very rigorous, so the exact
> +effect will vary somewhat depending on the input primaries and colorspace.
> +@table @option
> +@item brightness
> +Brightness boost, between @code{-1.0} and @code{1.0}. Defaults to @code{0.0}.
> +
> +@item contrast
> +Contrast gain, between @code{0.0} and @code{16.0}. Defaults to @code{1.0}.
> +
> +@item saturation
> +Saturation gain, between @code{0.0} and @code{16.0}. Defaults to @code{1.0}.
> +
> +@item hue
> +Hue shift in radians, between @code{-3.14} and @code{3.14}. Defaults to
> +@code{0.0}. This will rotate the UV subvector, defaulting to BT.709
> +coefficients for RGB inputs.
> +
> +@item gamma
> +Gamma adjustment, between @code{0.0} and @code{16.0}. Defaults to @code{1.0}.
> +
> +@item cones
> +Cone model to use for color blindness simulation. Accepts any combination of
> +@code{l}, @code{m} and @code{s}. Here are some examples:
> +@table @samp
> +@item m
> +Deuteranomaly / deuteranopia (affecting 3%-4% of the population)
> +@item l
> +Protanomaly / protanopia (affecting 1%-2% of the population)
> +@item l+m
> +Monochromacy (very rare)
> +@item l+m+s
> +Achromatopsy (complete loss of daytime vision, extremely rare)
> +@end table
> +
> +@item cone-strength
> +Gain factor for the cones specified by @code{cones}, between @code{0.0} and
> +@code{10.0}. A value of @code{1.0} results in no change to color vision. A
> +value of @code{0.0} (the default) simulates complete loss of those cones. Values
> +above @code{1.0} result in exaggerating the differences between cones, which
> +may help compensate for reduced color vision.
> +@end table
> +
> +@subsubsection Peak detection
> +To help deal with sources that only have static HDR10 metadata (or no tagging
> +whatsoever), libplacebo uses its own internal frame analysis compute shader to
> +analyze source frames and adapt the tone mapping function in realtime. If this
> +is too slow, or if exactly reproducible frame-perfect results are needed, it's
> +recommended to turn this feature off.
> +@table @option
> +@item peak_detect
> +Enable HDR peak detection. Ignores static MaxCLL/MaxFALL values in favor of
> +dynamic detection from the input. Note that the detected values do not get
> +written back to the output frames, they merely guide the internal tone mapping
> +process. Enabled by default.
> +
> +@item smoothing_period
> +Peak detection smoothing period, between @code{0.0} and @code{1000.0}. Higher
> +values result in peak detection becoming less responsive to changes in the
> +input. Defaults to @code{100.0}.
> +
> +@item minimum_peak
> +Lower bound on the detected peak (relative to SDR white), between @code{0.0}
> +and @code{100.0}. Defaults to @code{1.0}.
> +
> +@item scene_threshold_low
> +@item scene_threshold_high
> +Lower and upper thresholds for scene change detection. Expressed in a
> +logarithmic scale between @code{0.0} and @code{100.0}. Default to @code{5.5}
> +and @code{10.0}, respectively. Setting either to a negative value disables
> +this functionality.
> +
> +@item overshoot
> +Peak smoothing overshoot margin, between @code{0.0} and @code{1.0}. Provides a
> +safety margin to prevent clipping as a result of peak smoothing. Defaults to
> +@code{0.05}, corresponding to a margin of 5%.
> +@end table
> +
> +@subsubsection Tone mapping
> +The options in this section control how libplacebo performs tone-mapping and
> +gamut-mapping when dealing with mismatches between wide-gamut or HDR content.
> +In general, libplacebo relies on accurate source tagging and mastering display
> +gamut information to produce the best results.
> +@table @option
> +@item intent
> +Rendering intent to use when adapting between different primary color gamuts
> +(after tone-mapping).
> +@table @samp
> +@item perceptual
> +Perceptual gamut mapping. Currently equivalent to relative colorimetric.
> +@item relative
> +Relative colorimetric. This is the default.
> +@item absolute
> +Absolute colorimetric.
> +@item saturation
> +Saturation mapping. Forcibly stretches the source gamut to the target gamut.
> +@end table
> +
> +@item gamut_mode
> +How to handle out-of-gamut colors that can occur as a result of colorimetric
> +gamut mapping.
> +@table @samp
> +@item clip
> +Do nothing, simply clip out-of-range colors to the RGB volume. This is the
> +default.
> +@item warn
> +Highlight out-of-gamut pixels (by coloring them pink).
> +@item darken
> +Linearly reduces content brightness to preserves saturated details, followed by
> +clipping the remaining out-of-gamut colors. As the name implies, this makes
> +everything darker, but provides a good balance between preserving details and
> +colors.
> +@item desaturate
> +Hard-desaturates out-of-gamut colors towards white, while preserving the
> +luminance. Has a tendency to shift colors.
> +@end table
> +
> +@item tonemapping
> +Tone-mapping algorithm to use. Available values are:
> +@table @samp
> +@item auto
> +Automatic selection based on internal heuristics. This is the default.
> +@item clip
> +Performs no tone-mapping, just clips out-of-range colors. Retains perfect color
> +accuracy for in-range colors but completely destroys out-of-range information.
> +Does not perform any black point adaptation. Not configurable.
> +@item bt.2390
> +EETF from the ITU-R Report BT.2390, a hermite spline roll-off with linear
> +segment. The knee point offset is configurable. Note that this parameter
> +defaults to @code{1.0}, rather than the value of @code{0.5} from the ITU-R
> +spec.
> +@item bt.2446a
> +EETF from ITU-R Report BT.2446, method A. Designed for well-mastered HDR
> +sources. Can be used for both forward and inverse tone mapping. Not
> +configurable.
> +@item spline
> +Simple spline consisting of two polynomials, joined by a single pivot point.
> +The parameter gives the pivot point (in PQ space), defaulting to @code{0.30}.
> +Can be used for both forward and inverse tone mapping.
> +@item reinhard
> +Simple non-linear, global tone mapping algorithm. The parameter specifies the
> +local contrast coefficient at the display peak. Essentially, a parameter of
> +@code{0.5} implies that the reference white will be about half as bright as
> +when clipping. Defaults to @code{0.5}, which results in the simplest
> +formulation of this function.
> +@item mobius
> +Generalization of the reinhard tone mapping algorithm to support an additional
> +linear slope near black. The tone mapping parameter indicates the trade-off
> +between the linear section and the non-linear section. Essentially, for a given
> +parameter @var{x}, every color value below @var{x} will be mapped linearly,
> +while higher values get non-linearly tone-mapped. Values near @code{1.0} make
> +this curve behave like @code{clip}, while values near @code{0.0} make this
> +curve behave like @code{reinhard}. The default value is @code{0.3}, which
> +provides a good balance between colorimetric accuracy and preserving
> +out-of-gamut details.
> +@item hable
> +Piece-wise, filmic tone-mapping algorithm developed by John Hable for use in
> +Uncharted 2, inspired by a similar tone-mapping algorithm used by Kodak.
> +Popularized by its use in video games with HDR rendering. Preserves both dark
> +and bright details very well, but comes with the drawback of changing the
> +average brightness quite significantly. This is sort of similar to
> +@code{reinhard} with parameter @code{0.24}.
> +@item gamma
> +Fits a gamma (power) function to transfer between the source and target color
> +spaces, effectively resulting in a perceptual hard-knee joining two roughly
> +linear sections. This preserves details at all scales fairly accurately, but
> +can result in an image with a muted or dull appearance. The parameter is used
> +as the cutoff point, defaulting to @code{0.5}.
> +@item linear
> +Linearly stretches the input range to the output range, in PQ space. This will
> +preserve all details accurately, but results in a significantly different
> +average brightness. Can be used for inverse tone-mapping in addition to regular
> +tone-mapping. The parameter can be used as an additional linear gain
> +coefficient (defaulting to @code{1.0}).
> +@end table
> +
> +@item tonemapping_param
> +For tunable tone mapping functions, this parameter can be used to fine-tune the
> +curve behavior. Refer to the documentation of @code{tonemapping}. The default
> +value of @code{0.0} is replaced by the curve's preferred default setting.
> +
> +@item tonemapping_mode
> +This option determines how the tone mapping function specified by
> +@code{tonemapping} is applied to the colors in a scene. Possible values are:
> +@table @samp
> +@item auto
> +Automatic selection based on internal heuristics. This is the default.
> +@item rgb
> +Apply the function per-channel in the RGB colorspace.
> +Per-channel tone-mapping in RGB. Guarantees no clipping and heavily desaturates
> +the output, but distorts the colors quite significantly. Very similar to the
> +"Hollywood" look and feel.
> +@item max
> +Tone-mapping is performed on the brightest component found in the signal. Good
> +at preserving details in highlights, but has a tendency to crush blacks.
> +@item hybrid
> +Tone-map per-channel for highlights and linearly (luma-based) for
> +midtones/shadows, based on a fixed gamma @code{2.4} coefficient curve.
> +@item luma
> +Tone-map linearly on the luma component (CIE Y), and adjust (desaturate) the
> +chromaticities to compensate using a simple constant factor. This is
> +essentially the mode used in ITU-R BT.2446 method A.
> +@end table
> +
> +@item inverse_tonemapping
> +If enabled, this filter will also attempt stretching SDR signals to fill HDR
> +output color volumes. Disabled by default.
> +
> +@item tonemapping_crosstalk
> +Extra tone-mapping crosstalk factor, between @code{0.0} and @code{0.3}. This
> +can help reduce issues tone-mapping certain bright spectral colors. Defaults to
> +@code{0.04}.
> +
> +@item tonemapping_lut_size
> +Size of the tone-mapping LUT, between @code{2} and @code{1024}. Defaults to
> +@code{256}. Note that this figure is squared when combined with
> +@code{peak_detect}.
> +@end table
> +
> +@subsubsection Dithering
> +By default, libplacebo will dither whenever necessary, which includes rendering
> +to any integer format below 16-bit precision. It's recommended to always leave
> +this on, since not doing so may result in visible banding in the output, even
> +if the @code{debanding} filter is enabled. If maximum performance is needed,
> +use @code{ordered_fixed} instead of disabling dithering.
> +@table @option
> +@item dithering
> +Dithering method to use. Accepts the following values:
> +@table @samp
> +@item none
> +Disables dithering completely. May result in visible banding.
> +@item blue
> +Dither with pseudo-blue noise. This is the default.
> +@item ordered
> +Tunable ordered dither pattern.
> +@item ordered_fixed
> +Faster ordered dither with a fixed size of @code{6}. Texture-less.
> +@item white
> +Dither with white noise. Texture-less.
> +@end table
> +
> +@item dither_lut_size
> +Dither LUT size, as log base2 between @code{1} and @code{8}. Defaults to
> +@code{6}, corresponding to a LUT size of @code{64x64}.
> +
> +@item dither_temporal
> +Enables temporal dithering. Disabled by default.
> +@end table
> +
> +@subsubsection Custom shaders
> +libplacebo supports a number of custom shaders based on the mpv .hook GLSL
> +syntax. A collection of such shaders can be found here:
> +@url{https://github.com/mpv-player/mpv/wiki/User-Scripts#user-shaders}
> +
> +A full description of the mpv shader format is beyond the scope of this
> +section, but a summary can be found here:
> +@url{https://mpv.io/manual/master/#options-glsl-shader}
> +@table @option
> +@item custom_shader_path
> +Specifies a path to a custom shader file to load at runtime.
> +
> +@item custom_shader_bin
> +Specifies a complete custom shader as a raw string.
> +@end table
> +
> +@subsubsection Debugging / performance
> +All of the options in this section default off. They may be of assistance when
> +attempting to squeeze the maximum performance at the cost of quality.
> +@table @option
> +@item skip_aa
> +Disable anti-aliasing when downscaling.
> +
> +@item polar_cutoff
> +Truncate polar (EWA) scaler kernels below this absolute magnitude, between
> +@code{0.0} and @code{1.0}.
> +
> +@item disable_linear
> +Disable linear light scaling.
> +
> +@item disable_builtin
> +Disable built-in GPU sampling (forces LUT).
> +
> +@item force_icc_lut
> +Force the use of a full ICC 3DLUT for gamut mapping.
> +
> +@item disable_fbos
> +Forcibly disable FBOs, resulting in loss of almost all functionality, but
> +offering the maximum possible speed.
> +@end table
> +
> +@subsection Commands
> +This filter supports almost all of the above options as @ref{commands}.
> +
> +@subsection Examples
> +@itemize
> +@item
> +Initialize Vulkan device, upload, filter conversion to yuv420p, and download.
> +Note that in specific cases you can get around the need to perform format
> +conversion by specifying the correct @code{format} filter option corresponding
> +to the input frames.
> +@example
> +ffmpeg -i $INPUT -init_hw_device vulkan -vf hwupload,libplacebo=format=yuv420p,hwdownload,format=yuv420p $OUTPUT
> +@end example
> +
> +@item
> +Run this filter on the CPU, on systems with Mesa installed (and with the most
> +expensive options disabled):
> +@example
> +ffmpeg ... -init_hw_device vulkan:llvmpipe -vf libplacebo=upscaler=none:downscaler=none:peak_detect=false
> +@end example
> +
> +@item
> +Tone-map input to standard gamut BT.709 output:
> +@example
> +libplacebo=colorspace=bt709:color_primaries=bt709:color_trc=bt709:range=tv
> +@end example
> +
> +@item
> +Rescale input to fit into standard 1080p, with high quality scaling:
> +@example
> +libplacebo=w=1920:h=1080:force_original_aspect_ratio=decrease:normalize_sar=true:upscaler=ewa_lanczos:downscaler=ewa_lanczos
> +@end example
> +
> +@item
> +Convert input to standard sRGB JPEG:
> +@example
> +libplacebo=format=yuv420p:colorspace=bt470bg:color_primaries=bt709:color_trc=iec61966-2-1:range=pc
> +@end example
> +
> +@item
> +Use higher quality debanding settings:
> +@example
> +libplacebo=deband=true:deband_iterations=3:deband_radius=8:deband_threshold=6
> +@end example
> +@end itemize
> +
>  @section libvmaf
>  
>  Calulate the VMAF (Video Multi-Method Assessment Fusion) score for a
> -- 
> 2.35.1
> 
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".