From patchwork Mon May 15 08:59:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 41621 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:ba91:b0:105:feb:71f2 with SMTP id fb17csp1456377pzb; Mon, 15 May 2023 01:59:57 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6UpqD5fZ29Ulf9XsI8MdsGO2+axggNO7jxTs10OoqXQZUepOcqb8hA3OPUXAAF9HWfVrIF X-Received: by 2002:a17:907:3e27:b0:957:758e:df57 with SMTP id hp39-20020a1709073e2700b00957758edf57mr36261603ejc.65.1684141197418; Mon, 15 May 2023 01:59:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684141197; cv=none; d=google.com; s=arc-20160816; b=bp4IoOqJbKwKUegflTqZVGA3OtyIEez/AYjihl4Sh+sPeuUe8eH9a/iZPTxBkYvjze xlj4mY15kakM3DAy7JLf95K9Uj4JtuQJhjQqyV3z3JzAWEHjKtsJ0poY/sHDxbbuNNwu rIeUSiog992ZshHXO6UtcHhzLCGmbvPogyybiHVNcFW1kCgX6EuD1q010civYncyM2en oeWZkCg+IacxFOjr8kLhimTos1YSrQ+6d9Z7i3ft9UM+lgsFsDJxt5BLOXZ8oXBIEmjN iie2CS5EuNBdHg/ujjo5hMgCMcXT7nu5UnFVq7rGmgbAFy5p5hyjg6Xh/0KIrwhHU6au b5dQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=TBaqS7y1g8rvaO84GijMAsWA1pe3foe/oJLGfJDRO8E=; b=UN4cmtmqFot0xP+Bby3kpaEJhOMdJQ9kZ6gFUL29V9Ijc1V5KxVriBRXp53REkrxUW RE3Xl5f8G6S2ZDBNlu7fKI67qt6antBg4xuQd9wptskYi9AzrcGw2yMas99h+Bq5Ly0n xTHoHbXyi98NI7hiLkhgkU9OCpBI/Lg5WVzTi2G7Gcyx//6UIntGq/zs/Wz8ZiEVYp9o jDOgfdXFSTHrhhAtrtPfRa/pXPqmKj+RyFdl8igILrTcR8ELz34VgVuy0rmIFZuCM5YQ 9iFpClPkHmeQb7TRrE8b02oPxsTEG6JdSk11ssl8uINifoRiILBNyIVmJoDY9YEPooSX CFQg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=MbPXi2Le; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id y4-20020a17090614c400b0096978b2b5b0si11259888ejc.494.2023.05.15.01.59.56; Mon, 15 May 2023 01:59:57 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=MbPXi2Le; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id DF01668C0AE; Mon, 15 May 2023 11:59:52 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 87B9168C005 for ; Mon, 15 May 2023 11:59:46 +0300 (EEST) Received: from localhost (217-74-0-168.hsi.r-kom.net [217.74.0.168]) by haasn.dev (Postfix) with ESMTPSA id 80D4541EC3; Mon, 15 May 2023 10:59:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1684141185; bh=L74fNTclzytk9xn1KsJXZvCwiRR65dELmszAsEE2zWY=; h=From:To:Cc:Subject:Date:From; b=MbPXi2LeRxN4UdvpcG0s3EjAHlcpr5tOeT6mwN/DArX7UCthMbaQACbBLet5jG2ud Wy/5viGxYL4xxga3RVbZc8fPxXp5e3Lz8bsDtZ4C0LazHEGyYG+fsG7To7YJqusPAp f6F4sNZ3C7NgK0TBv9TofVjaoD+rrywv7dfKrggU= From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Mon, 15 May 2023 10:59:41 +0200 Message-Id: <20230515085943.31760-1-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.40.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/3] lavfi/vf_libplacebo: switch to new gamut mapping API X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: DKEEg8pSGBAP From: Niklas Haas Upstream deprecated the old ad-hoc, enum/intent-based gamut mapping API and added a new API based on colorimetrically accurate gamut mapping functions. The relevant change for us is the addition of several new modes, as well as deprecation of the old options. Update the documentation accordingly. --- doc/filters.texi | 47 ++++++++++--------- libavfilter/vf_libplacebo.c | 92 ++++++++++++++++++++++++++++++------- 2 files changed, 98 insertions(+), 41 deletions(-) diff --git a/doc/filters.texi b/doc/filters.texi index d935e62b3f..14cb31b95c 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -16332,37 +16332,36 @@ gamut-mapping when dealing with mismatches between wide-gamut or HDR content. In general, libplacebo relies on accurate source tagging and mastering display gamut information to produce the best results. @table @option -@item intent -Rendering intent to use when adapting between different primary color gamuts -(after tone-mapping). -@table @samp -@item perceptual -Perceptual gamut mapping. Currently equivalent to relative colorimetric. -@item relative -Relative colorimetric. This is the default. -@item absolute -Absolute colorimetric. -@item saturation -Saturation mapping. Forcibly stretches the source gamut to the target gamut. -@end table - @item gamut_mode How to handle out-of-gamut colors that can occur as a result of colorimetric gamut mapping. @table @samp @item clip -Do nothing, simply clip out-of-range colors to the RGB volume. This is the -default. -@item warn -Highlight out-of-gamut pixels (by coloring them pink). -@item darken -Linearly reduces content brightness to preserves saturated details, followed by -clipping the remaining out-of-gamut colors. As the name implies, this makes -everything darker, but provides a good balance between preserving details and -colors. +Do nothing, simply clip out-of-range colors to the RGB volume. Low quality but +extremely fast. +@item perceptual +Perceptually soft-clip colors to the gamut volume. This is the default. +@item relative +Relative colorimetric hard-clip. Similar to @code{perceptual} but without +the soft knee. +@item saturation +Saturation mapping, maps primaries directly to primaries in RGB space. +Not recommended except for artificial computer graphics for which a bright, +saturated display is desired. +@item absolute +Absolute colorimetric hard-clip. Performs no adjustment of the white point. @item desaturate Hard-desaturates out-of-gamut colors towards white, while preserving the -luminance. Has a tendency to shift colors. +luminance. Has a tendency to distort the visual appearance of bright objects. +@item darken +Linearly reduces content brightness to preserves saturated details, followed by +clipping the remaining out-of-gamut colors. +@item warn +Highlight out-of-gamut pixels (by inverting/marking them). +@item linear +Linearly reduces chromaticity of the entire image to make it fit within the +target color volume. Be careful when using this on BT.2020 sources without +proper mastering metadata, as doing so will lead to excessive desaturation. @end table @item tonemapping diff --git a/libavfilter/vf_libplacebo.c b/libavfilter/vf_libplacebo.c index 94e49aa465..fde8da0b9b 100644 --- a/libavfilter/vf_libplacebo.c +++ b/libavfilter/vf_libplacebo.c @@ -56,6 +56,19 @@ enum { TONE_MAP_COUNT, }; +enum { + GAMUT_MAP_CLIP, + GAMUT_MAP_PERCEPTUAL, + GAMUT_MAP_RELATIVE, + GAMUT_MAP_SATURATION, + GAMUT_MAP_ABSOLUTE, + GAMUT_MAP_DESATURATE, + GAMUT_MAP_DARKEN, + GAMUT_MAP_HIGHLIGHT, + GAMUT_MAP_LINEAR, + GAMUT_MAP_COUNT, +}; + static const struct pl_tone_map_function * const tonemapping_funcs[TONE_MAP_COUNT] = { [TONE_MAP_AUTO] = &pl_tone_map_auto, [TONE_MAP_CLIP] = &pl_tone_map_clip, @@ -73,6 +86,36 @@ static const struct pl_tone_map_function * const tonemapping_funcs[TONE_MAP_COUN [TONE_MAP_LINEAR] = &pl_tone_map_linear, }; +#if PL_API_VER >= 269 +static const struct pl_gamut_map_function * const gamut_funcs[GAMUT_MAP_COUNT] = { + [GAMUT_MAP_CLIP] = &pl_gamut_map_clip, + [GAMUT_MAP_PERCEPTUAL] = &pl_gamut_map_perceptual, + [GAMUT_MAP_RELATIVE] = &pl_gamut_map_relative, + [GAMUT_MAP_SATURATION] = &pl_gamut_map_saturation, + [GAMUT_MAP_ABSOLUTE] = &pl_gamut_map_absolute, + [GAMUT_MAP_DESATURATE] = &pl_gamut_map_desaturate, + [GAMUT_MAP_DARKEN] = &pl_gamut_map_darken, + [GAMUT_MAP_HIGHLIGHT] = &pl_gamut_map_highlight, + [GAMUT_MAP_LINEAR] = &pl_gamut_map_linear, +}; +#else +static const enum pl_gamut_mode gamut_modes[GAMUT_MAP_COUNT] = { + [GAMUT_MAP_CLIP] = PL_GAMUT_CLIP, + [GAMUT_MAP_PERCEPTUAL] = PL_GAMUT_CLIP, + [GAMUT_MAP_DESATURATE] = PL_GAMUT_DESATURATE, + [GAMUT_MAP_DARKEN] = PL_GAMUT_DARKEN, + [GAMUT_MAP_HIGHLIGHT] = PL_GAMUT_WARN, + /* Everything else defaults to PL_GAMUT_CLIP */ +}; + +static const enum pl_rendering_intent intents[GAMUT_MAP_COUNT] = { + [GAMUT_MAP_SATURATION] = PL_INTENT_SATURATION, + [GAMUT_MAP_RELATIVE] = PL_INTENT_RELATIVE_COLORIMETRIC, + [GAMUT_MAP_ABSOLUTE] = PL_INTENT_ABSOLUTE_COLORIMETRIC, + /* Everything else defaults to PL_INTENT_PERCEPTUAL */ +}; +#endif + static const char *const var_names[] = { "in_w", "iw", ///< width of the input video frame "in_h", "ih", ///< height of the input video frame @@ -203,7 +246,6 @@ typedef struct LibplaceboContext { /* pl_color_map_params */ struct pl_color_map_params color_map_params; - int intent; int gamut_mode; int tonemapping; float tonemapping_param; @@ -219,6 +261,7 @@ typedef struct LibplaceboContext { int gamut_warning; int gamut_clipping; int force_icc_lut; + int intent; #endif /* pl_dither_params */ @@ -314,7 +357,7 @@ static int update_settings(AVFilterContext *ctx) int err = 0; LibplaceboContext *s = ctx->priv; enum pl_tone_map_mode tonemapping_mode = s->tonemapping_mode; - enum pl_gamut_mode gamut_mode = s->gamut_mode; + int gamut_mode = s->gamut_mode; uint8_t color_rgba[4]; RET(av_parse_color(color_rgba, s->fillcolor, -1, s)); @@ -333,10 +376,16 @@ static int update_settings(AVFilterContext *ctx) } } + switch (s->intent) { + case PL_INTENT_SATURATION: gamut_mode = GAMUT_MAP_SATURATION; break; + case PL_INTENT_RELATIVE_COLORIMETRIC: gamut_mode = GAMUT_MAP_RELATIVE; break; + case PL_INTENT_ABSOLUTE_COLORIMETRIC: gamut_mode = GAMUT_MAP_ABSOLUTE; break; + } + if (s->gamut_warning) - gamut_mode = PL_GAMUT_WARN; + gamut_mode = GAMUT_MAP_HIGHLIGHT; if (s->gamut_clipping) - gamut_mode = PL_GAMUT_DESATURATE; + gamut_mode = GAMUT_MAP_DESATURATE; #endif s->deband_params = *pl_deband_params( @@ -363,8 +412,12 @@ static int update_settings(AVFilterContext *ctx) ); s->color_map_params = *pl_color_map_params( - .intent = s->intent, - .gamut_mode = gamut_mode, +#if PL_API_VER >= 269 + .gamut_mapping = gamut_funcs[gamut_mode], +#else + .gamut_mode = gamut_modes[gamut_mode], + .intent = intents[gamut_mode], +#endif .tone_mapping_function = tonemapping_funcs[s->tonemapping], .tone_mapping_param = s->tonemapping_param, .tone_mapping_mode = tonemapping_mode, @@ -1140,16 +1193,16 @@ static const AVOption libplacebo_options[] = { { "scene_threshold_high", "Scene change high threshold", OFFSET(scene_high), AV_OPT_TYPE_FLOAT, {.dbl = 10.0}, -1.0, 100.0, DYNAMIC }, { "overshoot", "Tone-mapping overshoot margin", OFFSET(overshoot), AV_OPT_TYPE_FLOAT, {.dbl = 0.05}, 0.0, 1.0, DYNAMIC }, - { "intent", "Rendering intent", OFFSET(intent), AV_OPT_TYPE_INT, {.i64 = PL_INTENT_RELATIVE_COLORIMETRIC}, 0, 3, DYNAMIC, "intent" }, - { "perceptual", "Perceptual", 0, AV_OPT_TYPE_CONST, {.i64 = PL_INTENT_PERCEPTUAL}, 0, 0, STATIC, "intent" }, - { "relative", "Relative colorimetric", 0, AV_OPT_TYPE_CONST, {.i64 = PL_INTENT_RELATIVE_COLORIMETRIC}, 0, 0, STATIC, "intent" }, - { "absolute", "Absolute colorimetric", 0, AV_OPT_TYPE_CONST, {.i64 = PL_INTENT_ABSOLUTE_COLORIMETRIC}, 0, 0, STATIC, "intent" }, - { "saturation", "Saturation mapping", 0, AV_OPT_TYPE_CONST, {.i64 = PL_INTENT_SATURATION}, 0, 0, STATIC, "intent" }, - { "gamut_mode", "Gamut-mapping mode", OFFSET(gamut_mode), AV_OPT_TYPE_INT, {.i64 = PL_GAMUT_CLIP}, 0, PL_GAMUT_MODE_COUNT - 1, DYNAMIC, "gamut_mode" }, - { "clip", "Hard-clip gamut boundary", 0, AV_OPT_TYPE_CONST, {.i64 = PL_GAMUT_CLIP}, 0, 0, STATIC, "gamut_mode" }, - { "warn", "Highlight out-of-gamut colors", 0, AV_OPT_TYPE_CONST, {.i64 = PL_GAMUT_WARN}, 0, 0, STATIC, "gamut_mode" }, - { "darken", "Darken image to fit gamut", 0, AV_OPT_TYPE_CONST, {.i64 = PL_GAMUT_DARKEN}, 0, 0, STATIC, "gamut_mode" }, - { "desaturate", "Colorimetrically desaturate colors", 0, AV_OPT_TYPE_CONST, {.i64 = PL_GAMUT_DESATURATE}, 0, 0, STATIC, "gamut_mode" }, + { "gamut_mode", "Gamut-mapping mode", OFFSET(gamut_mode), AV_OPT_TYPE_INT, {.i64 = GAMUT_MAP_PERCEPTUAL}, 0, GAMUT_MAP_COUNT - 1, DYNAMIC, "gamut_mode" }, + { "clip", "Hard-clip (RGB per-channel)", 0, AV_OPT_TYPE_CONST, {.i64 = GAMUT_MAP_CLIP}, 0, 0, STATIC, "gamut_mode" }, + { "perceptual", "Colorimetric soft clipping", 0, AV_OPT_TYPE_CONST, {.i64 = GAMUT_MAP_PERCEPTUAL}, 0, 0, STATIC, "gamut_mode" }, + { "relative", "Relative colorimetric clipping", 0, AV_OPT_TYPE_CONST, {.i64 = GAMUT_MAP_RELATIVE}, 0, 0, STATIC, "gamut_mode" }, + { "saturation", "Saturation mapping (RGB -> RGB)", 0, AV_OPT_TYPE_CONST, {.i64 = GAMUT_MAP_SATURATION}, 0, 0, STATIC, "gamut_mode" }, + { "absolute", "Absolute colorimetric clipping", 0, AV_OPT_TYPE_CONST, {.i64 = GAMUT_MAP_ABSOLUTE}, 0, 0, STATIC, "gamut_mode" }, + { "desaturate", "Colorimetrically desaturate colors towards white", 0, AV_OPT_TYPE_CONST, {.i64 = GAMUT_MAP_DESATURATE}, 0, 0, STATIC, "gamut_mode" }, + { "darken", "Colorimetric clip with bias towards darkening image to fit gamut", 0, AV_OPT_TYPE_CONST, {.i64 = GAMUT_MAP_DARKEN}, 0, 0, STATIC, "gamut_mode" }, + { "warn", "Highlight out-of-gamut colors", 0, AV_OPT_TYPE_CONST, {.i64 = GAMUT_MAP_HIGHLIGHT}, 0, 0, STATIC, "gamut_mode" }, + { "linear", "Linearly reduce chromaticity to fit gamut", 0, AV_OPT_TYPE_CONST, {.i64 = GAMUT_MAP_LINEAR}, 0, 0, STATIC, "gamut_mode" }, { "tonemapping", "Tone-mapping algorithm", OFFSET(tonemapping), AV_OPT_TYPE_INT, {.i64 = TONE_MAP_AUTO}, 0, TONE_MAP_COUNT - 1, DYNAMIC, "tonemap" }, { "auto", "Automatic selection", 0, AV_OPT_TYPE_CONST, {.i64 = TONE_MAP_AUTO}, 0, 0, STATIC, "tonemap" }, { "clip", "No tone mapping (clip", 0, AV_OPT_TYPE_CONST, {.i64 = TONE_MAP_CLIP}, 0, 0, STATIC, "tonemap" }, @@ -1181,7 +1234,12 @@ static const AVOption libplacebo_options[] = { { "desaturation_strength", "Desaturation strength", OFFSET(desat_str), AV_OPT_TYPE_FLOAT, {.dbl = -1.0}, -1.0, 1.0, DYNAMIC | AV_OPT_FLAG_DEPRECATED }, { "desaturation_exponent", "Desaturation exponent", OFFSET(desat_exp), AV_OPT_TYPE_FLOAT, {.dbl = -1.0}, -1.0, 10.0, DYNAMIC | AV_OPT_FLAG_DEPRECATED }, { "gamut_warning", "Highlight out-of-gamut colors", OFFSET(gamut_warning), AV_OPT_TYPE_BOOL, {.i64 = 0}, 0, 1, DYNAMIC | AV_OPT_FLAG_DEPRECATED }, - { "gamut_clipping", "Enable colorimetric gamut clipping", OFFSET(gamut_clipping), AV_OPT_TYPE_BOOL, {.i64 = 0}, 0, 1, DYNAMIC | AV_OPT_FLAG_DEPRECATED }, + { "gamut_clipping", "Enable desaturating colorimetric gamut clipping", OFFSET(gamut_clipping), AV_OPT_TYPE_BOOL, {.i64 = 0}, 0, 1, DYNAMIC | AV_OPT_FLAG_DEPRECATED }, + { "intent", "Rendering intent", OFFSET(intent), AV_OPT_TYPE_INT, {.i64 = PL_INTENT_PERCEPTUAL}, 0, 3, DYNAMIC | AV_OPT_FLAG_DEPRECATED, "intent" }, + { "perceptual", "Perceptual", 0, AV_OPT_TYPE_CONST, {.i64 = PL_INTENT_PERCEPTUAL}, 0, 0, STATIC, "intent" }, + { "relative", "Relative colorimetric", 0, AV_OPT_TYPE_CONST, {.i64 = PL_INTENT_RELATIVE_COLORIMETRIC}, 0, 0, STATIC, "intent" }, + { "absolute", "Absolute colorimetric", 0, AV_OPT_TYPE_CONST, {.i64 = PL_INTENT_ABSOLUTE_COLORIMETRIC}, 0, 0, STATIC, "intent" }, + { "saturation", "Saturation mapping", 0, AV_OPT_TYPE_CONST, {.i64 = PL_INTENT_SATURATION}, 0, 0, STATIC, "intent" }, #endif { "dithering", "Dither method to use", OFFSET(dithering), AV_OPT_TYPE_INT, {.i64 = PL_DITHER_BLUE_NOISE}, -1, PL_DITHER_METHOD_COUNT - 1, DYNAMIC, "dither" },