From patchwork Tue Apr 19 12:46:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 35345 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3b9e:b0:7d:cfb5:dc7c with SMTP id b30csp328270pzh; Tue, 19 Apr 2022 05:47:44 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyo248bzBKwZxGXKxMWX1sL/fJn//Mm1/df1rXi8HZED/MqOGVNSu0r3s0ZETckPhQrnwRl X-Received: by 2002:a05:6402:3509:b0:423:f906:8e2d with SMTP id b9-20020a056402350900b00423f9068e2dmr5110521edd.333.1650372464464; Tue, 19 Apr 2022 05:47:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1650372464; cv=none; d=google.com; s=arc-20160816; b=QHiO/MuILMHvvB6YxJEL3ZRi30jRJJVdXMSlb7pPf8yk096tw2WXKpeleAODUOXI9h /yRKkKBi/N3u0jczl4A9sgcwa+4O7KJ5RSiSIFJNAQvNC536RMqW5zMLBrty1PFaU3zF BMNch0c+1zIl3RyE6guGmOXPyqWonhC3/db0WGWoLAcq/6QBhRRjMnxBavJGcxfHG1bK 0BFB9wlGX8TFRDYXTtg24JTdlr4Tj/g3CJV9T8MohSmBjphNngcWGDwTcEd+aNNCSKa7 zUUZB+vad4vXBpnuiskk1k5oaaK9S55Cpg8AP9l27D5vIYHolkUqW5MVHL8s7CISFbi2 ShwA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=iMY2I3F0vpvA1FfKTD0t7WErmk1hq5D1AaaTqS0wgYY=; b=pUbD96cbDv8wqkSsjoTLjblWUS2TDs0183TCKmK09q6wuz8SxAlmzi0+d5rYSxYani H0Su24SISHYnX4x+D577bLGh6nr5+2j4Leg0/V0rVcw4qcRuRn71SikEdzvE+pBORUkd BIq8C6XDCo0ivg2smVI+rpyjwWrLIPzKVIV7iljZzcGJuz0kMnEZJFbwlspNk0aKTk+X kmcDQ8ljlYjGb2npcy0nxE9zNA41NM9l/oKH9819gKlQTZiVdrhD22r2WbeUxYyLf1RD C9J7LFS0Qc6qW5ZPv1nkFESy9+1pFbHuD96bBDcZXO/8ky1l6pV9JjrsH6m05uId5UFO HznA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=Dj0NURo9; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id h1-20020a1709066d8100b006e860384033si7588190ejt.106.2022.04.19.05.47.43; Tue, 19 Apr 2022 05:47:44 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=Dj0NURo9; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3FE7568A8CD; Tue, 19 Apr 2022 15:47:39 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4091E68B34E for ; Tue, 19 Apr 2022 15:47:32 +0300 (EEST) Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id 7FDC449CBD; Tue, 19 Apr 2022 14:47:31 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1650372451; bh=n8a2Gh51x2O3szqihYW9TL8Yrjt2gKROXnQbd5qSu+U=; h=From:To:Cc:Subject:Date:From; b=Dj0NURo9TSm1n6kGHEbfKK1njXgFLg4+b6RWqogJBNZRif3rauIAjdm+ArkIoIUjh tljbKUA4kf/On/1rEYU/rbI6qeNk2+FCv7Fgk2fbUz1OVIt/uuPV4YA38c1ZDjOkmC 459RK3L0JdsG7yv+vdbjY6QwgOwRJeccXpxOlqlw= From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Tue, 19 Apr 2022 14:46:53 +0200 Message-Id: <20220419124656.121638-1-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.35.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 1/5] lavfi: generalize colorspace coefficent helpers X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: KA1L5A4t2LTW From: Niklas Haas These are needed beyond just vf_colorspace, so give them a new home in colorspace.h. In addition to moving code around, also merge the white point and primary coefficients into a single struct to slightly increase the convenience and shrink the size of the new API by avoiding the need to introduce an extra function just to look up the white point as well. The only place the distinction matters is in a single enum comparison, which can just as well be a single memcpy - the difference is negligible. Signed-off-by: Niklas Haas --- Changes to v1 of this series: - minor cleanup here and there --- libavfilter/colorspace.c | 32 +++++++++++++++++ libavfilter/colorspace.h | 6 ++++ libavfilter/vf_colorspace.c | 70 ++++++------------------------------- 3 files changed, 49 insertions(+), 59 deletions(-) diff --git a/libavfilter/colorspace.c b/libavfilter/colorspace.c index 19616e4f12..25e99f4759 100644 --- a/libavfilter/colorspace.c +++ b/libavfilter/colorspace.c @@ -138,6 +138,38 @@ const struct LumaCoefficients *ff_get_luma_coefficients(enum AVColorSpace csp) return coeffs; } +#define WP_D65 { 0.3127, 0.3290 } +#define WP_C { 0.3100, 0.3160 } +#define WP_DCI { 0.3140, 0.3510 } +#define WP_E { 1/3.0f, 1/3.0f } + +static const struct ColorPrimaries color_primaries[AVCOL_PRI_NB] = { + [AVCOL_PRI_BT709] = { WP_D65, { 0.640, 0.330, 0.300, 0.600, 0.150, 0.060 } }, + [AVCOL_PRI_BT470M] = { WP_C, { 0.670, 0.330, 0.210, 0.710, 0.140, 0.080 } }, + [AVCOL_PRI_BT470BG] = { WP_D65, { 0.640, 0.330, 0.290, 0.600, 0.150, 0.060 } }, + [AVCOL_PRI_SMPTE170M] = { WP_D65, { 0.630, 0.340, 0.310, 0.595, 0.155, 0.070 } }, + [AVCOL_PRI_SMPTE240M] = { WP_D65, { 0.630, 0.340, 0.310, 0.595, 0.155, 0.070 } }, + [AVCOL_PRI_SMPTE428] = { WP_E, { 0.735, 0.265, 0.274, 0.718, 0.167, 0.009 } }, + [AVCOL_PRI_SMPTE431] = { WP_DCI, { 0.680, 0.320, 0.265, 0.690, 0.150, 0.060 } }, + [AVCOL_PRI_SMPTE432] = { WP_D65, { 0.680, 0.320, 0.265, 0.690, 0.150, 0.060 } }, + [AVCOL_PRI_FILM] = { WP_C, { 0.681, 0.319, 0.243, 0.692, 0.145, 0.049 } }, + [AVCOL_PRI_BT2020] = { WP_D65, { 0.708, 0.292, 0.170, 0.797, 0.131, 0.046 } }, + [AVCOL_PRI_JEDEC_P22] = { WP_D65, { 0.630, 0.340, 0.295, 0.605, 0.155, 0.077 } }, +}; + +const struct ColorPrimaries *ff_get_color_primaries(enum AVColorPrimaries prm) +{ + const struct ColorPrimaries *p; + + if (prm >= AVCOL_PRI_NB) + return NULL; + p = &color_primaries[prm]; + if (!p->prim.xr) + return NULL; + + return p; +} + void ff_fill_rgb2yuv_table(const struct LumaCoefficients *coeffs, double rgb2yuv[3][3]) { diff --git a/libavfilter/colorspace.h b/libavfilter/colorspace.h index a4c5078d5f..fc415fed05 100644 --- a/libavfilter/colorspace.h +++ b/libavfilter/colorspace.h @@ -37,6 +37,11 @@ struct WhitepointCoefficients { double xw, yw; }; +struct ColorPrimaries { + struct WhitepointCoefficients wp; + struct PrimaryCoefficients prim; +}; + void ff_matrix_invert_3x3(const double in[3][3], double out[3][3]); void ff_matrix_mul_3x3(double dst[3][3], const double src1[3][3], const double src2[3][3]); @@ -44,6 +49,7 @@ void ff_fill_rgb2xyz_table(const struct PrimaryCoefficients *coeffs, const struct WhitepointCoefficients *wp, double rgb2xyz[3][3]); +const struct ColorPrimaries *ff_get_color_primaries(enum AVColorPrimaries prm); const struct LumaCoefficients *ff_get_luma_coefficients(enum AVColorSpace csp); void ff_fill_rgb2yuv_table(const struct LumaCoefficients *coeffs, double rgb2yuv[3][3]); diff --git a/libavfilter/vf_colorspace.c b/libavfilter/vf_colorspace.c index 0bd8e2b0cf..3c8b3b20eb 100644 --- a/libavfilter/vf_colorspace.c +++ b/libavfilter/vf_colorspace.c @@ -55,14 +55,6 @@ enum Colorspace { CS_NB, }; -enum Whitepoint { - WP_D65, - WP_C, - WP_DCI, - WP_E, - WP_NB, -}; - enum WhitepointAdaptation { WP_ADAPT_BRADFORD, WP_ADAPT_VON_KRIES, @@ -110,11 +102,6 @@ static const enum AVColorSpace default_csp[CS_NB + 1] = { [CS_NB] = AVCOL_SPC_UNSPECIFIED, }; -struct ColorPrimaries { - enum Whitepoint wp; - struct PrimaryCoefficients coeff; -}; - struct TransferCharacteristics { double alpha, beta, gamma, delta; }; @@ -201,40 +188,6 @@ static const struct TransferCharacteristics * return coeffs; } -static const struct WhitepointCoefficients whitepoint_coefficients[WP_NB] = { - [WP_D65] = { 0.3127, 0.3290 }, - [WP_C] = { 0.3100, 0.3160 }, - [WP_DCI] = { 0.3140, 0.3510 }, - [WP_E] = { 1/3.0f, 1/3.0f }, -}; - -static const struct ColorPrimaries color_primaries[AVCOL_PRI_NB] = { - [AVCOL_PRI_BT709] = { WP_D65, { 0.640, 0.330, 0.300, 0.600, 0.150, 0.060 } }, - [AVCOL_PRI_BT470M] = { WP_C, { 0.670, 0.330, 0.210, 0.710, 0.140, 0.080 } }, - [AVCOL_PRI_BT470BG] = { WP_D65, { 0.640, 0.330, 0.290, 0.600, 0.150, 0.060 } }, - [AVCOL_PRI_SMPTE170M] = { WP_D65, { 0.630, 0.340, 0.310, 0.595, 0.155, 0.070 } }, - [AVCOL_PRI_SMPTE240M] = { WP_D65, { 0.630, 0.340, 0.310, 0.595, 0.155, 0.070 } }, - [AVCOL_PRI_SMPTE428] = { WP_E, { 0.735, 0.265, 0.274, 0.718, 0.167, 0.009 } }, - [AVCOL_PRI_SMPTE431] = { WP_DCI, { 0.680, 0.320, 0.265, 0.690, 0.150, 0.060 } }, - [AVCOL_PRI_SMPTE432] = { WP_D65, { 0.680, 0.320, 0.265, 0.690, 0.150, 0.060 } }, - [AVCOL_PRI_FILM] = { WP_C, { 0.681, 0.319, 0.243, 0.692, 0.145, 0.049 } }, - [AVCOL_PRI_BT2020] = { WP_D65, { 0.708, 0.292, 0.170, 0.797, 0.131, 0.046 } }, - [AVCOL_PRI_JEDEC_P22] = { WP_D65, { 0.630, 0.340, 0.295, 0.605, 0.155, 0.077 } }, -}; - -static const struct ColorPrimaries *get_color_primaries(enum AVColorPrimaries prm) -{ - const struct ColorPrimaries *p; - - if (prm >= AVCOL_PRI_NB) - return NULL; - p = &color_primaries[prm]; - if (!p->coeff.xr) - return NULL; - - return p; -} - static int fill_gamma_table(ColorSpaceContext *s) { int n; @@ -280,7 +233,8 @@ static int fill_gamma_table(ColorSpaceContext *s) * This function uses the Bradford mechanism. */ static void fill_whitepoint_conv_table(double out[3][3], enum WhitepointAdaptation wp_adapt, - enum Whitepoint src, enum Whitepoint dst) + const struct WhitepointCoefficients *wp_src, + const struct WhitepointCoefficients *wp_dst) { static const double ma_tbl[NB_WP_ADAPT_NON_IDENTITY][3][3] = { [WP_ADAPT_BRADFORD] = { @@ -294,9 +248,7 @@ static void fill_whitepoint_conv_table(double out[3][3], enum WhitepointAdaptati }, }; const double (*ma)[3] = ma_tbl[wp_adapt]; - const struct WhitepointCoefficients *wp_src = &whitepoint_coefficients[src]; double zw_src = 1.0 - wp_src->xw - wp_src->yw; - const struct WhitepointCoefficients *wp_dst = &whitepoint_coefficients[dst]; double zw_dst = 1.0 - wp_dst->xw - wp_dst->yw; double mai[3][3], fac[3][3], tmp[3][3]; double rs, gs, bs, rd, gd, bd; @@ -486,7 +438,7 @@ static int create_filtergraph(AVFilterContext *ctx, s->in_prm = default_prm[FFMIN(s->user_iall, CS_NB)]; if (s->user_iprm != AVCOL_PRI_UNSPECIFIED) s->in_prm = s->user_iprm; - s->in_primaries = get_color_primaries(s->in_prm); + s->in_primaries = ff_get_color_primaries(s->in_prm); if (!s->in_primaries) { av_log(ctx, AV_LOG_ERROR, "Unsupported input primaries %d (%s)\n", @@ -494,7 +446,7 @@ static int create_filtergraph(AVFilterContext *ctx, return AVERROR(EINVAL); } s->out_prm = out->color_primaries; - s->out_primaries = get_color_primaries(s->out_prm); + s->out_primaries = ff_get_color_primaries(s->out_prm); if (!s->out_primaries) { if (s->out_prm == AVCOL_PRI_UNSPECIFIED) { if (s->user_all == CS_UNSPECIFIED) { @@ -516,17 +468,17 @@ static int create_filtergraph(AVFilterContext *ctx, double rgb2xyz[3][3], xyz2rgb[3][3], rgb2rgb[3][3]; const struct WhitepointCoefficients *wp_out, *wp_in; - wp_out = &whitepoint_coefficients[s->out_primaries->wp]; - wp_in = &whitepoint_coefficients[s->in_primaries->wp]; - ff_fill_rgb2xyz_table(&s->out_primaries->coeff, wp_out, rgb2xyz); + wp_out = &s->out_primaries->wp; + wp_in = &s->in_primaries->wp; + ff_fill_rgb2xyz_table(&s->out_primaries->prim, wp_out, rgb2xyz); ff_matrix_invert_3x3(rgb2xyz, xyz2rgb); - ff_fill_rgb2xyz_table(&s->in_primaries->coeff, wp_in, rgb2xyz); - if (s->out_primaries->wp != s->in_primaries->wp && + ff_fill_rgb2xyz_table(&s->in_primaries->prim, wp_in, rgb2xyz); + if (memcmp(wp_in, wp_out, sizeof(*wp_in)) != 0 && s->wp_adapt != WP_ADAPT_IDENTITY) { double wpconv[3][3], tmp[3][3]; - fill_whitepoint_conv_table(wpconv, s->wp_adapt, s->in_primaries->wp, - s->out_primaries->wp); + fill_whitepoint_conv_table(wpconv, s->wp_adapt, &s->in_primaries->wp, + &s->out_primaries->wp); ff_matrix_mul_3x3(tmp, rgb2xyz, wpconv); ff_matrix_mul_3x3(rgb2rgb, tmp, xyz2rgb); } else { From patchwork Tue Apr 19 12:46:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 35346 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3b9e:b0:7d:cfb5:dc7c with SMTP id b30csp328345pzh; Tue, 19 Apr 2022 05:47:55 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxosdoYcuQqBbbSv9mSZLIvQvBhbTyPn6JOLNqDTs877fK6s1I49ZfI5KTQ2XHVcSjx2irz X-Received: by 2002:a17:906:4fc8:b0:6d8:5059:f998 with SMTP id i8-20020a1709064fc800b006d85059f998mr13819639ejw.487.1650372475392; Tue, 19 Apr 2022 05:47:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1650372475; cv=none; d=google.com; s=arc-20160816; b=PL5BtpOjzSIdU7a/PeB9srglYeUzEd+0tz9ncOpmxCnVZquRTvtQdrYoyhdAfhQOeJ 1U9fs8bjBFT+fsbggQX6QxAxDxUHUIJ6f9yvKKnDO/Fju7qg9vYcKGVKVBN0R+7GTOPO XyTMFoq5PigvG9vDY9bzgMrcp1s3C+lDfIi94HEeFLXXaHy5nzW6vInOO5+FFEHO8Tei n24gKtI54hlh0fAIjdld5LEA7zr7VhJ49c+8ofXwksTmKsg/ozEOJj36sD9TsqtoCQSa 7zSqUPNoGkxVFK+E2mVfwu4tV4ND/4u3rlzMVPQLDTQ61OP8SyC6ZDEI0D0Rpt4Mb7rD Latg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=+UPYwho+ZEyXk3swgaSjbwtsaYNduufS3c8UgviD/I0=; b=nd5++VYcg/33HJSjD0O9jDsQqGEr53SndEGs/lQmVeR64LO3lJmvdMcPUcigjGHKY1 du3W6XoPjDTOgPEOum6p6bC5wHfBjGEUuSvQhmEdQFsb8l4N5rJVlmOFMMzmn6BQ7EL+ QjRmSukbgqW5lEffgh/GLJ6z3waaCSXwlJboGSVewR1YwvSQG6r42ZQ7n3+BtObVJOgM QjJpKRLfds+PuwTHhcBKRMvZSg4b5ZG78Kv/OhSngqCGo+KdtJOih4fZUFOXS+IiqVaW uT+RyUU83sG87Oj0uQOqqtByK7hd2NZJSTTvMQi3+Sq8youjnGXgvInShYMnWyDKKM+M jQ1w== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=Gvtod5rp; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id br4-20020a170906d14400b006dfe7bc2fafsi8097613ejb.56.2022.04.19.05.47.55; Tue, 19 Apr 2022 05:47:55 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=Gvtod5rp; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5C41A68B36F; Tue, 19 Apr 2022 15:47:40 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3E46868B2DC for ; Tue, 19 Apr 2022 15:47:32 +0300 (EEST) Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id B7F6249CCB; Tue, 19 Apr 2022 14:47:31 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1650372451; bh=uHH7Ig+OyV2t/yoH7eeQ050z5rJHljVxxX91dh+mEVo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Gvtod5rpfJiDRgflYuTCurpSpVf0PsWZYHjHr8DGpeVcvff4gqSrnY0sbxiZARNs3 0Ic1BXchDM12QDJkxs7KCp1KhpVguz2ETRm33qaV809ABdalTESSywShAgiTXBm5XY W+EA6lSr6RzMlIda9QORTYQ4j2eoYZFwD1uhjgv8= From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Tue, 19 Apr 2022 14:46:54 +0200 Message-Id: <20220419124656.121638-2-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220419124656.121638-1-ffmpeg@haasn.xyz> References: <20220419124656.121638-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 2/5] lavfi: add ff_detect_color_primaries helper X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: gLB6R+musdy5 From: Niklas Haas Related to #9673, this helper exists to facilitate "guessing" the right primary tags from a given set of raw primary coefficients. The cutoff value of 0.001 was chosen by experimentation. The smallest "false positive" delta observed in practice was 0.023329, while the largest "false negative" delta was 0.00016. So, a value of 0.001 sits comfortably in the middle. Signed-off-by: Niklas Haas --- libavfilter/colorspace.c | 25 +++++++++++++++++++++++++ libavfilter/colorspace.h | 3 +++ 2 files changed, 28 insertions(+) diff --git a/libavfilter/colorspace.c b/libavfilter/colorspace.c index 25e99f4759..8d7b882375 100644 --- a/libavfilter/colorspace.c +++ b/libavfilter/colorspace.c @@ -170,6 +170,31 @@ const struct ColorPrimaries *ff_get_color_primaries(enum AVColorPrimaries prm) return p; } +enum AVColorPrimaries ff_detect_color_primaries(const struct ColorPrimaries *prm) +{ + double delta; + + for (enum AVColorPrimaries p = 0; p < AVCOL_PRI_NB; p++) { + const struct ColorPrimaries *ref = &color_primaries[p]; + if (!ref->prim.xr) + continue; + + delta = fabs(prm->prim.xr - ref->prim.xr) + + fabs(prm->prim.yr - ref->prim.yr) + + fabs(prm->prim.yg - ref->prim.yg) + + fabs(prm->prim.yg - ref->prim.yg) + + fabs(prm->prim.yb - ref->prim.yb) + + fabs(prm->prim.yb - ref->prim.yb) + + fabs(prm->wp.xw - ref->wp.xw) + + fabs(prm->wp.yw - ref->wp.yw); + + if (delta < 0.001) + return p; + } + + return AVCOL_PRI_UNSPECIFIED; +} + void ff_fill_rgb2yuv_table(const struct LumaCoefficients *coeffs, double rgb2yuv[3][3]) { diff --git a/libavfilter/colorspace.h b/libavfilter/colorspace.h index fc415fed05..6959133a49 100644 --- a/libavfilter/colorspace.h +++ b/libavfilter/colorspace.h @@ -49,6 +49,9 @@ void ff_fill_rgb2xyz_table(const struct PrimaryCoefficients *coeffs, const struct WhitepointCoefficients *wp, double rgb2xyz[3][3]); +/* Returns AVCOL_PRI_UNSPECIFIED if no clear match can be identified */ +enum AVColorPrimaries ff_detect_color_primaries(const struct ColorPrimaries *prm); + const struct ColorPrimaries *ff_get_color_primaries(enum AVColorPrimaries prm); const struct LumaCoefficients *ff_get_luma_coefficients(enum AVColorSpace csp); void ff_fill_rgb2yuv_table(const struct LumaCoefficients *coeffs, From patchwork Tue Apr 19 12:46:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 35347 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3b9e:b0:7d:cfb5:dc7c with SMTP id b30csp328434pzh; Tue, 19 Apr 2022 05:48:07 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzJJ4GP68Ktktp9nkkBZCFo2EOQEZy3PyxjLv6yfsUqv/MgX+kzHbODDh9JNKuyLiWfBGML X-Received: by 2002:a17:906:4546:b0:6e8:873a:22a8 with SMTP id s6-20020a170906454600b006e8873a22a8mr13517340ejq.711.1650372487224; Tue, 19 Apr 2022 05:48:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1650372487; cv=none; d=google.com; s=arc-20160816; b=XqpF6w8W7/DTb3hh0qRZVjYRLzfJHH1jc8inGfiTrGta0oClAmPXX8q9vjrhz5SG1M ozEqMXjDQ47FopHhEJ3dc0BuHljlOohSjR9/6Yk2qqNPW3ZaOGav/PfJnYo8jcAOFJO/ kvcSuhc8zRoNtoNrB/UZrqtIpo+m5R+5eW4/cE+jF5qJWSP/1CII0+U6nn3wb1S12IxZ LodMX1PlrNcHDfHHttSMbKbT6dys8pQSyPzFoL6D+ArbzsDJJLLYoqOkObOCsY3jgn8Y +WepGpfN1QBGzrE8kzbFHCJo/11ij6/a/CUwwIj3GPsob2lBgi2UqqcGgIbPO2OvtFiH ROQg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=yd4GKb+sRHBd+hxbNaxYTXnS+RLP8uNI8kq6Y4nnzwk=; b=wdwZDTdapGwLxeCKvc8RUMLPRykzTZiQPpo2QybvRJhm4flIkpQHvlEFe9G2PP3H74 AoZRBf8JBl08jULre4TQ5yIbggFkeUy0h+TTlmwVpJp3RuTUFtub1tr4VGh9ANMiNtHG 7NK6wcBDTNbbZOAz94jqVW3WTv26EZjDDaZFGAqt+FLccB6Ebn+6zOVHYbH+SOzlJSSA 65xv7y/4gC7P2zrx0jX3CHJ8GDUtV98TbxeMyosvy25OKfuv7LEJ1Kiqx8BdTuZk+wa5 8ln/o8eTQQmDrNiz9DIk+Q0hM/4MyiRKystmuuejVJiuQtPKQSKQJ01ljnymMz0O6U0R XZmQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=AHknVTyo; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id eb7-20020a0564020d0700b0042280e86243si9306609edb.414.2022.04.19.05.48.05; Tue, 19 Apr 2022 05:48:07 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=AHknVTyo; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4E6B468B375; Tue, 19 Apr 2022 15:47:41 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4281668B34F for ; Tue, 19 Apr 2022 15:47:32 +0300 (EEST) Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id EAE2E49E1E; Tue, 19 Apr 2022 14:47:31 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1650372452; bh=G5ZTZoSeVq6zi07UKk+NFU6OHocGaL2Z5cjDh6hw5h8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AHknVTyoFkdEf3RqcbpvyPQ2rrlDQyBeIAc32N28hzQArMK/1seppJq0QpR87dsDa /IqVhDDB9aFFNoWWLNA6j7Md0X23diQ+8E9IJU0+veKI6RTxjUXepS2rGpdSsszwPk /uhBY0K9JER3v0Em03Yfci7OHQsFOFHVYCzXgko0= From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Tue, 19 Apr 2022 14:46:55 +0200 Message-Id: <20220419124656.121638-3-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220419124656.121638-1-ffmpeg@haasn.xyz> References: <20220419124656.121638-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 3/5] lavfi: add ICC profile support via lcms2 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: NeIJXRzAw+bH From: Niklas Haas This introduces an optional dependency on lcms2 into FFmpeg. lcms2 is a widely used library for ICC profile handling, which apart from being used in almost all major image processing programs and video players, has also been deployed in browsers. As such, it's both widely available and well-tested. Add a few helpers to cover our major use cases. This commit merely introduces the helpers (and configure check), even though nothing uses them yet. It's worth pointing out that the reason the cmsToneCurves for each AVCOL_TRC are cached inside the context, is because constructing a cmsToneCurve requires evaluating the curve at 4096 (by default) grid points and constructing a LUT. So, we ideally only want to do this once per curve. This matters for e.g. ff_icc_profile_detect_transfer, which essentially compares a profile against all of these generated LUTs. Re-generating the LUTs for every iteration would be unnecessarily wasteful. The same consideration does not apply to e.g. cmsCreate*Profile, which is a very lightweight operation just involving struct allocation and setting a few pointers. The cutoff value of 0.01 was determined by experimentation. The lowest "false positive" delta I saw in practice was 0.13, and the largest "false negative" delta was 0.0008. So a value of 0.01 sits comfortaby almost exactly in the middle. Signed-off-by: Niklas Haas --- configure | 3 + libavfilter/fflcms2.c | 310 ++++++++++++++++++++++++++++++++++++++++++ libavfilter/fflcms2.h | 87 ++++++++++++ 3 files changed, 400 insertions(+) create mode 100644 libavfilter/fflcms2.c create mode 100644 libavfilter/fflcms2.h diff --git a/configure b/configure index 358a614854..c894efa600 100755 --- a/configure +++ b/configure @@ -215,6 +215,7 @@ External library support: --disable-iconv disable iconv [autodetect] --enable-jni enable JNI support [no] --enable-ladspa enable LADSPA audio filtering [no] + --enable-lcms2 enable ICC profile support via LittleCMS 2 [no] --enable-libaom enable AV1 video encoding/decoding via libaom [no] --enable-libaribb24 enable ARIB text and caption decoding via libaribb24 [no] --enable-libass enable libass subtitles rendering, @@ -1813,6 +1814,7 @@ EXTERNAL_LIBRARY_LIST=" gnutls jni ladspa + lcms2 libaom libass libbluray @@ -6506,6 +6508,7 @@ enabled gmp && require gmp gmp.h mpz_export -lgmp enabled gnutls && require_pkg_config gnutls gnutls gnutls/gnutls.h gnutls_global_init enabled jni && { [ $target_os = "android" ] && check_headers jni.h && enabled pthreads || die "ERROR: jni not found"; } enabled ladspa && require_headers "ladspa.h dlfcn.h" +enabled lcms2 && require_pkg_config lcms2 "lcms2 >= 2.13" lcms2.h cmsCreateContext enabled libaom && require_pkg_config libaom "aom >= 1.0.0" aom/aom_codec.h aom_codec_version enabled libaribb24 && { check_pkg_config libaribb24 "aribb24 > 1.0.3" "aribb24/aribb24.h" arib_instance_new || { enabled gpl && require_pkg_config libaribb24 aribb24 "aribb24/aribb24.h" arib_instance_new; } || diff --git a/libavfilter/fflcms2.c b/libavfilter/fflcms2.c new file mode 100644 index 0000000000..efc7cb5189 --- /dev/null +++ b/libavfilter/fflcms2.c @@ -0,0 +1,310 @@ +/* + * Copyright (c) 2022 Niklas Haas + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/color_utils.h" + +#include "fflcms2.h" + +static void log_cb(cmsContext ctx, cmsUInt32Number error, const char *str) +{ + FFIccContext *s = cmsGetContextUserData(ctx); + av_log(s->avctx, AV_LOG_ERROR, "lcms2: [%"PRIu32"] %s\n", error, str); +} + +int ff_icc_context_init(FFIccContext *s, void *avctx) +{ + memset(s, 0, sizeof(*s)); + s->avctx = avctx; + s->ctx = cmsCreateContext(NULL, s); + if (!s->ctx) + return AVERROR(ENOMEM); + + cmsSetLogErrorHandlerTHR(s->ctx, log_cb); + return 0; +} + +void ff_icc_context_uninit(FFIccContext *s) +{ + for (int i = 0; i < FF_ARRAY_ELEMS(s->curves); i++) + cmsFreeToneCurve(s->curves[i]); + cmsDeleteContext(s->ctx); + memset(s, 0, sizeof(*s)); +} + +static int get_curve(FFIccContext *s, enum AVColorTransferCharacteristic trc, + cmsToneCurve **out_curve) +{ + if (trc >= AVCOL_TRC_NB) + return AVERROR_INVALIDDATA; + + if (s->curves[trc]) + goto done; + + switch (trc) { + case AVCOL_TRC_LINEAR: + s->curves[trc] = cmsBuildGamma(s->ctx, 1.0); + break; + case AVCOL_TRC_GAMMA22: + s->curves[trc] = cmsBuildGamma(s->ctx, 2.2); + break; + case AVCOL_TRC_GAMMA28: + s->curves[trc] = cmsBuildGamma(s->ctx, 2.8); + break; + case AVCOL_TRC_BT709: + case AVCOL_TRC_SMPTE170M: + case AVCOL_TRC_BT2020_10: + case AVCOL_TRC_BT2020_12: + s->curves[trc] = cmsBuildParametricToneCurve(s->ctx, 4, (double[5]) { + /* γ = */ 1/0.45, + /* a = */ 1/1.099296826809442, + /* b = */ 1 - 1/1.099296826809442, + /* c = */ 1/4.5, + /* d = */ 4.5 * 0.018053968510807, + }); + break; + case AVCOL_TRC_SMPTE240M: + s->curves[trc] = cmsBuildParametricToneCurve(s->ctx, 4, (double[5]) { + /* γ = */ 1/0.45, + /* a = */ 1/1.1115, + /* b = */ 1 - 1/1.1115, + /* c = */ 1/4.0, + /* d = */ 4.0 * 0.0228, + }); + break; + case AVCOL_TRC_LOG: + s->curves[trc] = cmsBuildParametricToneCurve(s->ctx, 8, (double[5]) { + /* a = */ 1.0, + /* b = */ 10.0, + /* c = */ 2.0, + /* d = */ -1.0, + /* e = */ 0.0 + }); + break; + case AVCOL_TRC_LOG_SQRT: + s->curves[trc] = cmsBuildParametricToneCurve(s->ctx, 8, (double[5]) { + /* a = */ 1.0, + /* b = */ 10.0, + /* c = */ 2.5, + /* d = */ -1.0, + /* e = */ 0.0 + }); + break; + case AVCOL_TRC_IEC61966_2_1: + s->curves[trc] = cmsBuildParametricToneCurve(s->ctx, 4, (double[5]) { + /* γ = */ 2.4, + /* a = */ 1/1.055, + /* b = */ 1 - 1/1.055, + /* c = */ 1/12.92, + /* d = */ 12.92 * 0.0031308, + }); + break; + case AVCOL_TRC_SMPTE428: + s->curves[trc] = cmsBuildParametricToneCurve(s->ctx, 2, (double[3]) { + /* γ = */ 2.6, + /* a = */ pow(52.37/48.0, 1/2.6), + /* b = */ 0.0 + }); + break; + + /* Can't be represented using the existing parametric tone curves. + * FIXME: use cmsBuildTabulatedToneCurveFloat instead */ + case AVCOL_TRC_IEC61966_2_4: + case AVCOL_TRC_BT1361_ECG: + case AVCOL_TRC_SMPTE2084: + case AVCOL_TRC_ARIB_STD_B67: + return AVERROR_PATCHWELCOME; + + default: + return AVERROR_INVALIDDATA; + } + + if (!s->curves[trc]) + return AVERROR(ENOMEM); + +done: + *out_curve = s->curves[trc]; + return 0; +} + +int ff_icc_profile_generate(FFIccContext *s, + enum AVColorPrimaries color_prim, + enum AVColorTransferCharacteristic color_trc, + cmsHPROFILE *out_profile) +{ + cmsToneCurve *tonecurve; + const struct ColorPrimaries *prim; + int ret; + + if (!(prim = ff_get_color_primaries(color_prim))) + return AVERROR_INVALIDDATA; + if ((ret = get_curve(s, color_trc, &tonecurve)) < 0) + return ret; + + *out_profile = cmsCreateRGBProfileTHR(s->ctx, + &(cmsCIExyY) { prim->wp.xw, prim->wp.yw, 1.0 }, + &(cmsCIExyYTRIPLE) { + .Red = { prim->prim.xr, prim->prim.yr, 1.0 }, + .Green = { prim->prim.xg, prim->prim.yg, 1.0 }, + .Blue = { prim->prim.xb, prim->prim.yb, 1.0 }, + }, + (cmsToneCurve *[3]) { tonecurve, tonecurve, tonecurve } + ); + + return *out_profile == NULL ? AVERROR(ENOMEM) : 0; +} + +int ff_icc_profile_attach(FFIccContext *s, cmsHPROFILE profile, AVFrame *frame) +{ + cmsUInt32Number size; + AVBufferRef *buf; + + if (!cmsSaveProfileToMem(profile, NULL, &size)) + return AVERROR_EXTERNAL; + + buf = av_buffer_alloc(size); + if (!buf) + return AVERROR(ENOMEM); + + if (!cmsSaveProfileToMem(profile, buf->data, &size) || size != buf->size) { + av_buffer_unref(&buf); + return AVERROR_EXTERNAL; + } + + if (!av_frame_new_side_data_from_buf(frame, AV_FRAME_DATA_ICC_PROFILE, buf)) { + av_buffer_unref(&buf); + return AVERROR(ENOMEM); + } + + return 0; +} + +static av_always_inline void XYZ_xy(cmsCIEXYZ XYZ, double *x, double *y) +{ + double k = 1.0 / (XYZ.X + XYZ.Y + XYZ.Z); + *x = k * XYZ.X; + *y = k * XYZ.Y; +} + +int ff_icc_profile_read_primaries(FFIccContext *s, cmsHPROFILE profile, + struct ColorPrimaries *out_primaries) +{ + static const uint8_t testprimaries[4][3] = { + { 0xFF, 0, 0 }, /* red */ + { 0, 0xFF, 0 }, /* green */ + { 0, 0, 0xFF }, /* blue */ + { 0xFF, 0xFF, 0xFF }, /* white */ + }; + + struct WhitepointCoefficients *wp = &out_primaries->wp; + struct PrimaryCoefficients *prim = &out_primaries->prim; + cmsFloat64Number prev_adapt; + cmsHPROFILE xyz; + cmsHTRANSFORM tf; + cmsCIEXYZ dst[4]; + + xyz = cmsCreateXYZProfileTHR(s->ctx); + if (!xyz) + return AVERROR(ENOMEM); + + /* We need to use an unadapted observer to get the raw values */ + prev_adapt = cmsSetAdaptationStateTHR(s->ctx, 0.0); + tf = cmsCreateTransformTHR(s->ctx, profile, TYPE_RGB_8, xyz, TYPE_XYZ_DBL, + INTENT_ABSOLUTE_COLORIMETRIC, + /* Note: These flags mostly don't do anything + * anyway, but specify them regardless */ + cmsFLAGS_NOCACHE | + cmsFLAGS_NOOPTIMIZE | + cmsFLAGS_LOWRESPRECALC | + cmsFLAGS_GRIDPOINTS(2)); + cmsSetAdaptationStateTHR(s->ctx, prev_adapt); + cmsCloseProfile(xyz); + if (!tf) { + av_log(s->avctx, AV_LOG_ERROR, "Invalid ICC profile (e.g. CMYK)\n"); + return AVERROR_INVALIDDATA; + } + + cmsDoTransform(tf, testprimaries, dst, 4); + cmsDeleteTransform(tf); + XYZ_xy(dst[0], &prim->xr, &prim->yr); + XYZ_xy(dst[1], &prim->xg, &prim->yg); + XYZ_xy(dst[2], &prim->xb, &prim->yb); + XYZ_xy(dst[3], &wp->xw, &wp->yw); + return 0; +} + +int ff_icc_profile_detect_transfer(FFIccContext *s, cmsHPROFILE profile, + enum AVColorTransferCharacteristic *out_trc) +{ + /* 8-bit linear grayscale ramp */ + static const uint8_t testramp[16][3] = { + { 1, 1, 1}, /* avoid exact zero due to log100 etc. */ + { 17, 17, 17}, + { 34, 34, 34}, + { 51, 51, 51}, + { 68, 68, 68}, + { 85, 85, 85}, + { 02, 02, 02}, + {119, 119, 119}, + {136, 136, 136}, + {153, 153, 153}, + {170, 170, 170}, + {187, 187, 187}, + {204, 204, 204}, + {221, 221, 221}, + {238, 238, 238}, + {255, 255, 255}, + }; + + double dst[FF_ARRAY_ELEMS(testramp)]; + + for (enum AVColorTransferCharacteristic trc = 0; trc < AVCOL_TRC_NB; trc++) { + cmsToneCurve *tonecurve; + cmsHPROFILE ref; + cmsHTRANSFORM tf; + double delta = 0.0; + if (get_curve(s, trc, &tonecurve) < 0) + continue; + + ref = cmsCreateGrayProfileTHR(s->ctx, cmsD50_xyY(), tonecurve); + if (!ref) + return AVERROR(ENOMEM); + + tf = cmsCreateTransformTHR(s->ctx, profile, TYPE_RGB_8, ref, TYPE_GRAY_DBL, + INTENT_RELATIVE_COLORIMETRIC, + cmsFLAGS_NOCACHE | cmsFLAGS_NOOPTIMIZE); + cmsCloseProfile(ref); + if (!tf) { + av_log(s->avctx, AV_LOG_ERROR, "Invalid ICC profile (e.g. CMYK)\n"); + return AVERROR_INVALIDDATA; + } + + cmsDoTransform(tf, testramp, dst, FF_ARRAY_ELEMS(dst)); + cmsDeleteTransform(tf); + + for (int i = 0; i < FF_ARRAY_ELEMS(dst); i++) + delta += fabs(testramp[i][0] / 255.0 - dst[i]); + if (delta < 0.01) { + *out_trc = trc; + return 0; + } + } + + *out_trc = AVCOL_TRC_UNSPECIFIED; + return 0; +} diff --git a/libavfilter/fflcms2.h b/libavfilter/fflcms2.h new file mode 100644 index 0000000000..ad6c8c47cf --- /dev/null +++ b/libavfilter/fflcms2.h @@ -0,0 +1,87 @@ +/* + * Copyright (c) 2022 Niklas Haas + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * Various functions for dealing with ICC profiles + */ + +#ifndef AVFILTER_FFLCMS2_H +#define AVFILTER_FFLCMS2_H + +#include "libavutil/frame.h" +#include "libavutil/pixfmt.h" +#include "colorspace.h" + +#include + +typedef struct FFIccContext { + void *avctx; + cmsContext ctx; + cmsToneCurve *curves[AVCOL_TRC_NB]; /* tone curve cache */ +} FFIccContext; + +/** + * Initializes an FFIccContext. This must be done prior to using it. + * + * Returns 0 on success, or a negative error code. + */ +int ff_icc_context_init(FFIccContext *s, void *avctx); +void ff_icc_context_uninit(FFIccContext *s); + +/** + * Generate an ICC profile for a given combination of color primaries and + * transfer function. Both values must be set to valid entries (not + * "undefined") for this function to work. + * + * Returns 0 on success, or a negative error code. + */ +int ff_icc_profile_generate(FFIccContext *s, + enum AVColorPrimaries color_prim, + enum AVColorTransferCharacteristic color_trc, + cmsHPROFILE *out_profile); + +/** + * Attach an ICC profile to a frame. Helper wrapper around cmsSaveProfileToMem + * and av_frame_new_side_data_from_buf. + * + * Returns 0 on success, or a negative error code. + */ +int ff_icc_profile_attach(FFIccContext *s, cmsHPROFILE profile, AVFrame *frame); + +/** + * Read the color primaries and white point coefficients encoded by an ICC + * profile, and return the raw values in `out_primaries`. + * + * Returns 0 on success, or a negative error code. + */ +int ff_icc_profile_read_primaries(FFIccContext *s, cmsHPROFILE profile, + struct ColorPrimaries *out_primaries); + +/** + * Attempt detecting the transfer characteristic that best approximates the + * transfer function encoded by an ICC profile. Sets `out_trc` to + * AVCOL_TRC_UNSPECIFIED if no clear match can be identified. + * + * Returns 0 on success (including no match), or a negative error code. + */ +int ff_icc_profile_detect_transfer(FFIccContext *s, cmsHPROFILE profile, + enum AVColorTransferCharacteristic *out_trc); + +#endif /* AVFILTER_FFLCMS2_H */ From patchwork Tue Apr 19 12:46:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 35348 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3b9e:b0:7d:cfb5:dc7c with SMTP id b30csp328542pzh; Tue, 19 Apr 2022 05:48:18 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxxBXR4iSO7+KeUVOt/fYoZNX6u2KZTN4uXbtdDZTb9Xw/U4G/2bN+QPlQyn/pGHxkz9ntq X-Received: by 2002:a17:906:7c59:b0:6ef:7d6b:39fd with SMTP id g25-20020a1709067c5900b006ef7d6b39fdmr12709061ejp.230.1650372498141; Tue, 19 Apr 2022 05:48:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1650372498; cv=none; d=google.com; s=arc-20160816; b=kKjD47KB7C3vgvB1ULaHTjxQLPCsEF+GOiKZk0pZcB8gRaVnIVf8HJZQVzvjQELQxr 2xFGaU3WbNY1CwfwyKsIePh6Vd9meK9hyBn7kxFW1osL2EiTRdukJ8DWcEkOxxXKhMd9 TYVOlXTAVHY3GeKBkrzdGM/5EBi36OP1oztrXFdYAvgx/4uWAhTLQxNr3RJXeyTExefL RZITvNt8mmLsu5tTfY72v6R/qnNalH4g2yjvaMdIaSeXsA1VnpWitd6Cdec62h2KF80J 81xSssD/iUXxcomAKdZLfocEVFjUznAL8flB2LFEFfvjlwFRyTMd6z9chQMXCGw5ZuET TzUg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=NJNMpmsLo2ugY8ezIFH1345Rnv80iRsXwOqrolieSMw=; b=u0wqn//mCXWhx9vs1ZdRQach0r3yW9+z3SNmsE75vdC0JS0CBjIN7O9eDLkPBl46hv xkm0V54JbKLld27DOfWWkz0CYAyS+dDR2q4W1YQ1Zx276Hj0bI10jwh23simycxkkmAH v1SVrtktgLak7shA2wT2QHngOtVVNp6blPiLhfpHTjHpm6WX71RoD9TnoBqFUhuFZToq yz6pRf6WFxcPNPK20HyIIUUZ+jFqSjDMZODmUGKp3dbZfYAE2t5UeP+1tf9pNBQ5nBVW NVXfSBiYQ9b7kUD5Xk2EzVyGxSWjJdc3PQBkuh8ytlgdyzVQieDXjOWFsxQa78VjCgHt x/qA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=aF1jm4rO; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id d3-20020a170906c20300b006e88cd8b684si9178250ejz.727.2022.04.19.05.48.17; Tue, 19 Apr 2022 05:48:18 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=aF1jm4rO; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4820668B37C; Tue, 19 Apr 2022 15:47:42 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7CC4F68B34E for ; Tue, 19 Apr 2022 15:47:32 +0300 (EEST) Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id 2E28449E35; Tue, 19 Apr 2022 14:47:32 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1650372452; bh=KP+qYzFzmYOiHWJoIPh4dUSZFTIn4+3BW+dytD2WYUo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=aF1jm4rOh4PqaVDVPJLfs5s7mIwTC71ySjxYQ6qF+SeRoyLLFyb6EqI/5tMr3NdTE DHFpD+lMTIN5KG/Ort7KUdmLnRlZ0bHH00PBg/2mnEpxfnVefe/jqB1a3bRWS3jBE6 NmrjK9Z0mUm98qt4aNGzbd1Uuv1fc6WtuaaJZW7E= From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Tue, 19 Apr 2022 14:46:56 +0200 Message-Id: <20220419124656.121638-4-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220419124656.121638-1-ffmpeg@haasn.xyz> References: <20220419124656.121638-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 4/5] lavfi: add vf_iccgen for generating ICC profiles X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: XQc2jSwLXbz8 From: Niklas Haas This filter is designed to specifically cover the task of generating ICC profiles (and attaching them to output frames) on demand. Other tasks, such as ICC profile loading/stripping, or ICC profile application, are better left to separate filters (or included into e.g. vf_setparams). Signed-off-by: Niklas Haas --- configure | 1 + doc/filters.texi | 21 +++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/vf_iccgen.c | 181 +++++++++++++++++++++++++++++++++++++++ 5 files changed, 205 insertions(+) create mode 100644 libavfilter/vf_iccgen.c diff --git a/configure b/configure index c894efa600..44f8ff738e 100755 --- a/configure +++ b/configure @@ -3661,6 +3661,7 @@ gblur_vulkan_filter_deps="vulkan spirv_compiler" hflip_vulkan_filter_deps="vulkan spirv_compiler" histeq_filter_deps="gpl" hqdn3d_filter_deps="gpl" +iccgen_filter_deps="lcms2" interlace_filter_deps="gpl" kerndeint_filter_deps="gpl" ladspa_filter_deps="ladspa libdl" diff --git a/doc/filters.texi b/doc/filters.texi index a161754233..f29845890e 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -14342,6 +14342,27 @@ By default value is 0. The @code{hysteresis} filter also supports the @ref{framesync} options. +@section iccgen + +Generate ICC profiles and attach them to frames. + +This filter accepts the following options: + +@table @option +@item color_primaries +@item color_trc +Configure the colorspace that the ICC profile will be generated for. The +default value of @code{auto} infers the value from the input frame's metadata, +defaulting to BT.709/sRGB as appropriate. + +See the @ref{setparams} filter for a list of possible values, but note that +@code{unknown} are not valid values for this filter. + +@item force +If true, an ICC profile will be generated even if it would overwrite an +already existing ICC profile. Disabled by default. +@end table + @section identity Obtain the identity score between two input videos. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index b870ae2697..1f2b03fa9c 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -321,6 +321,7 @@ OBJS-$(CONFIG_HWMAP_FILTER) += vf_hwmap.o OBJS-$(CONFIG_HWUPLOAD_CUDA_FILTER) += vf_hwupload_cuda.o OBJS-$(CONFIG_HWUPLOAD_FILTER) += vf_hwupload.o OBJS-$(CONFIG_HYSTERESIS_FILTER) += vf_hysteresis.o framesync.o +OBJS-$(CONFIG_ICCGEN_FILTER) += vf_iccgen.o fflcms2.o colorspace.o OBJS-$(CONFIG_IDENTITY_FILTER) += vf_identity.o OBJS-$(CONFIG_IDET_FILTER) += vf_idet.o OBJS-$(CONFIG_IL_FILTER) += vf_il.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 366c5f4e7f..212615019d 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -304,6 +304,7 @@ extern const AVFilter ff_vf_hwmap; extern const AVFilter ff_vf_hwupload; extern const AVFilter ff_vf_hwupload_cuda; extern const AVFilter ff_vf_hysteresis; +extern const AVFilter ff_vf_iccgen; extern const AVFilter ff_vf_identity; extern const AVFilter ff_vf_idet; extern const AVFilter ff_vf_il; diff --git a/libavfilter/vf_iccgen.c b/libavfilter/vf_iccgen.c new file mode 100644 index 0000000000..67d02f5bc2 --- /dev/null +++ b/libavfilter/vf_iccgen.c @@ -0,0 +1,181 @@ +/* + * Copyright (c) 2022 Niklas Haas + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * filter for generating ICC profiles + */ + +#include + +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" + +#include "avfilter.h" +#include "fflcms2.h" +#include "internal.h" + +typedef struct IccGenContext { + const AVClass *class; + FFIccContext icc; + /* options */ + int color_prim; + int color_trc; + int force; + /* (cached) generated ICC profile */ + cmsHPROFILE profile; + int profile_prim; + int profile_trc; +} IccGenContext; + +#define OFFSET(x) offsetof(IccGenContext, x) +#define VF AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_FILTERING_PARAM + +static const AVOption iccgen_options[] = { + {"color_primaries", "select color primaries", OFFSET(color_prim), AV_OPT_TYPE_INT, {.i64=0}, 0, AVCOL_PRI_NB-1, VF, "color_primaries"}, + {"auto", "infer based on frame", 0, AV_OPT_TYPE_CONST, {.i64=0}, 0, 0, VF, "color_primaries"}, + {"bt709", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_PRI_BT709}, 0, 0, VF, "color_primaries"}, + {"bt470m", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_PRI_BT470M}, 0, 0, VF, "color_primaries"}, + {"bt470bg", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_PRI_BT470BG}, 0, 0, VF, "color_primaries"}, + {"smpte170m", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_PRI_SMPTE170M}, 0, 0, VF, "color_primaries"}, + {"smpte240m", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_PRI_SMPTE240M}, 0, 0, VF, "color_primaries"}, + {"film", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_PRI_FILM}, 0, 0, VF, "color_primaries"}, + {"bt2020", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_PRI_BT2020}, 0, 0, VF, "color_primaries"}, + {"smpte428", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_PRI_SMPTE428}, 0, 0, VF, "color_primaries"}, + {"smpte431", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_PRI_SMPTE431}, 0, 0, VF, "color_primaries"}, + {"smpte432", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_PRI_SMPTE432}, 0, 0, VF, "color_primaries"}, + {"jedec-p22", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_PRI_JEDEC_P22}, 0, 0, VF, "color_primaries"}, + {"ebu3213", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_PRI_EBU3213}, 0, 0, VF, "color_primaries"}, + {"color_trc", "select color transfer", OFFSET(color_trc), AV_OPT_TYPE_INT, {.i64=0}, 0, AVCOL_TRC_NB-1, VF, "color_trc"}, + {"auto", "infer based on frame", 0, AV_OPT_TYPE_CONST, {.i64=0}, 0, 0, VF, "color_trc"}, + {"bt709", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_TRC_BT709}, 0, 0, VF, "color_trc"}, + {"bt470m", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_TRC_GAMMA22}, 0, 0, VF, "color_trc"}, + {"bt470bg", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_TRC_GAMMA28}, 0, 0, VF, "color_trc"}, + {"smpte170m", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_TRC_SMPTE170M}, 0, 0, VF, "color_trc"}, + {"smpte240m", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_TRC_SMPTE240M}, 0, 0, VF, "color_trc"}, + {"linear", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_TRC_LINEAR}, 0, 0, VF, "color_trc"}, + {"iec61966-2-4", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_TRC_IEC61966_2_4}, 0, 0, VF, "color_trc"}, + {"bt1361e", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_TRC_BT1361_ECG}, 0, 0, VF, "color_trc"}, + {"iec61966-2-1", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_TRC_IEC61966_2_1}, 0, 0, VF, "color_trc"}, + {"bt2020-10", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_TRC_BT2020_10}, 0, 0, VF, "color_trc"}, + {"bt2020-12", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_TRC_BT2020_12}, 0, 0, VF, "color_trc"}, + {"smpte2084", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_TRC_SMPTE2084}, 0, 0, VF, "color_trc"}, + {"arib-std-b67", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_TRC_ARIB_STD_B67}, 0, 0, VF, "color_trc"}, + { "force", "overwrite existing ICC profile", OFFSET(force), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, VF }, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(iccgen); + +static av_cold void iccgen_uninit(AVFilterContext *avctx) +{ + IccGenContext *s = avctx->priv; + cmsCloseProfile(s->profile); + ff_icc_context_uninit(&s->icc); +} + +static av_cold int iccgen_init(AVFilterContext *avctx) +{ + IccGenContext *s = avctx->priv; + return ff_icc_context_init(&s->icc, avctx); +} + +static int iccgen_filter_frame(AVFilterLink *inlink, AVFrame *frame) +{ + AVFilterContext *avctx = inlink->dst; + IccGenContext *s = avctx->priv; + enum AVColorTransferCharacteristic trc; + enum AVColorPrimaries prim; + int ret; + + if (av_frame_get_side_data(frame, AV_FRAME_DATA_ICC_PROFILE)) { + if (s->force) { + av_frame_remove_side_data(frame, AV_FRAME_DATA_ICC_PROFILE); + } else { + return ff_filter_frame(inlink->dst->outputs[0], frame); + } + } + + trc = s->color_trc ? s->color_trc : frame->color_trc; + if (trc == AVCOL_TRC_UNSPECIFIED) { + const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(frame->format); + if (!desc) + return AVERROR_INVALIDDATA; + + if ((desc->flags & AV_PIX_FMT_FLAG_RGB) || frame->color_range == AVCOL_RANGE_JPEG) { + /* Default to sRGB for RGB or full-range content */ + trc = AVCOL_TRC_IEC61966_2_1; + } else { + /* Default to an ITU-R transfer depending on the bit-depth */ + trc = desc->comp[0].depth >= 12 ? AVCOL_TRC_BT2020_12 + : desc->comp[0].depth >= 10 ? AVCOL_TRC_BT2020_10 + : AVCOL_TRC_BT709; + } + } + + prim = s->color_prim ? s->color_prim : frame->color_primaries; + if (prim == AVCOL_PRI_UNSPECIFIED) { + /* Simply always default to sRGB/BT.709 primaries to avoid surprises */ + prim = AVCOL_PRI_BT709; + } + + if (s->profile && prim != s->profile_prim && trc != s->profile_trc) { + cmsCloseProfile(s->profile); + s->profile = NULL; + } + + if (!s->profile) { + if ((ret = ff_icc_profile_generate(&s->icc, prim, trc, &s->profile)) < 0) + return ret; + s->profile_prim = prim; + s->profile_trc = trc; + } + + if ((ret = ff_icc_profile_attach(&s->icc, s->profile, frame)) < 0) + return ret; + + return ff_filter_frame(inlink->dst->outputs[0], frame); +} + +static const AVFilterPad iccgen_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .filter_frame = iccgen_filter_frame, + }, +}; + +static const AVFilterPad iccgen_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + }, +}; + +const AVFilter ff_vf_iccgen = { + .name = "iccgen", + .description = NULL_IF_CONFIG_SMALL("Generate and attach ICC profiles."), + .priv_size = sizeof(IccGenContext), + .priv_class = &iccgen_class, + .flags = AVFILTER_FLAG_METADATA_ONLY, + .init = &iccgen_init, + .uninit = &iccgen_uninit, + FILTER_INPUTS(iccgen_inputs), + FILTER_OUTPUTS(iccgen_outputs), +}; From patchwork Tue Apr 19 12:46:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 35349 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3b9e:b0:7d:cfb5:dc7c with SMTP id b30csp328661pzh; Tue, 19 Apr 2022 05:48:33 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwL5d5euEN3fjcGDy7lpiTEA5eRIUFqqZb3i0HqDIFyEgb3tfYtHcGZL9vugruMIQvu/g6Y X-Received: by 2002:a17:907:72cc:b0:6e8:a2c1:3744 with SMTP id du12-20020a17090772cc00b006e8a2c13744mr13261343ejc.726.1650372510164; Tue, 19 Apr 2022 05:48:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1650372510; cv=none; d=google.com; s=arc-20160816; b=s2Gydfm+GnxvPBU4sdgORd4wwKvEo6zMLEtRLrksgBgrg4AdzoWx/Jc+hD2kId4r6o 1WbRKwX7LXR4F9IrUfEh3Jn7pma5KPCeTbxGfvaVvh1PkhWghh6MgLQrHuSS6YcRbzV1 xch2i23TOY29ytIdHLf6Eas2Fn+TJQKcDgkPSZeTQKfz4TKad0nkGL0J4ec3bEkSj0iQ 1B4GqP4BuD2Eb9GVTKChah3qndPH4HDpE6B5NLUONZnGgAGGM6DZxFvolzO2iCsKnvDH JXMW8eg0V9hzmD725NNInGWfeuGoS0gRL1WjejZvrSBp0yZ+DTjFOUvP1bJmCErm8WmC fpDA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=pTapCcVclr/q3uw0NiFsdo/AP5UDkdNUlb1KaflG+kc=; b=OjcP+rCclv4XZHvVuQJ/TWKYVpPMTmKe4UIPNcIwjWEMEm1d7hiRTRTfAsuFTQk+Xq BQtNr2k1Y5KBew/k5FWspFAvxLAWUPTxsy1XpOt5Id8sLzWSNB+nw4XX3E+UKAS3mbT8 GUIxowmpGAF7n9/FH1PoiteXA5drB81rPPp+YLtuEZmbWAge6laG3tj3VOteHxSaqRbz JXdcmcI8dAQIbyQ1pNzUkXsojXugq+iK270AgEyGddJND4wr4pNaGahDjJNp6V45AWdQ wI9CpGJmYzOwLJ/MhNK+REpF0Av31KnCSdtnuYON17HZqnnE0ydVVRkmD7hYHEpjxMNB K9Aw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=uOZsp1mL; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id o5-20020aa7dd45000000b0041e97a8af57si7704585edw.63.2022.04.19.05.48.29; Tue, 19 Apr 2022 05:48:30 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=uOZsp1mL; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3CEA068B380; Tue, 19 Apr 2022 15:47:43 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A6D4168B34E for ; Tue, 19 Apr 2022 15:47:32 +0300 (EEST) Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id 5FA8849E5F; Tue, 19 Apr 2022 14:47:32 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1650372452; bh=Z5828lygFJWE8fYYQGPNhHCOAKe4i4TB6CktSBYfTZk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uOZsp1mL02vvIZ9ius9hLU6GIxwV3STQmOiuJ9V98QhVm2mPmYb3bvmiI3wRUuAGd E5J/WPSj5H0/G4ovJtapqUqrFt6nu+GqCDmgENwUQyRzS++0pPV6RWzAVD4RWnhUuB EnhQqIzBDVSqXmyxGolKRZKUSDi87U3YXWmJNce0= From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Tue, 19 Apr 2022 14:46:57 +0200 Message-Id: <20220419124656.121638-5-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220419124656.121638-1-ffmpeg@haasn.xyz> References: <20220419124656.121638-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 5/5] lavfi: add vf_iccdetect for parsing ICC profiles X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: uFSDQ/c4g2IP From: Niklas Haas This filter is designed to parse embedded ICC profiles and attempt extracting colorspace tags from them, updating the AVFrame metadata accordingly. This is intentionally made a separate filter, rather than being part of libavcodec itself, so that it's an opt-in behavior for the time being. This also gives the user more flexibility to e.g. first attach an ICC profile and then also set the colorspace tags from it. This makes #9673 possible, though not automatic. Signed-off-by: Niklas Haas --- configure | 1 + doc/filters.texi | 14 ++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/vf_iccdetect.c | 142 +++++++++++++++++++++++++++++++++++++ 5 files changed, 159 insertions(+) create mode 100644 libavfilter/vf_iccdetect.c diff --git a/configure b/configure index 44f8ff738e..612c894f1b 100755 --- a/configure +++ b/configure @@ -3661,6 +3661,7 @@ gblur_vulkan_filter_deps="vulkan spirv_compiler" hflip_vulkan_filter_deps="vulkan spirv_compiler" histeq_filter_deps="gpl" hqdn3d_filter_deps="gpl" +iccdetect_filter_deps="lcms2" iccgen_filter_deps="lcms2" interlace_filter_deps="gpl" kerndeint_filter_deps="gpl" diff --git a/doc/filters.texi b/doc/filters.texi index f29845890e..85b1586486 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -14342,6 +14342,20 @@ By default value is 0. The @code{hysteresis} filter also supports the @ref{framesync} options. +@section iccdetect + +Detect the colorspace from an embedded ICC profile (if present), and update +the frame's tags accordingly. + +This filter accepts the following options: + +@table @option +@item force +If true, the frame's existing colorspace tags will always be overridden by +values detected from an ICC profile. Otherwise, they will only be assigned if +they contain @code{unknown}. Enabled by default. +@end table + @section iccgen Generate ICC profiles and attach them to frames. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 1f2b03fa9c..2c131e6af5 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -321,6 +321,7 @@ OBJS-$(CONFIG_HWMAP_FILTER) += vf_hwmap.o OBJS-$(CONFIG_HWUPLOAD_CUDA_FILTER) += vf_hwupload_cuda.o OBJS-$(CONFIG_HWUPLOAD_FILTER) += vf_hwupload.o OBJS-$(CONFIG_HYSTERESIS_FILTER) += vf_hysteresis.o framesync.o +OBJS-$(CONFIG_ICCDETECT_FILTER) += vf_iccdetect.o fflcms2.o colorspace.o OBJS-$(CONFIG_ICCGEN_FILTER) += vf_iccgen.o fflcms2.o colorspace.o OBJS-$(CONFIG_IDENTITY_FILTER) += vf_identity.o OBJS-$(CONFIG_IDET_FILTER) += vf_idet.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 212615019d..ebe12520b5 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -304,6 +304,7 @@ extern const AVFilter ff_vf_hwmap; extern const AVFilter ff_vf_hwupload; extern const AVFilter ff_vf_hwupload_cuda; extern const AVFilter ff_vf_hysteresis; +extern const AVFilter ff_vf_iccdetect; extern const AVFilter ff_vf_iccgen; extern const AVFilter ff_vf_identity; extern const AVFilter ff_vf_idet; diff --git a/libavfilter/vf_iccdetect.c b/libavfilter/vf_iccdetect.c new file mode 100644 index 0000000000..fb7871f035 --- /dev/null +++ b/libavfilter/vf_iccdetect.c @@ -0,0 +1,142 @@ +/* + * Copyright (c) 2022 Niklas Haas + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * filter for generating ICC profiles + */ + +#include + +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" + +#include "avfilter.h" +#include "fflcms2.h" +#include "internal.h" + +typedef struct IccDetectContext { + const AVClass *class; + FFIccContext icc; + int force; + /* (cached) detected ICC profile values */ + AVBufferRef *profile; + enum AVColorPrimaries profile_prim; + enum AVColorTransferCharacteristic profile_trc; +} IccDetectContext; + +#define OFFSET(x) offsetof(IccDetectContext, x) +#define VF AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_FILTERING_PARAM + +static const AVOption iccdetect_options[] = { + { "force", "overwrite existing tags", OFFSET(force), AV_OPT_TYPE_BOOL, {.i64=1}, 0, 1, VF }, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(iccdetect); + +static av_cold void iccdetect_uninit(AVFilterContext *avctx) +{ + IccDetectContext *s = avctx->priv; + av_buffer_unref(&s->profile); + ff_icc_context_uninit(&s->icc); +} + +static av_cold int iccdetect_init(AVFilterContext *avctx) +{ + IccDetectContext *s = avctx->priv; + return ff_icc_context_init(&s->icc, avctx); +} + +static int iccdetect_filter_frame(AVFilterLink *inlink, AVFrame *frame) +{ + AVFilterContext *avctx = inlink->dst; + IccDetectContext *s = avctx->priv; + const AVFrameSideData *sd; + struct ColorPrimaries coeffs; + cmsHPROFILE profile; + int ret; + + sd = av_frame_get_side_data(frame, AV_FRAME_DATA_ICC_PROFILE); + if (!sd) + return ff_filter_frame(inlink->dst->outputs[0], frame); + + if (s->profile && s->profile->data == sd->buf->data) { + /* No change from previous ICC profile */ + goto done; + } + + if ((ret = av_buffer_replace(&s->profile, sd->buf)) < 0) + return ret; + s->profile_prim = AVCOL_PRI_UNSPECIFIED; + s->profile_trc = AVCOL_TRC_UNSPECIFIED; + + profile = cmsOpenProfileFromMemTHR(s->icc.ctx, sd->data, sd->size); + if (!profile) + return AVERROR_INVALIDDATA; + + ret = ff_icc_profile_read_primaries(&s->icc, profile, &coeffs); + if (!ret) + ret = ff_icc_profile_detect_transfer(&s->icc, profile, &s->profile_trc); + cmsCloseProfile(profile); + if (ret < 0) + return ret; + + s->profile_prim = ff_detect_color_primaries(&coeffs); + +done: + if (s->profile_prim != AVCOL_PRI_UNSPECIFIED) { + if (s->force || frame->color_primaries == AVCOL_PRI_UNSPECIFIED) + frame->color_primaries = s->profile_prim; + } + + if (s->profile_trc != AVCOL_TRC_UNSPECIFIED) { + if (s->force || frame->color_trc == AVCOL_TRC_UNSPECIFIED) + frame->color_trc = s->profile_trc; + } + + return ff_filter_frame(inlink->dst->outputs[0], frame); +} + +static const AVFilterPad iccdetect_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .filter_frame = iccdetect_filter_frame, + }, +}; + +static const AVFilterPad iccdetect_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + }, +}; + +const AVFilter ff_vf_iccdetect = { + .name = "iccdetect", + .description = NULL_IF_CONFIG_SMALL("Detect and parse ICC profiles."), + .priv_size = sizeof(IccDetectContext), + .priv_class = &iccdetect_class, + .flags = AVFILTER_FLAG_METADATA_ONLY, + .init = &iccdetect_init, + .uninit = &iccdetect_uninit, + FILTER_INPUTS(iccdetect_inputs), + FILTER_OUTPUTS(iccdetect_outputs), +};