From patchwork Mon Apr 11 15:36:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 35255 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:671c:b0:7c:62c8:b2d1 with SMTP id q28csp1111539pzh; Mon, 11 Apr 2022 08:37:11 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyvj7TNRpf9vh/oo9wPgCEVK/6jF6S+sYpgIgBQbHylzkKJZ5r24FWpkVofddBK93Ar1DCl X-Received: by 2002:a05:6402:5254:b0:41d:7abe:3e08 with SMTP id t20-20020a056402525400b0041d7abe3e08mr6233908edd.153.1649691431522; Mon, 11 Apr 2022 08:37:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1649691431; cv=none; d=google.com; s=arc-20160816; b=SqyAxkIR2E2N6XLCyho8ZFA/Vn9SqcAyPlcxJANPpWyRzag1utpkBXiLulbhWrG9Uc iUhjID9EeKLQdutKgNe/C9lJySlum50U5x2n/RMp4EI6+wYFuJCBJSYVUprqHFjIRBUd nMXInz/oXZ94abaP00wmjXM4eMt05wVWR++oHF5vqvR8tAcIkgl2aK/XT+M20u5edUSx GmmUwx2AOKvvGFnYPLWKUUMAuz1JAvSoLPKgw+bynkNPAcUJbyi8GTjUowyFeZTItKjX iVaKjxoxiXA/WWs+f7OHiYj0Tiv+5Wgxbw4Up78Xj7JluaRjdwbZK6YQTL2cC1NwgGdE IScA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=lNwQTipKoMTDXVsuS2gKUfRN5nI0G0ZcU8so/MjKVjI=; b=YJ77sWzuMUbnVP2/vpbv4Kn5HeLqcFlOXEdQO0xvGLBUgeHb9mvlMIaYMjuhayrNRz AX/GJVQ39KSayTdceH+4unt3r5aW0sfJtqVr27avcXLTnuB277sTieXIw3pOHeb8gklp 5sMaWR4BEeE1gWmIwYp0QI1oNRI9HmiLwoy/tiFcKnhgQ9+cFqCHO04nhPtm92Umn4oC 0h0uHlSufXarIVQ4pCCpHoG1dI4TiL6DU4tcZD7X8a4aedNFXpU2Dg/iL43W1jn7NS0x 9GMkVrQ0T0ISvhOtiG4jwXdwuixx6WFIqw7niT2LKcxEmpEhKip+3vbbHkLxckQM4HW1 k/sw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=E1t+S91L; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id v3-20020a1709062f0300b006e093439ce5si7450064eji.644.2022.04.11.08.37.11; Mon, 11 Apr 2022 08:37:11 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=E1t+S91L; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6F7F468B295; Mon, 11 Apr 2022 18:37:08 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0682068B221 for ; Mon, 11 Apr 2022 18:37:02 +0300 (EEST) Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id BE88049DED; Mon, 11 Apr 2022 17:37:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1649691421; bh=wDbzAjxGCshjtUMXen7XPRU8AbVxoqhSJF3cLhxGkK0=; h=From:To:Cc:Subject:Date:From; b=E1t+S91LFK4XNAVOxSTNMu5bQfCkXc6ni4UP3/U0wWdQjhJScc5dcGYkumJHO2vNu Cvdv8i8qVdHC0JV5rA8Vd8PHiFPd6MrzdoKmYtZEk0gdiDG9A9EuKE+HGx3t5Pu++t Lygls9UWhd/qN2/cO6i3oiJZalAqBXH6lH0ssJ+U= From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Mon, 11 Apr 2022 17:36:50 +0200 Message-Id: <20220411153654.116722-1-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.35.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/5] lavfi: generalize colorspace coefficent helpers X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 7/vvsKHQF55P From: Niklas Haas These are needed beyond just vf_colorspace, so give them a new home in colorspace.h. In addition to moving code around, also merge the white point and primary coefficients into a single struct to slightly increase the convenience and shrink the size of the new API by avoiding the need to introduce an extra function just to look up the white point as well. The only place the distinction matters is in a single enum comparison, which can just as well be a single memcpy - the difference is negligible. Signed-off-by: Niklas Haas --- libavfilter/colorspace.c | 32 +++++++++++++++++ libavfilter/colorspace.h | 6 ++++ libavfilter/vf_colorspace.c | 70 ++++++------------------------------- 3 files changed, 49 insertions(+), 59 deletions(-) diff --git a/libavfilter/colorspace.c b/libavfilter/colorspace.c index 19616e4f12..25e99f4759 100644 --- a/libavfilter/colorspace.c +++ b/libavfilter/colorspace.c @@ -138,6 +138,38 @@ const struct LumaCoefficients *ff_get_luma_coefficients(enum AVColorSpace csp) return coeffs; } +#define WP_D65 { 0.3127, 0.3290 } +#define WP_C { 0.3100, 0.3160 } +#define WP_DCI { 0.3140, 0.3510 } +#define WP_E { 1/3.0f, 1/3.0f } + +static const struct ColorPrimaries color_primaries[AVCOL_PRI_NB] = { + [AVCOL_PRI_BT709] = { WP_D65, { 0.640, 0.330, 0.300, 0.600, 0.150, 0.060 } }, + [AVCOL_PRI_BT470M] = { WP_C, { 0.670, 0.330, 0.210, 0.710, 0.140, 0.080 } }, + [AVCOL_PRI_BT470BG] = { WP_D65, { 0.640, 0.330, 0.290, 0.600, 0.150, 0.060 } }, + [AVCOL_PRI_SMPTE170M] = { WP_D65, { 0.630, 0.340, 0.310, 0.595, 0.155, 0.070 } }, + [AVCOL_PRI_SMPTE240M] = { WP_D65, { 0.630, 0.340, 0.310, 0.595, 0.155, 0.070 } }, + [AVCOL_PRI_SMPTE428] = { WP_E, { 0.735, 0.265, 0.274, 0.718, 0.167, 0.009 } }, + [AVCOL_PRI_SMPTE431] = { WP_DCI, { 0.680, 0.320, 0.265, 0.690, 0.150, 0.060 } }, + [AVCOL_PRI_SMPTE432] = { WP_D65, { 0.680, 0.320, 0.265, 0.690, 0.150, 0.060 } }, + [AVCOL_PRI_FILM] = { WP_C, { 0.681, 0.319, 0.243, 0.692, 0.145, 0.049 } }, + [AVCOL_PRI_BT2020] = { WP_D65, { 0.708, 0.292, 0.170, 0.797, 0.131, 0.046 } }, + [AVCOL_PRI_JEDEC_P22] = { WP_D65, { 0.630, 0.340, 0.295, 0.605, 0.155, 0.077 } }, +}; + +const struct ColorPrimaries *ff_get_color_primaries(enum AVColorPrimaries prm) +{ + const struct ColorPrimaries *p; + + if (prm >= AVCOL_PRI_NB) + return NULL; + p = &color_primaries[prm]; + if (!p->prim.xr) + return NULL; + + return p; +} + void ff_fill_rgb2yuv_table(const struct LumaCoefficients *coeffs, double rgb2yuv[3][3]) { diff --git a/libavfilter/colorspace.h b/libavfilter/colorspace.h index a4c5078d5f..fc415fed05 100644 --- a/libavfilter/colorspace.h +++ b/libavfilter/colorspace.h @@ -37,6 +37,11 @@ struct WhitepointCoefficients { double xw, yw; }; +struct ColorPrimaries { + struct WhitepointCoefficients wp; + struct PrimaryCoefficients prim; +}; + void ff_matrix_invert_3x3(const double in[3][3], double out[3][3]); void ff_matrix_mul_3x3(double dst[3][3], const double src1[3][3], const double src2[3][3]); @@ -44,6 +49,7 @@ void ff_fill_rgb2xyz_table(const struct PrimaryCoefficients *coeffs, const struct WhitepointCoefficients *wp, double rgb2xyz[3][3]); +const struct ColorPrimaries *ff_get_color_primaries(enum AVColorPrimaries prm); const struct LumaCoefficients *ff_get_luma_coefficients(enum AVColorSpace csp); void ff_fill_rgb2yuv_table(const struct LumaCoefficients *coeffs, double rgb2yuv[3][3]); diff --git a/libavfilter/vf_colorspace.c b/libavfilter/vf_colorspace.c index 0bd8e2b0cf..3c8b3b20eb 100644 --- a/libavfilter/vf_colorspace.c +++ b/libavfilter/vf_colorspace.c @@ -55,14 +55,6 @@ enum Colorspace { CS_NB, }; -enum Whitepoint { - WP_D65, - WP_C, - WP_DCI, - WP_E, - WP_NB, -}; - enum WhitepointAdaptation { WP_ADAPT_BRADFORD, WP_ADAPT_VON_KRIES, @@ -110,11 +102,6 @@ static const enum AVColorSpace default_csp[CS_NB + 1] = { [CS_NB] = AVCOL_SPC_UNSPECIFIED, }; -struct ColorPrimaries { - enum Whitepoint wp; - struct PrimaryCoefficients coeff; -}; - struct TransferCharacteristics { double alpha, beta, gamma, delta; }; @@ -201,40 +188,6 @@ static const struct TransferCharacteristics * return coeffs; } -static const struct WhitepointCoefficients whitepoint_coefficients[WP_NB] = { - [WP_D65] = { 0.3127, 0.3290 }, - [WP_C] = { 0.3100, 0.3160 }, - [WP_DCI] = { 0.3140, 0.3510 }, - [WP_E] = { 1/3.0f, 1/3.0f }, -}; - -static const struct ColorPrimaries color_primaries[AVCOL_PRI_NB] = { - [AVCOL_PRI_BT709] = { WP_D65, { 0.640, 0.330, 0.300, 0.600, 0.150, 0.060 } }, - [AVCOL_PRI_BT470M] = { WP_C, { 0.670, 0.330, 0.210, 0.710, 0.140, 0.080 } }, - [AVCOL_PRI_BT470BG] = { WP_D65, { 0.640, 0.330, 0.290, 0.600, 0.150, 0.060 } }, - [AVCOL_PRI_SMPTE170M] = { WP_D65, { 0.630, 0.340, 0.310, 0.595, 0.155, 0.070 } }, - [AVCOL_PRI_SMPTE240M] = { WP_D65, { 0.630, 0.340, 0.310, 0.595, 0.155, 0.070 } }, - [AVCOL_PRI_SMPTE428] = { WP_E, { 0.735, 0.265, 0.274, 0.718, 0.167, 0.009 } }, - [AVCOL_PRI_SMPTE431] = { WP_DCI, { 0.680, 0.320, 0.265, 0.690, 0.150, 0.060 } }, - [AVCOL_PRI_SMPTE432] = { WP_D65, { 0.680, 0.320, 0.265, 0.690, 0.150, 0.060 } }, - [AVCOL_PRI_FILM] = { WP_C, { 0.681, 0.319, 0.243, 0.692, 0.145, 0.049 } }, - [AVCOL_PRI_BT2020] = { WP_D65, { 0.708, 0.292, 0.170, 0.797, 0.131, 0.046 } }, - [AVCOL_PRI_JEDEC_P22] = { WP_D65, { 0.630, 0.340, 0.295, 0.605, 0.155, 0.077 } }, -}; - -static const struct ColorPrimaries *get_color_primaries(enum AVColorPrimaries prm) -{ - const struct ColorPrimaries *p; - - if (prm >= AVCOL_PRI_NB) - return NULL; - p = &color_primaries[prm]; - if (!p->coeff.xr) - return NULL; - - return p; -} - static int fill_gamma_table(ColorSpaceContext *s) { int n; @@ -280,7 +233,8 @@ static int fill_gamma_table(ColorSpaceContext *s) * This function uses the Bradford mechanism. */ static void fill_whitepoint_conv_table(double out[3][3], enum WhitepointAdaptation wp_adapt, - enum Whitepoint src, enum Whitepoint dst) + const struct WhitepointCoefficients *wp_src, + const struct WhitepointCoefficients *wp_dst) { static const double ma_tbl[NB_WP_ADAPT_NON_IDENTITY][3][3] = { [WP_ADAPT_BRADFORD] = { @@ -294,9 +248,7 @@ static void fill_whitepoint_conv_table(double out[3][3], enum WhitepointAdaptati }, }; const double (*ma)[3] = ma_tbl[wp_adapt]; - const struct WhitepointCoefficients *wp_src = &whitepoint_coefficients[src]; double zw_src = 1.0 - wp_src->xw - wp_src->yw; - const struct WhitepointCoefficients *wp_dst = &whitepoint_coefficients[dst]; double zw_dst = 1.0 - wp_dst->xw - wp_dst->yw; double mai[3][3], fac[3][3], tmp[3][3]; double rs, gs, bs, rd, gd, bd; @@ -486,7 +438,7 @@ static int create_filtergraph(AVFilterContext *ctx, s->in_prm = default_prm[FFMIN(s->user_iall, CS_NB)]; if (s->user_iprm != AVCOL_PRI_UNSPECIFIED) s->in_prm = s->user_iprm; - s->in_primaries = get_color_primaries(s->in_prm); + s->in_primaries = ff_get_color_primaries(s->in_prm); if (!s->in_primaries) { av_log(ctx, AV_LOG_ERROR, "Unsupported input primaries %d (%s)\n", @@ -494,7 +446,7 @@ static int create_filtergraph(AVFilterContext *ctx, return AVERROR(EINVAL); } s->out_prm = out->color_primaries; - s->out_primaries = get_color_primaries(s->out_prm); + s->out_primaries = ff_get_color_primaries(s->out_prm); if (!s->out_primaries) { if (s->out_prm == AVCOL_PRI_UNSPECIFIED) { if (s->user_all == CS_UNSPECIFIED) { @@ -516,17 +468,17 @@ static int create_filtergraph(AVFilterContext *ctx, double rgb2xyz[3][3], xyz2rgb[3][3], rgb2rgb[3][3]; const struct WhitepointCoefficients *wp_out, *wp_in; - wp_out = &whitepoint_coefficients[s->out_primaries->wp]; - wp_in = &whitepoint_coefficients[s->in_primaries->wp]; - ff_fill_rgb2xyz_table(&s->out_primaries->coeff, wp_out, rgb2xyz); + wp_out = &s->out_primaries->wp; + wp_in = &s->in_primaries->wp; + ff_fill_rgb2xyz_table(&s->out_primaries->prim, wp_out, rgb2xyz); ff_matrix_invert_3x3(rgb2xyz, xyz2rgb); - ff_fill_rgb2xyz_table(&s->in_primaries->coeff, wp_in, rgb2xyz); - if (s->out_primaries->wp != s->in_primaries->wp && + ff_fill_rgb2xyz_table(&s->in_primaries->prim, wp_in, rgb2xyz); + if (memcmp(wp_in, wp_out, sizeof(*wp_in)) != 0 && s->wp_adapt != WP_ADAPT_IDENTITY) { double wpconv[3][3], tmp[3][3]; - fill_whitepoint_conv_table(wpconv, s->wp_adapt, s->in_primaries->wp, - s->out_primaries->wp); + fill_whitepoint_conv_table(wpconv, s->wp_adapt, &s->in_primaries->wp, + &s->out_primaries->wp); ff_matrix_mul_3x3(tmp, rgb2xyz, wpconv); ff_matrix_mul_3x3(rgb2rgb, tmp, xyz2rgb); } else { From patchwork Mon Apr 11 15:36:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 35256 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:671c:b0:7c:62c8:b2d1 with SMTP id q28csp1111629pzh; Mon, 11 Apr 2022 08:37:21 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzpleCmkLxzzaL9UJXiqzhAxL4VDRbGnZ1O9eoi4BfpvJmFp5kQHtwygeXNswThQd1k/id/ X-Received: by 2002:a17:906:478b:b0:6db:8b6e:d5de with SMTP id cw11-20020a170906478b00b006db8b6ed5demr31367273ejc.161.1649691440586; Mon, 11 Apr 2022 08:37:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1649691440; cv=none; d=google.com; s=arc-20160816; b=DTZqU/Ix0wUtFtUnvqCFERNy9KbuG+eqj2Lt02vtRXmDujFk165CmcvAy5L6nMiKco DWbT9X/kCoE3+zMWkhtSwsy8zTYNFG4WgSqZN0ojioUTmwRVcPLgO5POFnA/Y+4sMhgs vkF9Tc97oUwq2x8cTyF1AiGu3xlVi3ijvlIMn36KcgBhMCzYxYH/TtiVAX6XuVN3rRVB ftQ6jfrTubWXcy0N+dfsh92S7nSIHSrkYYjxnVYA4LPiZ1OXaUaC4I1Wn+6PCRNfTF6T +WyKZeHCJ3uw7Y/dHC3uq56C+ZkznBJg6RQ7qfJ8CnLLOppekHGpFA/rq439Xjdslgif Pj+w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=TApD+621N6ycokjyagd2k8YpgfBkgLXRWF8CKutf/OI=; b=RvbXZJLQuXU8VvOugiXyP5YszvIUZG7Xpl1VDxDFvI8nzrZMSJDl+BaZ86fwewb2Po qOukDAezII98rWl+4KRbctLl9aqmC7Ryf90SkuKdHHpIZPpfPCr37rBctp30YfY9J14W /TxwTi9E6x/Sb4AtaYHPYhM0/lfj44gTKv/hF5XGf32TSAcsRXSUkLfXAEj3U0Ao1wjL 6rbaoJlZAnvk0WIOSkwFA/Hfjkv0/EMLiXNKdbFtBngBytk1Y9fI5ppegiLw0LsSy2QU RqvUMzWr5rJhFTefNykfgKnoHemPfDWmZw+P5rlm7z1gGGHL9D45r+SjYKVZnSRc+Uau LFTw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b="SJuOMjB/"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id ko11-20020a170907986b00b006e88a65ed8csi3281676ejc.353.2022.04.11.08.37.19; Mon, 11 Apr 2022 08:37:20 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b="SJuOMjB/"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4D1EB68B2A9; Mon, 11 Apr 2022 18:37:09 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 36DFD68B22C for ; Mon, 11 Apr 2022 18:37:02 +0300 (EEST) Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id 025BC49DEE; Mon, 11 Apr 2022 17:37:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1649691422; bh=0xEp8OImyzjx222JDkL2IiVqvpI01Dmvwnotokm65+Q=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=SJuOMjB/UNfyXBYd6RyyHQrevijFCHGJqEuz7HfInsNhs9kgbVfCqnjgf9kGKst59 PI43665T3Am3up/Zasq4iWjVJrxlzQlu+SHBZB7bhueGjxck5Z2jl8nAXynpvDV9m9 unV9dYl9BE3YDXL6HEVTYaf8PJkIO7hUZ6vmaI6g= From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Mon, 11 Apr 2022 17:36:51 +0200 Message-Id: <20220411153654.116722-2-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220411153654.116722-1-ffmpeg@haasn.xyz> References: <20220411153654.116722-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/5] lavfi: add ff_detect_color_primaries helper X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: zaGPZsm+qUQz From: Niklas Haas Related to #9673, this helper exists to facilitate "guessing" the right primary tags from a given set of raw primary coefficients. The cutoff value of 0.001 was chosen by experimentation. The smallest "false negative" delta observed in practice was 0.023329, while the largest "false positive" delta was 0.00016. So, a value of 0.001 sits comfortably in the middle. Signed-off-by: Niklas Haas --- libavfilter/colorspace.c | 25 +++++++++++++++++++++++++ libavfilter/colorspace.h | 3 +++ 2 files changed, 28 insertions(+) diff --git a/libavfilter/colorspace.c b/libavfilter/colorspace.c index 25e99f4759..8d7b882375 100644 --- a/libavfilter/colorspace.c +++ b/libavfilter/colorspace.c @@ -170,6 +170,31 @@ const struct ColorPrimaries *ff_get_color_primaries(enum AVColorPrimaries prm) return p; } +enum AVColorPrimaries ff_detect_color_primaries(const struct ColorPrimaries *prm) +{ + double delta; + + for (enum AVColorPrimaries p = 0; p < AVCOL_PRI_NB; p++) { + const struct ColorPrimaries *ref = &color_primaries[p]; + if (!ref->prim.xr) + continue; + + delta = fabs(prm->prim.xr - ref->prim.xr) + + fabs(prm->prim.yr - ref->prim.yr) + + fabs(prm->prim.yg - ref->prim.yg) + + fabs(prm->prim.yg - ref->prim.yg) + + fabs(prm->prim.yb - ref->prim.yb) + + fabs(prm->prim.yb - ref->prim.yb) + + fabs(prm->wp.xw - ref->wp.xw) + + fabs(prm->wp.yw - ref->wp.yw); + + if (delta < 0.001) + return p; + } + + return AVCOL_PRI_UNSPECIFIED; +} + void ff_fill_rgb2yuv_table(const struct LumaCoefficients *coeffs, double rgb2yuv[3][3]) { diff --git a/libavfilter/colorspace.h b/libavfilter/colorspace.h index fc415fed05..6959133a49 100644 --- a/libavfilter/colorspace.h +++ b/libavfilter/colorspace.h @@ -49,6 +49,9 @@ void ff_fill_rgb2xyz_table(const struct PrimaryCoefficients *coeffs, const struct WhitepointCoefficients *wp, double rgb2xyz[3][3]); +/* Returns AVCOL_PRI_UNSPECIFIED if no clear match can be identified */ +enum AVColorPrimaries ff_detect_color_primaries(const struct ColorPrimaries *prm); + const struct ColorPrimaries *ff_get_color_primaries(enum AVColorPrimaries prm); const struct LumaCoefficients *ff_get_luma_coefficients(enum AVColorSpace csp); void ff_fill_rgb2yuv_table(const struct LumaCoefficients *coeffs, From patchwork Mon Apr 11 15:36:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 35257 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:671c:b0:7c:62c8:b2d1 with SMTP id q28csp1111714pzh; Mon, 11 Apr 2022 08:37:30 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzpxyX69WTweYuDhuVo8YTPf1XY2ywCQJrO/OMvyAkk+FjktmzPUCMTOAkLyZiEtTqb7C7K X-Received: by 2002:a17:906:174f:b0:6d0:5629:e4be with SMTP id d15-20020a170906174f00b006d05629e4bemr30468324eje.525.1649691450473; Mon, 11 Apr 2022 08:37:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1649691450; cv=none; d=google.com; s=arc-20160816; b=BvrAjwvW5tJOE2qw80wFpqoajTiEBkYPNiBgx/KAi/St8haiJj8dDFuPR3Scvdk2pe 4eR/xFZ6ZPiqF4kIngdThh2kwWuWmBQXTaBfEm3X+d39owfTJjjxZI92F5FSeM4LSwJi akXSA9qcHY1f1jn3rKK4NHF12RORci8qekzHqmcYEUYnTHQe+410yoULrRjaM5gD9oXw 8+yCP83gW1ZKE0vbtVDk8+yEXkW9LMNdmmXvGK2o7WgqpXcCwN1gIrUET+MShLqeXOEg vfedy5JVjVdevaBO9MnvQhsIN1vo3irl5PNKu3WZXjrAqVmvfSNguEjrsejC9gUIf9od Va0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=Fq11wQeZqbiXcz6mrGt3o6RpolOCfvWjh2AKUf+qG9s=; b=fu59s3wRwLbGv16TR/iU7lUPWvB3SDY+nHeYEvRNzE99RQPv/mes+KJmctJmN7jd/O SNjw3sDMbZMVkW1WECit9k4tko8+uqIDdFwSbNyFCyxHhsODKMGiQM+6jDKeuLFvrAve awP00XYPXh63iPyiq6HTFcxqYMux0HertT9BM7+j2Kev9+QivQW7VsCEnm83VcWPtexM aiSVN6G57pvc5cpFL6mEJjvRUvPaorZy0WLNeEvnI5OawwMnGF0XU4lERvCOLBrgPDfY fLoR3bU5glWsfh6wQvc2KXuwvtgpcZS+Cw2qdWuk1Rnbk2lnDgfquFnxfuAhVzKPVcJ6 nMNQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=L6CCnEhz; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id fr3-20020a170906890300b006df76385c98si6498403ejc.312.2022.04.11.08.37.29; Mon, 11 Apr 2022 08:37:30 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=L6CCnEhz; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3834368B28A; Mon, 11 Apr 2022 18:37:10 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8768F68B22C for ; Mon, 11 Apr 2022 18:37:02 +0300 (EEST) Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id 41B6449DEF; Mon, 11 Apr 2022 17:37:02 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1649691422; bh=O1+D4piVWQP3fjy16WD/0/NaG/QURczbpkwGRRm4/cQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=L6CCnEhzUZ7+oSk3DyKX/703rSRDQKG2OLR4vPgxsBdDGv9xlwe6yvcCHkVM+GsVe 5sOjLw4WZR3cUd6TQjt7RDJ+oSHEF4dwsfN8sIM5kLj8c1z2QCYDNtS1CE/v/uzBbg DfqjzIadjuTaK0II8HdicB+7d6IoeiEDIhRzeahE= From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Mon, 11 Apr 2022 17:36:52 +0200 Message-Id: <20220411153654.116722-3-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220411153654.116722-1-ffmpeg@haasn.xyz> References: <20220411153654.116722-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 3/5] lavfi: add ICC profile support via lcms2 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 4Bq0tFNRN3la From: Niklas Haas This introduces an optional dependency on lcms2 into FFmpeg. lcms2 is a widely used library for ICC profile handling, which apart from being used in almost all major image processing programs and video players, has also been deployed in browsers. As such, it's both widely available and well-tested. Add a few helpers to cover our major use cases. This commit merely introduces the helpers (and configure check), even though nothing uses them yet. It's worth pointing out that the reason the cmsToneCurves for each AVCOL_TRC are cached inside the context, is because constructing a cmsToneCurve requires evaluating the curve at 4096 (by default) grid points and constructing a LUT. So, we ideally only want to do this once per curve. This matters for e.g. ff_icc_profile_detect_transfer, which essentially compares a profile against all of these generated LUTs. Re-generating the LUTs for every iteration would be unnecessarily wasteful. The same consideration does not apply to e.g. cmsCreate*Profile, which is a very lightweight operation just involving struct allocation and setting a few pointers. The cutoff value of 0.01 was determined by experimentation. The lowest "false positive" delta I saw in practice was 0.13, and the largest "false negative" delta was 0.0008. So a value of 0.01 sits comfortaby almost exactly in the middle. Signed-off-by: Niklas Haas --- configure | 3 + libavfilter/fflcms2.c | 311 ++++++++++++++++++++++++++++++++++++++++++ libavfilter/fflcms2.h | 87 ++++++++++++ 3 files changed, 401 insertions(+) create mode 100644 libavfilter/fflcms2.c create mode 100644 libavfilter/fflcms2.h diff --git a/configure b/configure index 9c8965852b..1a9c3dcd3c 100755 --- a/configure +++ b/configure @@ -215,6 +215,7 @@ External library support: --disable-iconv disable iconv [autodetect] --enable-jni enable JNI support [no] --enable-ladspa enable LADSPA audio filtering [no] + --enable-lcms2 enable ICC profile support via LittleCMS 2 [no] --enable-libaom enable AV1 video encoding/decoding via libaom [no] --enable-libaribb24 enable ARIB text and caption decoding via libaribb24 [no] --enable-libass enable libass subtitles rendering, @@ -1813,6 +1814,7 @@ EXTERNAL_LIBRARY_LIST=" gnutls jni ladspa + lcms2 libaom libass libbluray @@ -6504,6 +6506,7 @@ enabled gmp && require gmp gmp.h mpz_export -lgmp enabled gnutls && require_pkg_config gnutls gnutls gnutls/gnutls.h gnutls_global_init enabled jni && { [ $target_os = "android" ] && check_headers jni.h && enabled pthreads || die "ERROR: jni not found"; } enabled ladspa && require_headers "ladspa.h dlfcn.h" +enabled lcms2 && require_pkg_config lcms2 "lcms2 >= 2.13" lcms2.h cmsCreateContext enabled libaom && require_pkg_config libaom "aom >= 1.0.0" aom/aom_codec.h aom_codec_version enabled libaribb24 && { check_pkg_config libaribb24 "aribb24 > 1.0.3" "aribb24/aribb24.h" arib_instance_new || { enabled gpl && require_pkg_config libaribb24 aribb24 "aribb24/aribb24.h" arib_instance_new; } || diff --git a/libavfilter/fflcms2.c b/libavfilter/fflcms2.c new file mode 100644 index 0000000000..afde5d87a8 --- /dev/null +++ b/libavfilter/fflcms2.c @@ -0,0 +1,311 @@ +/* + * Copyright (c) 2022 Niklas Haas + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/color_utils.h" + +#include "fflcms2.h" + +static void log_cb(cmsContext ctx, cmsUInt32Number error, const char *str) +{ + FFIccContext *s = cmsGetContextUserData(ctx); + av_log(s->avctx, AV_LOG_ERROR, "lcms2: [%"PRIu32"] %s\n", error, str); +} + +int ff_icc_context_init(FFIccContext *s, void *avctx) +{ + memset(s, 0, sizeof(*s)); + s->avctx = avctx; + s->ctx = cmsCreateContext(NULL, s); + if (!s->ctx) + return AVERROR(ENOMEM); + + cmsSetLogErrorHandlerTHR(s->ctx, log_cb); + return 0; +} + +void ff_icc_context_uninit(FFIccContext *s) +{ + for (int i = 0; i < FF_ARRAY_ELEMS(s->curves); i++) + cmsFreeToneCurve(s->curves[i]); + cmsDeleteContext(s->ctx); + memset(s, 0, sizeof(*s)); +} + +static int get_curve(FFIccContext *s, enum AVColorTransferCharacteristic trc, + cmsToneCurve **out_curve) +{ + if (trc >= AVCOL_TRC_NB) + return AVERROR_INVALIDDATA; + + if (s->curves[trc]) + goto done; + + switch (trc) { + case AVCOL_TRC_LINEAR: + s->curves[trc] = cmsBuildGamma(s->ctx, 1.0); + break; + case AVCOL_TRC_GAMMA22: + s->curves[trc] = cmsBuildGamma(s->ctx, 2.2); + break; + case AVCOL_TRC_GAMMA28: + s->curves[trc] = cmsBuildGamma(s->ctx, 2.8); + break; + case AVCOL_TRC_BT709: + case AVCOL_TRC_SMPTE170M: + case AVCOL_TRC_BT2020_10: + case AVCOL_TRC_BT2020_12: + s->curves[trc] = cmsBuildParametricToneCurve(s->ctx, 4, (double[5]) { + /* γ = */ 1/0.45, + /* a = */ 1/1.099296826809442, + /* b = */ 1 - 1/1.099296826809442, + /* c = */ 1/4.5, + /* d = */ 4.5 * 0.018053968510807, + }); + break; + case AVCOL_TRC_SMPTE240M: + s->curves[trc] = cmsBuildParametricToneCurve(s->ctx, 4, (double[5]) { + /* γ = */ 1/0.45, + /* a = */ 1/1.1115, + /* b = */ 1 - 1/1.1115, + /* c = */ 1/4.0, + /* d = */ 4.0 * 0.0228, + }); + break; + case AVCOL_TRC_LOG: + s->curves[trc] = cmsBuildParametricToneCurve(s->ctx, 8, (double[5]) { + /* a = */ 1.0, + /* b = */ 10.0, + /* c = */ 2.0, + /* d = */ -1.0, + /* e = */ 0.0 + }); + break; + case AVCOL_TRC_LOG_SQRT: + s->curves[trc] = cmsBuildParametricToneCurve(s->ctx, 8, (double[5]) { + /* a = */ 1.0, + /* b = */ 10.0, + /* c = */ 2.5, + /* d = */ -1.0, + /* e = */ 0.0 + }); + break; + case AVCOL_TRC_IEC61966_2_1: + s->curves[trc] = cmsBuildParametricToneCurve(s->ctx, 4, (double[5]) { + /* γ = */ 2.4, + /* a = */ 1/1.055, + /* b = */ 1 - 1/1.055, + /* c = */ 1/12.92, + /* d = */ 12.92 * 0.0031308, + }); + break; + case AVCOL_TRC_SMPTE428: + s->curves[trc] = cmsBuildParametricToneCurve(s->ctx, 2, (double[3]) { + /* γ = */ 2.6, + /* a = */ pow(52.37/48.0, 1/2.6), + /* b = */ 0.0 + }); + break; + + /* Can't be represented using the existing parametric tone curves. + * FIXME: use cmsBuildTabulatedToneCurveFloat instead */ + case AVCOL_TRC_IEC61966_2_4: + case AVCOL_TRC_BT1361_ECG: + case AVCOL_TRC_SMPTE2084: + case AVCOL_TRC_ARIB_STD_B67: + return AVERROR_PATCHWELCOME; + + default: + return AVERROR_INVALIDDATA; + } + + if (!s->curves[trc]) + return AVERROR(ENOMEM); + +done: + *out_curve = s->curves[trc]; + return 0; +} + +int ff_icc_profile_generate(FFIccContext *s, + enum AVColorPrimaries color_prim, + enum AVColorTransferCharacteristic color_trc, + cmsHPROFILE *out_profile) +{ + cmsToneCurve *tonecurve; + const struct ColorPrimaries *prim; + int ret; + + if (!(prim = ff_get_color_primaries(color_prim))) + return AVERROR_INVALIDDATA; + if ((ret = get_curve(s, color_trc, &tonecurve)) < 0) + return ret; + + *out_profile = cmsCreateRGBProfileTHR(s->ctx, + &(cmsCIExyY) { prim->wp.xw, prim->wp.yw, 1.0 }, + &(cmsCIExyYTRIPLE) { + .Red = { prim->prim.xr, prim->prim.yr, 1.0 }, + .Green = { prim->prim.xg, prim->prim.yg, 1.0 }, + .Blue = { prim->prim.xb, prim->prim.yb, 1.0 }, + }, + (cmsToneCurve *[3]) { tonecurve, tonecurve, tonecurve } + ); + + return *out_profile == NULL ? AVERROR_EXTERNAL : 0; +} + +int ff_icc_profile_attach(FFIccContext *s, cmsHPROFILE profile, AVFrame *frame) +{ + cmsUInt32Number size; + AVBufferRef *buf; + int ret; + + if (!cmsSaveProfileToMem(profile, NULL, &size)) + return AVERROR_EXTERNAL; + + buf = av_buffer_alloc(size); + if (!buf) + return AVERROR(ENOMEM); + + if (!cmsSaveProfileToMem(profile, buf->data, &size) || size != buf->size) { + av_buffer_unref(&buf); + return AVERROR_EXTERNAL; + } + + if (!av_frame_new_side_data_from_buf(frame, AV_FRAME_DATA_ICC_PROFILE, buf)) { + av_buffer_unref(&buf); + return AVERROR(ENOMEM); + } + + return ret; +} + +static av_always_inline void XYZ_xy(cmsCIEXYZ XYZ, double *x, double *y) +{ + double k = 1.0 / (XYZ.X + XYZ.Y + XYZ.Z); + *x = k * XYZ.X; + *y = k * XYZ.Y; +} + +int ff_icc_profile_read_primaries(FFIccContext *s, cmsHPROFILE profile, + struct ColorPrimaries *out_primaries) +{ + static const uint8_t testprimaries[4][3] = { + { 0xFF, 0, 0 }, /* red */ + { 0, 0xFF, 0 }, /* green */ + { 0, 0, 0xFF }, /* blue */ + { 0xFF, 0xFF, 0xFF }, /* white */ + }; + + struct WhitepointCoefficients *wp = &out_primaries->wp; + struct PrimaryCoefficients *prim = &out_primaries->prim; + cmsFloat64Number prev_adapt; + cmsHPROFILE xyz; + cmsHTRANSFORM tf; + cmsCIEXYZ dst[4]; + + xyz = cmsCreateXYZProfileTHR(s->ctx); + if (!xyz) + return AVERROR(ENOMEM); + + /* We need to use an unadapted observer to get the raw values */ + prev_adapt = cmsSetAdaptationStateTHR(s->ctx, 0.0); + tf = cmsCreateTransformTHR(s->ctx, profile, TYPE_RGB_8, xyz, TYPE_XYZ_DBL, + INTENT_ABSOLUTE_COLORIMETRIC, + /* Note: These flags mostly don't do anything + * anyway, but specify them regardless */ + cmsFLAGS_NOCACHE | + cmsFLAGS_NOOPTIMIZE | + cmsFLAGS_LOWRESPRECALC | + cmsFLAGS_GRIDPOINTS(2)); + cmsSetAdaptationStateTHR(s->ctx, prev_adapt); + cmsCloseProfile(xyz); + if (!tf) { + av_log(s->avctx, AV_LOG_ERROR, "Invalid ICC profile (e.g. CMYK)\n"); + return AVERROR_INVALIDDATA; + } + + cmsDoTransform(tf, testprimaries, dst, 4); + cmsDeleteTransform(tf); + XYZ_xy(dst[0], &prim->xr, &prim->yr); + XYZ_xy(dst[1], &prim->xg, &prim->yg); + XYZ_xy(dst[2], &prim->xb, &prim->yb); + XYZ_xy(dst[3], &wp->xw, &wp->yw); + return 0; +} + +int ff_icc_profile_detect_transfer(FFIccContext *s, cmsHPROFILE profile, + enum AVColorTransferCharacteristic *out_trc) +{ + /* 8-bit linear grayscale ramp */ + static const uint8_t testramp[16][3] = { + { 1, 1, 1}, /* avoid exact zero due to log100 etc. */ + { 17, 17, 17}, + { 34, 34, 34}, + { 51, 51, 51}, + { 68, 68, 68}, + { 85, 85, 85}, + { 02, 02, 02}, + {119, 119, 119}, + {136, 136, 136}, + {153, 153, 153}, + {170, 170, 170}, + {187, 187, 187}, + {204, 204, 204}, + {221, 221, 221}, + {238, 238, 238}, + {255, 255, 255}, + }; + + double dst[FF_ARRAY_ELEMS(testramp)]; + + for (enum AVColorTransferCharacteristic trc = 0; trc < AVCOL_TRC_NB; trc++) { + cmsToneCurve *tonecurve; + cmsHPROFILE ref; + cmsHTRANSFORM tf; + double delta = 0.0; + if (get_curve(s, trc, &tonecurve) < 0) + continue; + + ref = cmsCreateGrayProfileTHR(s->ctx, cmsD50_xyY(), tonecurve); + if (!ref) + return AVERROR(ENOMEM); + + tf = cmsCreateTransformTHR(s->ctx, profile, TYPE_RGB_8, ref, TYPE_GRAY_DBL, + INTENT_RELATIVE_COLORIMETRIC, + cmsFLAGS_NOCACHE | cmsFLAGS_NOOPTIMIZE); + cmsCloseProfile(ref); + if (!tf) { + av_log(s->avctx, AV_LOG_ERROR, "Invalid ICC profile (e.g. CMYK)\n"); + return AVERROR_INVALIDDATA; + } + + cmsDoTransform(tf, testramp, dst, FF_ARRAY_ELEMS(dst)); + cmsDeleteTransform(tf); + + for (int i = 0; i < FF_ARRAY_ELEMS(dst); i++) + delta += fabs(testramp[i][0] / 255.0 - dst[i]); + if (delta < 0.01) { + *out_trc = trc; + return 0; + } + } + + *out_trc = AVCOL_TRC_UNSPECIFIED; + return 0; +} diff --git a/libavfilter/fflcms2.h b/libavfilter/fflcms2.h new file mode 100644 index 0000000000..ad6c8c47cf --- /dev/null +++ b/libavfilter/fflcms2.h @@ -0,0 +1,87 @@ +/* + * Copyright (c) 2022 Niklas Haas + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * Various functions for dealing with ICC profiles + */ + +#ifndef AVFILTER_FFLCMS2_H +#define AVFILTER_FFLCMS2_H + +#include "libavutil/frame.h" +#include "libavutil/pixfmt.h" +#include "colorspace.h" + +#include + +typedef struct FFIccContext { + void *avctx; + cmsContext ctx; + cmsToneCurve *curves[AVCOL_TRC_NB]; /* tone curve cache */ +} FFIccContext; + +/** + * Initializes an FFIccContext. This must be done prior to using it. + * + * Returns 0 on success, or a negative error code. + */ +int ff_icc_context_init(FFIccContext *s, void *avctx); +void ff_icc_context_uninit(FFIccContext *s); + +/** + * Generate an ICC profile for a given combination of color primaries and + * transfer function. Both values must be set to valid entries (not + * "undefined") for this function to work. + * + * Returns 0 on success, or a negative error code. + */ +int ff_icc_profile_generate(FFIccContext *s, + enum AVColorPrimaries color_prim, + enum AVColorTransferCharacteristic color_trc, + cmsHPROFILE *out_profile); + +/** + * Attach an ICC profile to a frame. Helper wrapper around cmsSaveProfileToMem + * and av_frame_new_side_data_from_buf. + * + * Returns 0 on success, or a negative error code. + */ +int ff_icc_profile_attach(FFIccContext *s, cmsHPROFILE profile, AVFrame *frame); + +/** + * Read the color primaries and white point coefficients encoded by an ICC + * profile, and return the raw values in `out_primaries`. + * + * Returns 0 on success, or a negative error code. + */ +int ff_icc_profile_read_primaries(FFIccContext *s, cmsHPROFILE profile, + struct ColorPrimaries *out_primaries); + +/** + * Attempt detecting the transfer characteristic that best approximates the + * transfer function encoded by an ICC profile. Sets `out_trc` to + * AVCOL_TRC_UNSPECIFIED if no clear match can be identified. + * + * Returns 0 on success (including no match), or a negative error code. + */ +int ff_icc_profile_detect_transfer(FFIccContext *s, cmsHPROFILE profile, + enum AVColorTransferCharacteristic *out_trc); + +#endif /* AVFILTER_FFLCMS2_H */ From patchwork Mon Apr 11 15:36:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 35258 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:671c:b0:7c:62c8:b2d1 with SMTP id q28csp1111783pzh; Mon, 11 Apr 2022 08:37:39 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxEQwPj7nSPp8B5oUTATLwMIiO/aIYyVXw9nwum32ZGVI1/j46HnuD0mepq5g24GVuYYGbF X-Received: by 2002:a17:906:60c2:b0:6e7:681e:b4b7 with SMTP id f2-20020a17090660c200b006e7681eb4b7mr30306287ejk.130.1649691459739; Mon, 11 Apr 2022 08:37:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1649691459; cv=none; d=google.com; s=arc-20160816; b=VHaRMjVteX1KkmtuB7L7PW6OhvHm/sq67dxdHHGin/nvWgYNFvkSBag34jiQsp0eXt kfDcgMiEtw1/MS+sOkWmHU1CgV8nJeCJ3BJcIjasQXnfXGsqa8VvbeAVgdyoc3Tqf2FL g51JVjzK7t0DbMO1Y4n0oVQeltNpeoDf6TOxuajxyF1I+xlG3gL3ZvJ3u+JAVvpI7zXM /1cO189TuOMOLAqFbbOlDa5KbJWOQHhXxnV3As/qZJrt5oryJYHCSuZANfhGNBraFNFV GdWPR21Vj1hkEbdblv3s57HTUnmF17rjy59xju3t37PjkOPCZmA9i2Mc9R1IJy9AD1Dr 34iw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=BjRgtpNvSuDzQqHzYtUInrIRO0AHGmvxugXTCIW82bE=; b=o8nZfv8lEm4JXbv8g+nw5Z81Qx91wwbU3/9j/ihw8NiaS84wU6s46yoWdrC22gaYAs F/m81ohIdKNp4j0I+k2H6yfxF6MSc0eSIi9YJMFqSjbC2RkzGcrZQQqJnlFljuX6hVcf ryjD4BM7D+FfJEZvmqc/XfASgS1PyyZxsdHZP240VTwgD6CUCwZcYkGdxu/g6mPYqN3b +MgetvdJzg0+NyUsX1UGytprFLeLnCeDxt2o4O9ZmQMKxUzPbBs6Hi+u9O7HPqEUVVP1 HRb0erigDR9egWOzm1pQCkAzqy4YbHTz0emflr9bnJXkDNj5Ux7t8CqX3e3JD5KbBRH8 G+mg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=Q+j7GhAa; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id l24-20020a1709062a9800b006df76385d13si6873459eje.435.2022.04.11.08.37.39; Mon, 11 Apr 2022 08:37:39 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=Q+j7GhAa; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2934C68B2E2; Mon, 11 Apr 2022 18:37:11 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B43DE68B221 for ; Mon, 11 Apr 2022 18:37:02 +0300 (EEST) Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id 780C049DF0; Mon, 11 Apr 2022 17:37:02 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1649691422; bh=2bxE5glkuMRcxbEEw6cbJn/WcqR8LijIczZgdWiB/u4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Q+j7GhAa3pY7RviEL2n2uaQA4Wm4p3G/u7oTWZGcf6Sd7n3pMDzSbJvLpRuaRJiLu 5o4GgdV7+0FTgev4sherVEht+CSTwKMwkxXbo9VdfREC4djSzwpfKTvBX3C0KHiLld kG0+nwdCveIvKP2rGXz9InrJxyR6EsgEOUskmFhE= From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Mon, 11 Apr 2022 17:36:53 +0200 Message-Id: <20220411153654.116722-4-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220411153654.116722-1-ffmpeg@haasn.xyz> References: <20220411153654.116722-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 4/5] lavfi: add vf_iccgen for generating ICC profiles X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: TpCCZF3JXdyy From: Niklas Haas This filter is designed to specifically cover the task of generating ICC profiles (and attaching them to output frames) on demand. Other tasks, such as ICC profile loading/stripping, or ICC profile application, are better left to separate filters (or included into e.g. vf_setparams). Signed-off-by: Niklas Haas --- configure | 1 + doc/filters.texi | 21 +++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/vf_iccgen.c | 179 +++++++++++++++++++++++++++++++++++++++ 5 files changed, 203 insertions(+) create mode 100644 libavfilter/vf_iccgen.c diff --git a/configure b/configure index 1a9c3dcd3c..d64ccf7181 100755 --- a/configure +++ b/configure @@ -3659,6 +3659,7 @@ gblur_vulkan_filter_deps="vulkan spirv_compiler" hflip_vulkan_filter_deps="vulkan spirv_compiler" histeq_filter_deps="gpl" hqdn3d_filter_deps="gpl" +iccgen_filter_deps="lcms2" interlace_filter_deps="gpl" kerndeint_filter_deps="gpl" ladspa_filter_deps="ladspa libdl" diff --git a/doc/filters.texi b/doc/filters.texi index 4e9b0e0111..9673858355 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -14304,6 +14304,27 @@ By default value is 0. The @code{hysteresis} filter also supports the @ref{framesync} options. +@section iccgen + +Generate ICC profiles and attach them to frames. + +This filter accepts the following options: + +@table @option +@item color_primaries +@item color_trc +Configure the colorspace that the ICC profile will be generated for. The +default value of @code{auto} infers the value from the input frame's metadata, +defaulting to BT.709/sRGB as appropriate. + +See the @ref{setparams} filter for a list of possible values, but note that +@code{unknown} are not valid values for this filter. + +@item force +If true, an ICC profile will be generated even if it would overwrite an +already existing ICC profile. Disabled by default. +@end table + @section identity Obtain the identity score between two input videos. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index c4c946a988..8ffc53d751 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -320,6 +320,7 @@ OBJS-$(CONFIG_HWMAP_FILTER) += vf_hwmap.o OBJS-$(CONFIG_HWUPLOAD_CUDA_FILTER) += vf_hwupload_cuda.o OBJS-$(CONFIG_HWUPLOAD_FILTER) += vf_hwupload.o OBJS-$(CONFIG_HYSTERESIS_FILTER) += vf_hysteresis.o framesync.o +OBJS-$(CONFIG_ICCGEN_FILTER) += vf_iccgen.o fflcms2.o colorspace.o OBJS-$(CONFIG_IDENTITY_FILTER) += vf_identity.o OBJS-$(CONFIG_IDET_FILTER) += vf_idet.o OBJS-$(CONFIG_IL_FILTER) += vf_il.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 9fbaaacf47..d43f4b45b1 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -303,6 +303,7 @@ extern const AVFilter ff_vf_hwmap; extern const AVFilter ff_vf_hwupload; extern const AVFilter ff_vf_hwupload_cuda; extern const AVFilter ff_vf_hysteresis; +extern const AVFilter ff_vf_iccgen; extern const AVFilter ff_vf_identity; extern const AVFilter ff_vf_idet; extern const AVFilter ff_vf_il; diff --git a/libavfilter/vf_iccgen.c b/libavfilter/vf_iccgen.c new file mode 100644 index 0000000000..afc924e291 --- /dev/null +++ b/libavfilter/vf_iccgen.c @@ -0,0 +1,179 @@ +/* + * Copyright (c) 2022 Niklas Haas + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * filter for generating ICC profiles + */ + +#include + +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" + +#include "avfilter.h" +#include "fflcms2.h" +#include "internal.h" + +typedef struct IccGenContext { + const AVClass *class; + FFIccContext icc; + /* options */ + int color_prim; + int color_trc; + int force; + /* (cached) generated ICC profile */ + cmsHPROFILE profile; + int profile_prim; + int profile_trc; +} IccGenContext; + +#define OFFSET(x) offsetof(IccGenContext, x) +#define VF AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_FILTERING_PARAM + +static const AVOption iccgen_options[] = { + {"color_primaries", "select color primaries", OFFSET(color_prim), AV_OPT_TYPE_INT, {.i64=0}, 0, AVCOL_PRI_NB-1, VF, "color_primaries"}, + {"auto", "infer based on frame", 0, AV_OPT_TYPE_CONST, {.i64=0}, 0, 0, VF, "color_primaries"}, + {"bt709", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_PRI_BT709}, 0, 0, VF, "color_primaries"}, + {"bt470m", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_PRI_BT470M}, 0, 0, VF, "color_primaries"}, + {"bt470bg", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_PRI_BT470BG}, 0, 0, VF, "color_primaries"}, + {"smpte170m", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_PRI_SMPTE170M}, 0, 0, VF, "color_primaries"}, + {"smpte240m", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_PRI_SMPTE240M}, 0, 0, VF, "color_primaries"}, + {"film", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_PRI_FILM}, 0, 0, VF, "color_primaries"}, + {"bt2020", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_PRI_BT2020}, 0, 0, VF, "color_primaries"}, + {"smpte428", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_PRI_SMPTE428}, 0, 0, VF, "color_primaries"}, + {"smpte431", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_PRI_SMPTE431}, 0, 0, VF, "color_primaries"}, + {"smpte432", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_PRI_SMPTE432}, 0, 0, VF, "color_primaries"}, + {"jedec-p22", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_PRI_JEDEC_P22}, 0, 0, VF, "color_primaries"}, + {"ebu3213", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_PRI_EBU3213}, 0, 0, VF, "color_primaries"}, + {"color_trc", "select color transfer", OFFSET(color_trc), AV_OPT_TYPE_INT, {.i64=0}, 0, AVCOL_TRC_NB-1, VF, "color_trc"}, + {"auto", "infer based on frame", 0, AV_OPT_TYPE_CONST, {.i64=0}, 0, 0, VF, "color_trc"}, + {"bt709", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_TRC_BT709}, 0, 0, VF, "color_trc"}, + {"bt470m", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_TRC_GAMMA22}, 0, 0, VF, "color_trc"}, + {"bt470bg", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_TRC_GAMMA28}, 0, 0, VF, "color_trc"}, + {"smpte170m", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_TRC_SMPTE170M}, 0, 0, VF, "color_trc"}, + {"smpte240m", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_TRC_SMPTE240M}, 0, 0, VF, "color_trc"}, + {"linear", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_TRC_LINEAR}, 0, 0, VF, "color_trc"}, + {"iec61966-2-4", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_TRC_IEC61966_2_4}, 0, 0, VF, "color_trc"}, + {"bt1361e", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_TRC_BT1361_ECG}, 0, 0, VF, "color_trc"}, + {"iec61966-2-1", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_TRC_IEC61966_2_1}, 0, 0, VF, "color_trc"}, + {"bt2020-10", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_TRC_BT2020_10}, 0, 0, VF, "color_trc"}, + {"bt2020-12", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_TRC_BT2020_12}, 0, 0, VF, "color_trc"}, + {"smpte2084", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_TRC_SMPTE2084}, 0, 0, VF, "color_trc"}, + {"arib-std-b67", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_TRC_ARIB_STD_B67}, 0, 0, VF, "color_trc"}, + { "force", "overwrite existing ICC profile", OFFSET(force), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, VF }, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(iccgen); + +static av_cold void iccgen_uninit(AVFilterContext *avctx) +{ + IccGenContext *s = avctx->priv; + cmsCloseProfile(s->profile); + ff_icc_context_uninit(&s->icc); +} + +static av_cold int iccgen_init(AVFilterContext *avctx) +{ + IccGenContext *s = avctx->priv; + return ff_icc_context_init(&s->icc, avctx); +} + +static int iccgen_filter_frame(AVFilterLink *inlink, AVFrame *frame) +{ + AVFilterContext *avctx = inlink->dst; + IccGenContext *s = avctx->priv; + enum AVColorTransferCharacteristic trc; + enum AVColorPrimaries prim; + int ret; + + if (av_frame_get_side_data(frame, AV_FRAME_DATA_ICC_PROFILE)) { + if (s->force) { + av_frame_remove_side_data(frame, AV_FRAME_DATA_ICC_PROFILE); + } else { + return ff_filter_frame(inlink->dst->outputs[0], frame); + } + } + + trc = s->color_trc ? s->color_trc : frame->color_trc; + if (!trc || trc == AVCOL_TRC_UNSPECIFIED) { + const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(frame->format); + if (!desc) + return AVERROR_INVALIDDATA; + + if ((desc->flags & AV_PIX_FMT_FLAG_RGB) || frame->color_range == AVCOL_RANGE_JPEG) { + /* Default to sRGB for RGB or full-range content */ + trc = AVCOL_TRC_IEC61966_2_1; + } else { + /* Default to an ITU-R transfer depending on the bit-depth */ + trc = desc->comp[0].depth >= 12 ? AVCOL_TRC_BT2020_12 + : desc->comp[0].depth >= 10 ? AVCOL_TRC_BT2020_10 + : AVCOL_TRC_BT709; + } + } + + prim = s->color_prim ? s->color_prim : frame->color_primaries; + if (!prim || prim == AVCOL_PRI_UNSPECIFIED) { + /* Simply always default to sRGB/BT.709 primaries to avoid surprises */ + prim = AVCOL_PRI_BT709; + } + + if (s->profile && prim != s->profile_prim && trc != s->profile_trc) { + cmsCloseProfile(s->profile); + s->profile = NULL; + } + + if (!s->profile) { + if ((ret = ff_icc_profile_generate(&s->icc, prim, trc, &s->profile)) < 0) + return ret; + } + + if ((ret = ff_icc_profile_attach(&s->icc, s->profile, frame)) < 0) + return ret; + + return ff_filter_frame(inlink->dst->outputs[0], frame); +} + +static const AVFilterPad iccgen_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .filter_frame = iccgen_filter_frame, + }, +}; + +static const AVFilterPad iccgen_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + }, +}; + +const AVFilter ff_vf_iccgen = { + .name = "iccgen", + .description = NULL_IF_CONFIG_SMALL("Generate and attach ICC profiles."), + .priv_size = sizeof(IccGenContext), + .priv_class = &iccgen_class, + .flags = AVFILTER_FLAG_METADATA_ONLY, + .init = &iccgen_init, + .uninit = &iccgen_uninit, + FILTER_INPUTS(iccgen_inputs), + FILTER_OUTPUTS(iccgen_outputs), +}; From patchwork Mon Apr 11 15:36:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 35259 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:671c:b0:7c:62c8:b2d1 with SMTP id q28csp1111873pzh; Mon, 11 Apr 2022 08:37:50 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxIUvvtD6W4VFWV8YP/TtE8bHLoLJPU7JwuqQIDc4j9gXah0QR7UGHJK6p75DBWT7PPs9qI X-Received: by 2002:a17:906:5d03:b0:6df:a042:d6d5 with SMTP id g3-20020a1709065d0300b006dfa042d6d5mr30053979ejt.678.1649691470281; Mon, 11 Apr 2022 08:37:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1649691470; cv=none; d=google.com; s=arc-20160816; b=ZIZOoBOjtK3DRvfDmXk/DmiD93M6fYPgJ39yigTXrzih+sWvL1O9m0HyL4ECuMI6ID QEnAGwKdWmMYbpOkQzuuA3rB0hBiRuDqW6fE0HTfTyX8rrAD2dZjaX/nn7exseveb19N 1QoPxONuZs/je/y401uGFHHtzoqbJVdvhNzvgeYXGVkbUFOmlTMwq/+j66vFD8FSnlLY RrmYfE9SFyCHlYfLF9PGhl3t3L1d0BBOtV6AEoS4JvVSvRLJNBvqUSUhIlChBbEebKrp vGaPIACQQt8x+cjZVOV+W3W6TDhM8bSHXweTlxoUF7uxSUQdi1j+128J3UyqLO1R4EJJ tA1A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=JOppGXPzY0zebK0Revu4pvnSxyrXvkWHPsuAzzKyB8Y=; b=e1mPUQTdeFyRnV8D1zIsohTjc7TZN5w/PaGYQKo7YB+tj3wiPPpsbeq40pXMYn3GFv Eu0ezOQ2GdKo7jRR2J3tEbqDRyuDrasotcD9iKq9qUnvHCEr7vLgTkAJGjJB5zS7kLvq yjLYxuR40Odv9OSR3nvzz/PasCUebiTotPlW07WJTt1xiM0eChkqvHfspPFb75MnnCP1 XnFOi0hpjQkWjxZBS3wpILQhbl/62u0ouc8CwK/kFe1lM3+rK3W5g41lxiv1jv8aZ/Ck hEC+ZL+EUJo47vCLLBclyFEpKVRocy2odmJev79Zc8hDdE+7fCWaNx8yawszZgDqhxQh 89xA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=j8NfftXm; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id jg10-20020a170907970a00b006dfa8e11aa0si7068200ejc.792.2022.04.11.08.37.49; Mon, 11 Apr 2022 08:37:50 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=j8NfftXm; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0F8B568B272; Mon, 11 Apr 2022 18:37:12 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0C0DC68B221 for ; Mon, 11 Apr 2022 18:37:03 +0300 (EEST) Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id B2FAD49DF1; Mon, 11 Apr 2022 17:37:02 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1649691422; bh=HsWdBjj3qoewaacpPqjdIyTzNcK4Xk95j/1GoEXIyF4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=j8NfftXm5DG6PGjhVKBM/5cWfXl5wseecmynUCv/Fhhdm9Y6Bp+6ASIT9XtBJ0B9G dqStMTqh7FxXvPDbufdA2MnzN8dke0a97VxQiIs2oCQ1XdQFSu34b5knOsoLFbVf+X FFgArSF42MyZbkt44vWZfQrP/fAEvWeeT6lQv74c= From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Mon, 11 Apr 2022 17:36:54 +0200 Message-Id: <20220411153654.116722-5-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220411153654.116722-1-ffmpeg@haasn.xyz> References: <20220411153654.116722-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 5/5] lavfi: add vf_iccdetect for parsing ICC profiles X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: ylGOayRFnY4W From: Niklas Haas This filter is designed to parse embedded ICC profiles and attempt extracting colorspace tags from them, updating the AVFrame metadata accordingly. This is intentionally made a separate filter, rather than being part of libavcodec itself, so that it's an opt-in behavior for the time being. This also gives the user more flexibility to e.g. first attach an ICC profile and then also set the colorspace tags from it. This makes #9673 possible, though not automatic. Signed-off-by: Niklas Haas --- configure | 1 + doc/filters.texi | 14 ++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/vf_iccdetect.c | 141 +++++++++++++++++++++++++++++++++++++ 5 files changed, 158 insertions(+) create mode 100644 libavfilter/vf_iccdetect.c diff --git a/configure b/configure index d64ccf7181..c1e2c53b7f 100755 --- a/configure +++ b/configure @@ -3659,6 +3659,7 @@ gblur_vulkan_filter_deps="vulkan spirv_compiler" hflip_vulkan_filter_deps="vulkan spirv_compiler" histeq_filter_deps="gpl" hqdn3d_filter_deps="gpl" +iccdetect_filter_deps="lcms2" iccgen_filter_deps="lcms2" interlace_filter_deps="gpl" kerndeint_filter_deps="gpl" diff --git a/doc/filters.texi b/doc/filters.texi index 9673858355..67054458f9 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -14304,6 +14304,20 @@ By default value is 0. The @code{hysteresis} filter also supports the @ref{framesync} options. +@section iccdetect + +Detect the colorspace from an embedded ICC profile (if present), and update +the frame's tags accordingly. + +This filter accepts the following options: + +@table @option +@item force +If true, the frame's existing colorspace tags will always be overridden by +values detected from an ICC profile. Otherwise, they will only be assigned if +they contain @code{unknown}. Enabled by default. +@end table + @section iccgen Generate ICC profiles and attach them to frames. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 8ffc53d751..e82b0b373a 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -320,6 +320,7 @@ OBJS-$(CONFIG_HWMAP_FILTER) += vf_hwmap.o OBJS-$(CONFIG_HWUPLOAD_CUDA_FILTER) += vf_hwupload_cuda.o OBJS-$(CONFIG_HWUPLOAD_FILTER) += vf_hwupload.o OBJS-$(CONFIG_HYSTERESIS_FILTER) += vf_hysteresis.o framesync.o +OBJS-$(CONFIG_ICCDETECT_FILTER) += vf_iccdetect.o fflcms2.o colorspace.o OBJS-$(CONFIG_ICCGEN_FILTER) += vf_iccgen.o fflcms2.o colorspace.o OBJS-$(CONFIG_IDENTITY_FILTER) += vf_identity.o OBJS-$(CONFIG_IDET_FILTER) += vf_idet.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index d43f4b45b1..37661d26f9 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -303,6 +303,7 @@ extern const AVFilter ff_vf_hwmap; extern const AVFilter ff_vf_hwupload; extern const AVFilter ff_vf_hwupload_cuda; extern const AVFilter ff_vf_hysteresis; +extern const AVFilter ff_vf_iccdetect; extern const AVFilter ff_vf_iccgen; extern const AVFilter ff_vf_identity; extern const AVFilter ff_vf_idet; diff --git a/libavfilter/vf_iccdetect.c b/libavfilter/vf_iccdetect.c new file mode 100644 index 0000000000..8850857ad2 --- /dev/null +++ b/libavfilter/vf_iccdetect.c @@ -0,0 +1,141 @@ +/* + * Copyright (c) 2022 Niklas Haas + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * filter for generating ICC profiles + */ + +#include + +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" + +#include "avfilter.h" +#include "fflcms2.h" +#include "internal.h" + +typedef struct IccDetectContext { + const AVClass *class; + FFIccContext icc; + int force; + /* (cached) detected ICC profile values */ + AVBufferRef *profile; + enum AVColorPrimaries profile_prim; + enum AVColorTransferCharacteristic profile_trc; +} IccDetectContext; + +#define OFFSET(x) offsetof(IccDetectContext, x) +#define VF AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_FILTERING_PARAM + +static const AVOption iccdetect_options[] = { + { "force", "overwrite existing tags", OFFSET(force), AV_OPT_TYPE_BOOL, {.i64=1}, 0, 1, VF }, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(iccdetect); + +static av_cold void iccdetect_uninit(AVFilterContext *avctx) +{ + IccDetectContext *s = avctx->priv; + av_buffer_unref(&s->profile); + ff_icc_context_uninit(&s->icc); +} + +static av_cold int iccdetect_init(AVFilterContext *avctx) +{ + IccDetectContext *s = avctx->priv; + return ff_icc_context_init(&s->icc, avctx); +} + +static int iccdetect_filter_frame(AVFilterLink *inlink, AVFrame *frame) +{ + AVFilterContext *avctx = inlink->dst; + IccDetectContext *s = avctx->priv; + const AVFrameSideData *sd; + struct ColorPrimaries coeffs; + cmsHPROFILE profile; + int ret; + + sd = av_frame_get_side_data(frame, AV_FRAME_DATA_ICC_PROFILE); + if (!sd) + return ff_filter_frame(inlink->dst->outputs[0], frame); + + if (s->profile && s->profile->data == sd->buf->data) { + /* No change from previous ICC profile */ + goto done; + } + + av_buffer_replace(&s->profile, sd->buf); + s->profile_prim = AVCOL_PRI_UNSPECIFIED; + s->profile_trc = AVCOL_TRC_UNSPECIFIED; + + profile = cmsOpenProfileFromMemTHR(s->icc.ctx, sd->data, sd->size); + if (!profile) + return AVERROR_INVALIDDATA; + + ret = ff_icc_profile_read_primaries(&s->icc, profile, &coeffs); + if (!ret) + ret = ff_icc_profile_detect_transfer(&s->icc, profile, &s->profile_trc); + cmsCloseProfile(profile); + if (ret < 0) + return ret; + + s->profile_prim = ff_detect_color_primaries(&coeffs); + +done: + if (s->profile_prim != AVCOL_PRI_UNSPECIFIED) { + if (s->force || frame->color_primaries == AVCOL_PRI_UNSPECIFIED) + frame->color_primaries = s->profile_prim; + } + + if (s->profile_trc != AVCOL_TRC_UNSPECIFIED) { + if (s->force || frame->color_trc == AVCOL_TRC_UNSPECIFIED) + frame->color_trc = s->profile_trc; + } + + return ff_filter_frame(inlink->dst->outputs[0], frame); +} + +static const AVFilterPad iccdetect_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .filter_frame = iccdetect_filter_frame, + }, +}; + +static const AVFilterPad iccdetect_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + }, +}; + +const AVFilter ff_vf_iccdetect = { + .name = "iccdetect", + .description = NULL_IF_CONFIG_SMALL("Detect and parse ICC profiles."), + .priv_size = sizeof(IccDetectContext), + .priv_class = &iccdetect_class, + .flags = AVFILTER_FLAG_METADATA_ONLY, + .init = &iccdetect_init, + .uninit = &iccdetect_uninit, + FILTER_INPUTS(iccdetect_inputs), + FILTER_OUTPUTS(iccdetect_outputs), +};