From patchwork Tue May 3 15:44:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leo Izen X-Patchwork-Id: 35563 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:a885:b0:7f:4be2:bd17 with SMTP id ca5csp481112pzb; Tue, 3 May 2022 08:44:53 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwhAmnvfJepdXdf8W3LtDJslCiwH80UvqclkM1mWVUnx3m/dXMOo05F3LbPxqXpF/gP8ne4 X-Received: by 2002:a17:907:3d91:b0:6df:a01c:f7cd with SMTP id he17-20020a1709073d9100b006dfa01cf7cdmr16129087ejc.255.1651592693734; Tue, 03 May 2022 08:44:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1651592693; cv=none; d=google.com; s=arc-20160816; b=bykK/qLKwIFBP53p7pCYrhdmWrVtQe2WMCNbGUM02V9LLS0lsbL+JLq39u8DzeJOSq VpJU+7EU+MM1W0wncRpIZEiYwkHe/R4Rt8/Ipt9ln1/QdCvCvebRwQ1BDnkgkusuF0HI wEUvH1qLLQTMxRgRVkkbVcWI5Hv9+nzjsyxqNEZAkI7nprZRugeL/ixpBXjPMG7Xlm6S FEq6iczRM5/UIPdDNtTgFXX6vxNNxT9a4Aqm86j7SJTXK6Ts5e+0Pb+muQokkC/ufQws 0/Z1yBcQErrHIel2VlMGS0jznd6RNlNSGYvYJGiTMZwGm+av9ii7P0aWrE8b95higDGB FzaQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=YfaCJdmOq+Nj9LnPfQtRNPlZ5qZEn7cC93UbrjIC/Hk=; b=By9Uz2lq70spy7D3cvcC0BrUYEBmZgDaxkdFrfHL69GjDBXeZiu6vF1KeUM9YyWXvu jtISnrktqrRQlwCgoLSGeFAIMmxRraV9DeFNQldKFMZ5OIRvSY4tQER2kVGBStk+qPbM Spop0qzTgE3eU2gebxS1eP1XgiAwelLqHphpVq5ZgV/IGiDtSYjHalT7j7IxSxJpNHLU axDJAAzWR0GsD3sUDfo1sTrScBIACWmfY7pUTq2lMVEb2HTdRadlNIgY0YH8LETsFkqx qzynwNdYQ2nfdkLXj6HNGCOFc9ITDCACB5fx+4mW48/5mK4q7RpltksRHJvJ9/HYZ52+ jewg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=egSuQdef; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id y20-20020a056402271400b00427ba20da27si6444852edd.78.2022.05.03.08.44.53; Tue, 03 May 2022 08:44:53 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=egSuQdef; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3711168B2D4; Tue, 3 May 2022 18:44:44 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-io1-f49.google.com (mail-io1-f49.google.com [209.85.166.49]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A7B4B68B33F for ; Tue, 3 May 2022 18:44:37 +0300 (EEST) Received: by mail-io1-f49.google.com with SMTP id e15so19420905iob.3 for ; Tue, 03 May 2022 08:44:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cCKAHHjj2PJHARm3FDgpUVVgcHRiShLGy3H2EJ4Zbjo=; b=egSuQdefDx0g4N0/gNWxSv2KTaYITuVHBsHR3PEteDysFm7HpHp+RR8FG5/f3U1wWJ YQt3mC7Z4AtLbxU5qbUFd7O7i1Eg9/3ep7UweGBQV+rpHuOdDOcJBgF7CR0z9l+nSpKp QePPXyfkD6I5GHMSplJAPYhLZBAoSLndHf2j4AJAz63ikvIt52T71u4JMCByHNUnKpxN GoznN8mkHtUPHRdAAm+3re2KyuHB6043oUroW4TrFj8uRJyuIbBoFWV/PmMCmNadzUrA 9Co8YpFfGBUI06RNyv8jwH6WQa2tGSAe1MNRwjwsHPwYAPRxaGXurSoZddFVYKqZYWiM U8og== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cCKAHHjj2PJHARm3FDgpUVVgcHRiShLGy3H2EJ4Zbjo=; b=wk3yNvGI/szVma/puoc2odmpGGxKuvXwnlRV7S+1JzCoy6z8Eikn3sXKboWm6W3xrz P80fnF05oYiBJkY0+XU35hmdsqCy2UvqfU2ndyQYN/0P5wGK7QVONvIYx13RnhIkFcnI bkKgr16bg4ZXrfgMp91p1Q1BZeJw5uZMAtH0oYo34EPiZOLWvZCuLUUNsl8BPdonErb5 4hc2Wc8SP3KIPuFW/wcnmZa5YJmmjJH05aF+UZ6tylCCiCkMkMdAFm7WsPuhf5GQOoKR cNRIETcRwbboXopddMidwYKE5GdeCW0QpDbtScXxC75PItv8E1ZlYrymXw9xqOvS+YWd 0GvA== X-Gm-Message-State: AOAM530s+LKByllVjFyxc/HGQqiwoq2mXS4GcBB2YyyAciLwRLjfxrKi MeaNyCCc1DlHjSaMCNTJIvNigOwZTT8= X-Received: by 2002:a05:6638:3391:b0:32b:2407:8594 with SMTP id h17-20020a056638339100b0032b24078594mr6725869jav.283.1651592676428; Tue, 03 May 2022 08:44:36 -0700 (PDT) Received: from gauss.local (c-68-41-54-207.hsd1.mi.comcast.net. [68.41.54.207]) by smtp.gmail.com with ESMTPSA id l3-20020a026643000000b0032b3a7817e8sm3922228jaf.172.2022.05.03.08.44.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 03 May 2022 08:44:36 -0700 (PDT) From: Leo Izen To: ffmpeg-devel@ffmpeg.org Date: Tue, 3 May 2022 11:44:02 -0400 Message-Id: <20220503154403.132671-2-leo.izen@gmail.com> X-Mailer: git-send-email 2.36.0 In-Reply-To: <20220503154403.132671-1-leo.izen@gmail.com> References: <20220503154403.132671-1-leo.izen@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/2] avcodec/libjxldec: properly tag output colorspace X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Leo Izen Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: /E6ZD47qX+9Y Whether an ICC profile is present or not, the decoder should now properly tag the colorspace of pixel data received by the decoder. --- libavcodec/libjxl.h | 9 +++ libavcodec/libjxldec.c | 121 ++++++++++++++++++++++++++++++++++++++++- 2 files changed, 127 insertions(+), 3 deletions(-) diff --git a/libavcodec/libjxl.h b/libavcodec/libjxl.h index 5387c438fd..2b388dc45d 100644 --- a/libavcodec/libjxl.h +++ b/libavcodec/libjxl.h @@ -27,8 +27,17 @@ #ifndef AVCODEC_LIBJXL_H #define AVCODEC_LIBJXL_H +#include "libavutil/rational.h" + #include +#define FF_JXL_WP_D65_X av_make_q(3127, 10000) +#define FF_JXL_WP_D65_Y av_make_q(3290, 10000) +#define FF_JXL_WP_E_X av_make_q(1, 3) +#define FF_JXL_WP_E_Y av_make_q(1, 3) +#define FF_JXL_WP_DCI_X av_make_q(314, 1000) +#define FF_JXL_WP_DCI_Y av_make_q(351, 1000) + /** * Transform threadcount in ffmpeg to one used by libjxl. * diff --git a/libavcodec/libjxldec.c b/libavcodec/libjxldec.c index cd4bca3343..bd308590ab 100644 --- a/libavcodec/libjxldec.c +++ b/libavcodec/libjxldec.c @@ -28,9 +28,11 @@ #include "libavutil/buffer.h" #include "libavutil/common.h" #include "libavutil/error.h" +#include "libavutil/mastering_display_metadata.h" #include "libavutil/mem.h" #include "libavutil/pixdesc.h" #include "libavutil/pixfmt.h" +#include "libavutil/rational.h" #include "libavutil/frame.h" #include "avcodec.h" @@ -189,16 +191,129 @@ static int libjxl_decode_frame(AVCodecContext *avctx, AVFrame *frame, int *got_f continue; case JXL_DEC_COLOR_ENCODING: av_log(avctx, AV_LOG_DEBUG, "COLOR_ENCODING event emitted\n"); - jret = JxlDecoderGetICCProfileSize(ctx->decoder, &ctx->jxl_pixfmt, JXL_COLOR_PROFILE_TARGET_ORIGINAL, &iccp_len); + jret = JxlDecoderGetICCProfileSize(ctx->decoder, &ctx->jxl_pixfmt, JXL_COLOR_PROFILE_TARGET_DATA, &iccp_len); if (jret == JXL_DEC_SUCCESS && iccp_len > 0) { av_buffer_unref(&ctx->iccp); ctx->iccp = av_buffer_alloc(iccp_len); if (!ctx->iccp) return AVERROR(ENOMEM); - jret = JxlDecoderGetColorAsICCProfile(ctx->decoder, &ctx->jxl_pixfmt, JXL_COLOR_PROFILE_TARGET_ORIGINAL, ctx->iccp->data, iccp_len); - if (jret != JXL_DEC_SUCCESS) + jret = JxlDecoderGetColorAsICCProfile(ctx->decoder, &ctx->jxl_pixfmt, JXL_COLOR_PROFILE_TARGET_DATA, ctx->iccp->data, iccp_len); + if (jret != JXL_DEC_SUCCESS) { + av_log(avctx, AV_LOG_WARNING, "Unable to obtain ICCP from header\n"); av_buffer_unref(&ctx->iccp); + } } + avctx->color_range = frame->color_range = AVCOL_RANGE_JPEG; + if (ctx->iccp) { + /* if the ICCP is present, libjxl outputs sRGB */ + if (ctx->jxl_pixfmt.num_channels >= 3) + avctx->color_primaries = AVCOL_PRI_BT709; + /* linear sRGB if float values, standard sRGB if int values */ + avctx->color_trc = ctx->jxl_pixfmt.data_type == JXL_TYPE_FLOAT ? AVCOL_TRC_LINEAR : AVCOL_TRC_IEC61966_2_1; + } else { + JxlColorEncoding jxl_encoding; + jret = JxlDecoderGetColorAsEncodedProfile(ctx->decoder, &ctx->jxl_pixfmt, JXL_COLOR_PROFILE_TARGET_DATA, &jxl_encoding); + if (jret != JXL_DEC_SUCCESS) { + av_log(avctx, AV_LOG_WARNING, "Unable to obtain color encoding from header\n"); + continue; + } + + if (ctx->jxl_pixfmt.num_channels >= 3) { + AVMasteringDisplayMetadata *mdm = NULL; + + avctx->colorspace = frame->colorspace = AVCOL_SPC_RGB; + + /* dont set MDM if D65 white point is used */ + /* unless the custom primaries are set */ + if (jxl_encoding.primaries == JXL_PRIMARIES_CUSTOM || + jxl_encoding.white_point != JXL_WHITE_POINT_D65) { + mdm = av_mastering_display_metadata_create_side_data(frame); + if (!mdm) + return AVERROR(ENOMEM); + } + + switch (jxl_encoding.primaries) { + case JXL_PRIMARIES_SRGB: + avctx->color_primaries = AVCOL_PRI_BT709; + break; + case JXL_PRIMARIES_2100: + avctx->color_primaries = AVCOL_PRI_BT2020; + break; + case JXL_PRIMARIES_P3: + avctx->color_primaries = AVCOL_PRI_SMPTE431; + break; + case JXL_PRIMARIES_CUSTOM: + mdm->display_primaries[0][0] = av_d2q(jxl_encoding.primaries_red_xy[0], 100000); + mdm->display_primaries[0][1] = av_d2q(jxl_encoding.primaries_red_xy[1], 100000); + mdm->display_primaries[1][0] = av_d2q(jxl_encoding.primaries_green_xy[0], 100000); + mdm->display_primaries[1][1] = av_d2q(jxl_encoding.primaries_green_xy[1], 100000); + mdm->display_primaries[2][0] = av_d2q(jxl_encoding.primaries_blue_xy[0], 100000); + mdm->display_primaries[2][1] = av_d2q(jxl_encoding.primaries_blue_xy[1], 100000); + mdm->has_primaries = 1; + break; + default: + av_log(avctx, AV_LOG_WARNING, "Unknown JXL color primaries: %d\n", jxl_encoding.primaries); + avctx->color_primaries = AVCOL_PRI_UNSPECIFIED; + } + + switch (jxl_encoding.white_point) { + case JXL_WHITE_POINT_D65: + if (mdm) { + mdm->white_point[0] = FF_JXL_WP_D65_X; + mdm->white_point[1] = FF_JXL_WP_D65_Y; + } + break; + case JXL_WHITE_POINT_E: + mdm->white_point[0] = FF_JXL_WP_E_X; + mdm->white_point[1] = FF_JXL_WP_E_Y; + break; + case JXL_WHITE_POINT_DCI: + mdm->white_point[0] = FF_JXL_WP_DCI_X; + mdm->white_point[1] = FF_JXL_WP_DCI_Y; + break; + case JXL_WHITE_POINT_CUSTOM: + mdm->white_point[0] = av_d2q(jxl_encoding.white_point_xy[0], 100000); + mdm->white_point[1] = av_d2q(jxl_encoding.white_point_xy[1], 100000); + break; + default: + av_log(avctx, AV_LOG_WARNING, "Unknown JXL white point: %d\n", jxl_encoding.white_point); + } + } + + switch (jxl_encoding.transfer_function) { + case JXL_TRANSFER_FUNCTION_709: + avctx->color_trc = AVCOL_TRC_BT709; + break; + case JXL_TRANSFER_FUNCTION_LINEAR: + avctx->color_trc = AVCOL_TRC_LINEAR; + break; + case JXL_TRANSFER_FUNCTION_SRGB: + avctx->color_trc = AVCOL_TRC_IEC61966_2_1; + break; + case JXL_TRANSFER_FUNCTION_PQ: + case JXL_TRANSFER_FUNCTION_DCI: + avctx->color_trc = AVCOL_TRC_SMPTE428; + break; + case JXL_TRANSFER_FUNCTION_HLG: + avctx->color_trc = AVCOL_TRC_ARIB_STD_B67; + break; + case JXL_TRANSFER_FUNCTION_GAMMA: + if (jxl_encoding.gamma == 2.2) { + avctx->color_trc = AVCOL_TRC_GAMMA22; + } else if (jxl_encoding.gamma == 2.8) { + avctx->color_trc = AVCOL_TRC_GAMMA28; + } else { + av_log(avctx, AV_LOG_WARNING, "Unsupported gamma transfer: %f\n", jxl_encoding.gamma); + avctx->color_trc = AVCOL_TRC_UNSPECIFIED; + } + break; + default: + av_log(avctx, AV_LOG_WARNING, "Unknown transfer function: %d\n", jxl_encoding.transfer_function); + avctx->color_trc = AVCOL_TRC_UNSPECIFIED; + } + } + frame->color_trc = avctx->color_trc; + frame->color_primaries = avctx->color_primaries; continue; case JXL_DEC_NEED_IMAGE_OUT_BUFFER: av_log(avctx, AV_LOG_DEBUG, "NEED_IMAGE_OUT_BUFFER event emitted\n"); From patchwork Tue May 3 15:44:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leo Izen X-Patchwork-Id: 35564 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:a885:b0:7f:4be2:bd17 with SMTP id ca5csp481172pzb; Tue, 3 May 2022 08:45:04 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwL436FA28AWbWuOhSNSgKDlFqZ3QsoPxgKS7TeR14AQpJ0nRYXV8ZlfYCL56gCDDX5rYSW X-Received: by 2002:a17:907:7291:b0:6f4:28a8:9456 with SMTP id dt17-20020a170907729100b006f428a89456mr13662217ejc.688.1651592704234; Tue, 03 May 2022 08:45:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1651592704; cv=none; d=google.com; s=arc-20160816; b=Khv4dq73MgX9N1m+CJpj4NKZLLhog+qYrwmGehgyePhxEv7veYmkTqW7F0VXylcciz PsALIsL/PRuXWxJt3TL+U9NS02rvT5pSQzTT+hIjaLupgEWxNZTN6sG4P4+ZImLw7ejC D2mh8P4UkGtVa7tXg15U54JfwqraMhYgD9KMwlllWuc//MU2aGlwS/KyNVF+6N5JPGIS HpUS6vVzQu57iCUaQEeNYvxfZ+8ui2edVLQTgEVbUBZeaV91Qh9DOTtrVQi11W3wKLZt Z5J1qtCL3VL3W8eqLjiBHEoJVdMyHXcUDbBsD1JyghzjhHVCk19gSrMy25VPjLSBzYMN 9jkg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=UU+pwsohNZf/EYjxMxWwUTzLrJAKyXd5WIZm9ZC5aZ8=; b=xkBkgOCFSaxJrXyF2u1m1MRBLlA5eyAZ4tR5muuW3hCTqshKCBVNzPTabMVKVAYvfy 9Ak718WcBp/98XDCxiWTZorRIjpdaCjIBhcrf50nx3dYq6ycJNFCCPojTXYfPk+NbsTO w5RheC+NaGnY31AHF0buKzoHrpz7v/b0Sf2zeLtFMpwGYCngxykM2OtQNMb1WE5bhwOd MyqAf0SO0B8sCH+MSTCMMqiVZwG+88l6hTicm2s1auEVPlPuO1dTrc8Xz+edCi+CuRw+ Igch0B2PbDQBeOLa/7cW7k+UXgpPtZIRwBXGCMkk/XNS0povyrVht2X12TWPszOVnwT1 TZOQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=IA1+IDco; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id vw6-20020a170907058600b006f393afb98esi13908948ejb.509.2022.05.03.08.45.03; Tue, 03 May 2022 08:45:04 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=IA1+IDco; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 215FD68B374; Tue, 3 May 2022 18:44:45 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-io1-f53.google.com (mail-io1-f53.google.com [209.85.166.53]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B2AA068B2F2 for ; Tue, 3 May 2022 18:44:38 +0300 (EEST) Received: by mail-io1-f53.google.com with SMTP id m6so15212926iob.4 for ; Tue, 03 May 2022 08:44:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0v3b4FjEc8CcZlHY1uXrJBRnQbJZnwfu4mcEGicCX9A=; b=IA1+IDcofsV3SWACe4d7akPtsG4KZqp/mc3L0cJfCq/Zquode0OxHPU2xhPFvyXYEu UKvkx6ubJQYJWkSLbpshd0TDOWue67mxDMsfiRh3Sk2FMRq8CZ0i34cwn/hjUYCkxwIL FNDr6W/d5yNYdGGY5Q7GDJczwNanUvW2mTeT9QtXfd0GVJuOHFJibj9j+OTDwgEB5bBU NWFejK+iEIU3CIag17BmK4BQfdiHuMWmELLLvez0bh2aaWjxjyDINdKjLAca1QDOSkjb deIZhqOEhgqPGqUInIDzBX0daZlTIiUnkQhnpSLY2PIwzPe1mC2xjnN8BDuiYnfpQ22H s6OA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0v3b4FjEc8CcZlHY1uXrJBRnQbJZnwfu4mcEGicCX9A=; b=SOkuOvAMQoBLJ1GE08rqDNtg8G1daZ493nDjU4TtOPcNmdidehUnCDPOvZUYQuS5i5 IDcDVevTeskD9KveZ5hkh7Ow1tB7Aa1+UW2hF06zOkpE8Zn1+5qm/7n6MKS4rHu42lgs s8NwMBp/1RXyzwnJ09RGJb0SV+NAHYrsXBgPXK0hgTCq0yMcS0QIxrl8qUyK+PLgdSZc Tm2ouy60NaWptCG/qvq9VOEUAxIdi4hul7l2NPXJfrpgke+0mZ6lpwW1Jpz5b8Qrcx2f xybN/3o1R60FpHhBLyK4WiDzioxoIGodPXAvVktpFFXddFdGrQyoYXbrmAsj+TucDsD9 sYsg== X-Gm-Message-State: AOAM533CuKbFhp0ehABwk/dtym0nJcOoqlEmJS23MqBtc412xThyTJaW TirVWu8T3sTJUv0B+pZSOsRCurdAfX0= X-Received: by 2002:a6b:d003:0:b0:646:4652:bd57 with SMTP id x3-20020a6bd003000000b006464652bd57mr6323358ioa.51.1651592677088; Tue, 03 May 2022 08:44:37 -0700 (PDT) Received: from gauss.local (c-68-41-54-207.hsd1.mi.comcast.net. [68.41.54.207]) by smtp.gmail.com with ESMTPSA id l3-20020a026643000000b0032b3a7817e8sm3922228jaf.172.2022.05.03.08.44.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 03 May 2022 08:44:36 -0700 (PDT) From: Leo Izen To: ffmpeg-devel@ffmpeg.org Date: Tue, 3 May 2022 11:44:03 -0400 Message-Id: <20220503154403.132671-3-leo.izen@gmail.com> X-Mailer: git-send-email 2.36.0 In-Reply-To: <20220503154403.132671-1-leo.izen@gmail.com> References: <20220503154403.132671-1-leo.izen@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/2] avcodec/libjxlenc: properly read input colorspace X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Leo Izen Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: dA78f0kofbrx Whether an ICC profile is present or not, the libjxl encoder wrapper should now properly read colorspace tags and forward them to libjxl appropriately, rather than just assume sRGB as before. It will also print warnings when colorimetric assumptions are made about the input data. --- libavcodec/libjxlenc.c | 152 +++++++++++++++++++++++++++++++++-------- 1 file changed, 124 insertions(+), 28 deletions(-) diff --git a/libavcodec/libjxlenc.c b/libavcodec/libjxlenc.c index 8bebec6aeb..02ecdd7dc6 100644 --- a/libavcodec/libjxlenc.c +++ b/libavcodec/libjxlenc.c @@ -30,9 +30,11 @@ #include "libavutil/error.h" #include "libavutil/frame.h" #include "libavutil/libm.h" +#include "libavutil/mastering_display_metadata.h" #include "libavutil/opt.h" #include "libavutil/pixdesc.h" #include "libavutil/pixfmt.h" +#include "libavutil/rational.h" #include "libavutil/version.h" #include "avcodec.h" @@ -117,7 +119,7 @@ static int libjxl_init_jxl_encoder(AVCodecContext *avctx) return AVERROR_EXTERNAL; } - /* check for negative zero, our default */ + /* check for negative, our default */ if (ctx->distance < 0.0) { /* use ffmpeg.c -q option if passed */ if (avctx->flags & AV_CODEC_FLAG_QSCALE) @@ -133,7 +135,8 @@ static int libjxl_init_jxl_encoder(AVCodecContext *avctx) */ if (ctx->distance > 0.0 && ctx->distance < 0.01) ctx->distance = 0.01; - if (JxlEncoderOptionsSetDistance(ctx->options, ctx->distance) != JXL_ENC_SUCCESS) { + + if (JxlEncoderSetFrameDistance(ctx->options, ctx->distance) != JXL_ENC_SUCCESS) { av_log(avctx, AV_LOG_ERROR, "Failed to set distance: %f\n", ctx->distance); return AVERROR_EXTERNAL; } @@ -219,57 +222,150 @@ static int libjxl_encode_frame(AVCodecContext *avctx, AVPacket *pkt, const AVFra info.num_color_channels = jxl_fmt.num_channels - info.num_extra_channels; info.bits_per_sample = av_get_bits_per_pixel(pix_desc) / jxl_fmt.num_channels; info.alpha_bits = (info.num_extra_channels > 0) * info.bits_per_sample; + if (pix_desc->flags & AV_PIX_FMT_FLAG_FLOAT) { info.exponent_bits_per_sample = info.bits_per_sample > 16 ? 8 : 5; info.alpha_exponent_bits = info.alpha_bits ? info.exponent_bits_per_sample : 0; jxl_fmt.data_type = info.bits_per_sample > 16 ? JXL_TYPE_FLOAT : JXL_TYPE_FLOAT16; - JxlColorEncodingSetToLinearSRGB(&jxl_color, info.num_color_channels == 1); } else { info.exponent_bits_per_sample = 0; info.alpha_exponent_bits = 0; jxl_fmt.data_type = info.bits_per_sample <= 8 ? JXL_TYPE_UINT8 : JXL_TYPE_UINT16; - JxlColorEncodingSetToSRGB(&jxl_color, info.num_color_channels == 1); } - if (info.bits_per_sample > 16 - || info.xsize > (1 << 18) || info.ysize > (1 << 18) - || (info.xsize << 4) * (info.ysize << 4) > (1 << 20)) { - /* - * must upgrade codestream to level 10, from level 5 - * the encoder will not do this automatically - */ - if (JxlEncoderSetCodestreamLevel(ctx->encoder, 10) != JXL_ENC_SUCCESS) { - av_log(avctx, AV_LOG_ERROR, "Could not upgrade JXL Codestream level.\n"); - return AVERROR_EXTERNAL; + /* JPEG XL format itself does not support partial range */ + if (avctx->color_range == AVCOL_RANGE_MPEG) { + av_log(avctx, AV_LOG_ERROR, "This encoder does not support partial(tv) range, colors will be wrong!\n"); + } else if (avctx->color_range != AVCOL_RANGE_JPEG) { + av_log(avctx, AV_LOG_WARNING, "Unknown color range, assuming full\n"); + } + + /* rendering intent doesn't matter here + * but libjxl will whine if we don't set it */ + jxl_color.rendering_intent = JXL_RENDERING_INTENT_RELATIVE; + + switch (avctx->color_trc) { + case AVCOL_TRC_BT709: + jxl_color.transfer_function = JXL_TRANSFER_FUNCTION_709; + break; + case AVCOL_TRC_LINEAR: + jxl_color.transfer_function = JXL_TRANSFER_FUNCTION_LINEAR; + break; + case AVCOL_TRC_IEC61966_2_1: + jxl_color.transfer_function = JXL_TRANSFER_FUNCTION_SRGB; + break; + case AVCOL_TRC_SMPTE428: + jxl_color.transfer_function = JXL_TRANSFER_FUNCTION_DCI; + break; + case AVCOL_TRC_ARIB_STD_B67: + jxl_color.transfer_function = JXL_TRANSFER_FUNCTION_HLG; + break; + case AVCOL_TRC_GAMMA22: + jxl_color.transfer_function = JXL_TRANSFER_FUNCTION_GAMMA; + jxl_color.gamma = 2.2; + break; + case AVCOL_TRC_GAMMA28: + jxl_color.transfer_function = JXL_TRANSFER_FUNCTION_GAMMA; + jxl_color.gamma = 2.8; + break; + default: + if (pix_desc->flags & AV_PIX_FMT_FLAG_FLOAT) { + av_log(avctx, AV_LOG_WARNING, "Unknown transfer function, assuming Linear Light\n"); + jxl_color.transfer_function = JXL_TRANSFER_FUNCTION_LINEAR; + } else { + av_log(avctx, AV_LOG_WARNING, "Unknown transfer function, assuming IEC61966-2-1/sRGB\n"); + jxl_color.transfer_function = JXL_TRANSFER_FUNCTION_SRGB; } } - /* bitexact lossless requires there to be no XYB transform */ - info.uses_original_profile = ctx->distance == 0.0; + sd = av_frame_get_side_data(frame, AV_FRAME_DATA_MASTERING_DISPLAY_METADATA); + if (sd) { + AVMasteringDisplayMetadata *mdm = (AVMasteringDisplayMetadata *) sd; + if (!av_cmp_q(mdm->white_point[0], FF_JXL_WP_D65_X) && + !av_cmp_q(mdm->white_point[1], FF_JXL_WP_D65_Y)) { + jxl_color.white_point = JXL_WHITE_POINT_D65; + } else if (!av_cmp_q(mdm->white_point[0], FF_JXL_WP_E_X) && + !av_cmp_q(mdm->white_point[1], FF_JXL_WP_E_Y)) { + jxl_color.white_point = JXL_WHITE_POINT_E; + } else if (!av_cmp_q(mdm->white_point[0], FF_JXL_WP_DCI_X) && + !av_cmp_q(mdm->white_point[1], FF_JXL_WP_DCI_Y)) { + jxl_color.white_point = JXL_WHITE_POINT_DCI; + } else { + jxl_color.white_point = JXL_WHITE_POINT_CUSTOM; + jxl_color.white_point_xy[0] = av_q2d(mdm->white_point[0]); + jxl_color.white_point_xy[1] = av_q2d(mdm->white_point[1]); + } + } else { + /* assume D65 if otherwise unset, including for grayscale */ + jxl_color.white_point = JXL_WHITE_POINT_D65; + } - sd = av_frame_get_side_data(frame, AV_FRAME_DATA_ICC_PROFILE); - if (sd && sd->size && JxlEncoderSetICCProfile(ctx->encoder, sd->data, sd->size) != JXL_ENC_SUCCESS) { - av_log(avctx, AV_LOG_WARNING, "Could not set ICC Profile\n"); - } else if (info.uses_original_profile) { - /* - * the color encoding is not used if uses_original_profile is false - * this just works around a bug in libjxl 0.7.0 and lower - */ - if (JxlEncoderSetColorEncoding(ctx->encoder, &jxl_color) != JXL_ENC_SUCCESS) { - av_log(avctx, AV_LOG_ERROR, "Failed to set JxlColorEncoding\n"); - return AVERROR_EXTERNAL; + if (info.num_color_channels == 1) { + /* This should be implied to be honest + * but a libjxl but makes it fail otherwise */ + jxl_color.color_space = JXL_COLOR_SPACE_GRAY; + } else { + AVMasteringDisplayMetadata *mdm = NULL; + if (sd) + mdm = (AVMasteringDisplayMetadata *) sd; + + /* even if it internally uses XYB we set RGB here + * since that's the color space of the pixel data + * that we are feeding to libjxl */ + jxl_color.color_space = JXL_COLOR_SPACE_RGB; + + if (mdm && mdm->has_primaries) { + jxl_color.primaries = JXL_PRIMARIES_CUSTOM; + jxl_color.primaries_red_xy[0] = av_q2d(mdm->display_primaries[0][0]); + jxl_color.primaries_red_xy[1] = av_q2d(mdm->display_primaries[0][1]); + jxl_color.primaries_green_xy[0] = av_q2d(mdm->display_primaries[1][0]); + jxl_color.primaries_green_xy[1] = av_q2d(mdm->display_primaries[1][1]); + jxl_color.primaries_blue_xy[0] = av_q2d(mdm->display_primaries[2][0]); + jxl_color.primaries_blue_xy[1] = av_q2d(mdm->display_primaries[2][1]); + } else { + switch (avctx->color_primaries) { + case AVCOL_PRI_BT709: + jxl_color.primaries = JXL_PRIMARIES_SRGB; + break; + case AVCOL_PRI_BT2020: + jxl_color.primaries = JXL_PRIMARIES_2100; + break; + case AVCOL_PRI_SMPTE431: + jxl_color.primaries = JXL_PRIMARIES_P3; + break; + default: + av_log(avctx, AV_LOG_WARNING, "Unknown color primaries, assuming BT.709/sRGB\n"); + jxl_color.primaries = JXL_PRIMARIES_SRGB; + } } } + /* bitexact lossless requires there to be no XYB transform */ + info.uses_original_profile = ctx->distance == 0.0; + if (JxlEncoderSetBasicInfo(ctx->encoder, &info) != JXL_ENC_SUCCESS) { av_log(avctx, AV_LOG_ERROR, "Failed to set JxlBasicInfo\n"); return AVERROR_EXTERNAL; } + sd = av_frame_get_side_data(frame, AV_FRAME_DATA_ICC_PROFILE); + if (sd && sd->size && JxlEncoderSetICCProfile(ctx->encoder, sd->data, sd->size) != JXL_ENC_SUCCESS) + av_log(avctx, AV_LOG_WARNING, "Could not set ICC Profile\n"); + else if (JxlEncoderSetColorEncoding(ctx->encoder, &jxl_color) != JXL_ENC_SUCCESS) + av_log(avctx, AV_LOG_WARNING, "Failed to set JxlColorEncoding\n"); + + /* depending on basic info, level 10 might + * be required instead of level 5 */ + if (JxlEncoderGetRequiredCodestreamLevel(ctx->encoder) > 5) { + if (JxlEncoderSetCodestreamLevel(ctx->encoder, 10) != JXL_ENC_SUCCESS) + av_log(avctx, AV_LOG_WARNING, "Could not increase codestream level\n"); + } + jxl_fmt.endianness = JXL_NATIVE_ENDIAN; jxl_fmt.align = frame->linesize[0]; - if (JxlEncoderAddImageFrame(ctx->options, &jxl_fmt, frame->data[0], jxl_fmt.align * info.ysize) != JXL_ENC_SUCCESS) { + if (JxlEncoderAddImageFrame(ctx->options, &jxl_fmt, frame->data[0], + jxl_fmt.align * info.ysize) != JXL_ENC_SUCCESS) { av_log(avctx, AV_LOG_ERROR, "Failed to add Image Frame\n"); return AVERROR_EXTERNAL; }