From patchwork Fri Jun 24 16:50:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leo Izen X-Patchwork-Id: 36414 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:8b27:b0:88:1bbf:7fd2 with SMTP id l39csp421515pzh; Fri, 24 Jun 2022 09:51:21 -0700 (PDT) X-Google-Smtp-Source: AGRyM1tj3nIt5RhMgzE2+vCLM2qsO/kHgrnTEAeDY7aEvzCj9EO+ldtHVhkGR26ilxE4ttA6e3uL X-Received: by 2002:a05:6402:5255:b0:435:bec2:9e33 with SMTP id t21-20020a056402525500b00435bec29e33mr14948236edd.398.1656089481770; Fri, 24 Jun 2022 09:51:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1656089481; cv=none; d=google.com; s=arc-20160816; b=Q/LWJhNlJYuvUHf01n0Cw/OHOnj2AaQG6ervL40BL8s4l7mvf2HV6HbViGzvkSEM38 h5XNKVFfParTvJXPzMWb+bl6pvp5TLcP2PT58d3Euq7FoKitfzsCrWRwfKx6wlwVbUuj 5WSsDx/hSn6HyXJuZsNCy+AsHH4IB3qbjT3Ud4l7HjgUnl6eil+P7lIsvR7IiMSPNR/a iay/P7NvmQfCV7tgwYBp1D9DQ9fAnCJyqqE4Li43cRLH96kOzh1Vh7DtggAYhFXM5nBX UedO7KARf2bCm/A3BQFjE/gv8Ru8GHArkHgzGxuI8hrVioUyQJNVbKCuv5XZtFFo+Gwe yVxw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=95kf8H29+L3ixACInssqmWKfYCjNPtjbiBLVAbD2S8M=; b=tUrb7Ancq3OLjAimF/ROX/jZtQgCj0mjj1YSirKTHw6Gzmt3kGos/Iz0cNOXBYfN8M Hpl8rHBuLgwJd91qlXbN0T2B21RDPiOLNGSEcORQc6nHsOyuzU51xrKUlhNwLlFUIUZO cIrt+P3OxNEEoozwA4/ZMgjNyebTYyD1PIEtydWzc8ZdnnW19ZKyzfpJjjzjGN4wNkeE R+RiM8Xvz+4PdEVw9R4lgLnf9HZ4I9YEGK8H4I6sv1kXBdpE5mtzBUrKd7iHjARRkhnQ 1ozNnoPa3jpBTY6h05h5h+mSUB+TIUYgwbVNdctYPYlrOUI2asaScwBPIFKtqSD4/ZDm Qdrg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=GxM2X6IR; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id jg37-20020a170907972500b00711d8503c7csi3412361ejc.688.2022.06.24.09.51.21; Fri, 24 Jun 2022 09:51:21 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=GxM2X6IR; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9B0BF68B710; Fri, 24 Jun 2022 19:51:12 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qv1-f41.google.com (mail-qv1-f41.google.com [209.85.219.41]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6D25468B6E9 for ; Fri, 24 Jun 2022 19:51:05 +0300 (EEST) Received: by mail-qv1-f41.google.com with SMTP id 88so5364540qva.9 for ; Fri, 24 Jun 2022 09:51:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3cePHIAqUpr3Y5qE/30dlv6Vmz0yAaxZx3xC6mZPa5g=; b=GxM2X6IRLF4/qUL3hgHXDonApcySgoumAVIxsStMjHUl/P6KyD0ViDgR0AuhU4I/Jo 0GOrhgsT/C7zyd8JZVmkVIdbH/PcaBifIFTVEr9CAow03zdyS0YMX1YxktpVQ4LwWyco tScMXB9lbgNNjDL3NCwk+hWW0e4ID21U+MLRs7ESBgqEDp7R3V3VdJ2UaH/fuTNB6ONV DOfgEk+Wq9i0NSLE1LSObDfZXVMfcwlUqmkMrSyE6HDnchQtRLDvL6V3lShC+XXoQ7rv G5yiRcpp3H0j76LL1yE77g29YwHlKhlLTDd1q75FoTx9++zRKeeweh+nYQwQ1Qe+xepk aDfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3cePHIAqUpr3Y5qE/30dlv6Vmz0yAaxZx3xC6mZPa5g=; b=p/Rom2EAwy8QGQSgBGOLgVaU90SVMkFc2JwKBp/4LMCXF1dyueGuArlY2KKuZXUPqR J4uhhWNuMm1pUrkbNPUcHEQWUq5hqBGl6Vh8+IY2FYGPbjd7dO7CAf6U4c+Oqw6KxjLx QcdFnxcYE15YMJ6oX2rLyiRTapz8HlWYBisVyQ8z+AMHjd5iRrFJQ7KsyVRkUPjjPiEE qHc/IQRCIm7vrG2/Lui+HbYNeyIM9D00MRJx2hJRWVZobzTFFFi/px49k7kVeeI2PiDs +6TE+XGr0BTfHdT+OVyuQoldAXy9bSaGn6mJ1702AVHvSR211HAcsuCd0CHfHuZkayIb IvGA== X-Gm-Message-State: AJIora+CPsgamHOiWaIXFmxbxFYao9aTtl0g2o5DBwNeHXJSKcL2Jxod 8/IkBBmaON1iOE0uXrGO8TmN4fIi7Hg= X-Received: by 2002:ad4:458c:0:b0:46e:8282:3632 with SMTP id x12-20020ad4458c000000b0046e82823632mr33562874qvu.31.1656089463571; Fri, 24 Jun 2022 09:51:03 -0700 (PDT) Received: from gauss.local (c-68-41-54-207.hsd1.mi.comcast.net. [68.41.54.207]) by smtp.gmail.com with ESMTPSA id q15-20020ac8450f000000b003177f0fb61esm1588106qtn.75.2022.06.24.09.51.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 24 Jun 2022 09:51:02 -0700 (PDT) From: Leo Izen To: ffmpeg-devel@ffmpeg.org Date: Fri, 24 Jun 2022 12:50:55 -0400 Message-Id: <20220624165056.51700-2-leo.izen@gmail.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220624165056.51700-1-leo.izen@gmail.com> References: <20220624165056.51700-1-leo.izen@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v9 1/2] avcodec/libjxldec: properly tag output colorspace X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Leo Izen Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 5iI9+mZ0Uven Whether an ICC profile is present or not, the decoder should now properly tag the colorspace of pixel data received by the decoder. --- libavcodec/libjxldec.c | 214 +++++++++++++++++++++++++++++++++++++---- 1 file changed, 197 insertions(+), 17 deletions(-) diff --git a/libavcodec/libjxldec.c b/libavcodec/libjxldec.c index cd4bca3343..d516d3b0ac 100644 --- a/libavcodec/libjxldec.c +++ b/libavcodec/libjxldec.c @@ -27,6 +27,7 @@ #include "libavutil/avassert.h" #include "libavutil/buffer.h" #include "libavutil/common.h" +#include "libavutil/csp.h" #include "libavutil/error.h" #include "libavutil/mem.h" #include "libavutil/pixdesc.h" @@ -92,7 +93,7 @@ static av_cold int libjxl_decode_init(AVCodecContext *avctx) return libjxl_init_jxl_decoder(avctx); } -static enum AVPixelFormat libjxl_get_pix_fmt(AVCodecContext *avctx, JxlBasicInfo *basic_info, JxlPixelFormat *format) +static enum AVPixelFormat libjxl_get_pix_fmt(void *avctx, const JxlBasicInfo *basic_info, JxlPixelFormat *format) { format->endianness = JXL_NATIVE_ENDIAN; format->num_channels = basic_info->num_color_channels + (basic_info->alpha_bits > 0); @@ -129,11 +130,199 @@ static enum AVPixelFormat libjxl_get_pix_fmt(AVCodecContext *avctx, JxlBasicInfo return AV_PIX_FMT_NONE; } +static enum AVColorPrimaries libjxl_get_primaries(void *avctx, const JxlColorEncoding *jxl_color) +{ + AVColorPrimariesDesc desc; + enum AVColorPrimaries prim; + + /* libjxl populates these double values even if it uses an enum space */ + desc.prim.r.x = av_d2q(jxl_color->primaries_red_xy[0], 300000); + desc.prim.r.y = av_d2q(jxl_color->primaries_red_xy[1], 300000); + desc.prim.g.x = av_d2q(jxl_color->primaries_green_xy[0], 300000); + desc.prim.g.y = av_d2q(jxl_color->primaries_green_xy[1], 300000); + desc.prim.b.x = av_d2q(jxl_color->primaries_blue_xy[0], 300000); + desc.prim.b.y = av_d2q(jxl_color->primaries_blue_xy[1], 300000); + desc.wp.x = av_d2q(jxl_color->white_point_xy[0], 300000); + desc.wp.y = av_d2q(jxl_color->white_point_xy[1], 300000); + + prim = av_csp_primaries_id_from_desc(&desc); + if (prim == AVCOL_PRI_UNSPECIFIED) { + /* try D65 with the same primaries */ + /* BT.709 uses D65 white point */ + desc.wp = av_csp_primaries_desc_from_id(AVCOL_PRI_BT709)->wp; + av_log(avctx, AV_LOG_WARNING, "Changing unknown white point to D65\n"); + prim = av_csp_primaries_id_from_desc(&desc); + } + + return prim; +} + +static enum AVColorTransferCharacteristic libjxl_get_trc(void *avctx, const JxlColorEncoding *jxl_color) +{ + switch (jxl_color->transfer_function) { + case JXL_TRANSFER_FUNCTION_709: return AVCOL_TRC_BT709; + case JXL_TRANSFER_FUNCTION_LINEAR: return AVCOL_TRC_LINEAR; + case JXL_TRANSFER_FUNCTION_SRGB: return AVCOL_TRC_IEC61966_2_1; + case JXL_TRANSFER_FUNCTION_PQ: return AVCOL_TRC_SMPTE2084; + case JXL_TRANSFER_FUNCTION_DCI: return AVCOL_TRC_SMPTE428; + case JXL_TRANSFER_FUNCTION_HLG: return AVCOL_TRC_ARIB_STD_B67; + case JXL_TRANSFER_FUNCTION_GAMMA: + if (jxl_color->gamma > 2.199 && jxl_color->gamma < 2.201) + return AVCOL_TRC_GAMMA22; + else if (jxl_color->gamma > 2.799 && jxl_color->gamma < 2.801) + return AVCOL_TRC_GAMMA28; + else + av_log(avctx, AV_LOG_WARNING, "Unsupported gamma transfer: %f\n", jxl_color->gamma); + break; + default: + av_log(avctx, AV_LOG_WARNING, "Unknown transfer function: %d\n", jxl_color->transfer_function); + } + + return AVCOL_TRC_UNSPECIFIED; +} + +static int libjxl_get_icc(AVCodecContext *avctx) +{ + LibJxlDecodeContext *ctx = avctx->priv_data; + size_t icc_len; + JxlDecoderStatus jret; + /* an ICC profile is present, and we can meaningfully get it, + * because the pixel data is not XYB-encoded */ + jret = JxlDecoderGetICCProfileSize(ctx->decoder, &ctx->jxl_pixfmt, JXL_COLOR_PROFILE_TARGET_DATA, &icc_len); + if (jret == JXL_DEC_SUCCESS && icc_len > 0) { + av_buffer_unref(&ctx->iccp); + ctx->iccp = av_buffer_alloc(icc_len); + if (!ctx->iccp) + return AVERROR(ENOMEM); + jret = JxlDecoderGetColorAsICCProfile(ctx->decoder, &ctx->jxl_pixfmt, JXL_COLOR_PROFILE_TARGET_DATA, + ctx->iccp->data, icc_len); + if (jret != JXL_DEC_SUCCESS) { + av_log(avctx, AV_LOG_WARNING, "Unable to obtain ICC Profile\n"); + av_buffer_unref(&ctx->iccp); + } + } + + return 0; +} + +/* + * There's generally four cases when it comes to decoding a libjxl image + * with regard to color encoding: + * (a) There is an embedded ICC profile in the image, and the image is XYB-encoded. + * (b) There is an embedded ICC profile in the image, and the image is not XYB-encoded. + * (c) There is no embedded ICC profile, and FFmpeg supports the tagged colorspace. + * (d) There is no embedded ICC profile, and FFmpeg does not support the tagged colorspace. + * + * In case (b), we forward the pixel data as is and forward the ICC Profile as-is. + * In case (c), we request the pixel data in the space it's tagged as, + * and tag the space accordingly. + * In case (a), libjxl does not support getting the pixel data in the space described by the ICC + * profile, so instead we request the pixel data in BT.2020/PQ as it is the widest + * space that FFmpeg supports. + * In case (d), we also request wide-gamut pixel data as a fallback since FFmpeg doesn't support + * the custom primaries tagged in the space. + */ +static int libjxl_color_encoding_event(AVCodecContext *avctx, AVFrame *frame) +{ + LibJxlDecodeContext *ctx = avctx->priv_data; + JxlDecoderStatus jret; + int ret; + JxlColorEncoding jxl_color; + /* set this flag if we need to fall back on wide gamut */ + int fallback = 0; + + jret = JxlDecoderGetColorAsEncodedProfile(ctx->decoder, NULL, JXL_COLOR_PROFILE_TARGET_ORIGINAL, &jxl_color); + if (jret == JXL_DEC_SUCCESS) { + /* enum values describe the colors of this image */ + jret = JxlDecoderSetPreferredColorProfile(ctx->decoder, &jxl_color); + if (jret == JXL_DEC_SUCCESS) + jret = JxlDecoderGetColorAsEncodedProfile(ctx->decoder, &ctx->jxl_pixfmt, JXL_COLOR_PROFILE_TARGET_DATA, &jxl_color); + /* if we couldn't successfully request the pixel data space, we fall back on wide gamut */ + /* this code path is very unlikely to happen in practice */ + if (jret != JXL_DEC_SUCCESS) + fallback = 1; + } else { + /* an ICC Profile is present in the stream */ + if (ctx->basic_info.uses_original_profile) { + /* uses_original_profile is the same as !xyb_encoded */ + av_log(avctx, AV_LOG_VERBOSE, "Using embedded ICC Profile\n"); + if ((ret = libjxl_get_icc(avctx)) < 0) + return ret; + } else { + /* + * an XYB-encoded image with an embedded ICC profile can't always have the + * pixel data requested in the original space, so libjxl has no feature + * to allow this to happen, so we fall back on wide gamut + */ + fallback = 1; + } + } + + avctx->color_range = frame->color_range = AVCOL_RANGE_JPEG; + if (ctx->jxl_pixfmt.num_channels >= 3) + avctx->colorspace = AVCOL_SPC_RGB; + avctx->color_primaries = AVCOL_PRI_UNSPECIFIED; + avctx->color_trc = AVCOL_TRC_UNSPECIFIED; + + if (!ctx->iccp) { + /* checking enum values */ + if (!fallback) { + if (avctx->colorspace == AVCOL_SPC_RGB) + avctx->color_primaries = libjxl_get_primaries(avctx, &jxl_color); + avctx->color_trc = libjxl_get_trc(avctx, &jxl_color); + } + /* fall back on wide gamut if enum values fail */ + if (avctx->color_primaries == AVCOL_PRI_UNSPECIFIED) { + if (avctx->colorspace == AVCOL_SPC_RGB) { + av_log(avctx, AV_LOG_WARNING, "Falling back on wide gamut output\n"); + jxl_color.primaries = JXL_PRIMARIES_2100; + avctx->color_primaries = AVCOL_PRI_BT2020; + } + /* libjxl requires this set even for grayscale */ + jxl_color.white_point = JXL_WHITE_POINT_D65; + } + if (avctx->color_trc == AVCOL_TRC_UNSPECIFIED) { + if (ctx->jxl_pixfmt.data_type == JXL_TYPE_FLOAT + || ctx->jxl_pixfmt.data_type == JXL_TYPE_FLOAT16) { + av_log(avctx, AV_LOG_WARNING, "Falling back on Linear Light transfer\n"); + jxl_color.transfer_function = JXL_TRANSFER_FUNCTION_LINEAR; + avctx->color_trc = AVCOL_TRC_LINEAR; + } else { + av_log(avctx, AV_LOG_WARNING, "Falling back on iec61966-2-1/sRGB transfer\n"); + jxl_color.transfer_function = JXL_TRANSFER_FUNCTION_SRGB; + avctx->color_trc = AVCOL_TRC_IEC61966_2_1; + } + } + /* all colors will be in-gamut so we want accurate colors */ + jxl_color.rendering_intent = JXL_RENDERING_INTENT_RELATIVE; + jxl_color.color_space = avctx->colorspace == AVCOL_SPC_RGB ? JXL_COLOR_SPACE_RGB : JXL_COLOR_SPACE_GRAY; + jret = JxlDecoderSetPreferredColorProfile(ctx->decoder, &jxl_color); + if (jret != JXL_DEC_SUCCESS) { + av_log(avctx, AV_LOG_WARNING, "Unable to set fallback color encoding\n"); + /* + * This should only happen if there's a non-XYB encoded image with custom primaries + * embedded as enums and no embedded ICC Profile. + * In this case, libjxl will synthesize an ICC Profile for us. + */ + avctx->color_trc = AVCOL_TRC_UNSPECIFIED; + avctx->color_primaries = AVCOL_PRI_UNSPECIFIED; + if ((ret = libjxl_get_icc(avctx)) < 0) + return ret; + } + } + + frame->color_trc = avctx->color_trc; + frame->color_primaries = avctx->color_primaries; + frame->colorspace = avctx->colorspace; + + return 0; +} + static int libjxl_decode_frame(AVCodecContext *avctx, AVFrame *frame, int *got_frame, AVPacket *avpkt) { LibJxlDecodeContext *ctx = avctx->priv_data; uint8_t *buf = avpkt->data; - size_t remaining = avpkt->size, iccp_len; + size_t remaining = avpkt->size; JxlDecoderStatus jret; int ret; *got_frame = 0; @@ -183,30 +372,21 @@ static int libjxl_decode_frame(AVCodecContext *avctx, AVFrame *frame, int *got_f av_log(avctx, AV_LOG_ERROR, "Bad libjxl pixel format\n"); return AVERROR_EXTERNAL; } - ret = ff_set_dimensions(avctx, ctx->basic_info.xsize, ctx->basic_info.ysize); - if (ret < 0) + if ((ret = ff_set_dimensions(avctx, ctx->basic_info.xsize, ctx->basic_info.ysize)) < 0) return ret; continue; case JXL_DEC_COLOR_ENCODING: av_log(avctx, AV_LOG_DEBUG, "COLOR_ENCODING event emitted\n"); - jret = JxlDecoderGetICCProfileSize(ctx->decoder, &ctx->jxl_pixfmt, JXL_COLOR_PROFILE_TARGET_ORIGINAL, &iccp_len); - if (jret == JXL_DEC_SUCCESS && iccp_len > 0) { - av_buffer_unref(&ctx->iccp); - ctx->iccp = av_buffer_alloc(iccp_len); - if (!ctx->iccp) - return AVERROR(ENOMEM); - jret = JxlDecoderGetColorAsICCProfile(ctx->decoder, &ctx->jxl_pixfmt, JXL_COLOR_PROFILE_TARGET_ORIGINAL, ctx->iccp->data, iccp_len); - if (jret != JXL_DEC_SUCCESS) - av_buffer_unref(&ctx->iccp); - } + if ((ret = libjxl_color_encoding_event(avctx, frame)) < 0) + return ret; continue; case JXL_DEC_NEED_IMAGE_OUT_BUFFER: av_log(avctx, AV_LOG_DEBUG, "NEED_IMAGE_OUT_BUFFER event emitted\n"); - ret = ff_get_buffer(avctx, frame, 0); - if (ret < 0) + if ((ret = ff_get_buffer(avctx, frame, 0)) < 0) return ret; ctx->jxl_pixfmt.align = frame->linesize[0]; - if (JxlDecoderSetImageOutBuffer(ctx->decoder, &ctx->jxl_pixfmt, frame->data[0], frame->buf[0]->size) != JXL_DEC_SUCCESS) { + if (JxlDecoderSetImageOutBuffer(ctx->decoder, &ctx->jxl_pixfmt, frame->data[0], frame->buf[0]->size) + != JXL_DEC_SUCCESS) { av_log(avctx, AV_LOG_ERROR, "Bad libjxl dec need image out buffer event\n"); return AVERROR_EXTERNAL; } From patchwork Fri Jun 24 16:50:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leo Izen X-Patchwork-Id: 36415 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:8b27:b0:88:1bbf:7fd2 with SMTP id l39csp421623pzh; Fri, 24 Jun 2022 09:51:30 -0700 (PDT) X-Google-Smtp-Source: AGRyM1u8OchmsVOT6B9MJRYa4wAK3gE/vEwIdfacSOUqcibNogWFL/2+Kjfemjb4kaCjxNm3yR5Y X-Received: by 2002:a17:907:94d4:b0:722:e4b8:c2f2 with SMTP id dn20-20020a17090794d400b00722e4b8c2f2mr14069557ejc.527.1656089490469; Fri, 24 Jun 2022 09:51:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1656089490; cv=none; d=google.com; s=arc-20160816; b=auo7cyBUdPr3X3j4i68tQrGdzLupC3CHUrt2CvxK46RhmMdYrSNLr/q8NzGv4WCWJd /lfienI4dk1rDghHeprJErtumxO9lbQ/5ElQvgtysxjVM4zxMIPR0jyGbiljz0uXeK46 UIkOI+MIAPmNzev6KmvyarP9P4iLJ2j+AmqRi1bYO8dC+GV/MnuseZBsp3Q96t+Lti1d EXa5Ign1po4RKzfK5j6BaBQp64neqN9PSmyyyKyLgnQa3+NWXGDh9Ij3tSb1c7KxDI7c 2JNqdJWvUrvrY3xzqtnT+iViNNjd49e2UhHUnXEJSr/jLfLLqEnu4+lD+Uqvw6MOfSM0 zjBQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=UJTQe9zQkMGvymGSW4lxUwzsChSHYRgOFmKswP6T0kg=; b=Al2esjPMwWD9SWjd6CK6E6o4OFm+fAiJsLVseN2rixSBjHfGmDTB1KLqz6ZfEUegPD KIAS9XXaNTd2tEk3WaGGkzUBFJ+ukdIghw7viOWcPkgmmxknleAwfnWosWaC7TaTyrV1 jWWydoazqsM5P9l58y44AfMdmaYWaaYtagl9AUw6jflERfn1NKvG8f7890F3GY0+53Bl WVPWbQIl3I8mSnSg1WvajEUVPHibpGvP8RTtYA2AJohIqrybswncHxI9tb3cyeTccP+M peIldOM9YLZ98d/42cJj+d9WokO6XEP4s2PHsLdIredZutL8uCCw8hwCwhVMBza2oOwK Apwg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=P6lZRH6d; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id jg6-20020a170907970600b00726294f089fsi3271485ejc.229.2022.06.24.09.51.30; Fri, 24 Jun 2022 09:51:30 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=P6lZRH6d; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9D1FD68B74D; Fri, 24 Jun 2022 19:51:13 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qv1-f52.google.com (mail-qv1-f52.google.com [209.85.219.52]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 8A53268B6F1 for ; Fri, 24 Jun 2022 19:51:06 +0300 (EEST) Received: by mail-qv1-f52.google.com with SMTP id cs6so5382233qvb.6 for ; Fri, 24 Jun 2022 09:51:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Y9wlujvGLyMdnizwgAn/d+pdn9KTNfMcsctKuwvCTPI=; b=P6lZRH6duEuIlbcTgoqKmSMOwNpFOt4nL8FAFOBjXIHlnrU/CBOzjPiKj+NABX680B ThjW6wnznWFez2LU6xLDd9UbjRTVsMQYlujtLeCe4GbhIqPFjoqauJoGGsESQG9iTWQ5 YLzIzCZ6IHIZEQfzVp2IshlTnj3vCIixDzClf/1IcBGFl96BmCVwP44WWlRWNTFaIM9D Bt2Z28BGDwGKNK9aur4BISIZWL3nJfdrmZg/lS+uwU2bUE108bDnjdJ+vBWAi74d8DKX biKwm242jDvfgBvkH17j7S8Q+uwNecKDk1UrYb/RlUL4a0rt7ilqn8FLvqaNTZQe1VSP ojuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Y9wlujvGLyMdnizwgAn/d+pdn9KTNfMcsctKuwvCTPI=; b=sk54jLWwN09sRUyKqh6tcTBhSShBEfE3I6XAhdvjG2cboTxbkruZ8hbo0RDTc5XfP1 9xdkTRJPv390LsIqdvF9SvG9tie5JjhM+PGSSAPGBtdDQne+B4N9fPY/Ro299SZRtXy3 mSXWyE8KYGmjZZiFJxxSMQFkE9TmoDqCPzN1SgG98hHjj1U98CBbK7edy5lBZkAaN30d I/5pqno3iMeK0yO7Z3Zm67GolxgfmAwfd23CpsexQ7wNM85KcYL1XfZX7ubgkGw6kdn7 MoOHnzrwg7Hc3rMikTvv/7DOO33LRNRMAr+sjnd0MYvAUyo1kZSEwuJgA/UfVOmnEap5 tcrw== X-Gm-Message-State: AJIora+NiOx1XbC1uOrYzSIW4zl+ayu4SCOO3g4/7EKMSHkeFH+EQpIv x/qLSAP59O/hPW8/QxFQ4Mo7X4aOik4= X-Received: by 2002:a05:622a:118f:b0:317:cb6c:8027 with SMTP id m15-20020a05622a118f00b00317cb6c8027mr2632725qtk.305.1656089465262; Fri, 24 Jun 2022 09:51:05 -0700 (PDT) Received: from gauss.local (c-68-41-54-207.hsd1.mi.comcast.net. [68.41.54.207]) by smtp.gmail.com with ESMTPSA id q15-20020ac8450f000000b003177f0fb61esm1588106qtn.75.2022.06.24.09.51.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 24 Jun 2022 09:51:04 -0700 (PDT) From: Leo Izen To: ffmpeg-devel@ffmpeg.org Date: Fri, 24 Jun 2022 12:50:56 -0400 Message-Id: <20220624165056.51700-3-leo.izen@gmail.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220624165056.51700-1-leo.izen@gmail.com> References: <20220624165056.51700-1-leo.izen@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v9 2/2] avcodec/libjxlenc: properly read input colorspace X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Leo Izen Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: TcCjIGwtJf1A Whether an ICC profile is present or not, the libjxl encoder wrapper should now properly read colorspace tags and forward them to libjxl appropriately, rather than just assume sRGB as before. It will also print warnings when colorimetric assumptions are made about the input data. --- libavcodec/libjxlenc.c | 153 +++++++++++++++++++++++++++++++++-------- 1 file changed, 123 insertions(+), 30 deletions(-) diff --git a/libavcodec/libjxlenc.c b/libavcodec/libjxlenc.c index 8bebec6aeb..6a948cc3ae 100644 --- a/libavcodec/libjxlenc.c +++ b/libavcodec/libjxlenc.c @@ -27,6 +27,7 @@ #include #include "libavutil/avutil.h" +#include "libavutil/csp.h" #include "libavutil/error.h" #include "libavutil/frame.h" #include "libavutil/libm.h" @@ -117,7 +118,7 @@ static int libjxl_init_jxl_encoder(AVCodecContext *avctx) return AVERROR_EXTERNAL; } - /* check for negative zero, our default */ + /* check for negative, our default */ if (ctx->distance < 0.0) { /* use ffmpeg.c -q option if passed */ if (avctx->flags & AV_CODEC_FLAG_QSCALE) @@ -133,7 +134,7 @@ static int libjxl_init_jxl_encoder(AVCodecContext *avctx) */ if (ctx->distance > 0.0 && ctx->distance < 0.01) ctx->distance = 0.01; - if (JxlEncoderOptionsSetDistance(ctx->options, ctx->distance) != JXL_ENC_SUCCESS) { + if (JxlEncoderSetFrameDistance(ctx->options, ctx->distance) != JXL_ENC_SUCCESS) { av_log(avctx, AV_LOG_ERROR, "Failed to set distance: %f\n", ctx->distance); return AVERROR_EXTERNAL; } @@ -185,6 +186,52 @@ static av_cold int libjxl_encode_init(AVCodecContext *avctx) return 0; } +/** + * Populate a JxlColorEncoding with the given enum AVColorPrimaries. + * @return < 0 upon failure, >= 0 upon success + */ +static int libjxl_populate_primaries(JxlColorEncoding *jxl_color, enum AVColorPrimaries prm) +{ + const AVColorPrimariesDesc *desc; + + switch (prm) { + case AVCOL_PRI_BT709: + jxl_color->primaries = JXL_PRIMARIES_SRGB; + jxl_color->white_point = JXL_WHITE_POINT_D65; + return 0; + case AVCOL_PRI_BT2020: + jxl_color->primaries = JXL_PRIMARIES_2100; + jxl_color->white_point = JXL_WHITE_POINT_D65; + return 0; + case AVCOL_PRI_SMPTE431: + jxl_color->primaries = JXL_PRIMARIES_P3; + jxl_color->white_point = JXL_WHITE_POINT_DCI; + return 0; + case AVCOL_PRI_SMPTE432: + jxl_color->primaries = JXL_PRIMARIES_P3; + jxl_color->white_point = JXL_WHITE_POINT_D65; + return 0; + } + + desc = av_csp_primaries_desc_from_id(prm); + if (!desc) + return AVERROR(EINVAL); + + jxl_color->primaries = JXL_PRIMARIES_CUSTOM; + jxl_color->white_point = JXL_WHITE_POINT_CUSTOM; + + jxl_color->primaries_red_xy[0] = av_q2d(desc->prim.r.x); + jxl_color->primaries_red_xy[1] = av_q2d(desc->prim.r.y); + jxl_color->primaries_green_xy[0] = av_q2d(desc->prim.g.x); + jxl_color->primaries_green_xy[1] = av_q2d(desc->prim.g.y); + jxl_color->primaries_blue_xy[0] = av_q2d(desc->prim.b.x); + jxl_color->primaries_blue_xy[1] = av_q2d(desc->prim.b.y); + jxl_color->white_point_xy[0] = av_q2d(desc->wp.x); + jxl_color->white_point_xy[1] = av_q2d(desc->wp.y); + + return 0; +} + /** * Encode an entire frame. Currently animation, is not supported by * this encoder, so this will always reinitialize a new still image @@ -223,49 +270,95 @@ static int libjxl_encode_frame(AVCodecContext *avctx, AVPacket *pkt, const AVFra info.exponent_bits_per_sample = info.bits_per_sample > 16 ? 8 : 5; info.alpha_exponent_bits = info.alpha_bits ? info.exponent_bits_per_sample : 0; jxl_fmt.data_type = info.bits_per_sample > 16 ? JXL_TYPE_FLOAT : JXL_TYPE_FLOAT16; - JxlColorEncodingSetToLinearSRGB(&jxl_color, info.num_color_channels == 1); } else { info.exponent_bits_per_sample = 0; info.alpha_exponent_bits = 0; jxl_fmt.data_type = info.bits_per_sample <= 8 ? JXL_TYPE_UINT8 : JXL_TYPE_UINT16; - JxlColorEncodingSetToSRGB(&jxl_color, info.num_color_channels == 1); } - if (info.bits_per_sample > 16 - || info.xsize > (1 << 18) || info.ysize > (1 << 18) - || (info.xsize << 4) * (info.ysize << 4) > (1 << 20)) { - /* - * must upgrade codestream to level 10, from level 5 - * the encoder will not do this automatically - */ - if (JxlEncoderSetCodestreamLevel(ctx->encoder, 10) != JXL_ENC_SUCCESS) { - av_log(avctx, AV_LOG_ERROR, "Could not upgrade JXL Codestream level.\n"); - return AVERROR_EXTERNAL; - } - } + /* JPEG XL format itself does not support limited range */ + if (avctx->color_range == AVCOL_RANGE_MPEG || + avctx->color_range == AVCOL_RANGE_UNSPECIFIED && frame->color_range == AVCOL_RANGE_MPEG) + av_log(avctx, AV_LOG_WARNING, "This encoder does not support limited (tv) range, colors will be wrong!\n"); + else if (avctx->color_range != AVCOL_RANGE_JPEG && frame->color_range != AVCOL_RANGE_JPEG) + av_log(avctx, AV_LOG_WARNING, "Unknown color range, assuming full (pc)\n"); /* bitexact lossless requires there to be no XYB transform */ info.uses_original_profile = ctx->distance == 0.0; - sd = av_frame_get_side_data(frame, AV_FRAME_DATA_ICC_PROFILE); - if (sd && sd->size && JxlEncoderSetICCProfile(ctx->encoder, sd->data, sd->size) != JXL_ENC_SUCCESS) { - av_log(avctx, AV_LOG_WARNING, "Could not set ICC Profile\n"); - } else if (info.uses_original_profile) { - /* - * the color encoding is not used if uses_original_profile is false - * this just works around a bug in libjxl 0.7.0 and lower - */ - if (JxlEncoderSetColorEncoding(ctx->encoder, &jxl_color) != JXL_ENC_SUCCESS) { - av_log(avctx, AV_LOG_ERROR, "Failed to set JxlColorEncoding\n"); - return AVERROR_EXTERNAL; - } - } - if (JxlEncoderSetBasicInfo(ctx->encoder, &info) != JXL_ENC_SUCCESS) { av_log(avctx, AV_LOG_ERROR, "Failed to set JxlBasicInfo\n"); return AVERROR_EXTERNAL; } + /* rendering intent doesn't matter here + * but libjxl will whine if we don't set it */ + jxl_color.rendering_intent = JXL_RENDERING_INTENT_RELATIVE; + + switch (frame->color_trc && frame->color_trc != AVCOL_TRC_UNSPECIFIED + ? frame->color_trc : avctx->color_trc) { + case AVCOL_TRC_BT709: + jxl_color.transfer_function = JXL_TRANSFER_FUNCTION_709; + break; + case AVCOL_TRC_LINEAR: + jxl_color.transfer_function = JXL_TRANSFER_FUNCTION_LINEAR; + break; + case AVCOL_TRC_IEC61966_2_1: + jxl_color.transfer_function = JXL_TRANSFER_FUNCTION_SRGB; + break; + case AVCOL_TRC_SMPTE428: + jxl_color.transfer_function = JXL_TRANSFER_FUNCTION_DCI; + break; + case AVCOL_TRC_SMPTE2084: + jxl_color.transfer_function = JXL_TRANSFER_FUNCTION_PQ; + break; + case AVCOL_TRC_ARIB_STD_B67: + jxl_color.transfer_function = JXL_TRANSFER_FUNCTION_HLG; + break; + case AVCOL_TRC_GAMMA22: + jxl_color.transfer_function = JXL_TRANSFER_FUNCTION_GAMMA; + jxl_color.gamma = 2.2; + break; + case AVCOL_TRC_GAMMA28: + jxl_color.transfer_function = JXL_TRANSFER_FUNCTION_GAMMA; + jxl_color.gamma = 2.8; + break; + default: + if (pix_desc->flags & AV_PIX_FMT_FLAG_FLOAT) { + av_log(avctx, AV_LOG_WARNING, "Unknown transfer function, assuming Linear Light. Colors may be wrong.\n"); + jxl_color.transfer_function = JXL_TRANSFER_FUNCTION_LINEAR; + } else { + av_log(avctx, AV_LOG_WARNING, "Unknown transfer function, assuming IEC61966-2-1/sRGB. Colors may be wrong.\n"); + jxl_color.transfer_function = JXL_TRANSFER_FUNCTION_SRGB; + } + } + + /* This should be implied to be honest + * but a libjxl bug makes it fail otherwise */ + if (info.num_color_channels == 1) + jxl_color.color_space = JXL_COLOR_SPACE_GRAY; + else + jxl_color.color_space = JXL_COLOR_SPACE_RGB; + + ret = libjxl_populate_primaries(&jxl_color, + frame->color_primaries && frame->color_primaries != AVCOL_PRI_UNSPECIFIED + ? frame->color_primaries : avctx->color_primaries); + if (ret < 0) + return ret; + + sd = av_frame_get_side_data(frame, AV_FRAME_DATA_ICC_PROFILE); + if (sd && sd->size && JxlEncoderSetICCProfile(ctx->encoder, sd->data, sd->size) != JXL_ENC_SUCCESS) + av_log(avctx, AV_LOG_WARNING, "Could not set ICC Profile\n"); + if (JxlEncoderSetColorEncoding(ctx->encoder, &jxl_color) != JXL_ENC_SUCCESS) + av_log(avctx, AV_LOG_WARNING, "Failed to set JxlColorEncoding\n"); + + /* depending on basic info, level 10 might + * be required instead of level 5 */ + if (JxlEncoderGetRequiredCodestreamLevel(ctx->encoder) > 5) { + if (JxlEncoderSetCodestreamLevel(ctx->encoder, 10) != JXL_ENC_SUCCESS) + av_log(avctx, AV_LOG_WARNING, "Could not increase codestream level\n"); + } + jxl_fmt.endianness = JXL_NATIVE_ENDIAN; jxl_fmt.align = frame->linesize[0];