From patchwork Wed Apr 3 15:43:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 47757 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:9f96:b0:1a3:b6bb:3029 with SMTP id mm22csp485456pzb; Wed, 3 Apr 2024 08:44:27 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCVTWu2hjw61TyxeqGsUck+IxeJmvU11VeMnArTYunfRh2OnIP1OMdySFDjyFrCEay3puouxvUmsvCFSNrWStBF1qS95x7yHWkg5wQ== X-Google-Smtp-Source: AGHT+IGpHgF3WbTtAX1/I8QzpDMfwYMtNmdkHLYtTnZZo7cjzOeapcJDW+qUW0VUc7HAkO2+XPaq X-Received: by 2002:ac2:5208:0:b0:516:a1f8:8283 with SMTP id a8-20020ac25208000000b00516a1f88283mr6220228lfl.2.1712159067701; Wed, 03 Apr 2024 08:44:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1712159067; cv=none; d=google.com; s=arc-20160816; b=FnBD50mL0yr0T4Yu0ufHYuBCcBHmtCkTCBjaVI5+7pSkspeSqglBTNBAYGneoJled7 30QbcqKH0IWCTDGNU8HaMjm918tbI99aLYf6EcGUYUvn9MsU8nFEICe/tJp90G0TgwOF UknPVnDAF7glR7jrTx38gMQf+GIMHQVF5zv2JRVKq8H49ZpNeJa/zy+XnMn2BrAWfPL4 TT2MX32q9Arq+UtDAqwJJIkkenvEGL4bj0cZM5JRC0DzbaSLzBMMW+ouLaj9azhO+s5A gfJ0f3qJSdqYfxe6bJioiyL8CO5XSchZ8k093Vsv/mOHFzZglTjBhR+r8qbCUkw4+xqf AeNA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=brg+Xm1i7C1tVS06gT8iB8dt3gR38kThDy97C8Q2omo=; fh=xmAeKtysnShNOmkhiJmYkS30uw4Fu2hvBJ7qlIwukxQ=; b=A9jLoYA0px+6fJYnqFyVLcFpIWRaGIN+03a5x79A1QX8m28dJ20aKSuBOGElos9Ao1 H9ZVeA66idg3Ac4uzjQ86DKI4Nt96BT2X+bcnfGbSPQi70dXkndY17MqE4vqwX/5Du56 0qrKej/514OPQ1WfZxtVApOjGSwreGGm3CeWUxDjQsJDli4bHd8TczoIBUegR/RpEvCd RWBZZm6FPvGA0m9/ArP2XN4k0wmOxFiLYDq6ZbCcZ+R4L/AzikM5YLbRtvYjxLtmlygM HeLgXv2Df8ci/64aIVGEPOVOxvhx7U6ZlvktzfQSH4I8aV/f8sl0gafb4Pi014ioH0aV E1BA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=D5eGGkkH; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id u23-20020a170906655700b00a474be847e1si7009441ejn.758.2024.04.03.08.44.27; Wed, 03 Apr 2024 08:44:27 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=D5eGGkkH; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B2B3868D19B; Wed, 3 Apr 2024 18:43:45 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id F419A68D1BC for ; Wed, 3 Apr 2024 18:43:38 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1712159015; bh=/kwt6TXyy6QYrteGYB1Fzvj5cPHYs9UbqKWjc81sDQo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=D5eGGkkHya8p5IKO+/6vkF4Skt4UobEC1VeJsREGOAZl+ikyKxfUZwH9xU6otiwm3 3HUjyO6JgrhUMkK/4FGrSK1ztQv8RJ358o/hVXbwb4dmtuc8cr3nnRCYUmB5FkSaz2 pS8joEhgsKRbMn6TWuu5/Oq7eJr+UfMDYjOvZkzI= Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id 6043542A98; Wed, 3 Apr 2024 17:43:35 +0200 (CEST) From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Wed, 3 Apr 2024 17:43:26 +0200 Message-ID: <20240403154330.71585-6-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240403154330.71585-1-ffmpeg@haasn.xyz> References: <20240403154330.71585-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 06/10] avcodec/dovi_rpu: add ff_dovi_configure() X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: oWVqurKydnqv From: Niklas Haas We need to set up the configuration struct appropriately based on the codec type, colorspace metadata, and presence/absence of an EL (though, we currently don't support an EL). When present, we use the signalled RPU data header to help infer (and validate) the right values. --- libavcodec/dovi_rpu.c | 176 ++++++++++++++++++++++++++++++++++++++++++ libavcodec/dovi_rpu.h | 14 +++- 2 files changed, 189 insertions(+), 1 deletion(-) diff --git a/libavcodec/dovi_rpu.c b/libavcodec/dovi_rpu.c index 4da711d763e..b4e8d0cdea4 100644 --- a/libavcodec/dovi_rpu.c +++ b/libavcodec/dovi_rpu.c @@ -144,6 +144,182 @@ static int guess_hevc_profile(const AVDOVIRpuDataHeader *hdr) return 0; /* unknown */ } +static struct { + uint64_t pps; // maximum pixels per second + int width; // maximum width + int main; // maximum bitrate in main tier + int high; // maximum bitrate in high tier +} dv_levels[] = { + [1] = {1280*720*24, 1280, 20, 50}, + [2] = {1280*720*30, 1280, 20, 50}, + [3] = {1920*1080*24, 1920, 20, 70}, + [4] = {1920*1080*30, 2560, 20, 70}, + [5] = {1920*1080*60, 3840, 20, 70}, + [6] = {3840*2160*24, 3840, 25, 130}, + [7] = {3840*2160*30, 3840, 25, 130}, + [8] = {3840*2160*48, 3840, 40, 130}, + [9] = {3840*2160*60, 3840, 40, 130}, + [10] = {3840*2160*120, 3840, 60, 240}, + [11] = {3840*2160*120, 7680, 60, 240}, + [12] = {7680*4320*60, 7680, 120, 450}, + [13] = {7680*4320*120u, 7680, 240, 800}, +}; + +int ff_dovi_configure(DOVIContext *s, AVCodecContext *avctx) +{ + AVDOVIDecoderConfigurationRecord *cfg; + const AVDOVIRpuDataHeader *hdr = NULL; + const AVFrameSideData *sd; + int dv_profile, dv_level, bl_compat_id; + size_t cfg_size; + uint64_t pps; + + if (!avctx->dolbyvision) + goto skip; + + sd = av_frame_side_data_get(avctx->decoded_side_data, + avctx->nb_decoded_side_data, AV_FRAME_DATA_DOVI_METADATA); + + if (sd) + hdr = av_dovi_get_header((const AVDOVIMetadata *) sd->data); + + if (avctx->dolbyvision == FF_DOLBYVISION_AUTO && !hdr) + goto skip; + + switch (avctx->codec_id) { + case AV_CODEC_ID_AV1: dv_profile = 10; break; + case AV_CODEC_ID_H264: dv_profile = 9; break; + case AV_CODEC_ID_HEVC: dv_profile = hdr ? guess_hevc_profile(hdr) : 8; break; + default: + /* No other encoder should be calling this! */ + av_assert0(0); + return AVERROR_BUG; + } + + if (avctx->strict_std_compliance > FF_COMPLIANCE_UNOFFICIAL) { + if (dv_profile == 9) { + if (avctx->pix_fmt != AV_PIX_FMT_YUV420P) + dv_profile = 0; + } else { + if (avctx->pix_fmt != AV_PIX_FMT_YUV420P10) + dv_profile = 0; + } + } + + switch (dv_profile) { + case 0: /* None */ + bl_compat_id = -1; + break; + case 4: /* HEVC with enhancement layer */ + case 7: + if (avctx->dolbyvision > 0) { + av_log(s->logctx, AV_LOG_ERROR, "Coding of Dolby Vision enhancement " + "layers is currently unsupported."); + return AVERROR_PATCHWELCOME; + } else { + goto skip; + } + case 5: /* HEVC with proprietary IPTPQc2 */ + bl_compat_id = 0; + break; + case 10: + /* FIXME: check for proper H.273 tags once those are added */ + if (hdr && hdr->bl_video_full_range_flag) { + /* AV1 with proprietary IPTPQc2 */ + bl_compat_id = 0; + break; + } + /* fall through */ + case 8: /* HEVC (or AV1) with BL compatibility */ + if (avctx->colorspace == AVCOL_SPC_BT2020_NCL && + avctx->color_primaries == AVCOL_PRI_BT2020 && + avctx->color_trc == AVCOL_TRC_SMPTE2084) { + bl_compat_id = 1; + } else if (avctx->colorspace == AVCOL_SPC_BT2020_NCL && + avctx->color_primaries == AVCOL_PRI_BT2020 && + avctx->color_trc == AVCOL_TRC_ARIB_STD_B67) { + bl_compat_id = 4; + } else if (avctx->colorspace == AVCOL_SPC_BT709 && + avctx->color_primaries == AVCOL_PRI_BT709 && + avctx->color_trc == AVCOL_TRC_BT709) { + bl_compat_id = 2; + } else { + /* Not a valid colorspace combination */ + bl_compat_id = -1; + } + } + + if (!dv_profile || bl_compat_id < 0) { + if (avctx->dolbyvision > 0) { + av_log(s->logctx, AV_LOG_ERROR, "Dolby Vision enabled, but could " + "not determine profile and compaatibility mode. Double-check " + "colorspace and format settings for compatibility?\n"); + return AVERROR(EINVAL); + } + goto skip; + } + + pps = avctx->width * avctx->height; + if (avctx->framerate.num) { + pps = pps * avctx->framerate.num / avctx->framerate.den; + } else { + pps *= 25; /* sanity fallback */ + } + + dv_level = 0; + for (int i = 1; i < FF_ARRAY_ELEMS(dv_levels); i++) { + if (pps > dv_levels[i].pps) + continue; + if (avctx->width > dv_levels[i].width) + continue; + /* In theory, we should also test the bitrate when known, and + * distinguish between main and high tier. In practice, just ignore + * the bitrate constraints and hope they work out. This would ideally + * be handled by either the encoder or muxer directly. */ + dv_level = i; + break; + } + + if (!dv_level) { + if (avctx->strict_std_compliance >= FF_COMPLIANCE_STRICT) { + av_log(s->logctx, AV_LOG_ERROR, "Coded PPS (%"PRIu64") and width (%d) " + "exceed Dolby Vision limitations\n", pps, avctx->width); + return AVERROR(EINVAL); + } else { + av_log(s->logctx, AV_LOG_WARNING, "Coded PPS (%"PRIu64") and width (%d) " + "exceed Dolby Vision limitations. Ignoring, resulting file " + "may be non-conforming.\n", pps, avctx->width); + dv_level = FF_ARRAY_ELEMS(dv_levels) - 1; + } + } + + cfg = av_dovi_alloc(&cfg_size); + if (!cfg) + return AVERROR(ENOMEM); + + if (!av_packet_side_data_add(&avctx->coded_side_data, &avctx->nb_coded_side_data, + AV_PKT_DATA_DOVI_CONF, cfg, cfg_size, 0)) { + av_free(cfg); + return AVERROR(ENOMEM); + } + + cfg->dv_version_major = 1; + cfg->dv_version_minor = 0; + cfg->dv_profile = dv_profile; + cfg->dv_level = dv_level; + cfg->rpu_present_flag = 1; + cfg->el_present_flag = 0; + cfg->bl_present_flag = 1; + cfg->dv_bl_signal_compatibility_id = bl_compat_id; + + s->cfg = *cfg; + return 0; + +skip: + s->cfg = (AVDOVIDecoderConfigurationRecord) {0}; + return 0; +} + static inline uint64_t get_ue_coef(GetBitContext *gb, const AVDOVIRpuDataHeader *hdr) { uint64_t ipart; diff --git a/libavcodec/dovi_rpu.h b/libavcodec/dovi_rpu.h index 9a68e45bf1b..33e19dd037c 100644 --- a/libavcodec/dovi_rpu.h +++ b/libavcodec/dovi_rpu.h @@ -26,6 +26,7 @@ #include "libavutil/dovi_meta.h" #include "libavutil/frame.h" +#include "avcodec.h" #define DOVI_MAX_DM_ID 15 typedef struct DOVIContext { @@ -33,7 +34,8 @@ typedef struct DOVIContext { /** * Currently active dolby vision configuration, or {0} for none. - * Set by the user when decoding. + * Set by the user when decoding. Generated by ff_dovi_configure() + * when encoding. * * Note: sizeof(cfg) is not part of the libavutil ABI, so users should * never pass &cfg to any other library calls. This is included merely as @@ -96,4 +98,14 @@ int ff_dovi_rpu_parse(DOVIContext *s, const uint8_t *rpu, size_t rpu_size, */ int ff_dovi_attach_side_data(DOVIContext *s, AVFrame *frame); +/** + * Configure the encoder for Dolby Vision encoding. Generates a configuration + * record in s->cfg, and attaches it to avctx->coded_side_data. Sets the correct + * profile and compatibility ID based on the tagged AVCodecContext colorspace + * metadata, and the correct level based on the resolution and tagged framerate. + * + * Returns 0 or a negative error code. + */ +int ff_dovi_configure(DOVIContext *s, AVCodecContext *avctx); + #endif /* AVCODEC_DOVI_RPU_H */