From patchwork Tue Apr 9 12:57:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 47978 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:9c8d:b0:1a7:a0dc:8de5 with SMTP id mj13csp333267pzb; Tue, 9 Apr 2024 06:00:53 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCWaQGRAiLSjBZHxwqsGlXqT8Lhvj60h01yeauzJH6/Lhi/w5/DKNgx9Sg/0otFZmZubLsepo9kcxPFOknaR1U/5LCKBdy3vpOjtUg== X-Google-Smtp-Source: AGHT+IHHox/cA+GlikEQicldqPkqVCRLBKPpezaIMn2DCyJ9huCO7R7mPPzGLEv6TlX/L/KTwV2l X-Received: by 2002:a2e:3807:0:b0:2d2:246e:b373 with SMTP id f7-20020a2e3807000000b002d2246eb373mr7924487lja.5.1712667653204; Tue, 09 Apr 2024 06:00:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1712667653; cv=none; d=google.com; s=arc-20160816; b=fgtnqeVAtEOM8kM+H6ceWPrJQP0ZFu6BDGToKLnEj66Ova2wPEk6oAPtmREJ0JNoCq 2KJfEuduXqXzq3Eewm2Ap0Ta0z+UB6xsUBYJLfDK+9cns+9UNdupBljdxMgLjAIQ/bGE BVSHTtcShcKXbRidN0vxj7zZIw0jp9/kxOXMoPFX4iaQdleQOYqE89y7H6JbmEYj/YFf 54Np3gIZeSAksqzYpfT2S2/Ld8WFzRD7E1Mdi65hz+Uo2ljDpThsI8kq6U1XzklccNE2 EvUoOQ9xgG8wy9qEtPX8qddKkH2g5Q/n9A4+d1YpigPla+I78dZQe8wlcikMFzLTnETO c9JA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=poGcRvAuza6uY3Ra7wN6Gn3nwTFAmHVjDHlwb7PRCAY=; fh=xmAeKtysnShNOmkhiJmYkS30uw4Fu2hvBJ7qlIwukxQ=; b=WWdXZ/X/wnzMGiesC70imbpOf+7K0AhulP9U5aFEG46J2uGzaIgddt1Z/f4Iw5l9Qp 13fgMxpEtbvP/U0Sb5Iq1Dci5mhnDZFOL89QNEI+BKBiKSnWofr68wgwO3uOjQISxSDd 4iXZonxqbo2D9aAp8xjdgqDe6wwWtHBgdxOxSHQ6nbUgFdCM2T2YGrcg1DJ+bNR8hoEd EsrtQqS2NsKYRQSmlIwAxVsIGfwAKyyaAPuOPEDMzBbsmB96+KXgxUjEvE/Wd8tr+B97 2jDXGgSntLGywyHNepmizTmyjMu0mFD3QWzp1SlxZrfcurHe78Jy3QF96U+NN3W0+wol 9SQA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=XMH1KtpP; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id x20-20020a2e8814000000b002d8408c3f63si2761635ljh.226.2024.04.09.06.00.52; Tue, 09 Apr 2024 06:00:53 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=XMH1KtpP; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0D6DA68CD9D; Tue, 9 Apr 2024 15:59:31 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id DF33568D188 for ; Tue, 9 Apr 2024 15:59:21 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1712667557; bh=2HTTf6dYj9S+6fzcjjxcmVf2ux8oKFJ37FyeDSh+HSQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XMH1KtpPwc7q1JwvC5LG2b5VUQUld56kTuyU5SuynV6hoU2CRsjpxH0RbyXqZ+/va RaXJ09EtFbOnvUAmVAaN7fCbTBzyWR0huPgd8wILk3kbiJYyHb27ZJ8Mvlm6Lvu9Dj fXyl6XePVb7EP/PaN2qNDREloImWHfOM6YTOoUTA= Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id 774D342765; Tue, 9 Apr 2024 14:59:17 +0200 (CEST) From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Tue, 9 Apr 2024 14:57:25 +0200 Message-ID: <20240409125914.61149-6-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240409125914.61149-1-ffmpeg@haasn.xyz> References: <20240409125914.61149-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 05/11] avcodec/dovi_rpu: add ff_dovi_configure() X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: wH3jvD40qvtU From: Niklas Haas We need to set up the configuration struct appropriately based on the codec type, colorspace metadata, and presence/absence of an EL (though, we currently don't support an EL). When present, we use the signalled RPU data header to help infer (and validate) the right values. Behavior can be controlled by a new DOVIContext.enable flag. --- libavcodec/dovi_rpu.c | 176 ++++++++++++++++++++++++++++++++++++++++++ libavcodec/dovi_rpu.h | 23 +++++- 2 files changed, 198 insertions(+), 1 deletion(-) diff --git a/libavcodec/dovi_rpu.c b/libavcodec/dovi_rpu.c index 4da711d763e..d3a284c150d 100644 --- a/libavcodec/dovi_rpu.c +++ b/libavcodec/dovi_rpu.c @@ -144,6 +144,182 @@ static int guess_hevc_profile(const AVDOVIRpuDataHeader *hdr) return 0; /* unknown */ } +static struct { + uint64_t pps; // maximum pixels per second + int width; // maximum width + int main; // maximum bitrate in main tier + int high; // maximum bitrate in high tier +} dv_levels[] = { + [1] = {1280*720*24, 1280, 20, 50}, + [2] = {1280*720*30, 1280, 20, 50}, + [3] = {1920*1080*24, 1920, 20, 70}, + [4] = {1920*1080*30, 2560, 20, 70}, + [5] = {1920*1080*60, 3840, 20, 70}, + [6] = {3840*2160*24, 3840, 25, 130}, + [7] = {3840*2160*30, 3840, 25, 130}, + [8] = {3840*2160*48, 3840, 40, 130}, + [9] = {3840*2160*60, 3840, 40, 130}, + [10] = {3840*2160*120, 3840, 60, 240}, + [11] = {3840*2160*120, 7680, 60, 240}, + [12] = {7680*4320*60, 7680, 120, 450}, + [13] = {7680*4320*120u, 7680, 240, 800}, +}; + +int ff_dovi_configure(DOVIContext *s, AVCodecContext *avctx) +{ + AVDOVIDecoderConfigurationRecord *cfg; + const AVDOVIRpuDataHeader *hdr = NULL; + const AVFrameSideData *sd; + int dv_profile, dv_level, bl_compat_id; + size_t cfg_size; + uint64_t pps; + + if (!s->enable) + goto skip; + + sd = av_frame_side_data_get(avctx->decoded_side_data, + avctx->nb_decoded_side_data, AV_FRAME_DATA_DOVI_METADATA); + + if (sd) + hdr = av_dovi_get_header((const AVDOVIMetadata *) sd->data); + + if (s->enable == FF_DOVI_AUTOMATIC && !hdr) + goto skip; + + switch (avctx->codec_id) { + case AV_CODEC_ID_AV1: dv_profile = 10; break; + case AV_CODEC_ID_H264: dv_profile = 9; break; + case AV_CODEC_ID_HEVC: dv_profile = hdr ? guess_hevc_profile(hdr) : 8; break; + default: + /* No other encoder should be calling this! */ + av_assert0(0); + return AVERROR_BUG; + } + + if (avctx->strict_std_compliance > FF_COMPLIANCE_UNOFFICIAL) { + if (dv_profile == 9) { + if (avctx->pix_fmt != AV_PIX_FMT_YUV420P) + dv_profile = 0; + } else { + if (avctx->pix_fmt != AV_PIX_FMT_YUV420P10) + dv_profile = 0; + } + } + + switch (dv_profile) { + case 0: /* None */ + bl_compat_id = -1; + break; + case 4: /* HEVC with enhancement layer */ + case 7: + if (s->enable > 0) { + av_log(s->logctx, AV_LOG_ERROR, "Coding of Dolby Vision enhancement " + "layers is currently unsupported."); + return AVERROR_PATCHWELCOME; + } else { + goto skip; + } + case 5: /* HEVC with proprietary IPTPQc2 */ + bl_compat_id = 0; + break; + case 10: + /* FIXME: check for proper H.273 tags once those are added */ + if (hdr && hdr->bl_video_full_range_flag) { + /* AV1 with proprietary IPTPQc2 */ + bl_compat_id = 0; + break; + } + /* fall through */ + case 8: /* HEVC (or AV1) with BL compatibility */ + if (avctx->colorspace == AVCOL_SPC_BT2020_NCL && + avctx->color_primaries == AVCOL_PRI_BT2020 && + avctx->color_trc == AVCOL_TRC_SMPTE2084) { + bl_compat_id = 1; + } else if (avctx->colorspace == AVCOL_SPC_BT2020_NCL && + avctx->color_primaries == AVCOL_PRI_BT2020 && + avctx->color_trc == AVCOL_TRC_ARIB_STD_B67) { + bl_compat_id = 4; + } else if (avctx->colorspace == AVCOL_SPC_BT709 && + avctx->color_primaries == AVCOL_PRI_BT709 && + avctx->color_trc == AVCOL_TRC_BT709) { + bl_compat_id = 2; + } else { + /* Not a valid colorspace combination */ + bl_compat_id = -1; + } + } + + if (!dv_profile || bl_compat_id < 0) { + if (s->enable > 0) { + av_log(s->logctx, AV_LOG_ERROR, "Dolby Vision enabled, but could " + "not determine profile and compaatibility mode. Double-check " + "colorspace and format settings for compatibility?\n"); + return AVERROR(EINVAL); + } + goto skip; + } + + pps = avctx->width * avctx->height; + if (avctx->framerate.num) { + pps = pps * avctx->framerate.num / avctx->framerate.den; + } else { + pps *= 25; /* sanity fallback */ + } + + dv_level = 0; + for (int i = 1; i < FF_ARRAY_ELEMS(dv_levels); i++) { + if (pps > dv_levels[i].pps) + continue; + if (avctx->width > dv_levels[i].width) + continue; + /* In theory, we should also test the bitrate when known, and + * distinguish between main and high tier. In practice, just ignore + * the bitrate constraints and hope they work out. This would ideally + * be handled by either the encoder or muxer directly. */ + dv_level = i; + break; + } + + if (!dv_level) { + if (avctx->strict_std_compliance >= FF_COMPLIANCE_STRICT) { + av_log(s->logctx, AV_LOG_ERROR, "Coded PPS (%"PRIu64") and width (%d) " + "exceed Dolby Vision limitations\n", pps, avctx->width); + return AVERROR(EINVAL); + } else { + av_log(s->logctx, AV_LOG_WARNING, "Coded PPS (%"PRIu64") and width (%d) " + "exceed Dolby Vision limitations. Ignoring, resulting file " + "may be non-conforming.\n", pps, avctx->width); + dv_level = FF_ARRAY_ELEMS(dv_levels) - 1; + } + } + + cfg = av_dovi_alloc(&cfg_size); + if (!cfg) + return AVERROR(ENOMEM); + + if (!av_packet_side_data_add(&avctx->coded_side_data, &avctx->nb_coded_side_data, + AV_PKT_DATA_DOVI_CONF, cfg, cfg_size, 0)) { + av_free(cfg); + return AVERROR(ENOMEM); + } + + cfg->dv_version_major = 1; + cfg->dv_version_minor = 0; + cfg->dv_profile = dv_profile; + cfg->dv_level = dv_level; + cfg->rpu_present_flag = 1; + cfg->el_present_flag = 0; + cfg->bl_present_flag = 1; + cfg->dv_bl_signal_compatibility_id = bl_compat_id; + + s->cfg = *cfg; + return 0; + +skip: + s->cfg = (AVDOVIDecoderConfigurationRecord) {0}; + return 0; +} + static inline uint64_t get_ue_coef(GetBitContext *gb, const AVDOVIRpuDataHeader *hdr) { uint64_t ipart; diff --git a/libavcodec/dovi_rpu.h b/libavcodec/dovi_rpu.h index 9a68e45bf1b..56395707369 100644 --- a/libavcodec/dovi_rpu.h +++ b/libavcodec/dovi_rpu.h @@ -26,14 +26,25 @@ #include "libavutil/dovi_meta.h" #include "libavutil/frame.h" +#include "avcodec.h" #define DOVI_MAX_DM_ID 15 typedef struct DOVIContext { void *logctx; + /** + * Enable tri-state. + * + * For encoding, FF_DOVI_AUTOMATIC enables Dolby Vision only if + * avctx->decoded_side_data contains an AVDOVIMetadata. + */ +#define FF_DOVI_AUTOMATIC -1 + int enable; + /** * Currently active dolby vision configuration, or {0} for none. - * Set by the user when decoding. + * Set by the user when decoding. Generated by ff_dovi_configure() + * when encoding. * * Note: sizeof(cfg) is not part of the libavutil ABI, so users should * never pass &cfg to any other library calls. This is included merely as @@ -96,4 +107,14 @@ int ff_dovi_rpu_parse(DOVIContext *s, const uint8_t *rpu, size_t rpu_size, */ int ff_dovi_attach_side_data(DOVIContext *s, AVFrame *frame); +/** + * Configure the encoder for Dolby Vision encoding. Generates a configuration + * record in s->cfg, and attaches it to avctx->coded_side_data. Sets the correct + * profile and compatibility ID based on the tagged AVCodecContext colorspace + * metadata, and the correct level based on the resolution and tagged framerate. + * + * Returns 0 or a negative error code. + */ +int ff_dovi_configure(DOVIContext *s, AVCodecContext *avctx); + #endif /* AVCODEC_DOVI_RPU_H */