From patchwork Mon Dec 20 13:44:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 32748 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp4370452iog; Mon, 20 Dec 2021 05:45:07 -0800 (PST) X-Google-Smtp-Source: ABdhPJwbq8fjJdhd1JLOrZPUjBKZGKeGzflDrQMqYRhW5ue0cBwrM77XR7VrLi4UsQz6iLpnZlxo X-Received: by 2002:a17:906:f43:: with SMTP id h3mr12823594ejj.414.1640007907685; Mon, 20 Dec 2021 05:45:07 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1640007907; cv=none; d=google.com; s=arc-20160816; b=Oh3esXNMaPIHxU93ImSGOysskzOjzEmLaIa+FwRH+AIA2GeZIvS43pZ8S3kEAw6oFi IJg+aBm189wi2apMq4ge35F/k63NxbRJgqWV7ET14pL6jhOlptcIyoa7LwzprAKizFfc YEQabrnE4bSFTF6UTm6FTsqrMl7Kg0nGu1hsI3/dbactH03B+wuQ+UFAUPQlX2a8xNan oNWLhqw1TX4wWkZCqk1A/GRxnrYwVvmAOrkYkeCLS086z8+gPsTsbLoTZ1ui4gpZ2my0 BNJk6XpjYNdRNLRgiDWQw2r67WjaKT45y1QUS/7NIc/qjt1hQXtOGtCKtwn7qMQCJGtW yWlg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=8pHeHvob9hvtetyHoTirdVuYhzUbdEF5x1Y+S/mg7e4=; b=ORbMxWiRg4XeNfnBS/NLwfQIEZ0Y6E5etd2NhfhSAhpsPey1Kgm3gLJzB2ODOvsxEP Hok0xMNg5uwdFcT05OpWRYu775B5h7XKEJQOCccEUCQJi5ULh/iQhiSXlzqAdRH0Eh8j 1+iv0bKbxTKn/EGZLn+V6ofawDqjYaJ58DyDyMpIggXWiyqXwg10nPhlrLJkG6gwJqfp jyNW0iXKinHPLPO1b5rEgrXgSYcSB2BTjhmWvTNNXobj7wLecq5xG7FHVTQxQNb0pZ9c p7C6/OONPvPAaFwxRI6a1s70xGRRkAO2BVUSz8L8KZKddMf8mX4ulG5UenxOCeqT0JRz hI1g== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=mynH8A7n; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id c13si11973404ede.539.2021.12.20.05.45.07; Mon, 20 Dec 2021 05:45:07 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=mynH8A7n; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C002068AE46; Mon, 20 Dec 2021 15:44:31 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id DCE0868A7C8 for ; Mon, 20 Dec 2021 15:44:22 +0200 (EET) Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id A212B49361; Mon, 20 Dec 2021 14:44:22 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1640007862; bh=isVqXu9lGYGjUTL2cNtagi5zSn/ba8KNoDmP79ysfVo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mynH8A7ntgPXNxdvb7Y+dzXmkZqDflnWhRZCytULzadqwKJoMjLN+HZzjzb05QV04 zpoCJIT3FPowIEtZ5V18adsLz7ZJxI7ROpYIyKxwqSAfyI96jMj92uPcNII0q0KwCH A7JZxHIolj1wc6Z4j8mk7u/mhowMDnXBVdMdnVpk= From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Mon, 20 Dec 2021 14:44:18 +0100 Message-Id: <20211220134420.69078-4-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20211220134420.69078-1-ffmpeg@haasn.xyz> References: <20211220134420.69078-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v7 4/6] lavc: Implement Dolby Vision RPU parsing X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 9QU1kPVDWary From: Niklas Haas Based on a mixture of guesswork, partial documentation in patents, and reverse engineering of real-world samples. Confirmed working for all the samples I've thrown at it. Contains some annoying machinery to persist these values in between frames, which is needed in theory even though I've never actually seen a sample that relies on it in practice. May or may not work. Since the distinction matters greatly for parsing the color matrix values, this includes a small helper function to guess the right profile from the RPU itself in case the user has forgotten to forward the dovi configuration record to the decoder. (Which in practice, only ffmpeg.c and ffplay do..) Notable omissions / deviations: - CRC32 verification. This is based on the MPEG2 CRC32 type, which does not seem to be implemented in lavu. (And I don't care enough to do so) - Linear interpolation support. Nothing documents this (beyond its existence) and no samples use it, so impossible to implement. - All of the extension metadata blocks, but these contain values that seem largely congruent with ST2094, HDR10, or other existing forms of side data, so I will defer parsing/attaching them to a future commit. - The patent describes a mechanism for predicting coefficients from previous RPUs, but the bit for the flag whether to use the prediction deltas or signal entirely new coefficients does not seem to be present in actual RPUs, so we ignore this subsystem entirely. - In the patent's spec, the NLQ subsystem also loops over num_nlq_pivots, but even in the patent the number is hard-coded to one iteration rather than signalled. So we only store one set of coefs. Heavily influenced by https://github.com/quietvoid/dovi_tool Documentation drawn from US Patent 10,701,399 B2 and ETSI GS CCM 001 Signed-off-by: Niklas Haas --- configure | 2 + libavcodec/Makefile | 1 + libavcodec/dovi_rpu.c | 430 ++++++++++++++++++++++++++++++++++++++++++ libavcodec/dovi_rpu.h | 71 +++++++ 4 files changed, 504 insertions(+) create mode 100644 libavcodec/dovi_rpu.c create mode 100644 libavcodec/dovi_rpu.h diff --git a/configure b/configure index d9d41b2273..9a27ff9bbc 100755 --- a/configure +++ b/configure @@ -2434,6 +2434,7 @@ CONFIG_EXTRA=" cbs_vp9 dirac_parse dnn + dovi_rpu dvprofile exif faandct @@ -2706,6 +2707,7 @@ cbs_mpeg2_select="cbs" cbs_vp9_select="cbs" dct_select="rdft" dirac_parse_select="golomb" +dovi_rpu_select="golomb" dnn_suggest="libtensorflow libopenvino" dnn_deps="avformat swscale" error_resilience_select="me_cmp" diff --git a/libavcodec/Makefile b/libavcodec/Makefile index fb90ecea84..7364c7a91f 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -77,6 +77,7 @@ OBJS-$(CONFIG_CBS_MPEG2) += cbs_mpeg2.o OBJS-$(CONFIG_CBS_VP9) += cbs_vp9.o OBJS-$(CONFIG_CRYSTALHD) += crystalhd.o OBJS-$(CONFIG_DCT) += dct.o dct32_fixed.o dct32_float.o +OBJS-$(CONFIG_DOVI_RPU) += dovi_rpu.o OBJS-$(CONFIG_ERROR_RESILIENCE) += error_resilience.o OBJS-$(CONFIG_EXIF) += exif.o tiff_common.o OBJS-$(CONFIG_FAANDCT) += faandct.o diff --git a/libavcodec/dovi_rpu.c b/libavcodec/dovi_rpu.c new file mode 100644 index 0000000000..1dcd0f06f7 --- /dev/null +++ b/libavcodec/dovi_rpu.c @@ -0,0 +1,430 @@ +/* + * Dolby Vision RPU decoder + * + * Copyright (C) 2021 Jan Ekström + * Copyright (C) 2021 Niklas Haas + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/buffer.h" + +#include "dovi_rpu.h" +#include "golomb.h" +#include "get_bits.h" + +enum { + RPU_COEFF_FIXED = 0, + RPU_COEFF_FLOAT = 1, +}; + +/** + * Private contents of vdr_ref. + */ +typedef struct DOVIVdrRef { + AVDOVIDataMapping mapping; + AVDOVIColorMetadata color; +} DOVIVdrRef; + +void ff_dovi_ctx_unref(DOVIContext *s) +{ + for (int i = 0; i < FF_ARRAY_ELEMS(s->vdr_ref); i++) + av_buffer_unref(&s->vdr_ref[i]); + + /* Preserve the user-provided fields explicitly, reset everything else */ + *s = (DOVIContext) { + .avctx = s->avctx, + .config = s->config, + }; +} + +int ff_dovi_ctx_replace(DOVIContext *s, const DOVIContext *s0) +{ + int ret; + s->avctx = s0->avctx; + s->config = s0->config; + s->mapping = s0->mapping; + s->color = s0->color; + for (int i = 0; i < DOVI_MAX_DM_ID; i++) { + if ((ret = av_buffer_replace(&s->vdr_ref[i], s0->vdr_ref[i])) < 0) + goto fail; + } + + return 0; + +fail: + ff_dovi_ctx_unref(s); + return ret; +} + +int ff_dovi_attach_side_data(DOVIContext *s, AVFrame *frame) +{ + AVFrameSideData *sd; + AVBufferRef *buf; + AVDOVIMetadata *dovi; + size_t dovi_size; + + if (!s->mapping || !s->color) + return 0; /* incomplete dovi metadata */ + + dovi = av_dovi_metadata_alloc(&dovi_size); + if (!dovi) + return AVERROR(ENOMEM); + + buf = av_buffer_create((uint8_t *) dovi, dovi_size, NULL, NULL, 0); + if (!buf) { + av_free(dovi); + return AVERROR(ENOMEM); + } + + sd = av_frame_new_side_data_from_buf(frame, AV_FRAME_DATA_DOVI_METADATA, buf); + if (!sd) { + av_buffer_unref(&buf); + return AVERROR(ENOMEM); + } + + memcpy(&dovi->header, &s->header, sizeof(dovi->header)); + memcpy(&dovi->mapping, s->mapping, sizeof(dovi->mapping)); + memcpy(&dovi->color, s->color, sizeof(dovi->color)); + return 0; +} + +static int guess_profile(const AVDOVIRpuDataHeader *hdr) +{ + switch (hdr->vdr_rpu_profile) { + case 0: + if (hdr->bl_video_full_range_flag) + return 5; + break; + case 1: + if (hdr->el_spatial_resampling_filter_flag && !hdr->disable_residual_flag) { + if (hdr->vdr_bit_depth == 12) { + return 7; + } else { + return 4; + } + } else { + return 8; + } + } + + return 0; /* unknown */ +} + +static inline uint64_t get_ue_coef(GetBitContext *gb, const AVDOVIRpuDataHeader *hdr) +{ + uint64_t ipart; + union { uint32_t u32; float f32; } fpart; + + switch (hdr->coef_data_type) { + case RPU_COEFF_FIXED: + ipart = get_ue_golomb_long(gb); + fpart.u32 = get_bits_long(gb, hdr->coef_log2_denom); + return (ipart << hdr->coef_log2_denom) + fpart.u32; + + case RPU_COEFF_FLOAT: + fpart.u32 = get_bits_long(gb, 32); + return fpart.f32 * (1 << hdr->coef_log2_denom); + } + + return 0; /* unreachable */ +} + +static inline int64_t get_se_coef(GetBitContext *gb, const AVDOVIRpuDataHeader *hdr) +{ + int64_t ipart; + union { uint32_t u32; float f32; } fpart; + + switch (hdr->coef_data_type) { + case RPU_COEFF_FIXED: + ipart = get_se_golomb_long(gb); + fpart.u32 = get_bits_long(gb, hdr->coef_log2_denom); + return (ipart << hdr->coef_log2_denom) + fpart.u32; + + case RPU_COEFF_FLOAT: + fpart.u32 = get_bits_long(gb, 32); + return fpart.f32 * (1 << hdr->coef_log2_denom); + } + + return 0; /* unreachable */ +} + +#define VALIDATE(VAR, MIN, MAX) \ + do { \ + if (VAR < MIN || VAR > MAX) { \ + av_log(s->avctx, AV_LOG_ERROR, "RPU validation failed: " \ + #MIN" <= "#VAR" = %d <= "#MAX"\n", (int) VAR); \ + goto fail; \ + } \ + } while (0) + +int ff_dovi_rpu_parse(DOVIContext *s, const uint8_t *rpu, size_t rpu_size) +{ + AVDOVIRpuDataHeader *hdr = &s->header; + GetBitContext *gb = &(GetBitContext){0}; + DOVIVdrRef *vdr; + int ret; + + uint8_t nal_prefix; + uint8_t rpu_type; + uint8_t vdr_seq_info_present; + uint8_t vdr_dm_metadata_present; + uint8_t use_prev_vdr_rpu; + uint8_t use_nlq; + uint8_t profile; + if ((ret = init_get_bits8(gb, rpu, rpu_size)) < 0) + return ret; + + /* RPU header, common values */ + nal_prefix = get_bits(gb, 8); + VALIDATE(nal_prefix, 25, 25); + rpu_type = get_bits(gb, 6); + if (rpu_type != 2) { + av_log(s->avctx, AV_LOG_WARNING, "Unrecognized RPU type " + "%"PRIu8", ignoring\n", rpu_type); + return 0; + } + + hdr->rpu_type = rpu_type; + hdr->rpu_format = get_bits(gb, 11); + + /* Values specific to RPU type 2 */ + hdr->vdr_rpu_profile = get_bits(gb, 4); + hdr->vdr_rpu_level = get_bits(gb, 4); + + vdr_seq_info_present = get_bits1(gb); + if (vdr_seq_info_present) { + hdr->chroma_resampling_explicit_filter_flag = get_bits1(gb); + hdr->coef_data_type = get_bits(gb, 2); + VALIDATE(hdr->coef_data_type, RPU_COEFF_FIXED, RPU_COEFF_FLOAT); + switch (hdr->coef_data_type) { + case RPU_COEFF_FIXED: + hdr->coef_log2_denom = get_ue_golomb(gb); + VALIDATE(hdr->coef_log2_denom, 13, 32); + break; + case RPU_COEFF_FLOAT: + hdr->coef_log2_denom = 32; /* arbitrary, choose maximum precision */ + break; + } + + hdr->vdr_rpu_normalized_idc = get_bits(gb, 2); + hdr->bl_video_full_range_flag = get_bits1(gb); + + if ((hdr->rpu_format & 0x700) == 0) { + int bl_bit_depth_minus8 = get_ue_golomb_31(gb); + int el_bit_depth_minus8 = get_ue_golomb_31(gb); + int vdr_bit_depth_minus8 = get_ue_golomb_31(gb); + VALIDATE(bl_bit_depth_minus8, 0, 8); + VALIDATE(el_bit_depth_minus8, 0, 8); + VALIDATE(vdr_bit_depth_minus8, 0, 8); + hdr->bl_bit_depth = bl_bit_depth_minus8 + 8; + hdr->el_bit_depth = el_bit_depth_minus8 + 8; + hdr->vdr_bit_depth = vdr_bit_depth_minus8 + 8; + hdr->spatial_resampling_filter_flag = get_bits1(gb); + skip_bits(gb, 3); /* reserved_zero_3bits */ + hdr->el_spatial_resampling_filter_flag = get_bits1(gb); + hdr->disable_residual_flag = get_bits1(gb); + } + } + + if (!hdr->bl_bit_depth) { + av_log(s->avctx, AV_LOG_ERROR, "Missing RPU VDR sequence info?\n"); + goto fail; + } + + vdr_dm_metadata_present = get_bits1(gb); + use_prev_vdr_rpu = get_bits1(gb); + use_nlq = (hdr->rpu_format & 0x700) == 0 && !hdr->disable_residual_flag; + + profile = s->config ? s->config->dv_profile : guess_profile(hdr); + if (profile == 5 && use_nlq) { + av_log(s->avctx, AV_LOG_ERROR, "Profile 5 RPUs should not use NLQ\n"); + goto fail; + } + + if (use_prev_vdr_rpu) { + int prev_vdr_rpu_id = get_ue_golomb_31(gb); + VALIDATE(prev_vdr_rpu_id, 0, DOVI_MAX_DM_ID); + if (!s->vdr_ref[prev_vdr_rpu_id]) { + av_log(s->avctx, AV_LOG_ERROR, "Unknown previous RPU ID: %u\n", + prev_vdr_rpu_id); + goto fail; + } + vdr = (DOVIVdrRef *) s->vdr_ref[prev_vdr_rpu_id]->data; + s->mapping = &vdr->mapping; + } else { + int vdr_rpu_id = get_ue_golomb_31(gb); + VALIDATE(vdr_rpu_id, 0, DOVI_MAX_DM_ID); + if (!s->vdr_ref[vdr_rpu_id]) { + s->vdr_ref[vdr_rpu_id] = av_buffer_allocz(sizeof(DOVIVdrRef)); + if (!s->vdr_ref[vdr_rpu_id]) + return AVERROR(ENOMEM); + } + + vdr = (DOVIVdrRef *) s->vdr_ref[vdr_rpu_id]->data; + s->mapping = &vdr->mapping; + + vdr->mapping.vdr_rpu_id = vdr_rpu_id; + vdr->mapping.mapping_color_space = get_ue_golomb_31(gb); + vdr->mapping.mapping_chroma_format_idc = get_ue_golomb_31(gb); + + for (int c = 0; c < 3; c++) { + AVDOVIReshapingCurve *curve = &vdr->mapping.curves[c]; + int num_pivots_minus_2 = get_ue_golomb_31(gb); + int pivot = 0; + + VALIDATE(num_pivots_minus_2, 0, AV_DOVI_MAX_PIECES - 1); + curve->num_pivots = num_pivots_minus_2 + 2; + for (int i = 0; i < curve->num_pivots; i++) { + pivot += get_bits(gb, hdr->bl_bit_depth); + curve->pivots[i] = av_clip_uint16(pivot); + } + } + + if (use_nlq) { + vdr->mapping.nlq_method_idc = get_bits(gb, 3); + /** + * The patent mentions another legal value, NLQ_MU_LAW, but it's + * not documented anywhere how to parse or apply that type of NLQ. + */ + VALIDATE(vdr->mapping.nlq_method_idc, 0, AV_DOVI_NLQ_LINEAR_DZ); + } else { + vdr->mapping.nlq_method_idc = AV_DOVI_NLQ_NONE; + } + + vdr->mapping.num_x_partitions = get_ue_golomb_long(gb) + 1; + vdr->mapping.num_y_partitions = get_ue_golomb_long(gb) + 1; + /* End of rpu_data_header(), start of vdr_rpu_data_payload() */ + + for (int c = 0; c < 3; c++) { + AVDOVIReshapingCurve *curve = &vdr->mapping.curves[c]; + for (int i = 0; i < curve->num_pivots - 1; i++) { + int mapping_idc = get_ue_golomb_31(gb); + VALIDATE(mapping_idc, 0, 1); + curve->mapping_idc[i] = mapping_idc; + switch (mapping_idc) { + case AV_DOVI_MAPPING_POLYNOMIAL: { + int poly_order_minus1 = get_ue_golomb_31(gb); + VALIDATE(poly_order_minus1, 0, 1); + curve->poly_order[i] = poly_order_minus1 + 1; + if (poly_order_minus1 == 0) { + int linear_interp_flag = get_bits1(gb); + if (linear_interp_flag) { + /* lack of documentation/samples */ + avpriv_request_sample(s->avctx, "Dolby Vision " + "linear interpolation"); + ff_dovi_ctx_unref(s); + return AVERROR_PATCHWELCOME; + } + } + for (int k = 0; k <= curve->poly_order[i]; k++) + curve->poly_coef[i][k] = get_se_coef(gb, hdr); + break; + } + case AV_DOVI_MAPPING_MMR: { + int mmr_order_minus1 = get_bits(gb, 2); + VALIDATE(mmr_order_minus1, 0, 2); + curve->mmr_order[i] = mmr_order_minus1 + 1; + curve->mmr_constant[i] = get_se_coef(gb, hdr); + for (int j = 0; j < curve->mmr_order[i]; j++) { + for (int k = 0; k < 7; k++) + curve->mmr_coef[i][j][k] = get_se_coef(gb, hdr); + } + break; + } + } + } + } + + if (use_nlq) { + for (int c = 0; c < 3; c++) { + AVDOVINLQParams *nlq = &vdr->mapping.nlq[c]; + nlq->nlq_offset = get_bits(gb, hdr->el_bit_depth); + nlq->vdr_in_max = get_ue_coef(gb, hdr); + switch (vdr->mapping.nlq_method_idc) { + case AV_DOVI_NLQ_LINEAR_DZ: + nlq->linear_deadzone_slope = get_ue_coef(gb, hdr); + nlq->linear_deadzone_threshold = get_ue_coef(gb, hdr); + break; + } + } + } + } + + if (vdr_dm_metadata_present) { + AVDOVIColorMetadata *color; + int affected_dm_id = get_ue_golomb_31(gb); + int current_dm_id = get_ue_golomb_31(gb); + VALIDATE(affected_dm_id, 0, DOVI_MAX_DM_ID); + VALIDATE(current_dm_id, 0, DOVI_MAX_DM_ID); + if (!s->vdr_ref[affected_dm_id]) { + s->vdr_ref[affected_dm_id] = av_buffer_allocz(sizeof(DOVIVdrRef)); + if (!s->vdr_ref[affected_dm_id]) + return AVERROR(ENOMEM); + } + + if (!s->vdr_ref[current_dm_id]) { + av_log(s->avctx, AV_LOG_ERROR, "Unknown previous RPU DM ID: %u\n", + current_dm_id); + goto fail; + } + + /* Update current pointer based on current_dm_id */ + vdr = (DOVIVdrRef *) s->vdr_ref[current_dm_id]->data; + s->color = &vdr->color; + + /* Update values of affected_dm_id */ + vdr = (DOVIVdrRef *) s->vdr_ref[affected_dm_id]->data; + color = &vdr->color; + color->dm_metadata_id = affected_dm_id; + color->scene_refresh_flag = get_ue_golomb_31(gb); + for (int i = 0; i < 9; i++) + color->ycc_to_rgb_matrix[i] = av_make_q(get_sbits(gb, 16), 1 << 13); + for (int i = 0; i < 3; i++) { + int denom = profile == 4 ? (1 << 30) : (1 << 28); + unsigned offset = get_bits_long(gb, 32); + if (offset > INT_MAX) { + /* Ensure the result fits inside AVRational */ + offset >>= 1; + denom >>= 1; + } + color->ycc_to_rgb_offset[i] = av_make_q(offset, denom); + } + for (int i = 0; i < 9; i++) + color->rgb_to_lms_matrix[i] = av_make_q(get_sbits(gb, 16), 1 << 14); + + color->signal_eotf = get_bits(gb, 16); + color->signal_eotf_param0 = get_bits(gb, 16); + color->signal_eotf_param1 = get_bits(gb, 16); + color->signal_eotf_param2 = get_bits_long(gb, 32); + color->signal_bit_depth = get_bits(gb, 5); + VALIDATE(color->signal_bit_depth, 8, 16); + color->signal_color_space = get_bits(gb, 2); + color->signal_chroma_format = get_bits(gb, 2); + color->signal_full_range_flag = get_bits(gb, 2); + color->source_min_pq = get_bits(gb, 12); + color->source_max_pq = get_bits(gb, 12); + color->source_diagonal = get_bits(gb, 10); + } + + /* FIXME: verify CRC32, requires implementation of AV_CRC_32_MPEG_2 */ + return 0; + +fail: + ff_dovi_ctx_unref(s); /* don't leak potentially invalid state */ + return AVERROR(EINVAL); +} diff --git a/libavcodec/dovi_rpu.h b/libavcodec/dovi_rpu.h new file mode 100644 index 0000000000..8cabff2718 --- /dev/null +++ b/libavcodec/dovi_rpu.h @@ -0,0 +1,71 @@ +/* + * Dolby Vision RPU decoder + * + * Copyright (C) 2021 Jan Ekström + * Copyright (C) 2021 Niklas Haas + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_DOVI_RPU_H +#define AVCODEC_DOVI_RPU_H + +#include "libavutil/dovi_meta.h" +#include "avcodec.h" + +#define DOVI_MAX_DM_ID 15 +typedef struct DOVIContext { + AVCodecContext *avctx; + const AVDOVIDecoderConfigurationRecord *config; ///< provided by API user + AVBufferRef *vdr_ref[DOVI_MAX_DM_ID+1]; ///< decoded VDR data mappings + + /** + * Currently active RPU data header, updates on every dovi_rpu_parse(). + */ + AVDOVIRpuDataHeader header; + + /** + * Currently active data mappings, or NULL. Points into memory owned by the + * corresponding rpu/vdr_ref, which becomes invalid on the next call to + * dovi_rpu_parse. + */ + const AVDOVIDataMapping *mapping; + const AVDOVIColorMetadata *color; +} DOVIContext; + +/** + * Completely resets the internal state of a DOVIContext, but explicitly + * preserves any user-provided fields. + */ +void ff_dovi_ctx_unref(DOVIContext *s); + +int ff_dovi_ctx_replace(DOVIContext *s, const DOVIContext *s0); + +/** + * Parse the contents of a Dovi RPU NAL and update the parsed values in the + * DOVIContext struct. + * + * Returns 0 or an error code. + */ +int ff_dovi_rpu_parse(DOVIContext *s, const uint8_t *rpu, size_t rpu_size); + +/** + * Attach the decoded AVDOVIMetadata as side data to an AVFrame. + */ +int ff_dovi_attach_side_data(DOVIContext *s, AVFrame *frame); + +#endif /* AVCODEC_DOVI_RPU_H */