From patchwork Tue Apr 9 12:57:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 47972 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:9c8d:b0:1a7:a0dc:8de5 with SMTP id mj13csp332377pzb; Tue, 9 Apr 2024 05:59:46 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCUIsICKOVlBd6b8ZJbCVfUg7YvT7OtJ/ks0Xi/VaxhXwrQZkGofwbrM5DchizeowyPiLjb4u2IXwcXqs7nGXOiZViQrN0RWJIElQg== X-Google-Smtp-Source: AGHT+IEIMZ0Lboa9FWSuOts0nYXvZZ4ZW/taV6Z/RW+ReXYuj3tX59mGIQw4K1uLkJnAA1YZSQtE X-Received: by 2002:a17:907:72c2:b0:a4e:4ed4:5efb with SMTP id du2-20020a17090772c200b00a4e4ed45efbmr9294576ejc.3.1712667585972; Tue, 09 Apr 2024 05:59:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1712667585; cv=none; d=google.com; s=arc-20160816; b=WqZQZvnSwxv4ceHMJCxHcceO6dxBjaJc2+gs+0MeuvwlnaFapzDKgzaRrWFwQJgZB8 hZNor96bOginOAJ+0JoVmYuiQN3Sf4gHIWk0F2y1rJfot/EHBfDHypBm1LCgEA/ZiI7R 0YMhGoeFNamXRNK0VcpcP1fUCKK9eroubG2VmICq97f8NuZK8s7zgy9sfblbWgD2lrBg 6BuHzIa5vwYOy9iVaIbEPxS8w4zpUR2UIPpsTEF+D4Q9zYsvMQ2PBrmQvbKdQFlBovoi RVQrrgnw1gd/1TrRHkdp6942GjFmqBtnZ5CptG4WOmQ6msxgu6bAfRaEgft93iq9hTQu YLbw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=Nr60sPLLocu31//z42vwgAM67pj4HnQsh2tlSwHFx/g=; fh=xmAeKtysnShNOmkhiJmYkS30uw4Fu2hvBJ7qlIwukxQ=; b=ysjXrcHeceeAapy5VE9DrZT2TSwe5WDKehKjahZ64Ogkm4CuQdnMrZrQ0qGuGUiPTf iwdMvhtPMBGSH4ZDynUPMQvTxZj5T8nvlS2bCjMcWWxREZDhL917VrkhAsEcM+c6nSWp ymRUNDZ7/xN6TXBFr6o3RSoQiP4eg0Kf6kBylbkvK1wXD9PTpFj4G7WtCauUgwwHOhdW bFX9td/iYRiyO3Tk6MS3bKtIru5LuDwMoFzwd+bw1c6WzIsv4QEg+Yv6aFA/5cW3QSzR 8PIydpNY+3gNBgMHak7c5BmMNR+rso8mcne6TQRBxHEpHx1p3LqZ9fszHv1kVLAoJBAt ggOQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b="F/BBfvzm"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id y21-20020a17090668d500b00a51d67119c5si1900481ejr.587.2024.04.09.05.59.45; Tue, 09 Apr 2024 05:59:45 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b="F/BBfvzm"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C830168D158; Tue, 9 Apr 2024 15:59:24 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id CB9F268D07E for ; Tue, 9 Apr 2024 15:59:16 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1712667556; bh=ijv0Xrlxsq9W+vPwTlMVfK1siwcChBltlO7GF/HS00Q=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=F/BBfvzmPqNbml8b3+53qske5dwvwYW2VhcNwT+ZuDkioBA0cWePP9WQ3xTV/CHZJ uY/BfT/1UEr+1o9mYSTbfKMUfjyDJ1iCt1u98pa6wz48S9N+MOKp+iwjDusW/mCQ0H qEE8OiT/YpdB234Gg7xaW8xCCXYn+idcQ5GQf/ic= Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id 9369540292; Tue, 9 Apr 2024 14:59:16 +0200 (CEST) From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Tue, 9 Apr 2024 14:57:21 +0200 Message-ID: <20240409125914.61149-2-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240409125914.61149-1-ffmpeg@haasn.xyz> References: <20240409125914.61149-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 01/11] avcodec/dovi_rpu: store entire config record X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: JlJ+BTUiSEyL From: Niklas Haas And make it public. For encoding, users may also be interested in the configured level and compatibility ID. So generalize the dv_profile field and just expose the whole configuration record. This makes the already rather reductive ff_dovi_update_cfg() function almost wholly redundant, since users can just directly assign DOVIContext.cfg. --- libavcodec/av1dec.c | 6 +++--- libavcodec/dovi_rpu.c | 16 ++++------------ libavcodec/dovi_rpu.h | 21 ++++++++++++--------- libavcodec/hevcdec.c | 13 ++++++------- libavcodec/libdav1d.c | 6 +++--- 5 files changed, 28 insertions(+), 34 deletions(-) diff --git a/libavcodec/av1dec.c b/libavcodec/av1dec.c index 824725c031f..4c1405df779 100644 --- a/libavcodec/av1dec.c +++ b/libavcodec/av1dec.c @@ -888,10 +888,10 @@ static av_cold int av1_decode_init(AVCodecContext *avctx) } s->dovi.logctx = avctx; - s->dovi.dv_profile = 10; // default for AV1 + s->dovi.cfg.dv_profile = 10; // default for AV1 sd = ff_get_coded_side_data(avctx, AV_PKT_DATA_DOVI_CONF); - if (sd && sd->size > 0) - ff_dovi_update_cfg(&s->dovi, (AVDOVIDecoderConfigurationRecord *) sd->data); + if (sd && sd->size >= sizeof(s->dovi.cfg)) + s->dovi.cfg = *(AVDOVIDecoderConfigurationRecord *) sd->data; return ret; } diff --git a/libavcodec/dovi_rpu.c b/libavcodec/dovi_rpu.c index 9f7a6b00664..d95c7e03af9 100644 --- a/libavcodec/dovi_rpu.c +++ b/libavcodec/dovi_rpu.c @@ -64,7 +64,7 @@ void ff_dovi_ctx_flush(DOVIContext *s) *s = (DOVIContext) { .logctx = s->logctx, - .dv_profile = s->dv_profile, + .cfg = s->cfg, /* preserve temporary buffer */ .rpu_buf = s->rpu_buf, .rpu_buf_sz = s->rpu_buf_sz, @@ -74,22 +74,14 @@ void ff_dovi_ctx_flush(DOVIContext *s) void ff_dovi_ctx_replace(DOVIContext *s, const DOVIContext *s0) { s->logctx = s0->logctx; + s->cfg = s0->cfg; s->mapping = s0->mapping; s->color = s0->color; - s->dv_profile = s0->dv_profile; for (int i = 0; i <= DOVI_MAX_DM_ID; i++) ff_refstruct_replace(&s->vdr[i], s0->vdr[i]); ff_refstruct_replace(&s->ext_blocks, s0->ext_blocks); } -void ff_dovi_update_cfg(DOVIContext *s, const AVDOVIDecoderConfigurationRecord *cfg) -{ - if (!cfg) - return; - - s->dv_profile = cfg->dv_profile; -} - int ff_dovi_attach_side_data(DOVIContext *s, AVFrame *frame) { AVFrameSideData *sd; @@ -392,7 +384,7 @@ int ff_dovi_rpu_parse(DOVIContext *s, const uint8_t *rpu, size_t rpu_size, goto fail; /* Container */ - if (s->dv_profile == 10 /* dav1.10 */) { + if (s->cfg.dv_profile == 10 /* dav1.10 */) { /* DV inside AV1 re-uses an EMDF container skeleton, but with fixed * values - so we can effectively treat this as a magic byte sequence. * @@ -517,7 +509,7 @@ int ff_dovi_rpu_parse(DOVIContext *s, const uint8_t *rpu, size_t rpu_size, use_prev_vdr_rpu = get_bits1(gb); use_nlq = (hdr->rpu_format & 0x700) == 0 && !hdr->disable_residual_flag; - profile = s->dv_profile ? s->dv_profile : guess_profile(hdr); + profile = s->cfg.dv_profile ? s->cfg.dv_profile : guess_profile(hdr); if (profile == 5 && use_nlq) { av_log(s->logctx, AV_LOG_ERROR, "Profile 5 RPUs should not use NLQ\n"); goto fail; diff --git a/libavcodec/dovi_rpu.h b/libavcodec/dovi_rpu.h index 9f26f332ceb..9a68e45bf1b 100644 --- a/libavcodec/dovi_rpu.h +++ b/libavcodec/dovi_rpu.h @@ -31,6 +31,16 @@ typedef struct DOVIContext { void *logctx; + /** + * Currently active dolby vision configuration, or {0} for none. + * Set by the user when decoding. + * + * Note: sizeof(cfg) is not part of the libavutil ABI, so users should + * never pass &cfg to any other library calls. This is included merely as + * a way to look up the values of fields known at compile time. + */ + AVDOVIDecoderConfigurationRecord cfg; + /** * Currently active RPU data header, updates on every dovi_rpu_parse(). */ @@ -56,7 +66,6 @@ typedef struct DOVIContext { struct DOVIVdr *vdr[DOVI_MAX_DM_ID+1]; ///< RefStruct references uint8_t *rpu_buf; ///< temporary buffer unsigned rpu_buf_sz; - uint8_t dv_profile; } DOVIContext; @@ -68,17 +77,11 @@ void ff_dovi_ctx_replace(DOVIContext *s, const DOVIContext *s0); void ff_dovi_ctx_unref(DOVIContext *s); /** - * Partially reset the internal state. Resets per-frame state while preserving - * fields parsed from the configuration record. + * Partially reset the internal state. Resets per-frame state, but preserves + * the stream-wide configuration record. */ void ff_dovi_ctx_flush(DOVIContext *s); -/** - * Read the contents of an AVDOVIDecoderConfigurationRecord (usually provided - * by stream side data) and update internal state accordingly. - */ -void ff_dovi_update_cfg(DOVIContext *s, const AVDOVIDecoderConfigurationRecord *cfg); - /** * Parse the contents of a Dovi RPU NAL and update the parsed values in the * DOVIContext struct. diff --git a/libavcodec/hevcdec.c b/libavcodec/hevcdec.c index 727b02f0f40..4bc9e2afc1d 100644 --- a/libavcodec/hevcdec.c +++ b/libavcodec/hevcdec.c @@ -3364,14 +3364,13 @@ static int hevc_decode_frame(AVCodecContext *avctx, AVFrame *rframe, } sd = av_packet_get_side_data(avpkt, AV_PKT_DATA_DOVI_CONF, &sd_size); - if (sd && sd_size > 0) { - int old = s->dovi_ctx.dv_profile; - - ff_dovi_update_cfg(&s->dovi_ctx, (AVDOVIDecoderConfigurationRecord *) sd); + if (sd && sd_size >= sizeof(s->dovi_ctx.cfg)) { + int old = s->dovi_ctx.cfg.dv_profile; + s->dovi_ctx.cfg = *(AVDOVIDecoderConfigurationRecord *) sd; if (old) av_log(avctx, AV_LOG_DEBUG, "New DOVI configuration record from input packet (profile %d -> %u).\n", - old, s->dovi_ctx.dv_profile); + old, s->dovi_ctx.cfg.dv_profile); } s->ref = s->collocated_ref = NULL; @@ -3661,8 +3660,8 @@ static av_cold int hevc_decode_init(AVCodecContext *avctx) } sd = ff_get_coded_side_data(avctx, AV_PKT_DATA_DOVI_CONF); - if (sd && sd->size > 0) - ff_dovi_update_cfg(&s->dovi_ctx, (AVDOVIDecoderConfigurationRecord *) sd->data); + if (sd && sd->size >= sizeof(s->dovi_ctx.cfg)) + s->dovi_ctx.cfg = *(AVDOVIDecoderConfigurationRecord *) sd->data; } return 0; diff --git a/libavcodec/libdav1d.c b/libavcodec/libdav1d.c index 93c77ed9c6c..09fe767fb86 100644 --- a/libavcodec/libdav1d.c +++ b/libavcodec/libdav1d.c @@ -290,10 +290,10 @@ static av_cold int libdav1d_init(AVCodecContext *c) #endif dav1d->dovi.logctx = c; - dav1d->dovi.dv_profile = 10; // default for AV1 + dav1d->dovi.cfg.dv_profile = 10; // default for AV1 sd = ff_get_coded_side_data(c, AV_PKT_DATA_DOVI_CONF); - if (sd && sd->size > 0) - ff_dovi_update_cfg(&dav1d->dovi, (AVDOVIDecoderConfigurationRecord *) sd->data); + if (sd && sd->size >= sizeof(dav1d->dovi.cfg)) + dav1d->dovi.cfg = *(AVDOVIDecoderConfigurationRecord *) sd->data; return 0; } From patchwork Tue Apr 9 12:57:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 47973 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:9c8d:b0:1a7:a0dc:8de5 with SMTP id mj13csp332464pzb; Tue, 9 Apr 2024 05:59:55 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCXnVgkaz1l0whbdhKEaHqIpDsrJBqr2TPXdNB2e+9T+4SGDhNqfiFyQVKKWcM4PW4CFNUoTwq30uiUqd3kFut5ypf051h3fgdhLdg== X-Google-Smtp-Source: AGHT+IEVVqkktLCUUFMXjM3xkNCk6J4GkGvITrbnJz2S9Yw3JDP+2aX+TkGYTOCfh3zzAmmwDjx4 X-Received: by 2002:a17:906:f806:b0:a51:a47a:cc37 with SMTP id kh6-20020a170906f80600b00a51a47acc37mr6893364ejb.63.1712667594979; Tue, 09 Apr 2024 05:59:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1712667594; cv=none; d=google.com; s=arc-20160816; b=LWzgtzAZ/qojktAwcsRezpcADLkbJ+EoMcWceZuIeK4CEE8n+rg7DratGP3c3NP56r p00kF0I3Qja3DXlM8LBEwBZIoJcbLbModO1zDGOx4zmRwxvey0MTaO6LNSMWVkPaoybi bresqGGaPpbDkV3rWJwct2i3la4DHTsMcUvFfL5t47Bk7PouJdQClXRcMpozMeH78VmK 2ewPQM8Y5u3PI0pt9GXPyY2+IuJH8MMv5yV88v+tm5gzRZtZp0f/9olJGBdH7rA3r8d7 5lbFS3d71+6n2AAzMcleQUiqI2otz6LS3jTOtEUEPgeg3fzyQPm/yuy9MsCanXo6xBf7 yBDg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=iAhI5avL1F9G8teZCv3TamvCUeLJpT9gm9m0E0r90Qs=; fh=xmAeKtysnShNOmkhiJmYkS30uw4Fu2hvBJ7qlIwukxQ=; b=Bjx79HBZ2uqgX6QZnkvcDbCKG8cji7T5O90FCP3YSAgiNl9Mju+xe8ZPnEIT/GEPfi wrFGK17zewLIeW3bnni0xrqM5S58gPyOtuxYxfuX0wdAGVEwdsB70ybDqoGCInbDNwtt Q2+d9/Z2XEAPi+6ybGHqSXqVv/pExAYDFx3mH3bJ7vm6YzprduZwARlvCxt5TFYPLaep yu6LWXtw5bgD/gGhShuXBHKLSF/7mI/utXmbznAqRcayxpw0TviL/M+zg9pRgfeBrTUI K8M7sFL4H9nfHoranZascFx78pD32R7v06iV1oQocO+Q2mJsf0FrJ2/cp2gr/Fkxz8vh ozfA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=KZhprsbt; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id jz2-20020a170906bb0200b00a51c53685c0si3020542ejb.240.2024.04.09.05.59.54; Tue, 09 Apr 2024 05:59:54 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=KZhprsbt; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C60F568D15B; Tue, 9 Apr 2024 15:59:25 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 208D868D07E for ; Tue, 9 Apr 2024 15:59:17 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1712667556; bh=yKfuFVTB9ej0iDXyPsTjxqOUjHXCIpAUSk2ratlfxyI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=KZhprsbtNh3ZToSF1FBGiQGi0Erdv+AheSjeLXphFG7ZsOlfutHe7TSW/ObFO/X+M q8SpMi9sRMTKQem1C+5zDPrTj9SP+4zQbLITqReW/a79IA8BDZ6mo49ygz6m+PvotZ eAmGk74NY+Jit4U5Guq8xgajR/pi6fYanvfm0EHs= Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id C4D4E424FA; Tue, 9 Apr 2024 14:59:16 +0200 (CEST) From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Tue, 9 Apr 2024 14:57:22 +0200 Message-ID: <20240409125914.61149-3-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240409125914.61149-1-ffmpeg@haasn.xyz> References: <20240409125914.61149-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 02/11] avcodec/dovi_rpu: properly replace context header X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: AHViixwZuNxn From: Niklas Haas This was never set in ff_dovi_ctx_replace(), leading to possibly out-of-date when copying from a sub-thread to the main thread. --- libavcodec/dovi_rpu.c | 1 + 1 file changed, 1 insertion(+) diff --git a/libavcodec/dovi_rpu.c b/libavcodec/dovi_rpu.c index d95c7e03af9..bfb7b9fe661 100644 --- a/libavcodec/dovi_rpu.c +++ b/libavcodec/dovi_rpu.c @@ -75,6 +75,7 @@ void ff_dovi_ctx_replace(DOVIContext *s, const DOVIContext *s0) { s->logctx = s0->logctx; s->cfg = s0->cfg; + s->header = s0->header; s->mapping = s0->mapping; s->color = s0->color; for (int i = 0; i <= DOVI_MAX_DM_ID; i++) From patchwork Tue Apr 9 12:57:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 47974 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:9c8d:b0:1a7:a0dc:8de5 with SMTP id mj13csp332591pzb; Tue, 9 Apr 2024 06:00:06 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCX3zDj49mFRB5MnPXyWDdt/8B0oxrVoJ06oVZB1GSnxDFchS45d/IBrH8hAnHX0MHST7gdn3A0MRZrAP+bgYSYWdW9fUD+IgMhyzg== X-Google-Smtp-Source: AGHT+IHvu8d0Mv00GwnIOqgahC1tQ42nOyM6B6UDFsQnQWkbNRqgoYWVE8qiG4jprC3xHpz7OWSJ X-Received: by 2002:a05:6512:2101:b0:513:cfb8:8cb3 with SMTP id q1-20020a056512210100b00513cfb88cb3mr6797711lfr.1.1712667606315; Tue, 09 Apr 2024 06:00:06 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1712667606; cv=none; d=google.com; s=arc-20160816; b=hQH1ixZuwR8yerhfHMWuqN240CqBbvJvqk9ZQgBmFG7ErlmzI0wpxjWARb5LVpy6cx 8fZKIW7a5H5zl0IPcAlBDvWbqw9i+ZLqY76GGWM9GOuELqqlc4kHp4AU+xUD1iq7YMwY CSMUsWoPBlIj2WqYNXePphjcuTRxg1rawFPyXhKN44yO8zQJFLZ4zj2y8mszPRrz+w/1 RuFm4x5L/vNeBeUM2eRE/YJ8NwGPi27rOYlwgRDKq+KJTKRH8p4oWqeTvPJCrh339ZrG 5vPWDjFIFVv28vOQX/QPPOZlfiz2IZOp68dRewTMPJZpx/skMbyF9aGEB9Q2UMKe5ICJ XuUA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=EwmR47Yzu2TWBAIufKGL6iVR4ED040XE9w6ABZsoUL8=; fh=xmAeKtysnShNOmkhiJmYkS30uw4Fu2hvBJ7qlIwukxQ=; b=y9MDyRxtlrHMUEQVVWCAsV97y4Fcm/DUMHDTHMx6sCCUF25jf7pHSGOmrK7y31xaKr mPvXdogzq00MTe5oaP9wkJctGmzYCEbgCeR56S99bTKCt+muuE4zr+/QaVo40f84Ffzb qF386Ia//Vw9Kjm8k+t8yJn3q5UhJvQT2C/vT8QgaftXQpr1pTOngN+19jpuYW5022gW +JXxcCZ7tkQJqkIihsncDJXCHHudxDhsCaTwVcsKukG5I42mMHA9AVQrhCkX2setsjwv U12I2PcYDZi8c2836yJOBUoqkG+j/Y2x6FCE77PE+HmcKZTdYdOBfiZ69QiMkDMMu4vc BamQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=jmlFOEuF; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id hv8-20020a17090760c800b00a51ab9a254dsi4475253ejc.201.2024.04.09.06.00.04; Tue, 09 Apr 2024 06:00:06 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=jmlFOEuF; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D39DA68D171; Tue, 9 Apr 2024 15:59:26 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4720A68D07E for ; Tue, 9 Apr 2024 15:59:17 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1712667557; bh=7d78h44qfomf2wASovFHqpGHzlu1LfrAEiKMdkrspj0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jmlFOEuFhTUOrEZ0EOCnPUIl0ACfhl6mihLCHOd3/HnW9CL8MiE7eR66J2lGzDd97 rspcH5OXV10HDEipjp7N8f5T3dLPaeNm5v3Q2tCEM7iDguUURnIuHvEPrmSZGslZy5 jhw7ftwHMwVAAzO8ArGBO/FKRF0yrxoOQJqmIcYw= Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id 0F55842500; Tue, 9 Apr 2024 14:59:17 +0200 (CEST) From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Tue, 9 Apr 2024 14:57:23 +0200 Message-ID: <20240409125914.61149-4-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240409125914.61149-1-ffmpeg@haasn.xyz> References: <20240409125914.61149-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 03/11] avcodec/dovi_rpu: clarify error on missing RPU VDR X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: L/DJA+eWOyKu From: Niklas Haas The code was written under the misguided assumption that these fields would only be present when the value changes, however this does not match the actual patent specification, which says that streams are required to either always signal this metadata, or never signal it. That said, the specification does not really clarify what the defaults of these fields should be in the event that this metadata is missing, so without any sample file or other reference I don't wish to hazard a guess at how to interpret these fields. Fix the current behavior by making sure we always throw this error, even for files that have the vdr sequence info in one frame but are missing it in the next frame. --- libavcodec/dovi_rpu.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/libavcodec/dovi_rpu.c b/libavcodec/dovi_rpu.c index bfb7b9fe661..267e52ceb66 100644 --- a/libavcodec/dovi_rpu.c +++ b/libavcodec/dovi_rpu.c @@ -499,11 +499,11 @@ int ff_dovi_rpu_parse(DOVIContext *s, const uint8_t *rpu, size_t rpu_size, hdr->el_spatial_resampling_filter_flag = get_bits1(gb); hdr->disable_residual_flag = get_bits1(gb); } - } - - if (!hdr->bl_bit_depth) { - av_log(s->logctx, AV_LOG_ERROR, "Missing RPU VDR sequence info?\n"); - goto fail; + } else { + /* lack of documentation/samples */ + avpriv_request_sample(s->logctx, "Missing RPU VDR sequence info\n"); + ff_dovi_ctx_unref(s); + return AVERROR_PATCHWELCOME; } vdr_dm_metadata_present = get_bits1(gb); From patchwork Tue Apr 9 12:57:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 47975 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:9c8d:b0:1a7:a0dc:8de5 with SMTP id mj13csp332721pzb; Tue, 9 Apr 2024 06:00:17 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCU0OUnjCJAaGU5/K8ABxVdg7qPlSYKE+D4ZkOMYs8QGMEjuYBxmUIzQwbDco5XtSCQwjTkwyM4zRUMMRQ9wPkw0dc9imGSj1nYDIA== X-Google-Smtp-Source: AGHT+IF6LJzFNNLtWT7KCHs4g9ncJaRQ/8L4wEkFpN8r9Q6d/xyVTRFepiHgp0LwVZqVvIQWUe96 X-Received: by 2002:ac2:4207:0:b0:515:cedb:a518 with SMTP id y7-20020ac24207000000b00515cedba518mr7158312lfh.16.1712667616839; Tue, 09 Apr 2024 06:00:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1712667616; cv=none; d=google.com; s=arc-20160816; b=jk36lWR1DOaG7dqKN83SOpxwB35ipdfbfR4mobAQEROfuH8kTsv5gRA/+nONZX2akR KFEgt1wKBINn1PuTHeAUmjFTibRUSSA/x3jSW/hVHc3Bza2r/FHkHtCclMzASrU/4ilD vQA2j+wSQCe2h/mB7L+An3h0a9avn3V+9YUQWZIm5/pixMZVjOys1uIXVGlBDeKb0AUz FOzRIFUrxos9LInMCE/eYS4UBB6SbIbmYlz5QhDu3krGB/UrtbwXnTbf59HLhGlnXNB/ K72EUo1CIng+np8NQZGxjRo/NgS4m32vSs/vD9Qs/X566/nsZZjPT3UsaNUkRvX87ZJh UGMg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=0kqXGZikehXhcIo84km/gbpgOMmsOIIFzF6X8yD/j5w=; fh=xmAeKtysnShNOmkhiJmYkS30uw4Fu2hvBJ7qlIwukxQ=; b=kLlexJGEAnpFcMRYomL4EP4wZfP5Uubf8oiNTT8Sy+03U9PRAXIKOcADizFLnP6oEP ObykXZuE+Bdmh1ZMgEo7AUyi+S4gPDACiefGAxAkYPZLzmVQ0xNmuFSQDlGpjf6YsjXF jbxJAzLl871PfJ7XeEAclLg/jhyxzHIB8Gd/OcTePJLgW0cBdBCPJBWAZd4SDDDywF4T HMu9clZ0R7ZHEsh+39Fpk1ZJS3loCD08EaGSO77OtMRUGUOyGaoFujvcz7646NWUEsjV RKXug+fhf1fMhr7GBMf1Ac6cQHXDFhmySxHuClN5fFGMaF8WWka9/MI8G0oQ/5kRrEUA j6Zg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=Ra9ziqgl; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id i23-20020a508717000000b0056e0d2f4db1si4745005edb.324.2024.04.09.06.00.15; Tue, 09 Apr 2024 06:00:16 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=Ra9ziqgl; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id CC15268D1AC; Tue, 9 Apr 2024 15:59:27 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7622568D11E for ; Tue, 9 Apr 2024 15:59:17 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1712667557; bh=Sjb4FWSjYqvNI8yanOsl3A5OdmxY1nscObMNBeLKAXg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Ra9ziqgldCHjve+sxYfewPCIM9uGc3f00huEzP6qXXH8YrV8P8mYXef7DNYxko1OS xQhM+1sTfQb6m9bcqF+3FCfpk66GiFhS/XJ6LonDz8PbJNKGduqN1gK32dTfV+PXah 7KGrb+Tpjc0JXAJP6Vp5Rh1MAW2U29xettMTGNx8= Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id 40ECA42752; Tue, 9 Apr 2024 14:59:17 +0200 (CEST) From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Tue, 9 Apr 2024 14:57:24 +0200 Message-ID: <20240409125914.61149-5-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240409125914.61149-1-ffmpeg@haasn.xyz> References: <20240409125914.61149-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 04/11] avcodec/dovi_rpu: clarify semantics of guess_profile() X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: QL1Eun/nyWIF From: Niklas Haas This is based on HEVC only, H.264/AV1 use their own (hopefully correctly signalled) profiles. --- libavcodec/dovi_rpu.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/libavcodec/dovi_rpu.c b/libavcodec/dovi_rpu.c index 267e52ceb66..4da711d763e 100644 --- a/libavcodec/dovi_rpu.c +++ b/libavcodec/dovi_rpu.c @@ -121,7 +121,8 @@ int ff_dovi_attach_side_data(DOVIContext *s, AVFrame *frame) return 0; } -static int guess_profile(const AVDOVIRpuDataHeader *hdr) +/* Note: Only works for HEVC */ +static int guess_hevc_profile(const AVDOVIRpuDataHeader *hdr) { switch (hdr->vdr_rpu_profile) { case 0: @@ -510,7 +511,7 @@ int ff_dovi_rpu_parse(DOVIContext *s, const uint8_t *rpu, size_t rpu_size, use_prev_vdr_rpu = get_bits1(gb); use_nlq = (hdr->rpu_format & 0x700) == 0 && !hdr->disable_residual_flag; - profile = s->cfg.dv_profile ? s->cfg.dv_profile : guess_profile(hdr); + profile = s->cfg.dv_profile ? s->cfg.dv_profile : guess_hevc_profile(hdr); if (profile == 5 && use_nlq) { av_log(s->logctx, AV_LOG_ERROR, "Profile 5 RPUs should not use NLQ\n"); goto fail; From patchwork Tue Apr 9 12:57:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 47978 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:9c8d:b0:1a7:a0dc:8de5 with SMTP id mj13csp333267pzb; Tue, 9 Apr 2024 06:00:53 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCWaQGRAiLSjBZHxwqsGlXqT8Lhvj60h01yeauzJH6/Lhi/w5/DKNgx9Sg/0otFZmZubLsepo9kcxPFOknaR1U/5LCKBdy3vpOjtUg== X-Google-Smtp-Source: AGHT+IHHox/cA+GlikEQicldqPkqVCRLBKPpezaIMn2DCyJ9huCO7R7mPPzGLEv6TlX/L/KTwV2l X-Received: by 2002:a2e:3807:0:b0:2d2:246e:b373 with SMTP id f7-20020a2e3807000000b002d2246eb373mr7924487lja.5.1712667653204; Tue, 09 Apr 2024 06:00:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1712667653; cv=none; d=google.com; s=arc-20160816; b=fgtnqeVAtEOM8kM+H6ceWPrJQP0ZFu6BDGToKLnEj66Ova2wPEk6oAPtmREJ0JNoCq 2KJfEuduXqXzq3Eewm2Ap0Ta0z+UB6xsUBYJLfDK+9cns+9UNdupBljdxMgLjAIQ/bGE BVSHTtcShcKXbRidN0vxj7zZIw0jp9/kxOXMoPFX4iaQdleQOYqE89y7H6JbmEYj/YFf 54Np3gIZeSAksqzYpfT2S2/Ld8WFzRD7E1Mdi65hz+Uo2ljDpThsI8kq6U1XzklccNE2 EvUoOQ9xgG8wy9qEtPX8qddKkH2g5Q/n9A4+d1YpigPla+I78dZQe8wlcikMFzLTnETO c9JA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=poGcRvAuza6uY3Ra7wN6Gn3nwTFAmHVjDHlwb7PRCAY=; fh=xmAeKtysnShNOmkhiJmYkS30uw4Fu2hvBJ7qlIwukxQ=; b=WWdXZ/X/wnzMGiesC70imbpOf+7K0AhulP9U5aFEG46J2uGzaIgddt1Z/f4Iw5l9Qp 13fgMxpEtbvP/U0Sb5Iq1Dci5mhnDZFOL89QNEI+BKBiKSnWofr68wgwO3uOjQISxSDd 4iXZonxqbo2D9aAp8xjdgqDe6wwWtHBgdxOxSHQ6nbUgFdCM2T2YGrcg1DJ+bNR8hoEd EsrtQqS2NsKYRQSmlIwAxVsIGfwAKyyaAPuOPEDMzBbsmB96+KXgxUjEvE/Wd8tr+B97 2jDXGgSntLGywyHNepmizTmyjMu0mFD3QWzp1SlxZrfcurHe78Jy3QF96U+NN3W0+wol 9SQA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=XMH1KtpP; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id x20-20020a2e8814000000b002d8408c3f63si2761635ljh.226.2024.04.09.06.00.52; Tue, 09 Apr 2024 06:00:53 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=XMH1KtpP; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0D6DA68CD9D; Tue, 9 Apr 2024 15:59:31 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id DF33568D188 for ; Tue, 9 Apr 2024 15:59:21 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1712667557; bh=2HTTf6dYj9S+6fzcjjxcmVf2ux8oKFJ37FyeDSh+HSQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XMH1KtpPwc7q1JwvC5LG2b5VUQUld56kTuyU5SuynV6hoU2CRsjpxH0RbyXqZ+/va RaXJ09EtFbOnvUAmVAaN7fCbTBzyWR0huPgd8wILk3kbiJYyHb27ZJ8Mvlm6Lvu9Dj fXyl6XePVb7EP/PaN2qNDREloImWHfOM6YTOoUTA= Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id 774D342765; Tue, 9 Apr 2024 14:59:17 +0200 (CEST) From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Tue, 9 Apr 2024 14:57:25 +0200 Message-ID: <20240409125914.61149-6-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240409125914.61149-1-ffmpeg@haasn.xyz> References: <20240409125914.61149-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 05/11] avcodec/dovi_rpu: add ff_dovi_configure() X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: wH3jvD40qvtU From: Niklas Haas We need to set up the configuration struct appropriately based on the codec type, colorspace metadata, and presence/absence of an EL (though, we currently don't support an EL). When present, we use the signalled RPU data header to help infer (and validate) the right values. Behavior can be controlled by a new DOVIContext.enable flag. --- libavcodec/dovi_rpu.c | 176 ++++++++++++++++++++++++++++++++++++++++++ libavcodec/dovi_rpu.h | 23 +++++- 2 files changed, 198 insertions(+), 1 deletion(-) diff --git a/libavcodec/dovi_rpu.c b/libavcodec/dovi_rpu.c index 4da711d763e..d3a284c150d 100644 --- a/libavcodec/dovi_rpu.c +++ b/libavcodec/dovi_rpu.c @@ -144,6 +144,182 @@ static int guess_hevc_profile(const AVDOVIRpuDataHeader *hdr) return 0; /* unknown */ } +static struct { + uint64_t pps; // maximum pixels per second + int width; // maximum width + int main; // maximum bitrate in main tier + int high; // maximum bitrate in high tier +} dv_levels[] = { + [1] = {1280*720*24, 1280, 20, 50}, + [2] = {1280*720*30, 1280, 20, 50}, + [3] = {1920*1080*24, 1920, 20, 70}, + [4] = {1920*1080*30, 2560, 20, 70}, + [5] = {1920*1080*60, 3840, 20, 70}, + [6] = {3840*2160*24, 3840, 25, 130}, + [7] = {3840*2160*30, 3840, 25, 130}, + [8] = {3840*2160*48, 3840, 40, 130}, + [9] = {3840*2160*60, 3840, 40, 130}, + [10] = {3840*2160*120, 3840, 60, 240}, + [11] = {3840*2160*120, 7680, 60, 240}, + [12] = {7680*4320*60, 7680, 120, 450}, + [13] = {7680*4320*120u, 7680, 240, 800}, +}; + +int ff_dovi_configure(DOVIContext *s, AVCodecContext *avctx) +{ + AVDOVIDecoderConfigurationRecord *cfg; + const AVDOVIRpuDataHeader *hdr = NULL; + const AVFrameSideData *sd; + int dv_profile, dv_level, bl_compat_id; + size_t cfg_size; + uint64_t pps; + + if (!s->enable) + goto skip; + + sd = av_frame_side_data_get(avctx->decoded_side_data, + avctx->nb_decoded_side_data, AV_FRAME_DATA_DOVI_METADATA); + + if (sd) + hdr = av_dovi_get_header((const AVDOVIMetadata *) sd->data); + + if (s->enable == FF_DOVI_AUTOMATIC && !hdr) + goto skip; + + switch (avctx->codec_id) { + case AV_CODEC_ID_AV1: dv_profile = 10; break; + case AV_CODEC_ID_H264: dv_profile = 9; break; + case AV_CODEC_ID_HEVC: dv_profile = hdr ? guess_hevc_profile(hdr) : 8; break; + default: + /* No other encoder should be calling this! */ + av_assert0(0); + return AVERROR_BUG; + } + + if (avctx->strict_std_compliance > FF_COMPLIANCE_UNOFFICIAL) { + if (dv_profile == 9) { + if (avctx->pix_fmt != AV_PIX_FMT_YUV420P) + dv_profile = 0; + } else { + if (avctx->pix_fmt != AV_PIX_FMT_YUV420P10) + dv_profile = 0; + } + } + + switch (dv_profile) { + case 0: /* None */ + bl_compat_id = -1; + break; + case 4: /* HEVC with enhancement layer */ + case 7: + if (s->enable > 0) { + av_log(s->logctx, AV_LOG_ERROR, "Coding of Dolby Vision enhancement " + "layers is currently unsupported."); + return AVERROR_PATCHWELCOME; + } else { + goto skip; + } + case 5: /* HEVC with proprietary IPTPQc2 */ + bl_compat_id = 0; + break; + case 10: + /* FIXME: check for proper H.273 tags once those are added */ + if (hdr && hdr->bl_video_full_range_flag) { + /* AV1 with proprietary IPTPQc2 */ + bl_compat_id = 0; + break; + } + /* fall through */ + case 8: /* HEVC (or AV1) with BL compatibility */ + if (avctx->colorspace == AVCOL_SPC_BT2020_NCL && + avctx->color_primaries == AVCOL_PRI_BT2020 && + avctx->color_trc == AVCOL_TRC_SMPTE2084) { + bl_compat_id = 1; + } else if (avctx->colorspace == AVCOL_SPC_BT2020_NCL && + avctx->color_primaries == AVCOL_PRI_BT2020 && + avctx->color_trc == AVCOL_TRC_ARIB_STD_B67) { + bl_compat_id = 4; + } else if (avctx->colorspace == AVCOL_SPC_BT709 && + avctx->color_primaries == AVCOL_PRI_BT709 && + avctx->color_trc == AVCOL_TRC_BT709) { + bl_compat_id = 2; + } else { + /* Not a valid colorspace combination */ + bl_compat_id = -1; + } + } + + if (!dv_profile || bl_compat_id < 0) { + if (s->enable > 0) { + av_log(s->logctx, AV_LOG_ERROR, "Dolby Vision enabled, but could " + "not determine profile and compaatibility mode. Double-check " + "colorspace and format settings for compatibility?\n"); + return AVERROR(EINVAL); + } + goto skip; + } + + pps = avctx->width * avctx->height; + if (avctx->framerate.num) { + pps = pps * avctx->framerate.num / avctx->framerate.den; + } else { + pps *= 25; /* sanity fallback */ + } + + dv_level = 0; + for (int i = 1; i < FF_ARRAY_ELEMS(dv_levels); i++) { + if (pps > dv_levels[i].pps) + continue; + if (avctx->width > dv_levels[i].width) + continue; + /* In theory, we should also test the bitrate when known, and + * distinguish between main and high tier. In practice, just ignore + * the bitrate constraints and hope they work out. This would ideally + * be handled by either the encoder or muxer directly. */ + dv_level = i; + break; + } + + if (!dv_level) { + if (avctx->strict_std_compliance >= FF_COMPLIANCE_STRICT) { + av_log(s->logctx, AV_LOG_ERROR, "Coded PPS (%"PRIu64") and width (%d) " + "exceed Dolby Vision limitations\n", pps, avctx->width); + return AVERROR(EINVAL); + } else { + av_log(s->logctx, AV_LOG_WARNING, "Coded PPS (%"PRIu64") and width (%d) " + "exceed Dolby Vision limitations. Ignoring, resulting file " + "may be non-conforming.\n", pps, avctx->width); + dv_level = FF_ARRAY_ELEMS(dv_levels) - 1; + } + } + + cfg = av_dovi_alloc(&cfg_size); + if (!cfg) + return AVERROR(ENOMEM); + + if (!av_packet_side_data_add(&avctx->coded_side_data, &avctx->nb_coded_side_data, + AV_PKT_DATA_DOVI_CONF, cfg, cfg_size, 0)) { + av_free(cfg); + return AVERROR(ENOMEM); + } + + cfg->dv_version_major = 1; + cfg->dv_version_minor = 0; + cfg->dv_profile = dv_profile; + cfg->dv_level = dv_level; + cfg->rpu_present_flag = 1; + cfg->el_present_flag = 0; + cfg->bl_present_flag = 1; + cfg->dv_bl_signal_compatibility_id = bl_compat_id; + + s->cfg = *cfg; + return 0; + +skip: + s->cfg = (AVDOVIDecoderConfigurationRecord) {0}; + return 0; +} + static inline uint64_t get_ue_coef(GetBitContext *gb, const AVDOVIRpuDataHeader *hdr) { uint64_t ipart; diff --git a/libavcodec/dovi_rpu.h b/libavcodec/dovi_rpu.h index 9a68e45bf1b..56395707369 100644 --- a/libavcodec/dovi_rpu.h +++ b/libavcodec/dovi_rpu.h @@ -26,14 +26,25 @@ #include "libavutil/dovi_meta.h" #include "libavutil/frame.h" +#include "avcodec.h" #define DOVI_MAX_DM_ID 15 typedef struct DOVIContext { void *logctx; + /** + * Enable tri-state. + * + * For encoding, FF_DOVI_AUTOMATIC enables Dolby Vision only if + * avctx->decoded_side_data contains an AVDOVIMetadata. + */ +#define FF_DOVI_AUTOMATIC -1 + int enable; + /** * Currently active dolby vision configuration, or {0} for none. - * Set by the user when decoding. + * Set by the user when decoding. Generated by ff_dovi_configure() + * when encoding. * * Note: sizeof(cfg) is not part of the libavutil ABI, so users should * never pass &cfg to any other library calls. This is included merely as @@ -96,4 +107,14 @@ int ff_dovi_rpu_parse(DOVIContext *s, const uint8_t *rpu, size_t rpu_size, */ int ff_dovi_attach_side_data(DOVIContext *s, AVFrame *frame); +/** + * Configure the encoder for Dolby Vision encoding. Generates a configuration + * record in s->cfg, and attaches it to avctx->coded_side_data. Sets the correct + * profile and compatibility ID based on the tagged AVCodecContext colorspace + * metadata, and the correct level based on the resolution and tagged framerate. + * + * Returns 0 or a negative error code. + */ +int ff_dovi_configure(DOVIContext *s, AVCodecContext *avctx); + #endif /* AVCODEC_DOVI_RPU_H */ From patchwork Tue Apr 9 12:57:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 47977 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:9c8d:b0:1a7:a0dc:8de5 with SMTP id mj13csp333139pzb; Tue, 9 Apr 2024 06:00:44 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCUlN7FlJsl7JD3oWaulD3bU5y4z8jfeDSXwRHZGmX0LZyeRPbzwJ1kjpAZhpKAs8sdmy3ijtIHjdIqSTeJjaY83FxPjW2NaSjIQ8A== X-Google-Smtp-Source: AGHT+IHse/36bj5jzIxnl63l8rVUQeLHaDrAvLzkSNfAgLCJLcvI/7L62QGv5RYGN5eIcq5qA9ZM X-Received: by 2002:a17:906:4144:b0:a51:ce1f:21b2 with SMTP id l4-20020a170906414400b00a51ce1f21b2mr4158484ejk.6.1712667644514; Tue, 09 Apr 2024 06:00:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1712667644; cv=none; d=google.com; s=arc-20160816; b=xshR7ChQLIavj4aSogf68V+BBOyT4sR82lHA9PwOG9ol0TQilDAE2AHnxan5V1gADd 2AaPfjFhNt5tXsY/KLwke6+H61B3c6sl/Dx4vlNia6wCKmAWqaOazoayq5oTKRPOJfGz 9FS7/4MEAr/+4OQeOeKcjZeeL3hOwBnfDfoGMltzF84kigoueoN5ion5Zmn2jvUmWG+n PHJ1gfB61qKapCSB9TwVmE1PWDY7zhIKtI0trKoB96jNcflUL1eAR1q5fGF8DRDK+3ur oVRJhBqTuftSpc7FPDhHp2KT6dWHyWOZlu4OiXYZQlO5ysjpaan5waD9Nm1Zw4vZhR3r XrBw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=S1Z3j8+4vCqAAxa9Xnrjn/etMC6HQrDrcMqK6pZ17gU=; fh=xmAeKtysnShNOmkhiJmYkS30uw4Fu2hvBJ7qlIwukxQ=; b=fi+rsb4LSpNjajVOPcWTHBin5w2aCyWdJ+iybrIdhEVqlnUq/l3gSM72InuwjTbBR/ nBr3CyBa/ZfhmQlFU+3JQRB1MfBMc+K8gtcCDvip4erGtyPK9nrQ1gz9BV/W91lRNBeC GcucRx/OaueHYHUu/A4ywtJ9LKv24RBXWHjmqUEcOhr8syIjuArxjO0N9vq/Ip6/jEkR ATY0JAY8Dlnb9Tu0tg0p3d0cvfa0t0dVENaMkUbHSc9nI6t81DMuOjDqj7+h+Hjw3PLE VdI/oc2PW9Rgm+RlXHe+kjGWJZ9rfQoZ9QzKS4QKGK22hXPusKYRFkuqUSAXagFbudb1 I/Yg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=VVg9sxUR; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id qk20-20020a1709077f9400b00a46a9425fd0si4696862ejc.627.2024.04.09.06.00.38; Tue, 09 Apr 2024 06:00:44 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=VVg9sxUR; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0F6D168D18D; Tue, 9 Apr 2024 15:59:30 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D788568D180 for ; Tue, 9 Apr 2024 15:59:21 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1712667557; bh=rNULlvuFK9xeMtZfgFTffJsJSf/9BVVZ2dJqJEUvJas=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VVg9sxURIcPu+7LnG141Zt5YFGTDRLiJjCAdSbwQDQ4CCCE58EDadBthKdw7mrmBT pLE+f9ORk72h5UGMkod4in5EokQK82IBZ7YaTwMWFjg0XSXRtQvl+BL0MZEc2GOyjz cHnMBSmr4Mf/8ITdy//KdkdsxyIDs3DC0IkHsMp8= Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id AEA1142B27; Tue, 9 Apr 2024 14:59:17 +0200 (CEST) From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Tue, 9 Apr 2024 14:57:26 +0200 Message-ID: <20240409125914.61149-7-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240409125914.61149-1-ffmpeg@haasn.xyz> References: <20240409125914.61149-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 06/11] avcodec/dovi_rpu: make `enable` also affect decoding X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: zLwPZ4SxU7IP From: Niklas Haas This could be used by codecs to selectively disable parsing Dolby Vision RPUs, and is cheap to support. --- libavcodec/av1dec.c | 1 + libavcodec/dovi_rpu.c | 6 ++++++ libavcodec/dovi_rpu.h | 2 ++ libavcodec/hevcdec.c | 1 + libavcodec/libdav1d.c | 1 + 5 files changed, 11 insertions(+) diff --git a/libavcodec/av1dec.c b/libavcodec/av1dec.c index 4c1405df779..20865b4f129 100644 --- a/libavcodec/av1dec.c +++ b/libavcodec/av1dec.c @@ -1551,6 +1551,7 @@ static void av1_decode_flush(AVCodecContext *avctx) static const AVOption av1_options[] = { { "operating_point", "Select an operating point of the scalable bitstream", OFFSET(operating_point), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, AV1_MAX_OPERATING_POINTS - 1, VD }, + { "dolbyvision", "Decode Dolby Vision RPUs", OFFSET(dovi.enable), AV_OPT_TYPE_BOOL, {.i64 = 1 }, 0, 1, VD }, { NULL } }; diff --git a/libavcodec/dovi_rpu.c b/libavcodec/dovi_rpu.c index d3a284c150d..54994188a96 100644 --- a/libavcodec/dovi_rpu.c +++ b/libavcodec/dovi_rpu.c @@ -90,6 +90,9 @@ int ff_dovi_attach_side_data(DOVIContext *s, AVFrame *frame) AVDOVIMetadata *dovi; size_t dovi_size, ext_sz; + if (!s->enable) + return 0; + if (!s->mapping || !s->color) return 0; /* incomplete dovi metadata */ @@ -558,6 +561,9 @@ int ff_dovi_rpu_parse(DOVIContext *s, const uint8_t *rpu, size_t rpu_size, uint8_t use_nlq; uint8_t profile; + if (!s->enable) + return 0; + if (rpu_size < 5) goto fail; diff --git a/libavcodec/dovi_rpu.h b/libavcodec/dovi_rpu.h index 56395707369..8dc69a2733d 100644 --- a/libavcodec/dovi_rpu.h +++ b/libavcodec/dovi_rpu.h @@ -37,6 +37,8 @@ typedef struct DOVIContext { * * For encoding, FF_DOVI_AUTOMATIC enables Dolby Vision only if * avctx->decoded_side_data contains an AVDOVIMetadata. + * + * For decoding, FF_DOVI_AUTOMATIC has the same meaning as 1. */ #define FF_DOVI_AUTOMATIC -1 int enable; diff --git a/libavcodec/hevcdec.c b/libavcodec/hevcdec.c index 4bc9e2afc1d..651773260e3 100644 --- a/libavcodec/hevcdec.c +++ b/libavcodec/hevcdec.c @@ -3689,6 +3689,7 @@ static const AVOption options[] = { AV_OPT_TYPE_BOOL, {.i64 = 0}, 0, 1, PAR }, { "strict-displaywin", "stricly apply default display window size", OFFSET(apply_defdispwin), AV_OPT_TYPE_BOOL, {.i64 = 0}, 0, 1, PAR }, + { "dolbyvision", "Decode Dolby Vision RPUs", OFFSET(dovi_ctx.enable), AV_OPT_TYPE_BOOL, {.i64 = 1 }, 0, 1, PAR }, { NULL }, }; diff --git a/libavcodec/libdav1d.c b/libavcodec/libdav1d.c index 09fe767fb86..f9e1a181fc3 100644 --- a/libavcodec/libdav1d.c +++ b/libavcodec/libdav1d.c @@ -674,6 +674,7 @@ static const AVOption libdav1d_options[] = { { "filmgrain", "Apply Film Grain", OFFSET(apply_grain), AV_OPT_TYPE_BOOL, { .i64 = -1 }, -1, 1, VD | AV_OPT_FLAG_DEPRECATED }, { "oppoint", "Select an operating point of the scalable bitstream", OFFSET(operating_point), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 31, VD }, { "alllayers", "Output all spatial layers", OFFSET(all_layers), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, VD }, + { "dolbyvision", "Decode Dolby Vision RPUs", OFFSET(dovi.enable), AV_OPT_TYPE_BOOL, {.i64 = 1 }, 0, 1, VD }, { NULL } }; From patchwork Tue Apr 9 12:57:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 47980 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:9c8d:b0:1a7:a0dc:8de5 with SMTP id mj13csp333547pzb; Tue, 9 Apr 2024 06:01:14 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCWscHmJcprm8dC4KkXz+CPFrHG1WjWbBECTCxiP/C7SwSq2dH7BTCE/T3iMrFXObMidjAtPFCozCfdbO9TywKiY8vdJQSF3vB295g== X-Google-Smtp-Source: AGHT+IEg4tdbTR8zcVZ7VaW8B6hN7Uww4b/2YzGLN10iPHwHW/OSR2EAJ+pmHsM1CTv1bbQD5puv X-Received: by 2002:a05:6402:35cb:b0:56e:2171:a55d with SMTP id z11-20020a05640235cb00b0056e2171a55dmr9017859edc.0.1712667673992; Tue, 09 Apr 2024 06:01:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1712667673; cv=none; d=google.com; s=arc-20160816; b=ufiwRQikuUbpUIR2UEkV9Hg3RiWTd26AjzN9eq/jy70h182nnaKPdJ9aU4kV0q2VNQ mQMdqoJnRf7lWFsiAvvDKl53zBbS65K0g5ZSiZENTD7o8ImkQKSCazwjec2J2p2c2dj1 wKG4xTZxk4Q+4kDezx9LSZl7PlpebCJN3sHQWubmKuTy3uUOP8BdlaA+2JS7zP5Nvo4U 6cFc387oBl+6uAf6t9G/Szp1WjWM5TZGe+oZ/DaoKlK+dMhbibmrAZxb900ZNGQc6Smg PV/Oi6S2w8XY0SDaR3FUE50w5ADtkWfkAIsaeVdRm3zw6X9BbRflUOI3HtAglwzisGxP sbbA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=3Y8UtzGWUp4g0RkvQjP7Zp008o6PU1lja6f3HkSNmK8=; fh=xmAeKtysnShNOmkhiJmYkS30uw4Fu2hvBJ7qlIwukxQ=; b=iIhBJTkP1AYdwRFRQRN2+1RNLsjnNBJm3XPzeOmTvioQ3eABiyc6mZM5lIe1caPGP2 HBQCECzyzmdLSn3uJa/cnLrUNGeT551wdwuuwG9xPJoBpQxf79g05n9S5UcbPbObOOuG KT0ITyRcCenClv4YxkADhipE13nyn7xfXh3uJvzaQykaZlmSVcXp/5Arbthrq4dg+dct rpIchmfx40gMM9zwW9/6S/a1Nt7vQxUtCK8l86FNg6PSr8QaPCaXh40psneqA5PzVEOb THuoTp+S7JKPiqBnzX59wP0uPBR4YDVxPXxp85XwPIpv3kJ8kHijZwV4bkOjUFmU4llf 2Ycg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b="kNRlk/Rm"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id v1-20020a50d581000000b0056c18b21900si4852126edi.643.2024.04.09.06.01.13; Tue, 09 Apr 2024 06:01:13 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b="kNRlk/Rm"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C3E7368D1F2; Tue, 9 Apr 2024 15:59:32 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1654568D188 for ; Tue, 9 Apr 2024 15:59:22 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1712667557; bh=qarmrM/CV3AoJ0PzYL2tufUHnvOdo69hM9j7ieydumo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kNRlk/RmvX6hoRZaaJb18cWmAF9ehM5WQO45wsSxINjs2H02xCgUBL2vcCPZNzVcp KX2E/earc3/G/tcvvpT0KTp+INnpwmFG4qsx3JN/rmCXAbPorWw766nsyKtwWGjsQj qciIpoYrCSJPYlwwSNF6mePbHXhBy3y3vyUKgcd4= Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id DF8AB4301F; Tue, 9 Apr 2024 14:59:17 +0200 (CEST) From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Tue, 9 Apr 2024 14:57:27 +0200 Message-ID: <20240409125914.61149-8-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240409125914.61149-1-ffmpeg@haasn.xyz> References: <20240409125914.61149-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 07/11] avcodec/dovi_rpu: add ff_dovi_rpu_generate() X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: c+5hrLijGxYU From: Niklas Haas This function takes a decoded AVDOVIMetadata struct and turns it back into a binary RPU. Verified using existing tools, and matches the bitstream in official reference files. I decided to just roll the EMDF and NAL encapsulation into this function because the end user will need to do it otherwise anyways. --- libavcodec/dovi_rpu.c | 542 ++++++++++++++++++++++++++++++++++++++++++ libavcodec/dovi_rpu.h | 20 +- 2 files changed, 560 insertions(+), 2 deletions(-) diff --git a/libavcodec/dovi_rpu.c b/libavcodec/dovi_rpu.c index 54994188a96..272a5125b65 100644 --- a/libavcodec/dovi_rpu.c +++ b/libavcodec/dovi_rpu.c @@ -29,6 +29,9 @@ #include "dovi_rpu.h" #include "golomb.h" #include "get_bits.h" +#include "itut35.h" +#include "put_bits.h" +#include "put_golomb.h" #include "refstruct.h" enum { @@ -361,6 +364,42 @@ static inline int64_t get_se_coef(GetBitContext *gb, const AVDOVIRpuDataHeader * return 0; /* unreachable */ } +static inline void put_ue_coef(PutBitContext *pb, const AVDOVIRpuDataHeader *hdr, + uint64_t coef) +{ + union { uint32_t u32; float f32; } fpart; + + switch (hdr->coef_data_type) { + case RPU_COEFF_FIXED: + set_ue_golomb(pb, coef >> hdr->coef_log2_denom); + put_bits64(pb, hdr->coef_log2_denom, + coef & ((1LL << hdr->coef_log2_denom) - 1)); + break; + case RPU_COEFF_FLOAT: + fpart.f32 = coef / (float) (1LL << hdr->coef_log2_denom); + put_bits64(pb, hdr->coef_log2_denom, fpart.u32); + break; + } +} + +static inline void put_se_coef(PutBitContext *pb, const AVDOVIRpuDataHeader *hdr, + uint64_t coef) +{ + union { uint32_t u32; float f32; } fpart; + + switch (hdr->coef_data_type) { + case RPU_COEFF_FIXED: + set_se_golomb(pb, coef >> hdr->coef_log2_denom); + put_bits64(pb, hdr->coef_log2_denom, + coef & ((1LL << hdr->coef_log2_denom) - 1)); + break; + case RPU_COEFF_FLOAT: + fpart.f32 = coef / (float) (1LL << hdr->coef_log2_denom); + put_bits64(pb, hdr->coef_log2_denom, fpart.u32); + break; + } +} + static inline unsigned get_variable_bits(GetBitContext *gb, int n) { unsigned int value = get_bits(gb, n); @@ -891,3 +930,506 @@ fail: ff_dovi_ctx_unref(s); /* don't leak potentially invalid state */ return AVERROR_INVALIDDATA; } + +static int av_q2den(AVRational q, int den) +{ + if (q.den == den) + return q.num; + q = av_mul_q(q, av_make_q(den, 1)); + return (q.num + (q.den >> 1)) / q.den; +} + +static void generate_ext_v1(PutBitContext *pb, const AVDOVIDmData *dm) +{ + int ext_block_length, start_pos, pad_bits; + + switch (dm->level) { + case 1: ext_block_length = 5; break; + case 2: ext_block_length = 11; break; + case 4: ext_block_length = 3; break; + case 5: ext_block_length = 7; break; + case 6: ext_block_length = 8; break; + case 255: ext_block_length = 6; break; + default: return; + } + + set_ue_golomb(pb, ext_block_length); + put_bits(pb, 8, dm->level); + start_pos = put_bits_count(pb); + + switch (dm->level) { + case 1: + put_bits(pb, 12, dm->l1.min_pq); + put_bits(pb, 12, dm->l1.max_pq); + put_bits(pb, 12, dm->l1.avg_pq); + break; + case 2: + put_bits(pb, 12, dm->l2.target_max_pq); + put_bits(pb, 12, dm->l2.trim_slope); + put_bits(pb, 12, dm->l2.trim_offset); + put_bits(pb, 12, dm->l2.trim_power); + put_bits(pb, 12, dm->l2.trim_chroma_weight); + put_bits(pb, 12, dm->l2.trim_saturation_gain); + put_bits(pb, 13, dm->l2.ms_weight + 8192); + break; + case 4: + put_bits(pb, 12, dm->l4.anchor_pq); + put_bits(pb, 12, dm->l4.anchor_power); + break; + case 5: + put_bits(pb, 13, dm->l5.left_offset); + put_bits(pb, 13, dm->l5.right_offset); + put_bits(pb, 13, dm->l5.top_offset); + put_bits(pb, 13, dm->l5.bottom_offset); + break; + case 6: + put_bits(pb, 16, dm->l6.max_luminance); + put_bits(pb, 16, dm->l6.min_luminance); + put_bits(pb, 16, dm->l6.max_cll); + put_bits(pb, 16, dm->l6.max_fall); + break; + case 255: + put_bits(pb, 8, dm->l255.dm_run_mode); + put_bits(pb, 8, dm->l255.dm_run_version); + for (int i = 0; i < 4; i++) + put_bits(pb, 8, dm->l255.dm_debug[i]); + break; + } + + pad_bits = ext_block_length * 8 - (put_bits_count(pb) - start_pos); + av_assert1(pad_bits >= 0); + put_bits(pb, pad_bits, 0); +} + +static void put_cie_xy(PutBitContext *pb, AVCIExy xy) +{ + const int denom = 32767; + put_sbits(pb, 16, av_q2den(xy.x, denom)); + put_sbits(pb, 16, av_q2den(xy.y, denom)); +} + +#define ANY6(arr) (arr[0] || arr[1] || arr[2] || arr[3] || arr[4] || arr[5]) +#define ANY_XY(xy) (xy.x.num || xy.y.num) +#define ANY_CSP(csp) (ANY_XY(csp.prim.r) || ANY_XY(csp.prim.g) || \ + ANY_XY(csp.prim.b) || ANY_XY(csp.wp)) + +static void generate_ext_v2(PutBitContext *pb, const AVDOVIDmData *dm) +{ + int ext_block_length, start_pos, pad_bits; + + switch (dm->level) { + case 3: ext_block_length = 5; break; + case 8: + if (ANY6(dm->l8.hue_vector_field)) { + ext_block_length = 25; + } else if (ANY6(dm->l8.saturation_vector_field)) { + ext_block_length = 19; + } else if (dm->l8.clip_trim) { + ext_block_length = 13; + } else if (dm->l8.target_mid_contrast) { + ext_block_length = 12; + } else { + ext_block_length = 10; + } + break; + case 9: + if (ANY_CSP(dm->l9.source_display_primaries)) { + ext_block_length = 17; + } else { + ext_block_length = 1; + } + break; + case 10: + if (ANY_CSP(dm->l10.target_display_primaries)) { + ext_block_length = 21; + } else { + ext_block_length = 5; + } + break; + case 11: ext_block_length = 4; break; + case 254: ext_block_length = 2; break; + default: return; + } + + set_ue_golomb(pb, ext_block_length); + put_bits(pb, 8, dm->level); + start_pos = put_bits_count(pb); + + switch (dm->level) { + case 3: + put_bits(pb, 12, dm->l3.min_pq_offset); + put_bits(pb, 12, dm->l3.max_pq_offset); + put_bits(pb, 12, dm->l3.avg_pq_offset); + break; + case 8: + put_bits(pb, 8, dm->l8.target_display_index); + put_bits(pb, 12, dm->l8.trim_slope); + put_bits(pb, 12, dm->l8.trim_offset); + put_bits(pb, 12, dm->l8.trim_power); + put_bits(pb, 12, dm->l8.trim_chroma_weight); + put_bits(pb, 12, dm->l8.trim_saturation_gain); + put_bits(pb, 12, dm->l8.ms_weight + 8192); + if (ext_block_length < 12) + break; + put_bits(pb, 12, dm->l8.target_mid_contrast); + if (ext_block_length < 13) + break; + put_bits(pb, 12, dm->l8.clip_trim); + if (ext_block_length < 19) + break; + for (int i = 0; i < 6; i++) + put_bits(pb, 8, dm->l8.saturation_vector_field[i]); + if (ext_block_length < 25) + break; + for (int i = 0; i < 6; i++) + put_bits(pb, 8, dm->l8.hue_vector_field[i]); + break; + case 9: + put_bits(pb, 8, dm->l9.source_primary_index); + if (ext_block_length < 17) + break; + put_cie_xy(pb, dm->l9.source_display_primaries.prim.r); + put_cie_xy(pb, dm->l9.source_display_primaries.prim.g); + put_cie_xy(pb, dm->l9.source_display_primaries.prim.b); + put_cie_xy(pb, dm->l9.source_display_primaries.wp); + break; + case 10: + put_bits(pb, 8, dm->l10.target_display_index); + put_bits(pb, 12, dm->l10.target_max_pq); + put_bits(pb, 12, dm->l10.target_min_pq); + put_bits(pb, 8, dm->l10.target_primary_index); + if (ext_block_length < 21) + break; + put_cie_xy(pb, dm->l10.target_display_primaries.prim.r); + put_cie_xy(pb, dm->l10.target_display_primaries.prim.g); + put_cie_xy(pb, dm->l10.target_display_primaries.prim.b); + put_cie_xy(pb, dm->l10.target_display_primaries.wp); + break; + case 11: + put_bits(pb, 8, dm->l11.content_type); + put_bits(pb, 4, dm->l11.whitepoint); + put_bits(pb, 1, dm->l11.reference_mode_flag); + put_bits(pb, 3, 0); /* reserved */ + put_bits(pb, 2, dm->l11.sharpness); + put_bits(pb, 2, dm->l11.noise_reduction); + put_bits(pb, 2, dm->l11.mpeg_noise_reduction); + put_bits(pb, 2, dm->l11.frame_rate_conversion); + put_bits(pb, 2, dm->l11.brightness); + put_bits(pb, 2, dm->l11.color); + break; + case 254: + put_bits(pb, 8, dm->l254.dm_mode); + put_bits(pb, 8, dm->l254.dm_version_index); + break; + } + + pad_bits = ext_block_length * 8 - (put_bits_count(pb) - start_pos); + av_assert1(pad_bits >= 0); + put_bits(pb, pad_bits, 0); +} + +int ff_dovi_rpu_generate(DOVIContext *s, const AVDOVIMetadata *metadata, + uint8_t **out_rpu, int *out_size) +{ + PutBitContext *pb = &(PutBitContext){0}; + const AVDOVIRpuDataHeader *hdr; + const AVDOVIDataMapping *mapping; + const AVDOVIColorMetadata *color; + int vdr_dm_metadata_changed, vdr_rpu_id, use_prev_vdr_rpu, profile, + buffer_size, rpu_size, pad, zero_run; + int num_ext_blocks_v1, num_ext_blocks_v2; + uint32_t crc; + uint8_t *dst; + if (!metadata) { + *out_rpu = NULL; + *out_size = 0; + return 0; + } + + hdr = av_dovi_get_header(metadata); + mapping = av_dovi_get_mapping(metadata); + color = av_dovi_get_color(metadata); + av_assert0(s->cfg.dv_profile); + + if (hdr->rpu_type != 2) { + av_log(s->logctx, AV_LOG_ERROR, "Unhandled RPU type %"PRIu8"\n", + hdr->rpu_type); + return AVERROR_INVALIDDATA; + } + + vdr_rpu_id = -1; + for (int i = 0; i <= DOVI_MAX_DM_ID; i++) { + if (s->vdr[i] && !memcmp(&s->vdr[i]->mapping, mapping, sizeof(*mapping))) { + vdr_rpu_id = i; + break; + } else if (vdr_rpu_id < 0 && (!s->vdr[i] || i == DOVI_MAX_DM_ID)) { + vdr_rpu_id = i; + } + } + + if (!s->vdr[vdr_rpu_id]) { + s->vdr[vdr_rpu_id] = ff_refstruct_allocz(sizeof(DOVIVdr)); + if (!s->vdr[vdr_rpu_id]) + return AVERROR(ENOMEM); + } + + if (!s->vdr[color->dm_metadata_id]) { + s->vdr[color->dm_metadata_id] = ff_refstruct_allocz(sizeof(DOVIVdr)); + if (!s->vdr[color->dm_metadata_id]) + return AVERROR(ENOMEM); + } + + num_ext_blocks_v1 = num_ext_blocks_v2 = 0; + for (int i = 0; i < metadata->num_ext_blocks; i++) { + const AVDOVIDmData *dm = av_dovi_get_ext(metadata, i); + switch (dm->level) { + case 1: + case 2: + case 4: + case 5: + case 6: + case 255: + num_ext_blocks_v1++; + break; + case 3: + case 8: + case 9: + case 10: + case 11: + case 254: + num_ext_blocks_v2++; + break; + default: + av_log(s->logctx, AV_LOG_ERROR, "Invalid ext block level %d\n", + dm->level); + return AVERROR_INVALIDDATA; + } + } + + vdr_dm_metadata_changed = !s->color || memcmp(s->color, color, sizeof(*color)); + use_prev_vdr_rpu = !memcmp(&s->vdr[vdr_rpu_id]->mapping, mapping, sizeof(*mapping)); + + buffer_size = 12 /* vdr seq info */ + 5 /* CRC32 + terminator */; + buffer_size += num_ext_blocks_v1 * 13; + buffer_size += num_ext_blocks_v2 * 28; + if (!use_prev_vdr_rpu) { + buffer_size += 160; + for (int c = 0; c < 3; c++) { + for (int i = 0; i < mapping->curves[c].num_pivots - 1; i++) { + switch (mapping->curves[c].mapping_idc[i]) { + case AV_DOVI_MAPPING_POLYNOMIAL: buffer_size += 26; break; + case AV_DOVI_MAPPING_MMR: buffer_size += 177; break; + } + } + } + } + if (vdr_dm_metadata_changed) + buffer_size += 67; + + av_fast_padded_malloc(&s->rpu_buf, &s->rpu_buf_sz, buffer_size); + if (!s->rpu_buf) + return AVERROR(ENOMEM); + init_put_bits(pb, s->rpu_buf, s->rpu_buf_sz); + + /* RPU header */ + put_bits(pb, 6, hdr->rpu_type); + put_bits(pb, 11, hdr->rpu_format); + put_bits(pb, 4, hdr->vdr_rpu_profile); + put_bits(pb, 4, hdr->vdr_rpu_level); + put_bits(pb, 1, 1); /* vdr_seq_info_present */ + put_bits(pb, 1, hdr->chroma_resampling_explicit_filter_flag); + put_bits(pb, 2, hdr->coef_data_type); + if (hdr->coef_data_type == RPU_COEFF_FIXED) + set_ue_golomb(pb, hdr->coef_log2_denom); + put_bits(pb, 2, hdr->vdr_rpu_normalized_idc); + put_bits(pb, 1, hdr->bl_video_full_range_flag); + if ((hdr->rpu_format & 0x700) == 0) { + set_ue_golomb(pb, hdr->bl_bit_depth - 8); + set_ue_golomb(pb, hdr->el_bit_depth - 8); + set_ue_golomb(pb, hdr->vdr_bit_depth - 8); + put_bits(pb, 1, hdr->spatial_resampling_filter_flag); + put_bits(pb, 3, 0); /* reserved_zero_3bits */ + put_bits(pb, 1, hdr->el_spatial_resampling_filter_flag); + put_bits(pb, 1, hdr->disable_residual_flag); + } + s->header = *hdr; + + put_bits(pb, 1, vdr_dm_metadata_changed); + put_bits(pb, 1, use_prev_vdr_rpu); + set_ue_golomb(pb, vdr_rpu_id); + s->mapping = &s->vdr[vdr_rpu_id]->mapping; + + if (!use_prev_vdr_rpu) { + set_ue_golomb(pb, mapping->mapping_color_space); + set_ue_golomb(pb, mapping->mapping_chroma_format_idc); + for (int c = 0; c < 3; c++) { + const AVDOVIReshapingCurve *curve = &mapping->curves[c]; + int prev = 0; + set_ue_golomb(pb, curve->num_pivots - 2); + for (int i = 0; i < curve->num_pivots; i++) { + put_bits(pb, hdr->bl_bit_depth, curve->pivots[i] - prev); + prev = curve->pivots[i]; + } + } + + if (mapping->nlq_method_idc != AV_DOVI_NLQ_NONE) { + put_bits(pb, 3, mapping->nlq_method_idc); + put_bits(pb, hdr->bl_bit_depth, mapping->nlq_pivots[0]); + put_bits(pb, hdr->bl_bit_depth, mapping->nlq_pivots[1] - mapping->nlq_pivots[0]); + } + + set_ue_golomb(pb, mapping->num_x_partitions - 1); + set_ue_golomb(pb, mapping->num_y_partitions - 1); + + for (int c = 0; c < 3; c++) { + const AVDOVIReshapingCurve *curve = &mapping->curves[c]; + for (int i = 0; i < curve->num_pivots - 1; i++) { + set_ue_golomb(pb, curve->mapping_idc[i]); + switch (curve->mapping_idc[i]) { + case AV_DOVI_MAPPING_POLYNOMIAL: { + set_ue_golomb(pb, curve->poly_order[i] - 1); + if (curve->poly_order[i] == 1) + put_bits(pb, 1, 0); /* linear_interp_flag */ + for (int k = 0; k <= curve->poly_order[i]; k++) + put_se_coef(pb, hdr, curve->poly_coef[i][k]); + break; + } + case AV_DOVI_MAPPING_MMR: { + put_bits(pb, 2, curve->mmr_order[i] - 1); + put_se_coef(pb, hdr, curve->mmr_constant[i]); + for (int j = 0; j < curve->mmr_order[i]; j++) { + for (int k = 0; k < 7; k++) + put_se_coef(pb, hdr, curve->mmr_coef[i][j][k]); + } + break; + } + } + } + } + + if (mapping->nlq_method_idc != AV_DOVI_NLQ_NONE) { + for (int c = 0; c < 3; c++) { + const AVDOVINLQParams *nlq = &mapping->nlq[c]; + put_bits(pb, hdr->el_bit_depth, nlq->nlq_offset); + put_ue_coef(pb, hdr, nlq->vdr_in_max); + switch (mapping->nlq_method_idc) { + case AV_DOVI_NLQ_LINEAR_DZ: + put_ue_coef(pb, hdr, nlq->linear_deadzone_slope); + put_ue_coef(pb, hdr, nlq->linear_deadzone_threshold); + break; + } + } + } + + memcpy(&s->vdr[vdr_rpu_id]->mapping, mapping, sizeof(*mapping)); + } + + if (vdr_dm_metadata_changed) { + const int denom = profile == 4 ? (1 << 30) : (1 << 28); + set_ue_golomb(pb, color->dm_metadata_id); /* affected_dm_id */ + set_ue_golomb(pb, color->dm_metadata_id); /* current_dm_id */ + set_ue_golomb(pb, color->scene_refresh_flag); + for (int i = 0; i < 9; i++) + put_sbits(pb, 16, av_q2den(color->ycc_to_rgb_matrix[i], 1 << 13)); + for (int i = 0; i < 3; i++) + put_bits32(pb, av_q2den(color->ycc_to_rgb_offset[i], denom)); + for (int i = 0; i < 9; i++) + put_sbits(pb, 16, av_q2den(color->rgb_to_lms_matrix[i], 1 << 14)); + put_bits(pb, 16, color->signal_eotf); + put_bits(pb, 16, color->signal_eotf_param0); + put_bits(pb, 16, color->signal_eotf_param1); + put_bits32(pb, color->signal_eotf_param2); + put_bits(pb, 5, color->signal_bit_depth); + put_bits(pb, 2, color->signal_color_space); + put_bits(pb, 2, color->signal_chroma_format); + put_bits(pb, 2, color->signal_full_range_flag); + put_bits(pb, 12, color->source_min_pq); + put_bits(pb, 12, color->source_max_pq); + put_bits(pb, 10, color->source_diagonal); + + memcpy(&s->vdr[color->dm_metadata_id]->color, color, sizeof(*color)); + s->color = &s->vdr[color->dm_metadata_id]->color; + } + + set_ue_golomb(pb, num_ext_blocks_v1); + align_put_bits(pb); + for (int i = 0; i < metadata->num_ext_blocks; i++) + generate_ext_v1(pb, av_dovi_get_ext(metadata, i)); + + if (num_ext_blocks_v2) { + set_ue_golomb(pb, num_ext_blocks_v2); + align_put_bits(pb); + for (int i = 0; i < metadata->num_ext_blocks; i++) + generate_ext_v2(pb, av_dovi_get_ext(metadata, i)); + } + + flush_put_bits(pb); + crc = av_bswap32(av_crc(av_crc_get_table(AV_CRC_32_IEEE), -1, + s->rpu_buf, put_bytes_output(pb))); + put_bits32(pb, crc); + put_bits(pb, 8, 0x80); /* terminator */ + flush_put_bits(pb); + + rpu_size = put_bytes_output(pb); + switch (s->cfg.dv_profile) { + case 10: + /* AV1 uses T.35 OBU with EMDF header */ + *out_rpu = av_malloc(rpu_size + 15); + if (!*out_rpu) + return AVERROR(ENOMEM); + init_put_bits(pb, *out_rpu, rpu_size + 15); + put_bits(pb, 8, ITU_T_T35_COUNTRY_CODE_US); + put_bits(pb, 16, ITU_T_T35_PROVIDER_CODE_DOLBY); + put_bits32(pb, 0x800); /* provider_oriented_code */ + put_bits(pb, 27, 0x01be6841u); /* fixed EMDF header, see above */ + if (rpu_size > 0xFF) { + av_assert2(rpu_size <= 0x10000); + put_bits(pb, 8, (rpu_size >> 8) - 1); + put_bits(pb, 1, 1); /* read_more */ + put_bits(pb, 8, rpu_size & 0xFF); + put_bits(pb, 1, 0); + } else { + put_bits(pb, 8, rpu_size); + put_bits(pb, 1, 0); + } + ff_copy_bits(pb, s->rpu_buf, rpu_size * 8); + put_bits(pb, 17, 0x400); /* emdf payload id + emdf_protection */ + + pad = pb->bit_left & 7; + put_bits(pb, pad, (1 << pad) - 1); /* pad to next byte with 1 bits */ + flush_put_bits(pb); + *out_size = put_bytes_output(pb); + return 0; + + case 5: + case 8: + *out_rpu = dst = av_malloc(1 + rpu_size * 3 / 2); /* worst case */ + if (!*out_rpu) + return AVERROR(ENOMEM); + *dst++ = 25; /* NAL prefix */ + zero_run = 0; + for (int i = 0; i < rpu_size; i++) { + if (zero_run < 2) { + if (s->rpu_buf[i] == 0) { + zero_run++; + } else { + zero_run = 0; + } + } else { + if ((s->rpu_buf[i] & ~3) == 0) { + /* emulation prevention */ + *dst++ = 3; + } + zero_run = s->rpu_buf[i] == 0; + } + *dst++ = s->rpu_buf[i]; + } + *out_size = dst - *out_rpu; + return 0; + + default: + /* Should be unreachable */ + av_assert0(0); + return AVERROR_BUG; + } +} diff --git a/libavcodec/dovi_rpu.h b/libavcodec/dovi_rpu.h index 8dc69a2733d..1287dabd636 100644 --- a/libavcodec/dovi_rpu.h +++ b/libavcodec/dovi_rpu.h @@ -55,20 +55,22 @@ typedef struct DOVIContext { AVDOVIDecoderConfigurationRecord cfg; /** - * Currently active RPU data header, updates on every dovi_rpu_parse(). + * Currently active RPU data header, updates on every ff_dovi_rpu_parse() + * or ff_dovi_rpu_generate(). */ AVDOVIRpuDataHeader header; /** * Currently active data mappings, or NULL. Points into memory owned by the * corresponding rpu/vdr_ref, which becomes invalid on the next call to - * dovi_rpu_parse. + * ff_dovi_rpu_parse() or ff_dovi_rpu_generate(). */ const AVDOVIDataMapping *mapping; const AVDOVIColorMetadata *color; /** * Currently active extension blocks, updates on every ff_dovi_rpu_parse() + * or ff_dovi_rpu_generate(). */ AVDOVIDmData *ext_blocks; int num_ext_blocks; @@ -119,4 +121,18 @@ int ff_dovi_attach_side_data(DOVIContext *s, AVFrame *frame); */ int ff_dovi_configure(DOVIContext *s, AVCodecContext *avctx); +/** + * Synthesize a Dolby Vision RPU reflecting the current state. Note that this + * assumes all previous calls to `ff_dovi_rpu_generate` have been appropriately + * signalled, i.e. it will not re-send already transmitted redundant data. + * + * Mutates the internal state of DOVIContext to reflect the change. + * Returns 0 or a negative error code. + * + * This generates a fully formed RPU ready for inclusion in the bitstream, + * including the EMDF header (profile 10) or NAL encapsulation (otherwise). + */ +int ff_dovi_rpu_generate(DOVIContext *s, const AVDOVIMetadata *metadata, + uint8_t **out_rpu, int *out_size); + #endif /* AVCODEC_DOVI_RPU_H */ From patchwork Tue Apr 9 12:57:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 47979 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:9c8d:b0:1a7:a0dc:8de5 with SMTP id mj13csp333437pzb; Tue, 9 Apr 2024 06:01:05 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCVIW2sXROOOW1sQgMq88FJh88GX63k5V6/RFqQGPoX1dSrx+4vJfHNAU+RisNhZa9DfvrpLX2LvfIBaFGFJSUMVdhgu4BG+md22rA== X-Google-Smtp-Source: AGHT+IERkHoGzOXL0NikFIqvp7pafXBYbW+iPZa8V11PuJasVngqZ9+ezy9QsbMogJ2gbO8bp6dM X-Received: by 2002:a50:a6dc:0:b0:568:c6fd:4f50 with SMTP id f28-20020a50a6dc000000b00568c6fd4f50mr2644507edc.7.1712667665550; Tue, 09 Apr 2024 06:01:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1712667665; cv=none; d=google.com; s=arc-20160816; b=oV27E4U+mNRCKlBupzDDnpFi8jG/Ad2IQC5WIhrJdqBnukrEtS+AASB4vlLG91uzfl YhoFLDsi1eCfNql5xa6+8GzxONAVk+x6PnnSvcGKQje8e26vSyk9bVkDiYU7Ol0/FQrh s6bEfXWhzVe5piNJH/H1xlXVjf41TTWjlzZg3HNu7R9AsB5KRaRGrUP4qkSK2Xj+4z68 OcuQQBB7cQp3c7d8Sgg5AywaAl5R9D+T2D+zBOEXBhEEmyWQvn5ML6MebgHzswL3tGeg wsJPtKu/SkRjk7JOwL08RGpnbTXoCctiZwbBemrMAm4LtZFXhdbKpXNUF+Z3oI8cSLqg BRiQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=3fxPBVRCY76DKwlSILK4O6x0vfw/agwKqucbGUJ8tHQ=; fh=xmAeKtysnShNOmkhiJmYkS30uw4Fu2hvBJ7qlIwukxQ=; b=KFvt0cDf+rz3IIsMfYzajc+WTivT5e1h1KmicteZEf4co7/7UPbr8NhPbSDgC/L5EL g6EcOIwMG6fRtM6Lnc8vlGzxL9X7nJVBF9+RzsPgjmvl4VgSUY4VLG39FUHNen3Htsr9 FY3RRVJCmv5kyisOoefIQgmQhKbiLBwYMx2j5ixdelXt9wNQ7lqTq9Squq/LJfSN0Fdr d5lksZ/dKuogHpBdeAnlC0+WOxsaw/USSAloxKg8UntwLj96qmU+VcqSTjTU1+AmnLw9 b+QxK/EHHjtexohvok3pnr7ZB99SgcPrBjchvRR5MInq9hLaSX0J4LKX0IARfPbOCUCO m+OQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=WWNgZyAH; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id b29-20020a50ccdd000000b0056c49ebbb14si4788698edj.275.2024.04.09.06.01.02; Tue, 09 Apr 2024 06:01:05 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=WWNgZyAH; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E228968D1D8; Tue, 9 Apr 2024 15:59:31 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1BC5D68D18A for ; Tue, 9 Apr 2024 15:59:22 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1712667558; bh=AEStckKVyU7tZ92H7sOZKCzIyc/boUg/jr6V6WIiRbc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WWNgZyAH1ZcQhkQsS7/DuHU8VDNL5JQC25NTzABRwTPcOhEMpcgvdAnEqF+wjmIeD +jYKUBPNL/OLBkA4WJ6hKwsZlYN9yn+dfFLJzjQCJX/UB5r9jwA/WEEwzhgdzWS5eJ Nl7TSQyWO21Zx6hST6ChaPB2DbCohN7vRQiL+IfI= Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id 20F4B43315; Tue, 9 Apr 2024 14:59:18 +0200 (CEST) From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Tue, 9 Apr 2024 14:57:28 +0200 Message-ID: <20240409125914.61149-9-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240409125914.61149-1-ffmpeg@haasn.xyz> References: <20240409125914.61149-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 08/11] avformat/movenc: warn if dovi cfg ignored X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 2l1fzl0qBPXi From: Niklas Haas Since this is guarded behind strict unofficial, we should warn if the user feeds a dolby vision stream to this muxer, as it will otherwise result in a broken file. --- libavformat/movenc.c | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/libavformat/movenc.c b/libavformat/movenc.c index 15b65dcf96d..0f819214be9 100644 --- a/libavformat/movenc.c +++ b/libavformat/movenc.c @@ -2528,16 +2528,21 @@ static int mov_write_video_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContex const AVPacketSideData *spherical_mapping = av_packet_side_data_get(track->st->codecpar->coded_side_data, track->st->codecpar->nb_coded_side_data, AV_PKT_DATA_SPHERICAL); - const AVPacketSideData *dovi = av_packet_side_data_get(track->st->codecpar->coded_side_data, - track->st->codecpar->nb_coded_side_data, - AV_PKT_DATA_DOVI_CONF); - if (stereo_3d) mov_write_st3d_tag(s, pb, (AVStereo3D*)stereo_3d->data); if (spherical_mapping) mov_write_sv3d_tag(mov->fc, pb, (AVSphericalMapping*)spherical_mapping->data); - if (dovi) + } + + if (track->mode == MODE_MP4) { + const AVPacketSideData *dovi = av_packet_side_data_get(track->st->codecpar->coded_side_data, + track->st->codecpar->nb_coded_side_data, + AV_PKT_DATA_DOVI_CONF); + if (dovi && mov->fc->strict_std_compliance <= FF_COMPLIANCE_UNOFFICIAL) { mov_write_dvcc_dvvc_tag(s, pb, (AVDOVIDecoderConfigurationRecord *)dovi->data); + } else if (dovi) { + av_log(mov->fc, AV_LOG_WARNING, "Not writing 'dvcC'/'dvvC' box. Requires -strict unofficial.\n"); + } } if (track->par->sample_aspect_ratio.den && track->par->sample_aspect_ratio.num) { From patchwork Tue Apr 9 12:57:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 47981 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:9c8d:b0:1a7:a0dc:8de5 with SMTP id mj13csp333772pzb; Tue, 9 Apr 2024 06:01:29 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCWDNzyd6s67LPH+1QQduV4/fzbG3Naw1YGO7oTeCKT3aaI2eNNcb9soS2XE0N5NTNHefoFepvubDr53i9QStRL7C1RlbL3UlG8zBg== X-Google-Smtp-Source: AGHT+IEkD4JPQS94BH+mnMZotLbTr1dF0LU6VgVdLJLAsJJriUhii0Uqj5JXPcePW62CsNR78qHI X-Received: by 2002:a17:907:1b29:b0:a51:da1d:c0e1 with SMTP id mp41-20020a1709071b2900b00a51da1dc0e1mr4104194ejc.50.1712667689479; Tue, 09 Apr 2024 06:01:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1712667689; cv=none; d=google.com; s=arc-20160816; b=uqL6bi1h1+n3fNMeWjHzzWoKwLSHA6SO1N7QHLFr683GbxOdMgPQY5phBD1M6KGSaK 12rAW6DKbEI0TfbIAzWcJTS+PZ9a+n1MlfGL1+vaTMFP7tcmcYZEKiEJGUQLK6VgB+rj Bw5FZ4oP9oYVycWbEaMVvrtLULFsC889/CAa93HKEaOrl4LyPLFz90VYIrJ5a7GAu5UA /z5TyyZqbB/sYB3ZwZ5b+yaPD3quXm23IBHUIFpCs6ihremhhkn9hzKJ1VedSXQM9eMV 49Ak/30wLdV5GCks6GbNsOtHWpxvcgNR1EzNDxkghJ3RhknsUVMHm3k/6E2ziWXzfDyI eB5g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=pdyVCtNf+UBvyze1d0kOtYJNOPSrDcNiE/kwiiNon6k=; fh=xmAeKtysnShNOmkhiJmYkS30uw4Fu2hvBJ7qlIwukxQ=; b=Ac0d/inkLDfUDILe4Wv8Ej456xNAQdMvU7PM+mMfAyGJx+CtgkvrZ3Z97TNL6jVNDN qF1eWfQrm4nnZHHEEYLuHeJe+6fChWcpItlpnWHys/dgCRZqvZka+4SMbjf54zuAPi0o 8N8YlAE/18XTyjISU7etTHmJRvMq0egkb3QW/8d37WVVmBN/H9HZfcJ8ugSeHmJ4CkOm +pt6dBMlqIc2+l/G8jT7yfwPXff1NL1Tj4R0HJlIgbz6pvU8DQSrfEQbJ67tpBQB3hF2 8qTjXweGmGf6I0Agdyx2CDXRxlnN8i/IbA8VU33eHOE3XBeeS+kootinJ2wr/8tHhRRT a6TA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=DYl04w+1; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id j19-20020a170906255300b00a519d6027a0si4611314ejb.542.2024.04.09.06.01.24; Tue, 09 Apr 2024 06:01:29 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=DYl04w+1; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B583F68D1FC; Tue, 9 Apr 2024 15:59:33 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 473D468D188 for ; Tue, 9 Apr 2024 15:59:22 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1712667558; bh=SwkEC+YozGgsU3YLXl+zkKSjhor7Vl+YO1BAPz1Deb8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DYl04w+1lfLiVzzVl0j91w4Jbum8sANZMIRGwYm/sE6PmiJPPqYC1TvMm8eOZcvxT 61JbQz6hM831ahp5nDa5FaRcgWA5TvCByzH7SiV5N8j48DBDHhethPguIfzn/mArht VwpHnKjckMIhhbEbtmfGbhc2FDXW2kmLk+fHRBnQ= Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id 544AD433CD; Tue, 9 Apr 2024 14:59:18 +0200 (CEST) From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Tue, 9 Apr 2024 14:57:29 +0200 Message-ID: <20240409125914.61149-10-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240409125914.61149-1-ffmpeg@haasn.xyz> References: <20240409125914.61149-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 09/11] avcodec/libaomenc: implement dolby vision coding X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: TLhogeJgKIaH From: Niklas Haas --- libavcodec/libaomenc.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/libavcodec/libaomenc.c b/libavcodec/libaomenc.c index 4a71bba9c9c..b43a902a384 100644 --- a/libavcodec/libaomenc.c +++ b/libavcodec/libaomenc.c @@ -43,6 +43,7 @@ #include "avcodec.h" #include "bsf.h" #include "codec_internal.h" +#include "dovi_rpu.h" #include "encode.h" #include "internal.h" #include "libaom.h" @@ -70,6 +71,7 @@ struct FrameListData { typedef struct AOMEncoderContext { AVClass *class; AVBSFContext *bsf; + DOVIContext dovi; struct aom_codec_ctx encoder; struct aom_image rawimg; struct aom_fixed_buf twopass_stats; @@ -421,6 +423,7 @@ static av_cold int aom_free(AVCodecContext *avctx) av_freep(&avctx->stats_out); free_frame_list(ctx->coded_frame_list); av_bsf_free(&ctx->bsf); + ff_dovi_ctx_unref(&ctx->dovi); return 0; } @@ -989,6 +992,10 @@ static av_cold int aom_init(AVCodecContext *avctx, if (!cpb_props) return AVERROR(ENOMEM); + ctx->dovi.logctx = avctx; + if ((res = ff_dovi_configure(&ctx->dovi, avctx)) < 0) + return res; + if (avctx->flags & AV_CODEC_FLAG_GLOBAL_HEADER) { const AVBitStreamFilter *filter = av_bsf_get_by_name("extract_extradata"); int ret; @@ -1242,6 +1249,7 @@ static int aom_encode(AVCodecContext *avctx, AVPacket *pkt, unsigned long duration = 0; int res, coded_size; aom_enc_frame_flags_t flags = 0; + AVFrameSideData *sd; if (frame) { rawimg = &ctx->rawimg; @@ -1279,6 +1287,24 @@ FF_ENABLE_DEPRECATION_WARNINGS break; } + sd = av_frame_get_side_data(frame, AV_FRAME_DATA_DOVI_METADATA); + if (ctx->dovi.cfg.dv_profile && sd) { + const AVDOVIMetadata *metadata = (const AVDOVIMetadata *)sd->data; + uint8_t *t35; + int size; + if ((res = ff_dovi_rpu_generate(&ctx->dovi, metadata, &t35, &size)) < 0) + return res; + res = aom_img_add_metadata(rawimg, OBU_METADATA_TYPE_ITUT_T35, + t35, size, AOM_MIF_ANY_FRAME); + av_free(t35); + if (res != AOM_CODEC_OK) + return AVERROR(ENOMEM); + } else if (ctx->dovi.cfg.dv_profile) { + av_log(avctx, AV_LOG_ERROR, "Dolby Vision enabled, but received frame " + "without AV_FRAME_DATA_DOVI_METADATA\n"); + return AVERROR_INVALIDDATA; + } + if (frame->pict_type == AV_PICTURE_TYPE_I) flags |= AOM_EFLAG_FORCE_KF; } @@ -1459,6 +1485,8 @@ static const AVOption options[] = { { "ssim", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = AOM_TUNE_SSIM}, 0, 0, VE, .unit = "tune"}, FF_AV1_PROFILE_OPTS { "still-picture", "Encode in single frame mode (typically used for still AVIF images).", OFFSET(still_picture), AV_OPT_TYPE_BOOL, {.i64 = 0}, -1, 1, VE }, + { "dolbyvision", "Enable Dolby Vision RPU coding", OFFSET(dovi.enable), AV_OPT_TYPE_BOOL, {.i64 = FF_DOVI_AUTOMATIC }, -1, 1, VE, .unit = "dovi" }, + { "auto", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = FF_DOVI_AUTOMATIC}, .flags = VE, .unit = "dovi" }, { "enable-rect-partitions", "Enable rectangular partitions", OFFSET(enable_rect_partitions), AV_OPT_TYPE_BOOL, {.i64 = -1}, -1, 1, VE}, { "enable-1to4-partitions", "Enable 1:4/4:1 partitions", OFFSET(enable_1to4_partitions), AV_OPT_TYPE_BOOL, {.i64 = -1}, -1, 1, VE}, { "enable-ab-partitions", "Enable ab shape partitions", OFFSET(enable_ab_partitions), AV_OPT_TYPE_BOOL, {.i64 = -1}, -1, 1, VE}, From patchwork Tue Apr 9 12:57:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 47982 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:9c8d:b0:1a7:a0dc:8de5 with SMTP id mj13csp333892pzb; Tue, 9 Apr 2024 06:01:35 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCW07r/vK6trmjH13evloVZ33ESDCt4i5O1m+on0MldU77/SK9hoC3HmJTv1I0z8rabUU8EzQdisTpae2CvWeISqaWyYkTqsGIzjMA== X-Google-Smtp-Source: AGHT+IEBMcmfu9YPtMbD1EO+UEQ/9MM1eHCVQ55Xt2TZYphHwZ+jwOB6wIm36D+xAMlwiD0fzM4t X-Received: by 2002:a50:9313:0:b0:568:abe3:52b2 with SMTP id m19-20020a509313000000b00568abe352b2mr11561673eda.23.1712667695198; Tue, 09 Apr 2024 06:01:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1712667695; cv=none; d=google.com; s=arc-20160816; b=TIy4mjw+kHiDvCxucoCFvuqWD3W5X/g8T2ri9WTLFnAUijw+lBX5LL76BjfwiR8zmS nMzuvt9tBAlN5mvhmRWnTa7+qvfmdn0W8MCppBYvZwGO8ipl9syPVvNIjul6JEx98SwN un1qMFi21ZxoutP4jkktL+py/qfAzW2sG98nSMLCegCeJihtoe+sGfLHx3m1WrJ4GBki XCnvpvSCd3E46ASZtm9U9Wvf1x9EpuIpnYXVHFYivIpdv8siWsJGtfKm+bjleTmnN/AW ivmx5gjrsO2hCpIvE8sPD1POcraYatRseGtQsmZfPc0hHhk9svdggmVWPGgHr8Z/5Hlo A36Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=DrekKxU4nhzfxHvMn4NjWy6IxUYpHgvYCiCeyhRN8HA=; fh=xmAeKtysnShNOmkhiJmYkS30uw4Fu2hvBJ7qlIwukxQ=; b=CfTsOvB5GallZ2q591bOQkv+RK6lqXpHNucn1dT3FfIBJU+g6S+SjFkxCD+E9jiviK tBLMjI5uEdP5AxaN/xjw1VCxU3czX/aKUBMNgzS8OyQcrsBUusAnJO1R0WUBrR6Ey4UT Mfa7Ncls82PUskkUYUVondwCz91cerXQswosGinxOGi8Ve4+IezRbmHJV8WI7FILh4PV w4gtMn8fTceWK+Asub9vmz70w92YSbxWXrYvjsoBcviuEcMV1HFyn/kzpq9oJqMIFkcz 45EcZoUGBw7yDgwwPD9I3NFXOPAmREGmz1mLzlBvdAbxSy8uf/YR/jDsW/HVjWAjEvc+ k5Cg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=l0Tm3zG5; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 18-20020a508752000000b0056e21b42abasi4543272edv.331.2024.04.09.06.01.34; Tue, 09 Apr 2024 06:01:35 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=l0Tm3zG5; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9D3AD68D202; Tue, 9 Apr 2024 15:59:34 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5103B68D193 for ; Tue, 9 Apr 2024 15:59:22 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1712667558; bh=D26IeK/FHDP85BTDRl0SAKE6L40z3IIdUnKmSngg+tQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=l0Tm3zG5Bz0XVkV8Hs+xQ603ZnDfhV+LP3meQUoZopa1I/TekI3VQTR++O0GauKLl gob+ZDbGnwpxj8G1oazXKG9rtSuZDC54Gan/h74JHVp9IN9Mhorl1vAoq54U5ys+Pg idp2xO06PpbJco96kWonaG66kEMqWcKtUutS9NHU= Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id 842124344E; Tue, 9 Apr 2024 14:59:18 +0200 (CEST) From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Tue, 9 Apr 2024 14:57:30 +0200 Message-ID: <20240409125914.61149-11-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240409125914.61149-1-ffmpeg@haasn.xyz> References: <20240409125914.61149-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 10/11] avcodec/libx265: implement dolby vision coding X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: lTRVRoun7qdZ From: Niklas Haas libx265 supports these natively, we just need to attach the generated NALs to the x265picture, as well as setting the appropriate DV profile. --- libavcodec/libx265.c | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/libavcodec/libx265.c b/libavcodec/libx265.c index 0645cd20457..c4ceffff5d3 100644 --- a/libavcodec/libx265.c +++ b/libavcodec/libx265.c @@ -36,6 +36,7 @@ #include "libavutil/pixdesc.h" #include "avcodec.h" #include "codec_internal.h" +#include "dovi_rpu.h" #include "encode.h" #include "packet_internal.h" #include "atsc_a53.h" @@ -78,6 +79,8 @@ typedef struct libx265Context { * encounter a frame with ROI side data. */ int roi_warned; + + DOVIContext dovi; } libx265Context; static int is_keyframe(NalUnitType naltype) @@ -143,6 +146,8 @@ static av_cold int libx265_encode_close(AVCodecContext *avctx) if (ctx->encoder) ctx->api->encoder_close(ctx->encoder); + ff_dovi_ctx_unref(&ctx->dovi); + return 0; } @@ -526,6 +531,14 @@ FF_ENABLE_DEPRECATION_WARNINGS } } +#if X265_BUILD >= 167 + ctx->dovi.logctx = avctx; + if ((ret = ff_dovi_configure(&ctx->dovi, avctx)) < 0) + return ret; + ctx->params->dolbyProfile = ctx->dovi.cfg.dv_profile * 10 + + ctx->dovi.cfg.dv_bl_signal_compatibility_id; +#endif + ctx->encoder = ctx->api->encoder_open(ctx->params); if (!ctx->encoder) { av_log(avctx, AV_LOG_ERROR, "Cannot open libx265 encoder.\n"); @@ -629,6 +642,10 @@ static void free_picture(libx265Context *ctx, x265_picture *pic) for (int i = 0; i < sei->numPayloads; i++) av_free(sei->payloads[i].payload); +#if X265_BUILD >= 167 + av_free(pic->rpu.payload); +#endif + if (pic->userData) { int idx = (int)(intptr_t)pic->userData - 1; rd_release(ctx, idx); @@ -660,6 +677,7 @@ static int libx265_encode_frame(AVCodecContext *avctx, AVPacket *pkt, sei->numPayloads = 0; if (pic) { + AVFrameSideData *sd; ReorderedData *rd; int rd_idx; @@ -760,6 +778,24 @@ static int libx265_encode_frame(AVCodecContext *avctx, AVPacket *pkt, sei->numPayloads++; } } + +#if X265_BUILD >= 167 + sd = av_frame_get_side_data(pic, AV_FRAME_DATA_DOVI_METADATA); + if (ctx->dovi.cfg.dv_profile && sd) { + const AVDOVIMetadata *metadata = (const AVDOVIMetadata *)sd->data; + ret = ff_dovi_rpu_generate(&ctx->dovi, metadata, &x265pic.rpu.payload, + &x265pic.rpu.payloadSize); + if (ret < 0) { + free_picture(ctx, &x265pic); + return ret; + } + } else if (ctx->dovi.cfg.dv_profile) { + av_log(avctx, AV_LOG_ERROR, "Dolby Vision enabled, but received frame " + "without AV_FRAME_DATA_DOVI_METADATA"); + free_picture(ctx, &x265pic); + return AVERROR_INVALIDDATA; + } +#endif } ret = ctx->api->encoder_encode(ctx->encoder, &nal, &nnal, @@ -914,6 +950,10 @@ static const AVOption options[] = { { "udu_sei", "Use user data unregistered SEI if available", OFFSET(udu_sei), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, VE }, { "a53cc", "Use A53 Closed Captions (if available)", OFFSET(a53_cc), AV_OPT_TYPE_BOOL, { .i64 = 1 }, 0, 1, VE }, { "x265-params", "set the x265 configuration using a :-separated list of key=value parameters", OFFSET(x265_opts), AV_OPT_TYPE_DICT, { 0 }, 0, 0, VE }, +#if X265_BUILD >= 167 + { "dolbyvision", "Enable Dolby Vision RPU coding", OFFSET(dovi.enable), AV_OPT_TYPE_BOOL, {.i64 = FF_DOVI_AUTOMATIC }, -1, 1, VE, .unit = "dovi" }, + { "auto", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = FF_DOVI_AUTOMATIC}, .flags = VE, .unit = "dovi" }, +#endif { NULL } }; From patchwork Tue Apr 9 12:57:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 47976 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:9c8d:b0:1a7:a0dc:8de5 with SMTP id mj13csp332870pzb; Tue, 9 Apr 2024 06:00:27 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCVXBpZ9DtSH5LnyEcLE3sUkgmd/qMcJvqDf4kgoaNV8GeA3gZ0+xrtkUvGIsDA9NLBF/CE5y99I33dPfuh2rTxIxp+ZqgteYHUvCQ== X-Google-Smtp-Source: AGHT+IFZkof2b7Ups5RBhwV0Iegq8GmXOkphiddQUV3deH2pbKJqHZEebKN/TTnD49kglEOw/COs X-Received: by 2002:a17:907:da2:b0:a51:ee80:bae9 with SMTP id go34-20020a1709070da200b00a51ee80bae9mr2578154ejc.17.1712667626889; Tue, 09 Apr 2024 06:00:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1712667626; cv=none; d=google.com; s=arc-20160816; b=00OBX7jPKcHysdfKEhNHIpN1eqMUSFUoVgjYm7BhFPoQrIyo/4bTdDQvCtBfdpnuQH vNIwfLoztTOym/Oi9FCGuDBtK/8J9a8fC7W7nS2xTSwmVXfOMTw9/SyAvrZvwwQTbGfM mGFMhqYKQ5Y6sBjYTkqaXNLZD9pDG+vDoaT0+0CJM6YStAjvXTfmo9pCSxFZP0Pm4tU3 SGd87OhH/jI9YDdygfTDFhdyGp8s4XW9Vr3K0+sS7rQiXqse5yJhtCmYI1HErpXVnpXA 6biDzwRnCL655W7yJ4zygBb+i9Fv8SZeAj0tjZOigaDhK5aMj7g9Ww+TxTKhGKaOcp0H eFPg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=zzTt4238m8iKexxudNiZuf4dhOJNwFnctHZmF+HcgV0=; fh=xmAeKtysnShNOmkhiJmYkS30uw4Fu2hvBJ7qlIwukxQ=; b=hg7922pkJ1bJo6ogoQP5ggjJPnE8T8kT7fNGLaoksywHe1bvflCYyAC5r5ZuYgO6oV bRWmoQYzCshqD0ywpQdphoIg7VgURzYDymNotcFuu6fxSDHXsMclXuV62JWgV+VAFn+k GDaPbt5RBnwDAsjXhQMkMq88NlNwESo4UckUvxZgBQU9kEHsc4XePuWme2Vl0zZ5xBRA gw/o1uQioEhMePMOwQlyMECzgOwpzSmo95KAyZjVcLR8RGUV4PbLJ3yisatl+qZUAGnw YfGV/MYDfeTvDm1DCDnrUNF46d3JI+pfUwZsUs7s3GrSNbdb994jrS8xYoYsOo1qCJgI u5ag==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=Pc+qG5Sb; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id dt10-20020a170906b78a00b00a51d6337b52si2015142ejb.910.2024.04.09.06.00.26; Tue, 09 Apr 2024 06:00:26 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=Pc+qG5Sb; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D924D68D1C9; Tue, 9 Apr 2024 15:59:28 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7C01F68D0A4 for ; Tue, 9 Apr 2024 15:59:22 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1712667558; bh=IXMJJ50z0ADaWoB37Y/pzIk8sAAMbLPTdTfRisSC72w=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Pc+qG5SbhxavdLCcuhPym3H16OsxCGwQcMqnE5VtMevS3yhy7EvOyIQ2lIGTIxMIO 7ChMGM1UTtCKWTxzBbpVzhXZK81srwO5u5+Zev+FBUZIeMJMbzjylFqzixo9ZczvE4 W4a0546E7l14B5iByJz92U3cECXMTLd7M3fslmIs= Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id BA36E4367B; Tue, 9 Apr 2024 14:59:18 +0200 (CEST) From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Tue, 9 Apr 2024 14:57:31 +0200 Message-ID: <20240409125914.61149-12-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240409125914.61149-1-ffmpeg@haasn.xyz> References: <20240409125914.61149-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 11/11] avcodec/libsvtav1: implement dolby vision coding X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: IRAekEi7k4fE From: Niklas Haas --- libavcodec/libsvtav1.c | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/libavcodec/libsvtav1.c b/libavcodec/libsvtav1.c index 105c3369c0f..cd62103dba4 100644 --- a/libavcodec/libsvtav1.c +++ b/libavcodec/libsvtav1.c @@ -23,6 +23,7 @@ #include #include #include +#include #include "libavutil/common.h" #include "libavutil/frame.h" @@ -35,6 +36,7 @@ #include "libavutil/avassert.h" #include "codec_internal.h" +#include "dovi_rpu.h" #include "encode.h" #include "packet_internal.h" #include "avcodec.h" @@ -62,6 +64,8 @@ typedef struct SvtContext { EOS_STATUS eos_flag; + DOVIContext dovi; + // User options. AVDictionary *svtav1_opts; int enc_mode; @@ -418,6 +422,7 @@ static int read_in_data(EbSvtAv1EncConfiguration *param, const AVFrame *frame, in_data->cr_stride = AV_CEIL_RSHIFT(frame->linesize[2], bytes_shift); header_ptr->n_filled_len = frame_size; + svt_metadata_array_free(&header_ptr->metadata); return 0; } @@ -451,6 +456,11 @@ static av_cold int eb_enc_init(AVCodecContext *avctx) return svt_print_error(avctx, svt_ret, "Error initializing encoder"); } + svt_enc->dovi.logctx = avctx; + ret = ff_dovi_configure(&svt_enc->dovi, avctx); + if (ret < 0) + return ret; + if (avctx->flags & AV_CODEC_FLAG_GLOBAL_HEADER) { EbBufferHeaderType *headerPtr = NULL; @@ -486,6 +496,7 @@ static int eb_send_frame(AVCodecContext *avctx, const AVFrame *frame) { SvtContext *svt_enc = avctx->priv_data; EbBufferHeaderType *headerPtr = svt_enc->in_buf; + AVFrameSideData *sd; int ret; if (!frame) { @@ -524,6 +535,24 @@ static int eb_send_frame(AVCodecContext *avctx, const AVFrame *frame) if (avctx->gop_size == 1) headerPtr->pic_type = EB_AV1_KEY_PICTURE; + sd = av_frame_get_side_data(frame, AV_FRAME_DATA_DOVI_METADATA); + if (svt_enc->dovi.cfg.dv_profile && sd) { + const AVDOVIMetadata *metadata = (const AVDOVIMetadata *)sd->data; + uint8_t *t35; + int size; + if ((ret = ff_dovi_rpu_generate(&svt_enc->dovi, metadata, &t35, &size)) < 0) + return ret; + ret = svt_add_metadata(headerPtr, EB_AV1_METADATA_TYPE_ITUT_T35, t35, size); + av_free(t35); + if (ret < 0) + return AVERROR(ENOMEM); + } else if (svt_enc->dovi.cfg.dv_profile) { + av_log(avctx, AV_LOG_ERROR, "Dolby Vision enabled, but received frame " + "without AV_FRAME_DATA_DOVI_METADATA\n"); + return AVERROR_INVALIDDATA; + } + + svt_av1_enc_send_picture(svt_enc->svt_handle, headerPtr); return 0; @@ -644,11 +673,13 @@ static av_cold int eb_enc_close(AVCodecContext *avctx) } if (svt_enc->in_buf) { av_free(svt_enc->in_buf->p_buffer); + svt_metadata_array_free(&svt_enc->in_buf->metadata); av_freep(&svt_enc->in_buf); } av_buffer_pool_uninit(&svt_enc->pool); av_frame_free(&svt_enc->frame); + ff_dovi_ctx_unref(&svt_enc->dovi); return 0; } @@ -695,6 +726,9 @@ static const AVOption options[] = { AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 63, VE }, { "svtav1-params", "Set the SVT-AV1 configuration using a :-separated list of key=value parameters", OFFSET(svtav1_opts), AV_OPT_TYPE_DICT, { 0 }, 0, 0, VE }, + { "dolbyvision", "Enable Dolby Vision RPU coding", OFFSET(dovi.enable), AV_OPT_TYPE_BOOL, {.i64 = FF_DOVI_AUTOMATIC }, -1, 1, VE, .unit = "dovi" }, + { "auto", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = FF_DOVI_AUTOMATIC}, .flags = VE, .unit = "dovi" }, + {NULL}, };