From patchwork Wed Nov 10 14:22:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derek Buitenhuis X-Patchwork-Id: 31362 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:d206:0:0:0:0:0 with SMTP id q6csp504932iob; Wed, 10 Nov 2021 06:23:24 -0800 (PST) X-Google-Smtp-Source: ABdhPJwMrGoccTHLHL7gOK6XG7JQcRzqiNlpYrRm0xPQXhcEjFnR7pIUl8/oaoXq1HfWJZsRpMxw X-Received: by 2002:a17:906:6b81:: with SMTP id l1mr20872053ejr.479.1636554203961; Wed, 10 Nov 2021 06:23:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1636554203; cv=none; d=google.com; s=arc-20160816; b=V1j7kHqz5QH0kzoX56icP6qD90OdQ4kzNoQurk1B3K0MiqzAKY7s9RPzjcvTWLHITj R78V50q6W+jkVQD/3MI09P8gVy96Uz59WA/HCfMApkzFjzUI3yAVBrad4ZmWOiDYTuQK 2ts+Mf6pIqT99cSC1o3gRbd38Aojv2KRMwdn7Z53OhuiFC0j8B4xgOIEmYqlYTXwNO5n tUsyLXcK8aPr5N7XPxZzY1ut4GoON3zBAEgYf1ihC/F65EmMyrEq1vC9MJHoLyqCzXzk QvsoCrNN9Sk3xIpoZGlubwaKUAfbwO0sEme9n9gddhuVwOAdvjhoZHpn3L1dxAfBIFy0 LwMw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=mGgwJKHm2sxHPRBu6GW7Flj805G6kF9aGKHst4Q70CM=; b=b11TULZejngxdwCg0AoSgKqloepZP5CzkTSqATvCYKA5GDl8rqh8/AegwZAdjF2W8q dZI3bDbdCvrPDfHoH3+9YkXFU543zmdUJIEOkB5QsHgjTfd6nyfrhwBM6CbrTpOC0qMs 6nEEuWBri6ZccZi4RBZnz3UVFr79rxdyajGozqNxsy+tlOkC2cNMiuhTNOX9c03M5U3N vAss2/qmOfYeyMFVZw05F/1H0HjrctCqBSHe4s8y/gR8WzZMA8IIYiOBQPhSZigNe1pL 6sM9ZwlPYNM5x02HTCmE0owsakDeNkezhbhDeHX5rucuxWSJDJPHIkPiCxzoUL/+sIBJ ae/A== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=KKetx3EJ; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id go40si95406ejc.27.2021.11.10.06.23.20; Wed, 10 Nov 2021 06:23:23 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=KKetx3EJ; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id CAC5C68A993; Wed, 10 Nov 2021 16:22:56 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f45.google.com (mail-wr1-f45.google.com [209.85.221.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id BBC1D68A94A for ; Wed, 10 Nov 2021 16:22:49 +0200 (EET) Received: by mail-wr1-f45.google.com with SMTP id u1so4267406wru.13 for ; Wed, 10 Nov 2021 06:22:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=5KYqghl3+Y70hW/3o3Efm/jzDjpjHnTzvmFEOjdrdI8=; b=KKetx3EJQgRjPAqJrk5gx0XXCJMGv+e2eQaBNzaZsD4mG3OSv3LSWewfZ/OHfxWWmU Npa7v1CKlPvJBg9fjozxu1bdzkA3P/Ii0y4wFp4xd2ohgTa/R/m3dSI2QbuYwtsEaDTn gay6GEUEXp0QLMiKnyzyfdMzbe8qKpMFd/q/Lp3aZ2YsabBoHPVqNISIJNe9wlgq/viR VRn+KGOzkE4cOW2DmBjrUsLeUr+lDmPr8rYhXQ8lzaca1IfjxvvdKwuKkv+C50OSKm+N P34oSOtN9GKDQKJATvQkxYg9Qk4MyZsoAo2hSAGfpo46FpA6zr3V8uddt9bjsRsqFH7L nBrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5KYqghl3+Y70hW/3o3Efm/jzDjpjHnTzvmFEOjdrdI8=; b=3KLbOOB2W+bTh4yOTCEKKvGVp4+ttfF74HiZD1Hn1p1Ndq9sTDTM4RomQVWmvIvrVu ivmt7C84DwG0NFx4hFi0djrX+R7pw0UA0XSdcMDaIWNxeXKeWRoTRX1wbjrtWw3Z/bej SJu61xyY5BrNmADqeYqDiDH+AYbrMvHccd9vbp7WUZILkJMrdDsACuuzF58mtBuQI/pu +SyWnIH+CR6EWTOrc3Fdzvjyb4patU1rMIhXH3r7Q0xaBIATZW/H4QRpUDH+FyTYhwzf tuXrltfIK8Fr/nF0VPCkKRuc3717ENnb9oPxybpdW79gg4V4NFJwS9jo/LgOiXQbXuPh nf7A== X-Gm-Message-State: AOAM531tResDJuCK1Ljl2rHX8nufOWJCWzoI+PqEmsDU/FJKIs1KAbaA sOwtSzOnUvuuPCJ22BwWdKfy3JAEfB8= X-Received: by 2002:adf:f209:: with SMTP id p9mr93172wro.191.1636554168950; Wed, 10 Nov 2021 06:22:48 -0800 (PST) Received: from localhost.localdomain ([82.129.92.120]) by smtp.gmail.com with ESMTPSA id h27sm6644236wmc.43.2021.11.10.06.22.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Nov 2021 06:22:48 -0800 (PST) From: Derek Buitenhuis To: ffmpeg-devel@ffmpeg.org Date: Wed, 10 Nov 2021 14:22:36 +0000 Message-Id: <20211110142237.1667677-3-derek.buitenhuis@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211110142237.1667677-1-derek.buitenhuis@gmail.com> References: <20211110142237.1667677-1-derek.buitenhuis@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3 2/3] avcodec/hevcdec: Export Dolby Vision RPUs as side data X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 9l+Fr2lEuEN9 Signed-off-by: Derek Buitenhuis --- libavcodec/hevcdec.c | 39 +++++++++++++++++++++++++++++++++++++++ libavcodec/hevcdec.h | 2 ++ libavcodec/version.h | 2 +- 3 files changed, 42 insertions(+), 1 deletion(-) diff --git a/libavcodec/hevcdec.c b/libavcodec/hevcdec.c index 246ffd7d80..3e19f4ebed 100644 --- a/libavcodec/hevcdec.c +++ b/libavcodec/hevcdec.c @@ -2950,6 +2950,14 @@ static int set_side_data(HEVCContext *s) } } + if (s->rpu_buf) { + AVFrameSideData *rpu = av_frame_new_side_data_from_buf(out, AV_FRAME_DATA_DOVI_RPU_BUFFER, s->rpu_buf); + if (!rpu) + return AVERROR(ENOMEM); + + s->rpu_buf = NULL; + } + return 0; } @@ -3223,6 +3231,7 @@ static int decode_nal_unit(HEVCContext *s, const H2645NAL *nal) break; case HEVC_NAL_AUD: case HEVC_NAL_FD_NUT: + case HEVC_NAL_UNSPEC62: break; default: av_log(s->avctx, AV_LOG_INFO, @@ -3269,6 +3278,29 @@ static int decode_nal_units(HEVCContext *s, const uint8_t *buf, int length) } } + /* + * Check for RPU delimiter. + * + * Dolby Vision RPUs masquerade as unregistered NALs of type 62. + * + * We have to do this check here an create the rpu buffer, since RPUs are appended + * to the end of an AU; they are the last non-EOB/EOS NAL in the AU. + */ + if (s->pkt.nb_nals > 1 && s->pkt.nals[s->pkt.nb_nals - 1].type == HEVC_NAL_UNSPEC62 && + s->pkt.nals[s->pkt.nb_nals - 1].size > 2 && !s->pkt.nals[s->pkt.nb_nals - 1].nuh_layer_id + && !s->pkt.nals[s->pkt.nb_nals - 1].temporal_id) { + if (s->rpu_buf) { + av_buffer_unref(&s->rpu_buf); + av_log(s->avctx, AV_LOG_WARNING, "Multiple Dolby Vision RPUs found in one AU. Skipping previous.\n"); + } + + s->rpu_buf = av_buffer_alloc(s->pkt.nals[s->pkt.nb_nals - 1].raw_size - 2); + if (!s->rpu_buf) + return AVERROR(ENOMEM); + + memcpy(s->rpu_buf->data, s->pkt.nals[s->pkt.nb_nals - 1].raw_data + 2, s->pkt.nals[s->pkt.nb_nals - 1].raw_size - 2); + } + /* decode the NAL units */ for (i = 0; i < s->pkt.nb_nals; i++) { H2645NAL *nal = &s->pkt.nals[i]; @@ -3512,6 +3544,8 @@ static av_cold int hevc_decode_free(AVCodecContext *avctx) pic_arrays_free(s); + av_buffer_unref(&s->rpu_buf); + av_freep(&s->md5_ctx); av_freep(&s->cabac_state); @@ -3698,6 +3732,10 @@ static int hevc_update_thread_context(AVCodecContext *dst, if (ret < 0) return ret; + ret = av_buffer_replace(&s->rpu_buf, s0->rpu_buf); + if (ret < 0) + return ret; + s->sei.frame_packing = s0->sei.frame_packing; s->sei.display_orientation = s0->sei.display_orientation; s->sei.mastering_display = s0->sei.mastering_display; @@ -3754,6 +3792,7 @@ static void hevc_decode_flush(AVCodecContext *avctx) HEVCContext *s = avctx->priv_data; ff_hevc_flush_dpb(s); ff_hevc_reset_sei(&s->sei); + av_buffer_unref(&s->rpu_buf); s->max_ra = INT_MAX; s->eos = 1; } diff --git a/libavcodec/hevcdec.h b/libavcodec/hevcdec.h index 77fdf90da1..870ff178d4 100644 --- a/libavcodec/hevcdec.h +++ b/libavcodec/hevcdec.h @@ -572,6 +572,8 @@ typedef struct HEVCContext { int nal_length_size; ///< Number of bytes used for nal length (1, 2 or 4) int nuh_layer_id; + + AVBufferRef *rpu_buf; ///< 0 or 1 Dolby Vision RPUs. } HEVCContext; /** diff --git a/libavcodec/version.h b/libavcodec/version.h index 74b8baa5f3..76af066d32 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -28,7 +28,7 @@ #include "libavutil/version.h" #define LIBAVCODEC_VERSION_MAJOR 59 -#define LIBAVCODEC_VERSION_MINOR 12 +#define LIBAVCODEC_VERSION_MINOR 13 #define LIBAVCODEC_VERSION_MICRO 100 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \