From patchwork Mon Nov 1 20:32:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derek Buitenhuis X-Patchwork-Id: 31272 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a5e:a610:0:0:0:0:0 with SMTP id q16csp3117751ioi; Mon, 1 Nov 2021 13:33:15 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxa/lOf4gpJp+9EcqO8y92pN0MB/HEKoTuk1YxETxyPtvZvv32WdKtD8Ks/sE+brN73p/32 X-Received: by 2002:a17:906:f241:: with SMTP id gy1mr37582432ejb.192.1635798795471; Mon, 01 Nov 2021 13:33:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1635798795; cv=none; d=google.com; s=arc-20160816; b=eetZuXcMj79/TIDiv1TUHiNZHuUhFs1P6C70CVZywVZuDTENzlK+wftr1vGEXGq8Z0 0nffpgPd9e6dkrkB8l3au2TEj0njlM+IPQBgHDnPucFcr8JBCHDTwe2ZxulG7pXRHAXn 4crG+W3fUQyuMgtKm31U9koq7/8E7wzFuLr9dFGCx1RPjG0Db4Vju3VJoP7QnbfwJY20 utkvMFSX/Vl5i48cKcyDEag3eN6ME6hz6iGmDBQgKSz+2S3+U8gdrppll60rhdkzX0/u /tMdNZBmXbQxn3NwlC4Bw3CKxEvF3RE+17Y5UttQgOlXldxbEnNXoL4doibaIiHXN3Px cJUg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=lJUMK1wVWK6qKLzSr6enNyia5XN550Lb0+9js9W6f9M=; b=O+7tcLA1W6vr6BkcPoBtCF8yTtB8jgbR/jrCIYurkawC0ZJdNW4mfkfk8y2pWZGCwb a2hzR7wv/zw9wrAO8zUjnwFcvEFQ4GQhSS2pN7CSRuwOyz9n/N+0ha2se0ZDIXz/1d/J SHSOlum7niPMS8RSbIFYqoaeyj9ebNQ33gGbPAJkJlk6DBdBQM+x5ZDxUaP5IWivO1IN BwFS8MHF+ki0pcJb4rLb2MfpwYOCuVg6CsMge7UM9KeejM1DyBp+UMDsCFmHGqo/yuua NIqTBkfNBQaggKn+c2zyTH+3wIkwZx81VFXiP5Ma1k7Nc1sdhjkTBNNoXZ0g8zbF2sdI YgAQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=D2bNzVnI; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id l2si29068186ejo.671.2021.11.01.13.33.15; Mon, 01 Nov 2021 13:33:15 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=D2bNzVnI; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id CF18E68A8A3; Mon, 1 Nov 2021 22:32:50 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f47.google.com (mail-wr1-f47.google.com [209.85.221.47]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9375368A778 for ; Mon, 1 Nov 2021 22:32:43 +0200 (EET) Received: by mail-wr1-f47.google.com with SMTP id d27so11926130wrb.6 for ; Mon, 01 Nov 2021 13:32:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Ff/J4Fuf5gCtK5PuS4sHqvbAGog/r//bLYFmHxDGQ6s=; b=D2bNzVnImocrpVamzTO5wJ18kaZrwtVPdfBbbLyB+W609ldXdxFx4+ZSI37SD07fZq Ty7Hw70kg3HvgFGpqjXdLiwMA/M7SBRZY/ER5i/UXLaUwMtXvq2bLRyL0kVuX1v5lwqQ THM5DfZVOim7o5zMjUrTe1B+xjSbW2hovJXoM22fSLQ+EF71uvcUBGULWFdVf7/XsUUz /f/3NAzMykTl8Nr/V/+YiPhuaSMejdWn+i3aaMFDFkBgbxZPQeFWBn7VzeWoV1GrBES9 wakV5wYTiGURCZgNRW+sjFikNfjSveq1wrUBplrqGROMmnP+d0PGJXuMj4g6413fnOqj 4HRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Ff/J4Fuf5gCtK5PuS4sHqvbAGog/r//bLYFmHxDGQ6s=; b=hfSVdafaf67/jgAdie49iKgBxD+YoY8WnXjijqM7mRxlmLwwe3u4l5QnxA/v1TR8hc G+wTL3MCkOzbPQUXnx2EolBbmkFTjxw1c1RweNp+mCP4CJzeCMLtUnfq54iMf4Y3ETgK O0jJtzvzNISr03fIAcT38t4WnHIlVP6RAzAQHDU+bzZk2dKDregu5wokZjnWcRTi1/aA Ct9vKFT4V571O5ltVGxI7ENZRdJAUkuRQqKx5SaNk/d2696Gh6bTA0ugEu3nwqrq6KrY P2jjwOnzT67r5K6jiSRtLETij+mppfDoyorTgaAxJPCj4DcVrGL6P+w59C588Dy3W0NU rdEg== X-Gm-Message-State: AOAM5333VCfeCCbd4Vb8nYCZXFa1LLlEhqrhZRXiA0kdKgmkYo+LHANN 3CJM0HmiPeE+oLKT1R+crFMlb6hK+MY= X-Received: by 2002:a5d:680b:: with SMTP id w11mr33915846wru.345.1635798762754; Mon, 01 Nov 2021 13:32:42 -0700 (PDT) Received: from localhost.localdomain ([82.129.92.120]) by smtp.gmail.com with ESMTPSA id z15sm3801296wrr.65.2021.11.01.13.32.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Nov 2021 13:32:42 -0700 (PDT) From: Derek Buitenhuis To: ffmpeg-devel@ffmpeg.org Date: Mon, 1 Nov 2021 20:32:26 +0000 Message-Id: <20211101203227.1648805-3-derek.buitenhuis@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211101203227.1648805-1-derek.buitenhuis@gmail.com> References: <20211101203227.1648805-1-derek.buitenhuis@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/3 v2] avcodec/hevcdec: Export Dolby Vision RPUs as side data X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Io4eqvanj5kv Signed-off-by: Derek Buitenhuis --- libavcodec/h2645_parse.c | 28 ++++++++++++++++++++++++++++ libavcodec/hevcdec.c | 29 +++++++++++++++++++++++++++++ libavcodec/hevcdec.h | 2 ++ libavcodec/version.h | 2 +- 4 files changed, 60 insertions(+), 1 deletion(-) diff --git a/libavcodec/h2645_parse.c b/libavcodec/h2645_parse.c index 6fbe97ad4a..7adf5b7ef1 100644 --- a/libavcodec/h2645_parse.c +++ b/libavcodec/h2645_parse.c @@ -517,6 +517,34 @@ int ff_h2645_packet_split(H2645Packet *pkt, const uint8_t *buf, int length, pkt->nb_nals++; } + /* + * Due to limitions in avcodec's current frame threading code, it cannot + * handle adding frame side data in any place except before the slice has + * started decoding. Since Dolby Vision RPUs (which appear as NAL type 62) + * are appended to the AU, this is a poblematic. As a hack around this, we + * move any RPUs to before the first slice NAL. + */ + if (codec_id == AV_CODEC_ID_HEVC && pkt->nb_nals > 1 && pkt->nals[pkt->nb_nals - 1].type == HEVC_NAL_UNSPEC62 && + !pkt->nals[pkt->nb_nals - 1].nuh_layer_id && !pkt->nals[pkt->nb_nals - 1].temporal_id) { + + int first_non_slice = 0; + H2645NAL *tmp = av_malloc(pkt->nb_nals * sizeof(H2645NAL)); + if (!tmp) + return AVERROR(ENOMEM); + + for (int i = pkt->nb_nals - 1; i >= 0; i--) { + if (pkt->nals[i].type < HEVC_NAL_VPS) + first_non_slice = i; + tmp[i] = pkt->nals[i]; + } + + pkt->nals[first_non_slice] = pkt->nals[pkt->nb_nals - 1]; + for (int i = first_non_slice + 1; i < pkt->nb_nals; i++) + pkt->nals[i] = tmp[i - 1]; + + av_free(tmp); + } + return 0; } diff --git a/libavcodec/hevcdec.c b/libavcodec/hevcdec.c index 246ffd7d80..dd286deaa7 100644 --- a/libavcodec/hevcdec.c +++ b/libavcodec/hevcdec.c @@ -2950,6 +2950,14 @@ static int set_side_data(HEVCContext *s) } } + if (s->rpu_buf) { + AVFrameSideData *rpu = av_frame_new_side_data_from_buf(out, AV_FRAME_DATA_DOVI_RPU_BUFFER, s->rpu_buf); + if (!rpu) + return AVERROR(ENOMEM); + + s->rpu_buf = NULL; + } + return 0; } @@ -3224,6 +3232,20 @@ static int decode_nal_unit(HEVCContext *s, const H2645NAL *nal) case HEVC_NAL_AUD: case HEVC_NAL_FD_NUT: break; + case HEVC_NAL_UNSPEC62: + /* + * Check for RPU delimiter. + * + * Dolby Vision RPUs masquerade as unregistered NALs of type 62. + */ + if (nal->size > 2 && !nal->nuh_layer_id && !nal->temporal_id) { + s->rpu_buf = av_buffer_alloc(nal->raw_size - 2); + if (!s->rpu_buf) + return AVERROR(ENOMEM); + + memcpy(s->rpu_buf->data, nal->raw_data + 2, nal->raw_size - 2); + } + break; default: av_log(s->avctx, AV_LOG_INFO, "Skipping NAL unit %d\n", s->nal_unit_type); @@ -3512,6 +3534,8 @@ static av_cold int hevc_decode_free(AVCodecContext *avctx) pic_arrays_free(s); + av_buffer_unref(&s->rpu_buf); + av_freep(&s->md5_ctx); av_freep(&s->cabac_state); @@ -3698,6 +3722,10 @@ static int hevc_update_thread_context(AVCodecContext *dst, if (ret < 0) return ret; + ret = av_buffer_replace(&s->rpu_buf, s0->rpu_buf); + if (ret < 0) + return ret; + s->sei.frame_packing = s0->sei.frame_packing; s->sei.display_orientation = s0->sei.display_orientation; s->sei.mastering_display = s0->sei.mastering_display; @@ -3754,6 +3782,7 @@ static void hevc_decode_flush(AVCodecContext *avctx) HEVCContext *s = avctx->priv_data; ff_hevc_flush_dpb(s); ff_hevc_reset_sei(&s->sei); + av_buffer_unref(&s->rpu_buf); s->max_ra = INT_MAX; s->eos = 1; } diff --git a/libavcodec/hevcdec.h b/libavcodec/hevcdec.h index 77fdf90da1..870ff178d4 100644 --- a/libavcodec/hevcdec.h +++ b/libavcodec/hevcdec.h @@ -572,6 +572,8 @@ typedef struct HEVCContext { int nal_length_size; ///< Number of bytes used for nal length (1, 2 or 4) int nuh_layer_id; + + AVBufferRef *rpu_buf; ///< 0 or 1 Dolby Vision RPUs. } HEVCContext; /** diff --git a/libavcodec/version.h b/libavcodec/version.h index 74b8baa5f3..76af066d32 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -28,7 +28,7 @@ #include "libavutil/version.h" #define LIBAVCODEC_VERSION_MAJOR 59 -#define LIBAVCODEC_VERSION_MINOR 12 +#define LIBAVCODEC_VERSION_MINOR 13 #define LIBAVCODEC_VERSION_MICRO 100 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \