From patchwork Thu Oct 21 13:48:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derek Buitenhuis X-Patchwork-Id: 31200 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2084:0:0:0:0 with SMTP id a4csp625181ioa; Thu, 21 Oct 2021 06:49:17 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyn30Um5aAWMEMLKcqv4plHGNKGzDUsnHa0liwWp94X3Tb5MWtGwlGZkGr/acRynv53J9IJ X-Received: by 2002:aa7:d54e:: with SMTP id u14mr7884112edr.2.1634824157467; Thu, 21 Oct 2021 06:49:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1634824157; cv=none; d=google.com; s=arc-20160816; b=HbpSC5NaOa6C0OiryeNFklqH8jrz+qvaE+m5k5HzAtmAeU1luSlO/NlpbTp8QBttHH /01uvihrXGuQLW6/8UrJ6S9kHzpklxIgJq7JRn9kYNdM4GLVoeJVWZJc96p9Z6Phe100 ANYp+ij6aRLFW3ZTrjXngM/1T+q7mdleeeXG+9mXMVBsktVfpjuPgVbdbbA2n1w4pd/i +rRo7T0e87ekTYKD26xWr8KKL5ckvN+ReQdhNks1RLPrzS4fQuXWa0zE8gI0Qp+2K7lm Ew9BC3SmQEKo1osJ1erI2KpzWDMaEoq0VCMj0nR3lpbVcqADKfdoFNP8h1g/209V5vru r+zA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=YBGMK+Tsu3aksyYO0HwbuKV16GKYiDjpi7qVgoa4Oyk=; b=fKboy5do+DTdeVQz4G246NQ668V7QuwksjuBzllnuoIfXAyL2Ys3vdHNp6IfLmnkRJ 2F/nu43sM3RzX+LT9GkT7mJtRY/oYD3NTaMsXrSggP8m3RQkguKYBh1OIdEd2J4ZYXMV bOgZSsHPv+ltkjeitnxXLRvekENqezJMQfS/KRSofEYdtCuIRy8NSmFOQYjYsR2ZdfPr KfbuStA/ZbdUjBsinC12FNTOAS7wJd2wXJdvT2lVlJx3O8ALc+GfaSDVuRk845C0RpU9 UQpppq2cmMaKWBXCQRfFMJDXjzPDyaLYzXi/EnRpk8qKehvob6O3z5notcd62+L17RXY BdYw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=lDvvfE64; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id qw28si10653166ejc.645.2021.10.21.06.49.17; Thu, 21 Oct 2021 06:49:17 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=lDvvfE64; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5E5276899A7; Thu, 21 Oct 2021 16:49:09 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f46.google.com (mail-wm1-f46.google.com [209.85.128.46]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 772376807C5 for ; Thu, 21 Oct 2021 16:49:02 +0300 (EEST) Received: by mail-wm1-f46.google.com with SMTP id 67-20020a1c1946000000b0030d4c90fa87so790521wmz.2 for ; Thu, 21 Oct 2021 06:49:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=BxnUazshFHigBrhW+qTAn12j6ZOPlw/MNP8wPx0Z9Xc=; b=lDvvfE64ZkIfMXtp0oMhLGbyuuL17mrGmRVwfNhOifcWsEgAAklM/gmtq7DfmwqDJe HK0AT4yoDUhfgPfQN+ZN6YSbyJremSHq+Vi2lP9oH9gOYvUyDM7v3UJ4f6L8As9FLb8k XfxxrPuNpHdlvgil1XYQGgOZiwUcfkG42OxYEXYmwbVwocnFHjDHH3xK/8Tr8HKN/iCD +bboYdnKRnb3CUdb/WaFsVtzGijM/TtAmjRR126HWFvf1TgdEJWwCiJWoV0MTANz0QZh AZ0cOX1cby3wSfmvF4BlWVuHtKWpJ4V0VhgoADF79Mv8uDYzpIqydhWkZsenWRT7W2oe hjpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BxnUazshFHigBrhW+qTAn12j6ZOPlw/MNP8wPx0Z9Xc=; b=wA1hTdJSEGDDSQm5UVeasNOnsA7wEbK2oB+O4ttMqPfeELxIJ/TWtbzLBSstFXKcFB U2qC+PxPv5WFaCmexUkduaKCZoDEjXIx0gZWg4gE/lhk2OZvlXN9peIl9j3LE90NQwoD yWd9hSX8QJD9vw9xYDdU4z/fhs5a0qyRErabPknfHvXDKA8hck2XPpgs4JWe80tyZtYc zLWFQeQ27e4IQDJ61zhVpetluPTR9bOLH2iF1AcAGaC+WMRLAcP4YHwsqlE2gG2pyW7v xiL6ZIw++CfRCECTLGg3UHzA3iUNbWsd5oqugdhRQMiCdVoKsiRyvfsuSdgF0rw5bZmE cwIg== X-Gm-Message-State: AOAM531cycYkJJqRJHNtdShdS4qmvSNIa7bfwM0Ic7yCD3O2xZyb5gk2 mWQBydhahGNfYONiDMnpgOTffZGtIKM= X-Received: by 2002:a1c:4302:: with SMTP id q2mr21847159wma.133.1634824134662; Thu, 21 Oct 2021 06:48:54 -0700 (PDT) Received: from localhost.localdomain ([82.129.92.120]) by smtp.gmail.com with ESMTPSA id x21sm7938408wmc.14.2021.10.21.06.48.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Oct 2021 06:48:54 -0700 (PDT) From: Derek Buitenhuis To: ffmpeg-devel@ffmpeg.org Date: Thu, 21 Oct 2021 14:48:42 +0100 Message-Id: <20211021134843.1436241-3-derek.buitenhuis@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211021134843.1436241-1-derek.buitenhuis@gmail.com> References: <20211021134843.1436241-1-derek.buitenhuis@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/3] avcodec/hevcdec: Export Dolby Vision RPUs as side data X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: WL2VZwkeSOnU Signed-off-by: Derek Buitenhuis --- * The NAL reordering approach is a result of discussions with Anton and James a few months ago. * I've put the NAL reordering in ff_h2645_packet_split, rather than a bitstream filter for a few reasons: * I don't think having a decoder's behavior rely on the presence of a bitstream filter is good architecture / design. * This spliting is only usd or useful to decoding. --- libavcodec/h2645_parse.c | 28 ++++++++++++++++++++++++++++ libavcodec/hevcdec.c | 32 ++++++++++++++++++++++++++++++++ libavcodec/hevcdec.h | 3 +++ libavcodec/version.h | 2 +- 4 files changed, 64 insertions(+), 1 deletion(-) diff --git a/libavcodec/h2645_parse.c b/libavcodec/h2645_parse.c index 6fbe97ad4a..04348c3a73 100644 --- a/libavcodec/h2645_parse.c +++ b/libavcodec/h2645_parse.c @@ -517,6 +517,34 @@ int ff_h2645_packet_split(H2645Packet *pkt, const uint8_t *buf, int length, pkt->nb_nals++; } + /* + * Due to limitions in avcodec's current frame threading code, it cannot + * handle adding frame side data in any place except before the slice has + * started decoding. Since Dolby Vision RPUs (which appear as NAL type 62) + * are appended to the AU, this is a poblematic. As a hack around this, we + * move any RPUs to before the first slice NAL. + */ + if (codec_id == AV_CODEC_ID_HEVC && pkt->nb_nals > 1 && pkt->nals[pkt->nb_nals - 1].type == HEVC_NAL_UNSPEC62 && + pkt->nals[pkt->nb_nals - 1].data[0] == 0x7C && pkt->nals[pkt->nb_nals - 1].data[1] == 0x01) { + + int first_non_slice = 0; + H2645NAL *tmp = av_malloc(pkt->nb_nals * sizeof(H2645NAL)); + if (!tmp) + return AVERROR(ENOMEM); + + for (int i = pkt->nb_nals - 1; i >= 0; i--) { + if (pkt->nals[i].type < HEVC_NAL_VPS) + first_non_slice = i; + tmp[i] = pkt->nals[i]; + } + + pkt->nals[first_non_slice] = pkt->nals[pkt->nb_nals - 1]; + for (int i = first_non_slice + 1; i < pkt->nb_nals; i++) + pkt->nals[i] = tmp[i - 1]; + + av_free(tmp); + } + return 0; } diff --git a/libavcodec/hevcdec.c b/libavcodec/hevcdec.c index 246ffd7d80..5ccee2aa4e 100644 --- a/libavcodec/hevcdec.c +++ b/libavcodec/hevcdec.c @@ -2950,6 +2950,17 @@ static int set_side_data(HEVCContext *s) } } + if (s->rpu_buf) { + AVFrameSideData *rpu = av_frame_new_side_data(out, AV_FRAME_DATA_DOVI_RPU, s->rpu_buf_size); + if (!rpu) + return AVERROR(ENOMEM); + + memcpy(rpu->data, s->rpu_buf, s->rpu_buf_size); + + av_freep(&s->rpu_buf); + s->rpu_buf_size = 0; + } + return 0; } @@ -3224,6 +3235,23 @@ static int decode_nal_unit(HEVCContext *s, const H2645NAL *nal) case HEVC_NAL_AUD: case HEVC_NAL_FD_NUT: break; + case HEVC_NAL_UNSPEC62: + /* + * Check for RPU delimiter. + * + * Dolby Vision RPUs masquerade as unregistered NALs of type 62 and start with + * 0x7C01. + */ + if (nal->size > 2 && nal->data[0] == 0x7C && nal->data[1] == 0x01) { + s->rpu_buf = av_malloc(nal->raw_size - 2); + if (!s->rpu_buf) + return AVERROR(ENOMEM); + + memcpy(s->rpu_buf, nal->raw_data + 2, nal->raw_size - 2); + + s->rpu_buf_size = nal->raw_size - 2; + } + break; default: av_log(s->avctx, AV_LOG_INFO, "Skipping NAL unit %d\n", s->nal_unit_type); @@ -3512,6 +3540,8 @@ static av_cold int hevc_decode_free(AVCodecContext *avctx) pic_arrays_free(s); + av_freep(&s->rpu_buf); + av_freep(&s->md5_ctx); av_freep(&s->cabac_state); @@ -3754,6 +3784,8 @@ static void hevc_decode_flush(AVCodecContext *avctx) HEVCContext *s = avctx->priv_data; ff_hevc_flush_dpb(s); ff_hevc_reset_sei(&s->sei); + av_freep(&s->rpu_buf); + s->rpu_buf_size = 0; s->max_ra = INT_MAX; s->eos = 1; } diff --git a/libavcodec/hevcdec.h b/libavcodec/hevcdec.h index 77fdf90da1..8a9b516759 100644 --- a/libavcodec/hevcdec.h +++ b/libavcodec/hevcdec.h @@ -572,6 +572,9 @@ typedef struct HEVCContext { int nal_length_size; ///< Number of bytes used for nal length (1, 2 or 4) int nuh_layer_id; + + uint8_t *rpu_buf; ///< 0 or 1 Dolby Vision RPUs. + size_t rpu_buf_size; } HEVCContext; /** diff --git a/libavcodec/version.h b/libavcodec/version.h index 74b8baa5f3..76af066d32 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -28,7 +28,7 @@ #include "libavutil/version.h" #define LIBAVCODEC_VERSION_MAJOR 59 -#define LIBAVCODEC_VERSION_MINOR 12 +#define LIBAVCODEC_VERSION_MINOR 13 #define LIBAVCODEC_VERSION_MICRO 100 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \