From patchwork Thu Sep 19 04:43:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: yoonjoo X-Patchwork-Id: 51649 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:d154:0:b0:48e:c0f8:d0de with SMTP id bt20csp68917vqb; Wed, 18 Sep 2024 21:54:14 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCVgdb1WREvMBgv0tOvPFQDXRV82AskIlYyp4FrYUBPMnN8KgsC5Yi/bpTyuI/2zC2ELH8EeEAkmMEMwr6IRAXt3@gmail.com X-Google-Smtp-Source: AGHT+IFeOl4ELleBQS9ynGGbOktf35okR7KKVtLnJpc54v2d4wOOotK1pjeR6c7YRpI7UsahzMz2 X-Received: by 2002:a2e:a58a:0:b0:2f6:6101:5a6c with SMTP id 38308e7fff4ca-2f787da4fcdmr123493631fa.5.1726721653774; Wed, 18 Sep 2024 21:54:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1726721653; cv=none; d=google.com; s=arc-20240605; b=c7Et/e/y8UzWqDUH1y/YRpJkxOEpP3WIbcPiVZT+mOQJ2LuEAXLDICjSxY6EC3vd65 oy/dvdhkCZch0f4Hck41ccLNwjtxOLWj81APeGoj+q3NTVna9K9fYqzWjuQV8RBvuy9C 2UI3blH7NxL1EXKvsgvMGYdo9G8RA8hyin9jF2Us2vzujW4Y1zkCkZSksd873AMigXw0 ujI2vQlb/qkH8k5f34QKwKhk1hJlZwgUfpVl+0D6DMEryMmvpD/xWlzAu8ZkiDfdoqBB Z98qDIEqeL1LfWvme5PpM4uKMEoUqGjRAEy4S+4d2HawX5Ll2tg7sxI5XEXzGR4BRCt9 8zqQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:references:cms-type:content-language :thread-index:mime-version:message-id:date:to:from:dkim-signature :dkim-filter:delivered-to; bh=Cysa5sEJgH/1wnx34HPLWchmTYQPbuhO1vRrAwvg5us=; fh=5IeVwzS1vbVKjIV8MP3mnmnRtZGb8uteQ9r4QD2keV0=; b=KLcatmQNPP2Zz2POfHCQ8iR0bycXw2gLQFE2ewSp4K0rTVc9V8t1JqMdOG5zeDg2oC pHOUXEEBes/MCLbFq7g8dADnUQrJXUr7AYojXMBErAiwwWohy2N2a1D83VZfAheesoaz +bxRYnigofOepIXk/cF3ZVwTz+ENzLTdQAihEyNW6L+TwOxGzdKSkQ93lR491MNmCbuk SAhRL7EhCqxXpif+zDE9BP8YRdmVM7FTGd08Y6SQ16AwGcuFmEg3yFjqLXncN0V6doNs klh0w/dpYQPE36SPzBcui9SZQtShoW/rBQqbOzgK1oOYWG/UFR3+tQhSRVUaRDSHuqrf O+XA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@samsung.com header.s=mail20170921 header.b=KzBwd5H0; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=samsung.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 38308e7fff4ca-2f79d37952fsi32914201fa.428.2024.09.18.21.54.12; Wed, 18 Sep 2024 21:54:13 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@samsung.com header.s=mail20170921 header.b=KzBwd5H0; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=samsung.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7E8D568DCCA; Thu, 19 Sep 2024 07:43:45 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mailout4.samsung.com (mailout4.samsung.com [203.254.224.34]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E994B68D96B for ; Thu, 19 Sep 2024 07:43:37 +0300 (EEST) Received: from epcas1p1.samsung.com (unknown [182.195.41.45]) by mailout4.samsung.com (KnoxPortal) with ESMTP id 20240919044333epoutp04fbc93c36898ef0433060264c4b5de9b6~2i5zoAgdT1304113041epoutp04Y for ; Thu, 19 Sep 2024 04:43:33 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout4.samsung.com 20240919044333epoutp04fbc93c36898ef0433060264c4b5de9b6~2i5zoAgdT1304113041epoutp04Y DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1726721013; bh=JnhPQCxjSxKSkVZKUZL32g3Vt+9/hrAvlOa6Lg+d1Fg=; h=From:To:Subject:Date:References:From; b=KzBwd5H04uYVWipjtbrzTHvUm3t092umOL2Hexs7JiELN7/VQCuYHOSHUcNs9L8XD P/2pDwv00+sDgnKpvygDSUumMb842MKc6325VRg6gpd25YuXUaPZcjDEvlPRdlwucY spdJ9XwhqbUGJaNXtQktd4rsQ2meEPMTzDbJZUcA= Received: from epsmges1p5.samsung.com (unknown [182.195.42.57]) by epcas1p3.samsung.com (KnoxPortal) with ESMTP id 20240919044332epcas1p3a614a474b7779db42505dd0d64c249a5~2i5zSHd1S0363403634epcas1p3H for ; Thu, 19 Sep 2024 04:43:32 +0000 (GMT) Received: from epcas1p4.samsung.com ( [182.195.41.48]) by epsmges1p5.samsung.com (Symantec Messaging Gateway) with SMTP id DD.3E.09734.4FBABE66; Thu, 19 Sep 2024 13:43:32 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas1p2.samsung.com (KnoxPortal) with ESMTPA id 20240919044332epcas1p2b7f44559e425d9d8ef309d58a5e13478~2i5y4s1Pg0071500715epcas1p2D for ; Thu, 19 Sep 2024 04:43:32 +0000 (GMT) Received: from epsmgmc1p1new.samsung.com (unknown [182.195.42.40]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20240919044332epsmtrp1ba5bbaa9df72d737e35da100399a9f08~2i5y36qzw0780507805epsmtrp1T for ; Thu, 19 Sep 2024 04:43:32 +0000 (GMT) X-AuditID: b6c32a39-8f9ff70000002606-c8-66ebabf40f31 Received: from epsmtip1.samsung.com ( [182.195.34.30]) by epsmgmc1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 11.EB.07567.4FBABE66; Thu, 19 Sep 2024 13:43:32 +0900 (KST) Received: from yoonjookim02 (unknown [10.102.35.189]) by epsmtip1.samsung.com (KnoxPortal) with ESMTPA id 20240919044332epsmtip1b4fff0748abaee13ad241e30d9fd624f~2i5ytDr4P1859018590epsmtip1y for ; Thu, 19 Sep 2024 04:43:32 +0000 (GMT) From: "yoonjoo" To: Date: Thu, 19 Sep 2024 13:43:32 +0900 Message-ID: <007701db0a4e$7adc8200$70958600$@samsung.com> MIME-Version: 1.0 X-Mailer: Microsoft Outlook 16.0 X-Drm-Type: PERSONAL X-Msg-Type: PERSONAL Thread-Index: Adr3WhYOV5VjZKBPSu6auomPOU+A/A== Content-Language: ko X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrPIsWRmVeSWpSXmKPExsWy7bCmge6X1a/TDN4/07X49ukMswOjx59F m1kCGKO4bFJSczLLUov07RK4Mm4//MZU8Eau4uXaeYwNjLfEuhg5OSQETCR+T5nB2MXIxSEk sINR4s+nEywQzgQmiZ4ZM6GcKUwSXS+nsncxcoC1XJ7IDBHfyyhx5VYzG4TTwSQx59kEZpC5 bAKaEp/7dzOB2CICshKr/01hA7GFBXwk/l/YwQhiswioStyctZgVxOYVsJSY+vk3M4QtKHFy 5hMWEJtZwEhiyer5TBC2vMT2t3OYIe5WkPj5dBkrhC0i8XD1R3YYu/XeWWaIvXoSRz7sYoXo FZGY3dkGdrWEwDl2idmbpjNCNLhIbPy4AGqosMSr41ugBklJvOxvY4doaGaUuLV6K5TTwiix 7tw7Jogqe4nT345Cnccn8e5rDyskjHglOtqEIEpUJT6eOgs1VFri7K2NLBC2h8TuvWvZJzAq zkLy9CwkT89C8vQsJE8sYGRZxSiWWlCcm55abFhgqlecmFtcmpeul5yfu4kRnCa0LHcwTn/7 Qe8QIxMH4yFGCQ5mJRFe8Q8v04R4UxIrq1KL8uOLSnNSiw8xSnOwKInznrlSliokkJ5Ykpqd mlqQWgSTZeLglGpgSr0U6n74dmp6YJ2cWKHiRZ1FVtHvCjaslos8c67ezHT3m6qbR02Wvxb7 cvz9XQf5EwKSq0/PmG9nuHxLy+6Uzz3p323WJGYX9Nzq0vj8/t285s/7o9nFX5eu7pks3hcb t7pIyHfxiaf39SuOTuRx2z1h1awJu4qUNXcs/8X8V58tJCH/TLB8+KWWY7MDhL5du7s85JnS xqD/jWZLf9+OPeO6p3qL77be2I+sR/lm3kxxNtCNnJoo7b73u/Qe/QsvD/dXzGgrkVjZmVhU vDmppXB/VdjmrzxTVC8ETL6mY2zodzrGqFHuuw7TC7H4mbP7bjXdO2IbYrvySvaHxelX9039 rNU950m0zd1bjzkj2ZRYijMSDbWYi4oTAXqmvwqCAwAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrNLMWRmVeSWpSXmKPExsWy7bCSnO6X1a/TDJomC1l8+3SG2YHR48+i zSwBjFFcNimpOZllqUX6dglcGbcffmMqeCNX8XLtPMYGxltiXYwcHBICJhKXJzJ3MXJxCAns ZpRYuOwDM0RcWmLuBl0IU1ji8OFiiJI2Jom1R7vZuxg5OdgENCU+9+9mArFFBGQlVv+bwgZi Cwv4SPy/sIMRxGYRUJW4OWsxK4jNK2ApMfXzb2YIW1Di5MwnLCA2M9AJjYe7oWx5ie1v54DV SAgoSPx8uowVwhaReLj6IzuELSzx6vgWdph4672zUPVmEjP/rWGHuEdP4siHXawQM0UkZne2 MU9gFJmFZPUsJKtnIVk9C0nLAkaWVYySqQXFuem5yYYFhnmp5XrFibnFpXnpesn5uZsYweGv pbGD8d78f3qHGJk4GA8xSnAwK4nwin94mSbEm5JYWZValB9fVJqTWnyIUZqDRUmc13DG7BQh gfTEktTs1NSC1CKYLBMHp1QDk+7/vpViKiH+vD7qd+9vefvBzVxjhcvBgub412z9px/2zPu1 /TlX/uNrv16ckPl+e87Ni3O6PJKqnk0V/cOZ4ckulrnYpORyf/fHnc/ZViy9xafya24x/2KV GUxKcvnv52ht4l10ztdrxbsIg5Vvra8+eDer2GHeCoM1af4df4/LChSuqQo7EiB+L+tAF+Pi S9ISBsvzeJnir1yS/mZ54mRpd6POoeWruW7vEY3frl7bZ8UiK6a/a9LFoASLkoiX9344zn8n y70nO2PLX105LfZlZm73TFM1/hiFP3SqrIpVsxJtOaARvHDvQg2WYo8zTzbqlaarHvpo+3xe 9tK/Fpfn6b5xjLr+5OCK5IjfLUosxRmJhlrMRcWJAJBUKnfuAgAA X-CMS-MailID: 20240919044332epcas1p2b7f44559e425d9d8ef309d58a5e13478 X-Msg-Generator: CA CMS-TYPE: 101P X-CMS-RootMailID: 20240919044332epcas1p2b7f44559e425d9d8ef309d58a5e13478 References: Subject: [FFmpeg-devel] [PATCH] avcodec/cuviddec: Add handling HDR10+ sidedata on cuviddec. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: OHBrup94hjXX Implemented decoding of NAL units and handling HDR10+ sidedata by referring to hevcdec. Signed-off-by: yoonjoo --- libavcodec/cuviddec.c | 69 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 69 insertions(+) + } + } +} + static int cuvid_output_frame(AVCodecContext *avctx, AVFrame *frame) { CuvidContext *ctx = avctx->priv_data; @@ -497,6 +561,8 @@ static int cuvid_output_frame(AVCodecContext *avctx, AVFrame *frame) CuvidParsedFrame parsed_frame; CUdeviceptr mapped_frame = 0; int ret = 0, eret = 0; + uint8_t *pkt_data = avctx->internal->buffer_pkt->data; + int pkt_size = avctx->internal->buffer_pkt->size; av_log(avctx, AV_LOG_TRACE, "cuvid_output_frame\n"); @@ -512,6 +578,9 @@ static int cuvid_output_frame(AVCodecContext *avctx, AVFrame *frame) if (ret < 0 && ret != AVERROR_EOF) return ret; ret = cuvid_decode_packet(avctx, pkt); + + decode_nal_units(avctx, pkt_data, pkt_size, frame); + av_packet_unref(pkt); // cuvid_is_buffer_full() should avoid this. if (ret == AVERROR(EAGAIN)) diff --git a/libavcodec/cuviddec.c b/libavcodec/cuviddec.c index 3fae9c1..1b80b81 100644 --- a/libavcodec/cuviddec.c +++ b/libavcodec/cuviddec.c @@ -33,6 +33,7 @@ #include "libavutil/mem.h" #include "libavutil/opt.h" #include "libavutil/pixdesc.h" +#include "libavutil/hdr_dynamic_metadata.h" #include "avcodec.h" #include "bsf.h" @@ -41,6 +42,11 @@ #include "hwconfig.h" #include "nvdec.h" #include "internal.h" +#include "h2645_parse.h" +#include "bytestream.h" +#include "hevc/hevc.h" +#include "hevc/sei.h" +#include "hevc/ps.h" #if !NVDECAPI_CHECK_VERSION(9, 0) #define cudaVideoSurfaceFormat_YUV444 2 @@ -488,6 +494,64 @@ error: return 0; } +static void decode_nal_units(AVCodecContext* avctx, const uint8_t *buf, int length, AVFrame* frame) +{ + H2645Packet hpkt = { 0 }; + int is_nalff = 1; + int nal_length_size = 4; + HEVCSEI sei = { 0 }; + HEVCParamSets ps = { 0 }; + + av_log(avctx, AV_LOG_TRACE, "decode_nal_units\n"); + ff_h2645_packet_split(&hpkt, buf, length, avctx, is_nalff, nal_length_size, avctx->codec_id, 1, 0); + + for (int i = 0; i < hpkt.nb_nals; i++) { + H2645NAL* nal = &hpkt.nals[i]; + GetBitContext gb = nal->gb; + + av_log(avctx, AV_LOG_TRACE, "[%d/%d] NAL type = %d\n", i + 1, hpkt. nb_nals, nal->type); + + switch (nal->type) { + case HEVC_NAL_SEI_PREFIX: + { + int ret = ff_hevc_decode_nal_sei(&gb, avctx, &sei, &ps, nal- >type); + if (ret < 0) { + av_log(avctx, AV_LOG_WARNING, "Skipping invalid undecodable NALU: %d\n", nal->type); + return; + } + + if (sei.common.dynamic_hdr_plus.info) + av_frame_new_side_data_from_buf(frame, AV_FRAME_DATA_DYNAMIC_HDR_PLUS, sei.common.dynamic_hdr_plus.info); + break; + } + case HEVC_NAL_VPS: + case HEVC_NAL_SPS: + case HEVC_NAL_PPS: + case HEVC_NAL_TRAIL_R: + case HEVC_NAL_TRAIL_N: + case HEVC_NAL_TSA_N: + case HEVC_NAL_TSA_R: + case HEVC_NAL_STSA_N: + case HEVC_NAL_STSA_R: + case HEVC_NAL_BLA_W_LP: + case HEVC_NAL_BLA_W_RADL: + case HEVC_NAL_BLA_N_LP: + case HEVC_NAL_IDR_W_RADL: + case HEVC_NAL_IDR_N_LP: + case HEVC_NAL_CRA_NUT: + case HEVC_NAL_RADL_N: + case HEVC_NAL_RADL_R: + case HEVC_NAL_RASL_N: + case HEVC_NAL_RASL_R: + // these Nal types will be handled in 'cuvid_decode_packet()' + break; + default: + av_log(avctx, AV_LOG_INFO, "Skipping NAL unit %d\n", nal- >type); + break;