From patchwork Wed Oct 19 07:25:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Siedel X-Patchwork-Id: 38813 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp215861pzb; Wed, 19 Oct 2022 00:27:17 -0700 (PDT) X-Google-Smtp-Source: AMsMyM70CPeJjHMsFwNikJE7nx4vm5WJvLbJsBfFH153WqbtsWMd1JH064cyZ2T7poAIlpz7SgqU X-Received: by 2002:a17:907:a06d:b0:78d:ceae:ba5a with SMTP id ia13-20020a170907a06d00b0078dceaeba5amr5648920ejc.586.1666164437043; Wed, 19 Oct 2022 00:27:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666164437; cv=none; d=google.com; s=arc-20160816; b=mlJ7eSXeKSq76Hq0yAONyEjxKdwgHwTrJBYJ07rwUjUfs4iN+FmqTQRn44Vh1IA8z5 heKSSF8gbvvpRyWYNbw1K6SWWN5krHrQgxTbNSD/edZ5f/0RUD7hCy2WWpKVRDexQM4V eG6TFRTVmUFOmcEkmrTiguAkv++pV2FpCfAt10cSSp/BRppv+CB77fbdpX7sfrrfXKqY FdLQhGbqb0+pRSTWX2ipyTBSUrI0Q4WfEW6kiAsgPNbOyMk9mXG9LGxIg1Rv/++hFap5 m2oTFqlrnwula0TUUT5kEtpbK0+GDA6eN3BzEbCebnYzeJXdJkXNDmPZyomxmYivmxDQ c4FA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=xuBam7ySt2XS/grdwV+URCTeukWBf9RuH4c7eHh67n4=; b=muwl0mkhqlOdpS7Gj5gOk3+EpkOwnCynphN6eJ5G0b2iuDdoXkmVXifJhzhBKc0OVM ufWG96CnD6vef+QgATCub9t2X48OOOR9htmgr4FRhU4D08FJw1wOF+MTVqf640hjNrmv 8NMNVH23XZPhORO5+kH70XZfJdRJZB4lvwLP8hQiE6HracUHPBntVB/QlxE1W+pn9fjC 3Dx7zpYdKnHrFXkQ/aGmQjYiYyTeU25EtSH3bPxfiWrML/kH0X9faM+1sp6W09BS4Mi/ MBJsnVyMgGTgR5OGOD1CBzr7NNdFl4KxIZlEpUwWcSqJCQmiTcMnK612mHf4syH+uCHu jdng== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=3p9Yxq8N; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id lb25-20020a170907785900b00741d9a1e2dbsi12876590ejc.1004.2022.10.19.00.27.16; Wed, 19 Oct 2022 00:27:17 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=3p9Yxq8N; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1C3B568BBC9; Wed, 19 Oct 2022 10:26:36 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f43.google.com (mail-ej1-f43.google.com [209.85.218.43]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6281C68BD2E for ; Wed, 19 Oct 2022 10:26:28 +0300 (EEST) Received: by mail-ej1-f43.google.com with SMTP id fy4so37798930ejc.5 for ; Wed, 19 Oct 2022 00:26:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spin-digital-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=L7sPVx/ywHmQLDz6lTQZEC83ldqRDZu3e0mciEDAmXA=; b=3p9Yxq8NA2hNr3rrvbhP9L5pdm1QczcfcfF/WnqM19508u6ifk40+D3k8YnPuHwhac Z9phlE+w7ShaAm7NJeK04HepVzxUVHcHNA6jkc5SAP0v/hWZ1H2dcVzLE7IlPn/kfz+H TRLbFeGeJvXIz5sFL7/c7mMTTCT2Kxk4Ee9b0g7VnJdnDYmHUpn4ad21u4xbRgdsu9NZ KRbUh8gWpOqnt1H8te1SFxX8JmmqXEomNgIcb3oG3CEjXemdHB0wz1kpN8E4mPAIb3vB ALyogec9onMQJ8afW8z5EDBoRyaFZ++KUGNSj27tSpun3rYMbSLA8WgMHpkl+36k+/IZ Tg/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=L7sPVx/ywHmQLDz6lTQZEC83ldqRDZu3e0mciEDAmXA=; b=LlMLQ3R7jAUNy/VvzvFKr3rnX0FLDOComGE7mMmAsMe1VBb7XNrWTlHiGCetr/6hxB aBkdoy2mmvU4ZtybQi89N4KpJyd4qxxXJ42SXOOvt2fDhV8pRA3TzP/GsM8OKvKuViV2 ocgigwwKiT12HgOQ3ZoXmJ61uJwK01pFTyEJl9nAZ3V4mFbAASN8x/i3w4v93SbkFa6Q v7Z7HhyscRbOTORloTd3LDLXOlwPR7y0cKhI/uzVvYe6qZdVBsgC11Q+H1bVH+bGJ4kG fuSVpi5aL1TMuxfs5yxnqgAaMqhUFDH9uKPN/TObB9Jra7VGZLFsT0XlanK/j5oAwRVS YrQA== X-Gm-Message-State: ACrzQf3YeyBxhhRKsWfkCqyLz2nfyJV2VwZEBg0AZ6Tw84RmvRVg8xoF 8ZSASsNHnlyfIGZ2DEaRRAFrA2JYJycyAA== X-Received: by 2002:a17:907:a428:b0:78d:9fab:84fb with SMTP id sg40-20020a170907a42800b0078d9fab84fbmr5346088ejc.694.1666164387158; Wed, 19 Oct 2022 00:26:27 -0700 (PDT) Received: from thomas-win.localdomain ([213.138.44.237]) by smtp.gmail.com with ESMTPSA id p5-20020a170906498500b0078b83968ad4sm8515150eju.24.2022.10.19.00.26.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Oct 2022 00:26:26 -0700 (PDT) From: thomas.ff@spin-digital.com To: ffmpeg-devel@ffmpeg.org Date: Wed, 19 Oct 2022 09:25:01 +0200 Message-Id: <20221019072508.23460-5-thomas.ff@spin-digital.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221019072508.23460-1-thomas.ff@spin-digital.com> References: <20221019072508.23460-1-thomas.ff@spin-digital.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v1 04/11] avcodec: add bitstream parser for H266/VVC X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Thomas Siedel Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Nb7yFLRMZpAg From: Thomas Siedel Add nal parser ff_vvc_parser to parse vvc elementary bitstreams. Signed-off-by: Thomas Siedel --- configure | 3 + libavcodec/Makefile | 1 + libavcodec/h2645_parse.c | 71 ++++- libavcodec/parsers.c | 1 + libavcodec/vvc_parser.c | 588 +++++++++++++++++++++++++++++++++++++++ 5 files changed, 663 insertions(+), 1 deletion(-) create mode 100644 libavcodec/vvc_parser.c diff --git a/configure b/configure index f3fd91f592..13dca5cd9a 100755 --- a/configure +++ b/configure @@ -2446,6 +2446,7 @@ CONFIG_EXTRA=" cbs_av1 cbs_h264 cbs_h265 + cbs_h266 cbs_jpeg cbs_mpeg2 cbs_vp9 @@ -2727,6 +2728,7 @@ threads_if_any="$THREADS_LIST" cbs_av1_select="cbs" cbs_h264_select="cbs" cbs_h265_select="cbs" +cbs_h266_select="cbs" cbs_jpeg_select="cbs" cbs_mpeg2_select="cbs" cbs_vp9_select="cbs" @@ -3278,6 +3280,7 @@ hevc_parser_select="hevcparse" mpegaudio_parser_select="mpegaudioheader" mpeg4video_parser_select="h263dsp mpegvideodec qpeldsp" vc1_parser_select="vc1dsp" +vvc_parser_select="cbs_h266" # bitstream_filters aac_adtstoasc_bsf_select="adts_header mpeg4audio" diff --git a/libavcodec/Makefile b/libavcodec/Makefile index b3fcf173e9..3850e316ac 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1173,6 +1173,7 @@ OBJS-$(CONFIG_VC1_PARSER) += vc1_parser.o vc1.o vc1data.o \ OBJS-$(CONFIG_VP3_PARSER) += vp3_parser.o OBJS-$(CONFIG_VP8_PARSER) += vp8_parser.o OBJS-$(CONFIG_VP9_PARSER) += vp9_parser.o +OBJS-$(CONFIG_VVC_PARSER) += vvc_parser.o OBJS-$(CONFIG_WEBP_PARSER) += webp_parser.o OBJS-$(CONFIG_XBM_PARSER) += xbm_parser.o OBJS-$(CONFIG_XMA_PARSER) += xma_parser.o diff --git a/libavcodec/h2645_parse.c b/libavcodec/h2645_parse.c index 90944177c7..9fbeee3edd 100644 --- a/libavcodec/h2645_parse.c +++ b/libavcodec/h2645_parse.c @@ -27,6 +27,7 @@ #include "libavutil/mem.h" #include "bytestream.h" +#include "vvc.h" #include "hevc.h" #include "h264.h" #include "h2645_parse.h" @@ -145,6 +146,47 @@ nsc: return si; } +static const char *const vvc_nal_type_name[32] = { + "TRAIL_NUT", // VVC_TRAIL_NUT + "STSA_NUT", // VVC_STSA_NUT + "RADL_NUT", // VVC_RADL_NUT + "RASL_NUT", // VVC_RASL_NUT + "RSV_VCL_4", // VVC_RSV_VCL_4 + "RSV_VCL_5", // VVC_RSV_VCL_5 + "RSV_VCL_6", // VVC_RSV_VCL_6 + "IDR_W_RADL", // VVC_IDR_W_RADL + "IDR_N_LP", // VVC_IDR_N_LP + "CRA_NUT", // VVC_CRA_NUT + "GDR_NUT", // VVC_GDR_NUT + "RSV_IRAP_11", // VVC_RSV_IRAP_11 + "OPI_NUT", // VVC_OPI_NUT + "DCI_NUT", // VVC_DCI_NUT + "VPS_NUT", // VVC_VPS_NUT + "SPS_NUT", // VVC_SPS_NUT + "PPS_NUT", // VVC_PPS_NUT + "PREFIX_APS_NUT",// VVC_PREFIX_APS_NUT + "SUFFIX_APS_NUT",// VVC_SUFFIX_APS_NUT + "PH_NUT", // VVC_PH_NUT + "AUD_NUT", // VVC_AUD_NUT + "EOS_NUT", // VVC_EOS_NUT + "EOB_NUT", // VVC_EOB_NUT + "PREFIX_SEI_NUT",// VVC_PREFIX_SEI_NUT + "SUFFIX_SEI_NUT",// VVC_SUFFIX_SEI_NUT + "FD_NUT", // VVC_FD_NUT + "RSV_NVCL_26", // VVC_RSV_NVCL_26 + "RSV_NVCL_27", // VVC_RSV_NVCL_27 + "UNSPEC_28", // VVC_UNSPEC_28 + "UNSPEC_29", // VVC_UNSPEC_29 + "UNSPEC_30", // VVC_UNSPEC_30 + "UNSPEC_31", // VVC_UNSPEC_31 +}; + +static const char *vvc_nal_unit_name(int nal_type) +{ + av_assert0(nal_type >= 0 && nal_type < 32); + return vvc_nal_type_name[nal_type]; +} + static const char *const hevc_nal_type_name[64] = { "TRAIL_N", // HEVC_NAL_TRAIL_N "TRAIL_R", // HEVC_NAL_TRAIL_R @@ -293,6 +335,31 @@ static int get_bit_length(H2645NAL *nal, int min_size, int skip_trailing_zeros) * @return AVERROR_INVALIDDATA if the packet is not a valid NAL unit, * 0 otherwise */ +static int vvc_parse_nal_header(H2645NAL *nal, void *logctx) +{ + GetBitContext *gb = &nal->gb; + + if (get_bits1(gb) != 0) //forbidden_zero_bit + return AVERROR_INVALIDDATA; + + skip_bits1(gb); //nuh_reserved_zero_bit + + nal->nuh_layer_id = get_bits(gb, 6); + nal->type = get_bits(gb, 5); + nal->temporal_id = get_bits(gb, 3) - 1; + if (nal->temporal_id < 0) + return AVERROR_INVALIDDATA; + + if ((nal->type >= VVC_IDR_W_RADL && nal->type <= VVC_RSV_IRAP_11) && nal->temporal_id) + return AVERROR_INVALIDDATA; + + av_log(logctx, AV_LOG_DEBUG, + "nal_unit_type: %d(%s), nuh_layer_id: %d, temporal_id: %d\n", + nal->type, vvc_nal_unit_name(nal->type), nal->nuh_layer_id, nal->temporal_id); + + return 0; +} + static int hevc_parse_nal_header(H2645NAL *nal, void *logctx) { GetBitContext *gb = &nal->gb; @@ -509,7 +576,9 @@ int ff_h2645_packet_split(H2645Packet *pkt, const uint8_t *buf, int length, /* Reset type in case it contains a stale value from a previously parsed NAL */ nal->type = 0; - if (codec_id == AV_CODEC_ID_HEVC) + if (codec_id == AV_CODEC_ID_VVC) + ret = vvc_parse_nal_header(nal, logctx); + else if (codec_id == AV_CODEC_ID_HEVC) ret = hevc_parse_nal_header(nal, logctx); else ret = h264_parse_nal_header(nal, logctx); diff --git a/libavcodec/parsers.c b/libavcodec/parsers.c index d355808018..e5b01b7a59 100644 --- a/libavcodec/parsers.c +++ b/libavcodec/parsers.c @@ -74,6 +74,7 @@ extern const AVCodecParser ff_vorbis_parser; extern const AVCodecParser ff_vp3_parser; extern const AVCodecParser ff_vp8_parser; extern const AVCodecParser ff_vp9_parser; +extern const AVCodecParser ff_vvc_parser; extern const AVCodecParser ff_webp_parser; extern const AVCodecParser ff_xbm_parser; extern const AVCodecParser ff_xma_parser; diff --git a/libavcodec/vvc_parser.c b/libavcodec/vvc_parser.c new file mode 100644 index 0000000000..f7eb2340c5 --- /dev/null +++ b/libavcodec/vvc_parser.c @@ -0,0 +1,588 @@ +/* + * VVC parser + * + * Copyright (C) 2021 Nuo Mi + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "cbs.h" +#include "cbs_h266.h" +#include "internal.h" +#include "parser.h" +#include "decode.h" + +#define START_CODE 0x000001 ///< start_code_prefix_one_3bytes + +#define IS_SLICE(nut) (nut <= VVC_RASL_NUT || (nut >= VVC_IDR_W_RADL && nut <= VVC_GDR_NUT)) +#define IS_PH(nut) (nut == VVC_PH_NUT) +#define IS_IDR(nut) (nut == VVC_IDR_W_RADL || nut == VVC_IDR_N_LP) + + +typedef struct PuInfo { + AVBufferRef *sps_ref; + AVBufferRef *pps_ref; + AVBufferRef *slice_ref; + AVBufferRef *ph_ref; + + const H266RawPPS *pps; + const H266RawSPS *sps; + const H266RawPH *ph; + const H266RawSlice *slice; + int pic_type; +} PuInfo; + +typedef struct AuDetector { + uint8_t prev_layer_id; + int prev_tid0_poc; + int prev_poc; +} AuDetector; + +typedef struct VVCParserContext { + ParseContext pc; + CodedBitstreamContext *cbc; + + CodedBitstreamFragment picture_unit; + + PuInfo au_info; + AVPacket au; + AVPacket last_au; + + AuDetector au_detector; + + int parsed_extradata; +} VVCParserContext; + +static const enum AVPixelFormat pix_fmts_8bit[] = { + AV_PIX_FMT_GRAY8, AV_PIX_FMT_YUV420P, + AV_PIX_FMT_YUV422P, AV_PIX_FMT_YUV444P +}; + +static const enum AVPixelFormat pix_fmts_10bit[] = { + AV_PIX_FMT_GRAY10, AV_PIX_FMT_YUV420P10, + AV_PIX_FMT_YUV422P10, AV_PIX_FMT_YUV444P10 +}; + +static int get_format(const H266RawSPS *sps) +{ + switch (sps->sps_bitdepth_minus8) { + case 0: + return pix_fmts_8bit[sps->sps_chroma_format_idc]; + case 2: + return pix_fmts_10bit[sps->sps_chroma_format_idc]; + } + return AV_PIX_FMT_NONE; +} + +/** + * Find the end of the current frame in the bitstream. + * @return the position of the first byte of the next frame, or END_NOT_FOUND + */ +static int find_frame_end(AVCodecParserContext *s, const uint8_t *buf, + int buf_size) +{ + VVCParserContext *ctx = s->priv_data; + ParseContext *pc = &ctx->pc; + int i; + + for (i = 0; i < buf_size; i++) { + int nut; + + pc->state64 = (pc->state64 << 8) | buf[i]; + + if (((pc->state64 >> 3 * 8) & 0xFFFFFF) != START_CODE) + continue; + + nut = (pc->state64 >> (8 + 3)) & 0x1F; + // 7.4.2.4.3 and 7.4.2.4.4 + if ((nut >= VVC_OPI_NUT && nut <= VVC_PREFIX_APS_NUT && nut != VVC_PH_NUT) || + nut == VVC_AUD_NUT || (nut == VVC_PREFIX_SEI_NUT && !pc->frame_start_found) || nut == VVC_RSV_NVCL_26 || + nut == VVC_UNSPEC_28 || nut == VVC_UNSPEC_29) { + if (pc->frame_start_found) { + pc->frame_start_found = 0; + return i - 5; + } + } else if (nut == VVC_PH_NUT || IS_SLICE(nut)) { + int sh_picture_header_in_slice_header_flag = buf[i] >> 7; + + if (nut == VVC_PH_NUT || sh_picture_header_in_slice_header_flag) { + if (!pc->frame_start_found) { + pc->frame_start_found = 1; + } else { // First slice of next frame found + pc->frame_start_found = 0; + return i - 5; + } + } + } + } + return END_NOT_FOUND; +} + +static int get_pict_type(const CodedBitstreamFragment *pu) +{ + int has_p = 0; + for (int i = 0; i < pu->nb_units; i++) { + CodedBitstreamUnit *unit = &pu->units[i]; + if (IS_SLICE(unit->type)) { + const H266RawSlice *slice = unit->content; + uint8_t type = slice->header.sh_slice_type; + if (type == VVC_SLICE_TYPE_B) { + return AV_PICTURE_TYPE_B; + } + if (type == VVC_SLICE_TYPE_P) { + has_p = 1; + } + } + } + return has_p ? AV_PICTURE_TYPE_P : AV_PICTURE_TYPE_I; +} + +static void pu_info_unref(PuInfo *info) +{ + av_buffer_unref(&info->slice_ref); + av_buffer_unref(&info->ph_ref); + av_buffer_unref(&info->pps_ref); + av_buffer_unref(&info->sps_ref); + info->slice = NULL; + info->ph = NULL; + info->pps = NULL; + info->sps = NULL; + info->pic_type = AV_PICTURE_TYPE_NONE; +} + +static int pu_info_ref(PuInfo *dest, const PuInfo *src) +{ + pu_info_unref(dest); + dest->sps_ref = av_buffer_ref(src->sps_ref); + dest->pps_ref = av_buffer_ref(src->pps_ref); + if (src->ph_ref) + dest->ph_ref = av_buffer_ref(src->ph_ref); + dest->slice_ref = av_buffer_ref(src->slice_ref); + if (!dest->sps_ref || !dest->pps_ref || (src->ph_ref && !dest->ph_ref) || !dest->slice_ref) { + pu_info_unref(dest); + return AVERROR(ENOMEM); + } + + dest->sps = src->sps; + dest->pps = src->pps; + dest->ph = src->ph; + dest->slice = src->slice; + dest->pic_type = src->pic_type; + return 0; +} + +static int set_parser_ctx(AVCodecParserContext *ctx, AVCodecContext *avctx, + const PuInfo *pu) +{ + int ret, num = 0, den = 0; + static const uint8_t h266_sub_width_c[] = { + 1, 2, 2, 1 + }; + static const uint8_t h266_sub_height_c[] = { + 1, 2, 1, 1 + }; + const H266RawSPS *sps = pu->sps; + const H266RawPPS *pps = pu->pps; + //const H266RawPH *ph = pu->ph; + const H266RawNALUnitHeader *nal = &pu->slice->header.nal_unit_header; + + /* set some sane default values */ + ctx->pict_type = AV_PICTURE_TYPE_I; + ctx->key_frame = 0; + ctx->picture_structure = AV_PICTURE_STRUCTURE_FRAME; + + ctx->key_frame = nal->nal_unit_type == VVC_IDR_W_RADL || + nal->nal_unit_type == VVC_IDR_N_LP || + nal->nal_unit_type == VVC_CRA_NUT || + nal->nal_unit_type == VVC_GDR_NUT; + + ctx->coded_width = pps->pps_pic_width_in_luma_samples; + ctx->coded_height = pps->pps_pic_height_in_luma_samples; + ctx->width = pps->pps_pic_width_in_luma_samples - + (pps->pps_conf_win_left_offset + pps->pps_conf_win_right_offset) * + h266_sub_width_c[sps->sps_chroma_format_idc]; + ctx->height = pps->pps_pic_height_in_luma_samples - + (pps->pps_conf_win_top_offset + pps->pps_conf_win_bottom_offset) * + h266_sub_height_c[sps->sps_chroma_format_idc];; + ctx->pict_type = pu->pic_type; + ctx->format = get_format(sps); + + avctx->profile = sps->profile_tier_level.general_profile_idc; + avctx->level = sps->profile_tier_level.general_level_idc; + + avctx->colorspace = (enum AVColorSpace) sps->vui.vui_matrix_coeffs; + avctx->color_primaries = (enum AVColorPrimaries) sps->vui.vui_colour_primaries; + avctx->color_trc = (enum AVColorTransferCharacteristic) sps->vui.vui_transfer_characteristics; + avctx->color_range = sps->vui.vui_full_range_flag ? AVCOL_RANGE_JPEG : AVCOL_RANGE_MPEG; + + if (ctx->width != avctx->width || ctx->height != avctx->height) { + ret = ff_set_dimensions(avctx, ctx->width, ctx->height); + if (ret < 0) + return ret; + } + avctx->pix_fmt = ctx->format; + avctx->has_b_frames = (sps->sps_max_sublayers_minus1+1) > 2 ? 2 : sps->sps_max_sublayers_minus1; + avctx->max_b_frames = sps->sps_max_sublayers_minus1; + + if(sps->sps_ptl_dpb_hrd_params_present_flag && sps->sps_timing_hrd_params_present_flag) { + num = sps->sps_general_timing_hrd_parameters.num_units_in_tick; + den = sps->sps_general_timing_hrd_parameters.time_scale; + } else { + return 1; + } + if (num != 0 && den != 0) + av_reduce(&avctx->framerate.den, &avctx->framerate.num, + num, den, 1 << 30); + + if (avctx->framerate.num) + avctx->time_base = av_inv_q(av_mul_q(avctx->framerate, (AVRational){avctx->ticks_per_frame, 1})); + + return 1; +} + +static int set_ctx(AVCodecParserContext *ctx, AVCodecContext *avctx, const PuInfo *next_pu) +{ + VVCParserContext *s = ctx->priv_data; + int ret; + if (s->au_info.slice) { + if ((ret = set_parser_ctx(ctx, avctx, &s->au_info)) < 0) + return ret; + } + ret = pu_info_ref(&s->au_info, next_pu); + return ret; +} + +//8.3.1 Decoding process for picture order count. +//VTM did not follow the spec, and it's much simpler than spec. +//We follow the VTM. +static void get_slice_poc(VVCParserContext *s, int *poc, + const H266RawSPS *sps, + const H266RawPH *ph, const H266RawSliceHeader *slice, + void *log_ctx) +{ + int poc_msb, max_poc_lsb, poc_lsb; + AuDetector *d = &s->au_detector; + max_poc_lsb = 1 << (sps->sps_log2_max_pic_order_cnt_lsb_minus4 + 4); + poc_lsb = ph->ph_pic_order_cnt_lsb; + if (IS_IDR(slice->nal_unit_header.nal_unit_type)) { + if (ph->ph_poc_msb_cycle_present_flag) + poc_msb = ph->ph_poc_msb_cycle_val * max_poc_lsb; + else + poc_msb = 0; + } else { + int prev_poc = d->prev_tid0_poc; + int prev_poc_lsb = prev_poc & (max_poc_lsb - 1); + int prev_poc_msb = prev_poc - prev_poc_lsb; + if (ph->ph_poc_msb_cycle_present_flag) { + poc_msb = ph->ph_poc_msb_cycle_val * max_poc_lsb; + } else { + if ((poc_lsb < prev_poc_lsb) && ((prev_poc_lsb - poc_lsb) >= (max_poc_lsb / 2))) + poc_msb = prev_poc_msb + max_poc_lsb; + else if ((poc_lsb > prev_poc_lsb) && ((poc_lsb - prev_poc_lsb) > (max_poc_lsb / 2))) + poc_msb = prev_poc_msb - max_poc_lsb; + else + poc_msb = prev_poc_msb; + } + } + + *poc = poc_msb + poc_lsb; +} + +static void au_detector_init(AuDetector *d) +{ + d->prev_layer_id = UINT8_MAX; + d->prev_poc = INT_MAX; + d->prev_tid0_poc = INT_MAX; +} + +static int is_au_start(VVCParserContext *s, const PuInfo *pu, void *log_ctx) +{ + //7.4.2.4.3 + AuDetector *d = &s->au_detector; + const H266RawSPS *sps = pu->sps; + const H266RawNALUnitHeader *nal = &pu->slice->header.nal_unit_header; + const H266RawPH *ph = pu->ph; + const H266RawSlice *slice = pu->slice; + int ret, poc, nut; + + get_slice_poc(s, &poc, sps, ph, &slice->header, log_ctx); + + ret = (nal->nuh_layer_id <= d->prev_layer_id) || (poc != d->prev_poc); + + nut = nal->nal_unit_type; + d->prev_layer_id = nal->nuh_layer_id; + d->prev_poc = poc; + if (nal->nuh_temporal_id_plus1 == 1 && + !ph->ph_non_ref_pic_flag && nut != VVC_RADL_NUT && nut != VVC_RASL_NUT) { + d->prev_tid0_poc = poc; + } + return ret; +} + +static int get_pu_info(PuInfo *info, const CodedBitstreamH266Context *h266, + const CodedBitstreamFragment *pu, void *logctx) +{ + const H266RawNALUnitHeader *nal; + int ret; + + memset(info, 0, sizeof(*info)); + for (int i = 0; i < pu->nb_units; i++) { + nal = pu->units[i].content; + if (!nal) + continue; + if (IS_PH(nal->nal_unit_type)) { + info->ph = pu->units[i].content; + info->ph_ref = pu->units[i].content_ref; + } else if (IS_SLICE(nal->nal_unit_type)) { + info->slice = pu->units[i].content; + info->slice_ref = pu->units[i].content_ref; + if (info->slice->header.sh_picture_header_in_slice_header_flag) + info->ph = &info->slice->header.sh_picture_header; + if (!info->ph) { + av_log(logctx, AV_LOG_ERROR, + "can't find picture header in picture unit.\n"); + ret = AVERROR_INVALIDDATA; + goto error; + } + break; + } + } + if (!info->slice) { + av_log(logctx, AV_LOG_ERROR, + "can't find slice in picture unit.\n"); + ret = AVERROR_INVALIDDATA; + goto error; + } + info->pps = h266->pps[info->ph->ph_pic_parameter_set_id]; + if (!info->pps) { + av_log(logctx, AV_LOG_ERROR, "PPS id %d is not avaliable.\n", + info->ph->ph_pic_parameter_set_id); + ret = AVERROR_INVALIDDATA; + goto error; + } + info->pps_ref = h266->pps_ref[info->ph->ph_pic_parameter_set_id]; + info->sps = h266->sps[info->pps->pps_seq_parameter_set_id]; + if (!info->sps) { + av_log(logctx, AV_LOG_ERROR, "SPS id %d is not avaliable.\n", + info->pps->pps_seq_parameter_set_id); + ret = AVERROR_INVALIDDATA; + goto error; + } + info->sps_ref = h266->sps_ref[info->pps->pps_seq_parameter_set_id]; + info->pic_type = get_pict_type(pu); + return 0; +error: + memset(info, 0, sizeof(*info)); + return ret; +} + +static int append_au(AVPacket *pkt, const uint8_t *buf, int buf_size) +{ + int offset = pkt->size; + int ret; + if ((ret = av_grow_packet(pkt, buf_size)) < 0) + goto end; + memcpy(pkt->data + offset, buf, buf_size); +end: + return ret; +} + +/** + * Parse NAL units of found picture and decode some basic information. + * + * @param ctx parser context. + * @param avctx codec context. + * @param buf buffer with field/frame data. + * @param buf_size size of the buffer. + * @return < 0 for error, == 0 for a complete au, > 0 is not a completed au. + */ +static int parse_nal_units(AVCodecParserContext *ctx, const uint8_t *buf, + int buf_size, AVCodecContext *avctx) +{ + VVCParserContext *s = ctx->priv_data; + const CodedBitstreamH266Context *h266 = s->cbc->priv_data; + + CodedBitstreamFragment *pu = &s->picture_unit; + int ret; + PuInfo info; + + if (!buf_size) { + if (s->au.size) { + if ((ret = av_packet_ref(&s->last_au, &s->au)) < 0) + goto end; + av_packet_unref(&s->au); + return 0; + } + return 1; + } + + if ((ret = ff_cbs_read(s->cbc, pu, buf, buf_size))< 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to parse picture unit.\n"); + goto end; + } + if ((ret = get_pu_info(&info, h266, pu, avctx)) < 0) + goto end; + if (is_au_start(s, &info, avctx)) { + if ((ret = set_ctx(ctx, avctx, &info)) < 0) + goto end; + if ((ret = av_packet_ref(&s->last_au, &s->au)) < 0) + goto end; + av_packet_unref(&s->au); + } else { + //not a completed au + ret = 1; + } + if (append_au(&s->au, buf, buf_size) < 0) + ret = AVERROR(ENOMEM); +end: + ff_cbs_fragment_reset(pu); + return ret; +} + +/** + * Combine PU to AU + * + * @param ctx parser context. + * @param avctx codec context. + * @param buf buffer to a PU. + * @param buf_size size of the buffer. + * @return < 0 for error, == 0 a complete au, > 0 not a completed au. + */ + +static int combine_au(AVCodecParserContext *ctx, AVCodecContext *avctx, + const uint8_t **buf, int *buf_size) +{ + VVCParserContext *s = ctx->priv_data; + CodedBitstreamFragment *pu = &s->picture_unit; + int ret; + + s->cbc->log_ctx = avctx; + + if (avctx->extradata_size && !s->parsed_extradata) { + s->parsed_extradata = 1; + + if ((ret = ff_cbs_read(s->cbc, pu, avctx->extradata, avctx->extradata_size)) < 0) + av_log(avctx, AV_LOG_WARNING, "Failed to parse extradata.\n"); + + ff_cbs_fragment_reset(pu); + } + av_packet_unref(&s->last_au); + ret = parse_nal_units(ctx, *buf, *buf_size, avctx); + if (ret == 0) { + if (s->last_au.size) { + *buf = s->last_au.data; + *buf_size = s->last_au.size; + } else { + //no output + ret = 1; + } + } + s->cbc->log_ctx = NULL; + return ret; +} + +static int vvc_parser_parse(AVCodecParserContext *ctx, AVCodecContext *avctx, + const uint8_t **poutbuf, int *poutbuf_size, + const uint8_t *buf, int buf_size) +{ + int next; + VVCParserContext *s = ctx->priv_data; + ParseContext *pc = &s->pc; + + if (avctx->extradata && !s->parsed_extradata) { + av_log(avctx, AV_LOG_INFO, "extra data is not supported yet.\n"); + return AVERROR_PATCHWELCOME; + } + + if (ctx->flags & PARSER_FLAG_COMPLETE_FRAMES) { + next = buf_size; + } else { + int ret, flush = !buf_size; + next = find_frame_end(ctx, buf, buf_size); + if (ff_combine_frame(pc, next, &buf, &buf_size) < 0) + goto no_out; + ret = combine_au(ctx, avctx, &buf, &buf_size); + if (ret > 0 && flush) { + buf_size = 0; + ret = combine_au(ctx, avctx, &buf, &buf_size); + } + if (ret != 0) { + buf_size = next; + goto no_out; + } + } + *poutbuf = buf; + *poutbuf_size = buf_size; + return next; +no_out: + *poutbuf = NULL; + *poutbuf_size = 0; + return buf_size; +} + +static const CodedBitstreamUnitType decompose_unit_types[] = { + VVC_TRAIL_NUT, + VVC_STSA_NUT, + VVC_RADL_NUT, + VVC_RASL_NUT, + VVC_IDR_W_RADL, + VVC_IDR_N_LP, + VVC_CRA_NUT, + VVC_GDR_NUT, + VVC_VPS_NUT, + VVC_SPS_NUT, + VVC_PPS_NUT, + VVC_PH_NUT, + VVC_AUD_NUT, +}; + +static av_cold int vvc_parser_init(AVCodecParserContext *ctx) +{ + VVCParserContext *s = ctx->priv_data; + int ret; + + ret = ff_cbs_init(&s->cbc, AV_CODEC_ID_VVC, NULL); + if (ret < 0) + return ret; + au_detector_init(&s->au_detector); + + s->cbc->decompose_unit_types = decompose_unit_types; + s->cbc->nb_decompose_unit_types = FF_ARRAY_ELEMS(decompose_unit_types); + + return ret; +} + +static void vvc_parser_close(AVCodecParserContext *ctx) +{ + VVCParserContext *s = ctx->priv_data; + + pu_info_unref(&s->au_info); + av_packet_unref(&s->au); + av_packet_unref(&s->last_au); + ff_cbs_fragment_free(&s->picture_unit); + + ff_cbs_close(&s->cbc); + av_freep(&s->pc.buffer); +} + +AVCodecParser ff_vvc_parser = { + .codec_ids = { AV_CODEC_ID_VVC }, + .priv_data_size = sizeof(VVCParserContext), + .parser_init = vvc_parser_init, + .parser_close = vvc_parser_close, + .parser_parse = vvc_parser_parse, +};