From patchwork Thu Nov 3 12:21:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Siedel X-Patchwork-Id: 39138 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:999a:b0:a4:2148:650a with SMTP id ve26csp500403pzb; Thu, 3 Nov 2022 05:23:55 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7ZOyNHzpLgE2yZKs2fbC2o2CoQIiLPEpMgAYUt0F2WDUeuo+vdzm8fqB3y/TAQr0QYLWcw X-Received: by 2002:aa7:db01:0:b0:461:4dc9:74e with SMTP id t1-20020aa7db01000000b004614dc9074emr30053862eds.139.1667478234872; Thu, 03 Nov 2022 05:23:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667478234; cv=none; d=google.com; s=arc-20160816; b=okm9drIOoYhw5qtN4CGkK0lGAp3UfMJiEHxd1jvXCB5EjSGLIQOAgZcJiGKaBetL2m DjLbZH2N9oRNpfMTsiu6JQAQn5Ozny6z7+waqFH/FRXrgOT5NMc9o2/FhXK8owy9/fwd mtEiQ9IiWLe8+rteYLm3VbmSh/4obcfSjNpEwqOABoyI4d6XZQz8qGk/xVWmqjGv493B gZJedJWY0yQAujB6F2ZGPsL8qDF2gMPDQDqRcJnzJTfriArprrmjvbn4GDizO4lw7y5v Ytj3TpqPLC/Qj6AgRahbOs0w2zAZj7X2l14NFmhRUtwtlBKvGCzkBn5sju3HPSv2w58M Itiw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=0q+aJ1KzBoWK9wcS6uYkJlR+zpAKBUuV3yqNNdT4+fo=; b=xmynJZ+8CNoh1jF9WuKTDDpfiuglwHqdcPvaP66I4Ibc97tG9J3DndgKof6UC9l4e1 z+TAP2B08SwOCqEzahNuCJ9p+lIMvp+p0DU28hehpoomOJyNJK1ExiRFMRQXt8wB6IUi fkNaP2eZc6b2vZVsvn4hRN/ora6rDEwM3Svp2+m9hkNtaCG1wTb9eYpjJbPlnvi3yg22 QFTBVtkHdLBUuRUC+PtbTQ57gkEOqesf5brqi19DSJ3kwOp4nmc6ehh09JJ0OJKO6Wna U9T/WpZeInapikxv6+i707SPyxe+QxWXyrZi8VhP/V4uJ1A1rH0UvnB4r+wyTWfZT6aR 8sgQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=O47RYM+U; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id g10-20020a1709067c4a00b0073d8659db5csi781404ejp.966.2022.11.03.05.23.54; Thu, 03 Nov 2022 05:23:54 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=O47RYM+U; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id F1DCB68BF7D; Thu, 3 Nov 2022 14:22:46 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f52.google.com (mail-ed1-f52.google.com [209.85.208.52]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9D3CE68BF66 for ; Thu, 3 Nov 2022 14:22:39 +0200 (EET) Received: by mail-ed1-f52.google.com with SMTP id f7so2708944edc.6 for ; Thu, 03 Nov 2022 05:22:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spin-digital-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Tt7GZ+TZTug4t9C+hnB/umgSlV4twSPBFdEoo4L1GU8=; b=O47RYM+UDj6XWok8LHx5ugDPxn15aRPQoNospzRjrVz8Jr70LUasGQCJ4Eq/NBAlxf c2HgAYQiLPiNgVnK+IXZSamkxSly2Lfwsf1g3ZaTZtVkwTlthv2s2rIg2u6bf7k0ufOZ SR9zZNe60UE5AAq4JNjF7JZBwLNkUP9YnEHKuqh6SC7eLSwSUeMbWSNaSAVjpn+ykMO5 A5nbjxU6g5XSYgXTGb0dHRubnxxtTOK/3KelDcxmkYoXgkR15NT9QjBYJwv0NSp5+RSK DWUBXc9SaA6+y/OzrqM6uKyJbaD29KHP1PRCg4XWkREHKE4Uw9F8oMQNlqxy7IJ5CUGK 3rew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Tt7GZ+TZTug4t9C+hnB/umgSlV4twSPBFdEoo4L1GU8=; b=2C+OZuEp8gJRYcgow/LbG/gn3HgwhLsx28vZMHT70v75meP0E7m55IhcNlVEWyBN+9 pMiCatj8TF0TX5OFQGDkCpI/j/0UyOXcQ6C+XuP6hecegaRhIDv4r7GlLKbWyJTWk7op a86Zr6zuKpWA8GsUz1ENMtb3/3pnaihALWIiyY7b1IALX5PhCmxNKxxfXBlUmnBGeoIN SkdE97Vbc2PInz/fEr5fY2Vjn52MjEcvAwYzpQOopJLIihut/qyfMigKbEQfbeHOwYLZ AfnVKsva/jBFeIlN7gxFyimrl9GMb88tnYqAMYVCQZjR8ihaqat0f/HphNs0eD6WDzpD dl3Q== X-Gm-Message-State: ACrzQf3Bf+Np8uf1hdkzXAdfvQAUlM+lWibABR4oI2myoE0eu1KKoBwn TFOrnQ/hVz0922IvsmBe7HfI1bmJGbMYkQ== X-Received: by 2002:aa7:c452:0:b0:463:14dd:2093 with SMTP id n18-20020aa7c452000000b0046314dd2093mr26643606edr.48.1667478157882; Thu, 03 Nov 2022 05:22:37 -0700 (PDT) Received: from thomas-win.localdomain ([213.138.44.237]) by smtp.gmail.com with ESMTPSA id c7-20020a0564021f8700b0045c010d0584sm457117edc.47.2022.11.03.05.22.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 03 Nov 2022 05:22:37 -0700 (PDT) From: Thomas Siedel To: ffmpeg-devel@ffmpeg.org Date: Thu, 3 Nov 2022 13:21:55 +0100 Message-Id: <20221103122158.27169-8-thomas.ff@spin-digital.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221103122158.27169-1-thomas.ff@spin-digital.com> References: <20221103122158.27169-1-thomas.ff@spin-digital.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3 07/10] avcodec: add external decoder libvvdec for H266/VVC X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Thomas Siedel Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 7F3+MBgY6NkC Add external decoder VVdeC for H266/VVC decoding. Register new decoder libvvdec. Add vvc_parse_extradata to support parse/probe of vvcC stream input. Add vvc_paramset that implements the parser of vvcC configuration boxes. Add libvvdec to wrap the vvdec interface. Enable decoder by adding --enable-libvvdec in configure step. Signed-off-by: Thomas Siedel --- configure | 5 + libavcodec/Makefile | 1 + libavcodec/allcodecs.c | 1 + libavcodec/libvvdec.c | 511 ++++++++++++++++ libavcodec/vvc_paramset.c | 972 +++++++++++++++++++++++++++++++ libavcodec/vvc_paramset.h | 429 ++++++++++++++ libavcodec/vvc_parse_extradata.c | 242 ++++++++ libavcodec/vvc_parse_extradata.h | 36 ++ 8 files changed, 2197 insertions(+) create mode 100644 libavcodec/libvvdec.c create mode 100644 libavcodec/vvc_paramset.c create mode 100644 libavcodec/vvc_paramset.h create mode 100644 libavcodec/vvc_parse_extradata.c create mode 100644 libavcodec/vvc_parse_extradata.h diff --git a/configure b/configure index 7dbc939ed9..5893c81cf4 100755 --- a/configure +++ b/configure @@ -288,6 +288,7 @@ External library support: --enable-libvorbis enable Vorbis en/decoding via libvorbis, native implementation exists [no] --enable-libvpx enable VP8 and VP9 de/encoding via libvpx [no] + --enable-libvvdec enable VVC decoding via vvdec [no] --enable-libwebp enable WebP encoding via libwebp [no] --enable-libx264 enable H.264 encoding via x264 [no] --enable-libx265 enable HEVC encoding via x265 [no] @@ -1875,6 +1876,7 @@ EXTERNAL_LIBRARY_LIST=" libvmaf libvorbis libvpx + libvvdec libwebp libxml2 libzimg @@ -3405,6 +3407,8 @@ libvpx_vp8_decoder_deps="libvpx" libvpx_vp8_encoder_deps="libvpx" libvpx_vp9_decoder_deps="libvpx" libvpx_vp9_encoder_deps="libvpx" +libvvdec_decoder_deps="libvvdec" +libvvdec_decoder_select="vvc_mp4toannexb_bsf" libwebp_encoder_deps="libwebp" libwebp_anim_encoder_deps="libwebp" libx262_encoder_deps="libx262" @@ -6729,6 +6733,7 @@ enabled libvpx && { die "libvpx enabled but no supported decoders found" fi } +enabled libvvdec && require_pkg_config libvvdec "libvvdec >= 1.6.0" "vvdec/vvdec.h" vvdec_get_version enabled libwebp && { enabled libwebp_encoder && require_pkg_config libwebp "libwebp >= 0.2.0" webp/encode.h WebPGetEncoderVersion diff --git a/libavcodec/Makefile b/libavcodec/Makefile index c3789277e2..9d2107050f 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1103,6 +1103,7 @@ OBJS-$(CONFIG_LIBVPX_VP8_DECODER) += libvpxdec.o OBJS-$(CONFIG_LIBVPX_VP8_ENCODER) += libvpxenc.o OBJS-$(CONFIG_LIBVPX_VP9_DECODER) += libvpxdec.o libvpx.o OBJS-$(CONFIG_LIBVPX_VP9_ENCODER) += libvpxenc.o libvpx.o +OBJS-$(CONFIG_LIBVVDEC_DECODER) += libvvdec.o vvc_parse_extradata.o vvc_paramset.o OBJS-$(CONFIG_LIBWEBP_ENCODER) += libwebpenc_common.o libwebpenc.o OBJS-$(CONFIG_LIBWEBP_ANIM_ENCODER) += libwebpenc_common.o libwebpenc_animencoder.o OBJS-$(CONFIG_LIBX262_ENCODER) += libx264.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index 4f1d66cb0c..9813b291f4 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -793,6 +793,7 @@ extern const FFCodec ff_libvpx_vp8_encoder; extern const FFCodec ff_libvpx_vp8_decoder; extern FFCodec ff_libvpx_vp9_encoder; extern FFCodec ff_libvpx_vp9_decoder; +extern const FFCodec ff_libvvdec_decoder; /* preferred over libwebp */ extern const FFCodec ff_libwebp_anim_encoder; extern const FFCodec ff_libwebp_encoder; diff --git a/libavcodec/libvvdec.c b/libavcodec/libvvdec.c new file mode 100644 index 0000000000..28f3a548ca --- /dev/null +++ b/libavcodec/libvvdec.c @@ -0,0 +1,511 @@ +/* + * H.266 decoding using the VVdeC library + * + * Copyright (C) 2022, Thomas Siedel + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "config_components.h" + +#include + +#include "libavutil/common.h" +#include "libavutil/avutil.h" +#include "libavutil/pixdesc.h" +#include "libavutil/opt.h" +#include "libavutil/imgutils.h" +#include "libavutil/frame.h" +#include "libavutil/mastering_display_metadata.h" +#include "libavutil/log.h" + +#include "avcodec.h" +#include "codec_internal.h" +#include "decode.h" +#include "internal.h" +#include "profiles.h" + +#include "vvc_paramset.h" +#include "vvc_parse_extradata.h" + +typedef struct VVdeCContext { + AVClass *av_class; + vvdecDecoder* vvdecDec; + vvdecParams vvdecParams; + VVCParamSets ps; + int is_nalff; + int nal_length_size; + bool bFlush; + AVBufferPool *pools[3]; /** Pools for each data plane. */ + int pool_size[3]; +}VVdeCContext; + + +static void ff_vvdec_log_callback(void *avctx, int level, const char *fmt, va_list args ) +{ + vfprintf( level == 1 ? stderr : stdout, fmt, args ); +} + +static void* ff_vvdec_buffer_allocator( void *ctx, vvdecComponentType comp, uint32_t size, uint32_t alignment, void** allocator ) +{ + AVBufferRef *buf; + VVdeCContext *s; + int plane; + + uint32_t alignedsize = FFALIGN(size, alignment); + s = (VVdeCContext*)ctx; + plane = (int)comp; + + if ( plane < 0 || plane > 3 ) + return NULL; + + if ( alignedsize != s->pool_size[plane]) { + av_buffer_pool_uninit(&s->pools[plane]); + s->pools[plane] = av_buffer_pool_init(alignedsize, NULL); + if (!s->pools[plane]) { + s->pool_size[plane] = 0; + return NULL; + } + s->pool_size[plane] = alignedsize; + } + + buf = av_buffer_pool_get(s->pools[plane]); + if (!buf) + return NULL; + + *allocator = (void*)buf; + return buf->data; +} + +static void ff_vvdec_buffer_unref( void *ctx, void* allocator ) +{ + AVBufferRef *buf = (AVBufferRef*)allocator; + av_buffer_unref( &buf ); +} + +static void ff_vvdec_printParameterInfo( AVCodecContext *avctx, vvdecParams* params ) +{ + av_log(avctx, AV_LOG_DEBUG, "Version info: vvdec %s ( threads %d)\n",vvdec_get_version(), params->threads); +} + +static int ff_vvdec_set_pix_fmt(AVCodecContext *avctx, vvdecFrame* frame ) +{ + if( NULL != frame->picAttributes && NULL != frame->picAttributes->vui && + frame->picAttributes->vui->colourDescriptionPresentFlag ) { + avctx->color_trc = frame->picAttributes->vui->transferCharacteristics; + avctx->color_primaries = frame->picAttributes->vui->colourPrimaries; + avctx->colorspace = frame->picAttributes->vui->matrixCoefficients; + } + else { + avctx->color_primaries = AVCOL_PRI_UNSPECIFIED; + avctx->color_trc = AVCOL_TRC_UNSPECIFIED; + avctx->colorspace = AVCOL_SPC_UNSPECIFIED; + } + + if( NULL != frame->picAttributes && NULL != frame->picAttributes->vui && + frame->picAttributes->vui->videoSignalTypePresentFlag) { + avctx->color_range = frame->picAttributes->vui->videoFullRangeFlag ? + AVCOL_RANGE_JPEG : AVCOL_RANGE_MPEG; + } + else { + avctx->color_range = AVCOL_RANGE_MPEG; + } + + switch ( frame->colorFormat ) { + case VVDEC_CF_YUV420_PLANAR: + if (frame->bitDepth == 8) { + avctx->pix_fmt = frame->numPlanes == 1 ? + AV_PIX_FMT_GRAY8 : AV_PIX_FMT_YUV420P; + avctx->profile = FF_PROFILE_VVC_MAIN_10; + return 0; + } + else if (frame->bitDepth == 10) { + avctx->pix_fmt = frame->numPlanes == 1 ? + AV_PIX_FMT_GRAY10 : AV_PIX_FMT_YUV420P10; + avctx->profile = FF_PROFILE_VVC_MAIN_10; + return 0; + } + else { + return AVERROR_INVALIDDATA; + } + default: + return AVERROR_INVALIDDATA; + } +} + +static int set_side_data( AVCodecContext *avctx, AVFrame *avframe, vvdecFrame *frame ) +{ + vvdecSEI *sei; + VVdeCContext *s = (VVdeCContext*)avctx->priv_data; + + sei = vvdec_find_frame_sei( s->vvdecDec, VVDEC_MASTERING_DISPLAY_COLOUR_VOLUME, frame ); + if( sei ) { + // VVC uses a g,b,r ordering, which we convert to a more natural r,g,b + const int mapping[3] = {2, 0, 1}; + const int chroma_den = 50000; + const int luma_den = 10000; + int i; + vvdecSEIMasteringDisplayColourVolume* p; + AVMasteringDisplayMetadata *metadata = av_mastering_display_metadata_create_side_data(avframe); + p = (vvdecSEIMasteringDisplayColourVolume *)(sei->payload); + if ( p && metadata ) { + for (i = 0; i < 3; i++) { + const int j = mapping[i]; + metadata->display_primaries[i][0].num = p->primaries[j][0]; + metadata->display_primaries[i][0].den = chroma_den; + metadata->display_primaries[i][1].num = p->primaries[j][1]; + metadata->display_primaries[i][1].den = chroma_den; + } + metadata->white_point[0].num = p->whitePoint[0]; + metadata->white_point[0].den = chroma_den; + metadata->white_point[1].num = p->whitePoint[1]; + metadata->white_point[1].den = chroma_den; + + metadata->max_luminance.num = p->maxLuminance; + metadata->max_luminance.den = luma_den; + metadata->min_luminance.num = p->minLuminance; + metadata->min_luminance.den = luma_den; + metadata->has_luminance = 1; + metadata->has_primaries = 1; + + av_log(avctx, AV_LOG_DEBUG, "Mastering Display Metadata:\n"); + av_log(avctx, AV_LOG_DEBUG, + "r(%5.4f,%5.4f) g(%5.4f,%5.4f) b(%5.4f %5.4f) wp(%5.4f, %5.4f)\n", + av_q2d(metadata->display_primaries[0][0]), + av_q2d(metadata->display_primaries[0][1]), + av_q2d(metadata->display_primaries[1][0]), + av_q2d(metadata->display_primaries[1][1]), + av_q2d(metadata->display_primaries[2][0]), + av_q2d(metadata->display_primaries[2][1]), + av_q2d(metadata->white_point[0]), av_q2d(metadata->white_point[1])); + av_log(avctx, AV_LOG_DEBUG, + "min_luminance=%f, max_luminance=%f\n", + av_q2d(metadata->min_luminance), av_q2d(metadata->max_luminance)); + } + return 0; + } + + sei = vvdec_find_frame_sei( s->vvdecDec, VVDEC_CONTENT_LIGHT_LEVEL_INFO, frame ); + if( sei ) { + vvdecSEIContentLightLevelInfo* p = NULL; + AVContentLightMetadata *light = av_content_light_metadata_create_side_data(avframe); + p = (vvdecSEIContentLightLevelInfo *)(sei->payload); + if ( p && light) { + light->MaxCLL = p->maxContentLightLevel; + light->MaxFALL = p->maxPicAverageLightLevel; + } + + av_log(avctx, AV_LOG_DEBUG, "Content Light Level Metadata:\n"); + av_log(avctx, AV_LOG_DEBUG, "MaxCLL=%d, MaxFALL=%d\n", + light->MaxCLL, light->MaxFALL); + } + + return 0; +} + +static void export_stream_params( AVCodecContext *avctx, const VVCSPS *sps) +{ + avctx->coded_width = sps->pic_width_max_in_luma_samples; + avctx->coded_height = sps->pic_height_max_in_luma_samples; + avctx->width = sps->pic_width_max_in_luma_samples - + sps->conf_win_left_offset - + sps->conf_win_right_offset; + avctx->height = sps->pic_height_max_in_luma_samples - + sps->conf_win_top_offset - + sps->conf_win_bottom_offset; + avctx->has_b_frames = sps->max_sublayers; + avctx->profile = sps->profile_tier_level.general_profile_idc; + avctx->level = sps->profile_tier_level.general_level_idc; + avctx->pix_fmt = sps->pix_fmt; + + avctx->color_range = sps->vui.full_range_flag ? AVCOL_RANGE_JPEG : AVCOL_RANGE_MPEG; + + if (sps->vui.colour_description_present_flag) { + avctx->color_primaries = sps->vui.colour_primaries; + avctx->color_trc = sps->vui.transfer_characteristics; + avctx->colorspace = sps->vui.matrix_coeffs; + } else { + avctx->color_primaries = AVCOL_PRI_UNSPECIFIED; + avctx->color_trc = AVCOL_TRC_UNSPECIFIED; + avctx->colorspace = AVCOL_SPC_UNSPECIFIED; + } + + avctx->chroma_sample_location = AVCHROMA_LOC_UNSPECIFIED; + if (sps->chroma_format_idc == 1) { + if (sps->vui.chroma_loc_info_present_flag) { + if (sps->vui.chroma_sample_loc_type_top_field <= 5) + avctx->chroma_sample_location = sps->vui.chroma_sample_loc_type_top_field + 1; + } else + avctx->chroma_sample_location = AVCHROMA_LOC_LEFT; + } + + if (sps->timing_hrd_params_present_flag && + sps->general_timing_hrd_parameters.num_units_in_tick && + sps->general_timing_hrd_parameters.time_scale) { + av_reduce(&avctx->framerate.den, &avctx->framerate.num, + sps->general_timing_hrd_parameters.num_units_in_tick, + sps->general_timing_hrd_parameters.time_scale, INT_MAX); + } +} + +static int vvc_decode_extradata( AVCodecContext *avctx, uint8_t *buf, int length, int first) +{ + VVdeCContext *s = (VVdeCContext*)avctx->priv_data; + int ret; + + ret = ff_vvc_decode_extradata(buf, length, &s->ps, &s->is_nalff, + &s->nal_length_size, avctx->err_recognition, + false, avctx); + if (ret < 0) + return ret; + + if ( s->ps.sps != NULL ) + export_stream_params(avctx, s->ps.sps); + + return 0; +} + +static av_cold int ff_vvdec_decode_init(AVCodecContext *avctx) +{ + int i; + VVdeCContext *s = (VVdeCContext*)avctx->priv_data; + + vvdec_params_default( &s->vvdecParams ); + s->vvdecParams.logLevel = VVDEC_DETAILS; + + if ( av_log_get_level() >= AV_LOG_DEBUG ) s->vvdecParams.logLevel = VVDEC_DETAILS; + else if( av_log_get_level() >= AV_LOG_VERBOSE ) s->vvdecParams.logLevel = VVDEC_INFO; // VVDEC_INFO will output per picture info + else if( av_log_get_level() >= AV_LOG_INFO ) s->vvdecParams.logLevel = VVDEC_WARNING; // AV_LOG_INFO is ffmpeg default + else s->vvdecParams.logLevel = VVDEC_SILENT; + + if( avctx->thread_count > 0 ) + s->vvdecParams.threads = avctx->thread_count; // number of worker threads (should not exceed the number of physical cpu's) + else + s->vvdecParams.threads = -1; // get max cpus + + ff_vvdec_printParameterInfo( avctx, &s->vvdecParams ); + + // using buffer allocation by using AVBufferPool + s->vvdecParams.opaque = avctx->priv_data; + s->vvdecDec = vvdec_decoder_open_with_allocator( &s->vvdecParams, + ff_vvdec_buffer_allocator, ff_vvdec_buffer_unref ); + + + if( !s->vvdecDec ) { + av_log(avctx, AV_LOG_ERROR, "cannot init vvc decoder\n" ); + return -1; + } + + vvdec_set_logging_callback( s->vvdecDec, ff_vvdec_log_callback ); + + s->bFlush = false; + s->is_nalff = 0; + s->nal_length_size = 0; + + for (i = 0; i < FF_ARRAY_ELEMS(s->pools); i++) { + s->pools[i] = NULL; + s->pool_size[i] = 0; + } + + if (!avctx->internal->is_copy) { + if (avctx->extradata_size > 0 && avctx->extradata) { + int ret = vvc_decode_extradata(avctx, avctx->extradata, avctx->extradata_size, 1); + if (ret < 0) { + return ret; + } + } + } + + return 0; +} + +static av_cold int ff_vvdec_decode_close(AVCodecContext *avctx) +{ + VVdeCContext *s = (VVdeCContext*)avctx->priv_data; + + for (int i = 0; i < FF_ARRAY_ELEMS(s->pools); i++) { + av_buffer_pool_uninit(&s->pools[i]); + s->pool_size[i] = 0; + } + + if( 0 != vvdec_decoder_close(s->vvdecDec) ) { + av_log(avctx, AV_LOG_ERROR, "cannot close vvdec\n" ); + return -1; + } + + ff_vvc_ps_uninit( &s->ps ); + s->bFlush = false; + return 0; +} + +static av_cold int ff_vvdec_decode_frame( AVCodecContext *avctx, AVFrame *data, + int *got_frame, AVPacket *avpkt ) +{ + VVdeCContext *s = avctx->priv_data; + AVFrame *avframe = data; + + int ret = 0; + vvdecFrame *frame = NULL; + + if ( avframe ) { + if( !avpkt->size && !s->bFlush ) + s->bFlush = true; + + if( s->bFlush ) + ret = vvdec_flush( s->vvdecDec, &frame ); + else { + vvdecAccessUnit accessUnit; + vvdec_accessUnit_default( &accessUnit ); + accessUnit.payload = avpkt->data; + accessUnit.payloadSize = avpkt->size; + accessUnit.payloadUsedSize = avpkt->size; + + accessUnit.cts = avpkt->pts; accessUnit.ctsValid = true; + accessUnit.dts = avpkt->dts; accessUnit.dtsValid = true; + + ret = vvdec_decode( s->vvdecDec, &accessUnit, &frame ); + } + + if( ret < 0 ) { + if( ret == VVDEC_EOF ) + s->bFlush = true; + else if ( ret != VVDEC_TRY_AGAIN ) { + av_log(avctx, AV_LOG_ERROR, "error in vvdec::decode - ret:%d - %s\n", ret, vvdec_get_last_error(s->vvdecDec)); + return AVERROR(EINVAL); \ + } + } + else if( NULL != frame ) { + const uint8_t * src_data[4] = { frame->planes[0].ptr, frame->planes[1].ptr, frame->planes[2].ptr, NULL }; + const int src_linesizes[4] = { (int)frame->planes[0].stride, (int)frame->planes[1].stride, (int)frame->planes[2].stride, 0 }; + + if (( ret = ff_vvdec_set_pix_fmt(avctx, frame)) < 0) { + av_log(avctx, AV_LOG_ERROR, "Unsupported output colorspace (%d) / bit_depth (%d)\n", + frame->colorFormat, frame->bitDepth); + goto fail; + } + + if( avctx->pix_fmt != AV_PIX_FMT_YUV420P && avctx->pix_fmt != AV_PIX_FMT_YUV420P10LE ) { + av_log(avctx, AV_LOG_ERROR, "Unsupported output colorspace (%d) / bit_depth (%d)\n", + frame->colorFormat, frame->bitDepth ); + ret = AVERROR_INVALIDDATA; + goto fail; + } + + if ((int)frame->width != avctx->width || (int)frame->height != avctx->height) { + av_log(avctx, AV_LOG_INFO, "dimension change! %dx%d -> %dx%d\n", + avctx->width, avctx->height, frame->width, frame->height); + + ret = ff_set_dimensions(avctx, frame->width, frame->height); + if (ret < 0) + goto fail; + } + + if( frame->planes[0].allocator ) + avframe->buf[0] = av_buffer_ref( (AVBufferRef *)frame->planes[0].allocator); + if( frame->planes[1].allocator ) + avframe->buf[1] = av_buffer_ref( (AVBufferRef *)frame->planes[1].allocator); + if( frame->planes[2].allocator ) + avframe->buf[2] = av_buffer_ref( (AVBufferRef *)frame->planes[2].allocator); + + for (int i = 0; i < 4; i++) { + avframe->data[i] = (uint8_t *)src_data[i]; + avframe->linesize[i] = src_linesizes[i]; + } + + ret = ff_decode_frame_props(avctx, avframe); + if (ret < 0) + goto fail; + + if( frame->picAttributes ) { + avframe->key_frame = frame->picAttributes->isRefPic; + avframe->pict_type = (frame->picAttributes->sliceType != VVDEC_SLICETYPE_UNKNOWN) ? + frame->picAttributes->sliceType+1 : AV_PICTURE_TYPE_NONE; + } + + ret = set_side_data(avctx, avframe, frame ); + if (ret < 0) + goto fail; + + if( 0 != vvdec_frame_unref( s->vvdecDec, frame ) ) + av_log(avctx, AV_LOG_ERROR, "cannot free picture memory\n"); + + *got_frame = 1; + } + } + + return avpkt->size; + +fail: + if (frame) { + if( frame->planes[0].allocator ) + av_buffer_unref( (AVBufferRef **)&frame->planes[0].allocator ); + if( frame->planes[1].allocator ) + av_buffer_unref( (AVBufferRef **)&frame->planes[1].allocator ); + if( frame->planes[2].allocator ) + av_buffer_unref( (AVBufferRef **)&frame->planes[2].allocator ); + + vvdec_frame_unref( s->vvdecDec, frame ); + } + return ret; +} + +static av_cold void ff_vvdec_decode_flush(AVCodecContext *avctx) +{ + VVdeCContext *s = (VVdeCContext*)avctx->priv_data; + + if( 0 != vvdec_decoder_close(s->vvdecDec) ) + av_log(avctx, AV_LOG_ERROR, "cannot close vvdec during flush\n" ); + + s->vvdecDec = vvdec_decoder_open_with_allocator( &s->vvdecParams, ff_vvdec_buffer_allocator, ff_vvdec_buffer_unref ); + if( !s->vvdecDec ) + av_log(avctx, AV_LOG_ERROR, "cannot reinit vvdec during flush\n" ); + + vvdec_set_logging_callback( s->vvdecDec, ff_vvdec_log_callback ); + + s->bFlush = false; +} + +static const enum AVPixelFormat pix_fmts_vvc[] = { + AV_PIX_FMT_YUV420P, + AV_PIX_FMT_YUV420P10LE, + AV_PIX_FMT_NONE +}; + +static const AVClass class_libvvdec = { + .class_name = "libvvdec-vvc decoder", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, +}; + +FFCodec ff_libvvdec_decoder = { + .p.name = "libvvdec", + .p.long_name = "H.266 / VVC Decoder VVdeC", + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_VVC, + .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_OTHER_THREADS, + .p.profiles = NULL_IF_CONFIG_SMALL(ff_vvc_profiles), + .p.priv_class = &class_libvvdec, + .p.wrapper_name = "libvvdec", + .priv_data_size = sizeof(VVdeCContext), + .p.pix_fmts = pix_fmts_vvc, + .init = ff_vvdec_decode_init, + FF_CODEC_DECODE_CB(ff_vvdec_decode_frame), + .close = ff_vvdec_decode_close, + .flush = ff_vvdec_decode_flush, + .bsfs = "vvc_mp4toannexb", + .caps_internal = FF_CODEC_CAP_AUTO_THREADS, +}; diff --git a/libavcodec/vvc_paramset.c b/libavcodec/vvc_paramset.c new file mode 100644 index 0000000000..a016f57a37 --- /dev/null +++ b/libavcodec/vvc_paramset.c @@ -0,0 +1,972 @@ +/* + * VVC Parameter Set decoding + * + * Copyright (c) 2022, Thomas Siedel + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/imgutils.h" +#include "golomb.h" +#include "vvc_paramset.h" + +static void remove_sps(VVCParamSets *s, int id) +{ + if (s->sps_list[id]) { + if (s->sps == (const VVCSPS*)s->sps_list[id]->data) + s->sps = NULL; + + av_assert0(!(s->sps_list[id] && s->sps == (VVCSPS*)s->sps_list[id]->data)); + } + av_buffer_unref(&s->sps_list[id]); +} + +static int decode_general_constraints_info(GetBitContext *gb, AVCodecContext *avctx, + VVCGeneralConstraintsInfo *gci) +{ + int i; + gci->gci_present_flag = get_bits1(gb); + + if (gci->gci_present_flag) { + /* general */ + gci->gci_intra_only_constraint_flag = get_bits1(gb); + gci->gci_all_layers_independent_constraint_flag = get_bits1(gb); + gci->gci_one_au_only_constraint_flag = get_bits1(gb); + + /* picture format */ + gci->gci_sixteen_minus_max_bitdepth_constraint_idc = get_bits(gb,4); + gci->gci_three_minus_max_chroma_format_constraint_idc = get_bits(gb,2); + + /* NAL unit type related */ + gci->gci_no_mixed_nalu_types_in_pic_constraint_flag = get_bits1(gb); + gci->gci_no_trail_constraint_flag = get_bits1(gb); + gci->gci_no_stsa_constraint_flag = get_bits1(gb); + gci->gci_no_rasl_constraint_flag = get_bits1(gb); + gci->gci_no_radl_constraint_flag = get_bits1(gb); + gci->gci_no_idr_constraint_flag = get_bits1(gb); + gci->gci_no_cra_constraint_flag = get_bits1(gb); + gci->gci_no_gdr_constraint_flag = get_bits1(gb); + gci->gci_no_aps_constraint_flag = get_bits1(gb); + gci->gci_no_idr_rpl_constraint_flag = get_bits1(gb); + + /* tile, slice, subpicture partitioning */ + gci->gci_one_tile_per_pic_constraint_flag = get_bits1(gb); + gci->gci_pic_header_in_slice_header_constraint_flag = get_bits1(gb); + gci->gci_one_slice_per_pic_constraint_flag = get_bits1(gb); + gci->gci_no_rectangular_slice_constraint_flag = get_bits1(gb); + gci->gci_one_slice_per_subpic_constraint_flag = get_bits1(gb); + gci->gci_no_subpic_info_constraint_flag = get_bits1(gb); + + /* CTU and block partitioning */ + gci->gci_three_minus_max_log2_ctu_size_constraint_idc = get_bits(gb,2); + gci->gci_no_partition_constraints_override_constraint_flag = get_bits1(gb); + gci->gci_no_mtt_constraint_flag = get_bits1(gb); + gci->gci_no_qtbtt_dual_tree_intra_constraint_flag = get_bits1(gb); + + /* intra */ + gci->gci_no_palette_constraint_flag = get_bits1(gb); + gci->gci_no_ibc_constraint_flag = get_bits1(gb); + gci->gci_no_isp_constraint_flag = get_bits1(gb); + gci->gci_no_mrl_constraint_flag = get_bits1(gb); + gci->gci_no_mip_constraint_flag = get_bits1(gb); + gci->gci_no_cclm_constraint_flag = get_bits1(gb); + + /* inter */ + gci->gci_no_ref_pic_resampling_constraint_flag = get_bits1(gb); + gci->gci_no_res_change_in_clvs_constraint_flag = get_bits1(gb); + gci->gci_no_weighted_prediction_constraint_flag = get_bits1(gb); + gci->gci_no_ref_wraparound_constraint_flag = get_bits1(gb); + gci->gci_no_temporal_mvp_constraint_flag = get_bits1(gb); + gci->gci_no_sbtmvp_constraint_flag = get_bits1(gb); + gci->gci_no_amvr_constraint_flag = get_bits1(gb); + gci->gci_no_bdof_constraint_flag = get_bits1(gb); + gci->gci_no_smvd_constraint_flag = get_bits1(gb); + gci->gci_no_dmvr_constraint_flag = get_bits1(gb); + gci->gci_no_mmvd_constraint_flag = get_bits1(gb); + gci->gci_no_affine_motion_constraint_flag = get_bits1(gb); + gci->gci_no_prof_constraint_flag = get_bits1(gb); + gci->gci_no_bcw_constraint_flag = get_bits1(gb); + gci->gci_no_ciip_constraint_flag = get_bits1(gb); + gci->gci_no_gpm_constraint_flag = get_bits1(gb); + + /* transform, quantization, residual */ + gci->gci_no_luma_transform_size_64_constraint_flag = get_bits1(gb); + gci->gci_no_transform_skip_constraint_flag = get_bits1(gb); + gci->gci_no_bdpcm_constraint_flag = get_bits1(gb); + gci->gci_no_mts_constraint_flag = get_bits1(gb); + gci->gci_no_lfnst_constraint_flag = get_bits1(gb); + gci->gci_no_joint_cbcr_constraint_flag = get_bits1(gb); + gci->gci_no_sbt_constraint_flag = get_bits1(gb); + gci->gci_no_act_constraint_flag = get_bits1(gb); + gci->gci_no_explicit_scaling_list_constraint_flag = get_bits1(gb); + gci->gci_no_dep_quant_constraint_flag = get_bits1(gb); + gci->gci_no_sign_data_hiding_constraint_flag = get_bits1(gb); + gci->gci_no_cu_qp_delta_constraint_flag = get_bits1(gb); + gci->gci_no_chroma_qp_offset_constraint_flag = get_bits1(gb); + + /* loop filter */ + gci->gci_no_sao_constraint_flag = get_bits1(gb); + gci->gci_no_alf_constraint_flag = get_bits1(gb); + gci->gci_no_ccalf_constraint_flag = get_bits1(gb); + gci->gci_no_lmcs_constraint_flag = get_bits1(gb); + gci->gci_no_ladf_constraint_flag = get_bits1(gb); + gci->gci_no_virtual_boundaries_constraint_flag = get_bits1(gb); + gci->gci_num_reserved_bits = get_bits(gb,8); + for (i = 0; i < gci->gci_num_reserved_bits; i++) { + gci->gci_reserved_zero_bit[i] = get_bits1(gb); + } + } + + align_get_bits(gb); + + return 0; +} + + +static int decode_profile_tier_level(GetBitContext *gb, AVCodecContext *avctx, + VVCProfileTierLevel *ptl, + int profile_tier_present_flag, + int max_num_sub_layers_minus1 ) +{ + int i; + + if (profile_tier_present_flag) { + ptl->general_profile_idc = get_bits(gb, 7); + ptl->general_tier_flag = get_bits1(gb); + } + ptl->general_level_idc = get_bits(gb, 8); + ptl->ptl_frame_only_constraint_flag = get_bits1(gb); + ptl->ptl_multilayer_enabled_flag = get_bits1(gb); + + if (profile_tier_present_flag) { + decode_general_constraints_info(gb, avctx, &ptl->general_constraints_info); + } + + for (i = max_num_sub_layers_minus1 - 1; i >= 0; i--) { + ptl->ptl_sublayer_level_present_flag[i] = get_bits1(gb); + } + + align_get_bits(gb); + + for (i = max_num_sub_layers_minus1 - 1; i >= 0; i--) { + if (ptl->ptl_sublayer_level_present_flag[i]) + ptl->sublayer_level_idc[i] = get_bits(gb, 8); + } + + if (profile_tier_present_flag) + { + ptl->ptl_num_sub_profiles = get_bits(gb, 8); + for (i = 0; i < ptl->ptl_num_sub_profiles; i++) + ptl->general_sub_profile_idc[i] = get_bits_long(gb, 32); + } + + return 0; +} + +static int decode_dpb_parameters(GetBitContext *gb, AVCodecContext *avctx, + VVCDpbParameters *dpb, + uint8_t max_sublayers_minus1, + uint8_t sublayer_info_flag) +{ + int i; + for (i = (sublayer_info_flag ? 0 : max_sublayers_minus1); + i <= max_sublayers_minus1; i++) { + dpb->dpb_max_dec_pic_buffering_minus1[i] = get_ue_golomb(gb); + dpb->dpb_max_num_reorder_pics[i] = get_ue_golomb(gb); + dpb->dpb_max_latency_increase_plus1[i] = get_ue_golomb(gb); + } + return 0; +} + +static int decode_ref_pic_list_struct(GetBitContext *gb, AVCodecContext *avctx, + VVCRefPicListStruct *current, + uint8_t list_idx, uint8_t rpls_idx, + const VVCSPS *sps) +{ + int i, j, num_direct_ref_layers = 0; + + current->num_ref_entries = get_ue_golomb( gb); + if (sps->long_term_ref_pics_flag && + rpls_idx < sps->num_ref_pic_lists[list_idx] && + current->num_ref_entries > 0) + current->ltrp_in_header_flag = get_bits1(gb); + if (sps->long_term_ref_pics_flag && + rpls_idx == sps->num_ref_pic_lists[list_idx]) + current->ltrp_in_header_flag = 1; + for (i = 0, j = 0; i < current->num_ref_entries; i++) { + if (sps->inter_layer_prediction_enabled_flag) + current->inter_layer_ref_pic_flag[i] = get_bits1(gb); + else + current->inter_layer_ref_pic_flag[i] = 0; + + if (!current->inter_layer_ref_pic_flag[i]) { + if (sps->long_term_ref_pics_flag) + current->st_ref_pic_flag[i] = get_bits1(gb); + else + current->st_ref_pic_flag[i] = 1; + if (current->st_ref_pic_flag[i]) { + int abs_delta_poc_st; + current->abs_delta_poc_st[i] = get_ue_golomb( gb); + if ((sps->weighted_pred_flag || + sps->weighted_bipred_flag) && i != 0) + abs_delta_poc_st = current->abs_delta_poc_st[i]; + else + abs_delta_poc_st = current->abs_delta_poc_st[i] + 1; + if (abs_delta_poc_st > 0) + current->strp_entry_sign_flag[i] = get_bits1(gb); + } else { + if (!current->ltrp_in_header_flag) { + uint8_t bits = sps->log2_max_pic_order_cnt_lsb_minus4 + 4; + current->rpls_poc_lsb_lt[j] = get_bits(gb, bits ); + j++; + } + } + } else { + if (num_direct_ref_layers == 0) { + av_log(avctx, AV_LOG_ERROR, + "num_direct_ref_layers needs > 0.\n"); + return AVERROR_INVALIDDATA; + } + current->ilrp_idx[i] = get_ue_golomb( gb); + } + } + return 0; +} + +static int decode_general_timing_hrd_parameters(GetBitContext *gb, VVCGeneralTimingHrdParameters *current) +{ + current->num_units_in_tick = get_bits_long(gb,32); + current->time_scale = get_bits_long(gb,32); + current->general_nal_hrd_params_present_flag = get_bits1(gb); + current->general_vcl_hrd_params_present_flag = get_bits1(gb); + + if (current->general_nal_hrd_params_present_flag || + current->general_vcl_hrd_params_present_flag) { + current->general_same_pic_timing_in_all_ols_flag = get_bits1(gb); + current->general_du_hrd_params_present_flag = get_bits1(gb); + if (current->general_du_hrd_params_present_flag) + current->tick_divisor_minus2 = get_bits(gb,8); + current->bit_rate_scale = get_bits(gb,4); + current->cpb_size_scale = get_bits(gb,4); + if (current->general_du_hrd_params_present_flag) + current->cpb_size_du_scale = get_bits(gb,4); + current->hrd_cpb_cnt_minus1 = get_ue_golomb_long(gb); + } else { + current->general_du_hrd_params_present_flag = 0; + } + return 0; +} + +static int decode_sublayer_hrd_parameters(GetBitContext *gb, + VVCSubLayerHRDParameters *current, int sublayer_id, + const VVCGeneralTimingHrdParameters *general) +{ + int i; + for (i = 0; i <= general->hrd_cpb_cnt_minus1; i++) { + current->bit_rate_value_minus1[sublayer_id][i] = get_ue_golomb_long(gb); + current->cpb_size_value_minus1[sublayer_id][i] = get_ue_golomb_long(gb); + if (general->general_du_hrd_params_present_flag) { + current->cpb_size_du_value_minus1[sublayer_id][i] = get_ue_golomb_long(gb); + current->bit_rate_du_value_minus1[sublayer_id][i] = get_ue_golomb_long(gb); + } + current->cbr_flag[sublayer_id][i] = get_bits1(gb); + } + return 0; +} + +static int decode_ols_timing_hrd_parameters( GetBitContext *gb, + VVCOlsTimingHrdParameters *current, + uint8_t first_sublayer, uint8_t max_sublayers_minus1, + const VVCGeneralTimingHrdParameters *general) +{ + int i; + for (i = first_sublayer; i <= max_sublayers_minus1; i++) { + current->fixed_pic_rate_general_flag[i] = get_bits1(gb); + if (!current->fixed_pic_rate_general_flag[i]) + current->fixed_pic_rate_within_cvs_flag[i] = get_bits1(gb); + else + current->fixed_pic_rate_within_cvs_flag[i] = 1; + if (current->fixed_pic_rate_within_cvs_flag[i]) { + current->elemental_duration_in_tc_minus1[i] = get_ue_golomb_long(gb); + current->low_delay_hrd_flag[i] = 0; + } else if ((general->general_nal_hrd_params_present_flag || + general->general_vcl_hrd_params_present_flag) && + general->hrd_cpb_cnt_minus1 == 0) { + current->low_delay_hrd_flag[i] = get_bits1(gb); + } else { + current->low_delay_hrd_flag[i] = 0; + } + if (general->general_nal_hrd_params_present_flag) + decode_sublayer_hrd_parameters(gb, + ¤t->nal_sub_layer_hrd_parameters, + i, general); + if (general->general_vcl_hrd_params_present_flag) + decode_sublayer_hrd_parameters(gb, + ¤t->nal_sub_layer_hrd_parameters, i, general); + } + return 0; +} + +static int decode_vui(GetBitContext *gb, AVCodecContext *avctx, + VVCVUI* vui, uint8_t chroma_format_idc ) +{ + vui->progressive_source_flag = get_bits1(gb); + vui->interlaced_source_flag = get_bits1(gb); + vui->non_packed_constraint_flag = get_bits1(gb); + vui->non_projected_constraint_flag = get_bits1(gb); + vui->aspect_ratio_info_present_flag = get_bits1(gb); + if (vui->aspect_ratio_info_present_flag) { + vui->aspect_ratio_constant_flag = get_bits1(gb); + vui->aspect_ratio_idc = get_bits(gb, 8); + if (vui->aspect_ratio_idc == 255) { + vui->sar_width = get_bits(gb, 16); + vui->sar_height = get_bits(gb, 16); + } + } else { + vui->aspect_ratio_constant_flag = 0; + vui->aspect_ratio_idc = 0; + } + vui->overscan_info_present_flag = get_bits1(gb); + if (vui->overscan_info_present_flag) + vui->overscan_appropriate_flag = get_bits1(gb); + vui->colour_description_present_flag = get_bits1(gb); + if (vui->colour_description_present_flag) { + vui->colour_primaries = get_bits(gb, 8); + vui->transfer_characteristics = get_bits(gb, 8); + vui->matrix_coeffs = get_bits(gb, 8); + av_log(avctx, AV_LOG_DEBUG, "colour_primaries: %d transfer_characteristics: %d matrix_coeffs: %d \n", + vui->colour_primaries, vui->transfer_characteristics, vui->matrix_coeffs); + + vui->full_range_flag = get_bits1(gb); + } else { + vui->colour_primaries = 2; + vui->transfer_characteristics = 2; + vui->matrix_coeffs = 2; + vui->full_range_flag = 0; + } + vui->chroma_loc_info_present_flag = get_bits1(gb); + if (chroma_format_idc != 1 && vui->chroma_loc_info_present_flag) { + av_log(avctx, AV_LOG_ERROR, "chroma_format_idc == %d," + "chroma_loc_info_present_flag can't not be true", chroma_format_idc); + return AVERROR_INVALIDDATA; + } + if (vui->chroma_loc_info_present_flag) { + if (vui->progressive_source_flag && + !vui->interlaced_source_flag) { + vui->chroma_sample_loc_type_frame = get_ue_golomb(gb); + } else { + vui->chroma_sample_loc_type_top_field = get_ue_golomb(gb); + vui->chroma_sample_loc_type_bottom_field = get_ue_golomb(gb); + } + } else { + if (chroma_format_idc == 1) { + vui->chroma_sample_loc_type_frame = get_ue_golomb(gb); + vui->chroma_sample_loc_type_top_field = get_ue_golomb(gb); + vui->chroma_sample_loc_type_bottom_field = get_ue_golomb(gb); + } + } + return 0; +} + +static int map_pixel_format(AVCodecContext *avctx, VVCSPS *sps) +{ + const AVPixFmtDescriptor *desc; + switch (sps->bit_depth) { + case 8: + if (sps->chroma_format_idc == 0) sps->pix_fmt = AV_PIX_FMT_GRAY8; + if (sps->chroma_format_idc == 1) sps->pix_fmt = AV_PIX_FMT_YUV420P; + if (sps->chroma_format_idc == 2) sps->pix_fmt = AV_PIX_FMT_YUV422P; + if (sps->chroma_format_idc == 3) sps->pix_fmt = AV_PIX_FMT_YUV444P; + break; + case 9: + if (sps->chroma_format_idc == 0) sps->pix_fmt = AV_PIX_FMT_GRAY9; + if (sps->chroma_format_idc == 1) sps->pix_fmt = AV_PIX_FMT_YUV420P9; + if (sps->chroma_format_idc == 2) sps->pix_fmt = AV_PIX_FMT_YUV422P9; + if (sps->chroma_format_idc == 3) sps->pix_fmt = AV_PIX_FMT_YUV444P9; + break; + case 10: + if (sps->chroma_format_idc == 0) sps->pix_fmt = AV_PIX_FMT_GRAY10; + if (sps->chroma_format_idc == 1) sps->pix_fmt = AV_PIX_FMT_YUV420P10; + if (sps->chroma_format_idc == 2) sps->pix_fmt = AV_PIX_FMT_YUV422P10; + if (sps->chroma_format_idc == 3) sps->pix_fmt = AV_PIX_FMT_YUV444P10; + break; + case 12: + if (sps->chroma_format_idc == 0) sps->pix_fmt = AV_PIX_FMT_GRAY12; + if (sps->chroma_format_idc == 1) sps->pix_fmt = AV_PIX_FMT_YUV420P12; + if (sps->chroma_format_idc == 2) sps->pix_fmt = AV_PIX_FMT_YUV422P12; + if (sps->chroma_format_idc == 3) sps->pix_fmt = AV_PIX_FMT_YUV444P12; + break; + default: + av_log(avctx, AV_LOG_ERROR, + "The following bit-depths are currently specified: 8, 9, 10 and 12 bits, " + "chroma_format_idc is %d, depth is %d\n", + sps->chroma_format_idc, sps->bit_depth); + return AVERROR_INVALIDDATA; + } + + desc = av_pix_fmt_desc_get(sps->pix_fmt); + if (!desc) + return AVERROR(EINVAL); + + return 0; +} + +int ff_vvc_parse_sps(VVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, + int apply_defdispwin, AVCodecContext *avctx) +{ + int i, j; + unsigned int ctb_log2_size_y, ctb_size_y, max_num_merge_cand, + tmp_width_val, tmp_height_val; + + sps->sps_id = get_bits(gb, 4); + sps->vps_id = get_bits(gb, 4); + + *sps_id = sps->sps_id; + + sps->max_sublayers = get_bits(gb, 3) + 1; + sps->chroma_format_idc = get_bits(gb, 2); + sps->log2_ctu_size = get_bits(gb, 2) + 5; + + ctb_log2_size_y = sps->log2_ctu_size; + ctb_size_y = 1 << ctb_log2_size_y; + + sps->ptl_dpb_hrd_params_present_flag = get_bits1(gb); + if (sps->ptl_dpb_hrd_params_present_flag) + decode_profile_tier_level( gb, avctx, &sps->profile_tier_level, 1, sps->max_sublayers-1 ); + + sps->gdr_enabled_flag = get_bits1(gb); + sps->ref_pic_resampling_enabled_flag = get_bits1(gb); + + if (sps->ref_pic_resampling_enabled_flag) + sps->res_change_in_clvs_allowed_flag = get_bits1(gb); + else + sps->res_change_in_clvs_allowed_flag = 0; + + sps->pic_width_max_in_luma_samples = get_ue_golomb(gb); + sps->pic_height_max_in_luma_samples = get_ue_golomb(gb); + + sps->conformance_window_flag = get_bits1(gb); + + if (sps->conformance_window_flag) { + sps->conf_win_left_offset = get_ue_golomb(gb); + sps->conf_win_right_offset = get_ue_golomb(gb); + sps->conf_win_top_offset = get_ue_golomb(gb); + sps->conf_win_bottom_offset = get_ue_golomb(gb); + } else { + sps->conf_win_left_offset = 0; + sps->conf_win_right_offset = 0; + sps->conf_win_top_offset = 0; + sps->conf_win_bottom_offset = 0; + } + + tmp_width_val = (sps->pic_width_max_in_luma_samples + ctb_size_y -1)/ctb_size_y; + tmp_height_val = (sps->pic_height_max_in_luma_samples + ctb_size_y -1)/ctb_size_y; + + sps->subpic_info_present_flag = get_bits1(gb); + if (sps->subpic_info_present_flag) { + sps->num_subpics_minus1 = get_ue_golomb(gb); + if (sps->num_subpics_minus1 > 0) { + sps->independent_subpics_flag = get_bits1(gb); + sps->subpic_same_size_flag = get_bits1(gb); + } + } + + if (sps->num_subpics_minus1 > 0) { + int wlen = av_ceil_log2(tmp_width_val); + int hlen = av_ceil_log2(tmp_height_val); + if (sps->pic_width_max_in_luma_samples > ctb_size_y) + sps->subpic_width_minus1[0] = get_bits(gb, wlen ); + else + sps->subpic_width_minus1[0] = tmp_width_val - 1; + + if (sps->pic_height_max_in_luma_samples > ctb_size_y) + sps->subpic_height_minus1[0] = get_bits(gb, hlen ); + else + sps->subpic_height_minus1[0] = tmp_height_val; + + if (!sps->independent_subpics_flag) { + sps->subpic_treated_as_pic_flag[0] = get_bits1(gb); + sps->loop_filter_across_subpic_enabled_flag[0] = get_bits1(gb); + } else { + sps->subpic_treated_as_pic_flag[0] = 1; + sps->loop_filter_across_subpic_enabled_flag[0] = 1; + } + + for (i = 1; i <= sps->num_subpics_minus1; i++) { + if (!sps->subpic_same_size_flag) { + if (sps->pic_width_max_in_luma_samples > ctb_size_y) + sps->subpic_ctu_top_left_x[i] = get_bits(gb, wlen ); + else + sps->subpic_ctu_top_left_x[i] = 0; + + if (sps->pic_height_max_in_luma_samples > ctb_size_y) + sps->subpic_ctu_top_left_y[i] = get_bits(gb, hlen ); + else + sps->subpic_ctu_top_left_y[i] = 0; + + if (i < sps->num_subpics_minus1 && + sps->pic_width_max_in_luma_samples > ctb_size_y) { + sps->subpic_width_minus1[i] = get_bits(gb, wlen ); + } else { + sps->subpic_width_minus1[i] = tmp_width_val - sps->subpic_ctu_top_left_x[i] - 1; + } + if (i < sps->num_subpics_minus1 && + sps->pic_height_max_in_luma_samples > ctb_size_y) { + sps->subpic_height_minus1[i] = get_bits(gb, hlen ); + } else { + sps->subpic_height_minus1[i] = tmp_height_val - sps->subpic_ctu_top_left_y[i] - 1; + } + } else { + int num_subpic_cols = + tmp_width_val / (sps->subpic_width_minus1[0] + 1); + sps->subpic_ctu_top_left_x[i] = (i % num_subpic_cols) *(sps->subpic_width_minus1[0] + 1); + sps->subpic_ctu_top_left_y[i] = (i / num_subpic_cols) *(sps->subpic_height_minus1[0] + 1); + sps->subpic_width_minus1[i] = sps->subpic_width_minus1[0]; + sps->subpic_height_minus1[i] = sps->subpic_height_minus1[0]; + } + if (!sps->independent_subpics_flag) { + sps->subpic_treated_as_pic_flag[i] = get_bits1(gb); + sps->loop_filter_across_subpic_enabled_flag[i] = get_bits1(gb); + } else { + sps->subpic_treated_as_pic_flag[i] = 1; + sps->loop_filter_across_subpic_enabled_flag[i] = 0; + } + } + sps->subpic_id_len_minus1 = get_ue_golomb(gb ); + + if ((1 << (sps->subpic_id_len_minus1 + 1)) < + sps->num_subpics_minus1 + 1) { + av_log(avctx, AV_LOG_ERROR, + "sps->subpic_id_len_minus1(%d) is too small\n", + sps->subpic_id_len_minus1); + return AVERROR_INVALIDDATA; + } + sps->subpic_id_mapping_explicitly_signalled_flag = get_bits1(gb); + if (sps->subpic_id_mapping_explicitly_signalled_flag) { + sps->subpic_id_mapping_present_flag = get_bits1(gb); + if (sps->subpic_id_mapping_present_flag) { + for (i = 0; i <= sps->num_subpics_minus1; i++) + sps->subpic_id[i] = get_bits(gb, sps->subpic_id_len_minus1 + 1 ); + } + } else { + sps->subpic_ctu_top_left_x[0]= 0; + sps->subpic_ctu_top_left_y[0]= 0; + sps->subpic_width_minus1[0] = tmp_width_val - 1; + sps->subpic_height_minus1[0]= tmp_height_val - 1; + } + } else { + sps->num_subpics_minus1 = 0; + sps->independent_subpics_flag = 1; + sps->subpic_same_size_flag = 0; + sps->subpic_id_mapping_explicitly_signalled_flag= 0; + sps->subpic_ctu_top_left_x[0] = 0; + sps->subpic_ctu_top_left_y[0] = 0; + sps->subpic_width_minus1[0] = tmp_width_val - 1; + sps->subpic_height_minus1[0] = tmp_height_val - 1; + } + + sps->bit_depth = get_ue_golomb(gb ) + 8; + + sps->entropy_coding_sync_enabled_flag = get_bits1(gb); + sps->entry_point_offsets_present_flag = get_bits1(gb); + + sps->log2_max_pic_order_cnt_lsb_minus4 = get_bits(gb, 4 ); + + sps->poc_msb_cycle_flag = get_bits1(gb); + if (sps->poc_msb_cycle_flag) + sps->poc_msb_cycle_len_minus1 = get_ue_golomb(gb); + + sps->num_extra_ph_bytes = get_bits(gb,2); + + for (i = 0; i < FFMIN(16,(sps->num_extra_ph_bytes * 8)); i++) { + sps->extra_ph_bit_present_flag[i] = get_bits1(gb); + } + + sps->num_extra_sh_bytes = get_bits(gb,2); + for (i = 0; i < FFMIN(16,(sps->num_extra_sh_bytes * 8)); i++) { + sps->extra_sh_bit_present_flag[i] = get_bits1(gb); + } + + if (sps->ptl_dpb_hrd_params_present_flag) { + if (sps->max_sublayers > 1) + sps->sublayer_dpb_params_flag = get_bits1(gb); + else + sps->sublayer_dpb_params_flag = 0; + + decode_dpb_parameters( gb, avctx, &sps->dpb_params, + sps->max_sublayers-1, + sps->sublayer_dpb_params_flag); + } + + sps->log2_min_luma_coding_block_size_minus2 = get_ue_golomb(gb); + sps->partition_constraints_override_enabled_flag = get_bits1(gb); + sps->log2_diff_min_qt_min_cb_intra_slice_luma = get_ue_golomb(gb); + sps->max_mtt_hierarchy_depth_intra_slice_luma = get_ue_golomb(gb); + + if (sps->max_mtt_hierarchy_depth_intra_slice_luma != 0) { + sps->log2_diff_max_bt_min_qt_intra_slice_luma = get_ue_golomb(gb); + sps->log2_diff_max_tt_min_qt_intra_slice_luma = get_ue_golomb(gb); + } else { + sps->log2_diff_max_bt_min_qt_intra_slice_luma = 0; + sps->log2_diff_max_tt_min_qt_intra_slice_luma = 0; + } + + if (sps->chroma_format_idc != 0) { + sps->qtbtt_dual_tree_intra_flag = get_bits1(gb); + } else { + sps->qtbtt_dual_tree_intra_flag = 0; + } + + if (sps->qtbtt_dual_tree_intra_flag) { + sps->log2_diff_min_qt_min_cb_intra_slice_chroma = get_ue_golomb(gb); + sps->max_mtt_hierarchy_depth_intra_slice_chroma = get_ue_golomb(gb); + if (sps->max_mtt_hierarchy_depth_intra_slice_chroma != 0) { + sps->log2_diff_max_bt_min_qt_intra_slice_chroma = get_ue_golomb(gb); + sps->log2_diff_max_tt_min_qt_intra_slice_chroma = get_ue_golomb(gb); + } + } else { + sps->log2_diff_min_qt_min_cb_intra_slice_chroma = 0; + sps->max_mtt_hierarchy_depth_intra_slice_chroma = 0; + } + if (sps->max_mtt_hierarchy_depth_intra_slice_chroma == 0) { + sps->log2_diff_max_bt_min_qt_intra_slice_chroma = 0; + sps->log2_diff_max_tt_min_qt_intra_slice_chroma = 0; + } + + sps->log2_diff_min_qt_min_cb_inter_slice = get_ue_golomb(gb); + + sps->max_mtt_hierarchy_depth_inter_slice = get_ue_golomb(gb); + if (sps->max_mtt_hierarchy_depth_inter_slice != 0) { + sps->log2_diff_max_bt_min_qt_inter_slice = get_ue_golomb(gb); + sps->log2_diff_max_tt_min_qt_inter_slice = get_ue_golomb(gb); + } else { + sps->log2_diff_max_bt_min_qt_inter_slice = 0; + sps->log2_diff_max_tt_min_qt_inter_slice = 0; + } + + if (ctb_size_y > 32) + sps->max_luma_transform_size_64_flag = get_bits1(gb); + else + sps->max_luma_transform_size_64_flag = 0; + + sps->transform_skip_enabled_flag = get_bits1(gb); + if (sps->transform_skip_enabled_flag) { + sps->log2_transform_skip_max_size_minus2 = get_ue_golomb(gb); + sps->bdpcm_enabled_flag = get_bits1(gb); + } + + sps->mts_enabled_flag = get_bits1(gb); + if (sps->mts_enabled_flag) { + sps->explicit_mts_intra_enabled_flag = get_bits1(gb); + sps->explicit_mts_inter_enabled_flag = get_bits1(gb); + } else { + sps->explicit_mts_intra_enabled_flag = 0; + sps->explicit_mts_inter_enabled_flag = 0; + } + + sps->lfnst_enabled_flag = get_bits1(gb); + + if (sps->chroma_format_idc != 0) { + uint8_t num_qp_tables; + sps->joint_cbcr_enabled_flag = get_bits1(gb); + sps->same_qp_table_for_chroma_flag = get_bits1(gb); + num_qp_tables = sps->same_qp_table_for_chroma_flag ? + 1 : (sps->joint_cbcr_enabled_flag ? 3 : 2); + for (i = 0; i < num_qp_tables; i++) { + sps->qp_table_start_minus26[i] = get_se_golomb(gb); + sps->num_points_in_qp_table_minus1[i] = get_ue_golomb(gb); + for (j = 0; j <= sps->num_points_in_qp_table_minus1[i]; j++) { + sps->delta_qp_in_val_minus1[i][j] = get_ue_golomb(gb); + sps->delta_qp_diff_val[i][j] = get_ue_golomb(gb); + } + } + } else { + sps->joint_cbcr_enabled_flag = 0; + sps->same_qp_table_for_chroma_flag = 0; + } + + sps->sao_enabled_flag = get_bits1(gb); + sps->alf_enabled_flag = get_bits1(gb); + if (sps->alf_enabled_flag && sps->chroma_format_idc) + sps->ccalf_enabled_flag = get_bits1(gb); + else + sps->ccalf_enabled_flag = 0; + + sps->lmcs_enabled_flag = get_bits1(gb); + sps->weighted_pred_flag = get_bits1(gb); + sps->weighted_bipred_flag = get_bits1(gb); + sps->long_term_ref_pics_flag = get_bits1(gb); + if (sps->vps_id > 0) + sps->inter_layer_prediction_enabled_flag = get_bits1(gb); + else + sps->inter_layer_prediction_enabled_flag = 0; + sps->idr_rpl_present_flag = get_bits1(gb); + sps->rpl1_same_as_rpl0_flag = get_bits1(gb); + + for (i = 0; i < (sps->rpl1_same_as_rpl0_flag ? 1 : 2); i++) { + sps->num_ref_pic_lists[i] = get_ue_golomb(gb); + for (j = 0; j < sps->num_ref_pic_lists[i]; j++) + decode_ref_pic_list_struct( gb, avctx, + &sps->ref_pic_list_struct[i][j], + i, j, sps); + } + + if (sps->rpl1_same_as_rpl0_flag) { + sps->num_ref_pic_lists[1] = sps->num_ref_pic_lists[0]; + for (j = 0; j < sps->num_ref_pic_lists[0]; j++) + memcpy(&sps->ref_pic_list_struct[1][j], + &sps->ref_pic_list_struct[0][j], + sizeof(sps->ref_pic_list_struct[0][j])); + } + + sps->ref_wraparound_enabled_flag = get_bits1(gb); + + sps->temporal_mvp_enabled_flag = get_bits1(gb); + if (sps->temporal_mvp_enabled_flag) + sps->sbtmvp_enabled_flag = get_bits1(gb); + else + sps->sbtmvp_enabled_flag = 0; + + sps->amvr_enabled_flag = get_bits1(gb); + sps->bdof_enabled_flag = get_bits1(gb); + if (sps->bdof_enabled_flag) + sps->bdof_control_present_in_ph_flag = get_bits1(gb); + else + sps->bdof_control_present_in_ph_flag = 0; + + sps->smvd_enabled_flag = get_bits1(gb); + sps->dmvr_enabled_flag = get_bits1(gb); + if (sps->dmvr_enabled_flag) + sps->dmvr_control_present_in_ph_flag = get_bits1(gb); + else + sps->dmvr_control_present_in_ph_flag = 0; + + sps->mmvd_enabled_flag = get_bits1(gb); + if (sps->mmvd_enabled_flag) + sps->mmvd_fullpel_only_enabled_flag = get_bits1(gb); + else + sps->mmvd_fullpel_only_enabled_flag = 0; + + sps->six_minus_max_num_merge_cand = get_ue_golomb(gb); + max_num_merge_cand = 6 - sps->six_minus_max_num_merge_cand; + + sps->sbt_enabled_flag = get_bits1(gb); + + sps->affine_enabled_flag = get_bits1(gb); + if (sps->affine_enabled_flag) { + sps->five_minus_max_num_subblock_merge_cand = get_ue_golomb(gb); + sps->param_affine_enabled_flag = get_bits1(gb); + if (sps->amvr_enabled_flag) + sps->affine_amvr_enabled_flag = get_bits1(gb); + else + sps->affine_amvr_enabled_flag = 0; + sps->affine_prof_enabled_flag = get_bits1(gb); + if (sps->affine_prof_enabled_flag) + sps->prof_control_present_in_ph_flag = get_bits1(gb); + else + sps->prof_control_present_in_ph_flag = 0; + } else { + sps->param_affine_enabled_flag = 0; + sps->affine_amvr_enabled_flag = 0; + sps->affine_prof_enabled_flag = 0; + sps->prof_control_present_in_ph_flag = 0; + } + + sps->bcw_enabled_flag = get_bits1(gb); + sps->ciip_enabled_flag = get_bits1(gb); + + if (max_num_merge_cand >= 2) { + sps->gpm_enabled_flag = get_bits1(gb); + if (sps->gpm_enabled_flag && max_num_merge_cand >= 3) + sps->max_num_merge_cand_minus_max_num_gpm_cand = get_ue_golomb(gb); + } else { + sps->gpm_enabled_flag = 0; + } + + sps->log2_parallel_merge_level_minus2 = get_ue_golomb(gb); + + sps->isp_enabled_flag = get_bits1(gb); + sps->mrl_enabled_flag = get_bits1(gb); + sps->mip_enabled_flag = get_bits1(gb); + + if (sps->chroma_format_idc != 0) + sps->cclm_enabled_flag = get_bits1(gb); + else + sps->cclm_enabled_flag = 0; + if (sps->chroma_format_idc == 1) { + sps->chroma_horizontal_collocated_flag = get_bits1(gb); + sps->chroma_vertical_collocated_flag = get_bits1(gb); + } else { + sps->chroma_horizontal_collocated_flag = 0; + sps->chroma_vertical_collocated_flag = 0; + } + + sps->palette_enabled_flag = get_bits1(gb); + if (sps->chroma_format_idc == 3 && + !sps->max_luma_transform_size_64_flag) + sps->act_enabled_flag = get_bits1(gb); + else + sps->act_enabled_flag = 0; + if (sps->transform_skip_enabled_flag || + sps->palette_enabled_flag) + sps->min_qp_prime_ts = get_ue_golomb(gb); + + sps->ibc_enabled_flag = get_bits1(gb); + if (sps->ibc_enabled_flag) + sps->six_minus_max_num_ibc_merge_cand = get_ue_golomb(gb); + + sps->ladf_enabled_flag = get_bits1(gb); + if (sps->ladf_enabled_flag) { + sps->num_ladf_intervals_minus2 = get_bits( gb, 2 ); + sps->ladf_lowest_interval_qp_offset = get_se_golomb(gb); + for (i = 0; i < sps->num_ladf_intervals_minus2 + 1; i++) { + sps->ladf_qp_offset[i] = get_se_golomb(gb); + sps->ladf_delta_threshold_minus1[i] = get_ue_golomb(gb); + } + } + + sps->explicit_scaling_list_enabled_flag = get_bits1(gb); + if (sps->lfnst_enabled_flag && + sps->explicit_scaling_list_enabled_flag) + sps->scaling_matrix_for_lfnst_disabled_flag = get_bits1(gb); + + if (sps->act_enabled_flag && + sps->explicit_scaling_list_enabled_flag) + sps->scaling_matrix_for_alternative_colour_space_disabled_flag = get_bits1(gb); + else + sps->scaling_matrix_for_alternative_colour_space_disabled_flag = 0; + if (sps->scaling_matrix_for_alternative_colour_space_disabled_flag) + sps->scaling_matrix_designated_colour_space_flag = get_bits1(gb); + + sps->dep_quant_enabled_flag = get_bits1(gb); + sps->sign_data_hiding_enabled_flag = get_bits1(gb); + + sps->virtual_boundaries_enabled_flag = get_bits1(gb); + if (sps->virtual_boundaries_enabled_flag) { + sps->virtual_boundaries_present_flag = get_bits1(gb); + if (sps->virtual_boundaries_present_flag) { + sps->num_ver_virtual_boundaries = get_ue_golomb(gb); + for (i = 0; i < sps->num_ver_virtual_boundaries; i++) + sps->virtual_boundary_pos_x_minus1[i] = get_ue_golomb(gb); + for (i = 0; i < sps->num_hor_virtual_boundaries; i++) + sps->virtual_boundary_pos_y_minus1[i] = get_ue_golomb(gb); + } + } else { + sps->virtual_boundaries_present_flag = 0; + sps->num_ver_virtual_boundaries = 0; + sps->num_hor_virtual_boundaries = 0; + } + + if (sps->ptl_dpb_hrd_params_present_flag) { + sps->timing_hrd_params_present_flag = get_bits1(gb); + if (sps->timing_hrd_params_present_flag) { + uint8_t first_sublayer; + decode_general_timing_hrd_parameters(gb, &sps->general_timing_hrd_parameters); + + if (sps->max_sublayers > 1) + sps->sublayer_cpb_params_present_flag = get_bits1(gb); + else + sps->sublayer_cpb_params_present_flag = 0; + + first_sublayer = sps->sublayer_cpb_params_present_flag ? + 0 : sps->max_sublayers-1; + + decode_ols_timing_hrd_parameters ( gb, + &sps->ols_timing_hrd_parameters, first_sublayer, + sps->max_sublayers-1, + &sps->general_timing_hrd_parameters); + } + } + + sps->field_seq_flag = get_bits1(gb); + sps->vui_parameters_present_flag = get_bits1(gb); + if (sps->vui_parameters_present_flag) { + sps->vui_payload_size_minus1 = get_ue_golomb(gb); + align_get_bits(gb); + decode_vui( gb, avctx, &sps->vui, sps->chroma_format_idc); + } + + sps->extension_flag = get_bits1(gb); + // TODO: parse sps extension flag and read extension data + + map_pixel_format( avctx, sps); + + return 0; +} + +int ff_vvc_decode_nal_sps( GetBitContext *gb, AVCodecContext *avctx, + VVCParamSets *ps, int apply_defdispwin) +{ + unsigned int sps_id; + int ret; + VVCSPS *sps; + AVBufferRef *sps_buf = av_buffer_allocz(sizeof(*sps)); + + if (!sps_buf) + return AVERROR(ENOMEM); + sps = (VVCSPS*)sps_buf->data; + + ret = ff_vvc_parse_sps(sps, gb, &sps_id, + apply_defdispwin, + avctx); + if (ret < 0) { + av_buffer_unref(&sps_buf); + return ret; + } + + if (avctx->debug & FF_DEBUG_BITSTREAM) + { + av_log(avctx, AV_LOG_DEBUG, + "Parsed SPS: id %d; coded wxh: %dx%d; " + "pix_fmt: %s.\n", + sps->sps_id, sps->pic_width_max_in_luma_samples, sps->pic_height_max_in_luma_samples, + av_get_pix_fmt_name(sps->pix_fmt)); + } + + /* check if this is a repeat of an already parsed SPS, then keep the + * original one otherwise drop all PPSes that depend on it (PPS,VPS not implemented yet) */ + if (ps->sps_list[sps_id] && + !memcmp(ps->sps_list[sps_id]->data, sps_buf->data, sps_buf->size)) { + av_buffer_unref(&sps_buf); + } else { + remove_sps(ps, sps_id); + ps->sps_list[sps_id] = sps_buf; + ps->sps = (VVCSPS*)ps->sps_list[sps_id]->data; + } + + // TODO: read PPS flag and data + + return 0; +} + +void ff_vvc_ps_uninit(VVCParamSets *ps) +{ + int i; + + for (i = 0; i < FF_ARRAY_ELEMS(ps->sps_list); i++) + av_buffer_unref(&ps->sps_list[i]); + + // TODO: if PPS, VPS is implemended it must be unrefed as well + // for (i = 0; i < FF_ARRAY_ELEMS(ps->vps_list); i++) + // av_buffer_unref(&ps->vps_list[i]); + // for (i = 0; i < FF_ARRAY_ELEMS(ps->pps_list); i++) + // av_buffer_unref(&ps->pps_list[i]); + + ps->sps = NULL; + //ps->pps = NULL; + //ps->vps = NULL; +} diff --git a/libavcodec/vvc_paramset.h b/libavcodec/vvc_paramset.h new file mode 100644 index 0000000000..c4bb2e509b --- /dev/null +++ b/libavcodec/vvc_paramset.h @@ -0,0 +1,429 @@ +/* + * VVC parameter set parsing + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_VVC_PARAMSET_H +#define AVCODEC_VVC_PARAMSET_H + +#include + +#include "libavutil/buffer.h" +#include "libavutil/pixfmt.h" +#include "libavutil/rational.h" + +#include "avcodec.h" +#include "get_bits.h" +#include "vvc.h" +#include "cbs_h266.h" + + +typedef struct VVCGeneralConstraintsInfo { + uint8_t gci_present_flag; + /* general */ + uint8_t gci_intra_only_constraint_flag; + uint8_t gci_all_layers_independent_constraint_flag; + uint8_t gci_one_au_only_constraint_flag; + + /* picture format */ + uint8_t gci_sixteen_minus_max_bitdepth_constraint_idc; + uint8_t gci_three_minus_max_chroma_format_constraint_idc; + + /* NAL unit type related */ + uint8_t gci_no_mixed_nalu_types_in_pic_constraint_flag; + uint8_t gci_no_trail_constraint_flag; + uint8_t gci_no_stsa_constraint_flag; + uint8_t gci_no_rasl_constraint_flag; + uint8_t gci_no_radl_constraint_flag; + uint8_t gci_no_idr_constraint_flag; + uint8_t gci_no_cra_constraint_flag; + uint8_t gci_no_gdr_constraint_flag; + uint8_t gci_no_aps_constraint_flag; + uint8_t gci_no_idr_rpl_constraint_flag; + + /* tile, slice, subpicture partitioning */ + uint8_t gci_one_tile_per_pic_constraint_flag; + uint8_t gci_pic_header_in_slice_header_constraint_flag; + uint8_t gci_one_slice_per_pic_constraint_flag; + uint8_t gci_no_rectangular_slice_constraint_flag; + uint8_t gci_one_slice_per_subpic_constraint_flag; + uint8_t gci_no_subpic_info_constraint_flag; + + /* CTU and block partitioning */ + uint8_t gci_three_minus_max_log2_ctu_size_constraint_idc; + uint8_t gci_no_partition_constraints_override_constraint_flag; + uint8_t gci_no_mtt_constraint_flag; + uint8_t gci_no_qtbtt_dual_tree_intra_constraint_flag; + + /* intra */ + uint8_t gci_no_palette_constraint_flag; + uint8_t gci_no_ibc_constraint_flag; + uint8_t gci_no_isp_constraint_flag; + uint8_t gci_no_mrl_constraint_flag; + uint8_t gci_no_mip_constraint_flag; + uint8_t gci_no_cclm_constraint_flag; + + /* inter */ + uint8_t gci_no_ref_pic_resampling_constraint_flag; + uint8_t gci_no_res_change_in_clvs_constraint_flag;; + uint8_t gci_no_weighted_prediction_constraint_flag; + uint8_t gci_no_ref_wraparound_constraint_flag; + uint8_t gci_no_temporal_mvp_constraint_flag; + uint8_t gci_no_sbtmvp_constraint_flag; + uint8_t gci_no_amvr_constraint_flag; + uint8_t gci_no_bdof_constraint_flag; + uint8_t gci_no_smvd_constraint_flag; + uint8_t gci_no_dmvr_constraint_flag; + uint8_t gci_no_mmvd_constraint_flag; + uint8_t gci_no_affine_motion_constraint_flag; + uint8_t gci_no_prof_constraint_flag; + uint8_t gci_no_bcw_constraint_flag; + uint8_t gci_no_ciip_constraint_flag; + uint8_t gci_no_gpm_constraint_flag; + + /* transform, quantization, residual */ + uint8_t gci_no_luma_transform_size_64_constraint_flag; + uint8_t gci_no_transform_skip_constraint_flag; + uint8_t gci_no_bdpcm_constraint_flag; + uint8_t gci_no_mts_constraint_flag; + uint8_t gci_no_lfnst_constraint_flag; + uint8_t gci_no_joint_cbcr_constraint_flag; + uint8_t gci_no_sbt_constraint_flag; + uint8_t gci_no_act_constraint_flag; + uint8_t gci_no_explicit_scaling_list_constraint_flag; + uint8_t gci_no_dep_quant_constraint_flag; + uint8_t gci_no_sign_data_hiding_constraint_flag; + uint8_t gci_no_cu_qp_delta_constraint_flag; + uint8_t gci_no_chroma_qp_offset_constraint_flag; + + /* loop filter */ + uint8_t gci_no_sao_constraint_flag; + uint8_t gci_no_alf_constraint_flag; + uint8_t gci_no_ccalf_constraint_flag; + uint8_t gci_no_lmcs_constraint_flag; + uint8_t gci_no_ladf_constraint_flag; + uint8_t gci_no_virtual_boundaries_constraint_flag; + uint8_t gci_num_reserved_bits; + uint8_t gci_reserved_zero_bit[255]; +} VVCGeneralConstraintsInfo; + +typedef struct VVCProfileTierLevel { + uint8_t general_profile_idc; + uint8_t general_tier_flag; + uint8_t general_level_idc; + uint8_t ptl_frame_only_constraint_flag; + uint8_t ptl_multilayer_enabled_flag; + VVCGeneralConstraintsInfo general_constraints_info; + uint8_t ptl_sublayer_level_present_flag[VVC_MAX_SUBLAYERS - 1]; + uint8_t sublayer_level_idc[VVC_MAX_SUBLAYERS - 1]; + uint8_t ptl_num_sub_profiles; + uint32_t general_sub_profile_idc[VVC_MAX_SUB_PROFILES]; + + uint8_t ptl_reserved_zero_bit; +} VVCProfileTierLevel; + +typedef struct VVCDpbParameters { + uint8_t dpb_max_dec_pic_buffering_minus1[VVC_MAX_SUBLAYERS]; + uint8_t dpb_max_num_reorder_pics[VVC_MAX_SUBLAYERS]; + uint8_t dpb_max_latency_increase_plus1[VVC_MAX_SUBLAYERS]; +} VVCDpbParameters; + +typedef struct VVCRefPicListStruct { + uint8_t num_ref_entries; + uint8_t ltrp_in_header_flag; + uint8_t inter_layer_ref_pic_flag[VVC_MAX_REF_ENTRIES]; + uint8_t st_ref_pic_flag[VVC_MAX_REF_ENTRIES]; + uint8_t abs_delta_poc_st[VVC_MAX_REF_ENTRIES]; + uint8_t strp_entry_sign_flag[VVC_MAX_REF_ENTRIES]; + uint8_t rpls_poc_lsb_lt[VVC_MAX_REF_ENTRIES]; + uint8_t ilrp_idx[VVC_MAX_REF_ENTRIES]; +} VVCRefPicListStruct; + +typedef struct VVCGeneralTimingHrdParameters { + uint32_t num_units_in_tick; + uint32_t time_scale; + uint8_t general_nal_hrd_params_present_flag; + uint8_t general_vcl_hrd_params_present_flag; + uint8_t general_same_pic_timing_in_all_ols_flag; + uint8_t general_du_hrd_params_present_flag; + uint8_t tick_divisor_minus2; + uint8_t bit_rate_scale; + uint8_t cpb_size_scale; + uint8_t cpb_size_du_scale; + uint8_t hrd_cpb_cnt_minus1; +} VVCGeneralTimingHrdParameters; + +typedef struct VVCSubLayerHRDParameters { + uint32_t bit_rate_value_minus1[VVC_MAX_SUBLAYERS][VVC_MAX_CPB_CNT]; + uint32_t cpb_size_value_minus1[VVC_MAX_SUBLAYERS][VVC_MAX_CPB_CNT]; + uint32_t cpb_size_du_value_minus1[VVC_MAX_SUBLAYERS][VVC_MAX_CPB_CNT]; + uint32_t bit_rate_du_value_minus1[VVC_MAX_SUBLAYERS][VVC_MAX_CPB_CNT]; + uint8_t cbr_flag[VVC_MAX_SUBLAYERS][VVC_MAX_CPB_CNT]; +} VVCSubLayerHRDParameters; + +typedef struct VVCOlsTimingHrdParameters { + uint8_t fixed_pic_rate_general_flag[VVC_MAX_SUBLAYERS]; + uint8_t fixed_pic_rate_within_cvs_flag[VVC_MAX_SUBLAYERS]; + uint16_t elemental_duration_in_tc_minus1[VVC_MAX_SUBLAYERS]; + uint8_t low_delay_hrd_flag[VVC_MAX_SUBLAYERS]; + VVCSubLayerHRDParameters nal_sub_layer_hrd_parameters; + VVCSubLayerHRDParameters vcl_sub_layer_hrd_parameters; +} VVCOlsTimingHrdParameters; + + +typedef struct VVCVUI { + uint8_t progressive_source_flag; + uint8_t interlaced_source_flag; + uint8_t non_packed_constraint_flag; + uint8_t non_projected_constraint_flag; + + uint8_t aspect_ratio_info_present_flag; + uint8_t aspect_ratio_constant_flag; + uint8_t aspect_ratio_idc; + + uint16_t sar_width; + uint16_t sar_height;; + + uint8_t overscan_info_present_flag; + uint8_t overscan_appropriate_flag; + + uint8_t colour_description_present_flag; + uint8_t colour_primaries; + + uint8_t transfer_characteristics; + uint8_t matrix_coeffs; + uint8_t full_range_flag; + + uint8_t chroma_loc_info_present_flag; + uint8_t chroma_sample_loc_type_frame; + uint8_t chroma_sample_loc_type_top_field; + uint8_t chroma_sample_loc_type_bottom_field; + //VVCExtensionData extension_data; +} VVCVUI; + +typedef struct VVCSPS { + uint8_t sps_id; + uint8_t vps_id; + uint8_t max_sublayers; + uint8_t chroma_format_idc; + uint8_t log2_ctu_size; + uint8_t ptl_dpb_hrd_params_present_flag; + VVCProfileTierLevel profile_tier_level; + uint8_t gdr_enabled_flag; + uint8_t ref_pic_resampling_enabled_flag; + uint8_t res_change_in_clvs_allowed_flag; + + uint16_t pic_width_max_in_luma_samples; + uint16_t pic_height_max_in_luma_samples; + + uint8_t conformance_window_flag; + uint16_t conf_win_left_offset; + uint16_t conf_win_right_offset; + uint16_t conf_win_top_offset; + uint16_t conf_win_bottom_offset; + + uint8_t subpic_info_present_flag; + uint16_t num_subpics_minus1; + uint8_t independent_subpics_flag; + uint8_t subpic_same_size_flag; + uint16_t subpic_ctu_top_left_x[VVC_MAX_SLICES]; + uint16_t subpic_ctu_top_left_y[VVC_MAX_SLICES]; + uint16_t subpic_width_minus1[VVC_MAX_SLICES]; + uint16_t subpic_height_minus1[VVC_MAX_SLICES]; + uint8_t subpic_treated_as_pic_flag[VVC_MAX_SLICES]; + uint8_t loop_filter_across_subpic_enabled_flag[VVC_MAX_SLICES]; + uint8_t subpic_id_len_minus1; + uint8_t subpic_id_mapping_explicitly_signalled_flag; + uint8_t subpic_id_mapping_present_flag; + uint32_t subpic_id[VVC_MAX_SLICES]; + + + uint8_t bit_depth; + uint8_t entropy_coding_sync_enabled_flag; + uint8_t entry_point_offsets_present_flag; + + uint8_t log2_max_pic_order_cnt_lsb_minus4; + uint8_t poc_msb_cycle_flag; + uint8_t poc_msb_cycle_len_minus1; + + uint8_t num_extra_ph_bytes; + uint8_t extra_ph_bit_present_flag[16]; + + uint8_t num_extra_sh_bytes; + uint8_t extra_sh_bit_present_flag[16]; + + uint8_t sublayer_dpb_params_flag; + VVCDpbParameters dpb_params; + + uint8_t log2_min_luma_coding_block_size_minus2; + uint8_t partition_constraints_override_enabled_flag; + uint8_t log2_diff_min_qt_min_cb_intra_slice_luma; + uint8_t max_mtt_hierarchy_depth_intra_slice_luma; + uint8_t log2_diff_max_bt_min_qt_intra_slice_luma; + uint8_t log2_diff_max_tt_min_qt_intra_slice_luma; + + uint8_t qtbtt_dual_tree_intra_flag; + uint8_t log2_diff_min_qt_min_cb_intra_slice_chroma; + uint8_t max_mtt_hierarchy_depth_intra_slice_chroma; + uint8_t log2_diff_max_bt_min_qt_intra_slice_chroma; + uint8_t log2_diff_max_tt_min_qt_intra_slice_chroma; + + uint8_t log2_diff_min_qt_min_cb_inter_slice; + uint8_t max_mtt_hierarchy_depth_inter_slice; + uint8_t log2_diff_max_bt_min_qt_inter_slice; + uint8_t log2_diff_max_tt_min_qt_inter_slice; + + uint8_t max_luma_transform_size_64_flag; + + uint8_t transform_skip_enabled_flag; + uint8_t log2_transform_skip_max_size_minus2; + uint8_t bdpcm_enabled_flag; + + uint8_t mts_enabled_flag; + uint8_t explicit_mts_intra_enabled_flag; + uint8_t explicit_mts_inter_enabled_flag; + + uint8_t lfnst_enabled_flag; + + uint8_t joint_cbcr_enabled_flag; + uint8_t same_qp_table_for_chroma_flag; + + int8_t qp_table_start_minus26[VVC_MAX_SAMPLE_ARRAYS]; + uint8_t num_points_in_qp_table_minus1[VVC_MAX_SAMPLE_ARRAYS]; + uint8_t delta_qp_in_val_minus1[VVC_MAX_SAMPLE_ARRAYS][VVC_MAX_POINTS_IN_QP_TABLE]; + uint8_t delta_qp_diff_val[VVC_MAX_SAMPLE_ARRAYS][VVC_MAX_POINTS_IN_QP_TABLE]; + + uint8_t sao_enabled_flag; + uint8_t alf_enabled_flag; + uint8_t ccalf_enabled_flag; + uint8_t lmcs_enabled_flag; + uint8_t weighted_pred_flag; + uint8_t weighted_bipred_flag; + uint8_t long_term_ref_pics_flag; + uint8_t inter_layer_prediction_enabled_flag; + uint8_t idr_rpl_present_flag; + uint8_t rpl1_same_as_rpl0_flag; + + uint8_t num_ref_pic_lists[2]; + VVCRefPicListStruct ref_pic_list_struct[2][VVC_MAX_REF_PIC_LISTS]; + + uint8_t ref_wraparound_enabled_flag; + uint8_t temporal_mvp_enabled_flag; + uint8_t sbtmvp_enabled_flag; + uint8_t amvr_enabled_flag; + uint8_t bdof_enabled_flag; + uint8_t bdof_control_present_in_ph_flag; + uint8_t smvd_enabled_flag; + uint8_t dmvr_enabled_flag; + uint8_t dmvr_control_present_in_ph_flag; + uint8_t mmvd_enabled_flag; + uint8_t mmvd_fullpel_only_enabled_flag; + uint8_t six_minus_max_num_merge_cand; + uint8_t sbt_enabled_flag; + uint8_t affine_enabled_flag; + uint8_t five_minus_max_num_subblock_merge_cand; + uint8_t param_affine_enabled_flag; + uint8_t affine_amvr_enabled_flag; + uint8_t affine_prof_enabled_flag; + uint8_t prof_control_present_in_ph_flag; + uint8_t bcw_enabled_flag; + uint8_t ciip_enabled_flag; + uint8_t gpm_enabled_flag; + uint8_t max_num_merge_cand_minus_max_num_gpm_cand; + uint8_t log2_parallel_merge_level_minus2; + uint8_t isp_enabled_flag; + uint8_t mrl_enabled_flag; + uint8_t mip_enabled_flag; + uint8_t cclm_enabled_flag; + uint8_t chroma_horizontal_collocated_flag; + uint8_t chroma_vertical_collocated_flag; + uint8_t palette_enabled_flag; + uint8_t act_enabled_flag; + uint8_t min_qp_prime_ts; + uint8_t ibc_enabled_flag; + uint8_t six_minus_max_num_ibc_merge_cand; + uint8_t ladf_enabled_flag; + uint8_t num_ladf_intervals_minus2; + int8_t ladf_lowest_interval_qp_offset; + int8_t ladf_qp_offset[4]; + uint16_t ladf_delta_threshold_minus1[4]; + + uint8_t explicit_scaling_list_enabled_flag; + uint8_t scaling_matrix_for_lfnst_disabled_flag; + uint8_t scaling_matrix_for_alternative_colour_space_disabled_flag; + uint8_t scaling_matrix_designated_colour_space_flag; + uint8_t dep_quant_enabled_flag; + uint8_t sign_data_hiding_enabled_flag; + + uint8_t virtual_boundaries_enabled_flag; + uint8_t virtual_boundaries_present_flag; + uint8_t num_ver_virtual_boundaries; + uint16_t virtual_boundary_pos_x_minus1[3]; + uint8_t num_hor_virtual_boundaries; + uint16_t virtual_boundary_pos_y_minus1[3]; + + uint8_t timing_hrd_params_present_flag; + uint8_t sublayer_cpb_params_present_flag; + VVCGeneralTimingHrdParameters general_timing_hrd_parameters; + VVCOlsTimingHrdParameters ols_timing_hrd_parameters; + + uint8_t field_seq_flag; + uint8_t vui_parameters_present_flag; + uint16_t vui_payload_size_minus1; + VVCVUI vui; + + uint8_t extension_flag; + //H266RawExtensionData extension_data; /* TODO: read extension flag and data*/ + + enum AVPixelFormat pix_fmt; +} VVCSPS; + + +typedef struct VVCParamSets { + AVBufferRef *sps_list[VVC_MAX_SPS_COUNT]; + //AVBufferRef *vps_list[VVC_MAX_VPS_COUNT]; // TODO: since not needed, not implemented yet + //AVBufferRef *pps_list[VVC_MAX_PPS_COUNT]; // TODO: since not needed, not implemented yet + + /* currently active parameter sets */ + const VVCSPS *sps; + //const VVCVPS *vps; // TODO: since not needed, not implemented yet + //const VVCPPS *pps; // TODO: since not needed, not implemented yet +} VVCParamSets; + +/** + * Parse the SPS from the bitstream into the provided HEVCSPS struct. + * + * @param sps_id the SPS id will be written here + * @param apply_defdispwin if set 1, the default display window from the VUI + * will be applied to the video dimensions + */ +int ff_vvc_parse_sps(VVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, + int apply_defdispwin, AVCodecContext *avctx); + +int ff_vvc_decode_nal_sps(GetBitContext *gb, AVCodecContext *avctx, + VVCParamSets *ps, int apply_defdispwin); + +// TODO: since not needed, not implemented yet +//int ff_vvc_decode_nal_vps(GetBitContext *gb, AVCodecContext *avctx, +// VVCParamSets *ps); +//int ff_vvc_decode_nal_pps(GetBitContext *gb, AVCodecContext *avctx, +// VVCParamSets *ps); + +void ff_vvc_ps_uninit(VVCParamSets *ps); + +#endif /* AVCODEC_VVC_PARAMSET_H */ diff --git a/libavcodec/vvc_parse_extradata.c b/libavcodec/vvc_parse_extradata.c new file mode 100644 index 0000000000..c7cf008952 --- /dev/null +++ b/libavcodec/vvc_parse_extradata.c @@ -0,0 +1,242 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "bytestream.h" +#include "h2645_parse.h" +#include "vvc.h" +#include "vvc_parse_extradata.h" + +static int vvc_decode_nal_units(const uint8_t *buf, int buf_size, VVCParamSets *ps, + int is_nalff, int nal_length_size, + int err_recognition, int apply_defdispwin, void *logctx) +{ + int i; + int ret = 0; + H2645Packet pkt = { 0 }; + + ret = ff_h2645_packet_split(&pkt, buf, buf_size, logctx, is_nalff, + nal_length_size, AV_CODEC_ID_VVC, 1, 0); + if (ret < 0) { + goto done; + } + + for (i = 0; i < pkt.nb_nals; i++) { + H2645NAL *nal = &pkt.nals[i]; + if (nal->nuh_layer_id > 0) + continue; + + /* ignore everything except parameter sets and VCL NALUs */ + switch (nal->type) { + case VVC_VPS_NUT: + // TODO: since not needed yet, not implemented + //ret = ff_vvc_decode_nal_vps(&nal->gb, logctx, ps); + //if (ret < 0) + // goto done; + break; + case VVC_SPS_NUT: + ret = ff_vvc_decode_nal_sps(&nal->gb, logctx, ps, apply_defdispwin); + if (ret < 0) + goto done; + break; + case VVC_PPS_NUT: + // TODO: since not needed yet, not implemented + //ret = ff_vvc_decode_nal_pps(&nal->gb, logctx, ps); + //if (ret < 0) + // goto done; + break; + case VVC_PREFIX_SEI_NUT: + case VVC_SUFFIX_SEI_NUT: + // TODO: SEI decoding not implemented yet + break; + default: + av_log(logctx, AV_LOG_VERBOSE, "Ignoring NAL type %d in extradata\n", nal->type); + break; + } + } + +done: + ff_h2645_packet_uninit(&pkt); + if (err_recognition & AV_EF_EXPLODE) + return ret; + + return 0; +} + +int ff_vvc_decode_extradata(const uint8_t *data, int size, VVCParamSets *ps, + int *is_nalff, int *nal_length_size, + int err_recognition, int apply_defdispwin, void *logctx) +{ + int ret = 0; + GetByteContext gb; + + bytestream2_init(&gb, data, size); + + if (size > 3 && (data[0] || data[1] || data[2] > 1)) { + /* extradata is encoded as vvcC format. */ + int i, j, b, num_arrays, nal_len_size, num_sublayers, has_ptl; + + b = bytestream2_get_byte(&gb); + num_sublayers = (b >> 3) & 0x7; + nal_len_size = ((b >> 1) & 0x3) + 1; + has_ptl = b & 0x1; + + if (has_ptl) { + int max_picture_width = 0; + int max_picture_height = 0; + int avg_frame_rate = 0; + + int num_bytes_constraint_info; + int general_profile_idc; + int general_tier_flag; + int general_level_idc; + int ptl_frame_only_constraint_flag; + int ptl_multi_layer_enabled_flag; + int ptl_num_sub_profiles; + int temp2, temp3, temp4, temp5; + int ols_idx; + int constant_frame_rate; + int chroma_format_idc; + int bit_depth_minus8; + + temp2 = bytestream2_get_be16(&gb); + ols_idx = (temp2 >> 7) & 0x1ff; + num_sublayers = (temp2 >> 4) & 0x7; + constant_frame_rate = (temp2 >> 2) & 0x3; + chroma_format_idc = temp2 & 0x3; + bit_depth_minus8 = (bytestream2_get_byte(&gb) >> 5) & 0x7; + av_log(logctx, AV_LOG_TRACE, + "bit_depth_minus8 %d chroma_format_idc %d\n", + bit_depth_minus8, chroma_format_idc); + av_log(logctx, AV_LOG_TRACE, + "constant_frame_rate %d, ols_idx %d\n", + constant_frame_rate, ols_idx); + // VvcPTLRecord(num_sublayers) native_ptl + temp3 = bytestream2_get_byte(&gb); + num_bytes_constraint_info = (temp3) & 0x3f; + temp4 = bytestream2_get_byte(&gb); + general_profile_idc = (temp4 >> 1) & 0x7f; + general_tier_flag = (temp4) & 1; + general_level_idc = bytestream2_get_byte(&gb); + av_log(logctx, AV_LOG_TRACE, + "general_profile_idc %d, general_tier_flag %d, " + "general_level_idc %d, num_sublayers %d num_bytes_constraint_info %d\n", + general_profile_idc, general_tier_flag, general_level_idc, num_sublayers, + num_bytes_constraint_info); + + temp5 = bytestream2_get_byte(&gb); + ptl_frame_only_constraint_flag = (temp5 >> 7) & 0x1; + ptl_multi_layer_enabled_flag = (temp5 >> 6) & 0x1; + for (i = 0; i < num_bytes_constraint_info-1; i++) { + // unsigned int(8*num_bytes_constraint_info - 2) general_constraint_info; + bytestream2_get_byte(&gb); + } + + av_log(logctx, AV_LOG_TRACE, + "ptl_multi_layer_enabled_flag %d, ptl_frame_only_constraint_flag %d\n", + ptl_multi_layer_enabled_flag, ptl_frame_only_constraint_flag); + + if( num_sublayers > 1 ) { + int temp6 = bytestream2_get_byte(&gb); + uint8_t ptl_sublayer_level_present_flag[8] = {0}; + //uint8_t sublayer_level_idc[8] = {0}; + for (i=num_sublayers - 2; i >= 0; i--) { + ptl_sublayer_level_present_flag[i] = (temp6 >> (7 - (num_sublayers - 2 - i))) & 0x01; + } + for (i=num_sublayers-2; i >= 0; i--) { + if (ptl_sublayer_level_present_flag[i]) { + bytestream2_get_byte(&gb); // sublayer_level_idc[8] + } + } + } + + ptl_num_sub_profiles = bytestream2_get_byte(&gb); + for (j=0; j < ptl_num_sub_profiles; j++) { + // unsigned int(32) general_sub_profile_idc[j]; + bytestream2_get_be16(&gb); + bytestream2_get_be16(&gb); + } + + max_picture_width = bytestream2_get_be16(&gb); + max_picture_height = bytestream2_get_be16(&gb); + avg_frame_rate = bytestream2_get_be16(&gb); + av_log(logctx, AV_LOG_TRACE, + "max_picture_width %d, max_picture_height %d, avg_frame_rate %d\n", + max_picture_width, max_picture_height, avg_frame_rate); + } + + *is_nalff = 1; + + /* nal units in the vvcC always have length coded with 2 bytes, + * so set nal_length_size = 2 while parsing them */ + *nal_length_size = 2; + + num_arrays = bytestream2_get_byte(&gb); + + for (i = 0; i < num_arrays; i++) { + int cnt; + int type = bytestream2_get_byte(&gb) & 0x1f; + + if (type == VVC_OPI_NUT || type == VVC_DCI_NUT) + cnt = 1; + else + cnt = bytestream2_get_be16(&gb); + + av_log(logctx, AV_LOG_DEBUG, "nalu_type %d cnt %d\n", type, cnt); + + if (!(type == VVC_OPI_NUT || type == VVC_DCI_NUT || + type == VVC_VPS_NUT || type == VVC_SPS_NUT || type == VVC_PPS_NUT || + type == VVC_PREFIX_SEI_NUT || type == VVC_SUFFIX_SEI_NUT)) { + av_log(logctx, AV_LOG_ERROR, "Invalid NAL unit type in extradata: %d\n", + type); + ret = AVERROR_INVALIDDATA; + return ret; + } + + for (j = 0; j < cnt; j++) { + // +2 for the nal size field + int nalsize = bytestream2_peek_be16(&gb) + 2; + if (bytestream2_get_bytes_left(&gb) < nalsize) { + av_log(logctx, AV_LOG_ERROR, + "Invalid NAL unit size in extradata.\n"); + return AVERROR_INVALIDDATA; + } + + ret = vvc_decode_nal_units(gb.buffer, nalsize, ps, *is_nalff, + *nal_length_size, err_recognition, apply_defdispwin, + logctx); + if (ret < 0) { + av_log(logctx, AV_LOG_ERROR, + "Decoding nal unit %d %d from vvcC failed\n", type, i); + return ret; + } + bytestream2_skip(&gb, nalsize); + } + } + + /* store nal length size, that will be used to parse all other nals */ + *nal_length_size = nal_len_size; + } else { + *is_nalff = 0; + ret = vvc_decode_nal_units(data, size, ps, *is_nalff, *nal_length_size, + err_recognition, apply_defdispwin, logctx); + if (ret < 0) + return ret; + } + + return ret; +} diff --git a/libavcodec/vvc_parse_extradata.h b/libavcodec/vvc_parse_extradata.h new file mode 100644 index 0000000000..4adb7616b7 --- /dev/null +++ b/libavcodec/vvc_parse_extradata.h @@ -0,0 +1,36 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * H.266 parser code + */ + +#ifndef AVCODEC_VVC_PARSE_EXTRADATA_H +#define AVCODEC_VVC_PARSE_EXTRADATA_H + +#include + +#include "vvc_paramset.h" + + +int ff_vvc_decode_extradata(const uint8_t *data, int size, VVCParamSets *ps, + int *is_nalff, int *nal_length_size, + int err_recognition, int apply_defdispwin, void *logctx); + +#endif /* AVCODEC_VVC_PARSE_EXTRADATA_H */