From patchwork Wed Oct 19 07:25:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Siedel X-Patchwork-Id: 38820 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp216176pzb; Wed, 19 Oct 2022 00:28:26 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6wLRooag0tn366/s4FnDTq8LELJYLnCQCe6vUr0lheNUJC3bN0C7LMBuE9zlaVGz8AC3VH X-Received: by 2002:a17:907:971e:b0:78d:e7ed:7585 with SMTP id jg30-20020a170907971e00b0078de7ed7585mr5396658ejc.258.1666164506126; Wed, 19 Oct 2022 00:28:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666164506; cv=none; d=google.com; s=arc-20160816; b=MUGMMc+ZUGdzEbEFVgRLSSDHmi27tLj5DivssyqY2J/EXxJcaSRshP9snaGY0UrHM/ 2RTZM+yQ+czCM27Ye/KogqtOUrFBj0AhIkl+e1AM/AfOesa33IyXwphcuB514tt/ejgI UzspTSLbgxp/kigKzNPC5Yv5qqChp9Ud6bZBc/fAMAnTDAcyXnZRlYqJk7GVPWuqpr7B zUzeFlV5Z5oeg3itZIFiTjuMEa9BaRFI1Tb8g9kptKmMseln2bRd071W0Wy9pUh1gzlV omcICR9lxF+l2Y9PPEGJIRG4k7sJNcVVkD1B6m0M/VIckPI0DiQgL3exCZ9ZzjATetCj SoNQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=sAVuk7qSdnbUkiqtyKMvwzQ99eMlI582noA4jnjUWME=; b=HapC8uMaM4SQnK8bVX/y4B5GyHKF1wYt5+VeQrxsH7eS1OcDDPoSdV5Co35m0XZVBg Y4/oPTRUQKfzK4+5dyu6F4JxssC1DO2sKAbrQfKK8nolKP61EwVL7ENvvk4SOhWpDnC3 nc8VF6pl1lRFbpHzQ0yCMHxSJHVr05KmdJeXbU/LrnDG63Ox2tp6LsjQtHjnHg3hWkrj IPSbfivPDOFY0waBIH3MtVA6YwJ090jjGGP9TEEeALWbFJY86ZeDW0d1SQJwDZ+Xh9PK 9ykIpEa9ooM5JBZp9c19P0268xsaespgUR3d7t4wmLDJ+PS+u6ApptTClytpmbEs3H7a 51uA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b="vcVKRZP/"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id m15-20020a1709060d8f00b0073db9474d06si12410768eji.325.2022.10.19.00.28.25; Wed, 19 Oct 2022 00:28:26 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b="vcVKRZP/"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2638668BE0F; Wed, 19 Oct 2022 10:26:43 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f46.google.com (mail-ej1-f46.google.com [209.85.218.46]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5BA2E68BDF0 for ; Wed, 19 Oct 2022 10:26:33 +0300 (EEST) Received: by mail-ej1-f46.google.com with SMTP id d26so37792045eje.10 for ; Wed, 19 Oct 2022 00:26:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spin-digital-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=p4+u9fTv6uyp7C0EE9h76LOBGk31DB8sFpQvQt6hWsU=; b=vcVKRZP/gCNFKALG0hVMskMu8470zfvjD8OMisAdnjN7YPzKHtYM3GGb7ht0mnaRns Pjzv4AEkID6rknbI5X7ZbaztgQHstkpaOz7SM3cbgStVPmOiqaLllWiYNZobC6B0eXMA eVjPl0YdJQwGlUroqAEpxKEMxKZnePCBIaTg0VbRN5ENJeBRolVvAUpScAhtT6qmGniZ bhUhEqU5Y0qIZb+xTRZeKI61BLV7VSR5vzBgdYOUQfDcdsOVyd9yZnXkBF6vhO4I6NoI BR6u7rBFNbQALZ0F+pma29gzLt7f66hxaGWHcBo4VJSQgRhvbQMWFg02ROr4s599WdUg iG0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=p4+u9fTv6uyp7C0EE9h76LOBGk31DB8sFpQvQt6hWsU=; b=oVgwL0gT8sXBrJC1JRo/lOkluK5sXQsGDvVx1NKfsVEch/YrkWTAmbwswH4LCt3ZsL iGlCNhqVeGm+C6sYeyOeYuVObOyGGsxsGQx95vY36ixnH1KyXJ321rUK7Nsdt+g1HuUp PpDQ7H0JY3a82qhE5+4PsQziNEcx7IbJ+OUyXC+zcR+lZp4izzMQW0h7/HOKqW+UhgPr V9coewoUZw95T4xXkrlAHhHS4N5qhI4w1ld4y6kWc75VBGMbuyQjBm7cqp1ezGE0V/8C usN79rUWMkM+1UDS3kXFroiHTlOzydZbT6xNhDJbedoP7QpMqSqDkm/3Qx8+iarqgCtR F8nw== X-Gm-Message-State: ACrzQf3zsbuHmVd4q0ZfU8wztZ2oshNQJPiCzKXLe8QxDHUyRnVqCXsq dfczGoSQlyX7hzU/TCIstUgrte03JOS9Lw== X-Received: by 2002:a17:907:2da6:b0:78d:3cf1:9132 with SMTP id gt38-20020a1709072da600b0078d3cf19132mr5535776ejc.299.1666164391113; Wed, 19 Oct 2022 00:26:31 -0700 (PDT) Received: from thomas-win.localdomain ([213.138.44.237]) by smtp.gmail.com with ESMTPSA id p5-20020a170906498500b0078b83968ad4sm8515150eju.24.2022.10.19.00.26.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Oct 2022 00:26:30 -0700 (PDT) From: thomas.ff@spin-digital.com To: ffmpeg-devel@ffmpeg.org Date: Wed, 19 Oct 2022 09:25:05 +0200 Message-Id: <20221019072508.23460-9-thomas.ff@spin-digital.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221019072508.23460-1-thomas.ff@spin-digital.com> References: <20221019072508.23460-1-thomas.ff@spin-digital.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v1 08/11] avcodec: add external decoder libvvdec for H266/VVC X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Thomas Siedel Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: sLC0xuTbgdwU From: Thomas Siedel Add external decoder VVdeC for H266/VVC decoding. Register new decoder libvvdec Add vvc_parse_extradata to support parse/probe of vvcC stream input Add vvc_paramset that implements the parser of vvcC configuration boxes Add libvvdec to wrap the vvdec interface into ffmpeg libvvdec implements decoder option externmem to use AVBufferPool (default) or copy image after receiving from decoder Enable decoder by adding --enable-libvvdec in configure step Signed-off-by: Thomas Siedel --- configure | 5 + libavcodec/Makefile | 1 + libavcodec/allcodecs.c | 1 + libavcodec/libvvdec.c | 511 ++++++++++++++++ libavcodec/vvc_paramset.c | 972 +++++++++++++++++++++++++++++++ libavcodec/vvc_paramset.h | 429 ++++++++++++++ libavcodec/vvc_parse_extradata.c | 241 ++++++++ libavcodec/vvc_parse_extradata.h | 36 ++ 8 files changed, 2196 insertions(+) create mode 100644 libavcodec/libvvdec.c create mode 100644 libavcodec/vvc_paramset.c create mode 100644 libavcodec/vvc_paramset.h create mode 100644 libavcodec/vvc_parse_extradata.c create mode 100644 libavcodec/vvc_parse_extradata.h diff --git a/configure b/configure index 30df2cb0a4..289473782c 100755 --- a/configure +++ b/configure @@ -288,6 +288,7 @@ External library support: --enable-libvorbis enable Vorbis en/decoding via libvorbis, native implementation exists [no] --enable-libvpx enable VP8 and VP9 de/encoding via libvpx [no] + --enable-libvvdec enable VVC decoding via vvdec [no] --enable-libwebp enable WebP encoding via libwebp [no] --enable-libx264 enable H.264 encoding via x264 [no] --enable-libx265 enable HEVC encoding via x265 [no] @@ -1875,6 +1876,7 @@ EXTERNAL_LIBRARY_LIST=" libvmaf libvorbis libvpx + libvvdec libwebp libxml2 libzimg @@ -3402,6 +3404,8 @@ libvpx_vp8_decoder_deps="libvpx" libvpx_vp8_encoder_deps="libvpx" libvpx_vp9_decoder_deps="libvpx" libvpx_vp9_encoder_deps="libvpx" +libvvdec_decoder_deps="libvvdec" +libvvdec_decoder_select="vvc_mp4toannexb_bsf" libwebp_encoder_deps="libwebp" libwebp_anim_encoder_deps="libwebp" libx262_encoder_deps="libx262" @@ -6731,6 +6735,7 @@ enabled libvpx && { die "libvpx enabled but no supported decoders found" fi } +enabled libvvdec && require_pkg_config libvvdec "libvvdec >= 1.6.0" "vvdec/vvdec.h" vvdec_get_version enabled libwebp && { enabled libwebp_encoder && require_pkg_config libwebp "libwebp >= 0.2.0" webp/encode.h WebPGetEncoderVersion diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 53a8dc67cb..3cc258351c 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1102,6 +1102,7 @@ OBJS-$(CONFIG_LIBVPX_VP8_DECODER) += libvpxdec.o OBJS-$(CONFIG_LIBVPX_VP8_ENCODER) += libvpxenc.o OBJS-$(CONFIG_LIBVPX_VP9_DECODER) += libvpxdec.o libvpx.o OBJS-$(CONFIG_LIBVPX_VP9_ENCODER) += libvpxenc.o libvpx.o +OBJS-$(CONFIG_LIBVVDEC_DECODER) += libvvdec.o vvc_parse_extradata.o vvc_paramset.o OBJS-$(CONFIG_LIBWEBP_ENCODER) += libwebpenc_common.o libwebpenc.o OBJS-$(CONFIG_LIBWEBP_ANIM_ENCODER) += libwebpenc_common.o libwebpenc_animencoder.o OBJS-$(CONFIG_LIBX262_ENCODER) += libx264.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index 46ad3b5a25..58cf993785 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -793,6 +793,7 @@ extern const FFCodec ff_libvpx_vp8_encoder; extern const FFCodec ff_libvpx_vp8_decoder; extern FFCodec ff_libvpx_vp9_encoder; extern FFCodec ff_libvpx_vp9_decoder; +extern const FFCodec ff_libvvdec_decoder; /* preferred over libwebp */ extern const FFCodec ff_libwebp_anim_encoder; extern const FFCodec ff_libwebp_encoder; diff --git a/libavcodec/libvvdec.c b/libavcodec/libvvdec.c new file mode 100644 index 0000000000..28f3a548ca --- /dev/null +++ b/libavcodec/libvvdec.c @@ -0,0 +1,511 @@ +/* + * H.266 decoding using the VVdeC library + * + * Copyright (C) 2022, Thomas Siedel + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "config_components.h" + +#include + +#include "libavutil/common.h" +#include "libavutil/avutil.h" +#include "libavutil/pixdesc.h" +#include "libavutil/opt.h" +#include "libavutil/imgutils.h" +#include "libavutil/frame.h" +#include "libavutil/mastering_display_metadata.h" +#include "libavutil/log.h" + +#include "avcodec.h" +#include "codec_internal.h" +#include "decode.h" +#include "internal.h" +#include "profiles.h" + +#include "vvc_paramset.h" +#include "vvc_parse_extradata.h" + +typedef struct VVdeCContext { + AVClass *av_class; + vvdecDecoder* vvdecDec; + vvdecParams vvdecParams; + VVCParamSets ps; + int is_nalff; + int nal_length_size; + bool bFlush; + AVBufferPool *pools[3]; /** Pools for each data plane. */ + int pool_size[3]; +}VVdeCContext; + + +static void ff_vvdec_log_callback(void *avctx, int level, const char *fmt, va_list args ) +{ + vfprintf( level == 1 ? stderr : stdout, fmt, args ); +} + +static void* ff_vvdec_buffer_allocator( void *ctx, vvdecComponentType comp, uint32_t size, uint32_t alignment, void** allocator ) +{ + AVBufferRef *buf; + VVdeCContext *s; + int plane; + + uint32_t alignedsize = FFALIGN(size, alignment); + s = (VVdeCContext*)ctx; + plane = (int)comp; + + if ( plane < 0 || plane > 3 ) + return NULL; + + if ( alignedsize != s->pool_size[plane]) { + av_buffer_pool_uninit(&s->pools[plane]); + s->pools[plane] = av_buffer_pool_init(alignedsize, NULL); + if (!s->pools[plane]) { + s->pool_size[plane] = 0; + return NULL; + } + s->pool_size[plane] = alignedsize; + } + + buf = av_buffer_pool_get(s->pools[plane]); + if (!buf) + return NULL; + + *allocator = (void*)buf; + return buf->data; +} + +static void ff_vvdec_buffer_unref( void *ctx, void* allocator ) +{ + AVBufferRef *buf = (AVBufferRef*)allocator; + av_buffer_unref( &buf ); +} + +static void ff_vvdec_printParameterInfo( AVCodecContext *avctx, vvdecParams* params ) +{ + av_log(avctx, AV_LOG_DEBUG, "Version info: vvdec %s ( threads %d)\n",vvdec_get_version(), params->threads); +} + +static int ff_vvdec_set_pix_fmt(AVCodecContext *avctx, vvdecFrame* frame ) +{ + if( NULL != frame->picAttributes && NULL != frame->picAttributes->vui && + frame->picAttributes->vui->colourDescriptionPresentFlag ) { + avctx->color_trc = frame->picAttributes->vui->transferCharacteristics; + avctx->color_primaries = frame->picAttributes->vui->colourPrimaries; + avctx->colorspace = frame->picAttributes->vui->matrixCoefficients; + } + else { + avctx->color_primaries = AVCOL_PRI_UNSPECIFIED; + avctx->color_trc = AVCOL_TRC_UNSPECIFIED; + avctx->colorspace = AVCOL_SPC_UNSPECIFIED; + } + + if( NULL != frame->picAttributes && NULL != frame->picAttributes->vui && + frame->picAttributes->vui->videoSignalTypePresentFlag) { + avctx->color_range = frame->picAttributes->vui->videoFullRangeFlag ? + AVCOL_RANGE_JPEG : AVCOL_RANGE_MPEG; + } + else { + avctx->color_range = AVCOL_RANGE_MPEG; + } + + switch ( frame->colorFormat ) { + case VVDEC_CF_YUV420_PLANAR: + if (frame->bitDepth == 8) { + avctx->pix_fmt = frame->numPlanes == 1 ? + AV_PIX_FMT_GRAY8 : AV_PIX_FMT_YUV420P; + avctx->profile = FF_PROFILE_VVC_MAIN_10; + return 0; + } + else if (frame->bitDepth == 10) { + avctx->pix_fmt = frame->numPlanes == 1 ? + AV_PIX_FMT_GRAY10 : AV_PIX_FMT_YUV420P10; + avctx->profile = FF_PROFILE_VVC_MAIN_10; + return 0; + } + else { + return AVERROR_INVALIDDATA; + } + default: + return AVERROR_INVALIDDATA; + } +} + +static int set_side_data( AVCodecContext *avctx, AVFrame *avframe, vvdecFrame *frame ) +{ + vvdecSEI *sei; + VVdeCContext *s = (VVdeCContext*)avctx->priv_data; + + sei = vvdec_find_frame_sei( s->vvdecDec, VVDEC_MASTERING_DISPLAY_COLOUR_VOLUME, frame ); + if( sei ) { + // VVC uses a g,b,r ordering, which we convert to a more natural r,g,b + const int mapping[3] = {2, 0, 1}; + const int chroma_den = 50000; + const int luma_den = 10000; + int i; + vvdecSEIMasteringDisplayColourVolume* p; + AVMasteringDisplayMetadata *metadata = av_mastering_display_metadata_create_side_data(avframe); + p = (vvdecSEIMasteringDisplayColourVolume *)(sei->payload); + if ( p && metadata ) { + for (i = 0; i < 3; i++) { + const int j = mapping[i]; + metadata->display_primaries[i][0].num = p->primaries[j][0]; + metadata->display_primaries[i][0].den = chroma_den; + metadata->display_primaries[i][1].num = p->primaries[j][1]; + metadata->display_primaries[i][1].den = chroma_den; + } + metadata->white_point[0].num = p->whitePoint[0]; + metadata->white_point[0].den = chroma_den; + metadata->white_point[1].num = p->whitePoint[1]; + metadata->white_point[1].den = chroma_den; + + metadata->max_luminance.num = p->maxLuminance; + metadata->max_luminance.den = luma_den; + metadata->min_luminance.num = p->minLuminance; + metadata->min_luminance.den = luma_den; + metadata->has_luminance = 1; + metadata->has_primaries = 1; + + av_log(avctx, AV_LOG_DEBUG, "Mastering Display Metadata:\n"); + av_log(avctx, AV_LOG_DEBUG, + "r(%5.4f,%5.4f) g(%5.4f,%5.4f) b(%5.4f %5.4f) wp(%5.4f, %5.4f)\n", + av_q2d(metadata->display_primaries[0][0]), + av_q2d(metadata->display_primaries[0][1]), + av_q2d(metadata->display_primaries[1][0]), + av_q2d(metadata->display_primaries[1][1]), + av_q2d(metadata->display_primaries[2][0]), + av_q2d(metadata->display_primaries[2][1]), + av_q2d(metadata->white_point[0]), av_q2d(metadata->white_point[1])); + av_log(avctx, AV_LOG_DEBUG, + "min_luminance=%f, max_luminance=%f\n", + av_q2d(metadata->min_luminance), av_q2d(metadata->max_luminance)); + } + return 0; + } + + sei = vvdec_find_frame_sei( s->vvdecDec, VVDEC_CONTENT_LIGHT_LEVEL_INFO, frame ); + if( sei ) { + vvdecSEIContentLightLevelInfo* p = NULL; + AVContentLightMetadata *light = av_content_light_metadata_create_side_data(avframe); + p = (vvdecSEIContentLightLevelInfo *)(sei->payload); + if ( p && light) { + light->MaxCLL = p->maxContentLightLevel; + light->MaxFALL = p->maxPicAverageLightLevel; + } + + av_log(avctx, AV_LOG_DEBUG, "Content Light Level Metadata:\n"); + av_log(avctx, AV_LOG_DEBUG, "MaxCLL=%d, MaxFALL=%d\n", + light->MaxCLL, light->MaxFALL); + } + + return 0; +} + +static void export_stream_params( AVCodecContext *avctx, const VVCSPS *sps) +{ + avctx->coded_width = sps->pic_width_max_in_luma_samples; + avctx->coded_height = sps->pic_height_max_in_luma_samples; + avctx->width = sps->pic_width_max_in_luma_samples - + sps->conf_win_left_offset - + sps->conf_win_right_offset; + avctx->height = sps->pic_height_max_in_luma_samples - + sps->conf_win_top_offset - + sps->conf_win_bottom_offset; + avctx->has_b_frames = sps->max_sublayers; + avctx->profile = sps->profile_tier_level.general_profile_idc; + avctx->level = sps->profile_tier_level.general_level_idc; + avctx->pix_fmt = sps->pix_fmt; + + avctx->color_range = sps->vui.full_range_flag ? AVCOL_RANGE_JPEG : AVCOL_RANGE_MPEG; + + if (sps->vui.colour_description_present_flag) { + avctx->color_primaries = sps->vui.colour_primaries; + avctx->color_trc = sps->vui.transfer_characteristics; + avctx->colorspace = sps->vui.matrix_coeffs; + } else { + avctx->color_primaries = AVCOL_PRI_UNSPECIFIED; + avctx->color_trc = AVCOL_TRC_UNSPECIFIED; + avctx->colorspace = AVCOL_SPC_UNSPECIFIED; + } + + avctx->chroma_sample_location = AVCHROMA_LOC_UNSPECIFIED; + if (sps->chroma_format_idc == 1) { + if (sps->vui.chroma_loc_info_present_flag) { + if (sps->vui.chroma_sample_loc_type_top_field <= 5) + avctx->chroma_sample_location = sps->vui.chroma_sample_loc_type_top_field + 1; + } else + avctx->chroma_sample_location = AVCHROMA_LOC_LEFT; + } + + if (sps->timing_hrd_params_present_flag && + sps->general_timing_hrd_parameters.num_units_in_tick && + sps->general_timing_hrd_parameters.time_scale) { + av_reduce(&avctx->framerate.den, &avctx->framerate.num, + sps->general_timing_hrd_parameters.num_units_in_tick, + sps->general_timing_hrd_parameters.time_scale, INT_MAX); + } +} + +static int vvc_decode_extradata( AVCodecContext *avctx, uint8_t *buf, int length, int first) +{ + VVdeCContext *s = (VVdeCContext*)avctx->priv_data; + int ret; + + ret = ff_vvc_decode_extradata(buf, length, &s->ps, &s->is_nalff, + &s->nal_length_size, avctx->err_recognition, + false, avctx); + if (ret < 0) + return ret; + + if ( s->ps.sps != NULL ) + export_stream_params(avctx, s->ps.sps); + + return 0; +} + +static av_cold int ff_vvdec_decode_init(AVCodecContext *avctx) +{ + int i; + VVdeCContext *s = (VVdeCContext*)avctx->priv_data; + + vvdec_params_default( &s->vvdecParams ); + s->vvdecParams.logLevel = VVDEC_DETAILS; + + if ( av_log_get_level() >= AV_LOG_DEBUG ) s->vvdecParams.logLevel = VVDEC_DETAILS; + else if( av_log_get_level() >= AV_LOG_VERBOSE ) s->vvdecParams.logLevel = VVDEC_INFO; // VVDEC_INFO will output per picture info + else if( av_log_get_level() >= AV_LOG_INFO ) s->vvdecParams.logLevel = VVDEC_WARNING; // AV_LOG_INFO is ffmpeg default + else s->vvdecParams.logLevel = VVDEC_SILENT; + + if( avctx->thread_count > 0 ) + s->vvdecParams.threads = avctx->thread_count; // number of worker threads (should not exceed the number of physical cpu's) + else + s->vvdecParams.threads = -1; // get max cpus + + ff_vvdec_printParameterInfo( avctx, &s->vvdecParams ); + + // using buffer allocation by using AVBufferPool + s->vvdecParams.opaque = avctx->priv_data; + s->vvdecDec = vvdec_decoder_open_with_allocator( &s->vvdecParams, + ff_vvdec_buffer_allocator, ff_vvdec_buffer_unref ); + + + if( !s->vvdecDec ) { + av_log(avctx, AV_LOG_ERROR, "cannot init vvc decoder\n" ); + return -1; + } + + vvdec_set_logging_callback( s->vvdecDec, ff_vvdec_log_callback ); + + s->bFlush = false; + s->is_nalff = 0; + s->nal_length_size = 0; + + for (i = 0; i < FF_ARRAY_ELEMS(s->pools); i++) { + s->pools[i] = NULL; + s->pool_size[i] = 0; + } + + if (!avctx->internal->is_copy) { + if (avctx->extradata_size > 0 && avctx->extradata) { + int ret = vvc_decode_extradata(avctx, avctx->extradata, avctx->extradata_size, 1); + if (ret < 0) { + return ret; + } + } + } + + return 0; +} + +static av_cold int ff_vvdec_decode_close(AVCodecContext *avctx) +{ + VVdeCContext *s = (VVdeCContext*)avctx->priv_data; + + for (int i = 0; i < FF_ARRAY_ELEMS(s->pools); i++) { + av_buffer_pool_uninit(&s->pools[i]); + s->pool_size[i] = 0; + } + + if( 0 != vvdec_decoder_close(s->vvdecDec) ) { + av_log(avctx, AV_LOG_ERROR, "cannot close vvdec\n" ); + return -1; + } + + ff_vvc_ps_uninit( &s->ps ); + s->bFlush = false; + return 0; +} + +static av_cold int ff_vvdec_decode_frame( AVCodecContext *avctx, AVFrame *data, + int *got_frame, AVPacket *avpkt ) +{ + VVdeCContext *s = avctx->priv_data; + AVFrame *avframe = data; + + int ret = 0; + vvdecFrame *frame = NULL; + + if ( avframe ) { + if( !avpkt->size && !s->bFlush ) + s->bFlush = true; + + if( s->bFlush ) + ret = vvdec_flush( s->vvdecDec, &frame ); + else { + vvdecAccessUnit accessUnit; + vvdec_accessUnit_default( &accessUnit ); + accessUnit.payload = avpkt->data; + accessUnit.payloadSize = avpkt->size; + accessUnit.payloadUsedSize = avpkt->size; + + accessUnit.cts = avpkt->pts; accessUnit.ctsValid = true; + accessUnit.dts = avpkt->dts; accessUnit.dtsValid = true; + + ret = vvdec_decode( s->vvdecDec, &accessUnit, &frame ); + } + + if( ret < 0 ) { + if( ret == VVDEC_EOF ) + s->bFlush = true; + else if ( ret != VVDEC_TRY_AGAIN ) { + av_log(avctx, AV_LOG_ERROR, "error in vvdec::decode - ret:%d - %s\n", ret, vvdec_get_last_error(s->vvdecDec)); + return AVERROR(EINVAL); \ + } + } + else if( NULL != frame ) { + const uint8_t * src_data[4] = { frame->planes[0].ptr, frame->planes[1].ptr, frame->planes[2].ptr, NULL }; + const int src_linesizes[4] = { (int)frame->planes[0].stride, (int)frame->planes[1].stride, (int)frame->planes[2].stride, 0 }; + + if (( ret = ff_vvdec_set_pix_fmt(avctx, frame)) < 0) { + av_log(avctx, AV_LOG_ERROR, "Unsupported output colorspace (%d) / bit_depth (%d)\n", + frame->colorFormat, frame->bitDepth); + goto fail; + } + + if( avctx->pix_fmt != AV_PIX_FMT_YUV420P && avctx->pix_fmt != AV_PIX_FMT_YUV420P10LE ) { + av_log(avctx, AV_LOG_ERROR, "Unsupported output colorspace (%d) / bit_depth (%d)\n", + frame->colorFormat, frame->bitDepth ); + ret = AVERROR_INVALIDDATA; + goto fail; + } + + if ((int)frame->width != avctx->width || (int)frame->height != avctx->height) { + av_log(avctx, AV_LOG_INFO, "dimension change! %dx%d -> %dx%d\n", + avctx->width, avctx->height, frame->width, frame->height); + + ret = ff_set_dimensions(avctx, frame->width, frame->height); + if (ret < 0) + goto fail; + } + + if( frame->planes[0].allocator ) + avframe->buf[0] = av_buffer_ref( (AVBufferRef *)frame->planes[0].allocator); + if( frame->planes[1].allocator ) + avframe->buf[1] = av_buffer_ref( (AVBufferRef *)frame->planes[1].allocator); + if( frame->planes[2].allocator ) + avframe->buf[2] = av_buffer_ref( (AVBufferRef *)frame->planes[2].allocator); + + for (int i = 0; i < 4; i++) { + avframe->data[i] = (uint8_t *)src_data[i]; + avframe->linesize[i] = src_linesizes[i]; + } + + ret = ff_decode_frame_props(avctx, avframe); + if (ret < 0) + goto fail; + + if( frame->picAttributes ) { + avframe->key_frame = frame->picAttributes->isRefPic; + avframe->pict_type = (frame->picAttributes->sliceType != VVDEC_SLICETYPE_UNKNOWN) ? + frame->picAttributes->sliceType+1 : AV_PICTURE_TYPE_NONE; + } + + ret = set_side_data(avctx, avframe, frame ); + if (ret < 0) + goto fail; + + if( 0 != vvdec_frame_unref( s->vvdecDec, frame ) ) + av_log(avctx, AV_LOG_ERROR, "cannot free picture memory\n"); + + *got_frame = 1; + } + } + + return avpkt->size; + +fail: + if (frame) { + if( frame->planes[0].allocator ) + av_buffer_unref( (AVBufferRef **)&frame->planes[0].allocator ); + if( frame->planes[1].allocator ) + av_buffer_unref( (AVBufferRef **)&frame->planes[1].allocator ); + if( frame->planes[2].allocator ) + av_buffer_unref( (AVBufferRef **)&frame->planes[2].allocator ); + + vvdec_frame_unref( s->vvdecDec, frame ); + } + return ret; +} + +static av_cold void ff_vvdec_decode_flush(AVCodecContext *avctx) +{ + VVdeCContext *s = (VVdeCContext*)avctx->priv_data; + + if( 0 != vvdec_decoder_close(s->vvdecDec) ) + av_log(avctx, AV_LOG_ERROR, "cannot close vvdec during flush\n" ); + + s->vvdecDec = vvdec_decoder_open_with_allocator( &s->vvdecParams, ff_vvdec_buffer_allocator, ff_vvdec_buffer_unref ); + if( !s->vvdecDec ) + av_log(avctx, AV_LOG_ERROR, "cannot reinit vvdec during flush\n" ); + + vvdec_set_logging_callback( s->vvdecDec, ff_vvdec_log_callback ); + + s->bFlush = false; +} + +static const enum AVPixelFormat pix_fmts_vvc[] = { + AV_PIX_FMT_YUV420P, + AV_PIX_FMT_YUV420P10LE, + AV_PIX_FMT_NONE +}; + +static const AVClass class_libvvdec = { + .class_name = "libvvdec-vvc decoder", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, +}; + +FFCodec ff_libvvdec_decoder = { + .p.name = "libvvdec", + .p.long_name = "H.266 / VVC Decoder VVdeC", + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_VVC, + .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_OTHER_THREADS, + .p.profiles = NULL_IF_CONFIG_SMALL(ff_vvc_profiles), + .p.priv_class = &class_libvvdec, + .p.wrapper_name = "libvvdec", + .priv_data_size = sizeof(VVdeCContext), + .p.pix_fmts = pix_fmts_vvc, + .init = ff_vvdec_decode_init, + FF_CODEC_DECODE_CB(ff_vvdec_decode_frame), + .close = ff_vvdec_decode_close, + .flush = ff_vvdec_decode_flush, + .bsfs = "vvc_mp4toannexb", + .caps_internal = FF_CODEC_CAP_AUTO_THREADS, +}; diff --git a/libavcodec/vvc_paramset.c b/libavcodec/vvc_paramset.c new file mode 100644 index 0000000000..a016f57a37 --- /dev/null +++ b/libavcodec/vvc_paramset.c @@ -0,0 +1,972 @@ +/* + * VVC Parameter Set decoding + * + * Copyright (c) 2022, Thomas Siedel + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/imgutils.h" +#include "golomb.h" +#include "vvc_paramset.h" + +static void remove_sps(VVCParamSets *s, int id) +{ + if (s->sps_list[id]) { + if (s->sps == (const VVCSPS*)s->sps_list[id]->data) + s->sps = NULL; + + av_assert0(!(s->sps_list[id] && s->sps == (VVCSPS*)s->sps_list[id]->data)); + } + av_buffer_unref(&s->sps_list[id]); +} + +static int decode_general_constraints_info(GetBitContext *gb, AVCodecContext *avctx, + VVCGeneralConstraintsInfo *gci) +{ + int i; + gci->gci_present_flag = get_bits1(gb); + + if (gci->gci_present_flag) { + /* general */ + gci->gci_intra_only_constraint_flag = get_bits1(gb); + gci->gci_all_layers_independent_constraint_flag = get_bits1(gb); + gci->gci_one_au_only_constraint_flag = get_bits1(gb); + + /* picture format */ + gci->gci_sixteen_minus_max_bitdepth_constraint_idc = get_bits(gb,4); + gci->gci_three_minus_max_chroma_format_constraint_idc = get_bits(gb,2); + + /* NAL unit type related */ + gci->gci_no_mixed_nalu_types_in_pic_constraint_flag = get_bits1(gb); + gci->gci_no_trail_constraint_flag = get_bits1(gb); + gci->gci_no_stsa_constraint_flag = get_bits1(gb); + gci->gci_no_rasl_constraint_flag = get_bits1(gb); + gci->gci_no_radl_constraint_flag = get_bits1(gb); + gci->gci_no_idr_constraint_flag = get_bits1(gb); + gci->gci_no_cra_constraint_flag = get_bits1(gb); + gci->gci_no_gdr_constraint_flag = get_bits1(gb); + gci->gci_no_aps_constraint_flag = get_bits1(gb); + gci->gci_no_idr_rpl_constraint_flag = get_bits1(gb); + + /* tile, slice, subpicture partitioning */ + gci->gci_one_tile_per_pic_constraint_flag = get_bits1(gb); + gci->gci_pic_header_in_slice_header_constraint_flag = get_bits1(gb); + gci->gci_one_slice_per_pic_constraint_flag = get_bits1(gb); + gci->gci_no_rectangular_slice_constraint_flag = get_bits1(gb); + gci->gci_one_slice_per_subpic_constraint_flag = get_bits1(gb); + gci->gci_no_subpic_info_constraint_flag = get_bits1(gb); + + /* CTU and block partitioning */ + gci->gci_three_minus_max_log2_ctu_size_constraint_idc = get_bits(gb,2); + gci->gci_no_partition_constraints_override_constraint_flag = get_bits1(gb); + gci->gci_no_mtt_constraint_flag = get_bits1(gb); + gci->gci_no_qtbtt_dual_tree_intra_constraint_flag = get_bits1(gb); + + /* intra */ + gci->gci_no_palette_constraint_flag = get_bits1(gb); + gci->gci_no_ibc_constraint_flag = get_bits1(gb); + gci->gci_no_isp_constraint_flag = get_bits1(gb); + gci->gci_no_mrl_constraint_flag = get_bits1(gb); + gci->gci_no_mip_constraint_flag = get_bits1(gb); + gci->gci_no_cclm_constraint_flag = get_bits1(gb); + + /* inter */ + gci->gci_no_ref_pic_resampling_constraint_flag = get_bits1(gb); + gci->gci_no_res_change_in_clvs_constraint_flag = get_bits1(gb); + gci->gci_no_weighted_prediction_constraint_flag = get_bits1(gb); + gci->gci_no_ref_wraparound_constraint_flag = get_bits1(gb); + gci->gci_no_temporal_mvp_constraint_flag = get_bits1(gb); + gci->gci_no_sbtmvp_constraint_flag = get_bits1(gb); + gci->gci_no_amvr_constraint_flag = get_bits1(gb); + gci->gci_no_bdof_constraint_flag = get_bits1(gb); + gci->gci_no_smvd_constraint_flag = get_bits1(gb); + gci->gci_no_dmvr_constraint_flag = get_bits1(gb); + gci->gci_no_mmvd_constraint_flag = get_bits1(gb); + gci->gci_no_affine_motion_constraint_flag = get_bits1(gb); + gci->gci_no_prof_constraint_flag = get_bits1(gb); + gci->gci_no_bcw_constraint_flag = get_bits1(gb); + gci->gci_no_ciip_constraint_flag = get_bits1(gb); + gci->gci_no_gpm_constraint_flag = get_bits1(gb); + + /* transform, quantization, residual */ + gci->gci_no_luma_transform_size_64_constraint_flag = get_bits1(gb); + gci->gci_no_transform_skip_constraint_flag = get_bits1(gb); + gci->gci_no_bdpcm_constraint_flag = get_bits1(gb); + gci->gci_no_mts_constraint_flag = get_bits1(gb); + gci->gci_no_lfnst_constraint_flag = get_bits1(gb); + gci->gci_no_joint_cbcr_constraint_flag = get_bits1(gb); + gci->gci_no_sbt_constraint_flag = get_bits1(gb); + gci->gci_no_act_constraint_flag = get_bits1(gb); + gci->gci_no_explicit_scaling_list_constraint_flag = get_bits1(gb); + gci->gci_no_dep_quant_constraint_flag = get_bits1(gb); + gci->gci_no_sign_data_hiding_constraint_flag = get_bits1(gb); + gci->gci_no_cu_qp_delta_constraint_flag = get_bits1(gb); + gci->gci_no_chroma_qp_offset_constraint_flag = get_bits1(gb); + + /* loop filter */ + gci->gci_no_sao_constraint_flag = get_bits1(gb); + gci->gci_no_alf_constraint_flag = get_bits1(gb); + gci->gci_no_ccalf_constraint_flag = get_bits1(gb); + gci->gci_no_lmcs_constraint_flag = get_bits1(gb); + gci->gci_no_ladf_constraint_flag = get_bits1(gb); + gci->gci_no_virtual_boundaries_constraint_flag = get_bits1(gb); + gci->gci_num_reserved_bits = get_bits(gb,8); + for (i = 0; i < gci->gci_num_reserved_bits; i++) { + gci->gci_reserved_zero_bit[i] = get_bits1(gb); + } + } + + align_get_bits(gb); + + return 0; +} + + +static int decode_profile_tier_level(GetBitContext *gb, AVCodecContext *avctx, + VVCProfileTierLevel *ptl, + int profile_tier_present_flag, + int max_num_sub_layers_minus1 ) +{ + int i; + + if (profile_tier_present_flag) { + ptl->general_profile_idc = get_bits(gb, 7); + ptl->general_tier_flag = get_bits1(gb); + } + ptl->general_level_idc = get_bits(gb, 8); + ptl->ptl_frame_only_constraint_flag = get_bits1(gb); + ptl->ptl_multilayer_enabled_flag = get_bits1(gb); + + if (profile_tier_present_flag) { + decode_general_constraints_info(gb, avctx, &ptl->general_constraints_info); + } + + for (i = max_num_sub_layers_minus1 - 1; i >= 0; i--) { + ptl->ptl_sublayer_level_present_flag[i] = get_bits1(gb); + } + + align_get_bits(gb); + + for (i = max_num_sub_layers_minus1 - 1; i >= 0; i--) { + if (ptl->ptl_sublayer_level_present_flag[i]) + ptl->sublayer_level_idc[i] = get_bits(gb, 8); + } + + if (profile_tier_present_flag) + { + ptl->ptl_num_sub_profiles = get_bits(gb, 8); + for (i = 0; i < ptl->ptl_num_sub_profiles; i++) + ptl->general_sub_profile_idc[i] = get_bits_long(gb, 32); + } + + return 0; +} + +static int decode_dpb_parameters(GetBitContext *gb, AVCodecContext *avctx, + VVCDpbParameters *dpb, + uint8_t max_sublayers_minus1, + uint8_t sublayer_info_flag) +{ + int i; + for (i = (sublayer_info_flag ? 0 : max_sublayers_minus1); + i <= max_sublayers_minus1; i++) { + dpb->dpb_max_dec_pic_buffering_minus1[i] = get_ue_golomb(gb); + dpb->dpb_max_num_reorder_pics[i] = get_ue_golomb(gb); + dpb->dpb_max_latency_increase_plus1[i] = get_ue_golomb(gb); + } + return 0; +} + +static int decode_ref_pic_list_struct(GetBitContext *gb, AVCodecContext *avctx, + VVCRefPicListStruct *current, + uint8_t list_idx, uint8_t rpls_idx, + const VVCSPS *sps) +{ + int i, j, num_direct_ref_layers = 0; + + current->num_ref_entries = get_ue_golomb( gb); + if (sps->long_term_ref_pics_flag && + rpls_idx < sps->num_ref_pic_lists[list_idx] && + current->num_ref_entries > 0) + current->ltrp_in_header_flag = get_bits1(gb); + if (sps->long_term_ref_pics_flag && + rpls_idx == sps->num_ref_pic_lists[list_idx]) + current->ltrp_in_header_flag = 1; + for (i = 0, j = 0; i < current->num_ref_entries; i++) { + if (sps->inter_layer_prediction_enabled_flag) + current->inter_layer_ref_pic_flag[i] = get_bits1(gb); + else + current->inter_layer_ref_pic_flag[i] = 0; + + if (!current->inter_layer_ref_pic_flag[i]) { + if (sps->long_term_ref_pics_flag) + current->st_ref_pic_flag[i] = get_bits1(gb); + else + current->st_ref_pic_flag[i] = 1; + if (current->st_ref_pic_flag[i]) { + int abs_delta_poc_st; + current->abs_delta_poc_st[i] = get_ue_golomb( gb); + if ((sps->weighted_pred_flag || + sps->weighted_bipred_flag) && i != 0) + abs_delta_poc_st = current->abs_delta_poc_st[i]; + else + abs_delta_poc_st = current->abs_delta_poc_st[i] + 1; + if (abs_delta_poc_st > 0) + current->strp_entry_sign_flag[i] = get_bits1(gb); + } else { + if (!current->ltrp_in_header_flag) { + uint8_t bits = sps->log2_max_pic_order_cnt_lsb_minus4 + 4; + current->rpls_poc_lsb_lt[j] = get_bits(gb, bits ); + j++; + } + } + } else { + if (num_direct_ref_layers == 0) { + av_log(avctx, AV_LOG_ERROR, + "num_direct_ref_layers needs > 0.\n"); + return AVERROR_INVALIDDATA; + } + current->ilrp_idx[i] = get_ue_golomb( gb); + } + } + return 0; +} + +static int decode_general_timing_hrd_parameters(GetBitContext *gb, VVCGeneralTimingHrdParameters *current) +{ + current->num_units_in_tick = get_bits_long(gb,32); + current->time_scale = get_bits_long(gb,32); + current->general_nal_hrd_params_present_flag = get_bits1(gb); + current->general_vcl_hrd_params_present_flag = get_bits1(gb); + + if (current->general_nal_hrd_params_present_flag || + current->general_vcl_hrd_params_present_flag) { + current->general_same_pic_timing_in_all_ols_flag = get_bits1(gb); + current->general_du_hrd_params_present_flag = get_bits1(gb); + if (current->general_du_hrd_params_present_flag) + current->tick_divisor_minus2 = get_bits(gb,8); + current->bit_rate_scale = get_bits(gb,4); + current->cpb_size_scale = get_bits(gb,4); + if (current->general_du_hrd_params_present_flag) + current->cpb_size_du_scale = get_bits(gb,4); + current->hrd_cpb_cnt_minus1 = get_ue_golomb_long(gb); + } else { + current->general_du_hrd_params_present_flag = 0; + } + return 0; +} + +static int decode_sublayer_hrd_parameters(GetBitContext *gb, + VVCSubLayerHRDParameters *current, int sublayer_id, + const VVCGeneralTimingHrdParameters *general) +{ + int i; + for (i = 0; i <= general->hrd_cpb_cnt_minus1; i++) { + current->bit_rate_value_minus1[sublayer_id][i] = get_ue_golomb_long(gb); + current->cpb_size_value_minus1[sublayer_id][i] = get_ue_golomb_long(gb); + if (general->general_du_hrd_params_present_flag) { + current->cpb_size_du_value_minus1[sublayer_id][i] = get_ue_golomb_long(gb); + current->bit_rate_du_value_minus1[sublayer_id][i] = get_ue_golomb_long(gb); + } + current->cbr_flag[sublayer_id][i] = get_bits1(gb); + } + return 0; +} + +static int decode_ols_timing_hrd_parameters( GetBitContext *gb, + VVCOlsTimingHrdParameters *current, + uint8_t first_sublayer, uint8_t max_sublayers_minus1, + const VVCGeneralTimingHrdParameters *general) +{ + int i; + for (i = first_sublayer; i <= max_sublayers_minus1; i++) { + current->fixed_pic_rate_general_flag[i] = get_bits1(gb); + if (!current->fixed_pic_rate_general_flag[i]) + current->fixed_pic_rate_within_cvs_flag[i] = get_bits1(gb); + else + current->fixed_pic_rate_within_cvs_flag[i] = 1; + if (current->fixed_pic_rate_within_cvs_flag[i]) { + current->elemental_duration_in_tc_minus1[i] = get_ue_golomb_long(gb); + current->low_delay_hrd_flag[i] = 0; + } else if ((general->general_nal_hrd_params_present_flag || + general->general_vcl_hrd_params_present_flag) && + general->hrd_cpb_cnt_minus1 == 0) { + current->low_delay_hrd_flag[i] = get_bits1(gb); + } else { + current->low_delay_hrd_flag[i] = 0; + } + if (general->general_nal_hrd_params_present_flag) + decode_sublayer_hrd_parameters(gb, + ¤t->nal_sub_layer_hrd_parameters, + i, general); + if (general->general_vcl_hrd_params_present_flag) + decode_sublayer_hrd_parameters(gb, + ¤t->nal_sub_layer_hrd_parameters, i, general); + } + return 0; +} + +static int decode_vui(GetBitContext *gb, AVCodecContext *avctx, + VVCVUI* vui, uint8_t chroma_format_idc ) +{ + vui->progressive_source_flag = get_bits1(gb); + vui->interlaced_source_flag = get_bits1(gb); + vui->non_packed_constraint_flag = get_bits1(gb); + vui->non_projected_constraint_flag = get_bits1(gb); + vui->aspect_ratio_info_present_flag = get_bits1(gb); + if (vui->aspect_ratio_info_present_flag) { + vui->aspect_ratio_constant_flag = get_bits1(gb); + vui->aspect_ratio_idc = get_bits(gb, 8); + if (vui->aspect_ratio_idc == 255) { + vui->sar_width = get_bits(gb, 16); + vui->sar_height = get_bits(gb, 16); + } + } else { + vui->aspect_ratio_constant_flag = 0; + vui->aspect_ratio_idc = 0; + } + vui->overscan_info_present_flag = get_bits1(gb); + if (vui->overscan_info_present_flag) + vui->overscan_appropriate_flag = get_bits1(gb); + vui->colour_description_present_flag = get_bits1(gb); + if (vui->colour_description_present_flag) { + vui->colour_primaries = get_bits(gb, 8); + vui->transfer_characteristics = get_bits(gb, 8); + vui->matrix_coeffs = get_bits(gb, 8); + av_log(avctx, AV_LOG_DEBUG, "colour_primaries: %d transfer_characteristics: %d matrix_coeffs: %d \n", + vui->colour_primaries, vui->transfer_characteristics, vui->matrix_coeffs); + + vui->full_range_flag = get_bits1(gb); + } else { + vui->colour_primaries = 2; + vui->transfer_characteristics = 2; + vui->matrix_coeffs = 2; + vui->full_range_flag = 0; + } + vui->chroma_loc_info_present_flag = get_bits1(gb); + if (chroma_format_idc != 1 && vui->chroma_loc_info_present_flag) { + av_log(avctx, AV_LOG_ERROR, "chroma_format_idc == %d," + "chroma_loc_info_present_flag can't not be true", chroma_format_idc); + return AVERROR_INVALIDDATA; + } + if (vui->chroma_loc_info_present_flag) { + if (vui->progressive_source_flag && + !vui->interlaced_source_flag) { + vui->chroma_sample_loc_type_frame = get_ue_golomb(gb); + } else { + vui->chroma_sample_loc_type_top_field = get_ue_golomb(gb); + vui->chroma_sample_loc_type_bottom_field = get_ue_golomb(gb); + } + } else { + if (chroma_format_idc == 1) { + vui->chroma_sample_loc_type_frame = get_ue_golomb(gb); + vui->chroma_sample_loc_type_top_field = get_ue_golomb(gb); + vui->chroma_sample_loc_type_bottom_field = get_ue_golomb(gb); + } + } + return 0; +} + +static int map_pixel_format(AVCodecContext *avctx, VVCSPS *sps) +{ + const AVPixFmtDescriptor *desc; + switch (sps->bit_depth) { + case 8: + if (sps->chroma_format_idc == 0) sps->pix_fmt = AV_PIX_FMT_GRAY8; + if (sps->chroma_format_idc == 1) sps->pix_fmt = AV_PIX_FMT_YUV420P; + if (sps->chroma_format_idc == 2) sps->pix_fmt = AV_PIX_FMT_YUV422P; + if (sps->chroma_format_idc == 3) sps->pix_fmt = AV_PIX_FMT_YUV444P; + break; + case 9: + if (sps->chroma_format_idc == 0) sps->pix_fmt = AV_PIX_FMT_GRAY9; + if (sps->chroma_format_idc == 1) sps->pix_fmt = AV_PIX_FMT_YUV420P9; + if (sps->chroma_format_idc == 2) sps->pix_fmt = AV_PIX_FMT_YUV422P9; + if (sps->chroma_format_idc == 3) sps->pix_fmt = AV_PIX_FMT_YUV444P9; + break; + case 10: + if (sps->chroma_format_idc == 0) sps->pix_fmt = AV_PIX_FMT_GRAY10; + if (sps->chroma_format_idc == 1) sps->pix_fmt = AV_PIX_FMT_YUV420P10; + if (sps->chroma_format_idc == 2) sps->pix_fmt = AV_PIX_FMT_YUV422P10; + if (sps->chroma_format_idc == 3) sps->pix_fmt = AV_PIX_FMT_YUV444P10; + break; + case 12: + if (sps->chroma_format_idc == 0) sps->pix_fmt = AV_PIX_FMT_GRAY12; + if (sps->chroma_format_idc == 1) sps->pix_fmt = AV_PIX_FMT_YUV420P12; + if (sps->chroma_format_idc == 2) sps->pix_fmt = AV_PIX_FMT_YUV422P12; + if (sps->chroma_format_idc == 3) sps->pix_fmt = AV_PIX_FMT_YUV444P12; + break; + default: + av_log(avctx, AV_LOG_ERROR, + "The following bit-depths are currently specified: 8, 9, 10 and 12 bits, " + "chroma_format_idc is %d, depth is %d\n", + sps->chroma_format_idc, sps->bit_depth); + return AVERROR_INVALIDDATA; + } + + desc = av_pix_fmt_desc_get(sps->pix_fmt); + if (!desc) + return AVERROR(EINVAL); + + return 0; +} + +int ff_vvc_parse_sps(VVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, + int apply_defdispwin, AVCodecContext *avctx) +{ + int i, j; + unsigned int ctb_log2_size_y, ctb_size_y, max_num_merge_cand, + tmp_width_val, tmp_height_val; + + sps->sps_id = get_bits(gb, 4); + sps->vps_id = get_bits(gb, 4); + + *sps_id = sps->sps_id; + + sps->max_sublayers = get_bits(gb, 3) + 1; + sps->chroma_format_idc = get_bits(gb, 2); + sps->log2_ctu_size = get_bits(gb, 2) + 5; + + ctb_log2_size_y = sps->log2_ctu_size; + ctb_size_y = 1 << ctb_log2_size_y; + + sps->ptl_dpb_hrd_params_present_flag = get_bits1(gb); + if (sps->ptl_dpb_hrd_params_present_flag) + decode_profile_tier_level( gb, avctx, &sps->profile_tier_level, 1, sps->max_sublayers-1 ); + + sps->gdr_enabled_flag = get_bits1(gb); + sps->ref_pic_resampling_enabled_flag = get_bits1(gb); + + if (sps->ref_pic_resampling_enabled_flag) + sps->res_change_in_clvs_allowed_flag = get_bits1(gb); + else + sps->res_change_in_clvs_allowed_flag = 0; + + sps->pic_width_max_in_luma_samples = get_ue_golomb(gb); + sps->pic_height_max_in_luma_samples = get_ue_golomb(gb); + + sps->conformance_window_flag = get_bits1(gb); + + if (sps->conformance_window_flag) { + sps->conf_win_left_offset = get_ue_golomb(gb); + sps->conf_win_right_offset = get_ue_golomb(gb); + sps->conf_win_top_offset = get_ue_golomb(gb); + sps->conf_win_bottom_offset = get_ue_golomb(gb); + } else { + sps->conf_win_left_offset = 0; + sps->conf_win_right_offset = 0; + sps->conf_win_top_offset = 0; + sps->conf_win_bottom_offset = 0; + } + + tmp_width_val = (sps->pic_width_max_in_luma_samples + ctb_size_y -1)/ctb_size_y; + tmp_height_val = (sps->pic_height_max_in_luma_samples + ctb_size_y -1)/ctb_size_y; + + sps->subpic_info_present_flag = get_bits1(gb); + if (sps->subpic_info_present_flag) { + sps->num_subpics_minus1 = get_ue_golomb(gb); + if (sps->num_subpics_minus1 > 0) { + sps->independent_subpics_flag = get_bits1(gb); + sps->subpic_same_size_flag = get_bits1(gb); + } + } + + if (sps->num_subpics_minus1 > 0) { + int wlen = av_ceil_log2(tmp_width_val); + int hlen = av_ceil_log2(tmp_height_val); + if (sps->pic_width_max_in_luma_samples > ctb_size_y) + sps->subpic_width_minus1[0] = get_bits(gb, wlen ); + else + sps->subpic_width_minus1[0] = tmp_width_val - 1; + + if (sps->pic_height_max_in_luma_samples > ctb_size_y) + sps->subpic_height_minus1[0] = get_bits(gb, hlen ); + else + sps->subpic_height_minus1[0] = tmp_height_val; + + if (!sps->independent_subpics_flag) { + sps->subpic_treated_as_pic_flag[0] = get_bits1(gb); + sps->loop_filter_across_subpic_enabled_flag[0] = get_bits1(gb); + } else { + sps->subpic_treated_as_pic_flag[0] = 1; + sps->loop_filter_across_subpic_enabled_flag[0] = 1; + } + + for (i = 1; i <= sps->num_subpics_minus1; i++) { + if (!sps->subpic_same_size_flag) { + if (sps->pic_width_max_in_luma_samples > ctb_size_y) + sps->subpic_ctu_top_left_x[i] = get_bits(gb, wlen ); + else + sps->subpic_ctu_top_left_x[i] = 0; + + if (sps->pic_height_max_in_luma_samples > ctb_size_y) + sps->subpic_ctu_top_left_y[i] = get_bits(gb, hlen ); + else + sps->subpic_ctu_top_left_y[i] = 0; + + if (i < sps->num_subpics_minus1 && + sps->pic_width_max_in_luma_samples > ctb_size_y) { + sps->subpic_width_minus1[i] = get_bits(gb, wlen ); + } else { + sps->subpic_width_minus1[i] = tmp_width_val - sps->subpic_ctu_top_left_x[i] - 1; + } + if (i < sps->num_subpics_minus1 && + sps->pic_height_max_in_luma_samples > ctb_size_y) { + sps->subpic_height_minus1[i] = get_bits(gb, hlen ); + } else { + sps->subpic_height_minus1[i] = tmp_height_val - sps->subpic_ctu_top_left_y[i] - 1; + } + } else { + int num_subpic_cols = + tmp_width_val / (sps->subpic_width_minus1[0] + 1); + sps->subpic_ctu_top_left_x[i] = (i % num_subpic_cols) *(sps->subpic_width_minus1[0] + 1); + sps->subpic_ctu_top_left_y[i] = (i / num_subpic_cols) *(sps->subpic_height_minus1[0] + 1); + sps->subpic_width_minus1[i] = sps->subpic_width_minus1[0]; + sps->subpic_height_minus1[i] = sps->subpic_height_minus1[0]; + } + if (!sps->independent_subpics_flag) { + sps->subpic_treated_as_pic_flag[i] = get_bits1(gb); + sps->loop_filter_across_subpic_enabled_flag[i] = get_bits1(gb); + } else { + sps->subpic_treated_as_pic_flag[i] = 1; + sps->loop_filter_across_subpic_enabled_flag[i] = 0; + } + } + sps->subpic_id_len_minus1 = get_ue_golomb(gb ); + + if ((1 << (sps->subpic_id_len_minus1 + 1)) < + sps->num_subpics_minus1 + 1) { + av_log(avctx, AV_LOG_ERROR, + "sps->subpic_id_len_minus1(%d) is too small\n", + sps->subpic_id_len_minus1); + return AVERROR_INVALIDDATA; + } + sps->subpic_id_mapping_explicitly_signalled_flag = get_bits1(gb); + if (sps->subpic_id_mapping_explicitly_signalled_flag) { + sps->subpic_id_mapping_present_flag = get_bits1(gb); + if (sps->subpic_id_mapping_present_flag) { + for (i = 0; i <= sps->num_subpics_minus1; i++) + sps->subpic_id[i] = get_bits(gb, sps->subpic_id_len_minus1 + 1 ); + } + } else { + sps->subpic_ctu_top_left_x[0]= 0; + sps->subpic_ctu_top_left_y[0]= 0; + sps->subpic_width_minus1[0] = tmp_width_val - 1; + sps->subpic_height_minus1[0]= tmp_height_val - 1; + } + } else { + sps->num_subpics_minus1 = 0; + sps->independent_subpics_flag = 1; + sps->subpic_same_size_flag = 0; + sps->subpic_id_mapping_explicitly_signalled_flag= 0; + sps->subpic_ctu_top_left_x[0] = 0; + sps->subpic_ctu_top_left_y[0] = 0; + sps->subpic_width_minus1[0] = tmp_width_val - 1; + sps->subpic_height_minus1[0] = tmp_height_val - 1; + } + + sps->bit_depth = get_ue_golomb(gb ) + 8; + + sps->entropy_coding_sync_enabled_flag = get_bits1(gb); + sps->entry_point_offsets_present_flag = get_bits1(gb); + + sps->log2_max_pic_order_cnt_lsb_minus4 = get_bits(gb, 4 ); + + sps->poc_msb_cycle_flag = get_bits1(gb); + if (sps->poc_msb_cycle_flag) + sps->poc_msb_cycle_len_minus1 = get_ue_golomb(gb); + + sps->num_extra_ph_bytes = get_bits(gb,2); + + for (i = 0; i < FFMIN(16,(sps->num_extra_ph_bytes * 8)); i++) { + sps->extra_ph_bit_present_flag[i] = get_bits1(gb); + } + + sps->num_extra_sh_bytes = get_bits(gb,2); + for (i = 0; i < FFMIN(16,(sps->num_extra_sh_bytes * 8)); i++) { + sps->extra_sh_bit_present_flag[i] = get_bits1(gb); + } + + if (sps->ptl_dpb_hrd_params_present_flag) { + if (sps->max_sublayers > 1) + sps->sublayer_dpb_params_flag = get_bits1(gb); + else + sps->sublayer_dpb_params_flag = 0; + + decode_dpb_parameters( gb, avctx, &sps->dpb_params, + sps->max_sublayers-1, + sps->sublayer_dpb_params_flag); + } + + sps->log2_min_luma_coding_block_size_minus2 = get_ue_golomb(gb); + sps->partition_constraints_override_enabled_flag = get_bits1(gb); + sps->log2_diff_min_qt_min_cb_intra_slice_luma = get_ue_golomb(gb); + sps->max_mtt_hierarchy_depth_intra_slice_luma = get_ue_golomb(gb); + + if (sps->max_mtt_hierarchy_depth_intra_slice_luma != 0) { + sps->log2_diff_max_bt_min_qt_intra_slice_luma = get_ue_golomb(gb); + sps->log2_diff_max_tt_min_qt_intra_slice_luma = get_ue_golomb(gb); + } else { + sps->log2_diff_max_bt_min_qt_intra_slice_luma = 0; + sps->log2_diff_max_tt_min_qt_intra_slice_luma = 0; + } + + if (sps->chroma_format_idc != 0) { + sps->qtbtt_dual_tree_intra_flag = get_bits1(gb); + } else { + sps->qtbtt_dual_tree_intra_flag = 0; + } + + if (sps->qtbtt_dual_tree_intra_flag) { + sps->log2_diff_min_qt_min_cb_intra_slice_chroma = get_ue_golomb(gb); + sps->max_mtt_hierarchy_depth_intra_slice_chroma = get_ue_golomb(gb); + if (sps->max_mtt_hierarchy_depth_intra_slice_chroma != 0) { + sps->log2_diff_max_bt_min_qt_intra_slice_chroma = get_ue_golomb(gb); + sps->log2_diff_max_tt_min_qt_intra_slice_chroma = get_ue_golomb(gb); + } + } else { + sps->log2_diff_min_qt_min_cb_intra_slice_chroma = 0; + sps->max_mtt_hierarchy_depth_intra_slice_chroma = 0; + } + if (sps->max_mtt_hierarchy_depth_intra_slice_chroma == 0) { + sps->log2_diff_max_bt_min_qt_intra_slice_chroma = 0; + sps->log2_diff_max_tt_min_qt_intra_slice_chroma = 0; + } + + sps->log2_diff_min_qt_min_cb_inter_slice = get_ue_golomb(gb); + + sps->max_mtt_hierarchy_depth_inter_slice = get_ue_golomb(gb); + if (sps->max_mtt_hierarchy_depth_inter_slice != 0) { + sps->log2_diff_max_bt_min_qt_inter_slice = get_ue_golomb(gb); + sps->log2_diff_max_tt_min_qt_inter_slice = get_ue_golomb(gb); + } else { + sps->log2_diff_max_bt_min_qt_inter_slice = 0; + sps->log2_diff_max_tt_min_qt_inter_slice = 0; + } + + if (ctb_size_y > 32) + sps->max_luma_transform_size_64_flag = get_bits1(gb); + else + sps->max_luma_transform_size_64_flag = 0; + + sps->transform_skip_enabled_flag = get_bits1(gb); + if (sps->transform_skip_enabled_flag) { + sps->log2_transform_skip_max_size_minus2 = get_ue_golomb(gb); + sps->bdpcm_enabled_flag = get_bits1(gb); + } + + sps->mts_enabled_flag = get_bits1(gb); + if (sps->mts_enabled_flag) { + sps->explicit_mts_intra_enabled_flag = get_bits1(gb); + sps->explicit_mts_inter_enabled_flag = get_bits1(gb); + } else { + sps->explicit_mts_intra_enabled_flag = 0; + sps->explicit_mts_inter_enabled_flag = 0; + } + + sps->lfnst_enabled_flag = get_bits1(gb); + + if (sps->chroma_format_idc != 0) { + uint8_t num_qp_tables; + sps->joint_cbcr_enabled_flag = get_bits1(gb); + sps->same_qp_table_for_chroma_flag = get_bits1(gb); + num_qp_tables = sps->same_qp_table_for_chroma_flag ? + 1 : (sps->joint_cbcr_enabled_flag ? 3 : 2); + for (i = 0; i < num_qp_tables; i++) { + sps->qp_table_start_minus26[i] = get_se_golomb(gb); + sps->num_points_in_qp_table_minus1[i] = get_ue_golomb(gb); + for (j = 0; j <= sps->num_points_in_qp_table_minus1[i]; j++) { + sps->delta_qp_in_val_minus1[i][j] = get_ue_golomb(gb); + sps->delta_qp_diff_val[i][j] = get_ue_golomb(gb); + } + } + } else { + sps->joint_cbcr_enabled_flag = 0; + sps->same_qp_table_for_chroma_flag = 0; + } + + sps->sao_enabled_flag = get_bits1(gb); + sps->alf_enabled_flag = get_bits1(gb); + if (sps->alf_enabled_flag && sps->chroma_format_idc) + sps->ccalf_enabled_flag = get_bits1(gb); + else + sps->ccalf_enabled_flag = 0; + + sps->lmcs_enabled_flag = get_bits1(gb); + sps->weighted_pred_flag = get_bits1(gb); + sps->weighted_bipred_flag = get_bits1(gb); + sps->long_term_ref_pics_flag = get_bits1(gb); + if (sps->vps_id > 0) + sps->inter_layer_prediction_enabled_flag = get_bits1(gb); + else + sps->inter_layer_prediction_enabled_flag = 0; + sps->idr_rpl_present_flag = get_bits1(gb); + sps->rpl1_same_as_rpl0_flag = get_bits1(gb); + + for (i = 0; i < (sps->rpl1_same_as_rpl0_flag ? 1 : 2); i++) { + sps->num_ref_pic_lists[i] = get_ue_golomb(gb); + for (j = 0; j < sps->num_ref_pic_lists[i]; j++) + decode_ref_pic_list_struct( gb, avctx, + &sps->ref_pic_list_struct[i][j], + i, j, sps); + } + + if (sps->rpl1_same_as_rpl0_flag) { + sps->num_ref_pic_lists[1] = sps->num_ref_pic_lists[0]; + for (j = 0; j < sps->num_ref_pic_lists[0]; j++) + memcpy(&sps->ref_pic_list_struct[1][j], + &sps->ref_pic_list_struct[0][j], + sizeof(sps->ref_pic_list_struct[0][j])); + } + + sps->ref_wraparound_enabled_flag = get_bits1(gb); + + sps->temporal_mvp_enabled_flag = get_bits1(gb); + if (sps->temporal_mvp_enabled_flag) + sps->sbtmvp_enabled_flag = get_bits1(gb); + else + sps->sbtmvp_enabled_flag = 0; + + sps->amvr_enabled_flag = get_bits1(gb); + sps->bdof_enabled_flag = get_bits1(gb); + if (sps->bdof_enabled_flag) + sps->bdof_control_present_in_ph_flag = get_bits1(gb); + else + sps->bdof_control_present_in_ph_flag = 0; + + sps->smvd_enabled_flag = get_bits1(gb); + sps->dmvr_enabled_flag = get_bits1(gb); + if (sps->dmvr_enabled_flag) + sps->dmvr_control_present_in_ph_flag = get_bits1(gb); + else + sps->dmvr_control_present_in_ph_flag = 0; + + sps->mmvd_enabled_flag = get_bits1(gb); + if (sps->mmvd_enabled_flag) + sps->mmvd_fullpel_only_enabled_flag = get_bits1(gb); + else + sps->mmvd_fullpel_only_enabled_flag = 0; + + sps->six_minus_max_num_merge_cand = get_ue_golomb(gb); + max_num_merge_cand = 6 - sps->six_minus_max_num_merge_cand; + + sps->sbt_enabled_flag = get_bits1(gb); + + sps->affine_enabled_flag = get_bits1(gb); + if (sps->affine_enabled_flag) { + sps->five_minus_max_num_subblock_merge_cand = get_ue_golomb(gb); + sps->param_affine_enabled_flag = get_bits1(gb); + if (sps->amvr_enabled_flag) + sps->affine_amvr_enabled_flag = get_bits1(gb); + else + sps->affine_amvr_enabled_flag = 0; + sps->affine_prof_enabled_flag = get_bits1(gb); + if (sps->affine_prof_enabled_flag) + sps->prof_control_present_in_ph_flag = get_bits1(gb); + else + sps->prof_control_present_in_ph_flag = 0; + } else { + sps->param_affine_enabled_flag = 0; + sps->affine_amvr_enabled_flag = 0; + sps->affine_prof_enabled_flag = 0; + sps->prof_control_present_in_ph_flag = 0; + } + + sps->bcw_enabled_flag = get_bits1(gb); + sps->ciip_enabled_flag = get_bits1(gb); + + if (max_num_merge_cand >= 2) { + sps->gpm_enabled_flag = get_bits1(gb); + if (sps->gpm_enabled_flag && max_num_merge_cand >= 3) + sps->max_num_merge_cand_minus_max_num_gpm_cand = get_ue_golomb(gb); + } else { + sps->gpm_enabled_flag = 0; + } + + sps->log2_parallel_merge_level_minus2 = get_ue_golomb(gb); + + sps->isp_enabled_flag = get_bits1(gb); + sps->mrl_enabled_flag = get_bits1(gb); + sps->mip_enabled_flag = get_bits1(gb); + + if (sps->chroma_format_idc != 0) + sps->cclm_enabled_flag = get_bits1(gb); + else + sps->cclm_enabled_flag = 0; + if (sps->chroma_format_idc == 1) { + sps->chroma_horizontal_collocated_flag = get_bits1(gb); + sps->chroma_vertical_collocated_flag = get_bits1(gb); + } else { + sps->chroma_horizontal_collocated_flag = 0; + sps->chroma_vertical_collocated_flag = 0; + } + + sps->palette_enabled_flag = get_bits1(gb); + if (sps->chroma_format_idc == 3 && + !sps->max_luma_transform_size_64_flag) + sps->act_enabled_flag = get_bits1(gb); + else + sps->act_enabled_flag = 0; + if (sps->transform_skip_enabled_flag || + sps->palette_enabled_flag) + sps->min_qp_prime_ts = get_ue_golomb(gb); + + sps->ibc_enabled_flag = get_bits1(gb); + if (sps->ibc_enabled_flag) + sps->six_minus_max_num_ibc_merge_cand = get_ue_golomb(gb); + + sps->ladf_enabled_flag = get_bits1(gb); + if (sps->ladf_enabled_flag) { + sps->num_ladf_intervals_minus2 = get_bits( gb, 2 ); + sps->ladf_lowest_interval_qp_offset = get_se_golomb(gb); + for (i = 0; i < sps->num_ladf_intervals_minus2 + 1; i++) { + sps->ladf_qp_offset[i] = get_se_golomb(gb); + sps->ladf_delta_threshold_minus1[i] = get_ue_golomb(gb); + } + } + + sps->explicit_scaling_list_enabled_flag = get_bits1(gb); + if (sps->lfnst_enabled_flag && + sps->explicit_scaling_list_enabled_flag) + sps->scaling_matrix_for_lfnst_disabled_flag = get_bits1(gb); + + if (sps->act_enabled_flag && + sps->explicit_scaling_list_enabled_flag) + sps->scaling_matrix_for_alternative_colour_space_disabled_flag = get_bits1(gb); + else + sps->scaling_matrix_for_alternative_colour_space_disabled_flag = 0; + if (sps->scaling_matrix_for_alternative_colour_space_disabled_flag) + sps->scaling_matrix_designated_colour_space_flag = get_bits1(gb); + + sps->dep_quant_enabled_flag = get_bits1(gb); + sps->sign_data_hiding_enabled_flag = get_bits1(gb); + + sps->virtual_boundaries_enabled_flag = get_bits1(gb); + if (sps->virtual_boundaries_enabled_flag) { + sps->virtual_boundaries_present_flag = get_bits1(gb); + if (sps->virtual_boundaries_present_flag) { + sps->num_ver_virtual_boundaries = get_ue_golomb(gb); + for (i = 0; i < sps->num_ver_virtual_boundaries; i++) + sps->virtual_boundary_pos_x_minus1[i] = get_ue_golomb(gb); + for (i = 0; i < sps->num_hor_virtual_boundaries; i++) + sps->virtual_boundary_pos_y_minus1[i] = get_ue_golomb(gb); + } + } else { + sps->virtual_boundaries_present_flag = 0; + sps->num_ver_virtual_boundaries = 0; + sps->num_hor_virtual_boundaries = 0; + } + + if (sps->ptl_dpb_hrd_params_present_flag) { + sps->timing_hrd_params_present_flag = get_bits1(gb); + if (sps->timing_hrd_params_present_flag) { + uint8_t first_sublayer; + decode_general_timing_hrd_parameters(gb, &sps->general_timing_hrd_parameters); + + if (sps->max_sublayers > 1) + sps->sublayer_cpb_params_present_flag = get_bits1(gb); + else + sps->sublayer_cpb_params_present_flag = 0; + + first_sublayer = sps->sublayer_cpb_params_present_flag ? + 0 : sps->max_sublayers-1; + + decode_ols_timing_hrd_parameters ( gb, + &sps->ols_timing_hrd_parameters, first_sublayer, + sps->max_sublayers-1, + &sps->general_timing_hrd_parameters); + } + } + + sps->field_seq_flag = get_bits1(gb); + sps->vui_parameters_present_flag = get_bits1(gb); + if (sps->vui_parameters_present_flag) { + sps->vui_payload_size_minus1 = get_ue_golomb(gb); + align_get_bits(gb); + decode_vui( gb, avctx, &sps->vui, sps->chroma_format_idc); + } + + sps->extension_flag = get_bits1(gb); + // TODO: parse sps extension flag and read extension data + + map_pixel_format( avctx, sps); + + return 0; +} + +int ff_vvc_decode_nal_sps( GetBitContext *gb, AVCodecContext *avctx, + VVCParamSets *ps, int apply_defdispwin) +{ + unsigned int sps_id; + int ret; + VVCSPS *sps; + AVBufferRef *sps_buf = av_buffer_allocz(sizeof(*sps)); + + if (!sps_buf) + return AVERROR(ENOMEM); + sps = (VVCSPS*)sps_buf->data; + + ret = ff_vvc_parse_sps(sps, gb, &sps_id, + apply_defdispwin, + avctx); + if (ret < 0) { + av_buffer_unref(&sps_buf); + return ret; + } + + if (avctx->debug & FF_DEBUG_BITSTREAM) + { + av_log(avctx, AV_LOG_DEBUG, + "Parsed SPS: id %d; coded wxh: %dx%d; " + "pix_fmt: %s.\n", + sps->sps_id, sps->pic_width_max_in_luma_samples, sps->pic_height_max_in_luma_samples, + av_get_pix_fmt_name(sps->pix_fmt)); + } + + /* check if this is a repeat of an already parsed SPS, then keep the + * original one otherwise drop all PPSes that depend on it (PPS,VPS not implemented yet) */ + if (ps->sps_list[sps_id] && + !memcmp(ps->sps_list[sps_id]->data, sps_buf->data, sps_buf->size)) { + av_buffer_unref(&sps_buf); + } else { + remove_sps(ps, sps_id); + ps->sps_list[sps_id] = sps_buf; + ps->sps = (VVCSPS*)ps->sps_list[sps_id]->data; + } + + // TODO: read PPS flag and data + + return 0; +} + +void ff_vvc_ps_uninit(VVCParamSets *ps) +{ + int i; + + for (i = 0; i < FF_ARRAY_ELEMS(ps->sps_list); i++) + av_buffer_unref(&ps->sps_list[i]); + + // TODO: if PPS, VPS is implemended it must be unrefed as well + // for (i = 0; i < FF_ARRAY_ELEMS(ps->vps_list); i++) + // av_buffer_unref(&ps->vps_list[i]); + // for (i = 0; i < FF_ARRAY_ELEMS(ps->pps_list); i++) + // av_buffer_unref(&ps->pps_list[i]); + + ps->sps = NULL; + //ps->pps = NULL; + //ps->vps = NULL; +} diff --git a/libavcodec/vvc_paramset.h b/libavcodec/vvc_paramset.h new file mode 100644 index 0000000000..c4bb2e509b --- /dev/null +++ b/libavcodec/vvc_paramset.h @@ -0,0 +1,429 @@ +/* + * VVC parameter set parsing + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_VVC_PARAMSET_H +#define AVCODEC_VVC_PARAMSET_H + +#include + +#include "libavutil/buffer.h" +#include "libavutil/pixfmt.h" +#include "libavutil/rational.h" + +#include "avcodec.h" +#include "get_bits.h" +#include "vvc.h" +#include "cbs_h266.h" + + +typedef struct VVCGeneralConstraintsInfo { + uint8_t gci_present_flag; + /* general */ + uint8_t gci_intra_only_constraint_flag; + uint8_t gci_all_layers_independent_constraint_flag; + uint8_t gci_one_au_only_constraint_flag; + + /* picture format */ + uint8_t gci_sixteen_minus_max_bitdepth_constraint_idc; + uint8_t gci_three_minus_max_chroma_format_constraint_idc; + + /* NAL unit type related */ + uint8_t gci_no_mixed_nalu_types_in_pic_constraint_flag; + uint8_t gci_no_trail_constraint_flag; + uint8_t gci_no_stsa_constraint_flag; + uint8_t gci_no_rasl_constraint_flag; + uint8_t gci_no_radl_constraint_flag; + uint8_t gci_no_idr_constraint_flag; + uint8_t gci_no_cra_constraint_flag; + uint8_t gci_no_gdr_constraint_flag; + uint8_t gci_no_aps_constraint_flag; + uint8_t gci_no_idr_rpl_constraint_flag; + + /* tile, slice, subpicture partitioning */ + uint8_t gci_one_tile_per_pic_constraint_flag; + uint8_t gci_pic_header_in_slice_header_constraint_flag; + uint8_t gci_one_slice_per_pic_constraint_flag; + uint8_t gci_no_rectangular_slice_constraint_flag; + uint8_t gci_one_slice_per_subpic_constraint_flag; + uint8_t gci_no_subpic_info_constraint_flag; + + /* CTU and block partitioning */ + uint8_t gci_three_minus_max_log2_ctu_size_constraint_idc; + uint8_t gci_no_partition_constraints_override_constraint_flag; + uint8_t gci_no_mtt_constraint_flag; + uint8_t gci_no_qtbtt_dual_tree_intra_constraint_flag; + + /* intra */ + uint8_t gci_no_palette_constraint_flag; + uint8_t gci_no_ibc_constraint_flag; + uint8_t gci_no_isp_constraint_flag; + uint8_t gci_no_mrl_constraint_flag; + uint8_t gci_no_mip_constraint_flag; + uint8_t gci_no_cclm_constraint_flag; + + /* inter */ + uint8_t gci_no_ref_pic_resampling_constraint_flag; + uint8_t gci_no_res_change_in_clvs_constraint_flag;; + uint8_t gci_no_weighted_prediction_constraint_flag; + uint8_t gci_no_ref_wraparound_constraint_flag; + uint8_t gci_no_temporal_mvp_constraint_flag; + uint8_t gci_no_sbtmvp_constraint_flag; + uint8_t gci_no_amvr_constraint_flag; + uint8_t gci_no_bdof_constraint_flag; + uint8_t gci_no_smvd_constraint_flag; + uint8_t gci_no_dmvr_constraint_flag; + uint8_t gci_no_mmvd_constraint_flag; + uint8_t gci_no_affine_motion_constraint_flag; + uint8_t gci_no_prof_constraint_flag; + uint8_t gci_no_bcw_constraint_flag; + uint8_t gci_no_ciip_constraint_flag; + uint8_t gci_no_gpm_constraint_flag; + + /* transform, quantization, residual */ + uint8_t gci_no_luma_transform_size_64_constraint_flag; + uint8_t gci_no_transform_skip_constraint_flag; + uint8_t gci_no_bdpcm_constraint_flag; + uint8_t gci_no_mts_constraint_flag; + uint8_t gci_no_lfnst_constraint_flag; + uint8_t gci_no_joint_cbcr_constraint_flag; + uint8_t gci_no_sbt_constraint_flag; + uint8_t gci_no_act_constraint_flag; + uint8_t gci_no_explicit_scaling_list_constraint_flag; + uint8_t gci_no_dep_quant_constraint_flag; + uint8_t gci_no_sign_data_hiding_constraint_flag; + uint8_t gci_no_cu_qp_delta_constraint_flag; + uint8_t gci_no_chroma_qp_offset_constraint_flag; + + /* loop filter */ + uint8_t gci_no_sao_constraint_flag; + uint8_t gci_no_alf_constraint_flag; + uint8_t gci_no_ccalf_constraint_flag; + uint8_t gci_no_lmcs_constraint_flag; + uint8_t gci_no_ladf_constraint_flag; + uint8_t gci_no_virtual_boundaries_constraint_flag; + uint8_t gci_num_reserved_bits; + uint8_t gci_reserved_zero_bit[255]; +} VVCGeneralConstraintsInfo; + +typedef struct VVCProfileTierLevel { + uint8_t general_profile_idc; + uint8_t general_tier_flag; + uint8_t general_level_idc; + uint8_t ptl_frame_only_constraint_flag; + uint8_t ptl_multilayer_enabled_flag; + VVCGeneralConstraintsInfo general_constraints_info; + uint8_t ptl_sublayer_level_present_flag[VVC_MAX_SUBLAYERS - 1]; + uint8_t sublayer_level_idc[VVC_MAX_SUBLAYERS - 1]; + uint8_t ptl_num_sub_profiles; + uint32_t general_sub_profile_idc[VVC_MAX_SUB_PROFILES]; + + uint8_t ptl_reserved_zero_bit; +} VVCProfileTierLevel; + +typedef struct VVCDpbParameters { + uint8_t dpb_max_dec_pic_buffering_minus1[VVC_MAX_SUBLAYERS]; + uint8_t dpb_max_num_reorder_pics[VVC_MAX_SUBLAYERS]; + uint8_t dpb_max_latency_increase_plus1[VVC_MAX_SUBLAYERS]; +} VVCDpbParameters; + +typedef struct VVCRefPicListStruct { + uint8_t num_ref_entries; + uint8_t ltrp_in_header_flag; + uint8_t inter_layer_ref_pic_flag[VVC_MAX_REF_ENTRIES]; + uint8_t st_ref_pic_flag[VVC_MAX_REF_ENTRIES]; + uint8_t abs_delta_poc_st[VVC_MAX_REF_ENTRIES]; + uint8_t strp_entry_sign_flag[VVC_MAX_REF_ENTRIES]; + uint8_t rpls_poc_lsb_lt[VVC_MAX_REF_ENTRIES]; + uint8_t ilrp_idx[VVC_MAX_REF_ENTRIES]; +} VVCRefPicListStruct; + +typedef struct VVCGeneralTimingHrdParameters { + uint32_t num_units_in_tick; + uint32_t time_scale; + uint8_t general_nal_hrd_params_present_flag; + uint8_t general_vcl_hrd_params_present_flag; + uint8_t general_same_pic_timing_in_all_ols_flag; + uint8_t general_du_hrd_params_present_flag; + uint8_t tick_divisor_minus2; + uint8_t bit_rate_scale; + uint8_t cpb_size_scale; + uint8_t cpb_size_du_scale; + uint8_t hrd_cpb_cnt_minus1; +} VVCGeneralTimingHrdParameters; + +typedef struct VVCSubLayerHRDParameters { + uint32_t bit_rate_value_minus1[VVC_MAX_SUBLAYERS][VVC_MAX_CPB_CNT]; + uint32_t cpb_size_value_minus1[VVC_MAX_SUBLAYERS][VVC_MAX_CPB_CNT]; + uint32_t cpb_size_du_value_minus1[VVC_MAX_SUBLAYERS][VVC_MAX_CPB_CNT]; + uint32_t bit_rate_du_value_minus1[VVC_MAX_SUBLAYERS][VVC_MAX_CPB_CNT]; + uint8_t cbr_flag[VVC_MAX_SUBLAYERS][VVC_MAX_CPB_CNT]; +} VVCSubLayerHRDParameters; + +typedef struct VVCOlsTimingHrdParameters { + uint8_t fixed_pic_rate_general_flag[VVC_MAX_SUBLAYERS]; + uint8_t fixed_pic_rate_within_cvs_flag[VVC_MAX_SUBLAYERS]; + uint16_t elemental_duration_in_tc_minus1[VVC_MAX_SUBLAYERS]; + uint8_t low_delay_hrd_flag[VVC_MAX_SUBLAYERS]; + VVCSubLayerHRDParameters nal_sub_layer_hrd_parameters; + VVCSubLayerHRDParameters vcl_sub_layer_hrd_parameters; +} VVCOlsTimingHrdParameters; + + +typedef struct VVCVUI { + uint8_t progressive_source_flag; + uint8_t interlaced_source_flag; + uint8_t non_packed_constraint_flag; + uint8_t non_projected_constraint_flag; + + uint8_t aspect_ratio_info_present_flag; + uint8_t aspect_ratio_constant_flag; + uint8_t aspect_ratio_idc; + + uint16_t sar_width; + uint16_t sar_height;; + + uint8_t overscan_info_present_flag; + uint8_t overscan_appropriate_flag; + + uint8_t colour_description_present_flag; + uint8_t colour_primaries; + + uint8_t transfer_characteristics; + uint8_t matrix_coeffs; + uint8_t full_range_flag; + + uint8_t chroma_loc_info_present_flag; + uint8_t chroma_sample_loc_type_frame; + uint8_t chroma_sample_loc_type_top_field; + uint8_t chroma_sample_loc_type_bottom_field; + //VVCExtensionData extension_data; +} VVCVUI; + +typedef struct VVCSPS { + uint8_t sps_id; + uint8_t vps_id; + uint8_t max_sublayers; + uint8_t chroma_format_idc; + uint8_t log2_ctu_size; + uint8_t ptl_dpb_hrd_params_present_flag; + VVCProfileTierLevel profile_tier_level; + uint8_t gdr_enabled_flag; + uint8_t ref_pic_resampling_enabled_flag; + uint8_t res_change_in_clvs_allowed_flag; + + uint16_t pic_width_max_in_luma_samples; + uint16_t pic_height_max_in_luma_samples; + + uint8_t conformance_window_flag; + uint16_t conf_win_left_offset; + uint16_t conf_win_right_offset; + uint16_t conf_win_top_offset; + uint16_t conf_win_bottom_offset; + + uint8_t subpic_info_present_flag; + uint16_t num_subpics_minus1; + uint8_t independent_subpics_flag; + uint8_t subpic_same_size_flag; + uint16_t subpic_ctu_top_left_x[VVC_MAX_SLICES]; + uint16_t subpic_ctu_top_left_y[VVC_MAX_SLICES]; + uint16_t subpic_width_minus1[VVC_MAX_SLICES]; + uint16_t subpic_height_minus1[VVC_MAX_SLICES]; + uint8_t subpic_treated_as_pic_flag[VVC_MAX_SLICES]; + uint8_t loop_filter_across_subpic_enabled_flag[VVC_MAX_SLICES]; + uint8_t subpic_id_len_minus1; + uint8_t subpic_id_mapping_explicitly_signalled_flag; + uint8_t subpic_id_mapping_present_flag; + uint32_t subpic_id[VVC_MAX_SLICES]; + + + uint8_t bit_depth; + uint8_t entropy_coding_sync_enabled_flag; + uint8_t entry_point_offsets_present_flag; + + uint8_t log2_max_pic_order_cnt_lsb_minus4; + uint8_t poc_msb_cycle_flag; + uint8_t poc_msb_cycle_len_minus1; + + uint8_t num_extra_ph_bytes; + uint8_t extra_ph_bit_present_flag[16]; + + uint8_t num_extra_sh_bytes; + uint8_t extra_sh_bit_present_flag[16]; + + uint8_t sublayer_dpb_params_flag; + VVCDpbParameters dpb_params; + + uint8_t log2_min_luma_coding_block_size_minus2; + uint8_t partition_constraints_override_enabled_flag; + uint8_t log2_diff_min_qt_min_cb_intra_slice_luma; + uint8_t max_mtt_hierarchy_depth_intra_slice_luma; + uint8_t log2_diff_max_bt_min_qt_intra_slice_luma; + uint8_t log2_diff_max_tt_min_qt_intra_slice_luma; + + uint8_t qtbtt_dual_tree_intra_flag; + uint8_t log2_diff_min_qt_min_cb_intra_slice_chroma; + uint8_t max_mtt_hierarchy_depth_intra_slice_chroma; + uint8_t log2_diff_max_bt_min_qt_intra_slice_chroma; + uint8_t log2_diff_max_tt_min_qt_intra_slice_chroma; + + uint8_t log2_diff_min_qt_min_cb_inter_slice; + uint8_t max_mtt_hierarchy_depth_inter_slice; + uint8_t log2_diff_max_bt_min_qt_inter_slice; + uint8_t log2_diff_max_tt_min_qt_inter_slice; + + uint8_t max_luma_transform_size_64_flag; + + uint8_t transform_skip_enabled_flag; + uint8_t log2_transform_skip_max_size_minus2; + uint8_t bdpcm_enabled_flag; + + uint8_t mts_enabled_flag; + uint8_t explicit_mts_intra_enabled_flag; + uint8_t explicit_mts_inter_enabled_flag; + + uint8_t lfnst_enabled_flag; + + uint8_t joint_cbcr_enabled_flag; + uint8_t same_qp_table_for_chroma_flag; + + int8_t qp_table_start_minus26[VVC_MAX_SAMPLE_ARRAYS]; + uint8_t num_points_in_qp_table_minus1[VVC_MAX_SAMPLE_ARRAYS]; + uint8_t delta_qp_in_val_minus1[VVC_MAX_SAMPLE_ARRAYS][VVC_MAX_POINTS_IN_QP_TABLE]; + uint8_t delta_qp_diff_val[VVC_MAX_SAMPLE_ARRAYS][VVC_MAX_POINTS_IN_QP_TABLE]; + + uint8_t sao_enabled_flag; + uint8_t alf_enabled_flag; + uint8_t ccalf_enabled_flag; + uint8_t lmcs_enabled_flag; + uint8_t weighted_pred_flag; + uint8_t weighted_bipred_flag; + uint8_t long_term_ref_pics_flag; + uint8_t inter_layer_prediction_enabled_flag; + uint8_t idr_rpl_present_flag; + uint8_t rpl1_same_as_rpl0_flag; + + uint8_t num_ref_pic_lists[2]; + VVCRefPicListStruct ref_pic_list_struct[2][VVC_MAX_REF_PIC_LISTS]; + + uint8_t ref_wraparound_enabled_flag; + uint8_t temporal_mvp_enabled_flag; + uint8_t sbtmvp_enabled_flag; + uint8_t amvr_enabled_flag; + uint8_t bdof_enabled_flag; + uint8_t bdof_control_present_in_ph_flag; + uint8_t smvd_enabled_flag; + uint8_t dmvr_enabled_flag; + uint8_t dmvr_control_present_in_ph_flag; + uint8_t mmvd_enabled_flag; + uint8_t mmvd_fullpel_only_enabled_flag; + uint8_t six_minus_max_num_merge_cand; + uint8_t sbt_enabled_flag; + uint8_t affine_enabled_flag; + uint8_t five_minus_max_num_subblock_merge_cand; + uint8_t param_affine_enabled_flag; + uint8_t affine_amvr_enabled_flag; + uint8_t affine_prof_enabled_flag; + uint8_t prof_control_present_in_ph_flag; + uint8_t bcw_enabled_flag; + uint8_t ciip_enabled_flag; + uint8_t gpm_enabled_flag; + uint8_t max_num_merge_cand_minus_max_num_gpm_cand; + uint8_t log2_parallel_merge_level_minus2; + uint8_t isp_enabled_flag; + uint8_t mrl_enabled_flag; + uint8_t mip_enabled_flag; + uint8_t cclm_enabled_flag; + uint8_t chroma_horizontal_collocated_flag; + uint8_t chroma_vertical_collocated_flag; + uint8_t palette_enabled_flag; + uint8_t act_enabled_flag; + uint8_t min_qp_prime_ts; + uint8_t ibc_enabled_flag; + uint8_t six_minus_max_num_ibc_merge_cand; + uint8_t ladf_enabled_flag; + uint8_t num_ladf_intervals_minus2; + int8_t ladf_lowest_interval_qp_offset; + int8_t ladf_qp_offset[4]; + uint16_t ladf_delta_threshold_minus1[4]; + + uint8_t explicit_scaling_list_enabled_flag; + uint8_t scaling_matrix_for_lfnst_disabled_flag; + uint8_t scaling_matrix_for_alternative_colour_space_disabled_flag; + uint8_t scaling_matrix_designated_colour_space_flag; + uint8_t dep_quant_enabled_flag; + uint8_t sign_data_hiding_enabled_flag; + + uint8_t virtual_boundaries_enabled_flag; + uint8_t virtual_boundaries_present_flag; + uint8_t num_ver_virtual_boundaries; + uint16_t virtual_boundary_pos_x_minus1[3]; + uint8_t num_hor_virtual_boundaries; + uint16_t virtual_boundary_pos_y_minus1[3]; + + uint8_t timing_hrd_params_present_flag; + uint8_t sublayer_cpb_params_present_flag; + VVCGeneralTimingHrdParameters general_timing_hrd_parameters; + VVCOlsTimingHrdParameters ols_timing_hrd_parameters; + + uint8_t field_seq_flag; + uint8_t vui_parameters_present_flag; + uint16_t vui_payload_size_minus1; + VVCVUI vui; + + uint8_t extension_flag; + //H266RawExtensionData extension_data; /* TODO: read extension flag and data*/ + + enum AVPixelFormat pix_fmt; +} VVCSPS; + + +typedef struct VVCParamSets { + AVBufferRef *sps_list[VVC_MAX_SPS_COUNT]; + //AVBufferRef *vps_list[VVC_MAX_VPS_COUNT]; // TODO: since not needed, not implemented yet + //AVBufferRef *pps_list[VVC_MAX_PPS_COUNT]; // TODO: since not needed, not implemented yet + + /* currently active parameter sets */ + const VVCSPS *sps; + //const VVCVPS *vps; // TODO: since not needed, not implemented yet + //const VVCPPS *pps; // TODO: since not needed, not implemented yet +} VVCParamSets; + +/** + * Parse the SPS from the bitstream into the provided HEVCSPS struct. + * + * @param sps_id the SPS id will be written here + * @param apply_defdispwin if set 1, the default display window from the VUI + * will be applied to the video dimensions + */ +int ff_vvc_parse_sps(VVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, + int apply_defdispwin, AVCodecContext *avctx); + +int ff_vvc_decode_nal_sps(GetBitContext *gb, AVCodecContext *avctx, + VVCParamSets *ps, int apply_defdispwin); + +// TODO: since not needed, not implemented yet +//int ff_vvc_decode_nal_vps(GetBitContext *gb, AVCodecContext *avctx, +// VVCParamSets *ps); +//int ff_vvc_decode_nal_pps(GetBitContext *gb, AVCodecContext *avctx, +// VVCParamSets *ps); + +void ff_vvc_ps_uninit(VVCParamSets *ps); + +#endif /* AVCODEC_VVC_PARAMSET_H */ diff --git a/libavcodec/vvc_parse_extradata.c b/libavcodec/vvc_parse_extradata.c new file mode 100644 index 0000000000..4b7cbbe0b2 --- /dev/null +++ b/libavcodec/vvc_parse_extradata.c @@ -0,0 +1,241 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "bytestream.h" +#include "h2645_parse.h" +#include "vvc.h" +#include "vvc_parse_extradata.h" + +static int vvc_decode_nal_units(const uint8_t *buf, int buf_size, VVCParamSets *ps, + int is_nalff, int nal_length_size, + int err_recognition, int apply_defdispwin, void *logctx) +{ + int i; + int ret = 0; + H2645Packet pkt = { 0 }; + + ret = ff_h2645_packet_split(&pkt, buf, buf_size, logctx, is_nalff, + nal_length_size, AV_CODEC_ID_VVC, 1, 0); + if (ret < 0) { + goto done; + } + + for (i = 0; i < pkt.nb_nals; i++) { + H2645NAL *nal = &pkt.nals[i]; + if (nal->nuh_layer_id > 0) + continue; + + /* ignore everything except parameter sets and VCL NALUs */ + switch (nal->type) { + case VVC_VPS_NUT: + // TODO: since not needed yet, not implemented + //ret = ff_vvc_decode_nal_vps(&nal->gb, logctx, ps); + //if (ret < 0) + // goto done; + break; + case VVC_SPS_NUT: + ret = ff_vvc_decode_nal_sps(&nal->gb, logctx, ps, apply_defdispwin); + if (ret < 0) + goto done; + break; + case VVC_PPS_NUT: + // TODO: since not needed yet, not implemented + //ret = ff_vvc_decode_nal_pps(&nal->gb, logctx, ps); + //if (ret < 0) + // goto done; + break; + case VVC_PREFIX_SEI_NUT: + case VVC_SUFFIX_SEI_NUT: + // TODO: SEI decoding not implemented yet + break; + default: + av_log(logctx, AV_LOG_VERBOSE, "Ignoring NAL type %d in extradata\n", nal->type); + break; + } + } + +done: + ff_h2645_packet_uninit(&pkt); + if (err_recognition & AV_EF_EXPLODE) + return ret; + + return 0; +} + +int ff_vvc_decode_extradata(const uint8_t *data, int size, VVCParamSets *ps, + int *is_nalff, int *nal_length_size, + int err_recognition, int apply_defdispwin, void *logctx) +{ + int ret = 0; + GetByteContext gb; + + bytestream2_init(&gb, data, size); + + if (size > 3 && (data[0] || data[1] || data[2] > 1)) { + /* extradata is encoded as vvcC format. */ + int i, j, b, num_arrays, nal_len_size, num_sublayers, has_ptl; + + b = bytestream2_get_byte(&gb); + num_sublayers = (b >> 3) & 0x7; + nal_len_size = ((b >> 1) & 0x3) + 1; + has_ptl = b & 0x1; + + if (has_ptl) { + int max_picture_width = 0; + int max_picture_height = 0; + int avg_frame_rate = 0; + + int num_bytes_constraint_info; + int general_profile_idc; + int general_tier_flag; + int general_level_idc; + int ptl_frame_only_constraint_flag; + int ptl_multi_layer_enabled_flag; + int ptl_num_sub_profiles; + int temp2, temp3, temp4, temp5; + int ols_idx; + int constant_frame_rate; + int chroma_format_idc; + int bit_depth_minus8; + + temp2 = bytestream2_get_be16(&gb); + ols_idx = (temp2 >> 7) & 0x1ff; + num_sublayers = (temp2 >> 4) & 0x7; + constant_frame_rate = (temp2 >> 2) & 0x3; + chroma_format_idc = temp2 & 0x3; + bit_depth_minus8 = (bytestream2_get_byte(&gb) >> 5) & 0x7; + av_log(logctx, AV_LOG_TRACE, + "bit_depth_minus8 %d chroma_format_idc %d\n", + bit_depth_minus8, chroma_format_idc); + av_log(logctx, AV_LOG_TRACE, + "constant_frame_rate %d, ols_idx %d\n", + constant_frame_rate, ols_idx); + // VvcPTLRecord(num_sublayers) native_ptl + temp3 = bytestream2_get_byte(&gb); + num_bytes_constraint_info = (temp3) & 0x3f; + temp4 = bytestream2_get_byte(&gb); + general_profile_idc = (temp4 >> 1) & 0x7f; + general_tier_flag = (temp4) & 1; + general_level_idc = bytestream2_get_byte(&gb); + av_log(logctx, AV_LOG_TRACE, + "general_profile_idc %d, general_tier_flag %d, " + "general_level_idc %d, num_sublayers %d num_bytes_constraint_info %d\n", + general_profile_idc, general_tier_flag, general_level_idc, num_sublayers, + num_bytes_constraint_info); + + temp5 = bytestream2_get_byte(&gb); + ptl_frame_only_constraint_flag = (temp5 >> 7) & 0x1; + ptl_multi_layer_enabled_flag = (temp5 >> 6) & 0x1; + for (i = 0; i < num_bytes_constraint_info-1; i++) { + // unsigned int(8*num_bytes_constraint_info - 2) general_constraint_info; + bytestream2_get_byte(&gb); + } + + av_log(logctx, AV_LOG_TRACE, + "ptl_multi_layer_enabled_flag %d, ptl_frame_only_constraint_flag %d\n", + ptl_multi_layer_enabled_flag, ptl_frame_only_constraint_flag); + + if( num_sublayers > 1 ) { + int temp6 = bytestream2_get_byte(&gb); + uint8_t ptl_sublayer_level_present_flag[8] = {0}; + uint8_t sublayer_level_idc[8] = {0}; + for (i=num_sublayers - 2; i >= 0; i--) { + ptl_sublayer_level_present_flag[i] = (temp6 >> (7 - (num_sublayers - 2 - i))) & 0x01; + } + for (i=num_sublayers-2; i >= 0; i--) { + if (ptl_sublayer_level_present_flag[i]) { + sublayer_level_idc[i] = bytestream2_get_byte(&gb); // TODO: + } + } + } + + ptl_num_sub_profiles = bytestream2_get_byte(&gb); + for (j=0; j < ptl_num_sub_profiles; j++) { + // unsigned int(32) general_sub_profile_idc[j]; + bytestream2_get_be16(&gb); + bytestream2_get_be16(&gb); + } + + max_picture_width = bytestream2_get_be16(&gb); + max_picture_height = bytestream2_get_be16(&gb); + avg_frame_rate = bytestream2_get_be16(&gb); + av_log(logctx, AV_LOG_TRACE, + "max_picture_width %d, max_picture_height %d, avg_frame_rate %d\n", + max_picture_width, max_picture_height, avg_frame_rate); + } + + *is_nalff = 1; + + /* nal units in the vvcC always have length coded with 2 bytes, + * so set nal_length_size = 2 while parsing them */ + *nal_length_size = 2; + + num_arrays = bytestream2_get_byte(&gb); + + for (i = 0; i < num_arrays; i++) { + int cnt; + int type = bytestream2_get_byte(&gb) & 0x1f; + + if (type != VVC_OPI_NUT || type != VVC_DCI_NUT) + cnt = bytestream2_get_be16(&gb); + else + cnt = 1; + + av_log(logctx, AV_LOG_DEBUG, "nalu_type %d cnt %d\n", type, cnt); + + if (!(type == VVC_VPS_NUT || type == VVC_SPS_NUT || type == VVC_PPS_NUT || + type == VVC_PREFIX_SEI_NUT || type == VVC_SUFFIX_SEI_NUT)) { + av_log(logctx, AV_LOG_ERROR, "Invalid NAL unit type in extradata: %d\n", + type); + ret = AVERROR_INVALIDDATA; + return ret; + } + + for (j = 0; j < cnt; j++) { + // +2 for the nal size field + int nalsize = bytestream2_peek_be16(&gb) + 2; + if (bytestream2_get_bytes_left(&gb) < nalsize) { + av_log(logctx, AV_LOG_ERROR, + "Invalid NAL unit size in extradata.\n"); + return AVERROR_INVALIDDATA; + } + + ret = vvc_decode_nal_units(gb.buffer, nalsize, ps, *is_nalff, + *nal_length_size, err_recognition, apply_defdispwin, + logctx); + if (ret < 0) { + av_log(logctx, AV_LOG_ERROR, + "Decoding nal unit %d %d from vvcC failed\n", type, i); + return ret; + } + bytestream2_skip(&gb, nalsize); + } + } + + /* store nal length size, that will be used to parse all other nals */ + *nal_length_size = nal_len_size; + } else { + *is_nalff = 0; + ret = vvc_decode_nal_units(data, size, ps, *is_nalff, *nal_length_size, + err_recognition, apply_defdispwin, logctx); + if (ret < 0) + return ret; + } + + return ret; +} diff --git a/libavcodec/vvc_parse_extradata.h b/libavcodec/vvc_parse_extradata.h new file mode 100644 index 0000000000..4adb7616b7 --- /dev/null +++ b/libavcodec/vvc_parse_extradata.h @@ -0,0 +1,36 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * H.266 parser code + */ + +#ifndef AVCODEC_VVC_PARSE_EXTRADATA_H +#define AVCODEC_VVC_PARSE_EXTRADATA_H + +#include + +#include "vvc_paramset.h" + + +int ff_vvc_decode_extradata(const uint8_t *data, int size, VVCParamSets *ps, + int *is_nalff, int *nal_length_size, + int err_recognition, int apply_defdispwin, void *logctx); + +#endif /* AVCODEC_VVC_PARSE_EXTRADATA_H */