From patchwork Sat Jan 9 07:34:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Nuo Mi X-Patchwork-Id: 24863 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 64C90449441 for ; Sat, 9 Jan 2021 10:01:35 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 360B668A558; Sat, 9 Jan 2021 10:01:35 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pg1-f171.google.com (mail-pg1-f171.google.com [209.85.215.171]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 11D8868A558 for ; Sat, 9 Jan 2021 10:01:29 +0200 (EET) Received: by mail-pg1-f171.google.com with SMTP id 15so9159195pgx.7 for ; Sat, 09 Jan 2021 00:01:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8CyLsfbkMJcuqQ9xfDFvi3LQgeYy9cVbktSD6GAbgNA=; b=OV10NKpk8KCOk4VKcIsjJEhP1idqsux8TkTMZhv7vAy3YDtNwJpxew+XQOoFx4qpBT XTCFm1Eav9Dw9i3MQAUgfE4cu5ddGmqnZ9ugBxQXQa2L75Y3iMkASqzJ6AdD8jacye4H KdX9CPPZJHsI7/hfT6tEjiSxpyvrfCQvsZ7riA/wR7modnfK2aWoSTDGx47LtMZCdhl3 xMR6UShhTSXO/GdzAbsL2xWhuGhdW2MhlOxbLMHGEKaWo+c0oXI/eWWBP+gFkyQbxYjw Cvi6uKwxczPn2DyKJwrAwMHhfK7PZc5JCsWz0faYO2YBkwHz+lFC58HDvZIz1u6X4eyn nV7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8CyLsfbkMJcuqQ9xfDFvi3LQgeYy9cVbktSD6GAbgNA=; b=IA24W5D0HzC7BS+mZ4doZj4FjPJY6GY8Tr/bAY0YonEwawRlj93eP1PHfb1wJ1+gUG caL83NfrgJ8se8G+xAePWZDI7TfcjO1w9jSq5j7z66q6SKMyduh23xlNC4h9+s46XW2g txbVvNq3UMgMw08qwlzr1X+Tj77T/XXR/oyF0te4toTOr5iqVlYiedjR3CMNkSNHl4N5 0uJOCDwJIwZInDdWgLd7CKdOGFS8jXf2Nf+LPkhgagVzk7A+5hBHIY/G5vVJ95yb/SzS SUx7BzjS1zkRfbZPyO8eqfFrfGLa9U4txVNCYEgCK+TfIr5ehiascLM4PK3ddguvs6Fy TwrQ== X-Gm-Message-State: AOAM532GfclTreiwqC8NmQLWv6mQ3wW2jsQyFEaYXfWfyqen6dnFwo2D EKR4xx0SRuBVDQ4//c1PkpTcQ6nnSAlAhw== X-Google-Smtp-Source: ABdhPJyJyxfUnKiouYSsAMC609Cwz28l7mu48zNokA/gKrtUkVk5RZBVOdIobSCYmZMUiaMA5YlfnQ== X-Received: by 2002:a17:902:8f90:b029:db:fc74:c59a with SMTP id z16-20020a1709028f90b02900dbfc74c59amr10772688plo.58.1610177682350; Fri, 08 Jan 2021 23:34:42 -0800 (PST) Received: from Guangxin-PC.localdomain (23.83.245.51.16clouds.com. [23.83.245.51]) by smtp.gmail.com with ESMTPSA id m15sm11640120pfa.72.2021.01.08.23.34.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Jan 2021 23:34:41 -0800 (PST) From: Nuo Mi To: ffmpeg-devel@ffmpeg.org Date: Sat, 9 Jan 2021 15:34:11 +0800 Message-Id: <20210109073421.23721-2-nuomi2021@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210109073421.23721-1-nuomi2021@gmail.com> References: <20210109073421.23721-1-nuomi2021@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 01/11] avcodec/vvc: add shared header for vvc X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Nuo Mi Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/vvc.h | 124 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 124 insertions(+) create mode 100644 libavcodec/vvc.h diff --git a/libavcodec/vvc.h b/libavcodec/vvc.h new file mode 100644 index 0000000000..0bd2acac1d --- /dev/null +++ b/libavcodec/vvc.h @@ -0,0 +1,124 @@ +/* + * VVC shared code + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_VVC_H +#define AVCODEC_VVC_H + +/** + * Table 5 – NAL unit type codes and NAL unit type classes + * in T-REC-H.266-202008 + */ +enum VVCNALUnitType { + VVC_TRAIL_NUT = 0, + VVC_STSA_NUT = 1, + VVC_RADL_NUT = 2, + VVC_RASL_NUT = 3, + VVC_RSV_VCL_4 = 4, + VVC_RSV_VCL_5 = 5, + VVC_RSV_VCL_6 = 6, + VVC_IDR_W_RADL = 7, + VVC_IDR_N_LP = 8, + VVC_CRA_NUT = 9, + VVC_GDR_NUT = 10, + VVC_RSV_IRAP_11 = 11, + VVC_OPI_NUT = 12, + VVC_DCI_NUT = 13, + VVC_VPS_NUT = 14, + VVC_SPS_NUT = 15, + VVC_PPS_NUT = 16, + VVC_PREFIX_APS_NUT = 17, + VVC_SUFFIX_APS_NUT = 18, + VVC_PH_NUT = 19, + VVC_AUD_NUT = 20, + VVC_EOS_NUT = 21, + VVC_EOB_NUT = 22, + VVC_PREFIX_SEI_NUT = 23, + VVC_SUFFIX_SEI_NUT = 24, + VVC_FD_NUT = 25, + VVC_RSV_NVCL_26 = 26, + VVC_RSV_NVCL_27 = 27, + VVC_UNSPEC_28 = 28, + VVC_UNSPEC_29 = 29, + VVC_UNSPEC_30 = 30, + VVC_UNSPEC_31 = 31, +}; + +enum VVCSliceType { + VVC_SLICE_TYPE_B = 0, + VVC_SLICE_TYPE_P = 1, + VVC_SLICE_TYPE_I = 2, +}; + +enum { + VVC_MAX_PLANES = 3, + //7.4.3.3 The value of vps_max_sublayers_minus1 shall be in the range of 0 to 6, inclusive + VVC_MAX_SUBLAYERS = 7, + + // 7.3.2.3: vps_video_parameter_set_id is u(4). + VVC_MAX_VPS_COUNT = 16, + // 7.3.2.4: sps_seq_parameter_set_id is u(4) + VVC_MAX_SPS_COUNT = 16, + // 7.3.2.5: pps_pic_parameter_set_id is u(6) + VVC_MAX_PPS_COUNT = 64, + + // 7.4.4.1: ptl_num_sub_profiles is u(8) + VVC_MAX_SUB_PROFILES = 256, + + // A.4.2: according to (1577), MaxDpbSize is bounded above by 2 * maxDpbPicBuf(8) + VVC_MAX_DPB_SIZE = 16, + + //7.4.3.4 sps_num_ref_pic_lists in range [0, 64] + VVC_MAX_REF_PIC_LISTS = 64, + + //7.4.3.3 sps_num_points_in_qp_table_minus1[i] in range [0, 36 − sps_qp_table_start_minus26[i]], + //sps_qp_table_start_minus26[i] in range [sps_qp_table_start_minus26[i] −26 − QpBdOffset, 36] + //for 10 bitsQpBdOffset is 12, so sps_num_points_in_qp_table_minus1[i] in range [0, 74] + VVC_MAX_POINTS_IN_QP_TABLE = 75, + + // 7.4.6.1: hrd_cpb_cnt_minus1 is in [0, 31]. + VVC_MAX_CPB_CNT = 32, + + // A.4.1: the highest level allows a MaxLumaPs of 35 651 584. + VVC_MAX_LUMA_PS = 35651584, + // A.4.1: pic_width_in_luma_samples and pic_height_in_luma_samples are + // constrained to be not greater than sqrt(MaxLumaPs * 8). Hence height/ + // width are bounded above by sqrt(8 * 35651584) = 16888.2 samples. + VVC_MAX_WIDTH = 16888, + VVC_MAX_HEIGHT = 16888, + + // A.4.1: table A.1 allows at most 440 tiles for any au. + VVC_MAX_TILE_ROWS = 440, + // A.4.1: table A.1 allows at most 20 tile columns for any level. + VVC_MAX_TILE_COLUMNS = 20, + + // A.4.1 table A.1 allows at most 600 slice for any level. + VVC_MAX_SLICES = 600, + + // 7.4.8: in the worst case (tiles_enabled_flag and + // entropy_coding_sync_enabled_flag are both set), entry points can be + // placed at the beginning of every Ctb row in every tile, giving an + // upper bound of (num_tile_columns_minus1 + 1) * PicHeightInCtbsY - 1. + // Only a stream with very high resolution and perverse parameters could + // get near that, though, so set a lower limit here with the maximum + // possible value for 8K video (at most 135 32x32 Ctb rows). + VVC_MAX_ENTRY_POINTS = VVC_MAX_TILE_COLUMNS * 135, +}; + +#endif /* AVCODEC_VVC_H */ From patchwork Sat Jan 9 07:34:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nuo Mi X-Patchwork-Id: 24851 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 7359844A8AC for ; Sat, 9 Jan 2021 09:34:55 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 431BA68A588; Sat, 9 Jan 2021 09:34:55 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7D2D9689D8E for ; Sat, 9 Jan 2021 09:34:48 +0200 (EET) Received: by mail-pl1-f181.google.com with SMTP id q4so6884107plr.7 for ; Fri, 08 Jan 2021 23:34:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=R7wyn92aFPC1XWtLkHXzDJlPzZTqcLUeBb1mvmbEWg4=; b=Gd7alItFT/mPUnh1iBnLudt1nkBWOJ7AQbYnjMSqE89SDn2CqlvzW7JJ1uuJYhCi87 SqC851hyTzfFVRxRdpRz37fOHSzT+cFvX1ZDwibAL7ZHtZot1LUPAIFEy2uAND0myWgZ Vhb0OhT5mLIikc+K0QLOPTf3ciYKoHMmgTk8LPU0SSXAJWDROTxxb1NbuxLL4BJaqDtW 8V3RO0emlAgQaCW23Ug7Czb+RWXHz+/8wBoPiyURChk92OFRaS1K8WVp7R7wzvYeYdFs zJMBIc/HoWUw9UW5Nm9ML7do+E8do23BdjEotr7igsxif22NtMebeL+PAES8pi76q78k WYSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=R7wyn92aFPC1XWtLkHXzDJlPzZTqcLUeBb1mvmbEWg4=; b=OH9V05t0/d0p02eChS92Qc25pGbBK+xnYMs78uuvrLBO2u+o84H0q8IQoY4rsVsDL0 HuDUHyZCJWUWjAZT0MyZCTP7r+MUoedzDXqzbnF5kkqWwjD3/HWy/dH/96RzKSm/v+EH d51T1WyzO8V+9Fi2BsMWpZrrCEdaNEKf4QvdIgBtE0803D5xGALBsAXyiKy7/QgnNhL9 ZZWQGURmigdV37gnFpgRq0mpg+069csBBq1opRjHABscT+t/cBvBzOVm1wA6pn8LGjZw 5O/2E1I/dmEV8kbQ7PQn5glARipGfa8QSKLKjr4upEV3LLxUgxVG6jW4p082dLqwF/EQ kpag== X-Gm-Message-State: AOAM532yY6xmT0VRrzbF65bueIhi+W6zjLNy0FgFSBbeASCRukfwHAjG iYljMWOzIFZZnIP2Z6R5nQfFhuqNheeIQQ== X-Google-Smtp-Source: ABdhPJx3Aj0RSi+jFE4ObHSR/yHA4Ag5lL18B7hnG+3AZWRhSPy8ldRUgVGOn/uSj6OYRUZKP8h0xA== X-Received: by 2002:a17:90a:d307:: with SMTP id p7mr7680392pju.214.1610177686692; Fri, 08 Jan 2021 23:34:46 -0800 (PST) Received: from Guangxin-PC.localdomain (23.83.245.51.16clouds.com. [23.83.245.51]) by smtp.gmail.com with ESMTPSA id m15sm11640120pfa.72.2021.01.08.23.34.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Jan 2021 23:34:46 -0800 (PST) From: Nuo Mi To: ffmpeg-devel@ffmpeg.org Date: Sat, 9 Jan 2021 15:34:12 +0800 Message-Id: <20210109073421.23721-3-nuomi2021@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210109073421.23721-1-nuomi2021@gmail.com> References: <20210109073421.23721-1-nuomi2021@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 02/11] avcodec: add vvc codec id and profiles X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Nuo Mi Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/avcodec.h | 2 ++ libavcodec/codec_desc.c | 8 ++++++++ libavcodec/codec_id.h | 2 ++ libavcodec/profiles.c | 5 +++++ libavcodec/profiles.h | 1 + 5 files changed, 18 insertions(+) diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index 1d3099d50a..13a3191b53 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -1961,6 +1961,8 @@ typedef struct AVCodecContext { #define FF_PROFILE_HEVC_MAIN_STILL_PICTURE 3 #define FF_PROFILE_HEVC_REXT 4 +#define FF_PROFILE_VVC_MAIN_10 1 + #define FF_PROFILE_AV1_MAIN 0 #define FF_PROFILE_AV1_HIGH 1 #define FF_PROFILE_AV1_PROFESSIONAL 2 diff --git a/libavcodec/codec_desc.c b/libavcodec/codec_desc.c index 14757bf31b..a7594f9004 100644 --- a/libavcodec/codec_desc.c +++ b/libavcodec/codec_desc.c @@ -1426,6 +1426,14 @@ static const AVCodecDescriptor codec_descriptors[] = { .long_name = NULL_IF_CONFIG_SMALL("Microsoft Paint (MSP) version 2"), .props = AV_CODEC_PROP_INTRA_ONLY | AV_CODEC_PROP_LOSSLESS, }, + { + .id = AV_CODEC_ID_VVC, + .type = AVMEDIA_TYPE_VIDEO, + .name = "vvc", + .long_name = NULL_IF_CONFIG_SMALL("H.266 / VVC (Versatile Video Coding)"), + .props = AV_CODEC_PROP_LOSSY | AV_CODEC_PROP_REORDER, + .profiles = NULL_IF_CONFIG_SMALL(ff_vvc_profiles), + }, { .id = AV_CODEC_ID_Y41P, .type = AVMEDIA_TYPE_VIDEO, diff --git a/libavcodec/codec_id.h b/libavcodec/codec_id.h index 6133e03bb9..7a8a896bfe 100644 --- a/libavcodec/codec_id.h +++ b/libavcodec/codec_id.h @@ -244,6 +244,8 @@ enum AVCodecID { AV_CODEC_ID_PGX, AV_CODEC_ID_AVS3, AV_CODEC_ID_MSP2, + AV_CODEC_ID_VVC, +#define AV_CODEC_ID_H266 AV_CODEC_ID_VVC AV_CODEC_ID_Y41P = 0x8000, AV_CODEC_ID_AVRP, diff --git a/libavcodec/profiles.c b/libavcodec/profiles.c index e59a3a5c12..6dd6594319 100644 --- a/libavcodec/profiles.c +++ b/libavcodec/profiles.c @@ -74,6 +74,11 @@ const AVProfile ff_h264_profiles[] = { { FF_PROFILE_UNKNOWN }, }; +const AVProfile ff_vvc_profiles[] = { + { FF_PROFILE_VVC_MAIN_10, "Main 10" }, + { FF_PROFILE_UNKNOWN }, +}; + const AVProfile ff_hevc_profiles[] = { { FF_PROFILE_HEVC_MAIN, "Main" }, { FF_PROFILE_HEVC_MAIN_10, "Main 10" }, diff --git a/libavcodec/profiles.h b/libavcodec/profiles.h index 6baaba5701..41a19aa9ad 100644 --- a/libavcodec/profiles.h +++ b/libavcodec/profiles.h @@ -61,6 +61,7 @@ extern const AVProfile ff_dca_profiles[]; extern const AVProfile ff_dnxhd_profiles[]; extern const AVProfile ff_h264_profiles[]; extern const AVProfile ff_hevc_profiles[]; +extern const AVProfile ff_vvc_profiles[]; extern const AVProfile ff_jpeg2000_profiles[]; extern const AVProfile ff_mpeg2_video_profiles[]; extern const AVProfile ff_mpeg4_video_profiles[]; From patchwork Sat Jan 9 07:34:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nuo Mi X-Patchwork-Id: 24852 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 6F4CF44A8AC for ; Sat, 9 Jan 2021 09:34:58 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5509268A5CC; Sat, 9 Jan 2021 09:34:58 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C1CB368A5B4 for ; Sat, 9 Jan 2021 09:34:51 +0200 (EET) Received: by mail-pl1-f172.google.com with SMTP id j1so6894646pld.3 for ; Fri, 08 Jan 2021 23:34:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FzvXwFq3ACbLYFlJ1fyDQ9BVhIXMdAAZSYLpXDYBdz0=; b=mPhDviFRpQhEeUMz62yx0GJFx8op7pfZBH0aFmNF1AjO+FMWfHWcyrG5z75a0ZbFZe 2Dsmo51qK2ELQPyRTzz5hRKigfr6T9ZP85ktI2xAPbsEY6RZB1IGw8GYCpmig2qdIfdP wJ86vmLd+o2l+wbkT/S1B59gzBTvZlBzGyYqSnMwE59fl1hj0Rzwu9q8HD0J32bNTVDN +YIobyPbRz5E2m9nJ7oyCiVgvgOQnfadlIfqn8Agz8Guae4FNVHysCmTO7crl943oyTj +C7bTQvoM6YRzwKh/NAG0arswF0wWfWpUxKWWebaX5aKlWuy7lUEoO3zNYm3jLBgNnAu dVkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FzvXwFq3ACbLYFlJ1fyDQ9BVhIXMdAAZSYLpXDYBdz0=; b=uWuM2/TqauT8ZoCIRObgnDxqQ1mpe9r32INBFJqSJO++7Q9LciJI5EX6ZjRxy5fKLQ ghBOYSy7ojmbn0ro46UWOwPSKGW763b1el12TePpL/crU+5XEPTY0gyk4cgt9Ezu+xsj JlheBBKz8/tWfsv++WlFB3UgbPiZB3ItjYQ9VFqBU8A5EQZ0l9ePnh+591EnE57EqMil imUzmR2vQ4nBdRt9+k+C3i2+E0eoUGA0mCrhh2JOX7XqOb3Vn1SB3ciJiuGn9yRbgHQZ 6KESVY/p85dfajGngm6z4b1XDZbNi99Lsv9CBkJVw/aXlcOdPRGnv+HbZpNqdFttzMWI oAVQ== X-Gm-Message-State: AOAM532uQcH0+PdvYKAYOuoZ7P1WZieVGgUojkdwYRbPNsPyJxBuFzGY h2Qqva5BY2UrRdYFgV0UZAKr1RJFvr5oiQ== X-Google-Smtp-Source: ABdhPJxsXO4IjKxeF/DtWbJyc34MWffVKGItHQwcIE03wrFJmHTQbjViyKYVZaP6F2DJX2sh/46PcA== X-Received: by 2002:a17:90a:5298:: with SMTP id w24mr7583374pjh.182.1610177690058; Fri, 08 Jan 2021 23:34:50 -0800 (PST) Received: from Guangxin-PC.localdomain (23.83.245.51.16clouds.com. [23.83.245.51]) by smtp.gmail.com with ESMTPSA id m15sm11640120pfa.72.2021.01.08.23.34.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Jan 2021 23:34:49 -0800 (PST) From: Nuo Mi To: ffmpeg-devel@ffmpeg.org Date: Sat, 9 Jan 2021 15:34:13 +0800 Message-Id: <20210109073421.23721-4-nuomi2021@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210109073421.23721-1-nuomi2021@gmail.com> References: <20210109073421.23721-1-nuomi2021@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 03/11] avformat: add vvc raw demux X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Nuo Mi Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavformat/Makefile | 1 + libavformat/allformats.c | 1 + libavformat/vvcdec.c | 61 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 63 insertions(+) create mode 100644 libavformat/vvcdec.c diff --git a/libavformat/Makefile b/libavformat/Makefile index 3a8fbcbe5f..4a5406da38 100644 --- a/libavformat/Makefile +++ b/libavformat/Makefile @@ -563,6 +563,7 @@ OBJS-$(CONFIG_VOC_MUXER) += vocenc.o voc.o OBJS-$(CONFIG_VPK_DEMUXER) += vpk.o OBJS-$(CONFIG_VPLAYER_DEMUXER) += vplayerdec.o subtitles.o OBJS-$(CONFIG_VQF_DEMUXER) += vqf.o +OBJS-$(CONFIG_VVC_DEMUXER) += vvcdec.o rawdec.o OBJS-$(CONFIG_W64_DEMUXER) += wavdec.o w64.o pcm.o OBJS-$(CONFIG_W64_MUXER) += wavenc.o w64.o OBJS-$(CONFIG_WAV_DEMUXER) += wavdec.o pcm.o diff --git a/libavformat/allformats.c b/libavformat/allformats.c index 0e0caaad39..fdbf424e31 100644 --- a/libavformat/allformats.c +++ b/libavformat/allformats.c @@ -462,6 +462,7 @@ extern AVOutputFormat ff_voc_muxer; extern AVInputFormat ff_vpk_demuxer; extern AVInputFormat ff_vplayer_demuxer; extern AVInputFormat ff_vqf_demuxer; +extern AVInputFormat ff_vvc_demuxer; extern AVInputFormat ff_w64_demuxer; extern AVOutputFormat ff_w64_muxer; extern AVInputFormat ff_wav_demuxer; diff --git a/libavformat/vvcdec.c b/libavformat/vvcdec.c new file mode 100644 index 0000000000..149f39f28e --- /dev/null +++ b/libavformat/vvcdec.c @@ -0,0 +1,61 @@ +/* + * RAW VVC video demuxer + * Copyright (c) 2020 Nuo Mi + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavcodec/vvc.h" + +#include "avformat.h" +#include "rawdec.h" + +static int vvc_probe(const AVProbeData *p) +{ + uint32_t code = -1; + int sps = 0, pps = 0, irap = 0; + int i; + + for (i = 0; i < p->buf_size - 1; i++) { + code = (code << 8) + p->buf[i]; + if ((code & 0xffffff00) == 0x100) { + uint8_t nal2 = p->buf[i + 1]; + int type = (nal2 & 0xF8) >> 3; + + if (code & 0xc0) // forbidden_zero_bit and nuh_reserved_zero_bit + return 0; + + if ((nal2 & 0x7) == 0) // nuh_temporal_id_plus1 + return 0; + + switch (type) { + case VVC_SPS_NUT: sps++; break; + case VVC_PPS_NUT: pps++; break; + case VVC_IDR_N_LP: + case VVC_IDR_W_RADL: + case VVC_CRA_NUT: + case VVC_GDR_NUT: irap++; break; + } + } + } + + if (sps && pps && irap) + return AVPROBE_SCORE_EXTENSION + 1; // 1 more than .mpg + return 0; +} + +FF_DEF_RAWVIDEO_DEMUXER(vvc, "raw VVC video", vvc_probe, "h266,266,vvc", AV_CODEC_ID_VVC) From patchwork Sat Jan 9 07:34:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nuo Mi X-Patchwork-Id: 24864 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 61D3F449906 for ; Sat, 9 Jan 2021 10:02:51 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4C5A968A5EE; Sat, 9 Jan 2021 10:02:51 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-vs1-f47.google.com (mail-vs1-f47.google.com [209.85.217.47]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 18B9E68097A for ; Sat, 9 Jan 2021 10:02:45 +0200 (EET) Received: by mail-vs1-f47.google.com with SMTP id r24so6897664vsg.10 for ; Sat, 09 Jan 2021 00:02:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=TdDnjczr+F1pk0F6TetGMGqyzCNDUUOnRGZva22Wryk=; b=DW7S63GZaQ1SyzTjLPQ28KhKOZISvRaoJ6EzhoswlLGIlTGsxAjVnZtP6J6irp+8KP EQxoib0DH3qPU6Afo9uvqQhh44mIcnt5pYgIqcM85DVM3xDNVoy4h6vERMG/wDPAwUdK I+Dsota7ZXF2JT22Oq1QUtlZlrG0jxpwVFLiXuwIE92/XAasC8refayCvMeIYPIalBW0 y9psK/BWejx31q2w7o/OaylyeApg5lPzFiEfbpo9aQZFoPeL9qD5jzqCVLNeLTt9XGqL 7J7Xi2VnKeFZkySAZXEaJ7ByLjojYJJWm0eItkTxWvn+7x7rfpulhA4ZzwHz2b+pCMYz Kkdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TdDnjczr+F1pk0F6TetGMGqyzCNDUUOnRGZva22Wryk=; b=k8dQqQ61P65YR794Hoau03BFA2+NahGj9lkCxQlozi0b6qGdxivpM/iFcXpUmxgrdD dZ5j1nzlEk6tdYHBFR6DZigj+34jtTJklKVM+4UYMph1UzQ/40I5ILxmTl0+zsV0fLZs AIeZYEqmZ+P1juGS11pitt/eNaAYF7UCcn/CVxhNfOYvRhl+g27yTXu0Ru+cTHFno4kQ MQAVLeIAjOJTWzKc+e2RLbhskqgqEqJgK3sTCxClUhRtXeqUdbIqioH7UAU6oM11QOQU OO3sDlodvOwLZruNi1YT6M6yfDJPVEz7GKWVTdsaIBW/+GvhGQH9NLjnhudknlPmk6Ug evmQ== X-Gm-Message-State: AOAM531NmIDAn6aMaS0U1r9eRlmUhnoEzVtUw0Vm1Syj0pZ/mjP4twAW 3WZC956s9RHPlw2RGhwetc9L/SHu1Q74LA== X-Google-Smtp-Source: ABdhPJyhJzboTjWypKiGDINhQ9s0jm5UvkYYPke+R+8tD0Rq7pq6u/uPfZ5PXISjOVtVrG151kj3mw== X-Received: by 2002:a17:902:b688:b029:dc:240a:2bd7 with SMTP id c8-20020a170902b688b02900dc240a2bd7mr7514587pls.50.1610177693873; Fri, 08 Jan 2021 23:34:53 -0800 (PST) Received: from Guangxin-PC.localdomain (23.83.245.51.16clouds.com. [23.83.245.51]) by smtp.gmail.com with ESMTPSA id m15sm11640120pfa.72.2021.01.08.23.34.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Jan 2021 23:34:53 -0800 (PST) From: Nuo Mi To: ffmpeg-devel@ffmpeg.org Date: Sat, 9 Jan 2021 15:34:14 +0800 Message-Id: <20210109073421.23721-5-nuomi2021@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210109073421.23721-1-nuomi2021@gmail.com> References: <20210109073421.23721-1-nuomi2021@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 04/11] avcodec: add SEI enum for vvc X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Nuo Mi Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/vvc_sei.h | 47 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+) create mode 100644 libavcodec/vvc_sei.h diff --git a/libavcodec/vvc_sei.h b/libavcodec/vvc_sei.h new file mode 100644 index 0000000000..90724669de --- /dev/null +++ b/libavcodec/vvc_sei.h @@ -0,0 +1,47 @@ +/* + * H.266/VVC Supplementary Enhancement Information messages + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_VVC_SEI_H +#define AVCODEC_VVC_SEI_H + +/** + * SEI message types + */ +typedef enum { + VVC_SEI_TYPE_BUFFERING_PERIOD = 0, + VVC_SEI_TYPE_PICTURE_TIMING = 1, + VVC_SEI_TYPE_PAN_SCAN_RECT = 2, + VVC_SEI_TYPE_FILLER_PAYLOAD = 3, + VVC_SEI_TYPE_USER_DATA_REGISTERED_ITU_T_T35 = 4, + VVC_SEI_TYPE_USER_DATA_UNREGISTERED = 5, + VVC_SEI_TYPE_FILM_GRAIN_CHARACTERISTICS = 19, + VVC_SEI_TYPE_FRAME_PACKING = 45, + VVC_SEI_TYPE_PARAMETER_SETS_INCLUSION_INDICATION = 129, + VVC_SEI_TYPE_DECODING_UNIT_INFO = 130, + VVC_SEI_TYPE_DECODED_PICTURE_HASH = 132, + VVC_SEI_TYPE_SCALABLE_NESTING = 133, + VVC_SEI_TYPE_REGION_REFRESH_INFO = 134, + VVC_SEI_TYPE_TIME_CODE = 136, + VVC_SEI_TYPE_MASTERING_DISPLAY_INFO = 137, + VVC_SEI_TYPE_CONTENT_LIGHT_LEVEL_INFO = 144, + VVC_SEI_TYPE_ALTERNATIVE_TRANSFER_CHARACTERISTICS = 147, +} VVC_SEI_Type; + +#endif /* AVCODEC_VVC_SEI_H */ From patchwork Sat Jan 9 07:34:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nuo Mi X-Patchwork-Id: 24862 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 048AE44B4B3 for ; Sat, 9 Jan 2021 09:57:28 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id CA87968A5EE; Sat, 9 Jan 2021 09:57:27 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pg1-f179.google.com (mail-pg1-f179.google.com [209.85.215.179]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E9A876881CE for ; Sat, 9 Jan 2021 09:57:20 +0200 (EET) Received: by mail-pg1-f179.google.com with SMTP id g15so9148740pgu.9 for ; Fri, 08 Jan 2021 23:57:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=NzEeBgKKNr/amE4Cye8Pov+wLJnzTnH4fMHtfH2MrXA=; b=pIh8b8rQbsCOpaFoPzfYR2leIWiB9MQRW84alwLPJOZwtPwW8sUgQSTV0I/gQ8YML9 DP7CiNLLXQtP9wl9fLbYs+C6f3dZPc+edQLf7nf1hINiW7mX7T1kfW3j6ZTpBmGR1gIq V2jyrgdeTf0pHlfASpy8lanCf1vB2L/Fg0MzCnnCKUXei9O/4X7txgEAj8p3Vt7jAQjg 6ncyTmW278S+SowbGjaTbHM8lyRRE1+FbdE3n0SJAkNJrKHTgej5417drCO8hNWtV/CK bfoGuFoLzUr8voLQvZ/mhc6y2DPaAQjd3jy2vh3NwXurDRUYZXNhXgiQRlewMcxMmIzk um8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NzEeBgKKNr/amE4Cye8Pov+wLJnzTnH4fMHtfH2MrXA=; b=jvrGUfoQYyWXJ9gwwzY760zP14wC5INlMV6l28fidlzm6teDOI6KWLLB0kUdYBIfvQ nUeuADu5w7ru77uJWrGW0AA/wiyiH/LV0Q0mTi5/LfqGS2iT39+1ByNJR3gNo3ZUsD3X 1FJCkGDnqfm2QoirhT5Dfv8ZqA29PhOAh8fp4JMPbqqVxIHQ/2t+jcf959IKHA36CP5Q EEg3WKSJywDHotTKT4sLiByxjs7q3+JhWN4ymC404K2KQQ4iAkMbyw8sQ5QZaWvqX5Ie ENMN5pLthqoXq4kuhi1rHfvX0R4vmFrtkXa1DOX/EGBa4Xe0RCMC2FutAQp/VnCD/+qs JMfA== X-Gm-Message-State: AOAM533/8kG5bwgwZFuSNSR1BiPeS/SspHlvT8/7wKqusrwtv/BkcGBD N4W88UpO7CW501xO+KTRmF9MewHSscqGfA== X-Google-Smtp-Source: ABdhPJz268+IA4s/wlFwMTl+Xrkz7r3ESf5CsXr/XKqZzH3G13ATqRPjj/vwoz5QHlYepk5StEEu6g== X-Received: by 2002:a17:902:aa43:b029:dc:26a7:7391 with SMTP id c3-20020a170902aa43b02900dc26a77391mr7613907plr.51.1610177696977; Fri, 08 Jan 2021 23:34:56 -0800 (PST) Received: from Guangxin-PC.localdomain (23.83.245.51.16clouds.com. [23.83.245.51]) by smtp.gmail.com with ESMTPSA id m15sm11640120pfa.72.2021.01.08.23.34.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Jan 2021 23:34:56 -0800 (PST) From: Nuo Mi To: ffmpeg-devel@ffmpeg.org Date: Sat, 9 Jan 2021 15:34:15 +0800 Message-Id: <20210109073421.23721-6-nuomi2021@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210109073421.23721-1-nuomi2021@gmail.com> References: <20210109073421.23721-1-nuomi2021@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 05/11] avcodec/cbs_h265: fix undef SEI_TYPE_X X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Nuo Mi Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/cbs_h265_syntax_template.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/libavcodec/cbs_h265_syntax_template.c b/libavcodec/cbs_h265_syntax_template.c index 48fae82d04..c0e94683a2 100644 --- a/libavcodec/cbs_h265_syntax_template.c +++ b/libavcodec/cbs_h265_syntax_template.c @@ -2197,7 +2197,9 @@ static int FUNC(sei_payload)(CodedBitstreamContext *ctx, RWContext *rw, 1, 0, alternative_transfer_characteristics); SEI_TYPE_N(ALPHA_CHANNEL_INFO, 1, 0, alpha_channel_info); -#undef SEI_TYPE +#undef SEI_TYPE_N +#undef SEI_TYPE_S +#undef SEI_TYPE_E default: { #ifdef READ From patchwork Sat Jan 9 07:34:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nuo Mi X-Patchwork-Id: 24855 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 8C17B44AF8E for ; Sat, 9 Jan 2021 09:35:11 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6E42768A64D; Sat, 9 Jan 2021 09:35:11 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6357868A64D for ; Sat, 9 Jan 2021 09:35:09 +0200 (EET) Received: by mail-pl1-f180.google.com with SMTP id g3so6904404plp.2 for ; Fri, 08 Jan 2021 23:35:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=sHKBirnpH9yu/5PrHh4+aT5JqkUJbw0AITRNiEJfS1g=; b=JRQVnu0tUjOAfYQVcB+sJ5RkB4/gj7U4KPW5Pi5qBxho0sXNX32bfnrHvRowylamje qqZa1iOn0T0GW3lvMzEs1kJmYZYSn2iNY3PZoMHSTna5xEV3fx4Hwphd/fxtk123w9Fg 6ev9e2Wp0xPuH9Rqa2eQfke7ubA7S/ulmEKTeqvgBH+XSuWAVtbDh4QowxnSsHj43qV6 hXtT3uubCcJpIkhpuDt6H5cde0CHi3sc03JA+52p9PHN3HMPGyjqwlVTLttSDfIDZjsF X1JzKBvAxIICVLOuoEEYjFOaeSBwNXpiuD2lBUHwDSXgaE5QM+ovutvJVtkWisDdQlsN tUgw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=sHKBirnpH9yu/5PrHh4+aT5JqkUJbw0AITRNiEJfS1g=; b=ZG9KXIAbkIqjFr/Tr1I9f3k76y+aY1nVkrznB76XqWOk2DwEAnROdMoGSBKBR/2ws0 3croMoO4f9GQJvVjNDtyywJJszTtn1wTPaCteDUcUEnWfe3RS5hINGRm7aB6sMCq+SlP 9tRBHhuc0ZrKeI0/5EdbkMAnneBhgUrvUVvkPxyRprlpYJhaE165FcKjvkrc0oHvxHvD 6FovoeoWFuRLZN1XqXkg5b+wTBBajqmxATRSxYKAQQoVdbKV4emEHA+EAUDYB4ngJCix PeUAc2ncuxC8/E3+BA2sC1sZbFyJYJlSRiislAkHx6PrgsuRpH/q45xNSD3SYFEEAOmA 3sGQ== X-Gm-Message-State: AOAM533hcOMFlaadVhdyf+71Jw93FcPRqNSTPKOeJOp3xJKozILnDgNj bD9+IqEMBsj2jFksRXgeAppn6Fj3d4m7Ng== X-Google-Smtp-Source: ABdhPJyxewctRH63uv4BukvisIA6LOX7q9MBbvE4Y1e+bPGN+3AHlqMJocCVSqXw0hSlVGXKWcTXDw== X-Received: by 2002:a17:902:8341:b029:dc:27b:be2d with SMTP id z1-20020a1709028341b02900dc027bbe2dmr7713639pln.30.1610177705980; Fri, 08 Jan 2021 23:35:05 -0800 (PST) Received: from Guangxin-PC.localdomain (23.83.245.51.16clouds.com. [23.83.245.51]) by smtp.gmail.com with ESMTPSA id m15sm11640120pfa.72.2021.01.08.23.34.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Jan 2021 23:35:05 -0800 (PST) From: Nuo Mi To: ffmpeg-devel@ffmpeg.org Date: Sat, 9 Jan 2021 15:34:16 +0800 Message-Id: <20210109073421.23721-7-nuomi2021@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210109073421.23721-1-nuomi2021@gmail.com> References: <20210109073421.23721-1-nuomi2021@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 06/11] avcodec: add cbs for h266/vvc X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Nuo Mi Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- configure | 2 + libavcodec/Makefile | 1 + libavcodec/cbs.c | 6 + libavcodec/cbs_h2645.c | 373 ++++ libavcodec/cbs_h266.h | 840 ++++++++ libavcodec/cbs_h266_syntax_template.c | 2761 +++++++++++++++++++++++++ libavcodec/cbs_internal.h | 3 +- 7 files changed, 3985 insertions(+), 1 deletion(-) create mode 100644 libavcodec/cbs_h266.h create mode 100644 libavcodec/cbs_h266_syntax_template.c diff --git a/configure b/configure index 900505756b..4935625260 100755 --- a/configure +++ b/configure @@ -2355,6 +2355,7 @@ CONFIG_EXTRA=" cbs_av1 cbs_h264 cbs_h265 + cbs_h266 cbs_jpeg cbs_mpeg2 cbs_vp9 @@ -2623,6 +2624,7 @@ threads_if_any="$THREADS_LIST" cbs_av1_select="cbs" cbs_h264_select="cbs" cbs_h265_select="cbs" +cbs_h266_select="cbs" cbs_jpeg_select="cbs" cbs_mpeg2_select="cbs" cbs_vp9_select="cbs" diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 35318f4f4d..4b406adfce 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -73,6 +73,7 @@ OBJS-$(CONFIG_CBS) += cbs.o OBJS-$(CONFIG_CBS_AV1) += cbs_av1.o OBJS-$(CONFIG_CBS_H264) += cbs_h2645.o h2645_parse.o OBJS-$(CONFIG_CBS_H265) += cbs_h2645.o h2645_parse.o +OBJS-$(CONFIG_CBS_H266) += cbs_h2645.o h2645_parse.o OBJS-$(CONFIG_CBS_JPEG) += cbs_jpeg.o OBJS-$(CONFIG_CBS_MPEG2) += cbs_mpeg2.o OBJS-$(CONFIG_CBS_VP9) += cbs_vp9.o diff --git a/libavcodec/cbs.c b/libavcodec/cbs.c index ecf22b3fdb..6baf21d62b 100644 --- a/libavcodec/cbs.c +++ b/libavcodec/cbs.c @@ -39,6 +39,9 @@ static const CodedBitstreamType *const cbs_type_table[] = { #if CONFIG_CBS_H265 &ff_cbs_type_h265, #endif +#if CONFIG_CBS_H266 + &ff_cbs_type_h266, +#endif #if CONFIG_CBS_JPEG &ff_cbs_type_jpeg, #endif @@ -60,6 +63,9 @@ const enum AVCodecID ff_cbs_all_codec_ids[] = { #if CONFIG_CBS_H265 AV_CODEC_ID_H265, #endif +#if CONFIG_CBS_H266 + AV_CODEC_ID_H266, +#endif #if CONFIG_CBS_JPEG AV_CODEC_ID_MJPEG, #endif diff --git a/libavcodec/cbs_h2645.c b/libavcodec/cbs_h2645.c index 434322492c..2be46adda3 100644 --- a/libavcodec/cbs_h2645.c +++ b/libavcodec/cbs_h2645.c @@ -24,11 +24,14 @@ #include "cbs_internal.h" #include "cbs_h264.h" #include "cbs_h265.h" +#include "cbs_h266.h" #include "h264.h" #include "h264_sei.h" #include "h2645_parse.h" #include "hevc.h" #include "hevc_sei.h" +#include "vvc.h" +#include "vvc_sei.h" static int cbs_read_ue_golomb(CodedBitstreamContext *ctx, GetBitContext *gbc, @@ -256,6 +259,7 @@ static int cbs_h265_payload_extension_present(GetBitContext *gbc, uint32_t paylo #define FUNC_NAME(rw, codec, name) cbs_ ## codec ## _ ## rw ## _ ## name #define FUNC_H264(rw, name) FUNC_NAME(rw, h264, name) #define FUNC_H265(rw, name) FUNC_NAME(rw, h265, name) +#define FUNC_H266(rw, name) FUNC_NAME(rw, h266, name) #define SUBSCRIPTS(subs, ...) (subs > 0 ? ((int[subs + 1]){ subs, __VA_ARGS__ }) : NULL) @@ -364,6 +368,10 @@ static int cbs_h2645_read_more_rbsp_data(GetBitContext *gbc) #include "cbs_h265_syntax_template.c" #undef FUNC +#define FUNC(name) FUNC_H266(READWRITE, name) +#include "cbs_h266_syntax_template.c" +#undef FUNC + #undef READ #undef READWRITE #undef RWContext @@ -436,6 +444,10 @@ static int cbs_h2645_read_more_rbsp_data(GetBitContext *gbc) #include "cbs_h265_syntax_template.c" #undef FUNC +#define FUNC(name) FUNC_H266(READWRITE, name) +#include "cbs_h266_syntax_template.c" +#undef FUNC + #undef WRITE #undef READWRITE #undef RWContext @@ -679,6 +691,8 @@ cbs_h2645_replace_ps(4, PPS, pps, pic_parameter_set_id) cbs_h2645_replace_ps(5, VPS, vps, vps_video_parameter_set_id) cbs_h2645_replace_ps(5, SPS, sps, sps_seq_parameter_set_id) cbs_h2645_replace_ps(5, PPS, pps, pps_pic_parameter_set_id) +cbs_h2645_replace_ps(6, SPS, sps, sps_seq_parameter_set_id) +cbs_h2645_replace_ps(6, PPS, pps, pps_pic_parameter_set_id) static int cbs_h264_read_nal_unit(CodedBitstreamContext *ctx, CodedBitstreamUnit *unit) @@ -920,6 +934,135 @@ static int cbs_h265_read_nal_unit(CodedBitstreamContext *ctx, return 0; } +static int cbs_h266_replace_ph(CodedBitstreamContext *ctx, + CodedBitstreamUnit *unit) +{ + CodedBitstreamH266Context *priv = ctx->priv_data; + int err; + err = ff_cbs_make_unit_refcounted(ctx, unit); + if (err < 0) + return err; + av_buffer_unref(&priv->ph_ref); + av_assert0(unit->content_ref); + priv->ph_ref = av_buffer_ref(unit->content_ref); + if (!priv->ph_ref) + return AVERROR(ENOMEM); + priv->active_ph = priv->ph = (H266RawPH *)priv->ph_ref->data; + return 0; +} + +static int cbs_h266_read_nal_unit(CodedBitstreamContext *ctx, + CodedBitstreamUnit *unit) +{ + GetBitContext gbc; + int err; + + err = init_get_bits8(&gbc, unit->data, unit->data_size); + if (err < 0) + return err; + + err = ff_cbs_alloc_unit_content2(ctx, unit); + if (err < 0) + return err; + + switch (unit->type) { + case VVC_SPS_NUT: + { + H266RawSPS *sps = unit->content; + + err = cbs_h266_read_sps(ctx, &gbc, sps); + if (err < 0) + return err; + + err = cbs_h266_replace_sps(ctx, unit); + if (err < 0) + return err; + } + break; + + case VVC_PPS_NUT: + { + H266RawPPS *pps = unit->content; + + err = cbs_h266_read_pps(ctx, &gbc, pps); + if (err < 0) + return err; + + err = cbs_h266_replace_pps(ctx, unit); + if (err < 0) + return err; + } + break; + + case VVC_PH_NUT: + { + H266RawPH *ph = unit->content; + err = cbs_h266_read_ph(ctx, &gbc, ph); + if (err < 0) + return err; + + err = cbs_h266_replace_ph(ctx, unit); + if (err < 0) + return err; + } + break; + + case VVC_TRAIL_NUT: + case VVC_STSA_NUT: + case VVC_RADL_NUT: + case VVC_RASL_NUT: + case VVC_IDR_W_RADL: + case VVC_IDR_N_LP: + case VVC_CRA_NUT: + case VVC_GDR_NUT: + { + H266RawSlice *slice = unit->content; + int pos, len; + + err = cbs_h266_read_slice_header(ctx, &gbc, &slice->header); + if (err < 0) + return err; + + if (!cbs_h2645_read_more_rbsp_data(&gbc)) + return AVERROR_INVALIDDATA; + + pos = get_bits_count(&gbc); + len = unit->data_size; + + slice->data_size = len - pos / 8; + slice->data_ref = av_buffer_ref(unit->data_ref); + if (!slice->data_ref) + return AVERROR(ENOMEM); + slice->data = unit->data + pos / 8; + slice->data_bit_start = pos % 8; + } + break; + + case VVC_AUD_NUT: + { + err = cbs_h266_read_aud(ctx, &gbc, unit->content); + if (err < 0) + return err; + } + break; + + case VVC_PREFIX_SEI_NUT: + case VVC_SUFFIX_SEI_NUT: + { + err = cbs_h266_read_sei(ctx, &gbc, unit->content, + unit->type == VVC_PREFIX_SEI_NUT); + + if (err < 0) + return err; + } + break; + + default: + return AVERROR(ENOSYS); + } + return 0; +} + static int cbs_h2645_write_slice_data(CodedBitstreamContext *ctx, PutBitContext *pbc, const uint8_t *data, size_t data_size, int data_bit_start) @@ -1207,6 +1350,109 @@ static int cbs_h265_write_nal_unit(CodedBitstreamContext *ctx, return 0; } +static int cbs_h266_write_nal_unit(CodedBitstreamContext *ctx, + CodedBitstreamUnit *unit, + PutBitContext *pbc) +{ + int err; + + switch (unit->type) { + case VVC_SPS_NUT: + { + H266RawSPS *sps = unit->content; + + err = cbs_h266_write_sps(ctx, pbc, sps); + if (err < 0) + return err; + + err = cbs_h266_replace_sps(ctx, unit); + if (err < 0) + return err; + } + break; + + case VVC_PPS_NUT: + { + H266RawPPS *pps = unit->content; + + err = cbs_h266_write_pps(ctx, pbc, pps); + if (err < 0) + return err; + + err = cbs_h266_replace_pps(ctx, unit); + if (err < 0) + return err; + } + break; + + case VVC_PH_NUT: + { + H266RawPH *ph = unit->content; + err = cbs_h266_write_ph(ctx, pbc, ph); + if (err < 0) + return err; + + err = cbs_h266_replace_ph(ctx, unit); + if (err < 0) + return err; + } + break; + + case VVC_TRAIL_NUT: + case VVC_STSA_NUT: + case VVC_RADL_NUT: + case VVC_RASL_NUT: + case VVC_IDR_W_RADL: + case VVC_IDR_N_LP: + case VVC_CRA_NUT: + case VVC_GDR_NUT: + { + H266RawSlice *slice = unit->content; + + err = cbs_h266_write_slice_header(ctx, pbc, &slice->header); + if (err < 0) + return err; + + if (slice->data) { + err = cbs_h2645_write_slice_data(ctx, pbc, slice->data, + slice->data_size, + slice->data_bit_start); + if (err < 0) + return err; + } else { + // No slice data - that was just the header. + } + } + break; + + case VVC_AUD_NUT: + { + err = cbs_h266_write_aud(ctx, pbc, unit->content); + if (err < 0) + return err; + } + break; + + case VVC_PREFIX_SEI_NUT: + case VVC_SUFFIX_SEI_NUT: + { + err = cbs_h266_write_sei(ctx, pbc, unit->content, + unit->type == VVC_PREFIX_SEI_NUT); + + if (err < 0) + return err; + } + break; + + default: + av_log(ctx->log_ctx, AV_LOG_ERROR, "Write unimplemented for " + "NAL unit type %"PRIu32".\n", unit->type); + return AVERROR_PATCHWELCOME; + } + + return 0; +} + static int cbs_h2645_assemble_fragment(CodedBitstreamContext *ctx, CodedBitstreamFragment *frag) { @@ -1248,6 +1494,11 @@ static int cbs_h2645_assemble_fragment(CodedBitstreamContext *ctx, (unit->type == HEVC_NAL_VPS || unit->type == HEVC_NAL_SPS || unit->type == HEVC_NAL_PPS)) || + (ctx->codec->codec_id == AV_CODEC_ID_VVC && + (unit->type == VVC_VPS_NUT || + unit->type == VVC_SPS_NUT || + unit->type == VVC_PPS_NUT || + unit->type == VVC_PREFIX_APS_NUT)) || i == 0 /* (Assume this is the start of an access unit.) */) { // zero_byte data[dp++] = 0; @@ -1362,6 +1613,41 @@ static void cbs_h265_close(CodedBitstreamContext *ctx) av_buffer_unref(&h265->pps_ref[i]); } +static void cbs_h266_flush(CodedBitstreamContext *ctx) +{ + CodedBitstreamH266Context *h266 = ctx->priv_data; + + for (int i = 0; i < FF_ARRAY_ELEMS(h266->vps); i++) { + av_buffer_unref(&h266->vps_ref[i]); + h266->vps[i] = NULL; + } + for (int i = 0; i < FF_ARRAY_ELEMS(h266->sps); i++) { + av_buffer_unref(&h266->sps_ref[i]); + h266->sps[i] = NULL; + } + for (int i = 0; i < FF_ARRAY_ELEMS(h266->pps); i++) { + av_buffer_unref(&h266->pps_ref[i]); + h266->pps[i] = NULL; + } + av_buffer_unref(&h266->ph_ref); + h266->ph = NULL; + + h266->active_vps = NULL; + h266->active_sps = NULL; + h266->active_pps = NULL; + h266->active_ph = NULL; + +} + +static void cbs_h266_close(CodedBitstreamContext *ctx) +{ + CodedBitstreamH266Context *h266 = ctx->priv_data; + + cbs_h266_flush(ctx); + ff_h2645_packet_uninit(&h266->common.read_packet); + + } + static void cbs_h264_free_sei_payload(H264RawSEIPayload *payload) { switch (payload->payload_type) { @@ -1506,6 +1792,77 @@ static const CodedBitstreamUnitTypeDescriptor cbs_h265_unit_types[] = { CBS_UNIT_TYPE_END_OF_LIST }; +static void cbs_h266_free_sei(void *opaque, uint8_t *content) +{ +} + +static const CodedBitstreamUnitTypeDescriptor cbs_h266_unit_types[] = { + CBS_UNIT_TYPE_INTERNAL_REF(VVC_VPS_NUT, H266RawVPS, extension_data.data), + CBS_UNIT_TYPE_INTERNAL_REF(VVC_PPS_NUT, H266RawPPS, extension_data.data), + + CBS_UNIT_TYPE_POD(VVC_PH_NUT, H266RawPH), + CBS_UNIT_TYPE_POD(VVC_AUD_NUT, H266RawAUD), + + { + .nb_unit_types = 1, + .unit_types = { + VVC_SPS_NUT + }, + .content_type = CBS_CONTENT_TYPE_INTERNAL_REFS, + .content_size = sizeof(H266RawSPS), + .nb_ref_offsets = 2, + .ref_offsets = { + offsetof(H266RawSPS, extension_data.data), + offsetof(H266RawSPS, vui.extension_data.data), + }, + }, + + { + // Slices of non-IRAP pictures. + .nb_unit_types = 4, + .unit_types = { + VVC_TRAIL_NUT, + VVC_STSA_NUT, + VVC_RADL_NUT, + VVC_RASL_NUT + }, + + .content_type = CBS_CONTENT_TYPE_INTERNAL_REFS, + .content_size = sizeof(H266RawSlice), + .nb_ref_offsets = 1, + .ref_offsets = { offsetof(H266RawSlice, data) }, + }, + + { + // Slices of IRAP pictures. + .nb_unit_types = 4, + .unit_types = { + VVC_IDR_W_RADL, + VVC_IDR_N_LP, + VVC_CRA_NUT, + VVC_GDR_NUT + }, + + .content_type = CBS_CONTENT_TYPE_INTERNAL_REFS, + .content_size = sizeof(H266RawSlice), + .nb_ref_offsets = 1, + .ref_offsets = { offsetof(H266RawSlice, data) }, + }, + + { + .nb_unit_types = 2, + .unit_types = { + VVC_PREFIX_SEI_NUT, + VVC_SUFFIX_SEI_NUT + }, + .content_type = CBS_CONTENT_TYPE_COMPLEX, + .content_size = sizeof(H266RawSEI), + .content_free = &cbs_h266_free_sei, + }, + + CBS_UNIT_TYPE_END_OF_LIST +}; + const CodedBitstreamType ff_cbs_type_h264 = { .codec_id = AV_CODEC_ID_H264, @@ -1538,6 +1895,22 @@ const CodedBitstreamType ff_cbs_type_h265 = { .close = &cbs_h265_close, }; +const CodedBitstreamType ff_cbs_type_h266 = { + .codec_id = AV_CODEC_ID_VVC, + + .priv_data_size = sizeof(CodedBitstreamH266Context), + + .unit_types = cbs_h266_unit_types, + + .split_fragment = &cbs_h2645_split_fragment, + .read_unit = &cbs_h266_read_nal_unit, + .write_unit = &cbs_h266_write_nal_unit, + .assemble_fragment = &cbs_h2645_assemble_fragment, + + .flush = &cbs_h266_flush, + .close = &cbs_h266_close, +}; + int ff_cbs_h264_add_sei_message(CodedBitstreamFragment *au, H264RawSEIPayload *payload) { diff --git a/libavcodec/cbs_h266.h b/libavcodec/cbs_h266.h new file mode 100644 index 0000000000..90a57c05bd --- /dev/null +++ b/libavcodec/cbs_h266.h @@ -0,0 +1,840 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_CBS_H266_H +#define AVCODEC_CBS_H266_H + +#include +#include + +#include "cbs_h2645.h" +#include "vvc.h" + +enum { + // This limit is arbitrary - it is sufficient for one message of each + // type plus some repeats, and will therefore easily cover all sane + // streams. However, it is possible to make technically-valid streams + // for which it will fail (for example, by including a large number of + // user-data-unregistered messages). + VVC_MAX_SEI_PAYLOADS = 64, +}; + +typedef struct H266RawNALUnitHeader { + uint8_t nuh_layer_id; + uint8_t nal_unit_type; + uint8_t nuh_temporal_id_plus1; +} H266RawNALUnitHeader; + +typedef struct H266GeneralConstraintsInfo { + uint8_t gci_present_flag; + /* general */ + uint8_t gci_intra_only_constraint_flag; + uint8_t gci_all_layers_independent_constraint_flag; + uint8_t gci_one_au_only_constraint_flag; + + /* picture format */ + uint8_t gci_sixteen_minus_max_bitdepth_constraint_idc; + uint8_t gci_three_minus_max_chroma_format_constraint_idc; + + /* NAL unit type related */ + uint8_t gci_no_mixed_nalu_types_in_pic_constraint_flag; + uint8_t gci_no_trail_constraint_flag; + uint8_t gci_no_stsa_constraint_flag; + uint8_t gci_no_rasl_constraint_flag; + uint8_t gci_no_radl_constraint_flag; + uint8_t gci_no_idr_constraint_flag; + uint8_t gci_no_cra_constraint_flag; + uint8_t gci_no_gdr_constraint_flag; + uint8_t gci_no_aps_constraint_flag; + uint8_t gci_no_idr_rpl_constraint_flag; + + /* tile, slice, subpicture partitioning */ + uint8_t gci_one_tile_per_pic_constraint_flag; + uint8_t gci_pic_header_in_slice_header_constraint_flag; + uint8_t gci_one_slice_per_pic_constraint_flag; + uint8_t gci_no_rectangular_slice_constraint_flag; + uint8_t gci_one_slice_per_subpic_constraint_flag; + uint8_t gci_no_subpic_info_constraint_flag; + + /* CTU and block partitioning */ + uint8_t gci_three_minus_max_log2_ctu_size_constraint_idc; + uint8_t gci_no_partition_constraints_override_constraint_flag; + uint8_t gci_no_mtt_constraint_flag; + uint8_t gci_no_qtbtt_dual_tree_intra_constraint_flag; + + /* intra */ + uint8_t gci_no_palette_constraint_flag; + uint8_t gci_no_ibc_constraint_flag; + uint8_t gci_no_isp_constraint_flag; + uint8_t gci_no_mrl_constraint_flag; + uint8_t gci_no_mip_constraint_flag; + uint8_t gci_no_cclm_constraint_flag; + + /* inter */ + uint8_t gci_no_ref_pic_resampling_constraint_flag; + uint8_t gci_no_res_change_in_clvs_constraint_flag;; + uint8_t gci_no_weighted_prediction_constraint_flag; + uint8_t gci_no_ref_wraparound_constraint_flag; + uint8_t gci_no_temporal_mvp_constraint_flag; + uint8_t gci_no_sbtmvp_constraint_flag; + uint8_t gci_no_amvr_constraint_flag; + uint8_t gci_no_bdof_constraint_flag; + uint8_t gci_no_smvd_constraint_flag; + uint8_t gci_no_dmvr_constraint_flag; + uint8_t gci_no_mmvd_constraint_flag; + uint8_t gci_no_affine_motion_constraint_flag; + uint8_t gci_no_prof_constraint_flag; + uint8_t gci_no_bcw_constraint_flag; + uint8_t gci_no_ciip_constraint_flag; + uint8_t gci_no_gpm_constraint_flag; + + /* transform, quantization, residual */ + uint8_t gci_no_luma_transform_size_64_constraint_flag; + uint8_t gci_no_transform_skip_constraint_flag; + uint8_t gci_no_bdpcm_constraint_flag; + uint8_t gci_no_mts_constraint_flag; + uint8_t gci_no_lfnst_constraint_flag; + uint8_t gci_no_joint_cbcr_constraint_flag; + uint8_t gci_no_sbt_constraint_flag; + uint8_t gci_no_act_constraint_flag; + uint8_t gci_no_explicit_scaling_list_constraint_flag; + uint8_t gci_no_dep_quant_constraint_flag; + uint8_t gci_no_sign_data_hiding_constraint_flag; + uint8_t gci_no_cu_qp_delta_constraint_flag; + uint8_t gci_no_chroma_qp_offset_constraint_flag; + + /* loop filter */ + uint8_t gci_no_sao_constraint_flag; + uint8_t gci_no_alf_constraint_flag; + uint8_t gci_no_ccalf_constraint_flag; + uint8_t gci_no_lmcs_constraint_flag; + uint8_t gci_no_ladf_constraint_flag; + uint8_t gci_no_virtual_boundaries_constraint_flag; + uint8_t gci_num_reserved_bits; +} H266GeneralConstraintsInfo; + +typedef struct H266RawProfileTierLevel { + uint8_t general_profile_idc; + uint8_t general_tier_flag; + uint8_t general_level_idc; + uint8_t ptl_frame_only_constraint_flag; + uint8_t ptl_multilayer_enabled_flag; + H266GeneralConstraintsInfo general_constraints_info; + uint8_t ptl_sublayer_level_present_flag[VVC_MAX_SUBLAYERS - 1]; + uint8_t sublayer_level_idc[VVC_MAX_SUBLAYERS - 1]; + uint8_t ptl_num_sub_profiles; + uint32_t general_sub_profile_idc[VVC_MAX_SUB_PROFILES]; +} H266RawProfileTierLevel; + +typedef struct H266RawExtensionData { + uint8_t *data; + AVBufferRef *data_ref; + size_t bit_length; +} H266RawExtensionData; + +typedef struct H266DpbParameters { + uint8_t dpb_max_dec_pic_buffering_minus1[VVC_MAX_SUBLAYERS]; + uint8_t dpb_max_num_reorder_pics[VVC_MAX_SUBLAYERS]; + uint8_t dpb_max_latency_increase_plus1[VVC_MAX_SUBLAYERS]; +} H266DpbParameters; + +typedef struct H266RawVPS { + H266RawNALUnitHeader nal_unit_header; + + uint8_t vps_video_parameter_set_id; + + uint8_t vps_max_layers_minus1; + uint8_t vps_max_sublayers_minus1; + /*TODO add more*/ + H266RawExtensionData extension_data; +} H266RawVPS; +// +typedef struct H266RefPicListStruct { + uint8_t num_ref_entries; + uint8_t ltrp_in_header_flag; + uint8_t inter_layer_ref_pic_flag[VVC_MAX_DPB_SIZE + 13]; + uint8_t st_ref_pic_flag[VVC_MAX_DPB_SIZE + 13]; + uint8_t abs_delta_poc_st[VVC_MAX_DPB_SIZE + 13]; + uint8_t strp_entry_sign_flag[VVC_MAX_DPB_SIZE + 13]; + uint8_t rpls_poc_lsb_lt[VVC_MAX_DPB_SIZE + 13]; + uint8_t ilrp_idx[VVC_MAX_DPB_SIZE + 13]; +} H266RefPicListStruct; + +typedef struct H266RefPicLists { + uint8_t rpl_sps_flag[2]; + uint8_t rpl_idx[2]; + H266RefPicListStruct rpl_ref_list[2]; + uint16_t poc_lsb_lt[2][VVC_MAX_DPB_SIZE + 13]; + uint8_t delta_poc_msb_cycle_present_flag[2][VVC_MAX_DPB_SIZE + 13]; + uint16_t delta_poc_msb_cycle_lt[2][VVC_MAX_DPB_SIZE + 13]; +} H266RefPicLists; + +typedef struct H266RawGeneralTimingHrdParameters { + uint32_t num_units_in_tick; + uint32_t time_scale; + uint8_t general_nal_hrd_params_present_flag; + uint8_t general_vcl_hrd_params_present_flag; + uint8_t general_same_pic_timing_in_all_ols_flag; + uint8_t general_du_hrd_params_present_flag; + uint8_t tick_divisor_minus2; + uint8_t bit_rate_scale; + uint8_t cpb_size_scale; + uint8_t cpb_size_du_scale; + uint8_t hrd_cpb_cnt_minus1; +} H266RawGeneralTimingHrdParameters; + +typedef struct H266RawSubLayerHRDParameters { + uint32_t bit_rate_value_minus1[VVC_MAX_CPB_CNT]; + uint32_t cpb_size_value_minus1[VVC_MAX_CPB_CNT]; + uint32_t cpb_size_du_value_minus1[VVC_MAX_CPB_CNT]; + uint32_t bit_rate_du_value_minus1[VVC_MAX_CPB_CNT]; + uint8_t cbr_flag[VVC_MAX_CPB_CNT]; +} H266RawSubLayerHRDParameters; + +typedef struct H266RawOlsTimingHrdParameters { + uint8_t fixed_pic_rate_general_flag[VVC_MAX_SUBLAYERS]; + uint8_t fixed_pic_rate_within_cvs_flag[VVC_MAX_SUBLAYERS]; + uint16_t elemental_duration_in_tc_minus1[VVC_MAX_SUBLAYERS]; + uint8_t low_delay_hrd_flag[VVC_MAX_SUBLAYERS]; + H266RawSubLayerHRDParameters nal_sub_layer_hrd_parameters[VVC_MAX_SUBLAYERS]; + H266RawSubLayerHRDParameters vcl_sub_layer_hrd_parameters[VVC_MAX_SUBLAYERS]; +} H266RawOlsTimingHrdParameters; + +typedef struct H266RawVUI { + uint8_t vui_progressive_source_flag; + uint8_t vui_interlaced_source_flag; + uint8_t vui_non_packed_constraint_flag; + uint8_t vui_non_projected_constraint_flag; + + uint8_t vui_aspect_ratio_info_present_flag; + uint8_t vui_aspect_ratio_constant_flag; + uint8_t vui_aspect_ratio_idc; + + uint16_t vui_sar_width; + uint16_t vui_sar_height;; + + uint8_t vui_overscan_info_present_flag; + uint8_t vui_overscan_appropriate_flag; + + uint8_t vui_colour_description_present_flag; + uint8_t vui_colour_primaries; + + uint8_t vui_transfer_characteristics; + uint8_t vui_matrix_coeffs; + uint8_t vui_full_range_flag; + + uint8_t vui_chroma_loc_info_present_flag; + uint8_t vui_chroma_sample_loc_type_frame; + uint8_t vui_chroma_sample_loc_type_top_field; + uint8_t vui_chroma_sample_loc_type_bottom_field; + H266RawExtensionData extension_data; +} H266RawVUI; + +typedef struct H266RawSPS { + H266RawNALUnitHeader nal_unit_header; + + uint8_t sps_seq_parameter_set_id; + uint8_t sps_video_parameter_set_id; + uint8_t sps_max_sublayers_minus1; + uint8_t sps_chroma_format_idc; + uint8_t sps_log2_ctu_size_minus5; + uint8_t sps_ptl_dpb_hrd_params_present_flag; + H266RawProfileTierLevel profile_tier_level; + uint8_t sps_gdr_enabled_flag; + uint8_t sps_ref_pic_resampling_enabled_flag; + uint8_t sps_res_change_in_clvs_allowed_flag; + + uint16_t sps_pic_width_max_in_luma_samples; + uint16_t sps_pic_height_max_in_luma_samples; + + uint8_t sps_conformance_window_flag; + uint16_t sps_conf_win_left_offset; + uint16_t sps_conf_win_right_offset; + uint16_t sps_conf_win_top_offset; + uint16_t sps_conf_win_bottom_offset; + + uint8_t sps_subpic_info_present_flag; + uint16_t sps_num_subpics_minus1; + uint8_t sps_independent_subpics_flag; + uint8_t sps_subpic_same_size_flag; + uint16_t sps_subpic_ctu_top_left_x[VVC_MAX_SLICES]; + uint16_t sps_subpic_ctu_top_left_y[VVC_MAX_SLICES]; + uint16_t sps_subpic_width_minus1[VVC_MAX_SLICES]; + uint16_t sps_subpic_height_minus1[VVC_MAX_SLICES]; + uint8_t sps_subpic_treated_as_pic_flag[VVC_MAX_SLICES]; + uint8_t sps_loop_filter_across_subpic_enabled_flag[VVC_MAX_SLICES]; + uint8_t sps_subpic_id_len_minus1; + uint8_t sps_subpic_id_mapping_explicitly_signalled_flag; + uint8_t sps_subpic_id_mapping_present_flag; + uint32_t sps_subpic_id[VVC_MAX_SLICES]; + + + uint8_t sps_bitdepth_minus8; + uint8_t sps_entropy_coding_sync_enabled_flag; + uint8_t sps_entry_point_offsets_present_flag; + + uint8_t sps_log2_max_pic_order_cnt_lsb_minus4; + uint8_t sps_poc_msb_cycle_flag; + uint8_t sps_poc_msb_cycle_len_minus1; + + uint8_t sps_num_extra_ph_bytes; + uint8_t sps_extra_ph_bit_present_flag[16]; + + uint8_t sps_num_extra_sh_bytes; + uint8_t sps_extra_sh_bit_present_flag[16]; + + uint8_t sps_sublayer_dpb_params_flag; + H266DpbParameters sps_dpb_params; + + uint8_t sps_log2_min_luma_coding_block_size_minus2; + uint8_t sps_partition_constraints_override_enabled_flag; + uint8_t sps_log2_diff_min_qt_min_cb_intra_slice_luma; + uint8_t sps_max_mtt_hierarchy_depth_intra_slice_luma; + uint8_t sps_log2_diff_max_bt_min_qt_intra_slice_luma; + uint8_t sps_log2_diff_max_tt_min_qt_intra_slice_luma; + + uint8_t sps_qtbtt_dual_tree_intra_flag; + uint8_t sps_log2_diff_min_qt_min_cb_intra_slice_chroma; + uint8_t sps_max_mtt_hierarchy_depth_intra_slice_chroma; + uint8_t sps_log2_diff_max_bt_min_qt_intra_slice_chroma; + uint8_t sps_log2_diff_max_tt_min_qt_intra_slice_chroma; + + uint8_t sps_log2_diff_min_qt_min_cb_inter_slice; + uint8_t sps_max_mtt_hierarchy_depth_inter_slice; + uint8_t sps_log2_diff_max_bt_min_qt_inter_slice; + uint8_t sps_log2_diff_max_tt_min_qt_inter_slice; + + uint8_t sps_max_luma_transform_size_64_flag; + + uint8_t sps_transform_skip_enabled_flag; + uint8_t sps_log2_transform_skip_max_size_minus2; + uint8_t sps_bdpcm_enabled_flag; + + uint8_t sps_mts_enabled_flag; + uint8_t sps_explicit_mts_intra_enabled_flag; + uint8_t sps_explicit_mts_inter_enabled_flag; + + uint8_t sps_lfnst_enabled_flag; + + uint8_t sps_joint_cbcr_enabled_flag; + uint8_t sps_same_qp_table_for_chroma_flag; + + int8_t sps_qp_table_start_minus26[VVC_MAX_PLANES]; + uint8_t sps_num_points_in_qp_table_minus1[VVC_MAX_PLANES]; + uint8_t sps_delta_qp_in_val_minus1[VVC_MAX_PLANES][VVC_MAX_POINTS_IN_QP_TABLE]; + uint8_t sps_delta_qp_diff_val[VVC_MAX_PLANES][VVC_MAX_POINTS_IN_QP_TABLE]; + + uint8_t sps_sao_enabled_flag; + uint8_t sps_alf_enabled_flag; + uint8_t sps_ccalf_enabled_flag; + uint8_t sps_lmcs_enabled_flag; + uint8_t sps_weighted_pred_flag; + uint8_t sps_weighted_bipred_flag; + uint8_t sps_long_term_ref_pics_flag; + uint8_t sps_inter_layer_prediction_enabled_flag; + uint8_t sps_idr_rpl_present_flag; + uint8_t sps_rpl1_same_as_rpl0_flag; + + uint8_t sps_num_ref_pic_lists[2]; + H266RefPicListStruct sps_ref_pic_list_struct[2][VVC_MAX_REF_PIC_LISTS]; + + uint8_t sps_ref_wraparound_enabled_flag; + uint8_t sps_temporal_mvp_enabled_flag; + uint8_t sps_sbtmvp_enabled_flag; + uint8_t sps_amvr_enabled_flag; + uint8_t sps_bdof_enabled_flag; + uint8_t sps_bdof_control_present_in_ph_flag; + uint8_t sps_smvd_enabled_flag; + uint8_t sps_dmvr_enabled_flag; + uint8_t sps_dmvr_control_present_in_ph_flag; + uint8_t sps_mmvd_enabled_flag; + uint8_t sps_mmvd_fullpel_only_enabled_flag; + uint8_t sps_six_minus_max_num_merge_cand; + uint8_t sps_sbt_enabled_flag; + uint8_t sps_affine_enabled_flag; + uint8_t sps_five_minus_max_num_subblock_merge_cand; + uint8_t sps_6param_affine_enabled_flag; + uint8_t sps_affine_amvr_enabled_flag; + uint8_t sps_affine_prof_enabled_flag; + uint8_t sps_prof_control_present_in_ph_flag; + uint8_t sps_bcw_enabled_flag; + uint8_t sps_ciip_enabled_flag; + uint8_t sps_gpm_enabled_flag; + uint8_t sps_max_num_merge_cand_minus_max_num_gpm_cand; + uint8_t sps_log2_parallel_merge_level_minus2; + uint8_t sps_isp_enabled_flag; + uint8_t sps_mrl_enabled_flag; + uint8_t sps_mip_enabled_flag; + uint8_t sps_cclm_enabled_flag; + uint8_t sps_chroma_horizontal_collocated_flag; + uint8_t sps_chroma_vertical_collocated_flag; + uint8_t sps_palette_enabled_flag; + uint8_t sps_act_enabled_flag; + uint8_t sps_min_qp_prime_ts; + uint8_t sps_ibc_enabled_flag; + uint8_t sps_six_minus_max_num_ibc_merge_cand; + uint8_t sps_ladf_enabled_flag; + uint8_t sps_num_ladf_intervals_minus2; + int8_t sps_ladf_lowest_interval_qp_offset; + int8_t sps_ladf_qp_offset[4]; + uint16_t sps_ladf_delta_threshold_minus1[4]; + + uint8_t sps_explicit_scaling_list_enabled_flag; + uint8_t sps_scaling_matrix_for_lfnst_disabled_flag; + uint8_t sps_scaling_matrix_for_alternative_colour_space_disabled_flag; + uint8_t sps_scaling_matrix_designated_colour_space_flag; + uint8_t sps_dep_quant_enabled_flag; + uint8_t sps_sign_data_hiding_enabled_flag; + + uint8_t sps_virtual_boundaries_enabled_flag; + uint8_t sps_virtual_boundaries_present_flag; + uint8_t sps_num_ver_virtual_boundaries; + uint16_t sps_virtual_boundary_pos_x_minus1[3]; + uint8_t sps_num_hor_virtual_boundaries; + uint16_t sps_virtual_boundary_pos_y_minus1[3]; + + uint8_t sps_timing_hrd_params_present_flag; + uint8_t sps_sublayer_cpb_params_present_flag; + H266RawGeneralTimingHrdParameters sps_general_timing_hrd_parameters; + H266RawOlsTimingHrdParameters sps_ols_timing_hrd_parameters; + + uint8_t sps_field_seq_flag; + uint8_t sps_vui_parameters_present_flag; + uint16_t sps_vui_payload_size_minus1; + H266RawVUI vui; + + uint8_t sps_extension_flag; + + H266RawExtensionData extension_data; + +} H266RawSPS; + +typedef struct H266RawPPS { + H266RawNALUnitHeader nal_unit_header; + + uint8_t pps_pic_parameter_set_id; + uint8_t pps_seq_parameter_set_id; + uint8_t pps_mixed_nalu_types_in_pic_flag; + uint16_t pps_pic_width_in_luma_samples; + uint16_t pps_pic_height_in_luma_samples; + + uint8_t pps_conformance_window_flag; + uint16_t pps_conf_win_left_offset; + uint16_t pps_conf_win_right_offset; + uint16_t pps_conf_win_top_offset; + uint16_t pps_conf_win_bottom_offset; + + uint8_t pps_scaling_window_explicit_signalling_flag; + int16_t pps_scaling_win_left_offset; + int16_t pps_scaling_win_right_offset; + int16_t pps_scaling_win_top_offset; + int16_t pps_scaling_win_bottom_offset; + + uint8_t pps_output_flag_present_flag; + uint8_t pps_no_pic_partition_flag; + + uint8_t pps_subpic_id_mapping_present_flag; + uint16_t pps_num_subpics_minus1; + uint8_t pps_subpic_id_len_minus1; + uint16_t pps_subpic_id[VVC_MAX_SLICES]; + + uint8_t pps_log2_ctu_size_minus5; + uint8_t pps_num_exp_tile_columns_minus1; + uint8_t pps_num_exp_tile_rows_minus1; + uint16_t pps_tile_column_width_minus1[VVC_MAX_TILE_COLUMNS]; + uint16_t pps_tile_row_height_minus1[VVC_MAX_TILE_ROWS]; + + uint8_t pps_loop_filter_across_tiles_enabled_flag; + uint8_t pps_rect_slice_flag; + uint8_t pps_single_slice_per_subpic_flag; + + uint16_t pps_num_slices_in_pic_minus1; + uint8_t pps_tile_idx_delta_present_flag; + uint16_t pps_slice_width_in_tiles_minus1[VVC_MAX_SLICES]; + uint16_t pps_slice_height_in_tiles_minus1[VVC_MAX_SLICES]; + uint16_t pps_num_exp_slices_in_tile[VVC_MAX_SLICES]; + uint16_t pps_exp_slice_height_in_ctus_minus1[VVC_MAX_SLICES][VVC_MAX_TILE_ROWS]; + int16_t pps_tile_idx_delta_val[VVC_MAX_SLICES]; + + uint8_t pps_loop_filter_across_slices_enabled_flag; + uint8_t pps_cabac_init_present_flag; + uint8_t pps_num_ref_idx_default_active_minus1[2]; + uint8_t pps_rpl1_idx_present_flag; + uint8_t pps_weighted_pred_flag; + uint8_t pps_weighted_bipred_flag; + uint8_t pps_ref_wraparound_enabled_flag; + uint8_t pps_pic_width_minus_wraparound_offset; + int8_t pps_init_qp_minus26; + uint8_t pps_cu_qp_delta_enabled_flag; + uint8_t pps_chroma_tool_offsets_present_flag; + int8_t pps_cb_qp_offset; + int8_t pps_cr_qp_offset; + uint8_t pps_joint_cbcr_qp_offset_present_flag; + int8_t pps_joint_cbcr_qp_offset_value; + uint8_t pps_slice_chroma_qp_offsets_present_flag; + uint8_t pps_cu_chroma_qp_offset_list_enabled_flag; + uint8_t pps_chroma_qp_offset_list_len_minus1; + uint8_t pps_cb_qp_offset_list[6]; + uint8_t pps_cr_qp_offset_list[6]; + uint8_t pps_joint_cbcr_qp_offset_list[6]; + uint8_t pps_deblocking_filter_control_present_flag; + uint8_t pps_deblocking_filter_override_enabled_flag; + uint8_t pps_deblocking_filter_disabled_flag; + uint8_t pps_dbf_info_in_ph_flag; + + int8_t pps_luma_beta_offset_div2; + int8_t pps_luma_tc_offset_div2; + int8_t pps_cb_beta_offset_div2; + int8_t pps_cb_tc_offset_div2; + int8_t pps_cr_beta_offset_div2; + int8_t pps_cr_tc_offset_div2; + + uint8_t pps_rpl_info_in_ph_flag; + uint8_t pps_sao_info_in_ph_flag; + uint8_t pps_alf_info_in_ph_flag; + uint8_t pps_wp_info_in_ph_flag; + uint8_t pps_qp_delta_info_in_ph_flag; + + uint8_t pps_picture_header_extension_present_flag; + uint8_t pps_slice_header_extension_present_flag; + uint8_t pps_extension_flag; + H266RawExtensionData extension_data; + + //calculated value; + uint16_t num_tile_columns; + uint16_t num_tile_rows; + uint16_t num_tiles_in_pic; + uint16_t slice_height_in_ctus[VVC_MAX_SLICES]; + uint16_t num_slices_in_subpic[VVC_MAX_SLICES]; +} H266RawPPS; + +typedef struct H266RawAUD { + H266RawNALUnitHeader nal_unit_header; + uint8_t aud_irap_or_gdr_flag; + uint8_t aud_pic_type; +} H266RawAUD; + +typedef struct H266RawPredWeightTable { + uint8_t luma_log2_weight_denom; + int8_t delta_chroma_log2_weight_denom; + + uint8_t num_l0_weights; + uint8_t luma_weight_l0_flag[15]; + uint8_t chroma_weight_l0_flag[15]; + int8_t delta_luma_weight_l0[15]; + int8_t luma_offset_l0[15]; + int8_t delta_chroma_weight_l0[15][2]; + int16_t delta_chroma_offset_l0[15][2]; + + uint8_t num_l1_weights; + uint8_t luma_weight_l1_flag[15]; + uint8_t chroma_weight_l1_flag[15]; + int8_t delta_luma_weight_l1[15]; + int8_t luma_offset_l1[15]; + int8_t delta_chroma_weight_l1[15][2]; + int16_t delta_chroma_offset_l1[15][2]; +} H266RawPredWeightTable; + +typedef struct H266RawPH { + H266RawNALUnitHeader nal_unit_header; + uint8_t ph_gdr_or_irap_pic_flag; + uint8_t ph_non_ref_pic_flag; + uint8_t ph_gdr_pic_flag; + uint8_t ph_inter_slice_allowed_flag; + uint8_t ph_intra_slice_allowed_flag; + uint8_t ph_pic_parameter_set_id; + uint16_t ph_pic_order_cnt_lsb; + uint8_t ph_recovery_poc_cnt; + uint8_t ph_extra_bit[16]; + uint8_t ph_poc_msb_cycle_present_flag; + uint8_t ph_poc_msb_cycle_val; + + uint8_t ph_alf_enabled_flag; + uint8_t ph_num_alf_aps_ids_luma; + uint8_t ph_alf_aps_id_luma[8]; + uint8_t ph_alf_cb_enabled_flag; + uint8_t ph_alf_cr_enabled_flag; + uint8_t ph_alf_aps_id_chroma; + uint8_t ph_alf_cc_cb_enabled_flag; + uint8_t ph_alf_cc_cb_aps_id; + uint8_t ph_alf_cc_cr_enabled_flag; + uint8_t ph_alf_cc_cr_aps_id; + + uint8_t ph_lmcs_enabled_flag; + uint8_t ph_lmcs_aps_id; + uint8_t ph_chroma_residual_scale_flag; + uint8_t ph_explicit_scaling_list_enabled_flag; + uint8_t ph_scaling_list_aps_id; + + uint8_t ph_virtual_boundaries_present_flag; + uint8_t ph_num_ver_virtual_boundaries; + uint16_t ph_virtual_boundary_pos_x_minus1[3]; + uint8_t ph_num_hor_virtual_boundaries; + uint16_t ph_virtual_boundary_pos_y_minus1[3]; + + uint8_t ph_pic_output_flag; + H266RefPicLists ph_ref_pic_lists; + + uint8_t ph_partition_constraints_override_flag; + + uint8_t ph_log2_diff_min_qt_min_cb_intra_slice_luma; + uint8_t ph_max_mtt_hierarchy_depth_intra_slice_luma; + uint8_t ph_log2_diff_max_bt_min_qt_intra_slice_luma; + uint8_t ph_log2_diff_max_tt_min_qt_intra_slice_luma; + uint8_t ph_log2_diff_min_qt_min_cb_intra_slice_chroma; + + uint8_t ph_max_mtt_hierarchy_depth_intra_slice_chroma; + uint8_t ph_log2_diff_max_bt_min_qt_intra_slice_chroma; + uint8_t ph_log2_diff_max_tt_min_qt_intra_slice_chroma; + + uint8_t ph_cu_qp_delta_subdiv_intra_slice; + uint8_t ph_cu_chroma_qp_offset_subdiv_intra_slice; + + uint8_t ph_log2_diff_min_qt_min_cb_inter_slice; + uint8_t ph_max_mtt_hierarchy_depth_inter_slice; + uint8_t ph_log2_diff_max_bt_min_qt_inter_slice; + uint8_t ph_log2_diff_max_tt_min_qt_inter_slice; + uint8_t ph_cu_qp_delta_subdiv_inter_slice; + uint8_t ph_cu_chroma_qp_offset_subdiv_inter_slice; + + uint8_t ph_temporal_mvp_enabled_flag; + uint8_t ph_collocated_from_l0_flag; + uint8_t ph_collocated_ref_idx; + uint8_t ph_mmvd_fullpel_only_flag; + uint8_t ph_mvd_l1_zero_flag; + uint8_t ph_bdof_disabled_flag; + uint8_t ph_dmvr_disabled_flag; + uint8_t ph_prof_disabled_flag; + + H266RawPredWeightTable ph_pred_weight_table; + + int8_t ph_qp_delta; + uint8_t ph_joint_cbcr_sign_flag; + uint8_t ph_sao_luma_enabled_flag; + uint8_t ph_sao_chroma_enabled_flag; + + uint8_t ph_deblocking_params_present_flag; + uint8_t ph_deblocking_filter_disabled_flag; + int8_t ph_luma_beta_offset_div2; + int8_t ph_luma_tc_offset_div2; + int8_t ph_cb_beta_offset_div2; + int8_t ph_cb_tc_offset_div2; + int8_t ph_cr_beta_offset_div2; + int8_t ph_cr_tc_offset_div2; + + uint8_t ph_extension_length; + uint8_t ph_extension_data_byte[256]; +} H266RawPH; + +typedef struct H266RawSliceHeader { + H266RawNALUnitHeader nal_unit_header; + uint8_t sh_picture_header_in_slice_header_flag; + H266RawPH sh_picture_header; + + uint16_t sh_subpic_id; + uint16_t sh_slice_address; + uint8_t sh_extra_bit[16]; + uint8_t sh_num_tiles_in_slice_minus1; + uint8_t sh_slice_type; + uint8_t sh_no_output_of_prior_pics_flag; + + uint8_t sh_alf_enabled_flag; + uint8_t sh_num_alf_aps_ids_luma; + uint8_t sh_alf_aps_id_luma[8]; + uint8_t sh_alf_cb_enabled_flag; + uint8_t sh_alf_cr_enabled_flag; + uint8_t sh_alf_aps_id_chroma; + uint8_t sh_alf_cc_cb_enabled_flag; + uint8_t sh_alf_cc_cb_aps_id; + uint8_t sh_alf_cc_cr_enabled_flag; + uint8_t sh_alf_cc_cr_aps_id; + + uint8_t sh_lmcs_used_flag; + uint8_t sh_explicit_scaling_list_used_flag; + + H266RefPicLists sh_ref_pic_lists; + + uint8_t sh_num_ref_idx_active_override_flag; + uint8_t sh_num_ref_idx_active_minus1[2]; + uint8_t sh_cabac_init_flag; + uint8_t sh_collocated_from_l0_flag; + uint8_t sh_collocated_ref_idx; + + H266RawPredWeightTable sh_pred_weight_table; + + int8_t sh_qp_delta; + int8_t sh_cb_qp_offset; + int8_t sh_cr_qp_offset; + int8_t sh_joint_cbcr_qp_offset; + uint8_t sh_cu_chroma_qp_offset_enabled_flag; + + uint8_t sh_sao_luma_used_flag; + uint8_t sh_sao_chroma_used_flag; + + uint8_t sh_deblocking_params_present_flag; + uint8_t sh_deblocking_filter_disabled_flag; + int8_t sh_luma_beta_offset_div2; + int8_t sh_luma_tc_offset_div2; + int8_t sh_cb_beta_offset_div2; + int8_t sh_cb_tc_offset_div2; + int8_t sh_cr_beta_offset_div2; + int8_t sh_cr_tc_offset_div2; + uint8_t sh_dep_quant_used_flag; + + uint8_t sh_sign_data_hiding_used_flag; + uint8_t sh_ts_residual_coding_disabled_flag; + uint16_t sh_slice_header_extension_length; + uint8_t sh_slice_header_extension_data_byte[256]; + + uint8_t sh_entry_offset_len_minus1; + uint32_t sh_entry_point_offset_minus1[VVC_MAX_ENTRY_POINTS]; + +} H266RawSliceHeader; + + +typedef struct H266RawSlice { + H266RawSliceHeader header; + + uint8_t *data; + AVBufferRef *data_ref; + size_t data_size; + int data_bit_start; +} H266RawSlice; + + +typedef struct H266RawSEIBufferingPeriod { + uint8_t bp_seq_parameter_set_id; +} H266RawSEIBufferingPeriod; + +typedef struct H266RawSEIPicTiming { +} H266RawSEIPicTiming; + +typedef struct H266RawSEIPanScanRect { +} H266RawSEIPanScanRect; + +typedef struct H266RawSEIUserDataRegistered { + +} H266RawSEIUserDataRegistered; + +typedef struct H266RawSEIUserDataUnregistered { +} H266RawSEIUserDataUnregistered; + +typedef struct H266RawSEIRecoveryPoint { +} H266RawSEIRecoveryPoint; + +typedef struct H266RawSEIDisplayOrientation { +} H266RawSEIDisplayOrientation; + +typedef struct H266RawSEIActiveParameterSets { +} H266RawSEIActiveParameterSets; + +typedef struct H266RawSEIDecodedPictureHash { + uint8_t dph_sei_hash_type; + uint8_t dph_sei_single_component_flag; + uint8_t dph_sei_picture_md5[3][16]; + uint16_t dph_sei_picture_crc[3]; + uint32_t dph_sei_picture_checksum[3]; +} H266RawSEIDecodedPictureHash; + +typedef struct H266RawSEITimeCode { +} H266RawSEITimeCode; + +typedef struct H266RawSEIMasteringDisplayColourVolume { +} H266RawSEIMasteringDisplayColourVolume; + +typedef struct H266RawSEIContentLightLevelInfo { + uint16_t max_content_light_level; + uint16_t max_pic_average_light_level; +} H266RawSEIContentLightLevelInfo; + +typedef struct H266RawSEIAlternativeTransferCharacteristics { + uint8_t preferred_transfer_characteristics; +} H266RawSEIAlternativeTransferCharacteristics; + +typedef struct H266RawSEIAlphaChannelInfo { + uint8_t alpha_channel_cancel_flag; + uint8_t alpha_channel_use_idc; + uint8_t alpha_channel_bit_depth_minus8; + uint16_t alpha_transparent_value; + uint16_t alpha_opaque_value; + uint8_t alpha_channel_incr_flag; + uint8_t alpha_channel_clip_flag; + uint8_t alpha_channel_clip_type_flag; +} H266RawSEIAlphaChannelInfo; + +typedef struct H266RawSEIPayload { + uint32_t payload_type; + uint32_t payload_size; + union { + H266RawSEIBufferingPeriod buffering_period; + H266RawSEIPicTiming pic_timing; + H266RawSEIPanScanRect pan_scan_rect; + H266RawSEIUserDataRegistered user_data_registered; + H266RawSEIUserDataUnregistered user_data_unregistered; + H266RawSEIRecoveryPoint recovery_point; + H266RawSEIDisplayOrientation display_orientation; + H266RawSEIActiveParameterSets active_parameter_sets; + H266RawSEIDecodedPictureHash decoded_picture_hash; + H266RawSEITimeCode time_code; + H266RawSEIMasteringDisplayColourVolume mastering_display; + H266RawSEIContentLightLevelInfo content_light_level; + H266RawSEIAlternativeTransferCharacteristics + alternative_transfer_characteristics; + H266RawSEIAlphaChannelInfo alpha_channel_info; + struct { + uint8_t *data; + AVBufferRef *data_ref; + size_t data_length; + } other; + } payload; + H266RawExtensionData extension_data; +} H266RawSEIPayload; + +typedef struct H266RawSEI { + H266RawNALUnitHeader nal_unit_header; + + H266RawSEIPayload payload[VVC_MAX_SEI_PAYLOADS]; + uint8_t payload_count; +} H266RawSEI; + +typedef struct CodedBitstreamH266Context { + // Reader/writer context in common with the H.264 implementation. + CodedBitstreamH2645Context common; + + // All currently available parameter sets. These are updated when + // any parameter set NAL unit is read/written with this context. + AVBufferRef *vps_ref[VVC_MAX_VPS_COUNT]; + AVBufferRef *sps_ref[VVC_MAX_SPS_COUNT]; + AVBufferRef *pps_ref[VVC_MAX_PPS_COUNT]; + AVBufferRef *ph_ref; + H266RawVPS *vps[VVC_MAX_VPS_COUNT]; + H266RawSPS *sps[VVC_MAX_SPS_COUNT]; + H266RawPPS *pps[VVC_MAX_PPS_COUNT]; + H266RawPH *ph; + + // The currently active parameter sets. These are updated when any + // NAL unit refers to the relevant parameter set. These pointers + // must also be present in the arrays above. + const H266RawVPS *active_vps; + const H266RawSPS *active_sps; + const H266RawPPS *active_pps; + const H266RawPH *active_ph; +} CodedBitstreamH266Context; + + +#endif /* AVCODEC_CBS_H266_H */ diff --git a/libavcodec/cbs_h266_syntax_template.c b/libavcodec/cbs_h266_syntax_template.c new file mode 100644 index 0000000000..6a6defc8a5 --- /dev/null +++ b/libavcodec/cbs_h266_syntax_template.c @@ -0,0 +1,2761 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef H266_CEIL +#define H266_CEIL(v, a) (((v) + (a) - 1) / (a)) +#endif + +#ifndef H266_HAS_MULTI_SLICES_IN_TILE +#define H266_HAS_MULTI_SLICES_IN_TILE(pps, tile_y, slice) \ + (pps->pps_slice_width_in_tiles_minus1[slice] == 0 && \ + pps->pps_slice_height_in_tiles_minus1[slice] == 0 && \ + pps->pps_tile_row_height_minus1[tile_y] > 0) +#endif + +static int FUNC(rbsp_trailing_bits)(CodedBitstreamContext *ctx, RWContext *rw) +{ + int err; + + fixed(1, rbsp_stop_one_bit, 1); + while (byte_alignment(rw) != 0) + fixed(1, rbsp_alignment_zero_bit, 0); + return 0; +} + +static int FUNC(nal_unit_header)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawNALUnitHeader *current, + int expected_nal_unit_type) +{ + int err; + + fixed(1, forbidden_zero_bit, 0); + fixed(1, nuh_reserved_zero_bit, 0); + + u(6, nuh_layer_id, 0, 55); + + if (expected_nal_unit_type >= 0) + u(5, nal_unit_type, expected_nal_unit_type, + expected_nal_unit_type); + else + ub(5, nal_unit_type); + + u(3, nuh_temporal_id_plus1, 1, 7); + return 0; +} + +static int FUNC(byte_alignment)(CodedBitstreamContext *ctx, RWContext *rw) +{ + int err; + + fixed(1, alignment_bit_equal_to_one, 1); + while (byte_alignment(rw) != 0) + fixed(1, alignment_bit_equal_to_zero, 0); + return 0; +} + +static int FUNC(general_constraints_info)(CodedBitstreamContext *ctx, + RWContext *rw, + H266GeneralConstraintsInfo *current) +{ + int err, i; + + flag(gci_present_flag); + if (current->gci_present_flag) { + /* general */ + flag(gci_intra_only_constraint_flag); + flag(gci_all_layers_independent_constraint_flag); + flag(gci_one_au_only_constraint_flag); + + /* picture format */ + u(4, gci_sixteen_minus_max_bitdepth_constraint_idc, 0, 8); + ub(2, gci_three_minus_max_chroma_format_constraint_idc); + + /* NAL unit type related */ + flag(gci_no_mixed_nalu_types_in_pic_constraint_flag); + flag(gci_no_trail_constraint_flag); + flag(gci_no_stsa_constraint_flag); + flag(gci_no_rasl_constraint_flag); + flag(gci_no_radl_constraint_flag); + flag(gci_no_idr_constraint_flag); + flag(gci_no_cra_constraint_flag); + flag(gci_no_gdr_constraint_flag); + flag(gci_no_aps_constraint_flag); + flag(gci_no_idr_rpl_constraint_flag); + + /* tile, slice, subpicture partitioning */ + flag(gci_one_tile_per_pic_constraint_flag); + flag(gci_pic_header_in_slice_header_constraint_flag); + flag(gci_one_slice_per_pic_constraint_flag); + flag(gci_no_rectangular_slice_constraint_flag); + flag(gci_one_slice_per_subpic_constraint_flag); + flag(gci_no_subpic_info_constraint_flag); + + /* CTU and block partitioning */ + ub(2, gci_three_minus_max_log2_ctu_size_constraint_idc); + flag(gci_no_partition_constraints_override_constraint_flag); + flag(gci_no_mtt_constraint_flag); + flag(gci_no_qtbtt_dual_tree_intra_constraint_flag); + + /* intra */ + flag(gci_no_palette_constraint_flag); + flag(gci_no_ibc_constraint_flag); + flag(gci_no_isp_constraint_flag); + flag(gci_no_mrl_constraint_flag); + flag(gci_no_mip_constraint_flag); + flag(gci_no_cclm_constraint_flag); + + /* inter */ + flag(gci_no_ref_pic_resampling_constraint_flag); + flag(gci_no_res_change_in_clvs_constraint_flag); + flag(gci_no_weighted_prediction_constraint_flag); + flag(gci_no_ref_wraparound_constraint_flag); + flag(gci_no_temporal_mvp_constraint_flag); + flag(gci_no_sbtmvp_constraint_flag); + flag(gci_no_amvr_constraint_flag); + flag(gci_no_bdof_constraint_flag); + flag(gci_no_smvd_constraint_flag); + flag(gci_no_dmvr_constraint_flag); + flag(gci_no_mmvd_constraint_flag); + flag(gci_no_affine_motion_constraint_flag); + flag(gci_no_prof_constraint_flag); + flag(gci_no_bcw_constraint_flag); + flag(gci_no_ciip_constraint_flag); + flag(gci_no_gpm_constraint_flag); + + /* transform, quantization, residual */ + flag(gci_no_luma_transform_size_64_constraint_flag); + flag(gci_no_transform_skip_constraint_flag); + flag(gci_no_bdpcm_constraint_flag); + flag(gci_no_mts_constraint_flag); + flag(gci_no_lfnst_constraint_flag); + flag(gci_no_joint_cbcr_constraint_flag); + flag(gci_no_sbt_constraint_flag); + flag(gci_no_act_constraint_flag); + flag(gci_no_explicit_scaling_list_constraint_flag); + flag(gci_no_dep_quant_constraint_flag); + flag(gci_no_sign_data_hiding_constraint_flag); + flag(gci_no_cu_qp_delta_constraint_flag); + flag(gci_no_chroma_qp_offset_constraint_flag); + + /* loop filter */ + flag(gci_no_sao_constraint_flag); + flag(gci_no_alf_constraint_flag); + flag(gci_no_ccalf_constraint_flag); + flag(gci_no_lmcs_constraint_flag); + flag(gci_no_ladf_constraint_flag); + flag(gci_no_virtual_boundaries_constraint_flag); + ub(8, gci_num_reserved_bits); + for (i = 0; i < current->gci_num_reserved_bits; i++) { + fixed(1, gci_reserved_zero_bit, 1); + } + } + while (byte_alignment(rw) != 0) + fixed(1, gci_alignment_zero_bit, 0); + return 0; +} + +static int FUNC(profile_tier_level)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawProfileTierLevel *current, + int profile_tier_present_flag, + int max_num_sub_layers_minus1) +{ + int err, i; + + if (profile_tier_present_flag) { + ub(7, general_profile_idc); + flag(general_tier_flag); + } + ub(8, general_level_idc); + flag(ptl_frame_only_constraint_flag); + flag(ptl_multilayer_enabled_flag); + if (profile_tier_present_flag) { + CHECK(FUNC(general_constraints_info)(ctx, rw, + ¤t->general_constraints_info)); + } + for (i = max_num_sub_layers_minus1 - 1; i >= 0; i--) + flags(ptl_sublayer_level_present_flag[i], 1, i); + while (byte_alignment(rw) != 0) + fixed(1, ptl_reserved_zero_bit, 0); + for (i = max_num_sub_layers_minus1 - 1; i >= 0; i--) + if (current->ptl_sublayer_level_present_flag[i]) + ubs(8, sublayer_level_idc[i], 1, i); + if (profile_tier_present_flag) { + ub(8, ptl_num_sub_profiles); + for (i = 0; i < current->ptl_num_sub_profiles; i++) + ubs(32, general_sub_profile_idc[i], 1, i); + } + return 0; +} + +static int FUNC(vui_parameters)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawVUI *current) +{ + int err; + + flag(vui_progressive_source_flag); + flag(vui_interlaced_source_flag); + flag(vui_non_packed_constraint_flag); + flag(vui_non_projected_constraint_flag); + flag(vui_aspect_ratio_info_present_flag); + if (current->vui_aspect_ratio_info_present_flag) { + flag(vui_aspect_ratio_constant_flag); + ub(8, vui_aspect_ratio_idc); + if (current->vui_aspect_ratio_idc == 255) { + ub(16, vui_sar_width); + ub(16, vui_sar_height); + } + } else { + infer(vui_aspect_ratio_constant_flag, 0); + infer(vui_aspect_ratio_idc, 0); + } + flag(vui_overscan_info_present_flag); + if (current->vui_overscan_info_present_flag) + flag(vui_overscan_appropriate_flag); + flag(vui_colour_description_present_flag); + if (current->vui_colour_description_present_flag) { + ub(8, vui_colour_primaries); + ub(8, vui_transfer_characteristics); + ub(8, vui_matrix_coeffs); + flag(vui_full_range_flag); + } else { + infer(vui_colour_primaries, 2); + infer(vui_transfer_characteristics, 2); + infer(vui_matrix_coeffs, 2); + infer(vui_full_range_flag, 0); + } + flag(vui_chroma_loc_info_present_flag); + if (current->vui_chroma_loc_info_present_flag) { + if (current->vui_progressive_source_flag && + !current->vui_interlaced_source_flag) { + ue(vui_chroma_sample_loc_type_frame, 0, 6); + } else { + ue(vui_chroma_sample_loc_type_top_field, 0, 6); + ue(vui_chroma_sample_loc_type_bottom_field, 0, 6); + } + } + + return 0; +} + +static int FUNC(payload_extension)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawExtensionData *current, + uint32_t payload_size, int cur_pos) +{ + int err; + size_t byte_length, k; + +#ifdef READ + GetBitContext tmp; + int bits_left, payload_zero_bits; + + if (!cbs_h265_payload_extension_present(rw, payload_size, cur_pos)) + return 0; + + bits_left = 8 * payload_size - cur_pos; + tmp = *rw; + if (bits_left > 8) + skip_bits_long(&tmp, bits_left - 8); + payload_zero_bits = get_bits(&tmp, FFMIN(bits_left, 8)); + if (!payload_zero_bits) + return AVERROR_INVALIDDATA; + payload_zero_bits = ff_ctz(payload_zero_bits); + current->bit_length = bits_left - payload_zero_bits - 1; + allocate(current->data, (current->bit_length + 7) / 8); +#endif + + byte_length = (current->bit_length + 7) / 8; + for (k = 0; k < byte_length; k++) { + int length = FFMIN(current->bit_length - k * 8, 8); + xu(length, reserved_payload_extension_data, current->data[k], + 0, MAX_UINT_BITS(length), 0); + } + + return 0; +} + +static int FUNC(vui_payload)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawVUI *current, uint16_t vui_payload_size) +{ + int err; + int start_position, current_position; +#ifdef READ + start_position = get_bits_count(rw); +#else + start_position = put_bits_count(rw); +#endif + CHECK(FUNC(vui_parameters)(ctx, rw, current)); + +#ifdef READ + current_position = get_bits_count(rw) - start_position; +#else + current_position = put_bits_count(rw) - start_position; +#endif + if (current_position < 8 * vui_payload_size) { + CHECK(FUNC(payload_extension)(ctx, rw, ¤t->extension_data, + vui_payload_size, current_position)); + fixed(1, vui_payload_bit_equal_to_one, 1); + while (byte_alignment(rw) != 0) + fixed(1, vui_payload_bit_equal_to_zero, 0); + } + return 0; +} + +static int FUNC(extension_data)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawExtensionData *current) +{ + int err; + size_t k; +#ifdef READ + GetBitContext start; + uint8_t bit; + start = *rw; + for (k = 0; cbs_h2645_read_more_rbsp_data(rw); k++) + skip_bits(rw, 1); + current->bit_length = k; + if (k > 0) { + *rw = start; + allocate(current->data, (current->bit_length + 7) / 8); + for (k = 0; k < current->bit_length; k++) { + xu(1, extension_data, bit, 0, 1, 0); + current->data[k / 8] |= bit << (7 - k % 8); + } + } +#else + for (k = 0; k < current->bit_length; k++) + xu(1, extension_data, current->data[k / 8] >> (7 - k % 8) & 1, 0, 1, 0); +#endif + return 0; +} + +static int FUNC(dpb_parameters)(CodedBitstreamContext *ctx, RWContext *rw, + H266DpbParameters *current, + uint8_t max_sublayers_minus1, + uint8_t sublayer_info_flag) +{ + int err, i; + for (i = ( sublayer_info_flag ? 0 : max_sublayers_minus1); + i <= max_sublayers_minus1; i++) { + ues(dpb_max_dec_pic_buffering_minus1[i], + 0, VVC_MAX_DPB_SIZE - 1, 1, i); + ues(dpb_max_num_reorder_pics[i], + 0, current->dpb_max_dec_pic_buffering_minus1[i], 1, i); + ues(dpb_max_latency_increase_plus1[i], 0, UINT32_MAX - 1, 1, i); + } + return 0; +} + +static int FUNC(ref_pic_list_struct)(CodedBitstreamContext *ctx, RWContext *rw, + H266RefPicListStruct *current, + uint8_t list_idx, uint8_t rpls_idx, + const H266RawSPS *sps) +{ + + int err, i, j; + ue(num_ref_entries, 0, VVC_MAX_DPB_SIZE + 13); + if (sps->sps_long_term_ref_pics_flag && + rpls_idx < sps->sps_num_ref_pic_lists[list_idx] && + current->num_ref_entries > 0) + flag(ltrp_in_header_flag); + if (sps->sps_long_term_ref_pics_flag && + rpls_idx == sps->sps_num_ref_pic_lists[list_idx]) + infer(ltrp_in_header_flag, 1); + for (i = 0, j = 0; i < current->num_ref_entries; i++) { + if (sps->sps_inter_layer_prediction_enabled_flag) + flags(inter_layer_ref_pic_flag[i], 1, i); + else + infer(inter_layer_ref_pic_flag[i], 0); + + if (!current->inter_layer_ref_pic_flag[i]) { + if (sps->sps_long_term_ref_pics_flag) + flags(st_ref_pic_flag[i], 1, i); + else + infer(st_ref_pic_flag[i], 1); + if (current->st_ref_pic_flag[i]) { + int abs_delta_poc_st; + ues(abs_delta_poc_st[i], 0, MAX_UINT_BITS(15), 1, i); + if ((sps->sps_weighted_pred_flag || + sps->sps_weighted_bipred_flag) && i != 0) + abs_delta_poc_st = current->abs_delta_poc_st[i]; + else + abs_delta_poc_st = current->abs_delta_poc_st[i] + 1; + if (abs_delta_poc_st > 0) + flags(strp_entry_sign_flag[i], 1, i); + } else { + if (!current->ltrp_in_header_flag) { + uint8_t bits = sps->sps_log2_max_pic_order_cnt_lsb_minus4 + 4; + ubs(bits, rpls_poc_lsb_lt[j], 1, j); + j++; + } + } + } else { + //todo: check range when vps parser is ready. + ues(ilrp_idx[i], 0, MAX_UINT_BITS(8), 1, i); + } + } + return 0; +} + +static int FUNC(ref_pic_lists)(CodedBitstreamContext *ctx, RWContext *rw, + H266RefPicLists *current) +{ + CodedBitstreamH266Context *h266 = ctx->priv_data; + const H266RawSPS *sps = h266->active_sps; + const H266RawPPS *pps = h266->active_pps; + const H266RefPicListStruct *ref_list; + int err, i, j, num_ltrp_entries; + for (i = 0; i < 2; i++) { + if (sps->sps_num_ref_pic_lists[i] > 0 && + (i == 0 || (i == 1 && pps->pps_rpl1_idx_present_flag))) { + flags(rpl_sps_flag[i], 1, i); + } else { + if (sps->sps_num_ref_pic_lists[i] == 0) { + infer(rpl_sps_flag[i], 0); + } else { + if (!pps->pps_rpl1_idx_present_flag && i == 1) + infer(rpl_sps_flag[1], current->rpl_sps_flag[0]); + } + } + if (current->rpl_sps_flag[i]) { + if (sps->sps_num_ref_pic_lists[i] > 1 && + (i == 0 || (i == 1 && pps->pps_rpl1_idx_present_flag))) { + uint8_t bits = av_ceil_log2(sps->sps_num_ref_pic_lists[i]); + us(bits, rpl_idx[i], 0, sps->sps_num_ref_pic_lists[i] - 1, 1, i); + } else if (sps->sps_num_ref_pic_lists[i] == 1) { + infer(rpl_idx[i], 0); + } else if (i == 1 && !pps->pps_rpl1_idx_present_flag){ + infer(rpl_idx[1], current->rpl_idx[0]); + } else { + //how to handle this? or never happpend? + av_log(ctx->log_ctx, AV_LOG_ERROR, + "can't infer the rpl_idx[i]\n"); + return AVERROR_PATCHWELCOME; + } + memcpy(¤t->rpl_ref_list[i], + &sps->sps_ref_pic_list_struct[i][current->rpl_idx[i]], + sizeof(current->rpl_ref_list[i])); + } else { + CHECK(FUNC(ref_pic_list_struct)(ctx, rw, ¤t->rpl_ref_list[i], + i, sps->sps_num_ref_pic_lists[i], sps)); + } + ref_list = ¤t->rpl_ref_list[i]; + + num_ltrp_entries = 0; + for (int i = 0; i < ref_list->num_ref_entries; i++) { + if (!ref_list->inter_layer_ref_pic_flag[i]) { + if (!ref_list->st_ref_pic_flag[i]) { + num_ltrp_entries++; + } + } + } + + for (j = 0; j < num_ltrp_entries; j++) { + if (ref_list->ltrp_in_header_flag) { + ubs(sps->sps_log2_max_pic_order_cnt_lsb_minus4 + 4, + poc_lsb_lt[i][j], + 2, i, j); + } + flags(delta_poc_msb_cycle_present_flag[i][j], 2, i, j); + if (current->delta_poc_msb_cycle_present_flag[i][j]) { + uint32_t max = 1 << (32 - sps->sps_log2_max_pic_order_cnt_lsb_minus4 - 4); + ues(delta_poc_msb_cycle_lt[i][j], 0, max, 2, i, j); + } + } + } + return 0; +} + +static int FUNC(general_timing_hrd_parameters)(CodedBitstreamContext *ctx, + RWContext *rw, + H266RawGeneralTimingHrdParameters *current) +{ + int err; + ub(32, num_units_in_tick); + u(32, time_scale, 1, MAX_UINT_BITS(32)); + flag(general_nal_hrd_params_present_flag); + flag(general_vcl_hrd_params_present_flag); + + if (current->general_nal_hrd_params_present_flag || + current->general_vcl_hrd_params_present_flag) { + flag(general_same_pic_timing_in_all_ols_flag); + flag(general_du_hrd_params_present_flag); + if (current->general_du_hrd_params_present_flag) + ub(8, tick_divisor_minus2); + ub(4, bit_rate_scale); + ub(4, cpb_size_scale); + if (current->general_du_hrd_params_present_flag) + ub(4, cpb_size_du_scale); + ue(hrd_cpb_cnt_minus1, 0, 31); + } else { + //infer general_same_pic_timing_in_all_ols_flag? + infer(general_du_hrd_params_present_flag, 0); + } + return 0; +} + +static int FUNC(sublayer_hrd_parameters)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawSubLayerHRDParameters *current, int sublayer_id, + const H266RawGeneralTimingHrdParameters *general) +{ + int err, i; + for (i = 0; i <= general->hrd_cpb_cnt_minus1; i++) { + ues(bit_rate_value_minus1[i], 0, UINT32_MAX - 1, 2, sublayer_id, i); + ues(cpb_size_value_minus1[i], 0, UINT32_MAX - 1, 2, sublayer_id, i); + if (general->general_du_hrd_params_present_flag) { + ues(cpb_size_du_value_minus1[i], + 0, UINT32_MAX - 1, 2, sublayer_id, i); + ues(bit_rate_du_value_minus1[i], + 0, UINT32_MAX - 1, 2, sublayer_id, i); + } + flags(cbr_flag[i], 2, sublayer_id, i); + } + return 0; +} + +static int FUNC(ols_timing_hrd_parameters)(CodedBitstreamContext *ctx, + RWContext *rw, H266RawOlsTimingHrdParameters *current, + uint8_t first_sublayer, uint8_t max_sublayers_minus1, + const H266RawGeneralTimingHrdParameters *general) +{ + int err, i; + for (i = first_sublayer; i <= max_sublayers_minus1; i++) { + flags(fixed_pic_rate_general_flag[i], 1, i); + if (!current->fixed_pic_rate_general_flag[i]) + flags(fixed_pic_rate_within_cvs_flag[i], 1, i); + else + infer(fixed_pic_rate_within_cvs_flag[i], 1); + if (current->fixed_pic_rate_within_cvs_flag[i]) { + ues(elemental_duration_in_tc_minus1[i], 0, 2047, 1, i); + infer(low_delay_hrd_flag[i], 0); + } else if ((general->general_nal_hrd_params_present_flag || + general->general_vcl_hrd_params_present_flag) && + general->hrd_cpb_cnt_minus1 == 0) { + flags(low_delay_hrd_flag[i], 1, i); + } else { + infer(low_delay_hrd_flag[i], 0); + } + if (general->general_nal_hrd_params_present_flag) + CHECK(FUNC(sublayer_hrd_parameters)(ctx, rw, + ¤t->nal_sub_layer_hrd_parameters[i], + i, general)); + if (general->general_vcl_hrd_params_present_flag) + CHECK(FUNC(sublayer_hrd_parameters)(ctx, rw, + ¤t->nal_sub_layer_hrd_parameters[i], i, general)); + } + return 0; +} + +static int FUNC(sps_subpic)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawSPS *current) +{ + int err, i; + unsigned int ctb_size_y = 1 << (current->sps_log2_ctu_size_minus5 + 5); + int tmp_width_val, tmp_height_val; + + tmp_width_val = + H266_CEIL(current->sps_pic_width_max_in_luma_samples, ctb_size_y); + tmp_height_val = + H266_CEIL(current->sps_pic_height_max_in_luma_samples, ctb_size_y); + + flag(sps_subpic_info_present_flag); + if (!current->sps_subpic_info_present_flag) { + infer(sps_num_subpics_minus1, 0); + infer(sps_independent_subpics_flag, 1); + infer(sps_subpic_same_size_flag, 0); + infer(sps_subpic_id_mapping_explicitly_signalled_flag, 0); + infer(sps_subpic_ctu_top_left_x[0], 0); + infer(sps_subpic_ctu_top_left_y[0], 0); + infer(sps_subpic_width_minus1[0], tmp_width_val - 1); + infer(sps_subpic_height_minus1[0], tmp_height_val - 1); + return 0; + } + ue(sps_num_subpics_minus1, 1, VVC_MAX_SLICES - 1); + if (current->sps_num_subpics_minus1 > 0) { + flag(sps_independent_subpics_flag); + flag(sps_subpic_same_size_flag); + } + + if (current->sps_num_subpics_minus1 > 0) { + int wlen = av_ceil_log2(tmp_width_val); + int hlen = av_ceil_log2(tmp_height_val); + if (current->sps_pic_width_max_in_luma_samples > ctb_size_y) + ubs(wlen, sps_subpic_width_minus1[0], 1, 0); + else + infer(sps_subpic_width_minus1[i], tmp_width_val - 1); + if (current->sps_pic_height_max_in_luma_samples > ctb_size_y) + ubs(hlen, sps_subpic_height_minus1[0], 1, 0); + else + infer(sps_subpic_height_minus1[0], tmp_height_val); + if (!current->sps_independent_subpics_flag) { + flags(sps_subpic_treated_as_pic_flag[0], 1, 0); + flags(sps_loop_filter_across_subpic_enabled_flag[0], 1, 0); + } else { + infer(sps_subpic_treated_as_pic_flag[0], 1); + infer(sps_loop_filter_across_subpic_enabled_flag[0], 1); + } + for (i = 1; i <= current->sps_num_subpics_minus1; i++) { + if (!current->sps_subpic_same_size_flag) { + if (current->sps_pic_width_max_in_luma_samples > ctb_size_y) + ubs(wlen, sps_subpic_ctu_top_left_x[i], 1, i); + else + infer(sps_subpic_ctu_top_left_x[i], 0); + if (current->sps_pic_height_max_in_luma_samples > ctb_size_y) + ubs(hlen, sps_subpic_ctu_top_left_y[i], 1, i); + else + infer(sps_subpic_ctu_top_left_y[i], 0); + if (i < current->sps_num_subpics_minus1 && + current->sps_pic_width_max_in_luma_samples > ctb_size_y) { + ubs(wlen, sps_subpic_width_minus1[i], 1, i); + } else { + infer(sps_subpic_width_minus1[i], + tmp_width_val - current->sps_subpic_ctu_top_left_x[i] - 1); + } + if (i < current->sps_num_subpics_minus1 && + current->sps_pic_height_max_in_luma_samples > ctb_size_y) { + ubs(hlen, sps_subpic_height_minus1[i], 1, i); + } else { + infer(sps_subpic_height_minus1[i], + tmp_height_val - current->sps_subpic_ctu_top_left_y[i] - 1); + } + } else { + int num_subpic_cols = + tmp_width_val / (current->sps_subpic_width_minus1[0] + 1); + infer(sps_subpic_ctu_top_left_x[i], + (i % num_subpic_cols) * + (current->sps_subpic_width_minus1[0] + 1)); + infer(sps_subpic_ctu_top_left_y[i], + (i / num_subpic_cols) * + (current->sps_subpic_height_minus1[0] + 1)); + infer(sps_subpic_width_minus1[i], + current->sps_subpic_width_minus1[0]); + infer(sps_subpic_height_minus1[i], + current->sps_subpic_height_minus1[0]); + } + if (!current->sps_independent_subpics_flag) { + flags(sps_subpic_treated_as_pic_flag[i], 1, i); + flags(sps_loop_filter_across_subpic_enabled_flag[i], 1, i); + } else { + infer(sps_subpic_treated_as_pic_flag[i], 1); + infer(sps_loop_filter_across_subpic_enabled_flag[i], 0); + } + } + ue(sps_subpic_id_len_minus1, 0, 15); + if ((1 << (current->sps_subpic_id_len_minus1 + 1)) < + current->sps_num_subpics_minus1 + 1) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "sps_subpic_id_len_minus1(%d) is too small\n", + current->sps_subpic_id_len_minus1); + return AVERROR_INVALIDDATA; + } + flag(sps_subpic_id_mapping_explicitly_signalled_flag); + if (current->sps_subpic_id_mapping_explicitly_signalled_flag) { + flag(sps_subpic_id_mapping_present_flag); + if (current->sps_subpic_id_mapping_present_flag) { + for (i = 0; i <= current->sps_num_subpics_minus1; i++) { + ubs(current->sps_subpic_id_len_minus1 + 1, + sps_subpic_id[i], 1, i); + } + } + } + } else { + infer(sps_subpic_ctu_top_left_x[0], 0); + infer(sps_subpic_ctu_top_left_y[0], 0); + infer(sps_subpic_width_minus1[0], tmp_width_val - 1); + infer(sps_subpic_height_minus1[0], tmp_height_val - 1); + } + return 0; +} + +static int FUNC(sps)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawSPS *current) +{ + CodedBitstreamH266Context *h266 = ctx->priv_data; + const H266RawVPS *vps; + int err, i, j; + unsigned int ctb_log2_size_y, min_cb_log2_size_y, + min_qt_log2_size_intra_y, min_qt_log2_size_inter_y, + ctb_size_y, max_num_merge_cand; + uint8_t qp_bd_offset; + + static const uint8_t h266_sub_width_c[] = { + 1, 2, 2, 1 + }; + static const uint8_t h266_sub_height_c[] = { + 1, 2, 1, 1 + }; + + HEADER("Sequence Parameter Set"); + + CHECK(FUNC(nal_unit_header)(ctx, rw, + ¤t->nal_unit_header, VVC_SPS_NUT)); + + ub(4, sps_seq_parameter_set_id); + ub(4, sps_video_parameter_set_id); + if (!current->sps_video_parameter_set_id && !h266->vps[0]) { + H266RawVPS *vps0; + h266->vps_ref[0] = av_buffer_alloc(sizeof(H266RawVPS)); + if (!h266->vps_ref[0]) + return AVERROR(ENOMEM); + vps0 = h266->vps[0] = (H266RawVPS *)h266->vps_ref[0]->data; + vps0->vps_max_layers_minus1 = 0; + //todo: infer all things in 7.4.3.4 sps_video_parameter_set_id paragraph + } + h266->active_vps = vps = h266->vps[current->sps_video_parameter_set_id]; + + u(3, sps_max_sublayers_minus1, 0, VVC_MAX_SUBLAYERS - 1); + u(2, sps_chroma_format_idc, 0, 3); + u(2, sps_log2_ctu_size_minus5, 0, 3); + ctb_log2_size_y = current->sps_log2_ctu_size_minus5 + 5; + ctb_size_y = 1 << ctb_log2_size_y; + + flag(sps_ptl_dpb_hrd_params_present_flag); + if (current->sps_ptl_dpb_hrd_params_present_flag) { + CHECK(FUNC(profile_tier_level)(ctx, rw, ¤t->profile_tier_level, + 1, current->sps_max_sublayers_minus1)); + } + flag(sps_gdr_enabled_flag); + flag(sps_ref_pic_resampling_enabled_flag); + if (current->sps_ref_pic_resampling_enabled_flag) + flag(sps_res_change_in_clvs_allowed_flag); + else + infer(sps_res_change_in_clvs_allowed_flag, 0); + + ue(sps_pic_width_max_in_luma_samples, 1, VVC_MAX_WIDTH); + ue(sps_pic_height_max_in_luma_samples, 1, VVC_MAX_HEIGHT); + + flag(sps_conformance_window_flag); + if (current->sps_conformance_window_flag) { + uint8_t sub_width_c = h266_sub_width_c[current->sps_chroma_format_idc]; + uint8_t sub_height_c = h266_sub_height_c[current->sps_chroma_format_idc]; + uint16_t width = current->sps_pic_width_max_in_luma_samples / sub_width_c; + uint16_t height = current->sps_pic_height_max_in_luma_samples / sub_height_c; + ue(sps_conf_win_left_offset, 0, width); + ue(sps_conf_win_right_offset, 0, width - current->sps_conf_win_left_offset); + ue(sps_conf_win_top_offset, 0, height); + ue(sps_conf_win_bottom_offset, 0, height - current->sps_conf_win_top_offset); + } else { + infer(sps_conf_win_left_offset, 0); + infer(sps_conf_win_right_offset, 0); + infer(sps_conf_win_top_offset, 0); + infer(sps_conf_win_bottom_offset, 0); + } + + CHECK(FUNC(sps_subpic)(ctx, rw, current)); + + ue(sps_bitdepth_minus8, 0, 2); + qp_bd_offset = 6 * current->sps_bitdepth_minus8; + + flag(sps_entropy_coding_sync_enabled_flag); + flag(sps_entry_point_offsets_present_flag); + + u(4, sps_log2_max_pic_order_cnt_lsb_minus4, 0, 12); + flag(sps_poc_msb_cycle_flag); + if (current->sps_poc_msb_cycle_flag) + ue(sps_poc_msb_cycle_len_minus1, + 0, 32 - current->sps_log2_max_pic_order_cnt_lsb_minus4 - 5); + + u(2, sps_num_extra_ph_bytes, 0, 2); + for (i = 0; i < (current->sps_num_extra_ph_bytes * 8); i++) { + flags(sps_extra_ph_bit_present_flag[i], 1, i); + } + + u(2, sps_num_extra_sh_bytes, 0, 0); + for (i = 0; i < (current->sps_num_extra_sh_bytes * 8); i++) { + flags(sps_extra_sh_bit_present_flag[i], 1, i); + } + + if (current->sps_ptl_dpb_hrd_params_present_flag) { + if (current->sps_max_sublayers_minus1 > 0) + flag(sps_sublayer_dpb_params_flag); + else + infer(sps_sublayer_dpb_params_flag, 0); + CHECK(FUNC(dpb_parameters)(ctx, rw, ¤t->sps_dpb_params, + current->sps_max_sublayers_minus1, + current->sps_sublayer_dpb_params_flag)); + } + + ue(sps_log2_min_luma_coding_block_size_minus2, + 0, FFMIN(4, current->sps_log2_ctu_size_minus5 + 3)); + min_cb_log2_size_y = + current->sps_log2_min_luma_coding_block_size_minus2 + 2; + + flag(sps_partition_constraints_override_enabled_flag); + + ue(sps_log2_diff_min_qt_min_cb_intra_slice_luma, + 0, FFMIN(6, ctb_log2_size_y) - min_cb_log2_size_y); + min_qt_log2_size_intra_y = + current->sps_log2_diff_min_qt_min_cb_intra_slice_luma + + min_cb_log2_size_y; + + ue(sps_max_mtt_hierarchy_depth_intra_slice_luma, + 0, 2 * (ctb_log2_size_y - min_cb_log2_size_y)); + + if (current->sps_max_mtt_hierarchy_depth_intra_slice_luma != 0) { + ue(sps_log2_diff_max_bt_min_qt_intra_slice_luma, + 0, ctb_log2_size_y - min_qt_log2_size_intra_y); + ue(sps_log2_diff_max_tt_min_qt_intra_slice_luma, + 0, FFMIN(6, ctb_log2_size_y) - min_qt_log2_size_intra_y); + } else { + infer(sps_log2_diff_max_bt_min_qt_intra_slice_luma, 0); + infer(sps_log2_diff_max_tt_min_qt_intra_slice_luma, 0); + } + + if (current->sps_chroma_format_idc != 0) { + flag(sps_qtbtt_dual_tree_intra_flag); + } else { + infer(sps_qtbtt_dual_tree_intra_flag, 0); + } + + if (current->sps_qtbtt_dual_tree_intra_flag) { + ue(sps_log2_diff_min_qt_min_cb_intra_slice_chroma, + 0, FFMIN(6, ctb_log2_size_y) - min_cb_log2_size_y); + ue(sps_max_mtt_hierarchy_depth_intra_slice_chroma, + 0, 2 * (ctb_log2_size_y - min_cb_log2_size_y)); + if (current->sps_max_mtt_hierarchy_depth_intra_slice_chroma != 0) { + unsigned int min_qt_log2_size_intra_c = + current->sps_log2_diff_min_qt_min_cb_intra_slice_chroma + + min_cb_log2_size_y; + ue(sps_log2_diff_max_bt_min_qt_intra_slice_chroma, + 0, FFMIN(6, ctb_log2_size_y) - min_qt_log2_size_intra_c); + ue(sps_log2_diff_max_tt_min_qt_intra_slice_chroma, + 0, FFMIN(6, ctb_log2_size_y) - min_qt_log2_size_intra_c); + } + } else { + infer(sps_log2_diff_min_qt_min_cb_intra_slice_chroma, 0); + infer(sps_max_mtt_hierarchy_depth_intra_slice_chroma, 0); + } + if (current->sps_max_mtt_hierarchy_depth_intra_slice_chroma == 0) { + infer(sps_log2_diff_max_bt_min_qt_intra_slice_chroma, 0); + infer(sps_log2_diff_max_tt_min_qt_intra_slice_chroma, 0); + } + + ue(sps_log2_diff_min_qt_min_cb_inter_slice, + 0, FFMIN(6, ctb_log2_size_y) - min_cb_log2_size_y); + min_qt_log2_size_inter_y = + current->sps_log2_diff_min_qt_min_cb_inter_slice + min_cb_log2_size_y; + + ue(sps_max_mtt_hierarchy_depth_inter_slice, + 0, 2 * (ctb_log2_size_y - min_cb_log2_size_y)); + if (current->sps_max_mtt_hierarchy_depth_inter_slice != 0) { + ue(sps_log2_diff_max_bt_min_qt_inter_slice, + 0, ctb_log2_size_y - min_qt_log2_size_inter_y); + ue(sps_log2_diff_max_tt_min_qt_inter_slice, + 0, FFMIN(6, ctb_log2_size_y) - min_qt_log2_size_inter_y); + } else { + infer(sps_log2_diff_max_bt_min_qt_inter_slice, 0); + infer(sps_log2_diff_max_tt_min_qt_inter_slice, 0); + } + + if (ctb_size_y > 32) + flag(sps_max_luma_transform_size_64_flag); + else + infer(sps_max_luma_transform_size_64_flag, 0); + + flag(sps_transform_skip_enabled_flag); + if (current->sps_transform_skip_enabled_flag) { + ue(sps_log2_transform_skip_max_size_minus2, 0, 3); + flag(sps_bdpcm_enabled_flag); + } + + flag(sps_mts_enabled_flag); + if (current->sps_mts_enabled_flag) { + flag(sps_explicit_mts_intra_enabled_flag); + flag(sps_explicit_mts_inter_enabled_flag); + } else { + infer(sps_explicit_mts_intra_enabled_flag, 0); + infer(sps_explicit_mts_inter_enabled_flag, 0); + } + + flag(sps_lfnst_enabled_flag); + + if (current->sps_chroma_format_idc != 0) { + uint8_t num_qp_tables; + flag(sps_joint_cbcr_enabled_flag); + flag(sps_same_qp_table_for_chroma_flag); + num_qp_tables = current->sps_same_qp_table_for_chroma_flag ? + 1 : ( current->sps_joint_cbcr_enabled_flag ? 3 : 2); + for (i = 0; i < num_qp_tables; i++) { + ses(sps_qp_table_start_minus26[i], + -26 - qp_bd_offset, 36, 1, i); + ues(sps_num_points_in_qp_table_minus1[i], + 0, 36 - current->sps_qp_table_start_minus26[i], 1, i); + for (j = 0; j <= current->sps_num_points_in_qp_table_minus1[i]; j++) { + uint8_t max = MAX_UINT_BITS(8); + ues(sps_delta_qp_in_val_minus1[i][j], 0, max, 2, i, j); + ues(sps_delta_qp_diff_val[i][j], 0, max, 2, i, j); + } + } + } else { + infer(sps_joint_cbcr_enabled_flag, 0); + infer(sps_same_qp_table_for_chroma_flag, 0); + } + + flag(sps_sao_enabled_flag); + flag(sps_alf_enabled_flag); + if (current->sps_alf_enabled_flag && current->sps_chroma_format_idc) + flag(sps_ccalf_enabled_flag); + else + infer(sps_ccalf_enabled_flag, 0); + flag(sps_lmcs_enabled_flag); + flag(sps_weighted_pred_flag); + flag(sps_weighted_bipred_flag); + flag(sps_long_term_ref_pics_flag); + if (current->sps_video_parameter_set_id > 0) + flag(sps_inter_layer_prediction_enabled_flag); + else + infer(sps_inter_layer_prediction_enabled_flag, 0); + flag(sps_idr_rpl_present_flag); + flag(sps_rpl1_same_as_rpl0_flag); + + for (i = 0; i < (current->sps_rpl1_same_as_rpl0_flag ? 1 : 2); i++) { + ues(sps_num_ref_pic_lists[i], 0, VVC_MAX_REF_PIC_LISTS, 1, i); + for (j = 0; j < current->sps_num_ref_pic_lists[i]; j++) + CHECK(FUNC(ref_pic_list_struct)(ctx, rw, + ¤t->sps_ref_pic_list_struct[i][j], + i, j, current)); + } + + if (current->sps_rpl1_same_as_rpl0_flag) { + current->sps_num_ref_pic_lists[1] = current->sps_num_ref_pic_lists[0]; + for (j = 0; j < current->sps_num_ref_pic_lists[0]; j++) + memcpy(¤t->sps_ref_pic_list_struct[1][j], + ¤t->sps_ref_pic_list_struct[0][j], + sizeof(current->sps_ref_pic_list_struct[0][j])); + } + + flag(sps_ref_wraparound_enabled_flag); + + flag(sps_temporal_mvp_enabled_flag); + if (current->sps_temporal_mvp_enabled_flag) + flag(sps_sbtmvp_enabled_flag); + else + infer(sps_sbtmvp_enabled_flag, 0); + + flag(sps_amvr_enabled_flag); + flag(sps_bdof_enabled_flag); + if (current->sps_bdof_enabled_flag) + flag(sps_bdof_control_present_in_ph_flag); + else + infer(sps_bdof_control_present_in_ph_flag, 0); + + flag(sps_smvd_enabled_flag); + flag(sps_dmvr_enabled_flag); + if (current->sps_dmvr_enabled_flag) + flag(sps_dmvr_control_present_in_ph_flag); + else + infer(sps_dmvr_control_present_in_ph_flag, 0); + + flag(sps_mmvd_enabled_flag); + if (current->sps_mmvd_enabled_flag) + flag(sps_mmvd_fullpel_only_enabled_flag); + else + infer(sps_mmvd_fullpel_only_enabled_flag, 0); + + ue(sps_six_minus_max_num_merge_cand, 0, 5); + max_num_merge_cand = 6 - current->sps_six_minus_max_num_merge_cand; + + flag(sps_sbt_enabled_flag); + + flag(sps_affine_enabled_flag); + if (current->sps_affine_enabled_flag) { + ue(sps_five_minus_max_num_subblock_merge_cand, + 0, 5 - current->sps_sbtmvp_enabled_flag); + flag(sps_6param_affine_enabled_flag); + if (current->sps_amvr_enabled_flag) + flag(sps_affine_amvr_enabled_flag); + else + infer(sps_affine_amvr_enabled_flag, 0); + flag(sps_affine_prof_enabled_flag); + if (current->sps_affine_prof_enabled_flag) + flag(sps_prof_control_present_in_ph_flag); + else + infer(sps_prof_control_present_in_ph_flag, 0); + } else { + infer(sps_five_minus_max_num_subblock_merge_cand, 0); + infer(sps_6param_affine_enabled_flag, 0); + infer(sps_affine_amvr_enabled_flag, 0); + infer(sps_affine_prof_enabled_flag, 0); + infer(sps_prof_control_present_in_ph_flag, 0); + } + + flag(sps_bcw_enabled_flag); + flag(sps_ciip_enabled_flag); + + if (max_num_merge_cand >= 2) { + flag(sps_gpm_enabled_flag); + if (current->sps_gpm_enabled_flag && max_num_merge_cand >= 3) + ue(sps_max_num_merge_cand_minus_max_num_gpm_cand, + 0, max_num_merge_cand - 2); + } else { + infer(sps_gpm_enabled_flag, 0); + } + + ue(sps_log2_parallel_merge_level_minus2, 0, ctb_log2_size_y - 2); + + flag(sps_isp_enabled_flag); + flag(sps_mrl_enabled_flag); + flag(sps_mip_enabled_flag); + + if (current->sps_chroma_format_idc != 0) + flag(sps_cclm_enabled_flag); + else + infer(sps_cclm_enabled_flag, 0); + if (current->sps_chroma_format_idc == 1) { + flag(sps_chroma_horizontal_collocated_flag); + flag(sps_chroma_vertical_collocated_flag); + } else { + infer(sps_chroma_horizontal_collocated_flag, 1); + infer(sps_chroma_vertical_collocated_flag, 1); + } + + flag(sps_palette_enabled_flag); + if (current->sps_chroma_format_idc == 3 && + !current->sps_max_luma_transform_size_64_flag) + flag(sps_act_enabled_flag); + else + infer(sps_act_enabled_flag, 0); + if (current->sps_transform_skip_enabled_flag || + current->sps_palette_enabled_flag) + ue(sps_min_qp_prime_ts, 0, 8); + + flag(sps_ibc_enabled_flag); + if (current->sps_ibc_enabled_flag) { + ue(sps_six_minus_max_num_ibc_merge_cand, 0, 5); + } + + flag(sps_ladf_enabled_flag); + if (current->sps_ladf_enabled_flag) { + ub(2, sps_num_ladf_intervals_minus2); + se(sps_ladf_lowest_interval_qp_offset, -63, 63); + for (i = 0; i < current->sps_num_ladf_intervals_minus2 + 1; i++) { + ses(sps_ladf_qp_offset[i], -63, 63, 1, i); + ues(sps_ladf_delta_threshold_minus1[i], + 0, (2 << ( 8 + current->sps_bitdepth_minus8)) - 3, 1, i); + } + } + + flag(sps_explicit_scaling_list_enabled_flag); + if (current->sps_lfnst_enabled_flag && + current->sps_explicit_scaling_list_enabled_flag) + flag(sps_scaling_matrix_for_lfnst_disabled_flag); + + if (current->sps_act_enabled_flag && + current->sps_explicit_scaling_list_enabled_flag) + flag(sps_scaling_matrix_for_alternative_colour_space_disabled_flag); + else + infer(sps_scaling_matrix_for_alternative_colour_space_disabled_flag, 0); + if (current->sps_scaling_matrix_for_alternative_colour_space_disabled_flag) + flag(sps_scaling_matrix_designated_colour_space_flag); + + flag(sps_dep_quant_enabled_flag); + flag(sps_sign_data_hiding_enabled_flag); + + flag(sps_virtual_boundaries_enabled_flag); + if (current->sps_virtual_boundaries_enabled_flag) { + flag(sps_virtual_boundaries_present_flag); + if (current->sps_virtual_boundaries_present_flag) { + ue(sps_num_ver_virtual_boundaries, + 0, current->sps_pic_width_max_in_luma_samples <= 8 ? 0 : 3); + for (i = 0; i < current->sps_num_ver_virtual_boundaries; i++) + ues(sps_virtual_boundary_pos_x_minus1[i], + 0, (current->sps_pic_width_max_in_luma_samples + 7) / 8 - 2, + 1, i); + ue(sps_num_hor_virtual_boundaries, + 0, current->sps_pic_height_max_in_luma_samples <= 8 ? 0 : 3); + for (i = 0; i < current->sps_num_hor_virtual_boundaries; i++) + ues(sps_virtual_boundary_pos_y_minus1[i], + 0, (current->sps_pic_height_max_in_luma_samples + 7) / 8 - 2, + 1, i); + } + } else { + infer(sps_virtual_boundaries_present_flag, 0); + infer(sps_num_ver_virtual_boundaries, 0); + infer(sps_num_hor_virtual_boundaries, 0); + } + + if (current->sps_ptl_dpb_hrd_params_present_flag) { + flag(sps_timing_hrd_params_present_flag); + if (current->sps_timing_hrd_params_present_flag) { + uint8_t first_sublayer; + CHECK(FUNC(general_timing_hrd_parameters)(ctx, rw, + ¤t->sps_general_timing_hrd_parameters)); + if (current->sps_max_sublayers_minus1 > 0) + flag(sps_sublayer_cpb_params_present_flag); + else + infer(sps_sublayer_cpb_params_present_flag, 0); + first_sublayer = current->sps_sublayer_cpb_params_present_flag ? + 0 : current->sps_max_sublayers_minus1; + CHECK(FUNC(ols_timing_hrd_parameters)(ctx, rw, + ¤t->sps_ols_timing_hrd_parameters, first_sublayer, + current->sps_max_sublayers_minus1, + ¤t->sps_general_timing_hrd_parameters)); + } + } + + flag(sps_field_seq_flag); + flag(sps_vui_parameters_present_flag); + if (current->sps_vui_parameters_present_flag) { + ue(sps_vui_payload_size_minus1, 0, 1023); + while (byte_alignment(rw) != 0) + fixed(1, sps_vui_alignment_zero_bit, 0); + CHECK(FUNC(vui_payload)(ctx, rw, ¤t->vui, + current->sps_vui_payload_size_minus1 + 1)); + } + + flag(sps_extension_flag); + if (current->sps_extension_flag) + CHECK(FUNC(extension_data)(ctx, rw, ¤t->extension_data)); + + CHECK(FUNC(rbsp_trailing_bits)(ctx, rw)); + + return 0; +} + +static int FUNC(pps_slice_info)(CodedBitstreamContext *ctx, RWContext *rw, + const H266RawSPS *sps, H266RawPPS *current) +{ + int err, i, j; + uint16_t tile_idx = 0, tile_x, tile_y, ctu_x, ctu_y; + uint16_t slice_top_left_ctu_x[VVC_MAX_SLICES]; + uint16_t slice_top_left_ctu_y[VVC_MAX_SLICES]; + ue(pps_num_slices_in_pic_minus1, 0, VVC_MAX_SLICES - 1); + if (current->pps_num_slices_in_pic_minus1 > 1) + flag(pps_tile_idx_delta_present_flag); + else + infer(pps_tile_idx_delta_present_flag, 0); + for (i = 0; i < current->pps_num_slices_in_pic_minus1; i++) { + tile_x = tile_idx % current->num_tile_columns; + tile_y = tile_idx / current->num_tile_columns; + if (tile_x != current->num_tile_columns - 1) { + ues(pps_slice_width_in_tiles_minus1[i], + 0, current->num_tile_columns - 1, 1, i); + } else { + infer(pps_slice_width_in_tiles_minus1[i], 0); + } + if (tile_y != current->num_tile_rows - 1 && + (current->pps_tile_idx_delta_present_flag || tile_x == 0)) { + ues(pps_slice_height_in_tiles_minus1[i], + 0, current->num_tile_rows - 1, 1, i); + } else { + if (tile_y == current->num_tile_rows - 1) + infer(pps_slice_height_in_tiles_minus1[i], 0); + else + infer(pps_slice_height_in_tiles_minus1[i], + current->pps_slice_height_in_tiles_minus1[i-1]); + } + + ctu_x = ctu_y = 0; + for (j = 0; j < tile_x; j++) { + ctu_x += current->pps_tile_column_width_minus1[j] + 1; + } + for (j = 0; j < tile_y; j++) { + ctu_y += current->pps_tile_row_height_minus1[j] + 1; + } + if (H266_HAS_MULTI_SLICES_IN_TILE(current, tile_y, i)) { + int num_slices_in_tile, + uniform_slice_height, remaining_height_in_ctbs_y; + remaining_height_in_ctbs_y = + current->pps_tile_row_height_minus1[tile_y] + 1; + ues(pps_num_exp_slices_in_tile[i], + 0, current->pps_tile_row_height_minus1[tile_y], 1, i); + if (current->pps_num_exp_slices_in_tile[i] == 0) { + num_slices_in_tile = 1; + slice_top_left_ctu_x[i] = ctu_x; + slice_top_left_ctu_y[i] = ctu_y; + } else { + uint16_t slice_height_in_ctus; + for (j = 0; j < current->pps_num_exp_slices_in_tile[i]; j++) { + ues(pps_exp_slice_height_in_ctus_minus1[i][j], + 0, current->pps_tile_row_height_minus1[tile_y], 2, i, j); + slice_height_in_ctus = + current->pps_exp_slice_height_in_ctus_minus1[i][j] + 1; + + current->slice_height_in_ctus[i + j] = slice_height_in_ctus; + slice_top_left_ctu_x[i + j] = ctu_x; + slice_top_left_ctu_y[i + j] = ctu_y; + ctu_y += slice_height_in_ctus; + + remaining_height_in_ctbs_y -= slice_height_in_ctus; + } + uniform_slice_height = 1 + + (j == 0 ? current->pps_tile_row_height_minus1[tile_y] : + current->pps_exp_slice_height_in_ctus_minus1[i][j-1]); + while (remaining_height_in_ctbs_y > uniform_slice_height) { + current->slice_height_in_ctus[i + j] = uniform_slice_height; + slice_top_left_ctu_x[i + j] = ctu_x; + slice_top_left_ctu_y[i + j] = ctu_y; + ctu_y += uniform_slice_height; + + remaining_height_in_ctbs_y -= uniform_slice_height; + j++; + } + if (remaining_height_in_ctbs_y > 0) { + current->slice_height_in_ctus[i + j] = remaining_height_in_ctbs_y; + slice_top_left_ctu_x[i + j] = ctu_x; + slice_top_left_ctu_y[i + j] = ctu_y; + j++; + } + num_slices_in_tile = j; + } + i += num_slices_in_tile - 1; + } else { + uint16_t height= 0; + infer(pps_num_exp_slices_in_tile[i], 0); + for (j = 0; j <= current->pps_slice_height_in_tiles_minus1[i]; j++) { + height += current->pps_tile_row_height_minus1[tile_y + j] + 1; + } + current->slice_height_in_ctus[i] = height; + + slice_top_left_ctu_x[i] = ctu_x; + slice_top_left_ctu_y[i] = ctu_y; + } + if (i < current->pps_num_slices_in_pic_minus1) { + if (current->pps_tile_idx_delta_present_flag) { + ses(pps_tile_idx_delta_val[i], + -current->num_tiles_in_pic + 1, current->num_tiles_in_pic - 1, + 1, i); + if (current->pps_tile_idx_delta_val[i] == 0) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "pps_tile_idx_delta_val[i] shall not be equal to 0.\n"); + } + tile_idx += current->pps_tile_idx_delta_val[i]; + } else { + infer(pps_tile_idx_delta_val[i], 0); + tile_idx += current->pps_slice_width_in_tiles_minus1[i] + 1; + if (tile_idx % current->num_tile_columns == 0) { + tile_idx += current->pps_slice_height_in_tiles_minus1[i] * + current->num_tile_columns; + } + } + } + } + if (i == current->pps_num_slices_in_pic_minus1) { + uint16_t height= 0; + + tile_x = tile_idx % current->num_tile_columns; + tile_y = tile_idx / current->num_tile_columns; + + ctu_x = 0, ctu_y = 0; + for (j = 0; j < tile_x; j++) { + ctu_x += current->pps_tile_column_width_minus1[j] + 1; + } + for (j = 0; j < tile_y; j++) { + ctu_y += current->pps_tile_row_height_minus1[j] + 1; + } + slice_top_left_ctu_x[i] = ctu_x; + slice_top_left_ctu_y[i] = ctu_y; + + current->pps_slice_width_in_tiles_minus1[i] = + current->num_tile_columns - tile_x - 1; + current->pps_slice_height_in_tiles_minus1[i] = + current->num_tile_rows - tile_y - 1; + + for (j = 0; j <= current->pps_slice_height_in_tiles_minus1[i]; j++) { + height += current->pps_tile_row_height_minus1[tile_y + j] + 1; + } + current->slice_height_in_ctus[i] = height; + + infer(pps_num_exp_slices_in_tile[i], 0); + } + + //now, we got all slice information, let's resolve NumSlicesInSubpic + for (i = 0; i <= sps->sps_num_subpics_minus1; i++) { + current->num_slices_in_subpic[i] = 0; + for (j = 0; j <= current->pps_num_slices_in_pic_minus1; j++) { + uint16_t pos_x = 0, pos_y = 0; + pos_x = slice_top_left_ctu_x[j]; + pos_y = slice_top_left_ctu_y[j]; + if ((pos_x >= sps->sps_subpic_ctu_top_left_x[i]) && + (pos_x < sps->sps_subpic_ctu_top_left_x[i] + sps->sps_subpic_width_minus1[i] + 1) && + (pos_y >= sps->sps_subpic_ctu_top_left_y[i]) && + (pos_y < sps->sps_subpic_ctu_top_left_y[i] + sps->sps_subpic_height_minus1[i] + 1)) { + current->num_slices_in_subpic[i]++; + } + } + } + return 0; +} + +static int FUNC(pps_tile_info)(CodedBitstreamContext *ctx, RWContext *rw, + const H266RawSPS *sps, H266RawPPS *current) +{ + int err, i; + uint8_t ctb_size_y = 1 << (sps->sps_log2_ctu_size_minus5 + 5); + uint16_t pic_width_in_ctbs_y = + H266_CEIL(current->pps_pic_width_in_luma_samples, ctb_size_y); + uint16_t pic_height_in_ctbs_y = + H266_CEIL(current->pps_pic_height_in_luma_samples, ctb_size_y); + if (!current->pps_no_pic_partition_flag) { + unsigned int exp_tile_width = 0, exp_tile_height = 0; + unsigned int unified_size, remaining_size; + u(2, pps_log2_ctu_size_minus5, + sps->sps_log2_ctu_size_minus5, sps->sps_log2_ctu_size_minus5); + ue(pps_num_exp_tile_columns_minus1, + 0, FFMIN(pic_width_in_ctbs_y - 1, VVC_MAX_TILE_COLUMNS)); + ue(pps_num_exp_tile_rows_minus1, + 0, FFMIN(pic_height_in_ctbs_y - 1, VVC_MAX_TILE_ROWS)); + + for (i = 0; i <= current->pps_num_exp_tile_columns_minus1; i++) { + ues(pps_tile_column_width_minus1[i], + 0, pic_width_in_ctbs_y - 1, 1, i); + exp_tile_width += current->pps_tile_column_width_minus1[i] + 1; + } + remaining_size = pic_width_in_ctbs_y - exp_tile_width; + unified_size = (i == 0 ? pic_width_in_ctbs_y : + (current->pps_tile_column_width_minus1[i - 1] + 1)); + while (remaining_size > unified_size) { + current->pps_tile_column_width_minus1[i] = unified_size - 1; + remaining_size -= unified_size; + i++; + } + if (remaining_size > 0) { + current->pps_tile_column_width_minus1[i] = remaining_size - 1; + i++; + } + current->num_tile_columns = i; + + for (i = 0; i <= current->pps_num_exp_tile_rows_minus1; i++) { + ues(pps_tile_row_height_minus1[i], + 0, pic_height_in_ctbs_y - 1, 1, i); + exp_tile_height += current->pps_tile_row_height_minus1[i] + 1; + } + + remaining_size = pic_height_in_ctbs_y - exp_tile_height; + unified_size = (i == 0 ? pic_height_in_ctbs_y : + (current->pps_tile_row_height_minus1[i - 1] + 1)); + while (remaining_size > unified_size) { + current->pps_tile_row_height_minus1[i] = unified_size - 1; + remaining_size -= unified_size; + i++; + } + if (remaining_size > 0) { + current->pps_tile_row_height_minus1[i] = remaining_size - 1; + i++; + } + current->num_tile_rows = i; + + current->num_tiles_in_pic = + current->num_tile_columns * current->num_tile_rows; + if (current->num_tiles_in_pic > 1) { + flag(pps_loop_filter_across_tiles_enabled_flag); + flag(pps_rect_slice_flag); + } else { + infer(pps_loop_filter_across_tiles_enabled_flag, 0); + infer(pps_rect_slice_flag, 1); + } + if (current->pps_rect_slice_flag) + flag(pps_single_slice_per_subpic_flag); + else + infer(pps_single_slice_per_subpic_flag, 1); + if (current->pps_rect_slice_flag && + !current->pps_single_slice_per_subpic_flag) { + CHECK(FUNC(pps_slice_info)(ctx, rw, sps, current)); + } else { + if (current->pps_no_pic_partition_flag) + infer(pps_num_slices_in_pic_minus1, 0); + else if (current->pps_single_slice_per_subpic_flag) + infer(pps_num_slices_in_pic_minus1, sps->sps_num_subpics_minus1); + // else? + } + if (!current->pps_rect_slice_flag || + current->pps_single_slice_per_subpic_flag || + current->pps_num_slices_in_pic_minus1 > 0) + flag(pps_loop_filter_across_slices_enabled_flag); + else + infer(pps_loop_filter_across_slices_enabled_flag, 0); + } else { + infer(pps_num_exp_tile_columns_minus1, 0); + infer(pps_tile_column_width_minus1[0], pic_width_in_ctbs_y - 1); + infer(pps_num_exp_tile_rows_minus1, 0); + infer(pps_tile_row_height_minus1[0], pic_height_in_ctbs_y - 1); + infer(num_tile_columns, 1); + infer(num_tile_rows, 1); + infer(num_tiles_in_pic, 1); + } + return 0; +} + +static int FUNC(pps)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawPPS *current) +{ + + CodedBitstreamH266Context *h266 = ctx->priv_data; + const H266RawSPS *sps; + int err, i; + unsigned int min_cb_size_y, divisor, ctb_size_y; + uint8_t sub_width_c, sub_height_c, qp_bd_offset; + + static const uint8_t h266_sub_width_c[] = { + 1, 2, 2, 1 + }; + static const uint8_t h266_sub_height_c[] = { + 1, 2, 1, 1 + }; + + HEADER("Picture Parameter Set"); + + CHECK(FUNC(nal_unit_header)(ctx, rw, + ¤t->nal_unit_header, VVC_PPS_NUT)); + + ub(6, pps_pic_parameter_set_id); + ub(4, pps_seq_parameter_set_id); + sps = h266->sps[current->pps_seq_parameter_set_id]; + if (!sps) { + av_log(ctx->log_ctx, AV_LOG_ERROR, "SPS id %d not available.\n", + current->pps_seq_parameter_set_id); + return AVERROR_INVALIDDATA; + } + h266->active_sps = sps; + + flag(pps_mixed_nalu_types_in_pic_flag); + ue(pps_pic_width_in_luma_samples, + 1, sps->sps_pic_width_max_in_luma_samples); + ue(pps_pic_height_in_luma_samples, + 1, sps->sps_pic_height_max_in_luma_samples); + + min_cb_size_y = 1 << (sps->sps_log2_min_luma_coding_block_size_minus2 + 2); + divisor = FFMAX(min_cb_size_y, 8); + if (current->pps_pic_width_in_luma_samples % divisor || + current->pps_pic_height_in_luma_samples % divisor) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "Invalid dimensions: %ux%u not divisible " + "by %u, MinCbSizeY = %u.\n", current->pps_pic_width_in_luma_samples, + current->pps_pic_height_in_luma_samples, divisor, min_cb_size_y); + return AVERROR_INVALIDDATA; + } + if (!sps->sps_res_change_in_clvs_allowed_flag && + (current->pps_pic_width_in_luma_samples != + sps->sps_pic_width_max_in_luma_samples || + current->pps_pic_height_in_luma_samples != + sps->sps_pic_height_max_in_luma_samples)) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "Resoltuion change is not allowed, " + "in max resolution (%ux%u) mismatched with pps(%ux%u).\n", + sps->sps_pic_width_max_in_luma_samples, + sps->sps_pic_height_max_in_luma_samples, + current->pps_pic_width_in_luma_samples, + current->pps_pic_height_in_luma_samples); + return AVERROR_INVALIDDATA; + } + + ctb_size_y = 1 << (sps->sps_log2_ctu_size_minus5 + 5); + if (sps->sps_ref_wraparound_enabled_flag) { + if ((ctb_size_y / min_cb_size_y + 1) > + (current->pps_pic_width_in_luma_samples / min_cb_size_y - 1)) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "Invalid width(%u), ctb_size_y = %u, min_cb_size_y = %u.\n", + current->pps_pic_width_in_luma_samples, + ctb_size_y, min_cb_size_y); + return AVERROR_INVALIDDATA; + } + } + + flag(pps_conformance_window_flag); + if (current->pps_pic_width_in_luma_samples == + sps->sps_pic_width_max_in_luma_samples && + current->pps_pic_height_in_luma_samples == + sps->sps_pic_height_max_in_luma_samples && + current->pps_conformance_window_flag) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "Conformance window flag should not true.\n"); + return AVERROR_INVALIDDATA; + } + + sub_width_c = h266_sub_width_c[sps->sps_chroma_format_idc]; + sub_height_c = h266_sub_height_c[sps->sps_chroma_format_idc]; + if (current->pps_conformance_window_flag) { + ue(pps_conf_win_left_offset, + 0, current->pps_pic_width_in_luma_samples); + ue(pps_conf_win_right_offset, + 0, current->pps_pic_width_in_luma_samples); + ue(pps_conf_win_top_offset, + 0, current->pps_pic_height_in_luma_samples); + ue(pps_conf_win_bottom_offset, + 0, current->pps_pic_height_in_luma_samples); + if (sub_width_c * + (current->pps_conf_win_left_offset + + current->pps_conf_win_right_offset) >= + current->pps_pic_width_in_luma_samples || + sub_height_c * + (current->pps_conf_win_top_offset + + current->pps_conf_win_bottom_offset) >= + current->pps_pic_height_in_luma_samples) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "Invalid pps conformance window: (%u, %u, %u, %u), " + "resolution is %ux%u, sub wxh is %ux%u.\n", + current->pps_conf_win_left_offset, + current->pps_conf_win_right_offset, + current->pps_conf_win_top_offset, + current->pps_conf_win_bottom_offset, + current->pps_pic_width_in_luma_samples, + current->pps_pic_height_in_luma_samples, + sub_width_c, sub_height_c); + return AVERROR_INVALIDDATA; + } + } else { + if (current->pps_pic_width_in_luma_samples == + sps->sps_pic_width_max_in_luma_samples && + current->pps_pic_height_in_luma_samples == + sps->sps_pic_height_max_in_luma_samples) { + infer(pps_conf_win_left_offset, sps->sps_conf_win_left_offset); + infer(pps_conf_win_right_offset, sps->sps_conf_win_right_offset); + infer(pps_conf_win_top_offset, sps->sps_conf_win_top_offset); + infer(pps_conf_win_bottom_offset, sps->sps_conf_win_bottom_offset); + } else { + infer(pps_conf_win_left_offset, 0); + infer(pps_conf_win_right_offset, 0); + infer(pps_conf_win_top_offset, 0); + infer(pps_conf_win_bottom_offset, 0); + } + + } + + flag(pps_scaling_window_explicit_signalling_flag); + if (!sps->sps_ref_pic_resampling_enabled_flag && + current->pps_scaling_window_explicit_signalling_flag) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "Invalid data: sps_ref_pic_resampling_enabled_flag is false, " + "but pps_scaling_window_explicit_signalling_flag is true.\n"); + return AVERROR_INVALIDDATA; + } + if (current->pps_scaling_window_explicit_signalling_flag) { + se(pps_scaling_win_left_offset, + -current->pps_pic_width_in_luma_samples * 15 / sub_width_c, + current->pps_pic_width_in_luma_samples / sub_width_c); + se(pps_scaling_win_right_offset, + -current->pps_pic_width_in_luma_samples * 15 / sub_width_c, + current->pps_pic_width_in_luma_samples / sub_width_c); + se(pps_scaling_win_top_offset, + -current->pps_pic_height_in_luma_samples * 15 / sub_height_c, + current->pps_pic_height_in_luma_samples / sub_height_c); + se(pps_scaling_win_bottom_offset, + -current->pps_pic_height_in_luma_samples * 15 / sub_height_c, + current->pps_pic_height_in_luma_samples / sub_height_c); + } else { + infer(pps_scaling_win_left_offset, current->pps_conf_win_left_offset); + infer(pps_scaling_win_right_offset, current->pps_conf_win_right_offset); + infer(pps_scaling_win_top_offset, current->pps_conf_win_top_offset); + infer(pps_scaling_win_bottom_offset, current->pps_conf_win_bottom_offset); + } + + flag(pps_output_flag_present_flag); + flag(pps_no_pic_partition_flag); + flag(pps_subpic_id_mapping_present_flag); + + if (current->pps_subpic_id_mapping_present_flag) { + if (!current->pps_no_pic_partition_flag) { + ue(pps_num_subpics_minus1, + sps->sps_num_subpics_minus1, sps->sps_num_subpics_minus1); + } else { + infer(pps_num_subpics_minus1, 0); + } + ue(pps_subpic_id_len_minus1, sps->sps_subpic_id_len_minus1, sps->sps_subpic_id_len_minus1); + for (i = 0; i <= current->pps_num_subpics_minus1; i++) { + ubs(sps->sps_subpic_id_len_minus1 + 1, pps_subpic_id[i], 1, i); + } + } + CHECK(FUNC(pps_tile_info)(ctx, rw, sps, current)); + + flag(pps_cabac_init_present_flag); + for (i = 0; i < 2; i++) + ues(pps_num_ref_idx_default_active_minus1[i], 0, 14, 1, i); + flag(pps_rpl1_idx_present_flag); + flag(pps_weighted_pred_flag); + flag(pps_weighted_bipred_flag); + flag(pps_ref_wraparound_enabled_flag); + if (current->pps_ref_wraparound_enabled_flag) { + ue(pps_pic_width_minus_wraparound_offset, + 0, (current->pps_pic_width_in_luma_samples / min_cb_size_y) + - (ctb_size_y / min_cb_size_y) - 2); + } + + qp_bd_offset = 6 * sps->sps_bitdepth_minus8; + se(pps_init_qp_minus26, -(26 + qp_bd_offset), 37); + flag(pps_cu_qp_delta_enabled_flag); + flag(pps_chroma_tool_offsets_present_flag); + if (current->pps_chroma_tool_offsets_present_flag) { + se(pps_cb_qp_offset, -12, 12); + se(pps_cr_qp_offset, -12, 12); + flag(pps_joint_cbcr_qp_offset_present_flag); + if (current->pps_joint_cbcr_qp_offset_present_flag) + se(pps_joint_cbcr_qp_offset_value, -12, 12); + else + infer(pps_joint_cbcr_qp_offset_value, 0); + flag(pps_slice_chroma_qp_offsets_present_flag); + flag(pps_cu_chroma_qp_offset_list_enabled_flag); + if (current->pps_cu_chroma_qp_offset_list_enabled_flag) + return AVERROR_PATCHWELCOME; + } else { + infer(pps_cb_qp_offset, 0); + infer(pps_cr_qp_offset, 0); + infer(pps_joint_cbcr_qp_offset_present_flag, 0); + infer(pps_joint_cbcr_qp_offset_value, 0); + infer(pps_slice_chroma_qp_offsets_present_flag, 0); + infer(pps_cu_chroma_qp_offset_list_enabled_flag, 0); + } + flag(pps_deblocking_filter_control_present_flag); + if (current->pps_deblocking_filter_control_present_flag) { + flag(pps_deblocking_filter_override_enabled_flag); + flag(pps_deblocking_filter_disabled_flag); + if (!current->pps_no_pic_partition_flag && + current->pps_deblocking_filter_override_enabled_flag) + flag(pps_dbf_info_in_ph_flag); + else + infer(pps_dbf_info_in_ph_flag, 0); + if (!current->pps_deblocking_filter_disabled_flag) { + se(pps_luma_beta_offset_div2, -12, 12); + se(pps_luma_tc_offset_div2, -12, 12); + } else { + infer(pps_luma_beta_offset_div2, 0); + infer(pps_luma_tc_offset_div2, 0); + } + if (current->pps_chroma_tool_offsets_present_flag) { + se(pps_cb_beta_offset_div2, -12, 12); + se(pps_cb_tc_offset_div2, -12, 12); + se(pps_cr_beta_offset_div2, -12, 12); + se(pps_cr_tc_offset_div2, -12, 12); + } else { + infer(pps_cb_beta_offset_div2, 0); + infer(pps_cb_tc_offset_div2, 0); + infer(pps_cr_beta_offset_div2, current->pps_luma_beta_offset_div2); + infer(pps_cr_tc_offset_div2, current->pps_luma_tc_offset_div2); + } + } else { + infer(pps_deblocking_filter_override_enabled_flag, 0); + infer(pps_deblocking_filter_disabled_flag, 0); + infer(pps_dbf_info_in_ph_flag, 0); + infer(pps_luma_beta_offset_div2, 0); + infer(pps_luma_tc_offset_div2, 0); + infer(pps_cb_beta_offset_div2, 0); + infer(pps_cb_tc_offset_div2, 0); + infer(pps_cr_beta_offset_div2, 0); + infer(pps_cr_tc_offset_div2, 0); + } + + if (!current->pps_no_pic_partition_flag) { + flag(pps_rpl_info_in_ph_flag); + flag(pps_sao_info_in_ph_flag); + flag(pps_alf_info_in_ph_flag); + if ((current->pps_weighted_pred_flag || current->pps_weighted_bipred_flag) && + current->pps_rpl_info_in_ph_flag) + flag(pps_wp_info_in_ph_flag); + flag(pps_qp_delta_info_in_ph_flag); + } + flag(pps_picture_header_extension_present_flag); + flag(pps_slice_header_extension_present_flag); + + flag(pps_extension_flag); + if (current->pps_extension_flag) + CHECK(FUNC(extension_data)(ctx, rw, ¤t->extension_data)); + + CHECK(FUNC(rbsp_trailing_bits)(ctx, rw)); + return 0; +} + +static int FUNC(aud)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawAUD *current) +{ + int err; + + HEADER("Access Unit Delimiter"); + + CHECK(FUNC(nal_unit_header)(ctx, rw, + ¤t->nal_unit_header, VVC_AUD_NUT)); + + flag(aud_irap_or_gdr_flag); + u(3, aud_pic_type, 0, 2); + + CHECK(FUNC(rbsp_trailing_bits)(ctx, rw)); + return 0; +} + +static int FUNC(pred_weight_table)(CodedBitstreamContext *ctx, RWContext *rw, + const H266RawSPS *sps, + const H266RawPPS *pps, + const H266RefPicLists *ref_lists, + uint8_t num_ref_idx_active[2], + H266RawPredWeightTable *current) +{ + int err, i, j; + uint8_t num_weights_l0, num_weights_l1; + ue(luma_log2_weight_denom, 0, 7); + if (sps->sps_chroma_format_idc != 0) { + se(delta_chroma_log2_weight_denom, + -current->luma_log2_weight_denom, 7 - current->luma_log2_weight_denom); + } else { + infer(delta_chroma_log2_weight_denom, 0); + } + if (pps->pps_wp_info_in_ph_flag) + ue(num_l0_weights, 0, FFMIN(15, ref_lists->rpl_ref_list[0].num_ref_entries)); + else + infer(num_l0_weights, 0); + num_weights_l0 = pps->pps_wp_info_in_ph_flag ? + current->num_l0_weights : num_ref_idx_active[0]; + for (i = 0; i < num_weights_l0; i++) { + flags(luma_weight_l0_flag[i], 1, i); + } + if (sps->sps_chroma_format_idc != 0) { + for (i = 0; i < num_weights_l0; i++) + flags(chroma_weight_l0_flag[i], 1, i); + } + for (i = 0; i < num_weights_l0; i++) { + if (current->luma_weight_l0_flag[i]){ + ses(delta_luma_weight_l0[i], -128, 127, 1, i); + ses(luma_offset_l0[i], -128, 127, 1, i); + } else { + infer(delta_luma_weight_l0[i], 0); + infer(luma_offset_l0[i], 0); + } + if (current->chroma_weight_l0_flag[i]) { + for (j = 0; j < 2; j++) { + ses(delta_chroma_weight_l0[i][j], -128, 127, 2, i, j); + ses(delta_chroma_offset_l0[i][j], -4 * 128, 4 * 127, 2, i, j); + } + } + } + + if (pps->pps_weighted_bipred_flag && pps->pps_wp_info_in_ph_flag && + ref_lists->rpl_ref_list[1].num_ref_entries > 0) { + ue(num_l1_weights, 0, FFMIN(15, ref_lists->rpl_ref_list[1].num_ref_entries)); + } + if (!pps->pps_weighted_bipred_flag || + (pps->pps_wp_info_in_ph_flag && + ref_lists->rpl_ref_list[1].num_ref_entries == 0)) { + num_weights_l1 = 0; + } else if (pps->pps_wp_info_in_ph_flag) { + num_weights_l1 = current->num_l1_weights; + } else { + num_weights_l1 = num_ref_idx_active[1]; + } + + for (i = 0; i < num_weights_l1; i++) + flags(luma_weight_l1_flag[i], 1, i); + if (sps->sps_chroma_format_idc != 0) { + for (i = 0; i < num_weights_l1; i++) + flags(chroma_weight_l1_flag[i], 1, i); + } + for (i = 0; i < num_weights_l1; i++) { + if (current->luma_weight_l1_flag[i]){ + ses(delta_luma_weight_l1[i], -128, 127, 1, i); + ses(luma_offset_l1[i], -128, 127, 1, i); + } else { + infer(delta_luma_weight_l1[i], 0); + infer(luma_offset_l1[i], 0); + } + if (current->chroma_weight_l1_flag[i]) { + for (j = 0; j < 2; j++) { + ses(delta_chroma_weight_l1[i][j], -128, 127, 2, i, j); + ses(delta_chroma_offset_l1[i][j], -4 * 128, 4 * 127, 2, i, j); + } + } + } + return 0; +} + +static int FUNC(picture_header)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawPH *current) +{ + CodedBitstreamH266Context *h266 = ctx->priv_data; + const H266RawSPS *sps; + const H266RawPPS *pps; + int err, i; + unsigned int ctb_log2_size_y, min_cb_log2_size_y, + min_qt_log2_size_intra_y, min_qt_log2_size_inter_y; + uint8_t qp_bd_offset; + + flag(ph_gdr_or_irap_pic_flag); + flag(ph_non_ref_pic_flag); + if (current->ph_gdr_or_irap_pic_flag) + flag(ph_gdr_pic_flag); + else + infer(ph_gdr_pic_flag, 0); + flag(ph_inter_slice_allowed_flag); + if (current->ph_inter_slice_allowed_flag) + flag(ph_intra_slice_allowed_flag); + else + infer(ph_intra_slice_allowed_flag, 1); + ue(ph_pic_parameter_set_id, 0, VVC_MAX_PPS_COUNT - 1); + pps = h266->pps[current->ph_pic_parameter_set_id]; + if (!pps) { + av_log(ctx->log_ctx, AV_LOG_ERROR, "PPS id %d not available.\n", + current->ph_pic_parameter_set_id); + return AVERROR_INVALIDDATA; + } + sps = h266->sps[pps->pps_seq_parameter_set_id]; + if (!pps) { + av_log(ctx->log_ctx, AV_LOG_ERROR, "SPS id %d not available.\n", + pps->pps_seq_parameter_set_id); + return AVERROR_INVALIDDATA; + } + h266->active_sps = sps; + h266->active_pps = pps; + + ub(sps->sps_log2_max_pic_order_cnt_lsb_minus4 + 4, ph_pic_order_cnt_lsb); + if (current->ph_gdr_pic_flag) + ue(ph_recovery_poc_cnt, 0, 1 << (sps->sps_log2_max_pic_order_cnt_lsb_minus4 + 4)); + + for (i = 0; i < sps->sps_num_extra_ph_bytes * 8; i++) { + if (sps->sps_extra_ph_bit_present_flag[i]) + flags(ph_extra_bit[i], 1, i); + } + if (sps->sps_poc_msb_cycle_flag) { + flag(ph_poc_msb_cycle_present_flag); + if (current->ph_poc_msb_cycle_present_flag) + ub(sps->sps_poc_msb_cycle_len_minus1 + 1, ph_poc_msb_cycle_val); + } + if (sps->sps_alf_enabled_flag && pps->pps_alf_info_in_ph_flag) { + flag(ph_alf_enabled_flag); + if (current->ph_alf_enabled_flag) { + + ub(3, ph_num_alf_aps_ids_luma); + for (i = 0; i < current->ph_num_alf_aps_ids_luma; i++) + ubs(3, ph_alf_aps_id_luma[i], 1, i); + + if (sps->sps_chroma_format_idc != 0) { + flag(ph_alf_cb_enabled_flag); + flag(ph_alf_cr_enabled_flag); + } else { + infer(ph_alf_cb_enabled_flag, 0); + infer(ph_alf_cr_enabled_flag, 0); + } + + if (current->ph_alf_cb_enabled_flag || current->ph_alf_cr_enabled_flag) { + ub(3, ph_alf_aps_id_chroma); + } + + if (sps->sps_ccalf_enabled_flag) { + flag(ph_alf_cc_cb_enabled_flag); + if (current->ph_alf_cc_cb_enabled_flag) + ub(3, ph_alf_cc_cb_aps_id); + flag(ph_alf_cc_cr_enabled_flag); + if (current->ph_alf_cc_cr_enabled_flag) + ub(3, ph_alf_cc_cr_aps_id); + } + } + } else { + infer(ph_alf_enabled_flag, 0); + } + if (sps->sps_lmcs_enabled_flag) { + flag(ph_lmcs_enabled_flag); + if (current->ph_lmcs_enabled_flag) { + ub(2, ph_lmcs_aps_id); + if (sps->sps_chroma_format_idc != 0) + flag(ph_chroma_residual_scale_flag); + else + infer(ph_chroma_residual_scale_flag, 0); + } + } else { + infer(ph_lmcs_enabled_flag, 0); + infer(ph_chroma_residual_scale_flag, 0); + } + + if (sps->sps_explicit_scaling_list_enabled_flag) { + flag(ph_explicit_scaling_list_enabled_flag); + if (current->ph_explicit_scaling_list_enabled_flag) { + //todo: check the ph_scaling_list_aps_id range, when aps ready + ub(3, ph_scaling_list_aps_id); + } + } else { + infer(ph_explicit_scaling_list_enabled_flag, 0); + } + if (sps->sps_virtual_boundaries_enabled_flag && + !sps->sps_virtual_boundaries_present_flag) { + flag(ph_virtual_boundaries_present_flag); + if (current->ph_virtual_boundaries_present_flag) { + ue(ph_num_ver_virtual_boundaries, + 0, pps->pps_pic_width_in_luma_samples <= 8 ? 0 : 3); + for (i = 0; i < current->ph_num_ver_virtual_boundaries; i++) { + ues(ph_virtual_boundary_pos_x_minus1[i], + 0, (pps->pps_pic_width_in_luma_samples + 7) / 8 - 2, 1, i); + } + ue(ph_num_hor_virtual_boundaries, + 0, pps->pps_pic_height_in_luma_samples <= 8 ? 0 : 3); + for (i = 0; i < current->ph_num_hor_virtual_boundaries; i++) { + ues(ph_virtual_boundary_pos_y_minus1[i], + 0, (pps->pps_pic_height_in_luma_samples + 7) / 8 - 2, 1, i); + } + } else { + infer(ph_num_ver_virtual_boundaries, 0); + infer(ph_num_hor_virtual_boundaries, 0); + } + } + if (pps->pps_output_flag_present_flag && !current->ph_non_ref_pic_flag) + flag(ph_pic_output_flag); + else + infer(ph_pic_output_flag, 1); + if (pps->pps_rpl_info_in_ph_flag) { + CHECK(FUNC(ref_pic_lists)(ctx, rw, ¤t->ph_ref_pic_lists)); + } + if (sps->sps_partition_constraints_override_enabled_flag) + flag(ph_partition_constraints_override_flag); + else + infer(ph_partition_constraints_override_flag, 0); + + ctb_log2_size_y = sps->sps_log2_ctu_size_minus5 + 5; + min_cb_log2_size_y = sps->sps_log2_min_luma_coding_block_size_minus2 + 2; + if (current->ph_intra_slice_allowed_flag) { + if (current->ph_partition_constraints_override_flag) { + ue(ph_log2_diff_min_qt_min_cb_intra_slice_luma, + 0, FFMIN(6, ctb_log2_size_y) - min_cb_log2_size_y); + ue(ph_max_mtt_hierarchy_depth_intra_slice_luma, + 0, 2 * (ctb_log2_size_y - min_cb_log2_size_y)); + if (current->ph_max_mtt_hierarchy_depth_intra_slice_luma != 0) { + ue(ph_log2_diff_max_bt_min_qt_intra_slice_luma, + 0, ctb_log2_size_y - min_qt_log2_size_intra_y); + ue(ph_log2_diff_max_tt_min_qt_intra_slice_luma, + 0, FFMIN(6, ctb_log2_size_y) - min_qt_log2_size_intra_y); + } else { + infer(ph_log2_diff_max_bt_min_qt_intra_slice_luma, + sps->sps_log2_diff_max_bt_min_qt_intra_slice_luma); + infer(ph_log2_diff_max_tt_min_qt_intra_slice_luma, + sps->sps_log2_diff_max_tt_min_qt_intra_slice_luma); + } + if (sps->sps_qtbtt_dual_tree_intra_flag) { + ue(ph_log2_diff_min_qt_min_cb_intra_slice_chroma, + 0, FFMIN(6, ctb_log2_size_y) - min_cb_log2_size_y); + ue(ph_max_mtt_hierarchy_depth_intra_slice_chroma, + 0, 2 * (ctb_log2_size_y - min_cb_log2_size_y)); + if (sps->sps_max_mtt_hierarchy_depth_intra_slice_chroma != 0) { + unsigned int min_qt_log2_size_intra_c = + sps->sps_log2_diff_min_qt_min_cb_intra_slice_chroma + + min_cb_log2_size_y; + ue(ph_log2_diff_max_bt_min_qt_intra_slice_chroma, + 0, FFMIN(6, ctb_log2_size_y) - min_qt_log2_size_intra_c); + ue(ph_log2_diff_max_tt_min_qt_intra_slice_chroma, + 0, FFMIN(6, ctb_log2_size_y) - min_qt_log2_size_intra_c); + } else { + infer(ph_log2_diff_max_bt_min_qt_intra_slice_chroma, + sps->sps_log2_diff_max_bt_min_qt_intra_slice_chroma); + infer(ph_log2_diff_max_tt_min_qt_intra_slice_chroma, + sps->sps_log2_diff_max_tt_min_qt_intra_slice_chroma); + } + } + } else { + infer(ph_log2_diff_min_qt_min_cb_intra_slice_luma, + sps->sps_log2_diff_min_qt_min_cb_intra_slice_luma); + infer(ph_max_mtt_hierarchy_depth_intra_slice_luma, + sps->sps_max_mtt_hierarchy_depth_intra_slice_luma); + infer(ph_log2_diff_max_bt_min_qt_intra_slice_luma, + sps->sps_log2_diff_max_bt_min_qt_intra_slice_luma); + infer(ph_log2_diff_max_tt_min_qt_intra_slice_luma, + sps->sps_log2_diff_max_tt_min_qt_intra_slice_luma); + infer(ph_log2_diff_min_qt_min_cb_intra_slice_chroma, + sps->sps_log2_diff_min_qt_min_cb_intra_slice_chroma); + infer(ph_max_mtt_hierarchy_depth_intra_slice_chroma, + sps->sps_max_mtt_hierarchy_depth_intra_slice_chroma); + infer(ph_log2_diff_max_bt_min_qt_intra_slice_chroma, + sps->sps_log2_diff_max_bt_min_qt_intra_slice_chroma); + infer(ph_log2_diff_max_tt_min_qt_intra_slice_chroma, + sps->sps_log2_diff_max_tt_min_qt_intra_slice_chroma); + } + + min_qt_log2_size_intra_y = + current->ph_log2_diff_min_qt_min_cb_intra_slice_luma + ctb_log2_size_y; + if (pps->pps_cu_qp_delta_enabled_flag) + ue(ph_cu_qp_delta_subdiv_intra_slice, 0, + 2 * ( ctb_log2_size_y - min_qt_log2_size_intra_y + + current->ph_max_mtt_hierarchy_depth_intra_slice_luma)); + else + infer(ph_cu_qp_delta_subdiv_intra_slice, 0); + + if (pps->pps_cu_chroma_qp_offset_list_enabled_flag) + ue(ph_cu_chroma_qp_offset_subdiv_intra_slice, 0, + 2 * (ctb_log2_size_y - min_qt_log2_size_intra_y + + current->ph_max_mtt_hierarchy_depth_intra_slice_luma)); + else + infer(ph_cu_chroma_qp_offset_subdiv_intra_slice, 0); + } + if (current->ph_inter_slice_allowed_flag) { + if (current->ph_partition_constraints_override_flag) { + ue(ph_log2_diff_min_qt_min_cb_inter_slice, + 0, FFMIN(6, ctb_log2_size_y) - min_cb_log2_size_y); + min_qt_log2_size_inter_y = + current->ph_log2_diff_min_qt_min_cb_inter_slice + min_cb_log2_size_y; + ue(ph_max_mtt_hierarchy_depth_inter_slice, + 0, 2 * (ctb_log2_size_y - min_cb_log2_size_y)); + if (current->ph_max_mtt_hierarchy_depth_inter_slice != 0) { + ue(ph_log2_diff_max_bt_min_qt_inter_slice, + 0, ctb_log2_size_y - min_qt_log2_size_inter_y); + ue(ph_log2_diff_max_tt_min_qt_inter_slice, + 0, FFMIN(6, ctb_log2_size_y) - min_qt_log2_size_inter_y); + } + } else { + infer(ph_log2_diff_min_qt_min_cb_inter_slice, + sps->sps_log2_diff_min_qt_min_cb_inter_slice); + min_qt_log2_size_inter_y = + current->ph_log2_diff_min_qt_min_cb_inter_slice + min_cb_log2_size_y; + infer(ph_max_mtt_hierarchy_depth_inter_slice, + sps->sps_max_mtt_hierarchy_depth_inter_slice); + infer(ph_log2_diff_max_bt_min_qt_inter_slice, + sps->sps_log2_diff_max_bt_min_qt_inter_slice); + infer(ph_log2_diff_max_tt_min_qt_inter_slice, + sps->sps_log2_diff_max_tt_min_qt_inter_slice); + } + + if (pps->pps_cu_qp_delta_enabled_flag) + ue(ph_cu_qp_delta_subdiv_inter_slice, 0, + 2 * ( ctb_log2_size_y - min_qt_log2_size_inter_y + + current->ph_max_mtt_hierarchy_depth_inter_slice)); + else + infer(ph_cu_qp_delta_subdiv_inter_slice, 0); + + if (pps->pps_cu_chroma_qp_offset_list_enabled_flag) + ue(ph_cu_chroma_qp_offset_subdiv_inter_slice, 0, + 2 * (ctb_log2_size_y - min_qt_log2_size_inter_y + + current->ph_max_mtt_hierarchy_depth_inter_slice)); + else + infer(ph_cu_chroma_qp_offset_subdiv_inter_slice, 0); + if (sps->sps_temporal_mvp_enabled_flag) { + flag(ph_temporal_mvp_enabled_flag); + if (current->ph_temporal_mvp_enabled_flag && + pps->pps_rpl_info_in_ph_flag) { + if (current->ph_ref_pic_lists.rpl_ref_list[1].num_ref_entries > 0) + flag(ph_collocated_from_l0_flag); + else + infer(ph_collocated_from_l0_flag, 1); + if ((current->ph_collocated_from_l0_flag && + current->ph_ref_pic_lists.rpl_ref_list[0].num_ref_entries > 1) || + (!current->ph_collocated_from_l0_flag && + current->ph_ref_pic_lists.rpl_ref_list[1].num_ref_entries > 1)) { + unsigned int idx = current->ph_collocated_from_l0_flag ? 0 : 1; + ue(ph_collocated_ref_idx, 0, + current->ph_ref_pic_lists.rpl_ref_list[idx].num_ref_entries - 1); + } else { + infer(ph_collocated_ref_idx, 0); + } + } + } + if (sps->sps_mmvd_fullpel_only_enabled_flag) + flag(ph_mmvd_fullpel_only_flag); + else + infer(ph_mmvd_fullpel_only_flag, 0); + if (!pps->pps_rpl_info_in_ph_flag || + current->ph_ref_pic_lists.rpl_ref_list[1].num_ref_entries > 0) { + flag(ph_mvd_l1_zero_flag); + if (sps->sps_bdof_control_present_in_ph_flag) { + flag(ph_bdof_disabled_flag); + } else { + if (!sps->sps_bdof_control_present_in_ph_flag) + infer(ph_bdof_disabled_flag, 1 - sps->sps_bdof_enabled_flag); + else + infer(ph_bdof_disabled_flag, 1); + } + if (sps->sps_dmvr_control_present_in_ph_flag) { + flag(ph_dmvr_disabled_flag); + } else { + if (!sps->sps_dmvr_control_present_in_ph_flag) + infer(ph_dmvr_disabled_flag, 1 - sps->sps_dmvr_enabled_flag); + else + infer(ph_dmvr_disabled_flag, 1); + } + } else { + infer(ph_mvd_l1_zero_flag, 1); + } + if (sps->sps_prof_control_present_in_ph_flag) + flag(ph_prof_disabled_flag); + else + infer(ph_prof_disabled_flag, !sps->sps_affine_prof_enabled_flag); + if ((pps->pps_weighted_pred_flag || + pps->pps_weighted_bipred_flag) && + pps->pps_wp_info_in_ph_flag) { + + // if pps->pps_wp_info_in_ph_fla == 1 + // pred_weight_table will not use num_ref_idx_active + uint8_t num_ref_idx_active[2] = {0, 0}; + CHECK(FUNC(pred_weight_table)(ctx, rw, sps, pps, ¤t->ph_ref_pic_lists, + num_ref_idx_active, + ¤t->ph_pred_weight_table)); + } + } + + qp_bd_offset = 6 * sps->sps_bitdepth_minus8; + if (pps->pps_qp_delta_info_in_ph_flag) + se(ph_qp_delta, -qp_bd_offset - (26 + pps->pps_init_qp_minus26), + 63 - (26 + pps->pps_init_qp_minus26)); + + if (sps->sps_joint_cbcr_enabled_flag) + flag(ph_joint_cbcr_sign_flag); + else + infer(ph_joint_cbcr_sign_flag, 0); + if (sps->sps_sao_enabled_flag && pps->pps_sao_info_in_ph_flag) { + flag(ph_sao_luma_enabled_flag); + if (sps->sps_chroma_format_idc != 0) + flag(ph_sao_chroma_enabled_flag); + else + infer(ph_sao_chroma_enabled_flag, 0); + } else { + infer(ph_sao_luma_enabled_flag, 0); + infer(ph_sao_chroma_enabled_flag, 0); + } + + if (pps->pps_dbf_info_in_ph_flag) { + flag(ph_deblocking_params_present_flag); + if (current->ph_deblocking_params_present_flag) { + if (!pps->pps_deblocking_filter_disabled_flag) { + flag(ph_deblocking_filter_disabled_flag); + } else { + if (pps->pps_deblocking_filter_disabled_flag && + current->ph_deblocking_params_present_flag) { + infer(ph_deblocking_filter_disabled_flag, 0); + } else { + infer(ph_deblocking_filter_disabled_flag, + pps->pps_deblocking_filter_disabled_flag); + } + } + if (!current->ph_deblocking_filter_disabled_flag) { + se(ph_luma_beta_offset_div2, -12, 12); + se(ph_luma_tc_offset_div2, -12, 12); + if (pps->pps_chroma_tool_offsets_present_flag) { + se(ph_cb_beta_offset_div2, -12, 12); + se(ph_cb_tc_offset_div2, -12, 12); + se(ph_cr_beta_offset_div2, -12, 12); + se(ph_cr_tc_offset_div2, -12, 12); + } else { + infer(ph_cb_beta_offset_div2, current->ph_luma_beta_offset_div2); + infer(ph_cb_tc_offset_div2, current->ph_luma_tc_offset_div2); + infer(ph_cr_beta_offset_div2, current->ph_luma_beta_offset_div2); + infer(ph_cr_tc_offset_div2, current->ph_luma_tc_offset_div2); + } + } else { + infer(ph_luma_beta_offset_div2, pps->pps_luma_beta_offset_div2); + infer(ph_luma_tc_offset_div2, pps->pps_luma_tc_offset_div2); + if (pps->pps_chroma_tool_offsets_present_flag) { + infer(ph_cb_beta_offset_div2, pps->pps_cb_beta_offset_div2); + infer(ph_cb_tc_offset_div2, pps->pps_cb_tc_offset_div2); + infer(ph_cr_beta_offset_div2, pps->pps_cr_beta_offset_div2); + infer(ph_cr_tc_offset_div2, pps->pps_cr_tc_offset_div2); + } else { + infer(ph_cb_beta_offset_div2, current->ph_luma_beta_offset_div2); + infer(ph_cb_tc_offset_div2, current->ph_luma_tc_offset_div2); + infer(ph_cr_beta_offset_div2, current->ph_luma_beta_offset_div2); + infer(ph_cr_tc_offset_div2, current->ph_luma_tc_offset_div2); + } + } + } + } + + if (pps->pps_picture_header_extension_present_flag) { + ue(ph_extension_length, 0, 256); + for (i = 0; i < current->ph_extension_length; i++) + us(8, ph_extension_data_byte[i], 0x00, 0xff, 1, i); + } + + return 0; +} + +static int FUNC(ph)(CodedBitstreamContext *ctx, RWContext *rw, H266RawPH *current) +{ + int err; + + HEADER("Picture Header"); + + CHECK(FUNC(nal_unit_header)(ctx, rw, ¤t->nal_unit_header, VVC_PH_NUT)); + CHECK(FUNC(picture_header)(ctx, rw, current)); + CHECK(FUNC(rbsp_trailing_bits)(ctx, rw)); + return 0; +} + +static int FUNC(slice_header)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawSliceHeader *current) +{ + CodedBitstreamH266Context *h266 = ctx->priv_data; + const H266RawSPS *sps; + const H266RawPPS *pps; + const H266RawPH *ph; + const H266RefPicLists *ref_pic_lists; + int err, i; + uint8_t nal_unit_type, qp_bd_offset; + uint16_t curr_subpic_idx; + uint16_t num_slices_in_subpic; + + HEADER("Slice Header"); + + CHECK(FUNC(nal_unit_header)(ctx, rw, ¤t->nal_unit_header, -1)); + + flag(sh_picture_header_in_slice_header_flag); + if (current->sh_picture_header_in_slice_header_flag){ + CHECK(FUNC(picture_header)(ctx, rw, ¤t->sh_picture_header)); + if (!h266->ph_ref) { + h266->ph_ref = av_buffer_allocz(sizeof(H266RawPH)); + if (!h266->ph_ref) + return AVERROR(ENOMEM); + h266->active_ph = h266->ph = (H266RawPH*)h266->ph_ref->data; + } + memcpy(h266->ph, ¤t->sh_picture_header, sizeof(H266RawPH)); + } + sps = h266->active_sps; + pps = h266->active_pps; + ph = h266->active_ph; + + if (!sps || !pps || !ph) { + av_log(ctx->log_ctx, AV_LOG_ERROR, "no sps(%p)/pps(%p)/ph(%p).\n", sps, pps, ph); + return AVERROR_INVALIDDATA; + } + + if (sps->sps_subpic_info_present_flag) { + ub(sps->sps_subpic_id_len_minus1 + 1, sh_subpic_id); + if (sps->sps_subpic_id_mapping_explicitly_signalled_flag) { + for (i = 0; i <= sps->sps_num_subpics_minus1; i++) { + uint16_t subpic_id_val = pps->pps_subpic_id_mapping_present_flag ? + pps->pps_subpic_id[i] : sps->sps_subpic_id[i]; + if (subpic_id_val == current->sh_subpic_id) { + curr_subpic_idx = i; + break; + } + } + } else { + curr_subpic_idx = current->sh_subpic_id; + if (curr_subpic_idx > sps->sps_num_subpics_minus1) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "sh_subpic_id(%d) should in range [0, %d]\n", + curr_subpic_idx, sps->sps_num_subpics_minus1); + return AVERROR_INVALIDDATA; + } + } + } else { + curr_subpic_idx = 0; + } + + num_slices_in_subpic = pps->num_slices_in_subpic[curr_subpic_idx]; + + if ((pps->pps_rect_slice_flag && num_slices_in_subpic > 1) || + (!pps->pps_rect_slice_flag && pps->num_tiles_in_pic > 1)) { + unsigned int bits, max; + if (!pps->pps_rect_slice_flag) { + bits = av_ceil_log2(pps->num_tiles_in_pic); + max = pps->num_tiles_in_pic - 1; + } else { + bits = av_ceil_log2(num_slices_in_subpic); + max = num_slices_in_subpic - 1; + } + u(bits, sh_slice_address, 0, max); + } else { + infer(sh_slice_address, 0); + } + + for (i = 0; i < sps->sps_num_extra_sh_bytes * 8; i++) { + if (sps->sps_extra_sh_bit_present_flag[i]) + flags(sh_extra_bit[i], 1, i); + } + + if (!pps->pps_rect_slice_flag && + pps->num_tiles_in_pic - current->sh_slice_address > 1) + ue(sh_num_tiles_in_slice_minus1, 0, pps->num_tiles_in_pic - 1); + else + infer(sh_num_tiles_in_slice_minus1, 0); + + if (ph->ph_inter_slice_allowed_flag) + ue(sh_slice_type, 0, 2); + else + infer(sh_slice_type, 2); + + nal_unit_type = current->nal_unit_header.nal_unit_type; + if (nal_unit_type == VVC_IDR_W_RADL || + nal_unit_type == VVC_IDR_N_LP || + nal_unit_type == VVC_CRA_NUT || + nal_unit_type == VVC_GDR_NUT) + flag(sh_no_output_of_prior_pics_flag); + if (sps->sps_alf_enabled_flag && !pps->pps_alf_info_in_ph_flag) { + flag(sh_alf_enabled_flag); + if (current->sh_alf_enabled_flag) { + ub(3, sh_num_alf_aps_ids_luma); + for (i = 0; i < current->sh_num_alf_aps_ids_luma; i++) + ubs(3, sh_alf_aps_id_luma[i], 1, i); + if (sps->sps_chroma_format_idc != 0) { + flag(sh_alf_cb_enabled_flag); + flag(sh_alf_cr_enabled_flag); + } else { + infer(sh_alf_cb_enabled_flag, ph->ph_alf_cb_enabled_flag); + infer(sh_alf_cr_enabled_flag, ph->ph_alf_cr_enabled_flag); + } + if (current->sh_alf_cb_enabled_flag || + current->sh_alf_cr_enabled_flag) + ub(3, sh_alf_aps_id_chroma); + else + infer(sh_alf_aps_id_chroma, ph->ph_alf_aps_id_chroma); + if (sps->sps_ccalf_enabled_flag) { + flag(sh_alf_cc_cb_enabled_flag); + if (current->sh_alf_cc_cb_enabled_flag) + ub(3, sh_alf_cc_cb_aps_id); + else + infer(sh_alf_cc_cb_aps_id, ph->ph_alf_cc_cb_aps_id); + flag(sh_alf_cc_cr_enabled_flag); + if (current->sh_alf_cc_cr_enabled_flag) + ub(3, sh_alf_cc_cr_aps_id); + else + infer(sh_alf_cc_cr_aps_id, ph->ph_alf_cc_cr_aps_id); + } else { + infer(sh_alf_cc_cb_enabled_flag, ph->ph_alf_cc_cb_enabled_flag); + infer(sh_alf_cc_cr_enabled_flag, ph->ph_alf_cc_cr_enabled_flag); + infer(sh_alf_cc_cb_aps_id, ph->ph_alf_cc_cb_aps_id); + infer(sh_alf_cc_cr_aps_id, ph->ph_alf_cc_cr_aps_id); + } + } + } else { + infer(sh_alf_enabled_flag, 0); + } + + if (ph->ph_lmcs_enabled_flag && + !current->sh_picture_header_in_slice_header_flag) + flag(sh_lmcs_used_flag); + else + infer(sh_lmcs_used_flag, 0); + + if (ph->ph_explicit_scaling_list_enabled_flag && + !current->sh_picture_header_in_slice_header_flag) + flag(sh_explicit_scaling_list_used_flag); + else + infer(sh_explicit_scaling_list_used_flag, 0); + + if (!pps->pps_rpl_info_in_ph_flag && + ((nal_unit_type != VVC_IDR_W_RADL && + nal_unit_type != VVC_IDR_N_LP) || + sps->sps_idr_rpl_present_flag)) { + CHECK(FUNC(ref_pic_lists)(ctx, rw, ¤t->sh_ref_pic_lists)); + ref_pic_lists = ¤t->sh_ref_pic_lists; + } else { + ref_pic_lists = &h266->ph->ph_ref_pic_lists; + } + if ((current->sh_slice_type != VVC_SLICE_TYPE_I && + ref_pic_lists->rpl_ref_list[0].num_ref_entries > 1) || + (current->sh_slice_type == VVC_SLICE_TYPE_B && + ref_pic_lists->rpl_ref_list[1].num_ref_entries > 1)) { + flag(sh_num_ref_idx_active_override_flag); + if (current->sh_num_ref_idx_active_override_flag) { + for (i = 0; i < (current->sh_slice_type == VVC_SLICE_TYPE_B ? 2: 1); i++) + if (ref_pic_lists->rpl_ref_list[i].num_ref_entries > 1) + ues(sh_num_ref_idx_active_minus1[i], 0, 14, 1, i); + else + infer(sh_num_ref_idx_active_minus1[i], 0); + } + } else { + infer(sh_num_ref_idx_active_override_flag, 1); + } + if (current->sh_slice_type != VVC_SLICE_TYPE_I) { + if (pps->pps_cabac_init_present_flag) + flag(sh_cabac_init_flag); + else + infer(sh_cabac_init_flag, 0); + if (ph->ph_temporal_mvp_enabled_flag && !pps->pps_rpl_info_in_ph_flag) { + uint8_t num_ref_idx_active[2]; + for (i = 0; i < 2; i++) { + if (current->sh_slice_type == VVC_SLICE_TYPE_B || + (current->sh_slice_type == VVC_SLICE_TYPE_P && i == 0)) { + if (current->sh_num_ref_idx_active_override_flag) { + num_ref_idx_active[i] = current->sh_num_ref_idx_active_minus1[i] + 1; + } + else { + num_ref_idx_active[i] = + FFMIN(ref_pic_lists->rpl_ref_list[i].num_ref_entries, + pps->pps_num_ref_idx_default_active_minus1[i] + 1); + } + } else { + num_ref_idx_active[i] = 0; + } + } + + if (current->sh_slice_type == VVC_SLICE_TYPE_B) + flag(sh_collocated_from_l0_flag); + else + infer(sh_collocated_from_l0_flag, 1); + if ((current->sh_collocated_from_l0_flag && num_ref_idx_active[0] > 1) || + (!current->sh_collocated_from_l0_flag && num_ref_idx_active[1] > 1)) { + unsigned int idx = current->sh_collocated_from_l0_flag ? 0 : 1; + ue(sh_collocated_ref_idx, 0, num_ref_idx_active[idx] - 1); + } else { + infer(sh_collocated_ref_idx, 0); + } + if (!pps->pps_wp_info_in_ph_flag && + ((pps->pps_weighted_pred_flag && + current->sh_slice_type == VVC_SLICE_TYPE_P) || + (pps->pps_weighted_bipred_flag && + current->sh_slice_type == VVC_SLICE_TYPE_B))) { + CHECK(FUNC(pred_weight_table)(ctx, rw, sps, pps, ref_pic_lists, + num_ref_idx_active, + ¤t->sh_pred_weight_table)); + } + } + + } + qp_bd_offset = 6 * sps->sps_bitdepth_minus8; + if (!pps->pps_qp_delta_info_in_ph_flag) + se(sh_qp_delta, -qp_bd_offset - (26 + pps->pps_init_qp_minus26), + 63 - (26 + pps->pps_init_qp_minus26)); + if (pps->pps_slice_chroma_qp_offsets_present_flag) { + int8_t off; + + se(sh_cb_qp_offset, -12, 12); + off = pps->pps_cb_qp_offset + current->sh_cb_qp_offset; + if (off < -12 || off > 12) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "pps_cb_qp_offset + sh_cb_qp_offset (%d) not in range [-12, 12].\n", + off); + return AVERROR_INVALIDDATA; + } + + se(sh_cr_qp_offset, -12, 12); + off = pps->pps_cr_qp_offset + current->sh_cr_qp_offset; + if (off < -12 || off > 12) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "pps_cr_qp_offset + sh_cr_qp_offset (%d) not in range [-12, 12].\n", + off); + return AVERROR_INVALIDDATA; + } + + if (sps->sps_joint_cbcr_enabled_flag) { + se(sh_joint_cbcr_qp_offset, -12, 12); + off = pps->pps_joint_cbcr_qp_offset_value + current->sh_joint_cbcr_qp_offset; + if (off < -12 || off > 12) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "pps_joint_cbcr_qp_offset_value + sh_joint_cbcr_qp_offset (%d)" + "not in range [-12, 12]. \n", + off); + return AVERROR_INVALIDDATA; + } + } else { + infer(sh_joint_cbcr_qp_offset, 0); + } + } else { + infer(sh_cb_qp_offset, 0); + infer(sh_cr_qp_offset, 0); + infer(sh_joint_cbcr_qp_offset, 0); + } + if (pps->pps_cu_chroma_qp_offset_list_enabled_flag) + flag(sh_cu_chroma_qp_offset_enabled_flag); + else + infer(sh_cu_chroma_qp_offset_enabled_flag, 0); + if (sps->sps_sao_enabled_flag && !pps->pps_sao_info_in_ph_flag) { + flag(sh_sao_luma_used_flag); + if (sps->sps_chroma_format_idc != 0) + flag(sh_sao_chroma_used_flag); + else + infer(sh_sao_chroma_used_flag, ph->ph_sao_chroma_enabled_flag); + } else { + infer(sh_sao_luma_used_flag, ph->ph_sao_luma_enabled_flag); + infer(sh_sao_chroma_used_flag, ph->ph_sao_chroma_enabled_flag); + } + + if (pps->pps_deblocking_filter_override_enabled_flag && + !pps->pps_dbf_info_in_ph_flag) + flag(sh_deblocking_params_present_flag); + else + infer(sh_deblocking_params_present_flag, 0); + if (current->sh_deblocking_params_present_flag) { + if (!pps->pps_deblocking_filter_disabled_flag) + flag(sh_deblocking_filter_disabled_flag); + else + infer(sh_deblocking_filter_disabled_flag, + !(pps->pps_deblocking_filter_disabled_flag && + current->sh_deblocking_params_present_flag)); + if (!current->sh_deblocking_filter_disabled_flag) { + se(sh_luma_beta_offset_div2, -12, 12); + se(sh_luma_tc_offset_div2, -12, 12); + if (pps->pps_chroma_tool_offsets_present_flag) { + se(sh_cb_beta_offset_div2, -12, 12); + se(sh_cb_tc_offset_div2, -12, 12); + se(sh_cr_beta_offset_div2, -12, 12); + se(sh_cr_tc_offset_div2, -12, 12); + } else { + infer(sh_cb_beta_offset_div2, current->sh_luma_beta_offset_div2); + infer(sh_cb_tc_offset_div2, current->sh_luma_tc_offset_div2); + infer(sh_cr_beta_offset_div2, current->sh_luma_beta_offset_div2); + infer(sh_cr_tc_offset_div2, current->sh_luma_tc_offset_div2); + } + } else { + infer(sh_luma_beta_offset_div2, ph->ph_luma_beta_offset_div2); + infer(sh_luma_tc_offset_div2, ph->ph_luma_tc_offset_div2); + if (pps->pps_chroma_tool_offsets_present_flag) { + infer(sh_cb_beta_offset_div2, ph->ph_cb_beta_offset_div2); + infer(sh_cb_tc_offset_div2, ph->ph_cb_tc_offset_div2); + infer(sh_cr_beta_offset_div2, ph->ph_cr_beta_offset_div2); + infer(sh_cr_tc_offset_div2, ph->ph_cr_beta_offset_div2); + } else { + infer(sh_cb_beta_offset_div2, current->sh_luma_beta_offset_div2); + infer(sh_cb_tc_offset_div2, current->sh_luma_tc_offset_div2); + infer(sh_cr_beta_offset_div2, current->sh_luma_beta_offset_div2); + infer(sh_cr_tc_offset_div2, current->sh_luma_tc_offset_div2); + } + } + } + + if (sps->sps_dep_quant_enabled_flag) + flag(sh_dep_quant_used_flag); + else + infer(sh_dep_quant_used_flag, 0); + + if (sps->sps_sign_data_hiding_enabled_flag && + !current->sh_dep_quant_used_flag) + flag(sh_sign_data_hiding_used_flag); + else + infer(sh_sign_data_hiding_used_flag, 0); + + if (sps->sps_transform_skip_enabled_flag && + !current->sh_dep_quant_used_flag && + !current->sh_sign_data_hiding_used_flag) + flag(sh_ts_residual_coding_disabled_flag); + else + infer(sh_ts_residual_coding_disabled_flag, 0); + if (pps->pps_slice_header_extension_present_flag) { + ue(sh_slice_header_extension_length, 0, 256); + for (i = 0; i < current->sh_slice_header_extension_length; i++) + us(8, sh_slice_header_extension_data_byte[i], 0x00, 0xff, 1, i); + } + if (sps->sps_entry_point_offsets_present_flag) { + int num_entry_points = 0; + uint8_t entropy_sync = sps->sps_entropy_coding_sync_enabled_flag; + int height; + if (pps->pps_rect_slice_flag) { + int width_in_tiles; + int slice_idx = current->sh_slice_address; + for (i = 0; i < curr_subpic_idx; i++) { + slice_idx += pps->num_slices_in_subpic[i]; + } + width_in_tiles = + pps->pps_slice_width_in_tiles_minus1[slice_idx] + 1; + + if (entropy_sync) + height = pps->slice_height_in_ctus[slice_idx]; + else + height = pps->pps_slice_height_in_tiles_minus1[slice_idx] + 1; + + num_entry_points = width_in_tiles * height; + } else { + int tile_idx; + int tile_y; + for (tile_idx = current->sh_slice_address; + tile_idx <= current->sh_slice_address + current->sh_num_tiles_in_slice_minus1; + tile_idx++) { + tile_y = tile_idx / pps->num_tile_rows; + height = pps->pps_tile_row_height_minus1[tile_y] + 1; + num_entry_points += (entropy_sync ? height : 1); + } + } + num_entry_points--; + if (num_entry_points > VVC_MAX_ENTRY_POINTS) { + av_log(ctx->log_ctx, AV_LOG_ERROR, "Too many entry points: " + "%"PRIu16".\n", num_entry_points); + return AVERROR_PATCHWELCOME; + } + if (num_entry_points > 0) { + ue(sh_entry_offset_len_minus1, 0, 31); + for (i = 0; i < num_entry_points; i++) { + ubs(current->sh_entry_offset_len_minus1 + 1, + sh_entry_point_offset_minus1[i], 1, i); + } + } + } + CHECK(FUNC(byte_alignment)(ctx, rw)); + + return 0; +} + +static int FUNC(sei_decoded_picture_hash)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawSEIDecodedPictureHash *current) +{ + int err, c, i; + + HEADER("Decoded Picture Hash"); + + u(8, dph_sei_hash_type, 0, 2); + flag(dph_sei_single_component_flag); + fixed(7, ph_sei_reserved_zero_7bits, 0); + + for (c = 0; c < (current->dph_sei_single_component_flag ? 1 : 3); c++) { + if (current->dph_sei_hash_type == 0) { + for (i = 0; i < 16; i++) + us(8, dph_sei_picture_md5[c][i], 0x00, 0xff, 2, c, i); + } else if (current->dph_sei_hash_type == 1) { + us(16, dph_sei_picture_crc[c], 0x0000, 0xffff, 1, c); + } else if (current->dph_sei_hash_type == 2) { + us(32, dph_sei_picture_checksum[c], 0x00000000, 0xffffffff, 1, c); + } + } + return 0; +} + + +static int FUNC(sei_payload)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawSEIPayload *current, int prefix) +{ + int err, i; + int start_position, current_position; +#ifdef WRITE + int more_data = !!current->extension_data.bit_length; +#endif + +#ifdef READ + start_position = get_bits_count(rw); +#else + start_position = put_bits_count(rw); +#endif + + switch (current->payload_type) { +#define SEI_TYPE_CHECK_VALID(name, prefix_valid, suffix_valid) do { \ + if (prefix && !prefix_valid) { \ + av_log(ctx->log_ctx, AV_LOG_ERROR, "SEI type %s invalid " \ + "as prefix SEI!\n", #name); \ + return AVERROR_INVALIDDATA; \ + } \ + if (!prefix && !suffix_valid) { \ + av_log(ctx->log_ctx, AV_LOG_ERROR, "SEI type %s invalid " \ + "as suffix SEI!\n", #name); \ + return AVERROR_INVALIDDATA; \ + } \ + } while (0) +#define SEI_TYPE_N(type, prefix_valid, suffix_valid, name) \ + case VVC_SEI_TYPE_ ## type: \ + SEI_TYPE_CHECK_VALID(name, prefix_valid, suffix_valid); \ + CHECK(FUNC(sei_ ## name)(ctx, rw, ¤t->payload.name)); \ + break +#define SEI_TYPE_S(type, prefix_valid, suffix_valid, name) \ + case VVC_SEI_TYPE_ ## type: \ + SEI_TYPE_CHECK_VALID(name, prefix_valid, suffix_valid); \ + CHECK(FUNC(sei_ ## name)(ctx, rw, ¤t->payload.name, \ + ¤t->payload_size)); \ + break +#define SEI_TYPE_E(type, prefix_valid, suffix_valid, name) \ + case VVC_SEI_TYPE_ ## type: \ + SEI_TYPE_CHECK_VALID(name, prefix_valid, suffix_valid); \ + CHECK(FUNC(sei_ ## name)(ctx, rw, ¤t->payload.name, \ + ¤t->payload_size, \ + &more_data)); \ + break + + SEI_TYPE_N(DECODED_PICTURE_HASH, 0, 1, decoded_picture_hash); +#undef SEI_TYPE_N +#undef SEI_TYPE_S +#undef SEI_TYPE_E + default: + { +#ifdef READ + current->payload.other.data_length = current->payload_size; +#endif + allocate(current->payload.other.data, current->payload.other.data_length); + + for (i = 0; i < current->payload_size; i++) + xu(8, payload_byte[i], current->payload.other.data[i], 0, 255, + 1, i); + } + } + + // more_data_in_payload() +#ifdef READ + current_position = get_bits_count(rw) - start_position; + if (current_position < 8 * current->payload_size) { +#else + current_position = put_bits_count(rw) - start_position; + if (byte_alignment(rw) || more_data) { +#endif + CHECK(FUNC(payload_extension)(ctx, rw, ¤t->extension_data, + current->payload_size, current_position)); + fixed(1, bit_equal_to_one, 1); + while (byte_alignment(rw)) + fixed(1, bit_equal_to_zero, 0); + } + +#ifdef WRITE + current->payload_size = (put_bits_count(rw) - start_position) >> 3; +#endif + + return 0; +} + +static int FUNC(sei)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawSEI *current, int prefix) +{ + int err, k; + + if (prefix) + HEADER("Prefix Supplemental Enhancement Information"); + else + HEADER("Suffix Supplemental Enhancement Information"); + + CHECK(FUNC(nal_unit_header)(ctx, rw, ¤t->nal_unit_header, + prefix ? VVC_PREFIX_SEI_NUT + : VVC_SUFFIX_SEI_NUT)); + +#ifdef READ + for (k = 0; k < VVC_MAX_SEI_PAYLOADS; k++) { + uint32_t payload_type = 0; + uint32_t payload_size = 0; + uint32_t tmp; + + while (show_bits(rw, 8) == 0xff) { + fixed(8, payload_type_byte, 0xff); + payload_type += 255; + } + xu(8, payload_type_byte, tmp, 0, 254, 0); + payload_type += tmp; + + while (show_bits(rw, 8) == 0xff) { + fixed(8, payload_size_byte, 0xff); + payload_size += 255; + } + xu(8, payload_size_byte, tmp, 0, 254, 0); + payload_size += tmp; + + current->payload[k].payload_type = payload_type; + current->payload[k].payload_size = payload_size; + + current->payload_count++; + CHECK(FUNC(sei_payload)(ctx, rw, ¤t->payload[k], prefix)); + + if (!cbs_h2645_read_more_rbsp_data(rw)) + break; + } + if (k >= VVC_MAX_SEI_PAYLOADS) { + av_log(ctx->log_ctx, AV_LOG_ERROR, "Too many payloads in " + "SEI message: found %d.\n", k); + return AVERROR_INVALIDDATA; + } +#else + for (k = 0; k < current->payload_count; k++) { + PutBitContext start_state; + uint32_t tmp; + int need_size, i; + + // Somewhat clumsy: we write the payload twice when + // we don't know the size in advance. This will mess + // with trace output, but is otherwise harmless. + start_state = *rw; + need_size = !current->payload[k].payload_size; + for (i = 0; i < 1 + need_size; i++) { + *rw = start_state; + + tmp = current->payload[k].payload_type; + while (tmp >= 255) { + fixed(8, payload_type_byte, 0xff); + tmp -= 255; + } + xu(8, payload_type_byte, tmp, 0, 254, 0); + + tmp = current->payload[k].payload_size; + while (tmp >= 255) { + fixed(8, payload_type_byte, 0xff); + tmp -= 255; + } + xu(8, payload_type_byte, tmp, 0, 254, 0); + + CHECK(FUNC(sei_payload)(ctx, rw, ¤t->payload[k], prefix)); + } + } +#endif + + CHECK(FUNC(rbsp_trailing_bits)(ctx, rw)); + + return 0; +} diff --git a/libavcodec/cbs_internal.h b/libavcodec/cbs_internal.h index a392880036..118b1052d4 100644 --- a/libavcodec/cbs_internal.h +++ b/libavcodec/cbs_internal.h @@ -40,7 +40,7 @@ enum CBSContentType { enum { // Maximum number of unit types described by the same unit type // descriptor. - CBS_MAX_UNIT_TYPES = 3, + CBS_MAX_UNIT_TYPES = 4, // Maximum number of reference buffer offsets in any one unit. CBS_MAX_REF_OFFSETS = 2, // Special value used in a unit type descriptor to indicate that it @@ -204,6 +204,7 @@ int ff_cbs_write_signed(CodedBitstreamContext *ctx, PutBitContext *pbc, extern const CodedBitstreamType ff_cbs_type_av1; extern const CodedBitstreamType ff_cbs_type_h264; extern const CodedBitstreamType ff_cbs_type_h265; +extern const CodedBitstreamType ff_cbs_type_h266; extern const CodedBitstreamType ff_cbs_type_jpeg; extern const CodedBitstreamType ff_cbs_type_mpeg2; extern const CodedBitstreamType ff_cbs_type_vp9; From patchwork Sat Jan 9 07:34:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nuo Mi X-Patchwork-Id: 24856 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 8F6B844AF8E for ; Sat, 9 Jan 2021 09:35:13 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7110368A6B6; Sat, 9 Jan 2021 09:35:13 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C381768A67C for ; Sat, 9 Jan 2021 09:35:11 +0200 (EET) Received: by mail-pl1-f174.google.com with SMTP id be12so6890159plb.4 for ; Fri, 08 Jan 2021 23:35:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=63yfoVaS1RuYu8uRf39/W1AlMHHQUt8Zj7ULDJRfJ+k=; b=dwqWOQ+8B0EFYXgnJAhrwQmR1CVWrhGL5Y4pugMSFOHigeEvgRlyd3yNsKEJkr+xXe 1v+ZZnANXHp8Ul8guHXw5bkr/yt8xDZKBvnZXHNxHfQdJcGoEfE6g1cf00jGZm/HbtIo /fqkfYieAL+I2AZP+qr3TNbGSpULV5KKQuD2ObuXkCzrEmLapJTADEc0mC1ksaX7ZFpJ LwTsjUl93F2woZv3UB9bzKJ8ptD7lSF9eNuWP3MpkBKPMABMWtW4B7XS5OgVlG1OwOHs p2rs5/1Gbz+Dc+Lp4RjHfwlVDPe46G8r5umf7hIWrsFbemLPtgz94lhztp8QlnT/Xyo6 QW5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=63yfoVaS1RuYu8uRf39/W1AlMHHQUt8Zj7ULDJRfJ+k=; b=A5IVVqdoj1FXhUTLQmAFD92+Dqr/y8pz6OWlWlZy0td+wyvS/cTKjiBSgSg8p3XO3j CgH5bxbIR+VHGzQZtGrZP5qqMxqVkY8KbTmAIg/Yxn8g7v/btVZStHOT3M/LlzSHjfus sOXmwxwzvARR2JCeHSNySQ3CX6t32N1H1m8NhHDEB2hDMR2HXENE5y3NkJ6aUvP+WM2n AZKIrvOR1F/H7n8z+jsCqxrCicjnHJ1hibIzc4Ol966noawoBHwtk6+0KTzwa+PIqfNd IboN8GxwYTxpOaPp9rlXIT3zErM/t3CtLt+kWJJfPEMxDMJvON5gRsAAReI2htlcuLpt M4Rg== X-Gm-Message-State: AOAM533kDM11g8VuXX7dfEy9KcLK7Cy7kz2f+Xi4oO9LO52pSP8IUxC/ dQqrWIVdoqHimtmc2QcXqwTx2UiL7uVInw== X-Google-Smtp-Source: ABdhPJySRcYxKv/UODCNPWSTYFxPbst1NES09WNysC9Lnsv/M7ZKUubDFhUhwn52kmkP8D/sooAApQ== X-Received: by 2002:a17:90b:3687:: with SMTP id mj7mr7527272pjb.76.1610177709879; Fri, 08 Jan 2021 23:35:09 -0800 (PST) Received: from Guangxin-PC.localdomain (23.83.245.51.16clouds.com. [23.83.245.51]) by smtp.gmail.com with ESMTPSA id m15sm11640120pfa.72.2021.01.08.23.35.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Jan 2021 23:35:09 -0800 (PST) From: Nuo Mi To: ffmpeg-devel@ffmpeg.org Date: Sat, 9 Jan 2021 15:34:17 +0800 Message-Id: <20210109073421.23721-8-nuomi2021@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210109073421.23721-1-nuomi2021@gmail.com> References: <20210109073421.23721-1-nuomi2021@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 07/11] avcodec/h2645_parse: add nal header parser for h266/vvc X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Nuo Mi Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/h2645_parse.c | 74 ++++++++++++++++++++++++++++++++++++++-- 1 file changed, 71 insertions(+), 3 deletions(-) diff --git a/libavcodec/h2645_parse.c b/libavcodec/h2645_parse.c index a36ef4f5a0..35f9d035a9 100644 --- a/libavcodec/h2645_parse.c +++ b/libavcodec/h2645_parse.c @@ -1,5 +1,5 @@ /* - * H.264/HEVC common parsing code + * H.264/HEVC/VVC common parsing code * * This file is part of FFmpeg. * @@ -27,6 +27,7 @@ #include "libavutil/mem.h" #include "bytestream.h" +#include "vvc.h" #include "hevc.h" #include "h264.h" #include "h2645_parse.h" @@ -146,6 +147,47 @@ nsc: return si; } +static const char *const vvc_nal_type_name[32] = { + "TRAIL_NUT", // VVC_TRAIL_NUT + "STSA_NUT", // VVC_STSA_NUT + "RADL_NUT", // VVC_RADL_NUT + "RASL_NUT", // VVC_RASL_NUT + "RSV_VCL_4", // VVC_RSV_VCL_4 + "RSV_VCL_5", // VVC_RSV_VCL_5 + "RSV_VCL_6", // VVC_RSV_VCL_6 + "IDR_W_RADL", // VVC_IDR_W_RADL + "IDR_N_LP", // VVC_IDR_N_LP + "CRA_NUT", // VVC_CRA_NUT + "GDR_NUT", // VVC_GDR_NUT + "RSV_IRAP_11", // VVC_RSV_IRAP_11 + "OPI_NUT", // VVC_OPI_NUT + "DCI_NUT", // VVC_DCI_NUT + "VPS_NUT", // VVC_VPS_NUT + "SPS_NUT", // VVC_SPS_NUT + "PPS_NUT", // VVC_PPS_NUT + "PREFIX_APS_NUT",// VVC_PREFIX_APS_NUT + "SUFFIX_APS_NUT",// VVC_SUFFIX_APS_NUT + "PH_NUT", // VVC_PH_NUT + "AUD_NUT", // VVC_AUD_NUT + "EOS_NUT", // VVC_EOS_NUT + "EOB_NUT", // VVC_EOB_NUT + "PREFIX_SEI_NUT",// VVC_PREFIX_SEI_NUT + "SUFFIX_SEI_NUT",// VVC_SUFFIX_SEI_NUT + "FD_NUT", // VVC_FD_NUT + "RSV_NVCL_26", // VVC_RSV_NVCL_26 + "RSV_NVCL_27", // VVC_RSV_NVCL_27 + "UNSPEC_28", // VVC_UNSPEC_28 + "UNSPEC_29", // VVC_UNSPEC_29 + "UNSPEC_30", // VVC_UNSPEC_30 + "UNSPEC_31", // VVC_UNSPEC_31 +}; + +static const char *vvc_nal_unit_name(int nal_type) +{ + av_assert0(nal_type >= 0 && nal_type < 32); + return vvc_nal_type_name[nal_type]; +} + static const char *const hevc_nal_type_name[64] = { "TRAIL_N", // HEVC_NAL_TRAIL_N "TRAIL_R", // HEVC_NAL_TRAIL_R @@ -289,6 +331,31 @@ static int get_bit_length(H2645NAL *nal, int skip_trailing_zeros) * @return AVERROR_INVALIDDATA if the packet is not a valid NAL unit, * 0 otherwise */ +static int vvc_parse_nal_header(H2645NAL *nal, void *logctx) +{ + GetBitContext *gb = &nal->gb; + + if (get_bits1(gb) != 0) //forbidden_zero_bit + return AVERROR_INVALIDDATA; + + skip_bits1(gb); //nuh_reserved_zero_bit + + nal->nuh_layer_id = get_bits(gb, 6); + nal->type = get_bits(gb, 5); + nal->temporal_id = get_bits(gb, 3) - 1; + if (nal->temporal_id < 0) + return AVERROR_INVALIDDATA; + + if ((nal->type >= VVC_IDR_W_RADL && nal->type <= VVC_RSV_IRAP_11) && nal->temporal_id) + return AVERROR_INVALIDDATA; + + av_log(logctx, AV_LOG_DEBUG, + "nal_unit_type: %d(%s), nuh_layer_id: %d, temporal_id: %d\n", + nal->type, vvc_nal_unit_name(nal->type), nal->nuh_layer_id, nal->temporal_id); + + return 0; +} + static int hevc_parse_nal_header(H2645NAL *nal, void *logctx) { GetBitContext *gb = &nal->gb; @@ -503,8 +570,9 @@ int ff_h2645_packet_split(H2645Packet *pkt, const uint8_t *buf, int length, /* Reset type in case it contains a stale value from a previously parsed NAL */ nal->type = 0; - - if (codec_id == AV_CODEC_ID_HEVC) + if (codec_id == AV_CODEC_ID_VVC) + ret = vvc_parse_nal_header(nal, logctx); + else if (codec_id == AV_CODEC_ID_HEVC) ret = hevc_parse_nal_header(nal, logctx); else ret = h264_parse_nal_header(nal, logctx); From patchwork Sat Jan 9 07:34:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nuo Mi X-Patchwork-Id: 24857 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 72FFF44AF8E for ; Sat, 9 Jan 2021 09:35:17 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5CCD468A682; Sat, 9 Jan 2021 09:35:17 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf1-f176.google.com (mail-pf1-f176.google.com [209.85.210.176]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id BEA2F6805D7 for ; Sat, 9 Jan 2021 09:35:15 +0200 (EET) Received: by mail-pf1-f176.google.com with SMTP id c13so5031760pfi.12 for ; Fri, 08 Jan 2021 23:35:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/lX7+kdK6XYlabF6JYw6jMxVFcnwoml2rXkuZK6K1pA=; b=YNXkP2cJJwCFrpZdGzzju+PU0GqEINPyt2vPvaXWGx121+yD4puyPKjMrEfn2ppdpl NPTFqXKj3SU1vVsnvGuF6jEvphETwRhYUTr9hlHrT9wCZl2UA2J6qq2hwRGJRRKWVuST Y1y29QE4iQCq2zD1/pbHbBQajTZLoEqTeSMfg9fzn6MnISEj/E0tZpl37ckijaTf+KXV xlqJiNwsj2nFPEtJi6n13R5MtfkE6RYnD+MUMaRew6ZMocufQVMh4bhsKAmjFtSJrXx3 sOS2f8ItuCnMuUQZ5GTEkqKe4DS51TA5NUu3Nf8M+zx20RWkKm8qiAC028vC84lzNioB /sWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/lX7+kdK6XYlabF6JYw6jMxVFcnwoml2rXkuZK6K1pA=; b=GZ7ew3J8Mw+sb0CQiM2Vx6pRmlcWlFcRLhDUTmLcFrRG+WIf+266IUzkEm2hC4rz4L Zu0c9xB6yDBxm5RD9CTFR3HEywaTm1cTqEuNpmVkYz+dso8WvkLTrOD0cMlwZiiQb+vM VVTc+X6FvLWLv7js+oGBsyp2D0u3yj97vlkhuT9gkykLvvZ4Gs31V+ZqQn3mB8gUoO+r tjRFk4ftlQP6xzBIWu0edNZIoYPJWpWZUtNvS7G5Blh4q30UGeYiZWxV2Zf2073QRdK6 HtEKRwuqRT8VIqRAcL2X6JaeiYWIqutkY2Bz1NqpsMDDh3Okn4zWKXzRYCBdaswQX+n9 Aomg== X-Gm-Message-State: AOAM531ibgzvVLQdyyXilNs5c9YQpDkmIlufQQSbwQVM0Ahy/Qi0ncn2 kR5oqBqsxqkxlW/zYwlzPLByvAsvW4wFkA== X-Google-Smtp-Source: ABdhPJyJ5pJZyM879ZKbmQI+8nd7b6a5lNB4vz1n38Ar81ZLxBxLhvD75gOpD036ZRRQVJcTW6mejg== X-Received: by 2002:a63:ca51:: with SMTP id o17mr10630470pgi.314.1610177713780; Fri, 08 Jan 2021 23:35:13 -0800 (PST) Received: from Guangxin-PC.localdomain (23.83.245.51.16clouds.com. [23.83.245.51]) by smtp.gmail.com with ESMTPSA id m15sm11640120pfa.72.2021.01.08.23.35.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Jan 2021 23:35:13 -0800 (PST) From: Nuo Mi To: ffmpeg-devel@ffmpeg.org Date: Sat, 9 Jan 2021 15:34:18 +0800 Message-Id: <20210109073421.23721-9-nuomi2021@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210109073421.23721-1-nuomi2021@gmail.com> References: <20210109073421.23721-1-nuomi2021@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 08/11] avcodec: add vvc parser X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Nuo Mi Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- configure | 1 + libavcodec/Makefile | 1 + libavcodec/parsers.c | 1 + libavcodec/vvc_parser.c | 289 ++++++++++++++++++++++++++++++++++++++++ 4 files changed, 292 insertions(+) create mode 100644 libavcodec/vvc_parser.c diff --git a/configure b/configure index 4935625260..5ff743d9c2 100755 --- a/configure +++ b/configure @@ -3167,6 +3167,7 @@ mpegaudio_parser_select="mpegaudioheader" mpegvideo_parser_select="mpegvideo" mpeg4video_parser_select="h263dsp mpegvideo qpeldsp" vc1_parser_select="vc1dsp" +vcc_parser_select="cbs_h266" # bitstream_filters aac_adtstoasc_bsf_select="adts_header" diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 4b406adfce..30e9a92e9f 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1123,6 +1123,7 @@ OBJS-$(CONFIG_VC1_PARSER) += vc1_parser.o vc1.o vc1data.o \ OBJS-$(CONFIG_VP3_PARSER) += vp3_parser.o OBJS-$(CONFIG_VP8_PARSER) += vp8_parser.o OBJS-$(CONFIG_VP9_PARSER) += vp9_parser.o +OBJS-$(CONFIG_VVC_PARSER) += vvc_parser.o OBJS-$(CONFIG_WEBP_PARSER) += webp_parser.o OBJS-$(CONFIG_XMA_PARSER) += xma_parser.o diff --git a/libavcodec/parsers.c b/libavcodec/parsers.c index 83271d95a3..060c0931b6 100644 --- a/libavcodec/parsers.c +++ b/libavcodec/parsers.c @@ -70,6 +70,7 @@ extern AVCodecParser ff_vorbis_parser; extern AVCodecParser ff_vp3_parser; extern AVCodecParser ff_vp8_parser; extern AVCodecParser ff_vp9_parser; +extern AVCodecParser ff_vvc_parser; extern AVCodecParser ff_webp_parser; extern AVCodecParser ff_xma_parser; diff --git a/libavcodec/vvc_parser.c b/libavcodec/vvc_parser.c new file mode 100644 index 0000000000..3e1ab29185 --- /dev/null +++ b/libavcodec/vvc_parser.c @@ -0,0 +1,289 @@ +/* + * VVC parser + * + * Copyright (C) 2029 Nuo Mi + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "cbs.h" +#include "cbs_h266.h" +#include "internal.h" +#include "parser.h" + +#define START_CODE 0x000001 ///< start_code_prefix_one_3bytes + +#define IS_SLICE(nut) (nut <= VVC_RASL_NUT || (nut >= VVC_IDR_W_RADL && nut <= VVC_GDR_NUT)) + +typedef struct VVCParserContext { + ParseContext pc; + CodedBitstreamContext *cbc; + CodedBitstreamFragment picture_unit; + int parsed_extradata; +} VVCParserContext; + +static const enum AVPixelFormat pix_fmts_8bit[] = { + AV_PIX_FMT_GRAY8, AV_PIX_FMT_YUV420P, + AV_PIX_FMT_YUV422P, AV_PIX_FMT_YUV444P +}; + +static const enum AVPixelFormat pix_fmts_10bit[] = { + AV_PIX_FMT_GRAY10, AV_PIX_FMT_YUV420P10, + AV_PIX_FMT_YUV422P10, AV_PIX_FMT_YUV444P10 +}; + +static int get_format(const H266RawSPS* sps) +{ + switch (sps->sps_bitdepth_minus8) { + case 0: + return pix_fmts_8bit[sps->sps_chroma_format_idc]; + case 2: + return pix_fmts_10bit[sps->sps_chroma_format_idc]; + } + return AV_PIX_FMT_NONE; +} + +/** + * Find the end of the current frame in the bitstream. + * @return the position of the first byte of the next frame, or END_NOT_FOUND + */ +static int find_frame_end(AVCodecParserContext *s, const uint8_t *buf, + int buf_size) +{ + VVCParserContext *ctx = s->priv_data; + ParseContext *pc = &ctx->pc; + int i; + + for (i = 0; i < buf_size; i++) { + int nut; + + pc->state64 = (pc->state64 << 8) | buf[i]; + + if (((pc->state64 >> 3 * 8) & 0xFFFFFF) != START_CODE) + continue; + + nut = (pc->state64 >> (8 + 3)) & 0x1F; + // Beginning of access unit + if ((nut >= VVC_OPI_NUT && nut <= VVC_EOB_NUT && nut != VVC_PH_NUT) || + nut == VVC_PREFIX_SEI_NUT || + (nut >= VVC_RSV_NVCL_26 && nut <= VVC_UNSPEC_31)) { + if (pc->frame_start_found) { + pc->frame_start_found = 0; + return i - 5; + } + } else if (nut == VVC_PH_NUT || IS_SLICE(nut)) { + int sh_picture_header_in_slice_header_flag = buf[i] >> 7; + + if (nut == VVC_PH_NUT || sh_picture_header_in_slice_header_flag) { + if (!pc->frame_start_found) { + pc->frame_start_found = 1; + } else { // First slice of next frame found + pc->frame_start_found = 0; + return i - 5; + } + } + } + } + return END_NOT_FOUND; +} + +static int get_pict_type(const CodedBitstreamFragment *pu) +{ + int has_p = 0; + for (int i = 0; i < pu->nb_units; i++) { + CodedBitstreamUnit *unit = &pu->units[i]; + if (IS_SLICE(unit->type)) { + const H266RawSlice *slice = unit->content; + uint8_t type = slice->header.sh_slice_type; + if (type == VVC_SLICE_TYPE_B) { + return AV_PICTURE_TYPE_B; + } + if (type == VVC_SLICE_TYPE_P) { + has_p = 1; + } + } + } + return has_p ? AV_PICTURE_TYPE_P : AV_PICTURE_TYPE_I; +} + +/** + * Parse NAL units of found picture and decode some basic information. + * + * @param s parser context. + * @param avctx codec context. + * @param buf buffer with field/frame data. + * @param buf_size size of the buffer. + */ +static int parse_nal_units(AVCodecParserContext *ctx, const uint8_t *buf, + int buf_size, AVCodecContext *avctx) +{ + VVCParserContext *s = ctx->priv_data; + CodedBitstreamFragment *pu = &s->picture_unit; + CodedBitstreamH266Context *h266 = s->cbc->priv_data; + const H266RawSPS *sps; + const H266RawPPS *pps; + const H266RawPH *ph; + int ret, num = 0, den = 0; + static const uint8_t h266_sub_width_c[] = { + 1, 2, 2, 1 + }; + static const uint8_t h266_sub_height_c[] = { + 1, 2, 1, 1 + }; + + /* set some sane default values */ + ctx->pict_type = AV_PICTURE_TYPE_I; + ctx->key_frame = 0; + ctx->picture_structure = AV_PICTURE_STRUCTURE_FRAME; + + s->cbc->log_ctx = avctx; + + if (avctx->extradata_size && !s->parsed_extradata) { + s->parsed_extradata = 1; + + if ((ret = ff_cbs_read(s->cbc, pu, avctx->extradata, avctx->extradata_size)) < 0) + av_log(avctx, AV_LOG_WARNING, "Failed to parse extradata.\n"); + + ff_cbs_fragment_reset(pu); + } + + if ((ret = ff_cbs_read(s->cbc, pu, buf, buf_size))< 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to parse picture unit.\n"); + goto end; + } + + sps = h266->active_sps; + pps = h266->active_pps; + ph = h266->active_ph; + if (!h266->active_pps || !h266->active_ph) { + av_log(avctx, AV_LOG_ERROR, "No pss or ph header available\n"); + goto end; + } + + ctx->key_frame = ph->ph_gdr_or_irap_pic_flag; + ctx->coded_width = pps->pps_pic_width_in_luma_samples; + ctx->coded_height = pps->pps_pic_height_in_luma_samples; + ctx->width = pps->pps_pic_width_in_luma_samples - + (pps->pps_conf_win_left_offset + pps->pps_conf_win_right_offset) * + h266_sub_width_c[sps->sps_chroma_format_idc]; + ctx->height = pps->pps_pic_height_in_luma_samples - + (pps->pps_conf_win_top_offset + pps->pps_conf_win_bottom_offset) * + h266_sub_height_c[sps->sps_chroma_format_idc];; + ctx->pict_type = get_pict_type(pu); + ctx->format = get_format(sps); + avctx->profile = sps->profile_tier_level.general_profile_idc; + avctx->level = sps->profile_tier_level.general_level_idc; + + if(sps->sps_ptl_dpb_hrd_params_present_flag && sps->sps_timing_hrd_params_present_flag) { + num = sps->sps_general_timing_hrd_parameters.num_units_in_tick; + den = sps->sps_general_timing_hrd_parameters.time_scale; + } else { + goto end; + } + if (num != 0 && den != 0) + av_reduce(&avctx->framerate.den, &avctx->framerate.num, + num, den, 1 << 30); +end: + ff_cbs_fragment_reset(pu); + s->cbc->log_ctx = NULL; + return 0; +} + +static int vvc_parser_parse(AVCodecParserContext *s, AVCodecContext *avctx, + const uint8_t **poutbuf, int *poutbuf_size, + const uint8_t *buf, int buf_size) +{ + int next; + VVCParserContext *ctx = s->priv_data; + ParseContext *pc = &ctx->pc; + int is_dummy_buf = !buf_size; + const uint8_t *dummy_buf = buf; + + if (avctx->extradata && !ctx->parsed_extradata) { + av_log(avctx, AV_LOG_INFO, "extra data is not supported yet.\n"); + return AVERROR_PATCHWELCOME; + } + + if (s->flags & PARSER_FLAG_COMPLETE_FRAMES) { + next = buf_size; + } else { + next = find_frame_end(s, buf, buf_size); + if (ff_combine_frame(pc, next, &buf, &buf_size) < 0) { + *poutbuf = NULL; + *poutbuf_size = 0; + return buf_size; + } + } + + is_dummy_buf &= (dummy_buf == buf); + + if (!is_dummy_buf) + parse_nal_units(s, buf, buf_size, avctx); + + *poutbuf = buf; + *poutbuf_size = buf_size; + return next; + +} + +static const CodedBitstreamUnitType decompose_unit_types[] = { + VVC_TRAIL_NUT, + VVC_STSA_NUT, + VVC_RADL_NUT, + VVC_RASL_NUT, + VVC_IDR_W_RADL, + VVC_IDR_N_LP, + VVC_CRA_NUT, + VVC_GDR_NUT, + VVC_VPS_NUT, + VVC_SPS_NUT, + VVC_PPS_NUT, + VVC_PH_NUT, + VVC_AUD_NUT, +}; + +static av_cold int vvc_parser_init(AVCodecParserContext *ctx) +{ + VVCParserContext *s = ctx->priv_data; + int ret; + + ret = ff_cbs_init(&s->cbc, AV_CODEC_ID_VVC, NULL); + if (ret < 0) + return ret; + + s->cbc->decompose_unit_types = (CodedBitstreamUnitType *)decompose_unit_types; + s->cbc->nb_decompose_unit_types = FF_ARRAY_ELEMS(decompose_unit_types); + + return 0; +} + +static void vvc_parser_close(AVCodecParserContext *ctx) +{ + VVCParserContext *s = ctx->priv_data; + + ff_cbs_fragment_free(&s->picture_unit); + ff_cbs_close(&s->cbc); + av_freep(&s->pc.buffer); +} + +AVCodecParser ff_vvc_parser = { + .codec_ids = { AV_CODEC_ID_VVC }, + .priv_data_size = sizeof(VVCParserContext), + .parser_init = vvc_parser_init, + .parser_close = vvc_parser_close, + .parser_parse = vvc_parser_parse, +}; From patchwork Sat Jan 9 07:34:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nuo Mi X-Patchwork-Id: 24858 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 4DBCA44AF8E for ; Sat, 9 Jan 2021 09:35:21 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2B51868A711; Sat, 9 Jan 2021 09:35:21 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf1-f179.google.com (mail-pf1-f179.google.com [209.85.210.179]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 876DB68A68F for ; Sat, 9 Jan 2021 09:35:19 +0200 (EET) Received: by mail-pf1-f179.google.com with SMTP id m6so7722035pfm.6 for ; Fri, 08 Jan 2021 23:35:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5+ha6P9NBqPRZWcAsvZ4nXi0hv+nU6iKtqzdxMF54xI=; b=qUakvoPRsFpQXdMVZT7Uc4zE3tSVAVFx7H+fB2+6yD/wmoG09cFuVtip8cZSLpdgEW gBX5dkJD1031BzUDgTMbsQSBKtwqGRYN80o99xfCzNogHScTNhwI34N5eA22buQWI1io Ta25mR2KxUMWyeHKspjlKxD/KH/+08GCMWhkHsof1Sizou/NBFHEqwUSMDxBkRJC69tp c8Zusxx0FoktR4BUgeKO5ZMVY7KFIJsPzFqzAdLzidk6eDVDwo2ABinFvYmvTHKq8Gku LfIjc4ZmCfjq5rsejjfhHhIRpCz8My+Dx3BE/Cv2kxZdtekbqndAQxFlepISed8alwTj mALg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5+ha6P9NBqPRZWcAsvZ4nXi0hv+nU6iKtqzdxMF54xI=; b=iBh6JBYFggghJR1oaxvTNr9OmHUiIFrIpcdJlwCUVUBI442ov/dSY+lszjMGgsXpmU cSE+o7V59RmPbDEvSQz26Tb0wdhESckHwPRlIocgcxR5d4UwVPQAzYOq2Arrz3WU5tAW 9J45byCXO3rPX42ZBS2KXBhgF1g3k+GGylj9YAFBmRlcqJ4kg8JTZVWfWfgJUU1J6uo3 GUzfig7CeATHoxT3E4Y1pGlHkaCR3eDMcVPIv6adBsEdS1YTbyU+RJyBe4CVunBpbdyH S8MCf0CWrnZtcEQrNIfPARB1NSCDeGVUnvEGZ5tDACewuptGQOvLDLy+OvL/9JAn191l xuMg== X-Gm-Message-State: AOAM533d88tmQ2YWp7OVw5Ydqieok9QS1gJ6oshNvVT2RGavcM1jJixI kgAb/KVzhHTX1UqvG+WHUWv6XOT0tXFe2Q== X-Google-Smtp-Source: ABdhPJxfrV+dCIt/N6a0MEvP3wy8rSwrTT0XmKdNHdP8z+rnS6H9B8Wg8u1/gwRfiZnW6XiR99nnAg== X-Received: by 2002:a62:7bcd:0:b029:19e:9bb7:6db3 with SMTP id w196-20020a627bcd0000b029019e9bb76db3mr10407432pfc.34.1610177717595; Fri, 08 Jan 2021 23:35:17 -0800 (PST) Received: from Guangxin-PC.localdomain (23.83.245.51.16clouds.com. [23.83.245.51]) by smtp.gmail.com with ESMTPSA id m15sm11640120pfa.72.2021.01.08.23.35.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Jan 2021 23:35:17 -0800 (PST) From: Nuo Mi To: ffmpeg-devel@ffmpeg.org Date: Sat, 9 Jan 2021 15:34:19 +0800 Message-Id: <20210109073421.23721-10-nuomi2021@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210109073421.23721-1-nuomi2021@gmail.com> References: <20210109073421.23721-1-nuomi2021@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 09/11] avformat: add h266/vvc muxer X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Nuo Mi Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavformat/Makefile | 1 + libavformat/allformats.c | 1 + libavformat/rawenc.c | 25 +++++++++++++++++++++++++ 3 files changed, 27 insertions(+) diff --git a/libavformat/Makefile b/libavformat/Makefile index 4a5406da38..0253aa7d5a 100644 --- a/libavformat/Makefile +++ b/libavformat/Makefile @@ -564,6 +564,7 @@ OBJS-$(CONFIG_VPK_DEMUXER) += vpk.o OBJS-$(CONFIG_VPLAYER_DEMUXER) += vplayerdec.o subtitles.o OBJS-$(CONFIG_VQF_DEMUXER) += vqf.o OBJS-$(CONFIG_VVC_DEMUXER) += vvcdec.o rawdec.o +OBJS-$(CONFIG_VVC_MUXER) += rawenc.o OBJS-$(CONFIG_W64_DEMUXER) += wavdec.o w64.o pcm.o OBJS-$(CONFIG_W64_MUXER) += wavenc.o w64.o OBJS-$(CONFIG_WAV_DEMUXER) += wavdec.o pcm.o diff --git a/libavformat/allformats.c b/libavformat/allformats.c index fdbf424e31..ae9a98a18e 100644 --- a/libavformat/allformats.c +++ b/libavformat/allformats.c @@ -463,6 +463,7 @@ extern AVInputFormat ff_vpk_demuxer; extern AVInputFormat ff_vplayer_demuxer; extern AVInputFormat ff_vqf_demuxer; extern AVInputFormat ff_vvc_demuxer; +extern AVOutputFormat ff_vvc_muxer; extern AVInputFormat ff_w64_demuxer; extern AVOutputFormat ff_w64_muxer; extern AVInputFormat ff_wav_demuxer; diff --git a/libavformat/rawenc.c b/libavformat/rawenc.c index 32704f9bfd..5eb95f069c 100644 --- a/libavformat/rawenc.c +++ b/libavformat/rawenc.c @@ -372,6 +372,31 @@ AVOutputFormat ff_hevc_muxer = { }; #endif +#if CONFIG_VVC_MUXER +static int vvc_check_bitstream(struct AVFormatContext *s, const AVPacket *pkt) +{ + if (pkt->size >= 5 && AV_RB32(pkt->data) != 0x0000001 && + AV_RB24(pkt->data) != 0x000001) { + //TODO: fixed this after vvc codec defined in http://mp4ra.org/#/codecs + av_log(s, AV_LOG_ERROR, "vvc: mp4 to annexb is not supported\n"); + return AVERROR_PATCHWELCOME; + } + return 1; +} + +AVOutputFormat ff_vvc_muxer = { + .name = "vvc", + .long_name = NULL_IF_CONFIG_SMALL("raw VVC video"), + .extensions = "hevc,h266,266", + .audio_codec = AV_CODEC_ID_NONE, + .video_codec = AV_CODEC_ID_VVC, + .write_header = force_one_stream, + .write_packet = ff_raw_write_packet, + .check_bitstream = vvc_check_bitstream, + .flags = AVFMT_NOTIMESTAMPS, +}; +#endif + #if CONFIG_M4V_MUXER AVOutputFormat ff_m4v_muxer = { .name = "m4v", From patchwork Sat Jan 9 07:34:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nuo Mi X-Patchwork-Id: 24859 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 3653244AF8E for ; Sat, 9 Jan 2021 09:35:24 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1CEB868A71F; Sat, 9 Jan 2021 09:35:24 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf1-f177.google.com (mail-pf1-f177.google.com [209.85.210.177]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 10ED068A755 for ; Sat, 9 Jan 2021 09:35:22 +0200 (EET) Received: by mail-pf1-f177.google.com with SMTP id h186so7746196pfe.0 for ; Fri, 08 Jan 2021 23:35:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wClPzTSLRsJMiPJzf004FtGaIhxMtXHzJTwzWeDxPgY=; b=fVDqywLP99GUKpnSImc5E3NtH4A/uRJqcREaq5fLdfWRxUv2l84MCi5xfULP801iJ9 Qz0UZS6YeP8oTPITZh5z7l4u8XeNR/ISHSsiQORxncsD2D4Gmwfx5as9aDhtHsUC5s5g U2rbyQkSZgH7GDqXef/aiaiS5kzmmyhcxvmI7keC/4zti69SsgsUxB5zmng/KOXYIex3 JfcXKnnzdymdgXMjzEOfUVRq4SJMIkZKxJ+ucZN1Hv8W++E/zcLk7ILOAwSI9Zo040kR KHAPOgbj988wgijLzNdUlcBn7mLP6IYQHOc9Sclm8HDVuXBmsjKMhJ1t3xcs+FW+q0ci alKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wClPzTSLRsJMiPJzf004FtGaIhxMtXHzJTwzWeDxPgY=; b=pWWvPf0wF2EMQUZg88d6wpAt5eSdPul5rceA9bGxhxTFKaGsIvrejKDzKfyoFltWoa 7orx53uGFxWibp93qWmLwo9sErpRXA3Lgh3NYsLROouI7EP6MUFg6ekQf1Dq8mxArNBW 5gTy0zJwbkpW1cbE7LjsvbIC6/9B0P1H33jX4LPT3W10Y1v4St0Hp5XdmukkQSXBd+Q3 MimiYDHhiWKD5FV4yxW4k9wPMWptcOWa9dqDl0PG+9OWeHPjCofK0venBkVxvWDG16Zl gO8KirN4616qSTgm9P/FG8fthYEslJM85gUIsM7nJjjlPl/dT73FKOMB79bspObap/tX utJg== X-Gm-Message-State: AOAM530OXFLzFCDRsQkADUqQ0y5eNKQNhpde/EXad/CffsKdY9DkE66n i8rHEKf46DfRzbuMolfbhn/d3xC35dUXfQ== X-Google-Smtp-Source: ABdhPJyPIdDfrCW/Sy0/Ommx906CVIBG1lSWUibd2oXJQS2uOFlKkETM6ODBkm1kLUTAE7CxMQi8nA== X-Received: by 2002:a62:4d03:0:b029:1ac:6159:4572 with SMTP id a3-20020a624d030000b02901ac61594572mr7470384pfb.10.1610177721021; Fri, 08 Jan 2021 23:35:21 -0800 (PST) Received: from Guangxin-PC.localdomain (23.83.245.51.16clouds.com. [23.83.245.51]) by smtp.gmail.com with ESMTPSA id m15sm11640120pfa.72.2021.01.08.23.35.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Jan 2021 23:35:20 -0800 (PST) From: Nuo Mi To: ffmpeg-devel@ffmpeg.org Date: Sat, 9 Jan 2021 15:34:20 +0800 Message-Id: <20210109073421.23721-11-nuomi2021@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210109073421.23721-1-nuomi2021@gmail.com> References: <20210109073421.23721-1-nuomi2021@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 10/11] avcodec: add vvc metadata bsf X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Nuo Mi Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" use following command to test: ffmpeg -i in.bin -c:v copy -bsf vvc_metadata -f vvc out.bin 79.11%(197/249) can bit match with original clips 6.83%(17/249) are not bit match, the original clips has redundant emulation prevent bytes other all failed since vvdec failed to decode the first frame --- configure | 1 + libavcodec/Makefile | 1 + libavcodec/bitstream_filters.c | 1 + libavcodec/vvc_metadata_bsf.c | 243 +++++++++++++++++++++++++++++++++ 4 files changed, 246 insertions(+) create mode 100644 libavcodec/vvc_metadata_bsf.c diff --git a/configure b/configure index 5ff743d9c2..b41f2af151 100755 --- a/configure +++ b/configure @@ -3184,6 +3184,7 @@ mjpeg2jpeg_bsf_select="jpegtables" mpeg2_metadata_bsf_select="cbs_mpeg2" trace_headers_bsf_select="cbs" vp9_metadata_bsf_select="cbs_vp9" +vvc_metadata_bsf_select="cbs_h266" # external libraries aac_at_decoder_deps="audiotoolbox" diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 30e9a92e9f..d4cd64e43c 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1166,6 +1166,7 @@ OBJS-$(CONFIG_VP9_METADATA_BSF) += vp9_metadata_bsf.o OBJS-$(CONFIG_VP9_RAW_REORDER_BSF) += vp9_raw_reorder_bsf.o OBJS-$(CONFIG_VP9_SUPERFRAME_BSF) += vp9_superframe_bsf.o OBJS-$(CONFIG_VP9_SUPERFRAME_SPLIT_BSF) += vp9_superframe_split_bsf.o +OBJS-$(CONFIG_VVC_METADATA_BSF) += vvc_metadata_bsf.o # thread libraries OBJS-$(HAVE_LIBC_MSVCRT) += file_open.o diff --git a/libavcodec/bitstream_filters.c b/libavcodec/bitstream_filters.c index b26d6a910e..001a7bb3a4 100644 --- a/libavcodec/bitstream_filters.c +++ b/libavcodec/bitstream_filters.c @@ -60,6 +60,7 @@ extern const AVBitStreamFilter ff_vp9_metadata_bsf; extern const AVBitStreamFilter ff_vp9_raw_reorder_bsf; extern const AVBitStreamFilter ff_vp9_superframe_bsf; extern const AVBitStreamFilter ff_vp9_superframe_split_bsf; +extern const AVBitStreamFilter ff_vvc_metadata_bsf; #include "libavcodec/bsf_list.c" diff --git a/libavcodec/vvc_metadata_bsf.c b/libavcodec/vvc_metadata_bsf.c new file mode 100644 index 0000000000..a7deced47c --- /dev/null +++ b/libavcodec/vvc_metadata_bsf.c @@ -0,0 +1,243 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/common.h" +#include "libavutil/opt.h" + +#include "bsf.h" +#include "bsf_internal.h" +#include "cbs.h" +#include "cbs_h266.h" +#include "vvc.h" + +enum { + PASS, + INSERT, + REMOVE, +}; + +typedef struct VVCMetadataContext { + const AVClass *class; + + CodedBitstreamContext *input; + CodedBitstreamContext *output; + CodedBitstreamFragment access_unit; + + H266RawAUD aud_nal; + + int aud; + +} VVCMetadataContext; + +static int vvc_metadata_update_side_data(AVBSFContext *bsf, AVPacket *pkt) +{ + VVCMetadataContext *ctx = bsf->priv_data; + CodedBitstreamFragment *au = &ctx->access_unit; + uint8_t *side_data; + int side_data_size; + int err; + + side_data = av_packet_get_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA, + &side_data_size); + if (!side_data_size) + return 0; + + err = ff_cbs_read(ctx->input, au, side_data, side_data_size); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to read extradata from packet side data.\n"); + return err; + } + + err = ff_cbs_write_fragment_data(ctx->output, au); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to write extradata into packet side data.\n"); + return err; + } + + side_data = av_packet_new_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA, au->data_size); + if (!side_data) + return AVERROR(ENOMEM); + memcpy(side_data, au->data, au->data_size); + + ff_cbs_fragment_reset(au); + + return 0; +} + +static int vvc_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) +{ + VVCMetadataContext *ctx = bsf->priv_data; + CodedBitstreamFragment *au = &ctx->access_unit; + int err, i; + + err = ff_bsf_get_packet_ref(bsf, pkt); + if (err < 0) + return err; + + err = vvc_metadata_update_side_data(bsf, pkt); + if (err < 0) + goto fail; + + err = ff_cbs_read_packet(ctx->input, au, pkt); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to read packet.\n"); + goto fail; + } + + if (au->nb_units == 0) { + av_log(bsf, AV_LOG_ERROR, "No NAL units in packet.\n"); + err = AVERROR_INVALIDDATA; + goto fail; + } + + // If an AUD is present, it must be the first NAL unit. + if (au->units[0].type == VVC_AUD_NUT) { + if (ctx->aud == REMOVE) + ff_cbs_delete_unit(au, 0); + } else { + if (ctx->aud == INSERT) { + H266RawAUD *aud = &ctx->aud_nal; + int pic_type = 0, temporal_id = 8, layer_id = 0; + CodedBitstreamH266Context *h266 = ctx->input->priv_data; + for (i = 0; i < au->nb_units; i++) { + const H266RawNALUnitHeader *nal = au->units[i].content; + if (!nal) + continue; + if (nal->nuh_temporal_id_plus1 < temporal_id + 1) + temporal_id = nal->nuh_temporal_id_plus1 - 1; + + if (au->units[i].type <= VVC_RSV_IRAP_11) { + const H266RawSlice *slice = au->units[i].content; + layer_id = nal->nuh_layer_id; + if (slice->header.sh_slice_type == VVC_SLICE_TYPE_B && + pic_type < 2) + pic_type = 2; + if (slice->header.sh_slice_type == VVC_SLICE_TYPE_P && + pic_type < 1) + pic_type = 1; + } + } + + aud->nal_unit_header = (H266RawNALUnitHeader) { + .nal_unit_type = VVC_AUD_NUT, + .nuh_layer_id = layer_id, + .nuh_temporal_id_plus1 = temporal_id + 1, + }; + aud->aud_pic_type = pic_type; + aud->aud_irap_or_gdr_flag = h266->active_ph->ph_gdr_or_irap_pic_flag; + + err = ff_cbs_insert_unit_content(au, 0, VVC_AUD_NUT, aud, NULL); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to insert AUD.\n"); + goto fail; + } + } + } + + err = ff_cbs_write_packet(ctx->output, pkt, au); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to write packet.\n"); + goto fail; + } + +fail: + ff_cbs_fragment_reset(au); + + if (err < 0) + av_packet_unref(pkt); + + return err; +} + +static int vvc_metadata_init(AVBSFContext *bsf) +{ + VVCMetadataContext *ctx = bsf->priv_data; + CodedBitstreamFragment *au = &ctx->access_unit; + int err; + + err = ff_cbs_init(&ctx->input, AV_CODEC_ID_VVC, bsf); + if (err < 0) + return err; + err = ff_cbs_init(&ctx->output, AV_CODEC_ID_VVC, bsf); + if (err < 0) + return err; + + if (bsf->par_in->extradata) { + err = ff_cbs_read_extradata(ctx->input, au, bsf->par_in); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to read extradata.\n"); + goto fail; + } + + err = ff_cbs_write_extradata(ctx->output, bsf->par_out, au); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to write extradata.\n"); + goto fail; + } + } + +fail: + ff_cbs_fragment_reset(au); + return err; +} + +static void vvc_metadata_close(AVBSFContext *bsf) +{ + VVCMetadataContext *ctx = bsf->priv_data; + + ff_cbs_fragment_free(&ctx->access_unit); + ff_cbs_close(&ctx->input); + ff_cbs_close(&ctx->output); +} + +#define OFFSET(x) offsetof(VVCMetadataContext, x) +#define FLAGS (AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_BSF_PARAM) +static const AVOption vvc_metadata_options[] = { + { "aud", "Access Unit Delimiter NAL units", + OFFSET(aud), AV_OPT_TYPE_INT, + { .i64 = PASS }, PASS, REMOVE, FLAGS, "aud" }, + { "pass", NULL, 0, AV_OPT_TYPE_CONST, + { .i64 = PASS }, .flags = FLAGS, .unit = "aud" }, + { "insert", NULL, 0, AV_OPT_TYPE_CONST, + { .i64 = INSERT }, .flags = FLAGS, .unit = "aud" }, + { "remove", NULL, 0, AV_OPT_TYPE_CONST, + { .i64 = REMOVE }, .flags = FLAGS, .unit = "aud" }, + + { NULL } +}; + +static const AVClass vvc_metadata_class = { + .class_name = "vvc_metadata_bsf", + .item_name = av_default_item_name, + .option = vvc_metadata_options, + .version = LIBAVUTIL_VERSION_INT, +}; + +static const enum AVCodecID vvc_metadata_codec_ids[] = { + AV_CODEC_ID_VVC, AV_CODEC_ID_NONE, +}; + +const AVBitStreamFilter ff_vvc_metadata_bsf = { + .name = "vvc_metadata", + .priv_data_size = sizeof(VVCMetadataContext), + .priv_class = &vvc_metadata_class, + .init = &vvc_metadata_init, + .close = &vvc_metadata_close, + .filter = &vvc_metadata_filter, + .codec_ids = vvc_metadata_codec_ids, +}; From patchwork Sat Jan 9 07:34:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nuo Mi X-Patchwork-Id: 24860 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 4D7B144AF8E for ; Sat, 9 Jan 2021 09:35:29 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2361A68A760; Sat, 9 Jan 2021 09:35:29 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D3E8168A68F for ; Sat, 9 Jan 2021 09:35:27 +0200 (EET) Received: by mail-pf1-f181.google.com with SMTP id q20so4448339pfu.8 for ; Fri, 08 Jan 2021 23:35:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=69pu1zFGtZ/ejEbHhyh5vY+d00MC95wa1KJ7i99yJys=; b=VYho7zaFER/zsziUNodLZMsKxdeElEKwT/sk9mQjK2PHVFLL9wCbHMuRhxaThrrh+8 4xkd56mh5PYHFgpdcQg8wiecvUZ2bm/DwOuFvKfM5tw61jD6azkPaQcji3jFwnoWsDhg JzX0La6Xg/h8j9PHrGZPzneWL+yfqpd/ilXHBtF+ySSnhw9QWO7gF+/B3BPFKV8N1trj ObUDRFr7+cspNgcWVyYIx60Qe3FgJvwGkYX20zbQ8bgwtlGgWc9m7vKAxYcgX3ZDuut4 uXouxdx/9JIR4wo7yNXx+HkB+4ga/ifveDsXNYcXuzNJRqTVGXUNCINGa9LB7+ctlbgG 5Bdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=69pu1zFGtZ/ejEbHhyh5vY+d00MC95wa1KJ7i99yJys=; b=FuTo4L8HNkwRqStrKqePNtCq2uEm/zKvTkU38lO++6Fl6pWZjLOPn2TdJldJGOhhqq LBity+U8fyd5+rlq3CHgPLvPd/pPQo6k7VEaLdx8c6e7ZRHOXclld5g6r87OpjvW+lBV XUe2uE8H8drEXzhwRqYBmMBBjwJ8zx3xjlM/IwHae/fb31IfEAPhsFjZ8jkuYatODvM7 7BFbdDqrvIzE3EJ9iIRzVVv85qOz8KkEWhrXfZO+VqbSxFz5EFE1lJqJBLFarHWWXFX6 r6EFyvjyX4cpmObLFcP405q1bdzDEKFNQmDmxcfIpj34diUmDD29AhIxihUrLo8b2Qy6 sgkg== X-Gm-Message-State: AOAM531+O9xHnDZz2Rr2o95Qrrn5x0nr5zvjQs2u9hP9HGHjCDirv3dU 56ldORzt1aXt2KxT2FUTUS4Re9I6y4ARqw== X-Google-Smtp-Source: ABdhPJwhxxWi8i2CTUQJeBxTQrBnEwgRFCi1lVH2h6MI/cNb0DmAEXtAt5hQGWOGZasZh4+InWhk3A== X-Received: by 2002:a63:4764:: with SMTP id w36mr10572262pgk.127.1610177725616; Fri, 08 Jan 2021 23:35:25 -0800 (PST) Received: from Guangxin-PC.localdomain (23.83.245.51.16clouds.com. [23.83.245.51]) by smtp.gmail.com with ESMTPSA id m15sm11640120pfa.72.2021.01.08.23.35.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Jan 2021 23:35:25 -0800 (PST) From: Nuo Mi To: ffmpeg-devel@ffmpeg.org Date: Sat, 9 Jan 2021 15:34:21 +0800 Message-Id: <20210109073421.23721-12-nuomi2021@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210109073421.23721-1-nuomi2021@gmail.com> References: <20210109073421.23721-1-nuomi2021@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 11/11] avcodec: add vvdec H.266/VVC decoder X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Nuo Mi Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" you can download test clips here: https://www.itu.int/wftp3/av-arch/jvet-site/bitstream_exchange/VVC/under_test/VTM-11.0/ 76.71% (191/249) clips are md5 matched with VTM 11: passed: 10b400_A_Bytedance_2.bit 10b400_B_Bytedance_2.bit 8b400_A_Bytedance_2.bit 8b400_B_Bytedance_2.bit 8b420_A_Bytedance_2.bit 8b420_B_Bytedance_2.bit ACTPIC_A_Huawei_3.bit ACTPIC_B_Huawei_3.bit ACTPIC_C_Huawei_3.bit AFF_A_HUAWEI_2.bit AFF_B_HUAWEI_2.bit ALF_A_Huawei_3.bit ALF_B_Huawei_3.bit ALF_C_KDDI_2.bit ALF_D_Qualcomm_2.bit AMVR_A_HHI_3.bit AMVR_B_HHI_3.bit APSALF_A_Qualcomm_2.bit APSLMCS_A_Dolby_3.bit APSLMCS_B_Dolby_3.bit APSLMCS_C_Dolby_2.bit APSMULT_A_MediaTek_3.bit APSMULT_B_MediaTek_3.bit AUD_A_Broadcom_3.bit BCW_A_MediaTek_3.bit BCW_A_MediaTek_4.bit BDOF_A_MediaTek_3.bit BDOF_A_MediaTek_4.bit BDPCM_A_Orange_2.bit CCALF_A_Sharp_3.bit CCALF_B_Sharp_3.bit CCALF_C_Sharp_3.bit CCALF_D_Sharp_3.bit CCLM_A_KDDI_1.bit CIIP_A_MediaTek_3.bit CIIP_A_MediaTek_4.bit CodingToolsSets_A_Tencent_2.bit CodingToolsSets_B_Tencent_2.bit CodingToolsSets_C_Tencent_2.bit CodingToolsSets_D_Tencent_2.bit CROP_A_Panasonic_3.bit CROP_B_Panasonic_4.bit CST_A_MediaTek_3.bit CTU_A_MediaTek_3.bit CTU_A_MediaTek_4.bit CTU_B_MediaTek_3.bit CTU_B_MediaTek_4.bit CTU_C_MediaTek_3.bit CTU_C_MediaTek_4.bit CUBEMAP_A_MediaTek_3.bit CUBEMAP_B_MediaTek_3.bit CUBEMAP_C_MediaTek_3.bit DEBLOCKING_A_Sharp_3.bit DEBLOCKING_B_Sharp_2.bit DEBLOCKING_C_Huawei_3.bit DEBLOCKING_E_Ericsson_2.bit DEBLOCKING_E_Ericsson_3.bit DEBLOCKING_F_Ericsson_1.bit DEBLOCKING_F_Ericsson_2.bit DMVR_A_Huawei_3.bit DMVR_B_KDDI_3.bit DPB_A_Sharplabs_2.bit DPB_B_Sharplabs_2.bit DQ_A_HHI_3.bit ENT444HIGHTIER_A_Sony_3.bit ENT444HIGHTIER_B_Sony_3.bit ENT444HIGHTIER_C_Sony_3.bit ENT444HIGHTIER_D_Sony_3.bit ENT444MAINTIER_A_Sony_3.bit ENT444MAINTIER_B_Sony_3.bit ENT444MAINTIER_C_Sony_3.bit ENT444MAINTIER_D_Sony_3.bit ENTHIGHTIER_A_Sony_3.bit ENTHIGHTIER_B_Sony_3.bit ENTHIGHTIER_C_Sony_3.bit ENTHIGHTIER_D_Sony_3.bit ENTMAINTIER_A_Sony_3.bit ENTMAINTIER_B_Sony_3.bit ENTMAINTIER_C_Sony_3.bit ENTMAINTIER_D_Sony_3.bit ENTROPY_A_Chipsnmedia_2.bit ENTROPY_A_Qualcomm_2.bit ENTROPY_B_Sharp_2.bit ERP_A_MediaTek_3.bit FILLER_A_Bytedance_1.bit GPM_A_Alibaba_3.bit HLG_A_NHK_2.bit HLG_B_NHK_2.bit HRD_A_Fujitsu_3.bit HRD_B_Fujitsu_2.bit IBC_A_Tencent_2.bit IBC_B_Tencent_2.bit IBC_C_Tencent_2.bit IBC_D_Tencent_2.bit IP_B_Nokia_1.bit ISP_A_HHI_3.bit ISP_B_HHI_3.bit JCCR_A_Nokia_2.bit JCCR_B_Nokia_2.bit JCCR_C_HHI_3.bit JCCR_E_Nokia_1.bit JCCR_F_Nokia_1.bit LFNST_A_LGE_3.bit LFNST_B_LGE_3.bit LFNST_C_HHI_3.bit LMCS_A_Dolby_3.bit LOSSLESS_A_HHI_3.bit LOSSLESS_B_HHI_3.bit LTRP_A_ERICSSON_2.bit MERGE_A_Qualcomm_2.bit MERGE_B_Qualcomm_2.bit MERGE_C_Qualcomm_2.bit MERGE_D_Qualcomm_2.bit MERGE_E_Qualcomm_2.bit MERGE_F_Qualcomm_2.bit MERGE_G_Qualcomm_2.bit MERGE_H_Qualcomm_2.bit MERGE_I_Qualcomm_2.bit MERGE_J_Qualcomm_2.bit MIP_A_HHI_3.bit MIP_B_HHI_3.bit MPM_A_LGE_3.bit MRLP_A_HHI_2.bit MRLP_B_HHI_2.bit MTS_A_LGE_3.bit MTS_B_LGE_3.bit MTS_LFNST_A_LGE_3.bit MTS_LFNST_B_LGE_3.bit MVCOMP_A_Sharp_2.bit PDPC_A_Qualcomm_3.bit PDPC_B_Qualcomm_3.bit PDPC_C_Qualcomm_2.bit PHSH_B_Sharp_1.bit POC_A_Nokia_1.bit POUT_A_Sharplabs_2.bit PPS_B_Bytedance_1.bit PPS_C_Bytedance_1.bit PQ_A_Dolby_1.bit PROF_A_Interdigital_3.bit PROF_B_Interdigital_3.bit PSEXT_A_Nokia_2.bit PSEXT_B_Nokia_2.bit QTBTT_A_MediaTek_3.bit QTBTT_A_MediaTek_4.bit QUANT_A_Huawei_2.bit QUANT_B_Huawei_2.bit QUANT_C_Huawei_2.bit QUANT_D_Huawei_2.bit RAP_C_HHI_1.bit RAP_D_HHI_1.bit RPL_A_ERICSSON_2.bit SAO_A_SAMSUNG_3.bit SAO_B_SAMSUNG_3.bit SAO_C_SAMSUNG_3.bit SbTMVP_A_Bytedance_3.bit SbTMVP_B_Bytedance_3.bit SBT_A_HUAWEI_2.bit SCALING_A_InterDigital_1.bit SCALING_B_InterDigital_1.bit SCALING_C_InterDigital_1.bit SDH_A_Dolby_2.bit SLICES_A_HUAWEI_2.bit SMVD_A_HUAWEI_2.bit SPS_A_Bytedance_1.bit SPS_B_Bytedance_1.bit SPS_C_Bytedance_1.bit TEMPSCAL_A_Panasonic_4.bit TEMPSCAL_C_Panasonic_3.bit TILE_A_Nokia_2.bit TILE_B_Nokia_2.bit TILE_C_Nokia_2.bit TILE_D_Nokia_2.bit TILE_E_Nokia_2.bit TILE_F_Nokia_2.bit TILE_G_Nokia_2.bit TMVP_A_Chipsnmedia_3.bit TMVP_B_Chipsnmedia_3.bit TMVP_C_Chipsnmedia_3.bit TMVP_D_Chipsnmedia_3.bit TRANS_A_Chipsnmedia_2.bit TRANS_B_Chipsnmedia_2.bit TRANS_C_Chipsnmedia_2.bit TRANS_D_Chipsnmedia_2.bit WPP_A_Sharp_3.bit WPP_B_Sharp_2.bit WP_A_InterDigital_3.bit WP_B_InterDigital_3.bit WRAP_A_InterDigital_4.bit WRAP_B_InterDigital_4.bit WRAP_C_InterDigital_4.bit WRAP_D_InterDigital_4.bit --- configure | 5 +- libavcodec/Makefile | 1 + libavcodec/allcodecs.c | 1 + libavcodec/libvvdec.cpp | 244 ++++++++++++++++++++++++++++++++++++++++ 4 files changed, 250 insertions(+), 1 deletion(-) create mode 100644 libavcodec/libvvdec.cpp diff --git a/configure b/configure index b41f2af151..cefdff75fe 100755 --- a/configure +++ b/configure @@ -285,6 +285,7 @@ External library support: --enable-libvorbis enable Vorbis en/decoding via libvorbis, native implementation exists [no] --enable-libvpx enable VP8 and VP9 de/encoding via libvpx [no] + --enable-libvvdec enable VVC video decoding via libvvdec [no] --enable-libwebp enable WebP encoding via libwebp [no] --enable-libx264 enable H.264 encoding via x264 [no] --enable-libx265 enable HEVC encoding via x265 [no] @@ -1816,6 +1817,7 @@ EXTERNAL_LIBRARY_LIST=" libvmaf libvorbis libvpx + libvvdec libwebp libxml2 libzimg @@ -3284,6 +3286,7 @@ libvpx_vp8_decoder_deps="libvpx" libvpx_vp8_encoder_deps="libvpx" libvpx_vp9_decoder_deps="libvpx" libvpx_vp9_encoder_deps="libvpx" +libvvdec_decoder_deps="libvvdec" libwebp_encoder_deps="libwebp" libwebp_anim_encoder_deps="libwebp" libx262_encoder_deps="libx262" @@ -6461,7 +6464,7 @@ enabled libvpx && { die "libvpx enabled but no supported decoders found" fi } - +enabled libvvdec && require_pkg_config libvvdec "libvvdec >= 0.1.2" vvdec/version.h VVDEC_VERSION_MAJOR enabled libwebp && { enabled libwebp_encoder && require_pkg_config libwebp "libwebp >= 0.2.0" webp/encode.h WebPGetEncoderVersion enabled libwebp_anim_encoder && check_pkg_config libwebp_anim_encoder "libwebpmux >= 0.4.0" webp/mux.h WebPAnimEncoderOptionsInit; } diff --git a/libavcodec/Makefile b/libavcodec/Makefile index d4cd64e43c..cf734144f6 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1059,6 +1059,7 @@ OBJS-$(CONFIG_LIBVPX_VP8_DECODER) += libvpxdec.o OBJS-$(CONFIG_LIBVPX_VP8_ENCODER) += libvpxenc.o OBJS-$(CONFIG_LIBVPX_VP9_DECODER) += libvpxdec.o libvpx.o OBJS-$(CONFIG_LIBVPX_VP9_ENCODER) += libvpxenc.o libvpx.o +OBJS-$(CONFIG_LIBVVDEC_DECODER) += libvvdec.o OBJS-$(CONFIG_LIBWEBP_ENCODER) += libwebpenc_common.o libwebpenc.o OBJS-$(CONFIG_LIBWEBP_ANIM_ENCODER) += libwebpenc_common.o libwebpenc_animencoder.o OBJS-$(CONFIG_LIBX262_ENCODER) += libx264.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index f00d524747..545c38f541 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -752,6 +752,7 @@ extern AVCodec ff_libvpx_vp8_encoder; extern AVCodec ff_libvpx_vp8_decoder; extern AVCodec ff_libvpx_vp9_encoder; extern AVCodec ff_libvpx_vp9_decoder; +extern AVCodec ff_libvvdec_decoder; /* preferred over libwebp */ extern AVCodec ff_libwebp_anim_encoder; extern AVCodec ff_libwebp_encoder; diff --git a/libavcodec/libvvdec.cpp b/libavcodec/libvvdec.cpp new file mode 100644 index 0000000000..4229d45701 --- /dev/null +++ b/libavcodec/libvvdec.cpp @@ -0,0 +1,244 @@ +/* + * vvdec H.266/VVC decoder + * Copyright (c) 2020 Nuo Mi + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * VVC decoder support via libvvdec + */ + +#include "vvdec/vvdec.h" + +extern "C" { + +#include "libavutil/common.h" +#include "libavutil/imgutils.h" +#include "libavutil/internal.h" + +#include "avcodec.h" +#include "h2645_parse.h" +#include "internal.h" +#include "profiles.h" + +typedef struct VVDecContext { + AVCodecContext *avctx; + vvdec::VVDec* vvdec; + + H2645Packet pkt; + int is_nalff; ///< this flag is != 0 if bitstream is encapsulated + ///< as a format defined in 14496-15 + int nal_length_size; ///< Number of bytes used for nal length (1, 2 or 4) +} VVDecContext; + +} + +static int map_error(int vret) +{ + switch(vret) + { + case vvdec::VVDEC_OK : return 0; + case vvdec::VVDEC_ERR_UNSPECIFIED: return AVERROR_UNKNOWN; + case vvdec::VVDEC_ERR_INITIALIZE: return AVERROR_BUG; + case vvdec::VVDEC_ERR_ALLOCATE: return AVERROR(ENOMEM); + case vvdec::VVDEC_NOT_ENOUGH_MEM: return AVERROR(ENOMEM); + case vvdec::VVDEC_ERR_PARAMETER: return AVERROR(EINVAL); + case vvdec::VVDEC_ERR_NOT_SUPPORTED: return AVERROR(ENOSYS); + case vvdec::VVDEC_ERR_RESTART_REQUIRED: return AVERROR_BUG; + case vvdec::VVDEC_ERR_CPU: return AVERROR(ENOSYS); + case vvdec::VVDEC_TRY_AGAIN: return AVERROR(EAGAIN); + case vvdec::VVDEC_EOF: return AVERROR(EOF); + default: return AVERROR_UNKNOWN; + } +} + +static int check_vret(VVDecContext* s, int vret) { + vvdec::VVDec* vvdec = s->vvdec; + if (vret && vret != vvdec::VVDEC_EOF) { + av_log(s->avctx, AV_LOG_ERROR, "vvdec returns error: %s\n", vvdec->getErrorMsg(vret)); + } + return map_error(vret); +} + +static const enum AVPixelFormat pix_fmts_8bit[] = { + AV_PIX_FMT_GRAY8, AV_PIX_FMT_YUV420P, + AV_PIX_FMT_YUV422P, AV_PIX_FMT_YUV444P +}; + +static const enum AVPixelFormat pix_fmts_10bit[] = { + AV_PIX_FMT_GRAY10, AV_PIX_FMT_YUV420P10, + AV_PIX_FMT_YUV422P10, AV_PIX_FMT_YUV444P10 +}; + +static AVPixelFormat get_format(const vvdec::Frame* frame) +{ + if (frame->m_eColorFormat != vvdec::VVC_CF_INVALID) { + switch (frame->m_uiBitDepth) { + case 8: + return pix_fmts_8bit[frame->m_eColorFormat]; + case 10: + return pix_fmts_10bit[frame->m_eColorFormat]; + } + } + return AV_PIX_FMT_NONE; +} + +static int set_pix_fmt(VVDecContext* s, const vvdec::Frame* frame) +{ + AVPixelFormat format = get_format(frame); + if (format == AV_PIX_FMT_NONE) { + av_log(s->avctx, AV_LOG_ERROR, + "unsupported, depth = %d, color format = %d.\n", frame->m_uiBitDepth, frame->m_eColorFormat); + return AVERROR_INVALIDDATA; + } + s->avctx->pix_fmt = format; + return 0; +} + +static int copy_to_avframe(VVDecContext* s, const vvdec::Frame* src, AVFrame *dest) +{ + AVCodecContext* avctx = s->avctx; + int i, ret, width, height; + uint8_t *data[4]; + int linesize[4]; + + ret = set_pix_fmt(s, src); + if (ret < 0) + return ret; + width = (int)src->m_uiWidth; + height = (int)src->m_uiHeight; + if (width != avctx->width || height != avctx->height) { + av_log(avctx, AV_LOG_INFO, "dimension change! %dx%d -> %dx%d\n", + avctx->width, avctx->height, width, height); + if ((ret = ff_set_dimensions(avctx, width, height)) < 0) + return ret; + } + if ((ret = ff_get_buffer(avctx, dest, 0)) < 0) + return ret; + + for (i = 0; i < 3; i++) { + const vvdec::Component& plane = src->m_cComponent[i]; + data[i] = plane.m_pucBuffer; + linesize[i] = plane.m_iStride; + } + data[3] = 0; linesize[3] = 0; + + //TODO: conformance window? + av_image_copy(dest->data, dest->linesize, (const uint8_t **)data, linesize, + avctx->pix_fmt, width, height); + return 0; +} + +static int vvdec_decode(AVCodecContext *avctx, void *data, int *got_frame, + AVPacket *avpkt) +{ + VVDecContext *s = (VVDecContext *)avctx->priv_data; + vvdec::VVDec* vvdec = s->vvdec; + AVFrame *picture = (AVFrame*)data; + int vret, ret = 0; + vvdec::Frame* frame = NULL; + + if (!avpkt->size) { + vret = vvdec->flush(&frame); + if ((ret = check_vret(s, vret)) < 0) { + goto error_out; + } + } else { + uint8_t *new_extradata; + int new_extradata_size; + new_extradata = av_packet_get_side_data(avpkt, AV_PKT_DATA_NEW_EXTRADATA, + &new_extradata_size); + if (new_extradata && new_extradata_size > 0) { + return AVERROR_PATCHWELCOME; + } + + vvdec::AccessUnit au; + au.m_pucBuffer = (unsigned char*)avpkt->data; + au.m_iUsedSize = avpkt->size; + au.m_uiCts = avpkt->pts; + au.m_bCtsValid = true; + vret = vvdec->decode(au, &frame); + if (vret && vret != vvdec::VVDEC_TRY_AGAIN) { + if ((ret = check_vret(s, vret)) < 0) { + goto error_out; + } + } + + } + if (frame) { + ret = copy_to_avframe(s, frame, picture); + if (ret < 0) + goto error_out; + *got_frame = 1; + } + return 0; +error_out: + if (frame) { + vvdec->objectUnref(frame); + } + return ret; +} + +static av_cold int vvdec_free(AVCodecContext *avctx) +{ + VVDecContext *ctx = (VVDecContext *)avctx->priv_data; + vvdec::VVDec* vvdec = ctx->vvdec; + if (vvdec) { + vvdec->uninit(); + delete vvdec; + } + return 0; +} + +static av_cold int vvdec_init(AVCodecContext *avctx) +{ + VVDecContext *s = (VVDecContext *)avctx->priv_data; + vvdec::VVDecParameter param; + vvdec::VVDec* vvdec = new vvdec::VVDec; + int vret, ret = 0; + + av_log(avctx, AV_LOG_INFO, "VVDec version: %s\n", vvdec->getVersionNumber()); + + s->avctx = avctx; + s->vvdec = vvdec; + vret = vvdec->init(param); + if ((ret = check_vret(s, vret)) < 0) { + delete vvdec; + s->vvdec = NULL; + } + return ret; +} + +extern "C" { + +AVCodec ff_libvvdec_decoder = { + .name = "libvvdec", + .long_name = NULL_IF_CONFIG_SMALL("Fraunhofer Versatile Video Decoder (VVdeC)"), + .type = AVMEDIA_TYPE_VIDEO, + .id = AV_CODEC_ID_VVC, + .capabilities = AV_CODEC_CAP_AUTO_THREADS | AV_CODEC_CAP_DR1 | AV_CODEC_CAP_DELAY, + .profiles = NULL_IF_CONFIG_SMALL(ff_vvc_profiles), + .wrapper_name = "libvvdec", + .priv_data_size = sizeof(VVDecContext), + .init = vvdec_init, + .decode = vvdec_decode, + .close = vvdec_free, +}; + +}