From patchwork Wed Oct 19 07:24:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thomas Siedel X-Patchwork-Id: 38810 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp215704pzb; Wed, 19 Oct 2022 00:26:46 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6YTiou4xbPGiWyE1JHoFv4/ebt0vvWjAlX7fEhIdNkpEe5ylLeSDuWscrV3UXOZhHyNMmf X-Received: by 2002:a05:6402:1cc1:b0:45c:3a90:9499 with SMTP id ds1-20020a0564021cc100b0045c3a909499mr6226154edb.61.1666164406408; Wed, 19 Oct 2022 00:26:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666164406; cv=none; d=google.com; s=arc-20160816; b=etw1eDPe2mwftKVqOkhYzgHUa2ev/Fkf2IZzkdZz+P/s2C0mgdpcNngn7Mnw7+HWzR kUgIltMQNtNJVDg2fJTO71hGj0aAZxxuwaw+lNh2Dy2eSooo2EX8oeBD5u/WUhVsp4l6 Zm0yYTlfuc/ELkb3ribICG3lCbXdCg2QdNLb+Mb0yXKphvd1JVE7OgDDbJtWA3auPt2z jT+NMUg4UdbiklB3sMu0bZfmVwi0dU2w9mAKumwUod+vvKvpXFEiiYDTDlmhB/fD/UJt +Qe7ckL1PEFDsCS+PMT1nj0EjUOevwOh463fj4sGT8fEKpGexZCU8Y/S3PJBFzJyc1eu 5uNQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=Hnws1IGJMmMSIIXfU9x/6pImlQeG+jQAlrN9cyWvlyI=; b=a8lm6bTEn12UsX0Fmun+Xuu9tyhK+Wutp9n77ff8fL/f3DjYyEvlBo9SGD77whmIF1 Pn/JC06f40XDnjGZTj6GOF1ivOGp0ocUspkllrJnmpfFApSMy47LEtkCh8YKSG1Vb3Td nQf06ro8mIDNVtUl8VWea0BozmTHNZhUyhF9pUTF/4VrX5xbxyWXRG9aXWnYTdyVAH0l Pitmp+AAmPBsPt2E8sjIUIcK/L5FIUGBCajEu00r/axXH55foCehnkxhielPlib//Ua4 uDvCkg7derNnmPj0d58hmJAtYgIEqhVb/E/WjI3AnlHtzynU1lff2xC2y2zET01MhNet x9fQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=tEURL9Xu; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id dm25-20020a05640222d900b00458dcea145asi12502186edb.161.2022.10.19.00.26.45; Wed, 19 Oct 2022 00:26:46 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=tEURL9Xu; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B051368BD6C; Wed, 19 Oct 2022 10:26:32 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f49.google.com (mail-ed1-f49.google.com [209.85.208.49]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id F36D168BBC9 for ; Wed, 19 Oct 2022 10:26:24 +0300 (EEST) Received: by mail-ed1-f49.google.com with SMTP id l22so23902506edj.5 for ; Wed, 19 Oct 2022 00:26:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spin-digital-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xlsOuybVen4f96SBiCIcF54dALKHxXee+JiiMWKPUxI=; b=tEURL9XuWFuNBN/t2s1qINfZrCcadSPfjKGe8JPg2IXFTlxty9UZKmkcMGaXIkAE+k K+nJaOlfQnIG8ZYuQcVgMFz9C/3b2jRuwrE20HyxYJfRsI6+4J8iC6HMNk1gzPdpKfIV 9zLBOcCe3mrnUqoXR5Kb7d4my6fP235sd1rjKbTNBn1iQ6J/aSJzWVRaXfwJPNkCzQPh dsdSkBom+YU4CkkghHa6NlD3nu1vglMkRAZgdugmbhBI7+f9ZVng/D47GtGXLWuVzqio feva/YycK22paiO4ypRpICHrEo29JP0qN9f2mS2yjBarhqqn6SCNg7PXm5zDB5WwHzdT gOyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xlsOuybVen4f96SBiCIcF54dALKHxXee+JiiMWKPUxI=; b=FCuIV2CzeGlyrrJSyXGqhg83+0/9zcELJxX/Ks+b121c5E6VRxg47BaJIJeHVBamH+ WKLAvLfnvo4ltozhtBHV6khHpjf6MBXQ+LLiagBYJYlUfJ5fy6Az7+cyixh/UhC5Uq/1 QZXIoMxnG3whI1krIifcm8Kn9AfE55W9X7vh1tI9dGOMhk4Qd7hFjTeDzSuGZHinrRlr uSspsLMp6EJrP5I5mefbE+1XKOW0mCZcmB5zY+2RFoT8REXF69MemWj6FMcPxczPYL/0 DnF900bNPuzVoaGx2i+QA5lrFVQq1QnddM2u0zymWSHtIC/jlfrcQyRcg/yAbD8m96zv 0vPw== X-Gm-Message-State: ACrzQf3Kp2HfNKf5V8dsoXadmTfi9nzegTg8CSDUX3qGKPQoAD3Yih5S m4YvEt4g0wzRQA6wbsid8PHs24NiHdRnKQ== X-Received: by 2002:aa7:d289:0:b0:458:93a8:4789 with SMTP id w9-20020aa7d289000000b0045893a84789mr6105212edq.374.1666164383954; Wed, 19 Oct 2022 00:26:23 -0700 (PDT) Received: from thomas-win.localdomain ([213.138.44.237]) by smtp.gmail.com with ESMTPSA id p5-20020a170906498500b0078b83968ad4sm8515150eju.24.2022.10.19.00.26.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Oct 2022 00:26:23 -0700 (PDT) From: thomas.ff@spin-digital.com To: ffmpeg-devel@ffmpeg.org Date: Wed, 19 Oct 2022 09:24:58 +0200 Message-Id: <20221019072508.23460-2-thomas.ff@spin-digital.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221019072508.23460-1-thomas.ff@spin-digital.com> References: <20221019072508.23460-1-thomas.ff@spin-digital.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v1 01/11] avcodec: add enum types for H266/VVC X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Thomas Siedel Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: RU/elTm7jId6 From: Thomas Siedel Add needed types as nal unit types, slice types and vvc typedefs needed for parsing vvc high-level syntax Signed-off-by: Thomas Siedel --- libavcodec/vvc.h | 142 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 142 insertions(+) create mode 100644 libavcodec/vvc.h diff --git a/libavcodec/vvc.h b/libavcodec/vvc.h new file mode 100644 index 0000000000..19979b7017 --- /dev/null +++ b/libavcodec/vvc.h @@ -0,0 +1,142 @@ +/* + * VVC shared code + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_VVC_H +#define AVCODEC_VVC_H + +/** + * Table 5 – NAL unit type codes and NAL unit type classes + * in T-REC-H.266-202008 + */ +enum VVCNALUnitType { + VVC_TRAIL_NUT = 0, + VVC_STSA_NUT = 1, + VVC_RADL_NUT = 2, + VVC_RASL_NUT = 3, + VVC_RSV_VCL_4 = 4, + VVC_RSV_VCL_5 = 5, + VVC_RSV_VCL_6 = 6, + VVC_IDR_W_RADL = 7, + VVC_IDR_N_LP = 8, + VVC_CRA_NUT = 9, + VVC_GDR_NUT = 10, + VVC_RSV_IRAP_11 = 11, + VVC_OPI_NUT = 12, + VVC_DCI_NUT = 13, + VVC_VPS_NUT = 14, + VVC_SPS_NUT = 15, + VVC_PPS_NUT = 16, + VVC_PREFIX_APS_NUT = 17, + VVC_SUFFIX_APS_NUT = 18, + VVC_PH_NUT = 19, + VVC_AUD_NUT = 20, + VVC_EOS_NUT = 21, + VVC_EOB_NUT = 22, + VVC_PREFIX_SEI_NUT = 23, + VVC_SUFFIX_SEI_NUT = 24, + VVC_FD_NUT = 25, + VVC_RSV_NVCL_26 = 26, + VVC_RSV_NVCL_27 = 27, + VVC_UNSPEC_28 = 28, + VVC_UNSPEC_29 = 29, + VVC_UNSPEC_30 = 30, + VVC_UNSPEC_31 = 31, +}; + +enum VVCSliceType { + VVC_SLICE_TYPE_B = 0, + VVC_SLICE_TYPE_P = 1, + VVC_SLICE_TYPE_I = 2, +}; + +enum { + //6.2 we can have 3 sample arrays + VVC_MAX_SAMPLE_ARRAYS = 3, + + //7.4.3.3 vps_max_layers_minus1 is u(6) + VVC_MAX_LAYERS = 64, + + //7.4.3.3 The value of vps_max_sublayers_minus1 shall be in the range of 0 to 6, inclusive + VVC_MAX_SUBLAYERS = 7, + + //7.4.3.3 vps_num_ptls_minus1 is u(8) + VVC_MAX_PTLS = 256, + + //7.4.3.3 vps_num_output_layer_sets_minus2 is u(8) + VVC_MAX_TOTAL_NUM_OLSS = 257, + + // 7.3.2.3: vps_video_parameter_set_id is u(4). + VVC_MAX_VPS_COUNT = 16, + // 7.3.2.4: sps_seq_parameter_set_id is u(4) + VVC_MAX_SPS_COUNT = 16, + // 7.3.2.5: pps_pic_parameter_set_id is u(6) + VVC_MAX_PPS_COUNT = 64, + + // 7.4.4.1: ptl_num_sub_profiles is u(8) + VVC_MAX_SUB_PROFILES = 256, + + // A.4.2: according to (1577), MaxDpbSize is bounded above by 2 * maxDpbPicBuf(8) + VVC_MAX_DPB_SIZE = 16, + + //7.4.3.4 sps_num_ref_pic_lists in range [0, 64] + VVC_MAX_REF_PIC_LISTS = 64, + + //7.4.11 num_ref_entries in range [0, MaxDpbSize + 13] + VVC_MAX_REF_ENTRIES = VVC_MAX_DPB_SIZE + 13, + + //7.4.3.3 sps_num_points_in_qp_table_minus1[i] in range [0, 36 − sps_qp_table_start_minus26[i]], + //sps_qp_table_start_minus26[i] in range [sps_qp_table_start_minus26[i] −26 − QpBdOffset, 36] + //for 10 bitsQpBdOffset is 12, so sps_num_points_in_qp_table_minus1[i] in range [0, 74] + VVC_MAX_POINTS_IN_QP_TABLE = 75, + + // 7.4.6.1: hrd_cpb_cnt_minus1 is in [0, 31]. + VVC_MAX_CPB_CNT = 32, + + // A.4.1: the highest level allows a MaxLumaPs of 35 651 584. + VVC_MAX_LUMA_PS = 35651584, + + // A.4.1: pic_width_in_luma_samples and pic_height_in_luma_samples are + // constrained to be not greater than sqrt(MaxLumaPs * 8). Hence height/ + // width are bounded above by sqrt(8 * 35651584) = 16888.2 samples. + VVC_MAX_WIDTH = 16888, + VVC_MAX_HEIGHT = 16888, + + // A.4.1: table A.1 allows at most 440 tiles per au for any level. + VVC_MAX_TILES_PER_AU = 440, + // A.4.1: table A.1 did not define max tile rows. + // in worest a case, we can have 1x440 tiles picture. + VVC_MAX_TILE_ROWS = VVC_MAX_TILES_PER_AU, + // A.4.1: table A.1 allows at most 20 tile columns for any level. + VVC_MAX_TILE_COLUMNS = 20, + + // A.4.1 table A.1 allows at most 600 slice for any level. + VVC_MAX_SLICES = 600, + + // 7.4.8: in the worst case (!pps_no_pic_partition_flag and + // sps_entropy_coding_sync_enabled_flag are both true), entry points can be + // placed at the beginning of every Ctb row in every tile, giving an + // upper bound of (num_tile_columns_minus1 + 1) * PicHeightInCtbsY - 1. + // Only a stream with very high resolution and perverse parameters could + // get near that, though, so set a lower limit here with the maximum + // possible value for 8K video (at most 135 32x32 Ctb rows). + VVC_MAX_ENTRY_POINTS = VVC_MAX_TILE_COLUMNS * 135, +}; + +#endif /* AVCODEC_VVC_H */ \ No newline at end of file From patchwork Wed Oct 19 07:24:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Siedel X-Patchwork-Id: 38812 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp215805pzb; Wed, 19 Oct 2022 00:27:05 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5g8h/K5dm6q/TxIp9JWdXkik8WPqOvV+QlG34fP2WqNaRSRnzph4HLJBwWJEQchH5PM9br X-Received: by 2002:a17:907:2e0d:b0:78e:1d21:8001 with SMTP id ig13-20020a1709072e0d00b0078e1d218001mr5501948ejc.246.1666164425407; Wed, 19 Oct 2022 00:27:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666164425; cv=none; d=google.com; s=arc-20160816; b=f29ga+am5B/EsZ2Zn1YOjA+9aguyeDM92R7wAvpmQweFpk+W/VsX64kNr4/ScrwMp6 QGjCKh6zZDGK9CARCZ6zqRunFx/KhykVWnJxYz075dtgSeAlpqDSGrjRMwqxwyvQ4AIQ n/Xp+JQTLGNhVM56k6zgsN+9Nm72dc4rhvwbYrrS4C/K71Af9XTaKvrMzGm7fb+arvUQ nucRFcbZRBu1mYTmgosCtEnZ1A6vHtyROEMDczmVJQB6x1jdU+6Ru+/8PBErbeUmHKyp OGts794N8aZ5znWueNShQgbnEAihsmr3zowOObN+2ePejWj/iCU3RbDM4qzyAjRUlzgq y+SA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=bQGJBwlYqQe07hITq1pgL1gf+DWzHi74SxYeQ+HIa9c=; b=gXfRRBei8sxlzXBCcuoiITzO64KhgZ6elkpA5Vbz9jMOrtrOaOj0LQqbVobQXvssJd nPvyusXMCnF+wa5iiKGZw54lTuKDBbaWXYs9+Xy/H8sMzkQi57UzMCq8i/ftsG2k/JK8 dCV0QJKDLuMMtODx/L74jhUYfbts3HOrGZandI9+Y+Zjm5AkinMS1cBSZEMgaknN64Fk +0c7PEuuFqbBTT3Cz2rywQcxYrK2n6QlRwz2fpCmO4kKVU1J7RibzGEmh6dt5kAGukOo nXwloB2zRrKTY86fGjI+f9ewYgoRIwKp5fBRpzqey4iD9dxEQXWmiqPOuFavesasMl5w yXSw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=Sz33LBOH; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id y7-20020a50eb07000000b00443088b40b7si12332611edp.123.2022.10.19.00.27.04; Wed, 19 Oct 2022 00:27:05 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=Sz33LBOH; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E3EE368BDC8; Wed, 19 Oct 2022 10:26:34 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f47.google.com (mail-ej1-f47.google.com [209.85.218.47]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6068868BD6C for ; Wed, 19 Oct 2022 10:26:27 +0300 (EEST) Received: by mail-ej1-f47.google.com with SMTP id sc25so37788289ejc.12 for ; Wed, 19 Oct 2022 00:26:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spin-digital-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=n2XAUJHb9uyWsQmE7RCh8pmhxfrKCsBpv6lgRynKbbQ=; b=Sz33LBOHvpVkuYU8F0yzIiJfqaCgZ1CpilLw/r+ZztedTrgcIpJLFOfPkrKj0zc9gV WwImSc+Ia912CRycAuwmKn6olMzFl6C0LaU81jvghNnqsi/xQKMhFxPZ5/lXDB+EkoWN 42BMJC9xWRZpEbquwyFXv19MUiaVdliKnldKH3CpBmS8pMzn3sATYV5B2qRJx/zjxyon BJpLVc4y7D5RsHA7Vq7PFJfCtWB3B/i/vwHGXBALn0rF9zytFjNNxBbitE2jsiKfm6Q2 e3MjC1nP6t/aLlrJbc5X1NUzLRnn4Wu/6yF9FFQWyUPTiwhdJ0hYwPQD8OdJH8M2BxHK FfpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=n2XAUJHb9uyWsQmE7RCh8pmhxfrKCsBpv6lgRynKbbQ=; b=Ss17Wot36eR2biGOHtLxcNbSM7Z1lgEwqv5hp5V+F2ABYBUob3pJrCJ/7qU3HO6zH6 7KfAc/0y/iP4CqWLJWU37QznE0mAs/jSTFzF8+GilXNzCm202KBb46NQi9Z+vQbf+8ZQ C44Bi6xuQH+BzBmeUFW2cbZuVpjeixB9T2Jmo+PiGksXz5p6UL9feFSnJGxXjuvTPsq6 1NxMh2dmRsBgHvINJyka61KFYsqVKbHMBoN2MJB6Dwue04RVvsygWesMl8nJk1ecMybb uL3UMI23wYvfDOwsE89s6WwuRT+2KOJN86DeGR6GA2nF+Ui+v6KSqRYKDm4ycWQINl4s TxtA== X-Gm-Message-State: ACrzQf3syPGAQwQwC61QkXDzuBf5sLMPYBEwsQhREiierpwpGs6W/TAd o/Yj77PGtYk1gajh/M/IQU/rv8vopnBjbg== X-Received: by 2002:a17:906:5daa:b0:78e:2bc7:f1a3 with SMTP id n10-20020a1709065daa00b0078e2bc7f1a3mr5366413ejv.256.1666164385297; Wed, 19 Oct 2022 00:26:25 -0700 (PDT) Received: from thomas-win.localdomain ([213.138.44.237]) by smtp.gmail.com with ESMTPSA id p5-20020a170906498500b0078b83968ad4sm8515150eju.24.2022.10.19.00.26.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Oct 2022 00:26:24 -0700 (PDT) From: thomas.ff@spin-digital.com To: ffmpeg-devel@ffmpeg.org Date: Wed, 19 Oct 2022 09:24:59 +0200 Message-Id: <20221019072508.23460-3-thomas.ff@spin-digital.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221019072508.23460-1-thomas.ff@spin-digital.com> References: <20221019072508.23460-1-thomas.ff@spin-digital.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v1 02/11] avcodec: add cbs for H266/VVC X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Thomas Siedel Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: R57QTBeBRmtu From: Thomas Siedel Add CodedBitstreamContext structures to parse VPS,SPS,PPS in VVC nal units. Implement parsing(read) and writing of SPS,PPS,VPS,PH,AUD,SEI and slices (TRAIL,STSA,RADL,RASL,IDR,CRA,GDR) Signed-off-by: Thomas Siedel --- libavcodec/cbs_h2645.c | 384 +++- libavcodec/cbs_h266.h | 791 +++++++ libavcodec/cbs_h266_syntax_template.c | 3010 +++++++++++++++++++++++++ libavcodec/cbs_sei.c | 29 + 4 files changed, 4212 insertions(+), 2 deletions(-) create mode 100644 libavcodec/cbs_h266.h create mode 100644 libavcodec/cbs_h266_syntax_template.c diff --git a/libavcodec/cbs_h2645.c b/libavcodec/cbs_h2645.c index 4ee06003c3..a44aae6b9f 100644 --- a/libavcodec/cbs_h2645.c +++ b/libavcodec/cbs_h2645.c @@ -24,10 +24,16 @@ #include "cbs_internal.h" #include "cbs_h264.h" #include "cbs_h265.h" +#include "cbs_h266.h" #include "h264.h" #include "h2645_parse.h" #include "hevc.h" +#include "vvc.h" +static av_always_inline unsigned int h266_ceil(unsigned int v, unsigned int align) +{ + return (((v) + (align) - 1) / (align)); +} static int cbs_read_ue_golomb(CodedBitstreamContext *ctx, GetBitContext *gbc, const char *name, const int *subscripts, @@ -255,6 +261,7 @@ static int cbs_h265_payload_extension_present(GetBitContext *gbc, uint32_t paylo #define FUNC_NAME1(rw, codec, name) FUNC_NAME2(rw, codec, name) #define FUNC_H264(name) FUNC_NAME1(READWRITE, h264, name) #define FUNC_H265(name) FUNC_NAME1(READWRITE, h265, name) +#define FUNC_H266(name) FUNC_NAME1(READWRITE, h266, name) #define FUNC_SEI(name) FUNC_NAME1(READWRITE, sei, name) #define SUBSCRIPTS(subs, ...) (subs > 0 ? ((int[subs + 1]){ subs, __VA_ARGS__ }) : NULL) @@ -369,6 +376,10 @@ static int cbs_h2645_read_more_rbsp_data(GetBitContext *gbc) #include "cbs_h265_syntax_template.c" #undef FUNC +#define FUNC(name) FUNC_H266(name) +#include "cbs_h266_syntax_template.c" +#undef FUNC + #undef READ #undef READWRITE #undef RWContext @@ -447,6 +458,10 @@ static int cbs_h2645_read_more_rbsp_data(GetBitContext *gbc) #include "cbs_h265_syntax_template.c" #undef FUNC +#define FUNC(name) FUNC_H266(name) +#include "cbs_h266_syntax_template.c" +#undef FUNC + #undef WRITE #undef READWRITE #undef RWContext @@ -476,8 +491,9 @@ static int cbs_h2645_fragment_add_nals(CodedBitstreamContext *ctx, const H2645NAL *nal = &packet->nals[i]; AVBufferRef *ref; size_t size = nal->size; + enum AVCodecID codec_id = ctx->codec->codec_id; - if (nal->nuh_layer_id > 0) + if (codec_id != AV_CODEC_ID_VVC && nal->nuh_layer_id > 0) continue; // Remove trailing zeroes. @@ -513,6 +529,12 @@ static int cbs_h2645_split_fragment(CodedBitstreamContext *ctx, if (frag->data_size == 0) return 0; + if (codec_id == AV_CODEC_ID_VVC) { + //we deactive picture header here to avoid reuse previous au's ph. + CodedBitstreamH266Context *h266 = ctx->priv_data; + h266->priv.ph = NULL; + } + if (header && frag->data[0] && codec_id == AV_CODEC_ID_H264) { // AVCC header. size_t size, start, end; @@ -685,6 +707,26 @@ cbs_h2645_replace_ps(4, PPS, pps, pic_parameter_set_id) cbs_h2645_replace_ps(5, VPS, vps, vps_video_parameter_set_id) cbs_h2645_replace_ps(5, SPS, sps, sps_seq_parameter_set_id) cbs_h2645_replace_ps(5, PPS, pps, pps_pic_parameter_set_id) +cbs_h2645_replace_ps(6, VPS, vps, vps_video_parameter_set_id) +cbs_h2645_replace_ps(6, SPS, sps, sps_seq_parameter_set_id) +cbs_h2645_replace_ps(6, PPS, pps, pps_pic_parameter_set_id) + +static int cbs_h266_replace_ph(CodedBitstreamContext *ctx, + CodedBitstreamUnit *unit) +{ + CodedBitstreamH266Context *h266 = ctx->priv_data; + int err; + + h266->priv.ph = NULL; + err = ff_cbs_make_unit_refcounted(ctx, unit); + if (err < 0) + return err; + err = av_buffer_replace(&h266->priv.ph_ref, unit->content_ref); + if (err < 0) + return err; + h266->priv.ph = (H266RawPH*)h266->priv.ph_ref->data; + return 0; +} static int cbs_h264_read_nal_unit(CodedBitstreamContext *ctx, CodedBitstreamUnit *unit) @@ -926,6 +968,130 @@ static int cbs_h265_read_nal_unit(CodedBitstreamContext *ctx, return 0; } +static int cbs_h266_read_nal_unit(CodedBitstreamContext *ctx, + CodedBitstreamUnit *unit) +{ + GetBitContext gbc; + int err; + + err = init_get_bits8(&gbc, unit->data, unit->data_size); + if (err < 0) + return err; + + err = ff_cbs_alloc_unit_content(ctx, unit); + if (err < 0) + return err; + + switch (unit->type) { + case VVC_VPS_NUT: + { + H266RawVPS *vps = unit->content; + + err = cbs_h266_read_vps(ctx, &gbc, vps); + if (err < 0) + return err; + + err = cbs_h266_replace_vps(ctx, unit); + if (err < 0) + return err; + } + break; + case VVC_SPS_NUT: + { + H266RawSPS *sps = unit->content; + + err = cbs_h266_read_sps(ctx, &gbc, sps); + if (err < 0) + return err; + + err = cbs_h266_replace_sps(ctx, unit); + if (err < 0) + return err; + } + break; + + case VVC_PPS_NUT: + { + H266RawPPS *pps = unit->content; + + err = cbs_h266_read_pps(ctx, &gbc, pps); + if (err < 0) + return err; + + err = cbs_h266_replace_pps(ctx, unit); + if (err < 0) + return err; + } + break; + + case VVC_PH_NUT: + { + H266RawPH *ph = unit->content; + err = cbs_h266_read_ph(ctx, &gbc, ph); + if (err < 0) + return err; + err = cbs_h266_replace_ph(ctx, unit); + if (err < 0) + return err; + } + break; + + case VVC_TRAIL_NUT: + case VVC_STSA_NUT: + case VVC_RADL_NUT: + case VVC_RASL_NUT: + case VVC_IDR_W_RADL: + case VVC_IDR_N_LP: + case VVC_CRA_NUT: + case VVC_GDR_NUT: + { + H266RawSlice *slice = unit->content; + int pos, len; + + err = cbs_h266_read_slice_header(ctx, &gbc, &slice->header); + if (err < 0) + return err; + + if (!cbs_h2645_read_more_rbsp_data(&gbc)) + return AVERROR_INVALIDDATA; + + pos = get_bits_count(&gbc); + len = unit->data_size; + + slice->data_size = len - pos / 8; + slice->data_ref = av_buffer_ref(unit->data_ref); + if (!slice->data_ref) + return AVERROR(ENOMEM); + slice->data = unit->data + pos / 8; + slice->data_bit_start = pos % 8; + } + break; + + case VVC_AUD_NUT: + { + err = cbs_h266_read_aud(ctx, &gbc, unit->content); + if (err < 0) + return err; + } + break; + + case VVC_PREFIX_SEI_NUT: + case VVC_SUFFIX_SEI_NUT: + { + err = cbs_h266_read_sei(ctx, &gbc, unit->content, + unit->type == VVC_PREFIX_SEI_NUT); + + if (err < 0) + return err; + } + break; + + default: + return AVERROR(ENOSYS); + } + return 0; +} + static int cbs_h2645_write_slice_data(CodedBitstreamContext *ctx, PutBitContext *pbc, const uint8_t *data, size_t data_size, int data_bit_start) @@ -1213,11 +1379,127 @@ static int cbs_h265_write_nal_unit(CodedBitstreamContext *ctx, return 0; } +static int cbs_h266_write_nal_unit(CodedBitstreamContext *ctx, + CodedBitstreamUnit *unit, + PutBitContext *pbc) +{ + int err; + + switch (unit->type) { + case VVC_VPS_NUT: + { + H266RawVPS *vps = unit->content; + + err = cbs_h266_write_vps(ctx, pbc, vps); + if (err < 0) + return err; + + err = cbs_h266_replace_vps(ctx, unit); + if (err < 0) + return err; + } + break; + case VVC_SPS_NUT: + { + H266RawSPS *sps = unit->content; + + err = cbs_h266_write_sps(ctx, pbc, sps); + if (err < 0) + return err; + + err = cbs_h266_replace_sps(ctx, unit); + if (err < 0) + return err; + } + break; + + case VVC_PPS_NUT: + { + H266RawPPS *pps = unit->content; + + err = cbs_h266_write_pps(ctx, pbc, pps); + if (err < 0) + return err; + + err = cbs_h266_replace_pps(ctx, unit); + if (err < 0) + return err; + } + break; + + case VVC_PH_NUT: + { + H266RawPH *ph = unit->content; + err = cbs_h266_write_ph(ctx, pbc, ph); + if (err < 0) + return err; + + err = cbs_h266_replace_ph(ctx, unit); + if (err < 0) + return err; + } + break; + + case VVC_TRAIL_NUT: + case VVC_STSA_NUT: + case VVC_RADL_NUT: + case VVC_RASL_NUT: + case VVC_IDR_W_RADL: + case VVC_IDR_N_LP: + case VVC_CRA_NUT: + case VVC_GDR_NUT: + { + H266RawSlice *slice = unit->content; + + err = cbs_h266_write_slice_header(ctx, pbc, &slice->header); + if (err < 0) + return err; + + if (slice->data) { + err = cbs_h2645_write_slice_data(ctx, pbc, slice->data, + slice->data_size, + slice->data_bit_start); + if (err < 0) + return err; + } else { + // No slice data - that was just the header. + } + } + break; + + case VVC_AUD_NUT: + { + err = cbs_h266_write_aud(ctx, pbc, unit->content); + if (err < 0) + return err; + } + break; + + case VVC_PREFIX_SEI_NUT: + case VVC_SUFFIX_SEI_NUT: + { + err = cbs_h266_write_sei(ctx, pbc, unit->content, + unit->type == VVC_PREFIX_SEI_NUT); + + if (err < 0) + return err; + } + break; + + default: + av_log(ctx->log_ctx, AV_LOG_ERROR, "Write unimplemented for " + "NAL unit type %"PRIu32".\n", unit->type); + return AVERROR_PATCHWELCOME; + } + + return 0; +} + static int cbs_h2645_unit_requires_zero_byte(enum AVCodecID codec_id, CodedBitstreamUnitType type, int nal_unit_index) { - // Section B.1.2 in H.264, section B.2.2 in H.265. + // Section B.1.2 in H.264, section B.2.2 in H.265, H.266. if (nal_unit_index == 0) { // Assume that this is the first NAL unit in an access unit. return 1; @@ -1226,6 +1508,8 @@ static int cbs_h2645_unit_requires_zero_byte(enum AVCodecID codec_id, return type == H264_NAL_SPS || type == H264_NAL_PPS; if (codec_id == AV_CODEC_ID_HEVC) return type == HEVC_NAL_VPS || type == HEVC_NAL_SPS || type == HEVC_NAL_PPS; + if (codec_id == AV_CODEC_ID_VVC) + return type >= VVC_OPI_NUT && type <= VVC_SUFFIX_APS_NUT; return 0; } @@ -1377,6 +1661,35 @@ static void cbs_h265_close(CodedBitstreamContext *ctx) av_buffer_unref(&h265->pps_ref[i]); } +static void cbs_h266_flush(CodedBitstreamContext *ctx) +{ + CodedBitstreamH266Context *h266 = ctx->priv_data; + + for (int i = 0; i < FF_ARRAY_ELEMS(h266->vps); i++) { + av_buffer_unref(&h266->vps_ref[i]); + h266->vps[i] = NULL; + } + + for (int i = 0; i < FF_ARRAY_ELEMS(h266->sps); i++) { + av_buffer_unref(&h266->sps_ref[i]); + h266->sps[i] = NULL; + } + for (int i = 0; i < FF_ARRAY_ELEMS(h266->pps); i++) { + av_buffer_unref(&h266->pps_ref[i]); + h266->pps[i] = NULL; + } + av_buffer_unref(&h266->priv.ph_ref); + h266->priv.ph = NULL; +} + +static void cbs_h266_close(CodedBitstreamContext *ctx) +{ + CodedBitstreamH266Context *h266 = ctx->priv_data; + + cbs_h266_flush(ctx); + ff_h2645_packet_uninit(&h266->common.read_packet); + } + static void cbs_h264_free_sei(void *opaque, uint8_t *content) { H264RawSEI *sei = (H264RawSEI*)content; @@ -1431,6 +1744,44 @@ static const CodedBitstreamUnitTypeDescriptor cbs_h265_unit_types[] = { CBS_UNIT_TYPE_END_OF_LIST }; +static void cbs_h266_free_sei(void *opaque, uint8_t *content) +{ + H265RawSEI *sei = (H265RawSEI*)content; + ff_cbs_sei_free_message_list(&sei->message_list); + av_free(content); +} + +static const CodedBitstreamUnitTypeDescriptor cbs_h266_unit_types[] = { + CBS_UNIT_TYPE_INTERNAL_REF(VVC_VPS_NUT, H266RawVPS, extension_data.data), + CBS_UNIT_TYPE_INTERNAL_REF(VVC_PPS_NUT, H266RawPPS, extension_data.data), + + // SPS + { + .nb_unit_types = CBS_UNIT_TYPE_RANGE, + .unit_type.range.start = VVC_SPS_NUT, + .unit_type.range.end = VVC_SPS_NUT, + + .content_type = CBS_CONTENT_TYPE_INTERNAL_REFS, + .content_size = sizeof(H266RawSPS), + + .type.ref = { .nb_offsets = 2, + .offsets = { offsetof(H266RawSPS, extension_data.data), + offsetof(H266RawSPS, vui.extension_data.data) } }, + + }, + + CBS_UNIT_TYPE_POD(VVC_PH_NUT , H266RawPH), + CBS_UNIT_TYPE_POD(VVC_AUD_NUT, H266RawAUD), + + CBS_UNIT_RANGE_INTERNAL_REF(VVC_TRAIL_NUT, VVC_GDR_NUT, + H266RawSlice, data), + + CBS_UNIT_TYPES_COMPLEX((VVC_PREFIX_SEI_NUT, VVC_SUFFIX_SEI_NUT), + H266RawSEI, cbs_h266_free_sei), + + CBS_UNIT_TYPE_END_OF_LIST +}; + const CodedBitstreamType ff_cbs_type_h264 = { .codec_id = AV_CODEC_ID_H264, @@ -1463,6 +1814,22 @@ const CodedBitstreamType ff_cbs_type_h265 = { .close = &cbs_h265_close, }; +const CodedBitstreamType ff_cbs_type_h266 = { + .codec_id = AV_CODEC_ID_VVC, + + .priv_data_size = sizeof(CodedBitstreamH266Context), + + .unit_types = cbs_h266_unit_types, + + .split_fragment = &cbs_h2645_split_fragment, + .read_unit = &cbs_h266_read_nal_unit, + .write_unit = &cbs_h266_write_nal_unit, + .assemble_fragment = &cbs_h2645_assemble_fragment, + + .flush = &cbs_h266_flush, + .close = &cbs_h266_close, +}; + static const SEIMessageTypeDescriptor cbs_sei_common_types[] = { { SEI_TYPE_FILLER_PAYLOAD, @@ -1607,6 +1974,16 @@ static const SEIMessageTypeDescriptor cbs_sei_h265_types[] = { SEI_MESSAGE_TYPE_END }; +static const SEIMessageTypeDescriptor cbs_sei_h266_types[] = { + { + SEI_TYPE_DECODED_PICTURE_HASH, + 0, 1, + sizeof(H266RawSEIDecodedPictureHash), + SEI_MESSAGE_RW(h266, sei_decoded_picture_hash), + }, + SEI_MESSAGE_TYPE_END +}; + const SEIMessageTypeDescriptor *ff_cbs_sei_find_type(CodedBitstreamContext *ctx, int payload_type) { @@ -1625,6 +2002,9 @@ const SEIMessageTypeDescriptor *ff_cbs_sei_find_type(CodedBitstreamContext *ctx, case AV_CODEC_ID_H265: codec_list = cbs_sei_h265_types; break; + case AV_CODEC_ID_H266: + codec_list = cbs_sei_h266_types; + break; default: return NULL; } diff --git a/libavcodec/cbs_h266.h b/libavcodec/cbs_h266.h new file mode 100644 index 0000000000..f2f72f780d --- /dev/null +++ b/libavcodec/cbs_h266.h @@ -0,0 +1,791 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_CBS_H266_H +#define AVCODEC_CBS_H266_H + +#include +#include + +#include "cbs_h2645.h" +#include "cbs_sei.h" +#include "vvc.h" + +typedef struct H266RawNALUnitHeader { + uint8_t nuh_layer_id; + uint8_t nal_unit_type; + uint8_t nuh_temporal_id_plus1; + uint8_t nuh_reserved_zero_bit; +} H266RawNALUnitHeader; + +typedef struct H266GeneralConstraintsInfo { + uint8_t gci_present_flag; + /* general */ + uint8_t gci_intra_only_constraint_flag; + uint8_t gci_all_layers_independent_constraint_flag; + uint8_t gci_one_au_only_constraint_flag; + + /* picture format */ + uint8_t gci_sixteen_minus_max_bitdepth_constraint_idc; + uint8_t gci_three_minus_max_chroma_format_constraint_idc; + + /* NAL unit type related */ + uint8_t gci_no_mixed_nalu_types_in_pic_constraint_flag; + uint8_t gci_no_trail_constraint_flag; + uint8_t gci_no_stsa_constraint_flag; + uint8_t gci_no_rasl_constraint_flag; + uint8_t gci_no_radl_constraint_flag; + uint8_t gci_no_idr_constraint_flag; + uint8_t gci_no_cra_constraint_flag; + uint8_t gci_no_gdr_constraint_flag; + uint8_t gci_no_aps_constraint_flag; + uint8_t gci_no_idr_rpl_constraint_flag; + + /* tile, slice, subpicture partitioning */ + uint8_t gci_one_tile_per_pic_constraint_flag; + uint8_t gci_pic_header_in_slice_header_constraint_flag; + uint8_t gci_one_slice_per_pic_constraint_flag; + uint8_t gci_no_rectangular_slice_constraint_flag; + uint8_t gci_one_slice_per_subpic_constraint_flag; + uint8_t gci_no_subpic_info_constraint_flag; + + /* CTU and block partitioning */ + uint8_t gci_three_minus_max_log2_ctu_size_constraint_idc; + uint8_t gci_no_partition_constraints_override_constraint_flag; + uint8_t gci_no_mtt_constraint_flag; + uint8_t gci_no_qtbtt_dual_tree_intra_constraint_flag; + + /* intra */ + uint8_t gci_no_palette_constraint_flag; + uint8_t gci_no_ibc_constraint_flag; + uint8_t gci_no_isp_constraint_flag; + uint8_t gci_no_mrl_constraint_flag; + uint8_t gci_no_mip_constraint_flag; + uint8_t gci_no_cclm_constraint_flag; + + /* inter */ + uint8_t gci_no_ref_pic_resampling_constraint_flag; + uint8_t gci_no_res_change_in_clvs_constraint_flag;; + uint8_t gci_no_weighted_prediction_constraint_flag; + uint8_t gci_no_ref_wraparound_constraint_flag; + uint8_t gci_no_temporal_mvp_constraint_flag; + uint8_t gci_no_sbtmvp_constraint_flag; + uint8_t gci_no_amvr_constraint_flag; + uint8_t gci_no_bdof_constraint_flag; + uint8_t gci_no_smvd_constraint_flag; + uint8_t gci_no_dmvr_constraint_flag; + uint8_t gci_no_mmvd_constraint_flag; + uint8_t gci_no_affine_motion_constraint_flag; + uint8_t gci_no_prof_constraint_flag; + uint8_t gci_no_bcw_constraint_flag; + uint8_t gci_no_ciip_constraint_flag; + uint8_t gci_no_gpm_constraint_flag; + + /* transform, quantization, residual */ + uint8_t gci_no_luma_transform_size_64_constraint_flag; + uint8_t gci_no_transform_skip_constraint_flag; + uint8_t gci_no_bdpcm_constraint_flag; + uint8_t gci_no_mts_constraint_flag; + uint8_t gci_no_lfnst_constraint_flag; + uint8_t gci_no_joint_cbcr_constraint_flag; + uint8_t gci_no_sbt_constraint_flag; + uint8_t gci_no_act_constraint_flag; + uint8_t gci_no_explicit_scaling_list_constraint_flag; + uint8_t gci_no_dep_quant_constraint_flag; + uint8_t gci_no_sign_data_hiding_constraint_flag; + uint8_t gci_no_cu_qp_delta_constraint_flag; + uint8_t gci_no_chroma_qp_offset_constraint_flag; + + /* loop filter */ + uint8_t gci_no_sao_constraint_flag; + uint8_t gci_no_alf_constraint_flag; + uint8_t gci_no_ccalf_constraint_flag; + uint8_t gci_no_lmcs_constraint_flag; + uint8_t gci_no_ladf_constraint_flag; + uint8_t gci_no_virtual_boundaries_constraint_flag; + uint8_t gci_num_reserved_bits; + uint8_t gci_reserved_zero_bit[255]; +} H266GeneralConstraintsInfo; + +typedef struct H266RawProfileTierLevel { + uint8_t general_profile_idc; + uint8_t general_tier_flag; + uint8_t general_level_idc; + uint8_t ptl_frame_only_constraint_flag; + uint8_t ptl_multilayer_enabled_flag; + H266GeneralConstraintsInfo general_constraints_info; + uint8_t ptl_sublayer_level_present_flag[VVC_MAX_SUBLAYERS - 1]; + uint8_t sublayer_level_idc[VVC_MAX_SUBLAYERS - 1]; + uint8_t ptl_num_sub_profiles; + uint32_t general_sub_profile_idc[VVC_MAX_SUB_PROFILES]; + + uint8_t ptl_reserved_zero_bit; +} H266RawProfileTierLevel; + +typedef struct H266RawExtensionData { + uint8_t *data; + AVBufferRef *data_ref; + size_t bit_length; +} H266RawExtensionData; + +typedef struct H266DpbParameters { + uint8_t dpb_max_dec_pic_buffering_minus1[VVC_MAX_SUBLAYERS]; + uint8_t dpb_max_num_reorder_pics[VVC_MAX_SUBLAYERS]; + uint8_t dpb_max_latency_increase_plus1[VVC_MAX_SUBLAYERS]; +} H266DpbParameters; + +typedef struct H266RefPicListStruct { + uint8_t num_ref_entries; + uint8_t ltrp_in_header_flag; + uint8_t inter_layer_ref_pic_flag[VVC_MAX_REF_ENTRIES]; + uint8_t st_ref_pic_flag[VVC_MAX_REF_ENTRIES]; + uint8_t abs_delta_poc_st[VVC_MAX_REF_ENTRIES]; + uint8_t strp_entry_sign_flag[VVC_MAX_REF_ENTRIES]; + uint8_t rpls_poc_lsb_lt[VVC_MAX_REF_ENTRIES]; + uint8_t ilrp_idx[VVC_MAX_REF_ENTRIES]; +} H266RefPicListStruct; + +typedef struct H266RefPicLists { + uint8_t rpl_sps_flag[2]; + uint8_t rpl_idx[2]; + H266RefPicListStruct rpl_ref_list[2]; + uint16_t poc_lsb_lt[2][VVC_MAX_REF_ENTRIES]; + uint8_t delta_poc_msb_cycle_present_flag[2][VVC_MAX_REF_ENTRIES]; + uint16_t delta_poc_msb_cycle_lt[2][VVC_MAX_REF_ENTRIES]; +} H266RefPicLists; + +typedef struct H266RawGeneralTimingHrdParameters { + uint32_t num_units_in_tick; + uint32_t time_scale; + uint8_t general_nal_hrd_params_present_flag; + uint8_t general_vcl_hrd_params_present_flag; + uint8_t general_same_pic_timing_in_all_ols_flag; + uint8_t general_du_hrd_params_present_flag; + uint8_t tick_divisor_minus2; + uint8_t bit_rate_scale; + uint8_t cpb_size_scale; + uint8_t cpb_size_du_scale; + uint8_t hrd_cpb_cnt_minus1; +} H266RawGeneralTimingHrdParameters; + +typedef struct H266RawSubLayerHRDParameters { + uint32_t bit_rate_value_minus1[VVC_MAX_SUBLAYERS][VVC_MAX_CPB_CNT]; + uint32_t cpb_size_value_minus1[VVC_MAX_SUBLAYERS][VVC_MAX_CPB_CNT]; + uint32_t cpb_size_du_value_minus1[VVC_MAX_SUBLAYERS][VVC_MAX_CPB_CNT]; + uint32_t bit_rate_du_value_minus1[VVC_MAX_SUBLAYERS][VVC_MAX_CPB_CNT]; + uint8_t cbr_flag[VVC_MAX_SUBLAYERS][VVC_MAX_CPB_CNT]; +} H266RawSubLayerHRDParameters; + +typedef struct H266RawOlsTimingHrdParameters { + uint8_t fixed_pic_rate_general_flag[VVC_MAX_SUBLAYERS]; + uint8_t fixed_pic_rate_within_cvs_flag[VVC_MAX_SUBLAYERS]; + uint16_t elemental_duration_in_tc_minus1[VVC_MAX_SUBLAYERS]; + uint8_t low_delay_hrd_flag[VVC_MAX_SUBLAYERS]; + H266RawSubLayerHRDParameters nal_sub_layer_hrd_parameters; + H266RawSubLayerHRDParameters vcl_sub_layer_hrd_parameters; +} H266RawOlsTimingHrdParameters; + +typedef struct H266RawVUI { + uint8_t vui_progressive_source_flag; + uint8_t vui_interlaced_source_flag; + uint8_t vui_non_packed_constraint_flag; + uint8_t vui_non_projected_constraint_flag; + + uint8_t vui_aspect_ratio_info_present_flag; + uint8_t vui_aspect_ratio_constant_flag; + uint8_t vui_aspect_ratio_idc; + + uint16_t vui_sar_width; + uint16_t vui_sar_height;; + + uint8_t vui_overscan_info_present_flag; + uint8_t vui_overscan_appropriate_flag; + + uint8_t vui_colour_description_present_flag; + uint8_t vui_colour_primaries; + + uint8_t vui_transfer_characteristics; + uint8_t vui_matrix_coeffs; + uint8_t vui_full_range_flag; + + uint8_t vui_chroma_loc_info_present_flag; + uint8_t vui_chroma_sample_loc_type_frame; + uint8_t vui_chroma_sample_loc_type_top_field; + uint8_t vui_chroma_sample_loc_type_bottom_field; + H266RawExtensionData extension_data; +} H266RawVUI; + +typedef struct H266RawVPS { + H266RawNALUnitHeader nal_unit_header; + + uint8_t vps_video_parameter_set_id; + uint8_t vps_max_layers_minus1; + uint8_t vps_max_sublayers_minus1; + uint8_t vps_default_ptl_dpb_hrd_max_tid_flag; + uint8_t vps_all_independent_layers_flag; + uint8_t vps_layer_id[VVC_MAX_LAYERS]; + uint8_t vps_independent_layer_flag[VVC_MAX_LAYERS]; + uint8_t vps_max_tid_ref_present_flag[VVC_MAX_LAYERS]; + uint8_t vps_direct_ref_layer_flag[VVC_MAX_LAYERS][VVC_MAX_LAYERS - 1]; + uint8_t vps_max_tid_il_ref_pics_plus1[VVC_MAX_LAYERS][VVC_MAX_LAYERS - 1]; + uint8_t vps_each_layer_is_an_ols_flag; + uint8_t vps_ols_mode_idc; + uint8_t vps_num_output_layer_sets_minus2; + uint8_t vps_ols_output_layer_flag[VVC_MAX_TOTAL_NUM_OLSS][VVC_MAX_LAYERS]; + + uint8_t vps_num_ptls_minus1; + uint8_t vps_pt_present_flag[VVC_MAX_PTLS]; + uint8_t vps_ptl_max_tid[VVC_MAX_PTLS]; + H266RawProfileTierLevel vps_profile_tier_level[VVC_MAX_PTLS]; + uint8_t vps_ols_ptl_idx[VVC_MAX_TOTAL_NUM_OLSS]; + + uint16_t vps_num_dpb_params_minus1; + uint8_t vps_sublayer_dpb_params_present_flag; + uint8_t vps_dpb_max_tid[VVC_MAX_TOTAL_NUM_OLSS]; + H266DpbParameters vps_dpb_params[VVC_MAX_TOTAL_NUM_OLSS]; + uint16_t vps_ols_dpb_pic_width[VVC_MAX_TOTAL_NUM_OLSS]; + uint16_t vps_ols_dpb_pic_height[VVC_MAX_TOTAL_NUM_OLSS]; + uint8_t vps_ols_dpb_chroma_format[VVC_MAX_TOTAL_NUM_OLSS]; + uint8_t vps_ols_dpb_bitdepth_minus8[VVC_MAX_TOTAL_NUM_OLSS]; + uint16_t vps_ols_dpb_params_idx[VVC_MAX_TOTAL_NUM_OLSS]; + + uint8_t vps_timing_hrd_params_present_flag; + H266RawGeneralTimingHrdParameters vps_general_timing_hrd_parameters; + uint8_t vps_sublayer_cpb_params_present_flag; + uint16_t vps_num_ols_timing_hrd_params_minus1; + uint8_t vps_hrd_max_tid[VVC_MAX_TOTAL_NUM_OLSS]; + H266RawOlsTimingHrdParameters vps_ols_timing_hrd_parameters; + uint8_t vps_ols_timing_hrd_idx[VVC_MAX_TOTAL_NUM_OLSS]; + + uint8_t vps_extension_flag; + H266RawExtensionData extension_data; +} H266RawVPS; + +typedef struct H266RawSPS { + H266RawNALUnitHeader nal_unit_header; + + uint8_t sps_seq_parameter_set_id; + uint8_t sps_video_parameter_set_id; + uint8_t sps_max_sublayers_minus1; + uint8_t sps_chroma_format_idc; + uint8_t sps_log2_ctu_size_minus5; + uint8_t sps_ptl_dpb_hrd_params_present_flag; + H266RawProfileTierLevel profile_tier_level; + uint8_t sps_gdr_enabled_flag; + uint8_t sps_ref_pic_resampling_enabled_flag; + uint8_t sps_res_change_in_clvs_allowed_flag; + + uint16_t sps_pic_width_max_in_luma_samples; + uint16_t sps_pic_height_max_in_luma_samples; + + uint8_t sps_conformance_window_flag; + uint16_t sps_conf_win_left_offset; + uint16_t sps_conf_win_right_offset; + uint16_t sps_conf_win_top_offset; + uint16_t sps_conf_win_bottom_offset; + + uint8_t sps_subpic_info_present_flag; + uint16_t sps_num_subpics_minus1; + uint8_t sps_independent_subpics_flag; + uint8_t sps_subpic_same_size_flag; + uint16_t sps_subpic_ctu_top_left_x[VVC_MAX_SLICES]; + uint16_t sps_subpic_ctu_top_left_y[VVC_MAX_SLICES]; + uint16_t sps_subpic_width_minus1[VVC_MAX_SLICES]; + uint16_t sps_subpic_height_minus1[VVC_MAX_SLICES]; + uint8_t sps_subpic_treated_as_pic_flag[VVC_MAX_SLICES]; + uint8_t sps_loop_filter_across_subpic_enabled_flag[VVC_MAX_SLICES]; + uint8_t sps_subpic_id_len_minus1; + uint8_t sps_subpic_id_mapping_explicitly_signalled_flag; + uint8_t sps_subpic_id_mapping_present_flag; + uint32_t sps_subpic_id[VVC_MAX_SLICES]; + + + uint8_t sps_bitdepth_minus8; + uint8_t sps_entropy_coding_sync_enabled_flag; + uint8_t sps_entry_point_offsets_present_flag; + + uint8_t sps_log2_max_pic_order_cnt_lsb_minus4; + uint8_t sps_poc_msb_cycle_flag; + uint8_t sps_poc_msb_cycle_len_minus1; + + uint8_t sps_num_extra_ph_bytes; + uint8_t sps_extra_ph_bit_present_flag[16]; + + uint8_t sps_num_extra_sh_bytes; + uint8_t sps_extra_sh_bit_present_flag[16]; + + uint8_t sps_sublayer_dpb_params_flag; + H266DpbParameters sps_dpb_params; + + uint8_t sps_log2_min_luma_coding_block_size_minus2; + uint8_t sps_partition_constraints_override_enabled_flag; + uint8_t sps_log2_diff_min_qt_min_cb_intra_slice_luma; + uint8_t sps_max_mtt_hierarchy_depth_intra_slice_luma; + uint8_t sps_log2_diff_max_bt_min_qt_intra_slice_luma; + uint8_t sps_log2_diff_max_tt_min_qt_intra_slice_luma; + + uint8_t sps_qtbtt_dual_tree_intra_flag; + uint8_t sps_log2_diff_min_qt_min_cb_intra_slice_chroma; + uint8_t sps_max_mtt_hierarchy_depth_intra_slice_chroma; + uint8_t sps_log2_diff_max_bt_min_qt_intra_slice_chroma; + uint8_t sps_log2_diff_max_tt_min_qt_intra_slice_chroma; + + uint8_t sps_log2_diff_min_qt_min_cb_inter_slice; + uint8_t sps_max_mtt_hierarchy_depth_inter_slice; + uint8_t sps_log2_diff_max_bt_min_qt_inter_slice; + uint8_t sps_log2_diff_max_tt_min_qt_inter_slice; + + uint8_t sps_max_luma_transform_size_64_flag; + + uint8_t sps_transform_skip_enabled_flag; + uint8_t sps_log2_transform_skip_max_size_minus2; + uint8_t sps_bdpcm_enabled_flag; + + uint8_t sps_mts_enabled_flag; + uint8_t sps_explicit_mts_intra_enabled_flag; + uint8_t sps_explicit_mts_inter_enabled_flag; + + uint8_t sps_lfnst_enabled_flag; + + uint8_t sps_joint_cbcr_enabled_flag; + uint8_t sps_same_qp_table_for_chroma_flag; + + int8_t sps_qp_table_start_minus26[VVC_MAX_SAMPLE_ARRAYS]; + uint8_t sps_num_points_in_qp_table_minus1[VVC_MAX_SAMPLE_ARRAYS]; + uint8_t sps_delta_qp_in_val_minus1[VVC_MAX_SAMPLE_ARRAYS][VVC_MAX_POINTS_IN_QP_TABLE]; + uint8_t sps_delta_qp_diff_val[VVC_MAX_SAMPLE_ARRAYS][VVC_MAX_POINTS_IN_QP_TABLE]; + + uint8_t sps_sao_enabled_flag; + uint8_t sps_alf_enabled_flag; + uint8_t sps_ccalf_enabled_flag; + uint8_t sps_lmcs_enabled_flag; + uint8_t sps_weighted_pred_flag; + uint8_t sps_weighted_bipred_flag; + uint8_t sps_long_term_ref_pics_flag; + uint8_t sps_inter_layer_prediction_enabled_flag; + uint8_t sps_idr_rpl_present_flag; + uint8_t sps_rpl1_same_as_rpl0_flag; + + uint8_t sps_num_ref_pic_lists[2]; + H266RefPicListStruct sps_ref_pic_list_struct[2][VVC_MAX_REF_PIC_LISTS]; + + uint8_t sps_ref_wraparound_enabled_flag; + uint8_t sps_temporal_mvp_enabled_flag; + uint8_t sps_sbtmvp_enabled_flag; + uint8_t sps_amvr_enabled_flag; + uint8_t sps_bdof_enabled_flag; + uint8_t sps_bdof_control_present_in_ph_flag; + uint8_t sps_smvd_enabled_flag; + uint8_t sps_dmvr_enabled_flag; + uint8_t sps_dmvr_control_present_in_ph_flag; + uint8_t sps_mmvd_enabled_flag; + uint8_t sps_mmvd_fullpel_only_enabled_flag; + uint8_t sps_six_minus_max_num_merge_cand; + uint8_t sps_sbt_enabled_flag; + uint8_t sps_affine_enabled_flag; + uint8_t sps_five_minus_max_num_subblock_merge_cand; + uint8_t sps_6param_affine_enabled_flag; + uint8_t sps_affine_amvr_enabled_flag; + uint8_t sps_affine_prof_enabled_flag; + uint8_t sps_prof_control_present_in_ph_flag; + uint8_t sps_bcw_enabled_flag; + uint8_t sps_ciip_enabled_flag; + uint8_t sps_gpm_enabled_flag; + uint8_t sps_max_num_merge_cand_minus_max_num_gpm_cand; + uint8_t sps_log2_parallel_merge_level_minus2; + uint8_t sps_isp_enabled_flag; + uint8_t sps_mrl_enabled_flag; + uint8_t sps_mip_enabled_flag; + uint8_t sps_cclm_enabled_flag; + uint8_t sps_chroma_horizontal_collocated_flag; + uint8_t sps_chroma_vertical_collocated_flag; + uint8_t sps_palette_enabled_flag; + uint8_t sps_act_enabled_flag; + uint8_t sps_min_qp_prime_ts; + uint8_t sps_ibc_enabled_flag; + uint8_t sps_six_minus_max_num_ibc_merge_cand; + uint8_t sps_ladf_enabled_flag; + uint8_t sps_num_ladf_intervals_minus2; + int8_t sps_ladf_lowest_interval_qp_offset; + int8_t sps_ladf_qp_offset[4]; + uint16_t sps_ladf_delta_threshold_minus1[4]; + + uint8_t sps_explicit_scaling_list_enabled_flag; + uint8_t sps_scaling_matrix_for_lfnst_disabled_flag; + uint8_t sps_scaling_matrix_for_alternative_colour_space_disabled_flag; + uint8_t sps_scaling_matrix_designated_colour_space_flag; + uint8_t sps_dep_quant_enabled_flag; + uint8_t sps_sign_data_hiding_enabled_flag; + + uint8_t sps_virtual_boundaries_enabled_flag; + uint8_t sps_virtual_boundaries_present_flag; + uint8_t sps_num_ver_virtual_boundaries; + uint16_t sps_virtual_boundary_pos_x_minus1[3]; + uint8_t sps_num_hor_virtual_boundaries; + uint16_t sps_virtual_boundary_pos_y_minus1[3]; + + uint8_t sps_timing_hrd_params_present_flag; + uint8_t sps_sublayer_cpb_params_present_flag; + H266RawGeneralTimingHrdParameters sps_general_timing_hrd_parameters; + H266RawOlsTimingHrdParameters sps_ols_timing_hrd_parameters; + + uint8_t sps_field_seq_flag; + uint8_t sps_vui_parameters_present_flag; + uint16_t sps_vui_payload_size_minus1; + H266RawVUI vui; + + uint8_t sps_extension_flag; + + H266RawExtensionData extension_data; + +} H266RawSPS; + +typedef struct H266RawPPS { + H266RawNALUnitHeader nal_unit_header; + + uint8_t pps_pic_parameter_set_id; + uint8_t pps_seq_parameter_set_id; + uint8_t pps_mixed_nalu_types_in_pic_flag; + uint16_t pps_pic_width_in_luma_samples; + uint16_t pps_pic_height_in_luma_samples; + + uint8_t pps_conformance_window_flag; + uint16_t pps_conf_win_left_offset; + uint16_t pps_conf_win_right_offset; + uint16_t pps_conf_win_top_offset; + uint16_t pps_conf_win_bottom_offset; + + uint8_t pps_scaling_window_explicit_signalling_flag; + int16_t pps_scaling_win_left_offset; + int16_t pps_scaling_win_right_offset; + int16_t pps_scaling_win_top_offset; + int16_t pps_scaling_win_bottom_offset; + + uint8_t pps_output_flag_present_flag; + uint8_t pps_no_pic_partition_flag; + + uint8_t pps_subpic_id_mapping_present_flag; + uint16_t pps_num_subpics_minus1; + uint8_t pps_subpic_id_len_minus1; + uint16_t pps_subpic_id[VVC_MAX_SLICES]; + + uint8_t pps_log2_ctu_size_minus5; + uint8_t pps_num_exp_tile_columns_minus1; + uint8_t pps_num_exp_tile_rows_minus1; + uint16_t pps_tile_column_width_minus1[VVC_MAX_TILE_COLUMNS]; + uint16_t pps_tile_row_height_minus1[VVC_MAX_TILE_ROWS]; + + uint8_t pps_loop_filter_across_tiles_enabled_flag; + uint8_t pps_rect_slice_flag; + uint8_t pps_single_slice_per_subpic_flag; + + uint16_t pps_num_slices_in_pic_minus1; + uint8_t pps_tile_idx_delta_present_flag; + uint16_t pps_slice_width_in_tiles_minus1[VVC_MAX_SLICES]; + uint16_t pps_slice_height_in_tiles_minus1[VVC_MAX_SLICES]; + uint16_t pps_num_exp_slices_in_tile[VVC_MAX_SLICES]; + uint16_t pps_exp_slice_height_in_ctus_minus1[VVC_MAX_SLICES][VVC_MAX_TILE_ROWS]; + int16_t pps_tile_idx_delta_val[VVC_MAX_SLICES]; + + uint8_t pps_loop_filter_across_slices_enabled_flag; + uint8_t pps_cabac_init_present_flag; + uint8_t pps_num_ref_idx_default_active_minus1[2]; + uint8_t pps_rpl1_idx_present_flag; + uint8_t pps_weighted_pred_flag; + uint8_t pps_weighted_bipred_flag; + uint8_t pps_ref_wraparound_enabled_flag; + uint16_t pps_pic_width_minus_wraparound_offset; + int8_t pps_init_qp_minus26; + uint8_t pps_cu_qp_delta_enabled_flag; + uint8_t pps_chroma_tool_offsets_present_flag; + int8_t pps_cb_qp_offset; + int8_t pps_cr_qp_offset; + uint8_t pps_joint_cbcr_qp_offset_present_flag; + int8_t pps_joint_cbcr_qp_offset_value; + uint8_t pps_slice_chroma_qp_offsets_present_flag; + uint8_t pps_cu_chroma_qp_offset_list_enabled_flag; + uint8_t pps_chroma_qp_offset_list_len_minus1; + uint8_t pps_cb_qp_offset_list[6]; + uint8_t pps_cr_qp_offset_list[6]; + uint8_t pps_joint_cbcr_qp_offset_list[6]; + uint8_t pps_deblocking_filter_control_present_flag; + uint8_t pps_deblocking_filter_override_enabled_flag; + uint8_t pps_deblocking_filter_disabled_flag; + uint8_t pps_dbf_info_in_ph_flag; + + int8_t pps_luma_beta_offset_div2; + int8_t pps_luma_tc_offset_div2; + int8_t pps_cb_beta_offset_div2; + int8_t pps_cb_tc_offset_div2; + int8_t pps_cr_beta_offset_div2; + int8_t pps_cr_tc_offset_div2; + + uint8_t pps_rpl_info_in_ph_flag; + uint8_t pps_sao_info_in_ph_flag; + uint8_t pps_alf_info_in_ph_flag; + uint8_t pps_wp_info_in_ph_flag; + uint8_t pps_qp_delta_info_in_ph_flag; + + uint8_t pps_picture_header_extension_present_flag; + uint8_t pps_slice_header_extension_present_flag; + uint8_t pps_extension_flag; + H266RawExtensionData extension_data; + + //calculated value; + uint16_t num_tile_columns; + uint16_t num_tile_rows; + uint16_t num_tiles_in_pic; + uint16_t slice_height_in_ctus[VVC_MAX_SLICES]; + uint16_t num_slices_in_subpic[VVC_MAX_SLICES]; +} H266RawPPS; + +typedef struct H266RawAUD { + H266RawNALUnitHeader nal_unit_header; + uint8_t aud_irap_or_gdr_flag; + uint8_t aud_pic_type; +} H266RawAUD; + +typedef struct H266RawPredWeightTable { + uint8_t luma_log2_weight_denom; + int8_t delta_chroma_log2_weight_denom; + + uint8_t num_l0_weights; + uint8_t luma_weight_l0_flag[15]; + uint8_t chroma_weight_l0_flag[15]; + int8_t delta_luma_weight_l0[15]; + int8_t luma_offset_l0[15]; + int8_t delta_chroma_weight_l0[15][2]; + int16_t delta_chroma_offset_l0[15][2]; + + uint8_t num_l1_weights; + uint8_t luma_weight_l1_flag[15]; + uint8_t chroma_weight_l1_flag[15]; + int8_t delta_luma_weight_l1[15]; + int8_t luma_offset_l1[15]; + int8_t delta_chroma_weight_l1[15][2]; + int16_t delta_chroma_offset_l1[15][2]; +} H266RawPredWeightTable; + +typedef struct H266RawPH { + H266RawNALUnitHeader nal_unit_header; + uint8_t ph_gdr_or_irap_pic_flag; + uint8_t ph_non_ref_pic_flag; + uint8_t ph_gdr_pic_flag; + uint8_t ph_inter_slice_allowed_flag; + uint8_t ph_intra_slice_allowed_flag; + uint8_t ph_pic_parameter_set_id; + uint16_t ph_pic_order_cnt_lsb; + uint8_t ph_recovery_poc_cnt; + uint8_t ph_extra_bit[16]; + uint8_t ph_poc_msb_cycle_present_flag; + uint8_t ph_poc_msb_cycle_val; + + uint8_t ph_alf_enabled_flag; + uint8_t ph_num_alf_aps_ids_luma; + uint8_t ph_alf_aps_id_luma[8]; + uint8_t ph_alf_cb_enabled_flag; + uint8_t ph_alf_cr_enabled_flag; + uint8_t ph_alf_aps_id_chroma; + uint8_t ph_alf_cc_cb_enabled_flag; + uint8_t ph_alf_cc_cb_aps_id; + uint8_t ph_alf_cc_cr_enabled_flag; + uint8_t ph_alf_cc_cr_aps_id; + + uint8_t ph_lmcs_enabled_flag; + uint8_t ph_lmcs_aps_id; + uint8_t ph_chroma_residual_scale_flag; + uint8_t ph_explicit_scaling_list_enabled_flag; + uint8_t ph_scaling_list_aps_id; + + uint8_t ph_virtual_boundaries_present_flag; + uint8_t ph_num_ver_virtual_boundaries; + uint16_t ph_virtual_boundary_pos_x_minus1[3]; + uint8_t ph_num_hor_virtual_boundaries; + uint16_t ph_virtual_boundary_pos_y_minus1[3]; + + uint8_t ph_pic_output_flag; + H266RefPicLists ph_ref_pic_lists; + + uint8_t ph_partition_constraints_override_flag; + + uint8_t ph_log2_diff_min_qt_min_cb_intra_slice_luma; + uint8_t ph_max_mtt_hierarchy_depth_intra_slice_luma; + uint8_t ph_log2_diff_max_bt_min_qt_intra_slice_luma; + uint8_t ph_log2_diff_max_tt_min_qt_intra_slice_luma; + uint8_t ph_log2_diff_min_qt_min_cb_intra_slice_chroma; + + uint8_t ph_max_mtt_hierarchy_depth_intra_slice_chroma; + uint8_t ph_log2_diff_max_bt_min_qt_intra_slice_chroma; + uint8_t ph_log2_diff_max_tt_min_qt_intra_slice_chroma; + + uint8_t ph_cu_qp_delta_subdiv_intra_slice; + uint8_t ph_cu_chroma_qp_offset_subdiv_intra_slice; + + uint8_t ph_log2_diff_min_qt_min_cb_inter_slice; + uint8_t ph_max_mtt_hierarchy_depth_inter_slice; + uint8_t ph_log2_diff_max_bt_min_qt_inter_slice; + uint8_t ph_log2_diff_max_tt_min_qt_inter_slice; + uint8_t ph_cu_qp_delta_subdiv_inter_slice; + uint8_t ph_cu_chroma_qp_offset_subdiv_inter_slice; + + uint8_t ph_temporal_mvp_enabled_flag; + uint8_t ph_collocated_from_l0_flag; + uint8_t ph_collocated_ref_idx; + uint8_t ph_mmvd_fullpel_only_flag; + uint8_t ph_mvd_l1_zero_flag; + uint8_t ph_bdof_disabled_flag; + uint8_t ph_dmvr_disabled_flag; + uint8_t ph_prof_disabled_flag; + + H266RawPredWeightTable ph_pred_weight_table; + + int8_t ph_qp_delta; + uint8_t ph_joint_cbcr_sign_flag; + uint8_t ph_sao_luma_enabled_flag; + uint8_t ph_sao_chroma_enabled_flag; + + uint8_t ph_deblocking_params_present_flag; + uint8_t ph_deblocking_filter_disabled_flag; + int8_t ph_luma_beta_offset_div2; + int8_t ph_luma_tc_offset_div2; + int8_t ph_cb_beta_offset_div2; + int8_t ph_cb_tc_offset_div2; + int8_t ph_cr_beta_offset_div2; + int8_t ph_cr_tc_offset_div2; + + uint8_t ph_extension_length; + uint8_t ph_extension_data_byte[256]; +} H266RawPH; + +typedef struct H266RawSliceHeader { + H266RawNALUnitHeader nal_unit_header; + uint8_t sh_picture_header_in_slice_header_flag; + H266RawPH sh_picture_header; + + uint16_t sh_subpic_id; + uint16_t sh_slice_address; + uint8_t sh_extra_bit[16]; + uint8_t sh_num_tiles_in_slice_minus1; + uint8_t sh_slice_type; + uint8_t sh_no_output_of_prior_pics_flag; + + uint8_t sh_alf_enabled_flag; + uint8_t sh_num_alf_aps_ids_luma; + uint8_t sh_alf_aps_id_luma[8]; + uint8_t sh_alf_cb_enabled_flag; + uint8_t sh_alf_cr_enabled_flag; + uint8_t sh_alf_aps_id_chroma; + uint8_t sh_alf_cc_cb_enabled_flag; + uint8_t sh_alf_cc_cb_aps_id; + uint8_t sh_alf_cc_cr_enabled_flag; + uint8_t sh_alf_cc_cr_aps_id; + + uint8_t sh_lmcs_used_flag; + uint8_t sh_explicit_scaling_list_used_flag; + + H266RefPicLists sh_ref_pic_lists; + + uint8_t sh_num_ref_idx_active_override_flag; + uint8_t sh_num_ref_idx_active_minus1[2]; + uint8_t sh_cabac_init_flag; + uint8_t sh_collocated_from_l0_flag; + uint8_t sh_collocated_ref_idx; + + H266RawPredWeightTable sh_pred_weight_table; + + int8_t sh_qp_delta; + int8_t sh_cb_qp_offset; + int8_t sh_cr_qp_offset; + int8_t sh_joint_cbcr_qp_offset; + uint8_t sh_cu_chroma_qp_offset_enabled_flag; + + uint8_t sh_sao_luma_used_flag; + uint8_t sh_sao_chroma_used_flag; + + uint8_t sh_deblocking_params_present_flag; + uint8_t sh_deblocking_filter_disabled_flag; + int8_t sh_luma_beta_offset_div2; + int8_t sh_luma_tc_offset_div2; + int8_t sh_cb_beta_offset_div2; + int8_t sh_cb_tc_offset_div2; + int8_t sh_cr_beta_offset_div2; + int8_t sh_cr_tc_offset_div2; + uint8_t sh_dep_quant_used_flag; + + uint8_t sh_sign_data_hiding_used_flag; + uint8_t sh_ts_residual_coding_disabled_flag; + uint16_t sh_slice_header_extension_length; + uint8_t sh_slice_header_extension_data_byte[256]; + + uint8_t sh_entry_offset_len_minus1; + uint32_t sh_entry_point_offset_minus1[VVC_MAX_ENTRY_POINTS]; + +} H266RawSliceHeader; + +typedef struct H266RawSlice { + H266RawSliceHeader header; + + uint8_t *data; + AVBufferRef *data_ref; + size_t data_size; + int data_bit_start; +} H266RawSlice; + +typedef struct H266RawSEIDecodedPictureHash { + uint8_t dph_sei_hash_type; + uint8_t dph_sei_single_component_flag; + uint8_t dph_sei_picture_md5[3][16]; + uint16_t dph_sei_picture_crc[3]; + uint32_t dph_sei_picture_checksum[3]; + + uint8_t dph_sei_reserved_zero_7bits; +} H266RawSEIDecodedPictureHash; + +typedef struct H266RawSEI { + H266RawNALUnitHeader nal_unit_header; + SEIRawMessageList message_list; +} H266RawSEI; + +typedef struct CodedBitstreamH266Context { + // Reader/writer context in common with the H.264 implementation. + CodedBitstreamH2645Context common; + + // All currently available parameter sets. These are updated when + // any parameter set NAL unit is read/written with this context. + AVBufferRef *vps_ref[VVC_MAX_VPS_COUNT]; + AVBufferRef *sps_ref[VVC_MAX_SPS_COUNT]; + AVBufferRef *pps_ref[VVC_MAX_PPS_COUNT]; + H266RawVPS *vps[VVC_MAX_SPS_COUNT]; + H266RawSPS *sps[VVC_MAX_SPS_COUNT]; + H266RawPPS *pps[VVC_MAX_PPS_COUNT]; + + struct { + AVBufferRef *ph_ref; + H266RawPH *ph; + } priv; + + // The currently active parameter sets. These are updated when any + // NAL unit refers to the relevant parameter set. These pointers + // must also be present in the arrays above. + const H266RawVPS *active_vps; + const H266RawSPS *active_sps; + const H266RawPPS *active_pps; + +} CodedBitstreamH266Context; + +#endif /* AVCODEC_CBS_H266_H */ diff --git a/libavcodec/cbs_h266_syntax_template.c b/libavcodec/cbs_h266_syntax_template.c new file mode 100644 index 0000000000..21e58fd9d1 --- /dev/null +++ b/libavcodec/cbs_h266_syntax_template.c @@ -0,0 +1,3010 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +static int FUNC(rbsp_trailing_bits)(CodedBitstreamContext *ctx, RWContext *rw) +{ + int err; + + fixed(1, rbsp_stop_one_bit, 1); + while (byte_alignment(rw) != 0) + fixed(1, rbsp_alignment_zero_bit, 0); + return 0; +} + +static int FUNC(nal_unit_header)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawNALUnitHeader *current, + int expected_nal_unit_type) +{ + int err; + + fixed(1, forbidden_zero_bit, 0); + flag(nuh_reserved_zero_bit); + + u(6, nuh_layer_id, 0, 55); + + if (expected_nal_unit_type >= 0) + u(5, nal_unit_type, expected_nal_unit_type, + expected_nal_unit_type); + else + ub(5, nal_unit_type); + + u(3, nuh_temporal_id_plus1, 1, 7); + return 0; +} + +static int FUNC(byte_alignment)(CodedBitstreamContext *ctx, RWContext *rw) +{ + int err; + + fixed(1, byte_alignment_bit_equal_to_one, 1); + while (byte_alignment(rw) != 0) + fixed(1, byte_alignment_bit_equal_to_zero, 0); + return 0; +} + +static int FUNC(general_constraints_info)(CodedBitstreamContext *ctx, + RWContext *rw, + H266GeneralConstraintsInfo *current) +{ + int err, i; + + flag(gci_present_flag); + if (current->gci_present_flag) { + /* general */ + flag(gci_intra_only_constraint_flag); + flag(gci_all_layers_independent_constraint_flag); + flag(gci_one_au_only_constraint_flag); + + /* picture format */ + u(4, gci_sixteen_minus_max_bitdepth_constraint_idc, 0, 8); + ub(2, gci_three_minus_max_chroma_format_constraint_idc); + + /* NAL unit type related */ + flag(gci_no_mixed_nalu_types_in_pic_constraint_flag); + flag(gci_no_trail_constraint_flag); + flag(gci_no_stsa_constraint_flag); + flag(gci_no_rasl_constraint_flag); + flag(gci_no_radl_constraint_flag); + flag(gci_no_idr_constraint_flag); + flag(gci_no_cra_constraint_flag); + flag(gci_no_gdr_constraint_flag); + flag(gci_no_aps_constraint_flag); + flag(gci_no_idr_rpl_constraint_flag); + + /* tile, slice, subpicture partitioning */ + flag(gci_one_tile_per_pic_constraint_flag); + flag(gci_pic_header_in_slice_header_constraint_flag); + flag(gci_one_slice_per_pic_constraint_flag); + flag(gci_no_rectangular_slice_constraint_flag); + flag(gci_one_slice_per_subpic_constraint_flag); + flag(gci_no_subpic_info_constraint_flag); + + /* CTU and block partitioning */ + ub(2, gci_three_minus_max_log2_ctu_size_constraint_idc); + flag(gci_no_partition_constraints_override_constraint_flag); + flag(gci_no_mtt_constraint_flag); + flag(gci_no_qtbtt_dual_tree_intra_constraint_flag); + + /* intra */ + flag(gci_no_palette_constraint_flag); + flag(gci_no_ibc_constraint_flag); + flag(gci_no_isp_constraint_flag); + flag(gci_no_mrl_constraint_flag); + flag(gci_no_mip_constraint_flag); + flag(gci_no_cclm_constraint_flag); + + /* inter */ + flag(gci_no_ref_pic_resampling_constraint_flag); + flag(gci_no_res_change_in_clvs_constraint_flag); + flag(gci_no_weighted_prediction_constraint_flag); + flag(gci_no_ref_wraparound_constraint_flag); + flag(gci_no_temporal_mvp_constraint_flag); + flag(gci_no_sbtmvp_constraint_flag); + flag(gci_no_amvr_constraint_flag); + flag(gci_no_bdof_constraint_flag); + flag(gci_no_smvd_constraint_flag); + flag(gci_no_dmvr_constraint_flag); + flag(gci_no_mmvd_constraint_flag); + flag(gci_no_affine_motion_constraint_flag); + flag(gci_no_prof_constraint_flag); + flag(gci_no_bcw_constraint_flag); + flag(gci_no_ciip_constraint_flag); + flag(gci_no_gpm_constraint_flag); + + /* transform, quantization, residual */ + flag(gci_no_luma_transform_size_64_constraint_flag); + flag(gci_no_transform_skip_constraint_flag); + flag(gci_no_bdpcm_constraint_flag); + flag(gci_no_mts_constraint_flag); + flag(gci_no_lfnst_constraint_flag); + flag(gci_no_joint_cbcr_constraint_flag); + flag(gci_no_sbt_constraint_flag); + flag(gci_no_act_constraint_flag); + flag(gci_no_explicit_scaling_list_constraint_flag); + flag(gci_no_dep_quant_constraint_flag); + flag(gci_no_sign_data_hiding_constraint_flag); + flag(gci_no_cu_qp_delta_constraint_flag); + flag(gci_no_chroma_qp_offset_constraint_flag); + + /* loop filter */ + flag(gci_no_sao_constraint_flag); + flag(gci_no_alf_constraint_flag); + flag(gci_no_ccalf_constraint_flag); + flag(gci_no_lmcs_constraint_flag); + flag(gci_no_ladf_constraint_flag); + flag(gci_no_virtual_boundaries_constraint_flag); + ub(8, gci_num_reserved_bits); + for (i = 0; i < current->gci_num_reserved_bits; i++) { + flags(gci_reserved_zero_bit[i], 1, i); + } + } + while (byte_alignment(rw) != 0) + fixed(1, gci_alignment_zero_bit, 0); + return 0; +} + +static int FUNC(profile_tier_level)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawProfileTierLevel *current, + int profile_tier_present_flag, + int max_num_sub_layers_minus1) +{ + int err, i; + + if (profile_tier_present_flag) { + ub(7, general_profile_idc); + flag(general_tier_flag); + } + ub(8, general_level_idc); + flag(ptl_frame_only_constraint_flag); + flag(ptl_multilayer_enabled_flag); + if (profile_tier_present_flag) { + CHECK(FUNC(general_constraints_info)(ctx, rw, + ¤t->general_constraints_info)); + } + for (i = max_num_sub_layers_minus1 - 1; i >= 0; i--) + flags(ptl_sublayer_level_present_flag[i], 1, i); + while (byte_alignment(rw) != 0) + flag(ptl_reserved_zero_bit); + for (i = max_num_sub_layers_minus1 - 1; i >= 0; i--) + if (current->ptl_sublayer_level_present_flag[i]) + ubs(8, sublayer_level_idc[i], 1, i); + if (profile_tier_present_flag) { + ub(8, ptl_num_sub_profiles); + for (i = 0; i < current->ptl_num_sub_profiles; i++) + ubs(32, general_sub_profile_idc[i], 1, i); + } + return 0; +} + +static int FUNC(vui_parameters_default)(CodedBitstreamContext *ctx, + RWContext *rw, H266RawVUI *current) +{ + //defined in D.8 + infer(vui_progressive_source_flag, 0); + infer(vui_interlaced_source_flag, 0); + + infer(vui_non_packed_constraint_flag, 0); + infer(vui_non_projected_constraint_flag, 0); + + infer(vui_aspect_ratio_constant_flag, 0); + infer(vui_aspect_ratio_idc, 0); + + infer(vui_overscan_info_present_flag, 0); + + infer(vui_colour_primaries, 2); + infer(vui_transfer_characteristics, 2); + infer(vui_matrix_coeffs, 2); + infer(vui_full_range_flag, 0); + + infer(vui_chroma_sample_loc_type_frame, 6); + infer(vui_chroma_sample_loc_type_top_field, 6); + infer(vui_chroma_sample_loc_type_bottom_field, 6); + return 0; +} + +static int FUNC(vui_parameters)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawVUI *current, uint8_t chroma_format_idc) +{ + int err; + + flag(vui_progressive_source_flag); + flag(vui_interlaced_source_flag); + flag(vui_non_packed_constraint_flag); + flag(vui_non_projected_constraint_flag); + flag(vui_aspect_ratio_info_present_flag); + if (current->vui_aspect_ratio_info_present_flag) { + flag(vui_aspect_ratio_constant_flag); + ub(8, vui_aspect_ratio_idc); + if (current->vui_aspect_ratio_idc == 255) { + ub(16, vui_sar_width); + ub(16, vui_sar_height); + } + } else { + infer(vui_aspect_ratio_constant_flag, 0); + infer(vui_aspect_ratio_idc, 0); + } + flag(vui_overscan_info_present_flag); + if (current->vui_overscan_info_present_flag) + flag(vui_overscan_appropriate_flag); + flag(vui_colour_description_present_flag); + if (current->vui_colour_description_present_flag) { + ub(8, vui_colour_primaries); + av_log(ctx->log_ctx, AV_LOG_DEBUG, "vui_colour_primaries == %d \n", current->vui_colour_primaries); + ub(8, vui_transfer_characteristics); + av_log(ctx->log_ctx, AV_LOG_DEBUG, "vui_transfer_characteristics == %d \n", current->vui_transfer_characteristics); + ub(8, vui_matrix_coeffs); + av_log(ctx->log_ctx, AV_LOG_DEBUG, "vui_matrix_coeffs == %d \n", current->vui_matrix_coeffs); + flag(vui_full_range_flag); + } else { + infer(vui_colour_primaries, 2); + infer(vui_transfer_characteristics, 2); + infer(vui_matrix_coeffs, 2); + infer(vui_full_range_flag, 0); + } + flag(vui_chroma_loc_info_present_flag); + if (chroma_format_idc != 1 && current->vui_chroma_loc_info_present_flag) { + av_log(ctx->log_ctx, AV_LOG_ERROR, "chroma_format_idc == %d," + "vui_chroma_loc_info_present_flag can't not be true", chroma_format_idc); + return AVERROR_INVALIDDATA; + } + if (current->vui_chroma_loc_info_present_flag) { + if (current->vui_progressive_source_flag && + !current->vui_interlaced_source_flag) { + ue(vui_chroma_sample_loc_type_frame, 0, 6); + } else { + ue(vui_chroma_sample_loc_type_top_field, 0, 6); + ue(vui_chroma_sample_loc_type_bottom_field, 0, 6); + } + } else { + if (chroma_format_idc == 1) { + infer(vui_chroma_sample_loc_type_frame, 6); + infer(vui_chroma_sample_loc_type_top_field, + current->vui_chroma_sample_loc_type_frame); + infer(vui_chroma_sample_loc_type_bottom_field, + current->vui_chroma_sample_loc_type_frame); + } + } + return 0; +} + +static int FUNC(payload_extension)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawExtensionData *current, + uint32_t payload_size, int cur_pos) +{ + int err; + size_t byte_length, k; + +#ifdef READ + GetBitContext tmp; + int bits_left, payload_zero_bits; + + if (!cbs_h265_payload_extension_present(rw, payload_size, cur_pos)) + return 0; + + bits_left = 8 * payload_size - cur_pos; + tmp = *rw; + if (bits_left > 8) + skip_bits_long(&tmp, bits_left - 8); + payload_zero_bits = get_bits(&tmp, FFMIN(bits_left, 8)); + if (!payload_zero_bits) + return AVERROR_INVALIDDATA; + payload_zero_bits = ff_ctz(payload_zero_bits); + current->bit_length = bits_left - payload_zero_bits - 1; + allocate(current->data, (current->bit_length + 7) / 8); +#endif + + byte_length = (current->bit_length + 7) / 8; + for (k = 0; k < byte_length; k++) { + int length = FFMIN(current->bit_length - k * 8, 8); + xu(length, reserved_payload_extension_data, current->data[k], + 0, MAX_UINT_BITS(length), 0); + } + + return 0; +} + +static int FUNC(vui_payload)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawVUI *current, uint16_t vui_payload_size, + uint8_t chroma_format_idc) +{ + int err; + int start_position, current_position; + + start_position = bit_position(rw); + CHECK(FUNC(vui_parameters)(ctx, rw, current, chroma_format_idc)); + current_position = bit_position(rw) - start_position; + + if (current_position < 8 * vui_payload_size) { + CHECK(FUNC(payload_extension)(ctx, rw, ¤t->extension_data, + vui_payload_size, current_position)); + fixed(1, vui_payload_bit_equal_to_one, 1); + while (byte_alignment(rw) != 0) + fixed(1, vui_payload_bit_equal_to_zero, 0); + } + return 0; +} + +static int FUNC(extension_data)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawExtensionData *current) +{ + int err; + size_t k; +#ifdef READ + GetBitContext start; + uint8_t bit; + start = *rw; + for (k = 0; cbs_h2645_read_more_rbsp_data(rw); k++) + skip_bits(rw, 1); + current->bit_length = k; + if (k > 0) { + *rw = start; + allocate(current->data, (current->bit_length + 7) / 8); + for (k = 0; k < current->bit_length; k++) { + xu(1, extension_data, bit, 0, 1, 0); + current->data[k / 8] |= bit << (7 - k % 8); + } + } +#else + for (k = 0; k < current->bit_length; k++) + xu(1, extension_data, current->data[k / 8] >> (7 - k % 8) & 1, 0, 1, 0); +#endif + return 0; +} + +static int FUNC(dpb_parameters)(CodedBitstreamContext *ctx, RWContext *rw, + H266DpbParameters *current, + uint8_t max_sublayers_minus1, + uint8_t sublayer_info_flag) +{ + int err, i; + for (i = (sublayer_info_flag ? 0 : max_sublayers_minus1); + i <= max_sublayers_minus1; i++) { + ues(dpb_max_dec_pic_buffering_minus1[i], + 0, VVC_MAX_DPB_SIZE - 1, 1, i); + ues(dpb_max_num_reorder_pics[i], + 0, current->dpb_max_dec_pic_buffering_minus1[i], 1, i); + ues(dpb_max_latency_increase_plus1[i], 0, UINT32_MAX - 1, 1, i); + } + return 0; +} + +static int FUNC(ref_pic_list_struct)(CodedBitstreamContext *ctx, RWContext *rw, + H266RefPicListStruct *current, + uint8_t list_idx, uint8_t rpls_idx, + const H266RawSPS *sps) +{ + CodedBitstreamH266Context *h266 = ctx->priv_data; + int err, i, j, general_layer_idx = -1, num_direct_ref_layers = 0; + const H266RawVPS *vps = h266->vps[sps->sps_video_parameter_set_id]; + + if (!vps) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "VPS id %d not available.\n", sps->sps_video_parameter_set_id); + return AVERROR_INVALIDDATA; + } + + //7.4.3.3 (29) + for (i = 0; i <= vps->vps_max_layers_minus1; i++) { + if (sps->nal_unit_header.nuh_layer_id == vps->vps_layer_id[i]) { + general_layer_idx = i; + break; + } + } + if (general_layer_idx < 0) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "vps_layer_id %d not available.\n", sps->nal_unit_header.nuh_layer_id); + return AVERROR_INVALIDDATA; + } + + //7.4.3.3 (28) + for (j = 0; j <= vps->vps_max_layers_minus1; j++) { + if (vps->vps_direct_ref_layer_flag[general_layer_idx][j]) + num_direct_ref_layers++; + } + + ue(num_ref_entries, 0, VVC_MAX_REF_ENTRIES); + if (sps->sps_long_term_ref_pics_flag && + rpls_idx < sps->sps_num_ref_pic_lists[list_idx] && + current->num_ref_entries > 0) + flag(ltrp_in_header_flag); + if (sps->sps_long_term_ref_pics_flag && + rpls_idx == sps->sps_num_ref_pic_lists[list_idx]) + infer(ltrp_in_header_flag, 1); + for (i = 0, j = 0; i < current->num_ref_entries; i++) { + if (sps->sps_inter_layer_prediction_enabled_flag) + flags(inter_layer_ref_pic_flag[i], 1, i); + else + infer(inter_layer_ref_pic_flag[i], 0); + + if (!current->inter_layer_ref_pic_flag[i]) { + if (sps->sps_long_term_ref_pics_flag) + flags(st_ref_pic_flag[i], 1, i); + else + infer(st_ref_pic_flag[i], 1); + if (current->st_ref_pic_flag[i]) { + int abs_delta_poc_st; + ues(abs_delta_poc_st[i], 0, MAX_UINT_BITS(15), 1, i); + if ((sps->sps_weighted_pred_flag || + sps->sps_weighted_bipred_flag) && i != 0) + abs_delta_poc_st = current->abs_delta_poc_st[i]; + else + abs_delta_poc_st = current->abs_delta_poc_st[i] + 1; + if (abs_delta_poc_st > 0) + flags(strp_entry_sign_flag[i], 1, i); + } else { + if (!current->ltrp_in_header_flag) { + uint8_t bits = sps->sps_log2_max_pic_order_cnt_lsb_minus4 + 4; + ubs(bits, rpls_poc_lsb_lt[j], 1, j); + j++; + } + } + } else { + if (num_direct_ref_layers == 0) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "num_direct_ref_layers needs > 0.\n"); + return AVERROR_INVALIDDATA; + } + ues(ilrp_idx[i], 0, num_direct_ref_layers - 1, 1, i); + } + } + return 0; +} + +static int FUNC(ref_pic_lists)(CodedBitstreamContext *ctx, RWContext *rw, + const H266RawSPS *sps, const H266RawPPS *pps, + H266RefPicLists *current) +{ + const H266RefPicListStruct *ref_list; + int err, i, j, num_ltrp_entries; + for (i = 0; i < 2; i++) { + if (sps->sps_num_ref_pic_lists[i] > 0 && + (i == 0 || (i == 1 && pps->pps_rpl1_idx_present_flag))) { + flags(rpl_sps_flag[i], 1, i); + } else { + if (sps->sps_num_ref_pic_lists[i] == 0) { + infer(rpl_sps_flag[i], 0); + } else { + if (!pps->pps_rpl1_idx_present_flag && i == 1) + infer(rpl_sps_flag[1], current->rpl_sps_flag[0]); + } + } + if (current->rpl_sps_flag[i]) { + if (sps->sps_num_ref_pic_lists[i] > 1 && + (i == 0 || (i == 1 && pps->pps_rpl1_idx_present_flag))) { + uint8_t bits = av_ceil_log2(sps->sps_num_ref_pic_lists[i]); + us(bits, rpl_idx[i], 0, sps->sps_num_ref_pic_lists[i] - 1, 1, i); + } else if (sps->sps_num_ref_pic_lists[i] == 1) { + infer(rpl_idx[i], 0); + } else if (i == 1 && !pps->pps_rpl1_idx_present_flag){ + infer(rpl_idx[1], current->rpl_idx[0]); + } else { + //how to handle this? or never happpend? + av_log(ctx->log_ctx, AV_LOG_ERROR, + "can't infer the rpl_idx[i]\n"); + return AVERROR_PATCHWELCOME; + } + memcpy(¤t->rpl_ref_list[i], + &sps->sps_ref_pic_list_struct[i][current->rpl_idx[i]], + sizeof(current->rpl_ref_list[i])); + } else { + CHECK(FUNC(ref_pic_list_struct)(ctx, rw, ¤t->rpl_ref_list[i], + i, sps->sps_num_ref_pic_lists[i], sps)); + } + ref_list = ¤t->rpl_ref_list[i]; + + num_ltrp_entries = 0; + for (int i = 0; i < ref_list->num_ref_entries; i++) { + if (!ref_list->inter_layer_ref_pic_flag[i]) { + if (!ref_list->st_ref_pic_flag[i]) { + num_ltrp_entries++; + } + } + } + + for (j = 0; j < num_ltrp_entries; j++) { + if (ref_list->ltrp_in_header_flag) { + ubs(sps->sps_log2_max_pic_order_cnt_lsb_minus4 + 4, + poc_lsb_lt[i][j], + 2, i, j); + } + flags(delta_poc_msb_cycle_present_flag[i][j], 2, i, j); + if (current->delta_poc_msb_cycle_present_flag[i][j]) { + uint32_t max = 1 << (32 - sps->sps_log2_max_pic_order_cnt_lsb_minus4 - 4); + ues(delta_poc_msb_cycle_lt[i][j], 0, max, 2, i, j); + } + } + } + return 0; +} + +static int FUNC(general_timing_hrd_parameters)(CodedBitstreamContext *ctx, + RWContext *rw, + H266RawGeneralTimingHrdParameters *current) +{ + int err; + ub(32, num_units_in_tick); + u(32, time_scale, 1, MAX_UINT_BITS(32)); + flag(general_nal_hrd_params_present_flag); + flag(general_vcl_hrd_params_present_flag); + + if (current->general_nal_hrd_params_present_flag || + current->general_vcl_hrd_params_present_flag) { + flag(general_same_pic_timing_in_all_ols_flag); + flag(general_du_hrd_params_present_flag); + if (current->general_du_hrd_params_present_flag) + ub(8, tick_divisor_minus2); + ub(4, bit_rate_scale); + ub(4, cpb_size_scale); + if (current->general_du_hrd_params_present_flag) + ub(4, cpb_size_du_scale); + ue(hrd_cpb_cnt_minus1, 0, 31); + } else { + //infer general_same_pic_timing_in_all_ols_flag? + infer(general_du_hrd_params_present_flag, 0); + } + return 0; +} + +static int FUNC(sublayer_hrd_parameters)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawSubLayerHRDParameters *current, int sublayer_id, + const H266RawGeneralTimingHrdParameters *general) +{ + int err, i; + for (i = 0; i <= general->hrd_cpb_cnt_minus1; i++) { + ues(bit_rate_value_minus1[sublayer_id][i], 0, UINT32_MAX - 1, 2, sublayer_id, i); + ues(cpb_size_value_minus1[sublayer_id][i], 0, UINT32_MAX - 1, 2, sublayer_id, i); + if (general->general_du_hrd_params_present_flag) { + ues(cpb_size_du_value_minus1[sublayer_id][i], + 0, UINT32_MAX - 1, 2, sublayer_id, i); + ues(bit_rate_du_value_minus1[sublayer_id][i], + 0, UINT32_MAX - 1, 2, sublayer_id, i); + } + flags(cbr_flag[sublayer_id][i], 2, sublayer_id, i); + } + return 0; +} + +static int FUNC(ols_timing_hrd_parameters)(CodedBitstreamContext *ctx, + RWContext *rw, H266RawOlsTimingHrdParameters *current, + uint8_t first_sublayer, uint8_t max_sublayers_minus1, + const H266RawGeneralTimingHrdParameters *general) +{ + int err, i; + for (i = first_sublayer; i <= max_sublayers_minus1; i++) { + flags(fixed_pic_rate_general_flag[i], 1, i); + if (!current->fixed_pic_rate_general_flag[i]) + flags(fixed_pic_rate_within_cvs_flag[i], 1, i); + else + infer(fixed_pic_rate_within_cvs_flag[i], 1); + if (current->fixed_pic_rate_within_cvs_flag[i]) { + ues(elemental_duration_in_tc_minus1[i], 0, 2047, 1, i); + infer(low_delay_hrd_flag[i], 0); + } else if ((general->general_nal_hrd_params_present_flag || + general->general_vcl_hrd_params_present_flag) && + general->hrd_cpb_cnt_minus1 == 0) { + flags(low_delay_hrd_flag[i], 1, i); + } else { + infer(low_delay_hrd_flag[i], 0); + } + if (general->general_nal_hrd_params_present_flag) + CHECK(FUNC(sublayer_hrd_parameters)(ctx, rw, + ¤t->nal_sub_layer_hrd_parameters, + i, general)); + if (general->general_vcl_hrd_params_present_flag) + CHECK(FUNC(sublayer_hrd_parameters)(ctx, rw, + ¤t->nal_sub_layer_hrd_parameters, i, general)); + } + return 0; +} + +static int FUNC(vps)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawVPS *current) +{ + int err, i, j, k; + uint16_t total_num_olss = 0; + uint8_t ols_mode_idc = 0; + uint16_t num_multi_layer_olss = 0; + uint8_t layer_included_in_ols_flag[VVC_MAX_TOTAL_NUM_OLSS][VVC_MAX_LAYERS]; + uint8_t num_ref_layers[VVC_MAX_LAYERS]; + uint8_t reference_layer_idx[VVC_MAX_LAYERS][VVC_MAX_LAYERS]; + + HEADER("Video Parameter Set"); + + CHECK(FUNC(nal_unit_header)(ctx, rw, + ¤t->nal_unit_header, VVC_VPS_NUT)); + + ub(4, vps_video_parameter_set_id); + if (current->vps_video_parameter_set_id == 0) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "vps_video_parameter_set_id should > 0.\n"); + return AVERROR_INVALIDDATA; + } + ub(6, vps_max_layers_minus1); + u(3, vps_max_sublayers_minus1, 0, 6); + if (current->vps_max_layers_minus1 > 0 && current->vps_max_sublayers_minus1 > 0) + flag(vps_default_ptl_dpb_hrd_max_tid_flag); + else + infer(vps_default_ptl_dpb_hrd_max_tid_flag, 1); + + if (current->vps_max_layers_minus1 > 0) + flag(vps_all_independent_layers_flag); + else + infer(vps_all_independent_layers_flag, 1); + + for (i = 0; i <= current->vps_max_layers_minus1; i++) { + ubs(6, vps_layer_id[i], 1, i); + if (i > 0 && current->vps_layer_id[i] <= current->vps_layer_id[i - 1]) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "vps_layer_id[%d](%d) should > vps_layer_id[%d](%d).\n", + i, current->vps_layer_id[i], i - 1, current->vps_layer_id[i - 1]); + return AVERROR_INVALIDDATA; + } + if (i > 0 && !current->vps_all_independent_layers_flag) { + flags(vps_independent_layer_flag[i], 1, i); + if (!current->vps_independent_layer_flag[i]) { + flags(vps_max_tid_ref_present_flag[i], 1, i); + for (j = 0; j < i; j++) { + flags(vps_direct_ref_layer_flag[i][j], 2, i, j); + if (current->vps_max_tid_ref_present_flag[i] && + current->vps_direct_ref_layer_flag[i][j]) { + ubs(3, vps_max_tid_il_ref_pics_plus1[i][j], 2, i, j); + } else { + infer(vps_max_tid_il_ref_pics_plus1[i][j], + current->vps_max_sublayers_minus1 + 1); + } + } + } else { + for (j = 0; j < i; j++) { + infer(vps_direct_ref_layer_flag[i][j], 0); + } + } + } else { + infer(vps_independent_layer_flag[i], 1); + for (j = 0; j < i; j++) { + infer(vps_direct_ref_layer_flag[i][j], 0); + } + } + } + + if (current->vps_max_layers_minus1 > 0) { + if (current->vps_all_independent_layers_flag) + flag(vps_each_layer_is_an_ols_flag); + else + infer(vps_each_layer_is_an_ols_flag, 0); + if (!current->vps_each_layer_is_an_ols_flag) { + if (!current->vps_all_independent_layers_flag) + ub(2, vps_ols_mode_idc); + else + infer(vps_ols_mode_idc, 2); + if (current->vps_ols_mode_idc == 2) { + ub(8, vps_num_output_layer_sets_minus2); + for (i = 1; i <= current->vps_num_output_layer_sets_minus2 + 1; i++) + for (j = 0; j <= current->vps_max_layers_minus1; j++) + flags(vps_ols_output_layer_flag[i][j], 2, i, j); + } + ols_mode_idc = current->vps_ols_mode_idc; + } else { + ols_mode_idc = 4; + } + if (ols_mode_idc == 4 || ols_mode_idc == 0 || ols_mode_idc == 1) + total_num_olss = current->vps_max_layers_minus1 + 1; + else if (ols_mode_idc = 2) + total_num_olss = current->vps_num_output_layer_sets_minus2 + 2; + else + av_log(ctx->log_ctx, AV_LOG_ERROR, "ols_mode_idc == 3, patch welcome"); + u(8, vps_num_ptls_minus1, 0, total_num_olss - 1); + } else { + infer(vps_each_layer_is_an_ols_flag, 1); + infer(vps_num_ptls_minus1, 0); + } + { + //let's get calc NumMultiLayerOlss + int m; + uint8_t dependency_flag[VVC_MAX_LAYERS][VVC_MAX_LAYERS]; + uint16_t num_output_layers_in_ols[VVC_MAX_TOTAL_NUM_OLSS]; + uint8_t num_sub_layers_in_layer_in_ols[VVC_MAX_TOTAL_NUM_OLSS][VVC_MAX_TOTAL_NUM_OLSS]; + uint8_t output_layer_idx[VVC_MAX_TOTAL_NUM_OLSS][VVC_MAX_LAYERS]; + + //7.4.3.3 vps_direct_ref_layer_flag section + for (i = 0; i <= current->vps_max_layers_minus1; i++) { + for (j = 0; j <= current->vps_max_layers_minus1; j++) { + dependency_flag[i][j] = current->vps_direct_ref_layer_flag[i][j]; + for (k = 0; k < i; k++) { + if (current->vps_direct_ref_layer_flag[i][k] && dependency_flag[k][j]) + dependency_flag[i][j] = 1; + } + } + } + for (i = 0; i <= current->vps_max_layers_minus1; i++) { + int r; + for (j = 0, r = 0; j <= current->vps_max_layers_minus1; j++) { + if (dependency_flag[i][j]) + reference_layer_idx[i][r++] = j; + } + num_ref_layers[i] = r; + } + + //7.4.3.3 vps_ols_output_layer_flag section + num_output_layers_in_ols[0] = 1; + num_sub_layers_in_layer_in_ols[0][0] = current->vps_ptl_max_tid[current->vps_ols_ptl_idx[0]] + 1; + for (i = 1; i < total_num_olss; i++) { + if (ols_mode_idc == 4 || ols_mode_idc == 0) { + num_output_layers_in_ols[i] = 1; + if (current->vps_each_layer_is_an_ols_flag) { + num_sub_layers_in_layer_in_ols[i][0] = + current->vps_ptl_max_tid[current->vps_ols_ptl_idx[i]] + 1; + } else { + num_sub_layers_in_layer_in_ols[i][i] = + current->vps_ptl_max_tid[current->vps_ols_ptl_idx[i]] + 1; + for (k = i - 1; k >= 0; k--) { + num_sub_layers_in_layer_in_ols[i][k] = 0; + for (m = k + 1; m <= i; m++) { + uint8_t max_sublayer_needed = + FFMIN(num_sub_layers_in_layer_in_ols[i][m], + current->vps_max_tid_il_ref_pics_plus1[m][k]); + if (current->vps_direct_ref_layer_flag[m][k] && + num_sub_layers_in_layer_in_ols[i][k] < max_sublayer_needed) + num_sub_layers_in_layer_in_ols[i][k] = max_sublayer_needed; + } + } + } + } else if (current->vps_ols_mode_idc == 1) { + num_output_layers_in_ols[i] = i + 1; + for (j = 0; j < num_output_layers_in_ols[i]; j++) { + num_sub_layers_in_layer_in_ols[i][j] = + current->vps_ptl_max_tid[current->vps_ols_ptl_idx[i]] + 1; + } + } else if (current->vps_ols_mode_idc == 2) { + uint8_t highest_included_layer = 0; + for (j = 0; j <= current->vps_max_layers_minus1; j++) { + layer_included_in_ols_flag[i][j] = 0; + num_sub_layers_in_layer_in_ols[i][j] = 0; + } + for (k = 0, j = 0; k <= current->vps_max_layers_minus1; k++) { + if (current->vps_ols_output_layer_flag[i][k]) { + layer_included_in_ols_flag[i][k] = 1; + highest_included_layer = k; + output_layer_idx[i][j] = k; + num_sub_layers_in_layer_in_ols[i][k] = + current->vps_ptl_max_tid[current->vps_ols_ptl_idx[i]] + 1; + } + } + num_output_layers_in_ols[i] = j; + for (j = 0; j < num_output_layers_in_ols[i]; j++) { + int idx = output_layer_idx[i][j]; + for (k = 0; k < num_ref_layers[idx]; k++) { + if (!layer_included_in_ols_flag[i][reference_layer_idx[idx][k]]) + layer_included_in_ols_flag[i][reference_layer_idx[idx][k]] = 1; + } + } + for (k = highest_included_layer - 1; k >= 0; k--) { + if (layer_included_in_ols_flag[i][k] && !current->vps_ols_output_layer_flag[i][k]) { + for (m = k + 1; m <= highest_included_layer; m++) { + uint8_t max_sublayer_needed = + FFMIN(num_sub_layers_in_layer_in_ols[i][m], + current->vps_max_tid_il_ref_pics_plus1[m][k]); + if (current->vps_direct_ref_layer_flag[m][k] && + layer_included_in_ols_flag[i][m] && + num_sub_layers_in_layer_in_ols[i][k] < max_sublayer_needed) + num_sub_layers_in_layer_in_ols[i][k] = max_sublayer_needed; + } + } + } + } + } + for (i = 1; i < total_num_olss; i++) { + int num_layers_in_ols = 0; + if (current->vps_each_layer_is_an_ols_flag) { + num_layers_in_ols = 1; + } else if (current->vps_ols_mode_idc == 0 || current->vps_ols_mode_idc == 1) { + num_layers_in_ols = i + 1; + } else if (current->vps_ols_mode_idc == 2) { + for (k = 0, j = 0; k <= current->vps_max_layers_minus1; k++) { + if (layer_included_in_ols_flag[i][k]) + j++; + num_layers_in_ols = j; + } + } + if (num_layers_in_ols > 1) { + num_multi_layer_olss++; + } + } + } + + for (i = 0; i <= current->vps_num_ptls_minus1; i++) { + if (i > 0) + flags(vps_pt_present_flag[i], 1, i); + else + infer(vps_pt_present_flag[i], 1); + + if (!current->vps_default_ptl_dpb_hrd_max_tid_flag) + us(3, vps_ptl_max_tid[i], 0, current->vps_max_sublayers_minus1, 1, i); + else + infer(vps_ptl_max_tid[i], current->vps_max_sublayers_minus1); + } + while (byte_alignment(rw) != 0) + fixed(1, vps_ptl_alignment_zero_bit, 0); + for (i = 0; i <= current->vps_num_ptls_minus1; i++) { + CHECK(FUNC(profile_tier_level)(ctx, rw, + current->vps_profile_tier_level + i, + current->vps_pt_present_flag[i], + current->vps_ptl_max_tid[i])); + } + for (i = 0; i < total_num_olss; i++) { + if (current->vps_num_ptls_minus1 > 0 && + current->vps_num_ptls_minus1 + 1 != total_num_olss) { + us(8, vps_ols_ptl_idx[i], 0, current->vps_num_ptls_minus1, 1, i); + } + else if (current->vps_num_ptls_minus1 == 0) { + infer(vps_ols_ptl_idx[i], 0); + } else { + infer(vps_ols_ptl_idx[i], i); + } + } + + if (!current->vps_each_layer_is_an_ols_flag) { + uint16_t vps_num_dpb_params; + ue(vps_num_dpb_params_minus1, 0, num_multi_layer_olss - 1); + if (current->vps_each_layer_is_an_ols_flag) + vps_num_dpb_params = 0; + else + vps_num_dpb_params = current->vps_num_dpb_params_minus1 + 1; + + if (current->vps_max_sublayers_minus1 > 0) + flag(vps_sublayer_dpb_params_present_flag); + else + infer(vps_sublayer_dpb_params_present_flag, 0); + + for (i = 0; i < vps_num_dpb_params; i++) { + if (!current->vps_default_ptl_dpb_hrd_max_tid_flag) + us(3, vps_dpb_max_tid[i], 0, current->vps_max_sublayers_minus1, 1, i); + else + infer(vps_dpb_max_tid[i], current->vps_max_sublayers_minus1); + CHECK(FUNC(dpb_parameters)(ctx, rw, current->vps_dpb_params + i, + current->vps_dpb_max_tid[i], + current->vps_sublayer_dpb_params_present_flag)); + } + for (i = 0; i < num_multi_layer_olss; i++) { + ues(vps_ols_dpb_pic_width[i], 0, UINT16_MAX, 1, i); + ues(vps_ols_dpb_pic_height[i], 0, UINT16_MAX, 1, i); + ubs(2, vps_ols_dpb_chroma_format[i], 1, i); + ues(vps_ols_dpb_bitdepth_minus8[i], 0, 2, 1, i); + if (vps_num_dpb_params > 1 && vps_num_dpb_params != num_multi_layer_olss) + ues(vps_ols_dpb_params_idx[i], 0, vps_num_dpb_params - 1, 1, i); + else if (vps_num_dpb_params == 1) + infer(vps_ols_dpb_params_idx[i], 0); + else + infer(vps_ols_dpb_params_idx[i], i); + } + flag(vps_timing_hrd_params_present_flag); + if (current->vps_timing_hrd_params_present_flag) { + CHECK(FUNC(general_timing_hrd_parameters)(ctx, rw, + ¤t->vps_general_timing_hrd_parameters)); + if (current->vps_max_sublayers_minus1 > 0) + flag(vps_sublayer_cpb_params_present_flag); + else + infer(vps_sublayer_cpb_params_present_flag, 0); + ue(vps_num_ols_timing_hrd_params_minus1, 0, num_multi_layer_olss - 1); + for (i = 0; i <= current->vps_num_ols_timing_hrd_params_minus1; i++) { + uint8_t first_sublayer; + if (!current->vps_default_ptl_dpb_hrd_max_tid_flag) + us(3, vps_hrd_max_tid[i], 0, current->vps_max_sublayers_minus1, 1, i); + else + infer(vps_hrd_max_tid[i], current->vps_max_sublayers_minus1); + first_sublayer = current->vps_sublayer_cpb_params_present_flag ? + 0 : current->vps_hrd_max_tid[i]; + CHECK(FUNC(ols_timing_hrd_parameters)(ctx, rw, + ¤t->vps_ols_timing_hrd_parameters, first_sublayer, + current->vps_max_sublayers_minus1, + ¤t->vps_general_timing_hrd_parameters)); + + } + if (current->vps_num_ols_timing_hrd_params_minus1 > 0 && + current-> vps_num_ols_timing_hrd_params_minus1 + 1 != num_multi_layer_olss) { + for (i = 0; i < num_multi_layer_olss; i++) { + ues(vps_ols_timing_hrd_idx[i], 0, + current->vps_num_ols_timing_hrd_params_minus1, 1, i); + } + } else if (current->vps_num_ols_timing_hrd_params_minus1 == 0) { + for (i = 0; i < num_multi_layer_olss; i++) + infer(vps_ols_timing_hrd_idx[i], 0); + } else { + for (i = 0; i < num_multi_layer_olss; i++) + infer(vps_ols_timing_hrd_idx[i], i); + } + } + } + + flag(vps_extension_flag); + if (current->vps_extension_flag) + CHECK(FUNC(extension_data)(ctx, rw, ¤t->extension_data)); + CHECK(FUNC(rbsp_trailing_bits)(ctx, rw)); + + return 0; +} + + +static int FUNC(sps)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawSPS *current) +{ + CodedBitstreamH266Context *h266 = ctx->priv_data; + int err, i, j; + unsigned int ctb_log2_size_y, min_cb_log2_size_y, + min_qt_log2_size_intra_y, min_qt_log2_size_inter_y, + ctb_size_y, max_num_merge_cand, tmp_width_val, tmp_height_val; + uint8_t qp_bd_offset; + + static const uint8_t h266_sub_width_c[] = { + 1, 2, 2, 1 + }; + static const uint8_t h266_sub_height_c[] = { + 1, 2, 1, 1 + }; + + HEADER("Sequence Parameter Set"); + + CHECK(FUNC(nal_unit_header)(ctx, rw, + ¤t->nal_unit_header, VVC_SPS_NUT)); + + ub(4, sps_seq_parameter_set_id); + ub(4, sps_video_parameter_set_id); + if (current->sps_video_parameter_set_id == 0 && !h266->vps_ref[0]) { + H266RawVPS *vps; + AVBufferRef *ref = av_buffer_allocz(sizeof(H266RawVPS)); + if (!ref) { + return AVERROR(ENOMEM); + } + vps = (H266RawVPS*)ref->data; + vps->vps_max_layers_minus1 = 0; + vps->vps_independent_layer_flag[0] = 1; + vps->vps_layer_id[0] = current->nal_unit_header.nuh_layer_id; + h266->vps_ref[0] = ref; + h266->vps[0] = vps; + } + + u(3, sps_max_sublayers_minus1, 0, VVC_MAX_SUBLAYERS - 1); + u(2, sps_chroma_format_idc, 0, 3); + u(2, sps_log2_ctu_size_minus5, 0, 3); + ctb_log2_size_y = current->sps_log2_ctu_size_minus5 + 5; + ctb_size_y = 1 << ctb_log2_size_y; + + flag(sps_ptl_dpb_hrd_params_present_flag); + if (current->sps_ptl_dpb_hrd_params_present_flag) { + CHECK(FUNC(profile_tier_level)(ctx, rw, ¤t->profile_tier_level, + 1, current->sps_max_sublayers_minus1)); + } + flag(sps_gdr_enabled_flag); + flag(sps_ref_pic_resampling_enabled_flag); + if (current->sps_ref_pic_resampling_enabled_flag) + flag(sps_res_change_in_clvs_allowed_flag); + else + infer(sps_res_change_in_clvs_allowed_flag, 0); + + ue(sps_pic_width_max_in_luma_samples, 1, VVC_MAX_WIDTH); + ue(sps_pic_height_max_in_luma_samples, 1, VVC_MAX_HEIGHT); + + flag(sps_conformance_window_flag); + if (current->sps_conformance_window_flag) { + uint8_t sub_width_c = h266_sub_width_c[current->sps_chroma_format_idc]; + uint8_t sub_height_c = h266_sub_height_c[current->sps_chroma_format_idc]; + uint16_t width = current->sps_pic_width_max_in_luma_samples / sub_width_c; + uint16_t height = current->sps_pic_height_max_in_luma_samples / sub_height_c; + ue(sps_conf_win_left_offset, 0, width); + ue(sps_conf_win_right_offset, 0, width - current->sps_conf_win_left_offset); + ue(sps_conf_win_top_offset, 0, height); + ue(sps_conf_win_bottom_offset, 0, height - current->sps_conf_win_top_offset); + } else { + infer(sps_conf_win_left_offset, 0); + infer(sps_conf_win_right_offset, 0); + infer(sps_conf_win_top_offset, 0); + infer(sps_conf_win_bottom_offset, 0); + } + + tmp_width_val = + h266_ceil(current->sps_pic_width_max_in_luma_samples, ctb_size_y); + tmp_height_val = + h266_ceil(current->sps_pic_height_max_in_luma_samples, ctb_size_y); + + flag(sps_subpic_info_present_flag); + if (current->sps_subpic_info_present_flag) { + ue(sps_num_subpics_minus1, 1, VVC_MAX_SLICES - 1); + if (current->sps_num_subpics_minus1 > 0) { + flag(sps_independent_subpics_flag); + flag(sps_subpic_same_size_flag); + } + + if (current->sps_num_subpics_minus1 > 0) { + int wlen = av_ceil_log2(tmp_width_val); + int hlen = av_ceil_log2(tmp_height_val); + if (current->sps_pic_width_max_in_luma_samples > ctb_size_y) + ubs(wlen, sps_subpic_width_minus1[0], 1, 0); + else + infer(sps_subpic_width_minus1[i], tmp_width_val - 1); + if (current->sps_pic_height_max_in_luma_samples > ctb_size_y) + ubs(hlen, sps_subpic_height_minus1[0], 1, 0); + else + infer(sps_subpic_height_minus1[0], tmp_height_val); + if (!current->sps_independent_subpics_flag) { + flags(sps_subpic_treated_as_pic_flag[0], 1, 0); + flags(sps_loop_filter_across_subpic_enabled_flag[0], 1, 0); + } else { + infer(sps_subpic_treated_as_pic_flag[0], 1); + infer(sps_loop_filter_across_subpic_enabled_flag[0], 1); + } + for (i = 1; i <= current->sps_num_subpics_minus1; i++) { + if (!current->sps_subpic_same_size_flag) { + if (current->sps_pic_width_max_in_luma_samples > ctb_size_y) + ubs(wlen, sps_subpic_ctu_top_left_x[i], 1, i); + else + infer(sps_subpic_ctu_top_left_x[i], 0); + if (current->sps_pic_height_max_in_luma_samples > ctb_size_y) + ubs(hlen, sps_subpic_ctu_top_left_y[i], 1, i); + else + infer(sps_subpic_ctu_top_left_y[i], 0); + if (i < current->sps_num_subpics_minus1 && + current->sps_pic_width_max_in_luma_samples > ctb_size_y) { + ubs(wlen, sps_subpic_width_minus1[i], 1, i); + } else { + infer(sps_subpic_width_minus1[i], + tmp_width_val - current->sps_subpic_ctu_top_left_x[i] - 1); + } + if (i < current->sps_num_subpics_minus1 && + current->sps_pic_height_max_in_luma_samples > ctb_size_y) { + ubs(hlen, sps_subpic_height_minus1[i], 1, i); + } else { + infer(sps_subpic_height_minus1[i], + tmp_height_val - current->sps_subpic_ctu_top_left_y[i] - 1); + } + } else { + int num_subpic_cols = + tmp_width_val / (current->sps_subpic_width_minus1[0] + 1); + infer(sps_subpic_ctu_top_left_x[i], + (i % num_subpic_cols) * + (current->sps_subpic_width_minus1[0] + 1)); + infer(sps_subpic_ctu_top_left_y[i], + (i / num_subpic_cols) * + (current->sps_subpic_height_minus1[0] + 1)); + infer(sps_subpic_width_minus1[i], + current->sps_subpic_width_minus1[0]); + infer(sps_subpic_height_minus1[i], + current->sps_subpic_height_minus1[0]); + } + if (!current->sps_independent_subpics_flag) { + flags(sps_subpic_treated_as_pic_flag[i], 1, i); + flags(sps_loop_filter_across_subpic_enabled_flag[i], 1, i); + } else { + infer(sps_subpic_treated_as_pic_flag[i], 1); + infer(sps_loop_filter_across_subpic_enabled_flag[i], 0); + } + } + ue(sps_subpic_id_len_minus1, 0, 15); + if ((1 << (current->sps_subpic_id_len_minus1 + 1)) < + current->sps_num_subpics_minus1 + 1) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "sps_subpic_id_len_minus1(%d) is too small\n", + current->sps_subpic_id_len_minus1); + return AVERROR_INVALIDDATA; + } + flag(sps_subpic_id_mapping_explicitly_signalled_flag); + if (current->sps_subpic_id_mapping_explicitly_signalled_flag) { + flag(sps_subpic_id_mapping_present_flag); + if (current->sps_subpic_id_mapping_present_flag) { + for (i = 0; i <= current->sps_num_subpics_minus1; i++) { + ubs(current->sps_subpic_id_len_minus1 + 1, + sps_subpic_id[i], 1, i); + } + } + } + } else { + infer(sps_subpic_ctu_top_left_x[0], 0); + infer(sps_subpic_ctu_top_left_y[0], 0); + infer(sps_subpic_width_minus1[0], tmp_width_val - 1); + infer(sps_subpic_height_minus1[0], tmp_height_val - 1); + } + } else { + infer(sps_num_subpics_minus1, 0); + infer(sps_independent_subpics_flag, 1); + infer(sps_subpic_same_size_flag, 0); + infer(sps_subpic_id_mapping_explicitly_signalled_flag, 0); + infer(sps_subpic_ctu_top_left_x[0], 0); + infer(sps_subpic_ctu_top_left_y[0], 0); + infer(sps_subpic_width_minus1[0], tmp_width_val - 1); + infer(sps_subpic_height_minus1[0], tmp_height_val - 1); + } + + + ue(sps_bitdepth_minus8, 0, 2); + qp_bd_offset = 6 * current->sps_bitdepth_minus8; + + flag(sps_entropy_coding_sync_enabled_flag); + flag(sps_entry_point_offsets_present_flag); + + u(4, sps_log2_max_pic_order_cnt_lsb_minus4, 0, 12); + flag(sps_poc_msb_cycle_flag); + if (current->sps_poc_msb_cycle_flag) + ue(sps_poc_msb_cycle_len_minus1, + 0, 32 - current->sps_log2_max_pic_order_cnt_lsb_minus4 - 5); + + u(2, sps_num_extra_ph_bytes, 0, 2); + for (i = 0; i < (current->sps_num_extra_ph_bytes * 8); i++) { + flags(sps_extra_ph_bit_present_flag[i], 1, i); + } + + u(2, sps_num_extra_sh_bytes, 0, 2); + for (i = 0; i < (current->sps_num_extra_sh_bytes * 8); i++) { + flags(sps_extra_sh_bit_present_flag[i], 1, i); + } + + if (current->sps_ptl_dpb_hrd_params_present_flag) { + if (current->sps_max_sublayers_minus1 > 0) + flag(sps_sublayer_dpb_params_flag); + else + infer(sps_sublayer_dpb_params_flag, 0); + CHECK(FUNC(dpb_parameters)(ctx, rw, ¤t->sps_dpb_params, + current->sps_max_sublayers_minus1, + current->sps_sublayer_dpb_params_flag)); + } + + ue(sps_log2_min_luma_coding_block_size_minus2, + 0, FFMIN(4, current->sps_log2_ctu_size_minus5 + 3)); + min_cb_log2_size_y = + current->sps_log2_min_luma_coding_block_size_minus2 + 2; + + flag(sps_partition_constraints_override_enabled_flag); + + ue(sps_log2_diff_min_qt_min_cb_intra_slice_luma, + 0, FFMIN(6, ctb_log2_size_y) - min_cb_log2_size_y); + min_qt_log2_size_intra_y = + current->sps_log2_diff_min_qt_min_cb_intra_slice_luma + + min_cb_log2_size_y; + + ue(sps_max_mtt_hierarchy_depth_intra_slice_luma, + 0, 2 * (ctb_log2_size_y - min_cb_log2_size_y)); + + if (current->sps_max_mtt_hierarchy_depth_intra_slice_luma != 0) { + ue(sps_log2_diff_max_bt_min_qt_intra_slice_luma, + 0, ctb_log2_size_y - min_qt_log2_size_intra_y); + ue(sps_log2_diff_max_tt_min_qt_intra_slice_luma, + 0, FFMIN(6, ctb_log2_size_y) - min_qt_log2_size_intra_y); + } else { + infer(sps_log2_diff_max_bt_min_qt_intra_slice_luma, 0); + infer(sps_log2_diff_max_tt_min_qt_intra_slice_luma, 0); + } + + if (current->sps_chroma_format_idc != 0) { + flag(sps_qtbtt_dual_tree_intra_flag); + } else { + infer(sps_qtbtt_dual_tree_intra_flag, 0); + } + + if (current->sps_qtbtt_dual_tree_intra_flag) { + ue(sps_log2_diff_min_qt_min_cb_intra_slice_chroma, + 0, FFMIN(6, ctb_log2_size_y) - min_cb_log2_size_y); + ue(sps_max_mtt_hierarchy_depth_intra_slice_chroma, + 0, 2 * (ctb_log2_size_y - min_cb_log2_size_y)); + if (current->sps_max_mtt_hierarchy_depth_intra_slice_chroma != 0) { + unsigned int min_qt_log2_size_intra_c = + current->sps_log2_diff_min_qt_min_cb_intra_slice_chroma + + min_cb_log2_size_y; + ue(sps_log2_diff_max_bt_min_qt_intra_slice_chroma, + 0, FFMIN(6, ctb_log2_size_y) - min_qt_log2_size_intra_c); + ue(sps_log2_diff_max_tt_min_qt_intra_slice_chroma, + 0, FFMIN(6, ctb_log2_size_y) - min_qt_log2_size_intra_c); + } + } else { + infer(sps_log2_diff_min_qt_min_cb_intra_slice_chroma, 0); + infer(sps_max_mtt_hierarchy_depth_intra_slice_chroma, 0); + } + if (current->sps_max_mtt_hierarchy_depth_intra_slice_chroma == 0) { + infer(sps_log2_diff_max_bt_min_qt_intra_slice_chroma, 0); + infer(sps_log2_diff_max_tt_min_qt_intra_slice_chroma, 0); + } + + ue(sps_log2_diff_min_qt_min_cb_inter_slice, + 0, FFMIN(6, ctb_log2_size_y) - min_cb_log2_size_y); + min_qt_log2_size_inter_y = + current->sps_log2_diff_min_qt_min_cb_inter_slice + min_cb_log2_size_y; + + ue(sps_max_mtt_hierarchy_depth_inter_slice, + 0, 2 * (ctb_log2_size_y - min_cb_log2_size_y)); + if (current->sps_max_mtt_hierarchy_depth_inter_slice != 0) { + ue(sps_log2_diff_max_bt_min_qt_inter_slice, + 0, ctb_log2_size_y - min_qt_log2_size_inter_y); + ue(sps_log2_diff_max_tt_min_qt_inter_slice, + 0, FFMIN(6, ctb_log2_size_y) - min_qt_log2_size_inter_y); + } else { + infer(sps_log2_diff_max_bt_min_qt_inter_slice, 0); + infer(sps_log2_diff_max_tt_min_qt_inter_slice, 0); + } + + if (ctb_size_y > 32) + flag(sps_max_luma_transform_size_64_flag); + else + infer(sps_max_luma_transform_size_64_flag, 0); + + flag(sps_transform_skip_enabled_flag); + if (current->sps_transform_skip_enabled_flag) { + ue(sps_log2_transform_skip_max_size_minus2, 0, 3); + flag(sps_bdpcm_enabled_flag); + } + + flag(sps_mts_enabled_flag); + if (current->sps_mts_enabled_flag) { + flag(sps_explicit_mts_intra_enabled_flag); + flag(sps_explicit_mts_inter_enabled_flag); + } else { + infer(sps_explicit_mts_intra_enabled_flag, 0); + infer(sps_explicit_mts_inter_enabled_flag, 0); + } + + flag(sps_lfnst_enabled_flag); + + if (current->sps_chroma_format_idc != 0) { + uint8_t num_qp_tables; + flag(sps_joint_cbcr_enabled_flag); + flag(sps_same_qp_table_for_chroma_flag); + num_qp_tables = current->sps_same_qp_table_for_chroma_flag ? + 1 : (current->sps_joint_cbcr_enabled_flag ? 3 : 2); + for (i = 0; i < num_qp_tables; i++) { + ses(sps_qp_table_start_minus26[i], + -26 - qp_bd_offset, 36, 1, i); + ues(sps_num_points_in_qp_table_minus1[i], + 0, 36 - current->sps_qp_table_start_minus26[i], 1, i); + for (j = 0; j <= current->sps_num_points_in_qp_table_minus1[i]; j++) { + uint8_t max = MAX_UINT_BITS(8); + ues(sps_delta_qp_in_val_minus1[i][j], 0, max, 2, i, j); + ues(sps_delta_qp_diff_val[i][j], 0, max, 2, i, j); + } + } + } else { + infer(sps_joint_cbcr_enabled_flag, 0); + infer(sps_same_qp_table_for_chroma_flag, 0); + } + + flag(sps_sao_enabled_flag); + flag(sps_alf_enabled_flag); + if (current->sps_alf_enabled_flag && current->sps_chroma_format_idc) + flag(sps_ccalf_enabled_flag); + else + infer(sps_ccalf_enabled_flag, 0); + flag(sps_lmcs_enabled_flag); + flag(sps_weighted_pred_flag); + flag(sps_weighted_bipred_flag); + flag(sps_long_term_ref_pics_flag); + if (current->sps_video_parameter_set_id > 0) + flag(sps_inter_layer_prediction_enabled_flag); + else + infer(sps_inter_layer_prediction_enabled_flag, 0); + flag(sps_idr_rpl_present_flag); + flag(sps_rpl1_same_as_rpl0_flag); + + for (i = 0; i < (current->sps_rpl1_same_as_rpl0_flag ? 1 : 2); i++) { + ues(sps_num_ref_pic_lists[i], 0, VVC_MAX_REF_PIC_LISTS, 1, i); + for (j = 0; j < current->sps_num_ref_pic_lists[i]; j++) + CHECK(FUNC(ref_pic_list_struct)(ctx, rw, + ¤t->sps_ref_pic_list_struct[i][j], + i, j, current)); + } + + if (current->sps_rpl1_same_as_rpl0_flag) { + current->sps_num_ref_pic_lists[1] = current->sps_num_ref_pic_lists[0]; + for (j = 0; j < current->sps_num_ref_pic_lists[0]; j++) + memcpy(¤t->sps_ref_pic_list_struct[1][j], + ¤t->sps_ref_pic_list_struct[0][j], + sizeof(current->sps_ref_pic_list_struct[0][j])); + } + + flag(sps_ref_wraparound_enabled_flag); + + flag(sps_temporal_mvp_enabled_flag); + if (current->sps_temporal_mvp_enabled_flag) + flag(sps_sbtmvp_enabled_flag); + else + infer(sps_sbtmvp_enabled_flag, 0); + + flag(sps_amvr_enabled_flag); + flag(sps_bdof_enabled_flag); + if (current->sps_bdof_enabled_flag) + flag(sps_bdof_control_present_in_ph_flag); + else + infer(sps_bdof_control_present_in_ph_flag, 0); + + flag(sps_smvd_enabled_flag); + flag(sps_dmvr_enabled_flag); + if (current->sps_dmvr_enabled_flag) + flag(sps_dmvr_control_present_in_ph_flag); + else + infer(sps_dmvr_control_present_in_ph_flag, 0); + + flag(sps_mmvd_enabled_flag); + if (current->sps_mmvd_enabled_flag) + flag(sps_mmvd_fullpel_only_enabled_flag); + else + infer(sps_mmvd_fullpel_only_enabled_flag, 0); + + ue(sps_six_minus_max_num_merge_cand, 0, 5); + max_num_merge_cand = 6 - current->sps_six_minus_max_num_merge_cand; + + flag(sps_sbt_enabled_flag); + + flag(sps_affine_enabled_flag); + if (current->sps_affine_enabled_flag) { + ue(sps_five_minus_max_num_subblock_merge_cand, + 0, 5 - current->sps_sbtmvp_enabled_flag); + flag(sps_6param_affine_enabled_flag); + if (current->sps_amvr_enabled_flag) + flag(sps_affine_amvr_enabled_flag); + else + infer(sps_affine_amvr_enabled_flag, 0); + flag(sps_affine_prof_enabled_flag); + if (current->sps_affine_prof_enabled_flag) + flag(sps_prof_control_present_in_ph_flag); + else + infer(sps_prof_control_present_in_ph_flag, 0); + } else { + infer(sps_6param_affine_enabled_flag, 0); + infer(sps_affine_amvr_enabled_flag, 0); + infer(sps_affine_prof_enabled_flag, 0); + infer(sps_prof_control_present_in_ph_flag, 0); + } + + flag(sps_bcw_enabled_flag); + flag(sps_ciip_enabled_flag); + + if (max_num_merge_cand >= 2) { + flag(sps_gpm_enabled_flag); + if (current->sps_gpm_enabled_flag && max_num_merge_cand >= 3) + ue(sps_max_num_merge_cand_minus_max_num_gpm_cand, + 0, max_num_merge_cand - 2); + } else { + infer(sps_gpm_enabled_flag, 0); + } + + ue(sps_log2_parallel_merge_level_minus2, 0, ctb_log2_size_y - 2); + + flag(sps_isp_enabled_flag); + flag(sps_mrl_enabled_flag); + flag(sps_mip_enabled_flag); + + if (current->sps_chroma_format_idc != 0) + flag(sps_cclm_enabled_flag); + else + infer(sps_cclm_enabled_flag, 0); + if (current->sps_chroma_format_idc == 1) { + flag(sps_chroma_horizontal_collocated_flag); + flag(sps_chroma_vertical_collocated_flag); + } else { + infer(sps_chroma_horizontal_collocated_flag, 1); + infer(sps_chroma_vertical_collocated_flag, 1); + } + + flag(sps_palette_enabled_flag); + if (current->sps_chroma_format_idc == 3 && + !current->sps_max_luma_transform_size_64_flag) + flag(sps_act_enabled_flag); + else + infer(sps_act_enabled_flag, 0); + if (current->sps_transform_skip_enabled_flag || + current->sps_palette_enabled_flag) + ue(sps_min_qp_prime_ts, 0, 8); + + flag(sps_ibc_enabled_flag); + if (current->sps_ibc_enabled_flag) + ue(sps_six_minus_max_num_ibc_merge_cand, 0, 5); + + flag(sps_ladf_enabled_flag); + if (current->sps_ladf_enabled_flag) { + ub(2, sps_num_ladf_intervals_minus2); + se(sps_ladf_lowest_interval_qp_offset, -63, 63); + for (i = 0; i < current->sps_num_ladf_intervals_minus2 + 1; i++) { + ses(sps_ladf_qp_offset[i], -63, 63, 1, i); + ues(sps_ladf_delta_threshold_minus1[i], + 0, (2 << (8 + current->sps_bitdepth_minus8)) - 3, 1, i); + } + } + + flag(sps_explicit_scaling_list_enabled_flag); + if (current->sps_lfnst_enabled_flag && + current->sps_explicit_scaling_list_enabled_flag) + flag(sps_scaling_matrix_for_lfnst_disabled_flag); + + if (current->sps_act_enabled_flag && + current->sps_explicit_scaling_list_enabled_flag) + flag(sps_scaling_matrix_for_alternative_colour_space_disabled_flag); + else + infer(sps_scaling_matrix_for_alternative_colour_space_disabled_flag, 0); + if (current->sps_scaling_matrix_for_alternative_colour_space_disabled_flag) + flag(sps_scaling_matrix_designated_colour_space_flag); + + flag(sps_dep_quant_enabled_flag); + flag(sps_sign_data_hiding_enabled_flag); + + flag(sps_virtual_boundaries_enabled_flag); + if (current->sps_virtual_boundaries_enabled_flag) { + flag(sps_virtual_boundaries_present_flag); + if (current->sps_virtual_boundaries_present_flag) { + ue(sps_num_ver_virtual_boundaries, + 0, current->sps_pic_width_max_in_luma_samples <= 8 ? 0 : 3); + for (i = 0; i < current->sps_num_ver_virtual_boundaries; i++) + ues(sps_virtual_boundary_pos_x_minus1[i], + 0, (current->sps_pic_width_max_in_luma_samples + 7) / 8 - 2, + 1, i); + ue(sps_num_hor_virtual_boundaries, + 0, current->sps_pic_height_max_in_luma_samples <= 8 ? 0 : 3); + for (i = 0; i < current->sps_num_hor_virtual_boundaries; i++) + ues(sps_virtual_boundary_pos_y_minus1[i], + 0, (current->sps_pic_height_max_in_luma_samples + 7) / 8 - 2, + 1, i); + } + } else { + infer(sps_virtual_boundaries_present_flag, 0); + infer(sps_num_ver_virtual_boundaries, 0); + infer(sps_num_hor_virtual_boundaries, 0); + } + + if (current->sps_ptl_dpb_hrd_params_present_flag) { + flag(sps_timing_hrd_params_present_flag); + if (current->sps_timing_hrd_params_present_flag) { + uint8_t first_sublayer; + CHECK(FUNC(general_timing_hrd_parameters)(ctx, rw, + ¤t->sps_general_timing_hrd_parameters)); + if (current->sps_max_sublayers_minus1 > 0) + flag(sps_sublayer_cpb_params_present_flag); + else + infer(sps_sublayer_cpb_params_present_flag, 0); + first_sublayer = current->sps_sublayer_cpb_params_present_flag ? + 0 : current->sps_max_sublayers_minus1; + CHECK(FUNC(ols_timing_hrd_parameters)(ctx, rw, + ¤t->sps_ols_timing_hrd_parameters, first_sublayer, + current->sps_max_sublayers_minus1, + ¤t->sps_general_timing_hrd_parameters)); + } + } + + flag(sps_field_seq_flag); + flag(sps_vui_parameters_present_flag); + if (current->sps_vui_parameters_present_flag) { + ue(sps_vui_payload_size_minus1, 0, 1023); + while (byte_alignment(rw) != 0) + fixed(1, sps_vui_alignment_zero_bit, 0); + CHECK(FUNC(vui_payload)(ctx, rw, ¤t->vui, + current->sps_vui_payload_size_minus1 + 1, + current->sps_chroma_format_idc)); + } else { + CHECK(FUNC(vui_parameters_default)(ctx, rw, ¤t->vui)); + } + flag(sps_extension_flag); + if (current->sps_extension_flag) + CHECK(FUNC(extension_data)(ctx, rw, ¤t->extension_data)); + + CHECK(FUNC(rbsp_trailing_bits)(ctx, rw)); + + return 0; +} + +static int FUNC(pps)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawPPS *current) +{ + + CodedBitstreamH266Context *h266 = ctx->priv_data; + const H266RawSPS *sps; + int err, i; + unsigned int min_cb_size_y, divisor, ctb_size_y, + pic_width_in_ctbs_y, pic_height_in_ctbs_y; + uint8_t sub_width_c, sub_height_c, qp_bd_offset; + + static const uint8_t h266_sub_width_c[] = { + 1, 2, 2, 1 + }; + static const uint8_t h266_sub_height_c[] = { + 1, 2, 1, 1 + }; + + HEADER("Picture Parameter Set"); + + CHECK(FUNC(nal_unit_header)(ctx, rw, + ¤t->nal_unit_header, VVC_PPS_NUT)); + + ub(6, pps_pic_parameter_set_id); + ub(4, pps_seq_parameter_set_id); + sps = h266->sps[current->pps_seq_parameter_set_id]; + if (!sps) { + av_log(ctx->log_ctx, AV_LOG_ERROR, "SPS id %d not available.\n", + current->pps_seq_parameter_set_id); + return AVERROR_INVALIDDATA; + } + + flag(pps_mixed_nalu_types_in_pic_flag); + ue(pps_pic_width_in_luma_samples, + 1, sps->sps_pic_width_max_in_luma_samples); + ue(pps_pic_height_in_luma_samples, + 1, sps->sps_pic_height_max_in_luma_samples); + + min_cb_size_y = 1 << (sps->sps_log2_min_luma_coding_block_size_minus2 + 2); + divisor = FFMAX(min_cb_size_y, 8); + if (current->pps_pic_width_in_luma_samples % divisor || + current->pps_pic_height_in_luma_samples % divisor) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "Invalid dimensions: %ux%u not divisible " + "by %u, MinCbSizeY = %u.\n", current->pps_pic_width_in_luma_samples, + current->pps_pic_height_in_luma_samples, divisor, min_cb_size_y); + return AVERROR_INVALIDDATA; + } + if (!sps->sps_res_change_in_clvs_allowed_flag && + (current->pps_pic_width_in_luma_samples != + sps->sps_pic_width_max_in_luma_samples || + current->pps_pic_height_in_luma_samples != + sps->sps_pic_height_max_in_luma_samples)) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "Resoltuion change is not allowed, " + "in max resolution (%ux%u) mismatched with pps(%ux%u).\n", + sps->sps_pic_width_max_in_luma_samples, + sps->sps_pic_height_max_in_luma_samples, + current->pps_pic_width_in_luma_samples, + current->pps_pic_height_in_luma_samples); + return AVERROR_INVALIDDATA; + } + + ctb_size_y = 1 << (sps->sps_log2_ctu_size_minus5 + 5); + if (sps->sps_ref_wraparound_enabled_flag) { + if ((ctb_size_y / min_cb_size_y + 1) > + (current->pps_pic_width_in_luma_samples / min_cb_size_y - 1)) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "Invalid width(%u), ctb_size_y = %u, min_cb_size_y = %u.\n", + current->pps_pic_width_in_luma_samples, + ctb_size_y, min_cb_size_y); + return AVERROR_INVALIDDATA; + } + } + + flag(pps_conformance_window_flag); + if (current->pps_pic_width_in_luma_samples == + sps->sps_pic_width_max_in_luma_samples && + current->pps_pic_height_in_luma_samples == + sps->sps_pic_height_max_in_luma_samples && + current->pps_conformance_window_flag) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "Conformance window flag should not true.\n"); + return AVERROR_INVALIDDATA; + } + + sub_width_c = h266_sub_width_c[sps->sps_chroma_format_idc]; + sub_height_c = h266_sub_height_c[sps->sps_chroma_format_idc]; + if (current->pps_conformance_window_flag) { + ue(pps_conf_win_left_offset, + 0, current->pps_pic_width_in_luma_samples); + ue(pps_conf_win_right_offset, + 0, current->pps_pic_width_in_luma_samples); + ue(pps_conf_win_top_offset, + 0, current->pps_pic_height_in_luma_samples); + ue(pps_conf_win_bottom_offset, + 0, current->pps_pic_height_in_luma_samples); + if (sub_width_c * + (current->pps_conf_win_left_offset + + current->pps_conf_win_right_offset) >= + current->pps_pic_width_in_luma_samples || + sub_height_c * + (current->pps_conf_win_top_offset + + current->pps_conf_win_bottom_offset) >= + current->pps_pic_height_in_luma_samples) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "Invalid pps conformance window: (%u, %u, %u, %u), " + "resolution is %ux%u, sub wxh is %ux%u.\n", + current->pps_conf_win_left_offset, + current->pps_conf_win_right_offset, + current->pps_conf_win_top_offset, + current->pps_conf_win_bottom_offset, + current->pps_pic_width_in_luma_samples, + current->pps_pic_height_in_luma_samples, + sub_width_c, sub_height_c); + return AVERROR_INVALIDDATA; + } + } else { + if (current->pps_pic_width_in_luma_samples == + sps->sps_pic_width_max_in_luma_samples && + current->pps_pic_height_in_luma_samples == + sps->sps_pic_height_max_in_luma_samples) { + infer(pps_conf_win_left_offset, sps->sps_conf_win_left_offset); + infer(pps_conf_win_right_offset, sps->sps_conf_win_right_offset); + infer(pps_conf_win_top_offset, sps->sps_conf_win_top_offset); + infer(pps_conf_win_bottom_offset, sps->sps_conf_win_bottom_offset); + } else { + infer(pps_conf_win_left_offset, 0); + infer(pps_conf_win_right_offset, 0); + infer(pps_conf_win_top_offset, 0); + infer(pps_conf_win_bottom_offset, 0); + } + + } + + flag(pps_scaling_window_explicit_signalling_flag); + if (!sps->sps_ref_pic_resampling_enabled_flag && + current->pps_scaling_window_explicit_signalling_flag) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "Invalid data: sps_ref_pic_resampling_enabled_flag is false, " + "but pps_scaling_window_explicit_signalling_flag is true.\n"); + return AVERROR_INVALIDDATA; + } + if (current->pps_scaling_window_explicit_signalling_flag) { + se(pps_scaling_win_left_offset, + -current->pps_pic_width_in_luma_samples * 15 / sub_width_c, + current->pps_pic_width_in_luma_samples / sub_width_c); + se(pps_scaling_win_right_offset, + -current->pps_pic_width_in_luma_samples * 15 / sub_width_c, + current->pps_pic_width_in_luma_samples / sub_width_c); + se(pps_scaling_win_top_offset, + -current->pps_pic_height_in_luma_samples * 15 / sub_height_c, + current->pps_pic_height_in_luma_samples / sub_height_c); + se(pps_scaling_win_bottom_offset, + -current->pps_pic_height_in_luma_samples * 15 / sub_height_c, + current->pps_pic_height_in_luma_samples / sub_height_c); + } else { + infer(pps_scaling_win_left_offset, current->pps_conf_win_left_offset); + infer(pps_scaling_win_right_offset, current->pps_conf_win_right_offset); + infer(pps_scaling_win_top_offset, current->pps_conf_win_top_offset); + infer(pps_scaling_win_bottom_offset, current->pps_conf_win_bottom_offset); + } + + flag(pps_output_flag_present_flag); + flag(pps_no_pic_partition_flag); + flag(pps_subpic_id_mapping_present_flag); + + if (current->pps_subpic_id_mapping_present_flag) { + if (!current->pps_no_pic_partition_flag) { + ue(pps_num_subpics_minus1, + sps->sps_num_subpics_minus1, sps->sps_num_subpics_minus1); + } else { + infer(pps_num_subpics_minus1, 0); + } + ue(pps_subpic_id_len_minus1, sps->sps_subpic_id_len_minus1, sps->sps_subpic_id_len_minus1); + for (i = 0; i <= current->pps_num_subpics_minus1; i++) { + ubs(sps->sps_subpic_id_len_minus1 + 1, pps_subpic_id[i], 1, i); + } + } + + pic_width_in_ctbs_y = + h266_ceil(current->pps_pic_width_in_luma_samples, ctb_size_y); + pic_height_in_ctbs_y = + h266_ceil(current->pps_pic_height_in_luma_samples, ctb_size_y); + if (!current->pps_no_pic_partition_flag) { + unsigned int exp_tile_width = 0, exp_tile_height = 0; + unsigned int unified_size, remaining_size; + + u(2, pps_log2_ctu_size_minus5, + sps->sps_log2_ctu_size_minus5, sps->sps_log2_ctu_size_minus5); + ue(pps_num_exp_tile_columns_minus1, + 0, FFMIN(pic_width_in_ctbs_y - 1, VVC_MAX_TILE_COLUMNS - 1)); + ue(pps_num_exp_tile_rows_minus1, + 0, FFMIN(pic_height_in_ctbs_y - 1, VVC_MAX_TILE_ROWS - 1)); + + for (i = 0; i <= current->pps_num_exp_tile_columns_minus1; i++) { + ues(pps_tile_column_width_minus1[i], + 0, pic_width_in_ctbs_y - exp_tile_width - 1, 1, i); + exp_tile_width += current->pps_tile_column_width_minus1[i] + 1; + } + remaining_size = pic_width_in_ctbs_y - exp_tile_width; + unified_size = (i == 0 ? pic_width_in_ctbs_y : + (current->pps_tile_column_width_minus1[i - 1] + 1)); + current->num_tile_columns = i + h266_ceil(remaining_size, unified_size); + if (current->num_tile_columns > VVC_MAX_TILE_COLUMNS) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "NumTileColumns(%d) large than %d.\n", + current->num_tile_columns, VVC_MAX_TILE_COLUMNS); + return AVERROR_INVALIDDATA; + } + while (remaining_size > unified_size) { + current->pps_tile_column_width_minus1[i] = unified_size - 1; + remaining_size -= unified_size; + i++; + } + if (remaining_size > 0) { + current->pps_tile_column_width_minus1[i] = remaining_size - 1; + } + + for (i = 0; i <= current->pps_num_exp_tile_rows_minus1; i++) { + ues(pps_tile_row_height_minus1[i], + 0, pic_height_in_ctbs_y - 1, 1, i); + exp_tile_height += current->pps_tile_row_height_minus1[i] + 1; + } + remaining_size = pic_height_in_ctbs_y - exp_tile_height; + unified_size = (i == 0 ? pic_height_in_ctbs_y : + (current->pps_tile_row_height_minus1[i - 1] + 1)); + current->num_tile_rows = i + h266_ceil(remaining_size, unified_size); + if (current->num_tile_rows > VVC_MAX_TILE_ROWS) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "NumTileRows(%d) large than %d, consider increase VVC_MAX_TILE_ROWS.\n", + current->num_tile_rows, VVC_MAX_TILE_ROWS); + return AVERROR_INVALIDDATA; + } + while (remaining_size > unified_size) { + current->pps_tile_row_height_minus1[i] = unified_size - 1; + remaining_size -= unified_size; + i++; + } + if (remaining_size > 0) { + current->pps_tile_row_height_minus1[i] = remaining_size - 1; + } + + current->num_tiles_in_pic = + current->num_tile_columns * current->num_tile_rows; + if (current->num_tiles_in_pic > VVC_MAX_TILES_PER_AU) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "NumTilesInPic(%d) large than %d.\n", + current->num_tiles_in_pic, VVC_MAX_TILES_PER_AU); + return AVERROR_INVALIDDATA; + } + + if (current->num_tiles_in_pic > 1) { + flag(pps_loop_filter_across_tiles_enabled_flag); + flag(pps_rect_slice_flag); + } else { + infer(pps_loop_filter_across_tiles_enabled_flag, 0); + infer(pps_rect_slice_flag, 1); + } + if (current->pps_rect_slice_flag) + flag(pps_single_slice_per_subpic_flag); + else + infer(pps_single_slice_per_subpic_flag, 1); + if (current->pps_rect_slice_flag && + !current->pps_single_slice_per_subpic_flag) { + int j; + uint16_t tile_idx = 0, tile_x, tile_y, ctu_x, ctu_y; + uint16_t slice_top_left_ctu_x[VVC_MAX_SLICES]; + uint16_t slice_top_left_ctu_y[VVC_MAX_SLICES]; + ue(pps_num_slices_in_pic_minus1, 0, VVC_MAX_SLICES - 1); + if (current->pps_num_slices_in_pic_minus1 > 1) + flag(pps_tile_idx_delta_present_flag); + else + infer(pps_tile_idx_delta_present_flag, 0); + for (i = 0; i < current->pps_num_slices_in_pic_minus1; i++) { + tile_x = tile_idx % current->num_tile_columns; + tile_y = tile_idx / current->num_tile_columns; + if (tile_x != current->num_tile_columns - 1) { + ues(pps_slice_width_in_tiles_minus1[i], + 0, current->num_tile_columns - 1, 1, i); + } else { + infer(pps_slice_width_in_tiles_minus1[i], 0); + } + if (tile_y != current->num_tile_rows - 1 && + (current->pps_tile_idx_delta_present_flag || tile_x == 0)) { + ues(pps_slice_height_in_tiles_minus1[i], + 0, current->num_tile_rows - 1, 1, i); + } else { + if (tile_y == current->num_tile_rows - 1) + infer(pps_slice_height_in_tiles_minus1[i], 0); + else + infer(pps_slice_height_in_tiles_minus1[i], + current->pps_slice_height_in_tiles_minus1[i-1]); + } + + ctu_x = ctu_y = 0; + for (j = 0; j < tile_x; j++) { + ctu_x += current->pps_tile_column_width_minus1[j] + 1; + } + for (j = 0; j < tile_y; j++) { + ctu_y += current->pps_tile_row_height_minus1[j] + 1; + } + if (current->pps_slice_width_in_tiles_minus1[i] == 0 && + current->pps_slice_height_in_tiles_minus1[i] == 0 && + current->pps_tile_row_height_minus1[tile_y] > 0) { + int num_slices_in_tile, + uniform_slice_height, remaining_height_in_ctbs_y; + remaining_height_in_ctbs_y = + current->pps_tile_row_height_minus1[tile_y] + 1; + ues(pps_num_exp_slices_in_tile[i], + 0, current->pps_tile_row_height_minus1[tile_y], 1, i); + if (current->pps_num_exp_slices_in_tile[i] == 0) { + num_slices_in_tile = 1; + slice_top_left_ctu_x[i] = ctu_x; + slice_top_left_ctu_y[i] = ctu_y; + } else { + uint16_t slice_height_in_ctus; + for (j = 0; j < current->pps_num_exp_slices_in_tile[i]; j++) { + ues(pps_exp_slice_height_in_ctus_minus1[i][j], + 0, current->pps_tile_row_height_minus1[tile_y], 2, i, j); + slice_height_in_ctus = + current->pps_exp_slice_height_in_ctus_minus1[i][j] + 1; + + current->slice_height_in_ctus[i + j] = slice_height_in_ctus; + slice_top_left_ctu_x[i + j] = ctu_x; + slice_top_left_ctu_y[i + j] = ctu_y; + ctu_y += slice_height_in_ctus; + + remaining_height_in_ctbs_y -= slice_height_in_ctus; + } + uniform_slice_height = 1 + + (j == 0 ? current->pps_tile_row_height_minus1[tile_y] : + current->pps_exp_slice_height_in_ctus_minus1[i][j-1]); + while (remaining_height_in_ctbs_y > uniform_slice_height) { + current->slice_height_in_ctus[i + j] = uniform_slice_height; + slice_top_left_ctu_x[i + j] = ctu_x; + slice_top_left_ctu_y[i + j] = ctu_y; + ctu_y += uniform_slice_height; + + remaining_height_in_ctbs_y -= uniform_slice_height; + j++; + } + if (remaining_height_in_ctbs_y > 0) { + current->slice_height_in_ctus[i + j] = remaining_height_in_ctbs_y; + slice_top_left_ctu_x[i + j] = ctu_x; + slice_top_left_ctu_y[i + j] = ctu_y; + j++; + } + num_slices_in_tile = j; + } + i += num_slices_in_tile - 1; + } else { + uint16_t height= 0; + infer(pps_num_exp_slices_in_tile[i], 0); + for (j = 0; j <= current->pps_slice_height_in_tiles_minus1[i]; j++) { + height += current->pps_tile_row_height_minus1[tile_y + j] + 1; + } + current->slice_height_in_ctus[i] = height; + + slice_top_left_ctu_x[i] = ctu_x; + slice_top_left_ctu_y[i] = ctu_y; + } + if (i < current->pps_num_slices_in_pic_minus1) { + if (current->pps_tile_idx_delta_present_flag) { + ses(pps_tile_idx_delta_val[i], + -current->num_tiles_in_pic + 1, current->num_tiles_in_pic - 1, + 1, i); + if (current->pps_tile_idx_delta_val[i] == 0) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "pps_tile_idx_delta_val[i] shall not be equal to 0.\n"); + } + tile_idx += current->pps_tile_idx_delta_val[i]; + } else { + infer(pps_tile_idx_delta_val[i], 0); + tile_idx += current->pps_slice_width_in_tiles_minus1[i] + 1; + if (tile_idx % current->num_tile_columns == 0) { + tile_idx += current->pps_slice_height_in_tiles_minus1[i] * + current->num_tile_columns; + } + } + } + } + if (i == current->pps_num_slices_in_pic_minus1) { + uint16_t height= 0; + + tile_x = tile_idx % current->num_tile_columns; + tile_y = tile_idx / current->num_tile_columns; + + ctu_x = 0, ctu_y = 0; + for (j = 0; j < tile_x; j++) { + ctu_x += current->pps_tile_column_width_minus1[j] + 1; + } + for (j = 0; j < tile_y; j++) { + ctu_y += current->pps_tile_row_height_minus1[j] + 1; + } + slice_top_left_ctu_x[i] = ctu_x; + slice_top_left_ctu_y[i] = ctu_y; + + current->pps_slice_width_in_tiles_minus1[i] = + current->num_tile_columns - tile_x - 1; + current->pps_slice_height_in_tiles_minus1[i] = + current->num_tile_rows - tile_y - 1; + + for (j = 0; j <= current->pps_slice_height_in_tiles_minus1[i]; j++) { + height += current->pps_tile_row_height_minus1[tile_y + j] + 1; + } + current->slice_height_in_ctus[i] = height; + + infer(pps_num_exp_slices_in_tile[i], 0); + } + + //now, we got all slice information, let's resolve NumSlicesInSubpic + for (i = 0; i <= sps->sps_num_subpics_minus1; i++) { + current->num_slices_in_subpic[i] = 0; + for (j = 0; j <= current->pps_num_slices_in_pic_minus1; j++) { + uint16_t pos_x = 0, pos_y = 0; + pos_x = slice_top_left_ctu_x[j]; + pos_y = slice_top_left_ctu_y[j]; + if ((pos_x >= sps->sps_subpic_ctu_top_left_x[i]) && + (pos_x < sps->sps_subpic_ctu_top_left_x[i] + sps->sps_subpic_width_minus1[i] + 1) && + (pos_y >= sps->sps_subpic_ctu_top_left_y[i]) && + (pos_y < sps->sps_subpic_ctu_top_left_y[i] + sps->sps_subpic_height_minus1[i] + 1)) { + current->num_slices_in_subpic[i]++; + } + } + } + } else { + if (current->pps_no_pic_partition_flag) + infer(pps_num_slices_in_pic_minus1, 0); + else if (current->pps_single_slice_per_subpic_flag) + infer(pps_num_slices_in_pic_minus1, sps->sps_num_subpics_minus1); + // else? + } + if (!current->pps_rect_slice_flag || + current->pps_single_slice_per_subpic_flag || + current->pps_num_slices_in_pic_minus1 > 0) + flag(pps_loop_filter_across_slices_enabled_flag); + else + infer(pps_loop_filter_across_slices_enabled_flag, 0); + } else { + infer(pps_num_exp_tile_columns_minus1, 0); + infer(pps_tile_column_width_minus1[0], pic_width_in_ctbs_y - 1); + infer(pps_num_exp_tile_rows_minus1, 0); + infer(pps_tile_row_height_minus1[0], pic_height_in_ctbs_y - 1); + infer(num_tile_columns, 1); + infer(num_tile_rows, 1); + infer(num_tiles_in_pic, 1); + } + + flag(pps_cabac_init_present_flag); + for (i = 0; i < 2; i++) + ues(pps_num_ref_idx_default_active_minus1[i], 0, 14, 1, i); + flag(pps_rpl1_idx_present_flag); + flag(pps_weighted_pred_flag); + flag(pps_weighted_bipred_flag); + flag(pps_ref_wraparound_enabled_flag); + if (current->pps_ref_wraparound_enabled_flag) { + ue(pps_pic_width_minus_wraparound_offset, + 0, (current->pps_pic_width_in_luma_samples / min_cb_size_y) + - (ctb_size_y / min_cb_size_y) - 2); + } + + qp_bd_offset = 6 * sps->sps_bitdepth_minus8; + se(pps_init_qp_minus26, -(26 + qp_bd_offset), 37); + flag(pps_cu_qp_delta_enabled_flag); + flag(pps_chroma_tool_offsets_present_flag); + if (current->pps_chroma_tool_offsets_present_flag) { + se(pps_cb_qp_offset, -12, 12); + se(pps_cr_qp_offset, -12, 12); + flag(pps_joint_cbcr_qp_offset_present_flag); + if (current->pps_joint_cbcr_qp_offset_present_flag) + se(pps_joint_cbcr_qp_offset_value, -12, 12); + else + infer(pps_joint_cbcr_qp_offset_value, 0); + flag(pps_slice_chroma_qp_offsets_present_flag); + flag(pps_cu_chroma_qp_offset_list_enabled_flag); + if (current->pps_cu_chroma_qp_offset_list_enabled_flag) { + ue(pps_chroma_qp_offset_list_len_minus1, 0, 5); + for (i = 0; i <= current->pps_chroma_qp_offset_list_len_minus1; i++) { + ses(pps_cb_qp_offset_list[i], -12, 12, 1, i); + ses(pps_cr_qp_offset_list[i], -12, 12, 1, i); + if (current->pps_joint_cbcr_qp_offset_present_flag) + ses(pps_joint_cbcr_qp_offset_list[i], -12, 12, 1, i); + else + infer(pps_joint_cbcr_qp_offset_list[i], 0); + } + } + } else { + infer(pps_cb_qp_offset, 0); + infer(pps_cr_qp_offset, 0); + infer(pps_joint_cbcr_qp_offset_present_flag, 0); + infer(pps_joint_cbcr_qp_offset_value, 0); + infer(pps_slice_chroma_qp_offsets_present_flag, 0); + infer(pps_cu_chroma_qp_offset_list_enabled_flag, 0); + } + flag(pps_deblocking_filter_control_present_flag); + if (current->pps_deblocking_filter_control_present_flag) { + flag(pps_deblocking_filter_override_enabled_flag); + flag(pps_deblocking_filter_disabled_flag); + if (!current->pps_no_pic_partition_flag && + current->pps_deblocking_filter_override_enabled_flag) + flag(pps_dbf_info_in_ph_flag); + else + infer(pps_dbf_info_in_ph_flag, 0); + if (!current->pps_deblocking_filter_disabled_flag) { + se(pps_luma_beta_offset_div2, -12, 12); + se(pps_luma_tc_offset_div2, -12, 12); + if (current->pps_chroma_tool_offsets_present_flag) { + se(pps_cb_beta_offset_div2, -12, 12); + se(pps_cb_tc_offset_div2, -12, 12); + se(pps_cr_beta_offset_div2, -12, 12); + se(pps_cr_tc_offset_div2, -12, 12); + } else { + infer(pps_cb_beta_offset_div2, current->pps_luma_beta_offset_div2); + infer(pps_cb_tc_offset_div2, current->pps_luma_tc_offset_div2); + infer(pps_cr_beta_offset_div2, current->pps_luma_beta_offset_div2); + infer(pps_cr_tc_offset_div2, current->pps_luma_tc_offset_div2); + } + } else { + infer(pps_luma_beta_offset_div2, 0); + infer(pps_luma_tc_offset_div2, 0); + infer(pps_cb_beta_offset_div2, 0); + infer(pps_cb_tc_offset_div2, 0); + infer(pps_cr_beta_offset_div2, 0); + infer(pps_cr_tc_offset_div2, 0); + } + } else { + infer(pps_deblocking_filter_override_enabled_flag, 0); + infer(pps_deblocking_filter_disabled_flag, 0); + infer(pps_dbf_info_in_ph_flag, 0); + infer(pps_luma_beta_offset_div2, 0); + infer(pps_luma_tc_offset_div2, 0); + infer(pps_cb_beta_offset_div2, 0); + infer(pps_cb_tc_offset_div2, 0); + infer(pps_cr_beta_offset_div2, 0); + infer(pps_cr_tc_offset_div2, 0); + } + + if (!current->pps_no_pic_partition_flag) { + flag(pps_rpl_info_in_ph_flag); + flag(pps_sao_info_in_ph_flag); + flag(pps_alf_info_in_ph_flag); + if ((current->pps_weighted_pred_flag || current->pps_weighted_bipred_flag) && + current->pps_rpl_info_in_ph_flag) + flag(pps_wp_info_in_ph_flag); + flag(pps_qp_delta_info_in_ph_flag); + } + flag(pps_picture_header_extension_present_flag); + flag(pps_slice_header_extension_present_flag); + + flag(pps_extension_flag); + if (current->pps_extension_flag) + CHECK(FUNC(extension_data)(ctx, rw, ¤t->extension_data)); + + CHECK(FUNC(rbsp_trailing_bits)(ctx, rw)); + return 0; +} + +static int FUNC(aud)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawAUD *current) +{ + int err; + + HEADER("Access Unit Delimiter"); + + CHECK(FUNC(nal_unit_header)(ctx, rw, + ¤t->nal_unit_header, VVC_AUD_NUT)); + + flag(aud_irap_or_gdr_flag); + u(3, aud_pic_type, 0, 2); + + CHECK(FUNC(rbsp_trailing_bits)(ctx, rw)); + return 0; +} + +static int FUNC(pred_weight_table)(CodedBitstreamContext *ctx, RWContext *rw, + const H266RawSPS *sps, + const H266RawPPS *pps, + const H266RefPicLists *ref_lists, + uint8_t num_ref_idx_active[2], + H266RawPredWeightTable *current) +{ + int err, i, j; + uint8_t num_weights_l0, num_weights_l1; + ue(luma_log2_weight_denom, 0, 7); + if (sps->sps_chroma_format_idc != 0) { + se(delta_chroma_log2_weight_denom, + -current->luma_log2_weight_denom, 7 - current->luma_log2_weight_denom); + } else { + infer(delta_chroma_log2_weight_denom, 0); + } + if (pps->pps_wp_info_in_ph_flag) + ue(num_l0_weights, 0, FFMIN(15, ref_lists->rpl_ref_list[0].num_ref_entries)); + else + infer(num_l0_weights, 0); + num_weights_l0 = pps->pps_wp_info_in_ph_flag ? + current->num_l0_weights : num_ref_idx_active[0]; + for (i = 0; i < num_weights_l0; i++) { + flags(luma_weight_l0_flag[i], 1, i); + } + if (sps->sps_chroma_format_idc != 0) { + for (i = 0; i < num_weights_l0; i++) + flags(chroma_weight_l0_flag[i], 1, i); + } + for (i = 0; i < num_weights_l0; i++) { + if (current->luma_weight_l0_flag[i]){ + ses(delta_luma_weight_l0[i], -128, 127, 1, i); + ses(luma_offset_l0[i], -128, 127, 1, i); + } else { + infer(delta_luma_weight_l0[i], 0); + infer(luma_offset_l0[i], 0); + } + if (current->chroma_weight_l0_flag[i]) { + for (j = 0; j < 2; j++) { + ses(delta_chroma_weight_l0[i][j], -128, 127, 2, i, j); + ses(delta_chroma_offset_l0[i][j], -4 * 128, 4 * 127, 2, i, j); + } + } + } + + if (pps->pps_weighted_bipred_flag && pps->pps_wp_info_in_ph_flag && + ref_lists->rpl_ref_list[1].num_ref_entries > 0) { + ue(num_l1_weights, 0, FFMIN(15, ref_lists->rpl_ref_list[1].num_ref_entries)); + } + if (!pps->pps_weighted_bipred_flag || + (pps->pps_wp_info_in_ph_flag && + ref_lists->rpl_ref_list[1].num_ref_entries == 0)) { + num_weights_l1 = 0; + } else if (pps->pps_wp_info_in_ph_flag) { + num_weights_l1 = current->num_l1_weights; + } else { + num_weights_l1 = num_ref_idx_active[1]; + } + + for (i = 0; i < num_weights_l1; i++) + flags(luma_weight_l1_flag[i], 1, i); + if (sps->sps_chroma_format_idc != 0) { + for (i = 0; i < num_weights_l1; i++) + flags(chroma_weight_l1_flag[i], 1, i); + } + for (i = 0; i < num_weights_l1; i++) { + if (current->luma_weight_l1_flag[i]){ + ses(delta_luma_weight_l1[i], -128, 127, 1, i); + ses(luma_offset_l1[i], -128, 127, 1, i); + } else { + infer(delta_luma_weight_l1[i], 0); + infer(luma_offset_l1[i], 0); + } + if (current->chroma_weight_l1_flag[i]) { + for (j = 0; j < 2; j++) { + ses(delta_chroma_weight_l1[i][j], -128, 127, 2, i, j); + ses(delta_chroma_offset_l1[i][j], -4 * 128, 4 * 127, 2, i, j); + } + } + } + return 0; +} + +static int FUNC(picture_header)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawPH *current) +{ + CodedBitstreamH266Context *h266 = ctx->priv_data; + const H266RawVPS *vps; + const H266RawSPS *sps; + const H266RawPPS *pps; + int err, i; + unsigned int ctb_log2_size_y, min_cb_log2_size_y, + min_qt_log2_size_intra_y, min_qt_log2_size_inter_y; + uint8_t qp_bd_offset; + + flag(ph_gdr_or_irap_pic_flag); + flag(ph_non_ref_pic_flag); + if (current->ph_gdr_or_irap_pic_flag) + flag(ph_gdr_pic_flag); + else + infer(ph_gdr_pic_flag, 0); + flag(ph_inter_slice_allowed_flag); + if (current->ph_inter_slice_allowed_flag) + flag(ph_intra_slice_allowed_flag); + else + infer(ph_intra_slice_allowed_flag, 1); + ue(ph_pic_parameter_set_id, 0, VVC_MAX_PPS_COUNT - 1); + pps = h266->pps[current->ph_pic_parameter_set_id]; + if (!pps) { + av_log(ctx->log_ctx, AV_LOG_ERROR, "PPS id %d not available.\n", + current->ph_pic_parameter_set_id); + return AVERROR_INVALIDDATA; + } + sps = h266->sps[pps->pps_seq_parameter_set_id]; + if (!sps) { + av_log(ctx->log_ctx, AV_LOG_ERROR, "SPS id %d not available.\n", + pps->pps_seq_parameter_set_id); + return AVERROR_INVALIDDATA; + } + vps = h266->vps[sps->sps_video_parameter_set_id]; + if (!vps) { + av_log(ctx->log_ctx, AV_LOG_ERROR, "VPS id %d not available.\n", + sps->sps_video_parameter_set_id); + return AVERROR_INVALIDDATA; + } + + ub(sps->sps_log2_max_pic_order_cnt_lsb_minus4 + 4, ph_pic_order_cnt_lsb); + if (current->ph_gdr_pic_flag) + ue(ph_recovery_poc_cnt, 0, 1 << (sps->sps_log2_max_pic_order_cnt_lsb_minus4 + 4)); + + for (i = 0; i < sps->sps_num_extra_ph_bytes * 8; i++) { + if (sps->sps_extra_ph_bit_present_flag[i]) + flags(ph_extra_bit[i], 1, i); + } + if (sps->sps_poc_msb_cycle_flag) { + flag(ph_poc_msb_cycle_present_flag); + if (current->ph_poc_msb_cycle_present_flag) + ub(sps->sps_poc_msb_cycle_len_minus1 + 1, ph_poc_msb_cycle_val); + } + if (sps->sps_alf_enabled_flag && pps->pps_alf_info_in_ph_flag) { + flag(ph_alf_enabled_flag); + if (current->ph_alf_enabled_flag) { + + ub(3, ph_num_alf_aps_ids_luma); + for (i = 0; i < current->ph_num_alf_aps_ids_luma; i++) + ubs(3, ph_alf_aps_id_luma[i], 1, i); + + if (sps->sps_chroma_format_idc != 0) { + flag(ph_alf_cb_enabled_flag); + flag(ph_alf_cr_enabled_flag); + } else { + infer(ph_alf_cb_enabled_flag, 0); + infer(ph_alf_cr_enabled_flag, 0); + } + + if (current->ph_alf_cb_enabled_flag || current->ph_alf_cr_enabled_flag) { + ub(3, ph_alf_aps_id_chroma); + } + + if (sps->sps_ccalf_enabled_flag) { + flag(ph_alf_cc_cb_enabled_flag); + if (current->ph_alf_cc_cb_enabled_flag) + ub(3, ph_alf_cc_cb_aps_id); + flag(ph_alf_cc_cr_enabled_flag); + if (current->ph_alf_cc_cr_enabled_flag) + ub(3, ph_alf_cc_cr_aps_id); + } + } + } else { + infer(ph_alf_enabled_flag, 0); + } + if (sps->sps_lmcs_enabled_flag) { + flag(ph_lmcs_enabled_flag); + if (current->ph_lmcs_enabled_flag) { + ub(2, ph_lmcs_aps_id); + if (sps->sps_chroma_format_idc != 0) + flag(ph_chroma_residual_scale_flag); + else + infer(ph_chroma_residual_scale_flag, 0); + } + } else { + infer(ph_lmcs_enabled_flag, 0); + infer(ph_chroma_residual_scale_flag, 0); + } + + if (sps->sps_explicit_scaling_list_enabled_flag) { + flag(ph_explicit_scaling_list_enabled_flag); + if (current->ph_explicit_scaling_list_enabled_flag) { + //todo: check the ph_scaling_list_aps_id range, when aps ready + ub(3, ph_scaling_list_aps_id); + } + } else { + infer(ph_explicit_scaling_list_enabled_flag, 0); + } + if (sps->sps_virtual_boundaries_enabled_flag && + !sps->sps_virtual_boundaries_present_flag) { + flag(ph_virtual_boundaries_present_flag); + if (current->ph_virtual_boundaries_present_flag) { + ue(ph_num_ver_virtual_boundaries, + 0, pps->pps_pic_width_in_luma_samples <= 8 ? 0 : 3); + for (i = 0; i < current->ph_num_ver_virtual_boundaries; i++) { + ues(ph_virtual_boundary_pos_x_minus1[i], + 0, (pps->pps_pic_width_in_luma_samples + 7) / 8 - 2, 1, i); + } + ue(ph_num_hor_virtual_boundaries, + 0, pps->pps_pic_height_in_luma_samples <= 8 ? 0 : 3); + for (i = 0; i < current->ph_num_hor_virtual_boundaries; i++) { + ues(ph_virtual_boundary_pos_y_minus1[i], + 0, (pps->pps_pic_height_in_luma_samples + 7) / 8 - 2, 1, i); + } + } else { + infer(ph_num_ver_virtual_boundaries, 0); + infer(ph_num_hor_virtual_boundaries, 0); + } + } + if (pps->pps_output_flag_present_flag && !current->ph_non_ref_pic_flag) + flag(ph_pic_output_flag); + else + infer(ph_pic_output_flag, 1); + if (pps->pps_rpl_info_in_ph_flag) { + CHECK(FUNC(ref_pic_lists)(ctx, rw, sps, pps, ¤t->ph_ref_pic_lists)); + } + if (sps->sps_partition_constraints_override_enabled_flag) + flag(ph_partition_constraints_override_flag); + else + infer(ph_partition_constraints_override_flag, 0); + + ctb_log2_size_y = sps->sps_log2_ctu_size_minus5 + 5; + min_cb_log2_size_y = sps->sps_log2_min_luma_coding_block_size_minus2 + 2; + if (current->ph_intra_slice_allowed_flag) { + if (current->ph_partition_constraints_override_flag) { + ue(ph_log2_diff_min_qt_min_cb_intra_slice_luma, + 0, FFMIN(6, ctb_log2_size_y) - min_cb_log2_size_y); + ue(ph_max_mtt_hierarchy_depth_intra_slice_luma, + 0, 2 * (ctb_log2_size_y - min_cb_log2_size_y)); + if (current->ph_max_mtt_hierarchy_depth_intra_slice_luma != 0) { + ue(ph_log2_diff_max_bt_min_qt_intra_slice_luma, + 0, ctb_log2_size_y - min_qt_log2_size_intra_y); + ue(ph_log2_diff_max_tt_min_qt_intra_slice_luma, + 0, FFMIN(6, ctb_log2_size_y) - min_qt_log2_size_intra_y); + } else { + infer(ph_log2_diff_max_bt_min_qt_intra_slice_luma, + sps->sps_log2_diff_max_bt_min_qt_intra_slice_luma); + infer(ph_log2_diff_max_tt_min_qt_intra_slice_luma, + sps->sps_log2_diff_max_tt_min_qt_intra_slice_luma); + } + if (sps->sps_qtbtt_dual_tree_intra_flag) { + ue(ph_log2_diff_min_qt_min_cb_intra_slice_chroma, + 0, FFMIN(6, ctb_log2_size_y) - min_cb_log2_size_y); + ue(ph_max_mtt_hierarchy_depth_intra_slice_chroma, + 0, 2 * (ctb_log2_size_y - min_cb_log2_size_y)); + if (sps->sps_max_mtt_hierarchy_depth_intra_slice_chroma != 0) { + unsigned int min_qt_log2_size_intra_c = + sps->sps_log2_diff_min_qt_min_cb_intra_slice_chroma + + min_cb_log2_size_y; + ue(ph_log2_diff_max_bt_min_qt_intra_slice_chroma, + 0, FFMIN(6, ctb_log2_size_y) - min_qt_log2_size_intra_c); + ue(ph_log2_diff_max_tt_min_qt_intra_slice_chroma, + 0, FFMIN(6, ctb_log2_size_y) - min_qt_log2_size_intra_c); + } else { + infer(ph_log2_diff_max_bt_min_qt_intra_slice_chroma, + sps->sps_log2_diff_max_bt_min_qt_intra_slice_chroma); + infer(ph_log2_diff_max_tt_min_qt_intra_slice_chroma, + sps->sps_log2_diff_max_tt_min_qt_intra_slice_chroma); + } + } + } else { + infer(ph_log2_diff_min_qt_min_cb_intra_slice_luma, + sps->sps_log2_diff_min_qt_min_cb_intra_slice_luma); + infer(ph_max_mtt_hierarchy_depth_intra_slice_luma, + sps->sps_max_mtt_hierarchy_depth_intra_slice_luma); + infer(ph_log2_diff_max_bt_min_qt_intra_slice_luma, + sps->sps_log2_diff_max_bt_min_qt_intra_slice_luma); + infer(ph_log2_diff_max_tt_min_qt_intra_slice_luma, + sps->sps_log2_diff_max_tt_min_qt_intra_slice_luma); + infer(ph_log2_diff_min_qt_min_cb_intra_slice_chroma, + sps->sps_log2_diff_min_qt_min_cb_intra_slice_chroma); + infer(ph_max_mtt_hierarchy_depth_intra_slice_chroma, + sps->sps_max_mtt_hierarchy_depth_intra_slice_chroma); + infer(ph_log2_diff_max_bt_min_qt_intra_slice_chroma, + sps->sps_log2_diff_max_bt_min_qt_intra_slice_chroma); + infer(ph_log2_diff_max_tt_min_qt_intra_slice_chroma, + sps->sps_log2_diff_max_tt_min_qt_intra_slice_chroma); + } + + min_qt_log2_size_intra_y = + current->ph_log2_diff_min_qt_min_cb_intra_slice_luma + min_cb_log2_size_y; + if (pps->pps_cu_qp_delta_enabled_flag) + ue(ph_cu_qp_delta_subdiv_intra_slice, 0, + 2 * (ctb_log2_size_y - min_qt_log2_size_intra_y + + current->ph_max_mtt_hierarchy_depth_intra_slice_luma)); + else + infer(ph_cu_qp_delta_subdiv_intra_slice, 0); + + if (pps->pps_cu_chroma_qp_offset_list_enabled_flag) + ue(ph_cu_chroma_qp_offset_subdiv_intra_slice, 0, + 2 * (ctb_log2_size_y - min_qt_log2_size_intra_y + + current->ph_max_mtt_hierarchy_depth_intra_slice_luma)); + else + infer(ph_cu_chroma_qp_offset_subdiv_intra_slice, 0); + } + if (current->ph_inter_slice_allowed_flag) { + if (current->ph_partition_constraints_override_flag) { + ue(ph_log2_diff_min_qt_min_cb_inter_slice, + 0, FFMIN(6, ctb_log2_size_y) - min_cb_log2_size_y); + min_qt_log2_size_inter_y = + current->ph_log2_diff_min_qt_min_cb_inter_slice + min_cb_log2_size_y; + ue(ph_max_mtt_hierarchy_depth_inter_slice, + 0, 2 * (ctb_log2_size_y - min_cb_log2_size_y)); + if (current->ph_max_mtt_hierarchy_depth_inter_slice != 0) { + ue(ph_log2_diff_max_bt_min_qt_inter_slice, + 0, ctb_log2_size_y - min_qt_log2_size_inter_y); + ue(ph_log2_diff_max_tt_min_qt_inter_slice, + 0, FFMIN(6, ctb_log2_size_y) - min_qt_log2_size_inter_y); + } + } else { + infer(ph_log2_diff_min_qt_min_cb_inter_slice, + sps->sps_log2_diff_min_qt_min_cb_inter_slice); + min_qt_log2_size_inter_y = + current->ph_log2_diff_min_qt_min_cb_inter_slice + min_cb_log2_size_y; + infer(ph_max_mtt_hierarchy_depth_inter_slice, + sps->sps_max_mtt_hierarchy_depth_inter_slice); + infer(ph_log2_diff_max_bt_min_qt_inter_slice, + sps->sps_log2_diff_max_bt_min_qt_inter_slice); + infer(ph_log2_diff_max_tt_min_qt_inter_slice, + sps->sps_log2_diff_max_tt_min_qt_inter_slice); + } + + if (pps->pps_cu_qp_delta_enabled_flag) + ue(ph_cu_qp_delta_subdiv_inter_slice, 0, + 2 * (ctb_log2_size_y - min_qt_log2_size_inter_y + + current->ph_max_mtt_hierarchy_depth_inter_slice)); + else + infer(ph_cu_qp_delta_subdiv_inter_slice, 0); + + if (pps->pps_cu_chroma_qp_offset_list_enabled_flag) + ue(ph_cu_chroma_qp_offset_subdiv_inter_slice, 0, + 2 * (ctb_log2_size_y - min_qt_log2_size_inter_y + + current->ph_max_mtt_hierarchy_depth_inter_slice)); + else + infer(ph_cu_chroma_qp_offset_subdiv_inter_slice, 0); + if (sps->sps_temporal_mvp_enabled_flag) { + flag(ph_temporal_mvp_enabled_flag); + if (current->ph_temporal_mvp_enabled_flag && + pps->pps_rpl_info_in_ph_flag) { + if (current->ph_ref_pic_lists.rpl_ref_list[1].num_ref_entries > 0) + flag(ph_collocated_from_l0_flag); + else + infer(ph_collocated_from_l0_flag, 1); + if ((current->ph_collocated_from_l0_flag && + current->ph_ref_pic_lists.rpl_ref_list[0].num_ref_entries > 1) || + (!current->ph_collocated_from_l0_flag && + current->ph_ref_pic_lists.rpl_ref_list[1].num_ref_entries > 1)) { + unsigned int idx = current->ph_collocated_from_l0_flag ? 0 : 1; + ue(ph_collocated_ref_idx, 0, + current->ph_ref_pic_lists.rpl_ref_list[idx].num_ref_entries - 1); + } else { + infer(ph_collocated_ref_idx, 0); + } + } + } + if (sps->sps_mmvd_fullpel_only_enabled_flag) + flag(ph_mmvd_fullpel_only_flag); + else + infer(ph_mmvd_fullpel_only_flag, 0); + if (!pps->pps_rpl_info_in_ph_flag || + current->ph_ref_pic_lists.rpl_ref_list[1].num_ref_entries > 0) { + flag(ph_mvd_l1_zero_flag); + if (sps->sps_bdof_control_present_in_ph_flag) { + flag(ph_bdof_disabled_flag); + } else { + if (!sps->sps_bdof_control_present_in_ph_flag) + infer(ph_bdof_disabled_flag, 1 - sps->sps_bdof_enabled_flag); + else + infer(ph_bdof_disabled_flag, 1); + } + if (sps->sps_dmvr_control_present_in_ph_flag) { + flag(ph_dmvr_disabled_flag); + } else { + if (!sps->sps_dmvr_control_present_in_ph_flag) + infer(ph_dmvr_disabled_flag, 1 - sps->sps_dmvr_enabled_flag); + else + infer(ph_dmvr_disabled_flag, 1); + } + } else { + infer(ph_mvd_l1_zero_flag, 1); + } + if (sps->sps_prof_control_present_in_ph_flag) + flag(ph_prof_disabled_flag); + else + infer(ph_prof_disabled_flag, !sps->sps_affine_prof_enabled_flag); + if ((pps->pps_weighted_pred_flag || + pps->pps_weighted_bipred_flag) && + pps->pps_wp_info_in_ph_flag) { + + // if pps->pps_wp_info_in_ph_fla == 1 + // pred_weight_table will not use num_ref_idx_active + uint8_t num_ref_idx_active[2] = {0, 0}; + CHECK(FUNC(pred_weight_table)(ctx, rw, sps, pps, ¤t->ph_ref_pic_lists, + num_ref_idx_active, + ¤t->ph_pred_weight_table)); + } + } + + qp_bd_offset = 6 * sps->sps_bitdepth_minus8; + if (pps->pps_qp_delta_info_in_ph_flag) + se(ph_qp_delta, -qp_bd_offset - (26 + pps->pps_init_qp_minus26), + 63 - (26 + pps->pps_init_qp_minus26)); + + if (sps->sps_joint_cbcr_enabled_flag) + flag(ph_joint_cbcr_sign_flag); + else + infer(ph_joint_cbcr_sign_flag, 0); + if (sps->sps_sao_enabled_flag && pps->pps_sao_info_in_ph_flag) { + flag(ph_sao_luma_enabled_flag); + if (sps->sps_chroma_format_idc != 0) + flag(ph_sao_chroma_enabled_flag); + else + infer(ph_sao_chroma_enabled_flag, 0); + } else { + infer(ph_sao_luma_enabled_flag, 0); + infer(ph_sao_chroma_enabled_flag, 0); + } + + if (pps->pps_dbf_info_in_ph_flag) { + flag(ph_deblocking_params_present_flag); + if (current->ph_deblocking_params_present_flag) { + if (!pps->pps_deblocking_filter_disabled_flag) { + flag(ph_deblocking_filter_disabled_flag); + } else { + if (pps->pps_deblocking_filter_disabled_flag && + current->ph_deblocking_params_present_flag) { + infer(ph_deblocking_filter_disabled_flag, 0); + } else { + infer(ph_deblocking_filter_disabled_flag, + pps->pps_deblocking_filter_disabled_flag); + } + } + if (!current->ph_deblocking_filter_disabled_flag) { + se(ph_luma_beta_offset_div2, -12, 12); + se(ph_luma_tc_offset_div2, -12, 12); + if (pps->pps_chroma_tool_offsets_present_flag) { + se(ph_cb_beta_offset_div2, -12, 12); + se(ph_cb_tc_offset_div2, -12, 12); + se(ph_cr_beta_offset_div2, -12, 12); + se(ph_cr_tc_offset_div2, -12, 12); + } else { + infer(ph_cb_beta_offset_div2, current->ph_luma_beta_offset_div2); + infer(ph_cb_tc_offset_div2, current->ph_luma_tc_offset_div2); + infer(ph_cr_beta_offset_div2, current->ph_luma_beta_offset_div2); + infer(ph_cr_tc_offset_div2, current->ph_luma_tc_offset_div2); + } + } else { + infer(ph_luma_beta_offset_div2, pps->pps_luma_beta_offset_div2); + infer(ph_luma_tc_offset_div2, pps->pps_luma_tc_offset_div2); + if (pps->pps_chroma_tool_offsets_present_flag) { + infer(ph_cb_beta_offset_div2, pps->pps_cb_beta_offset_div2); + infer(ph_cb_tc_offset_div2, pps->pps_cb_tc_offset_div2); + infer(ph_cr_beta_offset_div2, pps->pps_cr_beta_offset_div2); + infer(ph_cr_tc_offset_div2, pps->pps_cr_tc_offset_div2); + } else { + infer(ph_cb_beta_offset_div2, current->ph_luma_beta_offset_div2); + infer(ph_cb_tc_offset_div2, current->ph_luma_tc_offset_div2); + infer(ph_cr_beta_offset_div2, current->ph_luma_beta_offset_div2); + infer(ph_cr_tc_offset_div2, current->ph_luma_tc_offset_div2); + } + } + } + } + + if (pps->pps_picture_header_extension_present_flag) { + ue(ph_extension_length, 0, 256); + for (i = 0; i < current->ph_extension_length; i++) + us(8, ph_extension_data_byte[i], 0x00, 0xff, 1, i); + } + + return 0; +} + +static int FUNC(ph)(CodedBitstreamContext *ctx, RWContext *rw, H266RawPH *current) +{ + int err; + + HEADER("Picture Header"); + + CHECK(FUNC(nal_unit_header)(ctx, rw, ¤t->nal_unit_header, VVC_PH_NUT)); + CHECK(FUNC(picture_header)(ctx, rw, current)); + CHECK(FUNC(rbsp_trailing_bits)(ctx, rw)); + return 0; +} + +static int FUNC(slice_header)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawSliceHeader *current) +{ + CodedBitstreamH266Context *h266 = ctx->priv_data; + const H266RawSPS *sps; + const H266RawPPS *pps; + const H266RawPH *ph; + const H266RefPicLists *ref_pic_lists; + int err, i; + uint8_t nal_unit_type, qp_bd_offset; + uint16_t curr_subpic_idx; + uint16_t num_slices_in_subpic; + + HEADER("Slice Header"); + + CHECK(FUNC(nal_unit_header)(ctx, rw, ¤t->nal_unit_header, -1)); + + flag(sh_picture_header_in_slice_header_flag); + if (current->sh_picture_header_in_slice_header_flag){ + CHECK(FUNC(picture_header)(ctx, rw, ¤t->sh_picture_header)); + ph = ¤t->sh_picture_header; + //7.4.8 if sh_picture_header_in_slice_header_flag is true, we do not have PH NAL unit + h266->priv.ph = NULL; + } else { + ph = h266->priv.ph; + if (!ph) { + av_log(ctx->log_ctx, AV_LOG_ERROR, "Picture header not available.\n"); + return AVERROR_INVALIDDATA; + } + } + + pps = h266->pps[ph->ph_pic_parameter_set_id]; + if (!pps) { + av_log(ctx->log_ctx, AV_LOG_ERROR, "PPS id %d not available.\n", + ph->ph_pic_parameter_set_id); + return AVERROR_INVALIDDATA; + } + sps = h266->sps[pps->pps_seq_parameter_set_id]; + if (!sps) { + av_log(ctx->log_ctx, AV_LOG_ERROR, "SPS id %d not available.\n", + pps->pps_seq_parameter_set_id); + return AVERROR_INVALIDDATA; + } + + if (sps->sps_subpic_info_present_flag) { + ub(sps->sps_subpic_id_len_minus1 + 1, sh_subpic_id); + if (sps->sps_subpic_id_mapping_explicitly_signalled_flag) { + for (i = 0; i <= sps->sps_num_subpics_minus1; i++) { + uint16_t subpic_id_val = pps->pps_subpic_id_mapping_present_flag ? + pps->pps_subpic_id[i] : sps->sps_subpic_id[i]; + if (subpic_id_val == current->sh_subpic_id) { + curr_subpic_idx = i; + break; + } + } + } else { + curr_subpic_idx = current->sh_subpic_id; + if (curr_subpic_idx > sps->sps_num_subpics_minus1) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "sh_subpic_id(%d) should in range [0, %d]\n", + curr_subpic_idx, sps->sps_num_subpics_minus1); + return AVERROR_INVALIDDATA; + } + } + } else { + curr_subpic_idx = 0; + } + + num_slices_in_subpic = pps->num_slices_in_subpic[curr_subpic_idx]; + + if ((pps->pps_rect_slice_flag && num_slices_in_subpic > 1) || + (!pps->pps_rect_slice_flag && pps->num_tiles_in_pic > 1)) { + unsigned int bits, max; + if (!pps->pps_rect_slice_flag) { + bits = av_ceil_log2(pps->num_tiles_in_pic); + max = pps->num_tiles_in_pic - 1; + } else { + bits = av_ceil_log2(num_slices_in_subpic); + max = num_slices_in_subpic - 1; + } + u(bits, sh_slice_address, 0, max); + } else { + infer(sh_slice_address, 0); + } + + for (i = 0; i < sps->sps_num_extra_sh_bytes * 8; i++) { + if (sps->sps_extra_sh_bit_present_flag[i]) + flags(sh_extra_bit[i], 1, i); + } + + if (!pps->pps_rect_slice_flag && + pps->num_tiles_in_pic - current->sh_slice_address > 1) + ue(sh_num_tiles_in_slice_minus1, 0, pps->num_tiles_in_pic - 1); + else + infer(sh_num_tiles_in_slice_minus1, 0); + + if (ph->ph_inter_slice_allowed_flag) + ue(sh_slice_type, 0, 2); + else + infer(sh_slice_type, 2); + + nal_unit_type = current->nal_unit_header.nal_unit_type; + if (nal_unit_type == VVC_IDR_W_RADL || + nal_unit_type == VVC_IDR_N_LP || + nal_unit_type == VVC_CRA_NUT || + nal_unit_type == VVC_GDR_NUT) + flag(sh_no_output_of_prior_pics_flag); + if (sps->sps_alf_enabled_flag && !pps->pps_alf_info_in_ph_flag) { + flag(sh_alf_enabled_flag); + if (current->sh_alf_enabled_flag) { + ub(3, sh_num_alf_aps_ids_luma); + for (i = 0; i < current->sh_num_alf_aps_ids_luma; i++) + ubs(3, sh_alf_aps_id_luma[i], 1, i); + if (sps->sps_chroma_format_idc != 0) { + flag(sh_alf_cb_enabled_flag); + flag(sh_alf_cr_enabled_flag); + } else { + infer(sh_alf_cb_enabled_flag, ph->ph_alf_cb_enabled_flag); + infer(sh_alf_cr_enabled_flag, ph->ph_alf_cr_enabled_flag); + } + if (current->sh_alf_cb_enabled_flag || + current->sh_alf_cr_enabled_flag) + ub(3, sh_alf_aps_id_chroma); + else + infer(sh_alf_aps_id_chroma, ph->ph_alf_aps_id_chroma); + if (sps->sps_ccalf_enabled_flag) { + flag(sh_alf_cc_cb_enabled_flag); + if (current->sh_alf_cc_cb_enabled_flag) + ub(3, sh_alf_cc_cb_aps_id); + else + infer(sh_alf_cc_cb_aps_id, ph->ph_alf_cc_cb_aps_id); + flag(sh_alf_cc_cr_enabled_flag); + if (current->sh_alf_cc_cr_enabled_flag) + ub(3, sh_alf_cc_cr_aps_id); + else + infer(sh_alf_cc_cr_aps_id, ph->ph_alf_cc_cr_aps_id); + } else { + infer(sh_alf_cc_cb_enabled_flag, ph->ph_alf_cc_cb_enabled_flag); + infer(sh_alf_cc_cr_enabled_flag, ph->ph_alf_cc_cr_enabled_flag); + infer(sh_alf_cc_cb_aps_id, ph->ph_alf_cc_cb_aps_id); + infer(sh_alf_cc_cr_aps_id, ph->ph_alf_cc_cr_aps_id); + } + } + } else { + infer(sh_alf_enabled_flag, 0); + } + + if (ph->ph_lmcs_enabled_flag && + !current->sh_picture_header_in_slice_header_flag) + flag(sh_lmcs_used_flag); + else + infer(sh_lmcs_used_flag, 0); + + if (ph->ph_explicit_scaling_list_enabled_flag && + !current->sh_picture_header_in_slice_header_flag) + flag(sh_explicit_scaling_list_used_flag); + else + infer(sh_explicit_scaling_list_used_flag, 0); + + if (!pps->pps_rpl_info_in_ph_flag && + ((nal_unit_type != VVC_IDR_W_RADL && + nal_unit_type != VVC_IDR_N_LP) || + sps->sps_idr_rpl_present_flag)) { + CHECK(FUNC(ref_pic_lists)(ctx, rw, sps, pps, ¤t->sh_ref_pic_lists)); + ref_pic_lists = ¤t->sh_ref_pic_lists; + } else { + ref_pic_lists = &h266->priv.ph->ph_ref_pic_lists; + } + if ((current->sh_slice_type != VVC_SLICE_TYPE_I && + ref_pic_lists->rpl_ref_list[0].num_ref_entries > 1) || + (current->sh_slice_type == VVC_SLICE_TYPE_B && + ref_pic_lists->rpl_ref_list[1].num_ref_entries > 1)) { + flag(sh_num_ref_idx_active_override_flag); + if (current->sh_num_ref_idx_active_override_flag) { + for (i = 0; i < (current->sh_slice_type == VVC_SLICE_TYPE_B ? 2: 1); i++) + if (ref_pic_lists->rpl_ref_list[i].num_ref_entries > 1) + ues(sh_num_ref_idx_active_minus1[i], 0, 14, 1, i); + else + infer(sh_num_ref_idx_active_minus1[i], 0); + } + } else { + infer(sh_num_ref_idx_active_override_flag, 1); + } + if (current->sh_slice_type != VVC_SLICE_TYPE_I) { + if (pps->pps_cabac_init_present_flag) + flag(sh_cabac_init_flag); + else + infer(sh_cabac_init_flag, 0); + if (ph->ph_temporal_mvp_enabled_flag && !pps->pps_rpl_info_in_ph_flag) { + uint8_t num_ref_idx_active[2]; + for (i = 0; i < 2; i++) { + if (current->sh_slice_type == VVC_SLICE_TYPE_B || + (current->sh_slice_type == VVC_SLICE_TYPE_P && i == 0)) { + if (current->sh_num_ref_idx_active_override_flag) { + num_ref_idx_active[i] = current->sh_num_ref_idx_active_minus1[i] + 1; + } + else { + num_ref_idx_active[i] = + FFMIN(ref_pic_lists->rpl_ref_list[i].num_ref_entries, + pps->pps_num_ref_idx_default_active_minus1[i] + 1); + } + } else { + num_ref_idx_active[i] = 0; + } + } + + if (current->sh_slice_type == VVC_SLICE_TYPE_B) + flag(sh_collocated_from_l0_flag); + else + infer(sh_collocated_from_l0_flag, 1); + if ((current->sh_collocated_from_l0_flag && num_ref_idx_active[0] > 1) || + (!current->sh_collocated_from_l0_flag && num_ref_idx_active[1] > 1)) { + unsigned int idx = current->sh_collocated_from_l0_flag ? 0 : 1; + ue(sh_collocated_ref_idx, 0, num_ref_idx_active[idx] - 1); + } else { + infer(sh_collocated_ref_idx, 0); + } + if (!pps->pps_wp_info_in_ph_flag && + ((pps->pps_weighted_pred_flag && + current->sh_slice_type == VVC_SLICE_TYPE_P) || + (pps->pps_weighted_bipred_flag && + current->sh_slice_type == VVC_SLICE_TYPE_B))) { + CHECK(FUNC(pred_weight_table)(ctx, rw, sps, pps, ref_pic_lists, + num_ref_idx_active, + ¤t->sh_pred_weight_table)); + } + } + + } + qp_bd_offset = 6 * sps->sps_bitdepth_minus8; + if (!pps->pps_qp_delta_info_in_ph_flag) + se(sh_qp_delta, -qp_bd_offset - (26 + pps->pps_init_qp_minus26), + 63 - (26 + pps->pps_init_qp_minus26)); + if (pps->pps_slice_chroma_qp_offsets_present_flag) { + int8_t off; + + se(sh_cb_qp_offset, -12, 12); + off = pps->pps_cb_qp_offset + current->sh_cb_qp_offset; + if (off < -12 || off > 12) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "pps_cb_qp_offset + sh_cb_qp_offset (%d) not in range [-12, 12].\n", + off); + return AVERROR_INVALIDDATA; + } + + se(sh_cr_qp_offset, -12, 12); + off = pps->pps_cr_qp_offset + current->sh_cr_qp_offset; + if (off < -12 || off > 12) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "pps_cr_qp_offset + sh_cr_qp_offset (%d) not in range [-12, 12].\n", + off); + return AVERROR_INVALIDDATA; + } + + if (sps->sps_joint_cbcr_enabled_flag) { + se(sh_joint_cbcr_qp_offset, -12, 12); + off = pps->pps_joint_cbcr_qp_offset_value + current->sh_joint_cbcr_qp_offset; + if (off < -12 || off > 12) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "pps_joint_cbcr_qp_offset_value + sh_joint_cbcr_qp_offset (%d)" + "not in range [-12, 12]. \n", + off); + return AVERROR_INVALIDDATA; + } + } else { + infer(sh_joint_cbcr_qp_offset, 0); + } + } else { + infer(sh_cb_qp_offset, 0); + infer(sh_cr_qp_offset, 0); + infer(sh_joint_cbcr_qp_offset, 0); + } + if (pps->pps_cu_chroma_qp_offset_list_enabled_flag) + flag(sh_cu_chroma_qp_offset_enabled_flag); + else + infer(sh_cu_chroma_qp_offset_enabled_flag, 0); + if (sps->sps_sao_enabled_flag && !pps->pps_sao_info_in_ph_flag) { + flag(sh_sao_luma_used_flag); + if (sps->sps_chroma_format_idc != 0) + flag(sh_sao_chroma_used_flag); + else + infer(sh_sao_chroma_used_flag, ph->ph_sao_chroma_enabled_flag); + } else { + infer(sh_sao_luma_used_flag, ph->ph_sao_luma_enabled_flag); + infer(sh_sao_chroma_used_flag, ph->ph_sao_chroma_enabled_flag); + } + + if (pps->pps_deblocking_filter_override_enabled_flag && + !pps->pps_dbf_info_in_ph_flag) + flag(sh_deblocking_params_present_flag); + else + infer(sh_deblocking_params_present_flag, 0); + if (current->sh_deblocking_params_present_flag) { + if (!pps->pps_deblocking_filter_disabled_flag) + flag(sh_deblocking_filter_disabled_flag); + else + infer(sh_deblocking_filter_disabled_flag, + !(pps->pps_deblocking_filter_disabled_flag && + current->sh_deblocking_params_present_flag)); + if (!current->sh_deblocking_filter_disabled_flag) { + se(sh_luma_beta_offset_div2, -12, 12); + se(sh_luma_tc_offset_div2, -12, 12); + if (pps->pps_chroma_tool_offsets_present_flag) { + se(sh_cb_beta_offset_div2, -12, 12); + se(sh_cb_tc_offset_div2, -12, 12); + se(sh_cr_beta_offset_div2, -12, 12); + se(sh_cr_tc_offset_div2, -12, 12); + } else { + infer(sh_cb_beta_offset_div2, current->sh_luma_beta_offset_div2); + infer(sh_cb_tc_offset_div2, current->sh_luma_tc_offset_div2); + infer(sh_cr_beta_offset_div2, current->sh_luma_beta_offset_div2); + infer(sh_cr_tc_offset_div2, current->sh_luma_tc_offset_div2); + } + } else { + infer(sh_luma_beta_offset_div2, ph->ph_luma_beta_offset_div2); + infer(sh_luma_tc_offset_div2, ph->ph_luma_tc_offset_div2); + if (pps->pps_chroma_tool_offsets_present_flag) { + infer(sh_cb_beta_offset_div2, ph->ph_cb_beta_offset_div2); + infer(sh_cb_tc_offset_div2, ph->ph_cb_tc_offset_div2); + infer(sh_cr_beta_offset_div2, ph->ph_cr_beta_offset_div2); + infer(sh_cr_tc_offset_div2, ph->ph_cr_beta_offset_div2); + } else { + infer(sh_cb_beta_offset_div2, current->sh_luma_beta_offset_div2); + infer(sh_cb_tc_offset_div2, current->sh_luma_tc_offset_div2); + infer(sh_cr_beta_offset_div2, current->sh_luma_beta_offset_div2); + infer(sh_cr_tc_offset_div2, current->sh_luma_tc_offset_div2); + } + } + } + + if (sps->sps_dep_quant_enabled_flag) + flag(sh_dep_quant_used_flag); + else + infer(sh_dep_quant_used_flag, 0); + + if (sps->sps_sign_data_hiding_enabled_flag && + !current->sh_dep_quant_used_flag) + flag(sh_sign_data_hiding_used_flag); + else + infer(sh_sign_data_hiding_used_flag, 0); + + if (sps->sps_transform_skip_enabled_flag && + !current->sh_dep_quant_used_flag && + !current->sh_sign_data_hiding_used_flag) + flag(sh_ts_residual_coding_disabled_flag); + else + infer(sh_ts_residual_coding_disabled_flag, 0); + if (pps->pps_slice_header_extension_present_flag) { + ue(sh_slice_header_extension_length, 0, 256); + for (i = 0; i < current->sh_slice_header_extension_length; i++) + us(8, sh_slice_header_extension_data_byte[i], 0x00, 0xff, 1, i); + } + if (sps->sps_entry_point_offsets_present_flag) { + int num_entry_points = 0; + uint8_t entropy_sync = sps->sps_entropy_coding_sync_enabled_flag; + int height; + if (pps->pps_rect_slice_flag) { + int width_in_tiles; + int slice_idx = current->sh_slice_address; + for (i = 0; i < curr_subpic_idx; i++) { + slice_idx += pps->num_slices_in_subpic[i]; + } + width_in_tiles = + pps->pps_slice_width_in_tiles_minus1[slice_idx] + 1; + + if (entropy_sync) + height = pps->slice_height_in_ctus[slice_idx]; + else + height = pps->pps_slice_height_in_tiles_minus1[slice_idx] + 1; + + num_entry_points = width_in_tiles * height; + } else { + int tile_idx; + int tile_y; + for (tile_idx = current->sh_slice_address; + tile_idx <= current->sh_slice_address + current->sh_num_tiles_in_slice_minus1; + tile_idx++) { + tile_y = tile_idx / pps->num_tile_rows; + height = pps->pps_tile_row_height_minus1[tile_y] + 1; + num_entry_points += (entropy_sync ? height : 1); + } + } + num_entry_points--; + if (num_entry_points > VVC_MAX_ENTRY_POINTS) { + av_log(ctx->log_ctx, AV_LOG_ERROR, "Too many entry points: " + "%"PRIu16".\n", num_entry_points); + return AVERROR_PATCHWELCOME; + } + if (num_entry_points > 0) { + ue(sh_entry_offset_len_minus1, 0, 31); + for (i = 0; i < num_entry_points; i++) { + ubs(current->sh_entry_offset_len_minus1 + 1, + sh_entry_point_offset_minus1[i], 1, i); + } + } + } + CHECK(FUNC(byte_alignment)(ctx, rw)); + + return 0; +} + +static int FUNC(sei_decoded_picture_hash)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawSEIDecodedPictureHash *current) +{ + int err, c_idx, i; + + HEADER("Decoded Picture Hash"); + + u(8, dph_sei_hash_type, 0, 2); + flag(dph_sei_single_component_flag); + ub(7, dph_sei_reserved_zero_7bits); + + for (c_idx = 0; c_idx < (current->dph_sei_single_component_flag ? 1 : 3); c_idx++) { + if (current->dph_sei_hash_type == 0) { + for (i = 0; i < 16; i++) + us(8, dph_sei_picture_md5[c_idx][i], 0x00, 0xff, 2, c_idx, i); + } else if (current->dph_sei_hash_type == 1) { + us(16, dph_sei_picture_crc[c_idx], 0x0000, 0xffff, 1, c_idx); + } else if (current->dph_sei_hash_type == 2) { + us(32, dph_sei_picture_checksum[c_idx], 0x00000000, 0xffffffff, 1, c_idx); + } + } + return 0; +} + +static int FUNC(sei)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawSEI *current, int prefix) +{ + int err; + + if (prefix) + HEADER("Prefix Supplemental Enhancement Information"); + else + HEADER("Suffix Supplemental Enhancement Information"); + + CHECK(FUNC(nal_unit_header)(ctx, rw, ¤t->nal_unit_header, + prefix ? VVC_PREFIX_SEI_NUT + : VVC_SUFFIX_SEI_NUT)); + + CHECK(FUNC_SEI(message_list)(ctx, rw, ¤t->message_list, prefix)); + + CHECK(FUNC(rbsp_trailing_bits)(ctx, rw)); + + return 0; +} diff --git a/libavcodec/cbs_sei.c b/libavcodec/cbs_sei.c index 50a513f592..bd7f6f4938 100644 --- a/libavcodec/cbs_sei.c +++ b/libavcodec/cbs_sei.c @@ -20,6 +20,7 @@ #include "cbs_internal.h" #include "cbs_h264.h" #include "cbs_h265.h" +#include "cbs_h266.h" #include "cbs_sei.h" static void cbs_free_user_data_registered(void *opaque, uint8_t *data) @@ -132,6 +133,13 @@ static int cbs_sei_get_unit(CodedBitstreamContext *ctx, else sei_type = HEVC_NAL_SEI_SUFFIX; break; + case AV_CODEC_ID_H266: + highest_vcl_type = VVC_RSV_IRAP_11; + if (prefix) + sei_type = VVC_PREFIX_SEI_NUT; + else + sei_type = VVC_SUFFIX_SEI_NUT; + break; default: return AVERROR(EINVAL); } @@ -207,6 +215,18 @@ static int cbs_sei_get_unit(CodedBitstreamContext *ctx, memcpy(unit->content, &sei, sizeof(sei)); } break; + case AV_CODEC_ID_H266: + { + H266RawSEI sei = { + .nal_unit_header = { + .nal_unit_type = sei_type, + .nuh_layer_id = 0, + .nuh_temporal_id_plus1 = 1, + }, + }; + memcpy(unit->content, &sei, sizeof(sei)); + } + break; default: av_assert0(0); } @@ -237,6 +257,15 @@ static int cbs_sei_get_message_list(CodedBitstreamContext *ctx, *list = &sei->message_list; } break; + case AV_CODEC_ID_H266: + { + H266RawSEI *sei = unit->content; + if (unit->type != VVC_PREFIX_SEI_NUT && + unit->type != VVC_SUFFIX_SEI_NUT) + return AVERROR(EINVAL); + *list = &sei->message_list; + } + break; default: return AVERROR(EINVAL); } From patchwork Wed Oct 19 07:25:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Siedel X-Patchwork-Id: 38811 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp215749pzb; Wed, 19 Oct 2022 00:26:56 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6GwxPczdhNhSsaKEjfZcsIGnwwMdCKVAs2ByJa5pl9jzmgSCqzSS8rCJFJ1GQ66RlsB390 X-Received: by 2002:a17:907:980e:b0:78d:b6d8:f53c with SMTP id ji14-20020a170907980e00b0078db6d8f53cmr5751424ejc.70.1666164415079; Wed, 19 Oct 2022 00:26:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666164415; cv=none; d=google.com; s=arc-20160816; b=Qs2AZ4Vp3+cv835rPGzLZ/Q570/fJUK2HublEHHePBcGawgjxX57XsbJdDpOGDioWX p+a8R4KYXVr0+DbskoayzUjEtLO+d7PD52TKp4kamzkd+JL2thFDAyzNBSddOHFWH3q1 bs08Z4xxYsOxzvEqEMO/Fmly+gPQkbgh0sPkDShq1RX3/oF1SMeWlRNnuhUsRikQDriH MZoj+M/K7Dy9STZ3y4QmnEYNts1qfhV8ZR/hPd79NxID0yHvLLYcVvakDyrfpiNsAUTX ZrZ4PkbvOmBg59mQ8wGJW/Juv8pAnb0c8GBpEu9YyjPK/kBQ9iaIYHENmuKxbXAlfqEC KMLg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=rNtl7ogIpk3iQh3ZZrAX9DpiRzpMAHh5iYo8aSXXkVA=; b=WnJfZbla9YcKbuWKDHH5WNqNyqKr1NCoTLY80gdGOPU6cEDmCKMlcyHrJdTHlP+KF9 x9A1p+EswB/ElVw5a3/HuOovf7TdJUDFufVQw8TpE/jz0/U086dQ1lHHXxNgnnBipeBd dGoAR0iX5AsRf4tfCZjCYfceIhdFR00AI2TNxNpU6RCJeHpibLypocZ/Sml6+L2XPiPO 1DgGORgtIajdCd47IynDdPu3kV5vSk/eYWRL+jXkj1D/+0hQiAfrCbfZZMcczNm8qisD /DdC6UMtZFqQRc3MduxppH/q1UUnAbJSN6TlXfl+le5hmwRhnnxk0lXbTEPCU06ei27N //ZQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=LyIGhiOn; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id nb35-20020a1709071ca300b0078dce6e5cf8si11120654ejc.436.2022.10.19.00.26.54; Wed, 19 Oct 2022 00:26:55 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=LyIGhiOn; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C482968BDD1; Wed, 19 Oct 2022 10:26:33 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f45.google.com (mail-ej1-f45.google.com [209.85.218.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 30BF568BC1C for ; Wed, 19 Oct 2022 10:26:27 +0300 (EEST) Received: by mail-ej1-f45.google.com with SMTP id 13so37745568ejn.3 for ; Wed, 19 Oct 2022 00:26:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spin-digital-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+AvceJpjdoHpInUCRZgLoGOxrnZHGAtQc6698NQEOZ0=; b=LyIGhiOnvgOsFGlk2aurtffGKQBkoKs8P0V2x8P1v5wc+3rFFmbXfeh/S84VGLW6/M W56vtLzcHoIe+jvBO+MrMreWpL7iexu3R8Fpb82DngKw+/4jgGc5ZGXmAYO0qCVPY4Pf jheCvGRKWQqx9bgS2qJCct3U53pL/KdRdz0/oo6p1Pyf5wXy6y5P9MjgEbmYuBQIiQjP ta8KRSWAmi6xC85k2x5nvk6J2M5R9AtmcJmmdgrqhFbLvU9cUXVkRv4cJSJRmBttChS1 N2d5j6p0Q28qmlEyaoZ0IyBFXcbbpjd3vEN+qTRUFiAm/opUxnHtFvS1sSKiDpxzKQRd JR5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+AvceJpjdoHpInUCRZgLoGOxrnZHGAtQc6698NQEOZ0=; b=TfIKnQfozdu1HCex1IcypLPzBL9Tibpmg8/z9JMNc9EFwCDbWNoQ1O5oB+p3XwDtPk hTGmvCQ67K77T0QUCbBWaqeYYbu4wVarlccm+rJOTXH5gfYENx6vdDJeWjEP2sy/DTZy JrM6Tf6l3WE5A9R+m2NO/MBboUOAr7pR79JlG5tMInohkke9/7JSlXRVJHIPEXntpk3I EGv/G5hZZn6BeVbSyeiJmMhwmhPC7aa08R2s21x7kdeoAL4g5MFYWPnK+TIa0AMlYIoo ul4EFyqLtKkOiyY4J1C+AqvuKXih48qrFRYS4Xn9ALPqSQtQ5UezvkmLXpKWd+znyWwK u2Aw== X-Gm-Message-State: ACrzQf3/fHz8bR1z+7VrloWdkEG5re7ptEesblUX5X09t7SjxCgHMR49 wlp0YDE7blRVw5/5C+lLPK9FvQ9tbFIHBw== X-Received: by 2002:a17:907:62a7:b0:789:48ea:ddb0 with SMTP id nd39-20020a17090762a700b0078948eaddb0mr5394683ejc.575.1666164386231; Wed, 19 Oct 2022 00:26:26 -0700 (PDT) Received: from thomas-win.localdomain ([213.138.44.237]) by smtp.gmail.com with ESMTPSA id p5-20020a170906498500b0078b83968ad4sm8515150eju.24.2022.10.19.00.26.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Oct 2022 00:26:25 -0700 (PDT) From: thomas.ff@spin-digital.com To: ffmpeg-devel@ffmpeg.org Date: Wed, 19 Oct 2022 09:25:00 +0200 Message-Id: <20221019072508.23460-4-thomas.ff@spin-digital.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221019072508.23460-1-thomas.ff@spin-digital.com> References: <20221019072508.23460-1-thomas.ff@spin-digital.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v1 03/11] avcodec: enable cbs for H266/VVC X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Thomas Siedel Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: VagvKBTCDsX6 From: Thomas Siedel Add ff_cbs_type_h266 to cbs types tables and AV_CODEC_ID_H266 to cbs codec ids. Change CBS_MAX_UNIT_TYPES to 8 as VVC implements 8 different slice types Signed-off-by: Thomas Siedel --- libavcodec/Makefile | 1 + libavcodec/cbs.c | 6 ++++++ libavcodec/cbs_internal.h | 3 ++- 3 files changed, 9 insertions(+), 1 deletion(-) diff --git a/libavcodec/Makefile b/libavcodec/Makefile index c7dc5da0f9..b3fcf173e9 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -76,6 +76,7 @@ OBJS-$(CONFIG_CBS) += cbs.o cbs_bsf.o OBJS-$(CONFIG_CBS_AV1) += cbs_av1.o OBJS-$(CONFIG_CBS_H264) += cbs_h2645.o cbs_sei.o h2645_parse.o OBJS-$(CONFIG_CBS_H265) += cbs_h2645.o cbs_sei.o h2645_parse.o +OBJS-$(CONFIG_CBS_H266) += cbs_h2645.o cbs_sei.o h2645_parse.o OBJS-$(CONFIG_CBS_JPEG) += cbs_jpeg.o OBJS-$(CONFIG_CBS_MPEG2) += cbs_mpeg2.o OBJS-$(CONFIG_CBS_VP9) += cbs_vp9.o diff --git a/libavcodec/cbs.c b/libavcodec/cbs.c index 8d6e3c3442..f01726aa37 100644 --- a/libavcodec/cbs.c +++ b/libavcodec/cbs.c @@ -40,6 +40,9 @@ static const CodedBitstreamType *const cbs_type_table[] = { #if CONFIG_CBS_H265 &ff_cbs_type_h265, #endif +#if CONFIG_CBS_H266 + &ff_cbs_type_h266, +#endif #if CONFIG_CBS_JPEG &ff_cbs_type_jpeg, #endif @@ -61,6 +64,9 @@ const enum AVCodecID ff_cbs_all_codec_ids[] = { #if CONFIG_CBS_H265 AV_CODEC_ID_H265, #endif +#if CONFIG_CBS_H266 + AV_CODEC_ID_H266, +#endif #if CONFIG_CBS_JPEG AV_CODEC_ID_MJPEG, #endif diff --git a/libavcodec/cbs_internal.h b/libavcodec/cbs_internal.h index 5ccba3c901..b741030da4 100644 --- a/libavcodec/cbs_internal.h +++ b/libavcodec/cbs_internal.h @@ -45,7 +45,7 @@ enum CBSContentType { enum { // Maximum number of unit types described by the same unit type // descriptor. - CBS_MAX_UNIT_TYPES = 3, + CBS_MAX_UNIT_TYPES = 8, // Maximum number of reference buffer offsets in any one unit. CBS_MAX_REF_OFFSETS = 2, // Special value used in a unit type descriptor to indicate that it @@ -242,6 +242,7 @@ int ff_cbs_write_signed(CodedBitstreamContext *ctx, PutBitContext *pbc, extern const CodedBitstreamType ff_cbs_type_av1; extern const CodedBitstreamType ff_cbs_type_h264; extern const CodedBitstreamType ff_cbs_type_h265; +extern const CodedBitstreamType ff_cbs_type_h266; extern const CodedBitstreamType ff_cbs_type_jpeg; extern const CodedBitstreamType ff_cbs_type_mpeg2; extern const CodedBitstreamType ff_cbs_type_vp9; From patchwork Wed Oct 19 07:25:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Siedel X-Patchwork-Id: 38813 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp215861pzb; Wed, 19 Oct 2022 00:27:17 -0700 (PDT) X-Google-Smtp-Source: AMsMyM70CPeJjHMsFwNikJE7nx4vm5WJvLbJsBfFH153WqbtsWMd1JH064cyZ2T7poAIlpz7SgqU X-Received: by 2002:a17:907:a06d:b0:78d:ceae:ba5a with SMTP id ia13-20020a170907a06d00b0078dceaeba5amr5648920ejc.586.1666164437043; Wed, 19 Oct 2022 00:27:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666164437; cv=none; d=google.com; s=arc-20160816; b=mlJ7eSXeKSq76Hq0yAONyEjxKdwgHwTrJBYJ07rwUjUfs4iN+FmqTQRn44Vh1IA8z5 heKSSF8gbvvpRyWYNbw1K6SWWN5krHrQgxTbNSD/edZ5f/0RUD7hCy2WWpKVRDexQM4V eG6TFRTVmUFOmcEkmrTiguAkv++pV2FpCfAt10cSSp/BRppv+CB77fbdpX7sfrrfXKqY FdLQhGbqb0+pRSTWX2ipyTBSUrI0Q4WfEW6kiAsgPNbOyMk9mXG9LGxIg1Rv/++hFap5 m2oTFqlrnwula0TUUT5kEtpbK0+GDA6eN3BzEbCebnYzeJXdJkXNDmPZyomxmYivmxDQ c4FA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=xuBam7ySt2XS/grdwV+URCTeukWBf9RuH4c7eHh67n4=; b=muwl0mkhqlOdpS7Gj5gOk3+EpkOwnCynphN6eJ5G0b2iuDdoXkmVXifJhzhBKc0OVM ufWG96CnD6vef+QgATCub9t2X48OOOR9htmgr4FRhU4D08FJw1wOF+MTVqf640hjNrmv 8NMNVH23XZPhORO5+kH70XZfJdRJZB4lvwLP8hQiE6HracUHPBntVB/QlxE1W+pn9fjC 3Dx7zpYdKnHrFXkQ/aGmQjYiYyTeU25EtSH3bPxfiWrML/kH0X9faM+1sp6W09BS4Mi/ MBJsnVyMgGTgR5OGOD1CBzr7NNdFl4KxIZlEpUwWcSqJCQmiTcMnK612mHf4syH+uCHu jdng== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=3p9Yxq8N; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id lb25-20020a170907785900b00741d9a1e2dbsi12876590ejc.1004.2022.10.19.00.27.16; Wed, 19 Oct 2022 00:27:17 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=3p9Yxq8N; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1C3B568BBC9; Wed, 19 Oct 2022 10:26:36 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f43.google.com (mail-ej1-f43.google.com [209.85.218.43]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6281C68BD2E for ; Wed, 19 Oct 2022 10:26:28 +0300 (EEST) Received: by mail-ej1-f43.google.com with SMTP id fy4so37798930ejc.5 for ; Wed, 19 Oct 2022 00:26:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spin-digital-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=L7sPVx/ywHmQLDz6lTQZEC83ldqRDZu3e0mciEDAmXA=; b=3p9Yxq8NA2hNr3rrvbhP9L5pdm1QczcfcfF/WnqM19508u6ifk40+D3k8YnPuHwhac Z9phlE+w7ShaAm7NJeK04HepVzxUVHcHNA6jkc5SAP0v/hWZ1H2dcVzLE7IlPn/kfz+H TRLbFeGeJvXIz5sFL7/c7mMTTCT2Kxk4Ee9b0g7VnJdnDYmHUpn4ad21u4xbRgdsu9NZ KRbUh8gWpOqnt1H8te1SFxX8JmmqXEomNgIcb3oG3CEjXemdHB0wz1kpN8E4mPAIb3vB ALyogec9onMQJ8afW8z5EDBoRyaFZ++KUGNSj27tSpun3rYMbSLA8WgMHpkl+36k+/IZ Tg/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=L7sPVx/ywHmQLDz6lTQZEC83ldqRDZu3e0mciEDAmXA=; b=LlMLQ3R7jAUNy/VvzvFKr3rnX0FLDOComGE7mMmAsMe1VBb7XNrWTlHiGCetr/6hxB aBkdoy2mmvU4ZtybQi89N4KpJyd4qxxXJ42SXOOvt2fDhV8pRA3TzP/GsM8OKvKuViV2 ocgigwwKiT12HgOQ3ZoXmJ61uJwK01pFTyEJl9nAZ3V4mFbAASN8x/i3w4v93SbkFa6Q v7Z7HhyscRbOTORloTd3LDLXOlwPR7y0cKhI/uzVvYe6qZdVBsgC11Q+H1bVH+bGJ4kG fuSVpi5aL1TMuxfs5yxnqgAaMqhUFDH9uKPN/TObB9Jra7VGZLFsT0XlanK/j5oAwRVS YrQA== X-Gm-Message-State: ACrzQf3YeyBxhhRKsWfkCqyLz2nfyJV2VwZEBg0AZ6Tw84RmvRVg8xoF 8ZSASsNHnlyfIGZ2DEaRRAFrA2JYJycyAA== X-Received: by 2002:a17:907:a428:b0:78d:9fab:84fb with SMTP id sg40-20020a170907a42800b0078d9fab84fbmr5346088ejc.694.1666164387158; Wed, 19 Oct 2022 00:26:27 -0700 (PDT) Received: from thomas-win.localdomain ([213.138.44.237]) by smtp.gmail.com with ESMTPSA id p5-20020a170906498500b0078b83968ad4sm8515150eju.24.2022.10.19.00.26.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Oct 2022 00:26:26 -0700 (PDT) From: thomas.ff@spin-digital.com To: ffmpeg-devel@ffmpeg.org Date: Wed, 19 Oct 2022 09:25:01 +0200 Message-Id: <20221019072508.23460-5-thomas.ff@spin-digital.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221019072508.23460-1-thomas.ff@spin-digital.com> References: <20221019072508.23460-1-thomas.ff@spin-digital.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v1 04/11] avcodec: add bitstream parser for H266/VVC X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Thomas Siedel Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Nb7yFLRMZpAg From: Thomas Siedel Add nal parser ff_vvc_parser to parse vvc elementary bitstreams. Signed-off-by: Thomas Siedel --- configure | 3 + libavcodec/Makefile | 1 + libavcodec/h2645_parse.c | 71 ++++- libavcodec/parsers.c | 1 + libavcodec/vvc_parser.c | 588 +++++++++++++++++++++++++++++++++++++++ 5 files changed, 663 insertions(+), 1 deletion(-) create mode 100644 libavcodec/vvc_parser.c diff --git a/configure b/configure index f3fd91f592..13dca5cd9a 100755 --- a/configure +++ b/configure @@ -2446,6 +2446,7 @@ CONFIG_EXTRA=" cbs_av1 cbs_h264 cbs_h265 + cbs_h266 cbs_jpeg cbs_mpeg2 cbs_vp9 @@ -2727,6 +2728,7 @@ threads_if_any="$THREADS_LIST" cbs_av1_select="cbs" cbs_h264_select="cbs" cbs_h265_select="cbs" +cbs_h266_select="cbs" cbs_jpeg_select="cbs" cbs_mpeg2_select="cbs" cbs_vp9_select="cbs" @@ -3278,6 +3280,7 @@ hevc_parser_select="hevcparse" mpegaudio_parser_select="mpegaudioheader" mpeg4video_parser_select="h263dsp mpegvideodec qpeldsp" vc1_parser_select="vc1dsp" +vvc_parser_select="cbs_h266" # bitstream_filters aac_adtstoasc_bsf_select="adts_header mpeg4audio" diff --git a/libavcodec/Makefile b/libavcodec/Makefile index b3fcf173e9..3850e316ac 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1173,6 +1173,7 @@ OBJS-$(CONFIG_VC1_PARSER) += vc1_parser.o vc1.o vc1data.o \ OBJS-$(CONFIG_VP3_PARSER) += vp3_parser.o OBJS-$(CONFIG_VP8_PARSER) += vp8_parser.o OBJS-$(CONFIG_VP9_PARSER) += vp9_parser.o +OBJS-$(CONFIG_VVC_PARSER) += vvc_parser.o OBJS-$(CONFIG_WEBP_PARSER) += webp_parser.o OBJS-$(CONFIG_XBM_PARSER) += xbm_parser.o OBJS-$(CONFIG_XMA_PARSER) += xma_parser.o diff --git a/libavcodec/h2645_parse.c b/libavcodec/h2645_parse.c index 90944177c7..9fbeee3edd 100644 --- a/libavcodec/h2645_parse.c +++ b/libavcodec/h2645_parse.c @@ -27,6 +27,7 @@ #include "libavutil/mem.h" #include "bytestream.h" +#include "vvc.h" #include "hevc.h" #include "h264.h" #include "h2645_parse.h" @@ -145,6 +146,47 @@ nsc: return si; } +static const char *const vvc_nal_type_name[32] = { + "TRAIL_NUT", // VVC_TRAIL_NUT + "STSA_NUT", // VVC_STSA_NUT + "RADL_NUT", // VVC_RADL_NUT + "RASL_NUT", // VVC_RASL_NUT + "RSV_VCL_4", // VVC_RSV_VCL_4 + "RSV_VCL_5", // VVC_RSV_VCL_5 + "RSV_VCL_6", // VVC_RSV_VCL_6 + "IDR_W_RADL", // VVC_IDR_W_RADL + "IDR_N_LP", // VVC_IDR_N_LP + "CRA_NUT", // VVC_CRA_NUT + "GDR_NUT", // VVC_GDR_NUT + "RSV_IRAP_11", // VVC_RSV_IRAP_11 + "OPI_NUT", // VVC_OPI_NUT + "DCI_NUT", // VVC_DCI_NUT + "VPS_NUT", // VVC_VPS_NUT + "SPS_NUT", // VVC_SPS_NUT + "PPS_NUT", // VVC_PPS_NUT + "PREFIX_APS_NUT",// VVC_PREFIX_APS_NUT + "SUFFIX_APS_NUT",// VVC_SUFFIX_APS_NUT + "PH_NUT", // VVC_PH_NUT + "AUD_NUT", // VVC_AUD_NUT + "EOS_NUT", // VVC_EOS_NUT + "EOB_NUT", // VVC_EOB_NUT + "PREFIX_SEI_NUT",// VVC_PREFIX_SEI_NUT + "SUFFIX_SEI_NUT",// VVC_SUFFIX_SEI_NUT + "FD_NUT", // VVC_FD_NUT + "RSV_NVCL_26", // VVC_RSV_NVCL_26 + "RSV_NVCL_27", // VVC_RSV_NVCL_27 + "UNSPEC_28", // VVC_UNSPEC_28 + "UNSPEC_29", // VVC_UNSPEC_29 + "UNSPEC_30", // VVC_UNSPEC_30 + "UNSPEC_31", // VVC_UNSPEC_31 +}; + +static const char *vvc_nal_unit_name(int nal_type) +{ + av_assert0(nal_type >= 0 && nal_type < 32); + return vvc_nal_type_name[nal_type]; +} + static const char *const hevc_nal_type_name[64] = { "TRAIL_N", // HEVC_NAL_TRAIL_N "TRAIL_R", // HEVC_NAL_TRAIL_R @@ -293,6 +335,31 @@ static int get_bit_length(H2645NAL *nal, int min_size, int skip_trailing_zeros) * @return AVERROR_INVALIDDATA if the packet is not a valid NAL unit, * 0 otherwise */ +static int vvc_parse_nal_header(H2645NAL *nal, void *logctx) +{ + GetBitContext *gb = &nal->gb; + + if (get_bits1(gb) != 0) //forbidden_zero_bit + return AVERROR_INVALIDDATA; + + skip_bits1(gb); //nuh_reserved_zero_bit + + nal->nuh_layer_id = get_bits(gb, 6); + nal->type = get_bits(gb, 5); + nal->temporal_id = get_bits(gb, 3) - 1; + if (nal->temporal_id < 0) + return AVERROR_INVALIDDATA; + + if ((nal->type >= VVC_IDR_W_RADL && nal->type <= VVC_RSV_IRAP_11) && nal->temporal_id) + return AVERROR_INVALIDDATA; + + av_log(logctx, AV_LOG_DEBUG, + "nal_unit_type: %d(%s), nuh_layer_id: %d, temporal_id: %d\n", + nal->type, vvc_nal_unit_name(nal->type), nal->nuh_layer_id, nal->temporal_id); + + return 0; +} + static int hevc_parse_nal_header(H2645NAL *nal, void *logctx) { GetBitContext *gb = &nal->gb; @@ -509,7 +576,9 @@ int ff_h2645_packet_split(H2645Packet *pkt, const uint8_t *buf, int length, /* Reset type in case it contains a stale value from a previously parsed NAL */ nal->type = 0; - if (codec_id == AV_CODEC_ID_HEVC) + if (codec_id == AV_CODEC_ID_VVC) + ret = vvc_parse_nal_header(nal, logctx); + else if (codec_id == AV_CODEC_ID_HEVC) ret = hevc_parse_nal_header(nal, logctx); else ret = h264_parse_nal_header(nal, logctx); diff --git a/libavcodec/parsers.c b/libavcodec/parsers.c index d355808018..e5b01b7a59 100644 --- a/libavcodec/parsers.c +++ b/libavcodec/parsers.c @@ -74,6 +74,7 @@ extern const AVCodecParser ff_vorbis_parser; extern const AVCodecParser ff_vp3_parser; extern const AVCodecParser ff_vp8_parser; extern const AVCodecParser ff_vp9_parser; +extern const AVCodecParser ff_vvc_parser; extern const AVCodecParser ff_webp_parser; extern const AVCodecParser ff_xbm_parser; extern const AVCodecParser ff_xma_parser; diff --git a/libavcodec/vvc_parser.c b/libavcodec/vvc_parser.c new file mode 100644 index 0000000000..f7eb2340c5 --- /dev/null +++ b/libavcodec/vvc_parser.c @@ -0,0 +1,588 @@ +/* + * VVC parser + * + * Copyright (C) 2021 Nuo Mi + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "cbs.h" +#include "cbs_h266.h" +#include "internal.h" +#include "parser.h" +#include "decode.h" + +#define START_CODE 0x000001 ///< start_code_prefix_one_3bytes + +#define IS_SLICE(nut) (nut <= VVC_RASL_NUT || (nut >= VVC_IDR_W_RADL && nut <= VVC_GDR_NUT)) +#define IS_PH(nut) (nut == VVC_PH_NUT) +#define IS_IDR(nut) (nut == VVC_IDR_W_RADL || nut == VVC_IDR_N_LP) + + +typedef struct PuInfo { + AVBufferRef *sps_ref; + AVBufferRef *pps_ref; + AVBufferRef *slice_ref; + AVBufferRef *ph_ref; + + const H266RawPPS *pps; + const H266RawSPS *sps; + const H266RawPH *ph; + const H266RawSlice *slice; + int pic_type; +} PuInfo; + +typedef struct AuDetector { + uint8_t prev_layer_id; + int prev_tid0_poc; + int prev_poc; +} AuDetector; + +typedef struct VVCParserContext { + ParseContext pc; + CodedBitstreamContext *cbc; + + CodedBitstreamFragment picture_unit; + + PuInfo au_info; + AVPacket au; + AVPacket last_au; + + AuDetector au_detector; + + int parsed_extradata; +} VVCParserContext; + +static const enum AVPixelFormat pix_fmts_8bit[] = { + AV_PIX_FMT_GRAY8, AV_PIX_FMT_YUV420P, + AV_PIX_FMT_YUV422P, AV_PIX_FMT_YUV444P +}; + +static const enum AVPixelFormat pix_fmts_10bit[] = { + AV_PIX_FMT_GRAY10, AV_PIX_FMT_YUV420P10, + AV_PIX_FMT_YUV422P10, AV_PIX_FMT_YUV444P10 +}; + +static int get_format(const H266RawSPS *sps) +{ + switch (sps->sps_bitdepth_minus8) { + case 0: + return pix_fmts_8bit[sps->sps_chroma_format_idc]; + case 2: + return pix_fmts_10bit[sps->sps_chroma_format_idc]; + } + return AV_PIX_FMT_NONE; +} + +/** + * Find the end of the current frame in the bitstream. + * @return the position of the first byte of the next frame, or END_NOT_FOUND + */ +static int find_frame_end(AVCodecParserContext *s, const uint8_t *buf, + int buf_size) +{ + VVCParserContext *ctx = s->priv_data; + ParseContext *pc = &ctx->pc; + int i; + + for (i = 0; i < buf_size; i++) { + int nut; + + pc->state64 = (pc->state64 << 8) | buf[i]; + + if (((pc->state64 >> 3 * 8) & 0xFFFFFF) != START_CODE) + continue; + + nut = (pc->state64 >> (8 + 3)) & 0x1F; + // 7.4.2.4.3 and 7.4.2.4.4 + if ((nut >= VVC_OPI_NUT && nut <= VVC_PREFIX_APS_NUT && nut != VVC_PH_NUT) || + nut == VVC_AUD_NUT || (nut == VVC_PREFIX_SEI_NUT && !pc->frame_start_found) || nut == VVC_RSV_NVCL_26 || + nut == VVC_UNSPEC_28 || nut == VVC_UNSPEC_29) { + if (pc->frame_start_found) { + pc->frame_start_found = 0; + return i - 5; + } + } else if (nut == VVC_PH_NUT || IS_SLICE(nut)) { + int sh_picture_header_in_slice_header_flag = buf[i] >> 7; + + if (nut == VVC_PH_NUT || sh_picture_header_in_slice_header_flag) { + if (!pc->frame_start_found) { + pc->frame_start_found = 1; + } else { // First slice of next frame found + pc->frame_start_found = 0; + return i - 5; + } + } + } + } + return END_NOT_FOUND; +} + +static int get_pict_type(const CodedBitstreamFragment *pu) +{ + int has_p = 0; + for (int i = 0; i < pu->nb_units; i++) { + CodedBitstreamUnit *unit = &pu->units[i]; + if (IS_SLICE(unit->type)) { + const H266RawSlice *slice = unit->content; + uint8_t type = slice->header.sh_slice_type; + if (type == VVC_SLICE_TYPE_B) { + return AV_PICTURE_TYPE_B; + } + if (type == VVC_SLICE_TYPE_P) { + has_p = 1; + } + } + } + return has_p ? AV_PICTURE_TYPE_P : AV_PICTURE_TYPE_I; +} + +static void pu_info_unref(PuInfo *info) +{ + av_buffer_unref(&info->slice_ref); + av_buffer_unref(&info->ph_ref); + av_buffer_unref(&info->pps_ref); + av_buffer_unref(&info->sps_ref); + info->slice = NULL; + info->ph = NULL; + info->pps = NULL; + info->sps = NULL; + info->pic_type = AV_PICTURE_TYPE_NONE; +} + +static int pu_info_ref(PuInfo *dest, const PuInfo *src) +{ + pu_info_unref(dest); + dest->sps_ref = av_buffer_ref(src->sps_ref); + dest->pps_ref = av_buffer_ref(src->pps_ref); + if (src->ph_ref) + dest->ph_ref = av_buffer_ref(src->ph_ref); + dest->slice_ref = av_buffer_ref(src->slice_ref); + if (!dest->sps_ref || !dest->pps_ref || (src->ph_ref && !dest->ph_ref) || !dest->slice_ref) { + pu_info_unref(dest); + return AVERROR(ENOMEM); + } + + dest->sps = src->sps; + dest->pps = src->pps; + dest->ph = src->ph; + dest->slice = src->slice; + dest->pic_type = src->pic_type; + return 0; +} + +static int set_parser_ctx(AVCodecParserContext *ctx, AVCodecContext *avctx, + const PuInfo *pu) +{ + int ret, num = 0, den = 0; + static const uint8_t h266_sub_width_c[] = { + 1, 2, 2, 1 + }; + static const uint8_t h266_sub_height_c[] = { + 1, 2, 1, 1 + }; + const H266RawSPS *sps = pu->sps; + const H266RawPPS *pps = pu->pps; + //const H266RawPH *ph = pu->ph; + const H266RawNALUnitHeader *nal = &pu->slice->header.nal_unit_header; + + /* set some sane default values */ + ctx->pict_type = AV_PICTURE_TYPE_I; + ctx->key_frame = 0; + ctx->picture_structure = AV_PICTURE_STRUCTURE_FRAME; + + ctx->key_frame = nal->nal_unit_type == VVC_IDR_W_RADL || + nal->nal_unit_type == VVC_IDR_N_LP || + nal->nal_unit_type == VVC_CRA_NUT || + nal->nal_unit_type == VVC_GDR_NUT; + + ctx->coded_width = pps->pps_pic_width_in_luma_samples; + ctx->coded_height = pps->pps_pic_height_in_luma_samples; + ctx->width = pps->pps_pic_width_in_luma_samples - + (pps->pps_conf_win_left_offset + pps->pps_conf_win_right_offset) * + h266_sub_width_c[sps->sps_chroma_format_idc]; + ctx->height = pps->pps_pic_height_in_luma_samples - + (pps->pps_conf_win_top_offset + pps->pps_conf_win_bottom_offset) * + h266_sub_height_c[sps->sps_chroma_format_idc];; + ctx->pict_type = pu->pic_type; + ctx->format = get_format(sps); + + avctx->profile = sps->profile_tier_level.general_profile_idc; + avctx->level = sps->profile_tier_level.general_level_idc; + + avctx->colorspace = (enum AVColorSpace) sps->vui.vui_matrix_coeffs; + avctx->color_primaries = (enum AVColorPrimaries) sps->vui.vui_colour_primaries; + avctx->color_trc = (enum AVColorTransferCharacteristic) sps->vui.vui_transfer_characteristics; + avctx->color_range = sps->vui.vui_full_range_flag ? AVCOL_RANGE_JPEG : AVCOL_RANGE_MPEG; + + if (ctx->width != avctx->width || ctx->height != avctx->height) { + ret = ff_set_dimensions(avctx, ctx->width, ctx->height); + if (ret < 0) + return ret; + } + avctx->pix_fmt = ctx->format; + avctx->has_b_frames = (sps->sps_max_sublayers_minus1+1) > 2 ? 2 : sps->sps_max_sublayers_minus1; + avctx->max_b_frames = sps->sps_max_sublayers_minus1; + + if(sps->sps_ptl_dpb_hrd_params_present_flag && sps->sps_timing_hrd_params_present_flag) { + num = sps->sps_general_timing_hrd_parameters.num_units_in_tick; + den = sps->sps_general_timing_hrd_parameters.time_scale; + } else { + return 1; + } + if (num != 0 && den != 0) + av_reduce(&avctx->framerate.den, &avctx->framerate.num, + num, den, 1 << 30); + + if (avctx->framerate.num) + avctx->time_base = av_inv_q(av_mul_q(avctx->framerate, (AVRational){avctx->ticks_per_frame, 1})); + + return 1; +} + +static int set_ctx(AVCodecParserContext *ctx, AVCodecContext *avctx, const PuInfo *next_pu) +{ + VVCParserContext *s = ctx->priv_data; + int ret; + if (s->au_info.slice) { + if ((ret = set_parser_ctx(ctx, avctx, &s->au_info)) < 0) + return ret; + } + ret = pu_info_ref(&s->au_info, next_pu); + return ret; +} + +//8.3.1 Decoding process for picture order count. +//VTM did not follow the spec, and it's much simpler than spec. +//We follow the VTM. +static void get_slice_poc(VVCParserContext *s, int *poc, + const H266RawSPS *sps, + const H266RawPH *ph, const H266RawSliceHeader *slice, + void *log_ctx) +{ + int poc_msb, max_poc_lsb, poc_lsb; + AuDetector *d = &s->au_detector; + max_poc_lsb = 1 << (sps->sps_log2_max_pic_order_cnt_lsb_minus4 + 4); + poc_lsb = ph->ph_pic_order_cnt_lsb; + if (IS_IDR(slice->nal_unit_header.nal_unit_type)) { + if (ph->ph_poc_msb_cycle_present_flag) + poc_msb = ph->ph_poc_msb_cycle_val * max_poc_lsb; + else + poc_msb = 0; + } else { + int prev_poc = d->prev_tid0_poc; + int prev_poc_lsb = prev_poc & (max_poc_lsb - 1); + int prev_poc_msb = prev_poc - prev_poc_lsb; + if (ph->ph_poc_msb_cycle_present_flag) { + poc_msb = ph->ph_poc_msb_cycle_val * max_poc_lsb; + } else { + if ((poc_lsb < prev_poc_lsb) && ((prev_poc_lsb - poc_lsb) >= (max_poc_lsb / 2))) + poc_msb = prev_poc_msb + max_poc_lsb; + else if ((poc_lsb > prev_poc_lsb) && ((poc_lsb - prev_poc_lsb) > (max_poc_lsb / 2))) + poc_msb = prev_poc_msb - max_poc_lsb; + else + poc_msb = prev_poc_msb; + } + } + + *poc = poc_msb + poc_lsb; +} + +static void au_detector_init(AuDetector *d) +{ + d->prev_layer_id = UINT8_MAX; + d->prev_poc = INT_MAX; + d->prev_tid0_poc = INT_MAX; +} + +static int is_au_start(VVCParserContext *s, const PuInfo *pu, void *log_ctx) +{ + //7.4.2.4.3 + AuDetector *d = &s->au_detector; + const H266RawSPS *sps = pu->sps; + const H266RawNALUnitHeader *nal = &pu->slice->header.nal_unit_header; + const H266RawPH *ph = pu->ph; + const H266RawSlice *slice = pu->slice; + int ret, poc, nut; + + get_slice_poc(s, &poc, sps, ph, &slice->header, log_ctx); + + ret = (nal->nuh_layer_id <= d->prev_layer_id) || (poc != d->prev_poc); + + nut = nal->nal_unit_type; + d->prev_layer_id = nal->nuh_layer_id; + d->prev_poc = poc; + if (nal->nuh_temporal_id_plus1 == 1 && + !ph->ph_non_ref_pic_flag && nut != VVC_RADL_NUT && nut != VVC_RASL_NUT) { + d->prev_tid0_poc = poc; + } + return ret; +} + +static int get_pu_info(PuInfo *info, const CodedBitstreamH266Context *h266, + const CodedBitstreamFragment *pu, void *logctx) +{ + const H266RawNALUnitHeader *nal; + int ret; + + memset(info, 0, sizeof(*info)); + for (int i = 0; i < pu->nb_units; i++) { + nal = pu->units[i].content; + if (!nal) + continue; + if (IS_PH(nal->nal_unit_type)) { + info->ph = pu->units[i].content; + info->ph_ref = pu->units[i].content_ref; + } else if (IS_SLICE(nal->nal_unit_type)) { + info->slice = pu->units[i].content; + info->slice_ref = pu->units[i].content_ref; + if (info->slice->header.sh_picture_header_in_slice_header_flag) + info->ph = &info->slice->header.sh_picture_header; + if (!info->ph) { + av_log(logctx, AV_LOG_ERROR, + "can't find picture header in picture unit.\n"); + ret = AVERROR_INVALIDDATA; + goto error; + } + break; + } + } + if (!info->slice) { + av_log(logctx, AV_LOG_ERROR, + "can't find slice in picture unit.\n"); + ret = AVERROR_INVALIDDATA; + goto error; + } + info->pps = h266->pps[info->ph->ph_pic_parameter_set_id]; + if (!info->pps) { + av_log(logctx, AV_LOG_ERROR, "PPS id %d is not avaliable.\n", + info->ph->ph_pic_parameter_set_id); + ret = AVERROR_INVALIDDATA; + goto error; + } + info->pps_ref = h266->pps_ref[info->ph->ph_pic_parameter_set_id]; + info->sps = h266->sps[info->pps->pps_seq_parameter_set_id]; + if (!info->sps) { + av_log(logctx, AV_LOG_ERROR, "SPS id %d is not avaliable.\n", + info->pps->pps_seq_parameter_set_id); + ret = AVERROR_INVALIDDATA; + goto error; + } + info->sps_ref = h266->sps_ref[info->pps->pps_seq_parameter_set_id]; + info->pic_type = get_pict_type(pu); + return 0; +error: + memset(info, 0, sizeof(*info)); + return ret; +} + +static int append_au(AVPacket *pkt, const uint8_t *buf, int buf_size) +{ + int offset = pkt->size; + int ret; + if ((ret = av_grow_packet(pkt, buf_size)) < 0) + goto end; + memcpy(pkt->data + offset, buf, buf_size); +end: + return ret; +} + +/** + * Parse NAL units of found picture and decode some basic information. + * + * @param ctx parser context. + * @param avctx codec context. + * @param buf buffer with field/frame data. + * @param buf_size size of the buffer. + * @return < 0 for error, == 0 for a complete au, > 0 is not a completed au. + */ +static int parse_nal_units(AVCodecParserContext *ctx, const uint8_t *buf, + int buf_size, AVCodecContext *avctx) +{ + VVCParserContext *s = ctx->priv_data; + const CodedBitstreamH266Context *h266 = s->cbc->priv_data; + + CodedBitstreamFragment *pu = &s->picture_unit; + int ret; + PuInfo info; + + if (!buf_size) { + if (s->au.size) { + if ((ret = av_packet_ref(&s->last_au, &s->au)) < 0) + goto end; + av_packet_unref(&s->au); + return 0; + } + return 1; + } + + if ((ret = ff_cbs_read(s->cbc, pu, buf, buf_size))< 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to parse picture unit.\n"); + goto end; + } + if ((ret = get_pu_info(&info, h266, pu, avctx)) < 0) + goto end; + if (is_au_start(s, &info, avctx)) { + if ((ret = set_ctx(ctx, avctx, &info)) < 0) + goto end; + if ((ret = av_packet_ref(&s->last_au, &s->au)) < 0) + goto end; + av_packet_unref(&s->au); + } else { + //not a completed au + ret = 1; + } + if (append_au(&s->au, buf, buf_size) < 0) + ret = AVERROR(ENOMEM); +end: + ff_cbs_fragment_reset(pu); + return ret; +} + +/** + * Combine PU to AU + * + * @param ctx parser context. + * @param avctx codec context. + * @param buf buffer to a PU. + * @param buf_size size of the buffer. + * @return < 0 for error, == 0 a complete au, > 0 not a completed au. + */ + +static int combine_au(AVCodecParserContext *ctx, AVCodecContext *avctx, + const uint8_t **buf, int *buf_size) +{ + VVCParserContext *s = ctx->priv_data; + CodedBitstreamFragment *pu = &s->picture_unit; + int ret; + + s->cbc->log_ctx = avctx; + + if (avctx->extradata_size && !s->parsed_extradata) { + s->parsed_extradata = 1; + + if ((ret = ff_cbs_read(s->cbc, pu, avctx->extradata, avctx->extradata_size)) < 0) + av_log(avctx, AV_LOG_WARNING, "Failed to parse extradata.\n"); + + ff_cbs_fragment_reset(pu); + } + av_packet_unref(&s->last_au); + ret = parse_nal_units(ctx, *buf, *buf_size, avctx); + if (ret == 0) { + if (s->last_au.size) { + *buf = s->last_au.data; + *buf_size = s->last_au.size; + } else { + //no output + ret = 1; + } + } + s->cbc->log_ctx = NULL; + return ret; +} + +static int vvc_parser_parse(AVCodecParserContext *ctx, AVCodecContext *avctx, + const uint8_t **poutbuf, int *poutbuf_size, + const uint8_t *buf, int buf_size) +{ + int next; + VVCParserContext *s = ctx->priv_data; + ParseContext *pc = &s->pc; + + if (avctx->extradata && !s->parsed_extradata) { + av_log(avctx, AV_LOG_INFO, "extra data is not supported yet.\n"); + return AVERROR_PATCHWELCOME; + } + + if (ctx->flags & PARSER_FLAG_COMPLETE_FRAMES) { + next = buf_size; + } else { + int ret, flush = !buf_size; + next = find_frame_end(ctx, buf, buf_size); + if (ff_combine_frame(pc, next, &buf, &buf_size) < 0) + goto no_out; + ret = combine_au(ctx, avctx, &buf, &buf_size); + if (ret > 0 && flush) { + buf_size = 0; + ret = combine_au(ctx, avctx, &buf, &buf_size); + } + if (ret != 0) { + buf_size = next; + goto no_out; + } + } + *poutbuf = buf; + *poutbuf_size = buf_size; + return next; +no_out: + *poutbuf = NULL; + *poutbuf_size = 0; + return buf_size; +} + +static const CodedBitstreamUnitType decompose_unit_types[] = { + VVC_TRAIL_NUT, + VVC_STSA_NUT, + VVC_RADL_NUT, + VVC_RASL_NUT, + VVC_IDR_W_RADL, + VVC_IDR_N_LP, + VVC_CRA_NUT, + VVC_GDR_NUT, + VVC_VPS_NUT, + VVC_SPS_NUT, + VVC_PPS_NUT, + VVC_PH_NUT, + VVC_AUD_NUT, +}; + +static av_cold int vvc_parser_init(AVCodecParserContext *ctx) +{ + VVCParserContext *s = ctx->priv_data; + int ret; + + ret = ff_cbs_init(&s->cbc, AV_CODEC_ID_VVC, NULL); + if (ret < 0) + return ret; + au_detector_init(&s->au_detector); + + s->cbc->decompose_unit_types = decompose_unit_types; + s->cbc->nb_decompose_unit_types = FF_ARRAY_ELEMS(decompose_unit_types); + + return ret; +} + +static void vvc_parser_close(AVCodecParserContext *ctx) +{ + VVCParserContext *s = ctx->priv_data; + + pu_info_unref(&s->au_info); + av_packet_unref(&s->au); + av_packet_unref(&s->last_au); + ff_cbs_fragment_free(&s->picture_unit); + + ff_cbs_close(&s->cbc); + av_freep(&s->pc.buffer); +} + +AVCodecParser ff_vvc_parser = { + .codec_ids = { AV_CODEC_ID_VVC }, + .priv_data_size = sizeof(VVCParserContext), + .parser_init = vvc_parser_init, + .parser_close = vvc_parser_close, + .parser_parse = vvc_parser_parse, +}; From patchwork Wed Oct 19 07:25:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Siedel X-Patchwork-Id: 38814 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp215906pzb; Wed, 19 Oct 2022 00:27:27 -0700 (PDT) X-Google-Smtp-Source: AMsMyM72iCYWii4laUk8hwr0Zh5MPWt3UkrWZ3GIByOdDFGLpPfFiGYtQjH+42DwH6jc979kAM3V X-Received: by 2002:a05:6402:450c:b0:443:6279:774f with SMTP id ez12-20020a056402450c00b004436279774fmr6203801edb.11.1666164447171; Wed, 19 Oct 2022 00:27:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666164447; cv=none; d=google.com; s=arc-20160816; b=VrkYZmSe7LxgMVCYYd8OvbrA7AbGdeeC3FN2SQ1aFGwCgwtVer5T0UT8GS8JUw/ZE1 jPkXKLdErQw24hCHUKClPjWr2YPg8s6qWNYhhd9aGHVx8++SzZOZOEzmxcrKTXCTyEKp 00R4FgFWtVcss0lE411HamAywGZ+Ar02FrVl0vM95t+k5g17nbwAruqLVX3aCY5LGkWE MWORceTtG85VZ/KcPeEaJhgGEJSoQa1FyIRDNK9ASxmsZQ0MXVjiainnfKdOCEudG5yB xmugDpqdSxD+TH8mgGdSUlKSj9pVaN1+tmSZQAkLLk4k6yXL0i3QTqRZl3qtRNLeVCN7 jJ9w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=wH1uG+/iXcKi3fC/qL1kIGpcQwFYDmIGvuZOIbT16yE=; b=qFs/YNtKHwuxNUZHfLWLQ7L0ze3zYROdX3Lk1X3EYHGZEHCZdFAZuW4bKVu1QZ74I9 tJ3AQkz8z16snls2q6wk3vAdlUAh84WEX7z1tK3uZe3izkDzSBI1sjNl3ncPhuim+OhQ k1fAlqGXh1vsYgw52fvt896C33J9dBxdTIOQXLFfxUhlslKYt6sOXSMeo8d/QENPMAYa d3MH68JKU9d3soLQaEr54o7N9v8h/7ErSo7vDcpcdRlLJcoI4AEtr77wiaKj/bzo//nL BhmpECHsmEd/dakNpuNRWoGJOGnUh43iRh/PH1ihlWoxi/tQIYVoyh+UhGaBQcpIYbqc x5Bg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=cBDUklPM; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id r5-20020a50d685000000b004592497f96asi12942648edi.125.2022.10.19.00.27.26; Wed, 19 Oct 2022 00:27:27 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=cBDUklPM; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2134868BD49; Wed, 19 Oct 2022 10:26:37 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f51.google.com (mail-ed1-f51.google.com [209.85.208.51]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 48BA268BDD5 for ; Wed, 19 Oct 2022 10:26:29 +0300 (EEST) Received: by mail-ed1-f51.google.com with SMTP id u21so23878019edi.9 for ; Wed, 19 Oct 2022 00:26:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spin-digital-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8BsMW+iBZNt6M5NUtsbpOqdH+cV6vwz5FQSaMtm7W2M=; b=cBDUklPMZZjkBowaixnsRkhK/yJCzmQ1fq1VQsDtJVp4sN6tl1lApCiPk2qidPup+r HJNt9SXOaUN+wMi2z0Mc3CYnDNr6RIc1K6Y8Elkx1ef7x5E5CVlGBPCvpNROcssd9kWC 88U7E1o00Ef7f7G77vCv1RuHby99+TVQSqG/HLJHPTyDb4WuYNvU7GQ8DMU1CtQAVKZE 5DjUTgLOb9tydSs1J9HXVJmZrresJWntBCXNK6zCjnz9OvubO2CuVcrRSu1bN2IxW3Px M2b7Esfdxgd3YXJlcJb0HUgOxKUnIALnzuqzyMAYyi+I5qvHtib9vr8TSj70Qh080z1o 9djQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8BsMW+iBZNt6M5NUtsbpOqdH+cV6vwz5FQSaMtm7W2M=; b=IN9hQFWcbbjo3+vcofAnjI7xSU/VsL92vOozk6E4VlG7wwP2KHUE6c7hCWtU3RKmFb ZyIchpHFLCbHdF2FwHWCkuuiOIbFnST6vFn+JCqR55TbQaOyTcRRHEXudRMIkqUykQaG F1PEAT9+22ceFx9pEAOHc6CC4qXpCHCW1as3qmUKMw6KLgaCmDvOZSQnavLvOr0CxcV9 OCuKjOhaYcwf2z3IqoyXQ5tq2+WYs9g9iPRqjZxXiO+h9uFrkQPSGBNzwYTE4y471Fnn PEBM/TkTXgGNdmpmic9/ntoy8NBn3HyK8zJz+dsUMZH1f43iSpACbtNDyZS5zR2pO4Vr ADsg== X-Gm-Message-State: ACrzQf1hQiyWZysTKIPwacJcRsc9GA9PJ0GnfXDLLhQafc13droks2j0 73xGwHhrQbcdISgnDxLjalsJTdYHDSi7cA== X-Received: by 2002:a05:6402:5024:b0:440:e4ad:f7b6 with SMTP id p36-20020a056402502400b00440e4adf7b6mr6186415eda.358.1666164388180; Wed, 19 Oct 2022 00:26:28 -0700 (PDT) Received: from thomas-win.localdomain ([213.138.44.237]) by smtp.gmail.com with ESMTPSA id p5-20020a170906498500b0078b83968ad4sm8515150eju.24.2022.10.19.00.26.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Oct 2022 00:26:27 -0700 (PDT) From: thomas.ff@spin-digital.com To: ffmpeg-devel@ffmpeg.org Date: Wed, 19 Oct 2022 09:25:02 +0200 Message-Id: <20221019072508.23460-6-thomas.ff@spin-digital.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221019072508.23460-1-thomas.ff@spin-digital.com> References: <20221019072508.23460-1-thomas.ff@spin-digital.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v1 05/11] avcodec: add MP4 to annexb support for H266/VVC X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Thomas Siedel Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: GJV7Z1lDsNiG From: Thomas Siedel Add parser for VVC MP4 to Annex B byte stream format. Signed-off-by: Thomas Siedel --- configure | 1 + libavcodec/Makefile | 2 + libavcodec/bitstream_filters.c | 2 + libavcodec/h266_metadata_bsf.c | 145 ++++++++++++++ libavcodec/vvc_mp4toannexb_bsf.c | 318 +++++++++++++++++++++++++++++++ 5 files changed, 468 insertions(+) create mode 100644 libavcodec/h266_metadata_bsf.c create mode 100644 libavcodec/vvc_mp4toannexb_bsf.c diff --git a/configure b/configure index 13dca5cd9a..30df2cb0a4 100755 --- a/configure +++ b/configure @@ -3298,6 +3298,7 @@ mjpeg2jpeg_bsf_select="jpegtables" mpeg2_metadata_bsf_select="cbs_mpeg2" trace_headers_bsf_select="cbs" vp9_metadata_bsf_select="cbs_vp9" +vvc_metadata_bsf_select="cbs_h266" # external libraries aac_at_decoder_deps="audiotoolbox" diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 3850e316ac..53a8dc67cb 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1222,6 +1222,8 @@ OBJS-$(CONFIG_VP9_METADATA_BSF) += vp9_metadata_bsf.o OBJS-$(CONFIG_VP9_RAW_REORDER_BSF) += vp9_raw_reorder_bsf.o OBJS-$(CONFIG_VP9_SUPERFRAME_BSF) += vp9_superframe_bsf.o OBJS-$(CONFIG_VP9_SUPERFRAME_SPLIT_BSF) += vp9_superframe_split_bsf.o +OBJS-$(CONFIG_VVC_METADATA_BSF) += h266_metadata_bsf.o +OBJS-$(CONFIG_VVC_MP4TOANNEXB_BSF) += vvc_mp4toannexb_bsf.o # thread libraries OBJS-$(HAVE_LIBC_MSVCRT) += file_open.o diff --git a/libavcodec/bitstream_filters.c b/libavcodec/bitstream_filters.c index a3bebefe5f..403884f3d7 100644 --- a/libavcodec/bitstream_filters.c +++ b/libavcodec/bitstream_filters.c @@ -64,6 +64,8 @@ extern const FFBitStreamFilter ff_vp9_metadata_bsf; extern const FFBitStreamFilter ff_vp9_raw_reorder_bsf; extern const FFBitStreamFilter ff_vp9_superframe_bsf; extern const FFBitStreamFilter ff_vp9_superframe_split_bsf; +extern const FFBitStreamFilter ff_vvc_mp4toannexb_bsf; +extern const FFBitStreamFilter ff_vvc_metadata_bsf; #include "libavcodec/bsf_list.c" diff --git a/libavcodec/h266_metadata_bsf.c b/libavcodec/h266_metadata_bsf.c new file mode 100644 index 0000000000..bc3c10f581 --- /dev/null +++ b/libavcodec/h266_metadata_bsf.c @@ -0,0 +1,145 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/common.h" +#include "libavutil/opt.h" + +#include "bsf.h" +#include "bsf_internal.h" +#include "cbs.h" +#include "cbs_bsf.h" +#include "cbs_h266.h" +#include "vvc.h" + +#define IS_SLICE(nut) (nut <= VVC_RASL_NUT || (nut >= VVC_IDR_W_RADL && nut <= VVC_GDR_NUT)) +#define IS_PH(nut) (nut == VVC_PH_NUT) + +typedef struct VVCMetadataContext { + CBSBSFContext common; + + H266RawAUD aud_nal; + + int aud; +} VVCMetadataContext; + +static int h266_metadata_update_fragment(AVBSFContext *bsf, AVPacket *pkt, + CodedBitstreamFragment *pu) +{ + VVCMetadataContext *ctx = bsf->priv_data; + int err, i; + + // If an AUD is present, it must be the first NAL unit. + if (pu->units[0].type == VVC_AUD_NUT) { + if (ctx->aud == BSF_ELEMENT_REMOVE) + ff_cbs_delete_unit(pu, 0); + } else { + if (ctx->aud == BSF_ELEMENT_INSERT) { + const H266RawSlice *first_slice = NULL; + const H266RawPH *ph = NULL; + H266RawAUD *aud = &ctx->aud_nal; + int pic_type = 0, temporal_id = 8, layer_id = 0; + for (i = 0; i < pu->nb_units; i++) { + const H266RawNALUnitHeader *nal = pu->units[i].content; + if (!nal) + continue; + if (nal->nuh_temporal_id_plus1 < temporal_id + 1) + temporal_id = nal->nuh_temporal_id_plus1 - 1; + if (IS_PH(nal->nal_unit_type)) { + ph = pu->units[i].content; + } else if(IS_SLICE(nal->nal_unit_type)) { + const H266RawSlice *slice = pu->units[i].content; + layer_id = nal->nuh_layer_id; + if (slice->header.sh_slice_type == VVC_SLICE_TYPE_B && + pic_type < 2) + pic_type = 2; + if (slice->header.sh_slice_type == VVC_SLICE_TYPE_P && + pic_type < 1) + pic_type = 1; + if (!first_slice) { + first_slice = slice; + if (first_slice->header.sh_picture_header_in_slice_header_flag) + ph = &first_slice->header.sh_picture_header; + else if (!ph) + break; + } + } + } + if (!ph) { + av_log(bsf, AV_LOG_ERROR, "no avaliable picture header"); + return AVERROR_INVALIDDATA; + } + + aud->nal_unit_header = (H266RawNALUnitHeader) { + .nal_unit_type = VVC_AUD_NUT, + .nuh_layer_id = layer_id, + .nuh_temporal_id_plus1 = temporal_id + 1, + }; + aud->aud_pic_type = pic_type; + aud->aud_irap_or_gdr_flag = ph->ph_gdr_or_irap_pic_flag; + + err = ff_cbs_insert_unit_content(pu, 0, VVC_AUD_NUT, aud, NULL); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to insert AUD.\n"); + return err; + } + } + } + return 0; +} + +static const CBSBSFType h266_metadata_type = { + .codec_id = AV_CODEC_ID_VVC, + .fragment_name = "access unit", + .unit_name = "NAL unit", + .update_fragment = &h266_metadata_update_fragment, +}; + +static int vvc_metadata_init(AVBSFContext *bsf) +{ + return ff_cbs_bsf_generic_init(bsf, &h266_metadata_type); +} + +#define OFFSET(x) offsetof(VVCMetadataContext, x) +#define FLAGS (AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_BSF_PARAM) +static const AVOption vvc_metadata_options[] = { + BSF_ELEMENT_OPTIONS_PIR("aud", "Access Unit Delimiter NAL units", + aud, FLAGS), + + { NULL } +}; + +static const AVClass vvc_metadata_class = { + .class_name = "vvc_metadata_bsf", + .item_name = av_default_item_name, + .option = vvc_metadata_options, + .version = LIBAVUTIL_VERSION_INT, +}; + +static const enum AVCodecID vvc_metadata_codec_ids[] = { + AV_CODEC_ID_VVC, AV_CODEC_ID_NONE, +}; + +const FFBitStreamFilter ff_vvc_metadata_bsf = { + .p.name = "vvc_metadata", + .p.codec_ids = vvc_metadata_codec_ids, + .p.priv_class = &vvc_metadata_class, + .priv_data_size = sizeof(VVCMetadataContext), + .init = &vvc_metadata_init, + .close = &ff_cbs_bsf_generic_close, + .filter = &ff_cbs_bsf_generic_filter, +}; diff --git a/libavcodec/vvc_mp4toannexb_bsf.c b/libavcodec/vvc_mp4toannexb_bsf.c new file mode 100644 index 0000000000..d6415dba7b --- /dev/null +++ b/libavcodec/vvc_mp4toannexb_bsf.c @@ -0,0 +1,318 @@ +/* + * VVC MP4 to Annex B byte stream format filter + * Copyright (c) 2022, Thomas Siedel + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include + +#include "libavutil/intreadwrite.h" +#include "libavutil/mem.h" + +#include "avcodec.h" +#include "bsf.h" +#include "bsf_internal.h" +#include "bytestream.h" +#include "vvc.h" + +#include "libavcodec/get_bits.h" + +#define MIN_VVCC_LENGTH 23 + +typedef struct VVCBSFContext { + uint8_t length_size; + int extradata_parsed; +} VVCBSFContext; + +static int vvc_extradata_to_annexb(AVBSFContext *ctx) +{ + GetByteContext gb; + int length_size, num_arrays, i, j; + int ret = 0; + int temp = 0; + int ptl_present; + + uint8_t *new_extradata = NULL; + size_t new_extradata_size = 0; + + int max_picture_width = 0; + int max_picture_height = 0; + int avg_frame_rate = 0; + + bytestream2_init(&gb, ctx->par_in->extradata, ctx->par_in->extradata_size); + temp = bytestream2_get_byte(&gb); + length_size = (( temp & 6) >> 1) + 1; + ptl_present = temp & 1; + if (ptl_present) { + int num_bytes_constraint_info; + int general_profile_idc; + int general_tier_flag; + int general_level_idc; + int ptl_frame_only_constraint_flag; + int ptl_multi_layer_enabled_flag; + int ptl_num_sub_profiles; + int temp3, temp4, temp5; + int temp2 = bytestream2_get_be16(&gb); + int ols_idx = (temp2 >> 7) & 0x1ff; + int num_sublayers = (temp2 >> 4) & 0x7; + int constant_frame_rate = (temp2 >> 2) & 0x3; + int chroma_format_idc = temp2 & 0x3; + int bit_depth_minus8 = (bytestream2_get_byte(&gb) >> 5) & 0x7; + av_log(ctx, AV_LOG_DEBUG, + "bit_depth_minus8 %d chroma_format_idc %d\n", bit_depth_minus8, chroma_format_idc); + av_log(ctx, AV_LOG_DEBUG, + "constant_frame_rate %d, ols_idx %d\n", constant_frame_rate, ols_idx); + // VvcPTLRecord(num_sublayers) native_ptl + temp3 = bytestream2_get_byte(&gb); + num_bytes_constraint_info = (temp3) & 0x3f; + temp4 = bytestream2_get_byte(&gb); + general_profile_idc = (temp4 >> 1) & 0x7f; + general_tier_flag = (temp4) & 1; + general_level_idc = bytestream2_get_byte(&gb); + av_log(ctx, AV_LOG_DEBUG, + "general_profile_idc %d, general_tier_flag %d, general_level_idc %d, num_sublayers %d num_bytes_constraint_info %d\n", general_profile_idc, general_tier_flag, general_level_idc, num_sublayers, num_bytes_constraint_info); + + temp5 = bytestream2_get_byte(&gb); + ptl_frame_only_constraint_flag = (temp5 >> 7) & 0x1; + ptl_multi_layer_enabled_flag = (temp5 >> 6) & 0x1; + for (i = 0; i < num_bytes_constraint_info-1; i++) { + // unsigned int(8*num_bytes_constraint_info - 2) general_constraint_info; + bytestream2_get_byte(&gb); + } + + av_log(ctx, AV_LOG_DEBUG, + "ptl_multi_layer_enabled_flag %d, ptl_frame_only_constraint_flag %d\n",ptl_multi_layer_enabled_flag, ptl_frame_only_constraint_flag); + + if( num_sublayers > 1 ) + { + int temp6 = bytestream2_get_byte(&gb); + uint8_t ptl_sublayer_level_present_flag[8] = {0}; + //uint8_t sublayer_level_idc[8] = {0}; + for (i=num_sublayers - 2; i >= 0; i--){ + ptl_sublayer_level_present_flag[i] = (temp6 >> (7 - (num_sublayers - 2 - i))) & 0x01; + } + // for (j=num_sublayers; j<=8 && num_sublayers > 1; j++) + // bit(1) ptl_reserved_zero_bit = 0; + for (i=num_sublayers-2; i >= 0; i--) + { + if (ptl_sublayer_level_present_flag[i]) { + //sublayer_level_idc[i] = bytestream2_get_byte(&gb); + } + } + } + + ptl_num_sub_profiles = bytestream2_get_byte(&gb); + for (j=0; j < ptl_num_sub_profiles; j++) { + // unsigned int(32) general_sub_profile_idc[j]; + bytestream2_get_be16(&gb); + bytestream2_get_be16(&gb); + } + + max_picture_width = bytestream2_get_be16(&gb); // unsigned_int(16) max_picture_width; + max_picture_height = bytestream2_get_be16(&gb); // unsigned_int(16) max_picture_height; + avg_frame_rate = bytestream2_get_be16(&gb); // unsigned int(16) avg_frame_rate; } + av_log(ctx, AV_LOG_DEBUG, + "max_picture_width %d, max_picture_height %d, avg_frame_rate %d\n", max_picture_width, max_picture_height, avg_frame_rate); + } + + num_arrays = bytestream2_get_byte(&gb); + + for (i = 0; i < num_arrays; i++) { + int cnt; + int type = bytestream2_get_byte(&gb) & 0x1f; + + if (type != VVC_OPI_NUT || type != VVC_DCI_NUT) + cnt = bytestream2_get_be16(&gb); + else + cnt = 1; + + av_log(ctx, AV_LOG_DEBUG, + "nalu_type %d cnt %d\n", type, cnt); + + if (!(type == VVC_VPS_NUT || type == VVC_SPS_NUT || type == VVC_PPS_NUT || + type == VVC_PREFIX_SEI_NUT || type == VVC_SUFFIX_SEI_NUT)) { + av_log(ctx, AV_LOG_ERROR, "Invalid NAL unit type in extradata: %d\n", type); + ret = AVERROR_INVALIDDATA; + goto fail; + } + + for (j = 0; j < cnt; j++) { + int nalu_len = bytestream2_get_be16(&gb); + + if (4 + AV_INPUT_BUFFER_PADDING_SIZE + nalu_len > SIZE_MAX - new_extradata_size) { + ret = AVERROR_INVALIDDATA; + goto fail; + } + ret = av_reallocp(&new_extradata, new_extradata_size + nalu_len + 4 + AV_INPUT_BUFFER_PADDING_SIZE); + if (ret < 0) + goto fail; + + AV_WB32(new_extradata + new_extradata_size, 1); // add the startcode + bytestream2_get_buffer(&gb, new_extradata + new_extradata_size + 4, nalu_len); + new_extradata_size += 4 + nalu_len; + memset(new_extradata + new_extradata_size, 0, AV_INPUT_BUFFER_PADDING_SIZE); + } + } + + av_freep(&ctx->par_out->extradata); + ctx->par_out->extradata = new_extradata; + ctx->par_out->extradata_size = new_extradata_size; + + if (!new_extradata_size) + av_log(ctx, AV_LOG_WARNING, "No parameter sets in the extradata\n"); + + return length_size; +fail: + av_freep(&new_extradata); + return ret; +} + +static int vvc_mp4toannexb_init(AVBSFContext *ctx) +{ + VVCBSFContext *s = ctx->priv_data; + int ret; + + if (ctx->par_in->extradata_size < MIN_VVCC_LENGTH || + AV_RB24(ctx->par_in->extradata) == 1 || + AV_RB32(ctx->par_in->extradata) == 1) { + av_log(ctx, AV_LOG_VERBOSE, "The input looks like it is Annex B already\n"); + } else { + ret = vvc_extradata_to_annexb(ctx); + if (ret < 0) + return ret; + s->length_size = ret; + s->extradata_parsed = 1; + } + + return 0; +} + +static int vvc_mp4toannexb_filter(AVBSFContext *ctx, AVPacket *out) +{ + VVCBSFContext *s = ctx->priv_data; + AVPacket *in; + GetByteContext gb; + + int is_irap = 0; + int added_extra = 0; + int i, ret = 0; + + ret = ff_bsf_get_packet(ctx, &in); + if (ret < 0) + return ret; + + if (!s->extradata_parsed) { + av_packet_move_ref(out, in); + av_packet_free(&in); + return 0; + } + + bytestream2_init(&gb, in->data, in->size); + + /* check if this packet contains an IRAP. The extradata will need to be added before any potential PH_NUT */ + while (bytestream2_get_bytes_left(&gb)) { + uint32_t nalu_size = 0; + int nalu_type; + + if (bytestream2_get_bytes_left(&gb) < s->length_size) { + ret = AVERROR_INVALIDDATA; + goto fail; + } + + for (i = 0; i < s->length_size; i++) + nalu_size = (nalu_size << 8) | bytestream2_get_byte(&gb); + + if (nalu_size < 2 || nalu_size > bytestream2_get_bytes_left(&gb)) { + ret = AVERROR_INVALIDDATA; + goto fail; + } + + nalu_type = (bytestream2_peek_be16(&gb) >> 3) & 0x1f; + is_irap = nalu_type >= VVC_IDR_W_RADL && nalu_type <= VVC_RSV_IRAP_11; + if (is_irap) { + break; + } + bytestream2_seek(&gb, nalu_size, SEEK_CUR); + } + + bytestream2_seek(&gb, 0, SEEK_SET); + while (bytestream2_get_bytes_left(&gb)) { + uint32_t nalu_size = 0; + int nalu_type; + int add_extradata, extra_size, prev_size; + + if (bytestream2_get_bytes_left(&gb) < s->length_size) { + ret = AVERROR_INVALIDDATA; + goto fail; + } + + for (i = 0; i < s->length_size; i++) + nalu_size = (nalu_size << 8) | bytestream2_get_byte(&gb); + + if (nalu_size < 2 || nalu_size > bytestream2_get_bytes_left(&gb)) { + ret = AVERROR_INVALIDDATA; + goto fail; + } + + nalu_type = (bytestream2_peek_be16(&gb) >> 3) & 0x1f; + + /* prepend extradata to IRAP frames */ + add_extradata = is_irap && nalu_type != VVC_AUD_NUT && !added_extra; + extra_size = add_extradata * ctx->par_out->extradata_size; + added_extra |= add_extradata; + + if (FFMIN(INT_MAX, SIZE_MAX) < 4ULL + nalu_size + extra_size) { + ret = AVERROR_INVALIDDATA; + goto fail; + } + + prev_size = out->size; + + ret = av_grow_packet(out, 4 + nalu_size + extra_size); + if (ret < 0) + goto fail; + + if (extra_size) + memcpy(out->data + prev_size, ctx->par_out->extradata, extra_size); + AV_WB32(out->data + prev_size + extra_size, 1); + bytestream2_get_buffer(&gb, out->data + prev_size + 4 + extra_size, nalu_size); + } + + ret = av_packet_copy_props(out, in); + if (ret < 0) + goto fail; + +fail: + if (ret < 0) + av_packet_unref(out); + av_packet_free(&in); + + return ret; +} + +static const enum AVCodecID codec_ids[] = { + AV_CODEC_ID_VVC, AV_CODEC_ID_NONE, +}; + +const FFBitStreamFilter ff_vvc_mp4toannexb_bsf = { + .p.name = "vvc_mp4toannexb", + .p.codec_ids = codec_ids, + .priv_data_size = sizeof(VVCBSFContext), + .init = vvc_mp4toannexb_init, + .filter = vvc_mp4toannexb_filter, +}; From patchwork Wed Oct 19 07:25:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thomas Siedel X-Patchwork-Id: 38817 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp216033pzb; Wed, 19 Oct 2022 00:27:55 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7i2ZR5rvskaJ/Rv8z5nK7BSsay8kIiZJBX0Tx63tm9JqrtX/sQAadIpk0nAXtw3qB6CLPx X-Received: by 2002:a17:907:97c2:b0:78d:accc:c0a9 with SMTP id js2-20020a17090797c200b0078dacccc0a9mr5450920ejc.312.1666164475364; Wed, 19 Oct 2022 00:27:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666164475; cv=none; d=google.com; s=arc-20160816; b=vAYbfrW8Pra/ZuggNJ9HhB1ho4c/srDW4GvtPBZNrO8xGvjx5br1J1xopjhmUjiw8K 0MjEM+2hDpWTDM6sNg6jj38X5tIEGji6359ApvRkObEZslzw2NsylhWwxtFlxbql/nRF oBK9CjvWbKL13Eep5Af0JD82yZxMlSjGrbXqTVevo0DXZc/Z6UDrqciqWLmEfka4t8as XxMpL13xRZVSkSaQ8u+uy4KBxe0ffUdcECflFKzYZTp4Y9w7kIlwHdjKejARVLq92Jl/ BWpG2AtIkn25rJBHouzO33iU1YOX/Y6hYU16xQx0GHfdjxnkdUpUyesX3nkvfqWdJx1D 3r0g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=grayBjtOGslz+pUW56BTv839p8K374W+d7zVwU8F8e8=; b=b4LIvRJRvIcBm0l3FhllBRky/D+OeoD9oi59Fr8hUsiJHppb1fdHZdGQX4W+z7SHbm 173UUkdC2HCmrXkq36IGdcMk5wgmmRTMUYP3tXmjxAI0GFuTSXN+iXfLEMHmwkJi5Tu4 NwAmO/pZ7eyah4d//3zJQB3zGjsQeO2FGzCZSjlX9Aas2ES0wfBlilbIzeNu6NLRKuNN heeXdGVwgqznnc/7yibXVjVRKm95j9tMnBYuQo2n47qCYwuf6+Kn0/x4nLp/SaAr5sg7 y5t3u8r9DjK+DbrK2XbvWAZ78VSgVEqTUO6R6BBT1037UmpO9se/n3VAVRlL5mL2xGFL 5u2g== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=spZHrFHC; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id h25-20020a056402095900b0045901fa87a6si11608603edz.449.2022.10.19.00.27.54; Wed, 19 Oct 2022 00:27:55 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=spZHrFHC; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1957D68BDFB; Wed, 19 Oct 2022 10:26:40 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f54.google.com (mail-ej1-f54.google.com [209.85.218.54]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 889A968BDDC for ; Wed, 19 Oct 2022 10:26:30 +0300 (EEST) Received: by mail-ej1-f54.google.com with SMTP id d26so37791822eje.10 for ; Wed, 19 Oct 2022 00:26:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spin-digital-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8vM6w6jtL0M8mszX/bMYSqQTo65+cfcN3YxFJnWgqPo=; b=spZHrFHC2AaKK3aAlbFo06zlB6f3Tw6X2O/q/wQWMkFemB3GZdnqisgx9too8XC0/0 e1SGPmZ+AqGZSmpGuz49vIyqX53ToaBLz0uXslmkDedwo23YZVja1B0Fw92eRWfkBIzT xsxzmrX6m4jt98stuL+3mOYKFjWU2A/oS/L1a2q69A8Z3JT+PwrROJlwHOF0XTJ1licZ 9RKO96b/wY5DyIavp40SJSiBUCq5xvMfnY3DXwzbKGCVqsivAxwpSlkUwceIXwh7Fef1 aA3QSBtWrCr9v3OOsP01TmhCw4G89tDpyYDR7cQDILAmQJmP9xeXPD6GrZFw5AK4h24+ O0FQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8vM6w6jtL0M8mszX/bMYSqQTo65+cfcN3YxFJnWgqPo=; b=YsxkZ69c9zb9u94hSgAVgWUokc3L1jjSZOd+Urg2F9yCCl9KTgZBoYG+wtMt27rnvb sa6XlRLNcizE774zU1K4sCKCk8zw/vHErCVX9tatDmS9kTy4KHbAtcQVH6EPUy3hbgV5 5e12QnPHqJUn1M4QdkN6xhcfUTPv3huAdyMHxelUaX2cnD8Yp50hwLj/zn3+q3UfDN37 bZDI19EEY5tuV4B54UfzI6J0H81mcMqyCVzcXBh3D6IS9Kfd/C7VMO4BeOWrW4W5Ve29 qE42AmvW+BJXwvnWfZ+AqETNvvh73hYaje48qGTTxXEY1+JDb2xhiVry13NcUyqqRTcK PEdQ== X-Gm-Message-State: ACrzQf2L4I6ZIphl6ucJbQ0dJvl/XqkRy8Py6FtJL/5fmTYfIUHR/rD/ Bup6FYBJo4s8V2LsqChN4lY2ssZLrmfuhw== X-Received: by 2002:a17:907:2d89:b0:78d:9c30:452b with SMTP id gt9-20020a1709072d8900b0078d9c30452bmr5184824ejc.533.1666164389122; Wed, 19 Oct 2022 00:26:29 -0700 (PDT) Received: from thomas-win.localdomain ([213.138.44.237]) by smtp.gmail.com with ESMTPSA id p5-20020a170906498500b0078b83968ad4sm8515150eju.24.2022.10.19.00.26.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Oct 2022 00:26:28 -0700 (PDT) From: thomas.ff@spin-digital.com To: ffmpeg-devel@ffmpeg.org Date: Wed, 19 Oct 2022 09:25:03 +0200 Message-Id: <20221019072508.23460-7-thomas.ff@spin-digital.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221019072508.23460-1-thomas.ff@spin-digital.com> References: <20221019072508.23460-1-thomas.ff@spin-digital.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v1 06/11] avformat: add demuxer and probe support for H266/VVC X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Thomas Siedel Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: UQBXXxvpHFFL From: Thomas Siedel Add demuxer to probe raw vvc and parse vvcc byte stream format. Signed-off-by: Thomas Siedel --- libavformat/Makefile | 1 + libavformat/allformats.c | 1 + libavformat/demux.c | 7 +- libavformat/vvc.c | 918 +++++++++++++++++++++++++++++++++++++++ libavformat/vvc.h | 99 +++++ libavformat/vvcdec.c | 61 +++ 6 files changed, 1085 insertions(+), 2 deletions(-) create mode 100644 libavformat/vvc.c create mode 100644 libavformat/vvc.h create mode 100644 libavformat/vvcdec.c diff --git a/libavformat/Makefile b/libavformat/Makefile index d7f198bf39..00ab4ded89 100644 --- a/libavformat/Makefile +++ b/libavformat/Makefile @@ -595,6 +595,7 @@ OBJS-$(CONFIG_VOC_MUXER) += vocenc.o voc.o OBJS-$(CONFIG_VPK_DEMUXER) += vpk.o OBJS-$(CONFIG_VPLAYER_DEMUXER) += vplayerdec.o subtitles.o OBJS-$(CONFIG_VQF_DEMUXER) += vqf.o +OBJS-$(CONFIG_VVC_DEMUXER) += vvcdec.o rawdec.o OBJS-$(CONFIG_W64_DEMUXER) += wavdec.o w64.o pcm.o OBJS-$(CONFIG_W64_MUXER) += wavenc.o w64.o OBJS-$(CONFIG_WAV_DEMUXER) += wavdec.o pcm.o diff --git a/libavformat/allformats.c b/libavformat/allformats.c index 47c419a009..a4e3822681 100644 --- a/libavformat/allformats.c +++ b/libavformat/allformats.c @@ -474,6 +474,7 @@ extern const AVOutputFormat ff_voc_muxer; extern const AVInputFormat ff_vpk_demuxer; extern const AVInputFormat ff_vplayer_demuxer; extern const AVInputFormat ff_vqf_demuxer; +extern const AVInputFormat ff_vvc_demuxer; extern const AVInputFormat ff_w64_demuxer; extern const AVOutputFormat ff_w64_muxer; extern const AVInputFormat ff_wav_demuxer; diff --git a/libavformat/demux.c b/libavformat/demux.c index 2dfd82a63c..8dbde23fcd 100644 --- a/libavformat/demux.c +++ b/libavformat/demux.c @@ -120,6 +120,7 @@ static int set_codec_from_probe_data(AVFormatContext *s, AVStream *st, { "mp3", AV_CODEC_ID_MP3, AVMEDIA_TYPE_AUDIO }, { "mpegvideo", AV_CODEC_ID_MPEG2VIDEO, AVMEDIA_TYPE_VIDEO }, { "truehd", AV_CODEC_ID_TRUEHD, AVMEDIA_TYPE_AUDIO }, + { "vvc", AV_CODEC_ID_VVC, AVMEDIA_TYPE_VIDEO }, { 0 } }; int score; @@ -743,7 +744,8 @@ static int64_t select_from_pts_buffer(AVStream *st, int64_t *pts_buffer, int64_t { FFStream *const sti = ffstream(st); int onein_oneout = st->codecpar->codec_id != AV_CODEC_ID_H264 && - st->codecpar->codec_id != AV_CODEC_ID_HEVC; + st->codecpar->codec_id != AV_CODEC_ID_HEVC && + st->codecpar->codec_id != AV_CODEC_ID_VVC; if (!onein_oneout) { int delay = sti->avctx->has_b_frames; @@ -933,7 +935,8 @@ static void compute_pkt_fields(AVFormatContext *s, AVStream *st, int64_t offset; AVRational duration; int onein_oneout = st->codecpar->codec_id != AV_CODEC_ID_H264 && - st->codecpar->codec_id != AV_CODEC_ID_HEVC; + st->codecpar->codec_id != AV_CODEC_ID_HEVC && + st->codecpar->codec_id != AV_CODEC_ID_VVC; if (s->flags & AVFMT_FLAG_NOFILLIN) return; diff --git a/libavformat/vvc.c b/libavformat/vvc.c new file mode 100644 index 0000000000..fd0527242f --- /dev/null +++ b/libavformat/vvc.c @@ -0,0 +1,918 @@ +/* + * VVC helper functions for muxers + * + * Copyright (C) 2022, Thomas Siedel + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavcodec/get_bits.h" +#include "libavcodec/golomb.h" +#include "libavcodec/vvc.h" +#include "libavutil/intreadwrite.h" +#include "avc.h" +#include "avio.h" +#include "avio_internal.h" +#include "vvc.h" + +typedef struct VVCCNALUnitArray { + uint8_t array_completeness; + uint8_t NAL_unit_type; + uint16_t num_nalus; + uint16_t *nal_unit_length; + uint8_t **nal_unit; +} VVCCNALUnitArray; + +typedef struct VVCPTLRecord { + uint8_t num_bytes_constraint_info; + uint8_t general_profile_idc; + uint8_t general_tier_flag; + uint8_t general_level_idc; + uint8_t ptl_frame_only_constraint_flag; + uint8_t ptl_multilayer_enabled_flag; + uint8_t *general_constraint_info; + uint8_t *ptl_sublayer_level_present_flag; + uint8_t *sublayer_level_idc; + uint8_t ptl_num_sub_profiles; + uint32_t *general_sub_profile_idc; +} VVCPTLRecord; + +typedef struct VVCDecoderConfigurationRecord { + uint8_t lengthSizeMinusOne; + uint8_t ptl_present_flag; + uint16_t ols_idx; + uint8_t num_sublayers; + uint8_t constant_frame_rate; + uint8_t chroma_format_idc; + uint8_t bit_depth_minus8; + VVCPTLRecord ptl; + uint16_t max_picture_width; + uint16_t max_picture_height; + uint16_t avg_frame_rate; + uint8_t num_of_arrays; + VVCCNALUnitArray *array; +} VVCDecoderConfigurationRecord; + +typedef struct VVCCProfileTierLevel { + uint8_t profile_idc; + uint8_t tier_flag; + uint8_t general_level_idc; + uint8_t ptl_frame_only_constraint_flag; + uint8_t ptl_multilayer_enabled_flag; +// general_constraint_info + uint8_t gci_present_flag; + unsigned __int128 gci_general_constraints; + uint8_t gci_num_reserved_bits; +// end general_constraint_info + uint8_t *ptl_sublayer_level_present_flag; + uint8_t *sublayer_level_idc; + uint8_t ptl_num_sub_profiles; + uint32_t *general_sub_profile_idc; +} VVCCProfileTierLevel; + + +static void vvcc_update_ptl(VVCDecoderConfigurationRecord *vvcc, + VVCCProfileTierLevel *ptl) +{ + /* + * The level indication general_level_idc must indicate a level of + * capability equal to or greater than the highest level indicated for the + * highest tier in all the parameter sets. + */ + if (vvcc->ptl.general_tier_flag < ptl->tier_flag) + vvcc->ptl.general_level_idc = ptl->general_level_idc; + else + vvcc->ptl.general_level_idc = FFMAX(vvcc->ptl.general_level_idc, ptl->general_level_idc); + + /* + * The tier indication general_tier_flag must indicate a tier equal to or + * greater than the highest tier indicated in all the parameter sets. + */ + vvcc->ptl.general_tier_flag = FFMAX(vvcc->ptl.general_tier_flag, ptl->tier_flag); + + /* + * The profile indication general_profile_idc must indicate a profile to + * which the stream associated with this configuration record conforms. + * + * If the sequence parameter sets are marked with different profiles, then + * the stream may need examination to determine which profile, if any, the + * entire stream conforms to. If the entire stream is not examined, or the + * examination reveals that there is no profile to which the entire stream + * conforms, then the entire stream must be split into two or more + * sub-streams with separate configuration records in which these rules can + * be met. + * + * Note: set the profile to the highest value for the sake of simplicity. + */ + vvcc->ptl.general_profile_idc = FFMAX(vvcc->ptl.general_profile_idc, ptl->profile_idc); + + /* + * Each bit in flags may only be set if all + * the parameter sets set that bit. + */ + vvcc->ptl.ptl_frame_only_constraint_flag &= ptl->ptl_frame_only_constraint_flag; + vvcc->ptl.ptl_multilayer_enabled_flag &= ptl->ptl_multilayer_enabled_flag; + + /* + * Constraints Info + * @todo implemnt correct contraint info parsing + */ + if(ptl->gci_present_flag) { + vvcc->ptl.num_bytes_constraint_info = 10 + ceil(ptl->gci_num_reserved_bits / 8.); + vvcc->ptl.general_constraint_info = (uint8_t *) malloc(sizeof(uint8_t)); + //*vvcc->ptl.general_constraint_info = ptl->gci_present_flag; + //*vvcc->ptl.general_constraint_info = *vvcc->general_constraint_info << 71 | ptl->gci_general_constraints; + //*vvcc->ptl.general_constraint_info = *vvcc->general_constraint_info << 8 | ptl->gci_num_reserved_bits; + *vvcc->ptl.general_constraint_info = *vvcc->ptl.general_constraint_info << (int)(8*ceil(ptl->gci_num_reserved_bits / 8.)); + } else { + vvcc->ptl.num_bytes_constraint_info = 1; + vvcc->ptl.general_constraint_info = (uint8_t *) malloc(vvcc->ptl.num_bytes_constraint_info*sizeof(uint8_t)); + *vvcc->ptl.general_constraint_info = 0x00; + } + + /* + * Each bit in flags may only be set if one of + * the parameter sets set that bit. + */ + vvcc->ptl.ptl_sublayer_level_present_flag = (uint8_t *) malloc(sizeof(uint8_t)*vvcc->num_sublayers-1); + vvcc->ptl.sublayer_level_idc = (uint8_t *) malloc(sizeof(uint8_t)*vvcc->num_sublayers-1); + + memset(vvcc->ptl.ptl_sublayer_level_present_flag, 0, sizeof(uint8_t)*vvcc->num_sublayers-1); + memset(vvcc->ptl.sublayer_level_idc, 0, sizeof(uint8_t)*vvcc->num_sublayers-1); + + for(int i = vvcc->num_sublayers - 2; i >= 0; i--) + { + vvcc->ptl.ptl_sublayer_level_present_flag[i] |= ptl->ptl_sublayer_level_present_flag[i]; + if(vvcc->ptl.ptl_sublayer_level_present_flag[i]){ + vvcc->ptl.sublayer_level_idc[i] = FFMAX(vvcc->ptl.sublayer_level_idc[i], ptl->sublayer_level_idc[i]); + } + else{ + if(i == vvcc->num_sublayers - 1){ + vvcc->ptl.sublayer_level_idc[i] = vvcc->ptl.general_level_idc; + }else{ + vvcc->ptl.sublayer_level_idc[i] = vvcc->ptl.sublayer_level_idc[i+1]; + } + } + } + + vvcc->ptl.ptl_num_sub_profiles = FFMAX(vvcc->ptl.ptl_num_sub_profiles, ptl->ptl_num_sub_profiles); + if( vvcc->ptl.ptl_num_sub_profiles ){ + vvcc->ptl.general_sub_profile_idc = (uint32_t *) malloc(sizeof(uint32_t)*vvcc->ptl.ptl_num_sub_profiles); + for(int i = 0; i < vvcc->ptl.ptl_num_sub_profiles; i++) { + vvcc->ptl.general_sub_profile_idc[i] = ptl->general_sub_profile_idc[i]; + } + } + else + { + vvcc->ptl.general_sub_profile_idc = (uint32_t *) malloc(sizeof(uint32_t)); + } +} + +static void vvcc_parse_ptl(GetBitContext *gb, + VVCDecoderConfigurationRecord *vvcc, + unsigned int profileTierPresentFlag, + unsigned int max_sub_layers_minus1) +{ + VVCCProfileTierLevel general_ptl; + + if(profileTierPresentFlag) { + general_ptl.profile_idc = get_bits(gb, 7); + general_ptl.tier_flag = get_bits1(gb); + } + general_ptl.general_level_idc = get_bits(gb, 8); + + general_ptl.ptl_frame_only_constraint_flag = get_bits1(gb); + general_ptl.ptl_multilayer_enabled_flag = get_bits1(gb); + if(profileTierPresentFlag) { // parse constraint info + general_ptl.gci_present_flag = get_bits1(gb); + if(general_ptl.gci_present_flag) { + general_ptl.gci_general_constraints = get_bits_long(gb, 71); + general_ptl.gci_num_reserved_bits = get_bits(gb, 8); + skip_bits(gb, general_ptl.gci_num_reserved_bits); + } + while(gb->index % 8 != 0) + skip_bits1(gb); + } + + general_ptl.ptl_sublayer_level_present_flag = (uint8_t *) malloc(sizeof(uint8_t)*max_sub_layers_minus1); + for(int i = max_sub_layers_minus1-1; i >= 0; i--) { + general_ptl.ptl_sublayer_level_present_flag[i] = get_bits1(gb); + } + while(gb->index%8 != 0) + skip_bits1(gb); + + general_ptl.sublayer_level_idc = (uint8_t *) malloc(sizeof(uint8_t)*max_sub_layers_minus1); + for(int i = max_sub_layers_minus1-1; i >= 0; i--) { + if( general_ptl.ptl_sublayer_level_present_flag[i] ) + general_ptl.sublayer_level_idc[i] = get_bits(gb, 8); + } + + if(profileTierPresentFlag) { + general_ptl.ptl_num_sub_profiles = get_bits(gb, 8); + if( general_ptl.ptl_num_sub_profiles) { + general_ptl.general_sub_profile_idc = (uint32_t *) malloc(sizeof(uint32_t)*general_ptl.ptl_num_sub_profiles); + for(int i = 0; i < general_ptl.ptl_num_sub_profiles; i++) { + general_ptl.general_sub_profile_idc[i] = get_bits_long(gb, 32); + } + } + else + { + general_ptl.general_sub_profile_idc = (uint32_t *) malloc(sizeof(uint32_t)); + } + } + + vvcc_update_ptl(vvcc, &general_ptl); + + free(general_ptl.ptl_sublayer_level_present_flag); + free(general_ptl.sublayer_level_idc); + free(general_ptl.general_sub_profile_idc); +} + +static int vvcc_parse_vps(GetBitContext *gb, + VVCDecoderConfigurationRecord *vvcc) +{ + unsigned int vps_max_layers_minus1; + unsigned int vps_max_sub_layers_minus1; + unsigned int vps_default_ptl_dpb_hrd_max_tid_flag; + unsigned int vps_all_independant_layer_flag; + unsigned int vps_each_layer_is_an_ols_flag; + unsigned int vps_ols_mode_idc; + + unsigned int *vps_pt_present_flag; + unsigned int *vps_ptl_max_tid; + unsigned int vps_num_ptls_minus1 = 0; + + + /* + * vps_video_parameter_set_id u(4) + */ + skip_bits(gb, 4); + + vps_max_layers_minus1 = get_bits(gb, 6); + vps_max_sub_layers_minus1 = get_bits(gb, 3); + + /* + * numTemporalLayers greater than 1 indicates that the stream to which this + * configuration record applies is temporally scalable and the contained + * number of temporal layers (also referred to as temporal sub-layer or + * sub-layer in ISO/IEC 23008-2) is equal to numTemporalLayers. Value 1 + * indicates that the stream is not temporally scalable. Value 0 indicates + * that it is unknown whether the stream is temporally scalable. + */ + vvcc->num_sublayers = FFMAX(vvcc->num_sublayers, + vps_max_sub_layers_minus1 + 1); + + + if(vps_max_layers_minus1 > 0 && vps_max_sub_layers_minus1 > 0) + vps_default_ptl_dpb_hrd_max_tid_flag = get_bits1(gb); //vps_default_ptl_dpb_hrd_max_tid_flags + if(vps_max_layers_minus1 > 0) + vps_all_independant_layer_flag = get_bits1(gb); + + for(int i =0; i <= vps_max_layers_minus1; i++) { + skip_bits(gb, 6); //vps_default_ptl_dpb_hrd_max_tid_flagsvps_layer_id[i] + if(i > 0 && !vps_all_independant_layer_flag) { + if(get_bits1(gb)) { // vps_independant_layer_flag + unsigned int vps_max_tid_ref_present_flag = get_bits1(gb); + for (int j =0; j 0) { + if(vps_all_independant_layer_flag) + vps_each_layer_is_an_ols_flag = get_bits1(gb); + if(vps_each_layer_is_an_ols_flag) { + if(!vps_all_independant_layer_flag) + vps_ols_mode_idc = get_bits(gb, 2); + if(vps_ols_mode_idc == 2) { + unsigned int vps_num_output_layer_sets_minus2 = get_bits(gb, 8); + for( int i = 1; i <= vps_num_output_layer_sets_minus2 +1; i++) { + for( int j = 0; j <= vps_max_layers_minus1; j++) { + skip_bits1(gb); + } + } + } + } + vps_num_ptls_minus1 = get_bits(gb, 8); + } + + vps_pt_present_flag = (unsigned int *) malloc(sizeof(unsigned int) * (vps_num_ptls_minus1+1)); + vps_ptl_max_tid = (unsigned int *) malloc(sizeof(unsigned int) * (vps_num_ptls_minus1+1)); + for(int i = 0; i <= vps_num_ptls_minus1; i++) { + if(i>0) + vps_pt_present_flag[i] = get_bits1(gb); + if(!vps_default_ptl_dpb_hrd_max_tid_flag) + vps_ptl_max_tid[i] = get_bits(gb, 3); + } + + while(gb->index%8 != 0) + skip_bits1(gb); + + for(int i = 0; i <= vps_num_ptls_minus1; i++) { + vvcc_parse_ptl(gb, vvcc, vps_pt_present_flag[i], vps_ptl_max_tid[i]); + } + + free(vps_pt_present_flag); + free(vps_ptl_max_tid); + + /* nothing useful for vvcc past this point */ + return 0; +} + +static int vvcc_parse_sps(GetBitContext *gb, + VVCDecoderConfigurationRecord *vvcc) +{ + unsigned int sps_max_sub_layers_minus1, log2_ctu_size_minus5; + //unsigned int num_short_term_ref_pic_sets, num_delta_pocs[VVC_MAX_REF_PIC_LISTS]; + //unsigned int sps_chroma_format_idc; + unsigned int sps_subpic_same_size_flag, sps_pic_height_max_in_luma_sample, sps_pic_width_max_in_luma_sample; + unsigned int sps_independant_subpics_flag; + unsigned int flag; + + skip_bits(gb, 8); // sps_seq_parameter_set_id && sps_video_parameter_set_id + sps_max_sub_layers_minus1 = get_bits (gb, 3); + + /* + * numTemporalLayers greater than 1 indicates that the stream to which this + * configuration record applies is temporally scalable and the contained + * number of temporal layers (also referred to as temporal sub-layer or + * sub-layer in ISO/IEC 23008-2) is equal to numTemporalLayers. Value 1 + * indicates that the stream is not temporally scalable. Value 0 indicates + * that it is unknown whether the stream is temporally scalable. + */ + vvcc->num_sublayers = FFMAX(vvcc->num_sublayers, + sps_max_sub_layers_minus1 + 1); + + vvcc->chroma_format_idc = get_bits(gb, 2); + log2_ctu_size_minus5 = get_bits(gb, 2); + + if(get_bits1(gb)) //sps_ptl_dpb_hrd_params_present_flag + vvcc_parse_ptl(gb, vvcc, 1, sps_max_sub_layers_minus1); + + flag = get_bits(gb, 1); //skip_bits1(gb); //sps_gdr_enabled_flag + flag = get_bits(gb, 1); //sps_ref_pic_resampling_enabled_flag + if(flag){ //sps_ref_pic_resampling_enabled_flag + flag = get_bits(gb, 1); //skip_bits1(gb); //sps_res_change_in_clvs_allowed_flag + } + + sps_pic_width_max_in_luma_sample = get_ue_golomb_long(gb); + vvcc->max_picture_width = FFMAX(vvcc->max_picture_width , sps_pic_width_max_in_luma_sample); + sps_pic_height_max_in_luma_sample= get_ue_golomb_long(gb); + vvcc->max_picture_height = FFMAX(vvcc->max_picture_height , sps_pic_height_max_in_luma_sample); + + if(get_bits1(gb)) { + get_ue_golomb_long(gb); // sps_conf_win_left_offset + get_ue_golomb_long(gb); // sps_conf_win_right_offset + get_ue_golomb_long(gb); // sps_conf_win_top_offset + get_ue_golomb_long(gb); // sps_conf_win_bottom_offset + } + + if(get_bits1(gb)) { // sps_subpic_info_present_flag + unsigned int sps_num_subpics_minus1 = get_ue_golomb_long(gb); + if(sps_num_subpics_minus1 > 0) { // sps_num_subpics_minus1 + sps_independant_subpics_flag = get_bits1(gb); + sps_subpic_same_size_flag = get_bits1(gb); + } + for(int i = 0; sps_num_subpics_minus1 > 0 && i <= sps_num_subpics_minus1; i++) { + if(!sps_subpic_same_size_flag || i == 0) { + int len = FFMIN(log2_ctu_size_minus5 +5, 16); + if(i > 0 && sps_pic_width_max_in_luma_sample > 128) + skip_bits(gb, len); + if(i > 0 && sps_pic_height_max_in_luma_sample > 128) + skip_bits(gb, len); + if(i < sps_num_subpics_minus1 && sps_pic_width_max_in_luma_sample > 128) + skip_bits(gb, len); + if(i < sps_num_subpics_minus1 && sps_pic_height_max_in_luma_sample > 128) + skip_bits(gb, len); + } + if(!sps_independant_subpics_flag) { + skip_bits(gb, 2); // sps_subpic_treated_as_pic_flag && sps_loop_filter_across_subpic_enabled_flag + } + } + get_ue_golomb_long(gb); // sps_subpic_id_len_minus1 + if(get_bits1(gb)) { // sps_subpic_id_mapping_explicitly_signalled_flag + if(get_bits1(gb)) // sps_subpic_id_mapping_present_flag + for(int i = 0; i <= sps_num_subpics_minus1; i++) { + skip_bits1(gb); // sps_subpic_id[i] + } + } + } + vvcc->bit_depth_minus8 = get_ue_golomb_long(gb); + + /* nothing useful for vvcc past this point */ + return 0; +} + +static int vvcc_parse_pps(GetBitContext *gb, + VVCDecoderConfigurationRecord *vvcc) +{ + + // Nothing of importance to parse in PPS + /* nothing useful for vvcc past this point */ + return 0; +} + +static void nal_unit_parse_header(GetBitContext *gb, uint8_t *nal_type) +{ + /* + * forbidden_zero_bit u(1) + * nuh_reserved_zero_bit u(1) + * nuh_layer_id u(6) + */ + skip_bits(gb, 8); + *nal_type = get_bits(gb, 5); + + /* + * nuh_temporal_id_plus1 u(3) + */ + skip_bits(gb, 3); +} + +static int vvcc_array_add_nal_unit(uint8_t *nal_buf, uint32_t nal_size, + uint8_t nal_type, int ps_array_completeness, + VVCDecoderConfigurationRecord *vvcc) +{ + int ret; + uint8_t index; + uint16_t num_nalus; + VVCCNALUnitArray *array; + + for (index = 0; index < vvcc->num_of_arrays; index++) + if (vvcc->array[index].NAL_unit_type == nal_type) + break; + + if (index >= vvcc->num_of_arrays) { + uint8_t i; + + ret = av_reallocp_array(&vvcc->array, index + 1, sizeof(VVCCNALUnitArray)); + if (ret < 0) + return ret; + + for (i = vvcc->num_of_arrays; i <= index; i++) + memset(&vvcc->array[i], 0, sizeof(VVCCNALUnitArray)); + vvcc->num_of_arrays = index + 1; + } + + array = &vvcc->array[index]; + num_nalus = array->num_nalus; + + ret = av_reallocp_array(&array->nal_unit, num_nalus + 1, sizeof(uint8_t*)); + if (ret < 0) + return ret; + + ret = av_reallocp_array(&array->nal_unit_length, num_nalus + 1, sizeof(uint16_t)); + if (ret < 0) + return ret; + + array->nal_unit [num_nalus] = nal_buf; + array->nal_unit_length[num_nalus] = nal_size; + array->NAL_unit_type = nal_type; + array->num_nalus++; + + /* + * When the sample entry name is ‘vvc1’, the default and mandatory value of + * array_completeness is 1 for arrays of all types of parameter sets, and 0 + * for all other arrays. When the sample entry name is ‘hev1’, the default + * value of array_completeness is 0 for all arrays. + */ + if (nal_type == VVC_VPS_NUT || nal_type == VVC_SPS_NUT || nal_type == VVC_PPS_NUT) + array->array_completeness = ps_array_completeness; + + return 0; +} + +static int vvcc_add_nal_unit(uint8_t *nal_buf, uint32_t nal_size, + int ps_array_completeness, + VVCDecoderConfigurationRecord *vvcc) +{ + int ret = 0; + GetBitContext gbc; + uint8_t nal_type; + uint8_t *rbsp_buf; + uint32_t rbsp_size; + + rbsp_buf = ff_nal_unit_extract_rbsp(nal_buf, nal_size, &rbsp_size, 2); + if (!rbsp_buf) { + ret = AVERROR(ENOMEM); + goto end; + } + + ret = init_get_bits8(&gbc, rbsp_buf, rbsp_size); + if (ret < 0) + goto end; + + nal_unit_parse_header(&gbc, &nal_type); + + /* + * Note: only 'declarative' SEI messages are allowed in + * vvcc. Perhaps the SEI playload type should be checked + * and non-declarative SEI messages discarded? + */ + switch (nal_type) { + case VVC_OPI_NUT: + case VVC_VPS_NUT: + case VVC_SPS_NUT: + case VVC_PPS_NUT: + case VVC_PREFIX_SEI_NUT: + case VVC_SUFFIX_SEI_NUT: + ret = vvcc_array_add_nal_unit(nal_buf, nal_size, nal_type, + ps_array_completeness, vvcc); + if (ret < 0) + goto end; + else if (nal_type == VVC_VPS_NUT) + ret = vvcc_parse_vps(&gbc, vvcc); + else if (nal_type == VVC_SPS_NUT) + ret = vvcc_parse_sps(&gbc, vvcc); + else if (nal_type == VVC_PPS_NUT) + ret = vvcc_parse_pps(&gbc, vvcc); + else if (nal_type == VVC_OPI_NUT) { + // not yet supported + } + if (ret < 0) + goto end; + break; + default: + ret = AVERROR_INVALIDDATA; + goto end; + } + +end: + av_free(rbsp_buf); + return ret; +} + +static void vvcc_init(VVCDecoderConfigurationRecord *vvcc) +{ + memset(vvcc, 0, sizeof(VVCDecoderConfigurationRecord)); + vvcc->lengthSizeMinusOne = 3; // 4 bytes + + vvcc->ptl.num_bytes_constraint_info = 1; + + vvcc->ptl_present_flag = 1; +} + +static void vvcc_close(VVCDecoderConfigurationRecord *vvcc) +{ + uint8_t i; + + for (i = 0; i < vvcc->num_of_arrays; i++) { + vvcc->array[i].num_nalus = 0; + av_freep(&vvcc->array[i].nal_unit); + av_freep(&vvcc->array[i].nal_unit_length); + } + + free(vvcc->ptl.general_constraint_info); + free(vvcc->ptl.ptl_sublayer_level_present_flag); + free(vvcc->ptl.sublayer_level_idc); + free(vvcc->ptl.general_sub_profile_idc); + + vvcc->num_of_arrays = 0; + av_freep(&vvcc->array); +} + +static int vvcc_write(AVIOContext *pb, VVCDecoderConfigurationRecord *vvcc) +{ + uint8_t i; + uint16_t j, vps_count = 0, sps_count = 0, pps_count = 0; + unsigned char *buf = NULL; + /* + * It's unclear how to properly compute these fields, so + * let's always set them to values meaning 'unspecified'. + */ + vvcc->avg_frame_rate = 0; + vvcc->constant_frame_rate = 1; + + av_log(NULL, AV_LOG_TRACE, "lengthSizeMinusOne: %"PRIu8"\n", + vvcc->lengthSizeMinusOne); + av_log(NULL, AV_LOG_TRACE, "ptl_present_flag: %"PRIu8"\n", + vvcc->ptl_present_flag); + av_log(NULL, AV_LOG_TRACE, "ols_idx: %"PRIu16"\n", + vvcc->ols_idx); + av_log(NULL, AV_LOG_TRACE, "num_sublayers: %"PRIu8"\n", + vvcc->num_sublayers); + av_log(NULL, AV_LOG_TRACE, "constant_frame_rate: %"PRIu8"\n", + vvcc->constant_frame_rate); + av_log(NULL, AV_LOG_TRACE, "chroma_format_idc: %"PRIu8"\n", + vvcc->chroma_format_idc); + + av_log(NULL, AV_LOG_TRACE, "bit_depth_minus8: %"PRIu8"\n", + vvcc->bit_depth_minus8); + av_log(NULL, AV_LOG_TRACE, "num_bytes_constraint_info: %"PRIu8"\n", + vvcc->ptl.num_bytes_constraint_info); + av_log(NULL, AV_LOG_TRACE, "general_profile_idc: %"PRIu8"\n", + vvcc->ptl.general_profile_idc); + av_log(NULL, AV_LOG_TRACE, "general_tier_flag: %"PRIu8"\n", + vvcc->ptl.general_tier_flag); + av_log(NULL, AV_LOG_TRACE, "general_level_idc: %"PRIu8"\n", + vvcc->ptl.general_level_idc); + av_log(NULL, AV_LOG_TRACE, "ptl_frame_only_constraint_flag: %"PRIu8"\n", + vvcc->ptl.ptl_frame_only_constraint_flag); + av_log(NULL, AV_LOG_TRACE, "ptl_multilayer_enabled_flag: %"PRIu8"\n", + vvcc->ptl.ptl_multilayer_enabled_flag); + av_log(NULL, AV_LOG_TRACE, "general_constraint_info: %"PRIu8"\n", + *vvcc->ptl.general_constraint_info); + + for (i = 0; i < vvcc->num_sublayers-1; i++) { + av_log(NULL, AV_LOG_TRACE, "ptl_sublayer_level_present_flag[%"PRIu8"]: %"PRIu8"\n", + i, vvcc->ptl.ptl_sublayer_level_present_flag[i]); + av_log(NULL, AV_LOG_TRACE, "sublayer_level_idc[%"PRIu8"]: %"PRIu8"\n", + i, vvcc->ptl.sublayer_level_idc[i]); + } + + av_log(NULL, AV_LOG_TRACE, "num_sub_profiles: %"PRIu8"\n", + vvcc->ptl.ptl_num_sub_profiles); + + for (i = 0; i < vvcc->ptl.ptl_num_sub_profiles; i++) { + av_log(NULL, AV_LOG_TRACE, "general_sub_profile_idc[%"PRIu8"]: %"PRIx32"\n", + i, vvcc->ptl.general_sub_profile_idc[i]); + } + + av_log(NULL, AV_LOG_TRACE, "max_picture_width: %"PRIu16"\n", + vvcc->max_picture_width); + av_log(NULL, AV_LOG_TRACE, "max_picture_height: %"PRIu16"\n", + vvcc->max_picture_height); + av_log(NULL, AV_LOG_TRACE, "avg_frame_rate: %"PRIu16"\n", + vvcc->avg_frame_rate); + + av_log(NULL, AV_LOG_TRACE, "num_of_arrays: %"PRIu8"\n", + vvcc->num_of_arrays); + for (i = 0; i < vvcc->num_of_arrays; i++) { + av_log(NULL, AV_LOG_TRACE, "array_completeness[%"PRIu8"]: %"PRIu8"\n", + i, vvcc->array[i].array_completeness); + av_log(NULL, AV_LOG_TRACE, "NAL_unit_type[%"PRIu8"]: %"PRIu8"\n", + i, vvcc->array[i].NAL_unit_type); + av_log(NULL, AV_LOG_TRACE, "num_nalus[%"PRIu8"]: %"PRIu16"\n", + i, vvcc->array[i].num_nalus); + for (j = 0; j < vvcc->array[i].num_nalus; j++) + av_log(NULL, AV_LOG_TRACE, + "nal_unit_length[%"PRIu8"][%"PRIu16"]: %"PRIu16"\n", + i, j, vvcc->array[i].nal_unit_length[j]); + } + + /* + * We need at least one of each: VPS and SPS. + */ + for (i = 0; i < vvcc->num_of_arrays; i++) + switch (vvcc->array[i].NAL_unit_type) { + case VVC_VPS_NUT: + vps_count += vvcc->array[i].num_nalus; + break; + case VVC_SPS_NUT: + sps_count += vvcc->array[i].num_nalus; + break; + case VVC_PPS_NUT: + pps_count += vvcc->array[i].num_nalus; + break; + default: + break; + } + + if (!sps_count || sps_count > VVC_MAX_SPS_COUNT) + return AVERROR_INVALIDDATA; + + /* bit(5) reserved = ‘11111’b; + unsigned int (2) LengthSizeMinusOne + unsigned int (1) ptl_present_flag */ + avio_w8(pb, vvcc->lengthSizeMinusOne << 1 | vvcc->ptl_present_flag | 0xf8); + + if(vvcc->ptl_present_flag) { + /* + * unsigned int(9) ols_idx; + * unsigned int(3) num_sublayers; + * unsigned int(2) constant_frame_rate; + * unsigned int(2) chroma_format_idc; */ + avio_wb16(pb, vvcc->ols_idx << 7 | vvcc->num_sublayers << 4 | vvcc->constant_frame_rate << 2 | vvcc->chroma_format_idc); + + /* unsigned int(3) bit_depth_minus8; + bit(5) reserved = ‘11111’b;*/ + avio_w8(pb, vvcc->bit_depth_minus8 << 5 | 0x1f); + + //VVCPTLRecord + + /* bit(2) reserved = ‘00’b; + unsigned int (6) num_bytes_constraint_info */ + avio_w8(pb, vvcc->ptl.num_bytes_constraint_info & 0x3f); + + /* unsigned int (7) general_profile_idc + unsigned int (1) general_tier_flag */ + avio_w8(pb, vvcc->ptl.general_profile_idc << 1 | vvcc->ptl.general_tier_flag); + + /* unsigned int (8) general_level_idc */ + avio_w8(pb, vvcc->ptl.general_level_idc); + + /* + * unsigned int (1) ptl_frame_only_constraint_flag + * unsigned int (1) ptl_multilayer_enabled_flag + * unsigned int (8*num_bytes_constraint_info -2) general_constraint_info */ + buf = (unsigned char *) malloc(sizeof(unsigned char) * vvcc->ptl.num_bytes_constraint_info); + *buf = vvcc->ptl.ptl_frame_only_constraint_flag << vvcc->ptl.num_bytes_constraint_info*8-1 | vvcc->ptl.ptl_multilayer_enabled_flag << vvcc->ptl.num_bytes_constraint_info*8-2 | *vvcc->ptl.general_constraint_info >> 2; + avio_write(pb, buf, vvcc->ptl.num_bytes_constraint_info); + free(buf); + + if(vvcc->num_sublayers > 1) { + uint8_t ptl_sublayer_level_present_flags = 0; + for(int i = vvcc->num_sublayers -2; i >= 0; i--) { + ptl_sublayer_level_present_flags = (ptl_sublayer_level_present_flags << 1 | vvcc->ptl.ptl_sublayer_level_present_flag[i]); + } + avio_w8(pb, ptl_sublayer_level_present_flags); + } + + for(int i = vvcc->num_sublayers -2; i >= 0; i--) { + if(vvcc->ptl.ptl_sublayer_level_present_flag[i]) + avio_w8(pb, vvcc->ptl.sublayer_level_idc[i]); + } + + /* unsigned int(8) num_sub_profiles; */ + avio_w8(pb, vvcc->ptl.ptl_num_sub_profiles); + + for(int j = 0; j < vvcc->ptl.ptl_num_sub_profiles; j++) { + /* unsigned int(32) general_sub_profile_idc[j]; */ + avio_wb32(pb, vvcc->ptl.general_sub_profile_idc[j]); + } + + //End of VvcPTLRecord + + /* + * unsigned int(16) max_picture_width;*/ + avio_wb16(pb, vvcc->max_picture_width); + + /* + * unsigned int(16) max_picture_height;*/ + avio_wb16(pb, vvcc->max_picture_height); + + /* + * unsigned int(16) avg_frame_rate; */ + avio_wb16(pb, vvcc->avg_frame_rate); + } + + /* unsigned int(8) num_of_arrays; */ + avio_w8(pb, vvcc->num_of_arrays); + + for (i = 0; i < vvcc->num_of_arrays; i++) { + /* + * bit(1) array_completeness; + * unsigned int(2) reserved = 0; + * unsigned int(5) NAL_unit_type; + */ + avio_w8(pb, vvcc->array[i].array_completeness << 7 | + vvcc->array[i].NAL_unit_type & 0x1f); + /* unsigned int(16) num_nalus; */ + if(vvcc->array[i].NAL_unit_type != VVC_DCI_NUT && vvcc->array[i].NAL_unit_type != VVC_OPI_NUT) + avio_wb16(pb, vvcc->array[i].num_nalus); + for (j = 0; j < vvcc->array[i].num_nalus; j++) { + /* unsigned int(16) nal_unit_length; */ + avio_wb16(pb, vvcc->array[i].nal_unit_length[j]); + + /* bit(8*nal_unit_length) nal_unit; */ + avio_write(pb, vvcc->array[i].nal_unit[j], vvcc->array[i].nal_unit_length[j]); + } + } + + return 0; +} + +int ff_vvc_annexb2mp4(AVIOContext *pb, const uint8_t *buf_in, + int size, int filter_ps, int *ps_count) +{ + int num_ps = 0, ret = 0; + uint8_t *buf, *end, *start = NULL; + + if (!filter_ps) { + ret = ff_avc_parse_nal_units(pb, buf_in, size); + goto end; + } + + ret = ff_avc_parse_nal_units_buf(buf_in, &start, &size); + if (ret < 0) + goto end; + + ret = 0; + buf = start; + end = start + size; + + while (end - buf > 4) { + uint32_t len = FFMIN(AV_RB32(buf), end - buf - 4); + uint8_t type = (buf[5] >> 3); + + buf += 4; + + switch (type) { + case VVC_VPS_NUT: + case VVC_SPS_NUT: + case VVC_PPS_NUT: + num_ps++; + break; + default: + ret += 4 + len; + avio_wb32(pb, len); + avio_write(pb, buf, len); + break; + } + + buf += len; + } + +end: + av_free(start); + if (ps_count) + *ps_count = num_ps; + return ret; +} + +int ff_vvc_annexb2mp4_buf(const uint8_t *buf_in, uint8_t **buf_out, + int *size, int filter_ps, int *ps_count) +{ + AVIOContext *pb; + int ret; + + ret = avio_open_dyn_buf(&pb); + if (ret < 0) + return ret; + + ret = ff_vvc_annexb2mp4(pb, buf_in, *size, filter_ps, ps_count); + if (ret < 0) { + ffio_free_dyn_buf(&pb); + return ret; + } + + *size = avio_close_dyn_buf(pb, buf_out); + + return 0; +} + +int ff_isom_write_vvcc(AVIOContext *pb, const uint8_t *data, + int size, int ps_array_completeness) +{ + VVCDecoderConfigurationRecord vvcc; + uint8_t *buf, *end, *start; + int ret; + + if (size < 6) { + /* We can't write a valid vvcc from the provided data */ + return AVERROR_INVALIDDATA; + } else if (*data == 1) { + /* Data is already vvcc-formatted */ + avio_write(pb, data, size); + return 0; + } else if (!(AV_RB24(data) == 1 || AV_RB32(data) == 1)) { + /* Not a valid Annex B start code prefix */ + return AVERROR_INVALIDDATA; + } + + ret = ff_avc_parse_nal_units_buf(data, &start, &size); + if (ret < 0) + return ret; + + vvcc_init(&vvcc); + + buf = start; + end = start + size; + + while (end - buf > 4) { + uint32_t len = FFMIN(AV_RB32(buf), end - buf - 4); + uint8_t type = (buf[5] >> 3); + + buf += 4; + + switch (type) { + case VVC_OPI_NUT: + case VVC_VPS_NUT: + case VVC_SPS_NUT: + case VVC_PPS_NUT: + case VVC_PREFIX_SEI_NUT: + case VVC_SUFFIX_SEI_NUT: + ret = vvcc_add_nal_unit(buf, len, ps_array_completeness, &vvcc); + if (ret < 0) + goto end; + break; + default: + break; + } + + buf += len; + } + + ret = vvcc_write(pb, &vvcc); + +end: + vvcc_close(&vvcc); + av_free(start); + return ret; +} diff --git a/libavformat/vvc.h b/libavformat/vvc.h new file mode 100644 index 0000000000..cdf4f6d3f5 --- /dev/null +++ b/libavformat/vvc.h @@ -0,0 +1,99 @@ +/* + * VVC helper functions for muxers + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * internal header for VVC (de)muxer utilities + */ + +#ifndef AVFORMAT_VVC_H +#define AVFORMAT_VVC_H + +#include +#include "avio.h" + +/** + * Writes Annex B formatted VVC NAL units to the provided AVIOContext. + * + * The NAL units are converted to an MP4-compatible format (start code prefixes + * are replaced by 4-byte size fields, as per ISO/IEC 14496-15). + * + * If filter_ps is non-zero, any VVC parameter sets found in the input will be + * discarded, and *ps_count will be set to the number of discarded PS NAL units. + * + * @param pb address of the AVIOContext where the data shall be written + * @param buf_in address of the buffer holding the input data + * @param size size (in bytes) of the input buffer + * @param filter_ps whether to write parameter set NAL units to the output (0) + * or to discard them (non-zero) + * @param ps_count address of the variable where the number of discarded + * parameter set NAL units shall be written, may be NULL + * @return the amount (in bytes) of data written in case of success, a negative + * value corresponding to an AVERROR code in case of failure + */ +int ff_vvc_annexb2mp4(AVIOContext *pb, const uint8_t *buf_in, + int size, int filter_ps, int *ps_count); + +/** + * Writes Annex B formatted VVC NAL units to a data buffer. + * + * The NAL units are converted to an MP4-compatible format (start code prefixes + * are replaced by 4-byte size fields, as per ISO/IEC 14496-15). + * + * If filter_ps is non-zero, any VVC parameter sets found in the input will be + * discarded, and *ps_count will be set to the number of discarded PS NAL units. + * + * On success, *size holds the size (in bytes) of the output data buffer. + * + * @param buf_in address of the buffer holding the input data + * @param size address of the variable holding the size (in bytes) of the input + * buffer (on input) and of the output buffer (on success) + * @param buf_out on success, address of the variable holding the address of + * the output buffer + * @param filter_ps whether to write parameter set NAL units to the output (0) + * or to discard them (non-zero) + * @param ps_count address of the variable where the number of discarded + * parameter set NAL units shall be written, may be NULL + * @return 0 in case of success, a negative value corresponding to an AVERROR + * code in case of failure + * @note *buf_out will be treated as uninitialized on input and won't be freed. + */ +int ff_vvc_annexb2mp4_buf(const uint8_t *buf_in, uint8_t **buf_out, + int *size, int filter_ps, int *ps_count); + +/** + * Writes VVC extradata (parameter sets, declarative SEI NAL units) to the + * provided AVIOContext. + * + * If the extradata is Annex B format, it gets converted to vvcC format before + * writing. + * + * @param pb address of the AVIOContext where the vvcC shall be written + * @param data address of the buffer holding the data needed to write the vvcC + * @param size size (in bytes) of the data buffer + * @param ps_array_completeness whether all parameter sets are in the vvcC (1) + * or there may be additional parameter sets in the bitstream (0) + * @return >=0 in case of success, a negative value corresponding to an AVERROR + * code in case of failure + */ +int ff_isom_write_vvcc(AVIOContext *pb, const uint8_t *data, + int size, int ps_array_completeness); + +#endif /* AVFORMAT_VVC_H */ diff --git a/libavformat/vvcdec.c b/libavformat/vvcdec.c new file mode 100644 index 0000000000..304f900966 --- /dev/null +++ b/libavformat/vvcdec.c @@ -0,0 +1,61 @@ +/* + * RAW VVC video demuxer + * Copyright (c) 2020 Nuo Mi + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavcodec/vvc.h" + +#include "avformat.h" +#include "rawdec.h" + +static int vvc_probe(const AVProbeData *p) +{ + uint32_t code = -1; + int sps = 0, pps = 0, irap = 0; + int i; + + for (i = 0; i < p->buf_size - 1; i++) { + code = (code << 8) + p->buf[i]; + if ((code & 0xffffff00) == 0x100) { + uint8_t nal2 = p->buf[i + 1]; + int type = (nal2 & 0xF8) >> 3; + + if (code & 0x80) // forbidden_zero_bit + return 0; + + if ((nal2 & 0x7) == 0) // nuh_temporal_id_plus1 + return 0; + + switch (type) { + case VVC_SPS_NUT: sps++; break; + case VVC_PPS_NUT: pps++; break; + case VVC_IDR_N_LP: + case VVC_IDR_W_RADL: + case VVC_CRA_NUT: + case VVC_GDR_NUT: irap++; break; + } + } + } + + if (sps && pps && irap) + return AVPROBE_SCORE_EXTENSION + 1; // 1 more than .mpg + return 0; +} + +FF_DEF_RAWVIDEO_DEMUXER(vvc, "raw VVC video", vvc_probe, "h266,266,vvc", AV_CODEC_ID_VVC) From patchwork Wed Oct 19 07:25:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Siedel X-Patchwork-Id: 38818 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp216091pzb; Wed, 19 Oct 2022 00:28:06 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4WtRt+NPnT4A+d49UBMl/IhSOmJezH1IsMDXdif7PeH1BcFHaUCNah5w9x4V/eII2Eo+Y3 X-Received: by 2002:a05:6402:51d0:b0:45d:5ba4:c3d2 with SMTP id r16-20020a05640251d000b0045d5ba4c3d2mr6137403edd.132.1666164485809; Wed, 19 Oct 2022 00:28:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666164485; cv=none; d=google.com; s=arc-20160816; b=EJXzsIkVjeiBCCN0td5okpm0u2a5bdKGXXd5bwiyuLQ1Lh60AeRlgGF2+143H4NFjO q27xDOER4HJ/LkMh+5ihKIqDF4q3JRs29zCtvo+ByxYOgmFB2WRflbpSkxPEUV/wT9Sq V4wz6klLeRRQ39MYTybCcFrMNyDLMtlM4uf4sQqOmS2egRisjrzQgTaCf2fE4gS08Ave ElyTTQGh0ZqAxeTFGKai+ye+F8bksEzgOrWYwt6BQHTeI84D/9BuQQpabjP5AH+hp2Pt MutzwWTQO8GUIvZtpQ6qM86rSKImReRDiz4wjviVB9+q2+IN52BnbhdOlGvZTkNmdYWh 7liQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=tYwpqcR29SrhzUGv7h5a3H5AUpFpPLH+mp/jHNYY+fk=; b=CgSoRRGfx3REGAm57GYbIOi6yiDLhg1EGHk2igjJYli83Pnt2eDJciBYtdZgJo3td+ X4OraP52YYvdKBOovMA/wMtw0VPu+8rrOlwWjUj4GyU/2tfW2xZRakgLMTJNhDtuPL8g 5RB7oMjNvtKJC+4TYMb1MbdO1epfHoLAyo7BfLKteIsogCEgPt9jk99fLSJ6hVarjsyy KhBME3NxbOjhawEwoLt6tF49M+q8WuKWj44qckxK5o8m+qSQSX6EMa9YVyfH3FgzR2d1 i4t0A3tQdzj0QaHVxO2ZdffrfA9HNYAvWpE9ugz/zfXdJuh6jDLdhIPQEINKbGaTvI5X 56WA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=NEESUUjY; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id g6-20020a1709064e4600b0077cb9bc7984si11613795ejw.181.2022.10.19.00.28.05; Wed, 19 Oct 2022 00:28:05 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=NEESUUjY; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1BD0C68BE06; Wed, 19 Oct 2022 10:26:41 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f52.google.com (mail-ed1-f52.google.com [209.85.208.52]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A2AB368BDDE for ; Wed, 19 Oct 2022 10:26:30 +0300 (EEST) Received: by mail-ed1-f52.google.com with SMTP id q19so23892210edd.10 for ; Wed, 19 Oct 2022 00:26:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spin-digital-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Vi8tP2cTrmJvwXXjoEzEPj0XCcKgDqCC/RabO8FGEy8=; b=NEESUUjYokl4JyjbQ4t8z6Nk81I4ULN4EcB22k331WXn7s4p06RLq0ZAodfi8d/gd2 nBOaclnU1AmDz7glGv/L3pyeuwMQ7PjxgYSTPzBWfjqbuqZYXuQAMqDe7Ebd/mjLfp87 oRumc/9H1++NZWBdfa+V/1NFwhV/FIV99TLPQXqtKlDgq2R8eBJ7AmSyqcLpHVyegIuJ 5bbf5Y4ujs0f/leFqJhJm1AOq6ed/9xdxv5tZzukyFg9yE1TqrY1FhrtD1Aa5Z3MfQGU Y67zNmQ1g+zDRwsP3Ja6W4Eg3BMPLm3DXZGU8FeCrZ67I6AG1jyLWxvcY/vCUt+313cG 3P5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Vi8tP2cTrmJvwXXjoEzEPj0XCcKgDqCC/RabO8FGEy8=; b=bstOSEFsjpmI2LrjkBz3rIv7ylluS73Uvh5mjdKEDhYL96ZvCPrRblib7FAZ2bh2Gr /fWDWXJVawjrW8jaxgmJxxiaM/J2nnY34lCCjSX/ucr5HjWk1e1ys1o22qEpUIShu37n YFm8fWPskG3fBexy3Gs0OZ0yCbIS2pb5ZP1KiN7anUuxpZ7dUYLwukSdyPbiSE5BvW9/ YQHud+tUb0JyvTfit6dn+vOoPH+uK9fMFD18OEy0fdPfN2N9c9ZoWUlBGTBnRUGDWo+d H/lf+ugK6lq46x24gSpW1kBboMMO6yTPN+752cSrNOV3+yiyBvmEjZtQVxb7oa9vEAVR 17iw== X-Gm-Message-State: ACrzQf2whdGsgeI7mdhSMYwcZw20D8+jAzkWLMkYLANGyZN1njnDj9ny gBiJaN8cW1BHsVDg3Jb9Ds5IVJ2OQyQetA== X-Received: by 2002:a05:6402:28cd:b0:459:19c3:43d0 with SMTP id ef13-20020a05640228cd00b0045919c343d0mr6049870edb.197.1666164390009; Wed, 19 Oct 2022 00:26:30 -0700 (PDT) Received: from thomas-win.localdomain ([213.138.44.237]) by smtp.gmail.com with ESMTPSA id p5-20020a170906498500b0078b83968ad4sm8515150eju.24.2022.10.19.00.26.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Oct 2022 00:26:29 -0700 (PDT) From: thomas.ff@spin-digital.com To: ffmpeg-devel@ffmpeg.org Date: Wed, 19 Oct 2022 09:25:04 +0200 Message-Id: <20221019072508.23460-8-thomas.ff@spin-digital.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221019072508.23460-1-thomas.ff@spin-digital.com> References: <20221019072508.23460-1-thomas.ff@spin-digital.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v1 07/11] avformat: add muxer support for H266/VVC X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Thomas Siedel Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: vngutXKzlNaV From: Thomas Siedel Add muxer for vvcc byte stream format. Add AV_CODEC_ID_VVC to ff_mp4_obj_type Add AV_CODEC_ID_VVC to ISO Media codec (VvcConfigurationBox vvi1, vvc1 defined in ISO/IEC 14496-15:2021) Add VvcConfigurationBox vvcC which extends FullBox type in ISO/IEC 14496-15:2021 Add ff_vvc_muxer to RAW muxers Signed-off-by: Thomas Siedel --- libavformat/Makefile | 7 ++++--- libavformat/allformats.c | 1 + libavformat/isom.c | 1 + libavformat/isom_tags.c | 3 +++ libavformat/mov.c | 6 ++++++ libavformat/movenc.c | 41 +++++++++++++++++++++++++++++++++++++++- libavformat/rawenc.c | 23 ++++++++++++++++++++++ 7 files changed, 78 insertions(+), 4 deletions(-) diff --git a/libavformat/Makefile b/libavformat/Makefile index 00ab4ded89..9ee2526eef 100644 --- a/libavformat/Makefile +++ b/libavformat/Makefile @@ -336,7 +336,7 @@ OBJS-$(CONFIG_MATROSKA_DEMUXER) += matroskadec.o matroska.o \ oggparsevorbis.o vorbiscomment.o \ qtpalette.o replaygain.o dovi_isom.o OBJS-$(CONFIG_MATROSKA_MUXER) += matroskaenc.o matroska.o \ - av1.o avc.o hevc.o \ + av1.o avc.o hevc.o vvc.o\ flacenc_header.o avlanguage.o \ vorbiscomment.o wv.o dovi_isom.o OBJS-$(CONFIG_MCA_DEMUXER) += mca.o @@ -358,7 +358,7 @@ OBJS-$(CONFIG_MODS_DEMUXER) += mods.o OBJS-$(CONFIG_MOFLEX_DEMUXER) += moflex.o OBJS-$(CONFIG_MOV_DEMUXER) += mov.o mov_chan.o mov_esds.o \ qtpalette.o replaygain.o dovi_isom.o -OBJS-$(CONFIG_MOV_MUXER) += movenc.o av1.o avc.o hevc.o vpcc.o \ +OBJS-$(CONFIG_MOV_MUXER) += movenc.o av1.o avc.o hevc.o vvc.o vpcc.o \ movenchint.o mov_chan.o rtp.o \ movenccenc.o movenc_ttml.o rawutils.o \ dovi_isom.o @@ -508,7 +508,7 @@ OBJS-$(CONFIG_RTP_MUXER) += rtp.o \ rtpenc_vp8.o \ rtpenc_vp9.o \ rtpenc_xiph.o \ - avc.o hevc.o + avc.o hevc.o vvc.o OBJS-$(CONFIG_RTSP_DEMUXER) += rtsp.o rtspdec.o httpauth.o \ urldecode.o OBJS-$(CONFIG_RTSP_MUXER) += rtsp.o rtspenc.o httpauth.o \ @@ -596,6 +596,7 @@ OBJS-$(CONFIG_VPK_DEMUXER) += vpk.o OBJS-$(CONFIG_VPLAYER_DEMUXER) += vplayerdec.o subtitles.o OBJS-$(CONFIG_VQF_DEMUXER) += vqf.o OBJS-$(CONFIG_VVC_DEMUXER) += vvcdec.o rawdec.o +OBJS-$(CONFIG_VVC_MUXER) += rawenc.o OBJS-$(CONFIG_W64_DEMUXER) += wavdec.o w64.o pcm.o OBJS-$(CONFIG_W64_MUXER) += wavenc.o w64.o OBJS-$(CONFIG_WAV_DEMUXER) += wavdec.o pcm.o diff --git a/libavformat/allformats.c b/libavformat/allformats.c index a4e3822681..006a7dc125 100644 --- a/libavformat/allformats.c +++ b/libavformat/allformats.c @@ -475,6 +475,7 @@ extern const AVInputFormat ff_vpk_demuxer; extern const AVInputFormat ff_vplayer_demuxer; extern const AVInputFormat ff_vqf_demuxer; extern const AVInputFormat ff_vvc_demuxer; +extern const AVOutputFormat ff_vvc_muxer; extern const AVInputFormat ff_w64_demuxer; extern const AVOutputFormat ff_w64_muxer; extern const AVInputFormat ff_wav_demuxer; diff --git a/libavformat/isom.c b/libavformat/isom.c index 6d019881e5..9fbccd4437 100644 --- a/libavformat/isom.c +++ b/libavformat/isom.c @@ -36,6 +36,7 @@ const AVCodecTag ff_mp4_obj_type[] = { { AV_CODEC_ID_MPEG4 , 0x20 }, { AV_CODEC_ID_H264 , 0x21 }, { AV_CODEC_ID_HEVC , 0x23 }, + { AV_CODEC_ID_VVC , 0x33 }, { AV_CODEC_ID_AAC , 0x40 }, { AV_CODEC_ID_MP4ALS , 0x40 }, /* 14496-3 ALS */ { AV_CODEC_ID_MPEG2VIDEO , 0x61 }, /* MPEG-2 Main */ diff --git a/libavformat/isom_tags.c b/libavformat/isom_tags.c index e2b80405cc..ec93bdc363 100644 --- a/libavformat/isom_tags.c +++ b/libavformat/isom_tags.c @@ -123,6 +123,9 @@ const AVCodecTag ff_codec_movvideo_tags[] = { { AV_CODEC_ID_HEVC, MKTAG('d', 'v', 'h', 'e') }, /* HEVC-based Dolby Vision derived from hev1 */ /* dvh1 is handled within mov.c */ + { AV_CODEC_ID_VVC, MKTAG('v', 'v', 'i', '1') }, /* VVC/H.266 which indicates parameter sets may be in ES */ + { AV_CODEC_ID_VVC, MKTAG('v', 'v', 'c', '1') }, /* VVC/H.266 which indicates parameter shall not be in ES */ + { AV_CODEC_ID_H264, MKTAG('a', 'v', 'c', '1') }, /* AVC-1/H.264 */ { AV_CODEC_ID_H264, MKTAG('a', 'v', 'c', '2') }, { AV_CODEC_ID_H264, MKTAG('a', 'v', 'c', '3') }, diff --git a/libavformat/mov.c b/libavformat/mov.c index 1f436e21d6..52719c9d9a 100644 --- a/libavformat/mov.c +++ b/libavformat/mov.c @@ -1962,6 +1962,11 @@ static int mov_read_glbl(MOVContext *c, AVIOContext *pb, MOVAtom atom) if ((uint64_t)atom.size > (1<<30)) return AVERROR_INVALIDDATA; + if (atom.type == MKTAG('v','v','c','C')) { + avio_rb32(pb); + atom.size -= 4; + } + if (atom.size >= 10) { // Broken files created by legacy versions of libavformat will // wrap a whole fiel atom inside of a glbl atom. @@ -7730,6 +7735,7 @@ static const MOVParseTableEntry mov_default_parse_table[] = { { MKTAG('s','g','p','d'), mov_read_sgpd }, { MKTAG('s','b','g','p'), mov_read_sbgp }, { MKTAG('h','v','c','C'), mov_read_glbl }, +{ MKTAG('v','v','c','C'), mov_read_glbl }, { MKTAG('u','u','i','d'), mov_read_uuid }, { MKTAG('C','i','n', 0x8e), mov_read_targa_y216 }, { MKTAG('f','r','e','e'), mov_read_free }, diff --git a/libavformat/movenc.c b/libavformat/movenc.c index 754f95912a..3d412d0079 100644 --- a/libavformat/movenc.c +++ b/libavformat/movenc.c @@ -59,6 +59,7 @@ #include "libavutil/color_utils.h" #include "libavutil/uuid.h" #include "hevc.h" +#include "vvc.h" #include "rtpenc.h" #include "mov_chan.h" #include "movenc_ttml.h" @@ -1392,6 +1393,23 @@ static int mov_write_hvcc_tag(AVIOContext *pb, MOVTrack *track) return update_size(pb, pos); } +static int mov_write_vvcc_tag(AVIOContext *pb, MOVTrack *track) +{ + int64_t pos = avio_tell(pb); + + avio_wb32(pb, 0); + ffio_wfourcc(pb, "vvcC"); + + avio_w8 (pb, 0); /* version */ + avio_wb24(pb, 0); /* flags */ + + if (track->tag == MKTAG('v','v','i','1')) + ff_isom_write_vvcc(pb, track->vos_data, track->vos_len, 1); + else + ff_isom_write_vvcc(pb, track->vos_data, track->vos_len, 0); + return update_size(pb, pos); +} + /* also used by all avid codecs (dv, imx, meridien) and their variants */ static int mov_write_avid_tag(AVIOContext *pb, MOVTrack *track) { @@ -2286,6 +2304,8 @@ static int mov_write_video_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContex avid = 1; } else if (track->par->codec_id == AV_CODEC_ID_HEVC) mov_write_hvcc_tag(pb, track); + else if (track->par->codec_id == AV_CODEC_ID_VVC) + mov_write_vvcc_tag(pb, track); else if (track->par->codec_id == AV_CODEC_ID_H264 && !TAG_IS_AVCI(track->tag)) { mov_write_avcc_tag(pb, track); if (track->mode == MODE_IPOD) @@ -6033,6 +6053,7 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) if ((par->codec_id == AV_CODEC_ID_DNXHD || par->codec_id == AV_CODEC_ID_H264 || par->codec_id == AV_CODEC_ID_HEVC || + par->codec_id == AV_CODEC_ID_VVC || par->codec_id == AV_CODEC_ID_TRUEHD) && !trk->vos_len && !TAG_IS_AVCI(trk->tag)) { /* copy frame to create needed atoms */ @@ -6096,6 +6117,18 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) size = ff_hevc_annexb2mp4(pb, pkt->data, pkt->size, 0, NULL); } } + } else if (par->codec_id == AV_CODEC_ID_VVC && trk->vos_len > 6 && + (AV_RB24(trk->vos_data) == 1 || AV_RB32(trk->vos_data) == 1)) { + /* extradata is Annex B, assume the bitstream is too and convert it */ + if (trk->hint_track >= 0 && trk->hint_track < mov->nb_streams) { + ret = ff_vvc_annexb2mp4_buf(pkt->data, &reformatted_data, + &size, 0, NULL); + if (ret < 0) + return ret; + avio_write(pb, reformatted_data, size); + } else { + size = ff_vvc_annexb2mp4(pb, pkt->data, pkt->size, 0, NULL); + } } else if (par->codec_id == AV_CODEC_ID_AV1) { if (trk->hint_track >= 0 && trk->hint_track < mov->nb_streams) { ret = ff_av1_filter_obus_buf(pkt->data, &reformatted_data, @@ -6142,6 +6175,9 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) } else if(par->codec_id == AV_CODEC_ID_HEVC && par->extradata_size > 21) { int nal_size_length = (par->extradata[21] & 0x3) + 1; ret = ff_mov_cenc_avc_write_nal_units(s, &trk->cenc, nal_size_length, pb, pkt->data, size); + } else if(par->codec_id == AV_CODEC_ID_VVC && par->extradata_size > 21) { + int nal_size_length = (par->extradata[21] & 0x3) + 1; + ret = ff_mov_cenc_avc_write_nal_units(s, &trk->cenc, nal_size_length, pb, pkt->data, size); } else { ret = ff_mov_cenc_write_packet(&trk->cenc, pb, pkt->data, size); } @@ -7224,7 +7260,8 @@ static int mov_init(AVFormatContext *s) if (mov->encryption_scheme == MOV_ENC_CENC_AES_CTR) { ret = ff_mov_cenc_init(&track->cenc, mov->encryption_key, - (track->par->codec_id == AV_CODEC_ID_H264 || track->par->codec_id == AV_CODEC_ID_HEVC), + (track->par->codec_id == AV_CODEC_ID_H264 || track->par->codec_id == AV_CODEC_ID_HEVC || + track->par->codec_id == AV_CODEC_ID_VVC), s->flags & AVFMT_FLAG_BITEXACT); if (ret) return ret; @@ -7692,6 +7729,8 @@ static const AVCodecTag codec_mp4_tags[] = { { AV_CODEC_ID_H264, MKTAG('a', 'v', 'c', '3') }, { AV_CODEC_ID_HEVC, MKTAG('h', 'e', 'v', '1') }, { AV_CODEC_ID_HEVC, MKTAG('h', 'v', 'c', '1') }, + { AV_CODEC_ID_VVC, MKTAG('v', 'v', 'c', '1') }, + { AV_CODEC_ID_VVC, MKTAG('v', 'v', 'i', '1') }, { AV_CODEC_ID_MPEG2VIDEO, MKTAG('m', 'p', '4', 'v') }, { AV_CODEC_ID_MPEG1VIDEO, MKTAG('m', 'p', '4', 'v') }, { AV_CODEC_ID_MJPEG, MKTAG('m', 'p', '4', 'v') }, diff --git a/libavformat/rawenc.c b/libavformat/rawenc.c index 267fce252d..7770d63d10 100644 --- a/libavformat/rawenc.c +++ b/libavformat/rawenc.c @@ -401,6 +401,29 @@ const AVOutputFormat ff_hevc_muxer = { }; #endif +#if CONFIG_VVC_MUXER +static int vvc_check_bitstream(AVFormatContext *s, AVStream *st, + const AVPacket *pkt) +{ + if (pkt->size >= 5 && AV_RB32(pkt->data) != 0x0000001 && + AV_RB24(pkt->data) != 0x000001) + return ff_stream_add_bitstream_filter(st, "vvc_mp4toannexb", NULL); + return 1; +} + +const AVOutputFormat ff_vvc_muxer = { + .name = "vvc", + .long_name = NULL_IF_CONFIG_SMALL("raw VVC video"), + .extensions = "vvc,h266,266", + .audio_codec = AV_CODEC_ID_NONE, + .video_codec = AV_CODEC_ID_VVC, + .init = force_one_stream, + .write_packet = ff_raw_write_packet, + .check_bitstream = vvc_check_bitstream, + .flags = AVFMT_NOTIMESTAMPS, +}; +#endif + #if CONFIG_M4V_MUXER const AVOutputFormat ff_m4v_muxer = { .name = "m4v", From patchwork Wed Oct 19 07:25:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Siedel X-Patchwork-Id: 38820 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp216176pzb; Wed, 19 Oct 2022 00:28:26 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6wLRooag0tn366/s4FnDTq8LELJYLnCQCe6vUr0lheNUJC3bN0C7LMBuE9zlaVGz8AC3VH X-Received: by 2002:a17:907:971e:b0:78d:e7ed:7585 with SMTP id jg30-20020a170907971e00b0078de7ed7585mr5396658ejc.258.1666164506126; Wed, 19 Oct 2022 00:28:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666164506; cv=none; d=google.com; s=arc-20160816; b=MUGMMc+ZUGdzEbEFVgRLSSDHmi27tLj5DivssyqY2J/EXxJcaSRshP9snaGY0UrHM/ 2RTZM+yQ+czCM27Ye/KogqtOUrFBj0AhIkl+e1AM/AfOesa33IyXwphcuB514tt/ejgI UzspTSLbgxp/kigKzNPC5Yv5qqChp9Ud6bZBc/fAMAnTDAcyXnZRlYqJk7GVPWuqpr7B zUzeFlV5Z5oeg3itZIFiTjuMEa9BaRFI1Tb8g9kptKmMseln2bRd071W0Wy9pUh1gzlV omcICR9lxF+l2Y9PPEGJIRG4k7sJNcVVkD1B6m0M/VIckPI0DiQgL3exCZ9ZzjATetCj SoNQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=sAVuk7qSdnbUkiqtyKMvwzQ99eMlI582noA4jnjUWME=; b=HapC8uMaM4SQnK8bVX/y4B5GyHKF1wYt5+VeQrxsH7eS1OcDDPoSdV5Co35m0XZVBg Y4/oPTRUQKfzK4+5dyu6F4JxssC1DO2sKAbrQfKK8nolKP61EwVL7ENvvk4SOhWpDnC3 nc8VF6pl1lRFbpHzQ0yCMHxSJHVr05KmdJeXbU/LrnDG63Ox2tp6LsjQtHjnHg3hWkrj IPSbfivPDOFY0waBIH3MtVA6YwJ090jjGGP9TEEeALWbFJY86ZeDW0d1SQJwDZ+Xh9PK 9ykIpEa9ooM5JBZp9c19P0268xsaespgUR3d7t4wmLDJ+PS+u6ApptTClytpmbEs3H7a 51uA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b="vcVKRZP/"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id m15-20020a1709060d8f00b0073db9474d06si12410768eji.325.2022.10.19.00.28.25; Wed, 19 Oct 2022 00:28:26 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b="vcVKRZP/"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2638668BE0F; Wed, 19 Oct 2022 10:26:43 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f46.google.com (mail-ej1-f46.google.com [209.85.218.46]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5BA2E68BDF0 for ; Wed, 19 Oct 2022 10:26:33 +0300 (EEST) Received: by mail-ej1-f46.google.com with SMTP id d26so37792045eje.10 for ; Wed, 19 Oct 2022 00:26:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spin-digital-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=p4+u9fTv6uyp7C0EE9h76LOBGk31DB8sFpQvQt6hWsU=; b=vcVKRZP/gCNFKALG0hVMskMu8470zfvjD8OMisAdnjN7YPzKHtYM3GGb7ht0mnaRns Pjzv4AEkID6rknbI5X7ZbaztgQHstkpaOz7SM3cbgStVPmOiqaLllWiYNZobC6B0eXMA eVjPl0YdJQwGlUroqAEpxKEMxKZnePCBIaTg0VbRN5ENJeBRolVvAUpScAhtT6qmGniZ bhUhEqU5Y0qIZb+xTRZeKI61BLV7VSR5vzBgdYOUQfDcdsOVyd9yZnXkBF6vhO4I6NoI BR6u7rBFNbQALZ0F+pma29gzLt7f66hxaGWHcBo4VJSQgRhvbQMWFg02ROr4s599WdUg iG0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=p4+u9fTv6uyp7C0EE9h76LOBGk31DB8sFpQvQt6hWsU=; b=oVgwL0gT8sXBrJC1JRo/lOkluK5sXQsGDvVx1NKfsVEch/YrkWTAmbwswH4LCt3ZsL iGlCNhqVeGm+C6sYeyOeYuVObOyGGsxsGQx95vY36ixnH1KyXJ321rUK7Nsdt+g1HuUp PpDQ7H0JY3a82qhE5+4PsQziNEcx7IbJ+OUyXC+zcR+lZp4izzMQW0h7/HOKqW+UhgPr V9coewoUZw95T4xXkrlAHhHS4N5qhI4w1ld4y6kWc75VBGMbuyQjBm7cqp1ezGE0V/8C usN79rUWMkM+1UDS3kXFroiHTlOzydZbT6xNhDJbedoP7QpMqSqDkm/3Qx8+iarqgCtR F8nw== X-Gm-Message-State: ACrzQf3zsbuHmVd4q0ZfU8wztZ2oshNQJPiCzKXLe8QxDHUyRnVqCXsq dfczGoSQlyX7hzU/TCIstUgrte03JOS9Lw== X-Received: by 2002:a17:907:2da6:b0:78d:3cf1:9132 with SMTP id gt38-20020a1709072da600b0078d3cf19132mr5535776ejc.299.1666164391113; Wed, 19 Oct 2022 00:26:31 -0700 (PDT) Received: from thomas-win.localdomain ([213.138.44.237]) by smtp.gmail.com with ESMTPSA id p5-20020a170906498500b0078b83968ad4sm8515150eju.24.2022.10.19.00.26.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Oct 2022 00:26:30 -0700 (PDT) From: thomas.ff@spin-digital.com To: ffmpeg-devel@ffmpeg.org Date: Wed, 19 Oct 2022 09:25:05 +0200 Message-Id: <20221019072508.23460-9-thomas.ff@spin-digital.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221019072508.23460-1-thomas.ff@spin-digital.com> References: <20221019072508.23460-1-thomas.ff@spin-digital.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v1 08/11] avcodec: add external decoder libvvdec for H266/VVC X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Thomas Siedel Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: sLC0xuTbgdwU From: Thomas Siedel Add external decoder VVdeC for H266/VVC decoding. Register new decoder libvvdec Add vvc_parse_extradata to support parse/probe of vvcC stream input Add vvc_paramset that implements the parser of vvcC configuration boxes Add libvvdec to wrap the vvdec interface into ffmpeg libvvdec implements decoder option externmem to use AVBufferPool (default) or copy image after receiving from decoder Enable decoder by adding --enable-libvvdec in configure step Signed-off-by: Thomas Siedel --- configure | 5 + libavcodec/Makefile | 1 + libavcodec/allcodecs.c | 1 + libavcodec/libvvdec.c | 511 ++++++++++++++++ libavcodec/vvc_paramset.c | 972 +++++++++++++++++++++++++++++++ libavcodec/vvc_paramset.h | 429 ++++++++++++++ libavcodec/vvc_parse_extradata.c | 241 ++++++++ libavcodec/vvc_parse_extradata.h | 36 ++ 8 files changed, 2196 insertions(+) create mode 100644 libavcodec/libvvdec.c create mode 100644 libavcodec/vvc_paramset.c create mode 100644 libavcodec/vvc_paramset.h create mode 100644 libavcodec/vvc_parse_extradata.c create mode 100644 libavcodec/vvc_parse_extradata.h diff --git a/configure b/configure index 30df2cb0a4..289473782c 100755 --- a/configure +++ b/configure @@ -288,6 +288,7 @@ External library support: --enable-libvorbis enable Vorbis en/decoding via libvorbis, native implementation exists [no] --enable-libvpx enable VP8 and VP9 de/encoding via libvpx [no] + --enable-libvvdec enable VVC decoding via vvdec [no] --enable-libwebp enable WebP encoding via libwebp [no] --enable-libx264 enable H.264 encoding via x264 [no] --enable-libx265 enable HEVC encoding via x265 [no] @@ -1875,6 +1876,7 @@ EXTERNAL_LIBRARY_LIST=" libvmaf libvorbis libvpx + libvvdec libwebp libxml2 libzimg @@ -3402,6 +3404,8 @@ libvpx_vp8_decoder_deps="libvpx" libvpx_vp8_encoder_deps="libvpx" libvpx_vp9_decoder_deps="libvpx" libvpx_vp9_encoder_deps="libvpx" +libvvdec_decoder_deps="libvvdec" +libvvdec_decoder_select="vvc_mp4toannexb_bsf" libwebp_encoder_deps="libwebp" libwebp_anim_encoder_deps="libwebp" libx262_encoder_deps="libx262" @@ -6731,6 +6735,7 @@ enabled libvpx && { die "libvpx enabled but no supported decoders found" fi } +enabled libvvdec && require_pkg_config libvvdec "libvvdec >= 1.6.0" "vvdec/vvdec.h" vvdec_get_version enabled libwebp && { enabled libwebp_encoder && require_pkg_config libwebp "libwebp >= 0.2.0" webp/encode.h WebPGetEncoderVersion diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 53a8dc67cb..3cc258351c 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1102,6 +1102,7 @@ OBJS-$(CONFIG_LIBVPX_VP8_DECODER) += libvpxdec.o OBJS-$(CONFIG_LIBVPX_VP8_ENCODER) += libvpxenc.o OBJS-$(CONFIG_LIBVPX_VP9_DECODER) += libvpxdec.o libvpx.o OBJS-$(CONFIG_LIBVPX_VP9_ENCODER) += libvpxenc.o libvpx.o +OBJS-$(CONFIG_LIBVVDEC_DECODER) += libvvdec.o vvc_parse_extradata.o vvc_paramset.o OBJS-$(CONFIG_LIBWEBP_ENCODER) += libwebpenc_common.o libwebpenc.o OBJS-$(CONFIG_LIBWEBP_ANIM_ENCODER) += libwebpenc_common.o libwebpenc_animencoder.o OBJS-$(CONFIG_LIBX262_ENCODER) += libx264.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index 46ad3b5a25..58cf993785 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -793,6 +793,7 @@ extern const FFCodec ff_libvpx_vp8_encoder; extern const FFCodec ff_libvpx_vp8_decoder; extern FFCodec ff_libvpx_vp9_encoder; extern FFCodec ff_libvpx_vp9_decoder; +extern const FFCodec ff_libvvdec_decoder; /* preferred over libwebp */ extern const FFCodec ff_libwebp_anim_encoder; extern const FFCodec ff_libwebp_encoder; diff --git a/libavcodec/libvvdec.c b/libavcodec/libvvdec.c new file mode 100644 index 0000000000..28f3a548ca --- /dev/null +++ b/libavcodec/libvvdec.c @@ -0,0 +1,511 @@ +/* + * H.266 decoding using the VVdeC library + * + * Copyright (C) 2022, Thomas Siedel + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "config_components.h" + +#include + +#include "libavutil/common.h" +#include "libavutil/avutil.h" +#include "libavutil/pixdesc.h" +#include "libavutil/opt.h" +#include "libavutil/imgutils.h" +#include "libavutil/frame.h" +#include "libavutil/mastering_display_metadata.h" +#include "libavutil/log.h" + +#include "avcodec.h" +#include "codec_internal.h" +#include "decode.h" +#include "internal.h" +#include "profiles.h" + +#include "vvc_paramset.h" +#include "vvc_parse_extradata.h" + +typedef struct VVdeCContext { + AVClass *av_class; + vvdecDecoder* vvdecDec; + vvdecParams vvdecParams; + VVCParamSets ps; + int is_nalff; + int nal_length_size; + bool bFlush; + AVBufferPool *pools[3]; /** Pools for each data plane. */ + int pool_size[3]; +}VVdeCContext; + + +static void ff_vvdec_log_callback(void *avctx, int level, const char *fmt, va_list args ) +{ + vfprintf( level == 1 ? stderr : stdout, fmt, args ); +} + +static void* ff_vvdec_buffer_allocator( void *ctx, vvdecComponentType comp, uint32_t size, uint32_t alignment, void** allocator ) +{ + AVBufferRef *buf; + VVdeCContext *s; + int plane; + + uint32_t alignedsize = FFALIGN(size, alignment); + s = (VVdeCContext*)ctx; + plane = (int)comp; + + if ( plane < 0 || plane > 3 ) + return NULL; + + if ( alignedsize != s->pool_size[plane]) { + av_buffer_pool_uninit(&s->pools[plane]); + s->pools[plane] = av_buffer_pool_init(alignedsize, NULL); + if (!s->pools[plane]) { + s->pool_size[plane] = 0; + return NULL; + } + s->pool_size[plane] = alignedsize; + } + + buf = av_buffer_pool_get(s->pools[plane]); + if (!buf) + return NULL; + + *allocator = (void*)buf; + return buf->data; +} + +static void ff_vvdec_buffer_unref( void *ctx, void* allocator ) +{ + AVBufferRef *buf = (AVBufferRef*)allocator; + av_buffer_unref( &buf ); +} + +static void ff_vvdec_printParameterInfo( AVCodecContext *avctx, vvdecParams* params ) +{ + av_log(avctx, AV_LOG_DEBUG, "Version info: vvdec %s ( threads %d)\n",vvdec_get_version(), params->threads); +} + +static int ff_vvdec_set_pix_fmt(AVCodecContext *avctx, vvdecFrame* frame ) +{ + if( NULL != frame->picAttributes && NULL != frame->picAttributes->vui && + frame->picAttributes->vui->colourDescriptionPresentFlag ) { + avctx->color_trc = frame->picAttributes->vui->transferCharacteristics; + avctx->color_primaries = frame->picAttributes->vui->colourPrimaries; + avctx->colorspace = frame->picAttributes->vui->matrixCoefficients; + } + else { + avctx->color_primaries = AVCOL_PRI_UNSPECIFIED; + avctx->color_trc = AVCOL_TRC_UNSPECIFIED; + avctx->colorspace = AVCOL_SPC_UNSPECIFIED; + } + + if( NULL != frame->picAttributes && NULL != frame->picAttributes->vui && + frame->picAttributes->vui->videoSignalTypePresentFlag) { + avctx->color_range = frame->picAttributes->vui->videoFullRangeFlag ? + AVCOL_RANGE_JPEG : AVCOL_RANGE_MPEG; + } + else { + avctx->color_range = AVCOL_RANGE_MPEG; + } + + switch ( frame->colorFormat ) { + case VVDEC_CF_YUV420_PLANAR: + if (frame->bitDepth == 8) { + avctx->pix_fmt = frame->numPlanes == 1 ? + AV_PIX_FMT_GRAY8 : AV_PIX_FMT_YUV420P; + avctx->profile = FF_PROFILE_VVC_MAIN_10; + return 0; + } + else if (frame->bitDepth == 10) { + avctx->pix_fmt = frame->numPlanes == 1 ? + AV_PIX_FMT_GRAY10 : AV_PIX_FMT_YUV420P10; + avctx->profile = FF_PROFILE_VVC_MAIN_10; + return 0; + } + else { + return AVERROR_INVALIDDATA; + } + default: + return AVERROR_INVALIDDATA; + } +} + +static int set_side_data( AVCodecContext *avctx, AVFrame *avframe, vvdecFrame *frame ) +{ + vvdecSEI *sei; + VVdeCContext *s = (VVdeCContext*)avctx->priv_data; + + sei = vvdec_find_frame_sei( s->vvdecDec, VVDEC_MASTERING_DISPLAY_COLOUR_VOLUME, frame ); + if( sei ) { + // VVC uses a g,b,r ordering, which we convert to a more natural r,g,b + const int mapping[3] = {2, 0, 1}; + const int chroma_den = 50000; + const int luma_den = 10000; + int i; + vvdecSEIMasteringDisplayColourVolume* p; + AVMasteringDisplayMetadata *metadata = av_mastering_display_metadata_create_side_data(avframe); + p = (vvdecSEIMasteringDisplayColourVolume *)(sei->payload); + if ( p && metadata ) { + for (i = 0; i < 3; i++) { + const int j = mapping[i]; + metadata->display_primaries[i][0].num = p->primaries[j][0]; + metadata->display_primaries[i][0].den = chroma_den; + metadata->display_primaries[i][1].num = p->primaries[j][1]; + metadata->display_primaries[i][1].den = chroma_den; + } + metadata->white_point[0].num = p->whitePoint[0]; + metadata->white_point[0].den = chroma_den; + metadata->white_point[1].num = p->whitePoint[1]; + metadata->white_point[1].den = chroma_den; + + metadata->max_luminance.num = p->maxLuminance; + metadata->max_luminance.den = luma_den; + metadata->min_luminance.num = p->minLuminance; + metadata->min_luminance.den = luma_den; + metadata->has_luminance = 1; + metadata->has_primaries = 1; + + av_log(avctx, AV_LOG_DEBUG, "Mastering Display Metadata:\n"); + av_log(avctx, AV_LOG_DEBUG, + "r(%5.4f,%5.4f) g(%5.4f,%5.4f) b(%5.4f %5.4f) wp(%5.4f, %5.4f)\n", + av_q2d(metadata->display_primaries[0][0]), + av_q2d(metadata->display_primaries[0][1]), + av_q2d(metadata->display_primaries[1][0]), + av_q2d(metadata->display_primaries[1][1]), + av_q2d(metadata->display_primaries[2][0]), + av_q2d(metadata->display_primaries[2][1]), + av_q2d(metadata->white_point[0]), av_q2d(metadata->white_point[1])); + av_log(avctx, AV_LOG_DEBUG, + "min_luminance=%f, max_luminance=%f\n", + av_q2d(metadata->min_luminance), av_q2d(metadata->max_luminance)); + } + return 0; + } + + sei = vvdec_find_frame_sei( s->vvdecDec, VVDEC_CONTENT_LIGHT_LEVEL_INFO, frame ); + if( sei ) { + vvdecSEIContentLightLevelInfo* p = NULL; + AVContentLightMetadata *light = av_content_light_metadata_create_side_data(avframe); + p = (vvdecSEIContentLightLevelInfo *)(sei->payload); + if ( p && light) { + light->MaxCLL = p->maxContentLightLevel; + light->MaxFALL = p->maxPicAverageLightLevel; + } + + av_log(avctx, AV_LOG_DEBUG, "Content Light Level Metadata:\n"); + av_log(avctx, AV_LOG_DEBUG, "MaxCLL=%d, MaxFALL=%d\n", + light->MaxCLL, light->MaxFALL); + } + + return 0; +} + +static void export_stream_params( AVCodecContext *avctx, const VVCSPS *sps) +{ + avctx->coded_width = sps->pic_width_max_in_luma_samples; + avctx->coded_height = sps->pic_height_max_in_luma_samples; + avctx->width = sps->pic_width_max_in_luma_samples - + sps->conf_win_left_offset - + sps->conf_win_right_offset; + avctx->height = sps->pic_height_max_in_luma_samples - + sps->conf_win_top_offset - + sps->conf_win_bottom_offset; + avctx->has_b_frames = sps->max_sublayers; + avctx->profile = sps->profile_tier_level.general_profile_idc; + avctx->level = sps->profile_tier_level.general_level_idc; + avctx->pix_fmt = sps->pix_fmt; + + avctx->color_range = sps->vui.full_range_flag ? AVCOL_RANGE_JPEG : AVCOL_RANGE_MPEG; + + if (sps->vui.colour_description_present_flag) { + avctx->color_primaries = sps->vui.colour_primaries; + avctx->color_trc = sps->vui.transfer_characteristics; + avctx->colorspace = sps->vui.matrix_coeffs; + } else { + avctx->color_primaries = AVCOL_PRI_UNSPECIFIED; + avctx->color_trc = AVCOL_TRC_UNSPECIFIED; + avctx->colorspace = AVCOL_SPC_UNSPECIFIED; + } + + avctx->chroma_sample_location = AVCHROMA_LOC_UNSPECIFIED; + if (sps->chroma_format_idc == 1) { + if (sps->vui.chroma_loc_info_present_flag) { + if (sps->vui.chroma_sample_loc_type_top_field <= 5) + avctx->chroma_sample_location = sps->vui.chroma_sample_loc_type_top_field + 1; + } else + avctx->chroma_sample_location = AVCHROMA_LOC_LEFT; + } + + if (sps->timing_hrd_params_present_flag && + sps->general_timing_hrd_parameters.num_units_in_tick && + sps->general_timing_hrd_parameters.time_scale) { + av_reduce(&avctx->framerate.den, &avctx->framerate.num, + sps->general_timing_hrd_parameters.num_units_in_tick, + sps->general_timing_hrd_parameters.time_scale, INT_MAX); + } +} + +static int vvc_decode_extradata( AVCodecContext *avctx, uint8_t *buf, int length, int first) +{ + VVdeCContext *s = (VVdeCContext*)avctx->priv_data; + int ret; + + ret = ff_vvc_decode_extradata(buf, length, &s->ps, &s->is_nalff, + &s->nal_length_size, avctx->err_recognition, + false, avctx); + if (ret < 0) + return ret; + + if ( s->ps.sps != NULL ) + export_stream_params(avctx, s->ps.sps); + + return 0; +} + +static av_cold int ff_vvdec_decode_init(AVCodecContext *avctx) +{ + int i; + VVdeCContext *s = (VVdeCContext*)avctx->priv_data; + + vvdec_params_default( &s->vvdecParams ); + s->vvdecParams.logLevel = VVDEC_DETAILS; + + if ( av_log_get_level() >= AV_LOG_DEBUG ) s->vvdecParams.logLevel = VVDEC_DETAILS; + else if( av_log_get_level() >= AV_LOG_VERBOSE ) s->vvdecParams.logLevel = VVDEC_INFO; // VVDEC_INFO will output per picture info + else if( av_log_get_level() >= AV_LOG_INFO ) s->vvdecParams.logLevel = VVDEC_WARNING; // AV_LOG_INFO is ffmpeg default + else s->vvdecParams.logLevel = VVDEC_SILENT; + + if( avctx->thread_count > 0 ) + s->vvdecParams.threads = avctx->thread_count; // number of worker threads (should not exceed the number of physical cpu's) + else + s->vvdecParams.threads = -1; // get max cpus + + ff_vvdec_printParameterInfo( avctx, &s->vvdecParams ); + + // using buffer allocation by using AVBufferPool + s->vvdecParams.opaque = avctx->priv_data; + s->vvdecDec = vvdec_decoder_open_with_allocator( &s->vvdecParams, + ff_vvdec_buffer_allocator, ff_vvdec_buffer_unref ); + + + if( !s->vvdecDec ) { + av_log(avctx, AV_LOG_ERROR, "cannot init vvc decoder\n" ); + return -1; + } + + vvdec_set_logging_callback( s->vvdecDec, ff_vvdec_log_callback ); + + s->bFlush = false; + s->is_nalff = 0; + s->nal_length_size = 0; + + for (i = 0; i < FF_ARRAY_ELEMS(s->pools); i++) { + s->pools[i] = NULL; + s->pool_size[i] = 0; + } + + if (!avctx->internal->is_copy) { + if (avctx->extradata_size > 0 && avctx->extradata) { + int ret = vvc_decode_extradata(avctx, avctx->extradata, avctx->extradata_size, 1); + if (ret < 0) { + return ret; + } + } + } + + return 0; +} + +static av_cold int ff_vvdec_decode_close(AVCodecContext *avctx) +{ + VVdeCContext *s = (VVdeCContext*)avctx->priv_data; + + for (int i = 0; i < FF_ARRAY_ELEMS(s->pools); i++) { + av_buffer_pool_uninit(&s->pools[i]); + s->pool_size[i] = 0; + } + + if( 0 != vvdec_decoder_close(s->vvdecDec) ) { + av_log(avctx, AV_LOG_ERROR, "cannot close vvdec\n" ); + return -1; + } + + ff_vvc_ps_uninit( &s->ps ); + s->bFlush = false; + return 0; +} + +static av_cold int ff_vvdec_decode_frame( AVCodecContext *avctx, AVFrame *data, + int *got_frame, AVPacket *avpkt ) +{ + VVdeCContext *s = avctx->priv_data; + AVFrame *avframe = data; + + int ret = 0; + vvdecFrame *frame = NULL; + + if ( avframe ) { + if( !avpkt->size && !s->bFlush ) + s->bFlush = true; + + if( s->bFlush ) + ret = vvdec_flush( s->vvdecDec, &frame ); + else { + vvdecAccessUnit accessUnit; + vvdec_accessUnit_default( &accessUnit ); + accessUnit.payload = avpkt->data; + accessUnit.payloadSize = avpkt->size; + accessUnit.payloadUsedSize = avpkt->size; + + accessUnit.cts = avpkt->pts; accessUnit.ctsValid = true; + accessUnit.dts = avpkt->dts; accessUnit.dtsValid = true; + + ret = vvdec_decode( s->vvdecDec, &accessUnit, &frame ); + } + + if( ret < 0 ) { + if( ret == VVDEC_EOF ) + s->bFlush = true; + else if ( ret != VVDEC_TRY_AGAIN ) { + av_log(avctx, AV_LOG_ERROR, "error in vvdec::decode - ret:%d - %s\n", ret, vvdec_get_last_error(s->vvdecDec)); + return AVERROR(EINVAL); \ + } + } + else if( NULL != frame ) { + const uint8_t * src_data[4] = { frame->planes[0].ptr, frame->planes[1].ptr, frame->planes[2].ptr, NULL }; + const int src_linesizes[4] = { (int)frame->planes[0].stride, (int)frame->planes[1].stride, (int)frame->planes[2].stride, 0 }; + + if (( ret = ff_vvdec_set_pix_fmt(avctx, frame)) < 0) { + av_log(avctx, AV_LOG_ERROR, "Unsupported output colorspace (%d) / bit_depth (%d)\n", + frame->colorFormat, frame->bitDepth); + goto fail; + } + + if( avctx->pix_fmt != AV_PIX_FMT_YUV420P && avctx->pix_fmt != AV_PIX_FMT_YUV420P10LE ) { + av_log(avctx, AV_LOG_ERROR, "Unsupported output colorspace (%d) / bit_depth (%d)\n", + frame->colorFormat, frame->bitDepth ); + ret = AVERROR_INVALIDDATA; + goto fail; + } + + if ((int)frame->width != avctx->width || (int)frame->height != avctx->height) { + av_log(avctx, AV_LOG_INFO, "dimension change! %dx%d -> %dx%d\n", + avctx->width, avctx->height, frame->width, frame->height); + + ret = ff_set_dimensions(avctx, frame->width, frame->height); + if (ret < 0) + goto fail; + } + + if( frame->planes[0].allocator ) + avframe->buf[0] = av_buffer_ref( (AVBufferRef *)frame->planes[0].allocator); + if( frame->planes[1].allocator ) + avframe->buf[1] = av_buffer_ref( (AVBufferRef *)frame->planes[1].allocator); + if( frame->planes[2].allocator ) + avframe->buf[2] = av_buffer_ref( (AVBufferRef *)frame->planes[2].allocator); + + for (int i = 0; i < 4; i++) { + avframe->data[i] = (uint8_t *)src_data[i]; + avframe->linesize[i] = src_linesizes[i]; + } + + ret = ff_decode_frame_props(avctx, avframe); + if (ret < 0) + goto fail; + + if( frame->picAttributes ) { + avframe->key_frame = frame->picAttributes->isRefPic; + avframe->pict_type = (frame->picAttributes->sliceType != VVDEC_SLICETYPE_UNKNOWN) ? + frame->picAttributes->sliceType+1 : AV_PICTURE_TYPE_NONE; + } + + ret = set_side_data(avctx, avframe, frame ); + if (ret < 0) + goto fail; + + if( 0 != vvdec_frame_unref( s->vvdecDec, frame ) ) + av_log(avctx, AV_LOG_ERROR, "cannot free picture memory\n"); + + *got_frame = 1; + } + } + + return avpkt->size; + +fail: + if (frame) { + if( frame->planes[0].allocator ) + av_buffer_unref( (AVBufferRef **)&frame->planes[0].allocator ); + if( frame->planes[1].allocator ) + av_buffer_unref( (AVBufferRef **)&frame->planes[1].allocator ); + if( frame->planes[2].allocator ) + av_buffer_unref( (AVBufferRef **)&frame->planes[2].allocator ); + + vvdec_frame_unref( s->vvdecDec, frame ); + } + return ret; +} + +static av_cold void ff_vvdec_decode_flush(AVCodecContext *avctx) +{ + VVdeCContext *s = (VVdeCContext*)avctx->priv_data; + + if( 0 != vvdec_decoder_close(s->vvdecDec) ) + av_log(avctx, AV_LOG_ERROR, "cannot close vvdec during flush\n" ); + + s->vvdecDec = vvdec_decoder_open_with_allocator( &s->vvdecParams, ff_vvdec_buffer_allocator, ff_vvdec_buffer_unref ); + if( !s->vvdecDec ) + av_log(avctx, AV_LOG_ERROR, "cannot reinit vvdec during flush\n" ); + + vvdec_set_logging_callback( s->vvdecDec, ff_vvdec_log_callback ); + + s->bFlush = false; +} + +static const enum AVPixelFormat pix_fmts_vvc[] = { + AV_PIX_FMT_YUV420P, + AV_PIX_FMT_YUV420P10LE, + AV_PIX_FMT_NONE +}; + +static const AVClass class_libvvdec = { + .class_name = "libvvdec-vvc decoder", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, +}; + +FFCodec ff_libvvdec_decoder = { + .p.name = "libvvdec", + .p.long_name = "H.266 / VVC Decoder VVdeC", + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_VVC, + .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_OTHER_THREADS, + .p.profiles = NULL_IF_CONFIG_SMALL(ff_vvc_profiles), + .p.priv_class = &class_libvvdec, + .p.wrapper_name = "libvvdec", + .priv_data_size = sizeof(VVdeCContext), + .p.pix_fmts = pix_fmts_vvc, + .init = ff_vvdec_decode_init, + FF_CODEC_DECODE_CB(ff_vvdec_decode_frame), + .close = ff_vvdec_decode_close, + .flush = ff_vvdec_decode_flush, + .bsfs = "vvc_mp4toannexb", + .caps_internal = FF_CODEC_CAP_AUTO_THREADS, +}; diff --git a/libavcodec/vvc_paramset.c b/libavcodec/vvc_paramset.c new file mode 100644 index 0000000000..a016f57a37 --- /dev/null +++ b/libavcodec/vvc_paramset.c @@ -0,0 +1,972 @@ +/* + * VVC Parameter Set decoding + * + * Copyright (c) 2022, Thomas Siedel + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/imgutils.h" +#include "golomb.h" +#include "vvc_paramset.h" + +static void remove_sps(VVCParamSets *s, int id) +{ + if (s->sps_list[id]) { + if (s->sps == (const VVCSPS*)s->sps_list[id]->data) + s->sps = NULL; + + av_assert0(!(s->sps_list[id] && s->sps == (VVCSPS*)s->sps_list[id]->data)); + } + av_buffer_unref(&s->sps_list[id]); +} + +static int decode_general_constraints_info(GetBitContext *gb, AVCodecContext *avctx, + VVCGeneralConstraintsInfo *gci) +{ + int i; + gci->gci_present_flag = get_bits1(gb); + + if (gci->gci_present_flag) { + /* general */ + gci->gci_intra_only_constraint_flag = get_bits1(gb); + gci->gci_all_layers_independent_constraint_flag = get_bits1(gb); + gci->gci_one_au_only_constraint_flag = get_bits1(gb); + + /* picture format */ + gci->gci_sixteen_minus_max_bitdepth_constraint_idc = get_bits(gb,4); + gci->gci_three_minus_max_chroma_format_constraint_idc = get_bits(gb,2); + + /* NAL unit type related */ + gci->gci_no_mixed_nalu_types_in_pic_constraint_flag = get_bits1(gb); + gci->gci_no_trail_constraint_flag = get_bits1(gb); + gci->gci_no_stsa_constraint_flag = get_bits1(gb); + gci->gci_no_rasl_constraint_flag = get_bits1(gb); + gci->gci_no_radl_constraint_flag = get_bits1(gb); + gci->gci_no_idr_constraint_flag = get_bits1(gb); + gci->gci_no_cra_constraint_flag = get_bits1(gb); + gci->gci_no_gdr_constraint_flag = get_bits1(gb); + gci->gci_no_aps_constraint_flag = get_bits1(gb); + gci->gci_no_idr_rpl_constraint_flag = get_bits1(gb); + + /* tile, slice, subpicture partitioning */ + gci->gci_one_tile_per_pic_constraint_flag = get_bits1(gb); + gci->gci_pic_header_in_slice_header_constraint_flag = get_bits1(gb); + gci->gci_one_slice_per_pic_constraint_flag = get_bits1(gb); + gci->gci_no_rectangular_slice_constraint_flag = get_bits1(gb); + gci->gci_one_slice_per_subpic_constraint_flag = get_bits1(gb); + gci->gci_no_subpic_info_constraint_flag = get_bits1(gb); + + /* CTU and block partitioning */ + gci->gci_three_minus_max_log2_ctu_size_constraint_idc = get_bits(gb,2); + gci->gci_no_partition_constraints_override_constraint_flag = get_bits1(gb); + gci->gci_no_mtt_constraint_flag = get_bits1(gb); + gci->gci_no_qtbtt_dual_tree_intra_constraint_flag = get_bits1(gb); + + /* intra */ + gci->gci_no_palette_constraint_flag = get_bits1(gb); + gci->gci_no_ibc_constraint_flag = get_bits1(gb); + gci->gci_no_isp_constraint_flag = get_bits1(gb); + gci->gci_no_mrl_constraint_flag = get_bits1(gb); + gci->gci_no_mip_constraint_flag = get_bits1(gb); + gci->gci_no_cclm_constraint_flag = get_bits1(gb); + + /* inter */ + gci->gci_no_ref_pic_resampling_constraint_flag = get_bits1(gb); + gci->gci_no_res_change_in_clvs_constraint_flag = get_bits1(gb); + gci->gci_no_weighted_prediction_constraint_flag = get_bits1(gb); + gci->gci_no_ref_wraparound_constraint_flag = get_bits1(gb); + gci->gci_no_temporal_mvp_constraint_flag = get_bits1(gb); + gci->gci_no_sbtmvp_constraint_flag = get_bits1(gb); + gci->gci_no_amvr_constraint_flag = get_bits1(gb); + gci->gci_no_bdof_constraint_flag = get_bits1(gb); + gci->gci_no_smvd_constraint_flag = get_bits1(gb); + gci->gci_no_dmvr_constraint_flag = get_bits1(gb); + gci->gci_no_mmvd_constraint_flag = get_bits1(gb); + gci->gci_no_affine_motion_constraint_flag = get_bits1(gb); + gci->gci_no_prof_constraint_flag = get_bits1(gb); + gci->gci_no_bcw_constraint_flag = get_bits1(gb); + gci->gci_no_ciip_constraint_flag = get_bits1(gb); + gci->gci_no_gpm_constraint_flag = get_bits1(gb); + + /* transform, quantization, residual */ + gci->gci_no_luma_transform_size_64_constraint_flag = get_bits1(gb); + gci->gci_no_transform_skip_constraint_flag = get_bits1(gb); + gci->gci_no_bdpcm_constraint_flag = get_bits1(gb); + gci->gci_no_mts_constraint_flag = get_bits1(gb); + gci->gci_no_lfnst_constraint_flag = get_bits1(gb); + gci->gci_no_joint_cbcr_constraint_flag = get_bits1(gb); + gci->gci_no_sbt_constraint_flag = get_bits1(gb); + gci->gci_no_act_constraint_flag = get_bits1(gb); + gci->gci_no_explicit_scaling_list_constraint_flag = get_bits1(gb); + gci->gci_no_dep_quant_constraint_flag = get_bits1(gb); + gci->gci_no_sign_data_hiding_constraint_flag = get_bits1(gb); + gci->gci_no_cu_qp_delta_constraint_flag = get_bits1(gb); + gci->gci_no_chroma_qp_offset_constraint_flag = get_bits1(gb); + + /* loop filter */ + gci->gci_no_sao_constraint_flag = get_bits1(gb); + gci->gci_no_alf_constraint_flag = get_bits1(gb); + gci->gci_no_ccalf_constraint_flag = get_bits1(gb); + gci->gci_no_lmcs_constraint_flag = get_bits1(gb); + gci->gci_no_ladf_constraint_flag = get_bits1(gb); + gci->gci_no_virtual_boundaries_constraint_flag = get_bits1(gb); + gci->gci_num_reserved_bits = get_bits(gb,8); + for (i = 0; i < gci->gci_num_reserved_bits; i++) { + gci->gci_reserved_zero_bit[i] = get_bits1(gb); + } + } + + align_get_bits(gb); + + return 0; +} + + +static int decode_profile_tier_level(GetBitContext *gb, AVCodecContext *avctx, + VVCProfileTierLevel *ptl, + int profile_tier_present_flag, + int max_num_sub_layers_minus1 ) +{ + int i; + + if (profile_tier_present_flag) { + ptl->general_profile_idc = get_bits(gb, 7); + ptl->general_tier_flag = get_bits1(gb); + } + ptl->general_level_idc = get_bits(gb, 8); + ptl->ptl_frame_only_constraint_flag = get_bits1(gb); + ptl->ptl_multilayer_enabled_flag = get_bits1(gb); + + if (profile_tier_present_flag) { + decode_general_constraints_info(gb, avctx, &ptl->general_constraints_info); + } + + for (i = max_num_sub_layers_minus1 - 1; i >= 0; i--) { + ptl->ptl_sublayer_level_present_flag[i] = get_bits1(gb); + } + + align_get_bits(gb); + + for (i = max_num_sub_layers_minus1 - 1; i >= 0; i--) { + if (ptl->ptl_sublayer_level_present_flag[i]) + ptl->sublayer_level_idc[i] = get_bits(gb, 8); + } + + if (profile_tier_present_flag) + { + ptl->ptl_num_sub_profiles = get_bits(gb, 8); + for (i = 0; i < ptl->ptl_num_sub_profiles; i++) + ptl->general_sub_profile_idc[i] = get_bits_long(gb, 32); + } + + return 0; +} + +static int decode_dpb_parameters(GetBitContext *gb, AVCodecContext *avctx, + VVCDpbParameters *dpb, + uint8_t max_sublayers_minus1, + uint8_t sublayer_info_flag) +{ + int i; + for (i = (sublayer_info_flag ? 0 : max_sublayers_minus1); + i <= max_sublayers_minus1; i++) { + dpb->dpb_max_dec_pic_buffering_minus1[i] = get_ue_golomb(gb); + dpb->dpb_max_num_reorder_pics[i] = get_ue_golomb(gb); + dpb->dpb_max_latency_increase_plus1[i] = get_ue_golomb(gb); + } + return 0; +} + +static int decode_ref_pic_list_struct(GetBitContext *gb, AVCodecContext *avctx, + VVCRefPicListStruct *current, + uint8_t list_idx, uint8_t rpls_idx, + const VVCSPS *sps) +{ + int i, j, num_direct_ref_layers = 0; + + current->num_ref_entries = get_ue_golomb( gb); + if (sps->long_term_ref_pics_flag && + rpls_idx < sps->num_ref_pic_lists[list_idx] && + current->num_ref_entries > 0) + current->ltrp_in_header_flag = get_bits1(gb); + if (sps->long_term_ref_pics_flag && + rpls_idx == sps->num_ref_pic_lists[list_idx]) + current->ltrp_in_header_flag = 1; + for (i = 0, j = 0; i < current->num_ref_entries; i++) { + if (sps->inter_layer_prediction_enabled_flag) + current->inter_layer_ref_pic_flag[i] = get_bits1(gb); + else + current->inter_layer_ref_pic_flag[i] = 0; + + if (!current->inter_layer_ref_pic_flag[i]) { + if (sps->long_term_ref_pics_flag) + current->st_ref_pic_flag[i] = get_bits1(gb); + else + current->st_ref_pic_flag[i] = 1; + if (current->st_ref_pic_flag[i]) { + int abs_delta_poc_st; + current->abs_delta_poc_st[i] = get_ue_golomb( gb); + if ((sps->weighted_pred_flag || + sps->weighted_bipred_flag) && i != 0) + abs_delta_poc_st = current->abs_delta_poc_st[i]; + else + abs_delta_poc_st = current->abs_delta_poc_st[i] + 1; + if (abs_delta_poc_st > 0) + current->strp_entry_sign_flag[i] = get_bits1(gb); + } else { + if (!current->ltrp_in_header_flag) { + uint8_t bits = sps->log2_max_pic_order_cnt_lsb_minus4 + 4; + current->rpls_poc_lsb_lt[j] = get_bits(gb, bits ); + j++; + } + } + } else { + if (num_direct_ref_layers == 0) { + av_log(avctx, AV_LOG_ERROR, + "num_direct_ref_layers needs > 0.\n"); + return AVERROR_INVALIDDATA; + } + current->ilrp_idx[i] = get_ue_golomb( gb); + } + } + return 0; +} + +static int decode_general_timing_hrd_parameters(GetBitContext *gb, VVCGeneralTimingHrdParameters *current) +{ + current->num_units_in_tick = get_bits_long(gb,32); + current->time_scale = get_bits_long(gb,32); + current->general_nal_hrd_params_present_flag = get_bits1(gb); + current->general_vcl_hrd_params_present_flag = get_bits1(gb); + + if (current->general_nal_hrd_params_present_flag || + current->general_vcl_hrd_params_present_flag) { + current->general_same_pic_timing_in_all_ols_flag = get_bits1(gb); + current->general_du_hrd_params_present_flag = get_bits1(gb); + if (current->general_du_hrd_params_present_flag) + current->tick_divisor_minus2 = get_bits(gb,8); + current->bit_rate_scale = get_bits(gb,4); + current->cpb_size_scale = get_bits(gb,4); + if (current->general_du_hrd_params_present_flag) + current->cpb_size_du_scale = get_bits(gb,4); + current->hrd_cpb_cnt_minus1 = get_ue_golomb_long(gb); + } else { + current->general_du_hrd_params_present_flag = 0; + } + return 0; +} + +static int decode_sublayer_hrd_parameters(GetBitContext *gb, + VVCSubLayerHRDParameters *current, int sublayer_id, + const VVCGeneralTimingHrdParameters *general) +{ + int i; + for (i = 0; i <= general->hrd_cpb_cnt_minus1; i++) { + current->bit_rate_value_minus1[sublayer_id][i] = get_ue_golomb_long(gb); + current->cpb_size_value_minus1[sublayer_id][i] = get_ue_golomb_long(gb); + if (general->general_du_hrd_params_present_flag) { + current->cpb_size_du_value_minus1[sublayer_id][i] = get_ue_golomb_long(gb); + current->bit_rate_du_value_minus1[sublayer_id][i] = get_ue_golomb_long(gb); + } + current->cbr_flag[sublayer_id][i] = get_bits1(gb); + } + return 0; +} + +static int decode_ols_timing_hrd_parameters( GetBitContext *gb, + VVCOlsTimingHrdParameters *current, + uint8_t first_sublayer, uint8_t max_sublayers_minus1, + const VVCGeneralTimingHrdParameters *general) +{ + int i; + for (i = first_sublayer; i <= max_sublayers_minus1; i++) { + current->fixed_pic_rate_general_flag[i] = get_bits1(gb); + if (!current->fixed_pic_rate_general_flag[i]) + current->fixed_pic_rate_within_cvs_flag[i] = get_bits1(gb); + else + current->fixed_pic_rate_within_cvs_flag[i] = 1; + if (current->fixed_pic_rate_within_cvs_flag[i]) { + current->elemental_duration_in_tc_minus1[i] = get_ue_golomb_long(gb); + current->low_delay_hrd_flag[i] = 0; + } else if ((general->general_nal_hrd_params_present_flag || + general->general_vcl_hrd_params_present_flag) && + general->hrd_cpb_cnt_minus1 == 0) { + current->low_delay_hrd_flag[i] = get_bits1(gb); + } else { + current->low_delay_hrd_flag[i] = 0; + } + if (general->general_nal_hrd_params_present_flag) + decode_sublayer_hrd_parameters(gb, + ¤t->nal_sub_layer_hrd_parameters, + i, general); + if (general->general_vcl_hrd_params_present_flag) + decode_sublayer_hrd_parameters(gb, + ¤t->nal_sub_layer_hrd_parameters, i, general); + } + return 0; +} + +static int decode_vui(GetBitContext *gb, AVCodecContext *avctx, + VVCVUI* vui, uint8_t chroma_format_idc ) +{ + vui->progressive_source_flag = get_bits1(gb); + vui->interlaced_source_flag = get_bits1(gb); + vui->non_packed_constraint_flag = get_bits1(gb); + vui->non_projected_constraint_flag = get_bits1(gb); + vui->aspect_ratio_info_present_flag = get_bits1(gb); + if (vui->aspect_ratio_info_present_flag) { + vui->aspect_ratio_constant_flag = get_bits1(gb); + vui->aspect_ratio_idc = get_bits(gb, 8); + if (vui->aspect_ratio_idc == 255) { + vui->sar_width = get_bits(gb, 16); + vui->sar_height = get_bits(gb, 16); + } + } else { + vui->aspect_ratio_constant_flag = 0; + vui->aspect_ratio_idc = 0; + } + vui->overscan_info_present_flag = get_bits1(gb); + if (vui->overscan_info_present_flag) + vui->overscan_appropriate_flag = get_bits1(gb); + vui->colour_description_present_flag = get_bits1(gb); + if (vui->colour_description_present_flag) { + vui->colour_primaries = get_bits(gb, 8); + vui->transfer_characteristics = get_bits(gb, 8); + vui->matrix_coeffs = get_bits(gb, 8); + av_log(avctx, AV_LOG_DEBUG, "colour_primaries: %d transfer_characteristics: %d matrix_coeffs: %d \n", + vui->colour_primaries, vui->transfer_characteristics, vui->matrix_coeffs); + + vui->full_range_flag = get_bits1(gb); + } else { + vui->colour_primaries = 2; + vui->transfer_characteristics = 2; + vui->matrix_coeffs = 2; + vui->full_range_flag = 0; + } + vui->chroma_loc_info_present_flag = get_bits1(gb); + if (chroma_format_idc != 1 && vui->chroma_loc_info_present_flag) { + av_log(avctx, AV_LOG_ERROR, "chroma_format_idc == %d," + "chroma_loc_info_present_flag can't not be true", chroma_format_idc); + return AVERROR_INVALIDDATA; + } + if (vui->chroma_loc_info_present_flag) { + if (vui->progressive_source_flag && + !vui->interlaced_source_flag) { + vui->chroma_sample_loc_type_frame = get_ue_golomb(gb); + } else { + vui->chroma_sample_loc_type_top_field = get_ue_golomb(gb); + vui->chroma_sample_loc_type_bottom_field = get_ue_golomb(gb); + } + } else { + if (chroma_format_idc == 1) { + vui->chroma_sample_loc_type_frame = get_ue_golomb(gb); + vui->chroma_sample_loc_type_top_field = get_ue_golomb(gb); + vui->chroma_sample_loc_type_bottom_field = get_ue_golomb(gb); + } + } + return 0; +} + +static int map_pixel_format(AVCodecContext *avctx, VVCSPS *sps) +{ + const AVPixFmtDescriptor *desc; + switch (sps->bit_depth) { + case 8: + if (sps->chroma_format_idc == 0) sps->pix_fmt = AV_PIX_FMT_GRAY8; + if (sps->chroma_format_idc == 1) sps->pix_fmt = AV_PIX_FMT_YUV420P; + if (sps->chroma_format_idc == 2) sps->pix_fmt = AV_PIX_FMT_YUV422P; + if (sps->chroma_format_idc == 3) sps->pix_fmt = AV_PIX_FMT_YUV444P; + break; + case 9: + if (sps->chroma_format_idc == 0) sps->pix_fmt = AV_PIX_FMT_GRAY9; + if (sps->chroma_format_idc == 1) sps->pix_fmt = AV_PIX_FMT_YUV420P9; + if (sps->chroma_format_idc == 2) sps->pix_fmt = AV_PIX_FMT_YUV422P9; + if (sps->chroma_format_idc == 3) sps->pix_fmt = AV_PIX_FMT_YUV444P9; + break; + case 10: + if (sps->chroma_format_idc == 0) sps->pix_fmt = AV_PIX_FMT_GRAY10; + if (sps->chroma_format_idc == 1) sps->pix_fmt = AV_PIX_FMT_YUV420P10; + if (sps->chroma_format_idc == 2) sps->pix_fmt = AV_PIX_FMT_YUV422P10; + if (sps->chroma_format_idc == 3) sps->pix_fmt = AV_PIX_FMT_YUV444P10; + break; + case 12: + if (sps->chroma_format_idc == 0) sps->pix_fmt = AV_PIX_FMT_GRAY12; + if (sps->chroma_format_idc == 1) sps->pix_fmt = AV_PIX_FMT_YUV420P12; + if (sps->chroma_format_idc == 2) sps->pix_fmt = AV_PIX_FMT_YUV422P12; + if (sps->chroma_format_idc == 3) sps->pix_fmt = AV_PIX_FMT_YUV444P12; + break; + default: + av_log(avctx, AV_LOG_ERROR, + "The following bit-depths are currently specified: 8, 9, 10 and 12 bits, " + "chroma_format_idc is %d, depth is %d\n", + sps->chroma_format_idc, sps->bit_depth); + return AVERROR_INVALIDDATA; + } + + desc = av_pix_fmt_desc_get(sps->pix_fmt); + if (!desc) + return AVERROR(EINVAL); + + return 0; +} + +int ff_vvc_parse_sps(VVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, + int apply_defdispwin, AVCodecContext *avctx) +{ + int i, j; + unsigned int ctb_log2_size_y, ctb_size_y, max_num_merge_cand, + tmp_width_val, tmp_height_val; + + sps->sps_id = get_bits(gb, 4); + sps->vps_id = get_bits(gb, 4); + + *sps_id = sps->sps_id; + + sps->max_sublayers = get_bits(gb, 3) + 1; + sps->chroma_format_idc = get_bits(gb, 2); + sps->log2_ctu_size = get_bits(gb, 2) + 5; + + ctb_log2_size_y = sps->log2_ctu_size; + ctb_size_y = 1 << ctb_log2_size_y; + + sps->ptl_dpb_hrd_params_present_flag = get_bits1(gb); + if (sps->ptl_dpb_hrd_params_present_flag) + decode_profile_tier_level( gb, avctx, &sps->profile_tier_level, 1, sps->max_sublayers-1 ); + + sps->gdr_enabled_flag = get_bits1(gb); + sps->ref_pic_resampling_enabled_flag = get_bits1(gb); + + if (sps->ref_pic_resampling_enabled_flag) + sps->res_change_in_clvs_allowed_flag = get_bits1(gb); + else + sps->res_change_in_clvs_allowed_flag = 0; + + sps->pic_width_max_in_luma_samples = get_ue_golomb(gb); + sps->pic_height_max_in_luma_samples = get_ue_golomb(gb); + + sps->conformance_window_flag = get_bits1(gb); + + if (sps->conformance_window_flag) { + sps->conf_win_left_offset = get_ue_golomb(gb); + sps->conf_win_right_offset = get_ue_golomb(gb); + sps->conf_win_top_offset = get_ue_golomb(gb); + sps->conf_win_bottom_offset = get_ue_golomb(gb); + } else { + sps->conf_win_left_offset = 0; + sps->conf_win_right_offset = 0; + sps->conf_win_top_offset = 0; + sps->conf_win_bottom_offset = 0; + } + + tmp_width_val = (sps->pic_width_max_in_luma_samples + ctb_size_y -1)/ctb_size_y; + tmp_height_val = (sps->pic_height_max_in_luma_samples + ctb_size_y -1)/ctb_size_y; + + sps->subpic_info_present_flag = get_bits1(gb); + if (sps->subpic_info_present_flag) { + sps->num_subpics_minus1 = get_ue_golomb(gb); + if (sps->num_subpics_minus1 > 0) { + sps->independent_subpics_flag = get_bits1(gb); + sps->subpic_same_size_flag = get_bits1(gb); + } + } + + if (sps->num_subpics_minus1 > 0) { + int wlen = av_ceil_log2(tmp_width_val); + int hlen = av_ceil_log2(tmp_height_val); + if (sps->pic_width_max_in_luma_samples > ctb_size_y) + sps->subpic_width_minus1[0] = get_bits(gb, wlen ); + else + sps->subpic_width_minus1[0] = tmp_width_val - 1; + + if (sps->pic_height_max_in_luma_samples > ctb_size_y) + sps->subpic_height_minus1[0] = get_bits(gb, hlen ); + else + sps->subpic_height_minus1[0] = tmp_height_val; + + if (!sps->independent_subpics_flag) { + sps->subpic_treated_as_pic_flag[0] = get_bits1(gb); + sps->loop_filter_across_subpic_enabled_flag[0] = get_bits1(gb); + } else { + sps->subpic_treated_as_pic_flag[0] = 1; + sps->loop_filter_across_subpic_enabled_flag[0] = 1; + } + + for (i = 1; i <= sps->num_subpics_minus1; i++) { + if (!sps->subpic_same_size_flag) { + if (sps->pic_width_max_in_luma_samples > ctb_size_y) + sps->subpic_ctu_top_left_x[i] = get_bits(gb, wlen ); + else + sps->subpic_ctu_top_left_x[i] = 0; + + if (sps->pic_height_max_in_luma_samples > ctb_size_y) + sps->subpic_ctu_top_left_y[i] = get_bits(gb, hlen ); + else + sps->subpic_ctu_top_left_y[i] = 0; + + if (i < sps->num_subpics_minus1 && + sps->pic_width_max_in_luma_samples > ctb_size_y) { + sps->subpic_width_minus1[i] = get_bits(gb, wlen ); + } else { + sps->subpic_width_minus1[i] = tmp_width_val - sps->subpic_ctu_top_left_x[i] - 1; + } + if (i < sps->num_subpics_minus1 && + sps->pic_height_max_in_luma_samples > ctb_size_y) { + sps->subpic_height_minus1[i] = get_bits(gb, hlen ); + } else { + sps->subpic_height_minus1[i] = tmp_height_val - sps->subpic_ctu_top_left_y[i] - 1; + } + } else { + int num_subpic_cols = + tmp_width_val / (sps->subpic_width_minus1[0] + 1); + sps->subpic_ctu_top_left_x[i] = (i % num_subpic_cols) *(sps->subpic_width_minus1[0] + 1); + sps->subpic_ctu_top_left_y[i] = (i / num_subpic_cols) *(sps->subpic_height_minus1[0] + 1); + sps->subpic_width_minus1[i] = sps->subpic_width_minus1[0]; + sps->subpic_height_minus1[i] = sps->subpic_height_minus1[0]; + } + if (!sps->independent_subpics_flag) { + sps->subpic_treated_as_pic_flag[i] = get_bits1(gb); + sps->loop_filter_across_subpic_enabled_flag[i] = get_bits1(gb); + } else { + sps->subpic_treated_as_pic_flag[i] = 1; + sps->loop_filter_across_subpic_enabled_flag[i] = 0; + } + } + sps->subpic_id_len_minus1 = get_ue_golomb(gb ); + + if ((1 << (sps->subpic_id_len_minus1 + 1)) < + sps->num_subpics_minus1 + 1) { + av_log(avctx, AV_LOG_ERROR, + "sps->subpic_id_len_minus1(%d) is too small\n", + sps->subpic_id_len_minus1); + return AVERROR_INVALIDDATA; + } + sps->subpic_id_mapping_explicitly_signalled_flag = get_bits1(gb); + if (sps->subpic_id_mapping_explicitly_signalled_flag) { + sps->subpic_id_mapping_present_flag = get_bits1(gb); + if (sps->subpic_id_mapping_present_flag) { + for (i = 0; i <= sps->num_subpics_minus1; i++) + sps->subpic_id[i] = get_bits(gb, sps->subpic_id_len_minus1 + 1 ); + } + } else { + sps->subpic_ctu_top_left_x[0]= 0; + sps->subpic_ctu_top_left_y[0]= 0; + sps->subpic_width_minus1[0] = tmp_width_val - 1; + sps->subpic_height_minus1[0]= tmp_height_val - 1; + } + } else { + sps->num_subpics_minus1 = 0; + sps->independent_subpics_flag = 1; + sps->subpic_same_size_flag = 0; + sps->subpic_id_mapping_explicitly_signalled_flag= 0; + sps->subpic_ctu_top_left_x[0] = 0; + sps->subpic_ctu_top_left_y[0] = 0; + sps->subpic_width_minus1[0] = tmp_width_val - 1; + sps->subpic_height_minus1[0] = tmp_height_val - 1; + } + + sps->bit_depth = get_ue_golomb(gb ) + 8; + + sps->entropy_coding_sync_enabled_flag = get_bits1(gb); + sps->entry_point_offsets_present_flag = get_bits1(gb); + + sps->log2_max_pic_order_cnt_lsb_minus4 = get_bits(gb, 4 ); + + sps->poc_msb_cycle_flag = get_bits1(gb); + if (sps->poc_msb_cycle_flag) + sps->poc_msb_cycle_len_minus1 = get_ue_golomb(gb); + + sps->num_extra_ph_bytes = get_bits(gb,2); + + for (i = 0; i < FFMIN(16,(sps->num_extra_ph_bytes * 8)); i++) { + sps->extra_ph_bit_present_flag[i] = get_bits1(gb); + } + + sps->num_extra_sh_bytes = get_bits(gb,2); + for (i = 0; i < FFMIN(16,(sps->num_extra_sh_bytes * 8)); i++) { + sps->extra_sh_bit_present_flag[i] = get_bits1(gb); + } + + if (sps->ptl_dpb_hrd_params_present_flag) { + if (sps->max_sublayers > 1) + sps->sublayer_dpb_params_flag = get_bits1(gb); + else + sps->sublayer_dpb_params_flag = 0; + + decode_dpb_parameters( gb, avctx, &sps->dpb_params, + sps->max_sublayers-1, + sps->sublayer_dpb_params_flag); + } + + sps->log2_min_luma_coding_block_size_minus2 = get_ue_golomb(gb); + sps->partition_constraints_override_enabled_flag = get_bits1(gb); + sps->log2_diff_min_qt_min_cb_intra_slice_luma = get_ue_golomb(gb); + sps->max_mtt_hierarchy_depth_intra_slice_luma = get_ue_golomb(gb); + + if (sps->max_mtt_hierarchy_depth_intra_slice_luma != 0) { + sps->log2_diff_max_bt_min_qt_intra_slice_luma = get_ue_golomb(gb); + sps->log2_diff_max_tt_min_qt_intra_slice_luma = get_ue_golomb(gb); + } else { + sps->log2_diff_max_bt_min_qt_intra_slice_luma = 0; + sps->log2_diff_max_tt_min_qt_intra_slice_luma = 0; + } + + if (sps->chroma_format_idc != 0) { + sps->qtbtt_dual_tree_intra_flag = get_bits1(gb); + } else { + sps->qtbtt_dual_tree_intra_flag = 0; + } + + if (sps->qtbtt_dual_tree_intra_flag) { + sps->log2_diff_min_qt_min_cb_intra_slice_chroma = get_ue_golomb(gb); + sps->max_mtt_hierarchy_depth_intra_slice_chroma = get_ue_golomb(gb); + if (sps->max_mtt_hierarchy_depth_intra_slice_chroma != 0) { + sps->log2_diff_max_bt_min_qt_intra_slice_chroma = get_ue_golomb(gb); + sps->log2_diff_max_tt_min_qt_intra_slice_chroma = get_ue_golomb(gb); + } + } else { + sps->log2_diff_min_qt_min_cb_intra_slice_chroma = 0; + sps->max_mtt_hierarchy_depth_intra_slice_chroma = 0; + } + if (sps->max_mtt_hierarchy_depth_intra_slice_chroma == 0) { + sps->log2_diff_max_bt_min_qt_intra_slice_chroma = 0; + sps->log2_diff_max_tt_min_qt_intra_slice_chroma = 0; + } + + sps->log2_diff_min_qt_min_cb_inter_slice = get_ue_golomb(gb); + + sps->max_mtt_hierarchy_depth_inter_slice = get_ue_golomb(gb); + if (sps->max_mtt_hierarchy_depth_inter_slice != 0) { + sps->log2_diff_max_bt_min_qt_inter_slice = get_ue_golomb(gb); + sps->log2_diff_max_tt_min_qt_inter_slice = get_ue_golomb(gb); + } else { + sps->log2_diff_max_bt_min_qt_inter_slice = 0; + sps->log2_diff_max_tt_min_qt_inter_slice = 0; + } + + if (ctb_size_y > 32) + sps->max_luma_transform_size_64_flag = get_bits1(gb); + else + sps->max_luma_transform_size_64_flag = 0; + + sps->transform_skip_enabled_flag = get_bits1(gb); + if (sps->transform_skip_enabled_flag) { + sps->log2_transform_skip_max_size_minus2 = get_ue_golomb(gb); + sps->bdpcm_enabled_flag = get_bits1(gb); + } + + sps->mts_enabled_flag = get_bits1(gb); + if (sps->mts_enabled_flag) { + sps->explicit_mts_intra_enabled_flag = get_bits1(gb); + sps->explicit_mts_inter_enabled_flag = get_bits1(gb); + } else { + sps->explicit_mts_intra_enabled_flag = 0; + sps->explicit_mts_inter_enabled_flag = 0; + } + + sps->lfnst_enabled_flag = get_bits1(gb); + + if (sps->chroma_format_idc != 0) { + uint8_t num_qp_tables; + sps->joint_cbcr_enabled_flag = get_bits1(gb); + sps->same_qp_table_for_chroma_flag = get_bits1(gb); + num_qp_tables = sps->same_qp_table_for_chroma_flag ? + 1 : (sps->joint_cbcr_enabled_flag ? 3 : 2); + for (i = 0; i < num_qp_tables; i++) { + sps->qp_table_start_minus26[i] = get_se_golomb(gb); + sps->num_points_in_qp_table_minus1[i] = get_ue_golomb(gb); + for (j = 0; j <= sps->num_points_in_qp_table_minus1[i]; j++) { + sps->delta_qp_in_val_minus1[i][j] = get_ue_golomb(gb); + sps->delta_qp_diff_val[i][j] = get_ue_golomb(gb); + } + } + } else { + sps->joint_cbcr_enabled_flag = 0; + sps->same_qp_table_for_chroma_flag = 0; + } + + sps->sao_enabled_flag = get_bits1(gb); + sps->alf_enabled_flag = get_bits1(gb); + if (sps->alf_enabled_flag && sps->chroma_format_idc) + sps->ccalf_enabled_flag = get_bits1(gb); + else + sps->ccalf_enabled_flag = 0; + + sps->lmcs_enabled_flag = get_bits1(gb); + sps->weighted_pred_flag = get_bits1(gb); + sps->weighted_bipred_flag = get_bits1(gb); + sps->long_term_ref_pics_flag = get_bits1(gb); + if (sps->vps_id > 0) + sps->inter_layer_prediction_enabled_flag = get_bits1(gb); + else + sps->inter_layer_prediction_enabled_flag = 0; + sps->idr_rpl_present_flag = get_bits1(gb); + sps->rpl1_same_as_rpl0_flag = get_bits1(gb); + + for (i = 0; i < (sps->rpl1_same_as_rpl0_flag ? 1 : 2); i++) { + sps->num_ref_pic_lists[i] = get_ue_golomb(gb); + for (j = 0; j < sps->num_ref_pic_lists[i]; j++) + decode_ref_pic_list_struct( gb, avctx, + &sps->ref_pic_list_struct[i][j], + i, j, sps); + } + + if (sps->rpl1_same_as_rpl0_flag) { + sps->num_ref_pic_lists[1] = sps->num_ref_pic_lists[0]; + for (j = 0; j < sps->num_ref_pic_lists[0]; j++) + memcpy(&sps->ref_pic_list_struct[1][j], + &sps->ref_pic_list_struct[0][j], + sizeof(sps->ref_pic_list_struct[0][j])); + } + + sps->ref_wraparound_enabled_flag = get_bits1(gb); + + sps->temporal_mvp_enabled_flag = get_bits1(gb); + if (sps->temporal_mvp_enabled_flag) + sps->sbtmvp_enabled_flag = get_bits1(gb); + else + sps->sbtmvp_enabled_flag = 0; + + sps->amvr_enabled_flag = get_bits1(gb); + sps->bdof_enabled_flag = get_bits1(gb); + if (sps->bdof_enabled_flag) + sps->bdof_control_present_in_ph_flag = get_bits1(gb); + else + sps->bdof_control_present_in_ph_flag = 0; + + sps->smvd_enabled_flag = get_bits1(gb); + sps->dmvr_enabled_flag = get_bits1(gb); + if (sps->dmvr_enabled_flag) + sps->dmvr_control_present_in_ph_flag = get_bits1(gb); + else + sps->dmvr_control_present_in_ph_flag = 0; + + sps->mmvd_enabled_flag = get_bits1(gb); + if (sps->mmvd_enabled_flag) + sps->mmvd_fullpel_only_enabled_flag = get_bits1(gb); + else + sps->mmvd_fullpel_only_enabled_flag = 0; + + sps->six_minus_max_num_merge_cand = get_ue_golomb(gb); + max_num_merge_cand = 6 - sps->six_minus_max_num_merge_cand; + + sps->sbt_enabled_flag = get_bits1(gb); + + sps->affine_enabled_flag = get_bits1(gb); + if (sps->affine_enabled_flag) { + sps->five_minus_max_num_subblock_merge_cand = get_ue_golomb(gb); + sps->param_affine_enabled_flag = get_bits1(gb); + if (sps->amvr_enabled_flag) + sps->affine_amvr_enabled_flag = get_bits1(gb); + else + sps->affine_amvr_enabled_flag = 0; + sps->affine_prof_enabled_flag = get_bits1(gb); + if (sps->affine_prof_enabled_flag) + sps->prof_control_present_in_ph_flag = get_bits1(gb); + else + sps->prof_control_present_in_ph_flag = 0; + } else { + sps->param_affine_enabled_flag = 0; + sps->affine_amvr_enabled_flag = 0; + sps->affine_prof_enabled_flag = 0; + sps->prof_control_present_in_ph_flag = 0; + } + + sps->bcw_enabled_flag = get_bits1(gb); + sps->ciip_enabled_flag = get_bits1(gb); + + if (max_num_merge_cand >= 2) { + sps->gpm_enabled_flag = get_bits1(gb); + if (sps->gpm_enabled_flag && max_num_merge_cand >= 3) + sps->max_num_merge_cand_minus_max_num_gpm_cand = get_ue_golomb(gb); + } else { + sps->gpm_enabled_flag = 0; + } + + sps->log2_parallel_merge_level_minus2 = get_ue_golomb(gb); + + sps->isp_enabled_flag = get_bits1(gb); + sps->mrl_enabled_flag = get_bits1(gb); + sps->mip_enabled_flag = get_bits1(gb); + + if (sps->chroma_format_idc != 0) + sps->cclm_enabled_flag = get_bits1(gb); + else + sps->cclm_enabled_flag = 0; + if (sps->chroma_format_idc == 1) { + sps->chroma_horizontal_collocated_flag = get_bits1(gb); + sps->chroma_vertical_collocated_flag = get_bits1(gb); + } else { + sps->chroma_horizontal_collocated_flag = 0; + sps->chroma_vertical_collocated_flag = 0; + } + + sps->palette_enabled_flag = get_bits1(gb); + if (sps->chroma_format_idc == 3 && + !sps->max_luma_transform_size_64_flag) + sps->act_enabled_flag = get_bits1(gb); + else + sps->act_enabled_flag = 0; + if (sps->transform_skip_enabled_flag || + sps->palette_enabled_flag) + sps->min_qp_prime_ts = get_ue_golomb(gb); + + sps->ibc_enabled_flag = get_bits1(gb); + if (sps->ibc_enabled_flag) + sps->six_minus_max_num_ibc_merge_cand = get_ue_golomb(gb); + + sps->ladf_enabled_flag = get_bits1(gb); + if (sps->ladf_enabled_flag) { + sps->num_ladf_intervals_minus2 = get_bits( gb, 2 ); + sps->ladf_lowest_interval_qp_offset = get_se_golomb(gb); + for (i = 0; i < sps->num_ladf_intervals_minus2 + 1; i++) { + sps->ladf_qp_offset[i] = get_se_golomb(gb); + sps->ladf_delta_threshold_minus1[i] = get_ue_golomb(gb); + } + } + + sps->explicit_scaling_list_enabled_flag = get_bits1(gb); + if (sps->lfnst_enabled_flag && + sps->explicit_scaling_list_enabled_flag) + sps->scaling_matrix_for_lfnst_disabled_flag = get_bits1(gb); + + if (sps->act_enabled_flag && + sps->explicit_scaling_list_enabled_flag) + sps->scaling_matrix_for_alternative_colour_space_disabled_flag = get_bits1(gb); + else + sps->scaling_matrix_for_alternative_colour_space_disabled_flag = 0; + if (sps->scaling_matrix_for_alternative_colour_space_disabled_flag) + sps->scaling_matrix_designated_colour_space_flag = get_bits1(gb); + + sps->dep_quant_enabled_flag = get_bits1(gb); + sps->sign_data_hiding_enabled_flag = get_bits1(gb); + + sps->virtual_boundaries_enabled_flag = get_bits1(gb); + if (sps->virtual_boundaries_enabled_flag) { + sps->virtual_boundaries_present_flag = get_bits1(gb); + if (sps->virtual_boundaries_present_flag) { + sps->num_ver_virtual_boundaries = get_ue_golomb(gb); + for (i = 0; i < sps->num_ver_virtual_boundaries; i++) + sps->virtual_boundary_pos_x_minus1[i] = get_ue_golomb(gb); + for (i = 0; i < sps->num_hor_virtual_boundaries; i++) + sps->virtual_boundary_pos_y_minus1[i] = get_ue_golomb(gb); + } + } else { + sps->virtual_boundaries_present_flag = 0; + sps->num_ver_virtual_boundaries = 0; + sps->num_hor_virtual_boundaries = 0; + } + + if (sps->ptl_dpb_hrd_params_present_flag) { + sps->timing_hrd_params_present_flag = get_bits1(gb); + if (sps->timing_hrd_params_present_flag) { + uint8_t first_sublayer; + decode_general_timing_hrd_parameters(gb, &sps->general_timing_hrd_parameters); + + if (sps->max_sublayers > 1) + sps->sublayer_cpb_params_present_flag = get_bits1(gb); + else + sps->sublayer_cpb_params_present_flag = 0; + + first_sublayer = sps->sublayer_cpb_params_present_flag ? + 0 : sps->max_sublayers-1; + + decode_ols_timing_hrd_parameters ( gb, + &sps->ols_timing_hrd_parameters, first_sublayer, + sps->max_sublayers-1, + &sps->general_timing_hrd_parameters); + } + } + + sps->field_seq_flag = get_bits1(gb); + sps->vui_parameters_present_flag = get_bits1(gb); + if (sps->vui_parameters_present_flag) { + sps->vui_payload_size_minus1 = get_ue_golomb(gb); + align_get_bits(gb); + decode_vui( gb, avctx, &sps->vui, sps->chroma_format_idc); + } + + sps->extension_flag = get_bits1(gb); + // TODO: parse sps extension flag and read extension data + + map_pixel_format( avctx, sps); + + return 0; +} + +int ff_vvc_decode_nal_sps( GetBitContext *gb, AVCodecContext *avctx, + VVCParamSets *ps, int apply_defdispwin) +{ + unsigned int sps_id; + int ret; + VVCSPS *sps; + AVBufferRef *sps_buf = av_buffer_allocz(sizeof(*sps)); + + if (!sps_buf) + return AVERROR(ENOMEM); + sps = (VVCSPS*)sps_buf->data; + + ret = ff_vvc_parse_sps(sps, gb, &sps_id, + apply_defdispwin, + avctx); + if (ret < 0) { + av_buffer_unref(&sps_buf); + return ret; + } + + if (avctx->debug & FF_DEBUG_BITSTREAM) + { + av_log(avctx, AV_LOG_DEBUG, + "Parsed SPS: id %d; coded wxh: %dx%d; " + "pix_fmt: %s.\n", + sps->sps_id, sps->pic_width_max_in_luma_samples, sps->pic_height_max_in_luma_samples, + av_get_pix_fmt_name(sps->pix_fmt)); + } + + /* check if this is a repeat of an already parsed SPS, then keep the + * original one otherwise drop all PPSes that depend on it (PPS,VPS not implemented yet) */ + if (ps->sps_list[sps_id] && + !memcmp(ps->sps_list[sps_id]->data, sps_buf->data, sps_buf->size)) { + av_buffer_unref(&sps_buf); + } else { + remove_sps(ps, sps_id); + ps->sps_list[sps_id] = sps_buf; + ps->sps = (VVCSPS*)ps->sps_list[sps_id]->data; + } + + // TODO: read PPS flag and data + + return 0; +} + +void ff_vvc_ps_uninit(VVCParamSets *ps) +{ + int i; + + for (i = 0; i < FF_ARRAY_ELEMS(ps->sps_list); i++) + av_buffer_unref(&ps->sps_list[i]); + + // TODO: if PPS, VPS is implemended it must be unrefed as well + // for (i = 0; i < FF_ARRAY_ELEMS(ps->vps_list); i++) + // av_buffer_unref(&ps->vps_list[i]); + // for (i = 0; i < FF_ARRAY_ELEMS(ps->pps_list); i++) + // av_buffer_unref(&ps->pps_list[i]); + + ps->sps = NULL; + //ps->pps = NULL; + //ps->vps = NULL; +} diff --git a/libavcodec/vvc_paramset.h b/libavcodec/vvc_paramset.h new file mode 100644 index 0000000000..c4bb2e509b --- /dev/null +++ b/libavcodec/vvc_paramset.h @@ -0,0 +1,429 @@ +/* + * VVC parameter set parsing + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_VVC_PARAMSET_H +#define AVCODEC_VVC_PARAMSET_H + +#include + +#include "libavutil/buffer.h" +#include "libavutil/pixfmt.h" +#include "libavutil/rational.h" + +#include "avcodec.h" +#include "get_bits.h" +#include "vvc.h" +#include "cbs_h266.h" + + +typedef struct VVCGeneralConstraintsInfo { + uint8_t gci_present_flag; + /* general */ + uint8_t gci_intra_only_constraint_flag; + uint8_t gci_all_layers_independent_constraint_flag; + uint8_t gci_one_au_only_constraint_flag; + + /* picture format */ + uint8_t gci_sixteen_minus_max_bitdepth_constraint_idc; + uint8_t gci_three_minus_max_chroma_format_constraint_idc; + + /* NAL unit type related */ + uint8_t gci_no_mixed_nalu_types_in_pic_constraint_flag; + uint8_t gci_no_trail_constraint_flag; + uint8_t gci_no_stsa_constraint_flag; + uint8_t gci_no_rasl_constraint_flag; + uint8_t gci_no_radl_constraint_flag; + uint8_t gci_no_idr_constraint_flag; + uint8_t gci_no_cra_constraint_flag; + uint8_t gci_no_gdr_constraint_flag; + uint8_t gci_no_aps_constraint_flag; + uint8_t gci_no_idr_rpl_constraint_flag; + + /* tile, slice, subpicture partitioning */ + uint8_t gci_one_tile_per_pic_constraint_flag; + uint8_t gci_pic_header_in_slice_header_constraint_flag; + uint8_t gci_one_slice_per_pic_constraint_flag; + uint8_t gci_no_rectangular_slice_constraint_flag; + uint8_t gci_one_slice_per_subpic_constraint_flag; + uint8_t gci_no_subpic_info_constraint_flag; + + /* CTU and block partitioning */ + uint8_t gci_three_minus_max_log2_ctu_size_constraint_idc; + uint8_t gci_no_partition_constraints_override_constraint_flag; + uint8_t gci_no_mtt_constraint_flag; + uint8_t gci_no_qtbtt_dual_tree_intra_constraint_flag; + + /* intra */ + uint8_t gci_no_palette_constraint_flag; + uint8_t gci_no_ibc_constraint_flag; + uint8_t gci_no_isp_constraint_flag; + uint8_t gci_no_mrl_constraint_flag; + uint8_t gci_no_mip_constraint_flag; + uint8_t gci_no_cclm_constraint_flag; + + /* inter */ + uint8_t gci_no_ref_pic_resampling_constraint_flag; + uint8_t gci_no_res_change_in_clvs_constraint_flag;; + uint8_t gci_no_weighted_prediction_constraint_flag; + uint8_t gci_no_ref_wraparound_constraint_flag; + uint8_t gci_no_temporal_mvp_constraint_flag; + uint8_t gci_no_sbtmvp_constraint_flag; + uint8_t gci_no_amvr_constraint_flag; + uint8_t gci_no_bdof_constraint_flag; + uint8_t gci_no_smvd_constraint_flag; + uint8_t gci_no_dmvr_constraint_flag; + uint8_t gci_no_mmvd_constraint_flag; + uint8_t gci_no_affine_motion_constraint_flag; + uint8_t gci_no_prof_constraint_flag; + uint8_t gci_no_bcw_constraint_flag; + uint8_t gci_no_ciip_constraint_flag; + uint8_t gci_no_gpm_constraint_flag; + + /* transform, quantization, residual */ + uint8_t gci_no_luma_transform_size_64_constraint_flag; + uint8_t gci_no_transform_skip_constraint_flag; + uint8_t gci_no_bdpcm_constraint_flag; + uint8_t gci_no_mts_constraint_flag; + uint8_t gci_no_lfnst_constraint_flag; + uint8_t gci_no_joint_cbcr_constraint_flag; + uint8_t gci_no_sbt_constraint_flag; + uint8_t gci_no_act_constraint_flag; + uint8_t gci_no_explicit_scaling_list_constraint_flag; + uint8_t gci_no_dep_quant_constraint_flag; + uint8_t gci_no_sign_data_hiding_constraint_flag; + uint8_t gci_no_cu_qp_delta_constraint_flag; + uint8_t gci_no_chroma_qp_offset_constraint_flag; + + /* loop filter */ + uint8_t gci_no_sao_constraint_flag; + uint8_t gci_no_alf_constraint_flag; + uint8_t gci_no_ccalf_constraint_flag; + uint8_t gci_no_lmcs_constraint_flag; + uint8_t gci_no_ladf_constraint_flag; + uint8_t gci_no_virtual_boundaries_constraint_flag; + uint8_t gci_num_reserved_bits; + uint8_t gci_reserved_zero_bit[255]; +} VVCGeneralConstraintsInfo; + +typedef struct VVCProfileTierLevel { + uint8_t general_profile_idc; + uint8_t general_tier_flag; + uint8_t general_level_idc; + uint8_t ptl_frame_only_constraint_flag; + uint8_t ptl_multilayer_enabled_flag; + VVCGeneralConstraintsInfo general_constraints_info; + uint8_t ptl_sublayer_level_present_flag[VVC_MAX_SUBLAYERS - 1]; + uint8_t sublayer_level_idc[VVC_MAX_SUBLAYERS - 1]; + uint8_t ptl_num_sub_profiles; + uint32_t general_sub_profile_idc[VVC_MAX_SUB_PROFILES]; + + uint8_t ptl_reserved_zero_bit; +} VVCProfileTierLevel; + +typedef struct VVCDpbParameters { + uint8_t dpb_max_dec_pic_buffering_minus1[VVC_MAX_SUBLAYERS]; + uint8_t dpb_max_num_reorder_pics[VVC_MAX_SUBLAYERS]; + uint8_t dpb_max_latency_increase_plus1[VVC_MAX_SUBLAYERS]; +} VVCDpbParameters; + +typedef struct VVCRefPicListStruct { + uint8_t num_ref_entries; + uint8_t ltrp_in_header_flag; + uint8_t inter_layer_ref_pic_flag[VVC_MAX_REF_ENTRIES]; + uint8_t st_ref_pic_flag[VVC_MAX_REF_ENTRIES]; + uint8_t abs_delta_poc_st[VVC_MAX_REF_ENTRIES]; + uint8_t strp_entry_sign_flag[VVC_MAX_REF_ENTRIES]; + uint8_t rpls_poc_lsb_lt[VVC_MAX_REF_ENTRIES]; + uint8_t ilrp_idx[VVC_MAX_REF_ENTRIES]; +} VVCRefPicListStruct; + +typedef struct VVCGeneralTimingHrdParameters { + uint32_t num_units_in_tick; + uint32_t time_scale; + uint8_t general_nal_hrd_params_present_flag; + uint8_t general_vcl_hrd_params_present_flag; + uint8_t general_same_pic_timing_in_all_ols_flag; + uint8_t general_du_hrd_params_present_flag; + uint8_t tick_divisor_minus2; + uint8_t bit_rate_scale; + uint8_t cpb_size_scale; + uint8_t cpb_size_du_scale; + uint8_t hrd_cpb_cnt_minus1; +} VVCGeneralTimingHrdParameters; + +typedef struct VVCSubLayerHRDParameters { + uint32_t bit_rate_value_minus1[VVC_MAX_SUBLAYERS][VVC_MAX_CPB_CNT]; + uint32_t cpb_size_value_minus1[VVC_MAX_SUBLAYERS][VVC_MAX_CPB_CNT]; + uint32_t cpb_size_du_value_minus1[VVC_MAX_SUBLAYERS][VVC_MAX_CPB_CNT]; + uint32_t bit_rate_du_value_minus1[VVC_MAX_SUBLAYERS][VVC_MAX_CPB_CNT]; + uint8_t cbr_flag[VVC_MAX_SUBLAYERS][VVC_MAX_CPB_CNT]; +} VVCSubLayerHRDParameters; + +typedef struct VVCOlsTimingHrdParameters { + uint8_t fixed_pic_rate_general_flag[VVC_MAX_SUBLAYERS]; + uint8_t fixed_pic_rate_within_cvs_flag[VVC_MAX_SUBLAYERS]; + uint16_t elemental_duration_in_tc_minus1[VVC_MAX_SUBLAYERS]; + uint8_t low_delay_hrd_flag[VVC_MAX_SUBLAYERS]; + VVCSubLayerHRDParameters nal_sub_layer_hrd_parameters; + VVCSubLayerHRDParameters vcl_sub_layer_hrd_parameters; +} VVCOlsTimingHrdParameters; + + +typedef struct VVCVUI { + uint8_t progressive_source_flag; + uint8_t interlaced_source_flag; + uint8_t non_packed_constraint_flag; + uint8_t non_projected_constraint_flag; + + uint8_t aspect_ratio_info_present_flag; + uint8_t aspect_ratio_constant_flag; + uint8_t aspect_ratio_idc; + + uint16_t sar_width; + uint16_t sar_height;; + + uint8_t overscan_info_present_flag; + uint8_t overscan_appropriate_flag; + + uint8_t colour_description_present_flag; + uint8_t colour_primaries; + + uint8_t transfer_characteristics; + uint8_t matrix_coeffs; + uint8_t full_range_flag; + + uint8_t chroma_loc_info_present_flag; + uint8_t chroma_sample_loc_type_frame; + uint8_t chroma_sample_loc_type_top_field; + uint8_t chroma_sample_loc_type_bottom_field; + //VVCExtensionData extension_data; +} VVCVUI; + +typedef struct VVCSPS { + uint8_t sps_id; + uint8_t vps_id; + uint8_t max_sublayers; + uint8_t chroma_format_idc; + uint8_t log2_ctu_size; + uint8_t ptl_dpb_hrd_params_present_flag; + VVCProfileTierLevel profile_tier_level; + uint8_t gdr_enabled_flag; + uint8_t ref_pic_resampling_enabled_flag; + uint8_t res_change_in_clvs_allowed_flag; + + uint16_t pic_width_max_in_luma_samples; + uint16_t pic_height_max_in_luma_samples; + + uint8_t conformance_window_flag; + uint16_t conf_win_left_offset; + uint16_t conf_win_right_offset; + uint16_t conf_win_top_offset; + uint16_t conf_win_bottom_offset; + + uint8_t subpic_info_present_flag; + uint16_t num_subpics_minus1; + uint8_t independent_subpics_flag; + uint8_t subpic_same_size_flag; + uint16_t subpic_ctu_top_left_x[VVC_MAX_SLICES]; + uint16_t subpic_ctu_top_left_y[VVC_MAX_SLICES]; + uint16_t subpic_width_minus1[VVC_MAX_SLICES]; + uint16_t subpic_height_minus1[VVC_MAX_SLICES]; + uint8_t subpic_treated_as_pic_flag[VVC_MAX_SLICES]; + uint8_t loop_filter_across_subpic_enabled_flag[VVC_MAX_SLICES]; + uint8_t subpic_id_len_minus1; + uint8_t subpic_id_mapping_explicitly_signalled_flag; + uint8_t subpic_id_mapping_present_flag; + uint32_t subpic_id[VVC_MAX_SLICES]; + + + uint8_t bit_depth; + uint8_t entropy_coding_sync_enabled_flag; + uint8_t entry_point_offsets_present_flag; + + uint8_t log2_max_pic_order_cnt_lsb_minus4; + uint8_t poc_msb_cycle_flag; + uint8_t poc_msb_cycle_len_minus1; + + uint8_t num_extra_ph_bytes; + uint8_t extra_ph_bit_present_flag[16]; + + uint8_t num_extra_sh_bytes; + uint8_t extra_sh_bit_present_flag[16]; + + uint8_t sublayer_dpb_params_flag; + VVCDpbParameters dpb_params; + + uint8_t log2_min_luma_coding_block_size_minus2; + uint8_t partition_constraints_override_enabled_flag; + uint8_t log2_diff_min_qt_min_cb_intra_slice_luma; + uint8_t max_mtt_hierarchy_depth_intra_slice_luma; + uint8_t log2_diff_max_bt_min_qt_intra_slice_luma; + uint8_t log2_diff_max_tt_min_qt_intra_slice_luma; + + uint8_t qtbtt_dual_tree_intra_flag; + uint8_t log2_diff_min_qt_min_cb_intra_slice_chroma; + uint8_t max_mtt_hierarchy_depth_intra_slice_chroma; + uint8_t log2_diff_max_bt_min_qt_intra_slice_chroma; + uint8_t log2_diff_max_tt_min_qt_intra_slice_chroma; + + uint8_t log2_diff_min_qt_min_cb_inter_slice; + uint8_t max_mtt_hierarchy_depth_inter_slice; + uint8_t log2_diff_max_bt_min_qt_inter_slice; + uint8_t log2_diff_max_tt_min_qt_inter_slice; + + uint8_t max_luma_transform_size_64_flag; + + uint8_t transform_skip_enabled_flag; + uint8_t log2_transform_skip_max_size_minus2; + uint8_t bdpcm_enabled_flag; + + uint8_t mts_enabled_flag; + uint8_t explicit_mts_intra_enabled_flag; + uint8_t explicit_mts_inter_enabled_flag; + + uint8_t lfnst_enabled_flag; + + uint8_t joint_cbcr_enabled_flag; + uint8_t same_qp_table_for_chroma_flag; + + int8_t qp_table_start_minus26[VVC_MAX_SAMPLE_ARRAYS]; + uint8_t num_points_in_qp_table_minus1[VVC_MAX_SAMPLE_ARRAYS]; + uint8_t delta_qp_in_val_minus1[VVC_MAX_SAMPLE_ARRAYS][VVC_MAX_POINTS_IN_QP_TABLE]; + uint8_t delta_qp_diff_val[VVC_MAX_SAMPLE_ARRAYS][VVC_MAX_POINTS_IN_QP_TABLE]; + + uint8_t sao_enabled_flag; + uint8_t alf_enabled_flag; + uint8_t ccalf_enabled_flag; + uint8_t lmcs_enabled_flag; + uint8_t weighted_pred_flag; + uint8_t weighted_bipred_flag; + uint8_t long_term_ref_pics_flag; + uint8_t inter_layer_prediction_enabled_flag; + uint8_t idr_rpl_present_flag; + uint8_t rpl1_same_as_rpl0_flag; + + uint8_t num_ref_pic_lists[2]; + VVCRefPicListStruct ref_pic_list_struct[2][VVC_MAX_REF_PIC_LISTS]; + + uint8_t ref_wraparound_enabled_flag; + uint8_t temporal_mvp_enabled_flag; + uint8_t sbtmvp_enabled_flag; + uint8_t amvr_enabled_flag; + uint8_t bdof_enabled_flag; + uint8_t bdof_control_present_in_ph_flag; + uint8_t smvd_enabled_flag; + uint8_t dmvr_enabled_flag; + uint8_t dmvr_control_present_in_ph_flag; + uint8_t mmvd_enabled_flag; + uint8_t mmvd_fullpel_only_enabled_flag; + uint8_t six_minus_max_num_merge_cand; + uint8_t sbt_enabled_flag; + uint8_t affine_enabled_flag; + uint8_t five_minus_max_num_subblock_merge_cand; + uint8_t param_affine_enabled_flag; + uint8_t affine_amvr_enabled_flag; + uint8_t affine_prof_enabled_flag; + uint8_t prof_control_present_in_ph_flag; + uint8_t bcw_enabled_flag; + uint8_t ciip_enabled_flag; + uint8_t gpm_enabled_flag; + uint8_t max_num_merge_cand_minus_max_num_gpm_cand; + uint8_t log2_parallel_merge_level_minus2; + uint8_t isp_enabled_flag; + uint8_t mrl_enabled_flag; + uint8_t mip_enabled_flag; + uint8_t cclm_enabled_flag; + uint8_t chroma_horizontal_collocated_flag; + uint8_t chroma_vertical_collocated_flag; + uint8_t palette_enabled_flag; + uint8_t act_enabled_flag; + uint8_t min_qp_prime_ts; + uint8_t ibc_enabled_flag; + uint8_t six_minus_max_num_ibc_merge_cand; + uint8_t ladf_enabled_flag; + uint8_t num_ladf_intervals_minus2; + int8_t ladf_lowest_interval_qp_offset; + int8_t ladf_qp_offset[4]; + uint16_t ladf_delta_threshold_minus1[4]; + + uint8_t explicit_scaling_list_enabled_flag; + uint8_t scaling_matrix_for_lfnst_disabled_flag; + uint8_t scaling_matrix_for_alternative_colour_space_disabled_flag; + uint8_t scaling_matrix_designated_colour_space_flag; + uint8_t dep_quant_enabled_flag; + uint8_t sign_data_hiding_enabled_flag; + + uint8_t virtual_boundaries_enabled_flag; + uint8_t virtual_boundaries_present_flag; + uint8_t num_ver_virtual_boundaries; + uint16_t virtual_boundary_pos_x_minus1[3]; + uint8_t num_hor_virtual_boundaries; + uint16_t virtual_boundary_pos_y_minus1[3]; + + uint8_t timing_hrd_params_present_flag; + uint8_t sublayer_cpb_params_present_flag; + VVCGeneralTimingHrdParameters general_timing_hrd_parameters; + VVCOlsTimingHrdParameters ols_timing_hrd_parameters; + + uint8_t field_seq_flag; + uint8_t vui_parameters_present_flag; + uint16_t vui_payload_size_minus1; + VVCVUI vui; + + uint8_t extension_flag; + //H266RawExtensionData extension_data; /* TODO: read extension flag and data*/ + + enum AVPixelFormat pix_fmt; +} VVCSPS; + + +typedef struct VVCParamSets { + AVBufferRef *sps_list[VVC_MAX_SPS_COUNT]; + //AVBufferRef *vps_list[VVC_MAX_VPS_COUNT]; // TODO: since not needed, not implemented yet + //AVBufferRef *pps_list[VVC_MAX_PPS_COUNT]; // TODO: since not needed, not implemented yet + + /* currently active parameter sets */ + const VVCSPS *sps; + //const VVCVPS *vps; // TODO: since not needed, not implemented yet + //const VVCPPS *pps; // TODO: since not needed, not implemented yet +} VVCParamSets; + +/** + * Parse the SPS from the bitstream into the provided HEVCSPS struct. + * + * @param sps_id the SPS id will be written here + * @param apply_defdispwin if set 1, the default display window from the VUI + * will be applied to the video dimensions + */ +int ff_vvc_parse_sps(VVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, + int apply_defdispwin, AVCodecContext *avctx); + +int ff_vvc_decode_nal_sps(GetBitContext *gb, AVCodecContext *avctx, + VVCParamSets *ps, int apply_defdispwin); + +// TODO: since not needed, not implemented yet +//int ff_vvc_decode_nal_vps(GetBitContext *gb, AVCodecContext *avctx, +// VVCParamSets *ps); +//int ff_vvc_decode_nal_pps(GetBitContext *gb, AVCodecContext *avctx, +// VVCParamSets *ps); + +void ff_vvc_ps_uninit(VVCParamSets *ps); + +#endif /* AVCODEC_VVC_PARAMSET_H */ diff --git a/libavcodec/vvc_parse_extradata.c b/libavcodec/vvc_parse_extradata.c new file mode 100644 index 0000000000..4b7cbbe0b2 --- /dev/null +++ b/libavcodec/vvc_parse_extradata.c @@ -0,0 +1,241 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "bytestream.h" +#include "h2645_parse.h" +#include "vvc.h" +#include "vvc_parse_extradata.h" + +static int vvc_decode_nal_units(const uint8_t *buf, int buf_size, VVCParamSets *ps, + int is_nalff, int nal_length_size, + int err_recognition, int apply_defdispwin, void *logctx) +{ + int i; + int ret = 0; + H2645Packet pkt = { 0 }; + + ret = ff_h2645_packet_split(&pkt, buf, buf_size, logctx, is_nalff, + nal_length_size, AV_CODEC_ID_VVC, 1, 0); + if (ret < 0) { + goto done; + } + + for (i = 0; i < pkt.nb_nals; i++) { + H2645NAL *nal = &pkt.nals[i]; + if (nal->nuh_layer_id > 0) + continue; + + /* ignore everything except parameter sets and VCL NALUs */ + switch (nal->type) { + case VVC_VPS_NUT: + // TODO: since not needed yet, not implemented + //ret = ff_vvc_decode_nal_vps(&nal->gb, logctx, ps); + //if (ret < 0) + // goto done; + break; + case VVC_SPS_NUT: + ret = ff_vvc_decode_nal_sps(&nal->gb, logctx, ps, apply_defdispwin); + if (ret < 0) + goto done; + break; + case VVC_PPS_NUT: + // TODO: since not needed yet, not implemented + //ret = ff_vvc_decode_nal_pps(&nal->gb, logctx, ps); + //if (ret < 0) + // goto done; + break; + case VVC_PREFIX_SEI_NUT: + case VVC_SUFFIX_SEI_NUT: + // TODO: SEI decoding not implemented yet + break; + default: + av_log(logctx, AV_LOG_VERBOSE, "Ignoring NAL type %d in extradata\n", nal->type); + break; + } + } + +done: + ff_h2645_packet_uninit(&pkt); + if (err_recognition & AV_EF_EXPLODE) + return ret; + + return 0; +} + +int ff_vvc_decode_extradata(const uint8_t *data, int size, VVCParamSets *ps, + int *is_nalff, int *nal_length_size, + int err_recognition, int apply_defdispwin, void *logctx) +{ + int ret = 0; + GetByteContext gb; + + bytestream2_init(&gb, data, size); + + if (size > 3 && (data[0] || data[1] || data[2] > 1)) { + /* extradata is encoded as vvcC format. */ + int i, j, b, num_arrays, nal_len_size, num_sublayers, has_ptl; + + b = bytestream2_get_byte(&gb); + num_sublayers = (b >> 3) & 0x7; + nal_len_size = ((b >> 1) & 0x3) + 1; + has_ptl = b & 0x1; + + if (has_ptl) { + int max_picture_width = 0; + int max_picture_height = 0; + int avg_frame_rate = 0; + + int num_bytes_constraint_info; + int general_profile_idc; + int general_tier_flag; + int general_level_idc; + int ptl_frame_only_constraint_flag; + int ptl_multi_layer_enabled_flag; + int ptl_num_sub_profiles; + int temp2, temp3, temp4, temp5; + int ols_idx; + int constant_frame_rate; + int chroma_format_idc; + int bit_depth_minus8; + + temp2 = bytestream2_get_be16(&gb); + ols_idx = (temp2 >> 7) & 0x1ff; + num_sublayers = (temp2 >> 4) & 0x7; + constant_frame_rate = (temp2 >> 2) & 0x3; + chroma_format_idc = temp2 & 0x3; + bit_depth_minus8 = (bytestream2_get_byte(&gb) >> 5) & 0x7; + av_log(logctx, AV_LOG_TRACE, + "bit_depth_minus8 %d chroma_format_idc %d\n", + bit_depth_minus8, chroma_format_idc); + av_log(logctx, AV_LOG_TRACE, + "constant_frame_rate %d, ols_idx %d\n", + constant_frame_rate, ols_idx); + // VvcPTLRecord(num_sublayers) native_ptl + temp3 = bytestream2_get_byte(&gb); + num_bytes_constraint_info = (temp3) & 0x3f; + temp4 = bytestream2_get_byte(&gb); + general_profile_idc = (temp4 >> 1) & 0x7f; + general_tier_flag = (temp4) & 1; + general_level_idc = bytestream2_get_byte(&gb); + av_log(logctx, AV_LOG_TRACE, + "general_profile_idc %d, general_tier_flag %d, " + "general_level_idc %d, num_sublayers %d num_bytes_constraint_info %d\n", + general_profile_idc, general_tier_flag, general_level_idc, num_sublayers, + num_bytes_constraint_info); + + temp5 = bytestream2_get_byte(&gb); + ptl_frame_only_constraint_flag = (temp5 >> 7) & 0x1; + ptl_multi_layer_enabled_flag = (temp5 >> 6) & 0x1; + for (i = 0; i < num_bytes_constraint_info-1; i++) { + // unsigned int(8*num_bytes_constraint_info - 2) general_constraint_info; + bytestream2_get_byte(&gb); + } + + av_log(logctx, AV_LOG_TRACE, + "ptl_multi_layer_enabled_flag %d, ptl_frame_only_constraint_flag %d\n", + ptl_multi_layer_enabled_flag, ptl_frame_only_constraint_flag); + + if( num_sublayers > 1 ) { + int temp6 = bytestream2_get_byte(&gb); + uint8_t ptl_sublayer_level_present_flag[8] = {0}; + uint8_t sublayer_level_idc[8] = {0}; + for (i=num_sublayers - 2; i >= 0; i--) { + ptl_sublayer_level_present_flag[i] = (temp6 >> (7 - (num_sublayers - 2 - i))) & 0x01; + } + for (i=num_sublayers-2; i >= 0; i--) { + if (ptl_sublayer_level_present_flag[i]) { + sublayer_level_idc[i] = bytestream2_get_byte(&gb); // TODO: + } + } + } + + ptl_num_sub_profiles = bytestream2_get_byte(&gb); + for (j=0; j < ptl_num_sub_profiles; j++) { + // unsigned int(32) general_sub_profile_idc[j]; + bytestream2_get_be16(&gb); + bytestream2_get_be16(&gb); + } + + max_picture_width = bytestream2_get_be16(&gb); + max_picture_height = bytestream2_get_be16(&gb); + avg_frame_rate = bytestream2_get_be16(&gb); + av_log(logctx, AV_LOG_TRACE, + "max_picture_width %d, max_picture_height %d, avg_frame_rate %d\n", + max_picture_width, max_picture_height, avg_frame_rate); + } + + *is_nalff = 1; + + /* nal units in the vvcC always have length coded with 2 bytes, + * so set nal_length_size = 2 while parsing them */ + *nal_length_size = 2; + + num_arrays = bytestream2_get_byte(&gb); + + for (i = 0; i < num_arrays; i++) { + int cnt; + int type = bytestream2_get_byte(&gb) & 0x1f; + + if (type != VVC_OPI_NUT || type != VVC_DCI_NUT) + cnt = bytestream2_get_be16(&gb); + else + cnt = 1; + + av_log(logctx, AV_LOG_DEBUG, "nalu_type %d cnt %d\n", type, cnt); + + if (!(type == VVC_VPS_NUT || type == VVC_SPS_NUT || type == VVC_PPS_NUT || + type == VVC_PREFIX_SEI_NUT || type == VVC_SUFFIX_SEI_NUT)) { + av_log(logctx, AV_LOG_ERROR, "Invalid NAL unit type in extradata: %d\n", + type); + ret = AVERROR_INVALIDDATA; + return ret; + } + + for (j = 0; j < cnt; j++) { + // +2 for the nal size field + int nalsize = bytestream2_peek_be16(&gb) + 2; + if (bytestream2_get_bytes_left(&gb) < nalsize) { + av_log(logctx, AV_LOG_ERROR, + "Invalid NAL unit size in extradata.\n"); + return AVERROR_INVALIDDATA; + } + + ret = vvc_decode_nal_units(gb.buffer, nalsize, ps, *is_nalff, + *nal_length_size, err_recognition, apply_defdispwin, + logctx); + if (ret < 0) { + av_log(logctx, AV_LOG_ERROR, + "Decoding nal unit %d %d from vvcC failed\n", type, i); + return ret; + } + bytestream2_skip(&gb, nalsize); + } + } + + /* store nal length size, that will be used to parse all other nals */ + *nal_length_size = nal_len_size; + } else { + *is_nalff = 0; + ret = vvc_decode_nal_units(data, size, ps, *is_nalff, *nal_length_size, + err_recognition, apply_defdispwin, logctx); + if (ret < 0) + return ret; + } + + return ret; +} diff --git a/libavcodec/vvc_parse_extradata.h b/libavcodec/vvc_parse_extradata.h new file mode 100644 index 0000000000..4adb7616b7 --- /dev/null +++ b/libavcodec/vvc_parse_extradata.h @@ -0,0 +1,36 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * H.266 parser code + */ + +#ifndef AVCODEC_VVC_PARSE_EXTRADATA_H +#define AVCODEC_VVC_PARSE_EXTRADATA_H + +#include + +#include "vvc_paramset.h" + + +int ff_vvc_decode_extradata(const uint8_t *data, int size, VVCParamSets *ps, + int *is_nalff, int *nal_length_size, + int err_recognition, int apply_defdispwin, void *logctx); + +#endif /* AVCODEC_VVC_PARSE_EXTRADATA_H */ From patchwork Wed Oct 19 07:25:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Siedel X-Patchwork-Id: 38819 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp216140pzb; Wed, 19 Oct 2022 00:28:16 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4bjFYZ/a9hN5Hs0J7T1tg9Rs6F1T4uhPHRuKY2b6StEFkF4QbDhi9SOkE8PIwkgGegoW1Z X-Received: by 2002:a05:6402:1f84:b0:455:27b8:27aa with SMTP id c4-20020a0564021f8400b0045527b827aamr6035755edc.243.1666164495909; Wed, 19 Oct 2022 00:28:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666164495; cv=none; d=google.com; s=arc-20160816; b=dkeXIc9udNW7tDGBUo+GaWUbxLHubwXO3jz5/FpCRieMTk3MlE22svzGG0Db4umY30 vLP6YTVx7QqljW+7EbnsNTdxQmFmpQzU/4KqHIWXUvHTJPKo1VKs74JwCml6eLv7Ajmm VStkgx3JtLern+DEGD6leuR1G7CggSnEJ0xXBX3dyhKA/T8VKDXCD+PMbazlPqhvNlBG sp6uqNHEVmvoXAvByOfObjs6HgdAVKXKUB19uxuLQ9QnIr5bggvyIUASIowt9X/dFITc RgCP1hZH8UM/SbCW4rRGmM8tcaOyib05P+F2EQekSmd5MjXXdMXQUkJlaLi7/NcC0o0P j0vA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=HTAcPI45wYm7pjbHD0GHXKga6vNtgoVbroRNaMvpU5k=; b=Q6HKMTXlveaOwOmZt7Nz6BQ9FlOfwOx+K000GV9H0F5xGjv3X2uAmDnuTP7lw+vGqE KeF/hsMdxzYVTgGk3+S8sfHe6rtHo+xhYPEP+uRYzHKGTrCp+fnG+epQ/cjyBq1X77ot f8R9gvvUjhng3kTpmZxZ9lD7XiuSdr1SWlGZyr9xAhjw35Pa7rVVtjljLD4qiktlBrBH 8vLqR8aMeKSW7AR5jX2T7Y8HMXDbpbD1S3ril/t+j/94xsz7+n+tIFISIHR8mzfUp3PD DbVzwUr/dL7Ho/t+0Yff67KO1gaeDy6Vq5tsjCJFFRT0T+wGJ7+Yi+LV8pzUNHI9Q1d1 OFnw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=z9ugb2f7; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id sa26-20020a1709076d1a00b0078d85177210si14215427ejc.756.2022.10.19.00.28.15; Wed, 19 Oct 2022 00:28:15 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=z9ugb2f7; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2025768BE0A; Wed, 19 Oct 2022 10:26:42 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f49.google.com (mail-ej1-f49.google.com [209.85.218.49]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 305E568BDF0 for ; Wed, 19 Oct 2022 10:26:33 +0300 (EEST) Received: by mail-ej1-f49.google.com with SMTP id y14so37797282ejd.9 for ; Wed, 19 Oct 2022 00:26:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spin-digital-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zOE6hMFqBeVwWCT/TiHomm0Bi1Zh67mr6XRIc2qGVW4=; b=z9ugb2f7HQ7NDnUH1nBGL8v7sRxX7qOxWZq2tjPa/urojDYYENs6izvHU+fGeHIHqf u7fEXvap1G6YA/EJCH0GEfeFet2woeANV/Yovw5qan6NVHFvjO0iB3hXBc7Roa9rHTBg jmCF8e9eYtfTSi+opbMpZ33C6mEJFxD8LNuklSxQf0VybElWD6kV2jKcxfXgLE2CaPBO ejZsxYiLGrZxpjiCHQ66O2uTH2ITpVUMwgsxxAolzRBZo51r8MhGBM6pcoCdEniwCnrn HlvyG0WXxy/89Qfbnxh6ixm1dljKrpzomILzU3YopUvQuVdoGmIG6msrT51qDm85hO04 PLBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zOE6hMFqBeVwWCT/TiHomm0Bi1Zh67mr6XRIc2qGVW4=; b=WUzFFBu4cSc/d0wOXZHSa6fvWnMj9c2aPE+MFR2TZa+E7LcXsttxlEgjNb2EgKOw52 VLf+FPccued/pndJi73KpmsY920qViIbq0vCcDeLlCMXTr71Rg11msisGFFpLUsfPQAy m8HLe/ry8chlTcd1CHQc+JvbKJjv3NIDm80i5Pq8hi+/6igoJzSAZRLe4p9trzF4tQBF SuXA0rGXJuHBPRwXVIOyUqLDLP+QQJuoKoAHRbwBw/w1iz0kU6+ztt5/epr504IOdkbb FqcU0Ohn2KxALtm9x3AW695BjcmXJZni/AEtd0gx2csKRqTv2sIu31BxhaU22Q4zvIJs Gsgw== X-Gm-Message-State: ACrzQf3mq9GYzZSf9mGsJ94zu7zTFkQzs1uEFLr+oCs4xZbgEVcPBWtm Q9b84BQulFlmxhnfGYG0RZJBHhI1lczTNA== X-Received: by 2002:a17:907:6d18:b0:78e:259d:c391 with SMTP id sa24-20020a1709076d1800b0078e259dc391mr5590501ejc.615.1666164392073; Wed, 19 Oct 2022 00:26:32 -0700 (PDT) Received: from thomas-win.localdomain ([213.138.44.237]) by smtp.gmail.com with ESMTPSA id p5-20020a170906498500b0078b83968ad4sm8515150eju.24.2022.10.19.00.26.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Oct 2022 00:26:31 -0700 (PDT) From: thomas.ff@spin-digital.com To: ffmpeg-devel@ffmpeg.org Date: Wed, 19 Oct 2022 09:25:06 +0200 Message-Id: <20221019072508.23460-10-thomas.ff@spin-digital.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221019072508.23460-1-thomas.ff@spin-digital.com> References: <20221019072508.23460-1-thomas.ff@spin-digital.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v1 09/11] avcodec: add external encoder libvvenc for H266/VVC X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Thomas Siedel Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: /oS27UNBQBxE From: Thomas Siedel Add external encoder VVenC for H266/VVC encoding. Register new encoder libvvencc Add libvvenc to wrap the vvenc interface into ffmpeg libvvenc implements encoder option: preset,qp,period,subjopt,vvenc-params,levelidc,tier Enable encoder by adding --enable-libvvenc in configure step Signed-off-by: Thomas Siedel --- configure | 5 + libavcodec/Makefile | 1 + libavcodec/allcodecs.c | 1 + libavcodec/libvvenc.c | 432 +++++++++++++++++++++++++++++++++++++++++ 4 files changed, 439 insertions(+) create mode 100644 libavcodec/libvvenc.c diff --git a/configure b/configure index 289473782c..ff85d2921b 100755 --- a/configure +++ b/configure @@ -289,6 +289,7 @@ External library support: native implementation exists [no] --enable-libvpx enable VP8 and VP9 de/encoding via libvpx [no] --enable-libvvdec enable VVC decoding via vvdec [no] + --enable-libvvenc enable VVC encoding via vvenc [no] --enable-libwebp enable WebP encoding via libwebp [no] --enable-libx264 enable H.264 encoding via x264 [no] --enable-libx265 enable HEVC encoding via x265 [no] @@ -1877,6 +1878,7 @@ EXTERNAL_LIBRARY_LIST=" libvorbis libvpx libvvdec + libvvenc libwebp libxml2 libzimg @@ -3406,6 +3408,8 @@ libvpx_vp9_decoder_deps="libvpx" libvpx_vp9_encoder_deps="libvpx" libvvdec_decoder_deps="libvvdec" libvvdec_decoder_select="vvc_mp4toannexb_bsf" +libvvenc_encoder_deps="libvvenc" +libvvenc_encoder_select="atsc_a53" libwebp_encoder_deps="libwebp" libwebp_anim_encoder_deps="libwebp" libx262_encoder_deps="libx262" @@ -6736,6 +6740,7 @@ enabled libvpx && { fi } enabled libvvdec && require_pkg_config libvvdec "libvvdec >= 1.6.0" "vvdec/vvdec.h" vvdec_get_version +enabled libvvenc && require_pkg_config libvvenc "libvvenc >= 1.6.1" "vvenc/vvenc.h" vvenc_get_version enabled libwebp && { enabled libwebp_encoder && require_pkg_config libwebp "libwebp >= 0.2.0" webp/encode.h WebPGetEncoderVersion diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 3cc258351c..4bc1a84260 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1103,6 +1103,7 @@ OBJS-$(CONFIG_LIBVPX_VP8_ENCODER) += libvpxenc.o OBJS-$(CONFIG_LIBVPX_VP9_DECODER) += libvpxdec.o libvpx.o OBJS-$(CONFIG_LIBVPX_VP9_ENCODER) += libvpxenc.o libvpx.o OBJS-$(CONFIG_LIBVVDEC_DECODER) += libvvdec.o vvc_parse_extradata.o vvc_paramset.o +OBJS-$(CONFIG_LIBVVENC_ENCODER) += libvvenc.o OBJS-$(CONFIG_LIBWEBP_ENCODER) += libwebpenc_common.o libwebpenc.o OBJS-$(CONFIG_LIBWEBP_ANIM_ENCODER) += libwebpenc_common.o libwebpenc_animencoder.o OBJS-$(CONFIG_LIBX262_ENCODER) += libx264.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index 58cf993785..f62c3cf902 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -794,6 +794,7 @@ extern const FFCodec ff_libvpx_vp8_decoder; extern FFCodec ff_libvpx_vp9_encoder; extern FFCodec ff_libvpx_vp9_decoder; extern const FFCodec ff_libvvdec_decoder; +extern const FFCodec ff_libvvenc_encoder; /* preferred over libwebp */ extern const FFCodec ff_libwebp_anim_encoder; extern const FFCodec ff_libwebp_encoder; diff --git a/libavcodec/libvvenc.c b/libavcodec/libvvenc.c new file mode 100644 index 0000000000..5a9412814c --- /dev/null +++ b/libavcodec/libvvenc.c @@ -0,0 +1,432 @@ +/* + * H.266 encoding using the VVenC library + * + * Copyright (C) 2022, Thomas Siedel + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "config_components.h" + +#include +#include + +#include "avcodec.h" +#include "codec_internal.h" +#include "encode.h" +#include "internal.h" +#include "packet_internal.h" +#include "profiles.h" + +#include "libavutil/avutil.h" +#include "libavutil/pixdesc.h" +#include "libavutil/opt.h" +#include "libavutil/common.h" +#include "libavutil/imgutils.h" +#include "libavutil/frame.h" +#include "libavutil/log.h" + +typedef struct VVenCOptions { + int preset; // preset 0: faster 4: slower + int qp; // quantization parameter 0-63 + int subjectiveOptimization; // perceptually motivated QP adaptation, XPSNR based + int intraRefreshSec; // intra period/refresh in seconds + int levelIdc; // vvc level_idc + int tier; // vvc tier + AVDictionary *vvenc_opts; +} VVenCOptions; + +typedef struct VVenCContext { + AVClass *av_class; + VVenCOptions options; // encoder options + vvencEncoder *vvencEnc; + vvencAccessUnit *pAU; + bool encodeDone; +}VVenCContext; + + +static av_cold void ff_vvenc_log_callback(void *avctx, int level, const char *fmt, va_list args ) +{ + vfprintf( level == 1 ? stderr : stdout, fmt, args ); +} + +static void ff_vvenc_internalLog( void* ctx, int level, const char* fmt, ... ) +{ + va_list args; + va_start( args, fmt ); + ff_vvenc_log_callback( ctx, level, fmt, args ); + va_end( args ); +} + +static av_cold int ff_vvenc_encode_init(AVCodecContext *avctx) +{ + int ret; + int framerate, qp, parse_ret; + VVenCContext *s; + vvenc_config params; + vvencPresetMode preset; + AVDictionaryEntry *en; + char statsfile[1024]="vvenc-rcstats.json"; + + s = (VVenCContext*)avctx->priv_data; + qp = (vvencPresetMode)s->options.qp; + preset = (vvencPresetMode)s->options.preset; + + if( avctx->flags & AV_CODEC_FLAG_INTERLACED_DCT ) { + av_log(avctx, AV_LOG_ERROR, "ff_vvenc_encode_init::init() interlaced encoding not supported yet\n"); + return AVERROR_INVALIDDATA; + } + + vvenc_config_default( ¶ms ); + + // set desired encoding options + framerate = avctx->time_base.den / avctx->time_base.num; + vvenc_init_default ( ¶ms, avctx->width, avctx->height, framerate, avctx->bit_rate, qp, preset ); + params.m_FrameRate = avctx->time_base.den; + params.m_FrameScale = avctx->time_base.num; + + params.m_verbosity = VVENC_VERBOSE; + if ( av_log_get_level() >= AV_LOG_DEBUG ) + params.m_verbosity = VVENC_DETAILS; + else if( av_log_get_level() >= AV_LOG_VERBOSE ) + params.m_verbosity = VVENC_NOTICE; // output per picture info + else if( av_log_get_level() >= AV_LOG_INFO ) + params.m_verbosity = VVENC_WARNING; // ffmpeg default ffmpeg loglevel + else + params.m_verbosity = VVENC_SILENT; + + if( avctx->ticks_per_frame == 1 ) { + params.m_TicksPerSecond = -1; // auto mode for ticks per frame = 1 + } + else{ + params.m_TicksPerSecond = ceil((avctx->time_base.den/(double)avctx->time_base.num)*(double)avctx->ticks_per_frame); + } + + if( avctx->thread_count > 0 ) + params.m_numThreads = avctx->thread_count; + + // GOP settings (IDR/CRA) + if( avctx->flags & AV_CODEC_FLAG_CLOSED_GOP ) + params.m_DecodingRefreshType = VVENC_DRT_IDR; + + if( avctx->gop_size == 1 ) { + params.m_GOPSize = 1; + params.m_IntraPeriod = 1; + } + else { + params.m_IntraPeriodSec = s->options.intraRefreshSec; + } + + params.m_usePerceptQPA = s->options.subjectiveOptimization; + params.m_level = (vvencLevel)s->options.levelIdc; + params.m_levelTier = (vvencTier)s->options.tier; + + params.m_AccessUnitDelimiter = true; + + params.m_internChromaFormat = VVENC_CHROMA_420; + switch( avctx->pix_fmt ) { + case AV_PIX_FMT_YUV420P : + params.m_inputBitDepth[0] = 8; + break; + case AV_PIX_FMT_YUV420P10LE: + params.m_inputBitDepth[0] = 10; + break; + default: { + av_log(avctx, AV_LOG_ERROR, "unsupported pixel format %s, choose yuv420p or yuv420p10le\n", av_get_pix_fmt_name(avctx->pix_fmt)); + return AVERROR(EINVAL); + break; + } + }; + + if( avctx->color_trc != AVCOL_TRC_UNSPECIFIED ) + params.m_transferCharacteristics = (int)avctx->color_trc; + if( avctx->color_primaries != AVCOL_PRI_UNSPECIFIED ) + params.m_colourPrimaries = (int)avctx->color_primaries; + if( avctx->colorspace != AVCOL_SPC_UNSPECIFIED ) + params.m_matrixCoefficients = (int)avctx->colorspace; + + params.m_RCNumPasses = 1; + en = NULL; + while ((en = av_dict_get(s->options.vvenc_opts, "", en, AV_DICT_IGNORE_SUFFIX))) { + av_log(avctx, AV_LOG_DEBUG, "vvenc_set_param: '%s:%s'\n", en->key, en->value); + parse_ret = vvenc_set_param(¶ms, en->key, en->value); + switch (parse_ret) + { + case VVENC_PARAM_BAD_NAME: + av_log(avctx, AV_LOG_WARNING,"Unknown vvenc option: %s.\n", en->key); + break; + case VVENC_PARAM_BAD_VALUE: + av_log(avctx, AV_LOG_WARNING, + "Invalid vvenc value for %s: %s.\n", en->key, en->value); + break; + default: + break; + } + + if (memcmp( en->key, "rcstatsfile", 11 ) == 0 || + memcmp( en->key, "RCStatsFile", 11 ) == 0) { + strncpy(statsfile, en->value, sizeof(statsfile)-1); + statsfile[sizeof(statsfile)-1] = '\0'; + } + } + + if( params.m_RCPass != -1 && params.m_RCNumPasses == 1 ) + params.m_RCNumPasses = 2; // enable 2pass mode + + s->vvencEnc = vvenc_encoder_create(); + if( NULL == s->vvencEnc ) { + av_log(avctx, AV_LOG_ERROR, "cannot create vvc encoder (vvenc)\n"); + return AVERROR(ENOMEM); + } + + vvenc_set_msg_callback( ¶ms, s->vvencEnc, ff_vvenc_log_callback ); + ret = vvenc_encoder_open( s->vvencEnc, ¶ms ); + if( 0 != ret ) { + av_log(avctx, AV_LOG_ERROR, "cannot open vvc encoder (vvenc): %s\n", vvenc_get_last_error(s->vvencEnc)); + vvenc_encoder_close( s->vvencEnc ); + return AVERROR(EINVAL); + } + + vvenc_get_config( s->vvencEnc, ¶ms ); // get the adapted config + + if( params.m_verbosity >= VVENC_DETAILS && av_log_get_level() < AV_LOG_DEBUG ) { + ff_vvenc_internalLog( avctx, params.m_verbosity, "vvenc version: %s\n", vvenc_get_version() ); + ff_vvenc_internalLog( avctx, params.m_verbosity, "vvenc info:\n%s\n", vvenc_get_config_as_string( ¶ms, VVENC_DETAILS) ); + } + else { + av_log(avctx, AV_LOG_DEBUG, "vvenc version: %s\n", vvenc_get_version()); + av_log(avctx, AV_LOG_DEBUG, "vvenc info:\n%s\n", vvenc_get_config_as_string( ¶ms, VVENC_DETAILS)); + } + + if( params.m_RCNumPasses == 2 ) { + ret = vvenc_init_pass( s->vvencEnc, params.m_RCPass-1, &statsfile[0] ); + if( 0 != ret ) { + av_log(avctx, AV_LOG_ERROR, "cannot init pass %d for vvc encoder (vvenc): %s\n", params.m_RCPass, vvenc_get_last_error(s->vvencEnc)); + vvenc_encoder_close( s->vvencEnc ); + return AVERROR(EINVAL); + } + } + + s->pAU = vvenc_accessUnit_alloc(); + vvenc_accessUnit_alloc_payload( s->pAU, avctx->width * avctx->height ); + + if (avctx->flags & AV_CODEC_FLAG_GLOBAL_HEADER) { + ret = vvenc_get_headers( s->vvencEnc, s->pAU ); + if( 0 != ret ) { + av_log(avctx, AV_LOG_ERROR, "cannot get headers (SPS,PPS) from vvc encoder(vvenc): %s\n", vvenc_get_last_error(s->vvencEnc)); + vvenc_encoder_close( s->vvencEnc ); + return AVERROR(EINVAL); + } + + if (s->pAU->payloadUsedSize <= 0) { + vvenc_encoder_close( s->vvencEnc ); + return AVERROR_INVALIDDATA; + } + + avctx->extradata_size = s->pAU->payloadUsedSize; + avctx->extradata = av_malloc(avctx->extradata_size + AV_INPUT_BUFFER_PADDING_SIZE); + if (!avctx->extradata) { + av_log(avctx, AV_LOG_ERROR, + "Cannot allocate VVC header of size %d.\n", avctx->extradata_size); + vvenc_encoder_close( s->vvencEnc ); + return AVERROR(ENOMEM); + } + + memcpy(avctx->extradata, s->pAU->payload, avctx->extradata_size); + memset(avctx->extradata + avctx->extradata_size, 0, AV_INPUT_BUFFER_PADDING_SIZE); + } + s->encodeDone = false; + return 0; +} + +static av_cold int ff_vvenc_encode_close(AVCodecContext *avctx) +{ + VVenCContext *s = (VVenCContext*)avctx->priv_data; + if( s->vvencEnc ) { + if( av_log_get_level() >= AV_LOG_VERBOSE ) + vvenc_print_summary(s->vvencEnc); + + if( 0 != vvenc_encoder_close(s->vvencEnc) ) { + av_log(avctx, AV_LOG_ERROR, "cannot close vvenc\n" ); + return -1; + } + } + + vvenc_accessUnit_free ( s->pAU, true ); + + return 0; +} + +static av_cold int ff_vvenc_encode_frame( AVCodecContext *avctx, AVPacket *pkt,const AVFrame *frame, int *got_packet ) +{ + VVenCContext *s = (VVenCContext*)avctx->priv_data; + vvencYUVBuffer* pyuvbuf; + vvencYUVBuffer yuvbuf; + int pict_type; + int ret; + + pyuvbuf = NULL; + if (frame) { + if( avctx->pix_fmt == AV_PIX_FMT_YUV420P10LE ) { + vvenc_YUVBuffer_default( &yuvbuf ); + yuvbuf.planes[0].ptr = (int16_t*)frame->data[0]; + yuvbuf.planes[1].ptr = (int16_t*)frame->data[1]; + yuvbuf.planes[2].ptr = (int16_t*)frame->data[2]; + + yuvbuf.planes[0].width = frame->width; + yuvbuf.planes[0].height = frame->height; + yuvbuf.planes[0].stride = frame->linesize[0] >> 1; // stride is used in samples (16bit) in vvenc, ffmpeg uses stride in bytes + + yuvbuf.planes[1].width = frame->width >>1; + yuvbuf.planes[1].height = frame->height >>1; + yuvbuf.planes[1].stride = frame->linesize[1] >> 1; + + yuvbuf.planes[2].width = frame->width >>1; + yuvbuf.planes[2].height = frame->height >>1; + yuvbuf.planes[2].stride = frame->linesize[2] >> 1; + + yuvbuf.cts = frame->pts; + yuvbuf.ctsValid = true; + pyuvbuf = &yuvbuf; + } + else { + av_log(avctx, AV_LOG_ERROR, "unsupported input colorspace! input must be yuv420p10le"); + return AVERROR(EINVAL); + } + } + + if( ! s->encodeDone ) { + ret = vvenc_encode( s->vvencEnc, pyuvbuf, s->pAU, &s->encodeDone ); + if( ret != 0 ) { + av_log(avctx, AV_LOG_ERROR, "error in vvenc::encode - ret:%d\n", ret); + return AVERROR(EINVAL); + } + } + else { + *got_packet = 0; + return 0; + } + + if( s->pAU->payloadUsedSize > 0 ) { + ret = ff_get_encode_buffer(avctx, pkt, s->pAU->payloadUsedSize, 0); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Error getting output packet.\n"); + return ret; + } + + memcpy( pkt->data, s->pAU->payload, s->pAU->payloadUsedSize ); + + if( s->pAU->ctsValid ) + pkt->pts = s->pAU->cts; + if( s->pAU->dtsValid ) + pkt->dts = s->pAU->dts; + pkt->flags |= AV_PKT_FLAG_KEY*s->pAU->rap; + + switch (s->pAU->sliceType) { + case VVENC_I_SLICE: + pict_type = AV_PICTURE_TYPE_I; + break; + case VVENC_P_SLICE: + pict_type = AV_PICTURE_TYPE_P; + break; + case VVENC_B_SLICE: + pict_type = AV_PICTURE_TYPE_B; + break; + default: + av_log(avctx, AV_LOG_ERROR, "Unknown picture type encountered.\n"); + return AVERROR_EXTERNAL; + } + + ff_side_data_set_encoder_stats(pkt, 0, NULL, 0, pict_type); + + *got_packet = 1; + + return 0; + } + else + { + *got_packet = 0; + return 0; + } + + return 0; +} + +static const enum AVPixelFormat pix_fmts_vvc[] = { + //AV_PIX_FMT_YUV420P, // TODO + AV_PIX_FMT_YUV420P10LE, + AV_PIX_FMT_NONE +}; + +#define OFFSET(x) offsetof(VVenCContext, x) +#define VE AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM +static const AVOption libvvenc_options[] = { + {"preset", "set encoding preset(0: faster - 4: slower", OFFSET( options.preset), AV_OPT_TYPE_INT, {.i64 = 2} , 0 , 4 , VE, "preset"}, + { "faster", "0", 0, AV_OPT_TYPE_CONST, {.i64 = VVENC_FASTER},INT_MIN, INT_MAX, VE, "preset" }, + { "fast", "1", 0, AV_OPT_TYPE_CONST, {.i64 = VVENC_FAST}, INT_MIN, INT_MAX, VE, "preset" }, + { "medium", "2", 0, AV_OPT_TYPE_CONST, {.i64 = VVENC_MEDIUM},INT_MIN, INT_MAX, VE, "preset" }, + { "slow", "3", 0, AV_OPT_TYPE_CONST, {.i64 = VVENC_SLOW}, INT_MIN, INT_MAX, VE, "preset" }, + { "slower", "4", 0, AV_OPT_TYPE_CONST, {.i64 = VVENC_SLOWER},INT_MIN, INT_MAX, VE, "preset" }, + {"qp" , "set quantization" , OFFSET(options.qp) , AV_OPT_TYPE_INT, {.i64 = 32}, 0 , 63 ,VE, "qp_mode" }, + {"period" , "set (intra) refresh period in seconds" , OFFSET(options.intraRefreshSec) , AV_OPT_TYPE_INT, {.i64 = 1}, 1 , INT_MAX ,VE,"irefreshsec" }, + {"subjopt", "set subjective (perceptually motivated) optimization" , OFFSET(options.subjectiveOptimization) , AV_OPT_TYPE_BOOL, {.i64 = 1}, 0 , 1, VE}, + {"vvenc-params", "set the vvenc configuration using a :-separated list of key=value parameters", OFFSET(options.vvenc_opts), AV_OPT_TYPE_DICT, { 0 }, 0, 0, VE }, + {"levelidc", "vvc level_idc", OFFSET( options.levelIdc), AV_OPT_TYPE_INT, {.i64 = 0}, 0, 105, VE, "levelidc"}, + {"0" , "auto" , 0, AV_OPT_TYPE_CONST, {.i64 = 0}, INT_MIN, INT_MAX, VE, "levelidc"}, + {"1" , "1" , 0, AV_OPT_TYPE_CONST, {.i64 = 16},INT_MIN, INT_MAX, VE, "levelidc"}, + {"2" , "2" , 0, AV_OPT_TYPE_CONST, {.i64 = 32},INT_MIN, INT_MAX, VE, "levelidc"}, + {"2.1" , "2.1" , 0, AV_OPT_TYPE_CONST, {.i64 = 35},INT_MIN, INT_MAX, VE, "levelidc"}, + {"3" , "3" , 0, AV_OPT_TYPE_CONST, {.i64 = 48},INT_MIN, INT_MAX, VE, "levelidc"}, + {"3.1" , "3.1" , 0, AV_OPT_TYPE_CONST, {.i64 = 51},INT_MIN, INT_MAX, VE, "levelidc"}, + {"4" , "4" , 0, AV_OPT_TYPE_CONST, {.i64 = 64},INT_MIN, INT_MAX, VE, "levelidc"}, + {"4.1" , "4.1" , 0, AV_OPT_TYPE_CONST, {.i64 = 67},INT_MIN, INT_MAX, VE, "levelidc"}, + {"5" , "5" , 0, AV_OPT_TYPE_CONST, {.i64 = 80},INT_MIN, INT_MAX, VE, "levelidc"}, + {"5.1" , "5.1" , 0, AV_OPT_TYPE_CONST, {.i64 = 83},INT_MIN, INT_MAX, VE, "levelidc"}, + {"5.2" , "5.2" , 0, AV_OPT_TYPE_CONST, {.i64 = 86},INT_MIN, INT_MAX, VE, "levelidc"}, + {"6" , "6" , 0, AV_OPT_TYPE_CONST, {.i64 = 96},INT_MIN, INT_MAX, VE, "levelidc"}, + {"6.1" , "6.1" , 0, AV_OPT_TYPE_CONST, {.i64 = 99},INT_MIN, INT_MAX, VE, "levelidc"}, + {"6.2" , "6.2" , 0, AV_OPT_TYPE_CONST, {.i64 = 102},INT_MIN, INT_MAX, VE, "levelidc"}, + {"6.3" , "6.3" , 0, AV_OPT_TYPE_CONST, {.i64 = 105},INT_MIN, INT_MAX, VE, "levelidc"}, + {"tier", "set vvc tier", OFFSET( options.tier), AV_OPT_TYPE_INT, {.i64 = 0}, 0 , 1 , VE, "tier"}, + {"main", "main" , 0, AV_OPT_TYPE_CONST, {.i64 = 0}, INT_MIN, INT_MAX, VE, "tier"}, + {"high", "high" , 0, AV_OPT_TYPE_CONST, {.i64 = 1}, INT_MIN, INT_MAX, VE, "tier"}, + {NULL} +}; + +static const AVClass class_libvvenc = { + .class_name = "libvvenc-vvc encoder", + .item_name = av_default_item_name, + .option = libvvenc_options, + .version = LIBAVUTIL_VERSION_INT, +}; + +FFCodec ff_libvvenc_encoder = { + .p.name = "libvvenc", + .p.long_name = "H.266 / VVC Encoder VVenC", + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_VVC, + .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_OTHER_THREADS, + .p.profiles = NULL_IF_CONFIG_SMALL(ff_vvc_profiles), + .p.priv_class = &class_libvvenc, + .p.wrapper_name = "libvvenc", + .priv_data_size = sizeof(VVenCContext), + .p.pix_fmts = pix_fmts_vvc, + .init = ff_vvenc_encode_init, + FF_CODEC_ENCODE_CB(ff_vvenc_encode_frame), + .close = ff_vvenc_encode_close, + .caps_internal = FF_CODEC_CAP_AUTO_THREADS, +}; From patchwork Wed Oct 19 07:25:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Siedel X-Patchwork-Id: 38815 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp215949pzb; Wed, 19 Oct 2022 00:27:36 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5hRriJDyC8XmIBv733ZjtYB0LHygFYZyNQlFVSUyQeCwE8ib1SecJMxJUfnqDWiSrodGGc X-Received: by 2002:a17:906:fe45:b0:791:9624:9ea4 with SMTP id wz5-20020a170906fe4500b0079196249ea4mr5462752ejb.147.1666164456785; Wed, 19 Oct 2022 00:27:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666164456; cv=none; d=google.com; s=arc-20160816; b=uDISsCkslUFYM/6dKIJ6etz/yU+0l+OwPSt7c0BdoT72H6X2jHQvye7+vmFkCKMnvv PgQ1AS0I0xZnEXu9Fvd952GiOzxz8XOJtFP0qV6xsfr5kbq350Akplc6OUmsQe/VGbil U6JSid3MI+kF5kBiIfrtnpXzrSWJB9PkeSBoyMLpFhdo9wiZiOCzCQcosrzi9wR6rCvZ e00EeShuE4chKgO91benIQIhC6ecSdhiy+JzQrcQCW9dtzC5O1CdR5l18Ox2zFNmJ83P h0btRJ3DqIgAcNbkMQmJ4ceec2IXCmxCCt5LuRFwEhccFp/3fQBJJLJlOtA9xPsrsSmk wafQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=AYLabUN88BhGwVG7EtdgmPsOmmjgUL9Ipt5TURNI7kA=; b=bll1AQGbvLav1bDXgNAt/mi+ie3a1zQ3lumugxOdusGPzy2j46fs2vokVjh8Ol+7T2 yFM54lmZxUMLxwyHxRpuFC53EeIubKtcW9VITEYLnGMwuKvZxPInHzizW+8+CRKjHRU4 QORrHRx+Oz9KgLi8cf3LWBfPaX4pGcmlvunpFoglfkb2jjFxVAfG+WAtuE94X42PixkB 7DBQt9lG/J4r7vEFh0oWZLOvecuKhuH+pI1sAp8gt36NS/xlM3rrtH5uv06T7hOeTaqr J1rU2oJdAZX6Fv6NvQVsSgpb7IN8cAdFAyV68nrYpEVMjv/GF2zUIm6bMw72F/15Iahn RYjQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=ZkjQl+Nb; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id n8-20020a05640205c800b004590ae88029si16505117edx.610.2022.10.19.00.27.36; Wed, 19 Oct 2022 00:27:36 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=ZkjQl+Nb; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3075768BDF6; Wed, 19 Oct 2022 10:26:38 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f54.google.com (mail-ed1-f54.google.com [209.85.208.54]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 72FFF68BDD9 for ; Wed, 19 Oct 2022 10:26:34 +0300 (EEST) Received: by mail-ed1-f54.google.com with SMTP id b12so23904453edd.6 for ; Wed, 19 Oct 2022 00:26:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spin-digital-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1jt2K2OTXU2SaqoNp5y9jAXDIqBrRohobOQSA2mdnaQ=; b=ZkjQl+NbatWmaLMLyQRRXKOl6bLGkDXRgHby1h0UjBftzJfjp0gNYSYWs27D+b2Wwu Q8ODfTzws+Q4XNoGBLY9CvOgD/Hr10JEX5oWDyHYduAwBUCd5KtiJFzU/4eNPYc2quVY b6n2JONa7rBBkr1jdFjiG4MENgvMGDT6SstFWcA6eJ612DbIIo+COcD0E6LG9nHlucXU ftwgopcx5g5csHskUh3OgJgp2sDmHovdqgHZVQK2Kw7UpcHDUusKZ71JnuCWbnA9Iose eeSbunFSYaK9F/am2lz+8YYxZiK/rhCQU10EYq5rQBuR+i7IU6qLP4icbuFwXbJCC42l YzJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1jt2K2OTXU2SaqoNp5y9jAXDIqBrRohobOQSA2mdnaQ=; b=DPWL/Kon4z8hRLyS6mpjqM/vyvTrFCgiK1I8wfWYTrPIMA0qLIMb53vL1E71OFIp6O 7IceN4sBuR8X2N8DAcOlSQ3K2oEQ/oTjcGZ1PKz8yuE336+D9M2EvL/0xjiUj/N5FBRC v06NkFgQC9pfioFjRm/CMEkgULROuIB3zaSwa/VEhmlFZK53/qx65cqj0hGwB/Gf3viN 8/jZJcYiEmPEDTJMZJKPuEmSJGmDfE+QipAwAX938SbCC8YEJDVmXFEsOmWE7rFPV8Tj 874CB7Bxf4VPTbN5tIXt1j7SbL7iMV3PBewwHWmnbfU5pIoyvOPdVA9MpCSoRRHVnyHI Motg== X-Gm-Message-State: ACrzQf3jzSksxPCA/Ijh7pRDQR6v3L67ZmfPZpw9RNdzudJT/W5Lg64Q +6dkBFNqRiwOdREcvYryCeCRu4VJW0gaaQ== X-Received: by 2002:a05:6402:2989:b0:44e:90d0:b9ff with SMTP id eq9-20020a056402298900b0044e90d0b9ffmr6047460edb.110.1666164393158; Wed, 19 Oct 2022 00:26:33 -0700 (PDT) Received: from thomas-win.localdomain ([213.138.44.237]) by smtp.gmail.com with ESMTPSA id p5-20020a170906498500b0078b83968ad4sm8515150eju.24.2022.10.19.00.26.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Oct 2022 00:26:32 -0700 (PDT) From: thomas.ff@spin-digital.com To: ffmpeg-devel@ffmpeg.org Date: Wed, 19 Oct 2022 09:25:07 +0200 Message-Id: <20221019072508.23460-11-thomas.ff@spin-digital.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221019072508.23460-1-thomas.ff@spin-digital.com> References: <20221019072508.23460-1-thomas.ff@spin-digital.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v1 10/11] avformat: add ts stream types for H266/VVC X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Thomas Siedel Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: dMFBg+hXSfUO From: Thomas Siedel Add transport stream stream type 0x33 for vvc. Add STREAM_TYPE_VIDEO_VVC to MPEG-1/2 and MPEG-2 transport stream Add basic transport stream support for TS mux/demux Signed-off-by: Thomas Siedel --- configure | 2 +- libavformat/mpeg.c | 3 ++ libavformat/mpeg.h | 1 + libavformat/mpegts.c | 2 ++ libavformat/mpegts.h | 1 + libavformat/mpegtsenc.c | 65 +++++++++++++++++++++++++++++++++++++++++ 6 files changed, 73 insertions(+), 1 deletion(-) diff --git a/configure b/configure index ff85d2921b..10a723eff9 100755 --- a/configure +++ b/configure @@ -3482,7 +3482,7 @@ mp3_demuxer_select="mpegaudio_parser" mp3_muxer_select="mpegaudioheader" mp4_muxer_select="mov_muxer" mpegts_demuxer_select="iso_media" -mpegts_muxer_select="ac3_parser adts_muxer latm_muxer h264_mp4toannexb_bsf hevc_mp4toannexb_bsf" +mpegts_muxer_select="ac3_parser adts_muxer latm_muxer h264_mp4toannexb_bsf hevc_mp4toannexb_bsf vvc_mp4toannexb_bsf" mpegtsraw_demuxer_select="mpegts_demuxer" mxf_muxer_select="pcm_rechunk_bsf" mxf_d10_muxer_select="mxf_muxer" diff --git a/libavformat/mpeg.c b/libavformat/mpeg.c index 864b08d8f8..970926df6b 100644 --- a/libavformat/mpeg.c +++ b/libavformat/mpeg.c @@ -546,6 +546,9 @@ redo: } else if (es_type == STREAM_TYPE_VIDEO_HEVC) { codec_id = AV_CODEC_ID_HEVC; type = AVMEDIA_TYPE_VIDEO; + } else if (es_type == STREAM_TYPE_VIDEO_VVC) { + codec_id = AV_CODEC_ID_VVC; + type = AVMEDIA_TYPE_VIDEO; } else if (es_type == STREAM_TYPE_AUDIO_AC3) { codec_id = AV_CODEC_ID_AC3; type = AVMEDIA_TYPE_AUDIO; diff --git a/libavformat/mpeg.h b/libavformat/mpeg.h index b635295776..20592eb184 100644 --- a/libavformat/mpeg.h +++ b/libavformat/mpeg.h @@ -56,6 +56,7 @@ #define STREAM_TYPE_VIDEO_MPEG4 0x10 #define STREAM_TYPE_VIDEO_H264 0x1b #define STREAM_TYPE_VIDEO_HEVC 0x24 +#define STREAM_TYPE_VIDEO_VVC 0x33 #define STREAM_TYPE_VIDEO_CAVS 0x42 #define STREAM_TYPE_AUDIO_AC3 0x81 diff --git a/libavformat/mpegts.c b/libavformat/mpegts.c index d97702fcd7..61a800c85f 100644 --- a/libavformat/mpegts.c +++ b/libavformat/mpegts.c @@ -811,6 +811,7 @@ static const StreamType ISO_types[] = { { 0x20, AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_H264 }, { 0x21, AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_JPEG2000 }, { 0x24, AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_HEVC }, + { 0x33, AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_VVC }, { 0x42, AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_CAVS }, { 0xd1, AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_DIRAC }, { 0xd2, AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_AVS2 }, @@ -865,6 +866,7 @@ static const StreamType REGD_types[] = { { MKTAG('D', 'T', 'S', '3'), AVMEDIA_TYPE_AUDIO, AV_CODEC_ID_DTS }, { MKTAG('E', 'A', 'C', '3'), AVMEDIA_TYPE_AUDIO, AV_CODEC_ID_EAC3 }, { MKTAG('H', 'E', 'V', 'C'), AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_HEVC }, + { MKTAG('V', 'V', 'C', ' '), AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_VVC }, { MKTAG('K', 'L', 'V', 'A'), AVMEDIA_TYPE_DATA, AV_CODEC_ID_SMPTE_KLV }, { MKTAG('I', 'D', '3', ' '), AVMEDIA_TYPE_DATA, AV_CODEC_ID_TIMED_ID3 }, { MKTAG('V', 'C', '-', '1'), AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_VC1 }, diff --git a/libavformat/mpegts.h b/libavformat/mpegts.h index a48f14e768..14ae312c50 100644 --- a/libavformat/mpegts.h +++ b/libavformat/mpegts.h @@ -128,6 +128,7 @@ #define STREAM_TYPE_METADATA 0x15 #define STREAM_TYPE_VIDEO_H264 0x1b #define STREAM_TYPE_VIDEO_HEVC 0x24 +#define STREAM_TYPE_VIDEO_VVC 0x33 #define STREAM_TYPE_VIDEO_CAVS 0x42 #define STREAM_TYPE_VIDEO_AVS2 0xd2 #define STREAM_TYPE_VIDEO_AVS3 0xd4 diff --git a/libavformat/mpegtsenc.c b/libavformat/mpegtsenc.c index 48d39e6a7d..4c32cddff1 100644 --- a/libavformat/mpegtsenc.c +++ b/libavformat/mpegtsenc.c @@ -366,6 +366,9 @@ static int get_dvb_stream_type(AVFormatContext *s, AVStream *st) case AV_CODEC_ID_HEVC: stream_type = STREAM_TYPE_VIDEO_HEVC; break; + case AV_CODEC_ID_VVC: + stream_type = STREAM_TYPE_VIDEO_VVC; + break; case AV_CODEC_ID_CAVS: stream_type = STREAM_TYPE_VIDEO_CAVS; break; @@ -462,6 +465,11 @@ static int get_m2ts_stream_type(AVFormatContext *s, AVStream *st) case AV_CODEC_ID_HEVC: stream_type = STREAM_TYPE_VIDEO_HEVC; break; + case AV_CODEC_ID_VVC: + av_log(s, AV_LOG_ERROR, + "MPEGTS VVC %s.\n", avcodec_get_name(st->codecpar->codec_id)); + stream_type = STREAM_TYPE_VIDEO_VVC; + break; case AV_CODEC_ID_PCM_BLURAY: stream_type = 0x80; break; @@ -1783,6 +1791,21 @@ static int check_hevc_startcode(AVFormatContext *s, const AVStream *st, const AV return 0; } +static int check_vvc_startcode(AVFormatContext *s, const AVStream *st, const AVPacket *pkt) +{ + if (pkt->size < 5 || AV_RB32(pkt->data) != 0x0000001 && AV_RB24(pkt->data) != 0x000001) { + if (!st->nb_frames) { + av_log(s, AV_LOG_ERROR, "VVC bitstream malformed, no startcode found\n"); + return AVERROR_PATCHWELCOME; + } + av_log(s, AV_LOG_WARNING, "VVC bitstream error, startcode missing, size %d", pkt->size); + if (pkt->size) + av_log(s, AV_LOG_WARNING, " data %08"PRIX32, AV_RB32(pkt->data)); + av_log(s, AV_LOG_WARNING, "\n"); + } + return 0; +} + /* Based on GStreamer's gst-plugins-base/ext/ogg/gstoggstream.c * Released under the LGPL v2.1+, written by * Vincent Penquerc'h @@ -1975,6 +1998,42 @@ static int mpegts_write_packet_internal(AVFormatContext *s, AVPacket *pkt) buf = data; size = pkt->size + 7 + extradd; } + } else if (st->codecpar->codec_id == AV_CODEC_ID_VVC) { + const uint8_t *p = buf, *buf_end = p + size; + uint32_t state = -1; + uint32_t naltype = -1; + int extradd = (pkt->flags & AV_PKT_FLAG_KEY) ? st->codecpar->extradata_size : 0; + int ret = check_vvc_startcode(s, st, pkt); + if (ret < 0) + return ret; + + if (extradd && AV_RB24(st->codecpar->extradata) > 1) + extradd = 0; + + do { + p = avpriv_find_start_code(p, buf_end, &state); + // state contains byte after start code, p points 2 bytes after start code + naltype = ((AV_RB8(p)>>3) & 0x1F); + av_log(s, AV_LOG_TRACE, "nal %"PRId32"\n", naltype ); + if (naltype == 14 ) // VPS + extradd = 0; + } while (p < buf_end && naltype != 20 && naltype >= 12); + + if (naltype >= 12) + extradd = 0; + if (naltype != 20) { // AUD NAL + data = av_malloc(pkt->size + 7 + extradd); + if (!data) + return AVERROR(ENOMEM); + memcpy(data + 7, st->codecpar->extradata, extradd); + memcpy(data + 7 + extradd, pkt->data, pkt->size); + AV_WB32(data, 0x00000001); + data[4] = 20; + data[5] = 1; + data[6] = 0x50; // any slice type (0x4) + rbsp stop one bit + buf = data; + size = pkt->size + 7 + extradd; + } } else if (st->codecpar->codec_id == AV_CODEC_ID_OPUS) { if (pkt->size < 2) { av_log(s, AV_LOG_ERROR, "Opus packet too short\n"); @@ -2231,6 +2290,12 @@ static int mpegts_check_bitstream(AVFormatContext *s, AVStream *st, (st->codecpar->extradata_size > 0 && st->codecpar->extradata[0] == 1))) ret = ff_stream_add_bitstream_filter(st, "hevc_mp4toannexb", NULL); + } else if (st->codecpar->codec_id == AV_CODEC_ID_VVC) { + if (pkt->size >= 5 && AV_RB32(pkt->data) != 0x0000001 && + (AV_RB24(pkt->data) != 0x000001 || + (st->codecpar->extradata_size > 0 && + st->codecpar->extradata[0] == 1))) + ret = ff_stream_add_bitstream_filter(st, "vvc_mp4toannexb", NULL); } return ret; From patchwork Wed Oct 19 07:25:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Siedel X-Patchwork-Id: 38816 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp215987pzb; Wed, 19 Oct 2022 00:27:45 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7p78iI55+JZM4f0fP/IJnoPki5h1xJABT7wIvDN2OYfuCiCOCDL0L6guwzbOdBQnD019j6 X-Received: by 2002:a05:6402:2949:b0:451:fabf:d88a with SMTP id ed9-20020a056402294900b00451fabfd88amr6026916edb.324.1666164465725; Wed, 19 Oct 2022 00:27:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666164465; cv=none; d=google.com; s=arc-20160816; b=ZPZuwDxiQsGqGLUafjt/ez+7C9EfUkxH8K4817DNcG4nB2SZcJEZMON+lmE1J9FxZW aSQ4l+1lH+/MUow+u6UA1WES5ZAQ09EHQpavYiltp8kJJZTROxeB6jqxWU25k9hJbEeU aUbuQmeSyQYzNA82HWlqsSfhdz2jDh+B9cemDve06YBBrLJevaVdY7C0KLVj51eCSo/t TckVurnalsq8tTq6P85N5Z8AtGv2zDSZlAr1/ux1f1/cXGd+FFR4R3JV1Z+SM5aPEF3w 1j4mNirpoj1z0mAAWBF6HhqnpOb4g2IP7fYlcFRerZd7MN2SZ9/3pO7OqJm3Z/oh9j1C rp8g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=xo7MQfKttBNZS/rALil6xqK09cUVvTrx/mJzKInPIJ0=; b=U1exizpuV4rrVLpLwBnnxRf3Cxdslo19IgnUFfQ/PfKlBnZgyKMdSqyOTPH2U8rNmo C5bfrEwT6m0IRjFAXM+wvFSb+93jRliNK/BW6CoJvfYYrnCgmc2cwogQLvPERKa0QgI6 cXrpj4nmW6aH0PkpwdwUs526LY2VUmDr+dU9gs8QSTx4waZ/2NGLv/AOm9/yCjKq/dVw v1jUGzvtB4TP4wDMA0X5AUMo1a7ZL84re5GjPMKB7xg5xKxpJo/ff+tTorSwaGh2JMav nLcoheNH5hKoB2vq9bgzbmKrtjrsJLWveCBm+uW5FuflM1n1GdF/h0Wnj/GE5EhNkWEp A5CQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=VBTCaay8; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id s17-20020aa7c551000000b0045c9dbe290csi12288463edr.406.2022.10.19.00.27.45; Wed, 19 Oct 2022 00:27:45 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=VBTCaay8; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 267BE68BDDF; Wed, 19 Oct 2022 10:26:39 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f52.google.com (mail-ej1-f52.google.com [209.85.218.52]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3F8B468BC1D for ; Wed, 19 Oct 2022 10:26:34 +0300 (EEST) Received: by mail-ej1-f52.google.com with SMTP id bj12so37775713ejb.13 for ; Wed, 19 Oct 2022 00:26:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spin-digital-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=sQfcDNzqwOR/WnYfmd1dCkacBdRVqThfJ4mW1iR2dGM=; b=VBTCaay8nPpTjdDbeDDuWQoQA3YsfsHj4xSzl4xVD3CwKsnGRjS+9QgrFuM/7jhLkW MWvI4+J15X9hcqu8mC8LrgCBB6L3YQZszth4yDD4ED3EGp0dvx6NM+ZxrZc/5E88HYtr zh97PNpXi9VhT170Tas0wlK7i3/G/jnSUb+MUNc5evf9iUgUOYNQi42In9k/zegsDZYU nKnroNcI/yeM2don7LFmz/21ZSQkMExpFrW5YaPNeD37nHRKQOs0tZKwd0BK550kp/2d gJhfluXHI5m4KMYk9dpVqGiTQtxa7NRrUXqmn2DF+gi8/iksFn4I+bljEM3cjuzXpH92 CwRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=sQfcDNzqwOR/WnYfmd1dCkacBdRVqThfJ4mW1iR2dGM=; b=1DDCQiZ7S4E+yMv5hJKW+WLy5GLtF29yX9nKoAzEM76A+qYolnHKhgYZXlcIMoVWsH nQptyf2VT30360Pc43BgsTy/rF3qpUPs4bau10y9k7tynNyvo1pJ1QKrVX+WcCKNLtyd hLu74Uyqa17oUOAuvYCi6J3ZYw09oFdP7xSK1aVQNc8BC832baYC+mnBEBUaPEEICopu kxrf0WzhkgH51DLvsGZxac/rDM2NRJMlgD755n+hAJGMbct50wlir0e3BWkuQ/LVwqpq YcUjvOxLSHxiT/iuSTEd082V6agK1ih/1BNFVndBRWRVGLMcJsc9f6ObQBieTjeXKy9x rqPw== X-Gm-Message-State: ACrzQf0MdzVm6+4+gLMiU59+tqyAupxwlzuztH4Ek5vEFp0A1OCkLC01 yJ64JaEwobZeLedWtqUELgiW01Ghr+MTLQ== X-Received: by 2002:a17:907:a059:b0:78d:1a16:fc5c with SMTP id gz25-20020a170907a05900b0078d1a16fc5cmr5514810ejc.31.1666164394053; Wed, 19 Oct 2022 00:26:34 -0700 (PDT) Received: from thomas-win.localdomain ([213.138.44.237]) by smtp.gmail.com with ESMTPSA id p5-20020a170906498500b0078b83968ad4sm8515150eju.24.2022.10.19.00.26.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Oct 2022 00:26:33 -0700 (PDT) From: thomas.ff@spin-digital.com To: ffmpeg-devel@ffmpeg.org Date: Wed, 19 Oct 2022 09:25:08 +0200 Message-Id: <20221019072508.23460-12-thomas.ff@spin-digital.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221019072508.23460-1-thomas.ff@spin-digital.com> References: <20221019072508.23460-1-thomas.ff@spin-digital.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v1 11/11] avcodec: increase minor version for H266/VVC X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Thomas Siedel Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 7Ha563hjU3u+ From: Thomas Siedel Increase avcodec minor version for vvc support. Signed-off-by: Thomas Siedel --- libavcodec/version.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavcodec/version.h b/libavcodec/version.h index f8abc803b6..86ac0f3871 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -29,7 +29,7 @@ #include "version_major.h" -#define LIBAVCODEC_VERSION_MINOR 51 +#define LIBAVCODEC_VERSION_MINOR 52 #define LIBAVCODEC_VERSION_MICRO 100 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \