From patchwork Mon Oct 24 14:06:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thomas Siedel X-Patchwork-Id: 38958 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2249003pzb; Mon, 24 Oct 2022 07:08:12 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5doA3empqerLpHBAxMTtOyMgKjENsJwxCivVMEfXVp4eq/Jyksq5Y3s4FORqJY9TCHggL+ X-Received: by 2002:a17:906:8470:b0:78d:b531:7d10 with SMTP id hx16-20020a170906847000b0078db5317d10mr28115164ejc.275.1666620481170; Mon, 24 Oct 2022 07:08:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666620481; cv=none; d=google.com; s=arc-20160816; b=ONk++m3kQOOGhYXwWqJdN/HHh9o5GX4dT5wn5ry/TSegh5tZO+1PLfvaE9L99SMXek 90N7tlSjCnobdTrBpSjXSTvI4Pr4TjP88Bq0bj5WPh4P7JMQWwr7AiMS37EYN3q1cv66 Wt67vLj+Uu32aZ2bt0Tt5HDt3sZVXU3yAooSmDTd4U1mSdZGC29TNRTVnr10TXgBXXb0 0OmUc593cq6qCH4cgERbge/tECIhv24aQGDoduXkdYhUoF8DXtdMqIYgeKeYu4tH11Ld Bs9RrcJbe81AgSg/0e4SVoQGMuDOv7WTMWsmpiMRHDyq/GVJMRlWNkUQy+Lauk7bQqDB qSHg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=gCVQ4Fg9OecPRKzAsZciAQy3sEyHFr3yur5Ehg8jOyc=; b=YzgSoYIGcY/v+ObGj1FCPUVl/NKYbB76/qJIIYGs22czs2hn/ST1fNXwgvkEn06naA Hsd8aiJjmjf+QFvgM7Br63313IdJoevJcGGM3vphnV8PM6+JEFC0WBErpvfbxsVyhtPB dIbZB1Wvs7va79P+H6VeDTsGoJNJdFxsd125ca8TrnllPnaH5FE609+LoFwuulE0BDBl hDULVFGiwlslfti/bF+0R9cVCJ1yS+0E5tgeGg3UtbE8GW1y0IjF83KRpwKZ2jdNzp1u uZlhMbNJTKtDIYwDD0zw9h2eJANioQX51zOOSbfhpf1FSi+VWX5bHkitVx+dQaE63Jbq BuLQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=2xeQAfoc; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id x23-20020a170906135700b007470b52f94esi23139346ejb.199.2022.10.24.07.07.56; Mon, 24 Oct 2022 07:08:01 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=2xeQAfoc; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2E9F368BBD7; Mon, 24 Oct 2022 17:07:44 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f50.google.com (mail-ed1-f50.google.com [209.85.208.50]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5AE3D68B948 for ; Mon, 24 Oct 2022 17:07:36 +0300 (EEST) Received: by mail-ed1-f50.google.com with SMTP id y69so12705937ede.5 for ; Mon, 24 Oct 2022 07:07:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spin-digital-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=w0eOI9TveFibamnZ+nCqukpJDuKppiccz3HdMuquqQ8=; b=2xeQAfocH6I6RIpoNP7SB44oK67qcmLh7ftVQwwYzONO0jSqnBAM8f12RMqZ3JrB27 ra5piCPQRAOPIhYoYXs5QIGFE2jz84vSSuYjpenCLNllAUz3bauvdDgxntHIZZ5irpyA ffHrMbeJYE2Q0EdK/45Sh8sHuCDUszsioBm6y9mfFZuv2prHIqVshlDjQNsMWw75IvWz ITFC++OLLgaHcsCJ3MJFQA/IPYSSTc+rYX15FDVwf/0DHJ7hpuVQwabv7D8wWXQ6DBjY Q5t34qFaIrZ5XP30kwIjhZEAFh8fFczSg1KF2utRsNU3uRh5hgck5glE8Y5dkddp3Tsb HWjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=w0eOI9TveFibamnZ+nCqukpJDuKppiccz3HdMuquqQ8=; b=x0WOH1sXdU4e3r8/7zK819KMKz7gDRbpvYUIA/LtD3DMJjK69LpDd+bFLYGnzf00wt ku5VLx/PJ4Bqq/RDrcxXp3hk5e8ma48fWEXzlXC7dWyexMWEHJ8JB2A2SVTArClhHrMP ODke55rEaOOSgVdWcm4czb3PS6tp6oZxbS/F7s78TugweC/chk3A6hmKr0DZo1zYNOgO VtVDWjsc85/p7UUdKcIDn5YbMPTWOsvmulfYhzlKP/8J+OK2yB4c29Vqtu446TuIMyz1 VSzcSmFmVkVWrIzwvY/pN9qjVkns8l5osJlcv3qAaqCMyTT8Bo/os5gux/pC1S5UqKzn sxCA== X-Gm-Message-State: ACrzQf2ksF9LvWvZHbWfM3Xk9Qkn6XYKOWFbmDtUzinw3lKzhBo0x0n6 Kz98Jx1mJJQvjldAwOPorNVxzMwJjZf/hw== X-Received: by 2002:a17:907:3da5:b0:78e:793:4084 with SMTP id he37-20020a1709073da500b0078e07934084mr28240858ejc.285.1666620455374; Mon, 24 Oct 2022 07:07:35 -0700 (PDT) Received: from thomas-win.localdomain ([213.138.44.237]) by smtp.gmail.com with ESMTPSA id s19-20020a170906455300b0079800b81709sm6056436ejq.219.2022.10.24.07.07.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Oct 2022 07:07:34 -0700 (PDT) From: Thomas Siedel To: ffmpeg-devel@ffmpeg.org Date: Mon, 24 Oct 2022 16:06:36 +0200 Message-Id: <20221024140645.4945-2-thomas.ff@spin-digital.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221024140645.4945-1-thomas.ff@spin-digital.com> References: <20221024140645.4945-1-thomas.ff@spin-digital.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 01/10] avcodec: add enum types for H266/VVC X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Thomas Siedel Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: X2WUKMS2rJPb Add needed types as nal unit types, slice types and vvc typedefs needed for parsing vvc high-level syntax Signed-off-by: Thomas Siedel --- libavcodec/vvc.h | 142 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 142 insertions(+) create mode 100644 libavcodec/vvc.h diff --git a/libavcodec/vvc.h b/libavcodec/vvc.h new file mode 100644 index 0000000000..19979b7017 --- /dev/null +++ b/libavcodec/vvc.h @@ -0,0 +1,142 @@ +/* + * VVC shared code + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_VVC_H +#define AVCODEC_VVC_H + +/** + * Table 5 – NAL unit type codes and NAL unit type classes + * in T-REC-H.266-202008 + */ +enum VVCNALUnitType { + VVC_TRAIL_NUT = 0, + VVC_STSA_NUT = 1, + VVC_RADL_NUT = 2, + VVC_RASL_NUT = 3, + VVC_RSV_VCL_4 = 4, + VVC_RSV_VCL_5 = 5, + VVC_RSV_VCL_6 = 6, + VVC_IDR_W_RADL = 7, + VVC_IDR_N_LP = 8, + VVC_CRA_NUT = 9, + VVC_GDR_NUT = 10, + VVC_RSV_IRAP_11 = 11, + VVC_OPI_NUT = 12, + VVC_DCI_NUT = 13, + VVC_VPS_NUT = 14, + VVC_SPS_NUT = 15, + VVC_PPS_NUT = 16, + VVC_PREFIX_APS_NUT = 17, + VVC_SUFFIX_APS_NUT = 18, + VVC_PH_NUT = 19, + VVC_AUD_NUT = 20, + VVC_EOS_NUT = 21, + VVC_EOB_NUT = 22, + VVC_PREFIX_SEI_NUT = 23, + VVC_SUFFIX_SEI_NUT = 24, + VVC_FD_NUT = 25, + VVC_RSV_NVCL_26 = 26, + VVC_RSV_NVCL_27 = 27, + VVC_UNSPEC_28 = 28, + VVC_UNSPEC_29 = 29, + VVC_UNSPEC_30 = 30, + VVC_UNSPEC_31 = 31, +}; + +enum VVCSliceType { + VVC_SLICE_TYPE_B = 0, + VVC_SLICE_TYPE_P = 1, + VVC_SLICE_TYPE_I = 2, +}; + +enum { + //6.2 we can have 3 sample arrays + VVC_MAX_SAMPLE_ARRAYS = 3, + + //7.4.3.3 vps_max_layers_minus1 is u(6) + VVC_MAX_LAYERS = 64, + + //7.4.3.3 The value of vps_max_sublayers_minus1 shall be in the range of 0 to 6, inclusive + VVC_MAX_SUBLAYERS = 7, + + //7.4.3.3 vps_num_ptls_minus1 is u(8) + VVC_MAX_PTLS = 256, + + //7.4.3.3 vps_num_output_layer_sets_minus2 is u(8) + VVC_MAX_TOTAL_NUM_OLSS = 257, + + // 7.3.2.3: vps_video_parameter_set_id is u(4). + VVC_MAX_VPS_COUNT = 16, + // 7.3.2.4: sps_seq_parameter_set_id is u(4) + VVC_MAX_SPS_COUNT = 16, + // 7.3.2.5: pps_pic_parameter_set_id is u(6) + VVC_MAX_PPS_COUNT = 64, + + // 7.4.4.1: ptl_num_sub_profiles is u(8) + VVC_MAX_SUB_PROFILES = 256, + + // A.4.2: according to (1577), MaxDpbSize is bounded above by 2 * maxDpbPicBuf(8) + VVC_MAX_DPB_SIZE = 16, + + //7.4.3.4 sps_num_ref_pic_lists in range [0, 64] + VVC_MAX_REF_PIC_LISTS = 64, + + //7.4.11 num_ref_entries in range [0, MaxDpbSize + 13] + VVC_MAX_REF_ENTRIES = VVC_MAX_DPB_SIZE + 13, + + //7.4.3.3 sps_num_points_in_qp_table_minus1[i] in range [0, 36 − sps_qp_table_start_minus26[i]], + //sps_qp_table_start_minus26[i] in range [sps_qp_table_start_minus26[i] −26 − QpBdOffset, 36] + //for 10 bitsQpBdOffset is 12, so sps_num_points_in_qp_table_minus1[i] in range [0, 74] + VVC_MAX_POINTS_IN_QP_TABLE = 75, + + // 7.4.6.1: hrd_cpb_cnt_minus1 is in [0, 31]. + VVC_MAX_CPB_CNT = 32, + + // A.4.1: the highest level allows a MaxLumaPs of 35 651 584. + VVC_MAX_LUMA_PS = 35651584, + + // A.4.1: pic_width_in_luma_samples and pic_height_in_luma_samples are + // constrained to be not greater than sqrt(MaxLumaPs * 8). Hence height/ + // width are bounded above by sqrt(8 * 35651584) = 16888.2 samples. + VVC_MAX_WIDTH = 16888, + VVC_MAX_HEIGHT = 16888, + + // A.4.1: table A.1 allows at most 440 tiles per au for any level. + VVC_MAX_TILES_PER_AU = 440, + // A.4.1: table A.1 did not define max tile rows. + // in worest a case, we can have 1x440 tiles picture. + VVC_MAX_TILE_ROWS = VVC_MAX_TILES_PER_AU, + // A.4.1: table A.1 allows at most 20 tile columns for any level. + VVC_MAX_TILE_COLUMNS = 20, + + // A.4.1 table A.1 allows at most 600 slice for any level. + VVC_MAX_SLICES = 600, + + // 7.4.8: in the worst case (!pps_no_pic_partition_flag and + // sps_entropy_coding_sync_enabled_flag are both true), entry points can be + // placed at the beginning of every Ctb row in every tile, giving an + // upper bound of (num_tile_columns_minus1 + 1) * PicHeightInCtbsY - 1. + // Only a stream with very high resolution and perverse parameters could + // get near that, though, so set a lower limit here with the maximum + // possible value for 8K video (at most 135 32x32 Ctb rows). + VVC_MAX_ENTRY_POINTS = VVC_MAX_TILE_COLUMNS * 135, +}; + +#endif /* AVCODEC_VVC_H */ \ No newline at end of file From patchwork Mon Oct 24 14:06:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Siedel X-Patchwork-Id: 38959 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2249140pzb; Mon, 24 Oct 2022 07:08:24 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5GP8hjzalDj3hvvjnbLb5w5zBtgqGDINGZsdwrwDVA1Ub7x2F5ShKZcetk34xxYMoR0Sv5 X-Received: by 2002:a05:6402:2707:b0:45c:d8a3:6cfc with SMTP id y7-20020a056402270700b0045cd8a36cfcmr31123120edd.269.1666620503233; Mon, 24 Oct 2022 07:08:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666620503; cv=none; d=google.com; s=arc-20160816; b=LcCUw8+y8/pr+iDiD7TaKiMGV5xtPeNlLb6zo0wLOHwxwuDvSed34UijLsZ4mzGPuv tMXJcLKy4YJOqmoxfw25fO74eXpX9udRR+YOqZWCvFR/s0xZy8n1AR0FCkf9l8MnlbCw o7aIwWH54fYXulYRVtynTbDHi6NtLr/QZvxzN7z/q54IwErTTZ8rHqgfmSm3cjbvHp7Q r5f5VozezvKVcwn7K9n0TAN1PZe4XYVAgrwbM6B4CA8LGJ56KrtGNfoMpMJjWIa+DQth rBrdzaDd211n3FIeVGtj5ltRtOoeiZK0VuvSfH3dGsMP04Vs2PRxMEUlDW18eHQWG9FW Hevg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=LlmXtK8UTCoT4ix9GsJYyIHxenHV2zZur4cdfYZs5Ww=; b=pM7TTbH1Bhb/x2LbFIIsK4r2YotMAPHB4Nii2HxRoT0mu06ljGQIjREXvoO5V3DAAi oDetfRHb8Em6Z7gOXVuZTjsPus7Hn/G9Svb3BLh9xDVuo+WFi00QWm75sKNMXuIrkQCY J6ME8tRKQLtTOnqFx3gXxwpzPJbq+pl1WJOwxjIsm3jxGp11bD6mN37n3EoFn9jLozsc 3xTMux3jD0xg4xDLmIu2xvxz6wlPtsxmDCKxy9Z0FIE/SGD3mHS1ANnoBIGO/z1ZteZT lg+wHfj6LvE/XhQdRiHzvJFNEh5ggKKRHjiaEpqFAkNpw5IBDD8JzQlqNzPO2TtCTTy3 /KkQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=6lRLm7EV; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id hs40-20020a1709073ea800b0078df2f22f80si30375465ejc.647.2022.10.24.07.08.20; Mon, 24 Oct 2022 07:08:23 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=6lRLm7EV; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4488F68BC0E; Mon, 24 Oct 2022 17:07:47 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f45.google.com (mail-ed1-f45.google.com [209.85.208.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1E07D68BBB4 for ; Mon, 24 Oct 2022 17:07:39 +0300 (EEST) Received: by mail-ed1-f45.google.com with SMTP id x2so4461410edd.2 for ; Mon, 24 Oct 2022 07:07:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spin-digital-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MygNmdPeJ/gM336rAHGAe5pzWw8OBEnJxpl5juSjBKQ=; b=6lRLm7EV5o1pfI4Ixa5BaUD7AiC4kXNktETRYVPrasAJQ5Xguvk/02V+VS9hAjPtew ID4MCnsM+KpAGxeMXPj+peQSuQB3i5Ho4avHlfDbiMwM/7X3N+Nz0QHi4SKltP26CjJD LillmAk9tcak5DCZJsUuTLi2xsr7ZP0x8ujBpfcfZZTl3aXLhjNSQNM4lKxqf3DPLr/X cyga5DCymbTZBDMzy6dQEyMVA4cMFAP5rRpMGHRQ+7c99GvK6WZNxjBCmCPR3ksVL2wl zLXWeTTa1aoydTMEXmgezvmwXBohxfKNqqlszEdD1BxjV+gEJbDoGvCYJqvePzyincDn FFsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MygNmdPeJ/gM336rAHGAe5pzWw8OBEnJxpl5juSjBKQ=; b=Lncl8y4RL++v8X6aFODz/rEw2vKsfXs5BnK3nNk8+k4NNUC61MhiL5wvHyCg0RzBXM LBkhjum20WJFG8ix8vGCgDAi8B6YPMH8g0XyBH10LEiI9MM1IfkqeSi2MgRvhe9B10CX 9ruaX+2mCKwuOQQObOewJs3TtwfksWJC/LK/B/f7+CxOYh6+HQrc80EBfvVWZNWDbs0i lwRgWbXRdm4L948U2f0tO/XEV2ai35uBtg+xw3oXnF8p6jQiQX4+VBy49BoXQ+d1cGoA LTnhjZD1hRykyRvN2NE0ApykxM0n4uwcc0poYHy1nD5U117ySa4cebK1MNQ4xJ8pqjKr YCvA== X-Gm-Message-State: ACrzQf3/rMKuZ/C3dabohyEjH5bLg+neTOPD5muxlbfy8bVL/hEoTk/p 5FYhLv/Xi1R6F3FQU9aYYc/j7me2BYKc4w== X-Received: by 2002:a17:907:25c9:b0:77b:a343:bd62 with SMTP id ae9-20020a17090725c900b0077ba343bd62mr28112972ejc.660.1666620456691; Mon, 24 Oct 2022 07:07:36 -0700 (PDT) Received: from thomas-win.localdomain ([213.138.44.237]) by smtp.gmail.com with ESMTPSA id s19-20020a170906455300b0079800b81709sm6056436ejq.219.2022.10.24.07.07.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Oct 2022 07:07:35 -0700 (PDT) From: Thomas Siedel To: ffmpeg-devel@ffmpeg.org Date: Mon, 24 Oct 2022 16:06:37 +0200 Message-Id: <20221024140645.4945-3-thomas.ff@spin-digital.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221024140645.4945-1-thomas.ff@spin-digital.com> References: <20221024140645.4945-1-thomas.ff@spin-digital.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 02/10] avcodec: add cbs for H266/VVC X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Thomas Siedel Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 6NbgWweAepKc Add CodedBitstreamContext structures to parse VPS,SPS,PPS in VVC nal units Implement parsing(read) and writing of SPS,PPS,VPS,PH,AUD,SEI and slices (TRAIL,STSA,RADL,RASL,IDR,CRA,GDR) Add ff_cbs_type_h266 to cbs types tables and AV_CODEC_ID_H266 to cbs codec ids Signed-off-by: Thomas Siedel --- libavcodec/cbs.c | 6 + libavcodec/cbs_h2645.c | 373 ++- libavcodec/cbs_h266.h | 791 +++++++ libavcodec/cbs_h266_syntax_template.c | 3010 +++++++++++++++++++++++++ libavcodec/cbs_internal.h | 1 + libavcodec/cbs_sei.c | 29 + 6 files changed, 4208 insertions(+), 2 deletions(-) create mode 100644 libavcodec/cbs_h266.h create mode 100644 libavcodec/cbs_h266_syntax_template.c diff --git a/libavcodec/cbs.c b/libavcodec/cbs.c index 504197e06d..dc9cec443b 100644 --- a/libavcodec/cbs.c +++ b/libavcodec/cbs.c @@ -40,6 +40,9 @@ static const CodedBitstreamType *const cbs_type_table[] = { #if CONFIG_CBS_H265 &ff_cbs_type_h265, #endif +#if CONFIG_CBS_H266 + &ff_cbs_type_h266, +#endif #if CONFIG_CBS_JPEG &ff_cbs_type_jpeg, #endif @@ -61,6 +64,9 @@ const enum AVCodecID ff_cbs_all_codec_ids[] = { #if CONFIG_CBS_H265 AV_CODEC_ID_H265, #endif +#if CONFIG_CBS_H266 + AV_CODEC_ID_H266, +#endif #if CONFIG_CBS_JPEG AV_CODEC_ID_MJPEG, #endif diff --git a/libavcodec/cbs_h2645.c b/libavcodec/cbs_h2645.c index 4ee06003c3..0167ee20e7 100644 --- a/libavcodec/cbs_h2645.c +++ b/libavcodec/cbs_h2645.c @@ -24,10 +24,16 @@ #include "cbs_internal.h" #include "cbs_h264.h" #include "cbs_h265.h" +#include "cbs_h266.h" #include "h264.h" #include "h2645_parse.h" #include "hevc.h" +#include "vvc.h" +static av_always_inline unsigned int h266_ceil(unsigned int v, unsigned int align) +{ + return (((v) + (align) - 1) / (align)); +} static int cbs_read_ue_golomb(CodedBitstreamContext *ctx, GetBitContext *gbc, const char *name, const int *subscripts, @@ -255,6 +261,7 @@ static int cbs_h265_payload_extension_present(GetBitContext *gbc, uint32_t paylo #define FUNC_NAME1(rw, codec, name) FUNC_NAME2(rw, codec, name) #define FUNC_H264(name) FUNC_NAME1(READWRITE, h264, name) #define FUNC_H265(name) FUNC_NAME1(READWRITE, h265, name) +#define FUNC_H266(name) FUNC_NAME1(READWRITE, h266, name) #define FUNC_SEI(name) FUNC_NAME1(READWRITE, sei, name) #define SUBSCRIPTS(subs, ...) (subs > 0 ? ((int[subs + 1]){ subs, __VA_ARGS__ }) : NULL) @@ -369,6 +376,10 @@ static int cbs_h2645_read_more_rbsp_data(GetBitContext *gbc) #include "cbs_h265_syntax_template.c" #undef FUNC +#define FUNC(name) FUNC_H266(name) +#include "cbs_h266_syntax_template.c" +#undef FUNC + #undef READ #undef READWRITE #undef RWContext @@ -447,6 +458,10 @@ static int cbs_h2645_read_more_rbsp_data(GetBitContext *gbc) #include "cbs_h265_syntax_template.c" #undef FUNC +#define FUNC(name) FUNC_H266(name) +#include "cbs_h266_syntax_template.c" +#undef FUNC + #undef WRITE #undef READWRITE #undef RWContext @@ -476,8 +491,9 @@ static int cbs_h2645_fragment_add_nals(CodedBitstreamContext *ctx, const H2645NAL *nal = &packet->nals[i]; AVBufferRef *ref; size_t size = nal->size; + enum AVCodecID codec_id = ctx->codec->codec_id; - if (nal->nuh_layer_id > 0) + if (codec_id != AV_CODEC_ID_VVC && nal->nuh_layer_id > 0) continue; // Remove trailing zeroes. @@ -513,6 +529,12 @@ static int cbs_h2645_split_fragment(CodedBitstreamContext *ctx, if (frag->data_size == 0) return 0; + if (codec_id == AV_CODEC_ID_VVC) { + //we deactive picture header here to avoid reuse previous au's ph. + CodedBitstreamH266Context *h266 = ctx->priv_data; + h266->priv.ph = NULL; + } + if (header && frag->data[0] && codec_id == AV_CODEC_ID_H264) { // AVCC header. size_t size, start, end; @@ -685,6 +707,26 @@ cbs_h2645_replace_ps(4, PPS, pps, pic_parameter_set_id) cbs_h2645_replace_ps(5, VPS, vps, vps_video_parameter_set_id) cbs_h2645_replace_ps(5, SPS, sps, sps_seq_parameter_set_id) cbs_h2645_replace_ps(5, PPS, pps, pps_pic_parameter_set_id) +cbs_h2645_replace_ps(6, VPS, vps, vps_video_parameter_set_id) +cbs_h2645_replace_ps(6, SPS, sps, sps_seq_parameter_set_id) +cbs_h2645_replace_ps(6, PPS, pps, pps_pic_parameter_set_id) + +static int cbs_h266_replace_ph(CodedBitstreamContext *ctx, + CodedBitstreamUnit *unit) +{ + CodedBitstreamH266Context *h266 = ctx->priv_data; + int err; + + h266->priv.ph = NULL; + err = ff_cbs_make_unit_refcounted(ctx, unit); + if (err < 0) + return err; + err = av_buffer_replace(&h266->priv.ph_ref, unit->content_ref); + if (err < 0) + return err; + h266->priv.ph = (H266RawPH*)h266->priv.ph_ref->data; + return 0; +} static int cbs_h264_read_nal_unit(CodedBitstreamContext *ctx, CodedBitstreamUnit *unit) @@ -926,6 +968,130 @@ static int cbs_h265_read_nal_unit(CodedBitstreamContext *ctx, return 0; } +static int cbs_h266_read_nal_unit(CodedBitstreamContext *ctx, + CodedBitstreamUnit *unit) +{ + GetBitContext gbc; + int err; + + err = init_get_bits8(&gbc, unit->data, unit->data_size); + if (err < 0) + return err; + + err = ff_cbs_alloc_unit_content(ctx, unit); + if (err < 0) + return err; + + switch (unit->type) { + case VVC_VPS_NUT: + { + H266RawVPS *vps = unit->content; + + err = cbs_h266_read_vps(ctx, &gbc, vps); + if (err < 0) + return err; + + err = cbs_h266_replace_vps(ctx, unit); + if (err < 0) + return err; + } + break; + case VVC_SPS_NUT: + { + H266RawSPS *sps = unit->content; + + err = cbs_h266_read_sps(ctx, &gbc, sps); + if (err < 0) + return err; + + err = cbs_h266_replace_sps(ctx, unit); + if (err < 0) + return err; + } + break; + + case VVC_PPS_NUT: + { + H266RawPPS *pps = unit->content; + + err = cbs_h266_read_pps(ctx, &gbc, pps); + if (err < 0) + return err; + + err = cbs_h266_replace_pps(ctx, unit); + if (err < 0) + return err; + } + break; + + case VVC_PH_NUT: + { + H266RawPH *ph = unit->content; + err = cbs_h266_read_ph(ctx, &gbc, ph); + if (err < 0) + return err; + err = cbs_h266_replace_ph(ctx, unit); + if (err < 0) + return err; + } + break; + + case VVC_TRAIL_NUT: + case VVC_STSA_NUT: + case VVC_RADL_NUT: + case VVC_RASL_NUT: + case VVC_IDR_W_RADL: + case VVC_IDR_N_LP: + case VVC_CRA_NUT: + case VVC_GDR_NUT: + { + H266RawSlice *slice = unit->content; + int pos, len; + + err = cbs_h266_read_slice_header(ctx, &gbc, &slice->header); + if (err < 0) + return err; + + if (!cbs_h2645_read_more_rbsp_data(&gbc)) + return AVERROR_INVALIDDATA; + + pos = get_bits_count(&gbc); + len = unit->data_size; + + slice->data_size = len - pos / 8; + slice->data_ref = av_buffer_ref(unit->data_ref); + if (!slice->data_ref) + return AVERROR(ENOMEM); + slice->data = unit->data + pos / 8; + slice->data_bit_start = pos % 8; + } + break; + + case VVC_AUD_NUT: + { + err = cbs_h266_read_aud(ctx, &gbc, unit->content); + if (err < 0) + return err; + } + break; + + case VVC_PREFIX_SEI_NUT: + case VVC_SUFFIX_SEI_NUT: + { + err = cbs_h266_read_sei(ctx, &gbc, unit->content, + unit->type == VVC_PREFIX_SEI_NUT); + + if (err < 0) + return err; + } + break; + + default: + return AVERROR(ENOSYS); + } + return 0; +} + static int cbs_h2645_write_slice_data(CodedBitstreamContext *ctx, PutBitContext *pbc, const uint8_t *data, size_t data_size, int data_bit_start) @@ -1213,11 +1379,127 @@ static int cbs_h265_write_nal_unit(CodedBitstreamContext *ctx, return 0; } +static int cbs_h266_write_nal_unit(CodedBitstreamContext *ctx, + CodedBitstreamUnit *unit, + PutBitContext *pbc) +{ + int err; + + switch (unit->type) { + case VVC_VPS_NUT: + { + H266RawVPS *vps = unit->content; + + err = cbs_h266_write_vps(ctx, pbc, vps); + if (err < 0) + return err; + + err = cbs_h266_replace_vps(ctx, unit); + if (err < 0) + return err; + } + break; + case VVC_SPS_NUT: + { + H266RawSPS *sps = unit->content; + + err = cbs_h266_write_sps(ctx, pbc, sps); + if (err < 0) + return err; + + err = cbs_h266_replace_sps(ctx, unit); + if (err < 0) + return err; + } + break; + + case VVC_PPS_NUT: + { + H266RawPPS *pps = unit->content; + + err = cbs_h266_write_pps(ctx, pbc, pps); + if (err < 0) + return err; + + err = cbs_h266_replace_pps(ctx, unit); + if (err < 0) + return err; + } + break; + + case VVC_PH_NUT: + { + H266RawPH *ph = unit->content; + err = cbs_h266_write_ph(ctx, pbc, ph); + if (err < 0) + return err; + + err = cbs_h266_replace_ph(ctx, unit); + if (err < 0) + return err; + } + break; + + case VVC_TRAIL_NUT: + case VVC_STSA_NUT: + case VVC_RADL_NUT: + case VVC_RASL_NUT: + case VVC_IDR_W_RADL: + case VVC_IDR_N_LP: + case VVC_CRA_NUT: + case VVC_GDR_NUT: + { + H266RawSlice *slice = unit->content; + + err = cbs_h266_write_slice_header(ctx, pbc, &slice->header); + if (err < 0) + return err; + + if (slice->data) { + err = cbs_h2645_write_slice_data(ctx, pbc, slice->data, + slice->data_size, + slice->data_bit_start); + if (err < 0) + return err; + } else { + // No slice data - that was just the header. + } + } + break; + + case VVC_AUD_NUT: + { + err = cbs_h266_write_aud(ctx, pbc, unit->content); + if (err < 0) + return err; + } + break; + + case VVC_PREFIX_SEI_NUT: + case VVC_SUFFIX_SEI_NUT: + { + err = cbs_h266_write_sei(ctx, pbc, unit->content, + unit->type == VVC_PREFIX_SEI_NUT); + + if (err < 0) + return err; + } + break; + + default: + av_log(ctx->log_ctx, AV_LOG_ERROR, "Write unimplemented for " + "NAL unit type %"PRIu32".\n", unit->type); + return AVERROR_PATCHWELCOME; + } + + return 0; +} + static int cbs_h2645_unit_requires_zero_byte(enum AVCodecID codec_id, CodedBitstreamUnitType type, int nal_unit_index) { - // Section B.1.2 in H.264, section B.2.2 in H.265. + // Section B.1.2 in H.264, section B.2.2 in H.265, H.266. if (nal_unit_index == 0) { // Assume that this is the first NAL unit in an access unit. return 1; @@ -1226,6 +1508,8 @@ static int cbs_h2645_unit_requires_zero_byte(enum AVCodecID codec_id, return type == H264_NAL_SPS || type == H264_NAL_PPS; if (codec_id == AV_CODEC_ID_HEVC) return type == HEVC_NAL_VPS || type == HEVC_NAL_SPS || type == HEVC_NAL_PPS; + if (codec_id == AV_CODEC_ID_VVC) + return type >= VVC_OPI_NUT && type <= VVC_SUFFIX_APS_NUT; return 0; } @@ -1377,6 +1661,35 @@ static void cbs_h265_close(CodedBitstreamContext *ctx) av_buffer_unref(&h265->pps_ref[i]); } +static void cbs_h266_flush(CodedBitstreamContext *ctx) +{ + CodedBitstreamH266Context *h266 = ctx->priv_data; + + for (int i = 0; i < FF_ARRAY_ELEMS(h266->vps); i++) { + av_buffer_unref(&h266->vps_ref[i]); + h266->vps[i] = NULL; + } + + for (int i = 0; i < FF_ARRAY_ELEMS(h266->sps); i++) { + av_buffer_unref(&h266->sps_ref[i]); + h266->sps[i] = NULL; + } + for (int i = 0; i < FF_ARRAY_ELEMS(h266->pps); i++) { + av_buffer_unref(&h266->pps_ref[i]); + h266->pps[i] = NULL; + } + av_buffer_unref(&h266->priv.ph_ref); + h266->priv.ph = NULL; +} + +static void cbs_h266_close(CodedBitstreamContext *ctx) +{ + CodedBitstreamH266Context *h266 = ctx->priv_data; + + cbs_h266_flush(ctx); + ff_h2645_packet_uninit(&h266->common.read_packet); + } + static void cbs_h264_free_sei(void *opaque, uint8_t *content) { H264RawSEI *sei = (H264RawSEI*)content; @@ -1431,6 +1744,33 @@ static const CodedBitstreamUnitTypeDescriptor cbs_h265_unit_types[] = { CBS_UNIT_TYPE_END_OF_LIST }; +static void cbs_h266_free_sei(void *opaque, uint8_t *content) +{ + H265RawSEI *sei = (H265RawSEI*)content; + ff_cbs_sei_free_message_list(&sei->message_list); + av_free(content); +} + +static const CodedBitstreamUnitTypeDescriptor cbs_h266_unit_types[] = { + CBS_UNIT_TYPE_INTERNAL_REF(VVC_VPS_NUT, H266RawVPS, extension_data.data), + CBS_UNIT_TYPE_INTERNAL_REF(VVC_SPS_NUT, H266RawSPS, extension_data.data), + CBS_UNIT_TYPE_INTERNAL_REF(VVC_PPS_NUT, H266RawPPS, extension_data.data), + + CBS_UNIT_TYPE_POD(VVC_PH_NUT , H266RawPH), + CBS_UNIT_TYPE_POD(VVC_AUD_NUT, H266RawAUD), + + CBS_UNIT_RANGE_INTERNAL_REF(VVC_TRAIL_NUT, VVC_RASL_NUT, + H266RawSlice, data), + + CBS_UNIT_RANGE_INTERNAL_REF(VVC_IDR_W_RADL, VVC_GDR_NUT, + H266RawSlice, data), + + CBS_UNIT_TYPES_COMPLEX((VVC_PREFIX_SEI_NUT, VVC_SUFFIX_SEI_NUT), + H266RawSEI, cbs_h266_free_sei), + + CBS_UNIT_TYPE_END_OF_LIST +}; + const CodedBitstreamType ff_cbs_type_h264 = { .codec_id = AV_CODEC_ID_H264, @@ -1463,6 +1803,22 @@ const CodedBitstreamType ff_cbs_type_h265 = { .close = &cbs_h265_close, }; +const CodedBitstreamType ff_cbs_type_h266 = { + .codec_id = AV_CODEC_ID_VVC, + + .priv_data_size = sizeof(CodedBitstreamH266Context), + + .unit_types = cbs_h266_unit_types, + + .split_fragment = &cbs_h2645_split_fragment, + .read_unit = &cbs_h266_read_nal_unit, + .write_unit = &cbs_h266_write_nal_unit, + .assemble_fragment = &cbs_h2645_assemble_fragment, + + .flush = &cbs_h266_flush, + .close = &cbs_h266_close, +}; + static const SEIMessageTypeDescriptor cbs_sei_common_types[] = { { SEI_TYPE_FILLER_PAYLOAD, @@ -1607,6 +1963,16 @@ static const SEIMessageTypeDescriptor cbs_sei_h265_types[] = { SEI_MESSAGE_TYPE_END }; +static const SEIMessageTypeDescriptor cbs_sei_h266_types[] = { + { + SEI_TYPE_DECODED_PICTURE_HASH, + 0, 1, + sizeof(H266RawSEIDecodedPictureHash), + SEI_MESSAGE_RW(h266, sei_decoded_picture_hash), + }, + SEI_MESSAGE_TYPE_END +}; + const SEIMessageTypeDescriptor *ff_cbs_sei_find_type(CodedBitstreamContext *ctx, int payload_type) { @@ -1625,6 +1991,9 @@ const SEIMessageTypeDescriptor *ff_cbs_sei_find_type(CodedBitstreamContext *ctx, case AV_CODEC_ID_H265: codec_list = cbs_sei_h265_types; break; + case AV_CODEC_ID_H266: + codec_list = cbs_sei_h266_types; + break; default: return NULL; } diff --git a/libavcodec/cbs_h266.h b/libavcodec/cbs_h266.h new file mode 100644 index 0000000000..f2f72f780d --- /dev/null +++ b/libavcodec/cbs_h266.h @@ -0,0 +1,791 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_CBS_H266_H +#define AVCODEC_CBS_H266_H + +#include +#include + +#include "cbs_h2645.h" +#include "cbs_sei.h" +#include "vvc.h" + +typedef struct H266RawNALUnitHeader { + uint8_t nuh_layer_id; + uint8_t nal_unit_type; + uint8_t nuh_temporal_id_plus1; + uint8_t nuh_reserved_zero_bit; +} H266RawNALUnitHeader; + +typedef struct H266GeneralConstraintsInfo { + uint8_t gci_present_flag; + /* general */ + uint8_t gci_intra_only_constraint_flag; + uint8_t gci_all_layers_independent_constraint_flag; + uint8_t gci_one_au_only_constraint_flag; + + /* picture format */ + uint8_t gci_sixteen_minus_max_bitdepth_constraint_idc; + uint8_t gci_three_minus_max_chroma_format_constraint_idc; + + /* NAL unit type related */ + uint8_t gci_no_mixed_nalu_types_in_pic_constraint_flag; + uint8_t gci_no_trail_constraint_flag; + uint8_t gci_no_stsa_constraint_flag; + uint8_t gci_no_rasl_constraint_flag; + uint8_t gci_no_radl_constraint_flag; + uint8_t gci_no_idr_constraint_flag; + uint8_t gci_no_cra_constraint_flag; + uint8_t gci_no_gdr_constraint_flag; + uint8_t gci_no_aps_constraint_flag; + uint8_t gci_no_idr_rpl_constraint_flag; + + /* tile, slice, subpicture partitioning */ + uint8_t gci_one_tile_per_pic_constraint_flag; + uint8_t gci_pic_header_in_slice_header_constraint_flag; + uint8_t gci_one_slice_per_pic_constraint_flag; + uint8_t gci_no_rectangular_slice_constraint_flag; + uint8_t gci_one_slice_per_subpic_constraint_flag; + uint8_t gci_no_subpic_info_constraint_flag; + + /* CTU and block partitioning */ + uint8_t gci_three_minus_max_log2_ctu_size_constraint_idc; + uint8_t gci_no_partition_constraints_override_constraint_flag; + uint8_t gci_no_mtt_constraint_flag; + uint8_t gci_no_qtbtt_dual_tree_intra_constraint_flag; + + /* intra */ + uint8_t gci_no_palette_constraint_flag; + uint8_t gci_no_ibc_constraint_flag; + uint8_t gci_no_isp_constraint_flag; + uint8_t gci_no_mrl_constraint_flag; + uint8_t gci_no_mip_constraint_flag; + uint8_t gci_no_cclm_constraint_flag; + + /* inter */ + uint8_t gci_no_ref_pic_resampling_constraint_flag; + uint8_t gci_no_res_change_in_clvs_constraint_flag;; + uint8_t gci_no_weighted_prediction_constraint_flag; + uint8_t gci_no_ref_wraparound_constraint_flag; + uint8_t gci_no_temporal_mvp_constraint_flag; + uint8_t gci_no_sbtmvp_constraint_flag; + uint8_t gci_no_amvr_constraint_flag; + uint8_t gci_no_bdof_constraint_flag; + uint8_t gci_no_smvd_constraint_flag; + uint8_t gci_no_dmvr_constraint_flag; + uint8_t gci_no_mmvd_constraint_flag; + uint8_t gci_no_affine_motion_constraint_flag; + uint8_t gci_no_prof_constraint_flag; + uint8_t gci_no_bcw_constraint_flag; + uint8_t gci_no_ciip_constraint_flag; + uint8_t gci_no_gpm_constraint_flag; + + /* transform, quantization, residual */ + uint8_t gci_no_luma_transform_size_64_constraint_flag; + uint8_t gci_no_transform_skip_constraint_flag; + uint8_t gci_no_bdpcm_constraint_flag; + uint8_t gci_no_mts_constraint_flag; + uint8_t gci_no_lfnst_constraint_flag; + uint8_t gci_no_joint_cbcr_constraint_flag; + uint8_t gci_no_sbt_constraint_flag; + uint8_t gci_no_act_constraint_flag; + uint8_t gci_no_explicit_scaling_list_constraint_flag; + uint8_t gci_no_dep_quant_constraint_flag; + uint8_t gci_no_sign_data_hiding_constraint_flag; + uint8_t gci_no_cu_qp_delta_constraint_flag; + uint8_t gci_no_chroma_qp_offset_constraint_flag; + + /* loop filter */ + uint8_t gci_no_sao_constraint_flag; + uint8_t gci_no_alf_constraint_flag; + uint8_t gci_no_ccalf_constraint_flag; + uint8_t gci_no_lmcs_constraint_flag; + uint8_t gci_no_ladf_constraint_flag; + uint8_t gci_no_virtual_boundaries_constraint_flag; + uint8_t gci_num_reserved_bits; + uint8_t gci_reserved_zero_bit[255]; +} H266GeneralConstraintsInfo; + +typedef struct H266RawProfileTierLevel { + uint8_t general_profile_idc; + uint8_t general_tier_flag; + uint8_t general_level_idc; + uint8_t ptl_frame_only_constraint_flag; + uint8_t ptl_multilayer_enabled_flag; + H266GeneralConstraintsInfo general_constraints_info; + uint8_t ptl_sublayer_level_present_flag[VVC_MAX_SUBLAYERS - 1]; + uint8_t sublayer_level_idc[VVC_MAX_SUBLAYERS - 1]; + uint8_t ptl_num_sub_profiles; + uint32_t general_sub_profile_idc[VVC_MAX_SUB_PROFILES]; + + uint8_t ptl_reserved_zero_bit; +} H266RawProfileTierLevel; + +typedef struct H266RawExtensionData { + uint8_t *data; + AVBufferRef *data_ref; + size_t bit_length; +} H266RawExtensionData; + +typedef struct H266DpbParameters { + uint8_t dpb_max_dec_pic_buffering_minus1[VVC_MAX_SUBLAYERS]; + uint8_t dpb_max_num_reorder_pics[VVC_MAX_SUBLAYERS]; + uint8_t dpb_max_latency_increase_plus1[VVC_MAX_SUBLAYERS]; +} H266DpbParameters; + +typedef struct H266RefPicListStruct { + uint8_t num_ref_entries; + uint8_t ltrp_in_header_flag; + uint8_t inter_layer_ref_pic_flag[VVC_MAX_REF_ENTRIES]; + uint8_t st_ref_pic_flag[VVC_MAX_REF_ENTRIES]; + uint8_t abs_delta_poc_st[VVC_MAX_REF_ENTRIES]; + uint8_t strp_entry_sign_flag[VVC_MAX_REF_ENTRIES]; + uint8_t rpls_poc_lsb_lt[VVC_MAX_REF_ENTRIES]; + uint8_t ilrp_idx[VVC_MAX_REF_ENTRIES]; +} H266RefPicListStruct; + +typedef struct H266RefPicLists { + uint8_t rpl_sps_flag[2]; + uint8_t rpl_idx[2]; + H266RefPicListStruct rpl_ref_list[2]; + uint16_t poc_lsb_lt[2][VVC_MAX_REF_ENTRIES]; + uint8_t delta_poc_msb_cycle_present_flag[2][VVC_MAX_REF_ENTRIES]; + uint16_t delta_poc_msb_cycle_lt[2][VVC_MAX_REF_ENTRIES]; +} H266RefPicLists; + +typedef struct H266RawGeneralTimingHrdParameters { + uint32_t num_units_in_tick; + uint32_t time_scale; + uint8_t general_nal_hrd_params_present_flag; + uint8_t general_vcl_hrd_params_present_flag; + uint8_t general_same_pic_timing_in_all_ols_flag; + uint8_t general_du_hrd_params_present_flag; + uint8_t tick_divisor_minus2; + uint8_t bit_rate_scale; + uint8_t cpb_size_scale; + uint8_t cpb_size_du_scale; + uint8_t hrd_cpb_cnt_minus1; +} H266RawGeneralTimingHrdParameters; + +typedef struct H266RawSubLayerHRDParameters { + uint32_t bit_rate_value_minus1[VVC_MAX_SUBLAYERS][VVC_MAX_CPB_CNT]; + uint32_t cpb_size_value_minus1[VVC_MAX_SUBLAYERS][VVC_MAX_CPB_CNT]; + uint32_t cpb_size_du_value_minus1[VVC_MAX_SUBLAYERS][VVC_MAX_CPB_CNT]; + uint32_t bit_rate_du_value_minus1[VVC_MAX_SUBLAYERS][VVC_MAX_CPB_CNT]; + uint8_t cbr_flag[VVC_MAX_SUBLAYERS][VVC_MAX_CPB_CNT]; +} H266RawSubLayerHRDParameters; + +typedef struct H266RawOlsTimingHrdParameters { + uint8_t fixed_pic_rate_general_flag[VVC_MAX_SUBLAYERS]; + uint8_t fixed_pic_rate_within_cvs_flag[VVC_MAX_SUBLAYERS]; + uint16_t elemental_duration_in_tc_minus1[VVC_MAX_SUBLAYERS]; + uint8_t low_delay_hrd_flag[VVC_MAX_SUBLAYERS]; + H266RawSubLayerHRDParameters nal_sub_layer_hrd_parameters; + H266RawSubLayerHRDParameters vcl_sub_layer_hrd_parameters; +} H266RawOlsTimingHrdParameters; + +typedef struct H266RawVUI { + uint8_t vui_progressive_source_flag; + uint8_t vui_interlaced_source_flag; + uint8_t vui_non_packed_constraint_flag; + uint8_t vui_non_projected_constraint_flag; + + uint8_t vui_aspect_ratio_info_present_flag; + uint8_t vui_aspect_ratio_constant_flag; + uint8_t vui_aspect_ratio_idc; + + uint16_t vui_sar_width; + uint16_t vui_sar_height;; + + uint8_t vui_overscan_info_present_flag; + uint8_t vui_overscan_appropriate_flag; + + uint8_t vui_colour_description_present_flag; + uint8_t vui_colour_primaries; + + uint8_t vui_transfer_characteristics; + uint8_t vui_matrix_coeffs; + uint8_t vui_full_range_flag; + + uint8_t vui_chroma_loc_info_present_flag; + uint8_t vui_chroma_sample_loc_type_frame; + uint8_t vui_chroma_sample_loc_type_top_field; + uint8_t vui_chroma_sample_loc_type_bottom_field; + H266RawExtensionData extension_data; +} H266RawVUI; + +typedef struct H266RawVPS { + H266RawNALUnitHeader nal_unit_header; + + uint8_t vps_video_parameter_set_id; + uint8_t vps_max_layers_minus1; + uint8_t vps_max_sublayers_minus1; + uint8_t vps_default_ptl_dpb_hrd_max_tid_flag; + uint8_t vps_all_independent_layers_flag; + uint8_t vps_layer_id[VVC_MAX_LAYERS]; + uint8_t vps_independent_layer_flag[VVC_MAX_LAYERS]; + uint8_t vps_max_tid_ref_present_flag[VVC_MAX_LAYERS]; + uint8_t vps_direct_ref_layer_flag[VVC_MAX_LAYERS][VVC_MAX_LAYERS - 1]; + uint8_t vps_max_tid_il_ref_pics_plus1[VVC_MAX_LAYERS][VVC_MAX_LAYERS - 1]; + uint8_t vps_each_layer_is_an_ols_flag; + uint8_t vps_ols_mode_idc; + uint8_t vps_num_output_layer_sets_minus2; + uint8_t vps_ols_output_layer_flag[VVC_MAX_TOTAL_NUM_OLSS][VVC_MAX_LAYERS]; + + uint8_t vps_num_ptls_minus1; + uint8_t vps_pt_present_flag[VVC_MAX_PTLS]; + uint8_t vps_ptl_max_tid[VVC_MAX_PTLS]; + H266RawProfileTierLevel vps_profile_tier_level[VVC_MAX_PTLS]; + uint8_t vps_ols_ptl_idx[VVC_MAX_TOTAL_NUM_OLSS]; + + uint16_t vps_num_dpb_params_minus1; + uint8_t vps_sublayer_dpb_params_present_flag; + uint8_t vps_dpb_max_tid[VVC_MAX_TOTAL_NUM_OLSS]; + H266DpbParameters vps_dpb_params[VVC_MAX_TOTAL_NUM_OLSS]; + uint16_t vps_ols_dpb_pic_width[VVC_MAX_TOTAL_NUM_OLSS]; + uint16_t vps_ols_dpb_pic_height[VVC_MAX_TOTAL_NUM_OLSS]; + uint8_t vps_ols_dpb_chroma_format[VVC_MAX_TOTAL_NUM_OLSS]; + uint8_t vps_ols_dpb_bitdepth_minus8[VVC_MAX_TOTAL_NUM_OLSS]; + uint16_t vps_ols_dpb_params_idx[VVC_MAX_TOTAL_NUM_OLSS]; + + uint8_t vps_timing_hrd_params_present_flag; + H266RawGeneralTimingHrdParameters vps_general_timing_hrd_parameters; + uint8_t vps_sublayer_cpb_params_present_flag; + uint16_t vps_num_ols_timing_hrd_params_minus1; + uint8_t vps_hrd_max_tid[VVC_MAX_TOTAL_NUM_OLSS]; + H266RawOlsTimingHrdParameters vps_ols_timing_hrd_parameters; + uint8_t vps_ols_timing_hrd_idx[VVC_MAX_TOTAL_NUM_OLSS]; + + uint8_t vps_extension_flag; + H266RawExtensionData extension_data; +} H266RawVPS; + +typedef struct H266RawSPS { + H266RawNALUnitHeader nal_unit_header; + + uint8_t sps_seq_parameter_set_id; + uint8_t sps_video_parameter_set_id; + uint8_t sps_max_sublayers_minus1; + uint8_t sps_chroma_format_idc; + uint8_t sps_log2_ctu_size_minus5; + uint8_t sps_ptl_dpb_hrd_params_present_flag; + H266RawProfileTierLevel profile_tier_level; + uint8_t sps_gdr_enabled_flag; + uint8_t sps_ref_pic_resampling_enabled_flag; + uint8_t sps_res_change_in_clvs_allowed_flag; + + uint16_t sps_pic_width_max_in_luma_samples; + uint16_t sps_pic_height_max_in_luma_samples; + + uint8_t sps_conformance_window_flag; + uint16_t sps_conf_win_left_offset; + uint16_t sps_conf_win_right_offset; + uint16_t sps_conf_win_top_offset; + uint16_t sps_conf_win_bottom_offset; + + uint8_t sps_subpic_info_present_flag; + uint16_t sps_num_subpics_minus1; + uint8_t sps_independent_subpics_flag; + uint8_t sps_subpic_same_size_flag; + uint16_t sps_subpic_ctu_top_left_x[VVC_MAX_SLICES]; + uint16_t sps_subpic_ctu_top_left_y[VVC_MAX_SLICES]; + uint16_t sps_subpic_width_minus1[VVC_MAX_SLICES]; + uint16_t sps_subpic_height_minus1[VVC_MAX_SLICES]; + uint8_t sps_subpic_treated_as_pic_flag[VVC_MAX_SLICES]; + uint8_t sps_loop_filter_across_subpic_enabled_flag[VVC_MAX_SLICES]; + uint8_t sps_subpic_id_len_minus1; + uint8_t sps_subpic_id_mapping_explicitly_signalled_flag; + uint8_t sps_subpic_id_mapping_present_flag; + uint32_t sps_subpic_id[VVC_MAX_SLICES]; + + + uint8_t sps_bitdepth_minus8; + uint8_t sps_entropy_coding_sync_enabled_flag; + uint8_t sps_entry_point_offsets_present_flag; + + uint8_t sps_log2_max_pic_order_cnt_lsb_minus4; + uint8_t sps_poc_msb_cycle_flag; + uint8_t sps_poc_msb_cycle_len_minus1; + + uint8_t sps_num_extra_ph_bytes; + uint8_t sps_extra_ph_bit_present_flag[16]; + + uint8_t sps_num_extra_sh_bytes; + uint8_t sps_extra_sh_bit_present_flag[16]; + + uint8_t sps_sublayer_dpb_params_flag; + H266DpbParameters sps_dpb_params; + + uint8_t sps_log2_min_luma_coding_block_size_minus2; + uint8_t sps_partition_constraints_override_enabled_flag; + uint8_t sps_log2_diff_min_qt_min_cb_intra_slice_luma; + uint8_t sps_max_mtt_hierarchy_depth_intra_slice_luma; + uint8_t sps_log2_diff_max_bt_min_qt_intra_slice_luma; + uint8_t sps_log2_diff_max_tt_min_qt_intra_slice_luma; + + uint8_t sps_qtbtt_dual_tree_intra_flag; + uint8_t sps_log2_diff_min_qt_min_cb_intra_slice_chroma; + uint8_t sps_max_mtt_hierarchy_depth_intra_slice_chroma; + uint8_t sps_log2_diff_max_bt_min_qt_intra_slice_chroma; + uint8_t sps_log2_diff_max_tt_min_qt_intra_slice_chroma; + + uint8_t sps_log2_diff_min_qt_min_cb_inter_slice; + uint8_t sps_max_mtt_hierarchy_depth_inter_slice; + uint8_t sps_log2_diff_max_bt_min_qt_inter_slice; + uint8_t sps_log2_diff_max_tt_min_qt_inter_slice; + + uint8_t sps_max_luma_transform_size_64_flag; + + uint8_t sps_transform_skip_enabled_flag; + uint8_t sps_log2_transform_skip_max_size_minus2; + uint8_t sps_bdpcm_enabled_flag; + + uint8_t sps_mts_enabled_flag; + uint8_t sps_explicit_mts_intra_enabled_flag; + uint8_t sps_explicit_mts_inter_enabled_flag; + + uint8_t sps_lfnst_enabled_flag; + + uint8_t sps_joint_cbcr_enabled_flag; + uint8_t sps_same_qp_table_for_chroma_flag; + + int8_t sps_qp_table_start_minus26[VVC_MAX_SAMPLE_ARRAYS]; + uint8_t sps_num_points_in_qp_table_minus1[VVC_MAX_SAMPLE_ARRAYS]; + uint8_t sps_delta_qp_in_val_minus1[VVC_MAX_SAMPLE_ARRAYS][VVC_MAX_POINTS_IN_QP_TABLE]; + uint8_t sps_delta_qp_diff_val[VVC_MAX_SAMPLE_ARRAYS][VVC_MAX_POINTS_IN_QP_TABLE]; + + uint8_t sps_sao_enabled_flag; + uint8_t sps_alf_enabled_flag; + uint8_t sps_ccalf_enabled_flag; + uint8_t sps_lmcs_enabled_flag; + uint8_t sps_weighted_pred_flag; + uint8_t sps_weighted_bipred_flag; + uint8_t sps_long_term_ref_pics_flag; + uint8_t sps_inter_layer_prediction_enabled_flag; + uint8_t sps_idr_rpl_present_flag; + uint8_t sps_rpl1_same_as_rpl0_flag; + + uint8_t sps_num_ref_pic_lists[2]; + H266RefPicListStruct sps_ref_pic_list_struct[2][VVC_MAX_REF_PIC_LISTS]; + + uint8_t sps_ref_wraparound_enabled_flag; + uint8_t sps_temporal_mvp_enabled_flag; + uint8_t sps_sbtmvp_enabled_flag; + uint8_t sps_amvr_enabled_flag; + uint8_t sps_bdof_enabled_flag; + uint8_t sps_bdof_control_present_in_ph_flag; + uint8_t sps_smvd_enabled_flag; + uint8_t sps_dmvr_enabled_flag; + uint8_t sps_dmvr_control_present_in_ph_flag; + uint8_t sps_mmvd_enabled_flag; + uint8_t sps_mmvd_fullpel_only_enabled_flag; + uint8_t sps_six_minus_max_num_merge_cand; + uint8_t sps_sbt_enabled_flag; + uint8_t sps_affine_enabled_flag; + uint8_t sps_five_minus_max_num_subblock_merge_cand; + uint8_t sps_6param_affine_enabled_flag; + uint8_t sps_affine_amvr_enabled_flag; + uint8_t sps_affine_prof_enabled_flag; + uint8_t sps_prof_control_present_in_ph_flag; + uint8_t sps_bcw_enabled_flag; + uint8_t sps_ciip_enabled_flag; + uint8_t sps_gpm_enabled_flag; + uint8_t sps_max_num_merge_cand_minus_max_num_gpm_cand; + uint8_t sps_log2_parallel_merge_level_minus2; + uint8_t sps_isp_enabled_flag; + uint8_t sps_mrl_enabled_flag; + uint8_t sps_mip_enabled_flag; + uint8_t sps_cclm_enabled_flag; + uint8_t sps_chroma_horizontal_collocated_flag; + uint8_t sps_chroma_vertical_collocated_flag; + uint8_t sps_palette_enabled_flag; + uint8_t sps_act_enabled_flag; + uint8_t sps_min_qp_prime_ts; + uint8_t sps_ibc_enabled_flag; + uint8_t sps_six_minus_max_num_ibc_merge_cand; + uint8_t sps_ladf_enabled_flag; + uint8_t sps_num_ladf_intervals_minus2; + int8_t sps_ladf_lowest_interval_qp_offset; + int8_t sps_ladf_qp_offset[4]; + uint16_t sps_ladf_delta_threshold_minus1[4]; + + uint8_t sps_explicit_scaling_list_enabled_flag; + uint8_t sps_scaling_matrix_for_lfnst_disabled_flag; + uint8_t sps_scaling_matrix_for_alternative_colour_space_disabled_flag; + uint8_t sps_scaling_matrix_designated_colour_space_flag; + uint8_t sps_dep_quant_enabled_flag; + uint8_t sps_sign_data_hiding_enabled_flag; + + uint8_t sps_virtual_boundaries_enabled_flag; + uint8_t sps_virtual_boundaries_present_flag; + uint8_t sps_num_ver_virtual_boundaries; + uint16_t sps_virtual_boundary_pos_x_minus1[3]; + uint8_t sps_num_hor_virtual_boundaries; + uint16_t sps_virtual_boundary_pos_y_minus1[3]; + + uint8_t sps_timing_hrd_params_present_flag; + uint8_t sps_sublayer_cpb_params_present_flag; + H266RawGeneralTimingHrdParameters sps_general_timing_hrd_parameters; + H266RawOlsTimingHrdParameters sps_ols_timing_hrd_parameters; + + uint8_t sps_field_seq_flag; + uint8_t sps_vui_parameters_present_flag; + uint16_t sps_vui_payload_size_minus1; + H266RawVUI vui; + + uint8_t sps_extension_flag; + + H266RawExtensionData extension_data; + +} H266RawSPS; + +typedef struct H266RawPPS { + H266RawNALUnitHeader nal_unit_header; + + uint8_t pps_pic_parameter_set_id; + uint8_t pps_seq_parameter_set_id; + uint8_t pps_mixed_nalu_types_in_pic_flag; + uint16_t pps_pic_width_in_luma_samples; + uint16_t pps_pic_height_in_luma_samples; + + uint8_t pps_conformance_window_flag; + uint16_t pps_conf_win_left_offset; + uint16_t pps_conf_win_right_offset; + uint16_t pps_conf_win_top_offset; + uint16_t pps_conf_win_bottom_offset; + + uint8_t pps_scaling_window_explicit_signalling_flag; + int16_t pps_scaling_win_left_offset; + int16_t pps_scaling_win_right_offset; + int16_t pps_scaling_win_top_offset; + int16_t pps_scaling_win_bottom_offset; + + uint8_t pps_output_flag_present_flag; + uint8_t pps_no_pic_partition_flag; + + uint8_t pps_subpic_id_mapping_present_flag; + uint16_t pps_num_subpics_minus1; + uint8_t pps_subpic_id_len_minus1; + uint16_t pps_subpic_id[VVC_MAX_SLICES]; + + uint8_t pps_log2_ctu_size_minus5; + uint8_t pps_num_exp_tile_columns_minus1; + uint8_t pps_num_exp_tile_rows_minus1; + uint16_t pps_tile_column_width_minus1[VVC_MAX_TILE_COLUMNS]; + uint16_t pps_tile_row_height_minus1[VVC_MAX_TILE_ROWS]; + + uint8_t pps_loop_filter_across_tiles_enabled_flag; + uint8_t pps_rect_slice_flag; + uint8_t pps_single_slice_per_subpic_flag; + + uint16_t pps_num_slices_in_pic_minus1; + uint8_t pps_tile_idx_delta_present_flag; + uint16_t pps_slice_width_in_tiles_minus1[VVC_MAX_SLICES]; + uint16_t pps_slice_height_in_tiles_minus1[VVC_MAX_SLICES]; + uint16_t pps_num_exp_slices_in_tile[VVC_MAX_SLICES]; + uint16_t pps_exp_slice_height_in_ctus_minus1[VVC_MAX_SLICES][VVC_MAX_TILE_ROWS]; + int16_t pps_tile_idx_delta_val[VVC_MAX_SLICES]; + + uint8_t pps_loop_filter_across_slices_enabled_flag; + uint8_t pps_cabac_init_present_flag; + uint8_t pps_num_ref_idx_default_active_minus1[2]; + uint8_t pps_rpl1_idx_present_flag; + uint8_t pps_weighted_pred_flag; + uint8_t pps_weighted_bipred_flag; + uint8_t pps_ref_wraparound_enabled_flag; + uint16_t pps_pic_width_minus_wraparound_offset; + int8_t pps_init_qp_minus26; + uint8_t pps_cu_qp_delta_enabled_flag; + uint8_t pps_chroma_tool_offsets_present_flag; + int8_t pps_cb_qp_offset; + int8_t pps_cr_qp_offset; + uint8_t pps_joint_cbcr_qp_offset_present_flag; + int8_t pps_joint_cbcr_qp_offset_value; + uint8_t pps_slice_chroma_qp_offsets_present_flag; + uint8_t pps_cu_chroma_qp_offset_list_enabled_flag; + uint8_t pps_chroma_qp_offset_list_len_minus1; + uint8_t pps_cb_qp_offset_list[6]; + uint8_t pps_cr_qp_offset_list[6]; + uint8_t pps_joint_cbcr_qp_offset_list[6]; + uint8_t pps_deblocking_filter_control_present_flag; + uint8_t pps_deblocking_filter_override_enabled_flag; + uint8_t pps_deblocking_filter_disabled_flag; + uint8_t pps_dbf_info_in_ph_flag; + + int8_t pps_luma_beta_offset_div2; + int8_t pps_luma_tc_offset_div2; + int8_t pps_cb_beta_offset_div2; + int8_t pps_cb_tc_offset_div2; + int8_t pps_cr_beta_offset_div2; + int8_t pps_cr_tc_offset_div2; + + uint8_t pps_rpl_info_in_ph_flag; + uint8_t pps_sao_info_in_ph_flag; + uint8_t pps_alf_info_in_ph_flag; + uint8_t pps_wp_info_in_ph_flag; + uint8_t pps_qp_delta_info_in_ph_flag; + + uint8_t pps_picture_header_extension_present_flag; + uint8_t pps_slice_header_extension_present_flag; + uint8_t pps_extension_flag; + H266RawExtensionData extension_data; + + //calculated value; + uint16_t num_tile_columns; + uint16_t num_tile_rows; + uint16_t num_tiles_in_pic; + uint16_t slice_height_in_ctus[VVC_MAX_SLICES]; + uint16_t num_slices_in_subpic[VVC_MAX_SLICES]; +} H266RawPPS; + +typedef struct H266RawAUD { + H266RawNALUnitHeader nal_unit_header; + uint8_t aud_irap_or_gdr_flag; + uint8_t aud_pic_type; +} H266RawAUD; + +typedef struct H266RawPredWeightTable { + uint8_t luma_log2_weight_denom; + int8_t delta_chroma_log2_weight_denom; + + uint8_t num_l0_weights; + uint8_t luma_weight_l0_flag[15]; + uint8_t chroma_weight_l0_flag[15]; + int8_t delta_luma_weight_l0[15]; + int8_t luma_offset_l0[15]; + int8_t delta_chroma_weight_l0[15][2]; + int16_t delta_chroma_offset_l0[15][2]; + + uint8_t num_l1_weights; + uint8_t luma_weight_l1_flag[15]; + uint8_t chroma_weight_l1_flag[15]; + int8_t delta_luma_weight_l1[15]; + int8_t luma_offset_l1[15]; + int8_t delta_chroma_weight_l1[15][2]; + int16_t delta_chroma_offset_l1[15][2]; +} H266RawPredWeightTable; + +typedef struct H266RawPH { + H266RawNALUnitHeader nal_unit_header; + uint8_t ph_gdr_or_irap_pic_flag; + uint8_t ph_non_ref_pic_flag; + uint8_t ph_gdr_pic_flag; + uint8_t ph_inter_slice_allowed_flag; + uint8_t ph_intra_slice_allowed_flag; + uint8_t ph_pic_parameter_set_id; + uint16_t ph_pic_order_cnt_lsb; + uint8_t ph_recovery_poc_cnt; + uint8_t ph_extra_bit[16]; + uint8_t ph_poc_msb_cycle_present_flag; + uint8_t ph_poc_msb_cycle_val; + + uint8_t ph_alf_enabled_flag; + uint8_t ph_num_alf_aps_ids_luma; + uint8_t ph_alf_aps_id_luma[8]; + uint8_t ph_alf_cb_enabled_flag; + uint8_t ph_alf_cr_enabled_flag; + uint8_t ph_alf_aps_id_chroma; + uint8_t ph_alf_cc_cb_enabled_flag; + uint8_t ph_alf_cc_cb_aps_id; + uint8_t ph_alf_cc_cr_enabled_flag; + uint8_t ph_alf_cc_cr_aps_id; + + uint8_t ph_lmcs_enabled_flag; + uint8_t ph_lmcs_aps_id; + uint8_t ph_chroma_residual_scale_flag; + uint8_t ph_explicit_scaling_list_enabled_flag; + uint8_t ph_scaling_list_aps_id; + + uint8_t ph_virtual_boundaries_present_flag; + uint8_t ph_num_ver_virtual_boundaries; + uint16_t ph_virtual_boundary_pos_x_minus1[3]; + uint8_t ph_num_hor_virtual_boundaries; + uint16_t ph_virtual_boundary_pos_y_minus1[3]; + + uint8_t ph_pic_output_flag; + H266RefPicLists ph_ref_pic_lists; + + uint8_t ph_partition_constraints_override_flag; + + uint8_t ph_log2_diff_min_qt_min_cb_intra_slice_luma; + uint8_t ph_max_mtt_hierarchy_depth_intra_slice_luma; + uint8_t ph_log2_diff_max_bt_min_qt_intra_slice_luma; + uint8_t ph_log2_diff_max_tt_min_qt_intra_slice_luma; + uint8_t ph_log2_diff_min_qt_min_cb_intra_slice_chroma; + + uint8_t ph_max_mtt_hierarchy_depth_intra_slice_chroma; + uint8_t ph_log2_diff_max_bt_min_qt_intra_slice_chroma; + uint8_t ph_log2_diff_max_tt_min_qt_intra_slice_chroma; + + uint8_t ph_cu_qp_delta_subdiv_intra_slice; + uint8_t ph_cu_chroma_qp_offset_subdiv_intra_slice; + + uint8_t ph_log2_diff_min_qt_min_cb_inter_slice; + uint8_t ph_max_mtt_hierarchy_depth_inter_slice; + uint8_t ph_log2_diff_max_bt_min_qt_inter_slice; + uint8_t ph_log2_diff_max_tt_min_qt_inter_slice; + uint8_t ph_cu_qp_delta_subdiv_inter_slice; + uint8_t ph_cu_chroma_qp_offset_subdiv_inter_slice; + + uint8_t ph_temporal_mvp_enabled_flag; + uint8_t ph_collocated_from_l0_flag; + uint8_t ph_collocated_ref_idx; + uint8_t ph_mmvd_fullpel_only_flag; + uint8_t ph_mvd_l1_zero_flag; + uint8_t ph_bdof_disabled_flag; + uint8_t ph_dmvr_disabled_flag; + uint8_t ph_prof_disabled_flag; + + H266RawPredWeightTable ph_pred_weight_table; + + int8_t ph_qp_delta; + uint8_t ph_joint_cbcr_sign_flag; + uint8_t ph_sao_luma_enabled_flag; + uint8_t ph_sao_chroma_enabled_flag; + + uint8_t ph_deblocking_params_present_flag; + uint8_t ph_deblocking_filter_disabled_flag; + int8_t ph_luma_beta_offset_div2; + int8_t ph_luma_tc_offset_div2; + int8_t ph_cb_beta_offset_div2; + int8_t ph_cb_tc_offset_div2; + int8_t ph_cr_beta_offset_div2; + int8_t ph_cr_tc_offset_div2; + + uint8_t ph_extension_length; + uint8_t ph_extension_data_byte[256]; +} H266RawPH; + +typedef struct H266RawSliceHeader { + H266RawNALUnitHeader nal_unit_header; + uint8_t sh_picture_header_in_slice_header_flag; + H266RawPH sh_picture_header; + + uint16_t sh_subpic_id; + uint16_t sh_slice_address; + uint8_t sh_extra_bit[16]; + uint8_t sh_num_tiles_in_slice_minus1; + uint8_t sh_slice_type; + uint8_t sh_no_output_of_prior_pics_flag; + + uint8_t sh_alf_enabled_flag; + uint8_t sh_num_alf_aps_ids_luma; + uint8_t sh_alf_aps_id_luma[8]; + uint8_t sh_alf_cb_enabled_flag; + uint8_t sh_alf_cr_enabled_flag; + uint8_t sh_alf_aps_id_chroma; + uint8_t sh_alf_cc_cb_enabled_flag; + uint8_t sh_alf_cc_cb_aps_id; + uint8_t sh_alf_cc_cr_enabled_flag; + uint8_t sh_alf_cc_cr_aps_id; + + uint8_t sh_lmcs_used_flag; + uint8_t sh_explicit_scaling_list_used_flag; + + H266RefPicLists sh_ref_pic_lists; + + uint8_t sh_num_ref_idx_active_override_flag; + uint8_t sh_num_ref_idx_active_minus1[2]; + uint8_t sh_cabac_init_flag; + uint8_t sh_collocated_from_l0_flag; + uint8_t sh_collocated_ref_idx; + + H266RawPredWeightTable sh_pred_weight_table; + + int8_t sh_qp_delta; + int8_t sh_cb_qp_offset; + int8_t sh_cr_qp_offset; + int8_t sh_joint_cbcr_qp_offset; + uint8_t sh_cu_chroma_qp_offset_enabled_flag; + + uint8_t sh_sao_luma_used_flag; + uint8_t sh_sao_chroma_used_flag; + + uint8_t sh_deblocking_params_present_flag; + uint8_t sh_deblocking_filter_disabled_flag; + int8_t sh_luma_beta_offset_div2; + int8_t sh_luma_tc_offset_div2; + int8_t sh_cb_beta_offset_div2; + int8_t sh_cb_tc_offset_div2; + int8_t sh_cr_beta_offset_div2; + int8_t sh_cr_tc_offset_div2; + uint8_t sh_dep_quant_used_flag; + + uint8_t sh_sign_data_hiding_used_flag; + uint8_t sh_ts_residual_coding_disabled_flag; + uint16_t sh_slice_header_extension_length; + uint8_t sh_slice_header_extension_data_byte[256]; + + uint8_t sh_entry_offset_len_minus1; + uint32_t sh_entry_point_offset_minus1[VVC_MAX_ENTRY_POINTS]; + +} H266RawSliceHeader; + +typedef struct H266RawSlice { + H266RawSliceHeader header; + + uint8_t *data; + AVBufferRef *data_ref; + size_t data_size; + int data_bit_start; +} H266RawSlice; + +typedef struct H266RawSEIDecodedPictureHash { + uint8_t dph_sei_hash_type; + uint8_t dph_sei_single_component_flag; + uint8_t dph_sei_picture_md5[3][16]; + uint16_t dph_sei_picture_crc[3]; + uint32_t dph_sei_picture_checksum[3]; + + uint8_t dph_sei_reserved_zero_7bits; +} H266RawSEIDecodedPictureHash; + +typedef struct H266RawSEI { + H266RawNALUnitHeader nal_unit_header; + SEIRawMessageList message_list; +} H266RawSEI; + +typedef struct CodedBitstreamH266Context { + // Reader/writer context in common with the H.264 implementation. + CodedBitstreamH2645Context common; + + // All currently available parameter sets. These are updated when + // any parameter set NAL unit is read/written with this context. + AVBufferRef *vps_ref[VVC_MAX_VPS_COUNT]; + AVBufferRef *sps_ref[VVC_MAX_SPS_COUNT]; + AVBufferRef *pps_ref[VVC_MAX_PPS_COUNT]; + H266RawVPS *vps[VVC_MAX_SPS_COUNT]; + H266RawSPS *sps[VVC_MAX_SPS_COUNT]; + H266RawPPS *pps[VVC_MAX_PPS_COUNT]; + + struct { + AVBufferRef *ph_ref; + H266RawPH *ph; + } priv; + + // The currently active parameter sets. These are updated when any + // NAL unit refers to the relevant parameter set. These pointers + // must also be present in the arrays above. + const H266RawVPS *active_vps; + const H266RawSPS *active_sps; + const H266RawPPS *active_pps; + +} CodedBitstreamH266Context; + +#endif /* AVCODEC_CBS_H266_H */ diff --git a/libavcodec/cbs_h266_syntax_template.c b/libavcodec/cbs_h266_syntax_template.c new file mode 100644 index 0000000000..21e58fd9d1 --- /dev/null +++ b/libavcodec/cbs_h266_syntax_template.c @@ -0,0 +1,3010 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +static int FUNC(rbsp_trailing_bits)(CodedBitstreamContext *ctx, RWContext *rw) +{ + int err; + + fixed(1, rbsp_stop_one_bit, 1); + while (byte_alignment(rw) != 0) + fixed(1, rbsp_alignment_zero_bit, 0); + return 0; +} + +static int FUNC(nal_unit_header)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawNALUnitHeader *current, + int expected_nal_unit_type) +{ + int err; + + fixed(1, forbidden_zero_bit, 0); + flag(nuh_reserved_zero_bit); + + u(6, nuh_layer_id, 0, 55); + + if (expected_nal_unit_type >= 0) + u(5, nal_unit_type, expected_nal_unit_type, + expected_nal_unit_type); + else + ub(5, nal_unit_type); + + u(3, nuh_temporal_id_plus1, 1, 7); + return 0; +} + +static int FUNC(byte_alignment)(CodedBitstreamContext *ctx, RWContext *rw) +{ + int err; + + fixed(1, byte_alignment_bit_equal_to_one, 1); + while (byte_alignment(rw) != 0) + fixed(1, byte_alignment_bit_equal_to_zero, 0); + return 0; +} + +static int FUNC(general_constraints_info)(CodedBitstreamContext *ctx, + RWContext *rw, + H266GeneralConstraintsInfo *current) +{ + int err, i; + + flag(gci_present_flag); + if (current->gci_present_flag) { + /* general */ + flag(gci_intra_only_constraint_flag); + flag(gci_all_layers_independent_constraint_flag); + flag(gci_one_au_only_constraint_flag); + + /* picture format */ + u(4, gci_sixteen_minus_max_bitdepth_constraint_idc, 0, 8); + ub(2, gci_three_minus_max_chroma_format_constraint_idc); + + /* NAL unit type related */ + flag(gci_no_mixed_nalu_types_in_pic_constraint_flag); + flag(gci_no_trail_constraint_flag); + flag(gci_no_stsa_constraint_flag); + flag(gci_no_rasl_constraint_flag); + flag(gci_no_radl_constraint_flag); + flag(gci_no_idr_constraint_flag); + flag(gci_no_cra_constraint_flag); + flag(gci_no_gdr_constraint_flag); + flag(gci_no_aps_constraint_flag); + flag(gci_no_idr_rpl_constraint_flag); + + /* tile, slice, subpicture partitioning */ + flag(gci_one_tile_per_pic_constraint_flag); + flag(gci_pic_header_in_slice_header_constraint_flag); + flag(gci_one_slice_per_pic_constraint_flag); + flag(gci_no_rectangular_slice_constraint_flag); + flag(gci_one_slice_per_subpic_constraint_flag); + flag(gci_no_subpic_info_constraint_flag); + + /* CTU and block partitioning */ + ub(2, gci_three_minus_max_log2_ctu_size_constraint_idc); + flag(gci_no_partition_constraints_override_constraint_flag); + flag(gci_no_mtt_constraint_flag); + flag(gci_no_qtbtt_dual_tree_intra_constraint_flag); + + /* intra */ + flag(gci_no_palette_constraint_flag); + flag(gci_no_ibc_constraint_flag); + flag(gci_no_isp_constraint_flag); + flag(gci_no_mrl_constraint_flag); + flag(gci_no_mip_constraint_flag); + flag(gci_no_cclm_constraint_flag); + + /* inter */ + flag(gci_no_ref_pic_resampling_constraint_flag); + flag(gci_no_res_change_in_clvs_constraint_flag); + flag(gci_no_weighted_prediction_constraint_flag); + flag(gci_no_ref_wraparound_constraint_flag); + flag(gci_no_temporal_mvp_constraint_flag); + flag(gci_no_sbtmvp_constraint_flag); + flag(gci_no_amvr_constraint_flag); + flag(gci_no_bdof_constraint_flag); + flag(gci_no_smvd_constraint_flag); + flag(gci_no_dmvr_constraint_flag); + flag(gci_no_mmvd_constraint_flag); + flag(gci_no_affine_motion_constraint_flag); + flag(gci_no_prof_constraint_flag); + flag(gci_no_bcw_constraint_flag); + flag(gci_no_ciip_constraint_flag); + flag(gci_no_gpm_constraint_flag); + + /* transform, quantization, residual */ + flag(gci_no_luma_transform_size_64_constraint_flag); + flag(gci_no_transform_skip_constraint_flag); + flag(gci_no_bdpcm_constraint_flag); + flag(gci_no_mts_constraint_flag); + flag(gci_no_lfnst_constraint_flag); + flag(gci_no_joint_cbcr_constraint_flag); + flag(gci_no_sbt_constraint_flag); + flag(gci_no_act_constraint_flag); + flag(gci_no_explicit_scaling_list_constraint_flag); + flag(gci_no_dep_quant_constraint_flag); + flag(gci_no_sign_data_hiding_constraint_flag); + flag(gci_no_cu_qp_delta_constraint_flag); + flag(gci_no_chroma_qp_offset_constraint_flag); + + /* loop filter */ + flag(gci_no_sao_constraint_flag); + flag(gci_no_alf_constraint_flag); + flag(gci_no_ccalf_constraint_flag); + flag(gci_no_lmcs_constraint_flag); + flag(gci_no_ladf_constraint_flag); + flag(gci_no_virtual_boundaries_constraint_flag); + ub(8, gci_num_reserved_bits); + for (i = 0; i < current->gci_num_reserved_bits; i++) { + flags(gci_reserved_zero_bit[i], 1, i); + } + } + while (byte_alignment(rw) != 0) + fixed(1, gci_alignment_zero_bit, 0); + return 0; +} + +static int FUNC(profile_tier_level)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawProfileTierLevel *current, + int profile_tier_present_flag, + int max_num_sub_layers_minus1) +{ + int err, i; + + if (profile_tier_present_flag) { + ub(7, general_profile_idc); + flag(general_tier_flag); + } + ub(8, general_level_idc); + flag(ptl_frame_only_constraint_flag); + flag(ptl_multilayer_enabled_flag); + if (profile_tier_present_flag) { + CHECK(FUNC(general_constraints_info)(ctx, rw, + ¤t->general_constraints_info)); + } + for (i = max_num_sub_layers_minus1 - 1; i >= 0; i--) + flags(ptl_sublayer_level_present_flag[i], 1, i); + while (byte_alignment(rw) != 0) + flag(ptl_reserved_zero_bit); + for (i = max_num_sub_layers_minus1 - 1; i >= 0; i--) + if (current->ptl_sublayer_level_present_flag[i]) + ubs(8, sublayer_level_idc[i], 1, i); + if (profile_tier_present_flag) { + ub(8, ptl_num_sub_profiles); + for (i = 0; i < current->ptl_num_sub_profiles; i++) + ubs(32, general_sub_profile_idc[i], 1, i); + } + return 0; +} + +static int FUNC(vui_parameters_default)(CodedBitstreamContext *ctx, + RWContext *rw, H266RawVUI *current) +{ + //defined in D.8 + infer(vui_progressive_source_flag, 0); + infer(vui_interlaced_source_flag, 0); + + infer(vui_non_packed_constraint_flag, 0); + infer(vui_non_projected_constraint_flag, 0); + + infer(vui_aspect_ratio_constant_flag, 0); + infer(vui_aspect_ratio_idc, 0); + + infer(vui_overscan_info_present_flag, 0); + + infer(vui_colour_primaries, 2); + infer(vui_transfer_characteristics, 2); + infer(vui_matrix_coeffs, 2); + infer(vui_full_range_flag, 0); + + infer(vui_chroma_sample_loc_type_frame, 6); + infer(vui_chroma_sample_loc_type_top_field, 6); + infer(vui_chroma_sample_loc_type_bottom_field, 6); + return 0; +} + +static int FUNC(vui_parameters)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawVUI *current, uint8_t chroma_format_idc) +{ + int err; + + flag(vui_progressive_source_flag); + flag(vui_interlaced_source_flag); + flag(vui_non_packed_constraint_flag); + flag(vui_non_projected_constraint_flag); + flag(vui_aspect_ratio_info_present_flag); + if (current->vui_aspect_ratio_info_present_flag) { + flag(vui_aspect_ratio_constant_flag); + ub(8, vui_aspect_ratio_idc); + if (current->vui_aspect_ratio_idc == 255) { + ub(16, vui_sar_width); + ub(16, vui_sar_height); + } + } else { + infer(vui_aspect_ratio_constant_flag, 0); + infer(vui_aspect_ratio_idc, 0); + } + flag(vui_overscan_info_present_flag); + if (current->vui_overscan_info_present_flag) + flag(vui_overscan_appropriate_flag); + flag(vui_colour_description_present_flag); + if (current->vui_colour_description_present_flag) { + ub(8, vui_colour_primaries); + av_log(ctx->log_ctx, AV_LOG_DEBUG, "vui_colour_primaries == %d \n", current->vui_colour_primaries); + ub(8, vui_transfer_characteristics); + av_log(ctx->log_ctx, AV_LOG_DEBUG, "vui_transfer_characteristics == %d \n", current->vui_transfer_characteristics); + ub(8, vui_matrix_coeffs); + av_log(ctx->log_ctx, AV_LOG_DEBUG, "vui_matrix_coeffs == %d \n", current->vui_matrix_coeffs); + flag(vui_full_range_flag); + } else { + infer(vui_colour_primaries, 2); + infer(vui_transfer_characteristics, 2); + infer(vui_matrix_coeffs, 2); + infer(vui_full_range_flag, 0); + } + flag(vui_chroma_loc_info_present_flag); + if (chroma_format_idc != 1 && current->vui_chroma_loc_info_present_flag) { + av_log(ctx->log_ctx, AV_LOG_ERROR, "chroma_format_idc == %d," + "vui_chroma_loc_info_present_flag can't not be true", chroma_format_idc); + return AVERROR_INVALIDDATA; + } + if (current->vui_chroma_loc_info_present_flag) { + if (current->vui_progressive_source_flag && + !current->vui_interlaced_source_flag) { + ue(vui_chroma_sample_loc_type_frame, 0, 6); + } else { + ue(vui_chroma_sample_loc_type_top_field, 0, 6); + ue(vui_chroma_sample_loc_type_bottom_field, 0, 6); + } + } else { + if (chroma_format_idc == 1) { + infer(vui_chroma_sample_loc_type_frame, 6); + infer(vui_chroma_sample_loc_type_top_field, + current->vui_chroma_sample_loc_type_frame); + infer(vui_chroma_sample_loc_type_bottom_field, + current->vui_chroma_sample_loc_type_frame); + } + } + return 0; +} + +static int FUNC(payload_extension)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawExtensionData *current, + uint32_t payload_size, int cur_pos) +{ + int err; + size_t byte_length, k; + +#ifdef READ + GetBitContext tmp; + int bits_left, payload_zero_bits; + + if (!cbs_h265_payload_extension_present(rw, payload_size, cur_pos)) + return 0; + + bits_left = 8 * payload_size - cur_pos; + tmp = *rw; + if (bits_left > 8) + skip_bits_long(&tmp, bits_left - 8); + payload_zero_bits = get_bits(&tmp, FFMIN(bits_left, 8)); + if (!payload_zero_bits) + return AVERROR_INVALIDDATA; + payload_zero_bits = ff_ctz(payload_zero_bits); + current->bit_length = bits_left - payload_zero_bits - 1; + allocate(current->data, (current->bit_length + 7) / 8); +#endif + + byte_length = (current->bit_length + 7) / 8; + for (k = 0; k < byte_length; k++) { + int length = FFMIN(current->bit_length - k * 8, 8); + xu(length, reserved_payload_extension_data, current->data[k], + 0, MAX_UINT_BITS(length), 0); + } + + return 0; +} + +static int FUNC(vui_payload)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawVUI *current, uint16_t vui_payload_size, + uint8_t chroma_format_idc) +{ + int err; + int start_position, current_position; + + start_position = bit_position(rw); + CHECK(FUNC(vui_parameters)(ctx, rw, current, chroma_format_idc)); + current_position = bit_position(rw) - start_position; + + if (current_position < 8 * vui_payload_size) { + CHECK(FUNC(payload_extension)(ctx, rw, ¤t->extension_data, + vui_payload_size, current_position)); + fixed(1, vui_payload_bit_equal_to_one, 1); + while (byte_alignment(rw) != 0) + fixed(1, vui_payload_bit_equal_to_zero, 0); + } + return 0; +} + +static int FUNC(extension_data)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawExtensionData *current) +{ + int err; + size_t k; +#ifdef READ + GetBitContext start; + uint8_t bit; + start = *rw; + for (k = 0; cbs_h2645_read_more_rbsp_data(rw); k++) + skip_bits(rw, 1); + current->bit_length = k; + if (k > 0) { + *rw = start; + allocate(current->data, (current->bit_length + 7) / 8); + for (k = 0; k < current->bit_length; k++) { + xu(1, extension_data, bit, 0, 1, 0); + current->data[k / 8] |= bit << (7 - k % 8); + } + } +#else + for (k = 0; k < current->bit_length; k++) + xu(1, extension_data, current->data[k / 8] >> (7 - k % 8) & 1, 0, 1, 0); +#endif + return 0; +} + +static int FUNC(dpb_parameters)(CodedBitstreamContext *ctx, RWContext *rw, + H266DpbParameters *current, + uint8_t max_sublayers_minus1, + uint8_t sublayer_info_flag) +{ + int err, i; + for (i = (sublayer_info_flag ? 0 : max_sublayers_minus1); + i <= max_sublayers_minus1; i++) { + ues(dpb_max_dec_pic_buffering_minus1[i], + 0, VVC_MAX_DPB_SIZE - 1, 1, i); + ues(dpb_max_num_reorder_pics[i], + 0, current->dpb_max_dec_pic_buffering_minus1[i], 1, i); + ues(dpb_max_latency_increase_plus1[i], 0, UINT32_MAX - 1, 1, i); + } + return 0; +} + +static int FUNC(ref_pic_list_struct)(CodedBitstreamContext *ctx, RWContext *rw, + H266RefPicListStruct *current, + uint8_t list_idx, uint8_t rpls_idx, + const H266RawSPS *sps) +{ + CodedBitstreamH266Context *h266 = ctx->priv_data; + int err, i, j, general_layer_idx = -1, num_direct_ref_layers = 0; + const H266RawVPS *vps = h266->vps[sps->sps_video_parameter_set_id]; + + if (!vps) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "VPS id %d not available.\n", sps->sps_video_parameter_set_id); + return AVERROR_INVALIDDATA; + } + + //7.4.3.3 (29) + for (i = 0; i <= vps->vps_max_layers_minus1; i++) { + if (sps->nal_unit_header.nuh_layer_id == vps->vps_layer_id[i]) { + general_layer_idx = i; + break; + } + } + if (general_layer_idx < 0) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "vps_layer_id %d not available.\n", sps->nal_unit_header.nuh_layer_id); + return AVERROR_INVALIDDATA; + } + + //7.4.3.3 (28) + for (j = 0; j <= vps->vps_max_layers_minus1; j++) { + if (vps->vps_direct_ref_layer_flag[general_layer_idx][j]) + num_direct_ref_layers++; + } + + ue(num_ref_entries, 0, VVC_MAX_REF_ENTRIES); + if (sps->sps_long_term_ref_pics_flag && + rpls_idx < sps->sps_num_ref_pic_lists[list_idx] && + current->num_ref_entries > 0) + flag(ltrp_in_header_flag); + if (sps->sps_long_term_ref_pics_flag && + rpls_idx == sps->sps_num_ref_pic_lists[list_idx]) + infer(ltrp_in_header_flag, 1); + for (i = 0, j = 0; i < current->num_ref_entries; i++) { + if (sps->sps_inter_layer_prediction_enabled_flag) + flags(inter_layer_ref_pic_flag[i], 1, i); + else + infer(inter_layer_ref_pic_flag[i], 0); + + if (!current->inter_layer_ref_pic_flag[i]) { + if (sps->sps_long_term_ref_pics_flag) + flags(st_ref_pic_flag[i], 1, i); + else + infer(st_ref_pic_flag[i], 1); + if (current->st_ref_pic_flag[i]) { + int abs_delta_poc_st; + ues(abs_delta_poc_st[i], 0, MAX_UINT_BITS(15), 1, i); + if ((sps->sps_weighted_pred_flag || + sps->sps_weighted_bipred_flag) && i != 0) + abs_delta_poc_st = current->abs_delta_poc_st[i]; + else + abs_delta_poc_st = current->abs_delta_poc_st[i] + 1; + if (abs_delta_poc_st > 0) + flags(strp_entry_sign_flag[i], 1, i); + } else { + if (!current->ltrp_in_header_flag) { + uint8_t bits = sps->sps_log2_max_pic_order_cnt_lsb_minus4 + 4; + ubs(bits, rpls_poc_lsb_lt[j], 1, j); + j++; + } + } + } else { + if (num_direct_ref_layers == 0) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "num_direct_ref_layers needs > 0.\n"); + return AVERROR_INVALIDDATA; + } + ues(ilrp_idx[i], 0, num_direct_ref_layers - 1, 1, i); + } + } + return 0; +} + +static int FUNC(ref_pic_lists)(CodedBitstreamContext *ctx, RWContext *rw, + const H266RawSPS *sps, const H266RawPPS *pps, + H266RefPicLists *current) +{ + const H266RefPicListStruct *ref_list; + int err, i, j, num_ltrp_entries; + for (i = 0; i < 2; i++) { + if (sps->sps_num_ref_pic_lists[i] > 0 && + (i == 0 || (i == 1 && pps->pps_rpl1_idx_present_flag))) { + flags(rpl_sps_flag[i], 1, i); + } else { + if (sps->sps_num_ref_pic_lists[i] == 0) { + infer(rpl_sps_flag[i], 0); + } else { + if (!pps->pps_rpl1_idx_present_flag && i == 1) + infer(rpl_sps_flag[1], current->rpl_sps_flag[0]); + } + } + if (current->rpl_sps_flag[i]) { + if (sps->sps_num_ref_pic_lists[i] > 1 && + (i == 0 || (i == 1 && pps->pps_rpl1_idx_present_flag))) { + uint8_t bits = av_ceil_log2(sps->sps_num_ref_pic_lists[i]); + us(bits, rpl_idx[i], 0, sps->sps_num_ref_pic_lists[i] - 1, 1, i); + } else if (sps->sps_num_ref_pic_lists[i] == 1) { + infer(rpl_idx[i], 0); + } else if (i == 1 && !pps->pps_rpl1_idx_present_flag){ + infer(rpl_idx[1], current->rpl_idx[0]); + } else { + //how to handle this? or never happpend? + av_log(ctx->log_ctx, AV_LOG_ERROR, + "can't infer the rpl_idx[i]\n"); + return AVERROR_PATCHWELCOME; + } + memcpy(¤t->rpl_ref_list[i], + &sps->sps_ref_pic_list_struct[i][current->rpl_idx[i]], + sizeof(current->rpl_ref_list[i])); + } else { + CHECK(FUNC(ref_pic_list_struct)(ctx, rw, ¤t->rpl_ref_list[i], + i, sps->sps_num_ref_pic_lists[i], sps)); + } + ref_list = ¤t->rpl_ref_list[i]; + + num_ltrp_entries = 0; + for (int i = 0; i < ref_list->num_ref_entries; i++) { + if (!ref_list->inter_layer_ref_pic_flag[i]) { + if (!ref_list->st_ref_pic_flag[i]) { + num_ltrp_entries++; + } + } + } + + for (j = 0; j < num_ltrp_entries; j++) { + if (ref_list->ltrp_in_header_flag) { + ubs(sps->sps_log2_max_pic_order_cnt_lsb_minus4 + 4, + poc_lsb_lt[i][j], + 2, i, j); + } + flags(delta_poc_msb_cycle_present_flag[i][j], 2, i, j); + if (current->delta_poc_msb_cycle_present_flag[i][j]) { + uint32_t max = 1 << (32 - sps->sps_log2_max_pic_order_cnt_lsb_minus4 - 4); + ues(delta_poc_msb_cycle_lt[i][j], 0, max, 2, i, j); + } + } + } + return 0; +} + +static int FUNC(general_timing_hrd_parameters)(CodedBitstreamContext *ctx, + RWContext *rw, + H266RawGeneralTimingHrdParameters *current) +{ + int err; + ub(32, num_units_in_tick); + u(32, time_scale, 1, MAX_UINT_BITS(32)); + flag(general_nal_hrd_params_present_flag); + flag(general_vcl_hrd_params_present_flag); + + if (current->general_nal_hrd_params_present_flag || + current->general_vcl_hrd_params_present_flag) { + flag(general_same_pic_timing_in_all_ols_flag); + flag(general_du_hrd_params_present_flag); + if (current->general_du_hrd_params_present_flag) + ub(8, tick_divisor_minus2); + ub(4, bit_rate_scale); + ub(4, cpb_size_scale); + if (current->general_du_hrd_params_present_flag) + ub(4, cpb_size_du_scale); + ue(hrd_cpb_cnt_minus1, 0, 31); + } else { + //infer general_same_pic_timing_in_all_ols_flag? + infer(general_du_hrd_params_present_flag, 0); + } + return 0; +} + +static int FUNC(sublayer_hrd_parameters)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawSubLayerHRDParameters *current, int sublayer_id, + const H266RawGeneralTimingHrdParameters *general) +{ + int err, i; + for (i = 0; i <= general->hrd_cpb_cnt_minus1; i++) { + ues(bit_rate_value_minus1[sublayer_id][i], 0, UINT32_MAX - 1, 2, sublayer_id, i); + ues(cpb_size_value_minus1[sublayer_id][i], 0, UINT32_MAX - 1, 2, sublayer_id, i); + if (general->general_du_hrd_params_present_flag) { + ues(cpb_size_du_value_minus1[sublayer_id][i], + 0, UINT32_MAX - 1, 2, sublayer_id, i); + ues(bit_rate_du_value_minus1[sublayer_id][i], + 0, UINT32_MAX - 1, 2, sublayer_id, i); + } + flags(cbr_flag[sublayer_id][i], 2, sublayer_id, i); + } + return 0; +} + +static int FUNC(ols_timing_hrd_parameters)(CodedBitstreamContext *ctx, + RWContext *rw, H266RawOlsTimingHrdParameters *current, + uint8_t first_sublayer, uint8_t max_sublayers_minus1, + const H266RawGeneralTimingHrdParameters *general) +{ + int err, i; + for (i = first_sublayer; i <= max_sublayers_minus1; i++) { + flags(fixed_pic_rate_general_flag[i], 1, i); + if (!current->fixed_pic_rate_general_flag[i]) + flags(fixed_pic_rate_within_cvs_flag[i], 1, i); + else + infer(fixed_pic_rate_within_cvs_flag[i], 1); + if (current->fixed_pic_rate_within_cvs_flag[i]) { + ues(elemental_duration_in_tc_minus1[i], 0, 2047, 1, i); + infer(low_delay_hrd_flag[i], 0); + } else if ((general->general_nal_hrd_params_present_flag || + general->general_vcl_hrd_params_present_flag) && + general->hrd_cpb_cnt_minus1 == 0) { + flags(low_delay_hrd_flag[i], 1, i); + } else { + infer(low_delay_hrd_flag[i], 0); + } + if (general->general_nal_hrd_params_present_flag) + CHECK(FUNC(sublayer_hrd_parameters)(ctx, rw, + ¤t->nal_sub_layer_hrd_parameters, + i, general)); + if (general->general_vcl_hrd_params_present_flag) + CHECK(FUNC(sublayer_hrd_parameters)(ctx, rw, + ¤t->nal_sub_layer_hrd_parameters, i, general)); + } + return 0; +} + +static int FUNC(vps)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawVPS *current) +{ + int err, i, j, k; + uint16_t total_num_olss = 0; + uint8_t ols_mode_idc = 0; + uint16_t num_multi_layer_olss = 0; + uint8_t layer_included_in_ols_flag[VVC_MAX_TOTAL_NUM_OLSS][VVC_MAX_LAYERS]; + uint8_t num_ref_layers[VVC_MAX_LAYERS]; + uint8_t reference_layer_idx[VVC_MAX_LAYERS][VVC_MAX_LAYERS]; + + HEADER("Video Parameter Set"); + + CHECK(FUNC(nal_unit_header)(ctx, rw, + ¤t->nal_unit_header, VVC_VPS_NUT)); + + ub(4, vps_video_parameter_set_id); + if (current->vps_video_parameter_set_id == 0) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "vps_video_parameter_set_id should > 0.\n"); + return AVERROR_INVALIDDATA; + } + ub(6, vps_max_layers_minus1); + u(3, vps_max_sublayers_minus1, 0, 6); + if (current->vps_max_layers_minus1 > 0 && current->vps_max_sublayers_minus1 > 0) + flag(vps_default_ptl_dpb_hrd_max_tid_flag); + else + infer(vps_default_ptl_dpb_hrd_max_tid_flag, 1); + + if (current->vps_max_layers_minus1 > 0) + flag(vps_all_independent_layers_flag); + else + infer(vps_all_independent_layers_flag, 1); + + for (i = 0; i <= current->vps_max_layers_minus1; i++) { + ubs(6, vps_layer_id[i], 1, i); + if (i > 0 && current->vps_layer_id[i] <= current->vps_layer_id[i - 1]) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "vps_layer_id[%d](%d) should > vps_layer_id[%d](%d).\n", + i, current->vps_layer_id[i], i - 1, current->vps_layer_id[i - 1]); + return AVERROR_INVALIDDATA; + } + if (i > 0 && !current->vps_all_independent_layers_flag) { + flags(vps_independent_layer_flag[i], 1, i); + if (!current->vps_independent_layer_flag[i]) { + flags(vps_max_tid_ref_present_flag[i], 1, i); + for (j = 0; j < i; j++) { + flags(vps_direct_ref_layer_flag[i][j], 2, i, j); + if (current->vps_max_tid_ref_present_flag[i] && + current->vps_direct_ref_layer_flag[i][j]) { + ubs(3, vps_max_tid_il_ref_pics_plus1[i][j], 2, i, j); + } else { + infer(vps_max_tid_il_ref_pics_plus1[i][j], + current->vps_max_sublayers_minus1 + 1); + } + } + } else { + for (j = 0; j < i; j++) { + infer(vps_direct_ref_layer_flag[i][j], 0); + } + } + } else { + infer(vps_independent_layer_flag[i], 1); + for (j = 0; j < i; j++) { + infer(vps_direct_ref_layer_flag[i][j], 0); + } + } + } + + if (current->vps_max_layers_minus1 > 0) { + if (current->vps_all_independent_layers_flag) + flag(vps_each_layer_is_an_ols_flag); + else + infer(vps_each_layer_is_an_ols_flag, 0); + if (!current->vps_each_layer_is_an_ols_flag) { + if (!current->vps_all_independent_layers_flag) + ub(2, vps_ols_mode_idc); + else + infer(vps_ols_mode_idc, 2); + if (current->vps_ols_mode_idc == 2) { + ub(8, vps_num_output_layer_sets_minus2); + for (i = 1; i <= current->vps_num_output_layer_sets_minus2 + 1; i++) + for (j = 0; j <= current->vps_max_layers_minus1; j++) + flags(vps_ols_output_layer_flag[i][j], 2, i, j); + } + ols_mode_idc = current->vps_ols_mode_idc; + } else { + ols_mode_idc = 4; + } + if (ols_mode_idc == 4 || ols_mode_idc == 0 || ols_mode_idc == 1) + total_num_olss = current->vps_max_layers_minus1 + 1; + else if (ols_mode_idc = 2) + total_num_olss = current->vps_num_output_layer_sets_minus2 + 2; + else + av_log(ctx->log_ctx, AV_LOG_ERROR, "ols_mode_idc == 3, patch welcome"); + u(8, vps_num_ptls_minus1, 0, total_num_olss - 1); + } else { + infer(vps_each_layer_is_an_ols_flag, 1); + infer(vps_num_ptls_minus1, 0); + } + { + //let's get calc NumMultiLayerOlss + int m; + uint8_t dependency_flag[VVC_MAX_LAYERS][VVC_MAX_LAYERS]; + uint16_t num_output_layers_in_ols[VVC_MAX_TOTAL_NUM_OLSS]; + uint8_t num_sub_layers_in_layer_in_ols[VVC_MAX_TOTAL_NUM_OLSS][VVC_MAX_TOTAL_NUM_OLSS]; + uint8_t output_layer_idx[VVC_MAX_TOTAL_NUM_OLSS][VVC_MAX_LAYERS]; + + //7.4.3.3 vps_direct_ref_layer_flag section + for (i = 0; i <= current->vps_max_layers_minus1; i++) { + for (j = 0; j <= current->vps_max_layers_minus1; j++) { + dependency_flag[i][j] = current->vps_direct_ref_layer_flag[i][j]; + for (k = 0; k < i; k++) { + if (current->vps_direct_ref_layer_flag[i][k] && dependency_flag[k][j]) + dependency_flag[i][j] = 1; + } + } + } + for (i = 0; i <= current->vps_max_layers_minus1; i++) { + int r; + for (j = 0, r = 0; j <= current->vps_max_layers_minus1; j++) { + if (dependency_flag[i][j]) + reference_layer_idx[i][r++] = j; + } + num_ref_layers[i] = r; + } + + //7.4.3.3 vps_ols_output_layer_flag section + num_output_layers_in_ols[0] = 1; + num_sub_layers_in_layer_in_ols[0][0] = current->vps_ptl_max_tid[current->vps_ols_ptl_idx[0]] + 1; + for (i = 1; i < total_num_olss; i++) { + if (ols_mode_idc == 4 || ols_mode_idc == 0) { + num_output_layers_in_ols[i] = 1; + if (current->vps_each_layer_is_an_ols_flag) { + num_sub_layers_in_layer_in_ols[i][0] = + current->vps_ptl_max_tid[current->vps_ols_ptl_idx[i]] + 1; + } else { + num_sub_layers_in_layer_in_ols[i][i] = + current->vps_ptl_max_tid[current->vps_ols_ptl_idx[i]] + 1; + for (k = i - 1; k >= 0; k--) { + num_sub_layers_in_layer_in_ols[i][k] = 0; + for (m = k + 1; m <= i; m++) { + uint8_t max_sublayer_needed = + FFMIN(num_sub_layers_in_layer_in_ols[i][m], + current->vps_max_tid_il_ref_pics_plus1[m][k]); + if (current->vps_direct_ref_layer_flag[m][k] && + num_sub_layers_in_layer_in_ols[i][k] < max_sublayer_needed) + num_sub_layers_in_layer_in_ols[i][k] = max_sublayer_needed; + } + } + } + } else if (current->vps_ols_mode_idc == 1) { + num_output_layers_in_ols[i] = i + 1; + for (j = 0; j < num_output_layers_in_ols[i]; j++) { + num_sub_layers_in_layer_in_ols[i][j] = + current->vps_ptl_max_tid[current->vps_ols_ptl_idx[i]] + 1; + } + } else if (current->vps_ols_mode_idc == 2) { + uint8_t highest_included_layer = 0; + for (j = 0; j <= current->vps_max_layers_minus1; j++) { + layer_included_in_ols_flag[i][j] = 0; + num_sub_layers_in_layer_in_ols[i][j] = 0; + } + for (k = 0, j = 0; k <= current->vps_max_layers_minus1; k++) { + if (current->vps_ols_output_layer_flag[i][k]) { + layer_included_in_ols_flag[i][k] = 1; + highest_included_layer = k; + output_layer_idx[i][j] = k; + num_sub_layers_in_layer_in_ols[i][k] = + current->vps_ptl_max_tid[current->vps_ols_ptl_idx[i]] + 1; + } + } + num_output_layers_in_ols[i] = j; + for (j = 0; j < num_output_layers_in_ols[i]; j++) { + int idx = output_layer_idx[i][j]; + for (k = 0; k < num_ref_layers[idx]; k++) { + if (!layer_included_in_ols_flag[i][reference_layer_idx[idx][k]]) + layer_included_in_ols_flag[i][reference_layer_idx[idx][k]] = 1; + } + } + for (k = highest_included_layer - 1; k >= 0; k--) { + if (layer_included_in_ols_flag[i][k] && !current->vps_ols_output_layer_flag[i][k]) { + for (m = k + 1; m <= highest_included_layer; m++) { + uint8_t max_sublayer_needed = + FFMIN(num_sub_layers_in_layer_in_ols[i][m], + current->vps_max_tid_il_ref_pics_plus1[m][k]); + if (current->vps_direct_ref_layer_flag[m][k] && + layer_included_in_ols_flag[i][m] && + num_sub_layers_in_layer_in_ols[i][k] < max_sublayer_needed) + num_sub_layers_in_layer_in_ols[i][k] = max_sublayer_needed; + } + } + } + } + } + for (i = 1; i < total_num_olss; i++) { + int num_layers_in_ols = 0; + if (current->vps_each_layer_is_an_ols_flag) { + num_layers_in_ols = 1; + } else if (current->vps_ols_mode_idc == 0 || current->vps_ols_mode_idc == 1) { + num_layers_in_ols = i + 1; + } else if (current->vps_ols_mode_idc == 2) { + for (k = 0, j = 0; k <= current->vps_max_layers_minus1; k++) { + if (layer_included_in_ols_flag[i][k]) + j++; + num_layers_in_ols = j; + } + } + if (num_layers_in_ols > 1) { + num_multi_layer_olss++; + } + } + } + + for (i = 0; i <= current->vps_num_ptls_minus1; i++) { + if (i > 0) + flags(vps_pt_present_flag[i], 1, i); + else + infer(vps_pt_present_flag[i], 1); + + if (!current->vps_default_ptl_dpb_hrd_max_tid_flag) + us(3, vps_ptl_max_tid[i], 0, current->vps_max_sublayers_minus1, 1, i); + else + infer(vps_ptl_max_tid[i], current->vps_max_sublayers_minus1); + } + while (byte_alignment(rw) != 0) + fixed(1, vps_ptl_alignment_zero_bit, 0); + for (i = 0; i <= current->vps_num_ptls_minus1; i++) { + CHECK(FUNC(profile_tier_level)(ctx, rw, + current->vps_profile_tier_level + i, + current->vps_pt_present_flag[i], + current->vps_ptl_max_tid[i])); + } + for (i = 0; i < total_num_olss; i++) { + if (current->vps_num_ptls_minus1 > 0 && + current->vps_num_ptls_minus1 + 1 != total_num_olss) { + us(8, vps_ols_ptl_idx[i], 0, current->vps_num_ptls_minus1, 1, i); + } + else if (current->vps_num_ptls_minus1 == 0) { + infer(vps_ols_ptl_idx[i], 0); + } else { + infer(vps_ols_ptl_idx[i], i); + } + } + + if (!current->vps_each_layer_is_an_ols_flag) { + uint16_t vps_num_dpb_params; + ue(vps_num_dpb_params_minus1, 0, num_multi_layer_olss - 1); + if (current->vps_each_layer_is_an_ols_flag) + vps_num_dpb_params = 0; + else + vps_num_dpb_params = current->vps_num_dpb_params_minus1 + 1; + + if (current->vps_max_sublayers_minus1 > 0) + flag(vps_sublayer_dpb_params_present_flag); + else + infer(vps_sublayer_dpb_params_present_flag, 0); + + for (i = 0; i < vps_num_dpb_params; i++) { + if (!current->vps_default_ptl_dpb_hrd_max_tid_flag) + us(3, vps_dpb_max_tid[i], 0, current->vps_max_sublayers_minus1, 1, i); + else + infer(vps_dpb_max_tid[i], current->vps_max_sublayers_minus1); + CHECK(FUNC(dpb_parameters)(ctx, rw, current->vps_dpb_params + i, + current->vps_dpb_max_tid[i], + current->vps_sublayer_dpb_params_present_flag)); + } + for (i = 0; i < num_multi_layer_olss; i++) { + ues(vps_ols_dpb_pic_width[i], 0, UINT16_MAX, 1, i); + ues(vps_ols_dpb_pic_height[i], 0, UINT16_MAX, 1, i); + ubs(2, vps_ols_dpb_chroma_format[i], 1, i); + ues(vps_ols_dpb_bitdepth_minus8[i], 0, 2, 1, i); + if (vps_num_dpb_params > 1 && vps_num_dpb_params != num_multi_layer_olss) + ues(vps_ols_dpb_params_idx[i], 0, vps_num_dpb_params - 1, 1, i); + else if (vps_num_dpb_params == 1) + infer(vps_ols_dpb_params_idx[i], 0); + else + infer(vps_ols_dpb_params_idx[i], i); + } + flag(vps_timing_hrd_params_present_flag); + if (current->vps_timing_hrd_params_present_flag) { + CHECK(FUNC(general_timing_hrd_parameters)(ctx, rw, + ¤t->vps_general_timing_hrd_parameters)); + if (current->vps_max_sublayers_minus1 > 0) + flag(vps_sublayer_cpb_params_present_flag); + else + infer(vps_sublayer_cpb_params_present_flag, 0); + ue(vps_num_ols_timing_hrd_params_minus1, 0, num_multi_layer_olss - 1); + for (i = 0; i <= current->vps_num_ols_timing_hrd_params_minus1; i++) { + uint8_t first_sublayer; + if (!current->vps_default_ptl_dpb_hrd_max_tid_flag) + us(3, vps_hrd_max_tid[i], 0, current->vps_max_sublayers_minus1, 1, i); + else + infer(vps_hrd_max_tid[i], current->vps_max_sublayers_minus1); + first_sublayer = current->vps_sublayer_cpb_params_present_flag ? + 0 : current->vps_hrd_max_tid[i]; + CHECK(FUNC(ols_timing_hrd_parameters)(ctx, rw, + ¤t->vps_ols_timing_hrd_parameters, first_sublayer, + current->vps_max_sublayers_minus1, + ¤t->vps_general_timing_hrd_parameters)); + + } + if (current->vps_num_ols_timing_hrd_params_minus1 > 0 && + current-> vps_num_ols_timing_hrd_params_minus1 + 1 != num_multi_layer_olss) { + for (i = 0; i < num_multi_layer_olss; i++) { + ues(vps_ols_timing_hrd_idx[i], 0, + current->vps_num_ols_timing_hrd_params_minus1, 1, i); + } + } else if (current->vps_num_ols_timing_hrd_params_minus1 == 0) { + for (i = 0; i < num_multi_layer_olss; i++) + infer(vps_ols_timing_hrd_idx[i], 0); + } else { + for (i = 0; i < num_multi_layer_olss; i++) + infer(vps_ols_timing_hrd_idx[i], i); + } + } + } + + flag(vps_extension_flag); + if (current->vps_extension_flag) + CHECK(FUNC(extension_data)(ctx, rw, ¤t->extension_data)); + CHECK(FUNC(rbsp_trailing_bits)(ctx, rw)); + + return 0; +} + + +static int FUNC(sps)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawSPS *current) +{ + CodedBitstreamH266Context *h266 = ctx->priv_data; + int err, i, j; + unsigned int ctb_log2_size_y, min_cb_log2_size_y, + min_qt_log2_size_intra_y, min_qt_log2_size_inter_y, + ctb_size_y, max_num_merge_cand, tmp_width_val, tmp_height_val; + uint8_t qp_bd_offset; + + static const uint8_t h266_sub_width_c[] = { + 1, 2, 2, 1 + }; + static const uint8_t h266_sub_height_c[] = { + 1, 2, 1, 1 + }; + + HEADER("Sequence Parameter Set"); + + CHECK(FUNC(nal_unit_header)(ctx, rw, + ¤t->nal_unit_header, VVC_SPS_NUT)); + + ub(4, sps_seq_parameter_set_id); + ub(4, sps_video_parameter_set_id); + if (current->sps_video_parameter_set_id == 0 && !h266->vps_ref[0]) { + H266RawVPS *vps; + AVBufferRef *ref = av_buffer_allocz(sizeof(H266RawVPS)); + if (!ref) { + return AVERROR(ENOMEM); + } + vps = (H266RawVPS*)ref->data; + vps->vps_max_layers_minus1 = 0; + vps->vps_independent_layer_flag[0] = 1; + vps->vps_layer_id[0] = current->nal_unit_header.nuh_layer_id; + h266->vps_ref[0] = ref; + h266->vps[0] = vps; + } + + u(3, sps_max_sublayers_minus1, 0, VVC_MAX_SUBLAYERS - 1); + u(2, sps_chroma_format_idc, 0, 3); + u(2, sps_log2_ctu_size_minus5, 0, 3); + ctb_log2_size_y = current->sps_log2_ctu_size_minus5 + 5; + ctb_size_y = 1 << ctb_log2_size_y; + + flag(sps_ptl_dpb_hrd_params_present_flag); + if (current->sps_ptl_dpb_hrd_params_present_flag) { + CHECK(FUNC(profile_tier_level)(ctx, rw, ¤t->profile_tier_level, + 1, current->sps_max_sublayers_minus1)); + } + flag(sps_gdr_enabled_flag); + flag(sps_ref_pic_resampling_enabled_flag); + if (current->sps_ref_pic_resampling_enabled_flag) + flag(sps_res_change_in_clvs_allowed_flag); + else + infer(sps_res_change_in_clvs_allowed_flag, 0); + + ue(sps_pic_width_max_in_luma_samples, 1, VVC_MAX_WIDTH); + ue(sps_pic_height_max_in_luma_samples, 1, VVC_MAX_HEIGHT); + + flag(sps_conformance_window_flag); + if (current->sps_conformance_window_flag) { + uint8_t sub_width_c = h266_sub_width_c[current->sps_chroma_format_idc]; + uint8_t sub_height_c = h266_sub_height_c[current->sps_chroma_format_idc]; + uint16_t width = current->sps_pic_width_max_in_luma_samples / sub_width_c; + uint16_t height = current->sps_pic_height_max_in_luma_samples / sub_height_c; + ue(sps_conf_win_left_offset, 0, width); + ue(sps_conf_win_right_offset, 0, width - current->sps_conf_win_left_offset); + ue(sps_conf_win_top_offset, 0, height); + ue(sps_conf_win_bottom_offset, 0, height - current->sps_conf_win_top_offset); + } else { + infer(sps_conf_win_left_offset, 0); + infer(sps_conf_win_right_offset, 0); + infer(sps_conf_win_top_offset, 0); + infer(sps_conf_win_bottom_offset, 0); + } + + tmp_width_val = + h266_ceil(current->sps_pic_width_max_in_luma_samples, ctb_size_y); + tmp_height_val = + h266_ceil(current->sps_pic_height_max_in_luma_samples, ctb_size_y); + + flag(sps_subpic_info_present_flag); + if (current->sps_subpic_info_present_flag) { + ue(sps_num_subpics_minus1, 1, VVC_MAX_SLICES - 1); + if (current->sps_num_subpics_minus1 > 0) { + flag(sps_independent_subpics_flag); + flag(sps_subpic_same_size_flag); + } + + if (current->sps_num_subpics_minus1 > 0) { + int wlen = av_ceil_log2(tmp_width_val); + int hlen = av_ceil_log2(tmp_height_val); + if (current->sps_pic_width_max_in_luma_samples > ctb_size_y) + ubs(wlen, sps_subpic_width_minus1[0], 1, 0); + else + infer(sps_subpic_width_minus1[i], tmp_width_val - 1); + if (current->sps_pic_height_max_in_luma_samples > ctb_size_y) + ubs(hlen, sps_subpic_height_minus1[0], 1, 0); + else + infer(sps_subpic_height_minus1[0], tmp_height_val); + if (!current->sps_independent_subpics_flag) { + flags(sps_subpic_treated_as_pic_flag[0], 1, 0); + flags(sps_loop_filter_across_subpic_enabled_flag[0], 1, 0); + } else { + infer(sps_subpic_treated_as_pic_flag[0], 1); + infer(sps_loop_filter_across_subpic_enabled_flag[0], 1); + } + for (i = 1; i <= current->sps_num_subpics_minus1; i++) { + if (!current->sps_subpic_same_size_flag) { + if (current->sps_pic_width_max_in_luma_samples > ctb_size_y) + ubs(wlen, sps_subpic_ctu_top_left_x[i], 1, i); + else + infer(sps_subpic_ctu_top_left_x[i], 0); + if (current->sps_pic_height_max_in_luma_samples > ctb_size_y) + ubs(hlen, sps_subpic_ctu_top_left_y[i], 1, i); + else + infer(sps_subpic_ctu_top_left_y[i], 0); + if (i < current->sps_num_subpics_minus1 && + current->sps_pic_width_max_in_luma_samples > ctb_size_y) { + ubs(wlen, sps_subpic_width_minus1[i], 1, i); + } else { + infer(sps_subpic_width_minus1[i], + tmp_width_val - current->sps_subpic_ctu_top_left_x[i] - 1); + } + if (i < current->sps_num_subpics_minus1 && + current->sps_pic_height_max_in_luma_samples > ctb_size_y) { + ubs(hlen, sps_subpic_height_minus1[i], 1, i); + } else { + infer(sps_subpic_height_minus1[i], + tmp_height_val - current->sps_subpic_ctu_top_left_y[i] - 1); + } + } else { + int num_subpic_cols = + tmp_width_val / (current->sps_subpic_width_minus1[0] + 1); + infer(sps_subpic_ctu_top_left_x[i], + (i % num_subpic_cols) * + (current->sps_subpic_width_minus1[0] + 1)); + infer(sps_subpic_ctu_top_left_y[i], + (i / num_subpic_cols) * + (current->sps_subpic_height_minus1[0] + 1)); + infer(sps_subpic_width_minus1[i], + current->sps_subpic_width_minus1[0]); + infer(sps_subpic_height_minus1[i], + current->sps_subpic_height_minus1[0]); + } + if (!current->sps_independent_subpics_flag) { + flags(sps_subpic_treated_as_pic_flag[i], 1, i); + flags(sps_loop_filter_across_subpic_enabled_flag[i], 1, i); + } else { + infer(sps_subpic_treated_as_pic_flag[i], 1); + infer(sps_loop_filter_across_subpic_enabled_flag[i], 0); + } + } + ue(sps_subpic_id_len_minus1, 0, 15); + if ((1 << (current->sps_subpic_id_len_minus1 + 1)) < + current->sps_num_subpics_minus1 + 1) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "sps_subpic_id_len_minus1(%d) is too small\n", + current->sps_subpic_id_len_minus1); + return AVERROR_INVALIDDATA; + } + flag(sps_subpic_id_mapping_explicitly_signalled_flag); + if (current->sps_subpic_id_mapping_explicitly_signalled_flag) { + flag(sps_subpic_id_mapping_present_flag); + if (current->sps_subpic_id_mapping_present_flag) { + for (i = 0; i <= current->sps_num_subpics_minus1; i++) { + ubs(current->sps_subpic_id_len_minus1 + 1, + sps_subpic_id[i], 1, i); + } + } + } + } else { + infer(sps_subpic_ctu_top_left_x[0], 0); + infer(sps_subpic_ctu_top_left_y[0], 0); + infer(sps_subpic_width_minus1[0], tmp_width_val - 1); + infer(sps_subpic_height_minus1[0], tmp_height_val - 1); + } + } else { + infer(sps_num_subpics_minus1, 0); + infer(sps_independent_subpics_flag, 1); + infer(sps_subpic_same_size_flag, 0); + infer(sps_subpic_id_mapping_explicitly_signalled_flag, 0); + infer(sps_subpic_ctu_top_left_x[0], 0); + infer(sps_subpic_ctu_top_left_y[0], 0); + infer(sps_subpic_width_minus1[0], tmp_width_val - 1); + infer(sps_subpic_height_minus1[0], tmp_height_val - 1); + } + + + ue(sps_bitdepth_minus8, 0, 2); + qp_bd_offset = 6 * current->sps_bitdepth_minus8; + + flag(sps_entropy_coding_sync_enabled_flag); + flag(sps_entry_point_offsets_present_flag); + + u(4, sps_log2_max_pic_order_cnt_lsb_minus4, 0, 12); + flag(sps_poc_msb_cycle_flag); + if (current->sps_poc_msb_cycle_flag) + ue(sps_poc_msb_cycle_len_minus1, + 0, 32 - current->sps_log2_max_pic_order_cnt_lsb_minus4 - 5); + + u(2, sps_num_extra_ph_bytes, 0, 2); + for (i = 0; i < (current->sps_num_extra_ph_bytes * 8); i++) { + flags(sps_extra_ph_bit_present_flag[i], 1, i); + } + + u(2, sps_num_extra_sh_bytes, 0, 2); + for (i = 0; i < (current->sps_num_extra_sh_bytes * 8); i++) { + flags(sps_extra_sh_bit_present_flag[i], 1, i); + } + + if (current->sps_ptl_dpb_hrd_params_present_flag) { + if (current->sps_max_sublayers_minus1 > 0) + flag(sps_sublayer_dpb_params_flag); + else + infer(sps_sublayer_dpb_params_flag, 0); + CHECK(FUNC(dpb_parameters)(ctx, rw, ¤t->sps_dpb_params, + current->sps_max_sublayers_minus1, + current->sps_sublayer_dpb_params_flag)); + } + + ue(sps_log2_min_luma_coding_block_size_minus2, + 0, FFMIN(4, current->sps_log2_ctu_size_minus5 + 3)); + min_cb_log2_size_y = + current->sps_log2_min_luma_coding_block_size_minus2 + 2; + + flag(sps_partition_constraints_override_enabled_flag); + + ue(sps_log2_diff_min_qt_min_cb_intra_slice_luma, + 0, FFMIN(6, ctb_log2_size_y) - min_cb_log2_size_y); + min_qt_log2_size_intra_y = + current->sps_log2_diff_min_qt_min_cb_intra_slice_luma + + min_cb_log2_size_y; + + ue(sps_max_mtt_hierarchy_depth_intra_slice_luma, + 0, 2 * (ctb_log2_size_y - min_cb_log2_size_y)); + + if (current->sps_max_mtt_hierarchy_depth_intra_slice_luma != 0) { + ue(sps_log2_diff_max_bt_min_qt_intra_slice_luma, + 0, ctb_log2_size_y - min_qt_log2_size_intra_y); + ue(sps_log2_diff_max_tt_min_qt_intra_slice_luma, + 0, FFMIN(6, ctb_log2_size_y) - min_qt_log2_size_intra_y); + } else { + infer(sps_log2_diff_max_bt_min_qt_intra_slice_luma, 0); + infer(sps_log2_diff_max_tt_min_qt_intra_slice_luma, 0); + } + + if (current->sps_chroma_format_idc != 0) { + flag(sps_qtbtt_dual_tree_intra_flag); + } else { + infer(sps_qtbtt_dual_tree_intra_flag, 0); + } + + if (current->sps_qtbtt_dual_tree_intra_flag) { + ue(sps_log2_diff_min_qt_min_cb_intra_slice_chroma, + 0, FFMIN(6, ctb_log2_size_y) - min_cb_log2_size_y); + ue(sps_max_mtt_hierarchy_depth_intra_slice_chroma, + 0, 2 * (ctb_log2_size_y - min_cb_log2_size_y)); + if (current->sps_max_mtt_hierarchy_depth_intra_slice_chroma != 0) { + unsigned int min_qt_log2_size_intra_c = + current->sps_log2_diff_min_qt_min_cb_intra_slice_chroma + + min_cb_log2_size_y; + ue(sps_log2_diff_max_bt_min_qt_intra_slice_chroma, + 0, FFMIN(6, ctb_log2_size_y) - min_qt_log2_size_intra_c); + ue(sps_log2_diff_max_tt_min_qt_intra_slice_chroma, + 0, FFMIN(6, ctb_log2_size_y) - min_qt_log2_size_intra_c); + } + } else { + infer(sps_log2_diff_min_qt_min_cb_intra_slice_chroma, 0); + infer(sps_max_mtt_hierarchy_depth_intra_slice_chroma, 0); + } + if (current->sps_max_mtt_hierarchy_depth_intra_slice_chroma == 0) { + infer(sps_log2_diff_max_bt_min_qt_intra_slice_chroma, 0); + infer(sps_log2_diff_max_tt_min_qt_intra_slice_chroma, 0); + } + + ue(sps_log2_diff_min_qt_min_cb_inter_slice, + 0, FFMIN(6, ctb_log2_size_y) - min_cb_log2_size_y); + min_qt_log2_size_inter_y = + current->sps_log2_diff_min_qt_min_cb_inter_slice + min_cb_log2_size_y; + + ue(sps_max_mtt_hierarchy_depth_inter_slice, + 0, 2 * (ctb_log2_size_y - min_cb_log2_size_y)); + if (current->sps_max_mtt_hierarchy_depth_inter_slice != 0) { + ue(sps_log2_diff_max_bt_min_qt_inter_slice, + 0, ctb_log2_size_y - min_qt_log2_size_inter_y); + ue(sps_log2_diff_max_tt_min_qt_inter_slice, + 0, FFMIN(6, ctb_log2_size_y) - min_qt_log2_size_inter_y); + } else { + infer(sps_log2_diff_max_bt_min_qt_inter_slice, 0); + infer(sps_log2_diff_max_tt_min_qt_inter_slice, 0); + } + + if (ctb_size_y > 32) + flag(sps_max_luma_transform_size_64_flag); + else + infer(sps_max_luma_transform_size_64_flag, 0); + + flag(sps_transform_skip_enabled_flag); + if (current->sps_transform_skip_enabled_flag) { + ue(sps_log2_transform_skip_max_size_minus2, 0, 3); + flag(sps_bdpcm_enabled_flag); + } + + flag(sps_mts_enabled_flag); + if (current->sps_mts_enabled_flag) { + flag(sps_explicit_mts_intra_enabled_flag); + flag(sps_explicit_mts_inter_enabled_flag); + } else { + infer(sps_explicit_mts_intra_enabled_flag, 0); + infer(sps_explicit_mts_inter_enabled_flag, 0); + } + + flag(sps_lfnst_enabled_flag); + + if (current->sps_chroma_format_idc != 0) { + uint8_t num_qp_tables; + flag(sps_joint_cbcr_enabled_flag); + flag(sps_same_qp_table_for_chroma_flag); + num_qp_tables = current->sps_same_qp_table_for_chroma_flag ? + 1 : (current->sps_joint_cbcr_enabled_flag ? 3 : 2); + for (i = 0; i < num_qp_tables; i++) { + ses(sps_qp_table_start_minus26[i], + -26 - qp_bd_offset, 36, 1, i); + ues(sps_num_points_in_qp_table_minus1[i], + 0, 36 - current->sps_qp_table_start_minus26[i], 1, i); + for (j = 0; j <= current->sps_num_points_in_qp_table_minus1[i]; j++) { + uint8_t max = MAX_UINT_BITS(8); + ues(sps_delta_qp_in_val_minus1[i][j], 0, max, 2, i, j); + ues(sps_delta_qp_diff_val[i][j], 0, max, 2, i, j); + } + } + } else { + infer(sps_joint_cbcr_enabled_flag, 0); + infer(sps_same_qp_table_for_chroma_flag, 0); + } + + flag(sps_sao_enabled_flag); + flag(sps_alf_enabled_flag); + if (current->sps_alf_enabled_flag && current->sps_chroma_format_idc) + flag(sps_ccalf_enabled_flag); + else + infer(sps_ccalf_enabled_flag, 0); + flag(sps_lmcs_enabled_flag); + flag(sps_weighted_pred_flag); + flag(sps_weighted_bipred_flag); + flag(sps_long_term_ref_pics_flag); + if (current->sps_video_parameter_set_id > 0) + flag(sps_inter_layer_prediction_enabled_flag); + else + infer(sps_inter_layer_prediction_enabled_flag, 0); + flag(sps_idr_rpl_present_flag); + flag(sps_rpl1_same_as_rpl0_flag); + + for (i = 0; i < (current->sps_rpl1_same_as_rpl0_flag ? 1 : 2); i++) { + ues(sps_num_ref_pic_lists[i], 0, VVC_MAX_REF_PIC_LISTS, 1, i); + for (j = 0; j < current->sps_num_ref_pic_lists[i]; j++) + CHECK(FUNC(ref_pic_list_struct)(ctx, rw, + ¤t->sps_ref_pic_list_struct[i][j], + i, j, current)); + } + + if (current->sps_rpl1_same_as_rpl0_flag) { + current->sps_num_ref_pic_lists[1] = current->sps_num_ref_pic_lists[0]; + for (j = 0; j < current->sps_num_ref_pic_lists[0]; j++) + memcpy(¤t->sps_ref_pic_list_struct[1][j], + ¤t->sps_ref_pic_list_struct[0][j], + sizeof(current->sps_ref_pic_list_struct[0][j])); + } + + flag(sps_ref_wraparound_enabled_flag); + + flag(sps_temporal_mvp_enabled_flag); + if (current->sps_temporal_mvp_enabled_flag) + flag(sps_sbtmvp_enabled_flag); + else + infer(sps_sbtmvp_enabled_flag, 0); + + flag(sps_amvr_enabled_flag); + flag(sps_bdof_enabled_flag); + if (current->sps_bdof_enabled_flag) + flag(sps_bdof_control_present_in_ph_flag); + else + infer(sps_bdof_control_present_in_ph_flag, 0); + + flag(sps_smvd_enabled_flag); + flag(sps_dmvr_enabled_flag); + if (current->sps_dmvr_enabled_flag) + flag(sps_dmvr_control_present_in_ph_flag); + else + infer(sps_dmvr_control_present_in_ph_flag, 0); + + flag(sps_mmvd_enabled_flag); + if (current->sps_mmvd_enabled_flag) + flag(sps_mmvd_fullpel_only_enabled_flag); + else + infer(sps_mmvd_fullpel_only_enabled_flag, 0); + + ue(sps_six_minus_max_num_merge_cand, 0, 5); + max_num_merge_cand = 6 - current->sps_six_minus_max_num_merge_cand; + + flag(sps_sbt_enabled_flag); + + flag(sps_affine_enabled_flag); + if (current->sps_affine_enabled_flag) { + ue(sps_five_minus_max_num_subblock_merge_cand, + 0, 5 - current->sps_sbtmvp_enabled_flag); + flag(sps_6param_affine_enabled_flag); + if (current->sps_amvr_enabled_flag) + flag(sps_affine_amvr_enabled_flag); + else + infer(sps_affine_amvr_enabled_flag, 0); + flag(sps_affine_prof_enabled_flag); + if (current->sps_affine_prof_enabled_flag) + flag(sps_prof_control_present_in_ph_flag); + else + infer(sps_prof_control_present_in_ph_flag, 0); + } else { + infer(sps_6param_affine_enabled_flag, 0); + infer(sps_affine_amvr_enabled_flag, 0); + infer(sps_affine_prof_enabled_flag, 0); + infer(sps_prof_control_present_in_ph_flag, 0); + } + + flag(sps_bcw_enabled_flag); + flag(sps_ciip_enabled_flag); + + if (max_num_merge_cand >= 2) { + flag(sps_gpm_enabled_flag); + if (current->sps_gpm_enabled_flag && max_num_merge_cand >= 3) + ue(sps_max_num_merge_cand_minus_max_num_gpm_cand, + 0, max_num_merge_cand - 2); + } else { + infer(sps_gpm_enabled_flag, 0); + } + + ue(sps_log2_parallel_merge_level_minus2, 0, ctb_log2_size_y - 2); + + flag(sps_isp_enabled_flag); + flag(sps_mrl_enabled_flag); + flag(sps_mip_enabled_flag); + + if (current->sps_chroma_format_idc != 0) + flag(sps_cclm_enabled_flag); + else + infer(sps_cclm_enabled_flag, 0); + if (current->sps_chroma_format_idc == 1) { + flag(sps_chroma_horizontal_collocated_flag); + flag(sps_chroma_vertical_collocated_flag); + } else { + infer(sps_chroma_horizontal_collocated_flag, 1); + infer(sps_chroma_vertical_collocated_flag, 1); + } + + flag(sps_palette_enabled_flag); + if (current->sps_chroma_format_idc == 3 && + !current->sps_max_luma_transform_size_64_flag) + flag(sps_act_enabled_flag); + else + infer(sps_act_enabled_flag, 0); + if (current->sps_transform_skip_enabled_flag || + current->sps_palette_enabled_flag) + ue(sps_min_qp_prime_ts, 0, 8); + + flag(sps_ibc_enabled_flag); + if (current->sps_ibc_enabled_flag) + ue(sps_six_minus_max_num_ibc_merge_cand, 0, 5); + + flag(sps_ladf_enabled_flag); + if (current->sps_ladf_enabled_flag) { + ub(2, sps_num_ladf_intervals_minus2); + se(sps_ladf_lowest_interval_qp_offset, -63, 63); + for (i = 0; i < current->sps_num_ladf_intervals_minus2 + 1; i++) { + ses(sps_ladf_qp_offset[i], -63, 63, 1, i); + ues(sps_ladf_delta_threshold_minus1[i], + 0, (2 << (8 + current->sps_bitdepth_minus8)) - 3, 1, i); + } + } + + flag(sps_explicit_scaling_list_enabled_flag); + if (current->sps_lfnst_enabled_flag && + current->sps_explicit_scaling_list_enabled_flag) + flag(sps_scaling_matrix_for_lfnst_disabled_flag); + + if (current->sps_act_enabled_flag && + current->sps_explicit_scaling_list_enabled_flag) + flag(sps_scaling_matrix_for_alternative_colour_space_disabled_flag); + else + infer(sps_scaling_matrix_for_alternative_colour_space_disabled_flag, 0); + if (current->sps_scaling_matrix_for_alternative_colour_space_disabled_flag) + flag(sps_scaling_matrix_designated_colour_space_flag); + + flag(sps_dep_quant_enabled_flag); + flag(sps_sign_data_hiding_enabled_flag); + + flag(sps_virtual_boundaries_enabled_flag); + if (current->sps_virtual_boundaries_enabled_flag) { + flag(sps_virtual_boundaries_present_flag); + if (current->sps_virtual_boundaries_present_flag) { + ue(sps_num_ver_virtual_boundaries, + 0, current->sps_pic_width_max_in_luma_samples <= 8 ? 0 : 3); + for (i = 0; i < current->sps_num_ver_virtual_boundaries; i++) + ues(sps_virtual_boundary_pos_x_minus1[i], + 0, (current->sps_pic_width_max_in_luma_samples + 7) / 8 - 2, + 1, i); + ue(sps_num_hor_virtual_boundaries, + 0, current->sps_pic_height_max_in_luma_samples <= 8 ? 0 : 3); + for (i = 0; i < current->sps_num_hor_virtual_boundaries; i++) + ues(sps_virtual_boundary_pos_y_minus1[i], + 0, (current->sps_pic_height_max_in_luma_samples + 7) / 8 - 2, + 1, i); + } + } else { + infer(sps_virtual_boundaries_present_flag, 0); + infer(sps_num_ver_virtual_boundaries, 0); + infer(sps_num_hor_virtual_boundaries, 0); + } + + if (current->sps_ptl_dpb_hrd_params_present_flag) { + flag(sps_timing_hrd_params_present_flag); + if (current->sps_timing_hrd_params_present_flag) { + uint8_t first_sublayer; + CHECK(FUNC(general_timing_hrd_parameters)(ctx, rw, + ¤t->sps_general_timing_hrd_parameters)); + if (current->sps_max_sublayers_minus1 > 0) + flag(sps_sublayer_cpb_params_present_flag); + else + infer(sps_sublayer_cpb_params_present_flag, 0); + first_sublayer = current->sps_sublayer_cpb_params_present_flag ? + 0 : current->sps_max_sublayers_minus1; + CHECK(FUNC(ols_timing_hrd_parameters)(ctx, rw, + ¤t->sps_ols_timing_hrd_parameters, first_sublayer, + current->sps_max_sublayers_minus1, + ¤t->sps_general_timing_hrd_parameters)); + } + } + + flag(sps_field_seq_flag); + flag(sps_vui_parameters_present_flag); + if (current->sps_vui_parameters_present_flag) { + ue(sps_vui_payload_size_minus1, 0, 1023); + while (byte_alignment(rw) != 0) + fixed(1, sps_vui_alignment_zero_bit, 0); + CHECK(FUNC(vui_payload)(ctx, rw, ¤t->vui, + current->sps_vui_payload_size_minus1 + 1, + current->sps_chroma_format_idc)); + } else { + CHECK(FUNC(vui_parameters_default)(ctx, rw, ¤t->vui)); + } + flag(sps_extension_flag); + if (current->sps_extension_flag) + CHECK(FUNC(extension_data)(ctx, rw, ¤t->extension_data)); + + CHECK(FUNC(rbsp_trailing_bits)(ctx, rw)); + + return 0; +} + +static int FUNC(pps)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawPPS *current) +{ + + CodedBitstreamH266Context *h266 = ctx->priv_data; + const H266RawSPS *sps; + int err, i; + unsigned int min_cb_size_y, divisor, ctb_size_y, + pic_width_in_ctbs_y, pic_height_in_ctbs_y; + uint8_t sub_width_c, sub_height_c, qp_bd_offset; + + static const uint8_t h266_sub_width_c[] = { + 1, 2, 2, 1 + }; + static const uint8_t h266_sub_height_c[] = { + 1, 2, 1, 1 + }; + + HEADER("Picture Parameter Set"); + + CHECK(FUNC(nal_unit_header)(ctx, rw, + ¤t->nal_unit_header, VVC_PPS_NUT)); + + ub(6, pps_pic_parameter_set_id); + ub(4, pps_seq_parameter_set_id); + sps = h266->sps[current->pps_seq_parameter_set_id]; + if (!sps) { + av_log(ctx->log_ctx, AV_LOG_ERROR, "SPS id %d not available.\n", + current->pps_seq_parameter_set_id); + return AVERROR_INVALIDDATA; + } + + flag(pps_mixed_nalu_types_in_pic_flag); + ue(pps_pic_width_in_luma_samples, + 1, sps->sps_pic_width_max_in_luma_samples); + ue(pps_pic_height_in_luma_samples, + 1, sps->sps_pic_height_max_in_luma_samples); + + min_cb_size_y = 1 << (sps->sps_log2_min_luma_coding_block_size_minus2 + 2); + divisor = FFMAX(min_cb_size_y, 8); + if (current->pps_pic_width_in_luma_samples % divisor || + current->pps_pic_height_in_luma_samples % divisor) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "Invalid dimensions: %ux%u not divisible " + "by %u, MinCbSizeY = %u.\n", current->pps_pic_width_in_luma_samples, + current->pps_pic_height_in_luma_samples, divisor, min_cb_size_y); + return AVERROR_INVALIDDATA; + } + if (!sps->sps_res_change_in_clvs_allowed_flag && + (current->pps_pic_width_in_luma_samples != + sps->sps_pic_width_max_in_luma_samples || + current->pps_pic_height_in_luma_samples != + sps->sps_pic_height_max_in_luma_samples)) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "Resoltuion change is not allowed, " + "in max resolution (%ux%u) mismatched with pps(%ux%u).\n", + sps->sps_pic_width_max_in_luma_samples, + sps->sps_pic_height_max_in_luma_samples, + current->pps_pic_width_in_luma_samples, + current->pps_pic_height_in_luma_samples); + return AVERROR_INVALIDDATA; + } + + ctb_size_y = 1 << (sps->sps_log2_ctu_size_minus5 + 5); + if (sps->sps_ref_wraparound_enabled_flag) { + if ((ctb_size_y / min_cb_size_y + 1) > + (current->pps_pic_width_in_luma_samples / min_cb_size_y - 1)) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "Invalid width(%u), ctb_size_y = %u, min_cb_size_y = %u.\n", + current->pps_pic_width_in_luma_samples, + ctb_size_y, min_cb_size_y); + return AVERROR_INVALIDDATA; + } + } + + flag(pps_conformance_window_flag); + if (current->pps_pic_width_in_luma_samples == + sps->sps_pic_width_max_in_luma_samples && + current->pps_pic_height_in_luma_samples == + sps->sps_pic_height_max_in_luma_samples && + current->pps_conformance_window_flag) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "Conformance window flag should not true.\n"); + return AVERROR_INVALIDDATA; + } + + sub_width_c = h266_sub_width_c[sps->sps_chroma_format_idc]; + sub_height_c = h266_sub_height_c[sps->sps_chroma_format_idc]; + if (current->pps_conformance_window_flag) { + ue(pps_conf_win_left_offset, + 0, current->pps_pic_width_in_luma_samples); + ue(pps_conf_win_right_offset, + 0, current->pps_pic_width_in_luma_samples); + ue(pps_conf_win_top_offset, + 0, current->pps_pic_height_in_luma_samples); + ue(pps_conf_win_bottom_offset, + 0, current->pps_pic_height_in_luma_samples); + if (sub_width_c * + (current->pps_conf_win_left_offset + + current->pps_conf_win_right_offset) >= + current->pps_pic_width_in_luma_samples || + sub_height_c * + (current->pps_conf_win_top_offset + + current->pps_conf_win_bottom_offset) >= + current->pps_pic_height_in_luma_samples) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "Invalid pps conformance window: (%u, %u, %u, %u), " + "resolution is %ux%u, sub wxh is %ux%u.\n", + current->pps_conf_win_left_offset, + current->pps_conf_win_right_offset, + current->pps_conf_win_top_offset, + current->pps_conf_win_bottom_offset, + current->pps_pic_width_in_luma_samples, + current->pps_pic_height_in_luma_samples, + sub_width_c, sub_height_c); + return AVERROR_INVALIDDATA; + } + } else { + if (current->pps_pic_width_in_luma_samples == + sps->sps_pic_width_max_in_luma_samples && + current->pps_pic_height_in_luma_samples == + sps->sps_pic_height_max_in_luma_samples) { + infer(pps_conf_win_left_offset, sps->sps_conf_win_left_offset); + infer(pps_conf_win_right_offset, sps->sps_conf_win_right_offset); + infer(pps_conf_win_top_offset, sps->sps_conf_win_top_offset); + infer(pps_conf_win_bottom_offset, sps->sps_conf_win_bottom_offset); + } else { + infer(pps_conf_win_left_offset, 0); + infer(pps_conf_win_right_offset, 0); + infer(pps_conf_win_top_offset, 0); + infer(pps_conf_win_bottom_offset, 0); + } + + } + + flag(pps_scaling_window_explicit_signalling_flag); + if (!sps->sps_ref_pic_resampling_enabled_flag && + current->pps_scaling_window_explicit_signalling_flag) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "Invalid data: sps_ref_pic_resampling_enabled_flag is false, " + "but pps_scaling_window_explicit_signalling_flag is true.\n"); + return AVERROR_INVALIDDATA; + } + if (current->pps_scaling_window_explicit_signalling_flag) { + se(pps_scaling_win_left_offset, + -current->pps_pic_width_in_luma_samples * 15 / sub_width_c, + current->pps_pic_width_in_luma_samples / sub_width_c); + se(pps_scaling_win_right_offset, + -current->pps_pic_width_in_luma_samples * 15 / sub_width_c, + current->pps_pic_width_in_luma_samples / sub_width_c); + se(pps_scaling_win_top_offset, + -current->pps_pic_height_in_luma_samples * 15 / sub_height_c, + current->pps_pic_height_in_luma_samples / sub_height_c); + se(pps_scaling_win_bottom_offset, + -current->pps_pic_height_in_luma_samples * 15 / sub_height_c, + current->pps_pic_height_in_luma_samples / sub_height_c); + } else { + infer(pps_scaling_win_left_offset, current->pps_conf_win_left_offset); + infer(pps_scaling_win_right_offset, current->pps_conf_win_right_offset); + infer(pps_scaling_win_top_offset, current->pps_conf_win_top_offset); + infer(pps_scaling_win_bottom_offset, current->pps_conf_win_bottom_offset); + } + + flag(pps_output_flag_present_flag); + flag(pps_no_pic_partition_flag); + flag(pps_subpic_id_mapping_present_flag); + + if (current->pps_subpic_id_mapping_present_flag) { + if (!current->pps_no_pic_partition_flag) { + ue(pps_num_subpics_minus1, + sps->sps_num_subpics_minus1, sps->sps_num_subpics_minus1); + } else { + infer(pps_num_subpics_minus1, 0); + } + ue(pps_subpic_id_len_minus1, sps->sps_subpic_id_len_minus1, sps->sps_subpic_id_len_minus1); + for (i = 0; i <= current->pps_num_subpics_minus1; i++) { + ubs(sps->sps_subpic_id_len_minus1 + 1, pps_subpic_id[i], 1, i); + } + } + + pic_width_in_ctbs_y = + h266_ceil(current->pps_pic_width_in_luma_samples, ctb_size_y); + pic_height_in_ctbs_y = + h266_ceil(current->pps_pic_height_in_luma_samples, ctb_size_y); + if (!current->pps_no_pic_partition_flag) { + unsigned int exp_tile_width = 0, exp_tile_height = 0; + unsigned int unified_size, remaining_size; + + u(2, pps_log2_ctu_size_minus5, + sps->sps_log2_ctu_size_minus5, sps->sps_log2_ctu_size_minus5); + ue(pps_num_exp_tile_columns_minus1, + 0, FFMIN(pic_width_in_ctbs_y - 1, VVC_MAX_TILE_COLUMNS - 1)); + ue(pps_num_exp_tile_rows_minus1, + 0, FFMIN(pic_height_in_ctbs_y - 1, VVC_MAX_TILE_ROWS - 1)); + + for (i = 0; i <= current->pps_num_exp_tile_columns_minus1; i++) { + ues(pps_tile_column_width_minus1[i], + 0, pic_width_in_ctbs_y - exp_tile_width - 1, 1, i); + exp_tile_width += current->pps_tile_column_width_minus1[i] + 1; + } + remaining_size = pic_width_in_ctbs_y - exp_tile_width; + unified_size = (i == 0 ? pic_width_in_ctbs_y : + (current->pps_tile_column_width_minus1[i - 1] + 1)); + current->num_tile_columns = i + h266_ceil(remaining_size, unified_size); + if (current->num_tile_columns > VVC_MAX_TILE_COLUMNS) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "NumTileColumns(%d) large than %d.\n", + current->num_tile_columns, VVC_MAX_TILE_COLUMNS); + return AVERROR_INVALIDDATA; + } + while (remaining_size > unified_size) { + current->pps_tile_column_width_minus1[i] = unified_size - 1; + remaining_size -= unified_size; + i++; + } + if (remaining_size > 0) { + current->pps_tile_column_width_minus1[i] = remaining_size - 1; + } + + for (i = 0; i <= current->pps_num_exp_tile_rows_minus1; i++) { + ues(pps_tile_row_height_minus1[i], + 0, pic_height_in_ctbs_y - 1, 1, i); + exp_tile_height += current->pps_tile_row_height_minus1[i] + 1; + } + remaining_size = pic_height_in_ctbs_y - exp_tile_height; + unified_size = (i == 0 ? pic_height_in_ctbs_y : + (current->pps_tile_row_height_minus1[i - 1] + 1)); + current->num_tile_rows = i + h266_ceil(remaining_size, unified_size); + if (current->num_tile_rows > VVC_MAX_TILE_ROWS) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "NumTileRows(%d) large than %d, consider increase VVC_MAX_TILE_ROWS.\n", + current->num_tile_rows, VVC_MAX_TILE_ROWS); + return AVERROR_INVALIDDATA; + } + while (remaining_size > unified_size) { + current->pps_tile_row_height_minus1[i] = unified_size - 1; + remaining_size -= unified_size; + i++; + } + if (remaining_size > 0) { + current->pps_tile_row_height_minus1[i] = remaining_size - 1; + } + + current->num_tiles_in_pic = + current->num_tile_columns * current->num_tile_rows; + if (current->num_tiles_in_pic > VVC_MAX_TILES_PER_AU) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "NumTilesInPic(%d) large than %d.\n", + current->num_tiles_in_pic, VVC_MAX_TILES_PER_AU); + return AVERROR_INVALIDDATA; + } + + if (current->num_tiles_in_pic > 1) { + flag(pps_loop_filter_across_tiles_enabled_flag); + flag(pps_rect_slice_flag); + } else { + infer(pps_loop_filter_across_tiles_enabled_flag, 0); + infer(pps_rect_slice_flag, 1); + } + if (current->pps_rect_slice_flag) + flag(pps_single_slice_per_subpic_flag); + else + infer(pps_single_slice_per_subpic_flag, 1); + if (current->pps_rect_slice_flag && + !current->pps_single_slice_per_subpic_flag) { + int j; + uint16_t tile_idx = 0, tile_x, tile_y, ctu_x, ctu_y; + uint16_t slice_top_left_ctu_x[VVC_MAX_SLICES]; + uint16_t slice_top_left_ctu_y[VVC_MAX_SLICES]; + ue(pps_num_slices_in_pic_minus1, 0, VVC_MAX_SLICES - 1); + if (current->pps_num_slices_in_pic_minus1 > 1) + flag(pps_tile_idx_delta_present_flag); + else + infer(pps_tile_idx_delta_present_flag, 0); + for (i = 0; i < current->pps_num_slices_in_pic_minus1; i++) { + tile_x = tile_idx % current->num_tile_columns; + tile_y = tile_idx / current->num_tile_columns; + if (tile_x != current->num_tile_columns - 1) { + ues(pps_slice_width_in_tiles_minus1[i], + 0, current->num_tile_columns - 1, 1, i); + } else { + infer(pps_slice_width_in_tiles_minus1[i], 0); + } + if (tile_y != current->num_tile_rows - 1 && + (current->pps_tile_idx_delta_present_flag || tile_x == 0)) { + ues(pps_slice_height_in_tiles_minus1[i], + 0, current->num_tile_rows - 1, 1, i); + } else { + if (tile_y == current->num_tile_rows - 1) + infer(pps_slice_height_in_tiles_minus1[i], 0); + else + infer(pps_slice_height_in_tiles_minus1[i], + current->pps_slice_height_in_tiles_minus1[i-1]); + } + + ctu_x = ctu_y = 0; + for (j = 0; j < tile_x; j++) { + ctu_x += current->pps_tile_column_width_minus1[j] + 1; + } + for (j = 0; j < tile_y; j++) { + ctu_y += current->pps_tile_row_height_minus1[j] + 1; + } + if (current->pps_slice_width_in_tiles_minus1[i] == 0 && + current->pps_slice_height_in_tiles_minus1[i] == 0 && + current->pps_tile_row_height_minus1[tile_y] > 0) { + int num_slices_in_tile, + uniform_slice_height, remaining_height_in_ctbs_y; + remaining_height_in_ctbs_y = + current->pps_tile_row_height_minus1[tile_y] + 1; + ues(pps_num_exp_slices_in_tile[i], + 0, current->pps_tile_row_height_minus1[tile_y], 1, i); + if (current->pps_num_exp_slices_in_tile[i] == 0) { + num_slices_in_tile = 1; + slice_top_left_ctu_x[i] = ctu_x; + slice_top_left_ctu_y[i] = ctu_y; + } else { + uint16_t slice_height_in_ctus; + for (j = 0; j < current->pps_num_exp_slices_in_tile[i]; j++) { + ues(pps_exp_slice_height_in_ctus_minus1[i][j], + 0, current->pps_tile_row_height_minus1[tile_y], 2, i, j); + slice_height_in_ctus = + current->pps_exp_slice_height_in_ctus_minus1[i][j] + 1; + + current->slice_height_in_ctus[i + j] = slice_height_in_ctus; + slice_top_left_ctu_x[i + j] = ctu_x; + slice_top_left_ctu_y[i + j] = ctu_y; + ctu_y += slice_height_in_ctus; + + remaining_height_in_ctbs_y -= slice_height_in_ctus; + } + uniform_slice_height = 1 + + (j == 0 ? current->pps_tile_row_height_minus1[tile_y] : + current->pps_exp_slice_height_in_ctus_minus1[i][j-1]); + while (remaining_height_in_ctbs_y > uniform_slice_height) { + current->slice_height_in_ctus[i + j] = uniform_slice_height; + slice_top_left_ctu_x[i + j] = ctu_x; + slice_top_left_ctu_y[i + j] = ctu_y; + ctu_y += uniform_slice_height; + + remaining_height_in_ctbs_y -= uniform_slice_height; + j++; + } + if (remaining_height_in_ctbs_y > 0) { + current->slice_height_in_ctus[i + j] = remaining_height_in_ctbs_y; + slice_top_left_ctu_x[i + j] = ctu_x; + slice_top_left_ctu_y[i + j] = ctu_y; + j++; + } + num_slices_in_tile = j; + } + i += num_slices_in_tile - 1; + } else { + uint16_t height= 0; + infer(pps_num_exp_slices_in_tile[i], 0); + for (j = 0; j <= current->pps_slice_height_in_tiles_minus1[i]; j++) { + height += current->pps_tile_row_height_minus1[tile_y + j] + 1; + } + current->slice_height_in_ctus[i] = height; + + slice_top_left_ctu_x[i] = ctu_x; + slice_top_left_ctu_y[i] = ctu_y; + } + if (i < current->pps_num_slices_in_pic_minus1) { + if (current->pps_tile_idx_delta_present_flag) { + ses(pps_tile_idx_delta_val[i], + -current->num_tiles_in_pic + 1, current->num_tiles_in_pic - 1, + 1, i); + if (current->pps_tile_idx_delta_val[i] == 0) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "pps_tile_idx_delta_val[i] shall not be equal to 0.\n"); + } + tile_idx += current->pps_tile_idx_delta_val[i]; + } else { + infer(pps_tile_idx_delta_val[i], 0); + tile_idx += current->pps_slice_width_in_tiles_minus1[i] + 1; + if (tile_idx % current->num_tile_columns == 0) { + tile_idx += current->pps_slice_height_in_tiles_minus1[i] * + current->num_tile_columns; + } + } + } + } + if (i == current->pps_num_slices_in_pic_minus1) { + uint16_t height= 0; + + tile_x = tile_idx % current->num_tile_columns; + tile_y = tile_idx / current->num_tile_columns; + + ctu_x = 0, ctu_y = 0; + for (j = 0; j < tile_x; j++) { + ctu_x += current->pps_tile_column_width_minus1[j] + 1; + } + for (j = 0; j < tile_y; j++) { + ctu_y += current->pps_tile_row_height_minus1[j] + 1; + } + slice_top_left_ctu_x[i] = ctu_x; + slice_top_left_ctu_y[i] = ctu_y; + + current->pps_slice_width_in_tiles_minus1[i] = + current->num_tile_columns - tile_x - 1; + current->pps_slice_height_in_tiles_minus1[i] = + current->num_tile_rows - tile_y - 1; + + for (j = 0; j <= current->pps_slice_height_in_tiles_minus1[i]; j++) { + height += current->pps_tile_row_height_minus1[tile_y + j] + 1; + } + current->slice_height_in_ctus[i] = height; + + infer(pps_num_exp_slices_in_tile[i], 0); + } + + //now, we got all slice information, let's resolve NumSlicesInSubpic + for (i = 0; i <= sps->sps_num_subpics_minus1; i++) { + current->num_slices_in_subpic[i] = 0; + for (j = 0; j <= current->pps_num_slices_in_pic_minus1; j++) { + uint16_t pos_x = 0, pos_y = 0; + pos_x = slice_top_left_ctu_x[j]; + pos_y = slice_top_left_ctu_y[j]; + if ((pos_x >= sps->sps_subpic_ctu_top_left_x[i]) && + (pos_x < sps->sps_subpic_ctu_top_left_x[i] + sps->sps_subpic_width_minus1[i] + 1) && + (pos_y >= sps->sps_subpic_ctu_top_left_y[i]) && + (pos_y < sps->sps_subpic_ctu_top_left_y[i] + sps->sps_subpic_height_minus1[i] + 1)) { + current->num_slices_in_subpic[i]++; + } + } + } + } else { + if (current->pps_no_pic_partition_flag) + infer(pps_num_slices_in_pic_minus1, 0); + else if (current->pps_single_slice_per_subpic_flag) + infer(pps_num_slices_in_pic_minus1, sps->sps_num_subpics_minus1); + // else? + } + if (!current->pps_rect_slice_flag || + current->pps_single_slice_per_subpic_flag || + current->pps_num_slices_in_pic_minus1 > 0) + flag(pps_loop_filter_across_slices_enabled_flag); + else + infer(pps_loop_filter_across_slices_enabled_flag, 0); + } else { + infer(pps_num_exp_tile_columns_minus1, 0); + infer(pps_tile_column_width_minus1[0], pic_width_in_ctbs_y - 1); + infer(pps_num_exp_tile_rows_minus1, 0); + infer(pps_tile_row_height_minus1[0], pic_height_in_ctbs_y - 1); + infer(num_tile_columns, 1); + infer(num_tile_rows, 1); + infer(num_tiles_in_pic, 1); + } + + flag(pps_cabac_init_present_flag); + for (i = 0; i < 2; i++) + ues(pps_num_ref_idx_default_active_minus1[i], 0, 14, 1, i); + flag(pps_rpl1_idx_present_flag); + flag(pps_weighted_pred_flag); + flag(pps_weighted_bipred_flag); + flag(pps_ref_wraparound_enabled_flag); + if (current->pps_ref_wraparound_enabled_flag) { + ue(pps_pic_width_minus_wraparound_offset, + 0, (current->pps_pic_width_in_luma_samples / min_cb_size_y) + - (ctb_size_y / min_cb_size_y) - 2); + } + + qp_bd_offset = 6 * sps->sps_bitdepth_minus8; + se(pps_init_qp_minus26, -(26 + qp_bd_offset), 37); + flag(pps_cu_qp_delta_enabled_flag); + flag(pps_chroma_tool_offsets_present_flag); + if (current->pps_chroma_tool_offsets_present_flag) { + se(pps_cb_qp_offset, -12, 12); + se(pps_cr_qp_offset, -12, 12); + flag(pps_joint_cbcr_qp_offset_present_flag); + if (current->pps_joint_cbcr_qp_offset_present_flag) + se(pps_joint_cbcr_qp_offset_value, -12, 12); + else + infer(pps_joint_cbcr_qp_offset_value, 0); + flag(pps_slice_chroma_qp_offsets_present_flag); + flag(pps_cu_chroma_qp_offset_list_enabled_flag); + if (current->pps_cu_chroma_qp_offset_list_enabled_flag) { + ue(pps_chroma_qp_offset_list_len_minus1, 0, 5); + for (i = 0; i <= current->pps_chroma_qp_offset_list_len_minus1; i++) { + ses(pps_cb_qp_offset_list[i], -12, 12, 1, i); + ses(pps_cr_qp_offset_list[i], -12, 12, 1, i); + if (current->pps_joint_cbcr_qp_offset_present_flag) + ses(pps_joint_cbcr_qp_offset_list[i], -12, 12, 1, i); + else + infer(pps_joint_cbcr_qp_offset_list[i], 0); + } + } + } else { + infer(pps_cb_qp_offset, 0); + infer(pps_cr_qp_offset, 0); + infer(pps_joint_cbcr_qp_offset_present_flag, 0); + infer(pps_joint_cbcr_qp_offset_value, 0); + infer(pps_slice_chroma_qp_offsets_present_flag, 0); + infer(pps_cu_chroma_qp_offset_list_enabled_flag, 0); + } + flag(pps_deblocking_filter_control_present_flag); + if (current->pps_deblocking_filter_control_present_flag) { + flag(pps_deblocking_filter_override_enabled_flag); + flag(pps_deblocking_filter_disabled_flag); + if (!current->pps_no_pic_partition_flag && + current->pps_deblocking_filter_override_enabled_flag) + flag(pps_dbf_info_in_ph_flag); + else + infer(pps_dbf_info_in_ph_flag, 0); + if (!current->pps_deblocking_filter_disabled_flag) { + se(pps_luma_beta_offset_div2, -12, 12); + se(pps_luma_tc_offset_div2, -12, 12); + if (current->pps_chroma_tool_offsets_present_flag) { + se(pps_cb_beta_offset_div2, -12, 12); + se(pps_cb_tc_offset_div2, -12, 12); + se(pps_cr_beta_offset_div2, -12, 12); + se(pps_cr_tc_offset_div2, -12, 12); + } else { + infer(pps_cb_beta_offset_div2, current->pps_luma_beta_offset_div2); + infer(pps_cb_tc_offset_div2, current->pps_luma_tc_offset_div2); + infer(pps_cr_beta_offset_div2, current->pps_luma_beta_offset_div2); + infer(pps_cr_tc_offset_div2, current->pps_luma_tc_offset_div2); + } + } else { + infer(pps_luma_beta_offset_div2, 0); + infer(pps_luma_tc_offset_div2, 0); + infer(pps_cb_beta_offset_div2, 0); + infer(pps_cb_tc_offset_div2, 0); + infer(pps_cr_beta_offset_div2, 0); + infer(pps_cr_tc_offset_div2, 0); + } + } else { + infer(pps_deblocking_filter_override_enabled_flag, 0); + infer(pps_deblocking_filter_disabled_flag, 0); + infer(pps_dbf_info_in_ph_flag, 0); + infer(pps_luma_beta_offset_div2, 0); + infer(pps_luma_tc_offset_div2, 0); + infer(pps_cb_beta_offset_div2, 0); + infer(pps_cb_tc_offset_div2, 0); + infer(pps_cr_beta_offset_div2, 0); + infer(pps_cr_tc_offset_div2, 0); + } + + if (!current->pps_no_pic_partition_flag) { + flag(pps_rpl_info_in_ph_flag); + flag(pps_sao_info_in_ph_flag); + flag(pps_alf_info_in_ph_flag); + if ((current->pps_weighted_pred_flag || current->pps_weighted_bipred_flag) && + current->pps_rpl_info_in_ph_flag) + flag(pps_wp_info_in_ph_flag); + flag(pps_qp_delta_info_in_ph_flag); + } + flag(pps_picture_header_extension_present_flag); + flag(pps_slice_header_extension_present_flag); + + flag(pps_extension_flag); + if (current->pps_extension_flag) + CHECK(FUNC(extension_data)(ctx, rw, ¤t->extension_data)); + + CHECK(FUNC(rbsp_trailing_bits)(ctx, rw)); + return 0; +} + +static int FUNC(aud)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawAUD *current) +{ + int err; + + HEADER("Access Unit Delimiter"); + + CHECK(FUNC(nal_unit_header)(ctx, rw, + ¤t->nal_unit_header, VVC_AUD_NUT)); + + flag(aud_irap_or_gdr_flag); + u(3, aud_pic_type, 0, 2); + + CHECK(FUNC(rbsp_trailing_bits)(ctx, rw)); + return 0; +} + +static int FUNC(pred_weight_table)(CodedBitstreamContext *ctx, RWContext *rw, + const H266RawSPS *sps, + const H266RawPPS *pps, + const H266RefPicLists *ref_lists, + uint8_t num_ref_idx_active[2], + H266RawPredWeightTable *current) +{ + int err, i, j; + uint8_t num_weights_l0, num_weights_l1; + ue(luma_log2_weight_denom, 0, 7); + if (sps->sps_chroma_format_idc != 0) { + se(delta_chroma_log2_weight_denom, + -current->luma_log2_weight_denom, 7 - current->luma_log2_weight_denom); + } else { + infer(delta_chroma_log2_weight_denom, 0); + } + if (pps->pps_wp_info_in_ph_flag) + ue(num_l0_weights, 0, FFMIN(15, ref_lists->rpl_ref_list[0].num_ref_entries)); + else + infer(num_l0_weights, 0); + num_weights_l0 = pps->pps_wp_info_in_ph_flag ? + current->num_l0_weights : num_ref_idx_active[0]; + for (i = 0; i < num_weights_l0; i++) { + flags(luma_weight_l0_flag[i], 1, i); + } + if (sps->sps_chroma_format_idc != 0) { + for (i = 0; i < num_weights_l0; i++) + flags(chroma_weight_l0_flag[i], 1, i); + } + for (i = 0; i < num_weights_l0; i++) { + if (current->luma_weight_l0_flag[i]){ + ses(delta_luma_weight_l0[i], -128, 127, 1, i); + ses(luma_offset_l0[i], -128, 127, 1, i); + } else { + infer(delta_luma_weight_l0[i], 0); + infer(luma_offset_l0[i], 0); + } + if (current->chroma_weight_l0_flag[i]) { + for (j = 0; j < 2; j++) { + ses(delta_chroma_weight_l0[i][j], -128, 127, 2, i, j); + ses(delta_chroma_offset_l0[i][j], -4 * 128, 4 * 127, 2, i, j); + } + } + } + + if (pps->pps_weighted_bipred_flag && pps->pps_wp_info_in_ph_flag && + ref_lists->rpl_ref_list[1].num_ref_entries > 0) { + ue(num_l1_weights, 0, FFMIN(15, ref_lists->rpl_ref_list[1].num_ref_entries)); + } + if (!pps->pps_weighted_bipred_flag || + (pps->pps_wp_info_in_ph_flag && + ref_lists->rpl_ref_list[1].num_ref_entries == 0)) { + num_weights_l1 = 0; + } else if (pps->pps_wp_info_in_ph_flag) { + num_weights_l1 = current->num_l1_weights; + } else { + num_weights_l1 = num_ref_idx_active[1]; + } + + for (i = 0; i < num_weights_l1; i++) + flags(luma_weight_l1_flag[i], 1, i); + if (sps->sps_chroma_format_idc != 0) { + for (i = 0; i < num_weights_l1; i++) + flags(chroma_weight_l1_flag[i], 1, i); + } + for (i = 0; i < num_weights_l1; i++) { + if (current->luma_weight_l1_flag[i]){ + ses(delta_luma_weight_l1[i], -128, 127, 1, i); + ses(luma_offset_l1[i], -128, 127, 1, i); + } else { + infer(delta_luma_weight_l1[i], 0); + infer(luma_offset_l1[i], 0); + } + if (current->chroma_weight_l1_flag[i]) { + for (j = 0; j < 2; j++) { + ses(delta_chroma_weight_l1[i][j], -128, 127, 2, i, j); + ses(delta_chroma_offset_l1[i][j], -4 * 128, 4 * 127, 2, i, j); + } + } + } + return 0; +} + +static int FUNC(picture_header)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawPH *current) +{ + CodedBitstreamH266Context *h266 = ctx->priv_data; + const H266RawVPS *vps; + const H266RawSPS *sps; + const H266RawPPS *pps; + int err, i; + unsigned int ctb_log2_size_y, min_cb_log2_size_y, + min_qt_log2_size_intra_y, min_qt_log2_size_inter_y; + uint8_t qp_bd_offset; + + flag(ph_gdr_or_irap_pic_flag); + flag(ph_non_ref_pic_flag); + if (current->ph_gdr_or_irap_pic_flag) + flag(ph_gdr_pic_flag); + else + infer(ph_gdr_pic_flag, 0); + flag(ph_inter_slice_allowed_flag); + if (current->ph_inter_slice_allowed_flag) + flag(ph_intra_slice_allowed_flag); + else + infer(ph_intra_slice_allowed_flag, 1); + ue(ph_pic_parameter_set_id, 0, VVC_MAX_PPS_COUNT - 1); + pps = h266->pps[current->ph_pic_parameter_set_id]; + if (!pps) { + av_log(ctx->log_ctx, AV_LOG_ERROR, "PPS id %d not available.\n", + current->ph_pic_parameter_set_id); + return AVERROR_INVALIDDATA; + } + sps = h266->sps[pps->pps_seq_parameter_set_id]; + if (!sps) { + av_log(ctx->log_ctx, AV_LOG_ERROR, "SPS id %d not available.\n", + pps->pps_seq_parameter_set_id); + return AVERROR_INVALIDDATA; + } + vps = h266->vps[sps->sps_video_parameter_set_id]; + if (!vps) { + av_log(ctx->log_ctx, AV_LOG_ERROR, "VPS id %d not available.\n", + sps->sps_video_parameter_set_id); + return AVERROR_INVALIDDATA; + } + + ub(sps->sps_log2_max_pic_order_cnt_lsb_minus4 + 4, ph_pic_order_cnt_lsb); + if (current->ph_gdr_pic_flag) + ue(ph_recovery_poc_cnt, 0, 1 << (sps->sps_log2_max_pic_order_cnt_lsb_minus4 + 4)); + + for (i = 0; i < sps->sps_num_extra_ph_bytes * 8; i++) { + if (sps->sps_extra_ph_bit_present_flag[i]) + flags(ph_extra_bit[i], 1, i); + } + if (sps->sps_poc_msb_cycle_flag) { + flag(ph_poc_msb_cycle_present_flag); + if (current->ph_poc_msb_cycle_present_flag) + ub(sps->sps_poc_msb_cycle_len_minus1 + 1, ph_poc_msb_cycle_val); + } + if (sps->sps_alf_enabled_flag && pps->pps_alf_info_in_ph_flag) { + flag(ph_alf_enabled_flag); + if (current->ph_alf_enabled_flag) { + + ub(3, ph_num_alf_aps_ids_luma); + for (i = 0; i < current->ph_num_alf_aps_ids_luma; i++) + ubs(3, ph_alf_aps_id_luma[i], 1, i); + + if (sps->sps_chroma_format_idc != 0) { + flag(ph_alf_cb_enabled_flag); + flag(ph_alf_cr_enabled_flag); + } else { + infer(ph_alf_cb_enabled_flag, 0); + infer(ph_alf_cr_enabled_flag, 0); + } + + if (current->ph_alf_cb_enabled_flag || current->ph_alf_cr_enabled_flag) { + ub(3, ph_alf_aps_id_chroma); + } + + if (sps->sps_ccalf_enabled_flag) { + flag(ph_alf_cc_cb_enabled_flag); + if (current->ph_alf_cc_cb_enabled_flag) + ub(3, ph_alf_cc_cb_aps_id); + flag(ph_alf_cc_cr_enabled_flag); + if (current->ph_alf_cc_cr_enabled_flag) + ub(3, ph_alf_cc_cr_aps_id); + } + } + } else { + infer(ph_alf_enabled_flag, 0); + } + if (sps->sps_lmcs_enabled_flag) { + flag(ph_lmcs_enabled_flag); + if (current->ph_lmcs_enabled_flag) { + ub(2, ph_lmcs_aps_id); + if (sps->sps_chroma_format_idc != 0) + flag(ph_chroma_residual_scale_flag); + else + infer(ph_chroma_residual_scale_flag, 0); + } + } else { + infer(ph_lmcs_enabled_flag, 0); + infer(ph_chroma_residual_scale_flag, 0); + } + + if (sps->sps_explicit_scaling_list_enabled_flag) { + flag(ph_explicit_scaling_list_enabled_flag); + if (current->ph_explicit_scaling_list_enabled_flag) { + //todo: check the ph_scaling_list_aps_id range, when aps ready + ub(3, ph_scaling_list_aps_id); + } + } else { + infer(ph_explicit_scaling_list_enabled_flag, 0); + } + if (sps->sps_virtual_boundaries_enabled_flag && + !sps->sps_virtual_boundaries_present_flag) { + flag(ph_virtual_boundaries_present_flag); + if (current->ph_virtual_boundaries_present_flag) { + ue(ph_num_ver_virtual_boundaries, + 0, pps->pps_pic_width_in_luma_samples <= 8 ? 0 : 3); + for (i = 0; i < current->ph_num_ver_virtual_boundaries; i++) { + ues(ph_virtual_boundary_pos_x_minus1[i], + 0, (pps->pps_pic_width_in_luma_samples + 7) / 8 - 2, 1, i); + } + ue(ph_num_hor_virtual_boundaries, + 0, pps->pps_pic_height_in_luma_samples <= 8 ? 0 : 3); + for (i = 0; i < current->ph_num_hor_virtual_boundaries; i++) { + ues(ph_virtual_boundary_pos_y_minus1[i], + 0, (pps->pps_pic_height_in_luma_samples + 7) / 8 - 2, 1, i); + } + } else { + infer(ph_num_ver_virtual_boundaries, 0); + infer(ph_num_hor_virtual_boundaries, 0); + } + } + if (pps->pps_output_flag_present_flag && !current->ph_non_ref_pic_flag) + flag(ph_pic_output_flag); + else + infer(ph_pic_output_flag, 1); + if (pps->pps_rpl_info_in_ph_flag) { + CHECK(FUNC(ref_pic_lists)(ctx, rw, sps, pps, ¤t->ph_ref_pic_lists)); + } + if (sps->sps_partition_constraints_override_enabled_flag) + flag(ph_partition_constraints_override_flag); + else + infer(ph_partition_constraints_override_flag, 0); + + ctb_log2_size_y = sps->sps_log2_ctu_size_minus5 + 5; + min_cb_log2_size_y = sps->sps_log2_min_luma_coding_block_size_minus2 + 2; + if (current->ph_intra_slice_allowed_flag) { + if (current->ph_partition_constraints_override_flag) { + ue(ph_log2_diff_min_qt_min_cb_intra_slice_luma, + 0, FFMIN(6, ctb_log2_size_y) - min_cb_log2_size_y); + ue(ph_max_mtt_hierarchy_depth_intra_slice_luma, + 0, 2 * (ctb_log2_size_y - min_cb_log2_size_y)); + if (current->ph_max_mtt_hierarchy_depth_intra_slice_luma != 0) { + ue(ph_log2_diff_max_bt_min_qt_intra_slice_luma, + 0, ctb_log2_size_y - min_qt_log2_size_intra_y); + ue(ph_log2_diff_max_tt_min_qt_intra_slice_luma, + 0, FFMIN(6, ctb_log2_size_y) - min_qt_log2_size_intra_y); + } else { + infer(ph_log2_diff_max_bt_min_qt_intra_slice_luma, + sps->sps_log2_diff_max_bt_min_qt_intra_slice_luma); + infer(ph_log2_diff_max_tt_min_qt_intra_slice_luma, + sps->sps_log2_diff_max_tt_min_qt_intra_slice_luma); + } + if (sps->sps_qtbtt_dual_tree_intra_flag) { + ue(ph_log2_diff_min_qt_min_cb_intra_slice_chroma, + 0, FFMIN(6, ctb_log2_size_y) - min_cb_log2_size_y); + ue(ph_max_mtt_hierarchy_depth_intra_slice_chroma, + 0, 2 * (ctb_log2_size_y - min_cb_log2_size_y)); + if (sps->sps_max_mtt_hierarchy_depth_intra_slice_chroma != 0) { + unsigned int min_qt_log2_size_intra_c = + sps->sps_log2_diff_min_qt_min_cb_intra_slice_chroma + + min_cb_log2_size_y; + ue(ph_log2_diff_max_bt_min_qt_intra_slice_chroma, + 0, FFMIN(6, ctb_log2_size_y) - min_qt_log2_size_intra_c); + ue(ph_log2_diff_max_tt_min_qt_intra_slice_chroma, + 0, FFMIN(6, ctb_log2_size_y) - min_qt_log2_size_intra_c); + } else { + infer(ph_log2_diff_max_bt_min_qt_intra_slice_chroma, + sps->sps_log2_diff_max_bt_min_qt_intra_slice_chroma); + infer(ph_log2_diff_max_tt_min_qt_intra_slice_chroma, + sps->sps_log2_diff_max_tt_min_qt_intra_slice_chroma); + } + } + } else { + infer(ph_log2_diff_min_qt_min_cb_intra_slice_luma, + sps->sps_log2_diff_min_qt_min_cb_intra_slice_luma); + infer(ph_max_mtt_hierarchy_depth_intra_slice_luma, + sps->sps_max_mtt_hierarchy_depth_intra_slice_luma); + infer(ph_log2_diff_max_bt_min_qt_intra_slice_luma, + sps->sps_log2_diff_max_bt_min_qt_intra_slice_luma); + infer(ph_log2_diff_max_tt_min_qt_intra_slice_luma, + sps->sps_log2_diff_max_tt_min_qt_intra_slice_luma); + infer(ph_log2_diff_min_qt_min_cb_intra_slice_chroma, + sps->sps_log2_diff_min_qt_min_cb_intra_slice_chroma); + infer(ph_max_mtt_hierarchy_depth_intra_slice_chroma, + sps->sps_max_mtt_hierarchy_depth_intra_slice_chroma); + infer(ph_log2_diff_max_bt_min_qt_intra_slice_chroma, + sps->sps_log2_diff_max_bt_min_qt_intra_slice_chroma); + infer(ph_log2_diff_max_tt_min_qt_intra_slice_chroma, + sps->sps_log2_diff_max_tt_min_qt_intra_slice_chroma); + } + + min_qt_log2_size_intra_y = + current->ph_log2_diff_min_qt_min_cb_intra_slice_luma + min_cb_log2_size_y; + if (pps->pps_cu_qp_delta_enabled_flag) + ue(ph_cu_qp_delta_subdiv_intra_slice, 0, + 2 * (ctb_log2_size_y - min_qt_log2_size_intra_y + + current->ph_max_mtt_hierarchy_depth_intra_slice_luma)); + else + infer(ph_cu_qp_delta_subdiv_intra_slice, 0); + + if (pps->pps_cu_chroma_qp_offset_list_enabled_flag) + ue(ph_cu_chroma_qp_offset_subdiv_intra_slice, 0, + 2 * (ctb_log2_size_y - min_qt_log2_size_intra_y + + current->ph_max_mtt_hierarchy_depth_intra_slice_luma)); + else + infer(ph_cu_chroma_qp_offset_subdiv_intra_slice, 0); + } + if (current->ph_inter_slice_allowed_flag) { + if (current->ph_partition_constraints_override_flag) { + ue(ph_log2_diff_min_qt_min_cb_inter_slice, + 0, FFMIN(6, ctb_log2_size_y) - min_cb_log2_size_y); + min_qt_log2_size_inter_y = + current->ph_log2_diff_min_qt_min_cb_inter_slice + min_cb_log2_size_y; + ue(ph_max_mtt_hierarchy_depth_inter_slice, + 0, 2 * (ctb_log2_size_y - min_cb_log2_size_y)); + if (current->ph_max_mtt_hierarchy_depth_inter_slice != 0) { + ue(ph_log2_diff_max_bt_min_qt_inter_slice, + 0, ctb_log2_size_y - min_qt_log2_size_inter_y); + ue(ph_log2_diff_max_tt_min_qt_inter_slice, + 0, FFMIN(6, ctb_log2_size_y) - min_qt_log2_size_inter_y); + } + } else { + infer(ph_log2_diff_min_qt_min_cb_inter_slice, + sps->sps_log2_diff_min_qt_min_cb_inter_slice); + min_qt_log2_size_inter_y = + current->ph_log2_diff_min_qt_min_cb_inter_slice + min_cb_log2_size_y; + infer(ph_max_mtt_hierarchy_depth_inter_slice, + sps->sps_max_mtt_hierarchy_depth_inter_slice); + infer(ph_log2_diff_max_bt_min_qt_inter_slice, + sps->sps_log2_diff_max_bt_min_qt_inter_slice); + infer(ph_log2_diff_max_tt_min_qt_inter_slice, + sps->sps_log2_diff_max_tt_min_qt_inter_slice); + } + + if (pps->pps_cu_qp_delta_enabled_flag) + ue(ph_cu_qp_delta_subdiv_inter_slice, 0, + 2 * (ctb_log2_size_y - min_qt_log2_size_inter_y + + current->ph_max_mtt_hierarchy_depth_inter_slice)); + else + infer(ph_cu_qp_delta_subdiv_inter_slice, 0); + + if (pps->pps_cu_chroma_qp_offset_list_enabled_flag) + ue(ph_cu_chroma_qp_offset_subdiv_inter_slice, 0, + 2 * (ctb_log2_size_y - min_qt_log2_size_inter_y + + current->ph_max_mtt_hierarchy_depth_inter_slice)); + else + infer(ph_cu_chroma_qp_offset_subdiv_inter_slice, 0); + if (sps->sps_temporal_mvp_enabled_flag) { + flag(ph_temporal_mvp_enabled_flag); + if (current->ph_temporal_mvp_enabled_flag && + pps->pps_rpl_info_in_ph_flag) { + if (current->ph_ref_pic_lists.rpl_ref_list[1].num_ref_entries > 0) + flag(ph_collocated_from_l0_flag); + else + infer(ph_collocated_from_l0_flag, 1); + if ((current->ph_collocated_from_l0_flag && + current->ph_ref_pic_lists.rpl_ref_list[0].num_ref_entries > 1) || + (!current->ph_collocated_from_l0_flag && + current->ph_ref_pic_lists.rpl_ref_list[1].num_ref_entries > 1)) { + unsigned int idx = current->ph_collocated_from_l0_flag ? 0 : 1; + ue(ph_collocated_ref_idx, 0, + current->ph_ref_pic_lists.rpl_ref_list[idx].num_ref_entries - 1); + } else { + infer(ph_collocated_ref_idx, 0); + } + } + } + if (sps->sps_mmvd_fullpel_only_enabled_flag) + flag(ph_mmvd_fullpel_only_flag); + else + infer(ph_mmvd_fullpel_only_flag, 0); + if (!pps->pps_rpl_info_in_ph_flag || + current->ph_ref_pic_lists.rpl_ref_list[1].num_ref_entries > 0) { + flag(ph_mvd_l1_zero_flag); + if (sps->sps_bdof_control_present_in_ph_flag) { + flag(ph_bdof_disabled_flag); + } else { + if (!sps->sps_bdof_control_present_in_ph_flag) + infer(ph_bdof_disabled_flag, 1 - sps->sps_bdof_enabled_flag); + else + infer(ph_bdof_disabled_flag, 1); + } + if (sps->sps_dmvr_control_present_in_ph_flag) { + flag(ph_dmvr_disabled_flag); + } else { + if (!sps->sps_dmvr_control_present_in_ph_flag) + infer(ph_dmvr_disabled_flag, 1 - sps->sps_dmvr_enabled_flag); + else + infer(ph_dmvr_disabled_flag, 1); + } + } else { + infer(ph_mvd_l1_zero_flag, 1); + } + if (sps->sps_prof_control_present_in_ph_flag) + flag(ph_prof_disabled_flag); + else + infer(ph_prof_disabled_flag, !sps->sps_affine_prof_enabled_flag); + if ((pps->pps_weighted_pred_flag || + pps->pps_weighted_bipred_flag) && + pps->pps_wp_info_in_ph_flag) { + + // if pps->pps_wp_info_in_ph_fla == 1 + // pred_weight_table will not use num_ref_idx_active + uint8_t num_ref_idx_active[2] = {0, 0}; + CHECK(FUNC(pred_weight_table)(ctx, rw, sps, pps, ¤t->ph_ref_pic_lists, + num_ref_idx_active, + ¤t->ph_pred_weight_table)); + } + } + + qp_bd_offset = 6 * sps->sps_bitdepth_minus8; + if (pps->pps_qp_delta_info_in_ph_flag) + se(ph_qp_delta, -qp_bd_offset - (26 + pps->pps_init_qp_minus26), + 63 - (26 + pps->pps_init_qp_minus26)); + + if (sps->sps_joint_cbcr_enabled_flag) + flag(ph_joint_cbcr_sign_flag); + else + infer(ph_joint_cbcr_sign_flag, 0); + if (sps->sps_sao_enabled_flag && pps->pps_sao_info_in_ph_flag) { + flag(ph_sao_luma_enabled_flag); + if (sps->sps_chroma_format_idc != 0) + flag(ph_sao_chroma_enabled_flag); + else + infer(ph_sao_chroma_enabled_flag, 0); + } else { + infer(ph_sao_luma_enabled_flag, 0); + infer(ph_sao_chroma_enabled_flag, 0); + } + + if (pps->pps_dbf_info_in_ph_flag) { + flag(ph_deblocking_params_present_flag); + if (current->ph_deblocking_params_present_flag) { + if (!pps->pps_deblocking_filter_disabled_flag) { + flag(ph_deblocking_filter_disabled_flag); + } else { + if (pps->pps_deblocking_filter_disabled_flag && + current->ph_deblocking_params_present_flag) { + infer(ph_deblocking_filter_disabled_flag, 0); + } else { + infer(ph_deblocking_filter_disabled_flag, + pps->pps_deblocking_filter_disabled_flag); + } + } + if (!current->ph_deblocking_filter_disabled_flag) { + se(ph_luma_beta_offset_div2, -12, 12); + se(ph_luma_tc_offset_div2, -12, 12); + if (pps->pps_chroma_tool_offsets_present_flag) { + se(ph_cb_beta_offset_div2, -12, 12); + se(ph_cb_tc_offset_div2, -12, 12); + se(ph_cr_beta_offset_div2, -12, 12); + se(ph_cr_tc_offset_div2, -12, 12); + } else { + infer(ph_cb_beta_offset_div2, current->ph_luma_beta_offset_div2); + infer(ph_cb_tc_offset_div2, current->ph_luma_tc_offset_div2); + infer(ph_cr_beta_offset_div2, current->ph_luma_beta_offset_div2); + infer(ph_cr_tc_offset_div2, current->ph_luma_tc_offset_div2); + } + } else { + infer(ph_luma_beta_offset_div2, pps->pps_luma_beta_offset_div2); + infer(ph_luma_tc_offset_div2, pps->pps_luma_tc_offset_div2); + if (pps->pps_chroma_tool_offsets_present_flag) { + infer(ph_cb_beta_offset_div2, pps->pps_cb_beta_offset_div2); + infer(ph_cb_tc_offset_div2, pps->pps_cb_tc_offset_div2); + infer(ph_cr_beta_offset_div2, pps->pps_cr_beta_offset_div2); + infer(ph_cr_tc_offset_div2, pps->pps_cr_tc_offset_div2); + } else { + infer(ph_cb_beta_offset_div2, current->ph_luma_beta_offset_div2); + infer(ph_cb_tc_offset_div2, current->ph_luma_tc_offset_div2); + infer(ph_cr_beta_offset_div2, current->ph_luma_beta_offset_div2); + infer(ph_cr_tc_offset_div2, current->ph_luma_tc_offset_div2); + } + } + } + } + + if (pps->pps_picture_header_extension_present_flag) { + ue(ph_extension_length, 0, 256); + for (i = 0; i < current->ph_extension_length; i++) + us(8, ph_extension_data_byte[i], 0x00, 0xff, 1, i); + } + + return 0; +} + +static int FUNC(ph)(CodedBitstreamContext *ctx, RWContext *rw, H266RawPH *current) +{ + int err; + + HEADER("Picture Header"); + + CHECK(FUNC(nal_unit_header)(ctx, rw, ¤t->nal_unit_header, VVC_PH_NUT)); + CHECK(FUNC(picture_header)(ctx, rw, current)); + CHECK(FUNC(rbsp_trailing_bits)(ctx, rw)); + return 0; +} + +static int FUNC(slice_header)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawSliceHeader *current) +{ + CodedBitstreamH266Context *h266 = ctx->priv_data; + const H266RawSPS *sps; + const H266RawPPS *pps; + const H266RawPH *ph; + const H266RefPicLists *ref_pic_lists; + int err, i; + uint8_t nal_unit_type, qp_bd_offset; + uint16_t curr_subpic_idx; + uint16_t num_slices_in_subpic; + + HEADER("Slice Header"); + + CHECK(FUNC(nal_unit_header)(ctx, rw, ¤t->nal_unit_header, -1)); + + flag(sh_picture_header_in_slice_header_flag); + if (current->sh_picture_header_in_slice_header_flag){ + CHECK(FUNC(picture_header)(ctx, rw, ¤t->sh_picture_header)); + ph = ¤t->sh_picture_header; + //7.4.8 if sh_picture_header_in_slice_header_flag is true, we do not have PH NAL unit + h266->priv.ph = NULL; + } else { + ph = h266->priv.ph; + if (!ph) { + av_log(ctx->log_ctx, AV_LOG_ERROR, "Picture header not available.\n"); + return AVERROR_INVALIDDATA; + } + } + + pps = h266->pps[ph->ph_pic_parameter_set_id]; + if (!pps) { + av_log(ctx->log_ctx, AV_LOG_ERROR, "PPS id %d not available.\n", + ph->ph_pic_parameter_set_id); + return AVERROR_INVALIDDATA; + } + sps = h266->sps[pps->pps_seq_parameter_set_id]; + if (!sps) { + av_log(ctx->log_ctx, AV_LOG_ERROR, "SPS id %d not available.\n", + pps->pps_seq_parameter_set_id); + return AVERROR_INVALIDDATA; + } + + if (sps->sps_subpic_info_present_flag) { + ub(sps->sps_subpic_id_len_minus1 + 1, sh_subpic_id); + if (sps->sps_subpic_id_mapping_explicitly_signalled_flag) { + for (i = 0; i <= sps->sps_num_subpics_minus1; i++) { + uint16_t subpic_id_val = pps->pps_subpic_id_mapping_present_flag ? + pps->pps_subpic_id[i] : sps->sps_subpic_id[i]; + if (subpic_id_val == current->sh_subpic_id) { + curr_subpic_idx = i; + break; + } + } + } else { + curr_subpic_idx = current->sh_subpic_id; + if (curr_subpic_idx > sps->sps_num_subpics_minus1) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "sh_subpic_id(%d) should in range [0, %d]\n", + curr_subpic_idx, sps->sps_num_subpics_minus1); + return AVERROR_INVALIDDATA; + } + } + } else { + curr_subpic_idx = 0; + } + + num_slices_in_subpic = pps->num_slices_in_subpic[curr_subpic_idx]; + + if ((pps->pps_rect_slice_flag && num_slices_in_subpic > 1) || + (!pps->pps_rect_slice_flag && pps->num_tiles_in_pic > 1)) { + unsigned int bits, max; + if (!pps->pps_rect_slice_flag) { + bits = av_ceil_log2(pps->num_tiles_in_pic); + max = pps->num_tiles_in_pic - 1; + } else { + bits = av_ceil_log2(num_slices_in_subpic); + max = num_slices_in_subpic - 1; + } + u(bits, sh_slice_address, 0, max); + } else { + infer(sh_slice_address, 0); + } + + for (i = 0; i < sps->sps_num_extra_sh_bytes * 8; i++) { + if (sps->sps_extra_sh_bit_present_flag[i]) + flags(sh_extra_bit[i], 1, i); + } + + if (!pps->pps_rect_slice_flag && + pps->num_tiles_in_pic - current->sh_slice_address > 1) + ue(sh_num_tiles_in_slice_minus1, 0, pps->num_tiles_in_pic - 1); + else + infer(sh_num_tiles_in_slice_minus1, 0); + + if (ph->ph_inter_slice_allowed_flag) + ue(sh_slice_type, 0, 2); + else + infer(sh_slice_type, 2); + + nal_unit_type = current->nal_unit_header.nal_unit_type; + if (nal_unit_type == VVC_IDR_W_RADL || + nal_unit_type == VVC_IDR_N_LP || + nal_unit_type == VVC_CRA_NUT || + nal_unit_type == VVC_GDR_NUT) + flag(sh_no_output_of_prior_pics_flag); + if (sps->sps_alf_enabled_flag && !pps->pps_alf_info_in_ph_flag) { + flag(sh_alf_enabled_flag); + if (current->sh_alf_enabled_flag) { + ub(3, sh_num_alf_aps_ids_luma); + for (i = 0; i < current->sh_num_alf_aps_ids_luma; i++) + ubs(3, sh_alf_aps_id_luma[i], 1, i); + if (sps->sps_chroma_format_idc != 0) { + flag(sh_alf_cb_enabled_flag); + flag(sh_alf_cr_enabled_flag); + } else { + infer(sh_alf_cb_enabled_flag, ph->ph_alf_cb_enabled_flag); + infer(sh_alf_cr_enabled_flag, ph->ph_alf_cr_enabled_flag); + } + if (current->sh_alf_cb_enabled_flag || + current->sh_alf_cr_enabled_flag) + ub(3, sh_alf_aps_id_chroma); + else + infer(sh_alf_aps_id_chroma, ph->ph_alf_aps_id_chroma); + if (sps->sps_ccalf_enabled_flag) { + flag(sh_alf_cc_cb_enabled_flag); + if (current->sh_alf_cc_cb_enabled_flag) + ub(3, sh_alf_cc_cb_aps_id); + else + infer(sh_alf_cc_cb_aps_id, ph->ph_alf_cc_cb_aps_id); + flag(sh_alf_cc_cr_enabled_flag); + if (current->sh_alf_cc_cr_enabled_flag) + ub(3, sh_alf_cc_cr_aps_id); + else + infer(sh_alf_cc_cr_aps_id, ph->ph_alf_cc_cr_aps_id); + } else { + infer(sh_alf_cc_cb_enabled_flag, ph->ph_alf_cc_cb_enabled_flag); + infer(sh_alf_cc_cr_enabled_flag, ph->ph_alf_cc_cr_enabled_flag); + infer(sh_alf_cc_cb_aps_id, ph->ph_alf_cc_cb_aps_id); + infer(sh_alf_cc_cr_aps_id, ph->ph_alf_cc_cr_aps_id); + } + } + } else { + infer(sh_alf_enabled_flag, 0); + } + + if (ph->ph_lmcs_enabled_flag && + !current->sh_picture_header_in_slice_header_flag) + flag(sh_lmcs_used_flag); + else + infer(sh_lmcs_used_flag, 0); + + if (ph->ph_explicit_scaling_list_enabled_flag && + !current->sh_picture_header_in_slice_header_flag) + flag(sh_explicit_scaling_list_used_flag); + else + infer(sh_explicit_scaling_list_used_flag, 0); + + if (!pps->pps_rpl_info_in_ph_flag && + ((nal_unit_type != VVC_IDR_W_RADL && + nal_unit_type != VVC_IDR_N_LP) || + sps->sps_idr_rpl_present_flag)) { + CHECK(FUNC(ref_pic_lists)(ctx, rw, sps, pps, ¤t->sh_ref_pic_lists)); + ref_pic_lists = ¤t->sh_ref_pic_lists; + } else { + ref_pic_lists = &h266->priv.ph->ph_ref_pic_lists; + } + if ((current->sh_slice_type != VVC_SLICE_TYPE_I && + ref_pic_lists->rpl_ref_list[0].num_ref_entries > 1) || + (current->sh_slice_type == VVC_SLICE_TYPE_B && + ref_pic_lists->rpl_ref_list[1].num_ref_entries > 1)) { + flag(sh_num_ref_idx_active_override_flag); + if (current->sh_num_ref_idx_active_override_flag) { + for (i = 0; i < (current->sh_slice_type == VVC_SLICE_TYPE_B ? 2: 1); i++) + if (ref_pic_lists->rpl_ref_list[i].num_ref_entries > 1) + ues(sh_num_ref_idx_active_minus1[i], 0, 14, 1, i); + else + infer(sh_num_ref_idx_active_minus1[i], 0); + } + } else { + infer(sh_num_ref_idx_active_override_flag, 1); + } + if (current->sh_slice_type != VVC_SLICE_TYPE_I) { + if (pps->pps_cabac_init_present_flag) + flag(sh_cabac_init_flag); + else + infer(sh_cabac_init_flag, 0); + if (ph->ph_temporal_mvp_enabled_flag && !pps->pps_rpl_info_in_ph_flag) { + uint8_t num_ref_idx_active[2]; + for (i = 0; i < 2; i++) { + if (current->sh_slice_type == VVC_SLICE_TYPE_B || + (current->sh_slice_type == VVC_SLICE_TYPE_P && i == 0)) { + if (current->sh_num_ref_idx_active_override_flag) { + num_ref_idx_active[i] = current->sh_num_ref_idx_active_minus1[i] + 1; + } + else { + num_ref_idx_active[i] = + FFMIN(ref_pic_lists->rpl_ref_list[i].num_ref_entries, + pps->pps_num_ref_idx_default_active_minus1[i] + 1); + } + } else { + num_ref_idx_active[i] = 0; + } + } + + if (current->sh_slice_type == VVC_SLICE_TYPE_B) + flag(sh_collocated_from_l0_flag); + else + infer(sh_collocated_from_l0_flag, 1); + if ((current->sh_collocated_from_l0_flag && num_ref_idx_active[0] > 1) || + (!current->sh_collocated_from_l0_flag && num_ref_idx_active[1] > 1)) { + unsigned int idx = current->sh_collocated_from_l0_flag ? 0 : 1; + ue(sh_collocated_ref_idx, 0, num_ref_idx_active[idx] - 1); + } else { + infer(sh_collocated_ref_idx, 0); + } + if (!pps->pps_wp_info_in_ph_flag && + ((pps->pps_weighted_pred_flag && + current->sh_slice_type == VVC_SLICE_TYPE_P) || + (pps->pps_weighted_bipred_flag && + current->sh_slice_type == VVC_SLICE_TYPE_B))) { + CHECK(FUNC(pred_weight_table)(ctx, rw, sps, pps, ref_pic_lists, + num_ref_idx_active, + ¤t->sh_pred_weight_table)); + } + } + + } + qp_bd_offset = 6 * sps->sps_bitdepth_minus8; + if (!pps->pps_qp_delta_info_in_ph_flag) + se(sh_qp_delta, -qp_bd_offset - (26 + pps->pps_init_qp_minus26), + 63 - (26 + pps->pps_init_qp_minus26)); + if (pps->pps_slice_chroma_qp_offsets_present_flag) { + int8_t off; + + se(sh_cb_qp_offset, -12, 12); + off = pps->pps_cb_qp_offset + current->sh_cb_qp_offset; + if (off < -12 || off > 12) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "pps_cb_qp_offset + sh_cb_qp_offset (%d) not in range [-12, 12].\n", + off); + return AVERROR_INVALIDDATA; + } + + se(sh_cr_qp_offset, -12, 12); + off = pps->pps_cr_qp_offset + current->sh_cr_qp_offset; + if (off < -12 || off > 12) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "pps_cr_qp_offset + sh_cr_qp_offset (%d) not in range [-12, 12].\n", + off); + return AVERROR_INVALIDDATA; + } + + if (sps->sps_joint_cbcr_enabled_flag) { + se(sh_joint_cbcr_qp_offset, -12, 12); + off = pps->pps_joint_cbcr_qp_offset_value + current->sh_joint_cbcr_qp_offset; + if (off < -12 || off > 12) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "pps_joint_cbcr_qp_offset_value + sh_joint_cbcr_qp_offset (%d)" + "not in range [-12, 12]. \n", + off); + return AVERROR_INVALIDDATA; + } + } else { + infer(sh_joint_cbcr_qp_offset, 0); + } + } else { + infer(sh_cb_qp_offset, 0); + infer(sh_cr_qp_offset, 0); + infer(sh_joint_cbcr_qp_offset, 0); + } + if (pps->pps_cu_chroma_qp_offset_list_enabled_flag) + flag(sh_cu_chroma_qp_offset_enabled_flag); + else + infer(sh_cu_chroma_qp_offset_enabled_flag, 0); + if (sps->sps_sao_enabled_flag && !pps->pps_sao_info_in_ph_flag) { + flag(sh_sao_luma_used_flag); + if (sps->sps_chroma_format_idc != 0) + flag(sh_sao_chroma_used_flag); + else + infer(sh_sao_chroma_used_flag, ph->ph_sao_chroma_enabled_flag); + } else { + infer(sh_sao_luma_used_flag, ph->ph_sao_luma_enabled_flag); + infer(sh_sao_chroma_used_flag, ph->ph_sao_chroma_enabled_flag); + } + + if (pps->pps_deblocking_filter_override_enabled_flag && + !pps->pps_dbf_info_in_ph_flag) + flag(sh_deblocking_params_present_flag); + else + infer(sh_deblocking_params_present_flag, 0); + if (current->sh_deblocking_params_present_flag) { + if (!pps->pps_deblocking_filter_disabled_flag) + flag(sh_deblocking_filter_disabled_flag); + else + infer(sh_deblocking_filter_disabled_flag, + !(pps->pps_deblocking_filter_disabled_flag && + current->sh_deblocking_params_present_flag)); + if (!current->sh_deblocking_filter_disabled_flag) { + se(sh_luma_beta_offset_div2, -12, 12); + se(sh_luma_tc_offset_div2, -12, 12); + if (pps->pps_chroma_tool_offsets_present_flag) { + se(sh_cb_beta_offset_div2, -12, 12); + se(sh_cb_tc_offset_div2, -12, 12); + se(sh_cr_beta_offset_div2, -12, 12); + se(sh_cr_tc_offset_div2, -12, 12); + } else { + infer(sh_cb_beta_offset_div2, current->sh_luma_beta_offset_div2); + infer(sh_cb_tc_offset_div2, current->sh_luma_tc_offset_div2); + infer(sh_cr_beta_offset_div2, current->sh_luma_beta_offset_div2); + infer(sh_cr_tc_offset_div2, current->sh_luma_tc_offset_div2); + } + } else { + infer(sh_luma_beta_offset_div2, ph->ph_luma_beta_offset_div2); + infer(sh_luma_tc_offset_div2, ph->ph_luma_tc_offset_div2); + if (pps->pps_chroma_tool_offsets_present_flag) { + infer(sh_cb_beta_offset_div2, ph->ph_cb_beta_offset_div2); + infer(sh_cb_tc_offset_div2, ph->ph_cb_tc_offset_div2); + infer(sh_cr_beta_offset_div2, ph->ph_cr_beta_offset_div2); + infer(sh_cr_tc_offset_div2, ph->ph_cr_beta_offset_div2); + } else { + infer(sh_cb_beta_offset_div2, current->sh_luma_beta_offset_div2); + infer(sh_cb_tc_offset_div2, current->sh_luma_tc_offset_div2); + infer(sh_cr_beta_offset_div2, current->sh_luma_beta_offset_div2); + infer(sh_cr_tc_offset_div2, current->sh_luma_tc_offset_div2); + } + } + } + + if (sps->sps_dep_quant_enabled_flag) + flag(sh_dep_quant_used_flag); + else + infer(sh_dep_quant_used_flag, 0); + + if (sps->sps_sign_data_hiding_enabled_flag && + !current->sh_dep_quant_used_flag) + flag(sh_sign_data_hiding_used_flag); + else + infer(sh_sign_data_hiding_used_flag, 0); + + if (sps->sps_transform_skip_enabled_flag && + !current->sh_dep_quant_used_flag && + !current->sh_sign_data_hiding_used_flag) + flag(sh_ts_residual_coding_disabled_flag); + else + infer(sh_ts_residual_coding_disabled_flag, 0); + if (pps->pps_slice_header_extension_present_flag) { + ue(sh_slice_header_extension_length, 0, 256); + for (i = 0; i < current->sh_slice_header_extension_length; i++) + us(8, sh_slice_header_extension_data_byte[i], 0x00, 0xff, 1, i); + } + if (sps->sps_entry_point_offsets_present_flag) { + int num_entry_points = 0; + uint8_t entropy_sync = sps->sps_entropy_coding_sync_enabled_flag; + int height; + if (pps->pps_rect_slice_flag) { + int width_in_tiles; + int slice_idx = current->sh_slice_address; + for (i = 0; i < curr_subpic_idx; i++) { + slice_idx += pps->num_slices_in_subpic[i]; + } + width_in_tiles = + pps->pps_slice_width_in_tiles_minus1[slice_idx] + 1; + + if (entropy_sync) + height = pps->slice_height_in_ctus[slice_idx]; + else + height = pps->pps_slice_height_in_tiles_minus1[slice_idx] + 1; + + num_entry_points = width_in_tiles * height; + } else { + int tile_idx; + int tile_y; + for (tile_idx = current->sh_slice_address; + tile_idx <= current->sh_slice_address + current->sh_num_tiles_in_slice_minus1; + tile_idx++) { + tile_y = tile_idx / pps->num_tile_rows; + height = pps->pps_tile_row_height_minus1[tile_y] + 1; + num_entry_points += (entropy_sync ? height : 1); + } + } + num_entry_points--; + if (num_entry_points > VVC_MAX_ENTRY_POINTS) { + av_log(ctx->log_ctx, AV_LOG_ERROR, "Too many entry points: " + "%"PRIu16".\n", num_entry_points); + return AVERROR_PATCHWELCOME; + } + if (num_entry_points > 0) { + ue(sh_entry_offset_len_minus1, 0, 31); + for (i = 0; i < num_entry_points; i++) { + ubs(current->sh_entry_offset_len_minus1 + 1, + sh_entry_point_offset_minus1[i], 1, i); + } + } + } + CHECK(FUNC(byte_alignment)(ctx, rw)); + + return 0; +} + +static int FUNC(sei_decoded_picture_hash)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawSEIDecodedPictureHash *current) +{ + int err, c_idx, i; + + HEADER("Decoded Picture Hash"); + + u(8, dph_sei_hash_type, 0, 2); + flag(dph_sei_single_component_flag); + ub(7, dph_sei_reserved_zero_7bits); + + for (c_idx = 0; c_idx < (current->dph_sei_single_component_flag ? 1 : 3); c_idx++) { + if (current->dph_sei_hash_type == 0) { + for (i = 0; i < 16; i++) + us(8, dph_sei_picture_md5[c_idx][i], 0x00, 0xff, 2, c_idx, i); + } else if (current->dph_sei_hash_type == 1) { + us(16, dph_sei_picture_crc[c_idx], 0x0000, 0xffff, 1, c_idx); + } else if (current->dph_sei_hash_type == 2) { + us(32, dph_sei_picture_checksum[c_idx], 0x00000000, 0xffffffff, 1, c_idx); + } + } + return 0; +} + +static int FUNC(sei)(CodedBitstreamContext *ctx, RWContext *rw, + H266RawSEI *current, int prefix) +{ + int err; + + if (prefix) + HEADER("Prefix Supplemental Enhancement Information"); + else + HEADER("Suffix Supplemental Enhancement Information"); + + CHECK(FUNC(nal_unit_header)(ctx, rw, ¤t->nal_unit_header, + prefix ? VVC_PREFIX_SEI_NUT + : VVC_SUFFIX_SEI_NUT)); + + CHECK(FUNC_SEI(message_list)(ctx, rw, ¤t->message_list, prefix)); + + CHECK(FUNC(rbsp_trailing_bits)(ctx, rw)); + + return 0; +} diff --git a/libavcodec/cbs_internal.h b/libavcodec/cbs_internal.h index e585c77934..2de5c243b6 100644 --- a/libavcodec/cbs_internal.h +++ b/libavcodec/cbs_internal.h @@ -245,6 +245,7 @@ int ff_cbs_write_signed(CodedBitstreamContext *ctx, PutBitContext *pbc, extern const CodedBitstreamType ff_cbs_type_av1; extern const CodedBitstreamType ff_cbs_type_h264; extern const CodedBitstreamType ff_cbs_type_h265; +extern const CodedBitstreamType ff_cbs_type_h266; extern const CodedBitstreamType ff_cbs_type_jpeg; extern const CodedBitstreamType ff_cbs_type_mpeg2; extern const CodedBitstreamType ff_cbs_type_vp9; diff --git a/libavcodec/cbs_sei.c b/libavcodec/cbs_sei.c index 50a513f592..bd7f6f4938 100644 --- a/libavcodec/cbs_sei.c +++ b/libavcodec/cbs_sei.c @@ -20,6 +20,7 @@ #include "cbs_internal.h" #include "cbs_h264.h" #include "cbs_h265.h" +#include "cbs_h266.h" #include "cbs_sei.h" static void cbs_free_user_data_registered(void *opaque, uint8_t *data) @@ -132,6 +133,13 @@ static int cbs_sei_get_unit(CodedBitstreamContext *ctx, else sei_type = HEVC_NAL_SEI_SUFFIX; break; + case AV_CODEC_ID_H266: + highest_vcl_type = VVC_RSV_IRAP_11; + if (prefix) + sei_type = VVC_PREFIX_SEI_NUT; + else + sei_type = VVC_SUFFIX_SEI_NUT; + break; default: return AVERROR(EINVAL); } @@ -207,6 +215,18 @@ static int cbs_sei_get_unit(CodedBitstreamContext *ctx, memcpy(unit->content, &sei, sizeof(sei)); } break; + case AV_CODEC_ID_H266: + { + H266RawSEI sei = { + .nal_unit_header = { + .nal_unit_type = sei_type, + .nuh_layer_id = 0, + .nuh_temporal_id_plus1 = 1, + }, + }; + memcpy(unit->content, &sei, sizeof(sei)); + } + break; default: av_assert0(0); } @@ -237,6 +257,15 @@ static int cbs_sei_get_message_list(CodedBitstreamContext *ctx, *list = &sei->message_list; } break; + case AV_CODEC_ID_H266: + { + H266RawSEI *sei = unit->content; + if (unit->type != VVC_PREFIX_SEI_NUT && + unit->type != VVC_SUFFIX_SEI_NUT) + return AVERROR(EINVAL); + *list = &sei->message_list; + } + break; default: return AVERROR(EINVAL); } From patchwork Mon Oct 24 14:06:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Siedel X-Patchwork-Id: 38960 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2249233pzb; Mon, 24 Oct 2022 07:08:33 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5gDf5SA8WZegf7ZxgUv39q+CK1AUvmq00SHLxeOi/AZCcV4lUCItdosODYdviCBX7VHEZW X-Received: by 2002:a05:6402:3588:b0:45d:7d14:baf2 with SMTP id y8-20020a056402358800b0045d7d14baf2mr30509176edc.1.1666620511999; Mon, 24 Oct 2022 07:08:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666620511; cv=none; d=google.com; s=arc-20160816; b=FCY/P7/EYGOdKdmlR6ekUunYSHPdxkLR7/CiHqszX03byqrF49ARzaQXdX3XEAYIo4 Qe2ZmM3vAwBRpQPQrIRk7aOSJJZ8vumG31hduozHAs92sGQaILE/dgk3s+qw+kNXN6QE e19+hKC7XLhKgNARIMK7FbL25O1znAtQGYIx+y3doEjwL/dGVimWja2ZcRxzMVj4etEL bfbQtCthJ00q1W+BQ3Qkkrfg17sWuJyJN224+CfED6/7rYBWkHtPqmADGEWAHCs46JAR c+XsnR68LKWdIQ+4vKEo3zTAvDR8o6u9QNMZO5cawq15V1zdI6GZ/w/TGTfjNEex5hkL 2W5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=qDO6s5ohJ9HyRRldlGV3pPslH1waBp9OGpufWTIvX8E=; b=MZGfbxNr1SMH8F1On1i7ZH1kvj6/5DjTxgTHxgly5rRo64p1XjeBq09cK2UNO/k7hg GPQMfYWr9+KN6Sf2Y8hNw/bLVxApal/h3ppuLU2p1CpVkWLBWQr2ny7F5S19PJxQlexk XFn9i2kwwwbDlsGNucUW25aDMraeSrGG/g2bhrPpey0g6GzwOJp6uRlhhar5ScGqOCVj 3ZiZxLhbP04eUCuYnEB/Mj/G19d6xoXu4pTO5UWQkf44iogb4E/+/alBAxN9HKEiHswg yoVsLupNAu2DsrKzbj3r5bPfwCsl+C7zfHdeneJWXIs81dFiCfrYNAu/LAjplSued7tY 7owg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=pFz1RvK5; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id i10-20020a1709064fca00b0078db1258ae9si32910204ejw.555.2022.10.24.07.08.07; Mon, 24 Oct 2022 07:08:31 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=pFz1RvK5; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2553C68BC0F; Mon, 24 Oct 2022 17:07:46 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f42.google.com (mail-ed1-f42.google.com [209.85.208.42]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1665768BAA4 for ; Mon, 24 Oct 2022 17:07:39 +0300 (EEST) Received: by mail-ed1-f42.google.com with SMTP id m15so31676324edb.13 for ; Mon, 24 Oct 2022 07:07:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spin-digital-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=oM8WV0VdUQid3EsjyuCgxIPSaFt9/RHYCYhL9qyWVgE=; b=pFz1RvK598Og+8e3jL5EgSHIZHhnI0BncADhf3s2PNC422fsZma+DFb2JSOZW8i2Pj WQx1ilU49yaK0X0eHcjyHMHaY9MPBTqx0mfCMkuCyi96YmMh44QBGI+7ylHdNs9lgC0w tovW8OKRXsvJ6bNUDGjh9tnUlKNYs8ZekTSAMBwPdUMm6c/qqOff8pfseBHd+gov4Arq hwC+kj1/MvYlZRrzR2NC9gqpbxaTv5sHUzvueQcjUsEwMBJEuTqtiJ9ru+Vh330PZk54 JbjokMntdb+jFo1yEKrfTKwHuWXVDOVd6Qb3Prb5U3z22gMyXXJRaFnejFKCXdeThZ52 jhXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oM8WV0VdUQid3EsjyuCgxIPSaFt9/RHYCYhL9qyWVgE=; b=14kJnh0jsbyHWP+lJN25Rf615RTwT8hNMe/ih+MNJO3Fb/l4c2RGygh0NSv/wUI/Qt aM7TUfuPZT1DjbV1oKUVXje1RFPXhKvom/5Gcz3vl23g4D/bwxTIClE653no7i6rGBaS iiJugCIBVsa8CPZbG0kWyO47wGEAFiTG1ITtnuY6U9AoyGVRWibLC0iZGtyRTshttn2F VoQXzH0kwHRslK0YkHS3SpWp7kdmrAwNoFpoarTNHxbxAV9n65lxcubkSvM2b1i1R1Lk HoHT1mKKzBMaSxL7mYRQHFzaUTt+pJJcpwD241EAwYOCQ+ADiYlI89G4JS0zSBJCds4r +cJg== X-Gm-Message-State: ACrzQf1UqDKAEkB2vfWjBxXdWh2HRngqxqN2s0YaqvBWR0dbSMNCFT5t pltuX0fS8B0zY5Nq3GTdw50UAKEH2oMzKA== X-Received: by 2002:a05:6402:50d4:b0:461:e349:56b2 with SMTP id h20-20020a05640250d400b00461e34956b2mr2659058edb.17.1666620457689; Mon, 24 Oct 2022 07:07:37 -0700 (PDT) Received: from thomas-win.localdomain ([213.138.44.237]) by smtp.gmail.com with ESMTPSA id s19-20020a170906455300b0079800b81709sm6056436ejq.219.2022.10.24.07.07.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Oct 2022 07:07:37 -0700 (PDT) From: Thomas Siedel To: ffmpeg-devel@ffmpeg.org Date: Mon, 24 Oct 2022 16:06:38 +0200 Message-Id: <20221024140645.4945-4-thomas.ff@spin-digital.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221024140645.4945-1-thomas.ff@spin-digital.com> References: <20221024140645.4945-1-thomas.ff@spin-digital.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 03/10] avcodec: add bitstream parser for H266/VVC X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Thomas Siedel Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 3GRgiZzO1xoy Add nal parser ff_vvc_parser to parse vvc elementary bitstreams. Signed-off-by: Thomas Siedel --- configure | 3 + libavcodec/Makefile | 2 + libavcodec/h2645_parse.c | 71 ++++- libavcodec/parsers.c | 1 + libavcodec/vvc_parser.c | 588 +++++++++++++++++++++++++++++++++++++++ 5 files changed, 664 insertions(+), 1 deletion(-) create mode 100644 libavcodec/vvc_parser.c diff --git a/configure b/configure index c5a466657f..691e8da6d0 100755 --- a/configure +++ b/configure @@ -2447,6 +2447,7 @@ CONFIG_EXTRA=" cbs_av1 cbs_h264 cbs_h265 + cbs_h266 cbs_jpeg cbs_mpeg2 cbs_vp9 @@ -2728,6 +2729,7 @@ threads_if_any="$THREADS_LIST" cbs_av1_select="cbs" cbs_h264_select="cbs" cbs_h265_select="cbs" +cbs_h266_select="cbs" cbs_jpeg_select="cbs" cbs_mpeg2_select="cbs" cbs_vp9_select="cbs" @@ -3282,6 +3284,7 @@ hevc_parser_select="hevcparse" mpegaudio_parser_select="mpegaudioheader" mpeg4video_parser_select="h263dsp mpegvideodec qpeldsp" vc1_parser_select="vc1dsp" +vvc_parser_select="cbs_h266" # bitstream_filters aac_adtstoasc_bsf_select="adts_header mpeg4audio" diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 7761620de7..092f714d67 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -76,6 +76,7 @@ OBJS-$(CONFIG_CBS) += cbs.o cbs_bsf.o OBJS-$(CONFIG_CBS_AV1) += cbs_av1.o OBJS-$(CONFIG_CBS_H264) += cbs_h2645.o cbs_sei.o h2645_parse.o OBJS-$(CONFIG_CBS_H265) += cbs_h2645.o cbs_sei.o h2645_parse.o +OBJS-$(CONFIG_CBS_H266) += cbs_h2645.o cbs_sei.o h2645_parse.o OBJS-$(CONFIG_CBS_JPEG) += cbs_jpeg.o OBJS-$(CONFIG_CBS_MPEG2) += cbs_mpeg2.o OBJS-$(CONFIG_CBS_VP9) += cbs_vp9.o @@ -1174,6 +1175,7 @@ OBJS-$(CONFIG_VC1_PARSER) += vc1_parser.o vc1.o vc1data.o \ OBJS-$(CONFIG_VP3_PARSER) += vp3_parser.o OBJS-$(CONFIG_VP8_PARSER) += vp8_parser.o OBJS-$(CONFIG_VP9_PARSER) += vp9_parser.o +OBJS-$(CONFIG_VVC_PARSER) += vvc_parser.o OBJS-$(CONFIG_WEBP_PARSER) += webp_parser.o OBJS-$(CONFIG_XBM_PARSER) += xbm_parser.o OBJS-$(CONFIG_XMA_PARSER) += xma_parser.o diff --git a/libavcodec/h2645_parse.c b/libavcodec/h2645_parse.c index 90944177c7..9fbeee3edd 100644 --- a/libavcodec/h2645_parse.c +++ b/libavcodec/h2645_parse.c @@ -27,6 +27,7 @@ #include "libavutil/mem.h" #include "bytestream.h" +#include "vvc.h" #include "hevc.h" #include "h264.h" #include "h2645_parse.h" @@ -145,6 +146,47 @@ nsc: return si; } +static const char *const vvc_nal_type_name[32] = { + "TRAIL_NUT", // VVC_TRAIL_NUT + "STSA_NUT", // VVC_STSA_NUT + "RADL_NUT", // VVC_RADL_NUT + "RASL_NUT", // VVC_RASL_NUT + "RSV_VCL_4", // VVC_RSV_VCL_4 + "RSV_VCL_5", // VVC_RSV_VCL_5 + "RSV_VCL_6", // VVC_RSV_VCL_6 + "IDR_W_RADL", // VVC_IDR_W_RADL + "IDR_N_LP", // VVC_IDR_N_LP + "CRA_NUT", // VVC_CRA_NUT + "GDR_NUT", // VVC_GDR_NUT + "RSV_IRAP_11", // VVC_RSV_IRAP_11 + "OPI_NUT", // VVC_OPI_NUT + "DCI_NUT", // VVC_DCI_NUT + "VPS_NUT", // VVC_VPS_NUT + "SPS_NUT", // VVC_SPS_NUT + "PPS_NUT", // VVC_PPS_NUT + "PREFIX_APS_NUT",// VVC_PREFIX_APS_NUT + "SUFFIX_APS_NUT",// VVC_SUFFIX_APS_NUT + "PH_NUT", // VVC_PH_NUT + "AUD_NUT", // VVC_AUD_NUT + "EOS_NUT", // VVC_EOS_NUT + "EOB_NUT", // VVC_EOB_NUT + "PREFIX_SEI_NUT",// VVC_PREFIX_SEI_NUT + "SUFFIX_SEI_NUT",// VVC_SUFFIX_SEI_NUT + "FD_NUT", // VVC_FD_NUT + "RSV_NVCL_26", // VVC_RSV_NVCL_26 + "RSV_NVCL_27", // VVC_RSV_NVCL_27 + "UNSPEC_28", // VVC_UNSPEC_28 + "UNSPEC_29", // VVC_UNSPEC_29 + "UNSPEC_30", // VVC_UNSPEC_30 + "UNSPEC_31", // VVC_UNSPEC_31 +}; + +static const char *vvc_nal_unit_name(int nal_type) +{ + av_assert0(nal_type >= 0 && nal_type < 32); + return vvc_nal_type_name[nal_type]; +} + static const char *const hevc_nal_type_name[64] = { "TRAIL_N", // HEVC_NAL_TRAIL_N "TRAIL_R", // HEVC_NAL_TRAIL_R @@ -293,6 +335,31 @@ static int get_bit_length(H2645NAL *nal, int min_size, int skip_trailing_zeros) * @return AVERROR_INVALIDDATA if the packet is not a valid NAL unit, * 0 otherwise */ +static int vvc_parse_nal_header(H2645NAL *nal, void *logctx) +{ + GetBitContext *gb = &nal->gb; + + if (get_bits1(gb) != 0) //forbidden_zero_bit + return AVERROR_INVALIDDATA; + + skip_bits1(gb); //nuh_reserved_zero_bit + + nal->nuh_layer_id = get_bits(gb, 6); + nal->type = get_bits(gb, 5); + nal->temporal_id = get_bits(gb, 3) - 1; + if (nal->temporal_id < 0) + return AVERROR_INVALIDDATA; + + if ((nal->type >= VVC_IDR_W_RADL && nal->type <= VVC_RSV_IRAP_11) && nal->temporal_id) + return AVERROR_INVALIDDATA; + + av_log(logctx, AV_LOG_DEBUG, + "nal_unit_type: %d(%s), nuh_layer_id: %d, temporal_id: %d\n", + nal->type, vvc_nal_unit_name(nal->type), nal->nuh_layer_id, nal->temporal_id); + + return 0; +} + static int hevc_parse_nal_header(H2645NAL *nal, void *logctx) { GetBitContext *gb = &nal->gb; @@ -509,7 +576,9 @@ int ff_h2645_packet_split(H2645Packet *pkt, const uint8_t *buf, int length, /* Reset type in case it contains a stale value from a previously parsed NAL */ nal->type = 0; - if (codec_id == AV_CODEC_ID_HEVC) + if (codec_id == AV_CODEC_ID_VVC) + ret = vvc_parse_nal_header(nal, logctx); + else if (codec_id == AV_CODEC_ID_HEVC) ret = hevc_parse_nal_header(nal, logctx); else ret = h264_parse_nal_header(nal, logctx); diff --git a/libavcodec/parsers.c b/libavcodec/parsers.c index d355808018..e5b01b7a59 100644 --- a/libavcodec/parsers.c +++ b/libavcodec/parsers.c @@ -74,6 +74,7 @@ extern const AVCodecParser ff_vorbis_parser; extern const AVCodecParser ff_vp3_parser; extern const AVCodecParser ff_vp8_parser; extern const AVCodecParser ff_vp9_parser; +extern const AVCodecParser ff_vvc_parser; extern const AVCodecParser ff_webp_parser; extern const AVCodecParser ff_xbm_parser; extern const AVCodecParser ff_xma_parser; diff --git a/libavcodec/vvc_parser.c b/libavcodec/vvc_parser.c new file mode 100644 index 0000000000..f7eb2340c5 --- /dev/null +++ b/libavcodec/vvc_parser.c @@ -0,0 +1,588 @@ +/* + * VVC parser + * + * Copyright (C) 2021 Nuo Mi + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "cbs.h" +#include "cbs_h266.h" +#include "internal.h" +#include "parser.h" +#include "decode.h" + +#define START_CODE 0x000001 ///< start_code_prefix_one_3bytes + +#define IS_SLICE(nut) (nut <= VVC_RASL_NUT || (nut >= VVC_IDR_W_RADL && nut <= VVC_GDR_NUT)) +#define IS_PH(nut) (nut == VVC_PH_NUT) +#define IS_IDR(nut) (nut == VVC_IDR_W_RADL || nut == VVC_IDR_N_LP) + + +typedef struct PuInfo { + AVBufferRef *sps_ref; + AVBufferRef *pps_ref; + AVBufferRef *slice_ref; + AVBufferRef *ph_ref; + + const H266RawPPS *pps; + const H266RawSPS *sps; + const H266RawPH *ph; + const H266RawSlice *slice; + int pic_type; +} PuInfo; + +typedef struct AuDetector { + uint8_t prev_layer_id; + int prev_tid0_poc; + int prev_poc; +} AuDetector; + +typedef struct VVCParserContext { + ParseContext pc; + CodedBitstreamContext *cbc; + + CodedBitstreamFragment picture_unit; + + PuInfo au_info; + AVPacket au; + AVPacket last_au; + + AuDetector au_detector; + + int parsed_extradata; +} VVCParserContext; + +static const enum AVPixelFormat pix_fmts_8bit[] = { + AV_PIX_FMT_GRAY8, AV_PIX_FMT_YUV420P, + AV_PIX_FMT_YUV422P, AV_PIX_FMT_YUV444P +}; + +static const enum AVPixelFormat pix_fmts_10bit[] = { + AV_PIX_FMT_GRAY10, AV_PIX_FMT_YUV420P10, + AV_PIX_FMT_YUV422P10, AV_PIX_FMT_YUV444P10 +}; + +static int get_format(const H266RawSPS *sps) +{ + switch (sps->sps_bitdepth_minus8) { + case 0: + return pix_fmts_8bit[sps->sps_chroma_format_idc]; + case 2: + return pix_fmts_10bit[sps->sps_chroma_format_idc]; + } + return AV_PIX_FMT_NONE; +} + +/** + * Find the end of the current frame in the bitstream. + * @return the position of the first byte of the next frame, or END_NOT_FOUND + */ +static int find_frame_end(AVCodecParserContext *s, const uint8_t *buf, + int buf_size) +{ + VVCParserContext *ctx = s->priv_data; + ParseContext *pc = &ctx->pc; + int i; + + for (i = 0; i < buf_size; i++) { + int nut; + + pc->state64 = (pc->state64 << 8) | buf[i]; + + if (((pc->state64 >> 3 * 8) & 0xFFFFFF) != START_CODE) + continue; + + nut = (pc->state64 >> (8 + 3)) & 0x1F; + // 7.4.2.4.3 and 7.4.2.4.4 + if ((nut >= VVC_OPI_NUT && nut <= VVC_PREFIX_APS_NUT && nut != VVC_PH_NUT) || + nut == VVC_AUD_NUT || (nut == VVC_PREFIX_SEI_NUT && !pc->frame_start_found) || nut == VVC_RSV_NVCL_26 || + nut == VVC_UNSPEC_28 || nut == VVC_UNSPEC_29) { + if (pc->frame_start_found) { + pc->frame_start_found = 0; + return i - 5; + } + } else if (nut == VVC_PH_NUT || IS_SLICE(nut)) { + int sh_picture_header_in_slice_header_flag = buf[i] >> 7; + + if (nut == VVC_PH_NUT || sh_picture_header_in_slice_header_flag) { + if (!pc->frame_start_found) { + pc->frame_start_found = 1; + } else { // First slice of next frame found + pc->frame_start_found = 0; + return i - 5; + } + } + } + } + return END_NOT_FOUND; +} + +static int get_pict_type(const CodedBitstreamFragment *pu) +{ + int has_p = 0; + for (int i = 0; i < pu->nb_units; i++) { + CodedBitstreamUnit *unit = &pu->units[i]; + if (IS_SLICE(unit->type)) { + const H266RawSlice *slice = unit->content; + uint8_t type = slice->header.sh_slice_type; + if (type == VVC_SLICE_TYPE_B) { + return AV_PICTURE_TYPE_B; + } + if (type == VVC_SLICE_TYPE_P) { + has_p = 1; + } + } + } + return has_p ? AV_PICTURE_TYPE_P : AV_PICTURE_TYPE_I; +} + +static void pu_info_unref(PuInfo *info) +{ + av_buffer_unref(&info->slice_ref); + av_buffer_unref(&info->ph_ref); + av_buffer_unref(&info->pps_ref); + av_buffer_unref(&info->sps_ref); + info->slice = NULL; + info->ph = NULL; + info->pps = NULL; + info->sps = NULL; + info->pic_type = AV_PICTURE_TYPE_NONE; +} + +static int pu_info_ref(PuInfo *dest, const PuInfo *src) +{ + pu_info_unref(dest); + dest->sps_ref = av_buffer_ref(src->sps_ref); + dest->pps_ref = av_buffer_ref(src->pps_ref); + if (src->ph_ref) + dest->ph_ref = av_buffer_ref(src->ph_ref); + dest->slice_ref = av_buffer_ref(src->slice_ref); + if (!dest->sps_ref || !dest->pps_ref || (src->ph_ref && !dest->ph_ref) || !dest->slice_ref) { + pu_info_unref(dest); + return AVERROR(ENOMEM); + } + + dest->sps = src->sps; + dest->pps = src->pps; + dest->ph = src->ph; + dest->slice = src->slice; + dest->pic_type = src->pic_type; + return 0; +} + +static int set_parser_ctx(AVCodecParserContext *ctx, AVCodecContext *avctx, + const PuInfo *pu) +{ + int ret, num = 0, den = 0; + static const uint8_t h266_sub_width_c[] = { + 1, 2, 2, 1 + }; + static const uint8_t h266_sub_height_c[] = { + 1, 2, 1, 1 + }; + const H266RawSPS *sps = pu->sps; + const H266RawPPS *pps = pu->pps; + //const H266RawPH *ph = pu->ph; + const H266RawNALUnitHeader *nal = &pu->slice->header.nal_unit_header; + + /* set some sane default values */ + ctx->pict_type = AV_PICTURE_TYPE_I; + ctx->key_frame = 0; + ctx->picture_structure = AV_PICTURE_STRUCTURE_FRAME; + + ctx->key_frame = nal->nal_unit_type == VVC_IDR_W_RADL || + nal->nal_unit_type == VVC_IDR_N_LP || + nal->nal_unit_type == VVC_CRA_NUT || + nal->nal_unit_type == VVC_GDR_NUT; + + ctx->coded_width = pps->pps_pic_width_in_luma_samples; + ctx->coded_height = pps->pps_pic_height_in_luma_samples; + ctx->width = pps->pps_pic_width_in_luma_samples - + (pps->pps_conf_win_left_offset + pps->pps_conf_win_right_offset) * + h266_sub_width_c[sps->sps_chroma_format_idc]; + ctx->height = pps->pps_pic_height_in_luma_samples - + (pps->pps_conf_win_top_offset + pps->pps_conf_win_bottom_offset) * + h266_sub_height_c[sps->sps_chroma_format_idc];; + ctx->pict_type = pu->pic_type; + ctx->format = get_format(sps); + + avctx->profile = sps->profile_tier_level.general_profile_idc; + avctx->level = sps->profile_tier_level.general_level_idc; + + avctx->colorspace = (enum AVColorSpace) sps->vui.vui_matrix_coeffs; + avctx->color_primaries = (enum AVColorPrimaries) sps->vui.vui_colour_primaries; + avctx->color_trc = (enum AVColorTransferCharacteristic) sps->vui.vui_transfer_characteristics; + avctx->color_range = sps->vui.vui_full_range_flag ? AVCOL_RANGE_JPEG : AVCOL_RANGE_MPEG; + + if (ctx->width != avctx->width || ctx->height != avctx->height) { + ret = ff_set_dimensions(avctx, ctx->width, ctx->height); + if (ret < 0) + return ret; + } + avctx->pix_fmt = ctx->format; + avctx->has_b_frames = (sps->sps_max_sublayers_minus1+1) > 2 ? 2 : sps->sps_max_sublayers_minus1; + avctx->max_b_frames = sps->sps_max_sublayers_minus1; + + if(sps->sps_ptl_dpb_hrd_params_present_flag && sps->sps_timing_hrd_params_present_flag) { + num = sps->sps_general_timing_hrd_parameters.num_units_in_tick; + den = sps->sps_general_timing_hrd_parameters.time_scale; + } else { + return 1; + } + if (num != 0 && den != 0) + av_reduce(&avctx->framerate.den, &avctx->framerate.num, + num, den, 1 << 30); + + if (avctx->framerate.num) + avctx->time_base = av_inv_q(av_mul_q(avctx->framerate, (AVRational){avctx->ticks_per_frame, 1})); + + return 1; +} + +static int set_ctx(AVCodecParserContext *ctx, AVCodecContext *avctx, const PuInfo *next_pu) +{ + VVCParserContext *s = ctx->priv_data; + int ret; + if (s->au_info.slice) { + if ((ret = set_parser_ctx(ctx, avctx, &s->au_info)) < 0) + return ret; + } + ret = pu_info_ref(&s->au_info, next_pu); + return ret; +} + +//8.3.1 Decoding process for picture order count. +//VTM did not follow the spec, and it's much simpler than spec. +//We follow the VTM. +static void get_slice_poc(VVCParserContext *s, int *poc, + const H266RawSPS *sps, + const H266RawPH *ph, const H266RawSliceHeader *slice, + void *log_ctx) +{ + int poc_msb, max_poc_lsb, poc_lsb; + AuDetector *d = &s->au_detector; + max_poc_lsb = 1 << (sps->sps_log2_max_pic_order_cnt_lsb_minus4 + 4); + poc_lsb = ph->ph_pic_order_cnt_lsb; + if (IS_IDR(slice->nal_unit_header.nal_unit_type)) { + if (ph->ph_poc_msb_cycle_present_flag) + poc_msb = ph->ph_poc_msb_cycle_val * max_poc_lsb; + else + poc_msb = 0; + } else { + int prev_poc = d->prev_tid0_poc; + int prev_poc_lsb = prev_poc & (max_poc_lsb - 1); + int prev_poc_msb = prev_poc - prev_poc_lsb; + if (ph->ph_poc_msb_cycle_present_flag) { + poc_msb = ph->ph_poc_msb_cycle_val * max_poc_lsb; + } else { + if ((poc_lsb < prev_poc_lsb) && ((prev_poc_lsb - poc_lsb) >= (max_poc_lsb / 2))) + poc_msb = prev_poc_msb + max_poc_lsb; + else if ((poc_lsb > prev_poc_lsb) && ((poc_lsb - prev_poc_lsb) > (max_poc_lsb / 2))) + poc_msb = prev_poc_msb - max_poc_lsb; + else + poc_msb = prev_poc_msb; + } + } + + *poc = poc_msb + poc_lsb; +} + +static void au_detector_init(AuDetector *d) +{ + d->prev_layer_id = UINT8_MAX; + d->prev_poc = INT_MAX; + d->prev_tid0_poc = INT_MAX; +} + +static int is_au_start(VVCParserContext *s, const PuInfo *pu, void *log_ctx) +{ + //7.4.2.4.3 + AuDetector *d = &s->au_detector; + const H266RawSPS *sps = pu->sps; + const H266RawNALUnitHeader *nal = &pu->slice->header.nal_unit_header; + const H266RawPH *ph = pu->ph; + const H266RawSlice *slice = pu->slice; + int ret, poc, nut; + + get_slice_poc(s, &poc, sps, ph, &slice->header, log_ctx); + + ret = (nal->nuh_layer_id <= d->prev_layer_id) || (poc != d->prev_poc); + + nut = nal->nal_unit_type; + d->prev_layer_id = nal->nuh_layer_id; + d->prev_poc = poc; + if (nal->nuh_temporal_id_plus1 == 1 && + !ph->ph_non_ref_pic_flag && nut != VVC_RADL_NUT && nut != VVC_RASL_NUT) { + d->prev_tid0_poc = poc; + } + return ret; +} + +static int get_pu_info(PuInfo *info, const CodedBitstreamH266Context *h266, + const CodedBitstreamFragment *pu, void *logctx) +{ + const H266RawNALUnitHeader *nal; + int ret; + + memset(info, 0, sizeof(*info)); + for (int i = 0; i < pu->nb_units; i++) { + nal = pu->units[i].content; + if (!nal) + continue; + if (IS_PH(nal->nal_unit_type)) { + info->ph = pu->units[i].content; + info->ph_ref = pu->units[i].content_ref; + } else if (IS_SLICE(nal->nal_unit_type)) { + info->slice = pu->units[i].content; + info->slice_ref = pu->units[i].content_ref; + if (info->slice->header.sh_picture_header_in_slice_header_flag) + info->ph = &info->slice->header.sh_picture_header; + if (!info->ph) { + av_log(logctx, AV_LOG_ERROR, + "can't find picture header in picture unit.\n"); + ret = AVERROR_INVALIDDATA; + goto error; + } + break; + } + } + if (!info->slice) { + av_log(logctx, AV_LOG_ERROR, + "can't find slice in picture unit.\n"); + ret = AVERROR_INVALIDDATA; + goto error; + } + info->pps = h266->pps[info->ph->ph_pic_parameter_set_id]; + if (!info->pps) { + av_log(logctx, AV_LOG_ERROR, "PPS id %d is not avaliable.\n", + info->ph->ph_pic_parameter_set_id); + ret = AVERROR_INVALIDDATA; + goto error; + } + info->pps_ref = h266->pps_ref[info->ph->ph_pic_parameter_set_id]; + info->sps = h266->sps[info->pps->pps_seq_parameter_set_id]; + if (!info->sps) { + av_log(logctx, AV_LOG_ERROR, "SPS id %d is not avaliable.\n", + info->pps->pps_seq_parameter_set_id); + ret = AVERROR_INVALIDDATA; + goto error; + } + info->sps_ref = h266->sps_ref[info->pps->pps_seq_parameter_set_id]; + info->pic_type = get_pict_type(pu); + return 0; +error: + memset(info, 0, sizeof(*info)); + return ret; +} + +static int append_au(AVPacket *pkt, const uint8_t *buf, int buf_size) +{ + int offset = pkt->size; + int ret; + if ((ret = av_grow_packet(pkt, buf_size)) < 0) + goto end; + memcpy(pkt->data + offset, buf, buf_size); +end: + return ret; +} + +/** + * Parse NAL units of found picture and decode some basic information. + * + * @param ctx parser context. + * @param avctx codec context. + * @param buf buffer with field/frame data. + * @param buf_size size of the buffer. + * @return < 0 for error, == 0 for a complete au, > 0 is not a completed au. + */ +static int parse_nal_units(AVCodecParserContext *ctx, const uint8_t *buf, + int buf_size, AVCodecContext *avctx) +{ + VVCParserContext *s = ctx->priv_data; + const CodedBitstreamH266Context *h266 = s->cbc->priv_data; + + CodedBitstreamFragment *pu = &s->picture_unit; + int ret; + PuInfo info; + + if (!buf_size) { + if (s->au.size) { + if ((ret = av_packet_ref(&s->last_au, &s->au)) < 0) + goto end; + av_packet_unref(&s->au); + return 0; + } + return 1; + } + + if ((ret = ff_cbs_read(s->cbc, pu, buf, buf_size))< 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to parse picture unit.\n"); + goto end; + } + if ((ret = get_pu_info(&info, h266, pu, avctx)) < 0) + goto end; + if (is_au_start(s, &info, avctx)) { + if ((ret = set_ctx(ctx, avctx, &info)) < 0) + goto end; + if ((ret = av_packet_ref(&s->last_au, &s->au)) < 0) + goto end; + av_packet_unref(&s->au); + } else { + //not a completed au + ret = 1; + } + if (append_au(&s->au, buf, buf_size) < 0) + ret = AVERROR(ENOMEM); +end: + ff_cbs_fragment_reset(pu); + return ret; +} + +/** + * Combine PU to AU + * + * @param ctx parser context. + * @param avctx codec context. + * @param buf buffer to a PU. + * @param buf_size size of the buffer. + * @return < 0 for error, == 0 a complete au, > 0 not a completed au. + */ + +static int combine_au(AVCodecParserContext *ctx, AVCodecContext *avctx, + const uint8_t **buf, int *buf_size) +{ + VVCParserContext *s = ctx->priv_data; + CodedBitstreamFragment *pu = &s->picture_unit; + int ret; + + s->cbc->log_ctx = avctx; + + if (avctx->extradata_size && !s->parsed_extradata) { + s->parsed_extradata = 1; + + if ((ret = ff_cbs_read(s->cbc, pu, avctx->extradata, avctx->extradata_size)) < 0) + av_log(avctx, AV_LOG_WARNING, "Failed to parse extradata.\n"); + + ff_cbs_fragment_reset(pu); + } + av_packet_unref(&s->last_au); + ret = parse_nal_units(ctx, *buf, *buf_size, avctx); + if (ret == 0) { + if (s->last_au.size) { + *buf = s->last_au.data; + *buf_size = s->last_au.size; + } else { + //no output + ret = 1; + } + } + s->cbc->log_ctx = NULL; + return ret; +} + +static int vvc_parser_parse(AVCodecParserContext *ctx, AVCodecContext *avctx, + const uint8_t **poutbuf, int *poutbuf_size, + const uint8_t *buf, int buf_size) +{ + int next; + VVCParserContext *s = ctx->priv_data; + ParseContext *pc = &s->pc; + + if (avctx->extradata && !s->parsed_extradata) { + av_log(avctx, AV_LOG_INFO, "extra data is not supported yet.\n"); + return AVERROR_PATCHWELCOME; + } + + if (ctx->flags & PARSER_FLAG_COMPLETE_FRAMES) { + next = buf_size; + } else { + int ret, flush = !buf_size; + next = find_frame_end(ctx, buf, buf_size); + if (ff_combine_frame(pc, next, &buf, &buf_size) < 0) + goto no_out; + ret = combine_au(ctx, avctx, &buf, &buf_size); + if (ret > 0 && flush) { + buf_size = 0; + ret = combine_au(ctx, avctx, &buf, &buf_size); + } + if (ret != 0) { + buf_size = next; + goto no_out; + } + } + *poutbuf = buf; + *poutbuf_size = buf_size; + return next; +no_out: + *poutbuf = NULL; + *poutbuf_size = 0; + return buf_size; +} + +static const CodedBitstreamUnitType decompose_unit_types[] = { + VVC_TRAIL_NUT, + VVC_STSA_NUT, + VVC_RADL_NUT, + VVC_RASL_NUT, + VVC_IDR_W_RADL, + VVC_IDR_N_LP, + VVC_CRA_NUT, + VVC_GDR_NUT, + VVC_VPS_NUT, + VVC_SPS_NUT, + VVC_PPS_NUT, + VVC_PH_NUT, + VVC_AUD_NUT, +}; + +static av_cold int vvc_parser_init(AVCodecParserContext *ctx) +{ + VVCParserContext *s = ctx->priv_data; + int ret; + + ret = ff_cbs_init(&s->cbc, AV_CODEC_ID_VVC, NULL); + if (ret < 0) + return ret; + au_detector_init(&s->au_detector); + + s->cbc->decompose_unit_types = decompose_unit_types; + s->cbc->nb_decompose_unit_types = FF_ARRAY_ELEMS(decompose_unit_types); + + return ret; +} + +static void vvc_parser_close(AVCodecParserContext *ctx) +{ + VVCParserContext *s = ctx->priv_data; + + pu_info_unref(&s->au_info); + av_packet_unref(&s->au); + av_packet_unref(&s->last_au); + ff_cbs_fragment_free(&s->picture_unit); + + ff_cbs_close(&s->cbc); + av_freep(&s->pc.buffer); +} + +AVCodecParser ff_vvc_parser = { + .codec_ids = { AV_CODEC_ID_VVC }, + .priv_data_size = sizeof(VVCParserContext), + .parser_init = vvc_parser_init, + .parser_close = vvc_parser_close, + .parser_parse = vvc_parser_parse, +}; From patchwork Mon Oct 24 14:06:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Siedel X-Patchwork-Id: 38961 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2249247pzb; Mon, 24 Oct 2022 07:08:34 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6IfIKlmYnR8KfcwH5Y1xQtkbUX8Make+PNYD7SS+7hPH1mguElYQTKFL0mXVfHRuu6ovdN X-Received: by 2002:a17:907:6e24:b0:791:905f:7881 with SMTP id sd36-20020a1709076e2400b00791905f7881mr27832767ejc.120.1666620513895; Mon, 24 Oct 2022 07:08:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666620513; cv=none; d=google.com; s=arc-20160816; b=udyvZvWjIapiBi7MGogGfTLgrk/dGUFoznU1Mc4Bu3uxVFp4XjuGcQQhJE/Ez2ibel 9KjfkGHabDJBscq9CeTPR1nZFy/BZBT1akHfYF3AuFLEslBVmQ0O495G+I+XnJ2b3xwf bqUm2oyZhEaPl8NEL4Hh1o2i8ZgIWP4Sop4n9+WXDsM+c9Wi/yMb8dxfu/zZijJp1ben rtvevGm2VArnK8KzkHKOi4HvWfxGMGFEpsKt/x8BTcbKEpgZ8woktK6+vwfdhTQQdje/ nYy4e9rryzeEIX8ClIzQP8Nu1T7S2MA1K+0UN8p3u3GFOZiLkqgjla9RFTRgNSGYCZUO c7Jg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=2l50Hzw1EM3O2ZCalAAMBt0no2hXRYYVJ8XIHDGxfwY=; b=VjcltGbkEIHMxVQEr6Di0hyeJSNV1YogvlLqR4/LchcJu2KAimxBs5di1CDoPWZ0cj bSRm51u5wRs8Kk/CwC32tO+f7HVQO7G8CO2cJwF2lFWAl9ggI7NXEkiutwvUK8Znumvu qrByqcz5+EjnsaFieRpicRcumChFy/MNPGiQeAxtS0msSvi4fMoPsGTH5rTzjqImzv0k bZv8g4EnnxyyMpf7mewHKA1ffBK39bPBsffBbj+inkKOwZrXRw/8+8lWAS4aAX+PgEnW 9GQciEp83NADBVmBHjDn78F6QsadAQy4aK1GJzRuq0+FQAe4BlpiO1KBddYOqs6txoHg bFfg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b="n/hX00dA"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id xc13-20020a170907074d00b007a0f4d6bf83si7418806ejb.305.2022.10.24.07.08.33; Mon, 24 Oct 2022 07:08:33 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b="n/hX00dA"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4E01B68BC38; Mon, 24 Oct 2022 17:07:48 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f45.google.com (mail-ed1-f45.google.com [209.85.208.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 019F668BB70 for ; Mon, 24 Oct 2022 17:07:39 +0300 (EEST) Received: by mail-ed1-f45.google.com with SMTP id y69so12706704ede.5 for ; Mon, 24 Oct 2022 07:07:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spin-digital-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PXbE27VibX1P48aG+5h/+6PdC0qKp3SkYZi12dRixCU=; b=n/hX00dAP0yah5wkjcC6kFGz53xAsSqgU0OFv56v69KzsKL8TERSwbtsC57beXfuBY ZU7z/bcjAdMsyRkzy1XulteFoNdzhapRzw2txZXg5cJfxMtLjLWHxhu5Ch91M32N7hbk ORH9Yt+UymZtvglZmnS+u0uugzo7aMBwDzng8ihfxKuXYYJ2CCU0k/SgnQmEPXE+O/Ar oAOmuyyuh9/dOZ/ZXyjSN85zr4rU+1/IO+yCgsoKDm53/hAFRBTKSL8W/bdm64DPtmTy jVgiwjFLl0Fn5u7NVbJSXsNF/j3WBaW8Mfl79tCuEeZQDuNBy1eB/s9dyfPGDBwEFxNk rJGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PXbE27VibX1P48aG+5h/+6PdC0qKp3SkYZi12dRixCU=; b=K8fjeZX8i6yLv1Yx98MnIV2tNeTaPJojrvvtFdhJ6P/iXkCHu6+0yplRX1+A98NNWy Vw3B6Ng2S7ToCopHsSs5BViTuzvpNesMkwYu7j2QbE0OW5083TpYUWYQ9ITOVntOrLAT 7TkeOKrVJ0RL0xdlDOsbTm4iqW3fxpWBSBjsHDZl/VFDGZDs0ZdIqYf4wwHS+WWcjVM1 smRvdZwezMdjSuhxjPqKubj0EW+dyl5LuMirniAsTvEv7jlTcI5Me6lss/O3BvsAD5bh lT0NLv9BtsJHyd9/nrBTnxgA4w153H/mic8b80C/EjEjzngaF8rPU3TGxbD8LT3tEq93 EZvA== X-Gm-Message-State: ACrzQf2Cq/F1q4UNaIOzZetfAWJMTViEJVVZEQQM26/xU+oPrWJtXoeb x8U5pBL460z9r80usVwJp1eqxknSYCGKpA== X-Received: by 2002:a05:6402:31f4:b0:461:604d:2607 with SMTP id dy20-20020a05640231f400b00461604d2607mr14552305edb.330.1666620458887; Mon, 24 Oct 2022 07:07:38 -0700 (PDT) Received: from thomas-win.localdomain ([213.138.44.237]) by smtp.gmail.com with ESMTPSA id s19-20020a170906455300b0079800b81709sm6056436ejq.219.2022.10.24.07.07.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Oct 2022 07:07:38 -0700 (PDT) From: Thomas Siedel To: ffmpeg-devel@ffmpeg.org Date: Mon, 24 Oct 2022 16:06:39 +0200 Message-Id: <20221024140645.4945-5-thomas.ff@spin-digital.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221024140645.4945-1-thomas.ff@spin-digital.com> References: <20221024140645.4945-1-thomas.ff@spin-digital.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 04/10] avcodec: add MP4 to annexb support for H266/VVC X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Thomas Siedel Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: thfPFCVcU5TN Add parser for VVC MP4 to Annex B byte stream format. Signed-off-by: Thomas Siedel --- configure | 1 + libavcodec/Makefile | 2 + libavcodec/bitstream_filters.c | 2 + libavcodec/h266_metadata_bsf.c | 145 ++++++++++++++ libavcodec/vvc_mp4toannexb_bsf.c | 318 +++++++++++++++++++++++++++++++ 5 files changed, 468 insertions(+) create mode 100644 libavcodec/h266_metadata_bsf.c create mode 100644 libavcodec/vvc_mp4toannexb_bsf.c diff --git a/configure b/configure index 691e8da6d0..4d54ce2a93 100755 --- a/configure +++ b/configure @@ -3302,6 +3302,7 @@ mjpeg2jpeg_bsf_select="jpegtables" mpeg2_metadata_bsf_select="cbs_mpeg2" trace_headers_bsf_select="cbs" vp9_metadata_bsf_select="cbs_vp9" +vvc_metadata_bsf_select="cbs_h266" # external libraries aac_at_decoder_deps="audiotoolbox" diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 092f714d67..c6652e5e09 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1224,6 +1224,8 @@ OBJS-$(CONFIG_VP9_METADATA_BSF) += vp9_metadata_bsf.o OBJS-$(CONFIG_VP9_RAW_REORDER_BSF) += vp9_raw_reorder_bsf.o OBJS-$(CONFIG_VP9_SUPERFRAME_BSF) += vp9_superframe_bsf.o OBJS-$(CONFIG_VP9_SUPERFRAME_SPLIT_BSF) += vp9_superframe_split_bsf.o +OBJS-$(CONFIG_VVC_METADATA_BSF) += h266_metadata_bsf.o +OBJS-$(CONFIG_VVC_MP4TOANNEXB_BSF) += vvc_mp4toannexb_bsf.o # thread libraries OBJS-$(HAVE_LIBC_MSVCRT) += file_open.o diff --git a/libavcodec/bitstream_filters.c b/libavcodec/bitstream_filters.c index a3bebefe5f..403884f3d7 100644 --- a/libavcodec/bitstream_filters.c +++ b/libavcodec/bitstream_filters.c @@ -64,6 +64,8 @@ extern const FFBitStreamFilter ff_vp9_metadata_bsf; extern const FFBitStreamFilter ff_vp9_raw_reorder_bsf; extern const FFBitStreamFilter ff_vp9_superframe_bsf; extern const FFBitStreamFilter ff_vp9_superframe_split_bsf; +extern const FFBitStreamFilter ff_vvc_mp4toannexb_bsf; +extern const FFBitStreamFilter ff_vvc_metadata_bsf; #include "libavcodec/bsf_list.c" diff --git a/libavcodec/h266_metadata_bsf.c b/libavcodec/h266_metadata_bsf.c new file mode 100644 index 0000000000..bc3c10f581 --- /dev/null +++ b/libavcodec/h266_metadata_bsf.c @@ -0,0 +1,145 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/common.h" +#include "libavutil/opt.h" + +#include "bsf.h" +#include "bsf_internal.h" +#include "cbs.h" +#include "cbs_bsf.h" +#include "cbs_h266.h" +#include "vvc.h" + +#define IS_SLICE(nut) (nut <= VVC_RASL_NUT || (nut >= VVC_IDR_W_RADL && nut <= VVC_GDR_NUT)) +#define IS_PH(nut) (nut == VVC_PH_NUT) + +typedef struct VVCMetadataContext { + CBSBSFContext common; + + H266RawAUD aud_nal; + + int aud; +} VVCMetadataContext; + +static int h266_metadata_update_fragment(AVBSFContext *bsf, AVPacket *pkt, + CodedBitstreamFragment *pu) +{ + VVCMetadataContext *ctx = bsf->priv_data; + int err, i; + + // If an AUD is present, it must be the first NAL unit. + if (pu->units[0].type == VVC_AUD_NUT) { + if (ctx->aud == BSF_ELEMENT_REMOVE) + ff_cbs_delete_unit(pu, 0); + } else { + if (ctx->aud == BSF_ELEMENT_INSERT) { + const H266RawSlice *first_slice = NULL; + const H266RawPH *ph = NULL; + H266RawAUD *aud = &ctx->aud_nal; + int pic_type = 0, temporal_id = 8, layer_id = 0; + for (i = 0; i < pu->nb_units; i++) { + const H266RawNALUnitHeader *nal = pu->units[i].content; + if (!nal) + continue; + if (nal->nuh_temporal_id_plus1 < temporal_id + 1) + temporal_id = nal->nuh_temporal_id_plus1 - 1; + if (IS_PH(nal->nal_unit_type)) { + ph = pu->units[i].content; + } else if(IS_SLICE(nal->nal_unit_type)) { + const H266RawSlice *slice = pu->units[i].content; + layer_id = nal->nuh_layer_id; + if (slice->header.sh_slice_type == VVC_SLICE_TYPE_B && + pic_type < 2) + pic_type = 2; + if (slice->header.sh_slice_type == VVC_SLICE_TYPE_P && + pic_type < 1) + pic_type = 1; + if (!first_slice) { + first_slice = slice; + if (first_slice->header.sh_picture_header_in_slice_header_flag) + ph = &first_slice->header.sh_picture_header; + else if (!ph) + break; + } + } + } + if (!ph) { + av_log(bsf, AV_LOG_ERROR, "no avaliable picture header"); + return AVERROR_INVALIDDATA; + } + + aud->nal_unit_header = (H266RawNALUnitHeader) { + .nal_unit_type = VVC_AUD_NUT, + .nuh_layer_id = layer_id, + .nuh_temporal_id_plus1 = temporal_id + 1, + }; + aud->aud_pic_type = pic_type; + aud->aud_irap_or_gdr_flag = ph->ph_gdr_or_irap_pic_flag; + + err = ff_cbs_insert_unit_content(pu, 0, VVC_AUD_NUT, aud, NULL); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to insert AUD.\n"); + return err; + } + } + } + return 0; +} + +static const CBSBSFType h266_metadata_type = { + .codec_id = AV_CODEC_ID_VVC, + .fragment_name = "access unit", + .unit_name = "NAL unit", + .update_fragment = &h266_metadata_update_fragment, +}; + +static int vvc_metadata_init(AVBSFContext *bsf) +{ + return ff_cbs_bsf_generic_init(bsf, &h266_metadata_type); +} + +#define OFFSET(x) offsetof(VVCMetadataContext, x) +#define FLAGS (AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_BSF_PARAM) +static const AVOption vvc_metadata_options[] = { + BSF_ELEMENT_OPTIONS_PIR("aud", "Access Unit Delimiter NAL units", + aud, FLAGS), + + { NULL } +}; + +static const AVClass vvc_metadata_class = { + .class_name = "vvc_metadata_bsf", + .item_name = av_default_item_name, + .option = vvc_metadata_options, + .version = LIBAVUTIL_VERSION_INT, +}; + +static const enum AVCodecID vvc_metadata_codec_ids[] = { + AV_CODEC_ID_VVC, AV_CODEC_ID_NONE, +}; + +const FFBitStreamFilter ff_vvc_metadata_bsf = { + .p.name = "vvc_metadata", + .p.codec_ids = vvc_metadata_codec_ids, + .p.priv_class = &vvc_metadata_class, + .priv_data_size = sizeof(VVCMetadataContext), + .init = &vvc_metadata_init, + .close = &ff_cbs_bsf_generic_close, + .filter = &ff_cbs_bsf_generic_filter, +}; diff --git a/libavcodec/vvc_mp4toannexb_bsf.c b/libavcodec/vvc_mp4toannexb_bsf.c new file mode 100644 index 0000000000..d6415dba7b --- /dev/null +++ b/libavcodec/vvc_mp4toannexb_bsf.c @@ -0,0 +1,318 @@ +/* + * VVC MP4 to Annex B byte stream format filter + * Copyright (c) 2022, Thomas Siedel + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include + +#include "libavutil/intreadwrite.h" +#include "libavutil/mem.h" + +#include "avcodec.h" +#include "bsf.h" +#include "bsf_internal.h" +#include "bytestream.h" +#include "vvc.h" + +#include "libavcodec/get_bits.h" + +#define MIN_VVCC_LENGTH 23 + +typedef struct VVCBSFContext { + uint8_t length_size; + int extradata_parsed; +} VVCBSFContext; + +static int vvc_extradata_to_annexb(AVBSFContext *ctx) +{ + GetByteContext gb; + int length_size, num_arrays, i, j; + int ret = 0; + int temp = 0; + int ptl_present; + + uint8_t *new_extradata = NULL; + size_t new_extradata_size = 0; + + int max_picture_width = 0; + int max_picture_height = 0; + int avg_frame_rate = 0; + + bytestream2_init(&gb, ctx->par_in->extradata, ctx->par_in->extradata_size); + temp = bytestream2_get_byte(&gb); + length_size = (( temp & 6) >> 1) + 1; + ptl_present = temp & 1; + if (ptl_present) { + int num_bytes_constraint_info; + int general_profile_idc; + int general_tier_flag; + int general_level_idc; + int ptl_frame_only_constraint_flag; + int ptl_multi_layer_enabled_flag; + int ptl_num_sub_profiles; + int temp3, temp4, temp5; + int temp2 = bytestream2_get_be16(&gb); + int ols_idx = (temp2 >> 7) & 0x1ff; + int num_sublayers = (temp2 >> 4) & 0x7; + int constant_frame_rate = (temp2 >> 2) & 0x3; + int chroma_format_idc = temp2 & 0x3; + int bit_depth_minus8 = (bytestream2_get_byte(&gb) >> 5) & 0x7; + av_log(ctx, AV_LOG_DEBUG, + "bit_depth_minus8 %d chroma_format_idc %d\n", bit_depth_minus8, chroma_format_idc); + av_log(ctx, AV_LOG_DEBUG, + "constant_frame_rate %d, ols_idx %d\n", constant_frame_rate, ols_idx); + // VvcPTLRecord(num_sublayers) native_ptl + temp3 = bytestream2_get_byte(&gb); + num_bytes_constraint_info = (temp3) & 0x3f; + temp4 = bytestream2_get_byte(&gb); + general_profile_idc = (temp4 >> 1) & 0x7f; + general_tier_flag = (temp4) & 1; + general_level_idc = bytestream2_get_byte(&gb); + av_log(ctx, AV_LOG_DEBUG, + "general_profile_idc %d, general_tier_flag %d, general_level_idc %d, num_sublayers %d num_bytes_constraint_info %d\n", general_profile_idc, general_tier_flag, general_level_idc, num_sublayers, num_bytes_constraint_info); + + temp5 = bytestream2_get_byte(&gb); + ptl_frame_only_constraint_flag = (temp5 >> 7) & 0x1; + ptl_multi_layer_enabled_flag = (temp5 >> 6) & 0x1; + for (i = 0; i < num_bytes_constraint_info-1; i++) { + // unsigned int(8*num_bytes_constraint_info - 2) general_constraint_info; + bytestream2_get_byte(&gb); + } + + av_log(ctx, AV_LOG_DEBUG, + "ptl_multi_layer_enabled_flag %d, ptl_frame_only_constraint_flag %d\n",ptl_multi_layer_enabled_flag, ptl_frame_only_constraint_flag); + + if( num_sublayers > 1 ) + { + int temp6 = bytestream2_get_byte(&gb); + uint8_t ptl_sublayer_level_present_flag[8] = {0}; + //uint8_t sublayer_level_idc[8] = {0}; + for (i=num_sublayers - 2; i >= 0; i--){ + ptl_sublayer_level_present_flag[i] = (temp6 >> (7 - (num_sublayers - 2 - i))) & 0x01; + } + // for (j=num_sublayers; j<=8 && num_sublayers > 1; j++) + // bit(1) ptl_reserved_zero_bit = 0; + for (i=num_sublayers-2; i >= 0; i--) + { + if (ptl_sublayer_level_present_flag[i]) { + //sublayer_level_idc[i] = bytestream2_get_byte(&gb); + } + } + } + + ptl_num_sub_profiles = bytestream2_get_byte(&gb); + for (j=0; j < ptl_num_sub_profiles; j++) { + // unsigned int(32) general_sub_profile_idc[j]; + bytestream2_get_be16(&gb); + bytestream2_get_be16(&gb); + } + + max_picture_width = bytestream2_get_be16(&gb); // unsigned_int(16) max_picture_width; + max_picture_height = bytestream2_get_be16(&gb); // unsigned_int(16) max_picture_height; + avg_frame_rate = bytestream2_get_be16(&gb); // unsigned int(16) avg_frame_rate; } + av_log(ctx, AV_LOG_DEBUG, + "max_picture_width %d, max_picture_height %d, avg_frame_rate %d\n", max_picture_width, max_picture_height, avg_frame_rate); + } + + num_arrays = bytestream2_get_byte(&gb); + + for (i = 0; i < num_arrays; i++) { + int cnt; + int type = bytestream2_get_byte(&gb) & 0x1f; + + if (type != VVC_OPI_NUT || type != VVC_DCI_NUT) + cnt = bytestream2_get_be16(&gb); + else + cnt = 1; + + av_log(ctx, AV_LOG_DEBUG, + "nalu_type %d cnt %d\n", type, cnt); + + if (!(type == VVC_VPS_NUT || type == VVC_SPS_NUT || type == VVC_PPS_NUT || + type == VVC_PREFIX_SEI_NUT || type == VVC_SUFFIX_SEI_NUT)) { + av_log(ctx, AV_LOG_ERROR, "Invalid NAL unit type in extradata: %d\n", type); + ret = AVERROR_INVALIDDATA; + goto fail; + } + + for (j = 0; j < cnt; j++) { + int nalu_len = bytestream2_get_be16(&gb); + + if (4 + AV_INPUT_BUFFER_PADDING_SIZE + nalu_len > SIZE_MAX - new_extradata_size) { + ret = AVERROR_INVALIDDATA; + goto fail; + } + ret = av_reallocp(&new_extradata, new_extradata_size + nalu_len + 4 + AV_INPUT_BUFFER_PADDING_SIZE); + if (ret < 0) + goto fail; + + AV_WB32(new_extradata + new_extradata_size, 1); // add the startcode + bytestream2_get_buffer(&gb, new_extradata + new_extradata_size + 4, nalu_len); + new_extradata_size += 4 + nalu_len; + memset(new_extradata + new_extradata_size, 0, AV_INPUT_BUFFER_PADDING_SIZE); + } + } + + av_freep(&ctx->par_out->extradata); + ctx->par_out->extradata = new_extradata; + ctx->par_out->extradata_size = new_extradata_size; + + if (!new_extradata_size) + av_log(ctx, AV_LOG_WARNING, "No parameter sets in the extradata\n"); + + return length_size; +fail: + av_freep(&new_extradata); + return ret; +} + +static int vvc_mp4toannexb_init(AVBSFContext *ctx) +{ + VVCBSFContext *s = ctx->priv_data; + int ret; + + if (ctx->par_in->extradata_size < MIN_VVCC_LENGTH || + AV_RB24(ctx->par_in->extradata) == 1 || + AV_RB32(ctx->par_in->extradata) == 1) { + av_log(ctx, AV_LOG_VERBOSE, "The input looks like it is Annex B already\n"); + } else { + ret = vvc_extradata_to_annexb(ctx); + if (ret < 0) + return ret; + s->length_size = ret; + s->extradata_parsed = 1; + } + + return 0; +} + +static int vvc_mp4toannexb_filter(AVBSFContext *ctx, AVPacket *out) +{ + VVCBSFContext *s = ctx->priv_data; + AVPacket *in; + GetByteContext gb; + + int is_irap = 0; + int added_extra = 0; + int i, ret = 0; + + ret = ff_bsf_get_packet(ctx, &in); + if (ret < 0) + return ret; + + if (!s->extradata_parsed) { + av_packet_move_ref(out, in); + av_packet_free(&in); + return 0; + } + + bytestream2_init(&gb, in->data, in->size); + + /* check if this packet contains an IRAP. The extradata will need to be added before any potential PH_NUT */ + while (bytestream2_get_bytes_left(&gb)) { + uint32_t nalu_size = 0; + int nalu_type; + + if (bytestream2_get_bytes_left(&gb) < s->length_size) { + ret = AVERROR_INVALIDDATA; + goto fail; + } + + for (i = 0; i < s->length_size; i++) + nalu_size = (nalu_size << 8) | bytestream2_get_byte(&gb); + + if (nalu_size < 2 || nalu_size > bytestream2_get_bytes_left(&gb)) { + ret = AVERROR_INVALIDDATA; + goto fail; + } + + nalu_type = (bytestream2_peek_be16(&gb) >> 3) & 0x1f; + is_irap = nalu_type >= VVC_IDR_W_RADL && nalu_type <= VVC_RSV_IRAP_11; + if (is_irap) { + break; + } + bytestream2_seek(&gb, nalu_size, SEEK_CUR); + } + + bytestream2_seek(&gb, 0, SEEK_SET); + while (bytestream2_get_bytes_left(&gb)) { + uint32_t nalu_size = 0; + int nalu_type; + int add_extradata, extra_size, prev_size; + + if (bytestream2_get_bytes_left(&gb) < s->length_size) { + ret = AVERROR_INVALIDDATA; + goto fail; + } + + for (i = 0; i < s->length_size; i++) + nalu_size = (nalu_size << 8) | bytestream2_get_byte(&gb); + + if (nalu_size < 2 || nalu_size > bytestream2_get_bytes_left(&gb)) { + ret = AVERROR_INVALIDDATA; + goto fail; + } + + nalu_type = (bytestream2_peek_be16(&gb) >> 3) & 0x1f; + + /* prepend extradata to IRAP frames */ + add_extradata = is_irap && nalu_type != VVC_AUD_NUT && !added_extra; + extra_size = add_extradata * ctx->par_out->extradata_size; + added_extra |= add_extradata; + + if (FFMIN(INT_MAX, SIZE_MAX) < 4ULL + nalu_size + extra_size) { + ret = AVERROR_INVALIDDATA; + goto fail; + } + + prev_size = out->size; + + ret = av_grow_packet(out, 4 + nalu_size + extra_size); + if (ret < 0) + goto fail; + + if (extra_size) + memcpy(out->data + prev_size, ctx->par_out->extradata, extra_size); + AV_WB32(out->data + prev_size + extra_size, 1); + bytestream2_get_buffer(&gb, out->data + prev_size + 4 + extra_size, nalu_size); + } + + ret = av_packet_copy_props(out, in); + if (ret < 0) + goto fail; + +fail: + if (ret < 0) + av_packet_unref(out); + av_packet_free(&in); + + return ret; +} + +static const enum AVCodecID codec_ids[] = { + AV_CODEC_ID_VVC, AV_CODEC_ID_NONE, +}; + +const FFBitStreamFilter ff_vvc_mp4toannexb_bsf = { + .p.name = "vvc_mp4toannexb", + .p.codec_ids = codec_ids, + .priv_data_size = sizeof(VVCBSFContext), + .init = vvc_mp4toannexb_init, + .filter = vvc_mp4toannexb_filter, +}; From patchwork Mon Oct 24 14:06:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thomas Siedel X-Patchwork-Id: 38963 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2249490pzb; Mon, 24 Oct 2022 07:08:57 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5AGT1kbjrI/2N/PGjqyAD2qFGj626UpEu/bwtq7tmT7UK7aCxA6YMam3oPIZTQoK0Qrh4D X-Received: by 2002:a17:907:2c68:b0:78d:eac6:2d0d with SMTP id ib8-20020a1709072c6800b0078deac62d0dmr28403321ejc.55.1666620536318; Mon, 24 Oct 2022 07:08:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666620536; cv=none; d=google.com; s=arc-20160816; b=Lge9Bo7QziXk7JY1ce4ELqWVvmq3oSGtE2fzxUSeLXsHkhEvQANsbsRkm5+lbeHGG4 nFiQk9Dw2ZWKTCvwbMGhuUIZ24niMOAlKWzVLo0gAx10a1QJZTvKvCXtAV2J3YN8xpwr wCurJJYo6QPZ0u/x4cNiucuPCDrcUOnavmO5ofQ6tbkCxgfdXZtuIM5a5zgVx04o8/ne aP7UPaKCRs/7yPgY6rVxzIipmzQEuiyt9gv6oXK/LDdZ/mjZnKGPKViZpskD62F7LjAx v7VVPL3sGOp61XyLdChdya6gLhdXPPOFc3RVNAaxzm8vY2z7fkPFJMQmpQAhZt9eKFaz aaNg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=f1EudIoubuUX52tTDIx5xtenQIVE9S4awm+R62V0xvM=; b=hd4xfvfxHROzNKJEU2ZVFMW8BMeeS2BRc5UWF+A8BghLC+/M/0VUoJry7tRY8yStJ7 J7tw0g1FPTtA5JYxm7PQ6q5lCMWw4Ze3M/9QPcXfUiGWbvCoLyysDM0k9ngwrgLgGoPZ 7BtMc2Hk7oHfYPL18TJuvTeDN2cuwrNjF/ikaLUIynEW1GK2iH85xeXgMjAS/i59+xRw +WJ30P2JjunK0S27xpWcyHbU3uiSLoATdm0XoClmEbDqtzKA1z/nbVsTSzXujEMagMFp liGVcOLFvByibGgdSXv/DN1l3NTaGJwr/qcy28dxwzV+QYLYzleb1EMgtg4QGGfZ/j5l UHhQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=G0DrzEwM; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id y10-20020a056402270a00b0045dc9b4c034si21646236edd.582.2022.10.24.07.08.54; Mon, 24 Oct 2022 07:08:56 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=G0DrzEwM; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 46AE668BB68; Mon, 24 Oct 2022 17:07:50 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f41.google.com (mail-ed1-f41.google.com [209.85.208.41]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 8C2BC68BB70 for ; Mon, 24 Oct 2022 17:07:41 +0300 (EEST) Received: by mail-ed1-f41.google.com with SMTP id a13so31810950edj.0 for ; Mon, 24 Oct 2022 07:07:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spin-digital-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=D+oa9p1iGqQcqQrVApoh0ws+BldKYrwNquLKngL4v9w=; b=G0DrzEwM+LxAuOXP9vDWBDOMX1Gb0z8dDz5bpUamYg6f9pVrrtro4kbbMS97fxSxY4 roQFFsQEqEg+xOtMM+7zeu3FPFEGjcEWylpsuiwHLIzmvSBfgsHigfW4J8RbovXUdquI 7LK5hUL9yHOZcGABCMR90jsbFlZu6pVDFBz0kr7iE8fxSXdFElguRlbByM2LusJKQufG 9G+1j2PC7Wm8gWsbWrNZggWc4i4DT06O9XlxEqinOLiGWqSZit9CfVhRzG6cWnHAqIHp u0CpggQZnqaYWRv9q5q+opLb6QCAHLVg8O+n7yOF6pH+yiW8V/K22GYsWziOHgaVv8Vm VhJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=D+oa9p1iGqQcqQrVApoh0ws+BldKYrwNquLKngL4v9w=; b=Y7vMLmDGAhboRTTIAQuNGKHa6uv7TSa/6q4hfPRlOx06hpV+3n5iT4M5cracoT601g j6Nf7IN0jUdSPyj5Lh9KU3E9BLFx08qLsmzBPaPrzNiy7r2yOP0S5Yvum/rL4am2or64 SY7UEodWxtxmG5+2cRyI+dPVTQZfYMw+7jXrK5h56IGmewq7Kf7F4R7aTlKJ4mswr/DP Z9BBvCdTF3vbOTu29tGxxDDgd+O8cYmaQiynqH9vV65bP9h3Apa/TPlR0NTttvzaa+hE 428b7QDNJ/VjYwJ8LYixY9IsKzr0Iz+uKuRxLOTSwG29rdB+7m5XSwZyYPxXvh/q1jX+ 69Pw== X-Gm-Message-State: ACrzQf3APzlXX+tqdjfHEXBB1nVTMIe0HUrLcHJfXvvGNc0CL0cDCKXx U5ezsDJP7gFwQ6Bqf356Ei8NNBG44G8pBA== X-Received: by 2002:a17:907:3201:b0:741:94f2:aeaf with SMTP id xg1-20020a170907320100b0074194f2aeafmr27182637ejb.505.1666620459937; Mon, 24 Oct 2022 07:07:39 -0700 (PDT) Received: from thomas-win.localdomain ([213.138.44.237]) by smtp.gmail.com with ESMTPSA id s19-20020a170906455300b0079800b81709sm6056436ejq.219.2022.10.24.07.07.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Oct 2022 07:07:39 -0700 (PDT) From: Thomas Siedel To: ffmpeg-devel@ffmpeg.org Date: Mon, 24 Oct 2022 16:06:40 +0200 Message-Id: <20221024140645.4945-6-thomas.ff@spin-digital.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221024140645.4945-1-thomas.ff@spin-digital.com> References: <20221024140645.4945-1-thomas.ff@spin-digital.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 05/10] avformat: add demuxer and probe support for H266/VVC X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Thomas Siedel Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 9Rg0K+fED2TG Add demuxer to probe raw vvc and parse vvcc byte stream format. Signed-off-by: Thomas Siedel --- libavformat/Makefile | 1 + libavformat/allformats.c | 1 + libavformat/demux.c | 7 +- libavformat/vvc.c | 919 +++++++++++++++++++++++++++++++++++++++ libavformat/vvc.h | 99 +++++ libavformat/vvcdec.c | 61 +++ 6 files changed, 1086 insertions(+), 2 deletions(-) create mode 100644 libavformat/vvc.c create mode 100644 libavformat/vvc.h create mode 100644 libavformat/vvcdec.c diff --git a/libavformat/Makefile b/libavformat/Makefile index d7f198bf39..00ab4ded89 100644 --- a/libavformat/Makefile +++ b/libavformat/Makefile @@ -595,6 +595,7 @@ OBJS-$(CONFIG_VOC_MUXER) += vocenc.o voc.o OBJS-$(CONFIG_VPK_DEMUXER) += vpk.o OBJS-$(CONFIG_VPLAYER_DEMUXER) += vplayerdec.o subtitles.o OBJS-$(CONFIG_VQF_DEMUXER) += vqf.o +OBJS-$(CONFIG_VVC_DEMUXER) += vvcdec.o rawdec.o OBJS-$(CONFIG_W64_DEMUXER) += wavdec.o w64.o pcm.o OBJS-$(CONFIG_W64_MUXER) += wavenc.o w64.o OBJS-$(CONFIG_WAV_DEMUXER) += wavdec.o pcm.o diff --git a/libavformat/allformats.c b/libavformat/allformats.c index 47c419a009..a4e3822681 100644 --- a/libavformat/allformats.c +++ b/libavformat/allformats.c @@ -474,6 +474,7 @@ extern const AVOutputFormat ff_voc_muxer; extern const AVInputFormat ff_vpk_demuxer; extern const AVInputFormat ff_vplayer_demuxer; extern const AVInputFormat ff_vqf_demuxer; +extern const AVInputFormat ff_vvc_demuxer; extern const AVInputFormat ff_w64_demuxer; extern const AVOutputFormat ff_w64_muxer; extern const AVInputFormat ff_wav_demuxer; diff --git a/libavformat/demux.c b/libavformat/demux.c index 2dfd82a63c..8dbde23fcd 100644 --- a/libavformat/demux.c +++ b/libavformat/demux.c @@ -120,6 +120,7 @@ static int set_codec_from_probe_data(AVFormatContext *s, AVStream *st, { "mp3", AV_CODEC_ID_MP3, AVMEDIA_TYPE_AUDIO }, { "mpegvideo", AV_CODEC_ID_MPEG2VIDEO, AVMEDIA_TYPE_VIDEO }, { "truehd", AV_CODEC_ID_TRUEHD, AVMEDIA_TYPE_AUDIO }, + { "vvc", AV_CODEC_ID_VVC, AVMEDIA_TYPE_VIDEO }, { 0 } }; int score; @@ -743,7 +744,8 @@ static int64_t select_from_pts_buffer(AVStream *st, int64_t *pts_buffer, int64_t { FFStream *const sti = ffstream(st); int onein_oneout = st->codecpar->codec_id != AV_CODEC_ID_H264 && - st->codecpar->codec_id != AV_CODEC_ID_HEVC; + st->codecpar->codec_id != AV_CODEC_ID_HEVC && + st->codecpar->codec_id != AV_CODEC_ID_VVC; if (!onein_oneout) { int delay = sti->avctx->has_b_frames; @@ -933,7 +935,8 @@ static void compute_pkt_fields(AVFormatContext *s, AVStream *st, int64_t offset; AVRational duration; int onein_oneout = st->codecpar->codec_id != AV_CODEC_ID_H264 && - st->codecpar->codec_id != AV_CODEC_ID_HEVC; + st->codecpar->codec_id != AV_CODEC_ID_HEVC && + st->codecpar->codec_id != AV_CODEC_ID_VVC; if (s->flags & AVFMT_FLAG_NOFILLIN) return; diff --git a/libavformat/vvc.c b/libavformat/vvc.c new file mode 100644 index 0000000000..b27a522009 --- /dev/null +++ b/libavformat/vvc.c @@ -0,0 +1,919 @@ +/* + * VVC helper functions for muxers + * + * Copyright (C) 2022, Thomas Siedel + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavcodec/get_bits.h" +#include "libavcodec/golomb.h" +#include "libavcodec/vvc.h" +#include "libavutil/intreadwrite.h" +#include "avc.h" +#include "avio.h" +#include "avio_internal.h" +#include "vvc.h" + +typedef struct VVCCNALUnitArray { + uint8_t array_completeness; + uint8_t NAL_unit_type; + uint16_t num_nalus; + uint16_t *nal_unit_length; + uint8_t **nal_unit; +} VVCCNALUnitArray; + +typedef struct VVCPTLRecord { + uint8_t num_bytes_constraint_info; + uint8_t general_profile_idc; + uint8_t general_tier_flag; + uint8_t general_level_idc; + uint8_t ptl_frame_only_constraint_flag; + uint8_t ptl_multilayer_enabled_flag; + uint8_t general_constraint_info[9]; + uint8_t *ptl_sublayer_level_present_flag; + uint8_t *sublayer_level_idc; + uint8_t ptl_num_sub_profiles; + uint32_t *general_sub_profile_idc; +} VVCPTLRecord; + +typedef struct VVCDecoderConfigurationRecord { + uint8_t lengthSizeMinusOne; + uint8_t ptl_present_flag; + uint16_t ols_idx; + uint8_t num_sublayers; + uint8_t constant_frame_rate; + uint8_t chroma_format_idc; + uint8_t bit_depth_minus8; + VVCPTLRecord ptl; + uint16_t max_picture_width; + uint16_t max_picture_height; + uint16_t avg_frame_rate; + uint8_t num_of_arrays; + VVCCNALUnitArray *array; +} VVCDecoderConfigurationRecord; + +typedef struct VVCCProfileTierLevel { + uint8_t profile_idc; + uint8_t tier_flag; + uint8_t general_level_idc; + uint8_t ptl_frame_only_constraint_flag; + uint8_t ptl_multilayer_enabled_flag; +// general_constraint_info + uint8_t gci_present_flag; + uint8_t gci_general_constraints[9]; + uint8_t gci_num_reserved_bits; +// end general_constraint_info + uint8_t *ptl_sublayer_level_present_flag; + uint8_t *sublayer_level_idc; + uint8_t ptl_num_sub_profiles; + uint32_t *general_sub_profile_idc; +} VVCCProfileTierLevel; + + +static void vvcc_update_ptl(VVCDecoderConfigurationRecord *vvcc, + VVCCProfileTierLevel *ptl) +{ + /* + * The level indication general_level_idc must indicate a level of + * capability equal to or greater than the highest level indicated for the + * highest tier in all the parameter sets. + */ + if (vvcc->ptl.general_tier_flag < ptl->tier_flag) + vvcc->ptl.general_level_idc = ptl->general_level_idc; + else + vvcc->ptl.general_level_idc = FFMAX(vvcc->ptl.general_level_idc, ptl->general_level_idc); + + /* + * The tier indication general_tier_flag must indicate a tier equal to or + * greater than the highest tier indicated in all the parameter sets. + */ + vvcc->ptl.general_tier_flag = FFMAX(vvcc->ptl.general_tier_flag, ptl->tier_flag); + + /* + * The profile indication general_profile_idc must indicate a profile to + * which the stream associated with this configuration record conforms. + * + * If the sequence parameter sets are marked with different profiles, then + * the stream may need examination to determine which profile, if any, the + * entire stream conforms to. If the entire stream is not examined, or the + * examination reveals that there is no profile to which the entire stream + * conforms, then the entire stream must be split into two or more + * sub-streams with separate configuration records in which these rules can + * be met. + * + * Note: set the profile to the highest value for the sake of simplicity. + */ + vvcc->ptl.general_profile_idc = FFMAX(vvcc->ptl.general_profile_idc, ptl->profile_idc); + + /* + * Each bit in flags may only be set if all + * the parameter sets set that bit. + */ + vvcc->ptl.ptl_frame_only_constraint_flag &= ptl->ptl_frame_only_constraint_flag; + vvcc->ptl.ptl_multilayer_enabled_flag &= ptl->ptl_multilayer_enabled_flag; + + /* + * Constraints Info + */ + if(ptl->gci_present_flag) { + vvcc->ptl.num_bytes_constraint_info = 9; + memcpy(&vvcc->ptl.general_constraint_info[0], &ptl->gci_general_constraints[0], sizeof(uint8_t)*9); + + } else { + vvcc->ptl.num_bytes_constraint_info = 1; + memset(&vvcc->ptl.general_constraint_info[0], 0, sizeof(uint8_t)*9); + } + + /* + * Each bit in flags may only be set if one of + * the parameter sets set that bit. + */ + vvcc->ptl.ptl_sublayer_level_present_flag = (uint8_t *) malloc(sizeof(uint8_t)*vvcc->num_sublayers-1); + vvcc->ptl.sublayer_level_idc = (uint8_t *) malloc(sizeof(uint8_t)*vvcc->num_sublayers-1); + + memset(vvcc->ptl.ptl_sublayer_level_present_flag, 0, sizeof(uint8_t)*vvcc->num_sublayers-1); + memset(vvcc->ptl.sublayer_level_idc, 0, sizeof(uint8_t)*vvcc->num_sublayers-1); + + for(int i = vvcc->num_sublayers - 2; i >= 0; i--) + { + vvcc->ptl.ptl_sublayer_level_present_flag[i] |= ptl->ptl_sublayer_level_present_flag[i]; + if(vvcc->ptl.ptl_sublayer_level_present_flag[i]){ + vvcc->ptl.sublayer_level_idc[i] = FFMAX(vvcc->ptl.sublayer_level_idc[i], ptl->sublayer_level_idc[i]); + } + else{ + if(i == vvcc->num_sublayers - 1){ + vvcc->ptl.sublayer_level_idc[i] = vvcc->ptl.general_level_idc; + }else{ + vvcc->ptl.sublayer_level_idc[i] = vvcc->ptl.sublayer_level_idc[i+1]; + } + } + } + + vvcc->ptl.ptl_num_sub_profiles = FFMAX(vvcc->ptl.ptl_num_sub_profiles, ptl->ptl_num_sub_profiles); + if( vvcc->ptl.ptl_num_sub_profiles ){ + vvcc->ptl.general_sub_profile_idc = (uint32_t *) malloc(sizeof(uint32_t)*vvcc->ptl.ptl_num_sub_profiles); + for(int i = 0; i < vvcc->ptl.ptl_num_sub_profiles; i++) { + vvcc->ptl.general_sub_profile_idc[i] = ptl->general_sub_profile_idc[i]; + } + } + else + { + vvcc->ptl.general_sub_profile_idc = (uint32_t *) malloc(sizeof(uint32_t)); + } +} + +static void vvcc_parse_ptl(GetBitContext *gb, + VVCDecoderConfigurationRecord *vvcc, + unsigned int profileTierPresentFlag, + unsigned int max_sub_layers_minus1) +{ + VVCCProfileTierLevel general_ptl; + int j; + + if(profileTierPresentFlag) { + general_ptl.profile_idc = get_bits(gb, 7); + general_ptl.tier_flag = get_bits1(gb); + } + general_ptl.general_level_idc = get_bits(gb, 8); + + general_ptl.ptl_frame_only_constraint_flag = get_bits1(gb); + general_ptl.ptl_multilayer_enabled_flag = get_bits1(gb); + if(profileTierPresentFlag) { // parse constraint info + general_ptl.gci_present_flag = get_bits1(gb); + if(general_ptl.gci_present_flag) { + for (j = 0; j < 8; j++) + general_ptl.gci_general_constraints[j] = get_bits(gb, 8); + general_ptl.gci_general_constraints[8] = 0; + general_ptl.gci_general_constraints[8] = get_bits(gb, 7); + + general_ptl.gci_num_reserved_bits = get_bits(gb, 8); + skip_bits(gb, general_ptl.gci_num_reserved_bits); + } + while(gb->index % 8 != 0) + skip_bits1(gb); + } + + general_ptl.ptl_sublayer_level_present_flag = (uint8_t *) malloc(sizeof(uint8_t)*max_sub_layers_minus1); + for(int i = max_sub_layers_minus1-1; i >= 0; i--) { + general_ptl.ptl_sublayer_level_present_flag[i] = get_bits1(gb); + } + while(gb->index%8 != 0) + skip_bits1(gb); + + general_ptl.sublayer_level_idc = (uint8_t *) malloc(sizeof(uint8_t)*max_sub_layers_minus1); + for(int i = max_sub_layers_minus1-1; i >= 0; i--) { + if( general_ptl.ptl_sublayer_level_present_flag[i] ) + general_ptl.sublayer_level_idc[i] = get_bits(gb, 8); + } + + if(profileTierPresentFlag) { + general_ptl.ptl_num_sub_profiles = get_bits(gb, 8); + if( general_ptl.ptl_num_sub_profiles) { + general_ptl.general_sub_profile_idc = (uint32_t *) malloc(sizeof(uint32_t)*general_ptl.ptl_num_sub_profiles); + for(int i = 0; i < general_ptl.ptl_num_sub_profiles; i++) { + general_ptl.general_sub_profile_idc[i] = get_bits_long(gb, 32); + } + } + else + { + general_ptl.general_sub_profile_idc = (uint32_t *) malloc(sizeof(uint32_t)); + } + } + + vvcc_update_ptl(vvcc, &general_ptl); + + free(general_ptl.ptl_sublayer_level_present_flag); + free(general_ptl.sublayer_level_idc); + free(general_ptl.general_sub_profile_idc); +} + +static int vvcc_parse_vps(GetBitContext *gb, + VVCDecoderConfigurationRecord *vvcc) +{ + unsigned int vps_max_layers_minus1; + unsigned int vps_max_sub_layers_minus1; + unsigned int vps_default_ptl_dpb_hrd_max_tid_flag; + unsigned int vps_all_independant_layer_flag; + unsigned int vps_each_layer_is_an_ols_flag; + unsigned int vps_ols_mode_idc; + + unsigned int *vps_pt_present_flag; + unsigned int *vps_ptl_max_tid; + unsigned int vps_num_ptls_minus1 = 0; + + + /* + * vps_video_parameter_set_id u(4) + */ + skip_bits(gb, 4); + + vps_max_layers_minus1 = get_bits(gb, 6); + vps_max_sub_layers_minus1 = get_bits(gb, 3); + + /* + * numTemporalLayers greater than 1 indicates that the stream to which this + * configuration record applies is temporally scalable and the contained + * number of temporal layers (also referred to as temporal sub-layer or + * sub-layer in ISO/IEC 23008-2) is equal to numTemporalLayers. Value 1 + * indicates that the stream is not temporally scalable. Value 0 indicates + * that it is unknown whether the stream is temporally scalable. + */ + vvcc->num_sublayers = FFMAX(vvcc->num_sublayers, + vps_max_sub_layers_minus1 + 1); + + + if(vps_max_layers_minus1 > 0 && vps_max_sub_layers_minus1 > 0) + vps_default_ptl_dpb_hrd_max_tid_flag = get_bits1(gb); //vps_default_ptl_dpb_hrd_max_tid_flags + if(vps_max_layers_minus1 > 0) + vps_all_independant_layer_flag = get_bits1(gb); + + for(int i =0; i <= vps_max_layers_minus1; i++) { + skip_bits(gb, 6); //vps_default_ptl_dpb_hrd_max_tid_flagsvps_layer_id[i] + if(i > 0 && !vps_all_independant_layer_flag) { + if(get_bits1(gb)) { // vps_independant_layer_flag + unsigned int vps_max_tid_ref_present_flag = get_bits1(gb); + for (int j =0; j 0) { + if(vps_all_independant_layer_flag) + vps_each_layer_is_an_ols_flag = get_bits1(gb); + if(vps_each_layer_is_an_ols_flag) { + if(!vps_all_independant_layer_flag) + vps_ols_mode_idc = get_bits(gb, 2); + if(vps_ols_mode_idc == 2) { + unsigned int vps_num_output_layer_sets_minus2 = get_bits(gb, 8); + for( int i = 1; i <= vps_num_output_layer_sets_minus2 +1; i++) { + for( int j = 0; j <= vps_max_layers_minus1; j++) { + skip_bits1(gb); + } + } + } + } + vps_num_ptls_minus1 = get_bits(gb, 8); + } + + vps_pt_present_flag = (unsigned int *) malloc(sizeof(unsigned int) * (vps_num_ptls_minus1+1)); + vps_ptl_max_tid = (unsigned int *) malloc(sizeof(unsigned int) * (vps_num_ptls_minus1+1)); + for(int i = 0; i <= vps_num_ptls_minus1; i++) { + if(i>0) + vps_pt_present_flag[i] = get_bits1(gb); + if(!vps_default_ptl_dpb_hrd_max_tid_flag) + vps_ptl_max_tid[i] = get_bits(gb, 3); + } + + while(gb->index%8 != 0) + skip_bits1(gb); + + for(int i = 0; i <= vps_num_ptls_minus1; i++) { + vvcc_parse_ptl(gb, vvcc, vps_pt_present_flag[i], vps_ptl_max_tid[i]); + } + + free(vps_pt_present_flag); + free(vps_ptl_max_tid); + + /* nothing useful for vvcc past this point */ + return 0; +} + +static int vvcc_parse_sps(GetBitContext *gb, + VVCDecoderConfigurationRecord *vvcc) +{ + unsigned int sps_max_sub_layers_minus1, log2_ctu_size_minus5; + //unsigned int num_short_term_ref_pic_sets, num_delta_pocs[VVC_MAX_REF_PIC_LISTS]; + //unsigned int sps_chroma_format_idc; + unsigned int sps_subpic_same_size_flag, sps_pic_height_max_in_luma_sample, sps_pic_width_max_in_luma_sample; + unsigned int sps_independant_subpics_flag; + unsigned int flag; + + skip_bits(gb, 8); // sps_seq_parameter_set_id && sps_video_parameter_set_id + sps_max_sub_layers_minus1 = get_bits (gb, 3); + + /* + * numTemporalLayers greater than 1 indicates that the stream to which this + * configuration record applies is temporally scalable and the contained + * number of temporal layers (also referred to as temporal sub-layer or + * sub-layer in ISO/IEC 23008-2) is equal to numTemporalLayers. Value 1 + * indicates that the stream is not temporally scalable. Value 0 indicates + * that it is unknown whether the stream is temporally scalable. + */ + vvcc->num_sublayers = FFMAX(vvcc->num_sublayers, + sps_max_sub_layers_minus1 + 1); + + vvcc->chroma_format_idc = get_bits(gb, 2); + log2_ctu_size_minus5 = get_bits(gb, 2); + + if(get_bits1(gb)) //sps_ptl_dpb_hrd_params_present_flag + vvcc_parse_ptl(gb, vvcc, 1, sps_max_sub_layers_minus1); + + flag = get_bits(gb, 1); //skip_bits1(gb); //sps_gdr_enabled_flag + flag = get_bits(gb, 1); //sps_ref_pic_resampling_enabled_flag + if(flag){ //sps_ref_pic_resampling_enabled_flag + flag = get_bits(gb, 1); //skip_bits1(gb); //sps_res_change_in_clvs_allowed_flag + } + + sps_pic_width_max_in_luma_sample = get_ue_golomb_long(gb); + vvcc->max_picture_width = FFMAX(vvcc->max_picture_width , sps_pic_width_max_in_luma_sample); + sps_pic_height_max_in_luma_sample= get_ue_golomb_long(gb); + vvcc->max_picture_height = FFMAX(vvcc->max_picture_height , sps_pic_height_max_in_luma_sample); + + if(get_bits1(gb)) { + get_ue_golomb_long(gb); // sps_conf_win_left_offset + get_ue_golomb_long(gb); // sps_conf_win_right_offset + get_ue_golomb_long(gb); // sps_conf_win_top_offset + get_ue_golomb_long(gb); // sps_conf_win_bottom_offset + } + + if(get_bits1(gb)) { // sps_subpic_info_present_flag + unsigned int sps_num_subpics_minus1 = get_ue_golomb_long(gb); + if(sps_num_subpics_minus1 > 0) { // sps_num_subpics_minus1 + sps_independant_subpics_flag = get_bits1(gb); + sps_subpic_same_size_flag = get_bits1(gb); + } + for(int i = 0; sps_num_subpics_minus1 > 0 && i <= sps_num_subpics_minus1; i++) { + if(!sps_subpic_same_size_flag || i == 0) { + int len = FFMIN(log2_ctu_size_minus5 +5, 16); + if(i > 0 && sps_pic_width_max_in_luma_sample > 128) + skip_bits(gb, len); + if(i > 0 && sps_pic_height_max_in_luma_sample > 128) + skip_bits(gb, len); + if(i < sps_num_subpics_minus1 && sps_pic_width_max_in_luma_sample > 128) + skip_bits(gb, len); + if(i < sps_num_subpics_minus1 && sps_pic_height_max_in_luma_sample > 128) + skip_bits(gb, len); + } + if(!sps_independant_subpics_flag) { + skip_bits(gb, 2); // sps_subpic_treated_as_pic_flag && sps_loop_filter_across_subpic_enabled_flag + } + } + get_ue_golomb_long(gb); // sps_subpic_id_len_minus1 + if(get_bits1(gb)) { // sps_subpic_id_mapping_explicitly_signalled_flag + if(get_bits1(gb)) // sps_subpic_id_mapping_present_flag + for(int i = 0; i <= sps_num_subpics_minus1; i++) { + skip_bits1(gb); // sps_subpic_id[i] + } + } + } + vvcc->bit_depth_minus8 = get_ue_golomb_long(gb); + + /* nothing useful for vvcc past this point */ + return 0; +} + +static int vvcc_parse_pps(GetBitContext *gb, + VVCDecoderConfigurationRecord *vvcc) +{ + + // Nothing of importance to parse in PPS + /* nothing useful for vvcc past this point */ + return 0; +} + +static void nal_unit_parse_header(GetBitContext *gb, uint8_t *nal_type) +{ + /* + * forbidden_zero_bit u(1) + * nuh_reserved_zero_bit u(1) + * nuh_layer_id u(6) + */ + skip_bits(gb, 8); + *nal_type = get_bits(gb, 5); + + /* + * nuh_temporal_id_plus1 u(3) + */ + skip_bits(gb, 3); +} + +static int vvcc_array_add_nal_unit(uint8_t *nal_buf, uint32_t nal_size, + uint8_t nal_type, int ps_array_completeness, + VVCDecoderConfigurationRecord *vvcc) +{ + int ret; + uint8_t index; + uint16_t num_nalus; + VVCCNALUnitArray *array; + + for (index = 0; index < vvcc->num_of_arrays; index++) + if (vvcc->array[index].NAL_unit_type == nal_type) + break; + + if (index >= vvcc->num_of_arrays) { + uint8_t i; + + ret = av_reallocp_array(&vvcc->array, index + 1, sizeof(VVCCNALUnitArray)); + if (ret < 0) + return ret; + + for (i = vvcc->num_of_arrays; i <= index; i++) + memset(&vvcc->array[i], 0, sizeof(VVCCNALUnitArray)); + vvcc->num_of_arrays = index + 1; + } + + array = &vvcc->array[index]; + num_nalus = array->num_nalus; + + ret = av_reallocp_array(&array->nal_unit, num_nalus + 1, sizeof(uint8_t*)); + if (ret < 0) + return ret; + + ret = av_reallocp_array(&array->nal_unit_length, num_nalus + 1, sizeof(uint16_t)); + if (ret < 0) + return ret; + + array->nal_unit [num_nalus] = nal_buf; + array->nal_unit_length[num_nalus] = nal_size; + array->NAL_unit_type = nal_type; + array->num_nalus++; + + /* + * When the sample entry name is ‘vvc1’, the default and mandatory value of + * array_completeness is 1 for arrays of all types of parameter sets, and 0 + * for all other arrays. When the sample entry name is ‘hev1’, the default + * value of array_completeness is 0 for all arrays. + */ + if (nal_type == VVC_VPS_NUT || nal_type == VVC_SPS_NUT || nal_type == VVC_PPS_NUT) + array->array_completeness = ps_array_completeness; + + return 0; +} + +static int vvcc_add_nal_unit(uint8_t *nal_buf, uint32_t nal_size, + int ps_array_completeness, + VVCDecoderConfigurationRecord *vvcc) +{ + int ret = 0; + GetBitContext gbc; + uint8_t nal_type; + uint8_t *rbsp_buf; + uint32_t rbsp_size; + + rbsp_buf = ff_nal_unit_extract_rbsp(nal_buf, nal_size, &rbsp_size, 2); + if (!rbsp_buf) { + ret = AVERROR(ENOMEM); + goto end; + } + + ret = init_get_bits8(&gbc, rbsp_buf, rbsp_size); + if (ret < 0) + goto end; + + nal_unit_parse_header(&gbc, &nal_type); + + /* + * Note: only 'declarative' SEI messages are allowed in + * vvcc. Perhaps the SEI playload type should be checked + * and non-declarative SEI messages discarded? + */ + switch (nal_type) { + case VVC_OPI_NUT: + case VVC_VPS_NUT: + case VVC_SPS_NUT: + case VVC_PPS_NUT: + case VVC_PREFIX_SEI_NUT: + case VVC_SUFFIX_SEI_NUT: + ret = vvcc_array_add_nal_unit(nal_buf, nal_size, nal_type, + ps_array_completeness, vvcc); + if (ret < 0) + goto end; + else if (nal_type == VVC_VPS_NUT) + ret = vvcc_parse_vps(&gbc, vvcc); + else if (nal_type == VVC_SPS_NUT) + ret = vvcc_parse_sps(&gbc, vvcc); + else if (nal_type == VVC_PPS_NUT) + ret = vvcc_parse_pps(&gbc, vvcc); + else if (nal_type == VVC_OPI_NUT) { + // not yet supported + } + if (ret < 0) + goto end; + break; + default: + ret = AVERROR_INVALIDDATA; + goto end; + } + +end: + av_free(rbsp_buf); + return ret; +} + +static void vvcc_init(VVCDecoderConfigurationRecord *vvcc) +{ + memset(vvcc, 0, sizeof(VVCDecoderConfigurationRecord)); + vvcc->lengthSizeMinusOne = 3; // 4 bytes + + vvcc->ptl.num_bytes_constraint_info = 1; + + vvcc->ptl_present_flag = 1; +} + +static void vvcc_close(VVCDecoderConfigurationRecord *vvcc) +{ + uint8_t i; + + for (i = 0; i < vvcc->num_of_arrays; i++) { + vvcc->array[i].num_nalus = 0; + av_freep(&vvcc->array[i].nal_unit); + av_freep(&vvcc->array[i].nal_unit_length); + } + + free(vvcc->ptl.ptl_sublayer_level_present_flag); + free(vvcc->ptl.sublayer_level_idc); + free(vvcc->ptl.general_sub_profile_idc); + + vvcc->num_of_arrays = 0; + av_freep(&vvcc->array); +} + +static int vvcc_write(AVIOContext *pb, VVCDecoderConfigurationRecord *vvcc) +{ + uint8_t i; + uint16_t j, vps_count = 0, sps_count = 0, pps_count = 0; + unsigned char *buf = NULL; + /* + * It's unclear how to properly compute these fields, so + * let's always set them to values meaning 'unspecified'. + */ + vvcc->avg_frame_rate = 0; + vvcc->constant_frame_rate = 1; + + av_log(NULL, AV_LOG_TRACE, "lengthSizeMinusOne: %"PRIu8"\n", + vvcc->lengthSizeMinusOne); + av_log(NULL, AV_LOG_TRACE, "ptl_present_flag: %"PRIu8"\n", + vvcc->ptl_present_flag); + av_log(NULL, AV_LOG_TRACE, "ols_idx: %"PRIu16"\n", + vvcc->ols_idx); + av_log(NULL, AV_LOG_TRACE, "num_sublayers: %"PRIu8"\n", + vvcc->num_sublayers); + av_log(NULL, AV_LOG_TRACE, "constant_frame_rate: %"PRIu8"\n", + vvcc->constant_frame_rate); + av_log(NULL, AV_LOG_TRACE, "chroma_format_idc: %"PRIu8"\n", + vvcc->chroma_format_idc); + + av_log(NULL, AV_LOG_TRACE, "bit_depth_minus8: %"PRIu8"\n", + vvcc->bit_depth_minus8); + av_log(NULL, AV_LOG_TRACE, "num_bytes_constraint_info: %"PRIu8"\n", + vvcc->ptl.num_bytes_constraint_info); + av_log(NULL, AV_LOG_TRACE, "general_profile_idc: %"PRIu8"\n", + vvcc->ptl.general_profile_idc); + av_log(NULL, AV_LOG_TRACE, "general_tier_flag: %"PRIu8"\n", + vvcc->ptl.general_tier_flag); + av_log(NULL, AV_LOG_TRACE, "general_level_idc: %"PRIu8"\n", + vvcc->ptl.general_level_idc); + av_log(NULL, AV_LOG_TRACE, "ptl_frame_only_constraint_flag: %"PRIu8"\n", + vvcc->ptl.ptl_frame_only_constraint_flag); + av_log(NULL, AV_LOG_TRACE, "ptl_multilayer_enabled_flag: %"PRIu8"\n", + vvcc->ptl.ptl_multilayer_enabled_flag); + for (i = 0; i < vvcc->ptl.num_bytes_constraint_info; i++) { + av_log(NULL, AV_LOG_TRACE, "general_constraint_info[%d]: %"PRIu8"\n", + i, vvcc->ptl.general_constraint_info[i]); + } + + for (i = 0; i < vvcc->num_sublayers-1; i++) { + av_log(NULL, AV_LOG_TRACE, "ptl_sublayer_level_present_flag[%"PRIu8"]: %"PRIu8"\n", + i, vvcc->ptl.ptl_sublayer_level_present_flag[i]); + av_log(NULL, AV_LOG_TRACE, "sublayer_level_idc[%"PRIu8"]: %"PRIu8"\n", + i, vvcc->ptl.sublayer_level_idc[i]); + } + + av_log(NULL, AV_LOG_TRACE, "num_sub_profiles: %"PRIu8"\n", + vvcc->ptl.ptl_num_sub_profiles); + + for (i = 0; i < vvcc->ptl.ptl_num_sub_profiles; i++) { + av_log(NULL, AV_LOG_TRACE, "general_sub_profile_idc[%"PRIu8"]: %"PRIx32"\n", + i, vvcc->ptl.general_sub_profile_idc[i]); + } + + av_log(NULL, AV_LOG_TRACE, "max_picture_width: %"PRIu16"\n", + vvcc->max_picture_width); + av_log(NULL, AV_LOG_TRACE, "max_picture_height: %"PRIu16"\n", + vvcc->max_picture_height); + av_log(NULL, AV_LOG_TRACE, "avg_frame_rate: %"PRIu16"\n", + vvcc->avg_frame_rate); + + av_log(NULL, AV_LOG_TRACE, "num_of_arrays: %"PRIu8"\n", + vvcc->num_of_arrays); + for (i = 0; i < vvcc->num_of_arrays; i++) { + av_log(NULL, AV_LOG_TRACE, "array_completeness[%"PRIu8"]: %"PRIu8"\n", + i, vvcc->array[i].array_completeness); + av_log(NULL, AV_LOG_TRACE, "NAL_unit_type[%"PRIu8"]: %"PRIu8"\n", + i, vvcc->array[i].NAL_unit_type); + av_log(NULL, AV_LOG_TRACE, "num_nalus[%"PRIu8"]: %"PRIu16"\n", + i, vvcc->array[i].num_nalus); + for (j = 0; j < vvcc->array[i].num_nalus; j++) + av_log(NULL, AV_LOG_TRACE, + "nal_unit_length[%"PRIu8"][%"PRIu16"]: %"PRIu16"\n", + i, j, vvcc->array[i].nal_unit_length[j]); + } + + /* + * We need at least one of each: VPS and SPS. + */ + for (i = 0; i < vvcc->num_of_arrays; i++) + switch (vvcc->array[i].NAL_unit_type) { + case VVC_VPS_NUT: + vps_count += vvcc->array[i].num_nalus; + break; + case VVC_SPS_NUT: + sps_count += vvcc->array[i].num_nalus; + break; + case VVC_PPS_NUT: + pps_count += vvcc->array[i].num_nalus; + break; + default: + break; + } + + if (!sps_count || sps_count > VVC_MAX_SPS_COUNT) + return AVERROR_INVALIDDATA; + + /* bit(5) reserved = ‘11111’b; + unsigned int (2) LengthSizeMinusOne + unsigned int (1) ptl_present_flag */ + avio_w8(pb, vvcc->lengthSizeMinusOne << 1 | vvcc->ptl_present_flag | 0xf8); + + if(vvcc->ptl_present_flag) { + /* + * unsigned int(9) ols_idx; + * unsigned int(3) num_sublayers; + * unsigned int(2) constant_frame_rate; + * unsigned int(2) chroma_format_idc; */ + avio_wb16(pb, vvcc->ols_idx << 7 | vvcc->num_sublayers << 4 | vvcc->constant_frame_rate << 2 | vvcc->chroma_format_idc); + + /* unsigned int(3) bit_depth_minus8; + bit(5) reserved = ‘11111’b;*/ + avio_w8(pb, vvcc->bit_depth_minus8 << 5 | 0x1f); + + //VVCPTLRecord + + /* bit(2) reserved = ‘00’b; + unsigned int (6) num_bytes_constraint_info */ + avio_w8(pb, vvcc->ptl.num_bytes_constraint_info & 0x3f); + + /* unsigned int (7) general_profile_idc + unsigned int (1) general_tier_flag */ + avio_w8(pb, vvcc->ptl.general_profile_idc << 1 | vvcc->ptl.general_tier_flag); + + /* unsigned int (8) general_level_idc */ + avio_w8(pb, vvcc->ptl.general_level_idc); + + /* + * unsigned int (1) ptl_frame_only_constraint_flag + * unsigned int (1) ptl_multilayer_enabled_flag + * unsigned int (8*num_bytes_constraint_info -2) general_constraint_info */ + buf = (unsigned char *) malloc(sizeof(unsigned char) * vvcc->ptl.num_bytes_constraint_info); + *buf = vvcc->ptl.ptl_frame_only_constraint_flag << vvcc->ptl.num_bytes_constraint_info*8-1 | vvcc->ptl.ptl_multilayer_enabled_flag << vvcc->ptl.num_bytes_constraint_info*8-2 | *vvcc->ptl.general_constraint_info >> 2; + avio_write(pb, buf, vvcc->ptl.num_bytes_constraint_info); + free(buf); + + if(vvcc->num_sublayers > 1) { + uint8_t ptl_sublayer_level_present_flags = 0; + for(int i = vvcc->num_sublayers -2; i >= 0; i--) { + ptl_sublayer_level_present_flags = (ptl_sublayer_level_present_flags << 1 | vvcc->ptl.ptl_sublayer_level_present_flag[i]); + } + avio_w8(pb, ptl_sublayer_level_present_flags); + } + + for(int i = vvcc->num_sublayers -2; i >= 0; i--) { + if(vvcc->ptl.ptl_sublayer_level_present_flag[i]) + avio_w8(pb, vvcc->ptl.sublayer_level_idc[i]); + } + + /* unsigned int(8) num_sub_profiles; */ + avio_w8(pb, vvcc->ptl.ptl_num_sub_profiles); + + for(int j = 0; j < vvcc->ptl.ptl_num_sub_profiles; j++) { + /* unsigned int(32) general_sub_profile_idc[j]; */ + avio_wb32(pb, vvcc->ptl.general_sub_profile_idc[j]); + } + + //End of VvcPTLRecord + + /* + * unsigned int(16) max_picture_width;*/ + avio_wb16(pb, vvcc->max_picture_width); + + /* + * unsigned int(16) max_picture_height;*/ + avio_wb16(pb, vvcc->max_picture_height); + + /* + * unsigned int(16) avg_frame_rate; */ + avio_wb16(pb, vvcc->avg_frame_rate); + } + + /* unsigned int(8) num_of_arrays; */ + avio_w8(pb, vvcc->num_of_arrays); + + for (i = 0; i < vvcc->num_of_arrays; i++) { + /* + * bit(1) array_completeness; + * unsigned int(2) reserved = 0; + * unsigned int(5) NAL_unit_type; + */ + avio_w8(pb, vvcc->array[i].array_completeness << 7 | + vvcc->array[i].NAL_unit_type & 0x1f); + /* unsigned int(16) num_nalus; */ + if(vvcc->array[i].NAL_unit_type != VVC_DCI_NUT && vvcc->array[i].NAL_unit_type != VVC_OPI_NUT) + avio_wb16(pb, vvcc->array[i].num_nalus); + for (j = 0; j < vvcc->array[i].num_nalus; j++) { + /* unsigned int(16) nal_unit_length; */ + avio_wb16(pb, vvcc->array[i].nal_unit_length[j]); + + /* bit(8*nal_unit_length) nal_unit; */ + avio_write(pb, vvcc->array[i].nal_unit[j], vvcc->array[i].nal_unit_length[j]); + } + } + + return 0; +} + +int ff_vvc_annexb2mp4(AVIOContext *pb, const uint8_t *buf_in, + int size, int filter_ps, int *ps_count) +{ + int num_ps = 0, ret = 0; + uint8_t *buf, *end, *start = NULL; + + if (!filter_ps) { + ret = ff_avc_parse_nal_units(pb, buf_in, size); + goto end; + } + + ret = ff_avc_parse_nal_units_buf(buf_in, &start, &size); + if (ret < 0) + goto end; + + ret = 0; + buf = start; + end = start + size; + + while (end - buf > 4) { + uint32_t len = FFMIN(AV_RB32(buf), end - buf - 4); + uint8_t type = (buf[5] >> 3); + + buf += 4; + + switch (type) { + case VVC_VPS_NUT: + case VVC_SPS_NUT: + case VVC_PPS_NUT: + num_ps++; + break; + default: + ret += 4 + len; + avio_wb32(pb, len); + avio_write(pb, buf, len); + break; + } + + buf += len; + } + +end: + av_free(start); + if (ps_count) + *ps_count = num_ps; + return ret; +} + +int ff_vvc_annexb2mp4_buf(const uint8_t *buf_in, uint8_t **buf_out, + int *size, int filter_ps, int *ps_count) +{ + AVIOContext *pb; + int ret; + + ret = avio_open_dyn_buf(&pb); + if (ret < 0) + return ret; + + ret = ff_vvc_annexb2mp4(pb, buf_in, *size, filter_ps, ps_count); + if (ret < 0) { + ffio_free_dyn_buf(&pb); + return ret; + } + + *size = avio_close_dyn_buf(pb, buf_out); + + return 0; +} + +int ff_isom_write_vvcc(AVIOContext *pb, const uint8_t *data, + int size, int ps_array_completeness) +{ + VVCDecoderConfigurationRecord vvcc; + uint8_t *buf, *end, *start; + int ret; + + if (size < 6) { + /* We can't write a valid vvcc from the provided data */ + return AVERROR_INVALIDDATA; + } else if (*data == 1) { + /* Data is already vvcc-formatted */ + avio_write(pb, data, size); + return 0; + } else if (!(AV_RB24(data) == 1 || AV_RB32(data) == 1)) { + /* Not a valid Annex B start code prefix */ + return AVERROR_INVALIDDATA; + } + + ret = ff_avc_parse_nal_units_buf(data, &start, &size); + if (ret < 0) + return ret; + + vvcc_init(&vvcc); + + buf = start; + end = start + size; + + while (end - buf > 4) { + uint32_t len = FFMIN(AV_RB32(buf), end - buf - 4); + uint8_t type = (buf[5] >> 3); + + buf += 4; + + switch (type) { + case VVC_OPI_NUT: + case VVC_VPS_NUT: + case VVC_SPS_NUT: + case VVC_PPS_NUT: + case VVC_PREFIX_SEI_NUT: + case VVC_SUFFIX_SEI_NUT: + ret = vvcc_add_nal_unit(buf, len, ps_array_completeness, &vvcc); + if (ret < 0) + goto end; + break; + default: + break; + } + + buf += len; + } + + ret = vvcc_write(pb, &vvcc); + +end: + vvcc_close(&vvcc); + av_free(start); + return ret; +} diff --git a/libavformat/vvc.h b/libavformat/vvc.h new file mode 100644 index 0000000000..cdf4f6d3f5 --- /dev/null +++ b/libavformat/vvc.h @@ -0,0 +1,99 @@ +/* + * VVC helper functions for muxers + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * internal header for VVC (de)muxer utilities + */ + +#ifndef AVFORMAT_VVC_H +#define AVFORMAT_VVC_H + +#include +#include "avio.h" + +/** + * Writes Annex B formatted VVC NAL units to the provided AVIOContext. + * + * The NAL units are converted to an MP4-compatible format (start code prefixes + * are replaced by 4-byte size fields, as per ISO/IEC 14496-15). + * + * If filter_ps is non-zero, any VVC parameter sets found in the input will be + * discarded, and *ps_count will be set to the number of discarded PS NAL units. + * + * @param pb address of the AVIOContext where the data shall be written + * @param buf_in address of the buffer holding the input data + * @param size size (in bytes) of the input buffer + * @param filter_ps whether to write parameter set NAL units to the output (0) + * or to discard them (non-zero) + * @param ps_count address of the variable where the number of discarded + * parameter set NAL units shall be written, may be NULL + * @return the amount (in bytes) of data written in case of success, a negative + * value corresponding to an AVERROR code in case of failure + */ +int ff_vvc_annexb2mp4(AVIOContext *pb, const uint8_t *buf_in, + int size, int filter_ps, int *ps_count); + +/** + * Writes Annex B formatted VVC NAL units to a data buffer. + * + * The NAL units are converted to an MP4-compatible format (start code prefixes + * are replaced by 4-byte size fields, as per ISO/IEC 14496-15). + * + * If filter_ps is non-zero, any VVC parameter sets found in the input will be + * discarded, and *ps_count will be set to the number of discarded PS NAL units. + * + * On success, *size holds the size (in bytes) of the output data buffer. + * + * @param buf_in address of the buffer holding the input data + * @param size address of the variable holding the size (in bytes) of the input + * buffer (on input) and of the output buffer (on success) + * @param buf_out on success, address of the variable holding the address of + * the output buffer + * @param filter_ps whether to write parameter set NAL units to the output (0) + * or to discard them (non-zero) + * @param ps_count address of the variable where the number of discarded + * parameter set NAL units shall be written, may be NULL + * @return 0 in case of success, a negative value corresponding to an AVERROR + * code in case of failure + * @note *buf_out will be treated as uninitialized on input and won't be freed. + */ +int ff_vvc_annexb2mp4_buf(const uint8_t *buf_in, uint8_t **buf_out, + int *size, int filter_ps, int *ps_count); + +/** + * Writes VVC extradata (parameter sets, declarative SEI NAL units) to the + * provided AVIOContext. + * + * If the extradata is Annex B format, it gets converted to vvcC format before + * writing. + * + * @param pb address of the AVIOContext where the vvcC shall be written + * @param data address of the buffer holding the data needed to write the vvcC + * @param size size (in bytes) of the data buffer + * @param ps_array_completeness whether all parameter sets are in the vvcC (1) + * or there may be additional parameter sets in the bitstream (0) + * @return >=0 in case of success, a negative value corresponding to an AVERROR + * code in case of failure + */ +int ff_isom_write_vvcc(AVIOContext *pb, const uint8_t *data, + int size, int ps_array_completeness); + +#endif /* AVFORMAT_VVC_H */ diff --git a/libavformat/vvcdec.c b/libavformat/vvcdec.c new file mode 100644 index 0000000000..304f900966 --- /dev/null +++ b/libavformat/vvcdec.c @@ -0,0 +1,61 @@ +/* + * RAW VVC video demuxer + * Copyright (c) 2020 Nuo Mi + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavcodec/vvc.h" + +#include "avformat.h" +#include "rawdec.h" + +static int vvc_probe(const AVProbeData *p) +{ + uint32_t code = -1; + int sps = 0, pps = 0, irap = 0; + int i; + + for (i = 0; i < p->buf_size - 1; i++) { + code = (code << 8) + p->buf[i]; + if ((code & 0xffffff00) == 0x100) { + uint8_t nal2 = p->buf[i + 1]; + int type = (nal2 & 0xF8) >> 3; + + if (code & 0x80) // forbidden_zero_bit + return 0; + + if ((nal2 & 0x7) == 0) // nuh_temporal_id_plus1 + return 0; + + switch (type) { + case VVC_SPS_NUT: sps++; break; + case VVC_PPS_NUT: pps++; break; + case VVC_IDR_N_LP: + case VVC_IDR_W_RADL: + case VVC_CRA_NUT: + case VVC_GDR_NUT: irap++; break; + } + } + } + + if (sps && pps && irap) + return AVPROBE_SCORE_EXTENSION + 1; // 1 more than .mpg + return 0; +} + +FF_DEF_RAWVIDEO_DEMUXER(vvc, "raw VVC video", vvc_probe, "h266,266,vvc", AV_CODEC_ID_VVC) From patchwork Mon Oct 24 14:06:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Siedel X-Patchwork-Id: 38964 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2249585pzb; Mon, 24 Oct 2022 07:09:06 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5ETfR69BkysL1CF2jQRRu/jBghmRigQawLYWRK6HiuU9TbbTKmpV9NWVgLPovzm+oF4dqd X-Received: by 2002:a17:906:8a4a:b0:78d:5ff6:7507 with SMTP id gx10-20020a1709068a4a00b0078d5ff67507mr27580893ejc.194.1666620545919; Mon, 24 Oct 2022 07:09:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666620545; cv=none; d=google.com; s=arc-20160816; b=V70ASMTs7DAO32Bn71k7+7rzMxz54xTOtZxjkLiBEcZ+inqrDPo00ULXyNOtpqOooA 51rNvWtgo3NccJqIYQY2BKcEvqysMfPJdm0PPG0gMu2WEKJcZe7S8VvOwfgJaWuCA3AY v6scUPO1kFP4T1TfXxyzAyXo0OZRL+50sXITeT18WnWdLRFssnBTkVxGsduH5RY1PNoC i2zs3ZK/5k4D9+j2M/kaJuqNK3O/ZMX+KeqVTiNA0U26q5iEDJIh5vms3XwDAVtWfJiy xHJbOyrZdYBGDScRPhvKkMvCApcqxCsoTImlW1HFFu7OUtgxYAfax2g6CWqjsn0Ch9B3 LtiA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=I+dezETIAYo2iKi9FmD5XWY/sL20nGM0KElKeEQh228=; b=u+SneJqkNXuxnUU7dC9FsdnmO45Yb26ZX0Tc4hpRzaRlhz6veLVOn4i84uESQphocj jCTTlYlTPIJhdpzofNEam7MZZ27GYfIGfvAEE5nI/CcD57or28ad1nZS/qqvPRUQcqTm 5G/lQAA0yVNVb1jcC0NvXCd1n/EyEye0DHPmkYt7kqj1zmzFDDc3eR6NzcZbidkmvfV6 PBUmmFRNKFsazUIHzpRVp7pm2qKsRsXAfmQGw13LQR0GSSnKfkjJW/5kUt4hjfEWV9sC 0IizpJmQHAFJLOwDGrozDaD1Y810ny7Z2dRTNJRVUc+evUfIhsQrZlikkLWXXkjrnNll ZkZw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=Ky3PpbXW; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id sh40-20020a1709076ea800b0078d288c1047si27864143ejc.841.2022.10.24.07.09.05; Mon, 24 Oct 2022 07:09:05 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=Ky3PpbXW; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7B1CA68BBFE; Mon, 24 Oct 2022 17:07:51 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f45.google.com (mail-ed1-f45.google.com [209.85.208.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id DCD1B68BC17 for ; Mon, 24 Oct 2022 17:07:41 +0300 (EEST) Received: by mail-ed1-f45.google.com with SMTP id b12so31767697edd.6 for ; Mon, 24 Oct 2022 07:07:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spin-digital-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QX78N43I566M9tXtnMI1jkReABMi3UXxhaW8slDxvAc=; b=Ky3PpbXW2H2iT5tuI17CIxhkX50QaNMhHL+xIEKnJJTZh39oYvO6MtbSOD2j8o2L4x BdaONEUiMLIxAu/uqyj+WW8b5fmxHWFbQWFmcOd7WPsrI/Hq820+XYqKoAw5gtwKOqoU dQANTGxzn1MiVhUJ/CkypbZQqx+tqlazNCf0Vm+zq4j+6UxACmQt8Dtv7A3/POZ+SW8E 9sTEyr1HNdLLroW4147Tl6bZVOGyWAuj3V91+sfvtA7UAA7ntitUWj2XHn4SCzolCgFK m16A29FpB8PSwdJX/Bn0lTO9M0Uup5w/KsJWGK/cJBdqMvZCO+AmnUcqZKDBggJVcpMq EFsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QX78N43I566M9tXtnMI1jkReABMi3UXxhaW8slDxvAc=; b=maJlKQK1ogK2cFtYrVW616CTENOib69/0dVDARs/EjyhKTQnAa3VrPPGXELUg/ngT3 KmU1jAzryzi9KiFS0fdNXvu6BL36N5zB1DruUTcDyJM9clgbbttz6LR+ou+zlOT9/iMy OC0gSGjuRfG6Y0nS0fJs7YAH0SfEvJUt5ykq2aiLcJdKCqoSiqqYzEKMUy498JUD4OJE NMe7M43JRS0nNXQLYIlcBVVfflDRUygtpXKRT4CHld/IxUGJcPWWO2yip02Wgx8XbjzJ jIc8mm7j8UhS+zKI5rEgrhHiXnDH+qOhevYbxZDrYkjwNMWoIXqARLdDnV5/RpZiLWb8 NswA== X-Gm-Message-State: ACrzQf0Wd7FB8hUw9ZJ8KcuSuBq20Z4wIAHXVrm0EVmPwzo4yBb0y9MM Cd4RbB++St9jSiBFkFwxtLz1am3lvN8rsQ== X-Received: by 2002:a05:6402:448c:b0:457:52eb:b57e with SMTP id er12-20020a056402448c00b0045752ebb57emr31449383edb.178.1666620460916; Mon, 24 Oct 2022 07:07:40 -0700 (PDT) Received: from thomas-win.localdomain ([213.138.44.237]) by smtp.gmail.com with ESMTPSA id s19-20020a170906455300b0079800b81709sm6056436ejq.219.2022.10.24.07.07.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Oct 2022 07:07:40 -0700 (PDT) From: Thomas Siedel To: ffmpeg-devel@ffmpeg.org Date: Mon, 24 Oct 2022 16:06:41 +0200 Message-Id: <20221024140645.4945-7-thomas.ff@spin-digital.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221024140645.4945-1-thomas.ff@spin-digital.com> References: <20221024140645.4945-1-thomas.ff@spin-digital.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 06/10] avformat: add muxer support for H266/VVC X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Thomas Siedel Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: cR4bYcys2oZC Add muxer for vvcc byte stream format. Add AV_CODEC_ID_VVC to ff_mp4_obj_type Add AV_CODEC_ID_VVC to ISO Media codec (VvcConfigurationBox vvi1, vvc1 defined in ISO/IEC 14496-15:2021) Add VvcConfigurationBox vvcC which extends FullBox type in ISO/IEC 14496-15:2021 Add ff_vvc_muxer to RAW muxers Signed-off-by: Thomas Siedel --- libavformat/Makefile | 7 ++++--- libavformat/allformats.c | 1 + libavformat/isom.c | 1 + libavformat/isom_tags.c | 3 +++ libavformat/mov.c | 6 ++++++ libavformat/movenc.c | 41 +++++++++++++++++++++++++++++++++++++++- libavformat/rawenc.c | 23 ++++++++++++++++++++++ 7 files changed, 78 insertions(+), 4 deletions(-) diff --git a/libavformat/Makefile b/libavformat/Makefile index 00ab4ded89..9ee2526eef 100644 --- a/libavformat/Makefile +++ b/libavformat/Makefile @@ -336,7 +336,7 @@ OBJS-$(CONFIG_MATROSKA_DEMUXER) += matroskadec.o matroska.o \ oggparsevorbis.o vorbiscomment.o \ qtpalette.o replaygain.o dovi_isom.o OBJS-$(CONFIG_MATROSKA_MUXER) += matroskaenc.o matroska.o \ - av1.o avc.o hevc.o \ + av1.o avc.o hevc.o vvc.o\ flacenc_header.o avlanguage.o \ vorbiscomment.o wv.o dovi_isom.o OBJS-$(CONFIG_MCA_DEMUXER) += mca.o @@ -358,7 +358,7 @@ OBJS-$(CONFIG_MODS_DEMUXER) += mods.o OBJS-$(CONFIG_MOFLEX_DEMUXER) += moflex.o OBJS-$(CONFIG_MOV_DEMUXER) += mov.o mov_chan.o mov_esds.o \ qtpalette.o replaygain.o dovi_isom.o -OBJS-$(CONFIG_MOV_MUXER) += movenc.o av1.o avc.o hevc.o vpcc.o \ +OBJS-$(CONFIG_MOV_MUXER) += movenc.o av1.o avc.o hevc.o vvc.o vpcc.o \ movenchint.o mov_chan.o rtp.o \ movenccenc.o movenc_ttml.o rawutils.o \ dovi_isom.o @@ -508,7 +508,7 @@ OBJS-$(CONFIG_RTP_MUXER) += rtp.o \ rtpenc_vp8.o \ rtpenc_vp9.o \ rtpenc_xiph.o \ - avc.o hevc.o + avc.o hevc.o vvc.o OBJS-$(CONFIG_RTSP_DEMUXER) += rtsp.o rtspdec.o httpauth.o \ urldecode.o OBJS-$(CONFIG_RTSP_MUXER) += rtsp.o rtspenc.o httpauth.o \ @@ -596,6 +596,7 @@ OBJS-$(CONFIG_VPK_DEMUXER) += vpk.o OBJS-$(CONFIG_VPLAYER_DEMUXER) += vplayerdec.o subtitles.o OBJS-$(CONFIG_VQF_DEMUXER) += vqf.o OBJS-$(CONFIG_VVC_DEMUXER) += vvcdec.o rawdec.o +OBJS-$(CONFIG_VVC_MUXER) += rawenc.o OBJS-$(CONFIG_W64_DEMUXER) += wavdec.o w64.o pcm.o OBJS-$(CONFIG_W64_MUXER) += wavenc.o w64.o OBJS-$(CONFIG_WAV_DEMUXER) += wavdec.o pcm.o diff --git a/libavformat/allformats.c b/libavformat/allformats.c index a4e3822681..006a7dc125 100644 --- a/libavformat/allformats.c +++ b/libavformat/allformats.c @@ -475,6 +475,7 @@ extern const AVInputFormat ff_vpk_demuxer; extern const AVInputFormat ff_vplayer_demuxer; extern const AVInputFormat ff_vqf_demuxer; extern const AVInputFormat ff_vvc_demuxer; +extern const AVOutputFormat ff_vvc_muxer; extern const AVInputFormat ff_w64_demuxer; extern const AVOutputFormat ff_w64_muxer; extern const AVInputFormat ff_wav_demuxer; diff --git a/libavformat/isom.c b/libavformat/isom.c index 6d019881e5..9fbccd4437 100644 --- a/libavformat/isom.c +++ b/libavformat/isom.c @@ -36,6 +36,7 @@ const AVCodecTag ff_mp4_obj_type[] = { { AV_CODEC_ID_MPEG4 , 0x20 }, { AV_CODEC_ID_H264 , 0x21 }, { AV_CODEC_ID_HEVC , 0x23 }, + { AV_CODEC_ID_VVC , 0x33 }, { AV_CODEC_ID_AAC , 0x40 }, { AV_CODEC_ID_MP4ALS , 0x40 }, /* 14496-3 ALS */ { AV_CODEC_ID_MPEG2VIDEO , 0x61 }, /* MPEG-2 Main */ diff --git a/libavformat/isom_tags.c b/libavformat/isom_tags.c index e2b80405cc..ec93bdc363 100644 --- a/libavformat/isom_tags.c +++ b/libavformat/isom_tags.c @@ -123,6 +123,9 @@ const AVCodecTag ff_codec_movvideo_tags[] = { { AV_CODEC_ID_HEVC, MKTAG('d', 'v', 'h', 'e') }, /* HEVC-based Dolby Vision derived from hev1 */ /* dvh1 is handled within mov.c */ + { AV_CODEC_ID_VVC, MKTAG('v', 'v', 'i', '1') }, /* VVC/H.266 which indicates parameter sets may be in ES */ + { AV_CODEC_ID_VVC, MKTAG('v', 'v', 'c', '1') }, /* VVC/H.266 which indicates parameter shall not be in ES */ + { AV_CODEC_ID_H264, MKTAG('a', 'v', 'c', '1') }, /* AVC-1/H.264 */ { AV_CODEC_ID_H264, MKTAG('a', 'v', 'c', '2') }, { AV_CODEC_ID_H264, MKTAG('a', 'v', 'c', '3') }, diff --git a/libavformat/mov.c b/libavformat/mov.c index 1f436e21d6..52719c9d9a 100644 --- a/libavformat/mov.c +++ b/libavformat/mov.c @@ -1962,6 +1962,11 @@ static int mov_read_glbl(MOVContext *c, AVIOContext *pb, MOVAtom atom) if ((uint64_t)atom.size > (1<<30)) return AVERROR_INVALIDDATA; + if (atom.type == MKTAG('v','v','c','C')) { + avio_rb32(pb); + atom.size -= 4; + } + if (atom.size >= 10) { // Broken files created by legacy versions of libavformat will // wrap a whole fiel atom inside of a glbl atom. @@ -7730,6 +7735,7 @@ static const MOVParseTableEntry mov_default_parse_table[] = { { MKTAG('s','g','p','d'), mov_read_sgpd }, { MKTAG('s','b','g','p'), mov_read_sbgp }, { MKTAG('h','v','c','C'), mov_read_glbl }, +{ MKTAG('v','v','c','C'), mov_read_glbl }, { MKTAG('u','u','i','d'), mov_read_uuid }, { MKTAG('C','i','n', 0x8e), mov_read_targa_y216 }, { MKTAG('f','r','e','e'), mov_read_free }, diff --git a/libavformat/movenc.c b/libavformat/movenc.c index 754f95912a..3d412d0079 100644 --- a/libavformat/movenc.c +++ b/libavformat/movenc.c @@ -59,6 +59,7 @@ #include "libavutil/color_utils.h" #include "libavutil/uuid.h" #include "hevc.h" +#include "vvc.h" #include "rtpenc.h" #include "mov_chan.h" #include "movenc_ttml.h" @@ -1392,6 +1393,23 @@ static int mov_write_hvcc_tag(AVIOContext *pb, MOVTrack *track) return update_size(pb, pos); } +static int mov_write_vvcc_tag(AVIOContext *pb, MOVTrack *track) +{ + int64_t pos = avio_tell(pb); + + avio_wb32(pb, 0); + ffio_wfourcc(pb, "vvcC"); + + avio_w8 (pb, 0); /* version */ + avio_wb24(pb, 0); /* flags */ + + if (track->tag == MKTAG('v','v','i','1')) + ff_isom_write_vvcc(pb, track->vos_data, track->vos_len, 1); + else + ff_isom_write_vvcc(pb, track->vos_data, track->vos_len, 0); + return update_size(pb, pos); +} + /* also used by all avid codecs (dv, imx, meridien) and their variants */ static int mov_write_avid_tag(AVIOContext *pb, MOVTrack *track) { @@ -2286,6 +2304,8 @@ static int mov_write_video_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContex avid = 1; } else if (track->par->codec_id == AV_CODEC_ID_HEVC) mov_write_hvcc_tag(pb, track); + else if (track->par->codec_id == AV_CODEC_ID_VVC) + mov_write_vvcc_tag(pb, track); else if (track->par->codec_id == AV_CODEC_ID_H264 && !TAG_IS_AVCI(track->tag)) { mov_write_avcc_tag(pb, track); if (track->mode == MODE_IPOD) @@ -6033,6 +6053,7 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) if ((par->codec_id == AV_CODEC_ID_DNXHD || par->codec_id == AV_CODEC_ID_H264 || par->codec_id == AV_CODEC_ID_HEVC || + par->codec_id == AV_CODEC_ID_VVC || par->codec_id == AV_CODEC_ID_TRUEHD) && !trk->vos_len && !TAG_IS_AVCI(trk->tag)) { /* copy frame to create needed atoms */ @@ -6096,6 +6117,18 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) size = ff_hevc_annexb2mp4(pb, pkt->data, pkt->size, 0, NULL); } } + } else if (par->codec_id == AV_CODEC_ID_VVC && trk->vos_len > 6 && + (AV_RB24(trk->vos_data) == 1 || AV_RB32(trk->vos_data) == 1)) { + /* extradata is Annex B, assume the bitstream is too and convert it */ + if (trk->hint_track >= 0 && trk->hint_track < mov->nb_streams) { + ret = ff_vvc_annexb2mp4_buf(pkt->data, &reformatted_data, + &size, 0, NULL); + if (ret < 0) + return ret; + avio_write(pb, reformatted_data, size); + } else { + size = ff_vvc_annexb2mp4(pb, pkt->data, pkt->size, 0, NULL); + } } else if (par->codec_id == AV_CODEC_ID_AV1) { if (trk->hint_track >= 0 && trk->hint_track < mov->nb_streams) { ret = ff_av1_filter_obus_buf(pkt->data, &reformatted_data, @@ -6142,6 +6175,9 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) } else if(par->codec_id == AV_CODEC_ID_HEVC && par->extradata_size > 21) { int nal_size_length = (par->extradata[21] & 0x3) + 1; ret = ff_mov_cenc_avc_write_nal_units(s, &trk->cenc, nal_size_length, pb, pkt->data, size); + } else if(par->codec_id == AV_CODEC_ID_VVC && par->extradata_size > 21) { + int nal_size_length = (par->extradata[21] & 0x3) + 1; + ret = ff_mov_cenc_avc_write_nal_units(s, &trk->cenc, nal_size_length, pb, pkt->data, size); } else { ret = ff_mov_cenc_write_packet(&trk->cenc, pb, pkt->data, size); } @@ -7224,7 +7260,8 @@ static int mov_init(AVFormatContext *s) if (mov->encryption_scheme == MOV_ENC_CENC_AES_CTR) { ret = ff_mov_cenc_init(&track->cenc, mov->encryption_key, - (track->par->codec_id == AV_CODEC_ID_H264 || track->par->codec_id == AV_CODEC_ID_HEVC), + (track->par->codec_id == AV_CODEC_ID_H264 || track->par->codec_id == AV_CODEC_ID_HEVC || + track->par->codec_id == AV_CODEC_ID_VVC), s->flags & AVFMT_FLAG_BITEXACT); if (ret) return ret; @@ -7692,6 +7729,8 @@ static const AVCodecTag codec_mp4_tags[] = { { AV_CODEC_ID_H264, MKTAG('a', 'v', 'c', '3') }, { AV_CODEC_ID_HEVC, MKTAG('h', 'e', 'v', '1') }, { AV_CODEC_ID_HEVC, MKTAG('h', 'v', 'c', '1') }, + { AV_CODEC_ID_VVC, MKTAG('v', 'v', 'c', '1') }, + { AV_CODEC_ID_VVC, MKTAG('v', 'v', 'i', '1') }, { AV_CODEC_ID_MPEG2VIDEO, MKTAG('m', 'p', '4', 'v') }, { AV_CODEC_ID_MPEG1VIDEO, MKTAG('m', 'p', '4', 'v') }, { AV_CODEC_ID_MJPEG, MKTAG('m', 'p', '4', 'v') }, diff --git a/libavformat/rawenc.c b/libavformat/rawenc.c index 267fce252d..7770d63d10 100644 --- a/libavformat/rawenc.c +++ b/libavformat/rawenc.c @@ -401,6 +401,29 @@ const AVOutputFormat ff_hevc_muxer = { }; #endif +#if CONFIG_VVC_MUXER +static int vvc_check_bitstream(AVFormatContext *s, AVStream *st, + const AVPacket *pkt) +{ + if (pkt->size >= 5 && AV_RB32(pkt->data) != 0x0000001 && + AV_RB24(pkt->data) != 0x000001) + return ff_stream_add_bitstream_filter(st, "vvc_mp4toannexb", NULL); + return 1; +} + +const AVOutputFormat ff_vvc_muxer = { + .name = "vvc", + .long_name = NULL_IF_CONFIG_SMALL("raw VVC video"), + .extensions = "vvc,h266,266", + .audio_codec = AV_CODEC_ID_NONE, + .video_codec = AV_CODEC_ID_VVC, + .init = force_one_stream, + .write_packet = ff_raw_write_packet, + .check_bitstream = vvc_check_bitstream, + .flags = AVFMT_NOTIMESTAMPS, +}; +#endif + #if CONFIG_M4V_MUXER const AVOutputFormat ff_m4v_muxer = { .name = "m4v", From patchwork Mon Oct 24 14:06:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Siedel X-Patchwork-Id: 38967 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2249901pzb; Mon, 24 Oct 2022 07:09:38 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4dC+mlaLlC6bR7ImQADrUH+UEn7ZfEuMulqqnUEBqAyeaAYWTjWPvNhLoE1DRmypvvI0u7 X-Received: by 2002:a17:907:7e90:b0:78d:50f7:8abd with SMTP id qb16-20020a1709077e9000b0078d50f78abdmr26768257ejc.523.1666620577013; Mon, 24 Oct 2022 07:09:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666620577; cv=none; d=google.com; s=arc-20160816; b=bL/28+mxhhfaNSHso8HFPWATEXlkQ9f2POwcLeFzgNgAuGD50DJOFm/id5s6WkOZ2J c4NFVt60xHeREg83jbEy6GBIUMTAuMH1G5KXY+YR6u2LWtgX4e6VnW1373TSFR5QT6cT EzPSZd5zwE2bdNO9BSl/7dQfTKbqjLcoZ4YLTB7dyxc+2yQlpXTw+tXKDIK3kIQ7Suul JOdWWd7Wvjqd9ijOhSXV+zLOR4TzBSq5462+/TJQAbGbVmgQXh9f0Q6nC27oIKxwomyk JDj1x2gH+4Cam/KIQsWQLWUPWSFamPrsK8EeSX3+/r+RpXij4V24ogTp9rfjOn5DGFxp FhGQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=G6/RDy8FbfOwEjWaB+3LT5YGPt0UUeqWXLvoH7heeGQ=; b=j6udHE7p3QzqER262cSDU/nmCf09uCN8jKg5TooggQ9h3GNRt4qkr7/5MvnHMvwmhZ VToHwVkxU1PI4iPdKxP+HiSJTvM2p51WCm++Xq8o3DiqkdmSDO04ujMVMb3VPGYJWjxN gIWqwKY5t+GSujYvfuT7TOZzw/vx5APFQYTRji5YeBeCxoCYIiuV/6qoY459nSuDxh8T TDZn1pXN3AK+BVyC/ckF1eK2Xy6Qz+xnG9eEQ7SH6lug7QgmBh91rDj+y/HJODh/nR8m 5sH/172mtQh8djqOAuzijmkZw3a5Du1Kx8WFB506CJ4FNIFgJljRHLLCwRImYvP/ExCB mlQg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=dMDt83Z8; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id sb24-20020a1709076d9800b007815a44de91si30383973ejc.771.2022.10.24.07.09.36; Mon, 24 Oct 2022 07:09:36 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=dMDt83Z8; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id CAD4168BC7E; Mon, 24 Oct 2022 17:07:56 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f52.google.com (mail-lf1-f52.google.com [209.85.167.52]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1AD2168BC91 for ; Mon, 24 Oct 2022 17:07:54 +0300 (EEST) Received: by mail-lf1-f52.google.com with SMTP id j4so17037953lfk.0 for ; Mon, 24 Oct 2022 07:07:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spin-digital-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=c6obKSxse9H4NIIh0ZmlX409MRPW2cvVKgUbFaeuFqI=; b=dMDt83Z8LoUbYdmI/W0lTz0Q9IH+zrHyanleLB3SdG2IrWmHmbfVgz/cvPF0Yi/wMe VJ/UBmZbwKi3XlivjcQLBKk+SybdEfV1XpQGrp5KFZUX9PfmvckblPx1uf8erwuMSE6g xHXmUJNF0wUVwox+H8Qry3HTJiQx5Zy22RjURAg2ShlpPQ1kYF13IxwCD6bX+Yd95kcq 9/8IhcIwob70hGaGkpunlbxFC3J7w8mAFik63MU4krcCs/5baC/m2Eyz02krUhw0XHFD /oiyYiVbqiS7V+djK7GeKUJwYpv8ADsfKxQF1cvf133+JTGXyOeDqg4svdbc4EwVwcjs HXaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=c6obKSxse9H4NIIh0ZmlX409MRPW2cvVKgUbFaeuFqI=; b=HUNRT1MdMksTm2lRS+WUN+UMCFcO5rE+4WZoJTEchZ3sKgxmQzgvkZj8lPrQq2TJY+ vWL22qKcLZ34vb8joKmQk2IAFhKTXuam1/6tvUi7QI3Ma8pT9R6QwT1FrsxkAghSRky5 UjYUOovwBxn2i1InEhh+tsEl/GXWCnV55tm/BgRNHc7XeXGStZZE6erDM2LCWFukgWyX rniTexG6tQvtarOKn28q8ecaEvVieWJsSQUfXe1yxLFwv7/QQCfBVC9TStrHR9qX2En0 a9PHv1d6MBfMOOW/mVRwTt6AhfftnPr53iWURP1YuS8QylsjsXI1lqAiCfVYEGLCBELs d1zQ== X-Gm-Message-State: ACrzQf2uCB3avOHSQanK6+tzyHerQgZ2UpKY5lNEqhFb9nGrtEN2zkSV SNN5fBthgQHy9sIdXFFpsWg+0Ql9FtCjXQ== X-Received: by 2002:a17:906:ee8e:b0:730:4a24:f311 with SMTP id wt14-20020a170906ee8e00b007304a24f311mr28443306ejb.420.1666620462023; Mon, 24 Oct 2022 07:07:42 -0700 (PDT) Received: from thomas-win.localdomain ([213.138.44.237]) by smtp.gmail.com with ESMTPSA id s19-20020a170906455300b0079800b81709sm6056436ejq.219.2022.10.24.07.07.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Oct 2022 07:07:41 -0700 (PDT) From: Thomas Siedel To: ffmpeg-devel@ffmpeg.org Date: Mon, 24 Oct 2022 16:06:42 +0200 Message-Id: <20221024140645.4945-8-thomas.ff@spin-digital.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221024140645.4945-1-thomas.ff@spin-digital.com> References: <20221024140645.4945-1-thomas.ff@spin-digital.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 07/10] avcodec: add external decoder libvvdec for H266/VVC X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Thomas Siedel Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: RfiXhUdcVhfU Add external decoder VVdeC for H266/VVC decoding. Register new decoder libvvdec Add vvc_parse_extradata to support parse/probe of vvcC stream input Add vvc_paramset that implements the parser of vvcC configuration boxes Add libvvdec to wrap the vvdec interface into ffmpeg libvvdec implements decoder option externmem to use AVBufferPool (default) or copy image after receiving from decoder Enable decoder by adding --enable-libvvdec in configure step Signed-off-by: Thomas Siedel --- configure | 5 + libavcodec/Makefile | 1 + libavcodec/allcodecs.c | 1 + libavcodec/libvvdec.c | 511 ++++++++++++++++ libavcodec/vvc_paramset.c | 972 +++++++++++++++++++++++++++++++ libavcodec/vvc_paramset.h | 429 ++++++++++++++ libavcodec/vvc_parse_extradata.c | 241 ++++++++ libavcodec/vvc_parse_extradata.h | 36 ++ 8 files changed, 2196 insertions(+) create mode 100644 libavcodec/libvvdec.c create mode 100644 libavcodec/vvc_paramset.c create mode 100644 libavcodec/vvc_paramset.h create mode 100644 libavcodec/vvc_parse_extradata.c create mode 100644 libavcodec/vvc_parse_extradata.h diff --git a/configure b/configure index 4d54ce2a93..978f15f772 100755 --- a/configure +++ b/configure @@ -288,6 +288,7 @@ External library support: --enable-libvorbis enable Vorbis en/decoding via libvorbis, native implementation exists [no] --enable-libvpx enable VP8 and VP9 de/encoding via libvpx [no] + --enable-libvvdec enable VVC decoding via vvdec [no] --enable-libwebp enable WebP encoding via libwebp [no] --enable-libx264 enable H.264 encoding via x264 [no] --enable-libx265 enable HEVC encoding via x265 [no] @@ -1875,6 +1876,7 @@ EXTERNAL_LIBRARY_LIST=" libvmaf libvorbis libvpx + libvvdec libwebp libxml2 libzimg @@ -3405,6 +3407,8 @@ libvpx_vp8_decoder_deps="libvpx" libvpx_vp8_encoder_deps="libvpx" libvpx_vp9_decoder_deps="libvpx" libvpx_vp9_encoder_deps="libvpx" +libvvdec_decoder_deps="libvvdec" +libvvdec_decoder_select="vvc_mp4toannexb_bsf" libwebp_encoder_deps="libwebp" libwebp_anim_encoder_deps="libwebp" libx262_encoder_deps="libx262" @@ -6735,6 +6739,7 @@ enabled libvpx && { die "libvpx enabled but no supported decoders found" fi } +enabled libvvdec && require_pkg_config libvvdec "libvvdec >= 1.6.0" "vvdec/vvdec.h" vvdec_get_version enabled libwebp && { enabled libwebp_encoder && require_pkg_config libwebp "libwebp >= 0.2.0" webp/encode.h WebPGetEncoderVersion diff --git a/libavcodec/Makefile b/libavcodec/Makefile index c6652e5e09..f4cdbc9be1 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1104,6 +1104,7 @@ OBJS-$(CONFIG_LIBVPX_VP8_DECODER) += libvpxdec.o OBJS-$(CONFIG_LIBVPX_VP8_ENCODER) += libvpxenc.o OBJS-$(CONFIG_LIBVPX_VP9_DECODER) += libvpxdec.o libvpx.o OBJS-$(CONFIG_LIBVPX_VP9_ENCODER) += libvpxenc.o libvpx.o +OBJS-$(CONFIG_LIBVVDEC_DECODER) += libvvdec.o vvc_parse_extradata.o vvc_paramset.o OBJS-$(CONFIG_LIBWEBP_ENCODER) += libwebpenc_common.o libwebpenc.o OBJS-$(CONFIG_LIBWEBP_ANIM_ENCODER) += libwebpenc_common.o libwebpenc_animencoder.o OBJS-$(CONFIG_LIBX262_ENCODER) += libx264.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index 4f1d66cb0c..9813b291f4 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -793,6 +793,7 @@ extern const FFCodec ff_libvpx_vp8_encoder; extern const FFCodec ff_libvpx_vp8_decoder; extern FFCodec ff_libvpx_vp9_encoder; extern FFCodec ff_libvpx_vp9_decoder; +extern const FFCodec ff_libvvdec_decoder; /* preferred over libwebp */ extern const FFCodec ff_libwebp_anim_encoder; extern const FFCodec ff_libwebp_encoder; diff --git a/libavcodec/libvvdec.c b/libavcodec/libvvdec.c new file mode 100644 index 0000000000..28f3a548ca --- /dev/null +++ b/libavcodec/libvvdec.c @@ -0,0 +1,511 @@ +/* + * H.266 decoding using the VVdeC library + * + * Copyright (C) 2022, Thomas Siedel + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "config_components.h" + +#include + +#include "libavutil/common.h" +#include "libavutil/avutil.h" +#include "libavutil/pixdesc.h" +#include "libavutil/opt.h" +#include "libavutil/imgutils.h" +#include "libavutil/frame.h" +#include "libavutil/mastering_display_metadata.h" +#include "libavutil/log.h" + +#include "avcodec.h" +#include "codec_internal.h" +#include "decode.h" +#include "internal.h" +#include "profiles.h" + +#include "vvc_paramset.h" +#include "vvc_parse_extradata.h" + +typedef struct VVdeCContext { + AVClass *av_class; + vvdecDecoder* vvdecDec; + vvdecParams vvdecParams; + VVCParamSets ps; + int is_nalff; + int nal_length_size; + bool bFlush; + AVBufferPool *pools[3]; /** Pools for each data plane. */ + int pool_size[3]; +}VVdeCContext; + + +static void ff_vvdec_log_callback(void *avctx, int level, const char *fmt, va_list args ) +{ + vfprintf( level == 1 ? stderr : stdout, fmt, args ); +} + +static void* ff_vvdec_buffer_allocator( void *ctx, vvdecComponentType comp, uint32_t size, uint32_t alignment, void** allocator ) +{ + AVBufferRef *buf; + VVdeCContext *s; + int plane; + + uint32_t alignedsize = FFALIGN(size, alignment); + s = (VVdeCContext*)ctx; + plane = (int)comp; + + if ( plane < 0 || plane > 3 ) + return NULL; + + if ( alignedsize != s->pool_size[plane]) { + av_buffer_pool_uninit(&s->pools[plane]); + s->pools[plane] = av_buffer_pool_init(alignedsize, NULL); + if (!s->pools[plane]) { + s->pool_size[plane] = 0; + return NULL; + } + s->pool_size[plane] = alignedsize; + } + + buf = av_buffer_pool_get(s->pools[plane]); + if (!buf) + return NULL; + + *allocator = (void*)buf; + return buf->data; +} + +static void ff_vvdec_buffer_unref( void *ctx, void* allocator ) +{ + AVBufferRef *buf = (AVBufferRef*)allocator; + av_buffer_unref( &buf ); +} + +static void ff_vvdec_printParameterInfo( AVCodecContext *avctx, vvdecParams* params ) +{ + av_log(avctx, AV_LOG_DEBUG, "Version info: vvdec %s ( threads %d)\n",vvdec_get_version(), params->threads); +} + +static int ff_vvdec_set_pix_fmt(AVCodecContext *avctx, vvdecFrame* frame ) +{ + if( NULL != frame->picAttributes && NULL != frame->picAttributes->vui && + frame->picAttributes->vui->colourDescriptionPresentFlag ) { + avctx->color_trc = frame->picAttributes->vui->transferCharacteristics; + avctx->color_primaries = frame->picAttributes->vui->colourPrimaries; + avctx->colorspace = frame->picAttributes->vui->matrixCoefficients; + } + else { + avctx->color_primaries = AVCOL_PRI_UNSPECIFIED; + avctx->color_trc = AVCOL_TRC_UNSPECIFIED; + avctx->colorspace = AVCOL_SPC_UNSPECIFIED; + } + + if( NULL != frame->picAttributes && NULL != frame->picAttributes->vui && + frame->picAttributes->vui->videoSignalTypePresentFlag) { + avctx->color_range = frame->picAttributes->vui->videoFullRangeFlag ? + AVCOL_RANGE_JPEG : AVCOL_RANGE_MPEG; + } + else { + avctx->color_range = AVCOL_RANGE_MPEG; + } + + switch ( frame->colorFormat ) { + case VVDEC_CF_YUV420_PLANAR: + if (frame->bitDepth == 8) { + avctx->pix_fmt = frame->numPlanes == 1 ? + AV_PIX_FMT_GRAY8 : AV_PIX_FMT_YUV420P; + avctx->profile = FF_PROFILE_VVC_MAIN_10; + return 0; + } + else if (frame->bitDepth == 10) { + avctx->pix_fmt = frame->numPlanes == 1 ? + AV_PIX_FMT_GRAY10 : AV_PIX_FMT_YUV420P10; + avctx->profile = FF_PROFILE_VVC_MAIN_10; + return 0; + } + else { + return AVERROR_INVALIDDATA; + } + default: + return AVERROR_INVALIDDATA; + } +} + +static int set_side_data( AVCodecContext *avctx, AVFrame *avframe, vvdecFrame *frame ) +{ + vvdecSEI *sei; + VVdeCContext *s = (VVdeCContext*)avctx->priv_data; + + sei = vvdec_find_frame_sei( s->vvdecDec, VVDEC_MASTERING_DISPLAY_COLOUR_VOLUME, frame ); + if( sei ) { + // VVC uses a g,b,r ordering, which we convert to a more natural r,g,b + const int mapping[3] = {2, 0, 1}; + const int chroma_den = 50000; + const int luma_den = 10000; + int i; + vvdecSEIMasteringDisplayColourVolume* p; + AVMasteringDisplayMetadata *metadata = av_mastering_display_metadata_create_side_data(avframe); + p = (vvdecSEIMasteringDisplayColourVolume *)(sei->payload); + if ( p && metadata ) { + for (i = 0; i < 3; i++) { + const int j = mapping[i]; + metadata->display_primaries[i][0].num = p->primaries[j][0]; + metadata->display_primaries[i][0].den = chroma_den; + metadata->display_primaries[i][1].num = p->primaries[j][1]; + metadata->display_primaries[i][1].den = chroma_den; + } + metadata->white_point[0].num = p->whitePoint[0]; + metadata->white_point[0].den = chroma_den; + metadata->white_point[1].num = p->whitePoint[1]; + metadata->white_point[1].den = chroma_den; + + metadata->max_luminance.num = p->maxLuminance; + metadata->max_luminance.den = luma_den; + metadata->min_luminance.num = p->minLuminance; + metadata->min_luminance.den = luma_den; + metadata->has_luminance = 1; + metadata->has_primaries = 1; + + av_log(avctx, AV_LOG_DEBUG, "Mastering Display Metadata:\n"); + av_log(avctx, AV_LOG_DEBUG, + "r(%5.4f,%5.4f) g(%5.4f,%5.4f) b(%5.4f %5.4f) wp(%5.4f, %5.4f)\n", + av_q2d(metadata->display_primaries[0][0]), + av_q2d(metadata->display_primaries[0][1]), + av_q2d(metadata->display_primaries[1][0]), + av_q2d(metadata->display_primaries[1][1]), + av_q2d(metadata->display_primaries[2][0]), + av_q2d(metadata->display_primaries[2][1]), + av_q2d(metadata->white_point[0]), av_q2d(metadata->white_point[1])); + av_log(avctx, AV_LOG_DEBUG, + "min_luminance=%f, max_luminance=%f\n", + av_q2d(metadata->min_luminance), av_q2d(metadata->max_luminance)); + } + return 0; + } + + sei = vvdec_find_frame_sei( s->vvdecDec, VVDEC_CONTENT_LIGHT_LEVEL_INFO, frame ); + if( sei ) { + vvdecSEIContentLightLevelInfo* p = NULL; + AVContentLightMetadata *light = av_content_light_metadata_create_side_data(avframe); + p = (vvdecSEIContentLightLevelInfo *)(sei->payload); + if ( p && light) { + light->MaxCLL = p->maxContentLightLevel; + light->MaxFALL = p->maxPicAverageLightLevel; + } + + av_log(avctx, AV_LOG_DEBUG, "Content Light Level Metadata:\n"); + av_log(avctx, AV_LOG_DEBUG, "MaxCLL=%d, MaxFALL=%d\n", + light->MaxCLL, light->MaxFALL); + } + + return 0; +} + +static void export_stream_params( AVCodecContext *avctx, const VVCSPS *sps) +{ + avctx->coded_width = sps->pic_width_max_in_luma_samples; + avctx->coded_height = sps->pic_height_max_in_luma_samples; + avctx->width = sps->pic_width_max_in_luma_samples - + sps->conf_win_left_offset - + sps->conf_win_right_offset; + avctx->height = sps->pic_height_max_in_luma_samples - + sps->conf_win_top_offset - + sps->conf_win_bottom_offset; + avctx->has_b_frames = sps->max_sublayers; + avctx->profile = sps->profile_tier_level.general_profile_idc; + avctx->level = sps->profile_tier_level.general_level_idc; + avctx->pix_fmt = sps->pix_fmt; + + avctx->color_range = sps->vui.full_range_flag ? AVCOL_RANGE_JPEG : AVCOL_RANGE_MPEG; + + if (sps->vui.colour_description_present_flag) { + avctx->color_primaries = sps->vui.colour_primaries; + avctx->color_trc = sps->vui.transfer_characteristics; + avctx->colorspace = sps->vui.matrix_coeffs; + } else { + avctx->color_primaries = AVCOL_PRI_UNSPECIFIED; + avctx->color_trc = AVCOL_TRC_UNSPECIFIED; + avctx->colorspace = AVCOL_SPC_UNSPECIFIED; + } + + avctx->chroma_sample_location = AVCHROMA_LOC_UNSPECIFIED; + if (sps->chroma_format_idc == 1) { + if (sps->vui.chroma_loc_info_present_flag) { + if (sps->vui.chroma_sample_loc_type_top_field <= 5) + avctx->chroma_sample_location = sps->vui.chroma_sample_loc_type_top_field + 1; + } else + avctx->chroma_sample_location = AVCHROMA_LOC_LEFT; + } + + if (sps->timing_hrd_params_present_flag && + sps->general_timing_hrd_parameters.num_units_in_tick && + sps->general_timing_hrd_parameters.time_scale) { + av_reduce(&avctx->framerate.den, &avctx->framerate.num, + sps->general_timing_hrd_parameters.num_units_in_tick, + sps->general_timing_hrd_parameters.time_scale, INT_MAX); + } +} + +static int vvc_decode_extradata( AVCodecContext *avctx, uint8_t *buf, int length, int first) +{ + VVdeCContext *s = (VVdeCContext*)avctx->priv_data; + int ret; + + ret = ff_vvc_decode_extradata(buf, length, &s->ps, &s->is_nalff, + &s->nal_length_size, avctx->err_recognition, + false, avctx); + if (ret < 0) + return ret; + + if ( s->ps.sps != NULL ) + export_stream_params(avctx, s->ps.sps); + + return 0; +} + +static av_cold int ff_vvdec_decode_init(AVCodecContext *avctx) +{ + int i; + VVdeCContext *s = (VVdeCContext*)avctx->priv_data; + + vvdec_params_default( &s->vvdecParams ); + s->vvdecParams.logLevel = VVDEC_DETAILS; + + if ( av_log_get_level() >= AV_LOG_DEBUG ) s->vvdecParams.logLevel = VVDEC_DETAILS; + else if( av_log_get_level() >= AV_LOG_VERBOSE ) s->vvdecParams.logLevel = VVDEC_INFO; // VVDEC_INFO will output per picture info + else if( av_log_get_level() >= AV_LOG_INFO ) s->vvdecParams.logLevel = VVDEC_WARNING; // AV_LOG_INFO is ffmpeg default + else s->vvdecParams.logLevel = VVDEC_SILENT; + + if( avctx->thread_count > 0 ) + s->vvdecParams.threads = avctx->thread_count; // number of worker threads (should not exceed the number of physical cpu's) + else + s->vvdecParams.threads = -1; // get max cpus + + ff_vvdec_printParameterInfo( avctx, &s->vvdecParams ); + + // using buffer allocation by using AVBufferPool + s->vvdecParams.opaque = avctx->priv_data; + s->vvdecDec = vvdec_decoder_open_with_allocator( &s->vvdecParams, + ff_vvdec_buffer_allocator, ff_vvdec_buffer_unref ); + + + if( !s->vvdecDec ) { + av_log(avctx, AV_LOG_ERROR, "cannot init vvc decoder\n" ); + return -1; + } + + vvdec_set_logging_callback( s->vvdecDec, ff_vvdec_log_callback ); + + s->bFlush = false; + s->is_nalff = 0; + s->nal_length_size = 0; + + for (i = 0; i < FF_ARRAY_ELEMS(s->pools); i++) { + s->pools[i] = NULL; + s->pool_size[i] = 0; + } + + if (!avctx->internal->is_copy) { + if (avctx->extradata_size > 0 && avctx->extradata) { + int ret = vvc_decode_extradata(avctx, avctx->extradata, avctx->extradata_size, 1); + if (ret < 0) { + return ret; + } + } + } + + return 0; +} + +static av_cold int ff_vvdec_decode_close(AVCodecContext *avctx) +{ + VVdeCContext *s = (VVdeCContext*)avctx->priv_data; + + for (int i = 0; i < FF_ARRAY_ELEMS(s->pools); i++) { + av_buffer_pool_uninit(&s->pools[i]); + s->pool_size[i] = 0; + } + + if( 0 != vvdec_decoder_close(s->vvdecDec) ) { + av_log(avctx, AV_LOG_ERROR, "cannot close vvdec\n" ); + return -1; + } + + ff_vvc_ps_uninit( &s->ps ); + s->bFlush = false; + return 0; +} + +static av_cold int ff_vvdec_decode_frame( AVCodecContext *avctx, AVFrame *data, + int *got_frame, AVPacket *avpkt ) +{ + VVdeCContext *s = avctx->priv_data; + AVFrame *avframe = data; + + int ret = 0; + vvdecFrame *frame = NULL; + + if ( avframe ) { + if( !avpkt->size && !s->bFlush ) + s->bFlush = true; + + if( s->bFlush ) + ret = vvdec_flush( s->vvdecDec, &frame ); + else { + vvdecAccessUnit accessUnit; + vvdec_accessUnit_default( &accessUnit ); + accessUnit.payload = avpkt->data; + accessUnit.payloadSize = avpkt->size; + accessUnit.payloadUsedSize = avpkt->size; + + accessUnit.cts = avpkt->pts; accessUnit.ctsValid = true; + accessUnit.dts = avpkt->dts; accessUnit.dtsValid = true; + + ret = vvdec_decode( s->vvdecDec, &accessUnit, &frame ); + } + + if( ret < 0 ) { + if( ret == VVDEC_EOF ) + s->bFlush = true; + else if ( ret != VVDEC_TRY_AGAIN ) { + av_log(avctx, AV_LOG_ERROR, "error in vvdec::decode - ret:%d - %s\n", ret, vvdec_get_last_error(s->vvdecDec)); + return AVERROR(EINVAL); \ + } + } + else if( NULL != frame ) { + const uint8_t * src_data[4] = { frame->planes[0].ptr, frame->planes[1].ptr, frame->planes[2].ptr, NULL }; + const int src_linesizes[4] = { (int)frame->planes[0].stride, (int)frame->planes[1].stride, (int)frame->planes[2].stride, 0 }; + + if (( ret = ff_vvdec_set_pix_fmt(avctx, frame)) < 0) { + av_log(avctx, AV_LOG_ERROR, "Unsupported output colorspace (%d) / bit_depth (%d)\n", + frame->colorFormat, frame->bitDepth); + goto fail; + } + + if( avctx->pix_fmt != AV_PIX_FMT_YUV420P && avctx->pix_fmt != AV_PIX_FMT_YUV420P10LE ) { + av_log(avctx, AV_LOG_ERROR, "Unsupported output colorspace (%d) / bit_depth (%d)\n", + frame->colorFormat, frame->bitDepth ); + ret = AVERROR_INVALIDDATA; + goto fail; + } + + if ((int)frame->width != avctx->width || (int)frame->height != avctx->height) { + av_log(avctx, AV_LOG_INFO, "dimension change! %dx%d -> %dx%d\n", + avctx->width, avctx->height, frame->width, frame->height); + + ret = ff_set_dimensions(avctx, frame->width, frame->height); + if (ret < 0) + goto fail; + } + + if( frame->planes[0].allocator ) + avframe->buf[0] = av_buffer_ref( (AVBufferRef *)frame->planes[0].allocator); + if( frame->planes[1].allocator ) + avframe->buf[1] = av_buffer_ref( (AVBufferRef *)frame->planes[1].allocator); + if( frame->planes[2].allocator ) + avframe->buf[2] = av_buffer_ref( (AVBufferRef *)frame->planes[2].allocator); + + for (int i = 0; i < 4; i++) { + avframe->data[i] = (uint8_t *)src_data[i]; + avframe->linesize[i] = src_linesizes[i]; + } + + ret = ff_decode_frame_props(avctx, avframe); + if (ret < 0) + goto fail; + + if( frame->picAttributes ) { + avframe->key_frame = frame->picAttributes->isRefPic; + avframe->pict_type = (frame->picAttributes->sliceType != VVDEC_SLICETYPE_UNKNOWN) ? + frame->picAttributes->sliceType+1 : AV_PICTURE_TYPE_NONE; + } + + ret = set_side_data(avctx, avframe, frame ); + if (ret < 0) + goto fail; + + if( 0 != vvdec_frame_unref( s->vvdecDec, frame ) ) + av_log(avctx, AV_LOG_ERROR, "cannot free picture memory\n"); + + *got_frame = 1; + } + } + + return avpkt->size; + +fail: + if (frame) { + if( frame->planes[0].allocator ) + av_buffer_unref( (AVBufferRef **)&frame->planes[0].allocator ); + if( frame->planes[1].allocator ) + av_buffer_unref( (AVBufferRef **)&frame->planes[1].allocator ); + if( frame->planes[2].allocator ) + av_buffer_unref( (AVBufferRef **)&frame->planes[2].allocator ); + + vvdec_frame_unref( s->vvdecDec, frame ); + } + return ret; +} + +static av_cold void ff_vvdec_decode_flush(AVCodecContext *avctx) +{ + VVdeCContext *s = (VVdeCContext*)avctx->priv_data; + + if( 0 != vvdec_decoder_close(s->vvdecDec) ) + av_log(avctx, AV_LOG_ERROR, "cannot close vvdec during flush\n" ); + + s->vvdecDec = vvdec_decoder_open_with_allocator( &s->vvdecParams, ff_vvdec_buffer_allocator, ff_vvdec_buffer_unref ); + if( !s->vvdecDec ) + av_log(avctx, AV_LOG_ERROR, "cannot reinit vvdec during flush\n" ); + + vvdec_set_logging_callback( s->vvdecDec, ff_vvdec_log_callback ); + + s->bFlush = false; +} + +static const enum AVPixelFormat pix_fmts_vvc[] = { + AV_PIX_FMT_YUV420P, + AV_PIX_FMT_YUV420P10LE, + AV_PIX_FMT_NONE +}; + +static const AVClass class_libvvdec = { + .class_name = "libvvdec-vvc decoder", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, +}; + +FFCodec ff_libvvdec_decoder = { + .p.name = "libvvdec", + .p.long_name = "H.266 / VVC Decoder VVdeC", + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_VVC, + .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_OTHER_THREADS, + .p.profiles = NULL_IF_CONFIG_SMALL(ff_vvc_profiles), + .p.priv_class = &class_libvvdec, + .p.wrapper_name = "libvvdec", + .priv_data_size = sizeof(VVdeCContext), + .p.pix_fmts = pix_fmts_vvc, + .init = ff_vvdec_decode_init, + FF_CODEC_DECODE_CB(ff_vvdec_decode_frame), + .close = ff_vvdec_decode_close, + .flush = ff_vvdec_decode_flush, + .bsfs = "vvc_mp4toannexb", + .caps_internal = FF_CODEC_CAP_AUTO_THREADS, +}; diff --git a/libavcodec/vvc_paramset.c b/libavcodec/vvc_paramset.c new file mode 100644 index 0000000000..a016f57a37 --- /dev/null +++ b/libavcodec/vvc_paramset.c @@ -0,0 +1,972 @@ +/* + * VVC Parameter Set decoding + * + * Copyright (c) 2022, Thomas Siedel + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/imgutils.h" +#include "golomb.h" +#include "vvc_paramset.h" + +static void remove_sps(VVCParamSets *s, int id) +{ + if (s->sps_list[id]) { + if (s->sps == (const VVCSPS*)s->sps_list[id]->data) + s->sps = NULL; + + av_assert0(!(s->sps_list[id] && s->sps == (VVCSPS*)s->sps_list[id]->data)); + } + av_buffer_unref(&s->sps_list[id]); +} + +static int decode_general_constraints_info(GetBitContext *gb, AVCodecContext *avctx, + VVCGeneralConstraintsInfo *gci) +{ + int i; + gci->gci_present_flag = get_bits1(gb); + + if (gci->gci_present_flag) { + /* general */ + gci->gci_intra_only_constraint_flag = get_bits1(gb); + gci->gci_all_layers_independent_constraint_flag = get_bits1(gb); + gci->gci_one_au_only_constraint_flag = get_bits1(gb); + + /* picture format */ + gci->gci_sixteen_minus_max_bitdepth_constraint_idc = get_bits(gb,4); + gci->gci_three_minus_max_chroma_format_constraint_idc = get_bits(gb,2); + + /* NAL unit type related */ + gci->gci_no_mixed_nalu_types_in_pic_constraint_flag = get_bits1(gb); + gci->gci_no_trail_constraint_flag = get_bits1(gb); + gci->gci_no_stsa_constraint_flag = get_bits1(gb); + gci->gci_no_rasl_constraint_flag = get_bits1(gb); + gci->gci_no_radl_constraint_flag = get_bits1(gb); + gci->gci_no_idr_constraint_flag = get_bits1(gb); + gci->gci_no_cra_constraint_flag = get_bits1(gb); + gci->gci_no_gdr_constraint_flag = get_bits1(gb); + gci->gci_no_aps_constraint_flag = get_bits1(gb); + gci->gci_no_idr_rpl_constraint_flag = get_bits1(gb); + + /* tile, slice, subpicture partitioning */ + gci->gci_one_tile_per_pic_constraint_flag = get_bits1(gb); + gci->gci_pic_header_in_slice_header_constraint_flag = get_bits1(gb); + gci->gci_one_slice_per_pic_constraint_flag = get_bits1(gb); + gci->gci_no_rectangular_slice_constraint_flag = get_bits1(gb); + gci->gci_one_slice_per_subpic_constraint_flag = get_bits1(gb); + gci->gci_no_subpic_info_constraint_flag = get_bits1(gb); + + /* CTU and block partitioning */ + gci->gci_three_minus_max_log2_ctu_size_constraint_idc = get_bits(gb,2); + gci->gci_no_partition_constraints_override_constraint_flag = get_bits1(gb); + gci->gci_no_mtt_constraint_flag = get_bits1(gb); + gci->gci_no_qtbtt_dual_tree_intra_constraint_flag = get_bits1(gb); + + /* intra */ + gci->gci_no_palette_constraint_flag = get_bits1(gb); + gci->gci_no_ibc_constraint_flag = get_bits1(gb); + gci->gci_no_isp_constraint_flag = get_bits1(gb); + gci->gci_no_mrl_constraint_flag = get_bits1(gb); + gci->gci_no_mip_constraint_flag = get_bits1(gb); + gci->gci_no_cclm_constraint_flag = get_bits1(gb); + + /* inter */ + gci->gci_no_ref_pic_resampling_constraint_flag = get_bits1(gb); + gci->gci_no_res_change_in_clvs_constraint_flag = get_bits1(gb); + gci->gci_no_weighted_prediction_constraint_flag = get_bits1(gb); + gci->gci_no_ref_wraparound_constraint_flag = get_bits1(gb); + gci->gci_no_temporal_mvp_constraint_flag = get_bits1(gb); + gci->gci_no_sbtmvp_constraint_flag = get_bits1(gb); + gci->gci_no_amvr_constraint_flag = get_bits1(gb); + gci->gci_no_bdof_constraint_flag = get_bits1(gb); + gci->gci_no_smvd_constraint_flag = get_bits1(gb); + gci->gci_no_dmvr_constraint_flag = get_bits1(gb); + gci->gci_no_mmvd_constraint_flag = get_bits1(gb); + gci->gci_no_affine_motion_constraint_flag = get_bits1(gb); + gci->gci_no_prof_constraint_flag = get_bits1(gb); + gci->gci_no_bcw_constraint_flag = get_bits1(gb); + gci->gci_no_ciip_constraint_flag = get_bits1(gb); + gci->gci_no_gpm_constraint_flag = get_bits1(gb); + + /* transform, quantization, residual */ + gci->gci_no_luma_transform_size_64_constraint_flag = get_bits1(gb); + gci->gci_no_transform_skip_constraint_flag = get_bits1(gb); + gci->gci_no_bdpcm_constraint_flag = get_bits1(gb); + gci->gci_no_mts_constraint_flag = get_bits1(gb); + gci->gci_no_lfnst_constraint_flag = get_bits1(gb); + gci->gci_no_joint_cbcr_constraint_flag = get_bits1(gb); + gci->gci_no_sbt_constraint_flag = get_bits1(gb); + gci->gci_no_act_constraint_flag = get_bits1(gb); + gci->gci_no_explicit_scaling_list_constraint_flag = get_bits1(gb); + gci->gci_no_dep_quant_constraint_flag = get_bits1(gb); + gci->gci_no_sign_data_hiding_constraint_flag = get_bits1(gb); + gci->gci_no_cu_qp_delta_constraint_flag = get_bits1(gb); + gci->gci_no_chroma_qp_offset_constraint_flag = get_bits1(gb); + + /* loop filter */ + gci->gci_no_sao_constraint_flag = get_bits1(gb); + gci->gci_no_alf_constraint_flag = get_bits1(gb); + gci->gci_no_ccalf_constraint_flag = get_bits1(gb); + gci->gci_no_lmcs_constraint_flag = get_bits1(gb); + gci->gci_no_ladf_constraint_flag = get_bits1(gb); + gci->gci_no_virtual_boundaries_constraint_flag = get_bits1(gb); + gci->gci_num_reserved_bits = get_bits(gb,8); + for (i = 0; i < gci->gci_num_reserved_bits; i++) { + gci->gci_reserved_zero_bit[i] = get_bits1(gb); + } + } + + align_get_bits(gb); + + return 0; +} + + +static int decode_profile_tier_level(GetBitContext *gb, AVCodecContext *avctx, + VVCProfileTierLevel *ptl, + int profile_tier_present_flag, + int max_num_sub_layers_minus1 ) +{ + int i; + + if (profile_tier_present_flag) { + ptl->general_profile_idc = get_bits(gb, 7); + ptl->general_tier_flag = get_bits1(gb); + } + ptl->general_level_idc = get_bits(gb, 8); + ptl->ptl_frame_only_constraint_flag = get_bits1(gb); + ptl->ptl_multilayer_enabled_flag = get_bits1(gb); + + if (profile_tier_present_flag) { + decode_general_constraints_info(gb, avctx, &ptl->general_constraints_info); + } + + for (i = max_num_sub_layers_minus1 - 1; i >= 0; i--) { + ptl->ptl_sublayer_level_present_flag[i] = get_bits1(gb); + } + + align_get_bits(gb); + + for (i = max_num_sub_layers_minus1 - 1; i >= 0; i--) { + if (ptl->ptl_sublayer_level_present_flag[i]) + ptl->sublayer_level_idc[i] = get_bits(gb, 8); + } + + if (profile_tier_present_flag) + { + ptl->ptl_num_sub_profiles = get_bits(gb, 8); + for (i = 0; i < ptl->ptl_num_sub_profiles; i++) + ptl->general_sub_profile_idc[i] = get_bits_long(gb, 32); + } + + return 0; +} + +static int decode_dpb_parameters(GetBitContext *gb, AVCodecContext *avctx, + VVCDpbParameters *dpb, + uint8_t max_sublayers_minus1, + uint8_t sublayer_info_flag) +{ + int i; + for (i = (sublayer_info_flag ? 0 : max_sublayers_minus1); + i <= max_sublayers_minus1; i++) { + dpb->dpb_max_dec_pic_buffering_minus1[i] = get_ue_golomb(gb); + dpb->dpb_max_num_reorder_pics[i] = get_ue_golomb(gb); + dpb->dpb_max_latency_increase_plus1[i] = get_ue_golomb(gb); + } + return 0; +} + +static int decode_ref_pic_list_struct(GetBitContext *gb, AVCodecContext *avctx, + VVCRefPicListStruct *current, + uint8_t list_idx, uint8_t rpls_idx, + const VVCSPS *sps) +{ + int i, j, num_direct_ref_layers = 0; + + current->num_ref_entries = get_ue_golomb( gb); + if (sps->long_term_ref_pics_flag && + rpls_idx < sps->num_ref_pic_lists[list_idx] && + current->num_ref_entries > 0) + current->ltrp_in_header_flag = get_bits1(gb); + if (sps->long_term_ref_pics_flag && + rpls_idx == sps->num_ref_pic_lists[list_idx]) + current->ltrp_in_header_flag = 1; + for (i = 0, j = 0; i < current->num_ref_entries; i++) { + if (sps->inter_layer_prediction_enabled_flag) + current->inter_layer_ref_pic_flag[i] = get_bits1(gb); + else + current->inter_layer_ref_pic_flag[i] = 0; + + if (!current->inter_layer_ref_pic_flag[i]) { + if (sps->long_term_ref_pics_flag) + current->st_ref_pic_flag[i] = get_bits1(gb); + else + current->st_ref_pic_flag[i] = 1; + if (current->st_ref_pic_flag[i]) { + int abs_delta_poc_st; + current->abs_delta_poc_st[i] = get_ue_golomb( gb); + if ((sps->weighted_pred_flag || + sps->weighted_bipred_flag) && i != 0) + abs_delta_poc_st = current->abs_delta_poc_st[i]; + else + abs_delta_poc_st = current->abs_delta_poc_st[i] + 1; + if (abs_delta_poc_st > 0) + current->strp_entry_sign_flag[i] = get_bits1(gb); + } else { + if (!current->ltrp_in_header_flag) { + uint8_t bits = sps->log2_max_pic_order_cnt_lsb_minus4 + 4; + current->rpls_poc_lsb_lt[j] = get_bits(gb, bits ); + j++; + } + } + } else { + if (num_direct_ref_layers == 0) { + av_log(avctx, AV_LOG_ERROR, + "num_direct_ref_layers needs > 0.\n"); + return AVERROR_INVALIDDATA; + } + current->ilrp_idx[i] = get_ue_golomb( gb); + } + } + return 0; +} + +static int decode_general_timing_hrd_parameters(GetBitContext *gb, VVCGeneralTimingHrdParameters *current) +{ + current->num_units_in_tick = get_bits_long(gb,32); + current->time_scale = get_bits_long(gb,32); + current->general_nal_hrd_params_present_flag = get_bits1(gb); + current->general_vcl_hrd_params_present_flag = get_bits1(gb); + + if (current->general_nal_hrd_params_present_flag || + current->general_vcl_hrd_params_present_flag) { + current->general_same_pic_timing_in_all_ols_flag = get_bits1(gb); + current->general_du_hrd_params_present_flag = get_bits1(gb); + if (current->general_du_hrd_params_present_flag) + current->tick_divisor_minus2 = get_bits(gb,8); + current->bit_rate_scale = get_bits(gb,4); + current->cpb_size_scale = get_bits(gb,4); + if (current->general_du_hrd_params_present_flag) + current->cpb_size_du_scale = get_bits(gb,4); + current->hrd_cpb_cnt_minus1 = get_ue_golomb_long(gb); + } else { + current->general_du_hrd_params_present_flag = 0; + } + return 0; +} + +static int decode_sublayer_hrd_parameters(GetBitContext *gb, + VVCSubLayerHRDParameters *current, int sublayer_id, + const VVCGeneralTimingHrdParameters *general) +{ + int i; + for (i = 0; i <= general->hrd_cpb_cnt_minus1; i++) { + current->bit_rate_value_minus1[sublayer_id][i] = get_ue_golomb_long(gb); + current->cpb_size_value_minus1[sublayer_id][i] = get_ue_golomb_long(gb); + if (general->general_du_hrd_params_present_flag) { + current->cpb_size_du_value_minus1[sublayer_id][i] = get_ue_golomb_long(gb); + current->bit_rate_du_value_minus1[sublayer_id][i] = get_ue_golomb_long(gb); + } + current->cbr_flag[sublayer_id][i] = get_bits1(gb); + } + return 0; +} + +static int decode_ols_timing_hrd_parameters( GetBitContext *gb, + VVCOlsTimingHrdParameters *current, + uint8_t first_sublayer, uint8_t max_sublayers_minus1, + const VVCGeneralTimingHrdParameters *general) +{ + int i; + for (i = first_sublayer; i <= max_sublayers_minus1; i++) { + current->fixed_pic_rate_general_flag[i] = get_bits1(gb); + if (!current->fixed_pic_rate_general_flag[i]) + current->fixed_pic_rate_within_cvs_flag[i] = get_bits1(gb); + else + current->fixed_pic_rate_within_cvs_flag[i] = 1; + if (current->fixed_pic_rate_within_cvs_flag[i]) { + current->elemental_duration_in_tc_minus1[i] = get_ue_golomb_long(gb); + current->low_delay_hrd_flag[i] = 0; + } else if ((general->general_nal_hrd_params_present_flag || + general->general_vcl_hrd_params_present_flag) && + general->hrd_cpb_cnt_minus1 == 0) { + current->low_delay_hrd_flag[i] = get_bits1(gb); + } else { + current->low_delay_hrd_flag[i] = 0; + } + if (general->general_nal_hrd_params_present_flag) + decode_sublayer_hrd_parameters(gb, + ¤t->nal_sub_layer_hrd_parameters, + i, general); + if (general->general_vcl_hrd_params_present_flag) + decode_sublayer_hrd_parameters(gb, + ¤t->nal_sub_layer_hrd_parameters, i, general); + } + return 0; +} + +static int decode_vui(GetBitContext *gb, AVCodecContext *avctx, + VVCVUI* vui, uint8_t chroma_format_idc ) +{ + vui->progressive_source_flag = get_bits1(gb); + vui->interlaced_source_flag = get_bits1(gb); + vui->non_packed_constraint_flag = get_bits1(gb); + vui->non_projected_constraint_flag = get_bits1(gb); + vui->aspect_ratio_info_present_flag = get_bits1(gb); + if (vui->aspect_ratio_info_present_flag) { + vui->aspect_ratio_constant_flag = get_bits1(gb); + vui->aspect_ratio_idc = get_bits(gb, 8); + if (vui->aspect_ratio_idc == 255) { + vui->sar_width = get_bits(gb, 16); + vui->sar_height = get_bits(gb, 16); + } + } else { + vui->aspect_ratio_constant_flag = 0; + vui->aspect_ratio_idc = 0; + } + vui->overscan_info_present_flag = get_bits1(gb); + if (vui->overscan_info_present_flag) + vui->overscan_appropriate_flag = get_bits1(gb); + vui->colour_description_present_flag = get_bits1(gb); + if (vui->colour_description_present_flag) { + vui->colour_primaries = get_bits(gb, 8); + vui->transfer_characteristics = get_bits(gb, 8); + vui->matrix_coeffs = get_bits(gb, 8); + av_log(avctx, AV_LOG_DEBUG, "colour_primaries: %d transfer_characteristics: %d matrix_coeffs: %d \n", + vui->colour_primaries, vui->transfer_characteristics, vui->matrix_coeffs); + + vui->full_range_flag = get_bits1(gb); + } else { + vui->colour_primaries = 2; + vui->transfer_characteristics = 2; + vui->matrix_coeffs = 2; + vui->full_range_flag = 0; + } + vui->chroma_loc_info_present_flag = get_bits1(gb); + if (chroma_format_idc != 1 && vui->chroma_loc_info_present_flag) { + av_log(avctx, AV_LOG_ERROR, "chroma_format_idc == %d," + "chroma_loc_info_present_flag can't not be true", chroma_format_idc); + return AVERROR_INVALIDDATA; + } + if (vui->chroma_loc_info_present_flag) { + if (vui->progressive_source_flag && + !vui->interlaced_source_flag) { + vui->chroma_sample_loc_type_frame = get_ue_golomb(gb); + } else { + vui->chroma_sample_loc_type_top_field = get_ue_golomb(gb); + vui->chroma_sample_loc_type_bottom_field = get_ue_golomb(gb); + } + } else { + if (chroma_format_idc == 1) { + vui->chroma_sample_loc_type_frame = get_ue_golomb(gb); + vui->chroma_sample_loc_type_top_field = get_ue_golomb(gb); + vui->chroma_sample_loc_type_bottom_field = get_ue_golomb(gb); + } + } + return 0; +} + +static int map_pixel_format(AVCodecContext *avctx, VVCSPS *sps) +{ + const AVPixFmtDescriptor *desc; + switch (sps->bit_depth) { + case 8: + if (sps->chroma_format_idc == 0) sps->pix_fmt = AV_PIX_FMT_GRAY8; + if (sps->chroma_format_idc == 1) sps->pix_fmt = AV_PIX_FMT_YUV420P; + if (sps->chroma_format_idc == 2) sps->pix_fmt = AV_PIX_FMT_YUV422P; + if (sps->chroma_format_idc == 3) sps->pix_fmt = AV_PIX_FMT_YUV444P; + break; + case 9: + if (sps->chroma_format_idc == 0) sps->pix_fmt = AV_PIX_FMT_GRAY9; + if (sps->chroma_format_idc == 1) sps->pix_fmt = AV_PIX_FMT_YUV420P9; + if (sps->chroma_format_idc == 2) sps->pix_fmt = AV_PIX_FMT_YUV422P9; + if (sps->chroma_format_idc == 3) sps->pix_fmt = AV_PIX_FMT_YUV444P9; + break; + case 10: + if (sps->chroma_format_idc == 0) sps->pix_fmt = AV_PIX_FMT_GRAY10; + if (sps->chroma_format_idc == 1) sps->pix_fmt = AV_PIX_FMT_YUV420P10; + if (sps->chroma_format_idc == 2) sps->pix_fmt = AV_PIX_FMT_YUV422P10; + if (sps->chroma_format_idc == 3) sps->pix_fmt = AV_PIX_FMT_YUV444P10; + break; + case 12: + if (sps->chroma_format_idc == 0) sps->pix_fmt = AV_PIX_FMT_GRAY12; + if (sps->chroma_format_idc == 1) sps->pix_fmt = AV_PIX_FMT_YUV420P12; + if (sps->chroma_format_idc == 2) sps->pix_fmt = AV_PIX_FMT_YUV422P12; + if (sps->chroma_format_idc == 3) sps->pix_fmt = AV_PIX_FMT_YUV444P12; + break; + default: + av_log(avctx, AV_LOG_ERROR, + "The following bit-depths are currently specified: 8, 9, 10 and 12 bits, " + "chroma_format_idc is %d, depth is %d\n", + sps->chroma_format_idc, sps->bit_depth); + return AVERROR_INVALIDDATA; + } + + desc = av_pix_fmt_desc_get(sps->pix_fmt); + if (!desc) + return AVERROR(EINVAL); + + return 0; +} + +int ff_vvc_parse_sps(VVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, + int apply_defdispwin, AVCodecContext *avctx) +{ + int i, j; + unsigned int ctb_log2_size_y, ctb_size_y, max_num_merge_cand, + tmp_width_val, tmp_height_val; + + sps->sps_id = get_bits(gb, 4); + sps->vps_id = get_bits(gb, 4); + + *sps_id = sps->sps_id; + + sps->max_sublayers = get_bits(gb, 3) + 1; + sps->chroma_format_idc = get_bits(gb, 2); + sps->log2_ctu_size = get_bits(gb, 2) + 5; + + ctb_log2_size_y = sps->log2_ctu_size; + ctb_size_y = 1 << ctb_log2_size_y; + + sps->ptl_dpb_hrd_params_present_flag = get_bits1(gb); + if (sps->ptl_dpb_hrd_params_present_flag) + decode_profile_tier_level( gb, avctx, &sps->profile_tier_level, 1, sps->max_sublayers-1 ); + + sps->gdr_enabled_flag = get_bits1(gb); + sps->ref_pic_resampling_enabled_flag = get_bits1(gb); + + if (sps->ref_pic_resampling_enabled_flag) + sps->res_change_in_clvs_allowed_flag = get_bits1(gb); + else + sps->res_change_in_clvs_allowed_flag = 0; + + sps->pic_width_max_in_luma_samples = get_ue_golomb(gb); + sps->pic_height_max_in_luma_samples = get_ue_golomb(gb); + + sps->conformance_window_flag = get_bits1(gb); + + if (sps->conformance_window_flag) { + sps->conf_win_left_offset = get_ue_golomb(gb); + sps->conf_win_right_offset = get_ue_golomb(gb); + sps->conf_win_top_offset = get_ue_golomb(gb); + sps->conf_win_bottom_offset = get_ue_golomb(gb); + } else { + sps->conf_win_left_offset = 0; + sps->conf_win_right_offset = 0; + sps->conf_win_top_offset = 0; + sps->conf_win_bottom_offset = 0; + } + + tmp_width_val = (sps->pic_width_max_in_luma_samples + ctb_size_y -1)/ctb_size_y; + tmp_height_val = (sps->pic_height_max_in_luma_samples + ctb_size_y -1)/ctb_size_y; + + sps->subpic_info_present_flag = get_bits1(gb); + if (sps->subpic_info_present_flag) { + sps->num_subpics_minus1 = get_ue_golomb(gb); + if (sps->num_subpics_minus1 > 0) { + sps->independent_subpics_flag = get_bits1(gb); + sps->subpic_same_size_flag = get_bits1(gb); + } + } + + if (sps->num_subpics_minus1 > 0) { + int wlen = av_ceil_log2(tmp_width_val); + int hlen = av_ceil_log2(tmp_height_val); + if (sps->pic_width_max_in_luma_samples > ctb_size_y) + sps->subpic_width_minus1[0] = get_bits(gb, wlen ); + else + sps->subpic_width_minus1[0] = tmp_width_val - 1; + + if (sps->pic_height_max_in_luma_samples > ctb_size_y) + sps->subpic_height_minus1[0] = get_bits(gb, hlen ); + else + sps->subpic_height_minus1[0] = tmp_height_val; + + if (!sps->independent_subpics_flag) { + sps->subpic_treated_as_pic_flag[0] = get_bits1(gb); + sps->loop_filter_across_subpic_enabled_flag[0] = get_bits1(gb); + } else { + sps->subpic_treated_as_pic_flag[0] = 1; + sps->loop_filter_across_subpic_enabled_flag[0] = 1; + } + + for (i = 1; i <= sps->num_subpics_minus1; i++) { + if (!sps->subpic_same_size_flag) { + if (sps->pic_width_max_in_luma_samples > ctb_size_y) + sps->subpic_ctu_top_left_x[i] = get_bits(gb, wlen ); + else + sps->subpic_ctu_top_left_x[i] = 0; + + if (sps->pic_height_max_in_luma_samples > ctb_size_y) + sps->subpic_ctu_top_left_y[i] = get_bits(gb, hlen ); + else + sps->subpic_ctu_top_left_y[i] = 0; + + if (i < sps->num_subpics_minus1 && + sps->pic_width_max_in_luma_samples > ctb_size_y) { + sps->subpic_width_minus1[i] = get_bits(gb, wlen ); + } else { + sps->subpic_width_minus1[i] = tmp_width_val - sps->subpic_ctu_top_left_x[i] - 1; + } + if (i < sps->num_subpics_minus1 && + sps->pic_height_max_in_luma_samples > ctb_size_y) { + sps->subpic_height_minus1[i] = get_bits(gb, hlen ); + } else { + sps->subpic_height_minus1[i] = tmp_height_val - sps->subpic_ctu_top_left_y[i] - 1; + } + } else { + int num_subpic_cols = + tmp_width_val / (sps->subpic_width_minus1[0] + 1); + sps->subpic_ctu_top_left_x[i] = (i % num_subpic_cols) *(sps->subpic_width_minus1[0] + 1); + sps->subpic_ctu_top_left_y[i] = (i / num_subpic_cols) *(sps->subpic_height_minus1[0] + 1); + sps->subpic_width_minus1[i] = sps->subpic_width_minus1[0]; + sps->subpic_height_minus1[i] = sps->subpic_height_minus1[0]; + } + if (!sps->independent_subpics_flag) { + sps->subpic_treated_as_pic_flag[i] = get_bits1(gb); + sps->loop_filter_across_subpic_enabled_flag[i] = get_bits1(gb); + } else { + sps->subpic_treated_as_pic_flag[i] = 1; + sps->loop_filter_across_subpic_enabled_flag[i] = 0; + } + } + sps->subpic_id_len_minus1 = get_ue_golomb(gb ); + + if ((1 << (sps->subpic_id_len_minus1 + 1)) < + sps->num_subpics_minus1 + 1) { + av_log(avctx, AV_LOG_ERROR, + "sps->subpic_id_len_minus1(%d) is too small\n", + sps->subpic_id_len_minus1); + return AVERROR_INVALIDDATA; + } + sps->subpic_id_mapping_explicitly_signalled_flag = get_bits1(gb); + if (sps->subpic_id_mapping_explicitly_signalled_flag) { + sps->subpic_id_mapping_present_flag = get_bits1(gb); + if (sps->subpic_id_mapping_present_flag) { + for (i = 0; i <= sps->num_subpics_minus1; i++) + sps->subpic_id[i] = get_bits(gb, sps->subpic_id_len_minus1 + 1 ); + } + } else { + sps->subpic_ctu_top_left_x[0]= 0; + sps->subpic_ctu_top_left_y[0]= 0; + sps->subpic_width_minus1[0] = tmp_width_val - 1; + sps->subpic_height_minus1[0]= tmp_height_val - 1; + } + } else { + sps->num_subpics_minus1 = 0; + sps->independent_subpics_flag = 1; + sps->subpic_same_size_flag = 0; + sps->subpic_id_mapping_explicitly_signalled_flag= 0; + sps->subpic_ctu_top_left_x[0] = 0; + sps->subpic_ctu_top_left_y[0] = 0; + sps->subpic_width_minus1[0] = tmp_width_val - 1; + sps->subpic_height_minus1[0] = tmp_height_val - 1; + } + + sps->bit_depth = get_ue_golomb(gb ) + 8; + + sps->entropy_coding_sync_enabled_flag = get_bits1(gb); + sps->entry_point_offsets_present_flag = get_bits1(gb); + + sps->log2_max_pic_order_cnt_lsb_minus4 = get_bits(gb, 4 ); + + sps->poc_msb_cycle_flag = get_bits1(gb); + if (sps->poc_msb_cycle_flag) + sps->poc_msb_cycle_len_minus1 = get_ue_golomb(gb); + + sps->num_extra_ph_bytes = get_bits(gb,2); + + for (i = 0; i < FFMIN(16,(sps->num_extra_ph_bytes * 8)); i++) { + sps->extra_ph_bit_present_flag[i] = get_bits1(gb); + } + + sps->num_extra_sh_bytes = get_bits(gb,2); + for (i = 0; i < FFMIN(16,(sps->num_extra_sh_bytes * 8)); i++) { + sps->extra_sh_bit_present_flag[i] = get_bits1(gb); + } + + if (sps->ptl_dpb_hrd_params_present_flag) { + if (sps->max_sublayers > 1) + sps->sublayer_dpb_params_flag = get_bits1(gb); + else + sps->sublayer_dpb_params_flag = 0; + + decode_dpb_parameters( gb, avctx, &sps->dpb_params, + sps->max_sublayers-1, + sps->sublayer_dpb_params_flag); + } + + sps->log2_min_luma_coding_block_size_minus2 = get_ue_golomb(gb); + sps->partition_constraints_override_enabled_flag = get_bits1(gb); + sps->log2_diff_min_qt_min_cb_intra_slice_luma = get_ue_golomb(gb); + sps->max_mtt_hierarchy_depth_intra_slice_luma = get_ue_golomb(gb); + + if (sps->max_mtt_hierarchy_depth_intra_slice_luma != 0) { + sps->log2_diff_max_bt_min_qt_intra_slice_luma = get_ue_golomb(gb); + sps->log2_diff_max_tt_min_qt_intra_slice_luma = get_ue_golomb(gb); + } else { + sps->log2_diff_max_bt_min_qt_intra_slice_luma = 0; + sps->log2_diff_max_tt_min_qt_intra_slice_luma = 0; + } + + if (sps->chroma_format_idc != 0) { + sps->qtbtt_dual_tree_intra_flag = get_bits1(gb); + } else { + sps->qtbtt_dual_tree_intra_flag = 0; + } + + if (sps->qtbtt_dual_tree_intra_flag) { + sps->log2_diff_min_qt_min_cb_intra_slice_chroma = get_ue_golomb(gb); + sps->max_mtt_hierarchy_depth_intra_slice_chroma = get_ue_golomb(gb); + if (sps->max_mtt_hierarchy_depth_intra_slice_chroma != 0) { + sps->log2_diff_max_bt_min_qt_intra_slice_chroma = get_ue_golomb(gb); + sps->log2_diff_max_tt_min_qt_intra_slice_chroma = get_ue_golomb(gb); + } + } else { + sps->log2_diff_min_qt_min_cb_intra_slice_chroma = 0; + sps->max_mtt_hierarchy_depth_intra_slice_chroma = 0; + } + if (sps->max_mtt_hierarchy_depth_intra_slice_chroma == 0) { + sps->log2_diff_max_bt_min_qt_intra_slice_chroma = 0; + sps->log2_diff_max_tt_min_qt_intra_slice_chroma = 0; + } + + sps->log2_diff_min_qt_min_cb_inter_slice = get_ue_golomb(gb); + + sps->max_mtt_hierarchy_depth_inter_slice = get_ue_golomb(gb); + if (sps->max_mtt_hierarchy_depth_inter_slice != 0) { + sps->log2_diff_max_bt_min_qt_inter_slice = get_ue_golomb(gb); + sps->log2_diff_max_tt_min_qt_inter_slice = get_ue_golomb(gb); + } else { + sps->log2_diff_max_bt_min_qt_inter_slice = 0; + sps->log2_diff_max_tt_min_qt_inter_slice = 0; + } + + if (ctb_size_y > 32) + sps->max_luma_transform_size_64_flag = get_bits1(gb); + else + sps->max_luma_transform_size_64_flag = 0; + + sps->transform_skip_enabled_flag = get_bits1(gb); + if (sps->transform_skip_enabled_flag) { + sps->log2_transform_skip_max_size_minus2 = get_ue_golomb(gb); + sps->bdpcm_enabled_flag = get_bits1(gb); + } + + sps->mts_enabled_flag = get_bits1(gb); + if (sps->mts_enabled_flag) { + sps->explicit_mts_intra_enabled_flag = get_bits1(gb); + sps->explicit_mts_inter_enabled_flag = get_bits1(gb); + } else { + sps->explicit_mts_intra_enabled_flag = 0; + sps->explicit_mts_inter_enabled_flag = 0; + } + + sps->lfnst_enabled_flag = get_bits1(gb); + + if (sps->chroma_format_idc != 0) { + uint8_t num_qp_tables; + sps->joint_cbcr_enabled_flag = get_bits1(gb); + sps->same_qp_table_for_chroma_flag = get_bits1(gb); + num_qp_tables = sps->same_qp_table_for_chroma_flag ? + 1 : (sps->joint_cbcr_enabled_flag ? 3 : 2); + for (i = 0; i < num_qp_tables; i++) { + sps->qp_table_start_minus26[i] = get_se_golomb(gb); + sps->num_points_in_qp_table_minus1[i] = get_ue_golomb(gb); + for (j = 0; j <= sps->num_points_in_qp_table_minus1[i]; j++) { + sps->delta_qp_in_val_minus1[i][j] = get_ue_golomb(gb); + sps->delta_qp_diff_val[i][j] = get_ue_golomb(gb); + } + } + } else { + sps->joint_cbcr_enabled_flag = 0; + sps->same_qp_table_for_chroma_flag = 0; + } + + sps->sao_enabled_flag = get_bits1(gb); + sps->alf_enabled_flag = get_bits1(gb); + if (sps->alf_enabled_flag && sps->chroma_format_idc) + sps->ccalf_enabled_flag = get_bits1(gb); + else + sps->ccalf_enabled_flag = 0; + + sps->lmcs_enabled_flag = get_bits1(gb); + sps->weighted_pred_flag = get_bits1(gb); + sps->weighted_bipred_flag = get_bits1(gb); + sps->long_term_ref_pics_flag = get_bits1(gb); + if (sps->vps_id > 0) + sps->inter_layer_prediction_enabled_flag = get_bits1(gb); + else + sps->inter_layer_prediction_enabled_flag = 0; + sps->idr_rpl_present_flag = get_bits1(gb); + sps->rpl1_same_as_rpl0_flag = get_bits1(gb); + + for (i = 0; i < (sps->rpl1_same_as_rpl0_flag ? 1 : 2); i++) { + sps->num_ref_pic_lists[i] = get_ue_golomb(gb); + for (j = 0; j < sps->num_ref_pic_lists[i]; j++) + decode_ref_pic_list_struct( gb, avctx, + &sps->ref_pic_list_struct[i][j], + i, j, sps); + } + + if (sps->rpl1_same_as_rpl0_flag) { + sps->num_ref_pic_lists[1] = sps->num_ref_pic_lists[0]; + for (j = 0; j < sps->num_ref_pic_lists[0]; j++) + memcpy(&sps->ref_pic_list_struct[1][j], + &sps->ref_pic_list_struct[0][j], + sizeof(sps->ref_pic_list_struct[0][j])); + } + + sps->ref_wraparound_enabled_flag = get_bits1(gb); + + sps->temporal_mvp_enabled_flag = get_bits1(gb); + if (sps->temporal_mvp_enabled_flag) + sps->sbtmvp_enabled_flag = get_bits1(gb); + else + sps->sbtmvp_enabled_flag = 0; + + sps->amvr_enabled_flag = get_bits1(gb); + sps->bdof_enabled_flag = get_bits1(gb); + if (sps->bdof_enabled_flag) + sps->bdof_control_present_in_ph_flag = get_bits1(gb); + else + sps->bdof_control_present_in_ph_flag = 0; + + sps->smvd_enabled_flag = get_bits1(gb); + sps->dmvr_enabled_flag = get_bits1(gb); + if (sps->dmvr_enabled_flag) + sps->dmvr_control_present_in_ph_flag = get_bits1(gb); + else + sps->dmvr_control_present_in_ph_flag = 0; + + sps->mmvd_enabled_flag = get_bits1(gb); + if (sps->mmvd_enabled_flag) + sps->mmvd_fullpel_only_enabled_flag = get_bits1(gb); + else + sps->mmvd_fullpel_only_enabled_flag = 0; + + sps->six_minus_max_num_merge_cand = get_ue_golomb(gb); + max_num_merge_cand = 6 - sps->six_minus_max_num_merge_cand; + + sps->sbt_enabled_flag = get_bits1(gb); + + sps->affine_enabled_flag = get_bits1(gb); + if (sps->affine_enabled_flag) { + sps->five_minus_max_num_subblock_merge_cand = get_ue_golomb(gb); + sps->param_affine_enabled_flag = get_bits1(gb); + if (sps->amvr_enabled_flag) + sps->affine_amvr_enabled_flag = get_bits1(gb); + else + sps->affine_amvr_enabled_flag = 0; + sps->affine_prof_enabled_flag = get_bits1(gb); + if (sps->affine_prof_enabled_flag) + sps->prof_control_present_in_ph_flag = get_bits1(gb); + else + sps->prof_control_present_in_ph_flag = 0; + } else { + sps->param_affine_enabled_flag = 0; + sps->affine_amvr_enabled_flag = 0; + sps->affine_prof_enabled_flag = 0; + sps->prof_control_present_in_ph_flag = 0; + } + + sps->bcw_enabled_flag = get_bits1(gb); + sps->ciip_enabled_flag = get_bits1(gb); + + if (max_num_merge_cand >= 2) { + sps->gpm_enabled_flag = get_bits1(gb); + if (sps->gpm_enabled_flag && max_num_merge_cand >= 3) + sps->max_num_merge_cand_minus_max_num_gpm_cand = get_ue_golomb(gb); + } else { + sps->gpm_enabled_flag = 0; + } + + sps->log2_parallel_merge_level_minus2 = get_ue_golomb(gb); + + sps->isp_enabled_flag = get_bits1(gb); + sps->mrl_enabled_flag = get_bits1(gb); + sps->mip_enabled_flag = get_bits1(gb); + + if (sps->chroma_format_idc != 0) + sps->cclm_enabled_flag = get_bits1(gb); + else + sps->cclm_enabled_flag = 0; + if (sps->chroma_format_idc == 1) { + sps->chroma_horizontal_collocated_flag = get_bits1(gb); + sps->chroma_vertical_collocated_flag = get_bits1(gb); + } else { + sps->chroma_horizontal_collocated_flag = 0; + sps->chroma_vertical_collocated_flag = 0; + } + + sps->palette_enabled_flag = get_bits1(gb); + if (sps->chroma_format_idc == 3 && + !sps->max_luma_transform_size_64_flag) + sps->act_enabled_flag = get_bits1(gb); + else + sps->act_enabled_flag = 0; + if (sps->transform_skip_enabled_flag || + sps->palette_enabled_flag) + sps->min_qp_prime_ts = get_ue_golomb(gb); + + sps->ibc_enabled_flag = get_bits1(gb); + if (sps->ibc_enabled_flag) + sps->six_minus_max_num_ibc_merge_cand = get_ue_golomb(gb); + + sps->ladf_enabled_flag = get_bits1(gb); + if (sps->ladf_enabled_flag) { + sps->num_ladf_intervals_minus2 = get_bits( gb, 2 ); + sps->ladf_lowest_interval_qp_offset = get_se_golomb(gb); + for (i = 0; i < sps->num_ladf_intervals_minus2 + 1; i++) { + sps->ladf_qp_offset[i] = get_se_golomb(gb); + sps->ladf_delta_threshold_minus1[i] = get_ue_golomb(gb); + } + } + + sps->explicit_scaling_list_enabled_flag = get_bits1(gb); + if (sps->lfnst_enabled_flag && + sps->explicit_scaling_list_enabled_flag) + sps->scaling_matrix_for_lfnst_disabled_flag = get_bits1(gb); + + if (sps->act_enabled_flag && + sps->explicit_scaling_list_enabled_flag) + sps->scaling_matrix_for_alternative_colour_space_disabled_flag = get_bits1(gb); + else + sps->scaling_matrix_for_alternative_colour_space_disabled_flag = 0; + if (sps->scaling_matrix_for_alternative_colour_space_disabled_flag) + sps->scaling_matrix_designated_colour_space_flag = get_bits1(gb); + + sps->dep_quant_enabled_flag = get_bits1(gb); + sps->sign_data_hiding_enabled_flag = get_bits1(gb); + + sps->virtual_boundaries_enabled_flag = get_bits1(gb); + if (sps->virtual_boundaries_enabled_flag) { + sps->virtual_boundaries_present_flag = get_bits1(gb); + if (sps->virtual_boundaries_present_flag) { + sps->num_ver_virtual_boundaries = get_ue_golomb(gb); + for (i = 0; i < sps->num_ver_virtual_boundaries; i++) + sps->virtual_boundary_pos_x_minus1[i] = get_ue_golomb(gb); + for (i = 0; i < sps->num_hor_virtual_boundaries; i++) + sps->virtual_boundary_pos_y_minus1[i] = get_ue_golomb(gb); + } + } else { + sps->virtual_boundaries_present_flag = 0; + sps->num_ver_virtual_boundaries = 0; + sps->num_hor_virtual_boundaries = 0; + } + + if (sps->ptl_dpb_hrd_params_present_flag) { + sps->timing_hrd_params_present_flag = get_bits1(gb); + if (sps->timing_hrd_params_present_flag) { + uint8_t first_sublayer; + decode_general_timing_hrd_parameters(gb, &sps->general_timing_hrd_parameters); + + if (sps->max_sublayers > 1) + sps->sublayer_cpb_params_present_flag = get_bits1(gb); + else + sps->sublayer_cpb_params_present_flag = 0; + + first_sublayer = sps->sublayer_cpb_params_present_flag ? + 0 : sps->max_sublayers-1; + + decode_ols_timing_hrd_parameters ( gb, + &sps->ols_timing_hrd_parameters, first_sublayer, + sps->max_sublayers-1, + &sps->general_timing_hrd_parameters); + } + } + + sps->field_seq_flag = get_bits1(gb); + sps->vui_parameters_present_flag = get_bits1(gb); + if (sps->vui_parameters_present_flag) { + sps->vui_payload_size_minus1 = get_ue_golomb(gb); + align_get_bits(gb); + decode_vui( gb, avctx, &sps->vui, sps->chroma_format_idc); + } + + sps->extension_flag = get_bits1(gb); + // TODO: parse sps extension flag and read extension data + + map_pixel_format( avctx, sps); + + return 0; +} + +int ff_vvc_decode_nal_sps( GetBitContext *gb, AVCodecContext *avctx, + VVCParamSets *ps, int apply_defdispwin) +{ + unsigned int sps_id; + int ret; + VVCSPS *sps; + AVBufferRef *sps_buf = av_buffer_allocz(sizeof(*sps)); + + if (!sps_buf) + return AVERROR(ENOMEM); + sps = (VVCSPS*)sps_buf->data; + + ret = ff_vvc_parse_sps(sps, gb, &sps_id, + apply_defdispwin, + avctx); + if (ret < 0) { + av_buffer_unref(&sps_buf); + return ret; + } + + if (avctx->debug & FF_DEBUG_BITSTREAM) + { + av_log(avctx, AV_LOG_DEBUG, + "Parsed SPS: id %d; coded wxh: %dx%d; " + "pix_fmt: %s.\n", + sps->sps_id, sps->pic_width_max_in_luma_samples, sps->pic_height_max_in_luma_samples, + av_get_pix_fmt_name(sps->pix_fmt)); + } + + /* check if this is a repeat of an already parsed SPS, then keep the + * original one otherwise drop all PPSes that depend on it (PPS,VPS not implemented yet) */ + if (ps->sps_list[sps_id] && + !memcmp(ps->sps_list[sps_id]->data, sps_buf->data, sps_buf->size)) { + av_buffer_unref(&sps_buf); + } else { + remove_sps(ps, sps_id); + ps->sps_list[sps_id] = sps_buf; + ps->sps = (VVCSPS*)ps->sps_list[sps_id]->data; + } + + // TODO: read PPS flag and data + + return 0; +} + +void ff_vvc_ps_uninit(VVCParamSets *ps) +{ + int i; + + for (i = 0; i < FF_ARRAY_ELEMS(ps->sps_list); i++) + av_buffer_unref(&ps->sps_list[i]); + + // TODO: if PPS, VPS is implemended it must be unrefed as well + // for (i = 0; i < FF_ARRAY_ELEMS(ps->vps_list); i++) + // av_buffer_unref(&ps->vps_list[i]); + // for (i = 0; i < FF_ARRAY_ELEMS(ps->pps_list); i++) + // av_buffer_unref(&ps->pps_list[i]); + + ps->sps = NULL; + //ps->pps = NULL; + //ps->vps = NULL; +} diff --git a/libavcodec/vvc_paramset.h b/libavcodec/vvc_paramset.h new file mode 100644 index 0000000000..c4bb2e509b --- /dev/null +++ b/libavcodec/vvc_paramset.h @@ -0,0 +1,429 @@ +/* + * VVC parameter set parsing + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_VVC_PARAMSET_H +#define AVCODEC_VVC_PARAMSET_H + +#include + +#include "libavutil/buffer.h" +#include "libavutil/pixfmt.h" +#include "libavutil/rational.h" + +#include "avcodec.h" +#include "get_bits.h" +#include "vvc.h" +#include "cbs_h266.h" + + +typedef struct VVCGeneralConstraintsInfo { + uint8_t gci_present_flag; + /* general */ + uint8_t gci_intra_only_constraint_flag; + uint8_t gci_all_layers_independent_constraint_flag; + uint8_t gci_one_au_only_constraint_flag; + + /* picture format */ + uint8_t gci_sixteen_minus_max_bitdepth_constraint_idc; + uint8_t gci_three_minus_max_chroma_format_constraint_idc; + + /* NAL unit type related */ + uint8_t gci_no_mixed_nalu_types_in_pic_constraint_flag; + uint8_t gci_no_trail_constraint_flag; + uint8_t gci_no_stsa_constraint_flag; + uint8_t gci_no_rasl_constraint_flag; + uint8_t gci_no_radl_constraint_flag; + uint8_t gci_no_idr_constraint_flag; + uint8_t gci_no_cra_constraint_flag; + uint8_t gci_no_gdr_constraint_flag; + uint8_t gci_no_aps_constraint_flag; + uint8_t gci_no_idr_rpl_constraint_flag; + + /* tile, slice, subpicture partitioning */ + uint8_t gci_one_tile_per_pic_constraint_flag; + uint8_t gci_pic_header_in_slice_header_constraint_flag; + uint8_t gci_one_slice_per_pic_constraint_flag; + uint8_t gci_no_rectangular_slice_constraint_flag; + uint8_t gci_one_slice_per_subpic_constraint_flag; + uint8_t gci_no_subpic_info_constraint_flag; + + /* CTU and block partitioning */ + uint8_t gci_three_minus_max_log2_ctu_size_constraint_idc; + uint8_t gci_no_partition_constraints_override_constraint_flag; + uint8_t gci_no_mtt_constraint_flag; + uint8_t gci_no_qtbtt_dual_tree_intra_constraint_flag; + + /* intra */ + uint8_t gci_no_palette_constraint_flag; + uint8_t gci_no_ibc_constraint_flag; + uint8_t gci_no_isp_constraint_flag; + uint8_t gci_no_mrl_constraint_flag; + uint8_t gci_no_mip_constraint_flag; + uint8_t gci_no_cclm_constraint_flag; + + /* inter */ + uint8_t gci_no_ref_pic_resampling_constraint_flag; + uint8_t gci_no_res_change_in_clvs_constraint_flag;; + uint8_t gci_no_weighted_prediction_constraint_flag; + uint8_t gci_no_ref_wraparound_constraint_flag; + uint8_t gci_no_temporal_mvp_constraint_flag; + uint8_t gci_no_sbtmvp_constraint_flag; + uint8_t gci_no_amvr_constraint_flag; + uint8_t gci_no_bdof_constraint_flag; + uint8_t gci_no_smvd_constraint_flag; + uint8_t gci_no_dmvr_constraint_flag; + uint8_t gci_no_mmvd_constraint_flag; + uint8_t gci_no_affine_motion_constraint_flag; + uint8_t gci_no_prof_constraint_flag; + uint8_t gci_no_bcw_constraint_flag; + uint8_t gci_no_ciip_constraint_flag; + uint8_t gci_no_gpm_constraint_flag; + + /* transform, quantization, residual */ + uint8_t gci_no_luma_transform_size_64_constraint_flag; + uint8_t gci_no_transform_skip_constraint_flag; + uint8_t gci_no_bdpcm_constraint_flag; + uint8_t gci_no_mts_constraint_flag; + uint8_t gci_no_lfnst_constraint_flag; + uint8_t gci_no_joint_cbcr_constraint_flag; + uint8_t gci_no_sbt_constraint_flag; + uint8_t gci_no_act_constraint_flag; + uint8_t gci_no_explicit_scaling_list_constraint_flag; + uint8_t gci_no_dep_quant_constraint_flag; + uint8_t gci_no_sign_data_hiding_constraint_flag; + uint8_t gci_no_cu_qp_delta_constraint_flag; + uint8_t gci_no_chroma_qp_offset_constraint_flag; + + /* loop filter */ + uint8_t gci_no_sao_constraint_flag; + uint8_t gci_no_alf_constraint_flag; + uint8_t gci_no_ccalf_constraint_flag; + uint8_t gci_no_lmcs_constraint_flag; + uint8_t gci_no_ladf_constraint_flag; + uint8_t gci_no_virtual_boundaries_constraint_flag; + uint8_t gci_num_reserved_bits; + uint8_t gci_reserved_zero_bit[255]; +} VVCGeneralConstraintsInfo; + +typedef struct VVCProfileTierLevel { + uint8_t general_profile_idc; + uint8_t general_tier_flag; + uint8_t general_level_idc; + uint8_t ptl_frame_only_constraint_flag; + uint8_t ptl_multilayer_enabled_flag; + VVCGeneralConstraintsInfo general_constraints_info; + uint8_t ptl_sublayer_level_present_flag[VVC_MAX_SUBLAYERS - 1]; + uint8_t sublayer_level_idc[VVC_MAX_SUBLAYERS - 1]; + uint8_t ptl_num_sub_profiles; + uint32_t general_sub_profile_idc[VVC_MAX_SUB_PROFILES]; + + uint8_t ptl_reserved_zero_bit; +} VVCProfileTierLevel; + +typedef struct VVCDpbParameters { + uint8_t dpb_max_dec_pic_buffering_minus1[VVC_MAX_SUBLAYERS]; + uint8_t dpb_max_num_reorder_pics[VVC_MAX_SUBLAYERS]; + uint8_t dpb_max_latency_increase_plus1[VVC_MAX_SUBLAYERS]; +} VVCDpbParameters; + +typedef struct VVCRefPicListStruct { + uint8_t num_ref_entries; + uint8_t ltrp_in_header_flag; + uint8_t inter_layer_ref_pic_flag[VVC_MAX_REF_ENTRIES]; + uint8_t st_ref_pic_flag[VVC_MAX_REF_ENTRIES]; + uint8_t abs_delta_poc_st[VVC_MAX_REF_ENTRIES]; + uint8_t strp_entry_sign_flag[VVC_MAX_REF_ENTRIES]; + uint8_t rpls_poc_lsb_lt[VVC_MAX_REF_ENTRIES]; + uint8_t ilrp_idx[VVC_MAX_REF_ENTRIES]; +} VVCRefPicListStruct; + +typedef struct VVCGeneralTimingHrdParameters { + uint32_t num_units_in_tick; + uint32_t time_scale; + uint8_t general_nal_hrd_params_present_flag; + uint8_t general_vcl_hrd_params_present_flag; + uint8_t general_same_pic_timing_in_all_ols_flag; + uint8_t general_du_hrd_params_present_flag; + uint8_t tick_divisor_minus2; + uint8_t bit_rate_scale; + uint8_t cpb_size_scale; + uint8_t cpb_size_du_scale; + uint8_t hrd_cpb_cnt_minus1; +} VVCGeneralTimingHrdParameters; + +typedef struct VVCSubLayerHRDParameters { + uint32_t bit_rate_value_minus1[VVC_MAX_SUBLAYERS][VVC_MAX_CPB_CNT]; + uint32_t cpb_size_value_minus1[VVC_MAX_SUBLAYERS][VVC_MAX_CPB_CNT]; + uint32_t cpb_size_du_value_minus1[VVC_MAX_SUBLAYERS][VVC_MAX_CPB_CNT]; + uint32_t bit_rate_du_value_minus1[VVC_MAX_SUBLAYERS][VVC_MAX_CPB_CNT]; + uint8_t cbr_flag[VVC_MAX_SUBLAYERS][VVC_MAX_CPB_CNT]; +} VVCSubLayerHRDParameters; + +typedef struct VVCOlsTimingHrdParameters { + uint8_t fixed_pic_rate_general_flag[VVC_MAX_SUBLAYERS]; + uint8_t fixed_pic_rate_within_cvs_flag[VVC_MAX_SUBLAYERS]; + uint16_t elemental_duration_in_tc_minus1[VVC_MAX_SUBLAYERS]; + uint8_t low_delay_hrd_flag[VVC_MAX_SUBLAYERS]; + VVCSubLayerHRDParameters nal_sub_layer_hrd_parameters; + VVCSubLayerHRDParameters vcl_sub_layer_hrd_parameters; +} VVCOlsTimingHrdParameters; + + +typedef struct VVCVUI { + uint8_t progressive_source_flag; + uint8_t interlaced_source_flag; + uint8_t non_packed_constraint_flag; + uint8_t non_projected_constraint_flag; + + uint8_t aspect_ratio_info_present_flag; + uint8_t aspect_ratio_constant_flag; + uint8_t aspect_ratio_idc; + + uint16_t sar_width; + uint16_t sar_height;; + + uint8_t overscan_info_present_flag; + uint8_t overscan_appropriate_flag; + + uint8_t colour_description_present_flag; + uint8_t colour_primaries; + + uint8_t transfer_characteristics; + uint8_t matrix_coeffs; + uint8_t full_range_flag; + + uint8_t chroma_loc_info_present_flag; + uint8_t chroma_sample_loc_type_frame; + uint8_t chroma_sample_loc_type_top_field; + uint8_t chroma_sample_loc_type_bottom_field; + //VVCExtensionData extension_data; +} VVCVUI; + +typedef struct VVCSPS { + uint8_t sps_id; + uint8_t vps_id; + uint8_t max_sublayers; + uint8_t chroma_format_idc; + uint8_t log2_ctu_size; + uint8_t ptl_dpb_hrd_params_present_flag; + VVCProfileTierLevel profile_tier_level; + uint8_t gdr_enabled_flag; + uint8_t ref_pic_resampling_enabled_flag; + uint8_t res_change_in_clvs_allowed_flag; + + uint16_t pic_width_max_in_luma_samples; + uint16_t pic_height_max_in_luma_samples; + + uint8_t conformance_window_flag; + uint16_t conf_win_left_offset; + uint16_t conf_win_right_offset; + uint16_t conf_win_top_offset; + uint16_t conf_win_bottom_offset; + + uint8_t subpic_info_present_flag; + uint16_t num_subpics_minus1; + uint8_t independent_subpics_flag; + uint8_t subpic_same_size_flag; + uint16_t subpic_ctu_top_left_x[VVC_MAX_SLICES]; + uint16_t subpic_ctu_top_left_y[VVC_MAX_SLICES]; + uint16_t subpic_width_minus1[VVC_MAX_SLICES]; + uint16_t subpic_height_minus1[VVC_MAX_SLICES]; + uint8_t subpic_treated_as_pic_flag[VVC_MAX_SLICES]; + uint8_t loop_filter_across_subpic_enabled_flag[VVC_MAX_SLICES]; + uint8_t subpic_id_len_minus1; + uint8_t subpic_id_mapping_explicitly_signalled_flag; + uint8_t subpic_id_mapping_present_flag; + uint32_t subpic_id[VVC_MAX_SLICES]; + + + uint8_t bit_depth; + uint8_t entropy_coding_sync_enabled_flag; + uint8_t entry_point_offsets_present_flag; + + uint8_t log2_max_pic_order_cnt_lsb_minus4; + uint8_t poc_msb_cycle_flag; + uint8_t poc_msb_cycle_len_minus1; + + uint8_t num_extra_ph_bytes; + uint8_t extra_ph_bit_present_flag[16]; + + uint8_t num_extra_sh_bytes; + uint8_t extra_sh_bit_present_flag[16]; + + uint8_t sublayer_dpb_params_flag; + VVCDpbParameters dpb_params; + + uint8_t log2_min_luma_coding_block_size_minus2; + uint8_t partition_constraints_override_enabled_flag; + uint8_t log2_diff_min_qt_min_cb_intra_slice_luma; + uint8_t max_mtt_hierarchy_depth_intra_slice_luma; + uint8_t log2_diff_max_bt_min_qt_intra_slice_luma; + uint8_t log2_diff_max_tt_min_qt_intra_slice_luma; + + uint8_t qtbtt_dual_tree_intra_flag; + uint8_t log2_diff_min_qt_min_cb_intra_slice_chroma; + uint8_t max_mtt_hierarchy_depth_intra_slice_chroma; + uint8_t log2_diff_max_bt_min_qt_intra_slice_chroma; + uint8_t log2_diff_max_tt_min_qt_intra_slice_chroma; + + uint8_t log2_diff_min_qt_min_cb_inter_slice; + uint8_t max_mtt_hierarchy_depth_inter_slice; + uint8_t log2_diff_max_bt_min_qt_inter_slice; + uint8_t log2_diff_max_tt_min_qt_inter_slice; + + uint8_t max_luma_transform_size_64_flag; + + uint8_t transform_skip_enabled_flag; + uint8_t log2_transform_skip_max_size_minus2; + uint8_t bdpcm_enabled_flag; + + uint8_t mts_enabled_flag; + uint8_t explicit_mts_intra_enabled_flag; + uint8_t explicit_mts_inter_enabled_flag; + + uint8_t lfnst_enabled_flag; + + uint8_t joint_cbcr_enabled_flag; + uint8_t same_qp_table_for_chroma_flag; + + int8_t qp_table_start_minus26[VVC_MAX_SAMPLE_ARRAYS]; + uint8_t num_points_in_qp_table_minus1[VVC_MAX_SAMPLE_ARRAYS]; + uint8_t delta_qp_in_val_minus1[VVC_MAX_SAMPLE_ARRAYS][VVC_MAX_POINTS_IN_QP_TABLE]; + uint8_t delta_qp_diff_val[VVC_MAX_SAMPLE_ARRAYS][VVC_MAX_POINTS_IN_QP_TABLE]; + + uint8_t sao_enabled_flag; + uint8_t alf_enabled_flag; + uint8_t ccalf_enabled_flag; + uint8_t lmcs_enabled_flag; + uint8_t weighted_pred_flag; + uint8_t weighted_bipred_flag; + uint8_t long_term_ref_pics_flag; + uint8_t inter_layer_prediction_enabled_flag; + uint8_t idr_rpl_present_flag; + uint8_t rpl1_same_as_rpl0_flag; + + uint8_t num_ref_pic_lists[2]; + VVCRefPicListStruct ref_pic_list_struct[2][VVC_MAX_REF_PIC_LISTS]; + + uint8_t ref_wraparound_enabled_flag; + uint8_t temporal_mvp_enabled_flag; + uint8_t sbtmvp_enabled_flag; + uint8_t amvr_enabled_flag; + uint8_t bdof_enabled_flag; + uint8_t bdof_control_present_in_ph_flag; + uint8_t smvd_enabled_flag; + uint8_t dmvr_enabled_flag; + uint8_t dmvr_control_present_in_ph_flag; + uint8_t mmvd_enabled_flag; + uint8_t mmvd_fullpel_only_enabled_flag; + uint8_t six_minus_max_num_merge_cand; + uint8_t sbt_enabled_flag; + uint8_t affine_enabled_flag; + uint8_t five_minus_max_num_subblock_merge_cand; + uint8_t param_affine_enabled_flag; + uint8_t affine_amvr_enabled_flag; + uint8_t affine_prof_enabled_flag; + uint8_t prof_control_present_in_ph_flag; + uint8_t bcw_enabled_flag; + uint8_t ciip_enabled_flag; + uint8_t gpm_enabled_flag; + uint8_t max_num_merge_cand_minus_max_num_gpm_cand; + uint8_t log2_parallel_merge_level_minus2; + uint8_t isp_enabled_flag; + uint8_t mrl_enabled_flag; + uint8_t mip_enabled_flag; + uint8_t cclm_enabled_flag; + uint8_t chroma_horizontal_collocated_flag; + uint8_t chroma_vertical_collocated_flag; + uint8_t palette_enabled_flag; + uint8_t act_enabled_flag; + uint8_t min_qp_prime_ts; + uint8_t ibc_enabled_flag; + uint8_t six_minus_max_num_ibc_merge_cand; + uint8_t ladf_enabled_flag; + uint8_t num_ladf_intervals_minus2; + int8_t ladf_lowest_interval_qp_offset; + int8_t ladf_qp_offset[4]; + uint16_t ladf_delta_threshold_minus1[4]; + + uint8_t explicit_scaling_list_enabled_flag; + uint8_t scaling_matrix_for_lfnst_disabled_flag; + uint8_t scaling_matrix_for_alternative_colour_space_disabled_flag; + uint8_t scaling_matrix_designated_colour_space_flag; + uint8_t dep_quant_enabled_flag; + uint8_t sign_data_hiding_enabled_flag; + + uint8_t virtual_boundaries_enabled_flag; + uint8_t virtual_boundaries_present_flag; + uint8_t num_ver_virtual_boundaries; + uint16_t virtual_boundary_pos_x_minus1[3]; + uint8_t num_hor_virtual_boundaries; + uint16_t virtual_boundary_pos_y_minus1[3]; + + uint8_t timing_hrd_params_present_flag; + uint8_t sublayer_cpb_params_present_flag; + VVCGeneralTimingHrdParameters general_timing_hrd_parameters; + VVCOlsTimingHrdParameters ols_timing_hrd_parameters; + + uint8_t field_seq_flag; + uint8_t vui_parameters_present_flag; + uint16_t vui_payload_size_minus1; + VVCVUI vui; + + uint8_t extension_flag; + //H266RawExtensionData extension_data; /* TODO: read extension flag and data*/ + + enum AVPixelFormat pix_fmt; +} VVCSPS; + + +typedef struct VVCParamSets { + AVBufferRef *sps_list[VVC_MAX_SPS_COUNT]; + //AVBufferRef *vps_list[VVC_MAX_VPS_COUNT]; // TODO: since not needed, not implemented yet + //AVBufferRef *pps_list[VVC_MAX_PPS_COUNT]; // TODO: since not needed, not implemented yet + + /* currently active parameter sets */ + const VVCSPS *sps; + //const VVCVPS *vps; // TODO: since not needed, not implemented yet + //const VVCPPS *pps; // TODO: since not needed, not implemented yet +} VVCParamSets; + +/** + * Parse the SPS from the bitstream into the provided HEVCSPS struct. + * + * @param sps_id the SPS id will be written here + * @param apply_defdispwin if set 1, the default display window from the VUI + * will be applied to the video dimensions + */ +int ff_vvc_parse_sps(VVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, + int apply_defdispwin, AVCodecContext *avctx); + +int ff_vvc_decode_nal_sps(GetBitContext *gb, AVCodecContext *avctx, + VVCParamSets *ps, int apply_defdispwin); + +// TODO: since not needed, not implemented yet +//int ff_vvc_decode_nal_vps(GetBitContext *gb, AVCodecContext *avctx, +// VVCParamSets *ps); +//int ff_vvc_decode_nal_pps(GetBitContext *gb, AVCodecContext *avctx, +// VVCParamSets *ps); + +void ff_vvc_ps_uninit(VVCParamSets *ps); + +#endif /* AVCODEC_VVC_PARAMSET_H */ diff --git a/libavcodec/vvc_parse_extradata.c b/libavcodec/vvc_parse_extradata.c new file mode 100644 index 0000000000..8011ec861b --- /dev/null +++ b/libavcodec/vvc_parse_extradata.c @@ -0,0 +1,241 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "bytestream.h" +#include "h2645_parse.h" +#include "vvc.h" +#include "vvc_parse_extradata.h" + +static int vvc_decode_nal_units(const uint8_t *buf, int buf_size, VVCParamSets *ps, + int is_nalff, int nal_length_size, + int err_recognition, int apply_defdispwin, void *logctx) +{ + int i; + int ret = 0; + H2645Packet pkt = { 0 }; + + ret = ff_h2645_packet_split(&pkt, buf, buf_size, logctx, is_nalff, + nal_length_size, AV_CODEC_ID_VVC, 1, 0); + if (ret < 0) { + goto done; + } + + for (i = 0; i < pkt.nb_nals; i++) { + H2645NAL *nal = &pkt.nals[i]; + if (nal->nuh_layer_id > 0) + continue; + + /* ignore everything except parameter sets and VCL NALUs */ + switch (nal->type) { + case VVC_VPS_NUT: + // TODO: since not needed yet, not implemented + //ret = ff_vvc_decode_nal_vps(&nal->gb, logctx, ps); + //if (ret < 0) + // goto done; + break; + case VVC_SPS_NUT: + ret = ff_vvc_decode_nal_sps(&nal->gb, logctx, ps, apply_defdispwin); + if (ret < 0) + goto done; + break; + case VVC_PPS_NUT: + // TODO: since not needed yet, not implemented + //ret = ff_vvc_decode_nal_pps(&nal->gb, logctx, ps); + //if (ret < 0) + // goto done; + break; + case VVC_PREFIX_SEI_NUT: + case VVC_SUFFIX_SEI_NUT: + // TODO: SEI decoding not implemented yet + break; + default: + av_log(logctx, AV_LOG_VERBOSE, "Ignoring NAL type %d in extradata\n", nal->type); + break; + } + } + +done: + ff_h2645_packet_uninit(&pkt); + if (err_recognition & AV_EF_EXPLODE) + return ret; + + return 0; +} + +int ff_vvc_decode_extradata(const uint8_t *data, int size, VVCParamSets *ps, + int *is_nalff, int *nal_length_size, + int err_recognition, int apply_defdispwin, void *logctx) +{ + int ret = 0; + GetByteContext gb; + + bytestream2_init(&gb, data, size); + + if (size > 3 && (data[0] || data[1] || data[2] > 1)) { + /* extradata is encoded as vvcC format. */ + int i, j, b, num_arrays, nal_len_size, num_sublayers, has_ptl; + + b = bytestream2_get_byte(&gb); + num_sublayers = (b >> 3) & 0x7; + nal_len_size = ((b >> 1) & 0x3) + 1; + has_ptl = b & 0x1; + + if (has_ptl) { + int max_picture_width = 0; + int max_picture_height = 0; + int avg_frame_rate = 0; + + int num_bytes_constraint_info; + int general_profile_idc; + int general_tier_flag; + int general_level_idc; + int ptl_frame_only_constraint_flag; + int ptl_multi_layer_enabled_flag; + int ptl_num_sub_profiles; + int temp2, temp3, temp4, temp5; + int ols_idx; + int constant_frame_rate; + int chroma_format_idc; + int bit_depth_minus8; + + temp2 = bytestream2_get_be16(&gb); + ols_idx = (temp2 >> 7) & 0x1ff; + num_sublayers = (temp2 >> 4) & 0x7; + constant_frame_rate = (temp2 >> 2) & 0x3; + chroma_format_idc = temp2 & 0x3; + bit_depth_minus8 = (bytestream2_get_byte(&gb) >> 5) & 0x7; + av_log(logctx, AV_LOG_TRACE, + "bit_depth_minus8 %d chroma_format_idc %d\n", + bit_depth_minus8, chroma_format_idc); + av_log(logctx, AV_LOG_TRACE, + "constant_frame_rate %d, ols_idx %d\n", + constant_frame_rate, ols_idx); + // VvcPTLRecord(num_sublayers) native_ptl + temp3 = bytestream2_get_byte(&gb); + num_bytes_constraint_info = (temp3) & 0x3f; + temp4 = bytestream2_get_byte(&gb); + general_profile_idc = (temp4 >> 1) & 0x7f; + general_tier_flag = (temp4) & 1; + general_level_idc = bytestream2_get_byte(&gb); + av_log(logctx, AV_LOG_TRACE, + "general_profile_idc %d, general_tier_flag %d, " + "general_level_idc %d, num_sublayers %d num_bytes_constraint_info %d\n", + general_profile_idc, general_tier_flag, general_level_idc, num_sublayers, + num_bytes_constraint_info); + + temp5 = bytestream2_get_byte(&gb); + ptl_frame_only_constraint_flag = (temp5 >> 7) & 0x1; + ptl_multi_layer_enabled_flag = (temp5 >> 6) & 0x1; + for (i = 0; i < num_bytes_constraint_info-1; i++) { + // unsigned int(8*num_bytes_constraint_info - 2) general_constraint_info; + bytestream2_get_byte(&gb); + } + + av_log(logctx, AV_LOG_TRACE, + "ptl_multi_layer_enabled_flag %d, ptl_frame_only_constraint_flag %d\n", + ptl_multi_layer_enabled_flag, ptl_frame_only_constraint_flag); + + if( num_sublayers > 1 ) { + int temp6 = bytestream2_get_byte(&gb); + uint8_t ptl_sublayer_level_present_flag[8] = {0}; + //uint8_t sublayer_level_idc[8] = {0}; + for (i=num_sublayers - 2; i >= 0; i--) { + ptl_sublayer_level_present_flag[i] = (temp6 >> (7 - (num_sublayers - 2 - i))) & 0x01; + } + for (i=num_sublayers-2; i >= 0; i--) { + if (ptl_sublayer_level_present_flag[i]) { + bytestream2_get_byte(&gb); // sublayer_level_idc[8] + } + } + } + + ptl_num_sub_profiles = bytestream2_get_byte(&gb); + for (j=0; j < ptl_num_sub_profiles; j++) { + // unsigned int(32) general_sub_profile_idc[j]; + bytestream2_get_be16(&gb); + bytestream2_get_be16(&gb); + } + + max_picture_width = bytestream2_get_be16(&gb); + max_picture_height = bytestream2_get_be16(&gb); + avg_frame_rate = bytestream2_get_be16(&gb); + av_log(logctx, AV_LOG_TRACE, + "max_picture_width %d, max_picture_height %d, avg_frame_rate %d\n", + max_picture_width, max_picture_height, avg_frame_rate); + } + + *is_nalff = 1; + + /* nal units in the vvcC always have length coded with 2 bytes, + * so set nal_length_size = 2 while parsing them */ + *nal_length_size = 2; + + num_arrays = bytestream2_get_byte(&gb); + + for (i = 0; i < num_arrays; i++) { + int cnt; + int type = bytestream2_get_byte(&gb) & 0x1f; + + if (type != VVC_OPI_NUT || type != VVC_DCI_NUT) + cnt = bytestream2_get_be16(&gb); + else + cnt = 1; + + av_log(logctx, AV_LOG_DEBUG, "nalu_type %d cnt %d\n", type, cnt); + + if (!(type == VVC_VPS_NUT || type == VVC_SPS_NUT || type == VVC_PPS_NUT || + type == VVC_PREFIX_SEI_NUT || type == VVC_SUFFIX_SEI_NUT)) { + av_log(logctx, AV_LOG_ERROR, "Invalid NAL unit type in extradata: %d\n", + type); + ret = AVERROR_INVALIDDATA; + return ret; + } + + for (j = 0; j < cnt; j++) { + // +2 for the nal size field + int nalsize = bytestream2_peek_be16(&gb) + 2; + if (bytestream2_get_bytes_left(&gb) < nalsize) { + av_log(logctx, AV_LOG_ERROR, + "Invalid NAL unit size in extradata.\n"); + return AVERROR_INVALIDDATA; + } + + ret = vvc_decode_nal_units(gb.buffer, nalsize, ps, *is_nalff, + *nal_length_size, err_recognition, apply_defdispwin, + logctx); + if (ret < 0) { + av_log(logctx, AV_LOG_ERROR, + "Decoding nal unit %d %d from vvcC failed\n", type, i); + return ret; + } + bytestream2_skip(&gb, nalsize); + } + } + + /* store nal length size, that will be used to parse all other nals */ + *nal_length_size = nal_len_size; + } else { + *is_nalff = 0; + ret = vvc_decode_nal_units(data, size, ps, *is_nalff, *nal_length_size, + err_recognition, apply_defdispwin, logctx); + if (ret < 0) + return ret; + } + + return ret; +} diff --git a/libavcodec/vvc_parse_extradata.h b/libavcodec/vvc_parse_extradata.h new file mode 100644 index 0000000000..4adb7616b7 --- /dev/null +++ b/libavcodec/vvc_parse_extradata.h @@ -0,0 +1,36 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * H.266 parser code + */ + +#ifndef AVCODEC_VVC_PARSE_EXTRADATA_H +#define AVCODEC_VVC_PARSE_EXTRADATA_H + +#include + +#include "vvc_paramset.h" + + +int ff_vvc_decode_extradata(const uint8_t *data, int size, VVCParamSets *ps, + int *is_nalff, int *nal_length_size, + int err_recognition, int apply_defdispwin, void *logctx); + +#endif /* AVCODEC_VVC_PARSE_EXTRADATA_H */ From patchwork Mon Oct 24 14:06:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Siedel X-Patchwork-Id: 38965 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2249666pzb; Mon, 24 Oct 2022 07:09:17 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4n107Y0rbS12fif6vpbWJJ7+kegBQa57fK4CzHmo9r+OX6GwgGs8UUKemYWk0dBXsd263y X-Received: by 2002:aa7:c504:0:b0:461:122b:882b with SMTP id o4-20020aa7c504000000b00461122b882bmr19844126edq.14.1666620556733; Mon, 24 Oct 2022 07:09:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666620556; cv=none; d=google.com; s=arc-20160816; b=d2ZmeYh1qLH3O3d1IgI3wn19oGHX3bB18WxlqUy0kHR/d8ta//HR4qt1zIAV38BJYv lKQD0yVC0jo5TmIzx9JBeCAaJA4eKpJDWwaULnWUCJtrC/H2dqVaMhqcfZStELHgCeNv o8v8Xbpm/NF4blPZlQQIkNvBw9/iZCIZnrVe6b0OuLRBpBHkl5U4vXxVn6fpp50iG84+ ho1OebN0iupDnqogx6yEmnlRaUbi4ThQHE0+s/Z8zMOwpfbVQnDI46XJEKtCmT8as5wq Ky5zPHtV1ECgccJ0TMsRgnz8WgwsCnU3Fed1Vvuwd3SxwfN1zZ/JRu/B2EfsP4PJeBbt o5Zw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=hSJe+Bd/F7VQEMaBZqt6RSX2xvdlFJ/TsEYx8BzcFVA=; b=boo/8xQ9Z4mfR3ZoYVBUbKuciqv68NPz4Jx4dPJoo9LnrFwzLd2ZswhNVLOkdsODEE yDFpMo8DZ0p1CSPs9rfEEsRNuBnWS63qJgBP4DZNen4Fag8XFA5jO6XkNVR2H7mp3vdm 7QOINVJMqGluNd6KEuXqszw6hs5HD6VVzbQ6QP4fsQQP1dpVC6wqwmAiPugJg8QSy0yx 0thasMuF9Rh6wE4a11Cd+9lwfR8MHSK2vlgni2dRQ/BYNzYUX7U/XFKa+tOjLTrMocnn wof4MM/mvmjdls/bCW8XDPEjBNo6Z58kOir1rd4bFtYFtQ7ovebgVsMjgR7tthDoGUvo WAPw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=iZ9OawnL; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id go12-20020a1709070d8c00b0078e319dcca7si27072489ejc.744.2022.10.24.07.09.15; Mon, 24 Oct 2022 07:09:16 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=iZ9OawnL; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B60F368BC8F; Mon, 24 Oct 2022 17:07:53 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f54.google.com (mail-ed1-f54.google.com [209.85.208.54]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 07E9968BC1A for ; Mon, 24 Oct 2022 17:07:44 +0300 (EEST) Received: by mail-ed1-f54.google.com with SMTP id a67so31683213edf.12 for ; Mon, 24 Oct 2022 07:07:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spin-digital-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yBSqmwOqHbiIXNQvMGGD88+8g7oAtY4yq7XdB8ARBPo=; b=iZ9OawnLZdmr3Af5gq6vvF6HlVk2PQEzc4GXS7IJ493qJuUm7j98yCUwVF9euw1jci 0zIrG4WlGf1yhvd6CghkFc9R+MaAQP9wcPJE55SP9QlLx6QW6V6lV2/v9rCvL9v62W54 s1QnlCe93Vn+7k39g+5FTiFjfoIvAkjDfJLDf5TQjQ4DAnHmcFTHblMPfvwWxkGntjcC T5tBO89nvZ6EbU0BDZvO1gL0c10i+ky14zVQELZ0SzfNXpsVT2+coYiO3YGfG6pk92T7 wWe/n4XzCJXdYvfmnNDOzjZeHgdrrkroLnjOw3EgMSw8GCCjwDnOtW+pQpFseDwgI8WA UX/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yBSqmwOqHbiIXNQvMGGD88+8g7oAtY4yq7XdB8ARBPo=; b=OZpnLacwJSOd9ySWNY7emrz/UgCuImk0wV6M1+rNIqjWq5NY8oT0F7G8Q75Z2xCAr9 /HLXr2dQQ5PwAOwCz94yw1gyv+/24bCcDKggz6NoMKHUT0dAqZ5ZTacsVn0faQsfpoWT 1cliM7Ou2i4sJu8SK1KVtW0lxCSoIB1LxDt6mjgicEju9rTzdV1BbeDzeWvjeRz2hEdt XHM8uIAcNV+svwiW1VkfY8oq9FXquaWIvVAvqumpcHCtk+KVgFACBQ2ShJ2UcLKAbP0B TR16y3Pv6WY7FnTc2WIkhGWgQ7xCZ9Kb1/d2G173yU8e87KH1hVYCtI1wFuXpMYuZlof 3VPg== X-Gm-Message-State: ACrzQf3llsdNTvt8p7fGFO1VYERfKwev+4M41JVBURvvgeE0GCpLxI3H 9hTU6jjf36jFvdPwIhYPc+AD59YirlcbPA== X-Received: by 2002:aa7:c648:0:b0:461:d9b4:687e with SMTP id z8-20020aa7c648000000b00461d9b4687emr3235082edr.91.1666620462989; Mon, 24 Oct 2022 07:07:42 -0700 (PDT) Received: from thomas-win.localdomain ([213.138.44.237]) by smtp.gmail.com with ESMTPSA id s19-20020a170906455300b0079800b81709sm6056436ejq.219.2022.10.24.07.07.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Oct 2022 07:07:42 -0700 (PDT) From: Thomas Siedel To: ffmpeg-devel@ffmpeg.org Date: Mon, 24 Oct 2022 16:06:43 +0200 Message-Id: <20221024140645.4945-9-thomas.ff@spin-digital.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221024140645.4945-1-thomas.ff@spin-digital.com> References: <20221024140645.4945-1-thomas.ff@spin-digital.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 08/10] avcodec: add external encoder libvvenc for H266/VVC X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Thomas Siedel Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: v+4+VFYsXqxP Add external encoder VVenC for H266/VVC encoding. Register new encoder libvvencc Add libvvenc to wrap the vvenc interface into ffmpeg libvvenc implements encoder option: preset,qp,period,subjopt,vvenc-params,levelidc,tier Enable encoder by adding --enable-libvvenc in configure step Signed-off-by: Thomas Siedel --- configure | 5 + libavcodec/Makefile | 1 + libavcodec/allcodecs.c | 1 + libavcodec/libvvenc.c | 442 +++++++++++++++++++++++++++++++++++++++++ 4 files changed, 449 insertions(+) create mode 100644 libavcodec/libvvenc.c diff --git a/configure b/configure index 978f15f772..f8f7965371 100755 --- a/configure +++ b/configure @@ -289,6 +289,7 @@ External library support: native implementation exists [no] --enable-libvpx enable VP8 and VP9 de/encoding via libvpx [no] --enable-libvvdec enable VVC decoding via vvdec [no] + --enable-libvvenc enable VVC encoding via vvenc [no] --enable-libwebp enable WebP encoding via libwebp [no] --enable-libx264 enable H.264 encoding via x264 [no] --enable-libx265 enable HEVC encoding via x265 [no] @@ -1877,6 +1878,7 @@ EXTERNAL_LIBRARY_LIST=" libvorbis libvpx libvvdec + libvvenc libwebp libxml2 libzimg @@ -3409,6 +3411,8 @@ libvpx_vp9_decoder_deps="libvpx" libvpx_vp9_encoder_deps="libvpx" libvvdec_decoder_deps="libvvdec" libvvdec_decoder_select="vvc_mp4toannexb_bsf" +libvvenc_encoder_deps="libvvenc" +libvvenc_encoder_select="atsc_a53" libwebp_encoder_deps="libwebp" libwebp_anim_encoder_deps="libwebp" libx262_encoder_deps="libx262" @@ -6740,6 +6744,7 @@ enabled libvpx && { fi } enabled libvvdec && require_pkg_config libvvdec "libvvdec >= 1.6.0" "vvdec/vvdec.h" vvdec_get_version +enabled libvvenc && require_pkg_config libvvenc "libvvenc >= 1.6.1" "vvenc/vvenc.h" vvenc_get_version enabled libwebp && { enabled libwebp_encoder && require_pkg_config libwebp "libwebp >= 0.2.0" webp/encode.h WebPGetEncoderVersion diff --git a/libavcodec/Makefile b/libavcodec/Makefile index f4cdbc9be1..badd266e66 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1105,6 +1105,7 @@ OBJS-$(CONFIG_LIBVPX_VP8_ENCODER) += libvpxenc.o OBJS-$(CONFIG_LIBVPX_VP9_DECODER) += libvpxdec.o libvpx.o OBJS-$(CONFIG_LIBVPX_VP9_ENCODER) += libvpxenc.o libvpx.o OBJS-$(CONFIG_LIBVVDEC_DECODER) += libvvdec.o vvc_parse_extradata.o vvc_paramset.o +OBJS-$(CONFIG_LIBVVENC_ENCODER) += libvvenc.o OBJS-$(CONFIG_LIBWEBP_ENCODER) += libwebpenc_common.o libwebpenc.o OBJS-$(CONFIG_LIBWEBP_ANIM_ENCODER) += libwebpenc_common.o libwebpenc_animencoder.o OBJS-$(CONFIG_LIBX262_ENCODER) += libx264.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index 9813b291f4..73c36f3134 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -794,6 +794,7 @@ extern const FFCodec ff_libvpx_vp8_decoder; extern FFCodec ff_libvpx_vp9_encoder; extern FFCodec ff_libvpx_vp9_decoder; extern const FFCodec ff_libvvdec_decoder; +extern const FFCodec ff_libvvenc_encoder; /* preferred over libwebp */ extern const FFCodec ff_libwebp_anim_encoder; extern const FFCodec ff_libwebp_encoder; diff --git a/libavcodec/libvvenc.c b/libavcodec/libvvenc.c new file mode 100644 index 0000000000..508fe6ceff --- /dev/null +++ b/libavcodec/libvvenc.c @@ -0,0 +1,442 @@ +/* + * H.266 encoding using the VVenC library + * + * Copyright (C) 2022, Thomas Siedel + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "config_components.h" + +#include +#include + +#include "avcodec.h" +#include "codec_internal.h" +#include "encode.h" +#include "internal.h" +#include "packet_internal.h" +#include "profiles.h" + +#include "libavutil/avutil.h" +#include "libavutil/pixdesc.h" +#include "libavutil/opt.h" +#include "libavutil/common.h" +#include "libavutil/imgutils.h" +#include "libavutil/frame.h" +#include "libavutil/log.h" + +typedef struct VVenCOptions { + int preset; // preset 0: faster 4: slower + int qp; // quantization parameter 0-63 + int subjectiveOptimization; // perceptually motivated QP adaptation, XPSNR based + int intraRefreshSec; // intra period/refresh in seconds + int levelIdc; // vvc level_idc + int tier; // vvc tier + AVDictionary *vvenc_opts; +} VVenCOptions; + +typedef struct VVenCContext { + AVClass *av_class; + VVenCOptions options; // encoder options + vvencEncoder *vvencEnc; + vvencAccessUnit *pAU; + bool encodeDone; +}VVenCContext; + + +static av_cold void ff_vvenc_log_callback(void *avctx, int level, const char *fmt, va_list args ) +{ + vfprintf( level == 1 ? stderr : stdout, fmt, args ); +} + +static void ff_vvenc_internalLog( void* ctx, int level, const char* fmt, ... ) +{ + va_list args; + va_start( args, fmt ); + ff_vvenc_log_callback( ctx, level, fmt, args ); + va_end( args ); +} + +static av_cold int ff_vvenc_encode_init(AVCodecContext *avctx) +{ + int ret; + int framerate, qp, parse_ret; + VVenCContext *s; + vvenc_config params; + vvencPresetMode preset; + AVDictionaryEntry *en; + char statsfile[1024]="vvenc-rcstats.json"; + + s = (VVenCContext*)avctx->priv_data; + qp = (vvencPresetMode)s->options.qp; + preset = (vvencPresetMode)s->options.preset; + + if( avctx->flags & AV_CODEC_FLAG_INTERLACED_DCT ) { + av_log(avctx, AV_LOG_ERROR, "ff_vvenc_encode_init::init() interlaced encoding not supported yet\n"); + return AVERROR_INVALIDDATA; + } + + vvenc_config_default( ¶ms ); + + // set desired encoding options + framerate = avctx->time_base.den / avctx->time_base.num; + vvenc_init_default ( ¶ms, avctx->width, avctx->height, framerate, avctx->bit_rate, qp, preset ); + params.m_FrameRate = avctx->time_base.den; + params.m_FrameScale = avctx->time_base.num; + + params.m_verbosity = VVENC_VERBOSE; + if ( av_log_get_level() >= AV_LOG_DEBUG ) + params.m_verbosity = VVENC_DETAILS; + else if( av_log_get_level() >= AV_LOG_VERBOSE ) + params.m_verbosity = VVENC_NOTICE; // output per picture info + else if( av_log_get_level() >= AV_LOG_INFO ) + params.m_verbosity = VVENC_WARNING; // ffmpeg default ffmpeg loglevel + else + params.m_verbosity = VVENC_SILENT; + + if( avctx->ticks_per_frame == 1 ) { + params.m_TicksPerSecond = -1; // auto mode for ticks per frame = 1 + } + else{ + params.m_TicksPerSecond = ceil((avctx->time_base.den/(double)avctx->time_base.num)*(double)avctx->ticks_per_frame); + } + + if( avctx->thread_count > 0 ) + params.m_numThreads = avctx->thread_count; + + // GOP settings (IDR/CRA) + if( avctx->flags & AV_CODEC_FLAG_CLOSED_GOP ) + params.m_DecodingRefreshType = VVENC_DRT_IDR; + + if( avctx->gop_size == 1 ) { + params.m_GOPSize = 1; + params.m_IntraPeriod = 1; + } + else { + params.m_IntraPeriodSec = s->options.intraRefreshSec; + } + + params.m_usePerceptQPA = s->options.subjectiveOptimization; + params.m_level = (vvencLevel)s->options.levelIdc; + params.m_levelTier = (vvencTier)s->options.tier; + + params.m_AccessUnitDelimiter = true; + + params.m_internChromaFormat = VVENC_CHROMA_420; + switch( avctx->pix_fmt ) { + case AV_PIX_FMT_YUV420P : + params.m_inputBitDepth[0] = 8; + break; + case AV_PIX_FMT_YUV420P10LE: + params.m_inputBitDepth[0] = 10; + break; + default: { + av_log(avctx, AV_LOG_ERROR, "unsupported pixel format %s, choose yuv420p or yuv420p10le\n", av_get_pix_fmt_name(avctx->pix_fmt)); + return AVERROR(EINVAL); + break; + } + }; + + if( avctx->color_primaries != AVCOL_PRI_UNSPECIFIED ) + params.m_colourPrimaries = (int)avctx->color_primaries; + if( avctx->colorspace != AVCOL_SPC_UNSPECIFIED ) + params.m_matrixCoefficients = (int)avctx->colorspace; + if( avctx->color_trc != AVCOL_TRC_UNSPECIFIED ) { + params.m_transferCharacteristics = (int)avctx->color_trc; + + if ( params.m_transferCharacteristics == 16 ) + params.m_HdrMode = (params.m_colourPrimaries == 9) ? VVENC_HDR_PQ_BT2020 + : VVENC_HDR_PQ; + else if ( params.m_transferCharacteristics == 14 && params.m_colourPrimaries == 9 ) + params.m_HdrMode = VVENC_HDR_HLG_BT2020; + else if ( params.m_transferCharacteristics == 1 && params.m_colourPrimaries == 1 && + params.m_matrixCoefficients == 1 ) + params.m_HdrMode = VVENC_HDR_HLG; + } + + params.m_RCNumPasses = 1; + en = NULL; + while ((en = av_dict_get(s->options.vvenc_opts, "", en, AV_DICT_IGNORE_SUFFIX))) { + av_log(avctx, AV_LOG_DEBUG, "vvenc_set_param: '%s:%s'\n", en->key, en->value); + parse_ret = vvenc_set_param(¶ms, en->key, en->value); + switch (parse_ret) + { + case VVENC_PARAM_BAD_NAME: + av_log(avctx, AV_LOG_WARNING,"Unknown vvenc option: %s.\n", en->key); + break; + case VVENC_PARAM_BAD_VALUE: + av_log(avctx, AV_LOG_WARNING, + "Invalid vvenc value for %s: %s.\n", en->key, en->value); + break; + default: + break; + } + + if (memcmp( en->key, "rcstatsfile", 11 ) == 0 || + memcmp( en->key, "RCStatsFile", 11 ) == 0) { + strncpy(statsfile, en->value, sizeof(statsfile)-1); + statsfile[sizeof(statsfile)-1] = '\0'; + } + } + + if( params.m_RCPass != -1 && params.m_RCNumPasses == 1 ) + params.m_RCNumPasses = 2; // enable 2pass mode + + s->vvencEnc = vvenc_encoder_create(); + if( NULL == s->vvencEnc ) { + av_log(avctx, AV_LOG_ERROR, "cannot create vvc encoder (vvenc)\n"); + return AVERROR(ENOMEM); + } + + vvenc_set_msg_callback( ¶ms, s->vvencEnc, ff_vvenc_log_callback ); + ret = vvenc_encoder_open( s->vvencEnc, ¶ms ); + if( 0 != ret ) { + av_log(avctx, AV_LOG_ERROR, "cannot open vvc encoder (vvenc): %s\n", vvenc_get_last_error(s->vvencEnc)); + vvenc_encoder_close( s->vvencEnc ); + return AVERROR(EINVAL); + } + + vvenc_get_config( s->vvencEnc, ¶ms ); // get the adapted config + + if( params.m_verbosity >= VVENC_DETAILS && av_log_get_level() < AV_LOG_DEBUG ) { + ff_vvenc_internalLog( avctx, params.m_verbosity, "vvenc version: %s\n", vvenc_get_version() ); + ff_vvenc_internalLog( avctx, params.m_verbosity, "vvenc info:\n%s\n", vvenc_get_config_as_string( ¶ms, VVENC_DETAILS) ); + } + else { + av_log(avctx, AV_LOG_DEBUG, "vvenc version: %s\n", vvenc_get_version()); + av_log(avctx, AV_LOG_DEBUG, "vvenc info:\n%s\n", vvenc_get_config_as_string( ¶ms, VVENC_DETAILS)); + } + + if( params.m_RCNumPasses == 2 ) { + ret = vvenc_init_pass( s->vvencEnc, params.m_RCPass-1, &statsfile[0] ); + if( 0 != ret ) { + av_log(avctx, AV_LOG_ERROR, "cannot init pass %d for vvc encoder (vvenc): %s\n", params.m_RCPass, vvenc_get_last_error(s->vvencEnc)); + vvenc_encoder_close( s->vvencEnc ); + return AVERROR(EINVAL); + } + } + + s->pAU = vvenc_accessUnit_alloc(); + vvenc_accessUnit_alloc_payload( s->pAU, avctx->width * avctx->height ); + + if (avctx->flags & AV_CODEC_FLAG_GLOBAL_HEADER) { + ret = vvenc_get_headers( s->vvencEnc, s->pAU ); + if( 0 != ret ) { + av_log(avctx, AV_LOG_ERROR, "cannot get headers (SPS,PPS) from vvc encoder(vvenc): %s\n", vvenc_get_last_error(s->vvencEnc)); + vvenc_encoder_close( s->vvencEnc ); + return AVERROR(EINVAL); + } + + if (s->pAU->payloadUsedSize <= 0) { + vvenc_encoder_close( s->vvencEnc ); + return AVERROR_INVALIDDATA; + } + + avctx->extradata_size = s->pAU->payloadUsedSize; + avctx->extradata = av_malloc(avctx->extradata_size + AV_INPUT_BUFFER_PADDING_SIZE); + if (!avctx->extradata) { + av_log(avctx, AV_LOG_ERROR, + "Cannot allocate VVC header of size %d.\n", avctx->extradata_size); + vvenc_encoder_close( s->vvencEnc ); + return AVERROR(ENOMEM); + } + + memcpy(avctx->extradata, s->pAU->payload, avctx->extradata_size); + memset(avctx->extradata + avctx->extradata_size, 0, AV_INPUT_BUFFER_PADDING_SIZE); + } + s->encodeDone = false; + return 0; +} + +static av_cold int ff_vvenc_encode_close(AVCodecContext *avctx) +{ + VVenCContext *s = (VVenCContext*)avctx->priv_data; + if( s->vvencEnc ) { + if( av_log_get_level() >= AV_LOG_VERBOSE ) + vvenc_print_summary(s->vvencEnc); + + if( 0 != vvenc_encoder_close(s->vvencEnc) ) { + av_log(avctx, AV_LOG_ERROR, "cannot close vvenc\n" ); + return -1; + } + } + + vvenc_accessUnit_free ( s->pAU, true ); + + return 0; +} + +static av_cold int ff_vvenc_encode_frame( AVCodecContext *avctx, AVPacket *pkt,const AVFrame *frame, int *got_packet ) +{ + VVenCContext *s = (VVenCContext*)avctx->priv_data; + vvencYUVBuffer* pyuvbuf; + vvencYUVBuffer yuvbuf; + int pict_type; + int ret; + + pyuvbuf = NULL; + if (frame) { + if( avctx->pix_fmt == AV_PIX_FMT_YUV420P10LE ) { + vvenc_YUVBuffer_default( &yuvbuf ); + yuvbuf.planes[0].ptr = (int16_t*)frame->data[0]; + yuvbuf.planes[1].ptr = (int16_t*)frame->data[1]; + yuvbuf.planes[2].ptr = (int16_t*)frame->data[2]; + + yuvbuf.planes[0].width = frame->width; + yuvbuf.planes[0].height = frame->height; + yuvbuf.planes[0].stride = frame->linesize[0] >> 1; // stride is used in samples (16bit) in vvenc, ffmpeg uses stride in bytes + + yuvbuf.planes[1].width = frame->width >>1; + yuvbuf.planes[1].height = frame->height >>1; + yuvbuf.planes[1].stride = frame->linesize[1] >> 1; + + yuvbuf.planes[2].width = frame->width >>1; + yuvbuf.planes[2].height = frame->height >>1; + yuvbuf.planes[2].stride = frame->linesize[2] >> 1; + + yuvbuf.cts = frame->pts; + yuvbuf.ctsValid = true; + pyuvbuf = &yuvbuf; + } + else { + av_log(avctx, AV_LOG_ERROR, "unsupported input colorspace! input must be yuv420p10le"); + return AVERROR(EINVAL); + } + } + + if( ! s->encodeDone ) { + ret = vvenc_encode( s->vvencEnc, pyuvbuf, s->pAU, &s->encodeDone ); + if( ret != 0 ) { + av_log(avctx, AV_LOG_ERROR, "error in vvenc::encode - ret:%d\n", ret); + return AVERROR(EINVAL); + } + } + else { + *got_packet = 0; + return 0; + } + + if( s->pAU->payloadUsedSize > 0 ) { + ret = ff_get_encode_buffer(avctx, pkt, s->pAU->payloadUsedSize, 0); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Error getting output packet.\n"); + return ret; + } + + memcpy( pkt->data, s->pAU->payload, s->pAU->payloadUsedSize ); + + if( s->pAU->ctsValid ) + pkt->pts = s->pAU->cts; + if( s->pAU->dtsValid ) + pkt->dts = s->pAU->dts; + pkt->flags |= AV_PKT_FLAG_KEY*s->pAU->rap; + + switch (s->pAU->sliceType) { + case VVENC_I_SLICE: + pict_type = AV_PICTURE_TYPE_I; + break; + case VVENC_P_SLICE: + pict_type = AV_PICTURE_TYPE_P; + break; + case VVENC_B_SLICE: + pict_type = AV_PICTURE_TYPE_B; + break; + default: + av_log(avctx, AV_LOG_ERROR, "Unknown picture type encountered.\n"); + return AVERROR_EXTERNAL; + } + + ff_side_data_set_encoder_stats(pkt, 0, NULL, 0, pict_type); + + *got_packet = 1; + + return 0; + } + else + { + *got_packet = 0; + return 0; + } + + return 0; +} + +static const enum AVPixelFormat pix_fmts_vvc[] = { + //AV_PIX_FMT_YUV420P, // TODO + AV_PIX_FMT_YUV420P10LE, + AV_PIX_FMT_NONE +}; + +#define OFFSET(x) offsetof(VVenCContext, x) +#define VE AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM +static const AVOption libvvenc_options[] = { + {"preset", "set encoding preset(0: faster - 4: slower", OFFSET( options.preset), AV_OPT_TYPE_INT, {.i64 = 2} , 0 , 4 , VE, "preset"}, + { "faster", "0", 0, AV_OPT_TYPE_CONST, {.i64 = VVENC_FASTER},INT_MIN, INT_MAX, VE, "preset" }, + { "fast", "1", 0, AV_OPT_TYPE_CONST, {.i64 = VVENC_FAST}, INT_MIN, INT_MAX, VE, "preset" }, + { "medium", "2", 0, AV_OPT_TYPE_CONST, {.i64 = VVENC_MEDIUM},INT_MIN, INT_MAX, VE, "preset" }, + { "slow", "3", 0, AV_OPT_TYPE_CONST, {.i64 = VVENC_SLOW}, INT_MIN, INT_MAX, VE, "preset" }, + { "slower", "4", 0, AV_OPT_TYPE_CONST, {.i64 = VVENC_SLOWER},INT_MIN, INT_MAX, VE, "preset" }, + {"qp" , "set quantization" , OFFSET(options.qp) , AV_OPT_TYPE_INT, {.i64 = 32}, 0 , 63 ,VE, "qp_mode" }, + {"period" , "set (intra) refresh period in seconds" , OFFSET(options.intraRefreshSec) , AV_OPT_TYPE_INT, {.i64 = 1}, 1 , INT_MAX ,VE,"irefreshsec" }, + {"subjopt", "set subjective (perceptually motivated) optimization" , OFFSET(options.subjectiveOptimization) , AV_OPT_TYPE_BOOL, {.i64 = 1}, 0 , 1, VE}, + {"vvenc-params", "set the vvenc configuration using a :-separated list of key=value parameters", OFFSET(options.vvenc_opts), AV_OPT_TYPE_DICT, { 0 }, 0, 0, VE }, + {"levelidc", "vvc level_idc", OFFSET( options.levelIdc), AV_OPT_TYPE_INT, {.i64 = 0}, 0, 105, VE, "levelidc"}, + {"0" , "auto" , 0, AV_OPT_TYPE_CONST, {.i64 = 0}, INT_MIN, INT_MAX, VE, "levelidc"}, + {"1" , "1" , 0, AV_OPT_TYPE_CONST, {.i64 = 16},INT_MIN, INT_MAX, VE, "levelidc"}, + {"2" , "2" , 0, AV_OPT_TYPE_CONST, {.i64 = 32},INT_MIN, INT_MAX, VE, "levelidc"}, + {"2.1" , "2.1" , 0, AV_OPT_TYPE_CONST, {.i64 = 35},INT_MIN, INT_MAX, VE, "levelidc"}, + {"3" , "3" , 0, AV_OPT_TYPE_CONST, {.i64 = 48},INT_MIN, INT_MAX, VE, "levelidc"}, + {"3.1" , "3.1" , 0, AV_OPT_TYPE_CONST, {.i64 = 51},INT_MIN, INT_MAX, VE, "levelidc"}, + {"4" , "4" , 0, AV_OPT_TYPE_CONST, {.i64 = 64},INT_MIN, INT_MAX, VE, "levelidc"}, + {"4.1" , "4.1" , 0, AV_OPT_TYPE_CONST, {.i64 = 67},INT_MIN, INT_MAX, VE, "levelidc"}, + {"5" , "5" , 0, AV_OPT_TYPE_CONST, {.i64 = 80},INT_MIN, INT_MAX, VE, "levelidc"}, + {"5.1" , "5.1" , 0, AV_OPT_TYPE_CONST, {.i64 = 83},INT_MIN, INT_MAX, VE, "levelidc"}, + {"5.2" , "5.2" , 0, AV_OPT_TYPE_CONST, {.i64 = 86},INT_MIN, INT_MAX, VE, "levelidc"}, + {"6" , "6" , 0, AV_OPT_TYPE_CONST, {.i64 = 96},INT_MIN, INT_MAX, VE, "levelidc"}, + {"6.1" , "6.1" , 0, AV_OPT_TYPE_CONST, {.i64 = 99},INT_MIN, INT_MAX, VE, "levelidc"}, + {"6.2" , "6.2" , 0, AV_OPT_TYPE_CONST, {.i64 = 102},INT_MIN, INT_MAX, VE, "levelidc"}, + {"6.3" , "6.3" , 0, AV_OPT_TYPE_CONST, {.i64 = 105},INT_MIN, INT_MAX, VE, "levelidc"}, + {"tier", "set vvc tier", OFFSET( options.tier), AV_OPT_TYPE_INT, {.i64 = 0}, 0 , 1 , VE, "tier"}, + {"main", "main" , 0, AV_OPT_TYPE_CONST, {.i64 = 0}, INT_MIN, INT_MAX, VE, "tier"}, + {"high", "high" , 0, AV_OPT_TYPE_CONST, {.i64 = 1}, INT_MIN, INT_MAX, VE, "tier"}, + {NULL} +}; + +static const AVClass class_libvvenc = { + .class_name = "libvvenc-vvc encoder", + .item_name = av_default_item_name, + .option = libvvenc_options, + .version = LIBAVUTIL_VERSION_INT, +}; + +FFCodec ff_libvvenc_encoder = { + .p.name = "libvvenc", + .p.long_name = "H.266 / VVC Encoder VVenC", + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_VVC, + .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_OTHER_THREADS, + .p.profiles = NULL_IF_CONFIG_SMALL(ff_vvc_profiles), + .p.priv_class = &class_libvvenc, + .p.wrapper_name = "libvvenc", + .priv_data_size = sizeof(VVenCContext), + .p.pix_fmts = pix_fmts_vvc, + .init = ff_vvenc_encode_init, + FF_CODEC_ENCODE_CB(ff_vvenc_encode_frame), + .close = ff_vvenc_encode_close, + .caps_internal = FF_CODEC_CAP_AUTO_THREADS, +}; From patchwork Mon Oct 24 14:06:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Siedel X-Patchwork-Id: 38962 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2249398pzb; Mon, 24 Oct 2022 07:08:48 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4eEfrvMWd7aJXB3xuffl5YGlL2y3Zz4b9AG61bNpKeknOUjBGXXqwpQn3YqV0umjlEjg0X X-Received: by 2002:a17:907:9485:b0:78e:119f:940 with SMTP id dm5-20020a170907948500b0078e119f0940mr26300513ejc.535.1666620526268; Mon, 24 Oct 2022 07:08:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666620526; cv=none; d=google.com; s=arc-20160816; b=c34EebJ6A4XkTlse57PWIz6kS6Eadi3gROIG1tZVxEGKeeuMHKFLRmVc73PtIefD3S /J8Xupe3krqx21DXsYQdROMFHrwa7iBxpNsCsGjtR/qqjdY3H8oxWS13pc9jM/Tk6aNx AU2tcuFoZrgDPHBqlOwBKMCEX+0whtY96JkxW4agUC1By29iRLU+3vM3OWS8lT/DYl0M /ILAutiqOqp/n59V7qAVQ+fQKbyLUgzNm975RKT4QNJT5h7/Pd00YshN8xOXcI7arLo1 2ZWjQY50IdQm5RkT2avvkotaqLsnTfmw6ABxAwfJYWquNuylJnjGbxUXnkOkySHRtdwE r+Tw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=KcIlFM7ozsvsq+0f0JJ2QkYwcISrtmkaNsboWzYRo+o=; b=VvYsSnUNGMfgZEXdi7H2+BK/rfP1J1jVTcZExOBZYcyMeFjQT8NswzxEhWOwALM4P+ 81cU/MJ+c7dnEVEx3VEaCJjgEcQI5gkkL7NHK8qIa+Rc3DFSq09pF/YZsxOQuV22vx3W 8EWSly9jbAEUZ4BpsiywgpFhzEEKeQdQ2EE1cE2+X+PZSHeZhhNRjsql08I8rdWJIJ38 gJbK0llGZXcOlaTCanjDcp5lGiRNmQfADk4fYjbhz0Dr5ciLF6VAbvTQazVYkbgZ6Rlb RKLMZvbDEeQ0VxpSyQM2UAn1ckTGgTBTE0R5Svxd9iRgawAPZ4K3SQD/e8hL2ice4Jz2 DYYg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=GfanVPOt; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id ds9-20020a170907724900b0078dee0fce23si492842ejc.123.2022.10.24.07.08.43; Mon, 24 Oct 2022 07:08:46 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=GfanVPOt; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 53E7368BC56; Mon, 24 Oct 2022 17:07:49 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f49.google.com (mail-ed1-f49.google.com [209.85.208.49]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4978A68BAA4 for ; Mon, 24 Oct 2022 17:07:45 +0300 (EEST) Received: by mail-ed1-f49.google.com with SMTP id b12so31768306edd.6 for ; Mon, 24 Oct 2022 07:07:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spin-digital-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=a2jXzkfQng4135oTacKgbS1JhR5EwtAeLLPBFitPFco=; b=GfanVPOtvlsHPzZLZohvKaT82fe5TgdkkiB9b9xxzKuan26ElcxcKJ02UA/dZ8Q9Kg AHh0UKHUesq+mH8xw6gjrBizC31Ao/XsAm5U3vfMlAgURHXEZJepY84IQ177SxlZl5aO 6IsTnk/LcxiFbQt/FZuBmq5FQ91bDlccO/m76f9NEcKmd5y6r7AMcN8oOGgwfQ3lT2V3 KtiAfBnkZpKz6LVC6wBCIUUxHtHLGxJ10Ki4pBBC7ISAM3OcHc0MVl4LN8Wv8BvyE87l xgP/fOS002q0EfmdNxRNC1gS5H5ZtTE8Q6ZPAauGagTJ2/66SlhjuXsoc4ETuL87SnAt uqcQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=a2jXzkfQng4135oTacKgbS1JhR5EwtAeLLPBFitPFco=; b=wiXjdH9/fgOODOPsnUdmz2fVMN4b8jOldh+HbDklgrPL/5QpYf8dRLRNbZ2ruAWVwm BF3yMaDtzd8Jo3MCyhg+4zeElLaJluuF8RixAo3g/VTMIoj5+btwi1AfZkWHB3ryq4aS BbX3UQkhtWhS2naNSPq69Mzox0o7TvFATHzav2X7qDrWtepYKpcgRu49L3QmOxCxyxkF 83bH6RPgqItN6icDnNlIOsECqgvJnP3apIWa/Uag9YMMgxXaGTceqUC3Jgu4U9EGQ1hj uBMUGRxu1TqRClN3QBqsnRfQDFrivxx7d9eryX1+GSpA6DPaxxoH8NW/M06Qgo6BUmi1 EJaw== X-Gm-Message-State: ACrzQf34lVutDZyyOcFc1GTfZKURy9zpH77jb5Pglb54ErlsTbCHNJKy qvLEFMfHHzM6wYr6MxIyM8oRrAXaY0BcmA== X-Received: by 2002:a17:906:4795:b0:794:8b93:2e33 with SMTP id cw21-20020a170906479500b007948b932e33mr22507661ejc.265.1666620463906; Mon, 24 Oct 2022 07:07:43 -0700 (PDT) Received: from thomas-win.localdomain ([213.138.44.237]) by smtp.gmail.com with ESMTPSA id s19-20020a170906455300b0079800b81709sm6056436ejq.219.2022.10.24.07.07.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Oct 2022 07:07:43 -0700 (PDT) From: Thomas Siedel To: ffmpeg-devel@ffmpeg.org Date: Mon, 24 Oct 2022 16:06:44 +0200 Message-Id: <20221024140645.4945-10-thomas.ff@spin-digital.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221024140645.4945-1-thomas.ff@spin-digital.com> References: <20221024140645.4945-1-thomas.ff@spin-digital.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 09/10] avformat: add ts stream types for H266/VVC X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Thomas Siedel Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 9Jv4aDudpz2g Add transport stream stream type 0x33 for vvc. Add STREAM_TYPE_VIDEO_VVC to MPEG-1/2 and MPEG-2 transport stream Add basic transport stream support for TS mux/demux Signed-off-by: Thomas Siedel --- configure | 2 +- libavformat/mpeg.c | 3 ++ libavformat/mpeg.h | 1 + libavformat/mpegts.c | 2 ++ libavformat/mpegts.h | 1 + libavformat/mpegtsenc.c | 65 +++++++++++++++++++++++++++++++++++++++++ 6 files changed, 73 insertions(+), 1 deletion(-) diff --git a/configure b/configure index f8f7965371..d8dca3656c 100755 --- a/configure +++ b/configure @@ -3485,7 +3485,7 @@ mp3_demuxer_select="mpegaudio_parser" mp3_muxer_select="mpegaudioheader" mp4_muxer_select="mov_muxer" mpegts_demuxer_select="iso_media" -mpegts_muxer_select="ac3_parser adts_muxer latm_muxer h264_mp4toannexb_bsf hevc_mp4toannexb_bsf" +mpegts_muxer_select="ac3_parser adts_muxer latm_muxer h264_mp4toannexb_bsf hevc_mp4toannexb_bsf vvc_mp4toannexb_bsf" mpegtsraw_demuxer_select="mpegts_demuxer" mxf_muxer_select="pcm_rechunk_bsf" mxf_d10_muxer_select="mxf_muxer" diff --git a/libavformat/mpeg.c b/libavformat/mpeg.c index 864b08d8f8..970926df6b 100644 --- a/libavformat/mpeg.c +++ b/libavformat/mpeg.c @@ -546,6 +546,9 @@ redo: } else if (es_type == STREAM_TYPE_VIDEO_HEVC) { codec_id = AV_CODEC_ID_HEVC; type = AVMEDIA_TYPE_VIDEO; + } else if (es_type == STREAM_TYPE_VIDEO_VVC) { + codec_id = AV_CODEC_ID_VVC; + type = AVMEDIA_TYPE_VIDEO; } else if (es_type == STREAM_TYPE_AUDIO_AC3) { codec_id = AV_CODEC_ID_AC3; type = AVMEDIA_TYPE_AUDIO; diff --git a/libavformat/mpeg.h b/libavformat/mpeg.h index b635295776..20592eb184 100644 --- a/libavformat/mpeg.h +++ b/libavformat/mpeg.h @@ -56,6 +56,7 @@ #define STREAM_TYPE_VIDEO_MPEG4 0x10 #define STREAM_TYPE_VIDEO_H264 0x1b #define STREAM_TYPE_VIDEO_HEVC 0x24 +#define STREAM_TYPE_VIDEO_VVC 0x33 #define STREAM_TYPE_VIDEO_CAVS 0x42 #define STREAM_TYPE_AUDIO_AC3 0x81 diff --git a/libavformat/mpegts.c b/libavformat/mpegts.c index d97702fcd7..61a800c85f 100644 --- a/libavformat/mpegts.c +++ b/libavformat/mpegts.c @@ -811,6 +811,7 @@ static const StreamType ISO_types[] = { { 0x20, AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_H264 }, { 0x21, AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_JPEG2000 }, { 0x24, AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_HEVC }, + { 0x33, AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_VVC }, { 0x42, AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_CAVS }, { 0xd1, AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_DIRAC }, { 0xd2, AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_AVS2 }, @@ -865,6 +866,7 @@ static const StreamType REGD_types[] = { { MKTAG('D', 'T', 'S', '3'), AVMEDIA_TYPE_AUDIO, AV_CODEC_ID_DTS }, { MKTAG('E', 'A', 'C', '3'), AVMEDIA_TYPE_AUDIO, AV_CODEC_ID_EAC3 }, { MKTAG('H', 'E', 'V', 'C'), AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_HEVC }, + { MKTAG('V', 'V', 'C', ' '), AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_VVC }, { MKTAG('K', 'L', 'V', 'A'), AVMEDIA_TYPE_DATA, AV_CODEC_ID_SMPTE_KLV }, { MKTAG('I', 'D', '3', ' '), AVMEDIA_TYPE_DATA, AV_CODEC_ID_TIMED_ID3 }, { MKTAG('V', 'C', '-', '1'), AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_VC1 }, diff --git a/libavformat/mpegts.h b/libavformat/mpegts.h index a48f14e768..14ae312c50 100644 --- a/libavformat/mpegts.h +++ b/libavformat/mpegts.h @@ -128,6 +128,7 @@ #define STREAM_TYPE_METADATA 0x15 #define STREAM_TYPE_VIDEO_H264 0x1b #define STREAM_TYPE_VIDEO_HEVC 0x24 +#define STREAM_TYPE_VIDEO_VVC 0x33 #define STREAM_TYPE_VIDEO_CAVS 0x42 #define STREAM_TYPE_VIDEO_AVS2 0xd2 #define STREAM_TYPE_VIDEO_AVS3 0xd4 diff --git a/libavformat/mpegtsenc.c b/libavformat/mpegtsenc.c index 48d39e6a7d..4c32cddff1 100644 --- a/libavformat/mpegtsenc.c +++ b/libavformat/mpegtsenc.c @@ -366,6 +366,9 @@ static int get_dvb_stream_type(AVFormatContext *s, AVStream *st) case AV_CODEC_ID_HEVC: stream_type = STREAM_TYPE_VIDEO_HEVC; break; + case AV_CODEC_ID_VVC: + stream_type = STREAM_TYPE_VIDEO_VVC; + break; case AV_CODEC_ID_CAVS: stream_type = STREAM_TYPE_VIDEO_CAVS; break; @@ -462,6 +465,11 @@ static int get_m2ts_stream_type(AVFormatContext *s, AVStream *st) case AV_CODEC_ID_HEVC: stream_type = STREAM_TYPE_VIDEO_HEVC; break; + case AV_CODEC_ID_VVC: + av_log(s, AV_LOG_ERROR, + "MPEGTS VVC %s.\n", avcodec_get_name(st->codecpar->codec_id)); + stream_type = STREAM_TYPE_VIDEO_VVC; + break; case AV_CODEC_ID_PCM_BLURAY: stream_type = 0x80; break; @@ -1783,6 +1791,21 @@ static int check_hevc_startcode(AVFormatContext *s, const AVStream *st, const AV return 0; } +static int check_vvc_startcode(AVFormatContext *s, const AVStream *st, const AVPacket *pkt) +{ + if (pkt->size < 5 || AV_RB32(pkt->data) != 0x0000001 && AV_RB24(pkt->data) != 0x000001) { + if (!st->nb_frames) { + av_log(s, AV_LOG_ERROR, "VVC bitstream malformed, no startcode found\n"); + return AVERROR_PATCHWELCOME; + } + av_log(s, AV_LOG_WARNING, "VVC bitstream error, startcode missing, size %d", pkt->size); + if (pkt->size) + av_log(s, AV_LOG_WARNING, " data %08"PRIX32, AV_RB32(pkt->data)); + av_log(s, AV_LOG_WARNING, "\n"); + } + return 0; +} + /* Based on GStreamer's gst-plugins-base/ext/ogg/gstoggstream.c * Released under the LGPL v2.1+, written by * Vincent Penquerc'h @@ -1975,6 +1998,42 @@ static int mpegts_write_packet_internal(AVFormatContext *s, AVPacket *pkt) buf = data; size = pkt->size + 7 + extradd; } + } else if (st->codecpar->codec_id == AV_CODEC_ID_VVC) { + const uint8_t *p = buf, *buf_end = p + size; + uint32_t state = -1; + uint32_t naltype = -1; + int extradd = (pkt->flags & AV_PKT_FLAG_KEY) ? st->codecpar->extradata_size : 0; + int ret = check_vvc_startcode(s, st, pkt); + if (ret < 0) + return ret; + + if (extradd && AV_RB24(st->codecpar->extradata) > 1) + extradd = 0; + + do { + p = avpriv_find_start_code(p, buf_end, &state); + // state contains byte after start code, p points 2 bytes after start code + naltype = ((AV_RB8(p)>>3) & 0x1F); + av_log(s, AV_LOG_TRACE, "nal %"PRId32"\n", naltype ); + if (naltype == 14 ) // VPS + extradd = 0; + } while (p < buf_end && naltype != 20 && naltype >= 12); + + if (naltype >= 12) + extradd = 0; + if (naltype != 20) { // AUD NAL + data = av_malloc(pkt->size + 7 + extradd); + if (!data) + return AVERROR(ENOMEM); + memcpy(data + 7, st->codecpar->extradata, extradd); + memcpy(data + 7 + extradd, pkt->data, pkt->size); + AV_WB32(data, 0x00000001); + data[4] = 20; + data[5] = 1; + data[6] = 0x50; // any slice type (0x4) + rbsp stop one bit + buf = data; + size = pkt->size + 7 + extradd; + } } else if (st->codecpar->codec_id == AV_CODEC_ID_OPUS) { if (pkt->size < 2) { av_log(s, AV_LOG_ERROR, "Opus packet too short\n"); @@ -2231,6 +2290,12 @@ static int mpegts_check_bitstream(AVFormatContext *s, AVStream *st, (st->codecpar->extradata_size > 0 && st->codecpar->extradata[0] == 1))) ret = ff_stream_add_bitstream_filter(st, "hevc_mp4toannexb", NULL); + } else if (st->codecpar->codec_id == AV_CODEC_ID_VVC) { + if (pkt->size >= 5 && AV_RB32(pkt->data) != 0x0000001 && + (AV_RB24(pkt->data) != 0x000001 || + (st->codecpar->extradata_size > 0 && + st->codecpar->extradata[0] == 1))) + ret = ff_stream_add_bitstream_filter(st, "vvc_mp4toannexb", NULL); } return ret; From patchwork Mon Oct 24 14:06:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Siedel X-Patchwork-Id: 38966 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2249888pzb; Mon, 24 Oct 2022 07:09:36 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5bpn8OZz4gNZghq7LxwwC9Klh7C2VIrEumzeoMN/EmgP7P+R1TonAcOh9qDx5aHJktQa+N X-Received: by 2002:a05:6402:550e:b0:456:f79f:2bed with SMTP id fi14-20020a056402550e00b00456f79f2bedmr31439527edb.106.1666620566289; Mon, 24 Oct 2022 07:09:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666620566; cv=none; d=google.com; s=arc-20160816; b=LZz+298s0H3akwOMMgwRXj4rtSd7QfEKE0TB6T4jD0kulkbcE0smCvTQlcZ1C9XlLc PM4rcJtUB2a6wMft+FK0kPPMEwGgXNrUQWTocAxVJ6/elDsXUTekf1d0nYGMS3H1gPR9 xWPAhYsn6X/wcZ/6ifL+Iwp1eCig/w0l/Uq9ik5CP2SPDjauiNN+qK2euf+yRCFqw5yO RdzHEU5Ave+qt6roUKwj0/P7dYTI/ibg7jTa76C/FDwAnSqik78j+WM4LOQJish3ciwe YhSs0Pnnt9S6rURaAX9DSO4nDWK54gw0ciQchxEsX8wTKUHWMrs9u0Dh87kF2zacdCeG RcZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=eg+wNxRiq9DOO3V9dOATfP/i0irwVqAMmlNSag/kH6A=; b=rQTWeDuzLzmyrmw+gTbBaKGAWzHbIJI/cZ8hhWa3ue6NjdOpOf3IGmxopZIioZYHId yBY66F5r0Fclkbm9k5u/l0aMQO/lkLR5c133n6CThgkhUXVPupujUuZfy6ei5PJEujtW 1N23FztBk0e2WoMfWIXrIsJamHy2xWwDOSEcd14SLRyIAXWLm4gfXmLqzAQnaOwD5k98 sypKKTn4Twsql+uDNiBwa8CaH9oambYO2EDZv5vysqBsyaqd+RVZyanJHVFr7aotU090 xLLlCwPZ5KoyV/ayYSvUkHlzYgimWAhr2zUxHZFGgfCctQ42Th5iZRFnXKAx6BbUz44u lsQg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=Ro4ST39P; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id ka3-20020a170907990300b0078c3603b4f6si22688984ejc.784.2022.10.24.07.09.25; Mon, 24 Oct 2022 07:09:26 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=Ro4ST39P; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id BF5D368BC8C; Mon, 24 Oct 2022 17:07:54 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f48.google.com (mail-ed1-f48.google.com [209.85.208.48]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CC82B68BC47 for ; Mon, 24 Oct 2022 17:07:45 +0300 (EEST) Received: by mail-ed1-f48.google.com with SMTP id m16so31572433edc.4 for ; Mon, 24 Oct 2022 07:07:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spin-digital-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bIxuAi/zK9n6L4ct9fNEX2sco1bprmCREnncAgcx8bw=; b=Ro4ST39PvdGi5213omVA3e9ZzLmEZGL7yo6wqBVvtWNRpbWb7N5O9KqEF/WiB2a/2U bBqijckRUgOJRUP8bw6LzGgVn33QlV+70GojM69Umpw309CfHc/CJsIBuzeJ2oOGMBhg 6ISK1dcNi6plStD1GmQkx8hQyDdeQINk80lHVD/Ivpie3GbZQ1042HqImr25Ib8uuaZb a//bZeMZTS7bd7qaU00ABN3Kl+Bi7aOLGxI5xgb/6d9r5Ns57RkS1N6teo1AtxT/ejY1 pQwACREzdaJX0UORnz+21WaYnrSCjsCF4pq7CH7+00RWoZkxVJqrVKFsQnC6ONAVa82N QZcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bIxuAi/zK9n6L4ct9fNEX2sco1bprmCREnncAgcx8bw=; b=GDfTKJrL/Ed8nTzFSz9od/CQjLziU9fenQZoUV4R6nlGXihuahV7tdnATdvXRwBym6 h5V+Ki4WKLUyQpyGWOiAWrABwrPykb85BZ1H3Y299Z47eHPTpdjf++cNz8mKgu6MeqIM skGw3pL5sS3N8BPhz3TAnJ1rwOxtTLq+TLQwKYuxega8GYtp+adHRZgroUG7LYNhd8EN 3XcTtlCxPY+jmohQ4yoCatYill2zQCUmNaL3vqz0BeheSwRMJqsyBF41lUxM6VkaqZRR felGTJkAQgB0fsriGcnioKybfoMNQ8SVFa0XWS9ukuHpkcBLQNKPdvEyqd0ji/A0YvUx YvkQ== X-Gm-Message-State: ACrzQf29LHQlRI3mCqWHvxDEqGUSfxvFxP8uMfI0IDhR5YzmJeeBpS7x LiTygw7ukqXEUhhb4oiLzSyehHOWhduw5Q== X-Received: by 2002:a05:6402:1d4f:b0:461:d2ed:788c with SMTP id dz15-20020a0564021d4f00b00461d2ed788cmr3797803edb.418.1666620464840; Mon, 24 Oct 2022 07:07:44 -0700 (PDT) Received: from thomas-win.localdomain ([213.138.44.237]) by smtp.gmail.com with ESMTPSA id s19-20020a170906455300b0079800b81709sm6056436ejq.219.2022.10.24.07.07.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Oct 2022 07:07:44 -0700 (PDT) From: Thomas Siedel To: ffmpeg-devel@ffmpeg.org Date: Mon, 24 Oct 2022 16:06:45 +0200 Message-Id: <20221024140645.4945-11-thomas.ff@spin-digital.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221024140645.4945-1-thomas.ff@spin-digital.com> References: <20221024140645.4945-1-thomas.ff@spin-digital.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 10/10] avcodec: increase minor version for H266/VVC X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Thomas Siedel Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 3dPR92cH/iPf Increase avcodec minor version for vvc support. Signed-off-by: Thomas Siedel --- libavcodec/version.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavcodec/version.h b/libavcodec/version.h index f8abc803b6..86ac0f3871 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -29,7 +29,7 @@ #include "version_major.h" -#define LIBAVCODEC_VERSION_MINOR 51 +#define LIBAVCODEC_VERSION_MINOR 52 #define LIBAVCODEC_VERSION_MICRO 100 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \