From patchwork Sat Jul 13 14:58:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 50527 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:2298:b0:482:c625:d099 with SMTP id fp24csp1439587vqb; Sat, 13 Jul 2024 09:16:48 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCXjF5Ks3a9uW9Zu1xobFBcf6C/mPPQzntznTjQIFjdMzCGUblXjFR2GRwGckn+XIBV3XB997gyOg5GvlenlTjde4sFAdkZFZm/2vA== X-Google-Smtp-Source: AGHT+IEnUPmG21Go+RAVwz9iFRKH4o5a15GCVuwtNsLA0ehJOgdmfor1H6fW1BU/LrdNPlHGeV52 X-Received: by 2002:a2e:95ce:0:b0:2ee:86c1:f743 with SMTP id 38308e7fff4ca-2eeb30e36a2mr89849931fa.15.1720887408169; Sat, 13 Jul 2024 09:16:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1720887408; cv=none; d=google.com; s=arc-20160816; b=V4JUvHNM3wlT3JMClDhWJzeDBQzMkPaZyIwJcf44Wlk8/LrZM/lujj1OERl1kYDPx5 wkrs6mlilzcxEnJiFdae9NvCi0t9QDvOwVjRzYE0RYu8W9TW1KivX78sLqw+cEoNadaa cNCNacsEad2kAdfGc0QgCMJWDQs+8DHkFCZzmjdjxN38p/SltTsesvh3wS1PesDv1Yul jxTxmzG2eh1Aii2N1x9sCaI6sitYrtbTyuVlqy2deVEqntrL0V3Ar17DJhKy34+RzTsD ik09tuB2C5fTdLbLywZUAJygYv+QFv45N2tKYlcuCTkU5khCraiIAWOw0EV40JoFjZgB 07Qw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=Z+AHAIlH1VOSiuXC/D9rjjh7wDe60j+xLjFw3JGXffY=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=Y/gtqTeK+dHZ7Irdfz3SUmw5DML4rKfzc8Qq4yDWULspvclTpYC8aujEJkJLzS6G0p pGqbHwbaAio0oV7Rehx0JLoRFfMoD+gm1f444IKVUWhi9RyGSGTbyxFseQ7YGawHD5Ou 5o8+iPHaf5IKyV7jtKMhbNA++JHq71YJK6nkwwbQ1Wsxdzf3LMfRRNXsW1W69hRbbjoC GgE5ZteMS8TyeQTryQcH31mbPvEpfoeqxzFPJ1TbPz85U4TFOF99U2tSK/U/Sf1s4+AJ SglW0+yD0QIKGxbld2SAw80hpOcYZ4MYLZ49AItU9MKWdDnT8i1XGlIfx3UubcdLJk8o DHFw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=EWP0PMyj; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 38308e7fff4ca-2eee192fe58si4346191fa.447.2024.07.13.09.16.47; Sat, 13 Jul 2024 09:16:48 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=EWP0PMyj; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C477368DBCA; Sat, 13 Jul 2024 17:59:14 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2845668D82A for ; Sat, 13 Jul 2024 17:59:06 +0300 (EEST) Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-1fb19ca5273so18227835ad.3 for ; Sat, 13 Jul 2024 07:59:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1720882743; x=1721487543; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=Gp6Mqpbsgv0lxXiThW7/4kMTO2ywv7q65SNKIHc7l4c=; b=EWP0PMyjWRNkKCNkNtW2aBBaWKy/vfxAzBsB/CIMZUPK34NQuHNcthHqoHQXYnpXnr ag3q7M62Xn3oEwiNbOCFqCrys0Vo8Wrsr3EXOjBhKR4VMoJYi4Pkfy8mvJrPYNJdZ/lP tfmNK4Q9JtKHp4e6Kd6ZhOwx6ZiqppemCLk7NeolmCOlphOPKIfzlztOxalhsZyAnAX4 ELO/HIok6ZHrTlk2xob6FXOvEcKP0K3pwSYr5cPTQ2pcXZwpqmkJ67UQ4vPVYE7EIdFc s1jZzo+qX+GxqwjNqMUoxvxfY9Jd6y82G3p05PWS1bcjhnx7W+5XaZesSQr/Tb9J6XGZ aRQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720882743; x=1721487543; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Gp6Mqpbsgv0lxXiThW7/4kMTO2ywv7q65SNKIHc7l4c=; b=JamFcSCvr2APWNzxerIGe27xEbO1MC1rxh72S68jwwsHrV+DzXFO3ZFZ6t9rSFE3tb 7FT77NrQ2XdkZ6SZUIDYjwHujhnTgrp5iJ5gpc887tQv6cxQ34PgLFdBMGhcUZoVXnFL YRPYL4hDm8dIBEAXgVGJePy4H1V7//X3rZ4CRHy4FJu5DGPzl/OO2g18iAhP3egbAsFk 1B+VIPMY1WG4Je7EjMshgn0B6w09BKGWXQHAIP33yS1hypcFG9Fvo3tbI845Bc0zSez0 M3jyUYmuw7KwhtjteqzTKIUmvUyrhgb9T+XMqcfvxwaHMe7MSbPxSKVCMGkFbvU5et5H h29Q== X-Gm-Message-State: AOJu0YzRg6CE2DJjKp626rpRmyduWl86GHXcPiUYCbEjr/uZjquR8wun phFoMlwpydRkOKt9eZ6KAx+wVuGcgQ39+6IyiE11sRw8nglID2DF216+xw== X-Received: by 2002:a17:902:ce8d:b0:1fb:83c5:cf8a with SMTP id d9443c01a7336-1fbb6d25184mr120695055ad.8.1720882743228; Sat, 13 Jul 2024 07:59:03 -0700 (PDT) Received: from localhost.localdomain ([190.194.167.233]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1fc0eea3cfbsm9800275ad.115.2024.07.13.07.59.01 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 13 Jul 2024 07:59:02 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Sat, 13 Jul 2024 11:58:44 -0300 Message-ID: <20240713145846.1331-2-jamrial@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240713145846.1331-1-jamrial@gmail.com> References: <20240713145846.1331-1-jamrial@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/4] avcodec/cbs_h265: add partial support for Multilayer extension fields in parameter set NALUs X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: CjUw0hGSs8V2 Signed-off-by: James Almer --- libavcodec/cbs_h2645.c | 3 +- libavcodec/cbs_h265.h | 44 ++++++++++ libavcodec/cbs_h265_syntax_template.c | 113 +++++++++++++++++++++++++- libavcodec/hevc/ps.h | 4 +- 4 files changed, 159 insertions(+), 5 deletions(-) diff --git a/libavcodec/cbs_h2645.c b/libavcodec/cbs_h2645.c index 5ec781ddab..828e56b8c0 100644 --- a/libavcodec/cbs_h2645.c +++ b/libavcodec/cbs_h2645.c @@ -499,7 +499,8 @@ static int cbs_h2645_fragment_add_nals(CodedBitstreamContext *ctx, size_t size = nal->size; enum AVCodecID codec_id = ctx->codec->codec_id; - if (codec_id != AV_CODEC_ID_VVC && nal->nuh_layer_id > 0) + if (codec_id == AV_CODEC_ID_HEVC && nal->nuh_layer_id > 0 && + (nal->type < HEVC_NAL_VPS || nal->type > HEVC_NAL_PPS)) continue; // Remove trailing zeroes. diff --git a/libavcodec/cbs_h265.h b/libavcodec/cbs_h265.h index 91a5a55317..afb942ced5 100644 --- a/libavcodec/cbs_h265.h +++ b/libavcodec/cbs_h265.h @@ -248,12 +248,16 @@ typedef struct H265RawSPS { uint8_t sps_video_parameter_set_id; uint8_t sps_max_sub_layers_minus1; + uint8_t sps_ext_or_max_sub_layers_minus1; uint8_t sps_temporal_id_nesting_flag; H265RawProfileTierLevel profile_tier_level; uint8_t sps_seq_parameter_set_id; + uint8_t update_rep_format_flag; + uint8_t sps_rep_format_idx; + uint8_t chroma_format_idc; uint8_t separate_colour_plane_flag; @@ -284,6 +288,8 @@ typedef struct H265RawSPS { uint8_t max_transform_hierarchy_depth_intra; uint8_t scaling_list_enabled_flag; + uint8_t sps_infer_scaling_list_flag; + uint8_t sps_scaling_list_ref_layer_id; uint8_t sps_scaling_list_data_present_flag; H265RawScalingList scaling_list; @@ -342,6 +348,9 @@ typedef struct H265RawSPS { uint8_t motion_vector_resolution_control_idc; uint8_t intra_boundary_filtering_disable_flag; + + // Multilayer extension. + uint8_t inter_view_mv_vert_constraint_flag; } H265RawSPS; typedef struct H265RawPPS { @@ -433,6 +442,41 @@ typedef struct H265RawPPS { uint8_t luma_bit_depth_entry_minus8; uint8_t chroma_bit_depth_entry_minus8; uint16_t pps_palette_predictor_initializers[3][128]; + + // Multilayer extension. + uint8_t poc_reset_info_present_flag; + uint8_t pps_infer_scaling_list_flag; + uint8_t pps_scaling_list_ref_layer_id; + uint8_t num_ref_loc_offsets; + uint8_t ref_loc_offset_layer_id[64]; + uint8_t scaled_ref_layer_offset_present_flag[64]; + int16_t scaled_ref_layer_left_offset[64]; + int16_t scaled_ref_layer_top_offset[64]; + int16_t scaled_ref_layer_right_offset[64]; + int16_t scaled_ref_layer_bottom_offset[64]; + uint8_t ref_region_offset_present_flag[64]; + int16_t ref_region_left_offset[64]; + int16_t ref_region_top_offset[64]; + int16_t ref_region_right_offset[64]; + int16_t ref_region_bottom_offset[64]; + uint8_t resample_phase_set_present_flag[64]; + uint8_t phase_hor_luma[64]; + uint8_t phase_ver_luma[64]; + uint8_t phase_hor_chroma_plus8[64]; + uint8_t phase_ver_chroma_plus8[64]; + uint8_t colour_mapping_enabled_flag; + uint8_t num_cm_ref_layers; + uint8_t cm_ref_layer_id[62]; + uint8_t cm_octant_depth; + uint8_t cm_y_part_num_log2; + uint8_t luma_bit_depth_cm_input; + uint8_t chroma_bit_depth_cm_input; + uint8_t luma_bit_depth_cm_output; + uint8_t chroma_bit_depth_cm_output; + uint8_t cm_res_quant_bits; + uint8_t cm_delta_flc_bits; + int8_t cm_adapt_threshold_u_delta; + int8_t cm_adapt_threshold_v_delta; } H265RawPPS; typedef struct H265RawAUD { diff --git a/libavcodec/cbs_h265_syntax_template.c b/libavcodec/cbs_h265_syntax_template.c index c6db439b3b..2f8ee0db10 100644 --- a/libavcodec/cbs_h265_syntax_template.c +++ b/libavcodec/cbs_h265_syntax_template.c @@ -747,6 +747,16 @@ static int FUNC(sps_scc_extension)(CodedBitstreamContext *ctx, RWContext *rw, return 0; } +static int FUNC(sps_multilayer_extension)(CodedBitstreamContext *ctx, RWContext *rw, + H265RawSPS *current) +{ + int err; + + flag(inter_view_mv_vert_constraint_flag); + + return 0; +} + static int FUNC(vui_parameters_default)(CodedBitstreamContext *ctx, RWContext *rw, H265RawVUI *current, H265RawSPS *sps) @@ -781,6 +791,7 @@ static int FUNC(sps)(CodedBitstreamContext *ctx, RWContext *rw, int err, i; unsigned int min_cb_log2_size_y, ctb_log2_size_y, min_cb_size_y, min_tb_log2_size_y; + unsigned int multi_layer_ext_sps_flag; HEADER("Sequence Parameter Set"); @@ -794,7 +805,17 @@ static int FUNC(sps)(CodedBitstreamContext *ctx, RWContext *rw, return AVERROR_INVALIDDATA; } + if (current->nal_unit_header.nuh_layer_id == 0) u(3, sps_max_sub_layers_minus1, 0, vps->vps_max_sub_layers_minus1); + else { + u(3, sps_ext_or_max_sub_layers_minus1, 0, HEVC_MAX_SUB_LAYERS); + infer(sps_max_sub_layers_minus1, current->sps_ext_or_max_sub_layers_minus1 == 7 + ? vps->vps_max_sub_layers_minus1 + : current->sps_ext_or_max_sub_layers_minus1); + } + multi_layer_ext_sps_flag = current->nal_unit_header.nuh_layer_id && + current->sps_ext_or_max_sub_layers_minus1 == 7; + if (!multi_layer_ext_sps_flag) { flag(sps_temporal_id_nesting_flag); if (vps->vps_temporal_id_nesting_flag && @@ -814,9 +835,20 @@ static int FUNC(sps)(CodedBitstreamContext *ctx, RWContext *rw, CHECK(FUNC(profile_tier_level)(ctx, rw, ¤t->profile_tier_level, 1, current->sps_max_sub_layers_minus1)); + } else { + if (current->sps_max_sub_layers_minus1 > 0) + infer(sps_temporal_id_nesting_flag, vps->vps_temporal_id_nesting_flag); + else + infer(sps_temporal_id_nesting_flag, 1); + } ue(sps_seq_parameter_set_id, 0, 15); + if (multi_layer_ext_sps_flag) { + flag(update_rep_format_flag); + if (current->update_rep_format_flag) + ub(8, sps_rep_format_idx); + } else { ue(chroma_format_idc, 0, 3); if (current->chroma_format_idc == 3) flag(separate_colour_plane_flag); @@ -841,9 +873,11 @@ static int FUNC(sps)(CodedBitstreamContext *ctx, RWContext *rw, ue(bit_depth_luma_minus8, 0, 8); ue(bit_depth_chroma_minus8, 0, 8); + } ue(log2_max_pic_order_cnt_lsb_minus4, 0, 12); + if (!multi_layer_ext_sps_flag) { flag(sps_sub_layer_ordering_info_present_flag); for (i = (current->sps_sub_layer_ordering_info_present_flag ? 0 : current->sps_max_sub_layers_minus1); @@ -865,6 +899,7 @@ static int FUNC(sps)(CodedBitstreamContext *ctx, RWContext *rw, current->sps_max_latency_increase_plus1[current->sps_max_sub_layers_minus1]); } } + } ue(log2_min_luma_coding_block_size_minus3, 0, 3); min_cb_log2_size_y = current->log2_min_luma_coding_block_size_minus3 + 3; @@ -895,9 +930,17 @@ static int FUNC(sps)(CodedBitstreamContext *ctx, RWContext *rw, flag(scaling_list_enabled_flag); if (current->scaling_list_enabled_flag) { + if (multi_layer_ext_sps_flag) + flag(sps_infer_scaling_list_flag); + else + infer(sps_infer_scaling_list_flag, 0); + if (current->sps_infer_scaling_list_flag) + ub(6, sps_scaling_list_ref_layer_id); + else { flag(sps_scaling_list_data_present_flag); if (current->sps_scaling_list_data_present_flag) CHECK(FUNC(scaling_list_data)(ctx, rw, ¤t->scaling_list)); + } } else { infer(sps_scaling_list_data_present_flag, 0); } @@ -955,7 +998,7 @@ static int FUNC(sps)(CodedBitstreamContext *ctx, RWContext *rw, if (current->sps_range_extension_flag) CHECK(FUNC(sps_range_extension)(ctx, rw, current)); if (current->sps_multilayer_extension_flag) - return AVERROR_PATCHWELCOME; + CHECK(FUNC(sps_multilayer_extension)(ctx, rw, current)); if (current->sps_3d_extension_flag) return AVERROR_PATCHWELCOME; if (current->sps_scc_extension_flag) @@ -996,6 +1039,72 @@ static int FUNC(pps_range_extension)(CodedBitstreamContext *ctx, RWContext *rw, return 0; } +static int FUNC(pps_multilayer_extension)(CodedBitstreamContext *ctx, RWContext *rw, + H265RawPPS *current) +{ + CodedBitstreamH265Context *h265 = ctx->priv_data; + const H265RawVPS *vps = h265->active_vps; + int offset; + int err, i; + + flag(poc_reset_info_present_flag); + flag(pps_infer_scaling_list_flag); + if (current->pps_infer_scaling_list_flag) + ub(6, pps_scaling_list_ref_layer_id); + + if (!vps) { + av_log(ctx->log_ctx, AV_LOG_ERROR, "VPS missing for PPS Multilayer Extension.\n"); + return AVERROR_INVALIDDATA; + } + + ue(num_ref_loc_offsets, 0, vps->vps_max_layers_minus1); + for (i = 0; i < current->num_ref_loc_offsets; i++) { + ubs(6, ref_loc_offset_layer_id[i], 1, i); + offset = current->ref_loc_offset_layer_id[i]; + flags(scaled_ref_layer_offset_present_flag[i], 1, i); + if (current->scaled_ref_layer_offset_present_flag[i]) { + ses(scaled_ref_layer_left_offset[offset], -16384, 16383, 1, offset); + ses(scaled_ref_layer_top_offset[offset], -16384, 16383, 1, offset); + ses(scaled_ref_layer_right_offset[offset], -16384, 16383, 1, offset); + ses(scaled_ref_layer_bottom_offset[offset], -16384, 16383, 1, offset); + } else { + infer(scaled_ref_layer_left_offset[offset], 0); + infer(scaled_ref_layer_top_offset[offset], 0); + infer(scaled_ref_layer_right_offset[offset], 0); + infer(scaled_ref_layer_bottom_offset[offset], 0); + } + flags(ref_region_offset_present_flag[i], 1, i); + if (current->ref_region_offset_present_flag[i]) { + ses(ref_region_left_offset[offset], -16384, 16383, 1, offset); + ses(ref_region_top_offset[offset], -16384, 16383, 1, offset); + ses(ref_region_right_offset[offset], -16384, 16383, 1, offset); + ses(ref_region_bottom_offset[offset], -16384, 16383, 1, offset); + } else { + infer(ref_region_left_offset[offset], 0); + infer(ref_region_top_offset[offset], 0); + infer(ref_region_right_offset[offset], 0); + infer(ref_region_bottom_offset[offset], 0); + } + flags(resample_phase_set_present_flag[i], 1, i); + if (current->resample_phase_set_present_flag[i]) { + ues(phase_hor_luma[offset], 0, 31, 1, offset); + ues(phase_ver_luma[offset], 0, 31, 1, offset); + ues(phase_hor_chroma_plus8[offset], 0, 63, 1, offset); + ues(phase_ver_chroma_plus8[offset], 0, 63, 1, offset); + } else { + infer(phase_hor_luma[offset], 0); + infer(phase_ver_luma[offset], 0); + infer(phase_hor_chroma_plus8[offset], 8); + } + } + + flag(colour_mapping_enabled_flag); + if (current->colour_mapping_enabled_flag) + return AVERROR_PATCHWELCOME; + + return 0; +} + static int FUNC(pps_scc_extension)(CodedBitstreamContext *ctx, RWContext *rw, H265RawPPS *current) { @@ -1146,7 +1255,7 @@ static int FUNC(pps)(CodedBitstreamContext *ctx, RWContext *rw, if (current->pps_range_extension_flag) CHECK(FUNC(pps_range_extension)(ctx, rw, current)); if (current->pps_multilayer_extension_flag) - return AVERROR_PATCHWELCOME; + CHECK(FUNC(pps_multilayer_extension)(ctx, rw, current)); if (current->pps_3d_extension_flag) return AVERROR_PATCHWELCOME; if (current->pps_scc_extension_flag) diff --git a/libavcodec/hevc/ps.h b/libavcodec/hevc/ps.h index 17395c5510..fab5a46273 100644 --- a/libavcodec/hevc/ps.h +++ b/libavcodec/hevc/ps.h @@ -390,8 +390,8 @@ typedef struct HEVCPPS { uint8_t resample_phase_set_present_flag[64]; uint8_t phase_hor_luma[64]; uint8_t phase_ver_luma[64]; - int8_t phase_hor_chroma[64]; - int8_t phase_ver_chroma[64]; + uint8_t phase_hor_chroma[64]; + uint8_t phase_ver_chroma[64]; uint8_t colour_mapping_enabled_flag; uint8_t num_cm_ref_layers; uint8_t cm_ref_layer_id[62];