From patchwork Sat Sep 14 10:45:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 51585 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:9fc3:0:b0:48e:c0f8:d0de with SMTP id k3csp303501vqy; Sat, 14 Sep 2024 04:19:24 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCUODdUETpcoRBnMr22rj0Sa0H4WVJsHTDSToXNebFtR24D7SD7fJsB48/oQEcpEqVM34Nf/g787qd7+DhINCpxL@gmail.com X-Google-Smtp-Source: AGHT+IG1QwF/3rxuVXUpNKa5ZaVc8Jt0cHmr/TQC+F6QkzN1iz8ihXaSFanQj3WJaI5rWW2FsEH9 X-Received: by 2002:a17:907:3f17:b0:a72:7a71:7f4f with SMTP id a640c23a62f3a-a902a3d2e92mr1024978166b.7.1726312763426; Sat, 14 Sep 2024 04:19:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1726312763; cv=none; d=google.com; s=arc-20240605; b=lXyLnHeN9uDwK9Drr/+v11sArGGIjiKMkB1zEmkj/WcTOgh1LRdh7JrnW5abuLziQJ HD40+h3/aZX7iyKlNoeZg21M2XOzumM2li38tyNLJHkIQr3Nscp6Asl1bNyeNOFg30qz udFWXTmfw15ljViIeief6nchfOrb2WnAqlHJk3aAgemMxxONpIYYMx0G9Pcsqx/Ynj+v EkowtevS0pEWQenblwpgRrq7+jh8Z9K48vK3LdEJ4Rx5DnBZl/DuRFo9dseCJIxmgOv/ JSrshI1jOxk9qclqGAJEoz7Zywt7Q8K0XBceVK+iRo60vJGDLxl0TLL2DfvHWuFgfNKF 4//g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=dRCp8vflKJ/y57ho6+86QF4wFPyDq+8sF6pRqyJghlg=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=CE0It1PqKXnvpJGu2pKaIOT2CuWth7vPNNGVhZ9WhVGePsBJRFDfMU/JqpBQzxcxba Sy0j4huM+nFuDFb6OSQmma0tQhpZOuoKxUMvLw0uQmQoRJUeAPpCIFXcweU2W1C1SRNi 0H0jor7bqfkyAmxZQjeaconM8Y4YUIc7n7FD5+XK2T7tPAFioi24qKj3+fsG1ZON/R7H sZfFh0H7zIKSCdeIe5EKIPrNPXlNB9getaSyT0wqBLsSoNwLsZHXuNWafxNd+CGGzoQM 6SELSBOIeeHyBSnta3r7vuF+dqFuI13fND5OKtUe7dblu89PJTJBjRsGWhibEmTgJQC4 0V3A==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@khirnov.net header.s=mail header.b=IyfTO1ok; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a906135931csi86933266b.948.2024.09.14.04.19.23; Sat, 14 Sep 2024 04:19:23 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@khirnov.net header.s=mail header.b=IyfTO1ok; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id ADDDA68DE34; Sat, 14 Sep 2024 14:11:04 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail1.khirnov.net (quelana.khirnov.net [94.230.150.81]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5FB4A68DC90 for ; Sat, 14 Sep 2024 14:10:53 +0300 (EEST) Authentication-Results: mail1.khirnov.net; dkim=pass (2048-bit key; unprotected) header.d=khirnov.net header.i=@khirnov.net header.a=rsa-sha256 header.s=mail header.b=IyfTO1ok; dkim-atps=neutral Received: from localhost (mail1.khirnov.net [IPv6:::1]) by mail1.khirnov.net (Postfix) with ESMTP id C7D754DDE for ; Sat, 14 Sep 2024 13:10:48 +0200 (CEST) Received: from mail1.khirnov.net ([IPv6:::1]) by localhost (mail1.khirnov.net [IPv6:::1]) (amavis, port 10024) with ESMTP id GIOiVe9p70ge for ; Sat, 14 Sep 2024 13:10:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=khirnov.net; s=mail; t=1726312246; bh=d4UmMs0eIWdvSP2Za1D4+cNA+2KYlrlRHb9d4DQ4Lb4=; h=From:To:Subject:Date:In-Reply-To:References:From; b=IyfTO1ok5yuNG8YakVIGVIf8e/wKgFTBws9XOpEOkVz1XTsaUBDt97+LSXV+S1j5u 4Tq8/if8tOEmPm1hCkWqmP1T/+N6ot0OIAYQ04zv9lvMaM5CChbnG442c3GXOF5b2k tA1e+WQ3PGz03QAMRSJg96yqVYnQqKI3r1Hw+Vxw34yO8Jagv5THgS3pjEFvK30XID +bPuKNbGEqYTSWxuYLP9SDkHYPYjNK2KqGsiRKdT+4Av5GoX7YBXi76O0FkPB4AX2h Lov8xgxNeKY+e2lcnHXTf/3dB+38kMF3sFyKd5LEqbF3GWnFgkILTPK2FvIRnEvJ8l oujMcJuqkanQA== Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail1.khirnov.net (Postfix) with ESMTPS id 5ED784E06 for ; Sat, 14 Sep 2024 13:10:46 +0200 (CEST) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:::1]) by libav.khirnov.net (Postfix) with ESMTP id 376673A19A6 for ; Sat, 14 Sep 2024 13:10:41 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Sat, 14 Sep 2024 12:45:33 +0200 Message-ID: <20240914111036.17164-9-anton@khirnov.net> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240914111036.17164-1-anton@khirnov.net> References: <20240914111036.17164-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 08/23] lavc/hevc/ps: implement SPS parsing for nuh_layer_id>0 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: pMBZwqzhDohv Cf. F.7.3.2.2 "Sequence parameter set RBSP syntax", which extends normal SPS parsing with special clauses depending on MultiLayerExtSpsFlag. --- libavcodec/hevc/hevcdec.c | 2 +- libavcodec/hevc/parse.c | 3 +- libavcodec/hevc/parser.c | 2 +- libavcodec/hevc/ps.c | 65 +++++++++++++++++++++++++++++++++++---- libavcodec/hevc/ps.h | 7 +++-- libavcodec/qsvenc_hevc.c | 2 +- 6 files changed, 68 insertions(+), 13 deletions(-) diff --git a/libavcodec/hevc/hevcdec.c b/libavcodec/hevc/hevcdec.c index d915d74d22..ebe2f5ad11 100644 --- a/libavcodec/hevc/hevcdec.c +++ b/libavcodec/hevc/hevcdec.c @@ -3260,7 +3260,7 @@ static int decode_nal_unit(HEVCContext *s, const H2645NAL *nal) break; case HEVC_NAL_SPS: ret = ff_hevc_decode_nal_sps(&gb, s->avctx, &s->ps, - s->apply_defdispwin); + nal->nuh_layer_id, s->apply_defdispwin); if (ret < 0) goto fail; break; diff --git a/libavcodec/hevc/parse.c b/libavcodec/hevc/parse.c index ec8d1aeacf..ad84b7b152 100644 --- a/libavcodec/hevc/parse.c +++ b/libavcodec/hevc/parse.c @@ -49,7 +49,8 @@ static int hevc_decode_nal_units(const uint8_t *buf, int buf_size, HEVCParamSets goto done; break; case HEVC_NAL_SPS: - ret = ff_hevc_decode_nal_sps(&nal->gb, logctx, ps, apply_defdispwin); + ret = ff_hevc_decode_nal_sps(&nal->gb, logctx, ps, + nal->nuh_layer_id, apply_defdispwin); if (ret < 0) goto done; break; diff --git a/libavcodec/hevc/parser.c b/libavcodec/hevc/parser.c index 2d14b4fae2..8c7444a162 100644 --- a/libavcodec/hevc/parser.c +++ b/libavcodec/hevc/parser.c @@ -209,7 +209,7 @@ static int parse_nal_units(AVCodecParserContext *s, const uint8_t *buf, ff_hevc_decode_nal_vps(gb, avctx, ps); break; case HEVC_NAL_SPS: - ff_hevc_decode_nal_sps(gb, avctx, ps, 1); + ff_hevc_decode_nal_sps(gb, avctx, ps, nal->nuh_layer_id, 1); break; case HEVC_NAL_PPS: ff_hevc_decode_nal_pps(gb, avctx, ps); diff --git a/libavcodec/hevc/ps.c b/libavcodec/hevc/ps.c index 81b3f9cd09..2d09500278 100644 --- a/libavcodec/hevc/ps.c +++ b/libavcodec/hevc/ps.c @@ -1148,12 +1148,12 @@ static int map_pixel_format(AVCodecContext *avctx, HEVCSPS *sps) } int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, - int apply_defdispwin, const HEVCVPS * const *vps_list, - AVCodecContext *avctx) + unsigned nuh_layer_id, int apply_defdispwin, + const HEVCVPS * const *vps_list, AVCodecContext *avctx) { HEVCWindow *ow; int ret = 0; - int bit_depth_chroma, start, num_comps; + int bit_depth_chroma, num_comps, multi_layer_ext; int i; // Coded parameters @@ -1170,16 +1170,29 @@ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, } sps->max_sub_layers = get_bits(gb, 3) + 1; + multi_layer_ext = nuh_layer_id > 0 && + sps->max_sub_layers == HEVC_MAX_SUB_LAYERS + 1; + if (multi_layer_ext) { + if (!sps->vps) + return AVERROR(EINVAL); + + sps->max_sub_layers = sps->vps->vps_max_sub_layers; + } if (sps->max_sub_layers > HEVC_MAX_SUB_LAYERS) { av_log(avctx, AV_LOG_ERROR, "sps_max_sub_layers out of range: %d\n", sps->max_sub_layers); return AVERROR_INVALIDDATA; } + if (!multi_layer_ext) { sps->temporal_id_nesting = get_bits(gb, 1); if ((ret = parse_ptl(gb, avctx, 1, &sps->ptl, sps->max_sub_layers)) < 0) return ret; + } else { + sps->temporal_id_nesting = sps->max_sub_layers > 1 ? + sps->vps->vps_max_sub_layers : 1; + } *sps_id = get_ue_golomb_long(gb); if (*sps_id >= HEVC_MAX_SPS_COUNT) { @@ -1187,6 +1200,28 @@ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, return AVERROR_INVALIDDATA; } + if (multi_layer_ext) { + const RepFormat *rf = &sps->vps->rep_format; + + if (get_bits1(gb) && // update_rep_format_flag + get_bits(gb, 8)) { // sps_rep_format_idx + av_log(avctx, AV_LOG_ERROR, "sps_rep_format_idx!=0\n"); + return AVERROR_PATCHWELCOME; + } + + sps->separate_colour_plane = rf->separate_colour_plane_flag; + sps->chroma_format_idc = sps->separate_colour_plane ? 0 : + rf->chroma_format_idc; + sps->bit_depth = rf->bit_depth_luma; + sps->width = rf->pic_width_in_luma_samples; + sps->height = rf->pic_height_in_luma_samples; + + sps->pic_conf_win.left_offset = rf->conf_win_left_offset; + sps->pic_conf_win.right_offset = rf->conf_win_right_offset; + sps->pic_conf_win.top_offset = rf->conf_win_top_offset; + sps->pic_conf_win.bottom_offset = rf->conf_win_bottom_offset; + + } else { sps->chroma_format_idc = get_ue_golomb_long(gb); if (sps->chroma_format_idc > 3U) { av_log(avctx, AV_LOG_ERROR, "chroma_format_idc %d is invalid\n", sps->chroma_format_idc); @@ -1228,7 +1263,6 @@ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, sps->pic_conf_win.top_offset = sps->pic_conf_win.bottom_offset = 0; } - sps->output_window = sps->pic_conf_win; } sps->bit_depth = get_ue_golomb_31(gb) + 8; @@ -1251,6 +1285,9 @@ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, return AVERROR_INVALIDDATA; } sps->bit_depth_chroma = bit_depth_chroma; + } + + sps->output_window = sps->pic_conf_win; ret = map_pixel_format(avctx, sps); if (ret < 0) @@ -1263,6 +1300,9 @@ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, return AVERROR_INVALIDDATA; } + if (!multi_layer_ext) { + int start; + sps->sublayer_ordering_info = get_bits1(gb); start = sps->sublayer_ordering_info ? 0 : sps->max_sub_layers - 1; for (i = start; i < sps->max_sub_layers; i++) { @@ -1292,6 +1332,13 @@ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, sps->temporal_layer[i].max_latency_increase = sps->temporal_layer[start].max_latency_increase; } } + } else { + for (int i = 0; i < sps->max_sub_layers; i++) { + sps->temporal_layer[i].max_dec_pic_buffering = sps->vps->dpb_size.max_dec_pic_buffering; + sps->temporal_layer[i].num_reorder_pics = sps->vps->dpb_size.max_num_reorder_pics; + sps->temporal_layer[i].max_latency_increase = sps->vps->dpb_size.max_latency_increase; + } + } sps->log2_min_cb_size = get_ue_golomb_long(gb) + 3; sps->log2_diff_max_min_coding_block_size = get_ue_golomb_long(gb); @@ -1328,6 +1375,11 @@ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, if (sps->scaling_list_enabled) { set_default_scaling_list_data(&sps->scaling_list); + if (multi_layer_ext && get_bits1(gb)) { // sps_infer_scaling_list_flag + av_log(avctx, AV_LOG_ERROR, "sps_infer_scaling_list_flag=1 not supported\n"); + return AVERROR_PATCHWELCOME; + } + if (get_bits1(gb)) { ret = scaling_list_data(gb, avctx, &sps->scaling_list, sps); if (ret < 0) @@ -1582,7 +1634,8 @@ static int compare_sps(const HEVCSPS *sps1, const HEVCSPS *sps2) } int ff_hevc_decode_nal_sps(GetBitContext *gb, AVCodecContext *avctx, - HEVCParamSets *ps, int apply_defdispwin) + HEVCParamSets *ps, unsigned nuh_layer_id, + int apply_defdispwin) { HEVCSPS *sps = ff_refstruct_alloc_ext(sizeof(*sps), 0, NULL, hevc_sps_free); unsigned int sps_id; @@ -1601,7 +1654,7 @@ int ff_hevc_decode_nal_sps(GetBitContext *gb, AVCodecContext *avctx, } ret = ff_hevc_parse_sps(sps, gb, &sps_id, - apply_defdispwin, + nuh_layer_id, apply_defdispwin, ps->vps_list, avctx); if (ret < 0) goto err; diff --git a/libavcodec/hevc/ps.h b/libavcodec/hevc/ps.h index bd1acf12e6..6f5b1f8755 100644 --- a/libavcodec/hevc/ps.h +++ b/libavcodec/hevc/ps.h @@ -521,13 +521,14 @@ typedef struct HEVCParamSets { * to an existing VPS */ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, - int apply_defdispwin, const HEVCVPS * const *vps_list, - AVCodecContext *avctx); + unsigned nuh_layer_id, int apply_defdispwin, + const HEVCVPS * const *vps_list, AVCodecContext *avctx); int ff_hevc_decode_nal_vps(GetBitContext *gb, AVCodecContext *avctx, HEVCParamSets *ps); int ff_hevc_decode_nal_sps(GetBitContext *gb, AVCodecContext *avctx, - HEVCParamSets *ps, int apply_defdispwin); + HEVCParamSets *ps, unsigned nuh_layer_id, + int apply_defdispwin); int ff_hevc_decode_nal_pps(GetBitContext *gb, AVCodecContext *avctx, HEVCParamSets *ps); diff --git a/libavcodec/qsvenc_hevc.c b/libavcodec/qsvenc_hevc.c index 2a397a2919..80fdf782a7 100644 --- a/libavcodec/qsvenc_hevc.c +++ b/libavcodec/qsvenc_hevc.c @@ -99,7 +99,7 @@ static int generate_fake_vps(QSVEncContext *q, AVCodecContext *avctx) } get_bits(&gb, 9); - ret = ff_hevc_parse_sps(&sps, &gb, &sps_id, 0, NULL, avctx); + ret = ff_hevc_parse_sps(&sps, &gb, &sps_id, 0, 0, NULL, avctx); av_freep(&sps_rbsp.rbsp_buffer); if (ret < 0) { av_log(avctx, AV_LOG_ERROR, "Error parsing the SPS\n");