From patchwork Tue Aug 27 15:05:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 51174 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:c944:0:b0:48e:c0f8:d0de with SMTP id k4csp364977vqt; Tue, 27 Aug 2024 09:20:16 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCW3TVHeeXLFchHNl8WBbS3WHfyViWXm/2H3BzUCtoEqVx38p+6Vi0pOhicBQn6j8Th7WtAsnxi/j6+7QFTF4BsO@gmail.com X-Google-Smtp-Source: AGHT+IFQJhH3JCYuaaaZJfkwFzW/Qe+tmvp9Ms4arDJ2a9ksZuPMpbF1WazNl3Bu/ydytefE9aQF X-Received: by 2002:a05:651c:b28:b0:2ef:2405:ff63 with SMTP id 38308e7fff4ca-2f4f493ee6amr56904701fa.5.1724775616122; Tue, 27 Aug 2024 09:20:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1724775616; cv=none; d=google.com; s=arc-20240605; b=QnhEXNJ1+0PiiLV20Vo3OdXUoixUklEz/bebyv/EC5Q3jlXQDIl8z+fHm7M92QmWtH V9BpoyiaDfzYiePK9QxBASpPB2h+AUwSBxHfH3K6b1tNVN/R9fVBqetPRoHznusLf9jI +PI9hJTyLv+TFvOxE0qkXLzWOJY6CJcSy81gKQuJ3R4E8UIzFUq+XIm8pyUo8osxYme3 vYfM91V2R683j+cGtQVgxYUVXrsJkfbHaxSyGTNjx878ekt0TBWfsZhccura7vHXAr3+ JWdxQaMqFl1jMDHXilWZDcs0Mpj66EhwNepS29ynNliYPhrpiw7q+cOOlkBROACMEkee Tuyw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=kq8NtY4VYTezY6RwtsxZWr1aQ52OHcg9ildcz9wSsuM=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=dnu1g+/+CecGgRVNy45It5GOjIkyTzW4K7L/NTzM9pp9FNFnq7BTKUMs7q7iRWf/nW zjLJIPaqSDd4MC2UGOs8qIH6aQcSnT0LOj6G8YCahqlhRnGx6JNre1CTVy4waQvzvpd5 JnTnQwwqyi+7jafDx5RRPHyKzjJi6Ps36IYJkVtqA+1peIMQ4NKR1hRQ+TJH/xGkf20A MqF3Nec42eHqYqT89LjLokWJZRY6aVPQFzp2ftcHEb3NnG0F4ShfaLWTiHMd2bt/5wzi x4yUeU7UOs1jOWkq8Xg2k0L1q0UEcS7SsMjJeuOQaIw5VLnC8CYruYMCmOTWS5WNbfrs cQOA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@khirnov.net header.s=mail header.b=JDiHbQZh; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 38308e7fff4ca-2f4048bb3besi37760141fa.489.2024.08.27.09.20.15; Tue, 27 Aug 2024 09:20:16 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@khirnov.net header.s=mail header.b=JDiHbQZh; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A4ADB68DDE5; Tue, 27 Aug 2024 18:43:06 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail1.khirnov.net (quelana.khirnov.net [94.230.150.81]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7316768DD91 for ; Tue, 27 Aug 2024 18:42:58 +0300 (EEST) Authentication-Results: mail1.khirnov.net; dkim=pass (2048-bit key; unprotected) header.d=khirnov.net header.i=@khirnov.net header.a=rsa-sha256 header.s=mail header.b=JDiHbQZh; dkim-atps=neutral Received: from localhost (mail1.khirnov.net [IPv6:::1]) by mail1.khirnov.net (Postfix) with ESMTP id ED3B44E04 for ; Tue, 27 Aug 2024 17:42:55 +0200 (CEST) Received: from mail1.khirnov.net ([IPv6:::1]) by localhost (mail1.khirnov.net [IPv6:::1]) (amavis, port 10024) with ESMTP id HfAS0CcTJcnp for ; Tue, 27 Aug 2024 17:42:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=khirnov.net; s=mail; t=1724773369; bh=CIlds5MoejAMZYk7GMz6ZiLWokqwfvD8A2+JR9NAp1I=; h=From:To:Subject:Date:In-Reply-To:References:From; b=JDiHbQZhuSIwoIDvTwWKonER+1OBrV2bGlq0lF3ZTo27Q3ul59QO/EeLTMVCILRmE agh8bLNf9f7uNto22h7jc7nEfUx0T0H3K6os6oSDGTjwK7f2fd3S7WSg8cfRNRSL3Z MPp0kwcfQ0JZpNdG1vSPjKcYxT6+d3+vCn7OZBBE9MMTUu5zNbsru73uxpD859AGSY XklzEiPXhoT2bBdDrC4sH6YnC4pzrgpPs0m4TMq1zPjkdhKl4rwJy2HF0tLouRShIL ubN24TD2598aJhvSlqIk8k1SRRXraic7pIxidSNrDvd19t0e4rS1/y6d0pRCQupxiW GUq2a8Mb3zGoA== Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail1.khirnov.net (Postfix) with ESMTPS id 6707F4E18 for ; Tue, 27 Aug 2024 17:42:49 +0200 (CEST) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:::1]) by libav.khirnov.net (Postfix) with ESMTP id 02BC03A30A6 for ; Tue, 27 Aug 2024 17:42:43 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Tue, 27 Aug 2024 17:05:08 +0200 Message-ID: <20240827154041.13846-30-anton@khirnov.net> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240827154041.13846-2-anton@khirnov.net> References: <20240827154041.13846-2-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 28/42] lavc/hevc/ps: implement SPS parsing for nuh_layer_id>0 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: zORiPkR2VJhR Cf. F.7.3.2.2 "Sequence parameter set RBSP syntax", which extends normal SPS parsing with special clauses depending on MultiLayerExtSpsFlag. --- libavcodec/hevc/hevcdec.c | 2 +- libavcodec/hevc/parse.c | 3 +- libavcodec/hevc/parser.c | 2 +- libavcodec/hevc/ps.c | 62 +++++++++++++++++++++++++++++++++++---- libavcodec/hevc/ps.h | 7 +++-- libavcodec/qsvenc_hevc.c | 2 +- 6 files changed, 65 insertions(+), 13 deletions(-) diff --git a/libavcodec/hevc/hevcdec.c b/libavcodec/hevc/hevcdec.c index 6b596f1573..260b9abef0 100644 --- a/libavcodec/hevc/hevcdec.c +++ b/libavcodec/hevc/hevcdec.c @@ -3256,7 +3256,7 @@ static int decode_nal_unit(HEVCContext *s, const H2645NAL *nal) break; case HEVC_NAL_SPS: ret = ff_hevc_decode_nal_sps(&gb, s->avctx, &s->ps, - s->apply_defdispwin); + nal->nuh_layer_id, s->apply_defdispwin); if (ret < 0) goto fail; break; diff --git a/libavcodec/hevc/parse.c b/libavcodec/hevc/parse.c index ec8d1aeacf..ad84b7b152 100644 --- a/libavcodec/hevc/parse.c +++ b/libavcodec/hevc/parse.c @@ -49,7 +49,8 @@ static int hevc_decode_nal_units(const uint8_t *buf, int buf_size, HEVCParamSets goto done; break; case HEVC_NAL_SPS: - ret = ff_hevc_decode_nal_sps(&nal->gb, logctx, ps, apply_defdispwin); + ret = ff_hevc_decode_nal_sps(&nal->gb, logctx, ps, + nal->nuh_layer_id, apply_defdispwin); if (ret < 0) goto done; break; diff --git a/libavcodec/hevc/parser.c b/libavcodec/hevc/parser.c index 8db56e259e..a10f38941b 100644 --- a/libavcodec/hevc/parser.c +++ b/libavcodec/hevc/parser.c @@ -209,7 +209,7 @@ static int parse_nal_units(AVCodecParserContext *s, const uint8_t *buf, ff_hevc_decode_nal_vps(gb, avctx, ps); break; case HEVC_NAL_SPS: - ff_hevc_decode_nal_sps(gb, avctx, ps, 1); + ff_hevc_decode_nal_sps(gb, avctx, ps, nal->nuh_layer_id, 1); break; case HEVC_NAL_PPS: ff_hevc_decode_nal_pps(gb, avctx, ps); diff --git a/libavcodec/hevc/ps.c b/libavcodec/hevc/ps.c index 0e084958be..0b34dd10a8 100644 --- a/libavcodec/hevc/ps.c +++ b/libavcodec/hevc/ps.c @@ -1145,12 +1145,12 @@ static int map_pixel_format(AVCodecContext *avctx, HEVCSPS *sps) } int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, - int apply_defdispwin, const HEVCVPS * const *vps_list, - AVCodecContext *avctx) + unsigned nuh_layer_id, int apply_defdispwin, + const HEVCVPS * const *vps_list, AVCodecContext *avctx) { HEVCWindow *ow; int ret = 0; - int bit_depth_chroma, start, num_comps; + int bit_depth_chroma, num_comps, multi_layer_ext; int i; // Coded parameters @@ -1167,16 +1167,26 @@ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, } sps->max_sub_layers = get_bits(gb, 3) + 1; + multi_layer_ext = nuh_layer_id > 0 && + sps->max_sub_layers == HEVC_MAX_SUB_LAYERS + 1; + if (multi_layer_ext) { + if (!sps->vps) + return AVERROR(EINVAL); + + sps->max_sub_layers = sps->vps->vps_max_sub_layers; + } if (sps->max_sub_layers > HEVC_MAX_SUB_LAYERS) { av_log(avctx, AV_LOG_ERROR, "sps_max_sub_layers out of range: %d\n", sps->max_sub_layers); return AVERROR_INVALIDDATA; } + if (!multi_layer_ext) { sps->temporal_id_nesting = get_bits(gb, 1); if ((ret = parse_ptl(gb, avctx, 1, &sps->ptl, sps->max_sub_layers)) < 0) return ret; + } *sps_id = get_ue_golomb_long(gb); if (*sps_id >= HEVC_MAX_SPS_COUNT) { @@ -1184,6 +1194,28 @@ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, return AVERROR_INVALIDDATA; } + if (multi_layer_ext) { + const RepFormat *rf = &sps->vps->rep_format; + + if (get_bits1(gb) && // update_rep_format_flag + get_bits(gb, 8)) { // sps_rep_format_idx + av_log(avctx, AV_LOG_ERROR, "sps_rep_format_idx!=0\n"); + return AVERROR_PATCHWELCOME; + } + + sps->separate_colour_plane = rf->separate_colour_plane_flag; + sps->chroma_format_idc = sps->separate_colour_plane ? 0 : + rf->chroma_format_idc; + sps->bit_depth = rf->bit_depth_luma; + sps->width = rf->pic_width_in_luma_samples; + sps->height = rf->pic_height_in_luma_samples; + + sps->pic_conf_win.left_offset = rf->conf_win_left_offset; + sps->pic_conf_win.right_offset = rf->conf_win_right_offset; + sps->pic_conf_win.top_offset = rf->conf_win_top_offset; + sps->pic_conf_win.bottom_offset = rf->conf_win_bottom_offset; + + } else { sps->chroma_format_idc = get_ue_golomb_long(gb); if (sps->chroma_format_idc > 3U) { av_log(avctx, AV_LOG_ERROR, "chroma_format_idc %d is invalid\n", sps->chroma_format_idc); @@ -1225,7 +1257,6 @@ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, sps->pic_conf_win.top_offset = sps->pic_conf_win.bottom_offset = 0; } - sps->output_window = sps->pic_conf_win; } sps->bit_depth = get_ue_golomb_31(gb) + 8; @@ -1248,6 +1279,9 @@ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, return AVERROR_INVALIDDATA; } sps->bit_depth_chroma = bit_depth_chroma; + } + + sps->output_window = sps->pic_conf_win; ret = map_pixel_format(avctx, sps); if (ret < 0) @@ -1260,6 +1294,9 @@ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, return AVERROR_INVALIDDATA; } + if (!multi_layer_ext) { + int start; + sps->sublayer_ordering_info = get_bits1(gb); start = sps->sublayer_ordering_info ? 0 : sps->max_sub_layers - 1; for (i = start; i < sps->max_sub_layers; i++) { @@ -1289,6 +1326,13 @@ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, sps->temporal_layer[i].max_latency_increase = sps->temporal_layer[start].max_latency_increase; } } + } else { + for (int i = 0; i < sps->max_sub_layers; i++) { + sps->temporal_layer[i].max_dec_pic_buffering = sps->vps->dpb_size.max_dec_pic_buffering; + sps->temporal_layer[i].num_reorder_pics = sps->vps->dpb_size.max_num_reorder_pics; + sps->temporal_layer[i].max_latency_increase = sps->vps->dpb_size.max_latency_increase; + } + } sps->log2_min_cb_size = get_ue_golomb_long(gb) + 3; sps->log2_diff_max_min_coding_block_size = get_ue_golomb_long(gb); @@ -1325,6 +1369,11 @@ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, if (sps->scaling_list_enabled) { set_default_scaling_list_data(&sps->scaling_list); + if (multi_layer_ext && get_bits1(gb)) { // sps_infer_scaling_list_flag + av_log(avctx, AV_LOG_ERROR, "sps_infer_scaling_list_flag=1 not supported\n"); + return AVERROR_PATCHWELCOME; + } + if (get_bits1(gb)) { ret = scaling_list_data(gb, avctx, &sps->scaling_list, sps); if (ret < 0) @@ -1579,7 +1628,8 @@ static int compare_sps(const HEVCSPS *sps1, const HEVCSPS *sps2) } int ff_hevc_decode_nal_sps(GetBitContext *gb, AVCodecContext *avctx, - HEVCParamSets *ps, int apply_defdispwin) + HEVCParamSets *ps, unsigned nuh_layer_id, + int apply_defdispwin) { HEVCSPS *sps = ff_refstruct_alloc_ext(sizeof(*sps), 0, NULL, hevc_sps_free); unsigned int sps_id; @@ -1598,7 +1648,7 @@ int ff_hevc_decode_nal_sps(GetBitContext *gb, AVCodecContext *avctx, } ret = ff_hevc_parse_sps(sps, gb, &sps_id, - apply_defdispwin, + nuh_layer_id, apply_defdispwin, ps->vps_list, avctx); if (ret < 0) goto err; diff --git a/libavcodec/hevc/ps.h b/libavcodec/hevc/ps.h index bd1acf12e6..6f5b1f8755 100644 --- a/libavcodec/hevc/ps.h +++ b/libavcodec/hevc/ps.h @@ -521,13 +521,14 @@ typedef struct HEVCParamSets { * to an existing VPS */ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, - int apply_defdispwin, const HEVCVPS * const *vps_list, - AVCodecContext *avctx); + unsigned nuh_layer_id, int apply_defdispwin, + const HEVCVPS * const *vps_list, AVCodecContext *avctx); int ff_hevc_decode_nal_vps(GetBitContext *gb, AVCodecContext *avctx, HEVCParamSets *ps); int ff_hevc_decode_nal_sps(GetBitContext *gb, AVCodecContext *avctx, - HEVCParamSets *ps, int apply_defdispwin); + HEVCParamSets *ps, unsigned nuh_layer_id, + int apply_defdispwin); int ff_hevc_decode_nal_pps(GetBitContext *gb, AVCodecContext *avctx, HEVCParamSets *ps); diff --git a/libavcodec/qsvenc_hevc.c b/libavcodec/qsvenc_hevc.c index e6c038e67d..1479d4cb8b 100644 --- a/libavcodec/qsvenc_hevc.c +++ b/libavcodec/qsvenc_hevc.c @@ -99,7 +99,7 @@ static int generate_fake_vps(QSVEncContext *q, AVCodecContext *avctx) } get_bits(&gb, 9); - ret = ff_hevc_parse_sps(&sps, &gb, &sps_id, 0, NULL, avctx); + ret = ff_hevc_parse_sps(&sps, &gb, &sps_id, 0, 0, NULL, avctx); av_freep(&sps_rbsp.rbsp_buffer); if (ret < 0) { av_log(avctx, AV_LOG_ERROR, "Error parsing the SPS\n");