From patchwork Wed Jul 3 21:26:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 50315 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:cc64:0:b0:482:c625:d099 with SMTP id k4csp3373449vqv; Wed, 3 Jul 2024 15:44:14 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCX4RB0Qwv93DqP2u70tVwujF4HLxwOfGOMPQQs6cxmTPpv3WsUXgwGUtLvYfiX5dua52umE/B7tXj0wXjl8fN29h21NNOiAFusN2g== X-Google-Smtp-Source: AGHT+IHT75taaCntQnatBl9yk1LsIx4MIboHci07bUFjF4M7NlciAibJOxPSHjHfl0NJCfY9YvWz X-Received: by 2002:a05:6402:174d:b0:58c:8c63:22ee with SMTP id 4fb4d7f45d1cf-58c8c632339mr1986747a12.42.1720046653757; Wed, 03 Jul 2024 15:44:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1720046653; cv=none; d=google.com; s=arc-20160816; b=rVyLIQo8jEXot5Gbn7oOzFV+pbL5SbMMqX+FXJ0unrw0xzbXqadEU/jHn1vzz2xugR MR6lDGJUdzWXNe5qZYBCxDzHx/j3yf1jFN3mvJtT9TnWmH45Om8QSHxi4NRkU8Srm3/z LgTJQVWmrdmUdv+Q5oNBGgOh5ZFYXAUPU8ndgenawr2i+s+xMv1kvvyFIDohWySYb3tk KaQuOCviSZ/+Y4nkku60nKqm+Oioi7TTQkFsNfwcTDcKELkdc0vRmKTQWgrHrfekWPx1 hAt6Ad++plaRMLmJGtyLWGZS4FmE2AaQ5UBgF0zDBoOZ8MxC1nQ7UUDrjMmFMlr7drnW tpLw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=9UCKBUOr2bEVy86m3GvLYtCZ5Czi8tS+Qkcrqt31Nag=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=upXJlCPryBP8Rx6ntnkJi7yi0kdeQmjzjAVv8OJW/mVD6i+2w7bMA/Or5YL6zgfDb+ 8rBilRi0WP31MHWbSMrbrEcYMz1mMjWgktruRjvYOAwQnvSXJIhLEkPxqHs5SfhmY3E8 F+tJcKJzlwFJ3aIbqZpp9bBDA2l82+dbIv7/pewi0DZGi3WbVuBuIWh7Im3nrLPC3gm+ ncFie0s2XQUtRfHw2Gb4lbPoBXBx8ELSDNOki42J69rDImuZQApJYfE0e8x7dTf9wXRY g9Sb/J0hEeOu65rUxy2+C+Q9PQCbpqgZ5621C5aF0dXSP2wUTPfjpnFvmigxKqsqViAR KZQw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=YgQkaja5; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 4fb4d7f45d1cf-58b87708e05si2049686a12.509.2024.07.03.15.44.13; Wed, 03 Jul 2024 15:44:13 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=YgQkaja5; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id CA0D168D9E9; Thu, 4 Jul 2024 00:26:50 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f50.google.com (mail-pj1-f50.google.com [209.85.216.50]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D438868D9C9 for ; Thu, 4 Jul 2024 00:26:38 +0300 (EEST) Received: by mail-pj1-f50.google.com with SMTP id 98e67ed59e1d1-2c5362c7c0bso15660a91.1 for ; Wed, 03 Jul 2024 14:26:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1720041997; x=1720646797; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=sk6Z2qCfqwzrfZz9Jz9nus+KotBOXNbFKGACxmi4gQs=; b=YgQkaja5q+J0Uor4wVCXWrROzBZlmaryXnpjP65YlQFJyF5jR3XPJN/F35GAqj7c16 0KZTv7CQ0ljYV0OWWq4Y7IrV6PikfoVyV4y/qNSxOnaexsGOeCwBIdhHA9fIivaT9xq3 vrEiG1SkiJPGIyAyS+9u8nRMwpsZUDU3+17Nw+XSL7AXiJYwg3erirRBrQE6jhZXf/Cu 0Fniobd06MBEb957C30FhVATUAt5NNe2DQXPulqoFkpjgTfS7r3fFWjDIXwcNQV6thDF 9g/VYw2HV74tiJ1HZTEZPlUXQkAMCQCUMiYHr3Zayp1CZzGysh7gFyVkJQrKXWmdDFjb Eeug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720041997; x=1720646797; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=sk6Z2qCfqwzrfZz9Jz9nus+KotBOXNbFKGACxmi4gQs=; b=dbRxZTN8bzQ9zXdJ/zB0AGXQypywJgOpkO4Rysde/xLm/Trk1mT04ZS2fzIuqAlRLR 8oZnUx0i3/ovJPopm/+6/6tslJYrQCqoY7ksP7Qay7+TdV5tYylKHnf1VEF99QFFLWKq w4drGZqJDq5Ebp3Ii56UMP8j5eCXscx7++AWML9tAkvoeLsS6CpU/KvIuCEmh6UgVlIL gauZuHPReYOOmuVLsGM4qaGl719NwwIG5kB7R+SUvWZvJ8ZSalfGOGr7vRXaIlUtwyzM +WrN8mVS1ei8YFX7k/FJNCFoV+Gk7wrCGdkOm8RRlisy/gHTncM3mznxFVmER70TOXfW xUeA== X-Gm-Message-State: AOJu0YwSjBduCaCd7SXWFEhLb9aALolGh7dlqg68O8fyBm3fQIVnqUv4 zmz/iEmUXJyGlGAtND8+ucVYDL4wskxPrpUIncUHgLZ8PUBi4co6Cium8A== X-Received: by 2002:a17:90b:214:b0:2c9:7803:1cf6 with SMTP id 98e67ed59e1d1-2c978031ee1mr2425404a91.20.1720041996509; Wed, 03 Jul 2024 14:26:36 -0700 (PDT) Received: from localhost.localdomain ([190.194.167.233]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2c91ce58877sm11370560a91.24.2024.07.03.14.26.34 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Jul 2024 14:26:35 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Wed, 3 Jul 2024 18:26:39 -0300 Message-ID: <20240703212648.48483-4-jamrial@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240703212648.48483-1-jamrial@gmail.com> References: <20240703212648.48483-1-jamrial@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 04/11] avformat/hevc: don't write NALUs with nuh_layer_id > 0 in hvcC boxes X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 9Vh8S+9IfrUs hvcC should only contain nuh_layer_id == 0 NALUs. Support for a box meant to contain higher layer NALUs will be added in a following patch. To achieve this, all sources are parsed and filtered, including hvcC source that until now were propagated untouched. This is reflected in how the affected reference files now have ps_array_completeness set to 0 instead of 1 for some non-PS NALUs. Signed-off-by: James Almer --- libavformat/hevc.c | 140 ++++++++++++++++----- libavformat/hevc.h | 3 +- tests/ref/fate/enhanced-flv-hevc | 4 +- tests/ref/fate/matroska-dovi-write-config8 | 4 +- tests/ref/lavf-fate/hevc.flv | 2 +- 5 files changed, 118 insertions(+), 35 deletions(-) diff --git a/libavformat/hevc.c b/libavformat/hevc.c index d6b9d233d9..651c3b4b1d 100644 --- a/libavformat/hevc.c +++ b/libavformat/hevc.c @@ -40,12 +40,15 @@ enum { NB_ARRAYS }; +#define FLAG_ARRAY_COMPLETENESS (1 << 0) +#define FLAG_IS_NALFF (1 << 1) + typedef struct HVCCNALUnitArray { uint8_t array_completeness; uint8_t NAL_unit_type; uint16_t numNalus; uint16_t *nalUnitLength; - uint8_t **nalUnit; + const uint8_t **nalUnit; } HVCCNALUnitArray; typedef struct HEVCDecoderConfigurationRecord { @@ -654,24 +657,26 @@ static int hvcc_parse_pps(GetBitContext *gb, return 0; } -static void nal_unit_parse_header(GetBitContext *gb, uint8_t *nal_type) +static void nal_unit_parse_header(GetBitContext *gb, uint8_t *nal_type, + uint8_t *nuh_layer_id) { skip_bits1(gb); // forbidden_zero_bit *nal_type = get_bits(gb, 6); + *nuh_layer_id = get_bits(gb, 6); /* - * nuh_layer_id u(6) * nuh_temporal_id_plus1 u(3) */ - skip_bits(gb, 9); + skip_bits(gb, 3); } -static int hvcc_array_add_nal_unit(uint8_t *nal_buf, uint32_t nal_size, - uint8_t nal_type, int ps_array_completeness, +static int hvcc_array_add_nal_unit(const uint8_t *nal_buf, uint32_t nal_size, + uint8_t nal_type, int flags, HVCCNALUnitArray *array) { int ret; + int ps_array_completeness = !!(flags & FLAG_ARRAY_COMPLETENESS); uint16_t numNalus = array->numNalus; ret = av_reallocp_array(&array->nalUnit, numNalus + 1, sizeof(uint8_t*)); @@ -699,14 +704,14 @@ static int hvcc_array_add_nal_unit(uint8_t *nal_buf, uint32_t nal_size, return 0; } -static int hvcc_add_nal_unit(uint8_t *nal_buf, uint32_t nal_size, - int ps_array_completeness, +static int hvcc_add_nal_unit(const uint8_t *nal_buf, uint32_t nal_size, HEVCDecoderConfigurationRecord *hvcc, - unsigned array_idx) + int flags, unsigned array_idx) { int ret = 0; + int is_nalff = !!(flags & FLAG_IS_NALFF); GetBitContext gbc; - uint8_t nal_type; + uint8_t nal_type, nuh_layer_id; uint8_t *rbsp_buf; uint32_t rbsp_size; @@ -720,7 +725,9 @@ static int hvcc_add_nal_unit(uint8_t *nal_buf, uint32_t nal_size, if (ret < 0) goto end; - nal_unit_parse_header(&gbc, &nal_type); + nal_unit_parse_header(&gbc, &nal_type, &nuh_layer_id); + if (nuh_layer_id > 0) + goto end; /* * Note: only 'declarative' SEI messages are allowed in @@ -728,12 +735,17 @@ static int hvcc_add_nal_unit(uint8_t *nal_buf, uint32_t nal_size, * and non-declarative SEI messages discarded? */ ret = hvcc_array_add_nal_unit(nal_buf, nal_size, nal_type, - ps_array_completeness, + flags, &hvcc->arrays[array_idx]); if (ret < 0) goto end; if (hvcc->arrays[array_idx].numNalus == 1) hvcc->numOfArrays++; + + /* Don't parse parameter sets. We already have the needed information*/ + if (is_nalff) + goto end; + if (nal_type == HEVC_NAL_VPS) ret = hvcc_parse_vps(&gbc, hvcc); else if (nal_type == HEVC_NAL_SPS) @@ -1041,20 +1053,100 @@ int ff_hevc_annexb2mp4_buf(const uint8_t *buf_in, uint8_t **buf_out, return 0; } +static int hvcc_parse_nal_unit(const uint8_t *buf, uint32_t len, int type, + HEVCDecoderConfigurationRecord *hvcc, + int flags) +{ + for (unsigned i = 0; i < FF_ARRAY_ELEMS(hvcc->arrays); i++) { + static const uint8_t array_idx_to_type[] = + { HEVC_NAL_VPS, HEVC_NAL_SPS, HEVC_NAL_PPS, + HEVC_NAL_SEI_PREFIX, HEVC_NAL_SEI_SUFFIX }; + + if (type == array_idx_to_type[i]) { + int ret = hvcc_add_nal_unit(buf, len, hvcc, flags, i); + if (ret < 0) + return ret; + break; + } + } + + return 0; +} + int ff_isom_write_hvcc(AVIOContext *pb, const uint8_t *data, int size, int ps_array_completeness) { HEVCDecoderConfigurationRecord hvcc; - uint8_t *buf, *end, *start; + uint8_t *buf, *end, *start = NULL; + int flags = !!ps_array_completeness * FLAG_ARRAY_COMPLETENESS; int ret; if (size < 6) { /* We can't write a valid hvcC from the provided data */ return AVERROR_INVALIDDATA; } else if (*data == 1) { - /* Data is already hvcC-formatted */ - avio_write(pb, data, size); - return 0; + /* Data is already hvcC-formatted. Parse the arrays to skip any NALU + with nuh_layer_id > 0 */ + GetBitContext gbc; + int num_arrays; + + if (size < 23) + return AVERROR_INVALIDDATA; + + ret = init_get_bits8(&gbc, data, size); + if (ret < 0) + return ret; + + hvcc_init(&hvcc); + skip_bits(&gbc, 8); // hvcc.configurationVersion + hvcc.general_profile_space = get_bits(&gbc, 2); + hvcc.general_tier_flag = get_bits1(&gbc); + hvcc.general_profile_idc = get_bits(&gbc, 5); + hvcc.general_profile_compatibility_flags = get_bits_long(&gbc, 32); + hvcc.general_constraint_indicator_flags = get_bits64(&gbc, 48); + hvcc.general_level_idc = get_bits(&gbc, 8); + skip_bits(&gbc, 4); // reserved + hvcc.min_spatial_segmentation_idc = get_bits(&gbc, 12); + skip_bits(&gbc, 6); // reserved + hvcc.parallelismType = get_bits(&gbc, 2); + skip_bits(&gbc, 6); // reserved + hvcc.chromaFormat = get_bits(&gbc, 2); + skip_bits(&gbc, 5); // reserved + hvcc.bitDepthLumaMinus8 = get_bits(&gbc, 3); + skip_bits(&gbc, 5); // reserved + hvcc.bitDepthChromaMinus8 = get_bits(&gbc, 3); + hvcc.avgFrameRate = get_bits(&gbc, 16); + hvcc.constantFrameRate = get_bits(&gbc, 2); + hvcc.numTemporalLayers = get_bits(&gbc, 3); + hvcc.temporalIdNested = get_bits1(&gbc); + hvcc.lengthSizeMinusOne = get_bits(&gbc, 2); + + flags |= FLAG_IS_NALFF; + + num_arrays = get_bits(&gbc, 8); + for (int i = 0; i < num_arrays; i++) { + int type, num_nalus; + + skip_bits(&gbc, 2); + type = get_bits(&gbc, 6); + num_nalus = get_bits(&gbc, 16); + for (int j = 0; j < num_nalus; j++) { + int len = get_bits(&gbc, 16); + + if (len > (get_bits_left(&gbc) / 8)) + goto end; + + ret = hvcc_parse_nal_unit(data + get_bits_count(&gbc) / 8, + len, type, &hvcc, flags); + if (ret < 0) + goto end; + + skip_bits_long(&gbc, len * 8); + } + } + + ret = hvcc_write(pb, &hvcc); + goto end; } else if (!(AV_RB24(data) == 1 || AV_RB32(data) == 1)) { /* Not a valid Annex B start code prefix */ return AVERROR_INVALIDDATA; @@ -1075,19 +1167,9 @@ int ff_isom_write_hvcc(AVIOContext *pb, const uint8_t *data, buf += 4; - for (unsigned i = 0; i < FF_ARRAY_ELEMS(hvcc.arrays); i++) { - static const uint8_t array_idx_to_type[] = - { HEVC_NAL_VPS, HEVC_NAL_SPS, HEVC_NAL_PPS, - HEVC_NAL_SEI_PREFIX, HEVC_NAL_SEI_SUFFIX }; - - if (type == array_idx_to_type[i]) { - ret = hvcc_add_nal_unit(buf, len, ps_array_completeness, - &hvcc, i); - if (ret < 0) - goto end; - break; - } - } + ret = hvcc_parse_nal_unit(buf, len, type, &hvcc, flags); + if (ret < 0) + goto end; buf += len; } diff --git a/libavformat/hevc.h b/libavformat/hevc.h index 0f56325c1c..cb66ac66ac 100644 --- a/libavformat/hevc.h +++ b/libavformat/hevc.h @@ -79,7 +79,8 @@ int ff_hevc_annexb2mp4_buf(const uint8_t *buf_in, uint8_t **buf_out, int *size, int filter_ps, int *ps_count); /** - * Writes HEVC extradata (parameter sets, declarative SEI NAL units) to the + * Writes HEVC extradata (parameter sets and declarative SEI NAL units with + * nuh_layer_id == 0, as a HEVCDecoderConfigurationRecord) to the * provided AVIOContext. * * If the extradata is Annex B format, it gets converted to hvcC format before diff --git a/tests/ref/fate/enhanced-flv-hevc b/tests/ref/fate/enhanced-flv-hevc index f011d38a30..f04905d06b 100644 --- a/tests/ref/fate/enhanced-flv-hevc +++ b/tests/ref/fate/enhanced-flv-hevc @@ -1,6 +1,6 @@ -0da54607064548fa1aae5695751f189c *tests/data/fate/enhanced-flv-hevc.flv +565cf155790db391137f81f619448477 *tests/data/fate/enhanced-flv-hevc.flv 3603038 tests/data/fate/enhanced-flv-hevc.flv -#extradata 0: 551, 0xa18acf66 +#extradata 0: 551, 0xb1ddcd66 #extradata 1: 2, 0x00340022 #tb 0: 1/1000 #media_type 0: video diff --git a/tests/ref/fate/matroska-dovi-write-config8 b/tests/ref/fate/matroska-dovi-write-config8 index 472cbed708..44ca015e0e 100644 --- a/tests/ref/fate/matroska-dovi-write-config8 +++ b/tests/ref/fate/matroska-dovi-write-config8 @@ -1,6 +1,6 @@ -0730145aa317d800cb4bde0e3a38bb8d *tests/data/fate/matroska-dovi-write-config8.matroska +3bd4b07d5af6153516e4c0e66a71c8c9 *tests/data/fate/matroska-dovi-write-config8.matroska 3600607 tests/data/fate/matroska-dovi-write-config8.matroska -#extradata 0: 551, 0xa18acf66 +#extradata 0: 551, 0xb1ddcd66 #extradata 1: 2, 0x00340022 #tb 0: 1/1000 #media_type 0: video diff --git a/tests/ref/lavf-fate/hevc.flv b/tests/ref/lavf-fate/hevc.flv index 1105d8eddb..e3962e0938 100644 --- a/tests/ref/lavf-fate/hevc.flv +++ b/tests/ref/lavf-fate/hevc.flv @@ -1,3 +1,3 @@ -39cf3df5fc3a9c50ab71a294f45663fe *tests/data/lavf-fate/lavf.hevc.flv +c9e8b5df15135d21bd2781558f32f269 *tests/data/lavf-fate/lavf.hevc.flv 11819 tests/data/lavf-fate/lavf.hevc.flv tests/data/lavf-fate/lavf.hevc.flv CRC=0xd29da885