From patchwork Wed Jun 22 10:53:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 36380 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:1a22:b0:84:42e0:ad30 with SMTP id cj34csp2759448pzb; Wed, 22 Jun 2022 03:53:41 -0700 (PDT) X-Google-Smtp-Source: AGRyM1tiwtEEQOEFgcBu0SdzSTtxzLbD02hxm5jmuZpTg+oGR2z53UEZJ0YzyW5esPIUdu5Y4fqB X-Received: by 2002:aa7:c7c4:0:b0:431:75d6:6b3 with SMTP id o4-20020aa7c7c4000000b0043175d606b3mr3385941eds.280.1655895220843; Wed, 22 Jun 2022 03:53:40 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id gs34-20020a1709072d2200b0071572d1bc73si24041316ejc.785.2022.06.22.03.53.39; Wed, 22 Jun 2022 03:53:40 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@outlook.com header.s=selector1 header.b=iaBMy8HK; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=outlook.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 821A168B00F; Wed, 22 Jun 2022 13:53:36 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-oln040092064081.outbound.protection.outlook.com [40.92.64.81]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 72B2968B308 for ; Wed, 22 Jun 2022 13:53:29 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eRSa20Np8DFcn5J2+vVRnOgUgGKtC0gUd1GPBReRMPjNj8L5Gdnj/Yfs6RKlsSFbCrDGLMUhup0wt6jAan3BV9Nfr/vJGEqjeMPLwEqt6KeR8YjaFxaHRp7SbDoP/CFPpWSA1rfdXpFxFvaNZ300jzftL28FKQPS2ckhD/mZyDYXcRwa6jsyp/dChbbxPLdZIS8xv1YIJZbThVfcOJEbC+R14fo+c5O54L2PVA1DD+iuJYWeWK9/w2r1FDF42LDoggXHb6PEdLQv9BCnmY/wGkSrVMGhSnIjLgRAYP1HKB72N/Wtgsx7LR0f2TMGUAYmS8eWLaCJJUadlbRJfio83w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=HC72nIqywu9npKPG/wEAzPqjELGmcogjq4ugvbuADb4=; b=l3OQzC4GGTR19PQuAkgf/ThCevNJ/m/TimGHzH7s2M8F2sQc8JYRDUMwAm1/Ghwa42DLmyMQ3A+tFz/pvwQZ/jio07ZltCkx87C17Z+06U5Taku6j0v8YfI6TPb/bMHtHj1LFksT0djGV53gwdb9LL6Qc+VfvosQycwu6mAWGeDTH/yB5MkUR5uEEK0Up+hg/ZvV6rLMxsNRb7OEK/H6KKzRTvjAHK1JhN3QYrhyNeKRXKPQ5a/bHf21cLGMGNiuPGKfzB77WqcAcMLAdOputfvUGc5os4og+BBv/+kXDSFknKztopYVJum+YbavLwhXOYfiUVNsKJFZDBFs6BP4sQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HC72nIqywu9npKPG/wEAzPqjELGmcogjq4ugvbuADb4=; b=iaBMy8HKvmR8VeCz1KA5PJarcgyP8GmmChHFU3uwcrsHFiULdZTNqxgqr+w+fYH8IsqshmloTN+v8SkMEZO1bbEkOpeI6H2Vf7xKm7oj40mEhlkVHKl7NwIUwUNKhG36LJTNWKADUoPzeJbbW6M6X5+qiNa6iRWfVZYXtvUcqoZlZ+9ysX+va9M53t8e/6aLuKqNpOrf8kPj6r2qak59iltxPwemtzB1f2UEBXjd0jeY9akRHEa4oq25O4tWxpKO6x0DTLvl8m88J3rjT2/Kndz4q+pcxvxhPOxjDGD2aCPxVt//HmiLrwGO/nLAW30v8q6GdHOdNaT5yQ95uKX8Ig== Received: from DB6PR0101MB2214.eurprd01.prod.exchangelabs.com (2603:10a6:4:42::27) by VI1PR01MB4912.eurprd01.prod.exchangelabs.com (2603:10a6:803:8f::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5353.18; Wed, 22 Jun 2022 10:53:26 +0000 Received: from DB6PR0101MB2214.eurprd01.prod.exchangelabs.com ([fe80::60b9:9f29:40cc:f01c]) by DB6PR0101MB2214.eurprd01.prod.exchangelabs.com ([fe80::60b9:9f29:40cc:f01c%10]) with mapi id 15.20.5353.022; Wed, 22 Jun 2022 10:53:26 +0000 From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Wed, 22 Jun 2022 12:53:18 +0200 Message-ID: X-Mailer: git-send-email 2.34.1 X-TMN: [/MFijk3bUWvovz7BFTHJvWRnGkfT8t9X] X-ClientProxiedBy: AS8PR07CA0042.eurprd07.prod.outlook.com (2603:10a6:20b:459::29) To DB6PR0101MB2214.eurprd01.prod.exchangelabs.com (2603:10a6:4:42::27) X-Microsoft-Original-Message-ID: <20220622105318.1256144-1-andreas.rheinhardt@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f32150a0-9667-4a0c-2c85-08da543d6f52 X-MS-Exchange-SLBlob-MailProps: S/btQ8cKWiRAkauAoWA5v5jYZEoFBVk6dVWLhutQVys7WnuH5iXrTyEOH9hrP9qT1Azs7CX0I6W0zjs4ShmnFaY4S8u0ux4G5hOKT1LuD1N4v8nBk95twCqf6UidfhcoF4wV56du0Ii8zDwK+tYISIr9sgdM+kMc+GS8D/uSrppdJ6UlaPp2yuiG96noJGKpG4zzR6hv07LIYijd748ePOrkFCESt7XFlF2+JZWaJuvduOb0SiWOX630w/FWJ7ju01wEt/X8qhcPNh1XnrFj04wCOvpAH+hQPYhAxjd7OiSoejJQmPv57eKebOlJlIRIL+y0Y2pw6krGuRNclM6unMe25Tafint+izPWMtSx/MQ3Ue2IDY33WIUQ9Bqo/r0LY25xBLtmKKDTgV6J4bns6k6oW+iI19+I8YFcp3skRaseMVcpcGle7oeZW0yIko+vpPCwMf86KDNV3ZNFHVmcVcbe72b76IF3nOhoMG3wULH13ztasXs5Y+Jag68exO7q5UePiJjVYmAOpKxLQHIy9mvpRkbe5ZMsR+L1hzErExV1iA/EhLyE5+0mKa/WSmfWgaV7H7sythUyP6KQMndK0sBdBRv7fgx4ia9hVJnwa9zQMqLK0+Wti3QWmu3AR8dPurXPBBwsh1OuGeaDHKbx9Q+BqHrt2omMtcOVFckzX0yuDLVe0leF7h8JKz1+tfVEmGiALKKioOkltuCJK9uUhNthDH/FtuI3+saQuP/zK3UXM46R65UbYV1MXxbyrS/xuC95J+G91FY= X-MS-TrafficTypeDiagnostic: VI1PR01MB4912:EE_ X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: qXezdgHz8OgOLizQ6gH6LmB0XGrSkZW+S57vdkFKOgLau03TKVUG6lsrcuM+LFo3eFs34ubJJOtQqW1Fw4A6tgzNr0VqoRgBWq9hZ33mZ2hHLslFWdU2Mc+/jjQ8O9Sa0DWj7sSdYa6TLncUqCtXXvWlnh1XpZ9NraQ0pY8jBAJSYa+zvE2RhL5uoA51PCbRJzwW8ci+wzdqU+7XmOpEA3eLF1OLZvZ3czHKfE/pISMWcjcKm4PYIT71qMDVF1ROn1oRYnOA9ZNcfYTALbaFsOwYppkCEFTvT2jV4rxP/mCu0UoZ9oiuwzJ4i6QdHyQPnng5yJ95lv+ZInMEAMv1JmewnxhWWPcqWWEpfsRXdsmd3hU/O7NPi4mQGt84hJ+HmNjLtibMT5r07KkiW6J78nwprOXi/RJBnU1hbaGQuu+CaaHCnjXd6DnRAQWBpDV16ThVEOC/qtbtBbNfIn/LQP4lSNzwzfxnszdA9fl87SyR9grkAMbSXqc3Arp5o6ShVGx8Qu6prp15WhX303MZLqksHsGfjZxOaLoMmRNNU45jW5j1/kNgPoQm/b2cVw8abfLzTW+ds2oJrCFdqprzrVIlEPxh90j0YO5diXgh9DAU9Lo54QphSmCsaT9tL5w3A8wdhsgT1MmlH4IUVLQreQ== X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: szAuTvtnAYwHD+oFrmIOIkjF5I3X/Yw6nqTWNXQWVxQgaPbi0u51A/ZKbCkj+ByuG6Xh3tRFp7DTCMrp0YAA0ijrIr+OWTlB5RIF8iG9VtisfJkcA5+DPFaPFb5veWrrT0u28mSPdd/3mkcQc5CGy/tHYZ9/vpoufso17MgeGAtUfpRw+TylA1dclZaoQ1LpQTCdriS3jyQqXVTLZUpVcSqzpyXfT7MUjbmDGkq8Ow+bXNqhgl/j6V406faTjjp5DcdyeIzfrsYxGtfVa0pQ61/ohHPOUWS7Wab8D8ePN9SEyS0B7bh5Lzyd+nz8odM2K7mPT4IW8FyWQEgHlLILSICMNi4tXDoTiHHAU2w/5q9It2ugjuuLbKsnzM0lA/NoPuE2Ei0V4XAMzimO16I8OC+nCcWsQAuxv94HHfqfR8C3VkbhrJhCgVPwPA6lYX5A0/PDwYZgr14nwN7iA26bSs6L0pyvSRqNMa/nHEgAAylnP+ndkV8J6qL6/4fxtRNPe52sW89s+dloslXSMr7HyBPRSrDDCTF2KX+hqCo53XXpcB/1MbQq3GsGldrb1AWHTWEhZP0JEawMuQVT0Lk7mtMdM77OADjCHO4K4hkNYVhtcez7gsaxmrZo6yg/EMwoFY1EavfagtFmgkztTwswzhi2mlI/DpucFjTboHaUhOGYJA1CLx7A8OKp5CWQ8VU4dJLpbnLtPwEogaQj/kDNaQQ5zhQiXNeqAEuhePtkDaseWhuLd62SAX5lqfie6PJiMj8erotreaEZuetYbumFdzHesIHQEXYw6IrDkorY30/7zXQiybrethA8aveCxa8c9SSHNEyxoYVhJrnSznQ8RvEzpzOy1Hq+xWgj+mS+drHQM11S3wQ+2YYHmj2J1jWVdcsq2cnr7UKfsfDAXznfSEz6eDP78/+oFoYjoerS4YhEhTisp1kX57EniITLS5+mH3he7lMoazgs50yFYXjTliUEqgCfEsM9URDyyUnEIrDDeE9ggw+ob3FWzOYCW7DhkeoyY8WHVjht/idYKSz0HvHZdoH5ZjXTcwuSvXrYkjO2/JlZVyxV3EzRVSNGkBL5/cMQHGXWPBi0g14UdKzcCnyXE/6EqnCXSxPwgp4PzY3gAvZN4WpvZ0tr7wcUw9Yr65bcSQZvZdi7lq8jYgThiQux6NHfWzhfK+8Hrv5xO3zwIzalIuA1lkSt96aSCgfzullQRW+obmtAe8M/clUj6wMxoAdjPZJBtfSnMudUqdV5QeruOxjJlElGULiVnx1nY+VC/xtl+X4RO9lYBK3cHP6AhVczlDRcuN6OVDiS2HL1HHC49XvfieQemXL1H6RgkmjHVk8s9b+LxiWMG/1s/NZYBp5rLOFnbQjk58v1n4G5VU3n9jTKDhYhpjk/vPRoh1jZFcnZe9srtCHVutYUfMxy1vXYJcA81YmMRsTbah5L6ugolAo6aGxDUPc84bElgXw5TGtGKAP4p5Wqj5iD7A== X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: f32150a0-9667-4a0c-2c85-08da543d6f52 X-MS-Exchange-CrossTenant-AuthSource: DB6PR0101MB2214.eurprd01.prod.exchangelabs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Jun 2022 10:53:26.7334 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR01MB4912 Subject: [FFmpeg-devel] [PATCH] avcodec/h2645_parse: Only trim RBSP trailing padding if it exists X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: c4sLSqQDXiWe It does not exist for NALUs for which the SODB is empty; it also does not exist for NALUs for which not even the complete header is present. The former category contains end of sequence and end of bitstream units. The latter category consists of one-byte HEVC units (the ordinary H.264 header is only one byte long). This commit therefore stops stripping RBSP trailing padding from the former type of unit and discards the latter type of unit altogether. This also fixes an assertion failure: Before this commit, a one-byte HEVC NALU from an ISOBMFF packet could pass all the checks in hevc_parse_nal_header() (because the first byte of the size field of the next unit is mistaken as containing the temporal_id); yet because the trailing padding bits were stripped, its actually had a size of less than eight bits; because h2645_parse.c uses the checked bitstream reader, the get_bits_count() of the GetBitContext is not 16 in this case; it is not even a multiple of eight and this can trigger an assert in ff_hevc_decode_nal_sei(). Fixes: Assertion failure Fixes: 46662/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HEVC_fuzzer-4947860854013952 Signed-off-by: Andreas Rheinhardt --- libavcodec/h2645_parse.c | 26 ++++++++++++++++---------- 1 file changed, 16 insertions(+), 10 deletions(-) diff --git a/libavcodec/h2645_parse.c b/libavcodec/h2645_parse.c index 03780680c6..dca91b24f3 100644 --- a/libavcodec/h2645_parse.c +++ b/libavcodec/h2645_parse.c @@ -259,10 +259,10 @@ static const char *h264_nal_unit_name(int nal_type) return h264_nal_type_name[nal_type]; } -static int get_bit_length(H2645NAL *nal, int skip_trailing_zeros) +static int get_bit_length(H2645NAL *nal, int min_size, int skip_trailing_zeros) { int size = nal->size; - int v; + int trailing_padding = 0; while (skip_trailing_zeros && size > 0 && nal->data[size - 1] == 0) size--; @@ -270,18 +270,23 @@ static int get_bit_length(H2645NAL *nal, int skip_trailing_zeros) if (!size) return 0; - v = nal->data[size - 1]; + if (size <= min_size) { + if (nal->size < min_size) + return AVERROR_INVALIDDATA; + size = min_size; + } else { + int v = nal->data[size - 1]; + /* remove the stop bit and following trailing zeros, + * or nothing for damaged bitstreams */ + if (v) + trailing_padding = ff_ctz(v) + 1; + } if (size > INT_MAX / 8) return AVERROR(ERANGE); size *= 8; - /* remove the stop bit and following trailing zeros, - * or nothing for damaged bitstreams */ - if (v) - size -= ff_ctz(v) + 1; - - return size; + return size - trailing_padding; } /** @@ -491,7 +496,8 @@ int ff_h2645_packet_split(H2645Packet *pkt, const uint8_t *buf, int length, bytestream2_peek_be32(&bc) == 0x000001E0) skip_trailing_zeros = 0; - nal->size_bits = get_bit_length(nal, skip_trailing_zeros); + nal->size_bits = get_bit_length(nal, 1 + (codec_id == AV_CODEC_ID_HEVC), + skip_trailing_zeros); if (nal->size <= 0 || nal->size_bits <= 0) continue;