From patchwork Mon Jul 1 12:16:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 50246 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:cc64:0:b0:482:c625:d099 with SMTP id k4csp1739133vqv; Mon, 1 Jul 2024 05:17:06 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCW0o9qHo0RmRmI3aAXlyj1NP6IZMgckCTpUwXZaQJ6U7bB0n7GJOLpEcacjIplgKuLcBx7P80plmpYdyUh1mNe8ZrHJ3iGabhklLw== X-Google-Smtp-Source: AGHT+IEJSB0F4pbCmAMx2CVLexUFOyEDUyMODTCbBvfMPVgncykLVbeoaLE2YcpDTUyRGSbEfnXf X-Received: by 2002:a05:6512:32ba:b0:52d:215c:4111 with SMTP id 2adb3069b0e04-52e826679b8mr3865317e87.24.1719836225811; Mon, 01 Jul 2024 05:17:05 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a72aaef5c93si396080266b.192.2024.07.01.05.17.05; Mon, 01 Jul 2024 05:17:05 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@outlook.com header.s=selector1 header.b=Nqq+iS1F; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=outlook.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 185E368D79C; Mon, 1 Jul 2024 15:16:56 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from EUR02-DB5-obe.outbound.protection.outlook.com (mail-db5eur02olkn2026.outbound.protection.outlook.com [40.92.50.26]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 67A9C68D794 for ; Mon, 1 Jul 2024 15:16:54 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=n7pjp6zNNvBcCI2UIB/NgYP0QOxFlafznJySxoiInYuG9/YEaB20loauAInB5VQJFTHhnIYC7s9mgmUYSV9fjSCo4L+gJH8CJ5Pl6z6I0XVhLXcDckZ7Nh7CMBI8BJEqlU1D2pBbAvQmi+XmO0ZOcAv0awmIHjMeZOslc6v5+DCN+b1OwU5TzUl83RtYVtfyG0S6gM+BimBx/jGvaTuuCVWTBeeDW6UBWGt4P2aXdmb6ojYb5740wPu+X5bVUuNI8EqWNIxdi4GgaSEc+VOkpu1T6oHDlPPNR+SJMokUZgcj/avBSPfAdKgvqe4R1QY1mRvXVolBWbe2h/MDwsbzyw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=OpIxh/7fozJm2/q265LTe1wdtLK9AM1wBnNL5yGFii8=; b=nZOSaIQbxMlticNDnDnMoaZ7x0PUB6Q2aTaqtpKhHm0XjFE5EtjaB72QpOmGFe9ZVSDldogEYKjTqYlWvnAJtAnE5PSNZ2fh2uhZJSdPLDG/nE6VctowtTu3vkPCMBg76O3PICE8fkFt2pVO6KgmyzyVl3s3MjuF5JPWb9LXGonII/WPhJ3ITuIgafjLNBmE37MtNa6OkXIt4RJPcU5q85ar+0Ww9f6HN7sBmsWQma8/rVeiHI4566oyg2gMEPMG0+pjqETDTJ2XXIgdRqfYvKRi9WxlR1IJhI43etbxPzNt9qZSB6GA2++PaBN0JHd6/4oRSQ2zwKwuz/h6bbK9Gw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OpIxh/7fozJm2/q265LTe1wdtLK9AM1wBnNL5yGFii8=; b=Nqq+iS1FbmLQG+ZgU4eGI9pJ8hRD/d15JoM3C6VaSlmk4I62dW19kyEqcIf7GwvxbtLX64o9vMz159VrvpXttlOts922qMrawzLIrWMijLtfpz4h11beTvSnDDhhIrFDTmcGr2O5JotlFmvs7SURrcwUvytpuqgOSecJmzfwafsg4fKr+ckd1OYiJR/HxermN+Y2XPea0FF4MX260L2Dxs/eNJTGQNpHLtQ3i2ia5f2kvSkkTaEo0eDg6eukcSu83V5Ws9CQ+Egebn2pPuAlLN5WVX0DxxXgiAQajRqC/lUc5ujQd5uADCaw5V2LwMj3STy8BdxEtr6RHaJOcKCr9A== Received: from GV1P250MB0737.EURP250.PROD.OUTLOOK.COM (2603:10a6:150:8e::17) by DU0P250MB0481.EURP250.PROD.OUTLOOK.COM (2603:10a6:10:349::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7719.32; Mon, 1 Jul 2024 12:16:52 +0000 Received: from GV1P250MB0737.EURP250.PROD.OUTLOOK.COM ([fe80::d6a1:e3af:a5f1:b614]) by GV1P250MB0737.EURP250.PROD.OUTLOOK.COM ([fe80::d6a1:e3af:a5f1:b614%5]) with mapi id 15.20.7719.029; Mon, 1 Jul 2024 12:16:52 +0000 From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Mon, 1 Jul 2024 14:16:01 +0200 Message-ID: X-Mailer: git-send-email 2.40.1 In-Reply-To: References: X-TMN: [cnQg98OJGaQdpqxPpSrYUYG7zFDk6R9AuDrxTKHdWQM=] X-ClientProxiedBy: ZR0P278CA0149.CHEP278.PROD.OUTLOOK.COM (2603:10a6:910:41::11) To GV1P250MB0737.EURP250.PROD.OUTLOOK.COM (2603:10a6:150:8e::17) X-Microsoft-Original-Message-ID: <20240701121610.3560848-3-andreas.rheinhardt@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: GV1P250MB0737:EE_|DU0P250MB0481:EE_ X-MS-Office365-Filtering-Correlation-Id: e4db7f32-74a7-4ecb-742e-08dc99c7b090 X-Microsoft-Antispam: BCL:0; ARA:14566002|8060799006|461199028|440099028|3412199025|1710799026; X-Microsoft-Antispam-Message-Info: DfSwelBAo1EIpyg7va1vOBSwk+ZKzJKGYmaQ1xDdDyn08dXAIb72Uogk+hPP1yIo7hbn3vjCZj8RLB0nQr63rNbwCCEd/uDSPBQK4TpuFgUraaHmEGu8AQTt2yoH0D4wgayTJ0QUtYxXT/raX8WWmThEmpH1UphceQgqc7gxvYodnS7cbBPj+iwWsOCLF+oz9Wb0rga96zjfZO5BooFWDRTP1GH0N+SpyHXMqnYpmCSOZ9YP3edtZ0tAAR8fPwid1YXFsUQC5Z3ckyvHj9jJ3k9LOiL1zczbNQ8TZ3HZU3xaCSQqPxGX5h567l2ePRlxnYORoHHhCGwIw+g3TXRdwkHaWrQW1TVpzuqYehTMXLBBFYGOGNbTQQc7JMOfSfDrSBRuuRebm4ruGTENvoAEXRhQh1hMBtWJ11uu1GjCaw3tWO1LHDRem1RA9s57EgB5SVgIXw3BzV0CNcC8Wldy/NLwKNsHEhyFBtQ4glszrmNpZCYk3BIozeWArNTap1ehqcU2Uu/UulYmSaN/zEsbI/gxoizzuaAalVQbBL56/xqnHz6UH4KGGzp1gRSETwXS1hzqQgF+uYQuywEecOcIyiKmEgw4hAtH1q2OEhHoYEO1g1BL2QYu0f7iVKUGF7PpQyFB/Fz7gOeMQ2/p2JKiFw== X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Tbo+5StZOA7HfFbW6S9pp5vMBOMI7OGSZYzOF+wOHjyaCFhD57JgA5XYP4Tp02n+LFN4sggz3WJt2nY3MO9MU9Mc8QLe9BUa7bpDhWk4S7pqvSoZzWd751PfaBBHrfdlvuFxrDZqIZUGDMITaETtRc3oUXttzsNY8oEExHQhjAaf+61XfcSKV2viDdX4swq1+N0XUG3pD8m99VmoLBJzYd52pa7vSOmQZbFOkcmEdT1W6Klg2U5R8pypFXTyMZIINZqb7SslpIQBaGgJLhdM9jbsEYg9jaaDVWYTmi7IcnHwIMAyAGMhGwIuIS8wICBB7/nPyrYD5CsmscK9EbC9NkkRKyQaXrUNI5ewRNviZh1OJbqivnZhU9xQU3rJDN+zIQ3vyHsSdf5G7DexVcOJUh+Ol9PaqwmdjbBuWKLlbvYZPLegtXRy8mAKDncZoX0l3VH5hgolk1QS2KW4PLJq3UbcyxDNki9hWCwX0pryrp3XiX/PmlM4q1Ng9SuD8dEzEo61+zAmXPFV4GfbFV/FXM//BEwFRR2dD5FuQY+Bl1GVyHaLlbuzAvudZkm4bcqUB0FUrpSu55RwzSkia8ENvQWxqucHI2DBxt2BFMwXtH71SsVoxuX2hGhOJ/64k2azGRsOSz6qMaomsIFA68lnAbVh2aG3S9ssw8CEbm5J0czlST2LQ9gXTvUnIp068qpzbvDuf3ttD7sJmCXKqtTjBL+Qf6XnR0MoUPj2mgZSvzwrd7HnfkU5JPiyEYZwVO0WLP+mBQxI/44Kk3hc+zpBZ7XH8kzUKioKVCB2QeGz3Rorv8gXMJwj0c9s2MwSt4fUD2fDks39IPZoKkBt9+9s+Fldjb9YPwaiEEh2EkRhTCwdAPrM9SoyR8DznKGkGcre4LhzWeYZyLIGdv0ZvcTlIBLXFEPXJnWYWYuaef16MWJtuFWWJMvgW8JCFld6TZMAgPwY04XQ1W10kZChlkR9mCHJ7n4rrg2VKuBUr5RFkRgvsKY8Zjov9eC72vsFoSKLvmjrTddXK9JA6L2qYMh/Ft9wEl3qH+DSxexDDfKk7vO4CjkyyJs+U1MmcZ7d/Vo+Pk5AylsWKl9eB5YB4ssJas6xJ55Xs2EzH/PmscFbLe0omCawQzFzkYTF+jlMsZBJ94v5f6psJT6KX2H4lew3XrKsetuw/nlFXWkXuMuj6m4oLvhYulB9G0BdxYGdX7dyGSBhE3OLywT576+2yDsrBQTbALLfu/U3YJPknJtqj5+82nQjWfra8+uZpb1KAphOD3AZPxCR9riA62/rR0bjog== X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: e4db7f32-74a7-4ecb-742e-08dc99c7b090 X-MS-Exchange-CrossTenant-AuthSource: GV1P250MB0737.EURP250.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Jul 2024 12:16:52.3349 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0P250MB0481 Subject: [FFmpeg-devel] [PATCH 04/13] avcodec/h263dec: Clean intra tables in decoder, not ff_mpv_reconstruct_mb X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: PKlL7jP7Sfsn This is a more appropriate place than a function designed to reconstruct a macroblock. It furthermore limits these checks to the codecs that actually need it (and removes it from e.g. RV10 and RV20 -- the latter actually uses these buffers, but only for intra-frames, so they don't need to be cleaned manually). This furthermore means that ff_mpv_reconstruct_mb() and therefore also the error-resilience code no longer needs block_index set. This fixes a crash caused by 65d5ccb808ec93de46a2458ea8cc082ce4460f34 when ff_mpv_reconstruct_mb() is called by VC-1 code without block_index being initialized properly (VC-1 uses and initializes block_index itself normally). Fixes: 69814/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VC1_fuzzer-4868081575329792 Fixes: heap-buffer-overflow Signed-off-by: Andreas Rheinhardt --- libavcodec/h263dec.c | 9 +++++++++ libavcodec/mpv_reconstruct_mb_template.c | 2 +- 2 files changed, 10 insertions(+), 1 deletion(-) diff --git a/libavcodec/h263dec.c b/libavcodec/h263dec.c index 0c23012584..3e9da23d3a 100644 --- a/libavcodec/h263dec.c +++ b/libavcodec/h263dec.c @@ -271,6 +271,15 @@ static int decode_slice(MpegEncContext *s) ff_tlog(NULL, "Decoding MB at %dx%d\n", s->mb_x, s->mb_y); ret = s->decode_mb(s, s->block); + if (s->h263_pred || s->h263_aic) { + int mb_xy = s->mb_y * s->mb_stride + s->mb_x; + if (!s->mb_intra) { + if (s->mbintra_table[mb_xy]) + ff_clean_intra_table_entries(s); + } else + s->mbintra_table[mb_xy] = 1; + } + if (s->pict_type != AV_PICTURE_TYPE_B) ff_h263_update_motion_val(s); diff --git a/libavcodec/mpv_reconstruct_mb_template.c b/libavcodec/mpv_reconstruct_mb_template.c index f1cb0d7989..981c837642 100644 --- a/libavcodec/mpv_reconstruct_mb_template.c +++ b/libavcodec/mpv_reconstruct_mb_template.c @@ -61,6 +61,7 @@ void mpv_reconstruct_mb_internal(MpegEncContext *s, int16_t block[12][64], s->cur_pic.qscale_table[mb_xy] = s->qscale; +#if IS_ENCODER /* update DC predictors for P macroblocks */ if (!s->mb_intra) { if (is_mpeg12 != DEFINITELY_MPEG12_H261 && (s->h263_pred || s->h263_aic)) { @@ -70,7 +71,6 @@ void mpv_reconstruct_mb_internal(MpegEncContext *s, int16_t block[12][64], } else if (is_mpeg12 != DEFINITELY_MPEG12_H261 && (s->h263_pred || s->h263_aic)) s->mbintra_table[mb_xy] = 1; -#if IS_ENCODER if ((s->avctx->flags & AV_CODEC_FLAG_PSNR) || s->frame_skip_threshold || s->frame_skip_factor || !((s->intra_only || s->pict_type == AV_PICTURE_TYPE_B) && s->avctx->mb_decision != FF_MB_DECISION_RD)) // FIXME precalc