From patchwork Wed Jan 26 21:34:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 33867 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2c4e:0:0:0:0 with SMTP id x14csp1882352iov; Wed, 26 Jan 2022 13:35:41 -0800 (PST) X-Google-Smtp-Source: ABdhPJwtLvfBjwknGPkdiFrQc4KbCFG+/euZrjL3/nF4MqhjofuAkOA5uVGXaIcvEegwN2ZixbPY X-Received: by 2002:a05:6402:519:: with SMTP id m25mr952783edv.130.1643232941035; Wed, 26 Jan 2022 13:35:41 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id ne33si261534ejc.3.2022.01.26.13.35.40; Wed, 26 Jan 2022 13:35:41 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@outlook.com header.s=selector1 header.b=eVKWT8xC; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=outlook.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3A4A268B143; Wed, 26 Jan 2022 23:35:19 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from EUR03-DB5-obe.outbound.protection.outlook.com (mail-oln040092071048.outbound.protection.outlook.com [40.92.71.48]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E63DD68B0D1 for ; Wed, 26 Jan 2022 23:35:16 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=EZaWJMGQZhGKyP6aU8a9PRRcTWoQ0pbiDMEmhx0uEL+DBxI93HX7roAM1usIKp3TaELd112S/PT2cTIuKfoXBeKzq5kNQlBDmjG6u63zk3xtTn3/2l8vTBH7/9w/p+TSJo79grWL3zkmKR8GLyoT0kWxp2etHz/kElE4c5qm6pjsSt321cjMTg0SeVa7tykdN9PjDmP+2nQEHzLdvCxhHoSH4TpWO4ln0PbMnuarU3+lFigO0V8eLnUXUu1VqeSAywKXMZe/oqYIyHJyINwPS+fvPb3DiAWadFgZNx3A6ciyvQ1cBKWPxxHRjcGz20XdX1l2jb5KDS6C3IMr0+73GA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=mUgM33gVqzbZCvp/+6DlhhY54ic4AqFPota2bcPDbPY=; b=XBurR4kH2hSAYviDWkNkrB/Ma0zvQOE5eAQ6xlmuv9wwvklFHfSHA9Mk9suf9gAUPJht9Jtj3ZX2Ew5BTVqgrW+pBaxxCHzRCi4BYCrqcdcxWWXvOYpRMgLxWCiiBG07aIzM7iUwy+1PGx0k7HqFijE5GSTtEVDSitFRPblyidQj0qzmclPTa4MSLlkfAlie4kpPZRFH0EWeW14+JOIDdZQs0E+pq9pZygIVY46qm//Dvpa5ky8mzP1fLRBXaXWKBoj5cwS65hpyUodswewtEITASBMo+8uW8fEhrZiBapXi2yv8y00/33e6w2mbwfhHGNqgCQAIv+4v37iFdFTJaw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mUgM33gVqzbZCvp/+6DlhhY54ic4AqFPota2bcPDbPY=; b=eVKWT8xCEI21rqjKZh7hmio/cSGRpHiOnGUwtwc0xnoDtVNljbUFvCCLPO2u2/jY2BvIcLDLGVFlfOqQKVxY1Fn+0p6dQjz+mEt5uZOLWjvYNMAESLNO/DTGYFVfmAoFQk31Vc1OjxeqFNIsAe9crSFwBMCPF+1C6LqJynRJmv+ljhYgF20NBSyTjJDoaNORIaTKc/ks/nRSXrZTFzpHdoDKDJN4QTKpen7Edz8uAHmZDrLlj49JsGZhSnyGnvnuJNh6TIwhZz8bL5YtlLUeJgyk9k0UQgQKWScP+S0lf0p6C223xkadXKH7t5j1i+nW0+K6aSgy2iA7ddo7u/ZyeQ== Received: from AM7PR03MB6660.eurprd03.prod.outlook.com (2603:10a6:20b:1c1::22) by DBAPR03MB6406.eurprd03.prod.outlook.com (2603:10a6:10:190::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4930.15; Wed, 26 Jan 2022 21:35:10 +0000 Received: from AM7PR03MB6660.eurprd03.prod.outlook.com ([fe80::ac56:2ff4:d304:ab22]) by AM7PR03MB6660.eurprd03.prod.outlook.com ([fe80::ac56:2ff4:d304:ab22%4]) with mapi id 15.20.4930.017; Wed, 26 Jan 2022 21:35:10 +0000 From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Wed, 26 Jan 2022 22:34:43 +0100 Message-ID: X-Mailer: git-send-email 2.32.0 In-Reply-To: References: X-TMN: [TT6QvKbL1zwvhK4jLC3VBDRHw2oK3Mtb] X-ClientProxiedBy: AM3PR03CA0066.eurprd03.prod.outlook.com (2603:10a6:207:5::24) To AM7PR03MB6660.eurprd03.prod.outlook.com (2603:10a6:20b:1c1::22) X-Microsoft-Original-Message-ID: <20220126213451.1887291-4-andreas.rheinhardt@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f0cc1fd0-6dc2-4404-a72a-08d9e113bab4 X-MS-TrafficTypeDiagnostic: DBAPR03MB6406:EE_ X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 3ZkHpOlfc2+LdjJNsA4+YiTogGri5QsYmw9LvOmn6QX8WYJ/bFJ5O4dpT4WERbDJ3yVeAsd0dj4vYmd846DyEGuZmV85Cm9RTs4kmhD+AQyJhot9Dpko3RV47B/8b6B/N+9jI9R4+dWplCSyJAW4Q0m/9XWOvEfsAC6p3YPV4Hugk0NupFUWcBbLF48oK4d2G4EgzsSV7DmXokoC9jnacLSHy/CBkH989Fw6Ag8eCWyI1rcHs67MocBw6c8+P0AZEPaIFgZKnjcVp+lAeQZjJld5ZLFzjWtR0uVwArkYxcwLqTZbohuEPKsopMlJn2wsHoVqPusz7TuBfxMw1wetWq+O1GYXYJBzWH/UNmpgQZbFnULRWUE2U4oQh1hgc83MoSb+eqcZ/dItMT/zdN95DAUtmwA8hO84doAVsByv+Aw7p9Y5kcRT5aIzhU588RK/x7wphiQnBQSUiIMkMYJf9dGvglEP487zsR2kHbiXj/Y3SFL+EK3rblZmRiBErftUt7ZHlkv7bezt/dl+e8H4zChfmPJNbWSovyzBW86vbDDpz7bPvC8180fBG+C6CcS9 X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: m8GW3xKNO7RnaGHEnqmpTpTAVOQI29pwZ3gc0mPrk6sDKhWUJRCn2MmL5rthgZJZRkgSpCLfXLm26W/tItJo/Ob7pLiZ6S25qmsaE1PBOzOskPX3DekERf0RY3n3x7YzC9Qsv0qOeTNvaiO7bxF12OtFnDOxpIzHroPc1e/JHZVOvYXcVlsSpdmqaPT9LjxsIBGLu/WZAxHCanKfXAqQwTpRRX05ZBMUqeXRJHZG3/SznAleQwscZf7bkJGTPJrPcTFYiKhfEJJ0ndCgDLk6MZKVpyis225Ni2f6g7Bb6+vgFEoP3isjPptwzvT5Dd8LwKG7+LbGkXjSve2XQJHBulucsJC0kdDVgHKoosIaoVXj/4iKM4wAwN8tEw6wLoJBHushK0RkETtM6rtqyF3AQG0L8oHPYvWVv4Rp9mUAwUYVAXR5Exw6ahbBeJclZmxDWuixYGsB6BELXXQ73ZgnSMSj3viwJ5rsmuEXJW4lc9CUhvWjWY5yMr5II26X74Z8Ax2Ot0YUOY9SdL+0C23u8n7RxNFKshGSnsVby1lPxtjpYICrE6iexCWaaKp3p9DlRa/BDtraUPEdQvS2pDCcl9jDvqvo7Y7g1v7Ac1EKoa5Vy9NxSWChW42F+eRoK6ybeTWBn7s5tWB+WkbKNNe9+l5laXPNzHylLPyM0uZbNGDsk2tCRn6AkIxYxWEjNaD5AZ8YG+gYj8feJmo3iqR4Fks2LY6GTQ2mCK3Jy8Ms2ILWTsFnfuGT0wkJzW8Zc5oARFskYqOYy2bEwf6ncDIkrgfG+Atw10+QYwAA8FBGiSW61W0ED25rXvfKRcJW3wYzMEsleDV4egCrWgM8yxIYukF1Em3EmW8kNCgwERh388OYF7NDCdN5zgJHK9Or8fUDpsxdDljQ4QF+3b1xhT/Ebxw8DUWHDM4oBCWK/LR+FtNVg0hsUq1+6ZEkjJ/LcmnzvWKPRvgg0AWC8b/ndGSn6g== X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: f0cc1fd0-6dc2-4404-a72a-08d9e113bab4 X-MS-Exchange-CrossTenant-AuthSource: AM7PR03MB6660.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Jan 2022 21:35:10.3602 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBAPR03MB6406 Subject: [FFmpeg-devel] [PATCH 25/33] avcodec/mpegvideo: Fix crash when using lowres with 10bit MPEG-4 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: M36Yxe/qpyzO In this case the macroblocks written to are smaller, yet the MPEG-4 Simple Studio Profile code for 10bit DPCM ignored this; e.g. in case of lowres = 2 or = 3, the sample mpeg4_sstp_dpcm.m4v from the FATE-suite reads beyond the end of the buffer. This commit fixes this by taking lowres into account. The DPCM macroblocks of the aforementioned sample look as good as can be expected after this patch; yet the non-DPCM coded macroblocks are simply corrupt. Signed-off-by: Andreas Rheinhardt --- libavcodec/mpegvideo.c | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/libavcodec/mpegvideo.c b/libavcodec/mpegvideo.c index 47603c2991..40494fe115 100644 --- a/libavcodec/mpegvideo.c +++ b/libavcodec/mpegvideo.c @@ -1629,13 +1629,17 @@ void mpv_reconstruct_mb_internal(MpegEncContext *s, int16_t block[12][64], uint16_t *dest_pcm[3] = {(uint16_t*)dest_y, (uint16_t*)dest_cb, (uint16_t*)dest_cr}; int linesize[3] = {dct_linesize, uvlinesize, uvlinesize}; for(i = 0; i < 3; i++) { + const int16_t *src = (*s->dpcm_macroblock)[i]; int idx = 0; int vsub = i ? s->chroma_y_shift : 0; int hsub = i ? s->chroma_x_shift : 0; - for(h = 0; h < (16 >> vsub); h++){ - for(w = 0; w < (16 >> hsub); w++) - dest_pcm[i][w] = (*s->dpcm_macroblock)[i][idx++]; + int lowres = lowres_flag ? s->avctx->lowres : 0; + int step = 1 << lowres; + for (h = 0; h < (16 >> (vsub + lowres)); h++){ + for (w = 0, idx = 0; w < (16 >> (hsub + lowres)); w++, idx += step) + dest_pcm[i][w] = src[idx]; dest_pcm[i] += linesize[i] / 2; + src += (16 >> hsub) * step; } } } else { @@ -1644,13 +1648,17 @@ void mpv_reconstruct_mb_internal(MpegEncContext *s, int16_t block[12][64], int linesize[3] = {dct_linesize, uvlinesize, uvlinesize}; av_assert2(s->dpcm_direction == -1); for(i = 0; i < 3; i++) { + const int16_t *src = (*s->dpcm_macroblock)[i]; int idx = 0; int vsub = i ? s->chroma_y_shift : 0; int hsub = i ? s->chroma_x_shift : 0; + int lowres = lowres_flag ? s->avctx->lowres : 0; + int step = 1 << lowres; dest_pcm[i] += (linesize[i] / 2) * ((16 >> vsub) - 1); - for (h = (16 >> vsub) - 1; h >= 0; h--) { - for (w = (16 >> hsub) - 1; w >= 0; w--) - dest_pcm[i][w] = (*s->dpcm_macroblock)[i][idx++]; + for (h = (16 >> (vsub + lowres)) - 1; h >= 0; h--){ + for (w = (16 >> (hsub + lowres)) - 1, idx = 0; w >= 0; w--, idx += step) + dest_pcm[i][w] = src[idx]; + src += step * (16 >> hsub); dest_pcm[i] -= linesize[i] / 2; } }