From patchwork Mon Nov 21 01:03:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 39367 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:999a:b0:a4:2148:650a with SMTP id ve26csp1831187pzb; Sun, 20 Nov 2022 17:03:32 -0800 (PST) X-Google-Smtp-Source: AA0mqf5aVf9vYMdBEUs3U9YyHahbw8w4ta5H7bZpUiOo5E8vbH4omige5dLS2M30jDXclJ+Sf+LK X-Received: by 2002:aa7:c841:0:b0:45d:2a5:2db8 with SMTP id g1-20020aa7c841000000b0045d02a52db8mr1690780edt.105.1668992612059; Sun, 20 Nov 2022 17:03:32 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id e12-20020a17090658cc00b007ae6869f9d9si9873359ejs.419.2022.11.20.17.03.31; Sun, 20 Nov 2022 17:03:32 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@outlook.com header.s=selector1 header.b="MQs43/Y3"; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=outlook.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D336F68BCAD; Mon, 21 Nov 2022 03:03:27 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-oln040092064073.outbound.protection.outlook.com [40.92.64.73]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 13F0568BC1A for ; Mon, 21 Nov 2022 03:03:20 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=OQYcl0fSQeHAlB9Ns4YgUqFOc+L8Gg9ZjVcb2m8xEnHAr0gY7bXbgEk3ZeMUs2VYgo2B/q4mBZT6ESuoVTgvuG1WyFpRdf2k7ZmtgCPKFig5pQe5QvGmnvqVjvCOVPtCwyd8HLgkj3ACkFj+v4hH8/+L4Y+fdmM8zO5TH25Tjd1f+1Jf0iUAZg3fbEpyaOrDiy76lG190cD9LQLuMsDGLCqlwW/fewx23jfaeabuqvGnm3b3m10kgMgRv7Cltt0uJSHP3WjKjenCd892bbQTArH3N7vZhK8VyeLyTxp25WXk3RM7NifJT1k5cZtzkAQaCxU/Gca/p45IWZ7JhNCj9w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=hpuX02MX3ReTp60dqZ54oABSpNSVJVDEkOtkoHnOlNY=; b=Uz5/vfyKZuTxNxl0y8LfS36H6SfKUXjcWWbqOm7gYPy5Q0APkx+b9OZ69otLNufzAtDH98LBlIiQQGxjeY8gD2rakWZYuSx1IJAtZOm3l/JegYDyk00mzks4rXYJMiw8WDFNDSu5xhgetepKYOPpB4fhz9P1VV5Ckq7fDsO4Nx213+2XBY1exm83PZoqQbZ0mwu/PNOsBI+2Hn62aiIjeyD7wJD5R726AgQlghFhHAmQ1c3n65/2qGdul3KO3vDw9bI61JUycPfDL8Fb1DdLRrWx6WNYnr6nWmeP3KYjaihP3PHTAlbAdNLWk7LnIm6btR4v24gGcpptLr3OxfkP8g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hpuX02MX3ReTp60dqZ54oABSpNSVJVDEkOtkoHnOlNY=; b=MQs43/Y3kaJ8nwdZ9HE8aLsy7f5uxxRkMCmhJoqQywdJ6UpnC2gLZppxNAc3sNf0+b1c2FZN6WuGgF+VwlALGm1j8e6RQ3S08TwlYGMJCxpFwSo8jRF4BDYZgdIjhbClOWyTvcVZwYQtMSCHrWMPqdeIW9mI/TPX8pz7vwr7ZHDL7mOGyruFn5B1k9qs3W7P//b8ufoCok7ESC9lImaKkRMsk+EedsG6KApm25f08a/NlWh3tBwbKkcucAEO1sILb6vKEAqaHyoQKBgM0IJbL3NqYNPCpweJmFkOsm1P+UUnHlB76BppWasEuizx2kQiCjUktdyfSghnflDV8mbkpA== Received: from AS8P250MB0744.EURP250.PROD.OUTLOOK.COM (2603:10a6:20b:541::14) by PRAP250MB0465.EURP250.PROD.OUTLOOK.COM (2603:10a6:102:278::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.15; Mon, 21 Nov 2022 01:03:19 +0000 Received: from AS8P250MB0744.EURP250.PROD.OUTLOOK.COM ([fe80::c37:cabe:316a:60b2]) by AS8P250MB0744.EURP250.PROD.OUTLOOK.COM ([fe80::c37:cabe:316a:60b2%4]) with mapi id 15.20.5834.011; Mon, 21 Nov 2022 01:03:19 +0000 From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Mon, 21 Nov 2022 02:03:25 +0100 Message-ID: X-Mailer: git-send-email 2.34.1 X-TMN: [tFKd4P7dBfIn0rs+bDMwBgq6AZm/m25vNQl54SQuxEQ=] X-ClientProxiedBy: FR3P281CA0151.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:a2::12) To AS8P250MB0744.EURP250.PROD.OUTLOOK.COM (2603:10a6:20b:541::14) X-Microsoft-Original-Message-ID: <20221121010330.2131526-1-andreas.rheinhardt@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AS8P250MB0744:EE_|PRAP250MB0465:EE_ X-MS-Office365-Filtering-Correlation-Id: 01685bc6-a02d-4474-7901-08dacb5c2dab X-MS-Exchange-SLBlob-MailProps: AZnQBsB9Xmo0aNP8rr9jcp/TekB2Cf+UGEm6saw4ic7gH3h3YydOCPlIMndbldDkxHoYlHzSR2XJm/GGgOQhhIcTbU5C5vvnWXt0vjSSfFfM/gGkdmrS6Zlw1jmvuUneqof+7Lvpl/kZmxeH7LK1YUCkNVKYSGmPoeLjBamPKXu2KaciJ3xvCCz8lyAKi846cxZrZQqVydK2G67CaHuE0bBDoB0WoJHk8VY4FgRcw3mgtTgZ5ern3qGZP4c0SIxjPD8bGGyyH0qpJJs+cEggxW7JM9eYyFeE7v+Vth851GEPTq83n1XCGFWImDG6C/Ge6n484b2J5I/nyw+6KXgerBhqSVtggSe4rPMYNiJZTskVS1dX62UCP8OAHeht1NySJJb0+mIGdxXB5wjCiHwDh+kQffBclLdzcwegK442iejOnwYzI7aTnffxGirMWSUGUjsxWlKamoSxeLBOxw0qdXCoGtebfy6ymod54vEqpRdoXQPAL4bZKKj+zXeajzYbjr3nNiPC0IiFVD48G4xnKzbbdZmj1oHMicQgluMgzBu1BMfqiCp0TkNLyWq6+TMtm1474i0hqhZgTRVIZzdmdbMwXtWtjVgrcT06ybJrPVo1yqx2Z8spq4n7H3ZZTEos8ouB97UobumJa62j14aqdXdKxjryKn9bqRd1sGQueZSR681/XA1Eoaqg3T0Ten8JzYgcp7qLVf+bRbqVvoUkruFq5JH1H/7pbdLcoB27IHXUJH+QbsbHyICXVUWFddRgj8zywGLH/eU= X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: B1zT7jdQdrTwIhXT+ufgC1xjh3/cfjA+Iiyggs/rqF0kToDoo1CYKtdLZNiAkIXN3Tx4ClQI+2iUxOtbwqJgHaRcWLCe64/1TwMvvV0Z6KTRHnIIkQNDbm5NjCsBuv3SZQEWT0DR0IWyQLrf10LxsCL1NOkgCijkq8fbk1tFb3fBktnHwlcEYHWGW4u/esr6I9ZR6d3JfatnNm39I9jnLRgSfFY4MEjJEqvJBieThNC46N1/wGsFQm81CI63VpIJOFPpJaks8Pn7DzeSDyEjRvskTJyBV9mJOrnwoNADNTPZaCUaVq5aqKe/L9ZEfcKXlRLqw/NDc8VaI/lxl03395MFyxj9L2AdN7NJ5l4r+uyC0dmmvQQgifbf5eDOZ8xiF9SBSdqa/lTE/J4TIP2U7MpMCfdKu2KHqNahj26WNv+QyW9gYu7/cxXIE9HuC5mZOzSY2n5yKOWgu3DdYECFvC21UzRenfr+5XvSLAMC6B7pTTiAqZKuhwwZUxmjBB89OgxfJixpV4ycc0gFODBdgf1aiVaW1luZZSeCQGtolw56hC4nbboHNqDA67/7xrt1w291YFW5jzi17Nzcz7Rhnw9Ff0v05fRvvGnnza1xBSpcx5qCxRPwPCCqQ0+K6xvQFjDjrZH6hDG+i+wpjSHT9w== X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: ayg5XbjvQLKxxPSJUrdNZczNx4tgt1IM9RRww5ud2y8Wijx7n6wB4PVsxybfcb0NZiSZ1jI0kQAOmaX1MGSWIgBXJhmt2xGMgscdp7cnUzZvoYnW+1K8l0InljumQS2FkyxCk0kCNKi+kkwunSo3/0n8cr/6JM/thCktS3+EhxoVuL83N2hInqH8pvSX/jr30fSDQ2ZjQLs9fA6ZwdJlz4IXke+5TGFwGhr9YwW4uCnYl5dPFOX3tGkI9IER31ldcMvSMcM1Hsaw68bbii1auKw0Sy/NRbtje6zvIpHbIXrYJFvIejr9o8/+b8kqpz6HET/ZtruTmkG4Gy6Gg46CVMSU2HRNg/V3p21qAV3rj9qQj2IgSsdWzHPDyyb1WNTUxTIfoxxachsqTrn4GS8pXm1+j3EMPp9kA5St/Y+Byy98pKucnYBBKIvqFNf6OemCVyEXpPiovH6lGJenZcn+H7VxCS5LdK5dsthZMP7b9ZqDmwnxaCG6nyWBQBhc+FKdlQ12JQWORQTqJcpNtcNTnFm8zhJpgOsWusv8eehYSPu8VHpBdNbyHM3VvjoNaaqMwot0s0PvvgWT2zQ0z4CRVntSZBil3wgQKUCi8gLiPFO7yndne9GB0rxk3/Kxid0sJRjntD9IXw9FAgSZB6JdyS72B/DGMHkjHI+5OTHsEh1MYLO2cvBBM5smQDemWfcEMlJrI3MzHxCHGGb85cuoNr60GLWbxvGuADT29yK2mpWSDoGmnKRv7JF6qhWQA32tnw1knloRBevw7GAVDcebG4vwlb/4BGZQiH1Rx+FeU38/RWc0L28WpNM8YEIOeDFVw4/205Kr7ccCoxsS/6TbZM7zqLv4bItfE/JiCAhkDdQ13RmF+8eg3aSGknsPOg9ItBje0vrOIH0yKkYK6C6x8MdXxre0aPoX7glJBbNKwzrIrG9d/yavl4V/x5z3DQiW0suoaF1IT4WK28v3kfvjVAlTlgugxn4HrglkJTlZ+TTq4UMezD9sQYoOJ2Dy3i1IY/AX4Y+Ow4Q7njkYNcJ1CWpu20qIFkPh7ex2u9UwIwInE3UsXQ3y1ev0AUyMYI4S6Nkmw5AvsRcaJhH05dfPcpGME6ClzUMtRnfVuC6lHJdQcI/9O7AGJFTTn7uNarP+IWlQT2q7vfWJ7iiI/I/pPZrJBHxvEAd2OA/YFPfDFZQtIjbUE/xFXNvzJklfmyAUeXPs2Tl2bShIzjZ3NYeQZbvYpmaUFqMuwBZsIhTEf/DmG4JYsx5jjccF/nBJu73EyNYZK8I/s7Gwza2vFhrOiRBYlUve+PHv08shvusNTAXlOvOy7M9Egd0zpphu2+XO X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 01685bc6-a02d-4474-7901-08dacb5c2dab X-MS-Exchange-CrossTenant-AuthSource: AS8P250MB0744.EURP250.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Nov 2022 01:03:19.1707 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: PRAP250MB0465 Subject: [FFmpeg-devel] [PATCH v2 1/6] avcodec/flashsvenc: Fix packet size calculation X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: UXkYvHlUPtON The earlier code did not account for the frame header as well as the block headers; furthermore, in case a large part of a block is unused (due to padding), the output size may exceed 3 * width * height (where the dimensions correspond to the visible pixels) due to the overhead of the zlib header, so use the padded dimensions to calculate the maximum packet size (which is also what the actual call to compress2() uses). Fixes ticket #10053. Signed-off-by: Andreas Rheinhardt --- libavcodec/flashsvenc.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/libavcodec/flashsvenc.c b/libavcodec/flashsvenc.c index 35793400fa..4cedb53fe4 100644 --- a/libavcodec/flashsvenc.c +++ b/libavcodec/flashsvenc.c @@ -54,11 +54,15 @@ #include "put_bits.h" #include "bytestream.h" +/* These values are hardcoded for now. */ +#define BLOCK_WIDTH (4 * 16U) +#define BLOCK_HEIGHT (4 * 16U) typedef struct FlashSVContext { AVCodecContext *avctx; uint8_t *previous_frame; int image_width, image_height; + unsigned packet_size; int block_width, block_height; uint8_t *encbuffer; int block_size; @@ -100,6 +104,7 @@ static av_cold int flashsv_encode_end(AVCodecContext *avctx) static av_cold int flashsv_encode_init(AVCodecContext *avctx) { FlashSVContext *s = avctx->priv_data; + int h_blocks, v_blocks, nb_blocks; s->avctx = avctx; @@ -114,6 +119,11 @@ static av_cold int flashsv_encode_init(AVCodecContext *avctx) s->image_width = avctx->width; s->image_height = avctx->height; + h_blocks = (s->image_width + BLOCK_WIDTH - 1) / BLOCK_WIDTH; + v_blocks = (s->image_height + BLOCK_WIDTH - 1) / BLOCK_WIDTH; + nb_blocks = h_blocks * v_blocks; + s->packet_size = 4 + nb_blocks * (2 + 3 * BLOCK_WIDTH * BLOCK_HEIGHT); + s->encbuffer = av_mallocz(s->image_width * s->image_height * 3); if (!s->encbuffer) { @@ -229,7 +239,8 @@ static int flashsv_encode_frame(AVCodecContext *avctx, AVPacket *pkt, I_frame = 1; } - if ((res = ff_alloc_packet(avctx, pkt, s->image_width * s->image_height * 3)) < 0) + res = ff_alloc_packet(avctx, pkt, s->packet_size); + if (res < 0) return res; pkt->size = encode_bitstream(s, p, pkt->data, pkt->size, opt_w * 16, opt_h * 16,