From patchwork Fri Sep 17 02:08:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 30305 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp1784842iov; Thu, 16 Sep 2021 19:09:34 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzB6C/fmbrKuhlbbtlLK/DV7m8KHQJ4WRic81LZDUz9DzLF+VCS/0YFBCqApn1XbvnRkbaG X-Received: by 2002:a17:906:dfe3:: with SMTP id lc3mr9675610ejc.478.1631844574326; Thu, 16 Sep 2021 19:09:34 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id j24si3286089ejo.673.2021.09.16.19.09.33; Thu, 16 Sep 2021 19:09:34 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@outlook.com header.s=selector1 header.b=VTx72r0h; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=outlook.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C6A1468B119; Fri, 17 Sep 2021 05:08:39 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-oln040092066061.outbound.protection.outlook.com [40.92.66.61]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7D25268B117 for ; Fri, 17 Sep 2021 05:08:35 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=gX97ualikj29EZhFZEd8tv+ajg9mcWeW0/DbVfY4Qccxq8H9vhYKSn1wwwVP4fVO+HlWk5ctBOzXuRbvorSz0KVJYb7EoXw1IWFxtXPTfhG/sJLcJSI3VSW/luERf8e4e7kPyZq3jhQZji85ZJpIEF3gZUh0P+QLCZuZl5bk3aJe0tOAmN+xwM2hQGwuUHFTjMX/Ay2Hy9rJHAEeTHCGrpLmNSV3AnHNQutqJzP3OzwnrVoTFoEfIeaf7eb1lbAIziXqpYplDQDv3VkKdP2b0QMR20nCADgb+ZE5Bf3bTjZOUbHNVO8DMEp+PbUXtNyja8AC/tBthytpYzdFnbaxLw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=diiu013vM5SbGzzPD771qjCCILvWTY9ZwDDde/FZhlA=; b=TwiJGVdyA7UA6V80noeQsQsmDk/tbva0gEL5zbzKNrh99Ig/PUvuPEcZk0IEoKHgf8vV3SWHARqYArKDOSHbCdwp6LXkC0j8xPUlvBZJCxMLZKMjeNRS+ZOM3FuwbkaMNgdBUgwDhyj6HIQ/I2+CiOZcP8VmaEJJHyU+CSEwhdgxvzVwI0wGv2k7+WkSWttu7mYzR9FMzVKEUcV/KkiZtMmy3kYJ82nIoWIUvWPWcbZU0Rp1T0C/YDjfRvTyaNdEyt6Yu98iwD/4ar1TwOm0QEsivMrd4N8/HMw3D9wWnKiBLePlu2SCAxHoQEZcfGIA4O2JorqXi3Cna4WMTDHklA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=diiu013vM5SbGzzPD771qjCCILvWTY9ZwDDde/FZhlA=; b=VTx72r0hskzL3hNa5aoRktSYTT6zbHFMUdVW2wkQNNLJx9fVq41IOmM+oblDuDhugxbhUSWQ4sx62jKd2okCPFNNhC876tFqBxiM5nSgMc8xHPbo3cvxJQ/U4uR8xFPWhpxSodtDw0PFX6mCyPKApFXQ1ew3DI6UqbkIYfG39f1S3KsboC2B/o1gA63NZzkz7pZ2yjQGuL4c/4K6Xv8ApDeW9A0AA0AjmhA4Xglel7x5YPV18Ow0WQJ3BTdEoD/TEpaQBweroYDpNQb2H9Q0lrjN6x3jfo8ROmKQwYdVrkzx/g2LYFTzgxwE3Dsy9QypLgTQ8Ik54VHYQqvGJ3dWbw== Received: from AM7PR03MB6660.eurprd03.prod.outlook.com (2603:10a6:20b:1c1::22) by AM6PR03MB4104.eurprd03.prod.outlook.com (2603:10a6:20b:18::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Fri, 17 Sep 2021 02:08:32 +0000 Received: from AM7PR03MB6660.eurprd03.prod.outlook.com ([fe80::787b:2156:ca99:fe00]) by AM7PR03MB6660.eurprd03.prod.outlook.com ([fe80::787b:2156:ca99:fe00%3]) with mapi id 15.20.4523.016; Fri, 17 Sep 2021 02:08:32 +0000 From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Fri, 17 Sep 2021 04:08:06 +0200 Message-ID: X-Mailer: git-send-email 2.30.2 In-Reply-To: References: X-TMN: [e4wM51gw8VQiuAMQwdldsRtEsYTFxZqP] X-ClientProxiedBy: AM8P191CA0018.EURP191.PROD.OUTLOOK.COM (2603:10a6:20b:21a::23) To AM7PR03MB6660.eurprd03.prod.outlook.com (2603:10a6:20b:1c1::22) X-Microsoft-Original-Message-ID: <20210917020808.275498-10-andreas.rheinhardt@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from sblaptop.fritz.box (188.192.142.38) by AM8P191CA0018.EURP191.PROD.OUTLOOK.COM (2603:10a6:20b:21a::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14 via Frontend Transport; Fri, 17 Sep 2021 02:08:32 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d260d6f2-a0fb-49ee-78f0-08d979800c99 X-MS-Exchange-SLBlob-MailProps: gjx25WM8ZNWDyOY6I9mBNa2F8wdSMbXWJ6/IrX7faoA2P2W6dWFqK4MAS1gRKjQt5Im3elirCusAmHQpYNq7g9Z63rK+XFYz0kJMtE9iwlHNjIZ056A8mY4b7btWfbMznHPGVbsU9aVYeA6/VJxbZovbg7B5LIasIyjUIsvlTxmcPPwRPUvQPP4cs7bzOeXVL6M6fBnE1GPRHsQycMY/GURKk4CeY93iemVwDj0HACQ8yuY+h5nEQVltThGnHuJk55RtIiPz4798Vx2DlDuUSbXxIz3vXK+01Vf3S3caJ99VNZUOGFzzP1MfyQrfJoLZMcOBc3HCAAShYZbyT9x1auAXM8194/Tb7rvSQTUy+GZSqb+0YdNFnc7Ou7w4mWIUqD2DNPuUviQ8Ll+xl3NMI5KdIC323SsM/nkUhwNF/7mxziUDwoyT0NayX8uVCxzM4xficlPH+6MdirUsfG0VLe6ckESAlVTkOubnezhAYuYBdFKoBh7jPd+/agVLghjNFJn+TfwEnEUvkninB5zo+Zd/OzXyFJOnemLavQDMF9c0PEMVoC842eS33ToGg27lx4OCTqo1UXfZnOabumsIDdTqj6HRGpSxosAwiz95B42Dke6rutbf80MvnRC/G/7lYNUAw6R5UbqMx5E+9d86kAkBYL7C0eu97mn/Xze4qWydm1ev+RahXEUh4JOQcc//dFb+phtCxk6V3bmAIuxw5+9eMhDz0XIJo4Y0+CktbrE= X-MS-TrafficTypeDiagnostic: AM6PR03MB4104: X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: RnnZ2KCZKdSFyZ0Ui8NeUEzE/pX5W+smocD1v7WJ142OyKJh2zAB8gzn6h6UvHHeHxm7+aKkdtMckiVuW1vNnsmGAKrSH1h26v02cFD51xVwc9HyFhEy79EkMuZ3LEytlNzjOs4M2O2mqx45BJ3dEisyS5FrUifnAqsF9JCzK0ZHptxtTAnwe9dKzTFjEwDLxctXo3IZhxOKxLp3GPzIMjyx4UJa4fVDec6rNQidQ6RMoSEwkUt1LEPcdpLgyLWcWNpoe3uHTQIt6rTvSUMEOHfL7tI9CGgYRa8euwif1b9yZTGqDUCaqPZqlQW6viz2y+ZDBt8AIHppCRiu3iHwNYHjEMbbmrh5sxEyuX2F99Uh+nKchDK/4RfpnTTLSBOzis/pBSujyvae3K45oXs9r+iivUSh0R/6Wns0+LGuaCYTTmaVlG3rmo2eEz3Xqf3j X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: RXH78GVOskmc88t1ho/BjGazfGTQm3sA/+m9mFqHP3qPubG/6NLZaCyzE1+hHE6CFTgeB9WcPI5/EtymFeeVAPLy9n2qRqcDw0o4B/MLKQW+kCevHPAM6Mku3CgZ8AZf3mX5FyWztn6LkxM27SvqxA== X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: d260d6f2-a0fb-49ee-78f0-08d979800c99 X-MS-Exchange-CrossTenant-AuthSource: AM7PR03MB6660.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 02:08:32.5637 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR03MB4104 Subject: [FFmpeg-devel] [PATCH 11/13] avcodec/msvideo1enc: Check all calls to avpriv_elbg_do() X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: xVS6VKSx807u Signed-off-by: Andreas Rheinhardt --- libavcodec/msvideo1enc.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/libavcodec/msvideo1enc.c b/libavcodec/msvideo1enc.c index d43013ba5f..79810ec8c3 100644 --- a/libavcodec/msvideo1enc.c +++ b/libavcodec/msvideo1enc.c @@ -118,8 +118,10 @@ static int encode_frame(AVCodecContext *avctx, AVPacket *pkt, } // try to find optimal value to fill whole 4x4 block score = 0; - avpriv_elbg_do(&c->elbg, c->block, 3, 16, c->avg, - 1, 1, c->output, &c->rnd); + ret = avpriv_elbg_do(&c->elbg, c->block, 3, 16, c->avg, + 1, 1, c->output, &c->rnd); + if (ret < 0) + return ret; if(c->avg[0] == 1) // red component = 1 will be written as skip code c->avg[0] = 0; for(j = 0; j < 4; j++){ @@ -138,8 +140,10 @@ static int encode_frame(AVCodecContext *avctx, AVPacket *pkt, } // search for optimal filling of 2-color block score = 0; - avpriv_elbg_do(&c->elbg, c->block, 3, 16, c->codebook, - 2, 1, c->output, &c->rnd); + ret = avpriv_elbg_do(&c->elbg, c->block, 3, 16, c->codebook, + 2, 1, c->output, &c->rnd); + if (ret < 0) + return ret; // last output value should be always 1, swap codebooks if needed if(!c->output[15]){ for(i = 0; i < 3; i++) @@ -164,9 +168,11 @@ static int encode_frame(AVCodecContext *avctx, AVPacket *pkt, // search for optimal filling of 2-color 2x2 subblocks score = 0; for(i = 0; i < 4; i++){ - avpriv_elbg_do(&c->elbg, c->block2 + i * 4 * 3, 3, 4, - c->codebook2 + i * 2 * 3, 2, 1, - c->output2 + i*4, &c->rnd); + ret = avpriv_elbg_do(&c->elbg, c->block2 + i * 4 * 3, 3, 4, + c->codebook2 + i * 2 * 3, 2, 1, + c->output2 + i * 4, &c->rnd); + if (ret < 0) + return ret; } // last value should be always 1, swap codebooks if needed if(!c->output2[15]){