From patchwork Sun Feb 14 19:36:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 25624 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id DBF9844912E for ; Sun, 14 Feb 2021 21:36:30 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B3632689A81; Sun, 14 Feb 2021 21:36:30 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f43.google.com (mail-wm1-f43.google.com [209.85.128.43]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C922B68804D for ; Sun, 14 Feb 2021 21:36:23 +0200 (EET) Received: by mail-wm1-f43.google.com with SMTP id o24so5987854wmh.5 for ; Sun, 14 Feb 2021 11:36:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=dt/XPkuy4kuvX6PYwsjjvNMDS0HrwGU+uIIkbcm/1Uw=; b=icZHjky0JKx/HO8OHU0jg6fFreviXST/7GtBa4wpLzacy6ZTpHRlBH9jbRTrPnm1Kg F2rtJIO9IMKdDYQ2K/rUfuqRkLiL560RW7oKNORvm82SiCI4u9c6214yDJPj1A3Fao59 5+o7EXUS/67CKx4OFXUJPPGORLet8cPMerKYc9mzySW5Zcd7MO0XKMBBKHWEEwp4bHfK /uMr/8upMBdN1bcZH8lnPflO1Vm4o3COY8VgMgpHZb0R94Z1K2rE0jCRDPaVy2MsrfEJ TvcQ0g8Y9uELtF6iht3+6sOJFP45XNnnEmLLv91VSxMD7NxAniol7pzLkQns3gnqokAO Hyxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=dt/XPkuy4kuvX6PYwsjjvNMDS0HrwGU+uIIkbcm/1Uw=; b=Sc9rCyIk/icY54hzupaCGb7Zbg78jy3TBqhypVMp5cmlrXNYt524v9xJjt4inA/CXs mKb3tRJ4ckOJUjTFj6YwaTTTbyWFp6J0zutfq0VQfHu9+x2wBg4Yne4pF4EFEYRK4YQ5 s6gm8gNJxP/CT7LfAljyDiX7YFpQon88ZjxTgzpD3LEoT6Qbh5vPFDFnf2pS8i6cVFF9 bDMEDbgwQvrGuZEr/+cdgVfVIqEHm1vAKdXTXy73crPBBl6ZblMWeHoo/eC/BZCEv3Gg kzYGjlltEe/UR7Fee+AZQJgIGlnI/f0b6o5Ep/bmlIT+gRT50RRfTEBsnpSTdUVfVMv0 Wj3g== X-Gm-Message-State: AOAM530yZrAtXe8Fc0q7BI6R9cXZ+n6UORKEHxaaaPtMNCJCxhwDvFdX EI9ZGQi1hZNAO+i8TEsJsRadoRwpr90= X-Google-Smtp-Source: ABdhPJyQcs6+n2xhuA8BwguFEfqepYaWPTa3LSrB8KmwMT8dD0KymrLgVjVfZcKTi3e+93AniptskA== X-Received: by 2002:a7b:cc0c:: with SMTP id f12mr11438827wmh.118.1613331383169; Sun, 14 Feb 2021 11:36:23 -0800 (PST) Received: from sblaptop.fritz.box (ipbcc1aa4b.dynamic.kabel-deutschland.de. [188.193.170.75]) by smtp.gmail.com with ESMTPSA id f7sm25969000wrw.81.2021.02.14.11.36.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 14 Feb 2021 11:36:22 -0800 (PST) From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Sun, 14 Feb 2021 20:36:15 +0100 Message-Id: <20210214193616.111213-1-andreas.rheinhardt@gmail.com> X-Mailer: git-send-email 2.27.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/2] avutil/video_enc_params: Combine overflow checks X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This patch also fixes a -Wtautological-constant-out-of-range-compare warning from Clang and a -Wtype-limits warning from GCC on systems where size_t is 64bits and unsigned 32bits. The reason for this seems to be that variable (whose value derives from sizeof() and can therefore be known at compile-time) is used instead of using sizeof() directly in the comparison. Signed-off-by: Andreas Rheinhardt --- Using (UINT_MAX > SIZE_MAX / sizeof(AVVideoBlockParams) && nb_blocks > SIZE_MAX / sizeof(AVVideoBlockParams)) would have fixed the warning with Clang, but not GCC. libavutil/video_enc_params.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/libavutil/video_enc_params.c b/libavutil/video_enc_params.c index c46c0f1dc6..b9cdafddbb 100644 --- a/libavutil/video_enc_params.c +++ b/libavutil/video_enc_params.c @@ -33,8 +33,7 @@ AVVideoEncParams *av_video_enc_params_alloc(enum AVVideoEncParamsType type, size_t size; size = sizeof(*par); - if (nb_blocks > SIZE_MAX / sizeof(AVVideoBlockParams) || - nb_blocks * sizeof(AVVideoBlockParams) > SIZE_MAX - size) + if (nb_blocks > (SIZE_MAX - size) / sizeof(AVVideoBlockParams)) return NULL; size += sizeof(AVVideoBlockParams) * nb_blocks;