From patchwork Fri Dec 13 14:22:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul B Mahol X-Patchwork-Id: 16773 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 7FA12446DB2 for ; Fri, 13 Dec 2019 16:30:52 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5ABE668A8E4; Fri, 13 Dec 2019 16:30:52 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f66.google.com (mail-wr1-f66.google.com [209.85.221.66]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 32CDA689FE7 for ; Fri, 13 Dec 2019 16:30:45 +0200 (EET) Received: by mail-wr1-f66.google.com with SMTP id z7so6814391wrl.13 for ; Fri, 13 Dec 2019 06:30:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id; bh=5ePClWcrINL9/RMdcFcOhjq8zgFKniqKbGTfUQ7WSoY=; b=jyy42180Rn777fxLI3C5O1lztbK8Cf6n54NhkMPcJ9n/xSF5U+Rs4120Hq63GromAJ s9IIK9IQS5tIYcpYXil0dbI9R8kLbPHSJiT7+mmpIvWlLAXtrReld9YHUYqdKK/TV41y lRRn5RM/zqal2bXfAtBoVVX/s4CEYnUq0ld6s/YIBZiQQthcQxQtKZFISfLIy2BVhblC J1hmUDAxf6iNq1gkQsAf0esThVjTJ2yvQ7EmiJEZQorvhBk/LqeiTV8R43nbvXi452Ql zeNh2NMdjFpNUc1fyE+Z+Oy/yJ4E6wocZ0FEwZY/buFBBn+ZHoMoxmIz4rlM1DR0/Nj9 Lnhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id; bh=5ePClWcrINL9/RMdcFcOhjq8zgFKniqKbGTfUQ7WSoY=; b=MQhiS5JYEV/vgc8tahRASNmvv+VQjIX6DGF0eTD4wnBAYrYxXq5d706hQ7rVa7IxK3 eBqX1aF8CMOCelGp5r7p7N45EAIXAnapF2MVCQ/Sf+OU+a4V7j3TN19JHftdmnSTwqaI dJGF9FBVipOZ4JA8Mb/PXg3wfd2ReS/ypKWapZziE+11+ZsJV6WSzRTSU85UMDl0RojS dLrAz1tJUwAxpg/Bu9I4TZ9yko04gB/9A9pK0d/y0fgsCXuJMf3BwW5fZv2FYGSXytMs amwta1aMX7qaCNor5fUl93v39BPCDiJCcNgvOOd/ZPQyl3Iu0axb8Yfmd3aUgod+kzrx GPXw== X-Gm-Message-State: APjAAAXzCwI7UHJ4w0ljIAR4X5IZ6O7x/K0s3BXjdY9Jw+Hp+mffbqU9 ATZY0yu+hv2eEf+3CG0oeV+kiJOF X-Google-Smtp-Source: APXvYqxH4to3nqresOFMHl8reLE1hBJ41NxUmQJMue4zXkpR8xM8gmrfk+xsc4sizQG1EtsbMFFjEw== X-Received: by 2002:a5d:4847:: with SMTP id n7mr13263349wrs.30.1576246946176; Fri, 13 Dec 2019 06:22:26 -0800 (PST) Received: from localhost.localdomain ([94.250.171.226]) by smtp.gmail.com with ESMTPSA id j12sm10446803wrt.55.2019.12.13.06.22.24 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Dec 2019 06:22:25 -0800 (PST) From: Paul B Mahol To: ffmpeg-devel@ffmpeg.org Date: Fri, 13 Dec 2019 15:22:17 +0100 Message-Id: <20191213142217.25352-1-onemda@gmail.com> X-Mailer: git-send-email 2.17.1 Subject: [FFmpeg-devel] [PATCH] avcodec/simple_idct_template: fix integer overflow X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Signed-off-by: Paul B Mahol --- libavcodec/simple_idct_template.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavcodec/simple_idct_template.c b/libavcodec/simple_idct_template.c index d8fcfd7c53..5ddd0b45a2 100644 --- a/libavcodec/simple_idct_template.c +++ b/libavcodec/simple_idct_template.c @@ -121,7 +121,7 @@ static inline void FUNC6(idctRowCondDC)(idctin *row, int extra_shift) // TODO: Add DC-only support for int32_t input #if IN_IDCT_DEPTH == 16 #if HAVE_FAST_64BIT -#define ROW0_MASK (0xffffLL << 48 * HAVE_BIGENDIAN) +#define ROW0_MASK (0xffffULL << 48 * HAVE_BIGENDIAN) if (((AV_RN64A(row) & ~ROW0_MASK) | AV_RN64A(row+4)) == 0) { uint64_t temp; if (DC_SHIFT - extra_shift >= 0) {