From patchwork Thu Sep 12 23:33:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Niedermayer X-Patchwork-Id: 51556 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:c541:0:b0:48e:c0f8:d0de with SMTP id f1csp564vqr; Thu, 12 Sep 2024 16:34:13 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCX0b/479mMtWcdqIb3o4ARTmlyv5DToI+2cRW746sTuR8lrBIwkrhK9JfDFMZPOZyVJxISjItXW3RMzpBJaRE7I@gmail.com X-Google-Smtp-Source: AGHT+IH1RB616I1fDLdAvnEL9clkZoY9ZrsYQ43T9W2BZO4DiKfnsrDyiEjIIsLEjLT9d6IFeuRC X-Received: by 2002:a17:906:fe04:b0:a8d:6921:e179 with SMTP id a640c23a62f3a-a90294e6753mr421427666b.39.1726184052903; Thu, 12 Sep 2024 16:34:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1726184052; cv=none; d=google.com; s=arc-20240605; b=DcqtaM+b7fOnlT2+L2lIEZ9lQudFooo0ZSsATQh6u/i2IYh9xPByRW1NceRrIisRsP 7GxBEHtq71n4cwrqSQpazr49qhJdyaQWdHH4TJuN7q1BkzxGjmfLk4/RS0s//2XHap48 V5d9sLIABZ8aPYSb7yvv0eT8YmM9wv4UPmvEKSWMvMhllUVKEi+MlpzCyVexQsIOqd9G HChZUVZvpzaIjN+OTMHF/TsJIV3TbZDm7dq7YF918wXAmQZ/TqZgllxjneKeeK4kyYtk 0Mszq3rEbVUaDpEtXL2oNv2vzgMxF+B05te6+XT3WDaflvoQiLGGlj3Mvp7k3/DFqRXy NKUA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=Qu2tSv9VPwutN94bCW30JgrxmUjjplosYq+iZ8G7Lgs=; fh=e5zN9xSzcxLA6bGo3lF+CqTbY/oLwzApV03EO/RBfgQ=; b=Vm5bq2egW0f/9KF037L60CN2XtC1q6/Ddj3dAhlbLe9nS5mCoeXRUxeSwGtntcDDWz YgfgvQzIQTdpin5Wt0bXbMZ4Ls6nKBTI2pDVJFJk5ah4KkKROinG9BadhwLfHArdZJd+ LW9r9XwhGgeRMYWConL2CdMROXfpSQQqz2H8kNb2mpwx443+pnVt91Dpmb7rwyQ14QAJ ScgOK7eFFhRUfmOeGgGJJPEL4s6Zct0g3XwHwR3Hm4pnVljAWMu3qBKmamJzG2j86ShR eIDgkZ1elPk/B9TRL5dtqcEglIpk6FGrejh4JrDjAYN5bEbPsLRUpH+dH2P5CtGc+EKa h9CA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@niedermayer.cc header.s=gm1 header.b=Khg43uYM; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a8d25d40eafsi943252666b.695.2024.09.12.16.34.12; Thu, 12 Sep 2024 16:34:12 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@niedermayer.cc header.s=gm1 header.b=Khg43uYM; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 15AAA68DD03; Fri, 13 Sep 2024 02:33:49 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [217.70.183.195]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0360568DBF5 for ; Fri, 13 Sep 2024 02:33:40 +0300 (EEST) Received: by mail.gandi.net (Postfix) with ESMTPSA id 44CA560002 for ; Thu, 12 Sep 2024 23:33:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=niedermayer.cc; s=gm1; t=1726184020; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3sBUsit0OVBv+rXhApoeJLu1Ms8KNv/1jaWQnf6Wei8=; b=Khg43uYM/EazQO3JXhHSGXkEvwQ9ygxIynQRAmtSkHNrMDHZUZg4RGelMrdJqvWNr4M9J3 YXdveh4Cr9Qz9Ni16o14sAPYW4HGcuxSlT7JBSnDhUfMXpBVk32+6q2yIOKo1/BmWl2Pc7 dnEJq6zXH6O3Jmj+Fq9xJsl/VjndyTMvYZBkBoxr1DwA+ZOJO4KodwbQyaHJ2v/68DqVFR GqzbPYJCUbx2qf414iOB5gswVdW51sTGFgu1RrZljEbrAzZzfM4rcI3BgVw8AuMrzz+wxf tZ5hZ6TmoJsTjzBRSyzNqIXJfSWT2ittx+Ri0Zm8SmcSeHcLo63stDELa7enfg== From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Fri, 13 Sep 2024 01:33:33 +0200 Message-ID: <20240912233337.2444412-3-michael@niedermayer.cc> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240912233337.2444412-1-michael@niedermayer.cc> References: <20240912233337.2444412-1-michael@niedermayer.cc> MIME-Version: 1.0 X-GND-Sasl: michael@niedermayer.cc Subject: [FFmpeg-devel] [PATCH 3/7] swscale/swscale: Use unsigned operation to avoid undefined behavior X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: eXkFmKLZfNAC I have not checked that the constant is correct, this just fixes the undefined behavior Fixes: signed integer overflow: -646656 * 3517 cannot be represented in type 'int Fixes: 70559/clusterfuzz-testcase-minimized-ffmpeg_SWS_fuzzer-5209368631508992 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer --- libswscale/swscale.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libswscale/swscale.c b/libswscale/swscale.c index df0d5708aa8..8b6a3a84b4e 100644 --- a/libswscale/swscale.c +++ b/libswscale/swscale.c @@ -224,7 +224,7 @@ static void lumRangeFromJpeg16_c(int16_t *_dst, int width) int i; int32_t *dst = (int32_t *) _dst; for (i = 0; i < width; i++) - dst[i] = (dst[i]*(14071/4) + (33561947<<4)/4)>>12; + dst[i] = ((int)(dst[i]*(14071U/4) + (33561947<<4)/4)) >> 12; }