From patchwork Wed Jan 22 14:52:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 17472 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id CEF72449947 for ; Wed, 22 Jan 2020 17:06:47 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B104068B129; Wed, 22 Jan 2020 17:06:47 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f66.google.com (mail-wr1-f66.google.com [209.85.221.66]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 24F0A6881B6 for ; Wed, 22 Jan 2020 17:06:41 +0200 (EET) Received: by mail-wr1-f66.google.com with SMTP id j42so7598073wrj.12 for ; Wed, 22 Jan 2020 07:06:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Qo+iUt84EDzw+gNy9ilcWEVS7Gj5yh2JzKwQM8I5hQI=; b=NA/0LKA90yiDCu2s7aMsuzUfS5u2ceTM3zyc+dpW3w85Q9KhgPOvVPhBnG5Mn/waVU kPzaGK5umUgWSkTB9oe80Jb7d0D2Q3Ni29HGyx0gkhN/20I6nF/I1FfF2veUYGhJl8ar NN5sHVUbxUsbNn+Acqo2w9UiQB7KN1r8tfW7D5AAfl2hxKBXCfqtmfJo/45GDTI91kEQ XnVRrWjIlFQAyRpAWdcoBJDQrF/7LbPdkfa39bx3xRYe48JoC9anIAD/7kELRyaYZFFl OCwgkVRftzG7fwEH9w+rlr5+vmqHSRE5RSQ7Ro3aDb0wtifG0XaeOPzA6iKrfHd5ghbO IC/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Qo+iUt84EDzw+gNy9ilcWEVS7Gj5yh2JzKwQM8I5hQI=; b=ZfcV+m7mQX/uy/z/k7zCFtg9IhKgnhZLsSjnd7QmwH4uW/AuoXS1rKziKX+Lvai9OD 6sSIeP5XXl28evCB3HlBEky06I9rB0NEyTl3cUrETFRzWAFS0w8hw/adq+9Jiu3hpcqn OaX93lp8YyA4mJlnoZ+5kkAMukS333p4A3WOFwILmEYwn56XJS0GvBdOhRMr2eIHW48+ ZbZ6KLh36chnuDdVdpsTJR7vX2DN7TbLAANIn3DBobmM5Z7hk500+AQNvZ17+gaW8PBu ndDWS1+NcK8C6KsIx+LmQr9+O/rPMqz8f9/NTPO8II+Xhg4+JMQo1NcQ/hpIwkHZrx5U j10Q== X-Gm-Message-State: APjAAAU7ES8q0dOh+srguuWGvBp+VfJonxSSPdr+3yaPjShOWU55MXQq tOHTmcD9ScZ7Aykqqmj7/FXk8Eom X-Google-Smtp-Source: APXvYqzeMdTJyo3i06ON4MriP/gJlOB0qpzKO09yhouE5cys4Mvv+ytrNrwl9/xhZgTbG1pDt6i3gA== X-Received: by 2002:adf:dc86:: with SMTP id r6mr11651683wrj.68.1579705201662; Wed, 22 Jan 2020 07:00:01 -0800 (PST) Received: from sblaptop.fritz.box (ipbcc08bbf.dynamic.kabel-deutschland.de. [188.192.139.191]) by smtp.gmail.com with ESMTPSA id t81sm4452092wmg.6.2020.01.22.07.00.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Jan 2020 07:00:01 -0800 (PST) From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Wed, 22 Jan 2020 15:52:08 +0100 Message-Id: <20200122145210.6898-2-andreas.rheinhardt@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200122145210.6898-1-andreas.rheinhardt@gmail.com> References: <20200122145210.6898-1-andreas.rheinhardt@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/4] avfilter/vf_paletteuse: Fix left shift outside of range of int X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" by keeping the variable uint32_t which in this situation is the natural type anyway. This affected the FATE-test filter-paletteuse-sierra2_4a. Signed-off-by: Andreas Rheinhardt --- libavfilter/vf_paletteuse.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/libavfilter/vf_paletteuse.c b/libavfilter/vf_paletteuse.c index ed128813d6..c9603601aa 100644 --- a/libavfilter/vf_paletteuse.c +++ b/libavfilter/vf_paletteuse.c @@ -158,9 +158,10 @@ static int query_formats(AVFilterContext *ctx) return 0; } -static av_always_inline int dither_color(uint32_t px, int er, int eg, int eb, int scale, int shift) +static av_always_inline uint32_t dither_color(uint32_t px, int er, int eg, + int eb, int scale, int shift) { - return av_clip_uint8( px >> 24 ) << 24 + return px >> 24 << 24 | av_clip_uint8((px >> 16 & 0xff) + ((er * scale) / (1<> 8 & 0xff) + ((eg * scale) / (1<