From patchwork Sat Sep 28 02:25:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 15355 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id D40B94475E8 for ; Sat, 28 Sep 2019 05:39:48 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id BAE4168A55C; Sat, 28 Sep 2019 05:39:48 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f68.google.com (mail-wr1-f68.google.com [209.85.221.68]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id F33326897DD for ; Sat, 28 Sep 2019 05:39:41 +0300 (EEST) Received: by mail-wr1-f68.google.com with SMTP id w12so5128139wro.5 for ; Fri, 27 Sep 2019 19:39:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=92gqDKqfRm1qdfUv/I2TotrQ2F8HcIG4ky/sz4lHyEc=; b=sx8XCUFBkRWHyMtvpReAnuycoLh4VFpovy+v+nxMJeEOq4erVPXhTC7LujoeD7bYKF E6i73NLj7F4G5PRS+OOInFz4rY0zWICzsdQPcnseGKWIDwemzwx/S0K5WCHELBSnmlaR 6PnGRHEOZsCmip2s+KDyT1irzrBb+hwEImc0fcXeane+58U7bp0T05sE3a4c8DMo0KB5 IM/h/Oc/1eQ1dlAv07CIedsQ12hsO0dVevwGOBvqD0Phvr+Y2AT7YUS2RLqiktAgl9bc kz/r5Kk/S78zp/93i++b1CuxjHeAjPnPmrjvSTn9YOHOZ7W0FkV2ntgtBq83ffW33Cnl udyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=92gqDKqfRm1qdfUv/I2TotrQ2F8HcIG4ky/sz4lHyEc=; b=Ufqb31AW6gtql26Ng29tTn4b84vsNHO8sg2FTGhExkDYxtuSf9C63S2O7MyrtjX0gp p/5z6j8+iKKmVNsp6ubbZUdnx599jzEDZRwtRccDF20Gk5Ry3mpr8wp1kFgiIbw/2iHB cxmBw+eJNHbZBsNh9BAeSNLMxj7szerkYfIsvPt9DGmdLpJs1uekeV3o0elCgrDkwjbs ca1cpgTmfv6SJdNJyMdVj7OJdXXqbqlar53tRb67zcE4eRtiQFcExW+lV0D+MB6bZQL5 3u20JJR+DlzY9fsGIiBBxvZKxAvic+5yCOoVM5NF+yOR2bSdG3wOU4hnbYAnOsYF21Iw se6Q== X-Gm-Message-State: APjAAAVAG7dIOxv2W7BZFOPpsyIYRi4BG6XsQbWKnwodfovq7U+bDTxw KiXntrf6Mo1BjknGSNT+o/x7LedV X-Google-Smtp-Source: APXvYqxMK+IQM/zw/PIMBGvJg3dj7VAUrgiD0GXN2q3J+JFOyNhNJ43/PK3XHVCGeau0Rc7IYDQwEA== X-Received: by 2002:adf:f406:: with SMTP id g6mr4859706wro.325.1569638034345; Fri, 27 Sep 2019 19:33:54 -0700 (PDT) Received: from sblaptop.fritz.box (ipbcc0f857.dynamic.kabel-deutschland.de. [188.192.248.87]) by smtp.gmail.com with ESMTPSA id r7sm4484231wrx.87.2019.09.27.19.33.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 27 Sep 2019 19:33:53 -0700 (PDT) From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Sat, 28 Sep 2019 04:25:57 +0200 Message-Id: <20190928022610.5903-2-andreas.rheinhardt@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190928022610.5903-1-andreas.rheinhardt@gmail.com> References: <20190928022610.5903-1-andreas.rheinhardt@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 02/15] avfilter/vf_hqx: Fix undefined left shifts of negative numbers X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Affected every usage of this filter; in particular, it affected the FATE-tests filter-2xbr, filter-3xbr and filter-4xbr. Signed-off-by: Andreas Rheinhardt --- libavfilter/vf_hqx.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavfilter/vf_hqx.c b/libavfilter/vf_hqx.c index 16a1be7bd4..4f768c7a13 100644 --- a/libavfilter/vf_hqx.c +++ b/libavfilter/vf_hqx.c @@ -523,7 +523,7 @@ static av_cold int init(AVFilterContext *ctx) int startg = FFMAX3(-bg, -rg, 0); int endg = FFMIN3(255-bg, 255-rg, 255); uint32_t y = (uint32_t)(( 299*rg + 1000*startg + 114*bg)/1000); - c = bg + (rg<<16) + 0x010101 * startg; + c = bg + rg * (1 << 16) + 0x010101 * startg; for (g = startg; g <= endg; g++) { hqx->rgbtoyuv[c] = ((y++) << 16) + (u << 8) + v; c+= 0x010101;