From patchwork Tue Sep 24 22:03:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 15272 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id E9605449BE7 for ; Wed, 25 Sep 2019 01:03:51 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D071368A40E; Wed, 25 Sep 2019 01:03:51 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f66.google.com (mail-wr1-f66.google.com [209.85.221.66]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 05BFC68A32E for ; Wed, 25 Sep 2019 01:03:41 +0300 (EEST) Received: by mail-wr1-f66.google.com with SMTP id h7so3801274wrw.8 for ; Tue, 24 Sep 2019 15:03:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=eVSSolozDNzm2ld5ayce9w6Mm7MgFFlInyM+I63lxxk=; b=habt+5ZNAoCOXnzj24UUWiLNrAOjU7c59ogsYBrkXss1i5fKsJtJucoVEaYLQb25ep /6RLhRJ9Ci/VY84NyYIfWc7Ikqg/sLs45OHrVYjpH9jN/8G3TinQzRh6VKcjneJG0alW ETfszZf1Ce8gDzlsmSvRBClR+DeiD9dMSW3iJTW6GwnCQiE3015tLhrxbu75OHZR2oCH RgDn3T63lmR4IASa2mxUjIGoUFrnkVcC+bAM4W2K8431SdQwhX87NoJV8zhGRh+hTUmP bcgCNQggmaTIxLOoo0RSF2esKS82Qf0QoiZtmoWUwjrS8eTJZxm/giy3FS8CnM6xq3H4 n3Wg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=eVSSolozDNzm2ld5ayce9w6Mm7MgFFlInyM+I63lxxk=; b=l9ehc8wGK/E+s0VcSpt+Zq0Tg1455gAXyn3qc36H0dMmqu4TQNRvpx/fIuGrMfWAhH aFZ3tOAKzeuofFVtLGjugQ+yzgNvkOmq+h3gyGrXEdt+StOnnb+f8A9MfLtKO9nbQvtg BSMsCmlQj8lKizuNfFABYF04bicjekoQF3SI5pthSCBF/IVF6tdRD+SuJygV6duzbbjc 0Cw3Ot4H/l6Q1DgwnDNHYmivN9SJqEi/mKIjXr8FL+6EonQR15HY4LdquA6uJ14cHfm8 FEmCTURYoaVVAabROFIoHZNcfs9ZF4I/WqVlO0bMqCMTnWSD0r9gcfpABfI2h0l6wxym Dedw== X-Gm-Message-State: APjAAAVjJCUSwNx7HjVdlPWLn0OHWce2DhTVl9zJEyqkPzynlMhKMmRN HR5A6Sgiw0Be2t57B/03JFd853QQ X-Google-Smtp-Source: APXvYqxSomvsSrV5dufsOjCE7wZZ1/HN7v4/kfNexYPKgqiHiUafJyz+jrmns5bcZ/As1R4CIOXRqw== X-Received: by 2002:adf:c7cf:: with SMTP id y15mr5086138wrg.54.1569362620185; Tue, 24 Sep 2019 15:03:40 -0700 (PDT) Received: from sblaptop.fritz.box ([188.192.248.87]) by smtp.gmail.com with ESMTPSA id l1sm3023808wrb.1.2019.09.24.15.03.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 24 Sep 2019 15:03:39 -0700 (PDT) From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Wed, 25 Sep 2019 00:03:02 +0200 Message-Id: <20190924220310.31157-8-andreas.rheinhardt@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190924220310.31157-1-andreas.rheinhardt@gmail.com> References: <20190924220310.31157-1-andreas.rheinhardt@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 07/15] avcodec/cavsdsp: Fix undefined left shifts of negative numbers X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Affected the ffmpeg-filter_colorkey FATE-test (but only if the C version of idct8_add is used and not e.g. the x86 SSE2 version). Signed-off-by: Andreas Rheinhardt --- The FFLSHIFT macro isn't used here, because the coefficients here look like weights, so that bit-shifting is actually not intended (but comes in handy when performing the calculations). libavcodec/cavsdsp.c | 52 ++++++++++++++++++++++---------------------- 1 file changed, 26 insertions(+), 26 deletions(-) diff --git a/libavcodec/cavsdsp.c b/libavcodec/cavsdsp.c index 90a67e910c..ba92121cc9 100644 --- a/libavcodec/cavsdsp.c +++ b/libavcodec/cavsdsp.c @@ -201,20 +201,20 @@ static void cavs_idct8_add_c(uint8_t *dst, int16_t *block, ptrdiff_t stride) src[0][0] += 8; for( i = 0; i < 8; i++ ) { - const int a0 = 3*src[i][1] - (src[i][7]<<1); - const int a1 = 3*src[i][3] + (src[i][5]<<1); - const int a2 = (src[i][3]<<1) - 3*src[i][5]; - const int a3 = (src[i][1]<<1) + 3*src[i][7]; + const int a0 = 3 * src[i][1] - 2 * src[i][7]; + const int a1 = 3 * src[i][3] + 2 * src[i][5]; + const int a2 = 2 * src[i][3] - 3 * src[i][5]; + const int a3 = 2 * src[i][1] + 3 * src[i][7]; - const int b4 = ((a0 + a1 + a3)<<1) + a1; - const int b5 = ((a0 - a1 + a2)<<1) + a0; - const int b6 = ((a3 - a2 - a1)<<1) + a3; - const int b7 = ((a0 - a2 - a3)<<1) - a2; + const int b4 = 2 * (a0 + a1 + a3) + a1; + const int b5 = 2 * (a0 - a1 + a2) + a0; + const int b6 = 2 * (a3 - a2 - a1) + a3; + const int b7 = 2 * (a0 - a2 - a3) - a2; - const int a7 = (src[i][2]<<2) - 10*src[i][6]; - const int a6 = (src[i][6]<<2) + 10*src[i][2]; - const int a5 = ((src[i][0] - src[i][4]) << 3) + 4; - const int a4 = ((src[i][0] + src[i][4]) << 3) + 4; + const int a7 = 4 * src[i][2] - 10 * src[i][6]; + const int a6 = 4 * src[i][6] + 10 * src[i][2]; + const int a5 = 8 * (src[i][0] - src[i][4]) + 4; + const int a4 = 8 * (src[i][0] + src[i][4]) + 4; const int b0 = a4 + a6; const int b1 = a5 + a7; @@ -231,20 +231,20 @@ static void cavs_idct8_add_c(uint8_t *dst, int16_t *block, ptrdiff_t stride) src[i][7] = (b0 - b4) >> 3; } for( i = 0; i < 8; i++ ) { - const int a0 = 3*src[1][i] - (src[7][i]<<1); - const int a1 = 3*src[3][i] + (src[5][i]<<1); - const int a2 = (src[3][i]<<1) - 3*src[5][i]; - const int a3 = (src[1][i]<<1) + 3*src[7][i]; - - const int b4 = ((a0 + a1 + a3)<<1) + a1; - const int b5 = ((a0 - a1 + a2)<<1) + a0; - const int b6 = ((a3 - a2 - a1)<<1) + a3; - const int b7 = ((a0 - a2 - a3)<<1) - a2; - - const int a7 = (src[2][i]<<2) - 10*src[6][i]; - const int a6 = (src[6][i]<<2) + 10*src[2][i]; - const int a5 = (src[0][i] - src[4][i]) << 3; - const int a4 = (src[0][i] + src[4][i]) << 3; + const int a0 = 3 * src[1][i] - 2 * src[7][i]; + const int a1 = 3 * src[3][i] + 2 * src[5][i]; + const int a2 = 2 * src[3][i] - 3 * src[5][i]; + const int a3 = 2 * src[1][i] + 3 * src[7][i]; + + const int b4 = 2 * (a0 + a1 + a3) + a1; + const int b5 = 2 * (a0 - a1 + a2) + a0; + const int b6 = 2 * (a3 - a2 - a1) + a3; + const int b7 = 2 * (a0 - a2 - a3) - a2; + + const int a7 = 4 * src[2][i] - 10 * src[6][i]; + const int a6 = 4 * src[6][i] + 10 * src[2][i]; + const int a5 = 8 * (src[0][i] - src[4][i]); + const int a4 = 8 * (src[0][i] + src[4][i]); const int b0 = a4 + a6; const int b1 = a5 + a7;