From patchwork Sun Sep 15 20:01:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 15079 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 4D5BA448D37 for ; Sun, 15 Sep 2019 23:10:06 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 21E066881E3; Sun, 15 Sep 2019 23:10:06 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f65.google.com (mail-wr1-f65.google.com [209.85.221.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4C15C68803C for ; Sun, 15 Sep 2019 23:09:59 +0300 (EEST) Received: by mail-wr1-f65.google.com with SMTP id q17so32077376wrx.10 for ; Sun, 15 Sep 2019 13:09:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=lP96w66MIuoJjmRTZslKVE6BGvsp9NQfuXVq3GkaEOU=; b=Hu17IekBwqkoHsOq1MUqjkGcrNMU+ZV/F9STG0Tq3dPNot0uQgHZp4xI9AdlDHoQ+p mUopGvRTwX4ZrODuHvgthatMzhVG/qv+m6R92dLUgKim32AJaQ5JjU+cTzLSqL9n+lAl 2fIhdStFgwEJxvOFFOsCZ0Q0ke9u4X9pQ8xYSeucTmP5d9Z38ghx02Y9c1APjJABirmS 3+wOkHwW2rqnb6gOoBBetvmv8qqNKuN25Qi+hMEulGPx8oz1HaRQ5oRu6oetFpK7z/6+ 308bgcbSKfwaJRVFxUnjKFjnNnTLAWw/CBJNiVeO4RCuK46BsgQx73OGsFtDVkcAIqun 4tEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=lP96w66MIuoJjmRTZslKVE6BGvsp9NQfuXVq3GkaEOU=; b=kDFUmIx/DjNPSurFxO5WFkJFM3CdJDJmE6mMmTynIY+YCGthA9FaYzIc+RdpVZw9Xk Jm1ZoajDQXLmPS6IQLoJTOcd7ECPaWUjB0byUD09p5DfzHTThMofbPyP7YHk/cq9IMlt PFWEMTfc25l1kvbWEFo8h2mWWLpUJCp3MwLVJSo0f5uIh+O/jIlitSyRZhFrmqAe9jMl e4Dc/s/S7ykq1BezKFwVEksjWrmhXyX++MBbRmicLU1E/ZD9m8VbrNWIt0E3/7/YIJg8 sHnKMFCIJM8muTp0brrJHg+2X/ovC765JshbycTxI1wwdHDWRFvIP256D3A5/vYRZdiK cigQ== X-Gm-Message-State: APjAAAUgEAFBUgsPewsjvLUR7HHYphISQSxLIiZjodrjEN4rqf5q01zb 1Xwolv6szrBPSymVZOEqJJagjKtrSXg= X-Google-Smtp-Source: APXvYqyxQOg+0nP1S+bo+yM4BlxEVeah8tnBSaconXl6hmAK150WbZDeUIBW5UQhxG+IvUSTJcl/6Q== X-Received: by 2002:adf:dcc1:: with SMTP id x1mr1882111wrm.332.1568577706013; Sun, 15 Sep 2019 13:01:46 -0700 (PDT) Received: from localhost.localdomain (ipbcc0f857.dynamic.kabel-deutschland.de. [188.192.248.87]) by smtp.gmail.com with ESMTPSA id m62sm12767943wmm.35.2019.09.15.13.01.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 15 Sep 2019 13:01:45 -0700 (PDT) From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Sun, 15 Sep 2019 22:01:20 +0200 Message-Id: <20190915200120.3784-1-andreas.rheinhardt@gmail.com> X-Mailer: git-send-email 2.21.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] avcodec/ttaenc: Fix undefined shift X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" ttaenc contained (1 << unary) - 1 as an argument for a function expecting an unsigned int. unary can be as big as 31 in this case. The type of the shift and the whole expression is int, because 1 fits into an integer, so that the behaviour is undefined if unary == 31 as the result of the shift can't be represented in an int §. Subtraction by 1 (which makes the result of the whole expression representable in an int) doesn't change that this is undefined (it usually leads to signed integer overflow which is undefined, too). The solution is simple: Make 1 unsigned to change the type of the whole expression to unsigned int (as the function expects anyway). Fixes ticket #8153. §: This of course presupposes the common int range of -2^31..2^31-1 Signed-off-by: Andreas Rheinhardt --- libavcodec/ttaenc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavcodec/ttaenc.c b/libavcodec/ttaenc.c index 3cc54d78c5..08a0d0483a 100644 --- a/libavcodec/ttaenc.c +++ b/libavcodec/ttaenc.c @@ -164,7 +164,7 @@ pkt_alloc: put_bits(&pb, 31, 0x7FFFFFFF); unary -= 31; } else { - put_bits(&pb, unary, (1 << unary) - 1); + put_bits(&pb, unary, (1U << unary) - 1); unary = 0; } } while (unary);