From patchwork Thu Jun 16 08:59:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thilo Borgmann X-Patchwork-Id: 36252 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:1a22:b0:84:42e0:ad30 with SMTP id cj34csp820122pzb; Thu, 16 Jun 2022 02:00:05 -0700 (PDT) X-Google-Smtp-Source: AGRyM1vjGMEpIXt3WtApBeFiEuytzjc/W7bCNdck0mQVwPnWzqqO5/MRThxuq0+KbUMV5lHXh23j X-Received: by 2002:a05:6402:3291:b0:42d:dd03:cbb1 with SMTP id f17-20020a056402329100b0042ddd03cbb1mr5007582eda.268.1655370005445; Thu, 16 Jun 2022 02:00:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1655370005; cv=none; d=google.com; s=arc-20160816; b=HpZViExahDXFITZsc6pn+YupZwqEeCsO/9y4FIu8Wj4LL+D8zyziQ62I3Zg7B0JSRz uleT424ddvfuezd2J8q/H2s8Hsil+JcQFqFOdFbp8/lwNdt7Fyz4QirGJGgtwvSQ3010 zqU7exEsvMyJe33CPjrSluDjERbTDADUGNnzuA9hkcaOHHgPWyOlARLJRPMD36aujH1B l8qdSmqV1ky1Yd/IIF/RnamidB4SnOUfofWxJl8nUkZgS7qF+WwOFW/TVEwvwUEJJa/e InGBciEGC3uMG6wS28ppb+gghWxo8JEz6+vqdaj3Vpf59as9CRejuiCpdimfT9bxdal8 KJjw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence:subject:from:to :content-language:mime-version:date:message-id:dkim-signature :delivered-to; bh=QlGPi+6iL2gizzP9Tzs9hgvY4yD5qBJFLaPXusOeZ+E=; b=Lp8D/n1OkOlDP8F4sEzCGoTnrG4vaRQG+EMrOuHuH5s6KCjzl5CdEhAgxw+fkHoQAA w8aFObHUUcHuA8ef4bUe0KkneTvEl2PyTRuimu89KsqQJzaIw1UyF7rX4nwCQZNpX9or EuRdYzLvlndVivZOx/NVXgrp1ObgSE7SaeCagLN+Y2QuAA+D8CkFXJ/Ud1GLMkX7Hcgc 0SBI3x8gPT4bc/o05+5cZ+8yuwLJeC5lkLWE1s64KdDWh3WMFV2TdVlcA/s/alyngZZI 4Y1fGKvNWIa1wGSqq6XftCX5gQBsB50y28t5iOpwnOvKmV59uEcAAs3CK/DG52O50J8r o3Dw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@mail.de header.s=mailde202009 header.b=ZZ9LVO5I; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=mail.de Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id h7-20020a170906530700b006feba25b350si1167826ejo.106.2022.06.16.02.00.04; Thu, 16 Jun 2022 02:00:05 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@mail.de header.s=mailde202009 header.b=ZZ9LVO5I; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=mail.de Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 87A4968B7B0; Thu, 16 Jun 2022 12:00:00 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from shout02.mail.de (shout02.mail.de [62.201.172.25]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5E86168B6FC for ; Thu, 16 Jun 2022 11:59:54 +0300 (EEST) Received: from postfix03.mail.de (postfix03.bt.mail.de [10.0.121.127]) by shout02.mail.de (Postfix) with ESMTP id A014EA351B for ; Thu, 16 Jun 2022 10:59:53 +0200 (CEST) Received: from smtp02.mail.de (smtp02.bt.mail.de [10.0.121.212]) by postfix03.mail.de (Postfix) with ESMTP id 8761D80299 for ; Thu, 16 Jun 2022 10:59:53 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=mail.de; s=mailde202009; t=1655369993; bh=2byVquCE0ItAKsHiycb6evWm2uGPoi2FgrMtFaZxn34=; h=Message-ID:Date:To:From:Subject:From:To:CC:Subject:Reply-To; b=ZZ9LVO5I27T1qofwmz1Bn4kj9z1UZO5+00EAstAoX8E0SUDlnKwsLuI1qcNaSp6eQ 2X9a+Xlg8/juWWtYvczAJ9CCq+9kvnVvZKDdAFfQqi1Kw4Gr4M7fxJSgSOryWStzfq Wrs0E2k63hLENzmBpOI+FtrY2F7OLUYcvdlstrruMqoscRmySTW7fkkm1MyeW+6HR8 9lncbmNU0rkDmz9hKBU2UJJwpqQVk3OMBlFEayMrnXQk9W5+ccZUJ6SSaUYcrRX8ZG z/0Po6f4+8iDbWhDkZprdPQ1IhaL0Kv8kWJ8mFfq2Zf4SFEZDU8XvmqTrIEMZ4bMxp DxZsSm1JOFKBg== Received: from [127.0.0.1] (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by smtp02.mail.de (Postfix) with ESMTPSA id 5DD58A0A5A for ; Thu, 16 Jun 2022 10:59:53 +0200 (CEST) Message-ID: Date: Thu, 16 Jun 2022 10:59:52 +0200 MIME-Version: 1.0 Content-Language: en-US To: FFmpeg development discussions and patches From: Thilo Borgmann X-purgate: clean X-purgate: This mail is considered clean (visit http://www.eleven.de for further information) X-purgate-type: clean X-purgate-Ad: Categorized by eleven eXpurgate (R) http://www.eleven.de X-purgate: This mail is considered clean (visit http://www.eleven.de for further information) X-purgate: clean X-purgate-size: 2781 X-purgate-ID: 154282::1655369993-0000737C-07D69D86/0/0 Subject: [FFmpeg-devel] [PATCH] lavc/dovi_rpu: Fix UB for possible left shift of negative values X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: fYi2UA5wOwdZ Hi, avoid left-shifting possibly negative values in case of RPU_COEFF_FLOAT. Changed RPU_COEFF_FIXED as well for consistency. -Thilo From 7eca2659b588c404f058eccef478c889555957fd Mon Sep 17 00:00:00 2001 From: Michael Goulet Date: Thu, 16 Jun 2022 10:25:28 +0200 Subject: [PATCH] lavc/dovi_rpu: Fix UB for possible left shift of negative values It is undefined to left-shift a negative value in the RPU_COEFF_FLOAT case. Changed the RPU_COEFF_FIXED case as well for consistency although its all positive values there. --- libavcodec/dovi_rpu.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libavcodec/dovi_rpu.c b/libavcodec/dovi_rpu.c index a87562c8a3..833ce9e705 100644 --- a/libavcodec/dovi_rpu.c +++ b/libavcodec/dovi_rpu.c @@ -153,7 +153,7 @@ static inline uint64_t get_ue_coef(GetBitContext *gb, const AVDOVIRpuDataHeader case RPU_COEFF_FIXED: ipart = get_ue_golomb_long(gb); fpart.u32 = get_bits_long(gb, hdr->coef_log2_denom); - return (ipart << hdr->coef_log2_denom) + fpart.u32; + return ipart * (1 << hdr->coef_log2_denom) + fpart.u32; case RPU_COEFF_FLOAT: fpart.u32 = get_bits_long(gb, 32); @@ -172,7 +172,7 @@ static inline int64_t get_se_coef(GetBitContext *gb, const AVDOVIRpuDataHeader * case RPU_COEFF_FIXED: ipart = get_se_golomb_long(gb); fpart.u32 = get_bits_long(gb, hdr->coef_log2_denom); - return (ipart << hdr->coef_log2_denom) + fpart.u32; + return ipart * (1 << hdr->coef_log2_denom) + fpart.u32; case RPU_COEFF_FLOAT: fpart.u32 = get_bits_long(gb, 32);