From patchwork Mon Jul 1 11:08:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Carl Eugen Hoyos X-Patchwork-Id: 13789 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 20B7A449A39 for ; Mon, 1 Jul 2019 14:08:54 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E3921689F2B; Mon, 1 Jul 2019 14:08:53 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-io1-f53.google.com (mail-io1-f53.google.com [209.85.166.53]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 82896689F27 for ; Mon, 1 Jul 2019 14:08:47 +0300 (EEST) Received: by mail-io1-f53.google.com with SMTP id s7so27763384iob.11 for ; Mon, 01 Jul 2019 04:08:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=Npn4sAPFMfMGd1lk4DYnq7pN7fodm8Vy34D5QHVKDz4=; b=Sroekdvcs8TWMrEMOKfFhK4+F9ULIc5C67pF3iB+VhJOX5JvFJmoKwwZcj7CYvykK4 HpfObvVAu2ncF0Nl+oaM3mfht03EBALcxMdPcK7IZBy9rJxfdPGItbTQv5DFl5dNfiLs SIvJWqnApFttWEnIjkV8mMaLAUEaQzImBbSbYzJAkcZp8HpIq9FpZWXNT9NG2DFW9upp Pj+vpOeL/qH82FKN+zUG42G/sz9U81UEgrW5X78XpZ62vCmyp35D0OkkaQPC04S3YoYT SpD4gfwAL9/HNIt/OE6Ebx+5ohGw2Fet2iz5g2jdWmqa7deers/ZjAlyyP9Girmpw3ML ajMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=Npn4sAPFMfMGd1lk4DYnq7pN7fodm8Vy34D5QHVKDz4=; b=tGZJsomuh9Ja8RogibeJrWQV53FpHVZrkgvXmPKmUjSS8q3/gT9JM0xpoJvHAYYjDL pOEubelQlxAqwzS+dlBKmK4fid3F9KLNwRzJ6tOLLXL2GYF7azCWZ1JTLjgK+U/Mf4o2 wHCp5dI6k6qiKzeMiqpUAO+KnhFvTTX1IEB8WZ6cTZsPqeB6pBmM8iWfELU5bWqsGDoa Q6cY00/n4YfgtfcqP8ZUqAF8PPIZGuI+Kv+EDTYklJA52+lXI54AS43QT8VvBBZpRACa bflEC6tJg0vQkOIrVK/ncypa6x8QnKX3/+d8Yo7VjikMoP39epaPEbrnrsz9/6fWSJAH C2FA== X-Gm-Message-State: APjAAAUgNNOgBm/fAVtMvFZGn9lndmTikcImYYvVw+QfGqQq+47h9RAy nBbmgrGJex5vOl1igjNvJct1vqLAwzBfeJrbKdNgbQ== X-Google-Smtp-Source: APXvYqwgxgSVWRTVDU+iWrmmZrB2Rq8FXSclDtl8sUiVYkJaEw8TweGVv8V5g/FT4XZXy1hemcbsUJJIVDdkJVd/aL4= X-Received: by 2002:a02:878a:: with SMTP id t10mr27973826jai.112.1561979325974; Mon, 01 Jul 2019 04:08:45 -0700 (PDT) MIME-Version: 1.0 From: Carl Eugen Hoyos Date: Mon, 1 Jul 2019 13:08:34 +0200 Message-ID: To: FFmpeg development discussions and patches Subject: [FFmpeg-devel] [PATCH]lavc/r210enc: Fix undefined behaviour encoding r10k X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Hi! Attached patch fixes an invalid left shift reported in ticket #7982. Please comment, Carl Eugen From cfe1ae9a1e95dee72ff0e0a45ce8caab98a1bd0b Mon Sep 17 00:00:00 2001 From: Carl Eugen Hoyos Date: Mon, 1 Jul 2019 13:06:02 +0200 Subject: [PATCH] lavc/r210enc: Fix undefined behaviour encoding r10k. Fixes the following ubsan error: libavcodec/r210enc.c:69:28: runtime error: left shift of 522 by 22 places cannot be represented in type 'int' Fixes ticket #7982. --- libavcodec/r210enc.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/libavcodec/r210enc.c b/libavcodec/r210enc.c index 02412f3684..95049089bc 100644 --- a/libavcodec/r210enc.c +++ b/libavcodec/r210enc.c @@ -60,9 +60,9 @@ static int encode_frame(AVCodecContext *avctx, AVPacket *pkt, uint16_t *srcb = (uint16_t *)srcb_line; for (j = 0; j < avctx->width; j++) { uint32_t pixel; - uint16_t r = *srcr++; - uint16_t g = *srcg++; - uint16_t b = *srcb++; + uint32_t r = *srcr++; + uint32_t g = *srcg++; + uint32_t b = *srcb++; if (avctx->codec_id == AV_CODEC_ID_R210) pixel = (r << 20) | (g << 10) | b; else -- 2.22.0