From patchwork Tue Sep 29 15:52:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 22656 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 98E1F44B682 for ; Tue, 29 Sep 2020 18:52:59 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 78ACD68A646; Tue, 29 Sep 2020 18:52:59 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f66.google.com (mail-wm1-f66.google.com [209.85.128.66]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5A69F687FAE for ; Tue, 29 Sep 2020 18:52:53 +0300 (EEST) Received: by mail-wm1-f66.google.com with SMTP id a9so5367600wmm.2 for ; Tue, 29 Sep 2020 08:52:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=sRidZ1lReRuefsfncxW1+6gaD0WG1IcXdYGY5ZeuEm0=; b=MAHCfr1KOQG3lVJHiAEvc0Kvpn3n4GRBDwS9jvkGi/wb5u7Ia57uza9yeOY2FOWfg6 lII4UBaMo7/2y0d5JB3cOARd3aIEhra9oYdaat0vuy50eGrdr9L32aSjTfVZyvc050wq uv8tk2y2520/vaDf8zvCawBClBriIFL8qNe/CfUd5fzMPgVV257UorE9beqkaKAsrMVJ kBHCCgRlnSSc6a8vVPZ4SJSHnYSU+aqB9d/R3e5LDS+pYWc3OuMwpp15i6BnKRsrgbrL LDLtlOGkormf+YTYqAfzPC8ou1/q5K1O8ArD81e3jpBfDtcEHFM4Xx3AM+jMXJ7QGMoO POUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=sRidZ1lReRuefsfncxW1+6gaD0WG1IcXdYGY5ZeuEm0=; b=gU4KDAi2qTGBi+fM12vNZUVnNdCLHJhRewBWSuSMPGIljjFzrNJHilY8YZVnn3i+sT S4cdssmrO7n0F0LFLE4uxFbjph0swrohbnBQwC4IYTAm65T/Rth85C6UYu/MuoceOqkT JQov70RcySiMOWPPizE41OqlQdmzM2JwkzBEV7wBhbaFlAIpKqGg1X34xpL+gVv6t99q IYWybPySpE7h4jm9CPnV7f0c5FwQIODYzI92HF0/swjSkfKGt6bBUAyhy9X/soXcX2y0 cLHBi6K41xZw9ER3wL9u3iIJTn2iAuPhDQjynU2mHZ618myhu+hoiWe0MnNnFXVMrBhz 1O4A== X-Gm-Message-State: AOAM533+ID1wpnUgmNTeXShHlX+oS8VNpqQHB2n8GlcUJ5Op60xqwf8X 8qz6oq7uS/ZQZLtS4yp3wcMcOp7+PkA= X-Google-Smtp-Source: ABdhPJy6cEx6EZurWKP5nh+NT2ahe09dKLL5oCS1eR24ETcOl4+dkybVz0uVFcEnUG5n8+kcXlqrCw== X-Received: by 2002:a1c:b7d7:: with SMTP id h206mr5643717wmf.159.1601394772480; Tue, 29 Sep 2020 08:52:52 -0700 (PDT) Received: from sblaptop.fritz.box (ipbcc1aa4b.dynamic.kabel-deutschland.de. [188.193.170.75]) by smtp.gmail.com with ESMTPSA id m4sm7286674wro.18.2020.09.29.08.52.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 29 Sep 2020 08:52:51 -0700 (PDT) From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Tue, 29 Sep 2020 17:52:43 +0200 Message-Id: <20200929155244.924968-1-andreas.rheinhardt@gmail.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/2] avcodec/dvenc: Fix undefined left shift of negative numbers X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" The earlier code was based on the assumption that AVFrame.linesize can not be negative. Fixes ticket #8280. Signed-off-by: Andreas Rheinhardt --- libavcodec/dvenc.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/libavcodec/dvenc.c b/libavcodec/dvenc.c index 3e98d1b38c..21a8b841f6 100644 --- a/libavcodec/dvenc.c +++ b/libavcodec/dvenc.c @@ -215,8 +215,8 @@ static av_always_inline int dv_guess_dct_mode(DVVideoContext *s, uint8_t *data, if (s->avctx->flags & AV_CODEC_FLAG_INTERLACED_DCT) { int ps = s->ildct_cmp(NULL, data, NULL, linesize, 8) - 400; if (ps > 0) { - int is = s->ildct_cmp(NULL, data, NULL, linesize << 1, 4) + - s->ildct_cmp(NULL, data + linesize, NULL, linesize << 1, 4); + int is = s->ildct_cmp(NULL, data, NULL, linesize * 2, 4) + + s->ildct_cmp(NULL, data + linesize, NULL, linesize * 2, 4); return ps > is; } } @@ -511,7 +511,7 @@ static av_always_inline int dv_init_enc_block(EncBlockInfo* bi, uint8_t *data, i if (data) { if (DV_PROFILE_IS_HD(s->sys)) { - s->get_pixels(blk, data, linesize << bi->dct_mode); + s->get_pixels(blk, data, linesize * (1 << bi->dct_mode)); s->fdct[0](blk); } else { bi->dct_mode = dv_guess_dct_mode(s, data, linesize); @@ -860,7 +860,7 @@ static int dv_encode_video_segment(AVCodecContext *avctx, void *arg) qnos[mb_index] = DV_PROFILE_IS_HD(s->sys) ? 1 : 15; - y_ptr = s->frame->data[0] + ((mb_y * s->frame->linesize[0] + mb_x) << 3); + y_ptr = s->frame->data[0] + (mb_y * s->frame->linesize[0] + mb_x) * 8; linesize = s->frame->linesize[0]; if (s->sys->height == 1080 && mb_y < 134) @@ -874,12 +874,12 @@ static int dv_encode_video_segment(AVCodecContext *avctx, void *arg) if ((s->sys->pix_fmt == AV_PIX_FMT_YUV420P) || (s->sys->pix_fmt == AV_PIX_FMT_YUV411P && mb_x >= (704 / 8)) || (s->sys->height >= 720 && mb_y != 134)) { - y_stride = s->frame->linesize[0] << (3*!enc_blk->dct_mode); + y_stride = s->frame->linesize[0] * (1 << (3*!enc_blk->dct_mode)); } else { y_stride = 16; } y_ptr = s->frame->data[0] + - ((mb_y * s->frame->linesize[0] + mb_x) << 3); + (mb_y * s->frame->linesize[0] + mb_x) * 8; linesize = s->frame->linesize[0]; if (s->sys->video_stype == 4) { /* SD 422 */ @@ -898,17 +898,17 @@ static int dv_encode_video_segment(AVCodecContext *avctx, void *arg) enc_blk += 4; /* initializing chrominance blocks */ - c_offset = (((mb_y >> (s->sys->pix_fmt == AV_PIX_FMT_YUV420P)) * s->frame->linesize[1] + - (mb_x >> ((s->sys->pix_fmt == AV_PIX_FMT_YUV411P) ? 2 : 1))) << 3); + c_offset = ((mb_y >> (s->sys->pix_fmt == AV_PIX_FMT_YUV420P)) * s->frame->linesize[1] + + (mb_x >> ((s->sys->pix_fmt == AV_PIX_FMT_YUV411P) ? 2 : 1))) * 8; for (j = 2; j; j--) { uint8_t *c_ptr = s->frame->data[j] + c_offset; linesize = s->frame->linesize[j]; - y_stride = (mb_y == 134) ? 8 : (s->frame->linesize[j] << (3*!enc_blk->dct_mode)); + y_stride = (mb_y == 134) ? 8 : (s->frame->linesize[j] * (1 << (3*!enc_blk->dct_mode))); if (s->sys->pix_fmt == AV_PIX_FMT_YUV411P && mb_x >= (704 / 8)) { uint8_t *d; uint8_t *b = scratch; for (i = 0; i < 8; i++) { - d = c_ptr + (linesize << 3); + d = c_ptr + linesize * 8; b[0] = c_ptr[0]; b[1] = c_ptr[1]; b[2] = c_ptr[2]; From patchwork Tue Sep 29 15:52:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 22657 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id F171E44B79E for ; Tue, 29 Sep 2020 18:53:27 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D875B68A98A; Tue, 29 Sep 2020 18:53:27 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f44.google.com (mail-wr1-f44.google.com [209.85.221.44]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id DB7BE6805A5 for ; Tue, 29 Sep 2020 18:53:21 +0300 (EEST) Received: by mail-wr1-f44.google.com with SMTP id o5so5952286wrn.13 for ; Tue, 29 Sep 2020 08:53:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/bn9qBfOfqgVMmMMazO3O+1J+Q+uJDavABGPOW/92lk=; b=u0Vn71imA870SwcfNIlo1tKUJOa78zG3tcBV4g7fO2EqPIqWHiV3KOscfVYvkSpuWa wz5+GduJOFlbCwNEUxDLhof7kXpFtvwh9pHsdaI93uRvb8d++QaMZ5X2GSjUiPRt/e4a SNVOzCIl7/XHZlLeFAR33Fn09NDgHYl0ulc2z4HZNuIh0tQC/4jfnGDsoIpu+ZUzHV8D RdNI1Q2xBDEwhCq6OfBap66lXQR68mwvMrOcD4L8Og7v8KNU+uf5afaRDn23M2KEpcJc Qsfj+3BQ4ZTJZ+d5qldjHSYyir3hWPsDBP/B9uuIRFxa2vNB50sfVQTY0kIXo1QNheAg koVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/bn9qBfOfqgVMmMMazO3O+1J+Q+uJDavABGPOW/92lk=; b=LzrnUT9MHE4Co9A51AjeaALKMiD0/a5MMUyszw7dyfG+iWakL9sT0sSgtQjphY+NGG 3H8mXWC3qy4rI8h5lva0t3FYeudeZlzi8ZP9TR9kefkthOP0rUPe9et+yUiWrZCRsM0V lUYtfEOhHnUbG0MpnABk7/e7o4TILNAq4aBxaib/pmoj0H63xIXGY5NO9Fc4cNtLOJCi nVH8LZDqZts5hrWg8jHYEhr+/uZAInJJEgwFaaZaEfAy2W98Utm8DQFm6ApFNuUPatRN yv1cIBJutynMInY0UemHZFCjr7MrVwqMGeWqouezlIl8NtPNObmnRGrrw3htIzunUW4V whNw== X-Gm-Message-State: AOAM530qmPG7bFdt0tFk74bKJIyHFtPB/BM8t7YrP2nb5iSwr70+TsCF YpAbmWcvZXWom/0l/5lHIxdf51uv0ZI= X-Google-Smtp-Source: ABdhPJwnl7+XbsV1mvzT4VHchAMoMwSshOLa+e+FSFjecOs9E4WrHIz8EtRW7s8KvcQSzNiwon/p6w== X-Received: by 2002:adf:f885:: with SMTP id u5mr4959024wrp.382.1601394801127; Tue, 29 Sep 2020 08:53:21 -0700 (PDT) Received: from sblaptop.fritz.box (ipbcc1aa4b.dynamic.kabel-deutschland.de. [188.193.170.75]) by smtp.gmail.com with ESMTPSA id m4sm7286674wro.18.2020.09.29.08.53.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 29 Sep 2020 08:53:20 -0700 (PDT) From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Tue, 29 Sep 2020 17:52:44 +0200 Message-Id: <20200929155244.924968-2-andreas.rheinhardt@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200929155244.924968-1-andreas.rheinhardt@gmail.com> References: <20200929155244.924968-1-andreas.rheinhardt@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/2] swresample/audioconvert: Fix left shift of negative value X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Fixes ticket #8219. Signed-off-by: Andreas Rheinhardt --- libswresample/audioconvert.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libswresample/audioconvert.c b/libswresample/audioconvert.c index d21fc8ef42..89ee7bfae3 100644 --- a/libswresample/audioconvert.c +++ b/libswresample/audioconvert.c @@ -59,7 +59,7 @@ CONV_FUNC(AV_SAMPLE_FMT_FLT, float , AV_SAMPLE_FMT_U8 , (*(const uint8_t*)pi - CONV_FUNC(AV_SAMPLE_FMT_DBL, double , AV_SAMPLE_FMT_U8 , (*(const uint8_t*)pi - 0x80)*(1.0 / (1<<7))) CONV_FUNC(AV_SAMPLE_FMT_U8 , uint8_t, AV_SAMPLE_FMT_S16, (*(const int16_t*)pi>>8) + 0x80) CONV_FUNC(AV_SAMPLE_FMT_S16, int16_t, AV_SAMPLE_FMT_S16, *(const int16_t*)pi) -CONV_FUNC(AV_SAMPLE_FMT_S32, int32_t, AV_SAMPLE_FMT_S16, *(const int16_t*)pi<<16) +CONV_FUNC(AV_SAMPLE_FMT_S32, int32_t, AV_SAMPLE_FMT_S16, *(const int16_t*)pi * (1 << 16)) CONV_FUNC(AV_SAMPLE_FMT_S64, int64_t, AV_SAMPLE_FMT_S16, (uint64_t)(*(const int16_t*)pi)<<48) CONV_FUNC(AV_SAMPLE_FMT_FLT, float , AV_SAMPLE_FMT_S16, *(const int16_t*)pi*(1.0f/ (1<<15))) CONV_FUNC(AV_SAMPLE_FMT_DBL, double , AV_SAMPLE_FMT_S16, *(const int16_t*)pi*(1.0 / (1<<15)))