From patchwork Thu Oct 10 09:47:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 15672 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 909EE446890 for ; Thu, 10 Oct 2019 12:55:47 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6E0D16881A8; Thu, 10 Oct 2019 12:55:47 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f67.google.com (mail-wm1-f67.google.com [209.85.128.67]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 36293688086 for ; Thu, 10 Oct 2019 12:55:41 +0300 (EEST) Received: by mail-wm1-f67.google.com with SMTP id 7so6232586wme.1 for ; Thu, 10 Oct 2019 02:55:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=V4ue1pU7r/Xzv4nsp5FfI5kzqOijVVls6u6l9mlcr+w=; b=euj/pQkhJ4OkfFP+OnQJBKe6OxQ0SYmKZP0eE5dO4x9kGezWgNsC64QrIvcJ5msyzM Ys53HiYsp6wmAlv5FHYD59FaGvUyv2yxRTiyPCVXDmXApSRDamVyfqzMbMqhZ5HnMsIj 3Qeo88mmPzYFSqwfyuI2/QsD+HxLoX87lpJ5WgPcyMO4bl4L6HnI09Qv0/Eqtusyp8Q9 T5QK2zm9EiTdK5FTHNgmyxZu9Zm2FQebpuhiuMlha0aFoXqlhoHm1jh8SvAlifk7rwx+ NncSeKdet8ub9NcKKpmXkZyV6o7Fqjxjn7H9blPxQqnx9h1pTf8MbiC8q4Z1L4ubj8+p nSpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=V4ue1pU7r/Xzv4nsp5FfI5kzqOijVVls6u6l9mlcr+w=; b=hXYq/nIBceQu52ZAz6g4QRBSGRpwvdPeb5g6vUv0sVALVvzBW1WPrgbXOscAEhlOq2 1ov3VzgOdOuad5270ZZreR2A/WXiBdk0ZmYwiQJ1uUdKa85aWZq2/SOIEOutD2CItrMY Jl6/4NDtLwzbACiEkHzuu/8Ue/5t5rfOwjwvSzZkVdqUwGGnrjJ5ESVSmFetahzhJwl8 GZgEdmkyv0pSd/x/1E1CWPTBNesRxfGeuYS4mbdSScWADp0K1Iu3mahPEKKqxk3O/S+k Jpl9ed287/dm2KmC+l0MN6mp0gXHEVrmvJJfpl4kbEqGM4Pm77mAVL0gzxb89ihNQs01 qZUw== X-Gm-Message-State: APjAAAWmFOoLryhWNTvRZCaQX/XFmyV5p5sWGPLEgQvD8ORGSb3cMXTy 3Wo5b0hAXhWBdf+M4Jwxk0ixjybb X-Google-Smtp-Source: APXvYqzLJW0xuSEqhGZzurnzf1TbzuQ+dhTgiH0jD77oI/kWr/nPLktWIJkqA+Dy4ZTI/Jpd275UOg== X-Received: by 2002:a05:600c:24c9:: with SMTP id 9mr7227614wmu.174.1570700877378; Thu, 10 Oct 2019 02:47:57 -0700 (PDT) Received: from sblaptop.fritz.box (ipbcc08937.dynamic.kabel-deutschland.de. [188.192.137.55]) by smtp.gmail.com with ESMTPSA id q22sm4510809wmj.5.2019.10.10.02.47.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Oct 2019 02:47:56 -0700 (PDT) From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Thu, 10 Oct 2019 11:47:41 +0200 Message-Id: <20191010094741.814-1-andreas.rheinhardt@gmail.com> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] avcodec/zmbvenc: Correct offset in buffer X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" zmbvenc allocates a buffer for a picture with padding on all four sides: The stride is choosen so large that it already contains padding on the right; the height also includes padding rows. The padding on the right of each row is also reused as padding for the left of the next row. So one still needs to add padding on the left for the first row. This is done by offsetting the actual pointer used to access the picture from the pointer returned by av_mallocz and the formula for this offset was wrong, because it ignored that a pixel can take more than one byte when calculating the offset resulting from the left padding of the first row. This fixes accesses outside of the allocated buffer that were reported in tickets #7980 and #7994. No writes were ever attempted outside of the buffer. Signed-off-by: Andreas Rheinhardt --- libavcodec/zmbvenc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavcodec/zmbvenc.c b/libavcodec/zmbvenc.c index 0e22ce687f..319381dd48 100644 --- a/libavcodec/zmbvenc.c +++ b/libavcodec/zmbvenc.c @@ -409,7 +409,7 @@ static av_cold int encode_init(AVCodecContext *avctx) */ c->pstride = FFALIGN((avctx->width + c->lrange) * c->bypp, 16); prev_size = FFALIGN(c->lrange * c->bypp, 16) + c->pstride * (c->lrange + avctx->height + c->urange); - prev_offset = FFALIGN(c->lrange, 16) + c->pstride * c->lrange; + prev_offset = FFALIGN(c->lrange * c->bypp, 16) + c->pstride * c->lrange; if (!(c->prev_buf = av_mallocz(prev_size))) { av_log(avctx, AV_LOG_ERROR, "Can't allocate picture.\n"); return AVERROR(ENOMEM);