From patchwork Wed Dec 5 21:21:12 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: matthew.w.fearnley@gmail.com X-Patchwork-Id: 11294 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id B010D44C931 for ; Thu, 6 Dec 2018 01:53:29 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6269068A5C8; Thu, 6 Dec 2018 01:53:21 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f66.google.com (mail-wr1-f66.google.com [209.85.221.66]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5209D68A0A0 for ; Wed, 5 Dec 2018 23:26:50 +0200 (EET) Received: by mail-wr1-f66.google.com with SMTP id r10so21163131wrs.10 for ; Wed, 05 Dec 2018 13:26:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=0Mpa6q5chwtda780v+4w8fbGa3SshtEvfalqExZLexA=; b=HzV2EX9XK+9/TKexOhhbTF1gJ0tIrbuuKb+Pu7Xy6MZ/jPLnfwOT2vtmzjPbxzwgCM UaZ03JXEFzP12P1rCEZxPzMA7EhaIE3LPvIFHoZfGrR4pwhIdoURiqKxWoENh9rdiQkg pG1Bj6r+It+CXqBBke7QWPtuXdISqK+3k9kOQ08LKbiocZkl3IDumu1SBU6yCWZqYVA8 80BDfmSVFzZITfqOMRjFP42b2+xYK+G4I8+VYXbInZbow7lXSvDhVW8uBxpK6LCCT5gK vCRzpURJEcsDXWskbBVSYdSry2Fd59huEQbpdRb+kd0dLiDZgL4iwsfxZtHDWAcriLPh L9/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=0Mpa6q5chwtda780v+4w8fbGa3SshtEvfalqExZLexA=; b=b90PfNEqbr5LiWHUDuY3zGYkYcDLPv4Xt5pzs+JVqtpikp78E1kbeNEMzOcqsU0Su1 +MGT9jpmebjIz51XSfsERzmTKqLaHiXKOaQhAOdTodcESwgsSjEZ81mRYim77Ch2SDBo v+z6UljuhR8cR2mSQyyx2nWaPBpX5zZwYgZB6JnYqjLUpBO9NWrXP294NB5q9BiRHyeK aOQsOxZzvtg/WBaq2E3U1OS0D9ZKhxRkF0BMbtobFvzZCurdoOTJkCbgK7ZhpdGZomQ9 h486buSeCD6AX+d4r9+4swo4+C/oUhPaTq4hMRlegvZwcnKWmQd3HEayrxDYlY3B8lY2 ci2A== X-Gm-Message-State: AA+aEWYYOxYx6T3wSmEzSazoBhTANO8yZkfKTNzKskWKiFQ3nJBtFEHl KIbVRprtVUfkM12XJ8JMMD0dKbDv X-Google-Smtp-Source: AFSGD/XkJbTL/R7OvyChOD2gSvyA1ceAocxtS6QJSutW1UhIxBfguJi7IMly3HokiJgtubOJRiOo8A== X-Received: by 2002:adf:a357:: with SMTP id d23mr24047289wrb.195.1544044897322; Wed, 05 Dec 2018 13:21:37 -0800 (PST) Received: from localhost.localdomain (cpc131498-bagu18-2-0-cust88.know.cable.virginm.net. [86.9.33.89]) by smtp.gmail.com with ESMTPSA id v5sm28366965wrn.71.2018.12.05.13.21.36 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 05 Dec 2018 13:21:36 -0800 (PST) From: matthew.w.fearnley@gmail.com To: ffmpeg-devel@ffmpeg.org Date: Wed, 5 Dec 2018 21:21:12 +0000 Message-Id: <20181205212112.30723-1-matthew.w.fearnley@gmail.com> X-Mailer: git-send-email 2.17.1 X-Mailman-Approved-At: Thu, 06 Dec 2018 01:53:20 +0200 Subject: [FFmpeg-devel] [PATCH] libavcodec/zmbvenc.c: don't allow motion estimation out of range. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Matthew Fearnley MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Matthew Fearnley The maximum allowable range for ZMBV motion estimation is [-64..63], since the dx,dy values are each stored in the upper 7 bits of a signed char. (Previously, the range was capped in the code to 127, resulting in an effective range of [-127..126]) Also fix a range error in the zmbv_me() for-loops ('<' instead of '<='), which made the limit asymmetrical [-N..N-1], and also prevented it from reaching the blocks touching the bottom/right edges. The range is now more symmetrical [-N..N], although this requires separate range caps of 64 and 63 for negative and positive dx,dy. Practically, this patch fixes graphical glitches e.g. when reencoding the Commander Keen sample video with me_range 65 or higher: ffmpeg -i keen4e_000.avi -c:v zmbv -me_range 65 keen4e_me65.avi (Glitches are worse with higher me_range values up to 127.) --- libavcodec/zmbvenc.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/libavcodec/zmbvenc.c b/libavcodec/zmbvenc.c index 4d9147657d..300a53b08e 100644 --- a/libavcodec/zmbvenc.c +++ b/libavcodec/zmbvenc.c @@ -45,7 +45,7 @@ typedef struct ZmbvEncContext { AVCodecContext *avctx; - int range; + int lrange, urange; uint8_t *comp_buf, *work_buf; uint8_t pal[768]; uint32_t pal2[256]; //for quick comparisons @@ -101,8 +101,8 @@ static int zmbv_me(ZmbvEncContext *c, uint8_t *src, int sstride, uint8_t *prev, bh = FFMIN(ZMBV_BLOCK, c->avctx->height - y); bv = block_cmp(c, src, sstride, prev, pstride, bw, bh, xored); if(!bv) return 0; - for(ty = FFMAX(y - c->range, 0); ty < FFMIN(y + c->range, c->avctx->height - bh); ty++){ - for(tx = FFMAX(x - c->range, 0); tx < FFMIN(x + c->range, c->avctx->width - bw); tx++){ + for(ty = FFMAX(y - c->lrange, 0); ty <= FFMIN(y + c->urange, c->avctx->height - bh); ty++){ + for(tx = FFMAX(x - c->lrange, 0); tx <= FFMIN(x + c->urange, c->avctx->width - bw); tx++){ if(tx == x && ty == y) continue; // we already tested this block dx = tx - x; dy = ty - y; @@ -285,9 +285,13 @@ static av_cold int encode_init(AVCodecContext *avctx) c->curfrm = 0; c->keyint = avctx->keyint_min; - c->range = 8; - if(avctx->me_range > 0) - c->range = FFMIN(avctx->me_range, 127); + + // Motion estimation range: maximum distance is -64..63 + c->lrange = c->urange = 8; + if(avctx->me_range > 0){ + c->lrange = FFMIN(avctx->me_range, 64); + c->urange = FFMIN(avctx->me_range, 63); + } if(avctx->compression_level >= 0) lvl = avctx->compression_level;