From patchwork Thu Jan 23 09:16:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Robert Deibel X-Patchwork-Id: 17480 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id E7CAF448582 for ; Thu, 23 Jan 2020 12:26:23 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D3DCE68B4E9; Thu, 23 Jan 2020 12:26:23 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f65.google.com (mail-wm1-f65.google.com [209.85.128.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 19976689E53 for ; Thu, 23 Jan 2020 11:23:04 +0200 (EET) Received: by mail-wm1-f65.google.com with SMTP id p17so1733301wmb.0 for ; Thu, 23 Jan 2020 01:23:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=N72nrJwl59jtm16375H2pn7iORhbUKlRhFl9T/aA0a4=; b=eBKwJLnI2ntcFXwBJhBYwNo8wJp/FrA61oBjQoDFGsEDdhLDyAfsj8MhQ1fB777CiI EqW/SzZDYzhBrZ8CXwxoA+kFR7iazS5YgV+6L8gQj9X19Z94nVRnD42ztr8HgMUTFMUu T1d47yrwtxhgHB4Sw1XN9Fp6zQLfEHmeh6gKP5PFc9rk/FQx0Vivl3m6tdej9sYdl66j 2OKkwJyEGoq0sTmhChFIMYqVXqamaqA7pv4bgyVj0M9iPr0kIPTq4a0MdpPUOPMAsYlu Q5Pk4Ty8HzcWbRj15kD/1t6+8Rrp84J2PmjWhaDtQayUaS5Dytjmt+L96/yMKIa/ZAvb uN3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=N72nrJwl59jtm16375H2pn7iORhbUKlRhFl9T/aA0a4=; b=Oj/B/k02LNbxHmpU5uDv9Kgu3vlqWBy17uVlZRg1FMCFOcFcpsKD9m13l7Bbm74sXH oB97IdELSgH3hA8kaxH7YRHnAPS+7T106RkLzMyB8scoBInRrN2fNqg6welFhzRde7eP k3pONrR+2OQ/l0SjHeDm4Fmit+hWqkTvfMRxcbQ7PuiEvuUf1Fvsxmm2Rbv3QvGrXMnb eC7vBqwE+OtCUU7xwty4N5rlLlAcTqrqNq5q39uTsQ48wHZwrZ3szMf9jindn7zsfDXY vkVGLoiaENwGHfUqd7jQhm9UuBMqpBvEJAmL1Jq+DMlrupGBirw2mw16Oe0BKjXNPWJW QKqw== X-Gm-Message-State: APjAAAXj9Cy67x7FaA0tLrxU/dXkJJaOcLpbjiQIhzQtuc6/BxUhCFMB RNxcu8kFH0S3gXFYLmpP3EPqaotOC80= X-Google-Smtp-Source: APXvYqwjUDu9VnNQP103DSYjbhnZfnnXo51RnGmgqNkfx1rdH4Jjhv345jxaNonv9OEMERY82Xmxhw== X-Received: by 2002:a7b:c85a:: with SMTP id c26mr3068427wml.107.1579771001992; Thu, 23 Jan 2020 01:16:41 -0800 (PST) Received: from lars.fritz.box (b2b-37-24-22-101.unitymedia.biz. [37.24.22.101]) by smtp.gmail.com with ESMTPSA id z21sm1986189wml.5.2020.01.23.01.16.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Jan 2020 01:16:41 -0800 (PST) From: Robert Deibel To: ffmpeg-devel@ffmpeg.org Date: Thu, 23 Jan 2020 10:16:14 +0100 Message-Id: <20200123091614.8880-1-deibel.robert@googlemail.com> X-Mailer: git-send-email 2.25.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] zoompan filter: fix shaking when zooming X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Robert Deibel Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Fix shaking of image when zoom is applied by the zoompan filter. Resolves ticket https://trac.ffmpeg.org/ticket/4298 --- libavfilter/vf_zoompan.c | 93 ++++++++++++++++++++++++++++++---------- 1 file changed, 70 insertions(+), 23 deletions(-) diff --git a/libavfilter/vf_zoompan.c b/libavfilter/vf_zoompan.c index 59c9b19ec8..4ae8c64cd0 100644 --- a/libavfilter/vf_zoompan.c +++ b/libavfilter/vf_zoompan.c @@ -150,16 +150,30 @@ static int config_output(AVFilterLink *outlink) return 0; } +/** + * Scales n to be a multiple of grid_size but minimally 2 * grid_size and divisable by two. + * + * Used to scale the width and height of a frame to fit with the subsampling grid. + * @param n The number to be scaled. + * @param grid_size the size of the grid. + * @return The scaled number divisable by 2 and minimally 2 * grid_size + */ +static int scale_to_grid(int n, uint8_t grid_size){ + return (((n + (1 << grid_size) * 2) & ~((1 << grid_size) - 1)) + 1) & ~1; + +} + static int output_single_frame(AVFilterContext *ctx, AVFrame *in, double *var_values, int i, double *zoom, double *dx, double *dy) { ZPContext *s = ctx->priv; AVFilterLink *outlink = ctx->outputs[0]; int64_t pts = s->frame_count; - int k, x, y, w, h, ret = 0; + int k, x, y, crop_x, crop_y, w, h, crop_w, crop_h, overscaled_w, overscaled_h, ret = 0; uint8_t *input[4]; int px[4], py[4]; AVFrame *out; + double dw, dh; var_values[VAR_PX] = s->x; var_values[VAR_PY] = s->y; @@ -173,32 +187,46 @@ static int output_single_frame(AVFilterContext *ctx, AVFrame *in, double *var_va *zoom = av_clipd(*zoom, 1, 10); var_values[VAR_ZOOM] = *zoom; - w = in->width * (1.0 / *zoom); - h = in->height * (1.0 / *zoom); + + // Keep track of double variables for correct calculation + w = dw = (double) in->width * (1.0 / *zoom); + h = dh = (double) in->height * (1.0 / *zoom); + + // width and height with additional pixels from subsampling "grid" + crop_w = scale_to_grid(w, s->desc->log2_chroma_w); + crop_h = scale_to_grid(h, s->desc->log2_chroma_h); *dx = av_expr_eval(s->x_expr, var_values, NULL); - x = *dx = av_clipd(*dx, 0, FFMAX(in->width - w, 0)); - var_values[VAR_X] = *dx; - x &= ~((1 << s->desc->log2_chroma_w) - 1); + crop_x = ceil(av_clipd(*dx - (((double) crop_w - w) / 2.0), 0, FFMAX(in->width - crop_w, 0))); + var_values[VAR_X] = *dx = av_clipd(*dx, 0, FFMAX(in->width - w, 0)); + crop_x &= ~((1 << s->desc->log2_chroma_w) - 1); // Masking LSBs making coordianate divisible *dy = av_expr_eval(s->y_expr, var_values, NULL); - y = *dy = av_clipd(*dy, 0, FFMAX(in->height - h, 0)); - var_values[VAR_Y] = *dy; - y &= ~((1 << s->desc->log2_chroma_h) - 1); + crop_y = ceil(av_clipd(*dy - (((double) crop_h - h)/ 2.0), 0, FFMAX(in->height - crop_h, 0))); + var_values[VAR_Y] = *dy = av_clipd(*dy, 0, FFMAX(in->height - h, 0)); + crop_y &= ~((1 << s->desc->log2_chroma_h) - 1); // Masking LSBs making coordianate divisible - out = ff_get_video_buffer(outlink, outlink->w, outlink->h); + overscaled_w = outlink->w + (crop_w - dw) * *zoom; + overscaled_h = outlink->h + (crop_h - dh) * *zoom; + + out = ff_get_video_buffer(outlink, overscaled_w, overscaled_h); if (!out) { ret = AVERROR(ENOMEM); return ret; } - px[1] = px[2] = AV_CEIL_RSHIFT(x, s->desc->log2_chroma_w); - px[0] = px[3] = x; + // Values for crop transform. Channel 1 and 2 are chroma plane, 0 luma plane, 3 alpha plane + px[1] = px[2] = AV_CEIL_RSHIFT(crop_x, s->desc->log2_chroma_w); + px[0] = px[3] = crop_x; - py[1] = py[2] = AV_CEIL_RSHIFT(y, s->desc->log2_chroma_h); - py[0] = py[3] = y; + py[1] = py[2] = AV_CEIL_RSHIFT(crop_y, s->desc->log2_chroma_h); + py[0] = py[3] = crop_y; + + // Crop data in input using px/py + for (k = 0; k<4; k++) + input[k] = in->data[k] + py[k] * in->linesize[k] + px[k]; s->sws = sws_alloc_context(); if (!s->sws) { @@ -206,21 +234,41 @@ static int output_single_frame(AVFilterContext *ctx, AVFrame *in, double *var_va goto error; } - for (k = 0; in->data[k]; k++) - input[k] = in->data[k] + py[k] * in->linesize[k] + px[k]; - - av_opt_set_int(s->sws, "srcw", w, 0); - av_opt_set_int(s->sws, "srch", h, 0); + // Set context variables. Used in scaling transform + av_opt_set_int(s->sws, "srcw", crop_w, 0); + av_opt_set_int(s->sws, "srch", crop_h, 0); av_opt_set_int(s->sws, "src_format", in->format, 0); - av_opt_set_int(s->sws, "dstw", outlink->w, 0); - av_opt_set_int(s->sws, "dsth", outlink->h, 0); + av_opt_set_int(s->sws, "dstw", overscaled_w, 0); + av_opt_set_int(s->sws, "dsth", overscaled_h, 0); av_opt_set_int(s->sws, "dst_format", outlink->format, 0); av_opt_set_int(s->sws, "sws_flags", SWS_BICUBIC, 0); if ((ret = sws_init_context(s->sws, NULL, NULL)) < 0) goto error; - sws_scale(s->sws, (const uint8_t *const *)&input, in->linesize, 0, h, out->data, out->linesize); + // Scale data in input to defined size and copy to out + sws_scale(s->sws, (const uint8_t *const *)&input, in->linesize, 0, crop_h, out->data, out->linesize); + + + // Calculate x and y with respect to rounding error. + *dx = ((crop_w - dw) * *zoom) / (((double) crop_w - dw) / (*dx - (double) crop_x)); + x = ceil(av_clipd(*dx, 0, FFMAX(overscaled_w - outlink->w, 0))); + x &= ~((1 << s->desc->log2_chroma_w) - 1); // Masking LSBs making coordianate divisible + + *dy = ((crop_h - dh) * *zoom) / (((double) crop_h - dh) / (*dy - (double) crop_y)); + y = ceil(av_clipd(*dy, 0, FFMAX(overscaled_h - outlink->h, 0))); + y &= ~((1 << s->desc->log2_chroma_h) - 1); // Masking LSBs making coordianate divisible + + // Values for crop transform. Channel 1 and 2 are chroma plane, 0 luma plane, 3 alpha plane + px[1] = px[2] = AV_CEIL_RSHIFT(x, s->desc->log2_chroma_w); + px[0] = px[3] = x; + + py[1] = py[2] = AV_CEIL_RSHIFT(y, s->desc->log2_chroma_h); + py[0] = py[3] = y; + + // Crop data in input using px/py + for (k = 0; k<4; k++) + out->data[k] = out->data[k] + py[k] * out->linesize[k] + px[k]; out->pts = pts; s->frame_count++; @@ -229,7 +277,6 @@ static int output_single_frame(AVFilterContext *ctx, AVFrame *in, double *var_va sws_freeContext(s->sws); s->sws = NULL; s->current_frame++; - if (s->current_frame >= s->nb_frames) { if (*dx != -1) s->x = *dx;