From patchwork Tue May 30 14:40:02 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Mark X-Patchwork-Id: 3762 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.10.2 with SMTP id 2csp1923319vsk; Tue, 30 May 2017 07:47:33 -0700 (PDT) X-Received: by 10.223.152.52 with SMTP id v49mr7644776wrb.60.1496155653460; Tue, 30 May 2017 07:47:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1496155653; cv=none; d=google.com; s=arc-20160816; b=HLEoHUbVUH3MzdJJX+V0xh826K+aEl9kTmf203Ym1iHkrWackQiRNgBbEkKgyNROpF JRfFLRYK8iRVSsUxVLVLtS0l1UlPEd/FskKTMOPn+cHSwrfkGRSaoqt3LA+MSQu79U0Y jmBryjr00IfivVn7MEBNBzh9o2m8kD2g+vrPD+l8tV5HsJxCzNnnZZq5tMud+/c1OuZW pXBDK6hvOOdcbnkI2oommaYkENLj5z7q3mZbtRrFJpGRxXQ1s9WHqW2vucIwHaNdvlnq Tcct0na4o1CCApruY1AdoQdf3ztZZ+whCuQw+JFJwWOip76PU+fT0y+2PN0p7duSKRfv lGyw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to:arc-authentication-results; bh=EpaRbOm0mOWJaXz7zKN9gO7coskw9/aF6QpePve0y7s=; b=syBErx/B9/uJweC/84REci/RmEhw3ZiMCOKU1z8BUWzh+uQIkmkK42MdYcvdIZOdHq xxNjMaZ9LZvRWGYhiY8mStqkLpX3gfvkrccqnRaG/dFOSaCGrfZ2wxFQ5t1EBftBarXz TxPaoBeX4+MXLwxxNlDInablOxSVefujclOO8FGvOpedXgA2pDJUWQA7OZ+wV8GVWCDX eVlDHK9zTyL6WdbTEZRF8jN3R9DYSgTq+eFbpDOkKPBoyW+Ru2MG1rC95KXUCa/VGoFv yiYcPT4FTrhRt5R/BgMuSM2eJd+7ZhT4aSCUJCGctDYi9+rbfJlem0hVXG/AksolTEvt 0e4Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id l103si307061wrc.218.2017.05.30.07.47.32; Tue, 30 May 2017 07:47:33 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D9E8768832B; Tue, 30 May 2017 17:47:24 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qk0-f193.google.com (mail-qk0-f193.google.com [209.85.220.193]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7F8B3680A18 for ; Tue, 30 May 2017 17:47:18 +0300 (EEST) Received: by mail-qk0-f193.google.com with SMTP id u75so12254635qka.1 for ; Tue, 30 May 2017 07:47:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=sl2SQe7kaF4PBtAxaYuTdHdpH8M7gbGzy1WpXVgEBA4=; b=sEvik2X3xM7HRIEb95Z1u+psufGf4AqtLaT554BBRHzmtN9L8WKYQY9qvHpCfYgTZh fGVp55nhn8m4cGRq26JV8JjLKN4b3jLDGSucWuUg6plnPSDMSrBQq+QLwYfszHYkqXlm rSdJusCXjyj8r7JS4o7f8yPMitdLMaLewPxKo8dCYVU+zLB8LmKD1WuEfUOV9XTj7VOy 4Yb2dFnC9eJKlT+vSO1Cg85zdu3awUAdt7K77492w8TN7JFqDWjE/FXEqiOug/9kmKCR ykSMC+pSMrHjDTGhuDKtsFPI1ptlSS+RmGzQNLt4aZ1zPcz1zBzIlN/4s4G0nx6o6QyP D54A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=sl2SQe7kaF4PBtAxaYuTdHdpH8M7gbGzy1WpXVgEBA4=; b=seiFkJfbqDNthq6G4vqjqq/sHBxCeotvMMa7fo9/TKFoEpU1pGySAOs/ugP8oKeSoj EJId5Frxz4vRwrdD1p5fb6JrPX+fITkMnuebFF2tI2J5qmV6+6NtXMvhdKlWahrcA5LK U+kXV0cazOC4WjvKazNhM8sdx+JJE8nj2CuJEDBqMWu0jNIkJhQEU+tlsLWJ0VRDKZRN pJtvwsq8+7e7F1aiWyyhdy+E7lQb6McVGc0ljq38vRivIc6D7kDCOsfgsySXlq2rTbyt tvaH1uwLu+r48v71nLsm9GQ+JW2GYrp4hk/WgC+lMJPqmMz0IGjXOtMmdtuVpHZn9IbY 3dlQ== X-Gm-Message-State: AODbwcD9LVJwx5P5BNF0gndpYRLSSHB/EHmh52ea0gBj0MzPW3i5i4lC lNzi+lkzc+ReZco8m6E= X-Received: by 10.233.223.196 with SMTP id t187mr24016094qkf.31.1496155216006; Tue, 30 May 2017 07:40:16 -0700 (PDT) Received: from Klingon.fios-router.home (pool-173-69-140-116.bltmmd.fios.verizon.net. [173.69.140.116]) by smtp.gmail.com with ESMTPSA id 21sm8191801qkj.12.2017.05.30.07.40.15 (version=TLS1 cipher=AES128-SHA bits=128/128); Tue, 30 May 2017 07:40:15 -0700 (PDT) From: Kevin Mark To: ffmpeg-devel@ffmpeg.org Date: Tue, 30 May 2017 10:40:02 -0400 Message-Id: <20170530144002.98505-1-kmark937@gmail.com> X-Mailer: git-send-email 2.13.0 In-Reply-To: <20170527141046.57488-1-kmark937@gmail.com> References: <20170527141046.57488-1-kmark937@gmail.com> Subject: [FFmpeg-devel] [PATCH] libavfilter/scale2ref: Add constants for the primary input X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Kevin Mark MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Variables pertaining to the main video are now available when using the scale2ref filter. This allows, as an example, scaling a video with another as a reference point while maintaining the original aspect ratio of the primary/non-reference video. Consider the following graph: scale2ref=iw/6:-1 [main][ref] This will scale [main] to 1/6 the width of [ref] while maintaining the aspect ratio. This works well when the AR of [ref] is equal to the AR of [main] only. What the above filter really does is maintain the AR of [ref] when scaling [main]. So in all non-same-AR situations [main] will appear stretched or compressed to conform to the same AR of the reference video. Without doing this calculation externally there is no way to scale in reference to another input while maintaining AR in libavfilter. To make this possible, we introduce eight new constants to be used in the w and h expressions only in the scale2ref filter: * main_w/main_h: width/height of the main input video * main_a: aspect ratio of the main input video * main_sar: sample aspect ratio of the main input video * main_dar: display aspect ratio of the main input video * main_hsub/main_vsub: horiz/vert chroma subsample vals of main * mdar: a shorthand alias of main_dar Of course, not all of these constants are needed for maintaining the AR, but adding additional constants in line of what is available for in/out allows for other scaling possibilities I have not imagined. So to now scale a video to 1/6 the size of another video using the width and maintaining its own aspect ratio you can do this: scale2ref=iw/6:ow/mdar [main][ref] This is ideal for picture-in-picture configurations where you could have a square or 4:3 video overlaid on a corner of a larger 16:9 feed all while keeping the scaled video in the corner at its correct aspect ratio and always the same size relative to the larger video. I've tried to re-use as much code as possible. I could not find a way to avoid duplication of the var_names array. It must now be kept in sync with the other (the normal one and the scale2ref one) for everything to work which does not seem ideal. For every new variable introduced/removed into/from the normal scale filter one must be added/removed to/from the scale2ref version. Suggestions on how to avoid var_names duplication are welcome. var_values has been increased to always be large enough for the additional scale2ref variables. I do not forsee this being a problem as the names variable will always be the correct size. From my understanding of av_expr_parse_and_eval it will stop processing variables when it runs out of names even though there may be additional (potentially uninitialized) entries in the values array. The ideal solution here would be using a variable-length array but that is unsupported in C90. This patch does not remove any functionality and is strictly a feature patch. There are no API changes. Behavior does not change for any previously valid inputs. The applicable documentation has also been updated. Signed-off-by: Kevin Mark --- doc/filters.texi | 26 ++++++++++++++++++- libavfilter/scale.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 93 insertions(+), 5 deletions(-) diff --git a/doc/filters.texi b/doc/filters.texi index 107fe61447..2cea6b74e6 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -12429,7 +12429,31 @@ Supersampling Scale (resize) the input video, based on a reference video. See the scale filter for available options, scale2ref supports the same but -uses the reference video instead of the main input as basis. +uses the reference video instead of the main input as basis. scale2ref also +supports the following additional constants for the @option{w} and +@option{h} options: + +@table @var +@item main_w +@item main_h +The main input video's width and height + +@item main_a +The same as @var{main_w} / @var{main_h} + +@item main_sar +The main input video's sample aspect ratio + +@item main_dar, mdar +The main input video's display aspect ratio. Calculated from +@code{(main_w / main_h) * main_sar}. + +@item main_hsub +@item main_vsub +The main input video's horizontal and vertical chroma subsample values. +For example for the pixel format "yuv422p" @var{hsub} is 2 and @var{vsub} +is 1. +@end table @subsection Examples diff --git a/libavfilter/scale.c b/libavfilter/scale.c index 50cd442849..552b7cbb04 100644 --- a/libavfilter/scale.c +++ b/libavfilter/scale.c @@ -60,6 +60,49 @@ enum var_name { VARS_NB }; +/** + * This must be kept in sync with var_names so that it is always a + * complete list of var_names with the scale2ref specific names + * appended. scale2ref values must appear in the order they appear + * in the var_name_scale2ref enum but also be below all of the + * non-scale2ref specific values. + */ +static const char *const var_names_scale2ref[] = { + "PI", + "PHI", + "E", + "in_w", "iw", + "in_h", "ih", + "out_w", "ow", + "out_h", "oh", + "a", + "sar", + "dar", + "hsub", + "vsub", + "ohsub", + "ovsub", + "main_w", + "main_h", + "main_a", + "main_sar", + "main_dar", "mdar", + "main_hsub", + "main_vsub", + NULL +}; + +enum var_name_scale2ref { + VAR_S2R_MAIN_W, + VAR_S2R_MAIN_H, + VAR_S2R_MAIN_A, + VAR_S2R_MAIN_SAR, + VAR_S2R_MAIN_DAR, VAR_S2R_MDAR, + VAR_S2R_MAIN_HSUB, + VAR_S2R_MAIN_VSUB, + VARS_S2R_NB +}; + int ff_scale_eval_dimensions(void *log_ctx, const char *w_expr, const char *h_expr, AVFilterLink *inlink, AVFilterLink *outlink, @@ -72,7 +115,16 @@ int ff_scale_eval_dimensions(void *log_ctx, int factor_w, factor_h; int eval_w, eval_h; int ret; - double var_values[VARS_NB], res; + const char scale2ref = outlink->src->inputs[1] == inlink; + double var_values[VARS_NB + VARS_S2R_NB], res; + const AVPixFmtDescriptor *main_desc; + const AVFilterLink *main; + const char *const *names = scale2ref ? var_names_scale2ref : var_names; + + if (scale2ref) { + main = outlink->src->inputs[0]; + main_desc = av_pix_fmt_desc_get(main->format); + } var_values[VAR_PI] = M_PI; var_values[VAR_PHI] = M_PHI; @@ -90,20 +142,32 @@ int ff_scale_eval_dimensions(void *log_ctx, var_values[VAR_OHSUB] = 1 << out_desc->log2_chroma_w; var_values[VAR_OVSUB] = 1 << out_desc->log2_chroma_h; + if (scale2ref) { + var_values[VARS_NB + VAR_S2R_MAIN_W] = main->w; + var_values[VARS_NB + VAR_S2R_MAIN_H] = main->h; + var_values[VARS_NB + VAR_S2R_MAIN_A] = (double) main->w / main->h; + var_values[VARS_NB + VAR_S2R_MAIN_SAR] = main->sample_aspect_ratio.num ? + (double) main->sample_aspect_ratio.num / main->sample_aspect_ratio.den : 1; + var_values[VARS_NB + VAR_S2R_MAIN_DAR] = var_values[VARS_NB + VAR_S2R_MDAR] = + var_values[VARS_NB + VAR_S2R_MAIN_A] * var_values[VARS_NB + VAR_S2R_MAIN_SAR]; + var_values[VARS_NB + VAR_S2R_MAIN_HSUB] = 1 << main_desc->log2_chroma_w; + var_values[VARS_NB + VAR_S2R_MAIN_VSUB] = 1 << main_desc->log2_chroma_h; + } + /* evaluate width and height */ av_expr_parse_and_eval(&res, (expr = w_expr), - var_names, var_values, + names, var_values, NULL, NULL, NULL, NULL, NULL, 0, log_ctx); eval_w = var_values[VAR_OUT_W] = var_values[VAR_OW] = res; if ((ret = av_expr_parse_and_eval(&res, (expr = h_expr), - var_names, var_values, + names, var_values, NULL, NULL, NULL, NULL, NULL, 0, log_ctx)) < 0) goto fail; eval_h = var_values[VAR_OUT_H] = var_values[VAR_OH] = res; /* evaluate again the width, as it may depend on the output height */ if ((ret = av_expr_parse_and_eval(&res, (expr = w_expr), - var_names, var_values, + names, var_values, NULL, NULL, NULL, NULL, NULL, 0, log_ctx)) < 0) goto fail; eval_w = res;