From patchwork Thu Aug 31 16:46:13 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashish Singh X-Patchwork-Id: 4922 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.15.201 with SMTP id 70csp1776148jao; Thu, 31 Aug 2017 09:46:38 -0700 (PDT) X-Received: by 10.223.164.90 with SMTP id e26mr3305035wra.62.1504197998042; Thu, 31 Aug 2017 09:46:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1504197997; cv=none; d=google.com; s=arc-20160816; b=BkqgRwkfflvxJNyQ1J5jTitQEaI8bIY2K8X1ZeFm0tNkE5fyOTVKdHZDLYwO1bEayW EKCeYmYxbm0if4psBmPyw1hGhkApsnL8C1MGwjGnwgc0/OmpxO2w1hI/a236YPCivvDu bFPsUfibNX02Pj9vU/nfBRzwFLH7LowT/ARp8QhCNsBghStFXIlatNqMypfC6D90Ao/q L/Zms8dgOiaXzQ/fW5vgXILJXL4pqsNLUFosOfSL+wi8Oba2LTInmz9pyd891VgAZpXL wIZxLd5gjjVkDwQebMUpuPFt/DMs9OHKKhnomLx9mEfp2gu+ue0TXCVTaTD8RXa5J44q NsXw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to:arc-authentication-results; bh=8dVFDIs9juaMQSdRFXBCVMhUatMJqLbPPwvXk/nzKjU=; b=G8v5dNUAommSThQovsq8wrmRsyEDC7f5upcQ/A5/m1nc5Ofvw+Ub3YiunKe4Kx4CgG L4VVz8s2HK8Zry/e+XzsVi3QuPOcq0yD5rocQNYWAW9ujYxOT433WvbgOzL29MUp8Kpa QV199syuraftogYUmFJG9KoZ7JJ6E840L3Dp1MpTRIXPlZ4TOcrT9hHyHD+l7aXvtYmh G5JW6ViJzf8aeeslMmxG1OXdtY8TBleqqSwAFF6ePKlrLSfVa+KyiNmyfFX3TKunuySU ozHIv+mqkYBxwD3U3OOkHK5S3k7HMobWf97WrCZKVdpjxP15+cxPM8JVonkRsmeCiGaa Cc/A== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=Fm0y6paw; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id e74si368141wme.144.2017.08.31.09.46.34; Thu, 31 Aug 2017 09:46:37 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=Fm0y6paw; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1B5A568A1BE; Thu, 31 Aug 2017 19:46:30 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pg0-f44.google.com (mail-pg0-f44.google.com [74.125.83.44]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A1B4D689D8B for ; Thu, 31 Aug 2017 19:46:23 +0300 (EEST) Received: by mail-pg0-f44.google.com with SMTP id r133so509584pgr.3 for ; Thu, 31 Aug 2017 09:46:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=mngCRYV/V7qzOJk0zpspesu5sqaQqn1bKtnzdZ3VYAA=; b=Fm0y6pawna1iqBgeg/1DieQKZPuoXebmY/j9aTb426NfbW7SbD4HWg6YDxra33nRjn DXNOU/6MDiRgyEskhzL6iufy4EtR+mZaKBp3XXX5JrWliGTz0age0ebI9Ee0SlMGK/d5 oa42H+arJDnEt2DIQtybB7D3cUVgrMfsXjpz4tHrIPQYCCyDg9fnsqfd5odVSXXQrHz3 8gO+1BsJHsKIoxEvwFe/HWNXFv1R8l2UW3F3k5R8bjSfsMXf6P+J1vlVR1DLeuMB8fvr ua78ObSzsWasTHIjwfwixBE0/J2Gpdb5aGMCju3pS1EBp5twBTdbI98VplqsQzAPCqwj aR2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=mngCRYV/V7qzOJk0zpspesu5sqaQqn1bKtnzdZ3VYAA=; b=D+fvIM0jhA9HlGtZ88pr2ScNVv860MpHHPiccDBBY9RQcPvOhADTup/ZwF325dhH8U 6SlsSX4lmKHOti41ef2DoGu/OSBEWoweqsRauWnyAylS1TEXpvOhA94FMgSrmNYGeaXT vV9RFndDfOf73Q0MGdNSQivST+1dADi8Q+m1r+h8KPWBC/5RsrLZA2gyYZ6Le2TP2jqg RU5cDpsJ/JtL5afsQTcGeNjN4oSYt51e1ItTkKIQbbdb6HXTn6UuqvC1cXG6vtjZ+Yjh /E4Vxdy+s5zr4VQcAbUr6ZFqpRp+AD+LkDtJEXUmSS9stU6Ml8rAc7gsnoPFvUGvz5gP jCcQ== X-Gm-Message-State: AHYfb5i/zv4bHmfbSb0cxkoUNZpCTo4wH6svK1RRKz51sizHRd3UiLO0 HiPxEJjkniY41fFvO6Y= X-Google-Smtp-Source: ADKCNb77Pf4ruWApAl6gZZ7UhT4EmiuCnKn9jjZdPjXVZDtnTosjt5LlU98YGHpnw0ONBb8DwWXTvw== X-Received: by 10.84.217.150 with SMTP id p22mr3282120pli.178.1504197981976; Thu, 31 Aug 2017 09:46:21 -0700 (PDT) Received: from localhost.localdomain ([49.44.51.56]) by smtp.gmail.com with ESMTPSA id l30sm186461pgc.61.2017.08.31.09.46.19 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 31 Aug 2017 09:46:21 -0700 (PDT) From: Ashish Pratap Singh To: ffmpeg-devel@ffmpeg.org Date: Thu, 31 Aug 2017 22:16:13 +0530 Message-Id: <1504197973-27434-1-git-send-email-ashk43712@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1504112047-12230-1-git-send-email-ashk43712@gmail.com> References: <1504112047-12230-1-git-send-email-ashk43712@gmail.com> Subject: [FFmpeg-devel] [PATCH] avfilter: add ADM filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Ashish Singh MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Ashish Singh Hi, This patch changes adm filter from dualinput to framesync2. Signed-off-by: Ashish Singh --- Changelog | 1 + doc/filters.texi | 15 + libavfilter/Makefile | 1 + libavfilter/adm.h | 75 +++++ libavfilter/allfilters.c | 1 + libavfilter/vf_adm.c | 748 +++++++++++++++++++++++++++++++++++++++++++++++ 6 files changed, 841 insertions(+) create mode 100644 libavfilter/adm.h create mode 100644 libavfilter/vf_adm.c diff --git a/Changelog b/Changelog index 8309417..c6a775c 100644 --- a/Changelog +++ b/Changelog @@ -40,6 +40,7 @@ version : They must always be used by name. - FITS demuxer and decoder - FITS muxer and encoder +- ADM video filter version 3.3: - CrystalHD decoder moved to new decode API diff --git a/doc/filters.texi b/doc/filters.texi index afcb99d..58dbbc7 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -4653,6 +4653,21 @@ build. Below is a description of the currently available video filters. +@section adm + +Obtain the average ADM/DLM (Detail Loss Metric) between two input videos. + +This filter takes two input videos. + +The obtained average ADM score is printed through the logging system. + +In the below example the input file @file{main.mpg} being processed is compared +with the reference file @file{ref.mpg}. + +@example +ffmpeg -i main.mpg -i ref.mpg -lavfi adm -f null - +@end example + @section alphaextract Extract the alpha component from the input as a grayscale video. This diff --git a/libavfilter/Makefile b/libavfilter/Makefile index ee840b0..d36ec89 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -126,6 +126,7 @@ OBJS-$(CONFIG_SINE_FILTER) += asrc_sine.o OBJS-$(CONFIG_ANULLSINK_FILTER) += asink_anullsink.o # video filters +OBJS-$(CONFIG_ADM_FILTER) += vf_adm.o framesync2.o OBJS-$(CONFIG_ALPHAEXTRACT_FILTER) += vf_extractplanes.o OBJS-$(CONFIG_ALPHAMERGE_FILTER) += vf_alphamerge.o OBJS-$(CONFIG_ASS_FILTER) += vf_subtitles.o diff --git a/libavfilter/adm.h b/libavfilter/adm.h new file mode 100644 index 0000000..862bd17 --- /dev/null +++ b/libavfilter/adm.h @@ -0,0 +1,75 @@ +/* + * Copyright (c) 2017 Ronald S. Bultje + * Copyright (c) 2017 Ashish Pratap Singh + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVFILTER_ADM_H +#define AVFILTER_ADM_H +/** Formula (1), page 1165 - display visual resolution (DVR), + * in pixels/degree of visual angle. This should be 56.55 + */ +#define R 56.55 +/** Percentage of frame to discard on all 4 sides */ +#define ADM_BORDER_FACTOR (0.1) + +#define N 15 + +typedef struct adm_dwt_band_t { + int16_t *band_a; /** Low-pass V + low-pass H. */ + int16_t *band_v; /** Low-pass V + high-pass H. */ + int16_t *band_h; /** High-pass V + low-pass H. */ + int16_t *band_d; /** High-pass V + high-pass H. */ +} adm_dwt_band_t; + +static const float dwt2_db2_coeffs_lo[4] = { + 0.482962913144690, 0.836516303737469, + 0.224143868041857, -0.129409522550921 +}; +static const float dwt2_db2_coeffs_hi[4] = { + -0.129409522550921, -0.224143868041857, + 0.836516303737469, -0.482962913144690 +}; + +static int32_t dwt2_db2_coeffs_lo_int[4]; +static int32_t dwt2_db2_coeffs_hi_int[4]; + +/** + * The following dwt basis function amplitudes, Q(lambda,theta), are taken from + * "Visibility of Wavelet Quantization Noise" + * by A. B. Watson, G. Y. Yang, J. A. Solomon and J. Villasenor + * IEEE Trans. on Image Processing, Vol. 6, No 8, Aug. 1997 + * Page 1172, Table V + * The table has been transposed, i.e. it can be used directly to obtain Q[lambda][theta] + * These amplitudes were calculated for the 7-9 biorthogonal wavelet basis + */ +static const float Q[4][2] = { + { 57.534645, 169.767410, }, + { 31.265896, 69.937431, }, + { 23.056629, 40.990150, }, + { 21.895033, 31.936741, }, +}; + +/** function to compute adm score */ +int compute_adm2(const void *ref, const void *main, int w, int h, + ptrdiff_t ref_stride, ptrdiff_t main_stride, double *score, + double *score_num, double *score_den, double *scores, + int16_t *data_buf, int16_t *temp_lo, int16_t *temp_hi, + uint8_t type); + +#endif /* AVFILTER_ADM_H */ diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 8b9b9a4..39ca9cc 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -138,6 +138,7 @@ static void register_all(void) REGISTER_FILTER(ANULLSINK, anullsink, asink); + REGISTER_FILTER(ADM, adm, vf); REGISTER_FILTER(ALPHAEXTRACT, alphaextract, vf); REGISTER_FILTER(ALPHAMERGE, alphamerge, vf); REGISTER_FILTER(ASS, ass, vf); diff --git a/libavfilter/vf_adm.c b/libavfilter/vf_adm.c new file mode 100644 index 0000000..1ba8e6e --- /dev/null +++ b/libavfilter/vf_adm.c @@ -0,0 +1,748 @@ +/* + * Copyright (c) 2017 Ronald S. Bultje + * Copyright (c) 2017 Ashish Pratap Singh + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * Calculate the ADM between two input videos. + */ + +#include "libavutil/avstring.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "avfilter.h" +#include "drawutils.h" +#include "formats.h" +#include "framesync2.h" +#include "internal.h" +#include "adm.h" +#include "video.h" +#include + +typedef struct ADMContext { + const AVClass *class; + FFFrameSync fs; + const AVPixFmtDescriptor *desc; + int width; + int height; + int16_t *data_buf; + int16_t *temp_lo; + int16_t *temp_hi; + double adm_sum; + uint64_t nb_frames; +} ADMContext; + +static const AVOption adm_options[] = { + { NULL } +}; + +FRAMESYNC_DEFINE_CLASS(adm, ADMContext, fs); + +#define MAX_ALIGN 32 +#define ALIGN_CEIL(x) ((x) + ((x) % MAX_ALIGN ? MAX_ALIGN - (x) % MAX_ALIGN : 0)) +#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM + +static float rcp(float x) +{ + float xi = _mm_cvtss_f32(_mm_rcp_ss(_mm_load_ss(&x))); + return xi + xi * (1.0 - x * xi); +} + +#define DIVS(n, d) ((n) * rcp(d)) + +static int32_t get_cube(int16_t val) +{ + return val * val * val; +} + +static int16_t adm_sum_cube(const int16_t *x, int w, int h, ptrdiff_t stride, + double border_factor) +{ + ptrdiff_t px_stride = stride / sizeof(int16_t); + int left = w * border_factor - 0.5; + int top = h * border_factor - 0.5; + int right = w - left; + int bottom = h - top; + + int i, j; + + int sum = 0; + + for (i = top; i < bottom; i++) { + for (j = left; j < right; j++) { + sum += get_cube(FFABS(x[i * px_stride + j])); + } + } + + return ceil(cbrt(sum)) + ceil(cbrt((bottom - top) * (right - left) / 32.0)); +} + +static void adm_decouple(const adm_dwt_band_t *ref, const adm_dwt_band_t *main, + const adm_dwt_band_t *r, const adm_dwt_band_t *a, + int w, int h, ptrdiff_t ref_stride, ptrdiff_t main_stride, + ptrdiff_t r_stride, ptrdiff_t a_stride) +{ + const float cos_1deg_sq = cos(1.0 * M_PI / 180.0) * cos(1.0 * M_PI / 180.0); + const float eps = 1e-30; + + ptrdiff_t ref_px_stride = ref_stride / sizeof(int16_t); + ptrdiff_t main_px_stride = main_stride / sizeof(int16_t); + ptrdiff_t r_px_stride = r_stride / sizeof(int16_t); + ptrdiff_t a_px_stride = a_stride / sizeof(int16_t); + + int oh, ov, od, th, tv, td; + float kh, kv, kd, tmph, tmpv, tmpd; + float ot_dp, o_mag_sq, t_mag_sq; + int angle_flag; + int i, j; + + for (i = 0; i < h; i++) { + for (j = 0; j < w; j++) { + oh = ref->band_h[i * ref_px_stride + j]; + ov = ref->band_v[i * ref_px_stride + j]; + od = ref->band_d[i * ref_px_stride + j]; + th = main->band_h[i * main_px_stride + j]; + tv = main->band_v[i * main_px_stride + j]; + td = main->band_d[i * main_px_stride + j]; + + kh = DIVS(th, oh + eps); + kv = DIVS(tv, ov + eps); + kd = DIVS(td, od + eps); + + kh = kh < 0 ? 0 : (kh > 1 ? 1 : kh); + kv = kv < 0 ? 0 : (kv > 1 ? 1 : kv); + kd = kd < 0 ? 0 : (kd > 1 ? 1 : kd); + + tmph = kh * oh; + tmpv = kv * ov; + tmpd = kd * od; + + ot_dp = oh * th + ov * tv; + o_mag_sq = oh * oh + ov * ov; + t_mag_sq = th * th + tv * tv; + + angle_flag = (ot_dp >= 0) && (ot_dp * ot_dp >= cos_1deg_sq * + o_mag_sq * t_mag_sq); + + if (angle_flag) { + tmph = th; + tmpv = tv; + tmpd = td; + } + + r->band_h[i * r_px_stride + j] = ceil(tmph); + r->band_v[i * r_px_stride + j] = ceil(tmpv); + r->band_d[i * r_px_stride + j] = ceil(tmpd); + + a->band_h[i * a_px_stride + j] = ceil(th - tmph); + a->band_v[i * a_px_stride + j] = ceil(tv - tmpv); + a->band_d[i * a_px_stride + j] = ceil(td - tmpd); + } + } +} + +static void adm_csf(const adm_dwt_band_t *src, const adm_dwt_band_t *dst, + int orig_h, int scale, int w, int h, ptrdiff_t src_stride, + ptrdiff_t dst_stride) +{ + const int16_t *src_angles[3] = { src->band_h, src->band_v, src->band_d }; + int16_t *dst_angles[3] = { dst->band_h, dst->band_v, dst->band_d }; + + const int16_t *src_ptr; + int16_t *dst_ptr; + + ptrdiff_t src_px_stride = src_stride / sizeof(int16_t); + ptrdiff_t dst_px_stride = dst_stride / sizeof(int16_t); + + uint16_t rfactor[3] = {lrint((1.0 / Q[scale][0]) * (1 << N)), + lrint((1.0 / Q[scale][0]) * (1 << N)), + lrint((1.0 / Q[scale][1]) * (1 << N))}; + + int i, j, theta; + + for (theta = 0; theta < 3; theta++) { + src_ptr = src_angles[theta]; + dst_ptr = dst_angles[theta]; + + for (i = 0; i < h; i++) { + for (j = 0; j < w; j++) { + dst_ptr[i * dst_px_stride + j] = (rfactor[theta] * + src_ptr[i * src_px_stride + j]) >> N; + } + } + } +} + +static void adm_cm_thresh(const adm_dwt_band_t *src, int16_t *dst, int w, int h, + ptrdiff_t src_stride, ptrdiff_t dst_stride) +{ + const int16_t *angles[3] = { src->band_h, src->band_v, src->band_d }; + const int16_t *src_ptr; + + ptrdiff_t src_px_stride = src_stride / sizeof(int16_t); + ptrdiff_t dst_px_stride = dst_stride / sizeof(int16_t); + + int filt_coeff, img_coeff; + + int theta, i, j, filt_i, filt_j, src_i, src_j; + + for (i = 0; i < h; i++) { + + for (j = 0; j < w; j++) { + dst[i * dst_px_stride + j] = 0; + } + + for (theta = 0; theta < 3; ++theta) { + src_ptr = angles[theta]; + + for (j = 0; j < w; j++) { + int sum = 0; + + for (filt_i = 0; filt_i < 3; filt_i++) { + for (filt_j = 0; filt_j < 3; filt_j++) { + filt_coeff = (lrint((filt_i == 1 && filt_j == 1) ? 1.0 / + 15.0 : 1.0 / 30.0) * (1 << N)); + + src_i = i - 1 + filt_i; + src_j = j - 1 + filt_j; + + src_i = FFABS(src_i); + if (src_i >= h) { + src_i = 2 * h - src_i - 1; + } + src_j = FFABS(src_j); + if (src_j >= w) { + src_j = 2 * w - src_j - 1; + } + img_coeff = FFABS(src_ptr[src_i * src_px_stride + src_j]); + + sum += filt_coeff * img_coeff; + } + } + + dst[i * dst_px_stride + j] += sum >> N; + } + } + } +} + +static void adm_cm(const adm_dwt_band_t *src, const adm_dwt_band_t *dst, + const int16_t *thresh, int w, int h, ptrdiff_t src_stride, + ptrdiff_t dst_stride, ptrdiff_t thresh_stride) +{ + ptrdiff_t src_px_stride = src_stride / sizeof(int16_t); + ptrdiff_t dst_px_stride = dst_stride / sizeof(int16_t); + ptrdiff_t thresh_px_stride = thresh_stride / sizeof(int16_t); + + int xh, xv, xd, thr; + + int i, j; + + for (i = 0; i < h; i++) { + for (j = 0; j < w; j++) { + xh = src->band_h[i * src_px_stride + j]; + xv = src->band_v[i * src_px_stride + j]; + xd = src->band_d[i * src_px_stride + j]; + thr = thresh[i * thresh_px_stride + j]; + + xh = FFABS(xh) - thr; + xv = FFABS(xv) - thr; + xd = FFABS(xd) - thr; + + xh = xh < 0 ? 0 : xh; + xv = xv < 0 ? 0 : xv; + xd = xd < 0 ? 0 : xd; + + dst->band_h[i * dst_px_stride + j] = xh; + dst->band_v[i * dst_px_stride + j] = xv; + dst->band_d[i * dst_px_stride + j] = xd; + } + } +} + +#define adm_dwt2_fn(type, bits) \ + static void adm_dwt2_##bits##bit(const type *src, const adm_dwt_band_t *dst, \ + int w, int h, ptrdiff_t src_stride, \ + ptrdiff_t dst_stride, int16_t *temp_lo, \ + int16_t* temp_hi) \ +{ \ + const int32_t *filter_lo = dwt2_db2_coeffs_lo_int; \ + const int32_t *filter_hi = dwt2_db2_coeffs_hi_int; \ + int filt_w = sizeof(dwt2_db2_coeffs_lo_int) / sizeof(int); \ + \ + ptrdiff_t src_px_stride = src_stride / sizeof(type); \ + ptrdiff_t dst_px_stride = dst_stride / sizeof(int16_t); \ + \ + int filt_coeff_lo, filt_coeff_hi, img_coeff; \ + \ + int i, j, filt_i, filt_j, src_i, src_j; \ + \ + for (i = 0; i < (h + 1) / 2; i++) { \ + /** Vertical pass. */ \ + for (j = 0; j < w; j++) { \ + int sum_lo = 0; \ + int sum_hi = 0; \ + \ + for (filt_i = 0; filt_i < filt_w; filt_i++) { \ + filt_coeff_lo = filter_lo[filt_i]; \ + filt_coeff_hi = filter_hi[filt_i]; \ + \ + src_i = 2 * i - 1 + filt_i; \ + \ + src_i = FFABS(src_i); \ + if (src_i >= h) { \ + src_i = 2 * h - src_i - 1; \ + } \ + \ + img_coeff = src[src_i * src_px_stride + j]; \ + \ + sum_lo += filt_coeff_lo * img_coeff; \ + sum_hi += filt_coeff_hi * img_coeff; \ + } \ + \ + temp_lo[j] = sum_lo >> N; \ + temp_hi[j] = sum_hi >> N; \ + } \ + \ + /** Horizontal pass (lo). */ \ + for (j = 0; j < (w + 1) / 2; j++) { \ + int sum_lo = 0; \ + int sum_hi = 0; \ + \ + for (filt_j = 0; filt_j < filt_w; filt_j++) { \ + filt_coeff_lo = filter_lo[filt_j]; \ + filt_coeff_hi = filter_hi[filt_j]; \ + \ + src_j = 2 * j - 1 + filt_j; \ + \ + src_j = FFABS(src_j); \ + if (src_j >= w) { \ + src_j = 2 * w - src_j - 1; \ + } \ + \ + img_coeff = temp_lo[src_j]; \ + \ + sum_lo += filt_coeff_lo * img_coeff; \ + sum_hi += filt_coeff_hi * img_coeff; \ + } \ + \ + dst->band_a[i * dst_px_stride + j] = sum_lo >> N; \ + dst->band_v[i * dst_px_stride + j] = sum_hi >> N; \ + } \ + \ + /** Horizontal pass (hi). */ \ + for (j = 0; j < (w + 1) / 2; j++) { \ + int sum_lo = 0; \ + int sum_hi = 0; \ + \ + for (filt_j = 0; filt_j < filt_w; filt_j++) { \ + filt_coeff_lo = filter_lo[filt_j]; \ + filt_coeff_hi = filter_hi[filt_j]; \ + \ + src_j = 2 * j - 1 + filt_j; \ + \ + src_j = FFABS(src_j); \ + if (src_j >= w) { \ + src_j = 2 * w - src_j - 1; \ + } \ + \ + img_coeff = temp_hi[src_j]; \ + \ + sum_lo += filt_coeff_lo * img_coeff; \ + sum_hi += filt_coeff_hi * img_coeff; \ + } \ + \ + dst->band_h[i * dst_px_stride + j] = sum_lo >> N; \ + dst->band_d[i * dst_px_stride + j] = sum_hi >> N; \ + } \ + } \ +} + +adm_dwt2_fn(uint8_t, 8); +adm_dwt2_fn(uint16_t, 10); +adm_dwt2_fn(int16_t, 32); + +static void adm_buffer_copy(const void *src, void *dst, int linewidth, int h, + ptrdiff_t src_stride, ptrdiff_t dst_stride) +{ + const char *src_p = src; + char *dst_p = dst; + int i; + + for (i = 0; i < h; i++) { + memcpy(dst_p, src_p, linewidth); + src_p += src_stride; + dst_p += dst_stride; + } +} + +static char *init_dwt_band(adm_dwt_band_t *band, char *data_top, size_t buf_sz) +{ + band->band_a = (int16_t *) data_top; + data_top += buf_sz; + band->band_h = (int16_t *) data_top; + data_top += buf_sz; + band->band_v = (int16_t *) data_top; + data_top += buf_sz; + band->band_d = (int16_t *) data_top; + data_top += buf_sz; + return data_top; +} + +int compute_adm2(const void *ref, const void *main, int w, int h, + ptrdiff_t ref_stride, ptrdiff_t main_stride, double *score, + double *score_num, double *score_den, double *scores, + int16_t *data_buf, int16_t *temp_lo, int16_t *temp_hi, + uint8_t type) +{ + double numden_limit = 1e-2 * (w * h) / (1920.0 * 1080.0); + + char *data_top; + + int16_t *ref_scale; + int16_t *main_scale; + + adm_dwt_band_t ref_dwt2; + adm_dwt_band_t main_dwt2; + + adm_dwt_band_t decouple_r; + adm_dwt_band_t decouple_a; + + adm_dwt_band_t csf_o; + adm_dwt_band_t csf_r; + adm_dwt_band_t csf_a; + + int16_t *mta; + + adm_dwt_band_t cm_r; + + const void *curr_ref_scale = ref; + const void *curr_main_scale = main; + ptrdiff_t curr_ref_stride = ref_stride; + ptrdiff_t curr_main_stride = main_stride; + + int orig_h = h; + + ptrdiff_t buf_stride = ALIGN_CEIL(((w + 1) / 2) * sizeof(int16_t)); + size_t buf_sz = (size_t)buf_stride * ((h + 1) / 2); + + double num = 0; + double den = 0; + + int scale; + int ret = 1; + + data_top = (char *) (data_buf); + + ref_scale = (int16_t *) data_top; + data_top += buf_sz; + main_scale = (int16_t *) data_top; + data_top += buf_sz; + + data_top = init_dwt_band(&ref_dwt2, data_top, buf_sz); + data_top = init_dwt_band(&main_dwt2, data_top, buf_sz); + data_top = init_dwt_band(&decouple_r, data_top, buf_sz); + data_top = init_dwt_band(&decouple_a, data_top, buf_sz); + data_top = init_dwt_band(&csf_o, data_top, buf_sz); + data_top = init_dwt_band(&csf_r, data_top, buf_sz); + data_top = init_dwt_band(&csf_a, data_top, buf_sz); + + mta = (int16_t *) data_top; + data_top += buf_sz; + + data_top = init_dwt_band(&cm_r, data_top, buf_sz); + + for (scale = 0; scale < 4; scale++) { + float num_scale = 0.0; + float den_scale = 0.0; + + if(!scale) { + if(type <= 8) { + adm_dwt2_8bit((const uint8_t *) curr_ref_scale, &ref_dwt2, w, + h, curr_ref_stride, buf_stride, temp_lo, temp_hi); + adm_dwt2_8bit((const uint8_t *) curr_main_scale, &main_dwt2, w, + h, curr_main_stride, buf_stride, temp_lo, temp_hi); + } else { + adm_dwt2_10bit((const uint16_t *) curr_ref_scale, &ref_dwt2, w, + h, curr_ref_stride, buf_stride, temp_lo, temp_hi); + adm_dwt2_10bit((const uint16_t *) curr_main_scale, &main_dwt2, w, + h, curr_main_stride, buf_stride, temp_lo, temp_hi); + } + } else{ + adm_dwt2_32bit((const int16_t *) curr_ref_scale, &ref_dwt2, w, h, + curr_ref_stride, buf_stride, temp_lo, temp_hi); + adm_dwt2_32bit((const int16_t *) curr_main_scale, &main_dwt2, w, h, + curr_main_stride, buf_stride, temp_lo, temp_hi); + } + + w = (w + 1) / 2; + h = (h + 1) / 2; + + adm_decouple(&ref_dwt2, &main_dwt2, &decouple_r, &decouple_a, w, h, + buf_stride, buf_stride, buf_stride, buf_stride); + + adm_csf(&ref_dwt2, &csf_o, orig_h, scale, w, h, buf_stride, buf_stride); + adm_csf(&decouple_r, &csf_r, orig_h, scale, w, h, buf_stride, buf_stride); + adm_csf(&decouple_a, &csf_a, orig_h, scale, w, h, buf_stride, buf_stride); + + adm_cm_thresh(&csf_a, mta, w, h, buf_stride, buf_stride); + adm_cm(&csf_r, &cm_r, mta, w, h, buf_stride, buf_stride, buf_stride); + + num_scale += adm_sum_cube(cm_r.band_h, w, h, buf_stride, ADM_BORDER_FACTOR); + num_scale += adm_sum_cube(cm_r.band_v, w, h, buf_stride, ADM_BORDER_FACTOR); + num_scale += adm_sum_cube(cm_r.band_d, w, h, buf_stride, ADM_BORDER_FACTOR); + + den_scale += adm_sum_cube(csf_o.band_h, w, h, buf_stride, ADM_BORDER_FACTOR); + den_scale += adm_sum_cube(csf_o.band_v, w, h, buf_stride, ADM_BORDER_FACTOR); + den_scale += adm_sum_cube(csf_o.band_d, w, h, buf_stride, ADM_BORDER_FACTOR); + + num += num_scale; + den += den_scale; + + adm_buffer_copy(ref_dwt2.band_a, ref_scale, w * sizeof(int16_t), h, + buf_stride, buf_stride); + adm_buffer_copy(main_dwt2.band_a, main_scale, w * sizeof(int16_t), h, + buf_stride, buf_stride); + + curr_ref_scale = ref_scale; + curr_main_scale = main_scale; + curr_ref_stride = buf_stride; + curr_main_stride = buf_stride; + + scores[2 * scale + 0] = num_scale; + scores[2 * scale + 1] = den_scale; + } + + num = num < numden_limit ? 0 : num; + den = den < numden_limit ? 0 : den; + + if (den == 0.0) { + *score = 1.0; + } else { + *score = num / den; + } + *score_num = num; + *score_den = den; + + ret = 0; + + return ret; +} + +static void set_meta(AVDictionary **metadata, const char *key, float d) +{ + char value[128]; + snprintf(value, sizeof(value), "%0.2f", d); + av_dict_set(metadata, key, value, 0); +} + +static int do_vmaf(FFFrameSync *fs) +{ + AVFilterContext *ctx = fs->parent; + ADMContext *s = ctx->priv; + AVFrame *main, *ref; + AVDictionary **metadata; + int ret; + double score = 0.0; + double score_num = 0; + double score_den = 0; + double scores[2 * 4]; + + int w = s->width; + int h = s->height; + + ptrdiff_t ref_stride, main_stride; + + ret = ff_framesync2_dualinput_get(fs, &main, &ref); + if (ret < 0) + return ret; + if (!ref) + return ff_filter_frame(ctx->outputs[0], main); + + metadata = &main->metadata; + + ref_stride = ref->linesize[0]; + main_stride = main->linesize[0]; + + compute_adm2(ref->data[0], main->data[0], w, h, ref_stride, main_stride, + &score, &score_num, &score_den, scores, s->data_buf, s->temp_lo, + s->temp_hi, s->desc->comp[0].depth); + + set_meta(metadata, "lavfi.adm.score", score); + + s->nb_frames++; + + s->adm_sum += score; + + return ff_filter_frame(ctx->outputs[0], main); +} + +static av_cold int init(AVFilterContext *ctx) +{ + ADMContext *s = ctx->priv; + + int i; + for(i = 0; i < 4; i++) { + dwt2_db2_coeffs_lo_int[i] = lrint(dwt2_db2_coeffs_lo[i] * (1 << N)); + dwt2_db2_coeffs_hi_int[i] = lrint(dwt2_db2_coeffs_hi[i] * (1 << N)); + } + + s->fs.on_event = do_vmaf; + + return 0; +} + +static int query_formats(AVFilterContext *ctx) +{ + static const enum AVPixelFormat pix_fmts[] = { + AV_PIX_FMT_YUV444P, AV_PIX_FMT_YUV422P, AV_PIX_FMT_YUV420P, + AV_PIX_FMT_YUV444P10LE, AV_PIX_FMT_YUV422P10LE, AV_PIX_FMT_YUV420P10LE, + AV_PIX_FMT_NONE + }; + + AVFilterFormats *fmts_list = ff_make_format_list(pix_fmts); + if (!fmts_list) + return AVERROR(ENOMEM); + return ff_set_common_formats(ctx, fmts_list); +} + +static int config_input_ref(AVFilterLink *inlink) +{ + AVFilterContext *ctx = inlink->dst; + ADMContext *s = ctx->priv; + ptrdiff_t buf_stride; + size_t buf_sz; + ptrdiff_t stride; + + if (ctx->inputs[0]->w != ctx->inputs[1]->w || + ctx->inputs[0]->h != ctx->inputs[1]->h) { + av_log(ctx, AV_LOG_ERROR, "Width and height of input videos must be same.\n"); + return AVERROR(EINVAL); + } + if (ctx->inputs[0]->format != ctx->inputs[1]->format) { + av_log(ctx, AV_LOG_ERROR, "Inputs must be of same pixel format.\n"); + return AVERROR(EINVAL); + } + + s->desc = av_pix_fmt_desc_get(inlink->format); + s->width = ctx->inputs[0]->w; + s->height = ctx->inputs[0]->h; + + buf_stride = ALIGN_CEIL(((s->width + 1) / 2) * sizeof(int16_t)); + buf_sz = (size_t)buf_stride * ((s->height + 1) / 2); + + if (SIZE_MAX / buf_sz < 35) { + av_log(ctx, AV_LOG_ERROR, "error: SIZE_MAX / buf_sz_one < 35"); + return AVERROR(EINVAL); + } + + if (!(s->data_buf = av_malloc(buf_sz * 35))) { + return AVERROR(ENOMEM); + } + + stride = ALIGN_CEIL(s->width * sizeof(int16_t)); + if (!(s->temp_lo = av_malloc(stride))) { + return AVERROR(ENOMEM); + } + + if (!(s->temp_hi = av_malloc(stride))) { + return AVERROR(ENOMEM); + } + + return 0; +} + + +static int config_output(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + ADMContext *s = ctx->priv; + AVFilterLink *mainlink = ctx->inputs[0]; + int ret; + + ret = ff_framesync2_init_dualinput(&s->fs, ctx); + if (ret < 0) + return ret; + outlink->w = mainlink->w; + outlink->h = mainlink->h; + outlink->time_base = mainlink->time_base; + outlink->sample_aspect_ratio = mainlink->sample_aspect_ratio; + outlink->frame_rate = mainlink->frame_rate; + if ((ret = ff_framesync2_configure(&s->fs)) < 0) + return ret; + + return 0; +} + +static int activate(AVFilterContext *ctx) +{ + ADMContext *s = ctx->priv; + return ff_framesync2_activate(&s->fs); +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + ADMContext *s = ctx->priv; + + ff_framesync2_uninit(&s->fs); + + if (s->nb_frames > 0) { + av_log(ctx, AV_LOG_INFO, "ADM AVG: %.3f\n", s->adm_sum / s->nb_frames); + } + + av_free(s->data_buf); + av_free(s->temp_lo); + av_free(s->temp_hi); +} + +static const AVFilterPad adm_inputs[] = { + { + .name = "main", + .type = AVMEDIA_TYPE_VIDEO, + },{ + .name = "reference", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_input_ref, + }, + { NULL } +}; + +static const AVFilterPad adm_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_output, + }, + { NULL } +}; + +AVFilter ff_vf_adm = { + .name = "adm", + .description = NULL_IF_CONFIG_SMALL("Calculate the ADM score between two video streams."), + .preinit = adm_framesync_preinit, + .init = init, + .uninit = uninit, + .query_formats = query_formats, + .activate = activate, + .priv_size = sizeof(ADMContext), + .priv_class = &adm_class, + .inputs = adm_inputs, + .outputs = adm_outputs, +};