From patchwork Sat Jul 29 08:46:29 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashish Singh X-Patchwork-Id: 4502 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.1.85 with SMTP id 82csp77661vsb; Sat, 29 Jul 2017 01:48:54 -0700 (PDT) X-Received: by 10.28.216.66 with SMTP id p63mr7595944wmg.17.1501318134345; Sat, 29 Jul 2017 01:48:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1501318134; cv=none; d=google.com; s=arc-20160816; b=qEZ1mAcsKusFrkp0G1MbbCb3Puv90SoSmPkluVIhCqoBcsvi54VldFQCpxVOt4IrD3 ji+ooVYfKif3vs7IOir1L2Vdo144FErsV0yq5vMwYyJGxbRAEeq+eXP27Z9ae3lEvrqx /mqsRUtaRTKZuDsue1mw+feh1lBV0fJek0vt0E17052XxShPOAPE/QokccVHZrZpF+Ny kAdFF3LphYsycIixKRuiYxizgf9ZKYsgksaA+zNlgO8mkzph22XIINBmR7IqWIfmnL9H LUZpELFlbnW6S86zRyd1BE+Tizdgy6Sc/5J/Dw9RuDx0SD06Z6gE8ChA9uflXk/wM/3g Lzrg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:dkim-signature :delivered-to:arc-authentication-results; bh=WU8ALkaKconahb2vNAWSRh555n2QHdpKRFopdWSBPB4=; b=IZQWBkIh5fuIJYFevwg/5I6YncxYCnu/bVm3gFgLyk7SCniEx7MqeVunz22F+qGOyL +sWgcKu/o0l471l0NLImCOeS00ukvVZKMvgMXXHVGP3e/Urw/j9jB0+HyhhFTDxxCMaY UwrlmpFImDUoDeyH36S/eDkoajsbLAiF4FHxXA+wsAPRE5mIF29cqdvNOHcCqD4/SKYQ Ttqmjo0ef4jQeFD5MCDHwUKkM9uAdNDRVwqh1r08dyH4ZYMG7m6LrEOtbDBTem8bASnO q7g9Di+hko2u30wah9c36v9kKA4ccukk15Ior9qeg+UdAeRsm8VQxeB6/C3qwYguHhK+ Zr9w== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.b=fe977HKv; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id x20si15827842wrd.233.2017.07.29.01.48.53; Sat, 29 Jul 2017 01:48:54 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.b=fe977HKv; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4D1CD689D4F; Sat, 29 Jul 2017 11:48:48 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pg0-f65.google.com (mail-pg0-f65.google.com [74.125.83.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 13B00689D3E for ; Sat, 29 Jul 2017 11:48:41 +0300 (EEST) Received: by mail-pg0-f65.google.com with SMTP id k190so10004228pgk.4 for ; Sat, 29 Jul 2017 01:48:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=G5UNuQkT2FMPlUpJHNgXWI4/hhDqNsKA96kuqaYtf7A=; b=fe977HKvpbewjg02QcdBKynXZrr4SzHpnty6jJFtd0ZPKv1Zlzq390aX2wO1eOcwhS akTLaeRJRSgPp8oP9cIUIQap/VgH1t5Mf4scwyY81TeItyGnYvBZeU4AzcHYN5oQqsxb CKB0KXvSMqT/5067U1edVTKt4gpxV/HrSihS6gi4BT9ZJyv4kwemi1+Z+TwrE+1xntRf 3chuH+GeeSPDRwDQ3TE8pQNxhtCtSY2hiqYiA/wA/IMJcYqh/wtu32hzX6qba/TDsVsN peO8dB04QJPSQYKAhuEXKbX6LTE5yKRCj8qLElsp3bKRsD785JzDr/6pgbTdeYkbhWmj eR0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=G5UNuQkT2FMPlUpJHNgXWI4/hhDqNsKA96kuqaYtf7A=; b=I2OLZkXvTm/aDaDZDiUgPgE0bLhu5uJHQCuHA1t4fluKgJl+nisFT7h+124K5L/R28 InvnboHFPHVyIAM828e0TZ23nzlURHOJ3KQYy/pLd/MkQIgO9EdLE+lGxBB08nbenfzL luK96fJS4Dstat9KOXFM/G4i9ttEr70rLqwyvNwn4JhgsRDjqyz2k0/mKZSgq34vucaP CKGlsvq9UW+4XnPVgBt2DSveBIglvcjNRfXeEJQi0BTfnETBn6KW4gWITbuRV0YyZM1f 24zhSC/l03LEJw4nC4uE64J1VzCVS59mmBghByV2kEOX34cwUgs0ZyQHjiR9uWT8zbni 9scw== X-Gm-Message-State: AIVw111nPUuINTU/ccpgE4y7UPKiVr8eVtRUsJr7qhcJInhqT2YztSI4 Zg7JSPpK/cE7XutM X-Received: by 10.99.55.79 with SMTP id g15mr9791297pgn.28.1501318120488; Sat, 29 Jul 2017 01:48:40 -0700 (PDT) Received: from localhost.localdomain ([117.244.56.65]) by smtp.gmail.com with ESMTPSA id 139sm6234278pgh.60.2017.07.29.01.48.36 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sat, 29 Jul 2017 01:48:39 -0700 (PDT) From: Ashish Pratap Singh To: ffmpeg-devel@ffmpeg.org Date: Sat, 29 Jul 2017 14:16:29 +0530 Message-Id: <1501317989-5062-1-git-send-email-ashk43712@gmail.com> X-Mailer: git-send-email 2.7.4 Subject: [FFmpeg-devel] [PATCH] avfilter: add VIF filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Ashish Singh MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Ashish Singh This is Visual Information Fidelity (VIF) filter and one of the component filters of VMAF. It outputs the average VIF score over all frames. Signed-off-by: Ashish Singh --- Changelog | 1 + doc/filters.texi | 19 ++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/vf_vif.c | 646 +++++++++++++++++++++++++++++++++++++++++++++++ libavfilter/vif.h | 30 +++ 6 files changed, 698 insertions(+) create mode 100644 libavfilter/vf_vif.c create mode 100644 libavfilter/vif.h diff --git a/Changelog b/Changelog index 187ae79..68900ca 100644 --- a/Changelog +++ b/Changelog @@ -29,6 +29,7 @@ version : - limiter video filter - libvmaf video filter - Dolby E decoder and SMPTE 337M demuxer +- vif video filter version 3.3: - CrystalHD decoder moved to new decode API diff --git a/doc/filters.texi b/doc/filters.texi index 2324b96..df5056f 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -15026,6 +15026,25 @@ For example, to vertically flip a video with @command{ffmpeg}: ffmpeg -i in.avi -vf "vflip" out.avi @end example +@section vif + +Obtain the average VIF (Visual Information Fidelity) between two input videos. + +This filter takes two input videos. + +Both input videos must have the same resolution and pixel format for +this filter to work correctly. Also it assumes that both inputs +have the same number of frames, which are compared one by one. + +The obtained average VIF score is printed through the logging system. + +In the below example the input file @file{main.mpg} being processed is compared +with the reference file @file{ref.mpg}. + +@example +ffmpeg -i main.mpg -i ref.mpg -lavfi vif -f null - +@end example + @anchor{vignette} @section vignette diff --git a/libavfilter/Makefile b/libavfilter/Makefile index ee16361..9999856 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -321,6 +321,7 @@ OBJS-$(CONFIG_VECTORSCOPE_FILTER) += vf_vectorscope.o OBJS-$(CONFIG_VFLIP_FILTER) += vf_vflip.o OBJS-$(CONFIG_VIDSTABDETECT_FILTER) += vidstabutils.o vf_vidstabdetect.o OBJS-$(CONFIG_VIDSTABTRANSFORM_FILTER) += vidstabutils.o vf_vidstabtransform.o +OBJS-$(CONFIG_VIF_FILTER) += vf_vif.o dualinput.o framesync.o OBJS-$(CONFIG_VIGNETTE_FILTER) += vf_vignette.o OBJS-$(CONFIG_VSTACK_FILTER) += vf_stack.o framesync.o OBJS-$(CONFIG_W3FDIF_FILTER) += vf_w3fdif.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index b1c2d11..bd342e3 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -332,6 +332,7 @@ static void register_all(void) REGISTER_FILTER(VFLIP, vflip, vf); REGISTER_FILTER(VIDSTABDETECT, vidstabdetect, vf); REGISTER_FILTER(VIDSTABTRANSFORM, vidstabtransform, vf); + REGISTER_FILTER(VIF, vif, vf); REGISTER_FILTER(VIGNETTE, vignette, vf); REGISTER_FILTER(VSTACK, vstack, vf); REGISTER_FILTER(W3FDIF, w3fdif, vf); diff --git a/libavfilter/vf_vif.c b/libavfilter/vf_vif.c new file mode 100644 index 0000000..4dfe77c --- /dev/null +++ b/libavfilter/vf_vif.c @@ -0,0 +1,646 @@ +/* + * Copyright (c) 2017 Ronald S. Bultje + * Copyright (c) 2017 Ashish Pratap Singh + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * Calculate VIF between two input videos. + */ + +#include "libavutil/avstring.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "avfilter.h" +#include "dualinput.h" +#include "drawutils.h" +#include "formats.h" +#include "internal.h" +#include "vif.h" +#include "video.h" + +typedef struct VIFContext { + const AVClass *class; + FFDualInputContext dinput; + const AVPixFmtDescriptor *desc; + int width; + int height; + float *data_buf; + float *temp; + float *ref_data; + float *main_data; + double vif_sum; + uint64_t nb_frames; +} VIFContext; + +#define OFFSET(x) offsetof(VIFContext, x) +#define MAX_ALIGN 32 +#define ALIGN_CEIL(x) ((x) + ((x) % MAX_ALIGN ? MAX_ALIGN - (x) % MAX_ALIGN : 0)) +#define OPT_RANGE_PIXEL_OFFSET (-128) + +static const AVOption vif_options[] = { + { NULL } +}; + +AVFILTER_DEFINE_CLASS(vif); + +const int vif_filter1d_width1[4] = { 17, 9, 5, 3 }; +const float vif_filter1d_table[4][17] = { + { 0x1.e8a77p-8, 0x1.d373b2p-7, 0x1.9a1cf6p-6, 0x1.49fd9ep-5, 0x1.e7092ep-5, + 0x1.49a044p-4, 0x1.99350ep-4, 0x1.d1e76ap-4, 0x1.e67f8p-4, 0x1.d1e76ap-4, + 0x1.99350ep-4, 0x1.49a044p-4, 0x1.e7092ep-5, 0x1.49fd9ep-5, 0x1.9a1cf6p-6, + 0x1.d373b2p-7, 0x1.e8a77p-8 }, + { 0x1.36efdap-6, 0x1.c9eaf8p-5, 0x1.ef4ac2p-4, 0x1.897424p-3, 0x1.cb1b88p-3, + 0x1.897424p-3, 0x1.ef4ac2p-4, 0x1.c9eaf8p-5, 0x1.36efdap-6 }, + { 0x1.be5f0ep-5, 0x1.f41fd6p-3, 0x1.9c4868p-2, 0x1.f41fd6p-3, 0x1.be5f0ep-5 }, + { 0x1.54be4p-3, 0x1.55a0ep-1, 0x1.54be4p-3 } +}; + +static void vif_dec2(const float *src, float *dst, int src_w, int src_h, + int src_stride, int dst_stride) +{ + int src_px_stride = src_stride / sizeof(float); + int dst_px_stride = dst_stride / sizeof(float); + + int i, j; + + /** decimation by 2 in each direction (after gaussian blur check) */ + for (i = 0; i < src_h / 2; i++) { + for (j = 0; j < src_w / 2; j++) { + dst[i * dst_px_stride + j] = src[(i * 2) * src_px_stride + (j * 2)]; + } + } +} + +static float vif_sum(const float *x, int w, int h, int stride) +{ + int px_stride = stride / sizeof(float); + int i, j; + + float sum = 0; + + for (i = 0; i < h; i++) { + float sum_inner = 0; + + for (j = 0; j < w; j++) { + sum_inner += x[i * px_stride + j]; + } + + sum += sum_inner; + } + + return sum; +} + +static void vif_statistic(const float *mu1_sq, const float *mu2_sq, + const float *mu1_mu2, const float *xx_filt, + const float *yy_filt, const float *xy_filt, + float *num, float *den, int w, int h, + int mu1_sq_stride, int mu2_sq_stride, + int mu1_mu2_stride, int xx_filt_stride, + int yy_filt_stride, int xy_filt_stride, + int num_stride, int den_stride) +{ + static const float sigma_nsq = 2; + static const float sigma_max_inv = 4.0/(255.0*255.0); + + int mu1_sq_px_stride = mu1_sq_stride / sizeof(float); + int mu2_sq_px_stride = mu2_sq_stride / sizeof(float); + int mu1_mu2_px_stride = mu1_mu2_stride / sizeof(float); + int xx_filt_px_stride = xx_filt_stride / sizeof(float); + int yy_filt_px_stride = yy_filt_stride / sizeof(float); + int xy_filt_px_stride = xy_filt_stride / sizeof(float); + int num_px_stride = num_stride / sizeof(float); + int den_px_stride = den_stride / sizeof(float); + + float mu1_sq_val, mu2_sq_val, mu1_mu2_val, xx_filt_val, yy_filt_val, xy_filt_val; + float sigma1_sq, sigma2_sq, sigma12, g, sv_sq; + float num_val, den_val; + int i, j; + + for (i = 0; i < h; i++) { + for (j = 0; j < w; j++) { + mu1_sq_val = mu1_sq[i * mu1_sq_px_stride + j]; + mu2_sq_val = mu2_sq[i * mu2_sq_px_stride + j]; + mu1_mu2_val = mu1_mu2[i * mu1_mu2_px_stride + j]; + xx_filt_val = xx_filt[i * xx_filt_px_stride + j]; + yy_filt_val = yy_filt[i * yy_filt_px_stride + j]; + xy_filt_val = xy_filt[i * xy_filt_px_stride + j]; + + sigma1_sq = xx_filt_val - mu1_sq_val; + sigma2_sq = yy_filt_val - mu2_sq_val; + sigma12 = xy_filt_val - mu1_mu2_val; + + if (sigma1_sq < sigma_nsq) { + num_val = 1.0 - sigma2_sq*sigma_max_inv; + den_val = 1.0; + } else { + sv_sq = (sigma2_sq + sigma_nsq) * sigma1_sq; + if( sigma12 < 0 ) { + num_val = 0.0; + } else { + g = sv_sq - sigma12 * sigma12; + num_val = log2f(sv_sq / g); + } + den_val = log2f(1.0f + sigma1_sq / sigma_nsq); + } + + num[i * num_px_stride + j] = num_val; + den[i * den_px_stride + j] = den_val; + } + } +} + +static void vif_xx_yy_xy(const float *x, const float *y, float *xx, float *yy, + float *xy, int w, int h, int xstride, int ystride, + int xxstride, int yystride, int xystride) +{ + int x_px_stride = xstride / sizeof(float); + int y_px_stride = ystride / sizeof(float); + int xx_px_stride = xxstride / sizeof(float); + int yy_px_stride = yystride / sizeof(float); + int xy_px_stride = xystride / sizeof(float); + + int i, j; + + float xval, yval, xxval, yyval, xyval; + + for (i = 0; i < h; i++) { + for (j = 0; j < w; j++) { + xval = x[i * x_px_stride + j]; + yval = y[i * y_px_stride + j]; + + xxval = xval * xval; + yyval = yval * yval; + xyval = xval * yval; + + xx[i * xx_px_stride + j] = xxval; + yy[i * yy_px_stride + j] = yyval; + xy[i * xy_px_stride + j] = xyval; + } + } +} + +static void vif_filter1d(const float *filter, const float *src, float *dst, + float *temp_buf, int w, int h, int src_stride, + int dst_stride, int filt_w, float *temp) +{ + int src_px_stride = src_stride / sizeof(float); + int dst_px_stride = dst_stride / sizeof(float); + + float filt_coeff, img_coeff; + + int i, j, filt_i, filt_j, ii, jj; + + for (i = 0; i < h; i++) { + /** Vertical pass. */ + for (j = 0; j < w; j++) { + float sum = 0; + + for (filt_i = 0; filt_i < filt_w; filt_i++) { + filt_coeff = filter[filt_i]; + + ii = i - filt_w / 2 + filt_i; + ii = ii < 0 ? -ii : (ii >= h ? 2 * h - ii - 1 : ii); + + img_coeff = src[ii * src_px_stride + j]; + + sum += filt_coeff * img_coeff; + } + + temp[j] = sum; + } + + /** Horizontal pass. */ + for (j = 0; j < w; j++) { + float sum = 0; + + for (filt_j = 0; filt_j < filt_w; filt_j++) { + filt_coeff = filter[filt_j]; + + jj = j - filt_w / 2 + filt_j; + jj = jj < 0 ? -jj : (jj >= w ? 2 * w - jj - 1 : jj); + + img_coeff = temp[jj]; + + sum += filt_coeff * img_coeff; + } + + dst[i * dst_px_stride + j] = sum; + } + } +} + +int compute_vif2(const float *ref, const float *main, int w, int h, + int ref_stride, int main_stride, double *score, + double *score_num, double *score_den, double *scores, + float *data_buf, float *temp) +{ + char *data_top; + + float *ref_scale; + float *main_scale; + float *ref_sq; + float *main_sq; + float *ref_main; + + float *mu1; + float *mu2; + float *mu1_sq; + float *mu2_sq; + float *mu1_mu2; + float *ref_sq_filt; + float *main_sq_filt; + float *ref_main_filt; + float *num_array; + float *den_array; + float *temp_buf; + + const float *curr_ref_scale = ref; + const float *curr_main_scale = main; + int curr_ref_stride = ref_stride; + int curr_main_stride = main_stride; + + int buf_stride = ALIGN_CEIL(w * sizeof(float)); + size_t buf_sz = (size_t)buf_stride * h; + + double num = 0; + double den = 0; + + int scale; + int ret = 1; + + data_top = (char *) data_buf; + + ref_scale = (float *) data_top; + data_top += buf_sz; + + main_scale = (float *) data_top; + data_top += buf_sz; + + ref_sq = (float *) data_top; + data_top += buf_sz; + + main_sq = (float *) data_top; + data_top += buf_sz; + + ref_main = (float *) data_top; + data_top += buf_sz; + + mu1 = (float *) data_top; + data_top += buf_sz; + + mu2 = (float *) data_top; + data_top += buf_sz; + + mu1_sq = (float *) data_top; + data_top += buf_sz; + + mu2_sq = (float *) data_top; + data_top += buf_sz; + + mu1_mu2 = (float *) data_top; + data_top += buf_sz; + + ref_sq_filt = (float *) data_top; + data_top += buf_sz; + + main_sq_filt = (float *) data_top; + data_top += buf_sz; + + ref_main_filt = (float *) data_top; + data_top += buf_sz; + + num_array = (float *) data_top; + data_top += buf_sz; + + den_array = (float *) data_top; + data_top += buf_sz; + + temp_buf = (float *) data_top; + data_top += buf_sz; + + for (scale = 0; scale < 4; scale++) { + const float *filter = vif_filter1d_table[scale]; + int filter_width = vif_filter1d_width1[scale]; + + int buf_valid_w = w; + int buf_valid_h = h; + + if (scale > 0) { + vif_filter1d(filter, curr_ref_scale, mu1, temp_buf, w, h, + curr_ref_stride, buf_stride, filter_width, temp); + vif_filter1d(filter, curr_main_scale, mu2, temp_buf, w, h, + curr_main_stride, buf_stride, filter_width, temp); + + vif_dec2(mu1, ref_scale, buf_valid_w, buf_valid_h, buf_stride, + buf_stride); + vif_dec2(mu2, main_scale, buf_valid_w, buf_valid_h, buf_stride, + buf_stride); + + w = buf_valid_w / 2; + h = buf_valid_h / 2; + + buf_valid_w = w; + buf_valid_h = h; + + curr_ref_scale = ref_scale; + curr_main_scale = main_scale; + + curr_ref_stride = buf_stride; + curr_main_stride = buf_stride; + } + + vif_filter1d(filter, curr_ref_scale, mu1, temp_buf, w, h, curr_ref_stride, + buf_stride, filter_width, temp); + vif_filter1d(filter, curr_main_scale, mu2, temp_buf, w, h, curr_main_stride, + buf_stride, filter_width, temp); + + vif_xx_yy_xy(mu1, mu2, mu1_sq, mu2_sq, mu1_mu2, w, h, buf_stride, + buf_stride, buf_stride, buf_stride, buf_stride); + + vif_xx_yy_xy(curr_ref_scale, curr_main_scale, ref_sq, main_sq, ref_main, + w, h, curr_ref_stride, curr_main_stride, buf_stride, + buf_stride, buf_stride); + + vif_filter1d(filter, ref_sq, ref_sq_filt, temp_buf, w, h, buf_stride, + buf_stride, filter_width, temp); + vif_filter1d(filter, main_sq, main_sq_filt, temp_buf, w, h, buf_stride, + buf_stride, filter_width, temp); + vif_filter1d(filter, ref_main, ref_main_filt, temp_buf, w, h, buf_stride, + buf_stride, filter_width, temp); + + vif_statistic(mu1_sq, mu2_sq, mu1_mu2, ref_sq_filt, main_sq_filt, + ref_main_filt, num_array, den_array, w, h, buf_stride, + buf_stride, buf_stride, buf_stride, buf_stride, + buf_stride, buf_stride, buf_stride); + + num = vif_sum(num_array, buf_valid_w, buf_valid_h, buf_stride); + den = vif_sum(den_array, buf_valid_w, buf_valid_h, buf_stride); + + scores[2*scale] = num; + scores[2*scale+1] = den; + } + + *score_num = 0.0; + *score_den = 0.0; + for (scale = 0; scale < 4; ++scale) { + *score_num += scores[2*scale]; + *score_den += scores[2*scale+1]; + } + + if (*score_den == 0.0) { + *score = 1.0f; + } else { + *score = (*score_num) / (*score_den); + } + + ret = 0; + + return ret; +} + +#define offset_fn(type, bits) \ + static void offset_##bits##bit(VIFContext *s, const AVFrame *ref, AVFrame *main, int stride) \ +{ \ + int w = s->width; \ + int h = s->height; \ + int i,j; \ + \ + int ref_stride = ref->linesize[0]; \ + int main_stride = main->linesize[0]; \ + \ + const type *ref_ptr = (const type *) ref->data[0]; \ + const type *main_ptr = (const type *) main->data[0]; \ + \ + float *ref_ptr_data = s->ref_data; \ + float *main_ptr_data = s->main_data; \ + \ + for(i = 0; i < h; i++) { \ + for(j = 0; j < w; j++) { \ + ref_ptr_data[j] = (float) ref_ptr[j] + OPT_RANGE_PIXEL_OFFSET; \ + main_ptr_data[j] = (float) main_ptr[j] + OPT_RANGE_PIXEL_OFFSET; \ + } \ + ref_ptr += ref_stride / sizeof(type); \ + ref_ptr_data += stride / sizeof(float); \ + main_ptr += main_stride / sizeof(type); \ + main_ptr_data += stride / sizeof(float); \ + } \ +} + +offset_fn(uint8_t, 8); +offset_fn(uint16_t, 10); + +static void set_meta(AVDictionary **metadata, const char *key, float d) +{ + char value[128]; + snprintf(value, sizeof(value), "%0.2f", d); + av_dict_set(metadata, key, value, 0); +} + +static AVFrame *do_vif(AVFilterContext *ctx, AVFrame *main, const AVFrame *ref) +{ + VIFContext *s = ctx->priv; + AVDictionary **metadata = &main->metadata; + + double score = 0.0; + double score_num = 0.0; + double score_den = 0.0; + double scores[8]; + + int w = s->width; + int h = s->height; + + double stride; + + stride = ALIGN_CEIL(w * sizeof(float)); + + /** Offset ref and main pixel by OPT_RANGE_PIXEL_OFFSET */ + if (s->desc->comp[0].depth <= 8) { + offset_8bit(s, ref, main, stride); + } else { + offset_10bit(s, ref, main, stride); + } + + compute_vif2(s->ref_data, s->main_data, w, h, stride, stride, &score, + &score_num, &score_den, scores, s->data_buf, s->temp); + + set_meta(metadata, "lavfi.vif.score", score); + + s->nb_frames++; + + s->vif_sum += score; + + return main; +} + +static av_cold int init(AVFilterContext *ctx) +{ + VIFContext *s = ctx->priv; + + s->dinput.process = do_vif; + + return 0; +} + +static int query_formats(AVFilterContext *ctx) +{ + static const enum AVPixelFormat pix_fmts[] = { + AV_PIX_FMT_YUV444P, AV_PIX_FMT_YUV422P, AV_PIX_FMT_YUV420P, + AV_PIX_FMT_YUV444P10LE, AV_PIX_FMT_YUV422P10LE, AV_PIX_FMT_YUV420P10LE, + AV_PIX_FMT_NONE + }; + + AVFilterFormats *fmts_list = ff_make_format_list(pix_fmts); + if (!fmts_list) + return AVERROR(ENOMEM); + return ff_set_common_formats(ctx, fmts_list); +} + +static int config_input_ref(AVFilterLink *inlink) +{ + AVFilterContext *ctx = inlink->dst; + VIFContext *s = ctx->priv; + int stride; + size_t data_sz; + + if (ctx->inputs[0]->w != ctx->inputs[1]->w || + ctx->inputs[0]->h != ctx->inputs[1]->h) { + av_log(ctx, AV_LOG_ERROR, "Width and height of input videos must be same.\n"); + return AVERROR(EINVAL); + } + if (ctx->inputs[0]->format != ctx->inputs[1]->format) { + av_log(ctx, AV_LOG_ERROR, "Inputs must be of same pixel format.\n"); + return AVERROR(EINVAL); + } + + s->desc = av_pix_fmt_desc_get(inlink->format); + s->width = ctx->inputs[0]->w; + s->height = ctx->inputs[0]->h; + + stride = ALIGN_CEIL(s->width * sizeof(float)); + data_sz = (size_t)stride * s->height; + + if (SIZE_MAX / data_sz < 15) { + av_log(ctx, AV_LOG_ERROR, "error: SIZE_MAX / buf_sz < 15\n"); + return AVERROR(EINVAL); + } + + if (!(s->data_buf = av_malloc(data_sz * 16))) { + av_log(ctx, AV_LOG_ERROR, "error: av_malloc failed for data_buf.\n"); + return AVERROR(ENOMEM); + } + if (!(s->ref_data = av_malloc(data_sz))) { + av_log(ctx, AV_LOG_ERROR, "error: av_malloc failed for ref_data.\n"); + return AVERROR(ENOMEM); + } + if (!(s->main_data = av_malloc(data_sz))) { + av_log(ctx, AV_LOG_ERROR, "error: av_malloc failed for main_data.\n"); + return AVERROR(ENOMEM); + } + if (!(s->temp = av_malloc(s->width * sizeof(float)))) { + av_log(ctx, AV_LOG_ERROR, "error: av_malloc failed for temp.\n"); + return AVERROR(ENOMEM); + } + + return 0; +} + + +static int config_output(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + VIFContext *s = ctx->priv; + AVFilterLink *mainlink = ctx->inputs[0]; + int ret; + + outlink->w = mainlink->w; + outlink->h = mainlink->h; + outlink->time_base = mainlink->time_base; + outlink->sample_aspect_ratio = mainlink->sample_aspect_ratio; + outlink->frame_rate = mainlink->frame_rate; + if ((ret = ff_dualinput_init(ctx, &s->dinput)) < 0) + return ret; + + return 0; +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *inpicref) +{ + VIFContext *s = inlink->dst->priv; + return ff_dualinput_filter_frame(&s->dinput, inlink, inpicref); +} + +static int request_frame(AVFilterLink *outlink) +{ + VIFContext *s = outlink->src->priv; + return ff_dualinput_request_frame(&s->dinput, outlink); +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + VIFContext *s = ctx->priv; + + if (s->nb_frames > 0) { + av_log(ctx, AV_LOG_INFO, "VIF AVG: %.3f\n", s->vif_sum / s->nb_frames); + } + + av_free(s->data_buf); + av_free(s->ref_data); + av_free(s->main_data); + av_free(s->temp); + + ff_dualinput_uninit(&s->dinput); +} + +static const AVFilterPad vif_inputs[] = { + { + .name = "main", + .type = AVMEDIA_TYPE_VIDEO, + .filter_frame = filter_frame, + },{ + .name = "reference", + .type = AVMEDIA_TYPE_VIDEO, + .filter_frame = filter_frame, + .config_props = config_input_ref, + }, + { NULL } +}; + +static const AVFilterPad vif_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_output, + .request_frame = request_frame, + }, + { NULL } +}; + +AVFilter ff_vf_vif = { + .name = "vif", + .description = NULL_IF_CONFIG_SMALL("Calculate the VIF between two video streams."), + .init = init, + .uninit = uninit, + .query_formats = query_formats, + .priv_size = sizeof(VIFContext), + .priv_class = &vif_class, + .inputs = vif_inputs, + .outputs = vif_outputs, +}; diff --git a/libavfilter/vif.h b/libavfilter/vif.h new file mode 100644 index 0000000..9074efa --- /dev/null +++ b/libavfilter/vif.h @@ -0,0 +1,30 @@ +/* + * Copyright (c) 2017 Ronald S. Bultje + * Copyright (c) 2017 Ashish Pratap Singh + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVFILTER_VIF_H +#define AVFILTER_VIF_H + +int compute_vif2(const float *ref, const float *main, int w, int h, + int ref_stride, int main_stride, double *score, + double *score_num, double *score_den, double *scores, + float *data_buf, float *temp); + +#endif /* AVFILTER_VIF_H */