diff mbox

[FFmpeg-devel] avfilter: add VIF filter

Message ID 1501317989-5062-1-git-send-email-ashk43712@gmail.com
State New
Headers show

Commit Message

Ashish Singh July 29, 2017, 8:46 a.m. UTC
From: Ashish Singh <ashk43712@gmail.com>

This is Visual Information Fidelity (VIF) filter and one of the component
filters of VMAF. It outputs the average VIF score over all frames.

Signed-off-by: Ashish Singh <ashk43712@gmail.com>
---
 Changelog                |   1 +
 doc/filters.texi         |  19 ++
 libavfilter/Makefile     |   1 +
 libavfilter/allfilters.c |   1 +
 libavfilter/vf_vif.c     | 646 +++++++++++++++++++++++++++++++++++++++++++++++
 libavfilter/vif.h        |  30 +++
 6 files changed, 698 insertions(+)
 create mode 100644 libavfilter/vf_vif.c
 create mode 100644 libavfilter/vif.h

Comments

Moritz Barsnick July 29, 2017, 9:45 a.m. UTC | #1
On Sat, Jul 29, 2017 at 14:16:29 +0530, Ashish Pratap Singh wrote:

> +Both input videos must have the same resolution and pixel format for
> +this filter to work correctly. Also it assumes that both inputs
> +have the same number of frames, which are compared one by one.

Nit, for all these filters: If the inputs are checked, you should
probably just write "must have the same resolution and pixel format",
or "must have the same resolution and pixel format to work". Since it's
checked, there's no chance of working incorrectly.

> +const int vif_filter1d_width1[4] = { 17, 9, 5, 3 };
> +const float vif_filter1d_table[4][17] = {
> +    { 0x1.e8a77p-8,  0x1.d373b2p-7, 0x1.9a1cf6p-6, 0x1.49fd9ep-5, 0x1.e7092ep-5,

My brain has a hard time interpreting scientific hex notation, but I
guess it's fine. ;-)

> +                num_val = 1.0 - sigma2_sq*sigma_max_inv;
> +                den_val = 1.0;

1.0f to avoid double promotion.

> +    if (*score_den == 0.0) {
> +        *score = 1.0f;

Nit: I know the compiler fixes the right hand side, but since you're
assigning to a double, you shouldn't force the const to a float. "0.0"
or "0" as you chose elsewhere is more consistent.

> +    int i,j; \
[...]
> +    for(i = 0; i < h; i++) { \
> +        for(j = 0; j < w; j++) { \

Minor nit: Whitespace. "for (", "i, j"

> +        av_log(ctx, AV_LOG_ERROR, "Width and height of input videos must be same.\n");

Either "must be the same" or "must be identical".

> +        av_log(ctx, AV_LOG_ERROR, "error: av_malloc failed for data_buf.\n");

Are these messages of any help to anyone?

Moritz
Paul B Mahol July 29, 2017, 9:51 a.m. UTC | #2
On 7/29/17, Moritz Barsnick <barsnick@gmx.net> wrote:
> On Sat, Jul 29, 2017 at 14:16:29 +0530, Ashish Pratap Singh wrote:
>
>> +Both input videos must have the same resolution and pixel format for
>> +this filter to work correctly. Also it assumes that both inputs
>> +have the same number of frames, which are compared one by one.
>
> Nit, for all these filters: If the inputs are checked, you should
> probably just write "must have the same resolution and pixel format",
> or "must have the same resolution and pixel format to work". Since it's
> checked, there's no chance of working incorrectly.
>
>> +const int vif_filter1d_width1[4] = { 17, 9, 5, 3 };
>> +const float vif_filter1d_table[4][17] = {
>> +    { 0x1.e8a77p-8,  0x1.d373b2p-7, 0x1.9a1cf6p-6, 0x1.49fd9ep-5,
>> 0x1.e7092ep-5,
>
> My brain has a hard time interpreting scientific hex notation, but I
> guess it's fine. ;-)
>
>> +                num_val = 1.0 - sigma2_sq*sigma_max_inv;
>> +                den_val = 1.0;
>
> 1.0f to avoid double promotion.
>
>> +    if (*score_den == 0.0) {
>> +        *score = 1.0f;
>
> Nit: I know the compiler fixes the right hand side, but since you're
> assigning to a double, you shouldn't force the const to a float. "0.0"
> or "0" as you chose elsewhere is more consistent.
>
>> +    int i,j; \
> [...]
>> +    for(i = 0; i < h; i++) { \
>> +        for(j = 0; j < w; j++) { \
>
> Minor nit: Whitespace. "for (", "i, j"
>
>> +        av_log(ctx, AV_LOG_ERROR, "Width and height of input videos must
>> be same.\n");
>
> Either "must be the same" or "must be identical".
>
>> +        av_log(ctx, AV_LOG_ERROR, "error: av_malloc failed for
>> data_buf.\n");
>
> Are these messages of any help to anyone?

Remove this one message.
Ashish Singh July 30, 2017, 5:46 p.m. UTC | #3
Hi,

On Sat, Jul 29, 2017 at 3:15 PM, Moritz Barsnick <barsnick@gmx.net> wrote:

> On Sat, Jul 29, 2017 at 14:16:29 +0530, Ashish Pratap Singh wrote:
>
> > +Both input videos must have the same resolution and pixel format for
> > +this filter to work correctly. Also it assumes that both inputs
> > +have the same number of frames, which are compared one by one.
>
> Nit, for all these filters: If the inputs are checked, you should
> probably just write "must have the same resolution and pixel format",
> or "must have the same resolution and pixel format to work". Since it's
> checked, there's no chance of working incorrectly.
>
> > +const int vif_filter1d_width1[4] = { 17, 9, 5, 3 };
> > +const float vif_filter1d_table[4][17] = {
> > +    { 0x1.e8a77p-8,  0x1.d373b2p-7, 0x1.9a1cf6p-6, 0x1.49fd9ep-5,
> 0x1.e7092ep-5,
>
> My brain has a hard time interpreting scientific hex notation, but I
> guess it's fine. ;-)
>
> Later it will be converted to fixed point.

> > +                num_val = 1.0 - sigma2_sq*sigma_max_inv;
> > +                den_val = 1.0;
>
> 1.0f to avoid double promotion.
>
> > +    if (*score_den == 0.0) {
> > +        *score = 1.0f;
>
> Nit: I know the compiler fixes the right hand side, but since you're
> assigning to a double, you shouldn't force the const to a float. "0.0"
> or "0" as you chose elsewhere is more consistent.
>
> Ok, point noted.

> > +    int i,j; \
> [...]
> > +    for(i = 0; i < h; i++) { \
> > +        for(j = 0; j < w; j++) { \
>
> Minor nit: Whitespace. "for (", "i, j"
>
> > +        av_log(ctx, AV_LOG_ERROR, "Width and height of input videos
> must be same.\n");
>
> Either "must be the same" or "must be identical".
>
> > +        av_log(ctx, AV_LOG_ERROR, "error: av_malloc failed for
> data_buf.\n");
>
> Are these messages of any help to anyone?
>
What would be an appropriate error message or should I don't write anything
at all?

>
> Moritz
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
Ronald S. Bultje July 30, 2017, 9:54 p.m. UTC | #4
Hi,

On Sun, Jul 30, 2017 at 1:46 PM, Ashish Pratap Singh <ashk43712@gmail.com>
wrote:

> On Sat, Jul 29, 2017 at 3:15 PM, Moritz Barsnick <barsnick@gmx.net> wrote:
> > On Sat, Jul 29, 2017 at 14:16:29 +0530, Ashish Pratap Singh wrote:
> > > +        av_log(ctx, AV_LOG_ERROR, "error: av_malloc failed for
> > data_buf.\n");
> >
> > Are these messages of any help to anyone?
> >
> What would be an appropriate error message or should I don't write anything
> at all?


For -ENOMEM, you don't need to write an error message, the system error
code is clear enough.

Ronald
Ashish Singh July 31, 2017, 9:15 a.m. UTC | #5
On Mon, Jul 31, 2017 at 3:24 AM, Ronald S. Bultje <rsbultje@gmail.com>
wrote:

> Hi,
>
> On Sun, Jul 30, 2017 at 1:46 PM, Ashish Pratap Singh <ashk43712@gmail.com>
> wrote:
>
> > On Sat, Jul 29, 2017 at 3:15 PM, Moritz Barsnick <barsnick@gmx.net>
> wrote:
> > > On Sat, Jul 29, 2017 at 14:16:29 +0530, Ashish Pratap Singh wrote:
> > > > +        av_log(ctx, AV_LOG_ERROR, "error: av_malloc failed for
> > > data_buf.\n");
> > >
> > > Are these messages of any help to anyone?
> > >
> > What would be an appropriate error message or should I don't write
> anything
> > at all?
>
>
> For -ENOMEM, you don't need to write an error message, the system error
> code is clear enough.
>
> Ok, thanks.

> Ronald
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
diff mbox

Patch

diff --git a/Changelog b/Changelog
index 187ae79..68900ca 100644
--- a/Changelog
+++ b/Changelog
@@ -29,6 +29,7 @@  version <next>:
 - limiter video filter
 - libvmaf video filter
 - Dolby E decoder and SMPTE 337M demuxer
+- vif video filter
 
 version 3.3:
 - CrystalHD decoder moved to new decode API
diff --git a/doc/filters.texi b/doc/filters.texi
index 2324b96..df5056f 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -15026,6 +15026,25 @@  For example, to vertically flip a video with @command{ffmpeg}:
 ffmpeg -i in.avi -vf "vflip" out.avi
 @end example
 
+@section vif
+
+Obtain the average VIF (Visual Information Fidelity) between two input videos.
+
+This filter takes two input videos.
+
+Both input videos must have the same resolution and pixel format for
+this filter to work correctly. Also it assumes that both inputs
+have the same number of frames, which are compared one by one.
+
+The obtained average VIF score is printed through the logging system.
+
+In the below example the input file @file{main.mpg} being processed is compared
+with the reference file @file{ref.mpg}.
+
+@example
+ffmpeg -i main.mpg -i ref.mpg -lavfi vif -f null -
+@end example
+
 @anchor{vignette}
 @section vignette
 
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index ee16361..9999856 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -321,6 +321,7 @@  OBJS-$(CONFIG_VECTORSCOPE_FILTER)            += vf_vectorscope.o
 OBJS-$(CONFIG_VFLIP_FILTER)                  += vf_vflip.o
 OBJS-$(CONFIG_VIDSTABDETECT_FILTER)          += vidstabutils.o vf_vidstabdetect.o
 OBJS-$(CONFIG_VIDSTABTRANSFORM_FILTER)       += vidstabutils.o vf_vidstabtransform.o
+OBJS-$(CONFIG_VIF_FILTER)                    += vf_vif.o dualinput.o framesync.o
 OBJS-$(CONFIG_VIGNETTE_FILTER)               += vf_vignette.o
 OBJS-$(CONFIG_VSTACK_FILTER)                 += vf_stack.o framesync.o
 OBJS-$(CONFIG_W3FDIF_FILTER)                 += vf_w3fdif.o
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index b1c2d11..bd342e3 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -332,6 +332,7 @@  static void register_all(void)
     REGISTER_FILTER(VFLIP,          vflip,          vf);
     REGISTER_FILTER(VIDSTABDETECT,  vidstabdetect,  vf);
     REGISTER_FILTER(VIDSTABTRANSFORM, vidstabtransform, vf);
+    REGISTER_FILTER(VIF,            vif,            vf);
     REGISTER_FILTER(VIGNETTE,       vignette,       vf);
     REGISTER_FILTER(VSTACK,         vstack,         vf);
     REGISTER_FILTER(W3FDIF,         w3fdif,         vf);
diff --git a/libavfilter/vf_vif.c b/libavfilter/vf_vif.c
new file mode 100644
index 0000000..4dfe77c
--- /dev/null
+++ b/libavfilter/vf_vif.c
@@ -0,0 +1,646 @@ 
+/*
+ * Copyright (c) 2017 Ronald S. Bultje <rsbultje@gmail.com>
+ * Copyright (c) 2017 Ashish Pratap Singh <ashk43712@gmail.com>
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+/**
+ * @file
+ * Calculate VIF between two input videos.
+ */
+
+#include "libavutil/avstring.h"
+#include "libavutil/opt.h"
+#include "libavutil/pixdesc.h"
+#include "avfilter.h"
+#include "dualinput.h"
+#include "drawutils.h"
+#include "formats.h"
+#include "internal.h"
+#include "vif.h"
+#include "video.h"
+
+typedef struct VIFContext {
+    const AVClass *class;
+    FFDualInputContext dinput;
+    const AVPixFmtDescriptor *desc;
+    int width;
+    int height;
+    float *data_buf;
+    float *temp;
+    float *ref_data;
+    float *main_data;
+    double vif_sum;
+    uint64_t nb_frames;
+} VIFContext;
+
+#define OFFSET(x) offsetof(VIFContext, x)
+#define MAX_ALIGN 32
+#define ALIGN_CEIL(x) ((x) + ((x) % MAX_ALIGN ? MAX_ALIGN - (x) % MAX_ALIGN : 0))
+#define OPT_RANGE_PIXEL_OFFSET (-128)
+
+static const AVOption vif_options[] = {
+    { NULL }
+};
+
+AVFILTER_DEFINE_CLASS(vif);
+
+const int vif_filter1d_width1[4] = { 17, 9, 5, 3 };
+const float vif_filter1d_table[4][17] = {
+    { 0x1.e8a77p-8,  0x1.d373b2p-7, 0x1.9a1cf6p-6, 0x1.49fd9ep-5, 0x1.e7092ep-5,
+      0x1.49a044p-4, 0x1.99350ep-4, 0x1.d1e76ap-4, 0x1.e67f8p-4,  0x1.d1e76ap-4,
+      0x1.99350ep-4, 0x1.49a044p-4, 0x1.e7092ep-5, 0x1.49fd9ep-5, 0x1.9a1cf6p-6,
+      0x1.d373b2p-7, 0x1.e8a77p-8 },
+    { 0x1.36efdap-6, 0x1.c9eaf8p-5, 0x1.ef4ac2p-4, 0x1.897424p-3, 0x1.cb1b88p-3,
+      0x1.897424p-3, 0x1.ef4ac2p-4, 0x1.c9eaf8p-5, 0x1.36efdap-6 },
+    { 0x1.be5f0ep-5, 0x1.f41fd6p-3, 0x1.9c4868p-2, 0x1.f41fd6p-3, 0x1.be5f0ep-5 },
+    { 0x1.54be4p-3,  0x1.55a0ep-1,  0x1.54be4p-3 }
+};
+
+static void vif_dec2(const float *src, float *dst, int src_w, int src_h,
+                     int src_stride, int dst_stride)
+{
+    int src_px_stride = src_stride / sizeof(float);
+    int dst_px_stride = dst_stride / sizeof(float);
+
+    int i, j;
+
+    /** decimation by 2 in each direction (after gaussian blur check) */
+    for (i = 0; i < src_h / 2; i++) {
+        for (j = 0; j < src_w / 2; j++) {
+            dst[i * dst_px_stride + j] = src[(i * 2) * src_px_stride + (j * 2)];
+        }
+    }
+}
+
+static float vif_sum(const float *x, int w, int h, int stride)
+{
+    int px_stride = stride / sizeof(float);
+    int i, j;
+
+    float sum = 0;
+
+    for (i = 0; i < h; i++) {
+        float sum_inner = 0;
+
+        for (j = 0; j < w; j++) {
+            sum_inner += x[i * px_stride + j];
+        }
+
+        sum += sum_inner;
+    }
+
+    return sum;
+}
+
+static void vif_statistic(const float *mu1_sq, const float *mu2_sq,
+                          const float *mu1_mu2, const float *xx_filt,
+                          const float *yy_filt, const float *xy_filt,
+                          float *num, float *den, int w, int h,
+                          int mu1_sq_stride, int mu2_sq_stride,
+                          int mu1_mu2_stride, int xx_filt_stride,
+                          int yy_filt_stride, int xy_filt_stride,
+                          int num_stride, int den_stride)
+{
+    static const float sigma_nsq = 2;
+    static const float sigma_max_inv = 4.0/(255.0*255.0);
+
+    int mu1_sq_px_stride  = mu1_sq_stride / sizeof(float);
+    int mu2_sq_px_stride  = mu2_sq_stride / sizeof(float);
+    int mu1_mu2_px_stride = mu1_mu2_stride / sizeof(float);
+    int xx_filt_px_stride = xx_filt_stride / sizeof(float);
+    int yy_filt_px_stride = yy_filt_stride / sizeof(float);
+    int xy_filt_px_stride = xy_filt_stride / sizeof(float);
+    int num_px_stride = num_stride / sizeof(float);
+    int den_px_stride = den_stride / sizeof(float);
+
+    float mu1_sq_val, mu2_sq_val, mu1_mu2_val, xx_filt_val, yy_filt_val, xy_filt_val;
+    float sigma1_sq, sigma2_sq, sigma12, g, sv_sq;
+    float num_val, den_val;
+    int i, j;
+
+    for (i = 0; i < h; i++) {
+        for (j = 0; j < w; j++) {
+            mu1_sq_val  = mu1_sq[i * mu1_sq_px_stride + j];
+            mu2_sq_val  = mu2_sq[i * mu2_sq_px_stride + j];
+            mu1_mu2_val = mu1_mu2[i * mu1_mu2_px_stride + j];
+            xx_filt_val = xx_filt[i * xx_filt_px_stride + j];
+            yy_filt_val = yy_filt[i * yy_filt_px_stride + j];
+            xy_filt_val = xy_filt[i * xy_filt_px_stride + j];
+
+            sigma1_sq = xx_filt_val - mu1_sq_val;
+            sigma2_sq = yy_filt_val - mu2_sq_val;
+            sigma12   = xy_filt_val - mu1_mu2_val;
+
+            if (sigma1_sq < sigma_nsq) {
+                num_val = 1.0 - sigma2_sq*sigma_max_inv;
+                den_val = 1.0;
+            } else {
+                sv_sq = (sigma2_sq + sigma_nsq) * sigma1_sq;
+                if( sigma12 < 0 ) {
+                    num_val = 0.0;
+                } else {
+                    g = sv_sq - sigma12 * sigma12;
+                    num_val = log2f(sv_sq / g);
+                }
+                den_val = log2f(1.0f + sigma1_sq / sigma_nsq);
+            }
+
+            num[i * num_px_stride + j] = num_val;
+            den[i * den_px_stride + j] = den_val;
+        }
+    }
+}
+
+static void vif_xx_yy_xy(const float *x, const float *y, float *xx, float *yy,
+                         float *xy, int w, int h, int xstride, int ystride,
+                         int xxstride, int yystride, int xystride)
+{
+    int x_px_stride = xstride / sizeof(float);
+    int y_px_stride = ystride / sizeof(float);
+    int xx_px_stride = xxstride / sizeof(float);
+    int yy_px_stride = yystride / sizeof(float);
+    int xy_px_stride = xystride / sizeof(float);
+
+    int i, j;
+
+    float xval, yval, xxval, yyval, xyval;
+
+    for (i = 0; i < h; i++) {
+        for (j = 0; j < w; j++) {
+            xval = x[i * x_px_stride + j];
+            yval = y[i * y_px_stride + j];
+
+            xxval = xval * xval;
+            yyval = yval * yval;
+            xyval = xval * yval;
+
+            xx[i * xx_px_stride + j] = xxval;
+            yy[i * yy_px_stride + j] = yyval;
+            xy[i * xy_px_stride + j] = xyval;
+        }
+    }
+}
+
+static void vif_filter1d(const float *filter, const float *src, float *dst,
+                         float *temp_buf, int w, int h, int src_stride,
+                         int dst_stride, int filt_w, float *temp)
+{
+    int src_px_stride = src_stride / sizeof(float);
+    int dst_px_stride = dst_stride / sizeof(float);
+
+    float filt_coeff, img_coeff;
+
+    int i, j, filt_i, filt_j, ii, jj;
+
+    for (i = 0; i < h; i++) {
+        /** Vertical pass. */
+        for (j = 0; j < w; j++) {
+            float sum = 0;
+
+            for (filt_i = 0; filt_i < filt_w; filt_i++) {
+                filt_coeff = filter[filt_i];
+
+                ii = i - filt_w / 2 + filt_i;
+                ii = ii < 0 ? -ii : (ii >= h ? 2 * h - ii - 1 : ii);
+
+                img_coeff = src[ii * src_px_stride + j];
+
+                sum += filt_coeff * img_coeff;
+            }
+
+            temp[j] = sum;
+        }
+
+        /** Horizontal pass. */
+        for (j = 0; j < w; j++) {
+            float sum = 0;
+
+            for (filt_j = 0; filt_j < filt_w; filt_j++) {
+                filt_coeff = filter[filt_j];
+
+                jj = j - filt_w / 2 + filt_j;
+                jj = jj < 0 ? -jj : (jj >= w ? 2 * w - jj - 1 : jj);
+
+                img_coeff = temp[jj];
+
+                sum += filt_coeff * img_coeff;
+            }
+
+            dst[i * dst_px_stride + j] = sum;
+        }
+    }
+}
+
+int compute_vif2(const float *ref, const float *main, int w, int h,
+                 int ref_stride, int main_stride, double *score,
+                 double *score_num, double *score_den, double *scores,
+                 float *data_buf, float *temp)
+{
+    char *data_top;
+
+    float *ref_scale;
+    float *main_scale;
+    float *ref_sq;
+    float *main_sq;
+    float *ref_main;
+
+    float *mu1;
+    float *mu2;
+    float *mu1_sq;
+    float *mu2_sq;
+    float *mu1_mu2;
+    float *ref_sq_filt;
+    float *main_sq_filt;
+    float *ref_main_filt;
+    float *num_array;
+    float *den_array;
+    float *temp_buf;
+
+    const float *curr_ref_scale = ref;
+    const float *curr_main_scale = main;
+    int curr_ref_stride = ref_stride;
+    int curr_main_stride = main_stride;
+
+    int buf_stride = ALIGN_CEIL(w * sizeof(float));
+    size_t buf_sz = (size_t)buf_stride * h;
+
+    double num = 0;
+    double den = 0;
+
+    int scale;
+    int ret = 1;
+
+    data_top = (char *) data_buf;
+
+    ref_scale = (float *) data_top;
+    data_top += buf_sz;
+
+    main_scale = (float *) data_top;
+    data_top += buf_sz;
+
+    ref_sq = (float *) data_top;
+    data_top += buf_sz;
+
+    main_sq = (float *) data_top;
+    data_top += buf_sz;
+
+    ref_main = (float *) data_top;
+    data_top += buf_sz;
+
+    mu1 = (float *) data_top;
+    data_top += buf_sz;
+
+    mu2 = (float *) data_top;
+    data_top += buf_sz;
+
+    mu1_sq = (float *) data_top;
+    data_top += buf_sz;
+
+    mu2_sq = (float *) data_top;
+    data_top += buf_sz;
+
+    mu1_mu2 = (float *) data_top;
+    data_top += buf_sz;
+
+    ref_sq_filt = (float *) data_top;
+    data_top += buf_sz;
+
+    main_sq_filt = (float *) data_top;
+    data_top += buf_sz;
+
+    ref_main_filt = (float *) data_top;
+    data_top += buf_sz;
+
+    num_array = (float *) data_top;
+    data_top += buf_sz;
+
+    den_array = (float *) data_top;
+    data_top += buf_sz;
+
+    temp_buf = (float *) data_top;
+    data_top += buf_sz;
+
+    for (scale = 0; scale < 4; scale++) {
+        const float *filter = vif_filter1d_table[scale];
+        int filter_width = vif_filter1d_width1[scale];
+
+        int buf_valid_w = w;
+        int buf_valid_h = h;
+
+        if (scale > 0) {
+            vif_filter1d(filter, curr_ref_scale, mu1, temp_buf, w, h,
+                         curr_ref_stride, buf_stride, filter_width, temp);
+            vif_filter1d(filter, curr_main_scale, mu2, temp_buf, w, h,
+                         curr_main_stride, buf_stride, filter_width, temp);
+
+            vif_dec2(mu1, ref_scale, buf_valid_w, buf_valid_h, buf_stride,
+                     buf_stride);
+            vif_dec2(mu2, main_scale, buf_valid_w, buf_valid_h, buf_stride,
+                     buf_stride);
+
+            w  = buf_valid_w / 2;
+            h  = buf_valid_h / 2;
+
+            buf_valid_w = w;
+            buf_valid_h = h;
+
+            curr_ref_scale = ref_scale;
+            curr_main_scale = main_scale;
+
+            curr_ref_stride = buf_stride;
+            curr_main_stride = buf_stride;
+        }
+
+        vif_filter1d(filter, curr_ref_scale, mu1, temp_buf, w, h, curr_ref_stride,
+                     buf_stride, filter_width, temp);
+        vif_filter1d(filter, curr_main_scale, mu2, temp_buf, w, h, curr_main_stride,
+                     buf_stride, filter_width, temp);
+
+        vif_xx_yy_xy(mu1, mu2, mu1_sq, mu2_sq, mu1_mu2, w, h, buf_stride,
+                     buf_stride, buf_stride, buf_stride, buf_stride);
+
+        vif_xx_yy_xy(curr_ref_scale, curr_main_scale, ref_sq, main_sq, ref_main,
+                     w, h, curr_ref_stride, curr_main_stride, buf_stride,
+                     buf_stride, buf_stride);
+
+        vif_filter1d(filter, ref_sq, ref_sq_filt, temp_buf, w, h, buf_stride,
+                     buf_stride, filter_width, temp);
+        vif_filter1d(filter, main_sq, main_sq_filt, temp_buf, w, h, buf_stride,
+                     buf_stride, filter_width, temp);
+        vif_filter1d(filter, ref_main, ref_main_filt, temp_buf, w, h, buf_stride,
+                     buf_stride, filter_width, temp);
+
+        vif_statistic(mu1_sq, mu2_sq, mu1_mu2, ref_sq_filt, main_sq_filt,
+                      ref_main_filt, num_array, den_array, w, h, buf_stride,
+                      buf_stride, buf_stride, buf_stride, buf_stride,
+                      buf_stride, buf_stride, buf_stride);
+
+        num = vif_sum(num_array, buf_valid_w, buf_valid_h, buf_stride);
+        den = vif_sum(den_array, buf_valid_w, buf_valid_h, buf_stride);
+
+        scores[2*scale] = num;
+        scores[2*scale+1] = den;
+    }
+
+    *score_num = 0.0;
+    *score_den = 0.0;
+    for (scale = 0; scale < 4; ++scale) {
+        *score_num += scores[2*scale];
+        *score_den += scores[2*scale+1];
+    }
+
+    if (*score_den == 0.0) {
+        *score = 1.0f;
+    } else {
+        *score = (*score_num) / (*score_den);
+    }
+
+    ret = 0;
+
+    return ret;
+}
+
+#define offset_fn(type, bits) \
+    static void offset_##bits##bit(VIFContext *s, const AVFrame *ref, AVFrame *main, int stride) \
+{ \
+    int w = s->width; \
+    int h = s->height; \
+    int i,j; \
+    \
+    int ref_stride = ref->linesize[0]; \
+    int main_stride = main->linesize[0]; \
+    \
+    const type *ref_ptr = (const type *) ref->data[0]; \
+    const type *main_ptr = (const type *) main->data[0]; \
+    \
+    float *ref_ptr_data = s->ref_data; \
+    float *main_ptr_data = s->main_data; \
+    \
+    for(i = 0; i < h; i++) { \
+        for(j = 0; j < w; j++) { \
+            ref_ptr_data[j] = (float) ref_ptr[j] + OPT_RANGE_PIXEL_OFFSET; \
+            main_ptr_data[j] = (float) main_ptr[j] + OPT_RANGE_PIXEL_OFFSET; \
+        } \
+        ref_ptr += ref_stride / sizeof(type); \
+        ref_ptr_data += stride / sizeof(float); \
+        main_ptr += main_stride / sizeof(type); \
+        main_ptr_data += stride / sizeof(float); \
+    } \
+}
+
+offset_fn(uint8_t, 8);
+offset_fn(uint16_t, 10);
+
+static void set_meta(AVDictionary **metadata, const char *key, float d)
+{
+    char value[128];
+    snprintf(value, sizeof(value), "%0.2f", d);
+    av_dict_set(metadata, key, value, 0);
+}
+
+static AVFrame *do_vif(AVFilterContext *ctx, AVFrame *main, const AVFrame *ref)
+{
+    VIFContext *s = ctx->priv;
+    AVDictionary **metadata = &main->metadata;
+
+    double score = 0.0;
+    double score_num = 0.0;
+    double score_den = 0.0;
+    double scores[8];
+
+    int w = s->width;
+    int h = s->height;
+
+    double stride;
+
+    stride = ALIGN_CEIL(w * sizeof(float));
+
+    /** Offset ref and main pixel by OPT_RANGE_PIXEL_OFFSET */
+    if (s->desc->comp[0].depth <= 8) {
+        offset_8bit(s, ref, main, stride);
+    } else {
+        offset_10bit(s, ref, main, stride);
+    }
+
+    compute_vif2(s->ref_data, s->main_data, w, h, stride, stride, &score,
+                 &score_num, &score_den, scores, s->data_buf, s->temp);
+
+    set_meta(metadata, "lavfi.vif.score", score);
+
+    s->nb_frames++;
+
+    s->vif_sum += score;
+
+    return main;
+}
+
+static av_cold int init(AVFilterContext *ctx)
+{
+    VIFContext *s = ctx->priv;
+
+    s->dinput.process = do_vif;
+
+    return 0;
+}
+
+static int query_formats(AVFilterContext *ctx)
+{
+    static const enum AVPixelFormat pix_fmts[] = {
+        AV_PIX_FMT_YUV444P, AV_PIX_FMT_YUV422P, AV_PIX_FMT_YUV420P,
+        AV_PIX_FMT_YUV444P10LE, AV_PIX_FMT_YUV422P10LE, AV_PIX_FMT_YUV420P10LE,
+        AV_PIX_FMT_NONE
+    };
+
+    AVFilterFormats *fmts_list = ff_make_format_list(pix_fmts);
+    if (!fmts_list)
+        return AVERROR(ENOMEM);
+    return ff_set_common_formats(ctx, fmts_list);
+}
+
+static int config_input_ref(AVFilterLink *inlink)
+{
+    AVFilterContext *ctx  = inlink->dst;
+    VIFContext *s = ctx->priv;
+    int stride;
+    size_t data_sz;
+
+    if (ctx->inputs[0]->w != ctx->inputs[1]->w ||
+        ctx->inputs[0]->h != ctx->inputs[1]->h) {
+        av_log(ctx, AV_LOG_ERROR, "Width and height of input videos must be same.\n");
+        return AVERROR(EINVAL);
+    }
+    if (ctx->inputs[0]->format != ctx->inputs[1]->format) {
+        av_log(ctx, AV_LOG_ERROR, "Inputs must be of same pixel format.\n");
+        return AVERROR(EINVAL);
+    }
+
+    s->desc = av_pix_fmt_desc_get(inlink->format);
+    s->width = ctx->inputs[0]->w;
+    s->height = ctx->inputs[0]->h;
+
+    stride = ALIGN_CEIL(s->width * sizeof(float));
+    data_sz = (size_t)stride * s->height;
+
+    if (SIZE_MAX / data_sz < 15) {
+        av_log(ctx, AV_LOG_ERROR, "error: SIZE_MAX / buf_sz < 15\n");
+        return AVERROR(EINVAL);
+    }
+
+    if (!(s->data_buf = av_malloc(data_sz * 16))) {
+        av_log(ctx, AV_LOG_ERROR, "error: av_malloc failed for data_buf.\n");
+        return AVERROR(ENOMEM);
+    }
+    if (!(s->ref_data = av_malloc(data_sz))) {
+        av_log(ctx, AV_LOG_ERROR, "error: av_malloc failed for ref_data.\n");
+        return AVERROR(ENOMEM);
+    }
+    if (!(s->main_data = av_malloc(data_sz))) {
+        av_log(ctx, AV_LOG_ERROR, "error: av_malloc failed for main_data.\n");
+        return AVERROR(ENOMEM);
+    }
+    if (!(s->temp = av_malloc(s->width * sizeof(float)))) {
+        av_log(ctx, AV_LOG_ERROR, "error: av_malloc failed for temp.\n");
+        return AVERROR(ENOMEM);
+    }
+
+    return 0;
+}
+
+
+static int config_output(AVFilterLink *outlink)
+{
+    AVFilterContext *ctx = outlink->src;
+    VIFContext *s = ctx->priv;
+    AVFilterLink *mainlink = ctx->inputs[0];
+    int ret;
+
+    outlink->w = mainlink->w;
+    outlink->h = mainlink->h;
+    outlink->time_base = mainlink->time_base;
+    outlink->sample_aspect_ratio = mainlink->sample_aspect_ratio;
+    outlink->frame_rate = mainlink->frame_rate;
+    if ((ret = ff_dualinput_init(ctx, &s->dinput)) < 0)
+        return ret;
+
+    return 0;
+}
+
+static int filter_frame(AVFilterLink *inlink, AVFrame *inpicref)
+{
+    VIFContext *s = inlink->dst->priv;
+    return ff_dualinput_filter_frame(&s->dinput, inlink, inpicref);
+}
+
+static int request_frame(AVFilterLink *outlink)
+{
+    VIFContext *s = outlink->src->priv;
+    return ff_dualinput_request_frame(&s->dinput, outlink);
+}
+
+static av_cold void uninit(AVFilterContext *ctx)
+{
+    VIFContext *s = ctx->priv;
+
+    if (s->nb_frames > 0) {
+        av_log(ctx, AV_LOG_INFO, "VIF AVG: %.3f\n", s->vif_sum / s->nb_frames);
+    }
+
+    av_free(s->data_buf);
+    av_free(s->ref_data);
+    av_free(s->main_data);
+    av_free(s->temp);
+
+    ff_dualinput_uninit(&s->dinput);
+}
+
+static const AVFilterPad vif_inputs[] = {
+    {
+        .name         = "main",
+        .type         = AVMEDIA_TYPE_VIDEO,
+        .filter_frame = filter_frame,
+    },{
+        .name         = "reference",
+        .type         = AVMEDIA_TYPE_VIDEO,
+        .filter_frame = filter_frame,
+        .config_props = config_input_ref,
+    },
+    { NULL }
+};
+
+static const AVFilterPad vif_outputs[] = {
+    {
+        .name          = "default",
+        .type          = AVMEDIA_TYPE_VIDEO,
+        .config_props  = config_output,
+        .request_frame = request_frame,
+    },
+    { NULL }
+};
+
+AVFilter ff_vf_vif = {
+    .name          = "vif",
+    .description   = NULL_IF_CONFIG_SMALL("Calculate the VIF between two video streams."),
+    .init          = init,
+    .uninit        = uninit,
+    .query_formats = query_formats,
+    .priv_size     = sizeof(VIFContext),
+    .priv_class    = &vif_class,
+    .inputs        = vif_inputs,
+    .outputs       = vif_outputs,
+};
diff --git a/libavfilter/vif.h b/libavfilter/vif.h
new file mode 100644
index 0000000..9074efa
--- /dev/null
+++ b/libavfilter/vif.h
@@ -0,0 +1,30 @@ 
+/*
+ * Copyright (c) 2017 Ronald S. Bultje <rsbultje@gmail.com>
+ * Copyright (c) 2017 Ashish Pratap Singh <ashk43712@gmail.com>
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#ifndef AVFILTER_VIF_H
+#define AVFILTER_VIF_H
+
+int compute_vif2(const float *ref, const float *main, int w, int h,
+                 int ref_stride, int main_stride, double *score,
+                 double *score_num, double *score_den, double *scores,
+                 float *data_buf, float *temp);
+
+#endif /* AVFILTER_VIF_H */