[FFmpeg-devel] avfilter: add ANSNR filter

A new filter ANSNR is added. libavfilter/Makefile is changed.
Run 'ffmpeg -i input1 -i input2 -lavfi ansnr -f null -' to get an overall score while per-frame value is stored but not printed.
This implementation is for constructing the vmaf filter later since ANSNR is one of individual tools used in vmaf.

Signed-off-by: Betty Wu <lumosomul@gmail.com>
---
 libavfilter/Makefile     |   1 +
 libavfilter/allfilters.c |   1 +
 libavfilter/vf_ansnr.c   | 727 +++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 729 insertions(+)
 create mode 100644 libavfilter/vf_ansnr.c

On Thu, Mar 30, 2017 at 11:06:36PM +0200, Betty Wu wrote:
> A new filter ANSNR is added. libavfilter/Makefile is changed.
> Run 'ffmpeg -i input1 -i input2 -lavfi ansnr -f null -' to get an overall score while per-frame value is stored but not printed.
> This implementation is for constructing the vmaf filter later since ANSNR is one of individual tools used in vmaf.
> 
> Signed-off-by: Betty Wu <lumosomul@gmail.com>
> ---
>

[...]

> +static void *aligned_malloc(size_t size, size_t alignment)
> +{
> +    void *ptr;
> +
> +    if (posix_memalign(&ptr, alignment, size))
> +        return 0;
> +    else
> +        return ptr;
> +}

please use av_malloc()

posix_memalign is not portable


> +
> +static void aligned_free(void *ptr)
> +{
> +    free(ptr);
> +}

use av_freep()

also please see tools/patcheck


[...]

2017-03-31 5:06 GMT+08:00 Betty Wu <lumosomul@gmail.com>:

> A new filter ANSNR is added. libavfilter/Makefile is changed.
> Run 'ffmpeg -i input1 -i input2 -lavfi ansnr -f null -' to get an overall
> score while per-frame value is stored but not printed.
> This implementation is for constructing the vmaf filter later since ANSNR
> is one of individual tools used in vmaf.
>
> Signed-off-by: Betty Wu <lumosomul@gmail.com>
> ---
>  libavfilter/Makefile     |   1 +
>  libavfilter/allfilters.c |   1 +
>  libavfilter/vf_ansnr.c   | 727 ++++++++++++++++++++++++++++++
> +++++++++++++++++
>  3 files changed, 729 insertions(+)
>  create mode 100644 libavfilter/vf_ansnr.c
>

need add options describes into document doc/filters.texi

>
> diff --git a/libavfilter/Makefile b/libavfilter/Makefile
> index 9c15ed62d2..5416e0f34f 100644
> --- a/libavfilter/Makefile
> +++ b/libavfilter/Makefile
> @@ -123,6 +123,7 @@ OBJS-$(CONFIG_ANULLSINK_FILTER)              +=
> asink_anullsink.o
>  # video filters
>  OBJS-$(CONFIG_ALPHAEXTRACT_FILTER)           += vf_extractplanes.o
>  OBJS-$(CONFIG_ALPHAMERGE_FILTER)             += vf_alphamerge.o
> +OBJS-$(CONFIG_ANSNR_FILTER)                  += vf_ansnr.o dualinput.o
> framesync.o
>  OBJS-$(CONFIG_ASS_FILTER)                    += vf_subtitles.o
>  OBJS-$(CONFIG_ATADENOISE_FILTER)             += vf_atadenoise.o
>  OBJS-$(CONFIG_AVGBLUR_FILTER)                += vf_avgblur.o
> diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
> index 64b634e8f3..fbd9ec026e 100644
> --- a/libavfilter/allfilters.c
> +++ b/libavfilter/allfilters.c
> @@ -134,6 +134,7 @@ static void register_all(void)
>
>      REGISTER_FILTER(ALPHAEXTRACT,   alphaextract,   vf);
>      REGISTER_FILTER(ALPHAMERGE,     alphamerge,     vf);
> +    REGISTER_FILTER(ANSNR,          ansnr,          vf);
>      REGISTER_FILTER(ASS,            ass,            vf);
>      REGISTER_FILTER(ATADENOISE,     atadenoise,     vf);
>      REGISTER_FILTER(AVGBLUR,        avgblur,        vf);
> diff --git a/libavfilter/vf_ansnr.c b/libavfilter/vf_ansnr.c
> new file mode 100644
> index 0000000000..3c53b726d2
> --- /dev/null
> +++ b/libavfilter/vf_ansnr.c
> @@ -0,0 +1,727 @@
> +/*
> + * Copyright 2016-2017 Netflix, Inc.
> + * Copyright (c) 2017 Betty Wu
> + *
> + * This file is part of FFmpeg.
> + *
> + * FFmpeg is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2.1 of the License, or (at your option) any later version.
> + *
> + * FFmpeg is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with FFmpeg; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
> 02110-1301 USA
> + */
> +
> +/*
> + * @file
> + * Caculate the ANSNR between two input videos.
> + * @author Betty Wu
> + */
> +
> +#include "libavutil/avstring.h"
> +#include "libavutil/opt.h"
> +#include "libavutil/pixdesc.h"
> +#include "avfilter.h"
> +#include "dualinput.h"
> +#include "drawutils.h"
> +#include "formats.h"
> +#include "internal.h"
> +#include "video.h"
> +
> +#define MAX_ALIGN 32
> +#define ALIGN_CEIL(x) ((x) + ((x) % MAX_ALIGN ? MAX_ALIGN - (x) %
> MAX_ALIGN : 0))
> +#define OPT_RANGE_PIXEL_OFFSET -128
> +
> +#ifndef ANSNR_OPTIONS_H_
> +#define ANSNR_OPTIONS_H_
> +
> +/* Whether to use border replication instead of zero extension. */
> +/* #define ANSNR_OPT_BORDER_REPLICATE */
> +
> +/* Whether to save intermediate results to files. */
> +/* #define ANSNR_OPT_DEBUG_DUMP */
> +
> +/* Whether to use a 1-D approximation of filters. */
> +/* #define ANSNR_OPT_FILTER_1D */
> +
> +/* Whether to normalize result by dividing against maximum ANSNR. */
> +/* #define ANSNR_OPT_NORMALIZE */
> +
> +/* Whether to use single precision for computation. */
> +#define ANSNR_OPT_SINGLE_PRECISION
> +//#define ANSNR_OPT_DOUBLE_PRECISION
> +
> +#endif /* ANSNR_OPTIONS_H_ */
> +
> +#ifndef ANSNR_TOOLS_H_
> +#define ANSNR_TOOLS_H_
> +#endif
> +
> +#ifdef ANSNR_OPT_SINGLE_PRECISION
> +  typedef float number_t;
> +  #define read_image_b       read_image_b2s
> +  #define read_image_w       read_image_w2s
> +  #define ansnr_filter1d_ref ansnr_filter1d_ref_s
> +  #define ansnr_filter1d_dis ansnr_filter1d_dis_s
> +  #define ansnr_filter2d_ref ansnr_filter2d_ref_s
> +  #define ansnr_filter2d_dis ansnr_filter2d_dis_s
> +  #define ansnr_filter1d     ansnr_filter1d_s
> +  #define ansnr_filter2d     ansnr_filter2d_s
> +  #define ansnr_mse          ansnr_mse_s
> +#else
> +  typedef double number_t;
> +  #define read_image_b       read_image_b2d
> +  #define read_image_w       read_image_w2d
> +  #define ansnr_filter1d_ref ansnr_filter1d_ref_d
> +  #define ansnr_filter1d_dis ansnr_filter1d_dis_d
> +  #define ansnr_filter2d_ref ansnr_filter2d_ref_d
> +  #define ansnr_filter2d_dis ansnr_filter2d_dis_d
> +  #define ansnr_filter1d     ansnr_filter1d_d
> +  #define ansnr_filter2d     ansnr_filter2d_d
> +  #define ansnr_mse          ansnr_mse_d
> +#endif
> +
> +typedef struct ANSNRContext {
> +    const AVClass *class;
> +    FFDualInputContext dinput;
> +    uint64_t nb_frames;
> +    FILE *stats_file;
> +    char *stats_file_str;
> +    int stats_version;
> +    int stats_header_written;
> +    int stats_add_max;
> +    int is_rgb;
> +    uint8_t rgba_map[4];
> +    double score;
> +    double score_total;
> +    char comps[4];
> +    int nb_components;
> +    int planewidth[4];
> +    int planeheight[4];
> +
> +} ANSNRContext;
> +#define OFFSET(x) offsetof(ANSNRContext, x)
> +#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM
> +
> +static const AVOption ansnr_options[] = {
> +    {"stats_file", "Set file where to store per-frame difference
> information", OFFSET(stats_file_str), AV_OPT_TYPE_STRING, {.str=NULL}, 0,
> 0, FLAGS },
> +    {"f",          "Set file where to store per-frame difference
> information", OFFSET(stats_file_str), AV_OPT_TYPE_STRING, {.str=NULL}, 0,
> 0, FLAGS },
> +    { NULL }
> +};
> +AVFILTER_DEFINE_CLASS(ansnr);
> +
> +
> +const int ansnr_filter1d_ref_width = 3;
> +const int ansnr_filter1d_dis_width = 5;
> +const int ansnr_filter2d_ref_width = 3;
> +const int ansnr_filter2d_dis_width = 5;
> +
> +
> +#ifdef ANSNR_OPT_SINGLE_PRECISION
> +const float ansnr_filter1d_ref_s[3] = { 0x1.00243ap-2, 0x1.ffb78cp-2,
> 0x1.00243ap-2 };
> +const float ansnr_filter1d_dis_s[5] = { 0x1.be5f0ep-5, 0x1.f41fd6p-3,
> 0x1.9c4868p-2, 0x1.f41fd6p-3, 0x1.be5f0ep-5 };
> +const float ansnr_filter2d_ref_s[3*3] = {
> +    1.0 / 16.0, 2.0 / 16.0, 1.0 / 16.0,
> +    2.0 / 16.0, 4.0 / 16.0, 2.0 / 16.0,
> +    1.0 / 16.0, 2.0 / 16.0, 1.0 / 16.0
> +};
> +const float ansnr_filter2d_dis_s[5*5] = {
> +    2.0 / 571.0,  7.0 / 571.0,  12.0 / 571.0,  7.0 / 571.0,  2.0 / 571.0,
> +    7.0 / 571.0,  31.0 / 571.0, 52.0 / 571.0,  31.0 / 571.0, 7.0 / 571.0,
> +    12.0 / 571.0, 52.0 / 571.0, 127.0 / 571.0, 52.0 / 571.0, 12.0 / 571.0,
> +    7.0 / 571.0,  31.0 / 571.0, 52.0 / 571.0,  31.0 / 571.0, 7.0 / 571.0,
> +    2.0 / 571.0,  7.0 / 571.0,  12.0 / 571.0,  7.0 / 571.0,  2.0 / 571.0
> +};
> +#else
> +const double ansnr_filter1d_ref_d[3] = { 0x1.00243aee6175bp-2,
> 0x1.ffb78a233d14ap-2, 0x1.00243aee6175bp-2 };
> +const double ansnr_filter1d_dis_d[5] = { 0x1.be5f0dc491a0fp-5,
> 0x1.f41fd54c58786p-3, 0x1.9c486742831f6p-2, 0x1.f41fd54c58786p-3,
> 0x1.be5f0dc491a0fp-5 };const double ansnr_filter2d_ref_d[3*3] = {
> +    1.0 / 16.0, 2.0 / 16.0, 1.0 / 16.0,
> +    2.0 / 16.0, 4.0 / 16.0, 2.0 / 16.0,
> +    1.0 / 16.0, 2.0 / 16.0, 1.0 / 16.0
> +};
> +const double ansnr_filter2d_dis_d[5*5] = {
> +    2.0 / 571.0,  7.0 / 571.0,  12.0 / 571.0,  7.0 / 571.0,  2.0 / 571.0,
> +    7.0 / 571.0,  31.0 / 571.0, 52.0 / 571.0,  31.0 / 571.0, 7.0 / 571.0,
> +    12.0 / 571.0, 52.0 / 571.0, 127.0 / 571.0, 52.0 / 571.0, 12.0 / 571.0,
> +    7.0 / 571.0,  31.0 / 571.0, 52.0 / 571.0,  31.0 / 571.0, 7.0 / 571.0,
> +    2.0 / 571.0,  7.0 / 571.0,  12.0 / 571.0,  7.0 / 571.0,  2.0 / 571.0
> +};
> +#endif
> +
> +static void *aligned_malloc(size_t size, size_t alignment)
> +{
> +    void *ptr;
> +
> +    if (posix_memalign(&ptr, alignment, size))
> +        return 0;
> +    else
> +        return ptr;
> +}
> +
> +static void aligned_free(void *ptr)
> +{
> +    free(ptr);
> +}
> +
> +#ifdef ANSNR_OPT_SINGLE_PRECISION
> +static void ansnr_mse_s(const float *ref, const float *dis, float *sig,
> float *noise, int w, int h, int ref_stride, int dis_stride)
> +{
> +    int ref_px_stride = ref_stride / sizeof(float);
> +    int dis_px_stride = dis_stride / sizeof(float);
> +    int i, j;
> +
> +    float ref_val, dis_val;
> +
> +    float sig_accum = 0;
> +    float noise_accum = 0;
> +
> +    for (i = 0; i < h; ++i) {
> +        float sig_accum_inner = 0;
> +        float noise_accum_inner = 0;
> +
> +        for (j = 0; j < w; ++j) {
> +            ref_val = ref[i * ref_px_stride + j];
> +            dis_val = dis[i * dis_px_stride + j];
> +            sig_accum_inner   += ref_val * ref_val;
> +            noise_accum_inner += (ref_val - dis_val) * (ref_val -
> dis_val);
> +        }
> +
> +        sig_accum   += sig_accum_inner;
> +        noise_accum += noise_accum_inner;
> +
> +    }
> +
> +    if (sig)
> +        *sig = sig_accum;
> +    if (noise)
> +        *noise = noise_accum;
> +}
> +
> +#ifdef ANSNR_OPT_FILTER_1D
> +static void ansnr_filter1d_s(const float *f, const uint8_t *src, float
> *dst, int w, int h, int src_stride, int dst_stride, int fwidth)
> +{
> +    int src_px_stride = src_stride / sizeof(float);
> +    int dst_px_stride = dst_stride / sizeof(float);
> +
> +    float *tmp = aligned_malloc(ALIGN_CEIL(w * sizeof(float)), MAX_ALIGN);
> +    float fcoeff, imgcoeff;
> +
> +    int i, j, fi, fj, ii, jj;
> +
> +    for (i = 0; i < h; ++i) {
> +        /* Vertical pass. */
> +        for (j = 0; j < w; ++j) {
> +            float accum = 0;
> +
> +            for (fi = 0; fi < fwidth; ++fi) {
> +                fcoeff = f[fi];
> +
> +                ii = i - fwidth / 2 + fi;
> +#ifdef ANSNR_OPT_BORDER_REPLICATE
> +                ii = ii < 0 ? 0 : (ii > h - 1 ? h - 1 : ii);
> +                imgcoeff = (float)src[ii * src_px_stride +
> j]+OPT_RANGE_PIXEL_OFFSET;
> +#else
> +                if (ii < 0) ii = -ii;
> +                else if (ii >= h) ii = 2 * h - ii - 1;
> +                imgcoeff = (float)src[ii * src_px_stride +
> j]+OPT_RANGE_PIXEL_OFFSET;
> +#endif
> +                accum += fcoeff * imgcoeff;
> +            }
> +
> +            tmp[j] = accum;
> +        }
> +
> +        /* Horizontal pass. */
> +        for (j = 0; j < w; ++j) {
> +            float accum = 0;
> +
> +            for (fj = 0; fj < fwidth; ++fj) {
> +                fcoeff = f[fj];
> +
> +                jj = j - fwidth / 2 + fj;
> +#ifdef ANSNR_OPT_BORDER_REPLICATE
> +                jj = jj < 0 ? 0 : (jj > w - 1 ? w - 1 : jj);
> +                imgcoeff = tmp[jj];
> +#else
> +                if (jj < 0) jj = -jj;
> +                else if (jj >= w) jj = 2 * w - jj - 1;
> +                imgcoeff = tmp[jj];
> +#endif
> +                accum += fcoeff * imgcoeff;
> +            }
> +
> +            dst[i * dst_px_stride + j] = accum;
> +        }
> +    }
> +
> +    aligned_free(tmp);
> +}
> +#else
> +static void ansnr_filter2d_s(const float *f, const uint8_t *src, float
> *dst, int w, int h, int src_stride, int dst_stride, int fwidth)
> +{
> +    int src_px_stride = src_stride / sizeof(float);
> +    int dst_px_stride = dst_stride / sizeof(float);
> +
> +    float fcoeff, imgcoeff;
> +    int i, j, fi, fj, ii, jj;
> +
> +    for (i = 0; i < h; ++i) {
> +        for (j = 0; j < w; ++j) {
> +            float accum = 0;
> +
> +            for (fi = 0; fi < fwidth; ++fi) {
> +                float accum_inner = 0;
> +
> +                for (fj = 0; fj < fwidth; ++fj) {
> +                    fcoeff = f[fi * fwidth + fj];
> +
> +                    ii = i - fwidth / 2 + fi;
> +                    jj = j - fwidth / 2 + fj;
> +#ifdef ANSNR_OPT_BORDER_REPLICATE
> +                    ii = ii < 0 ? 0 : (ii > h - 1 ? h - 1 : ii);
> +                    jj = jj < 0 ? 0 : (jj > w - 1 ? w - 1 : jj);
> +                    imgcoeff = (float)src[ii * src_px_stride +
> jj]+OPT_RANGE_PIXEL_OFFSET;
> +#else
> +                    if (ii < 0) ii = -ii;
> +                    else if (ii >= h) ii = 2 * h - ii - 1;
> +                    if (jj < 0) jj = -jj;
> +                    else if (jj >= w) jj = 2 * w - jj - 1;
> +                    imgcoeff = (float)src[ii * src_px_stride +
> jj]+OPT_RANGE_PIXEL_OFFSET;
> +#endif
> +                    accum_inner += fcoeff * imgcoeff;
> +                }
> +
> +                accum += accum_inner;
> +            }
> +
> +            dst[i * dst_px_stride + j] = accum;
> +        }
> +    }
> +}
> +#endif
> +#else
> +static void ansnr_mse_d(const double *ref, const double *dis, double
> *sig, double *noise, int w, int h, int ref_stride, int dis_stride)
> +{
> +    int ref_px_stride = ref_stride / sizeof(double);
> +    int dis_px_stride = dis_stride / sizeof(double);
> +    int i, j;
> +
> +    double ref_val, dis_val;
> +
> +    double sig_accum = 0;
> +    double noise_accum = 0;
> +
> +    for (i = 0; i < h; ++i) {
> +        double sig_accum_inner = 0;
> +        double noise_accum_inner = 0;
> +
> +        for (j = 0; j < w; ++j) {
> +            ref_val = ref[i * ref_px_stride + j];
> +            dis_val = dis[i * dis_px_stride + j];
> +
> +            sig_accum_inner   += ref_val * ref_val;
> +            noise_accum_inner += (ref_val - dis_val) * (ref_val -
> dis_val);
> +        }
> +
> +        sig_accum   += sig_accum_inner;
> +        noise_accum += noise_accum_inner;
> +    }
> +
> +    if (sig)
> +        *sig = sig_accum;
> +    if (noise)
> +        *noise = noise_accum;
> +}
> +#ifdef ANSNR_OPT_FILTER_1D
> +static void ansnr_filter1d_d(const double *f, const uint8_t *src, double
> *dst, int w, int h, int src_stride, int dst_stride, int fwidth)
> +{
> +    int src_px_stride = src_stride / sizeof(double);
> +    int dst_px_stride = dst_stride / sizeof(double);
> +
> +    double *tmp = aligned_malloc(ALIGN_CEIL(w * sizeof(double)),
> MAX_ALIGN);
> +    double fcoeff, imgcoeff;
> +
> +    int i, j, fi, fj, ii, jj;
> +
> +    for (i = 0; i < h; ++i) {
> +        /* Vertical pass. */
> +        for (j = 0; j < w; ++j) {
> +            double accum = 0;
> +
> +            for (fi = 0; fi < fwidth; ++fi) {
> +                fcoeff = f[fi];
> +
> +                ii = i - fwidth / 2 + fi;
> +#ifdef ANSNR_OPT_BORDER_REPLICATE
> +                ii = ii < 0 ? 0 : (ii > h - 1 ? h - 1 : ii);
> +                imgcoeff = (double)src[ii * src_px_stride +
> j]+OPT_RANGE_PIXEL_OFFSET;
> +#else
> +                if (ii < 0) ii = -ii;
> +                else if (ii >= h) ii = 2 * h - ii - 1;
> +                imgcoeff = (double)src[ii * src_px_stride +
> j]+OPT_RANGE_PIXEL_OFFSET;
> +#endif
> +                accum += fcoeff * imgcoeff;
> +            }
> +
> +            tmp[j] = accum;
> +        }
> +
> +        /* Horizontal pass. */
> +        for (j = 0; j < w; ++j) {
> +            double accum = 0;
> +
> +            for (fj = 0; fj < fwidth; ++fj) {
> +                fcoeff = f[fj];
> +
> +                jj = j - fwidth / 2 + fj;
> +#ifdef ANSNR_OPT_BORDER_REPLICATE
> +                jj = jj < 0 ? 0 : (jj > w - 1 ? w - 1 : jj);
> +                imgcoeff = tmp[jj];
> +#else
> +                if (jj < 0) jj = -jj;
> +                else if (jj >= w) jj = 2 * w - jj - 1;
> +                imgcoeff = tmp[jj];
> +#endif
> +                accum += fcoeff * imgcoeff;
> +            }
> +
> +            dst[i * dst_px_stride + j] = accum;
> +        }
> +    }
> +
> +    aligned_free(tmp);
> +}
> +#else
> +static void ansnr_filter2d_d(const double *f, const uint8_t *src, double
> *dst, int w, int h, int src_stride, int dst_stride, int fwidth)
> +{
> +    int src_px_stride = src_stride / sizeof(double);
> +    int dst_px_stride = dst_stride / sizeof(double);
> +
> +    double fcoeff, imgcoeff;
> +    int i, j, fi, fj, ii, jj;
> +
> +    for (i = 0; i < h; ++i) {
> +        for (j = 0; j < w; ++j) {
> +            double accum = 0;
> +
> +            for (fi = 0; fi < fwidth; ++fi) {
> +                double accum_inner = 0;
> +
> +                for (fj = 0; fj < fwidth; ++fj) {
> +                    fcoeff = f[fi * fwidth + fj];
> +
> +                    ii = i - fwidth / 2 + fi;
> +                    jj = j - fwidth / 2 + fj;
> +#ifdef ANSNR_OPT_BORDER_REPLICATE
> +                    ii = ii < 0 ? 0 : (ii > h - 1 ? h - 1 : ii);
> +                    jj = jj < 0 ? 0 : (jj > w - 1 ? w - 1 : jj);
> +                    imgcoeff = (double)src[ii * src_px_stride +
> jj]+OPT_RANGE_PIXEL_OFFSET;
> +#else
> +                    if (ii < 0) ii = -ii;
> +                    else if (ii >= h) ii = 2 * h - ii - 1;
> +                    if (jj < 0) jj = -jj;
> +                    else if (jj >= w) jj = 2 * w - jj - 1;
> +                    imgcoeff = (double)src[ii * src_px_stride +
> jj]+OPT_RANGE_PIXEL_OFFSET;
> +#endif
> +                    accum_inner += fcoeff * imgcoeff;
> +                }
> +
> +                accum += accum_inner;
> +            }
> +
> +            dst[i * dst_px_stride + j] = accum;
> +        }
> +    }
> +}
> +#endif
> +#endif
> +
> +static inline
> +number_t compute_ansnr(const uint8_t *ref, const uint8_t *dis, int w, int
> h, double psnr_max, number_t *out)
> +{
> +    double score=0;
> +    int ref_stride=ALIGN_CEIL(w * sizeof(number_t));
> +    int dis_stride=ref_stride;
> +    number_t *data_buf = 0;
> +    char *data_top;
> +
> +    number_t *ref_filtr;
> +    number_t *ref_filtd;
> +    number_t *dis_filtd;
> +
> +    number_t sig, noise;
> +
> +#ifdef ANSNR_OPT_NORMALIZE
> +    number_t noise_min;
> +#endif
> +
> +    int buf_stride = ALIGN_CEIL(w * sizeof(number_t));
> +    size_t buf_sz_one = (size_t)buf_stride * h;
> +
> +    if (SIZE_MAX / buf_sz_one < 3)
> +    {
> +        goto fail;
> +    }
>
code style should fix:  if (SIZE_MAX / buf_sz_one < 3) {

> +
> +    if (!(data_buf = aligned_malloc(buf_sz_one * 3, MAX_ALIGN)))
> +    {
> +        goto fail;
> +    }
>
same to above comment.

> +
> +    data_top = (char *)data_buf;
> +
> +    ref_filtr = (number_t *)data_top; data_top += buf_sz_one;
> +    ref_filtd = (number_t *)data_top; data_top += buf_sz_one;
> +    dis_filtd = (number_t *)data_top; data_top += buf_sz_one;
> +
> +#ifdef ANSNR_OPT_FILTER_1D
> +    ansnr_filter1d(ansnr_filter1d_ref, ref, ref_filtr, w, h, ref_stride,
> buf_stride, ansnr_filter1d_ref_width);
> +    ansnr_filter1d(ansnr_filter1d_dis, ref, ref_filtd, w, h, ref_stride,
> buf_stride, ansnr_filter1d_dis_width);
> +    ansnr_filter1d(ansnr_filter1d_dis, dis, dis_filtd, w, h, dis_stride,
> buf_stride, ansnr_filter1d_dis_width);
> +#else
> +    ansnr_filter2d(ansnr_filter2d_ref, ref, ref_filtr, w, h, ref_stride,
> buf_stride, ansnr_filter2d_ref_width);
> +    ansnr_filter2d(ansnr_filter2d_dis, ref, ref_filtd, w, h, ref_stride,
> buf_stride, ansnr_filter2d_dis_width);
> +    ansnr_filter2d(ansnr_filter2d_dis, dis, dis_filtd, w, h, dis_stride,
> buf_stride, ansnr_filter2d_dis_width);
> +#endif
> +
> +#ifdef ANSNR_OPT_DEBUG_DUMP
> +    write_image("stage/ref_filtr.bin", ref_filtr, w, h, buf_stride,
> sizeof(number_t));
> +    write_image("stage/ref_filtd.bin", ref_filtd, w, h, buf_stride,
> sizeof(number_t));
> +    write_image("stage/dis_filtd.bin", dis_filtd, w, h, buf_stride,
> sizeof(number_t));
> +#endif
> +
> +    ansnr_mse(ref_filtr, dis_filtd, &sig, &noise, w, h, buf_stride,
> buf_stride);
> +#ifdef ANSNR_OPT_NORMALIZE
> +    ansnr_mse(ref_filtr, ref_filtd, 0, &noise_min, w, h, buf_stride,
> buf_stride);
> +    score = 10.0 * log10(noise / (noise - noise_min));
> +    *out= noise / (noise - noise_min);
> +#else
> +    score = noise==0 ? psnr_max : 10.0 * log10(sig / noise);
> +    *out = noise==0 ? pow(10, psnr_max/10.0) : sig / noise;
>
What about use  score = noise? 10.0 * log10(sig / noise) : psnr_max;
*out = noise ? sig / noise : pow(10, psnr_max/10.0) ;

> +#endif
> +
> +fail:
> +    aligned_free(data_buf);
> +    return score;
> +}
> +static void set_meta(AVDictionary **metadata, const char *key, char comp,
> float d)
> +{
> +    char value[128];
> +    snprintf(value, sizeof(value), "%0.2f", d);
> +    if (comp) {
> +        char key2[128];
> +        snprintf(key2, sizeof(key2), "%s%c", key, comp);
> +        av_dict_set(metadata, key2, value, 0);
> +    } else {
> +        av_dict_set(metadata, key, value, 0);
> +    }
> +}
> +
> +
> +
> +
>
redundant newline

> +static int query_formats(AVFilterContext *ctx)
> +{
> +    static const enum AVPixelFormat pix_fmts[] = {
> +        AV_PIX_FMT_GRAY8, AV_PIX_FMT_GRAY16,
> +#define PF_NOALPHA(suf) AV_PIX_FMT_YUV420##suf,  AV_PIX_FMT_YUV422##suf,
> AV_PIX_FMT_YUV444##suf
> +#define PF_ALPHA(suf)   AV_PIX_FMT_YUVA420##suf, AV_PIX_FMT_YUVA422##suf,
> AV_PIX_FMT_YUVA444##suf
> +#define PF(suf)         PF_NOALPHA(suf), PF_ALPHA(suf)
> +        PF(P), PF(P9), PF(P10), PF_NOALPHA(P12), PF_NOALPHA(P14), PF(P16),
> +        AV_PIX_FMT_YUV440P, AV_PIX_FMT_YUV411P, AV_PIX_FMT_YUV410P,
> +        AV_PIX_FMT_YUVJ411P, AV_PIX_FMT_YUVJ420P, AV_PIX_FMT_YUVJ422P,
> +        AV_PIX_FMT_YUVJ440P, AV_PIX_FMT_YUVJ444P,
> +        AV_PIX_FMT_GBRP, AV_PIX_FMT_GBRP9, AV_PIX_FMT_GBRP10,
> +        AV_PIX_FMT_GBRP12, AV_PIX_FMT_GBRP14, AV_PIX_FMT_GBRP16,
> +        AV_PIX_FMT_GBRAP, AV_PIX_FMT_GBRAP16,
> +        AV_PIX_FMT_NONE
> +    };
> +
> +    AVFilterFormats *fmts_list = ff_make_format_list(pix_fmts);
> +    if (!fmts_list)
> +        return AVERROR(ENOMEM);
> +    return ff_set_common_formats(ctx, fmts_list);
> +}
>
need a new line.

> +static AVFrame *do_ansnr(AVFilterContext *ctx, AVFrame *main,
> +                        const AVFrame *ref)
> +{
> +    ANSNRContext *s = ctx->priv;
> +    AVDictionary **metadata = avpriv_frame_get_metadatap(main);
> +    double psnr_max;
> +    char *frame_format = (char *)av_get_pix_fmt_name(main->format);
> +    if( !strcmp(frame_format,"yuv420p") || !strcmp(frame_format,"yuv422p")
> || !strcmp(frame_format,"yuv444p"))
> +        psnr_max=60;
> +    if( !strcmp(frame_format,"yuv420p10le") || !strcmp(frame_format,"yuv422p10le")
> || !strcmp(frame_format,"yuv444p10le"))
> +        psnr_max=72;
> +    number_t out;
> +    double score = compute_ansnr(ref->data[0], main->data[0],
> s->planewidth[0], s->planeheight[0],  psnr_max, &out);
> +    s->nb_frames++;
> +    s->score=score;
> +    s->score_total += out;
> +
> +    set_meta(metadata, "lavfi.ansnr.All", 0, score);
> +
> +    if (s->stats_file) {
> +        fprintf(s->stats_file, "n:%"PRId64" ", s->nb_frames);
> +        fprintf(s->stats_file, "All:%f\n", score);
> +    }
> +
> +    return main;
> +}
> +
> +static av_cold int init(AVFilterContext *ctx)
> +{
> +    ANSNRContext *s = ctx->priv;
> +    s->score_total = 0;
> +    s->score = 0;
> +    if (s->stats_file_str) {
> +        if (s->stats_version < 2 && s->stats_add_max) {
> +            av_log(ctx, AV_LOG_ERROR,
> +                "stats_add_max was specified but stats_version < 2.\n" );
> +            return AVERROR(EINVAL);
> +        }
> +        if (!strcmp(s->stats_file_str, "-")) {
> +            s->stats_file = stdout;
> +        } else {
> +            s->stats_file = fopen(s->stats_file_str, "w");
> +            if (!s->stats_file) {
> +                int err = AVERROR(errno);
> +                char buf[128];
> +                av_strerror(err, buf, sizeof(buf));
> +                av_log(ctx, AV_LOG_ERROR, "Could not open stats file %s:
> %s\n",
> +                       s->stats_file_str, buf);
> +                return err;
> +            }
> +        }
> +    }
> +
> +    s->dinput.process = do_ansnr;
> +    return 0;
> +}
> +
> +
> +
>
redundant newline

> +static int config_input_ref(AVFilterLink *inlink)
> +{
> +    const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(inlink->format);
> +    AVFilterContext *ctx  = inlink->dst;
> +    ANSNRContext *s = ctx->priv;
> +
> +
>
redundant newline

> +    s->nb_components = desc->nb_components;
> +
> +    if (ctx->inputs[0]->w != ctx->inputs[1]->w ||
> +        ctx->inputs[0]->h != ctx->inputs[1]->h) {
> +        av_log(ctx, AV_LOG_ERROR, "Width and height of input videos must
> be same.\n");
> +        return AVERROR(EINVAL);
> +    }
> +    if (ctx->inputs[0]->format != ctx->inputs[1]->format) {
> +        av_log(ctx, AV_LOG_ERROR, "Inputs must be of same pixel
> format.\n");
> +        return AVERROR(EINVAL);
> +    }
> +    s->is_rgb = ff_fill_rgba_map(s->rgba_map, inlink->format) >= 0;
> +    s->comps[0] = s->is_rgb ? 'r' : 'y' ;
> +    s->comps[1] = s->is_rgb ? 'g' : 'u' ;
> +    s->comps[2] = s->is_rgb ? 'b' : 'v' ;
> +    s->comps[3] = 'a';
> +
> +    s->planeheight[1] = s->planeheight[2] = AV_CEIL_RSHIFT(inlink->h,
> desc->log2_chroma_h);
> +    s->planeheight[0] = s->planeheight[3] = inlink->h;
> +    s->planewidth[1]  = s->planewidth[2]  = AV_CEIL_RSHIFT(inlink->w,
> desc->log2_chroma_w);
> +    s->planewidth[0]  = s->planewidth[3]  = inlink->w;
> +    //if (ARCH_X86)
> +    //    ff_ansnr_init_x86(&s->dsp, desc->comp[0].depth);
>
if the code have no use, should remove them.

> +
> +    return 0;
> +}
> +
> +static int config_output(AVFilterLink *outlink)
> +{
> +    AVFilterContext *ctx = outlink->src;
> +    ANSNRContext *s = ctx->priv;
> +    AVFilterLink *mainlink = ctx->inputs[0];
> +    int ret;
> +
> +    outlink->w = mainlink->w;
> +    outlink->h = mainlink->h;
> +    outlink->time_base = mainlink->time_base;
> +    outlink->sample_aspect_ratio = mainlink->sample_aspect_ratio;
> +    outlink->frame_rate = mainlink->frame_rate;
> +
> +    if ((ret = ff_dualinput_init(ctx, &s->dinput)) < 0)
> +        return ret;
> +
> +    return 0;
>
maybe return ret will ok?

> +}
> +
> +static int filter_frame(AVFilterLink *inlink, AVFrame *inpicref)
> +{
> +    ANSNRContext *s = inlink->dst->priv;
> +    return ff_dualinput_filter_frame(&s->dinput, inlink, inpicref);
> +}
> +
> +static int request_frame(AVFilterLink *outlink)
> +{
> +    ANSNRContext *s = outlink->src->priv;
> +    return ff_dualinput_request_frame(&s->dinput, outlink);
> +}
> +
> +static av_cold void uninit(AVFilterContext *ctx)
> +{
> +    ANSNRContext *s = ctx->priv;
> +
> +    if (s->nb_frames > 0) {
> +        char buf[256];
> +        buf[0] = 0;
> +        av_log(ctx, AV_LOG_INFO, "ANSNR%s All:%f\n", buf,
> 10*log10(s->score_total));
> +    }
> +
> +    ff_dualinput_uninit(&s->dinput);
> +
> +    if (s->stats_file && s->stats_file != stdout)
> +        fclose(s->stats_file);
> +
> +}
> +
> +static const AVFilterPad ansnr_inputs[] = {
> +    {
> +        .name         = "main",
> +        .type         = AVMEDIA_TYPE_VIDEO,
> +        .filter_frame = filter_frame,
> +    },{
> +        .name         = "reference",
> +        .type         = AVMEDIA_TYPE_VIDEO,
> +        .filter_frame = filter_frame,
> +        .config_props = config_input_ref,
> +    },
> +    { NULL }
> +};
> +
> +static const AVFilterPad ansnr_outputs[] = {
> +    {
> +        .name          = "default",
> +        .type          = AVMEDIA_TYPE_VIDEO,
> +        .config_props  = config_output,
> +        .request_frame = request_frame,
> +    },
> +    { NULL }
> +};
> +
> +AVFilter ff_vf_ansnr = {
> +    .name          = "ansnr",
> +    .description   = NULL_IF_CONFIG_SMALL("Calculate the ANSNR between
> two video streams."),
> +    .init          = init,
> +    .uninit        = uninit,
> +    .query_formats = query_formats,
> +    .priv_size     = sizeof(ANSNRContext),
> +    .priv_class    = &ansnr_class,
> +    .inputs        = ansnr_inputs,
> +    .outputs       = ansnr_outputs,
> +};
> --
> 2.12.1
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>

On Thu, Mar 30, 2017 at 23:06:36 +0200, Betty Wu wrote:
> A new filter ANSNR is added. libavfilter/Makefile is changed.

That line isn't really needed in a commit message (too obvious).

> +/*
> + * Copyright 2016-2017 Netflix, Inc.
> + * Copyright (c) 2017 Betty Wu
> + *
> + * This file is part of FFmpeg.
> + *
> + * FFmpeg is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2.1 of the License, or (at your option) any later version.

Are you the copyright holder of this code? It looks very similar to
https://github.com/Netflix/vmaf/blob/master/feature/src/ansnr.c and its
dependencies, which are licensed Apache 2.0. IINAL, but you can't
"just" change the license, unless given permission to do so.

> +/* Whether to use border replication instead of zero extension. */
> +/* #define ANSNR_OPT_BORDER_REPLICATE */
> +
> +/* Whether to save intermediate results to files. */
> +/* #define ANSNR_OPT_DEBUG_DUMP */
> +
> +/* Whether to use a 1-D approximation of filters. */
> +/* #define ANSNR_OPT_FILTER_1D */
> +
> +/* Whether to normalize result by dividing against maximum ANSNR. */
> +/* #define ANSNR_OPT_NORMALIZE */
> +
> +/* Whether to use single precision for computation. */
> +#define ANSNR_OPT_SINGLE_PRECISION
> +//#define ANSNR_OPT_DOUBLE_PRECISION

This produces a lot of dead code. Should these not be run-time options,
so that the user can make use of all the code? (Single vs. double
precision could possibly be maintained as two separate filters in one
source file, instead of a run-time option, if there's any reasoning for
keeping both.)

> +#endif /* ANSNR_OPTIONS_H_ */

This shows that this is a verbatim copy of ansnr_options.h from the
named link, and that you don't know what these #define guards are good
for. ;-)


Just some random comments to the source code, I didn't try to
understand the functionality:

> +                    if (ii < 0) ii = -ii;

FFABS()

> +                    imgcoeff = (float)src[ii * src_px_stride + jj]+OPT_RANGE_PIXEL_OFFSET;

Please leave spaces around operators for readability.

> +    if (SIZE_MAX / buf_sz_one < 3)
> +    {
> +        goto fail;
> +    }

ffmpeg's bracket style is different (and some say they could be omitted
here, some say not).

> +    ref_filtr = (number_t *)data_top; data_top += buf_sz_one;

I believe it is preferred not to combine two statements onto one line.


> +    if( !strcmp(frame_format,"yuv420p") || !strcmp(frame_format,"yuv422p") || !strcmp(frame_format,"yuv444p"))
> +        psnr_max=60;
> +    if( !strcmp(frame_format,"yuv420p10le") || !strcmp(frame_format,"yuv422p10le") || !strcmp(frame_format,"yuv444p10le"))
> +        psnr_max=72;

Bracket style.

> +    s->score=score;

Whitespace.

> +    //if (ARCH_X86)
> +    //    ff_ansnr_init_x86(&s->dsp, desc->comp[0].depth);

Unused code shouldn't go in.

Another observation:
Is stats_version initialized anywhere?

Moritz