From patchwork Sat Oct 26 12:00:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ckennedy@ellation.com X-Patchwork-Id: 15973 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 8AB7A4496D8 for ; Sat, 26 Oct 2019 15:00:39 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5FD2E68AC2D; Sat, 26 Oct 2019 15:00:39 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-yw1-f52.google.com (mail-yw1-f52.google.com [209.85.161.52]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E860F68A742 for ; Sat, 26 Oct 2019 15:00:32 +0300 (EEST) Received: by mail-yw1-f52.google.com with SMTP id i123so1957701ywe.11 for ; Sat, 26 Oct 2019 05:00:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ellation.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=diZzI7tp9Kp7So7E/5+GXvIBADrGKxzQJTZB5PxRC7k=; b=GGIHf4xYIaaSJvNICBKThfFvN34IeQnUPmc03cgEycFpCan9fq8MrB/3HdONtCq4Hw UZ68uHNoukJ0xAotioWxx305uQ54Ygbo5ZniSSDKuMQosq6I4PsHjpF0bpYivKoGpx9F bMDPVmqkZ/lE7k9QArWMuI9FlYb3IYF6mB538= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=diZzI7tp9Kp7So7E/5+GXvIBADrGKxzQJTZB5PxRC7k=; b=RQCfajFErszKtEtTZ8hW11YIOrNQ0QArtXYV9cV/6GXg5nz99aUUfP8dd0XmeB/zIE 1u7h6EWADfTgINY9Z9jNUruyA7qeKuKxQf8LrZROtjIr2wLps+jCof+6+HPH/U4HiM2m UVTliv6wsQxeKl9wJic/MNyInkLogf+KoTkObzQEykxHAf0SK7GJjt8RmvK53eO82gmR HO2NqV1pN0RbhPku0tET+xDQdozZ+yxNjDAJ5MBDQu2K9bO7n1bBunEKWC+2/C7h7XgS tPlZUd33meroPjHgDX9706fg+BBKiGiBauxD+YMi/FrJVc7SClCU/HiiEAaqRMlCUA6s QpMA== X-Gm-Message-State: APjAAAWaOEqV22sT5hLzTLanK1z5hTwyV26PZzaD8eS2KwTTMYQTi3T6 4W9cBznG+84qIGU8nd6vsMUMZq8R+sVSqg== X-Google-Smtp-Source: APXvYqzLvtTtdAG0gtN3NuApapVegOI6KHXnlJ14E42pi2M3xqs/iC0usYGjK500pTGnmAY4Cyhcsg== X-Received: by 2002:a81:8087:: with SMTP id q129mr5678586ywf.191.1572091230555; Sat, 26 Oct 2019 05:00:30 -0700 (PDT) Received: from Christophers-MacBook-Pro.local.net (c-69-247-204-250.hsd1.mo.comcast.net. [69.247.204.250]) by smtp.gmail.com with ESMTPSA id f194sm6908571ywb.53.2019.10.26.05.00.29 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sat, 26 Oct 2019 05:00:29 -0700 (PDT) From: ckennedy@ellation.com To: ffmpeg-devel@ffmpeg.org Date: Sat, 26 Oct 2019 07:00:25 -0500 Message-Id: <20191026120025.24822-1-ckennedy@ellation.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] add phqm filter and img_hash X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Christopher Kennedy Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Christopher Kennedy this adds a phqm filter and OpenCV img_hash based resource usable by the phqm and future filters using image hash functionality from OpenCV. C++ to C handling so that full OpenCV functionality and API can be used instead of the C versions (which are incomplete and don't always exist). Example command line: ffmpeg -i encode.mp4 -i reference.mp4 \ -filter_complex "[0:v][1:v]phqm=stats_file=out.log" \ -y -f null /dev/null Signed-off-by: Christopher Kennedy --- Changelog | 1 + configure | 2 + libavfilter/Makefile | 2 + libavfilter/allfilters.c | 1 + libavfilter/img_hash.cpp | 98 ++++++++++++ libavfilter/img_hash.h | 46 ++++++ libavfilter/vf_phqm.c | 334 +++++++++++++++++++++++++++++++++++++++ 7 files changed, 484 insertions(+) create mode 100644 libavfilter/img_hash.cpp create mode 100644 libavfilter/img_hash.h create mode 100644 libavfilter/vf_phqm.c diff --git a/Changelog b/Changelog index 316589e336..4a22f77d37 100644 --- a/Changelog +++ b/Changelog @@ -17,6 +17,7 @@ version : - anlms filter - arnndn filter - bilateral filter +- phqm perceptual hash filter using OpenCV img_lib version 4.2: diff --git a/configure b/configure index 8413826f9e..e231d359bb 100755 --- a/configure +++ b/configure @@ -3497,6 +3497,8 @@ nlmeans_opencl_filter_deps="opencl" nnedi_filter_deps="gpl" ocr_filter_deps="libtesseract" ocv_filter_deps="libopencv" +phqm_filter_deps="libopencv" +phqm_filter_extralibs="-lstdc++ -lopencv_img_hash" openclsrc_filter_deps="opencl" overlay_opencl_filter_deps="opencl" overlay_qsv_filter_deps="libmfx" diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 63d2fba861..645e232b3e 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -325,6 +325,7 @@ OBJS-$(CONFIG_PERMS_FILTER) += f_perms.o OBJS-$(CONFIG_PERSPECTIVE_FILTER) += vf_perspective.o OBJS-$(CONFIG_PHASE_FILTER) += vf_phase.o OBJS-$(CONFIG_PHOTOSENSITIVITY_FILTER) += vf_photosensitivity.o +OBJS-$(CONFIG_PHQM_FILTER) += vf_phqm.o img_hash.o OBJS-$(CONFIG_PIXDESCTEST_FILTER) += vf_pixdesctest.o OBJS-$(CONFIG_PIXSCOPE_FILTER) += vf_datascope.o OBJS-$(CONFIG_PP_FILTER) += vf_pp.o @@ -498,6 +499,7 @@ OBJS-$(CONFIG_SHARED) += log2_tab.o SKIPHEADERS-$(CONFIG_QSVVPP) += qsvvpp.h SKIPHEADERS-$(CONFIG_OPENCL) += opencl.h SKIPHEADERS-$(CONFIG_VAAPI) += vaapi_vpp.h +SKIPHEADERS-$(CONFIG_LIBOPENCV) += img_hash.h TOOLS = graph2dot TESTPROGS = drawutils filtfmts formats integral diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index e4186f93db..f0fcaad235 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -309,6 +309,7 @@ extern AVFilter ff_vf_perms; extern AVFilter ff_vf_perspective; extern AVFilter ff_vf_phase; extern AVFilter ff_vf_photosensitivity; +extern AVFilter ff_vf_phqm; extern AVFilter ff_vf_pixdesctest; extern AVFilter ff_vf_pixscope; extern AVFilter ff_vf_pp; diff --git a/libavfilter/img_hash.cpp b/libavfilter/img_hash.cpp new file mode 100644 index 0000000000..4d5843da22 --- /dev/null +++ b/libavfilter/img_hash.cpp @@ -0,0 +1,98 @@ +/* + * Copyright (c) 2019 Christopher Kennedy + * + * OpenCV img_hash + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include +#include +#include +#include +#include + +#include + +#include "img_hash.h" +#include "libavutil/pixdesc.h" +extern "C" { +#include "avfilter.h" +} + +// convert from avframe to iplimage format +static int fill_iplimage_from_frame(IplImage *img, const AVFrame *frame, enum AVPixelFormat pixfmt) +{ + IplImage *tmpimg; + int depth = IPL_DEPTH_8U, channels_nb; + + switch (pixfmt) { + case AV_PIX_FMT_GRAY8: channels_nb = 1; break; + case AV_PIX_FMT_BGRA: channels_nb = 4; break; + case AV_PIX_FMT_BGR24: channels_nb = 3; break; + default: return -1; + } + + tmpimg = cvCreateImageHeader((CvSize){frame->width, frame->height}, depth, channels_nb); + *img = *tmpimg; + img->imageData = img->imageDataOrigin = (char *) frame->data[0]; + img->dataOrder = IPL_DATA_ORDER_PIXEL; + img->origin = IPL_ORIGIN_TL; + img->widthStep = frame->linesize[0]; + + return 0; +} + +// Get the score of two Video Frames by comparing the perceptual hashes and deriving a hamming distance +// showing how similar they are or different. lower score is better for most algorithms +extern "C" double getScore(const AVFrame *frame1, const AVFrame *frame2, enum AVPixelFormat pixfmt, int hash_type) { + cv::Ptr algo; + IplImage ipl1, ipl2; + cv::Mat h1; + cv::Mat h2; + cv::Mat m1; + cv::Mat m2; + + // Take FFmpeg video frame and convert into an IplImage for OpenCV + if (fill_iplimage_from_frame(&ipl1, frame1, pixfmt) != 0 || + fill_iplimage_from_frame(&ipl2, frame2, pixfmt) != 0) + return DBL_MAX; // Return an invalid value if either fails + + // Convert an IplImage to an Mat Image for OpenCV (newer format) + m1 = cv::cvarrToMat(&ipl1); + m2 = cv::cvarrToMat(&ipl2); + + // substantiate the hash type algorithm + switch (hash_type) { + case PHASH: algo = cv::img_hash::PHash::create(); break; + case AVERAGE: algo = cv::img_hash::AverageHash::create(); break; + case MARRHILDRETH: algo = cv::img_hash::MarrHildrethHash::create(); break; + case RADIALVARIANCE: algo = cv::img_hash::RadialVarianceHash::create(); break; + // BlockMeanHash support mode 0 and mode 1, they associate to + // mode 1 and mode 2 of PHash library + case BLOCKMEAN1: algo = cv::img_hash::BlockMeanHash::create(0); break; + case BLOCKMEAN2: algo = cv::img_hash::BlockMeanHash::create(1); break; + case COLORMOMENT: algo = cv::img_hash::ColorMomentHash::create(); break; + } + + // Compute the hash + algo->compute(m1, h1); + algo->compute(m2, h2); + + // Compare the hashes and return the hamming distance + return algo->compare(h1, h2); +} diff --git a/libavfilter/img_hash.h b/libavfilter/img_hash.h new file mode 100644 index 0000000000..76f55c3013 --- /dev/null +++ b/libavfilter/img_hash.h @@ -0,0 +1,46 @@ +/* + * Copyright (c) 2019 Christopher Kennedy + * + * PHQM Perceptual Hash Quality Metric + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVFILTER_IMG_HASH_H +#define AVFILTER_IMG_HASH_H + +#include "avfilter.h" + +#if defined(__cplusplus) +extern "C" +{ +#endif + +#define AVERAGE 0 +#define BLOCKMEAN1 1 +#define BLOCKMEAN2 2 +#define COLORMOMENT 3 +#define MARRHILDRETH 4 +#define PHASH 5 +#define RADIALVARIANCE 6 + +double getScore(const AVFrame *frame1, const AVFrame *frame2, enum AVPixelFormat pixfmt, int hash_type); +#if defined(__cplusplus) +} +#endif + +#endif diff --git a/libavfilter/vf_phqm.c b/libavfilter/vf_phqm.c new file mode 100644 index 0000000000..0930386b10 --- /dev/null +++ b/libavfilter/vf_phqm.c @@ -0,0 +1,334 @@ +/* + * Copyright (c) 2019 Christopher Kennedy + * + * PHQM Perceptual Hash Quality Metric + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * PHQM: Calculate the Image Hash Hamming Difference between two input videos. + */ + +#include +#include "libavutil/avstring.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "avfilter.h" +#include "drawutils.h" +#include "formats.h" +#include "framesync.h" +#include "internal.h" +#include "video.h" + +#include "img_hash.h" +#include "scene_sad.h" + +typedef struct PHQMContext { + const AVClass *class; + FFFrameSync fs; + double shd, hd, min_hd, max_hd, smin_hd, smax_hd; + uint64_t nb_shd; + uint64_t nb_frames; + FILE *stats_file; + char *stats_file_str; + int hash_type; + ff_scene_sad_fn sad; ///< Sum of the absolute difference function (scene detect only) + double prev_mafd; ///< previous MAFD (scene detect only) + AVFrame *prev_picref; ///< previous frame (scene detect only) + double scd_thresh; + double scene_score; +} PHQMContext; + +#define OFFSET(x) offsetof(PHQMContext, x) +#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM + +static const AVOption phqm_options[] = { + { "stats_file", "Set file where to store per-frame difference information.", OFFSET(stats_file_str), AV_OPT_TYPE_STRING, {.str=NULL}, 0, 0, FLAGS }, + { "f", "Set file where to store per-frame difference information.", OFFSET(stats_file_str), AV_OPT_TYPE_STRING, {.str=NULL}, 0, 0, FLAGS }, + { "scd_thresh", "Scene Change Detection Threshold.", OFFSET(scd_thresh), AV_OPT_TYPE_DOUBLE, {.dbl=0.5}, 0, 1, FLAGS }, + { "hash_type", "Type of Image Hash to use from OpenCV.", OFFSET(hash_type), AV_OPT_TYPE_INT, {.i64 = PHASH}, 0, 6, FLAGS, "hash_type" }, + { "average", "Average Hash", 0, AV_OPT_TYPE_CONST, {.i64 = AVERAGE}, 0, 0, FLAGS, "hash_type" }, + { "blockmean1", "Block Mean Hash 1", 0, AV_OPT_TYPE_CONST, {.i64 = BLOCKMEAN1}, 0, 0, FLAGS, "hash_type" }, + { "blockmean2", "Block Mean Hash 2", 0, AV_OPT_TYPE_CONST, {.i64 = BLOCKMEAN2}, 0, 0, FLAGS, "hash_type" }, + { "colormoment", "Color Moment Hash", 0, AV_OPT_TYPE_CONST, {.i64 = COLORMOMENT}, 0, 0, FLAGS, "hash_type" }, + { "marrhildreth", "Marr Hildreth Hash", 0, AV_OPT_TYPE_CONST, {.i64 = MARRHILDRETH}, 0, 0, FLAGS, "hash_type" }, + { "phash", "Perceptual Hash (PHash)", 0, AV_OPT_TYPE_CONST, {.i64 = PHASH}, 0, 0, FLAGS, "hash_type" }, + { "radialvariance", "Radial Variance Hash", 0, AV_OPT_TYPE_CONST, {.i64 = RADIALVARIANCE}, 0, 0, FLAGS, "hash_type" }, + { NULL } +}; + +FRAMESYNC_DEFINE_CLASS(phqm, PHQMContext, fs); + +static void set_meta(AVDictionary **metadata, const char *key, char comp, float d) +{ + char value[128]; + snprintf(value, sizeof(value), "%0.2f", d); + if (comp) { + char key2[128]; + snprintf(key2, sizeof(key2), "%s%c", key, comp); + av_dict_set(metadata, key2, value, 0); + } else { + av_dict_set(metadata, key, value, 0); + } +} + +static double get_scene_score(AVFilterContext *ctx, AVFrame *frame) +{ + double ret = 0.; + PHQMContext *s = ctx->priv; + AVFrame *prev_picref = s->prev_picref; + + if (prev_picref && + frame->height == prev_picref->height && + frame->width == prev_picref->width) { + uint64_t sad; + double mafd, diff; + + s->sad(prev_picref->data[0], prev_picref->linesize[0], frame->data[0], frame->linesize[0], frame->width * 3, frame->height, &sad); + emms_c(); + mafd = (double)sad / (frame->width * 3 * frame->height); + diff = fabs(mafd - s->prev_mafd); + ret = av_clipf(FFMIN(mafd, diff) / 100., 0, 1); + s->prev_mafd = mafd; + av_frame_free(&prev_picref); + } + s->prev_picref = av_frame_clone(frame); + return ret; +} + +static int do_phqm(FFFrameSync *fs) +{ + AVFilterContext *ctx = fs->parent; + PHQMContext *s = ctx->priv; + AVFrame *master, *ref; + double hd = 0.; + int ret; + double hd_limit = 1000000.; + AVDictionary **metadata; + + ret = ff_framesync_dualinput_get(fs, &master, &ref); + if (ret < 0) + return ret; + if (!ref) + return ff_filter_frame(ctx->outputs[0], master); + metadata = &master->metadata; + + + s->nb_frames++; + + /* scene change detection score */ + s->scene_score = get_scene_score(ctx, ref); + if (s->scene_score >= s->scd_thresh && s->nb_shd >= 48) { + av_log(s, AV_LOG_INFO, "ImgHashScene: n:%"PRId64"-%"PRId64" hd_avg:%0.3lf hd_min:%0.3lf hd_max:%0.3lf scd:%0.2lf\n", + (s->nb_frames - s->nb_shd), s->nb_frames - 1, (s->shd / s->nb_shd), s->smin_hd, s->smax_hd, s->scene_score); + s->shd = 0.; + s->nb_shd = 0; + s->smin_hd = 0.; + s->smax_hd = 0.; + } + + /* limit the highest value so we cut off at perceptual difference match */ + switch (s->hash_type) { + case PHASH: + case AVERAGE: hd_limit = 5; break; + case MARRHILDRETH: hd_limit = 30; break; + case RADIALVARIANCE: hd_limit = 0.9; break; + case BLOCKMEAN1: hd_limit = 12; break; + case BLOCKMEAN2: hd_limit = 48; break; + case COLORMOMENT: hd_limit = 8; break; + } + + /* get ref / enc perceptual hashes and calc hamming distance difference value */ + hd = getScore(ref, master, ref->format, s->hash_type); + if (hd == DBL_MAX) { + av_log(s, AV_LOG_ERROR, "Failure with handling pix_fmt of AVFrame for conversion to IPLimage.\n"); + return AVERROR(EINVAL); + } + s->hd += FFMIN(hd, hd_limit); + set_meta(metadata, "lavfi.phqm.phqm", 0, hd); + + /* scene hamming distance avg */ + s->shd += FFMIN(hd, hd_limit); + s->nb_shd++; + av_log(s, AV_LOG_DEBUG, "ImgHashFrame: hd:%0.3lf scd:%0.2lf\n", hd, s->scene_score); + + s->min_hd = FFMIN(s->min_hd, hd); + s->max_hd = FFMAX(s->max_hd, hd); + s->smin_hd = FFMIN(s->smin_hd, hd); + s->smax_hd = FFMAX(s->smax_hd, hd); + + if (s->stats_file) { + fprintf(s->stats_file, + "n:%"PRId64" phqm:%0.3f phqm_min:%0.3f phqm_max:%0.3f sad:%0.2f", + s->nb_frames, hd, s->min_hd, s->max_hd, s->scene_score); + fprintf(s->stats_file, "\n"); + } + + return ff_filter_frame(ctx->outputs[0], master); +} + +static av_cold int init(AVFilterContext *ctx) +{ + PHQMContext *s = ctx->priv; + + if (s->stats_file_str) { + if (!strcmp(s->stats_file_str, "-")) { + s->stats_file = stdout; + } else { + s->stats_file = fopen(s->stats_file_str, "w"); + if (!s->stats_file) { + int err = AVERROR(errno); + char buf[128]; + av_strerror(err, buf, sizeof(buf)); + av_log(ctx, AV_LOG_ERROR, "Could not open stats file %s: %s\n", + s->stats_file_str, buf); + return err; + } + } + } + + s->sad = ff_scene_sad_get_fn(8); + if (!s->sad) + return AVERROR(EINVAL); + + s->fs.on_event = do_phqm; + return 0; +} + +static int query_formats(AVFilterContext *ctx) +{ + PHQMContext *s = ctx->priv; + AVFilterFormats *fmts_list = NULL; + static const enum AVPixelFormat gray8_pix_fmts[] = { + AV_PIX_FMT_GRAY8, + AV_PIX_FMT_NONE + }; + static const enum AVPixelFormat bgr24_pix_fmts[] = { + AV_PIX_FMT_BGR24, + AV_PIX_FMT_NONE + }; + static const enum AVPixelFormat bgra_pix_fmts[] = { + AV_PIX_FMT_BGRA, + AV_PIX_FMT_NONE + }; + + switch (s->hash_type) { + case COLORMOMENT: fmts_list = ff_make_format_list(bgr24_pix_fmts); break; + case MARRHILDRETH: fmts_list = ff_make_format_list(bgra_pix_fmts); break; + /* all other hashes take the gray8 format */ + default: fmts_list = ff_make_format_list(gray8_pix_fmts); break; + } + if (!fmts_list) + return AVERROR(ENOMEM); + return ff_set_common_formats(ctx, fmts_list); +} + +static int config_input_ref(AVFilterLink *inlink) +{ + AVFilterContext *ctx = inlink->dst; + + if (ctx->inputs[0]->w != ctx->inputs[1]->w || + ctx->inputs[0]->h != ctx->inputs[1]->h) { + av_log(ctx, AV_LOG_ERROR, "Width and height of input videos must be same.\n"); + return AVERROR(EINVAL); + } + if (ctx->inputs[0]->format != ctx->inputs[1]->format) { + av_log(ctx, AV_LOG_ERROR, "Inputs must be of same pixel format.\n"); + return AVERROR(EINVAL); + } + + return 0; +} + +static int config_output(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + PHQMContext *s = ctx->priv; + AVFilterLink *mainlink = ctx->inputs[0]; + int ret; + + ret = ff_framesync_init_dualinput(&s->fs, ctx); + if (ret < 0) + return ret; + outlink->w = mainlink->w; + outlink->h = mainlink->h; + outlink->time_base = mainlink->time_base; + outlink->sample_aspect_ratio = mainlink->sample_aspect_ratio; + outlink->frame_rate = mainlink->frame_rate; + if ((ret = ff_framesync_configure(&s->fs)) < 0) + return ret; + + return 0; +} + +static int activate(AVFilterContext *ctx) +{ + PHQMContext *s = ctx->priv; + return ff_framesync_activate(&s->fs); +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + PHQMContext *s = ctx->priv; + + if (s->nb_frames > 0) + av_log(ctx, AV_LOG_WARNING, "PHQM average:%f min:%f max:%f\n", + s->hd / s->nb_frames, s->min_hd, s->max_hd); + + ff_framesync_uninit(&s->fs); + + if (s->stats_file && s->stats_file != stdout) + fclose(s->stats_file); + av_frame_free(&s->prev_picref); +} + +static const AVFilterPad phqm_inputs[] = { + { + .name = "main", + .type = AVMEDIA_TYPE_VIDEO, + },{ + .name = "reference", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_input_ref, + }, + { NULL } +}; + +static const AVFilterPad phqm_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_output, + }, + { NULL } +}; + +AVFilter ff_vf_phqm= { + .name = "phqm", + .description = NULL_IF_CONFIG_SMALL("PHQM: Calculate the Perceptual Hash Hamming Difference between two video streams."), + .preinit = phqm_framesync_preinit, + .init = init, + .uninit = uninit, + .query_formats = query_formats, + .activate = activate, + .priv_size = sizeof(PHQMContext), + .priv_class = &phqm_class, + .inputs = phqm_inputs, + .outputs = phqm_outputs, +};