From patchwork Mon Aug 9 10:29:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 29378 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:6c0f:0:0:0:0:0 with SMTP id a15csp2736315ioh; Mon, 9 Aug 2021 03:29:52 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy43Sr5ps/I0rEm3UoiHds1Cj/7UXVJ9aSFCDjPvoDrfPiAPwA2878377rQMI2RAXcNURMU X-Received: by 2002:aa7:d54b:: with SMTP id u11mr18314902edr.160.1628504991965; Mon, 09 Aug 2021 03:29:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1628504991; cv=none; d=google.com; s=arc-20160816; b=DtqcsdkG8oISPQ1CxgMq2GCcur/DSGeLx5GAz0cENKZTUSthNoAPVcS2xn+m0Y5opX tTnUnR9bpOGWN0ZvOZn1o/qxyvoKicjTb2UkXuKZ2nBZhUtAyjqw7/6Z8jGjEYC/djr1 rMfo49GPtB1+PIM5s9ihGHXPFw5Hc7ukbgk3NgmNNZ0/TA3oPxsrfYaGns6Y+woP/AN1 F6lMvWe0YpLLpwk4CqyrI+NyFjHzS+KarZrqMdhbbrwmHvpyfecJ/x+eTlzbetFCg3yj eGcsMjZ2QflF/FvmOPYx8adDYSTusTG6Ll8YmOXprmDXPtZ/2uNioqEkQEWjMZxklvqO ps2g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :delivered-to; bh=mdgQKXANdXjnZCbz7u7xmYbaY5ux2bzXwdq8X+jg9OI=; b=NH7kNZh7O7C8Cu7xWZKv/M/1sQSkMS1FdNnCNbYIYvyiHezFlzc0bH7GZHcgWqfO9r sUacjZ5fA3af6wOJBhnL3OqNaXMG6Ofc3FQUNUPKBeWvzulAEKnGdKXTIfaM2ijkElCu kP53uMvWRNCBz3G4dy+XNDAY/xuBHU5XN5qu7jICRocFmlkUcR+23peiLKbyhfzEr1th a94W+jrno4e8mSFHbKfuGNHCiYpda6cpM7c7iDuARj0W2FlZyQ9lsGPTsRu3wDvrBsyW Fle2ZAtC0qfGYunfJ/8d7NMazohCGGMaAbSCSx0FppPT7+4Y/hX6q9uaX+0F1CKDYJa4 cTxQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id y19si17364426ejp.323.2021.08.09.03.29.51; Mon, 09 Aug 2021 03:29:51 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B538668A2B9; Mon, 9 Aug 2021 13:29:37 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail0.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 26A8A6897E8 for ; Mon, 9 Aug 2021 13:29:30 +0300 (EEST) Received: from localhost (localhost [IPv6:::1]) by mail0.khirnov.net (Postfix) with ESMTP id E38902404C8 for ; Mon, 9 Aug 2021 12:29:29 +0200 (CEST) Received: from mail0.khirnov.net ([IPv6:::1]) by localhost (mail0.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id aTWJcoavUYTI for ; Mon, 9 Aug 2021 12:29:27 +0200 (CEST) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail0.khirnov.net (Postfix) with ESMTPS id AC4542404B2 for ; Mon, 9 Aug 2021 12:29:27 +0200 (CEST) Received: by libav.khirnov.net (Postfix, from userid 1000) id 846583A0BCA; Mon, 9 Aug 2021 12:29:27 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Mon, 9 Aug 2021 12:29:18 +0200 Message-Id: <20210809102919.387-1-anton@khirnov.net> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/2] lavfi: Add vf_ssim360 filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: +jogB8QTPxlU From: Shannon Chen Customized SSIM for various projections (and stereo formats) of 360 images and videos. Further contributions by: Ashok Mathew Kuruvilla Matthieu Patou Yu-Hui Wu Anton Khirnov Suggested-By: ffmpeg@fb.com Signed-off-by: Anton Khirnov --- Changelog | 1 + libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/version.h | 4 +- libavfilter/vf_ssim360.c | 1762 ++++++++++++++++++++++++++++++++++++++ 5 files changed, 1767 insertions(+), 2 deletions(-) create mode 100644 libavfilter/vf_ssim360.c diff --git a/Changelog b/Changelog index 1037688682..56791ba26d 100644 --- a/Changelog +++ b/Changelog @@ -9,6 +9,7 @@ version : - Argonaut Games CVG muxer - Concatf protocol - afwtdn audio filter +- ssim360 video filter version 4.4: diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 49c0c8342b..b0348ccfa3 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -433,6 +433,7 @@ OBJS-$(CONFIG_SPLIT_FILTER) += split.o OBJS-$(CONFIG_SPP_FILTER) += vf_spp.o qp_table.o OBJS-$(CONFIG_SR_FILTER) += vf_sr.o OBJS-$(CONFIG_SSIM_FILTER) += vf_ssim.o framesync.o +OBJS-$(CONFIG_SSIM360_FILTER) += vf_ssim360.o framesync.o OBJS-$(CONFIG_STEREO3D_FILTER) += vf_stereo3d.o OBJS-$(CONFIG_STREAMSELECT_FILTER) += f_streamselect.o framesync.o OBJS-$(CONFIG_SUBTITLES_FILTER) += vf_subtitles.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index ae74f9c891..76863c9ea1 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -413,6 +413,7 @@ extern const AVFilter ff_vf_split; extern const AVFilter ff_vf_spp; extern const AVFilter ff_vf_sr; extern const AVFilter ff_vf_ssim; +extern const AVFilter ff_vf_ssim360; extern const AVFilter ff_vf_stereo3d; extern const AVFilter ff_vf_streamselect; extern const AVFilter ff_vf_subtitles; diff --git a/libavfilter/version.h b/libavfilter/version.h index 75cd10dccd..67f2a5883c 100644 --- a/libavfilter/version.h +++ b/libavfilter/version.h @@ -30,8 +30,8 @@ #include "libavutil/version.h" #define LIBAVFILTER_VERSION_MAJOR 8 -#define LIBAVFILTER_VERSION_MINOR 1 -#define LIBAVFILTER_VERSION_MICRO 103 +#define LIBAVFILTER_VERSION_MINOR 2 +#define LIBAVFILTER_VERSION_MICRO 100 #define LIBAVFILTER_VERSION_INT AV_VERSION_INT(LIBAVFILTER_VERSION_MAJOR, \ diff --git a/libavfilter/vf_ssim360.c b/libavfilter/vf_ssim360.c new file mode 100644 index 0000000000..1e6216c630 --- /dev/null +++ b/libavfilter/vf_ssim360.c @@ -0,0 +1,1762 @@ +/** + * Copyright (c) 2015-2021, Facebook, Inc. + * All rights reserved. + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/* Computes the Structural Similarity Metric between two 360 video streams. + * original SSIM algorithm: + * Z. Wang, A. C. Bovik, H. R. Sheikh and E. P. Simoncelli, + * "Image quality assessment: From error visibility to structural similarity," + * IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600-612, Apr. 2004. + * + * To improve speed, this implementation uses the standard approximation of + * overlapped 8x8 block sums, rather than the original gaussian weights. + * + * To address warping from 360 projections for videos with same + * projection and resolution, the 8x8 blocks sampled are weighted by + * their location in the image. + * + * To apply SSIM across projections and video sizes, we render the video on to + * a flat "tape" from which the 8x8 are selected and compared. + */ + +/* + * @file + * Caculate the SSIM between two input 360 videos. + */ + +#include "libavutil/avstring.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "avfilter.h" +#include "drawutils.h" +#include "formats.h" +#include "internal.h" +#include "video.h" +#include "framesync.h" +#include + +#define RIGHT 0 +#define LEFT 1 +#define TOP 2 +#define BOTTOM 3 +#define FRONT 4 +#define BACK 5 + +#define DEFAULT_HEATMAP_W 32 +#define DEFAULT_HEATMAP_H 16 + +#define M_PI_F ((float)M_PI) +#define M_PI_2_F ((float)M_PI_2) +#define M_PI_4_F ((float)M_PI_4) +#define M_SQRT2_F ((float)M_SQRT2) + +#define DEFAULT_EXPANSION_COEF 1.01f + +static const float BARREL_THETA_RANGE = DEFAULT_EXPANSION_COEF * 2.0f * M_PI_F; +static const float BARREL_PHI_RANGE = DEFAULT_EXPANSION_COEF * M_PI_2_F; + +// Use fixed-point with 16 bit precision for fast bilinear math +static const int FIXED_POINT_PRECISION = 16; + +// Use 1MB per channel for the histogram to get 5-digit precise SSIM value +#define SSIM360_HIST_SIZE 131072 + +// The last number is a marker < 0 to mark end of list +static const double PERCENTILE_LIST[] = { + 1.0, 0.9, 0.8, 0.7, 0.6, + 0.5, 0.4, 0.3, 0.2, 0.1, 0, -1 +}; + +typedef enum StereoFormat { + STEREO_FORMAT_TB, + STEREO_FORMAT_LR, + STEREO_FORMAT_MONO, + STEREO_FORMAT_N +} StereoFormat; + +typedef enum Projection { + PROJECTION_CUBEMAP32, + PROJECTION_CUBEMAP23, + PROJECTION_BARREL, + PROJECTION_BARREL_SPLIT, + PROJECTION_EQUIRECT, + PROJECTION_N +} Projection; + +typedef struct Map2D { + int w, h; + double **value; +} Map2D; + +typedef struct HeatmapList { + Map2D map; + struct HeatmapList *next; +} HeatmapList; + +typedef struct SampleParams { + int stride; + int planewidth; + int planeheight; + int x_image_offset; + int y_image_offset; + int x_image_range; + int y_image_range; + int projection; + float expand_coef; +} SampleParams; + +typedef struct BilinearMap { + // Indices to the 4 samples to compute bilinear + int tli; + int tri; + int bli; + int bri; + + // Fixed point factors with which the above 4 sample vector's + // dot product needs to be computed for the final bilinear value + int tlf; + int trf; + int blf; + int brf; +} BilinearMap; + +typedef struct SSIM360Context { + const AVClass *class; + + FFFrameSync fs; + // Stats file configuration + FILE *stats_file; + char *stats_file_str; + + // Component properties + int nb_components; + double coefs[4]; + char comps[4]; + int max; + + // Channel configuration & properties + int compute_chroma; + + int is_rgb; + uint8_t rgba_map[4]; + + // Standard SSIM computation configuration & workspace + uint64_t frame_skip_ratio; + + int *temp; + uint64_t nb_ssim_frames; + uint64_t nb_net_frames; + double ssim360[4], ssim360_total; + double *ssim360_hist[4]; + double ssim360_hist_net[4]; + double ssim360_percentile_sum[4][256]; + + // 360 projection configuration & workspace + int ref_projection; + int main_projection; + int ref_stereo_format; + int main_stereo_format; + float ref_pad; + float main_pad; + int use_tape; + char *heatmap_path; + int default_heatmap_w; + int default_heatmap_h; + + Map2D density; + HeatmapList *heatmaps; + int ref_planewidth[4]; + int ref_planeheight[4]; + int main_planewidth[4]; + int main_planeheight[4]; + int tape_length[4]; + BilinearMap *ref_tape_map[4][2]; + BilinearMap *main_tape_map[4][2]; + float angular_resolution[4][2]; + double (*ssim360_plane)( + uint8_t *main, int main_stride, + uint8_t *ref, int ref_stride, + int width, int height, void *temp, + int max, Map2D density); +} SSIM360Context; + +#define OFFSET(x) offsetof(SSIM360Context, x) +#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM + +static const AVOption ssim360_options[] = { + { "stats_file", "Set file where to store per-frame difference information", + OFFSET(stats_file_str), AV_OPT_TYPE_STRING, {.str=NULL}, 0, 0, FLAGS }, + { "f", "Set file where to store per-frame difference information", + OFFSET(stats_file_str), AV_OPT_TYPE_STRING, {.str=NULL}, 0, 0, FLAGS }, + + { "compute_chroma", + "Specifies if non-luma channels must be computed", + OFFSET(compute_chroma), AV_OPT_TYPE_INT, {.i64 = 1}, + 0, 1, .flags = FLAGS }, + + { "frame_skip_ratio", + "Specifies the number of frames to be skipped from evaluation, for every evaluated frame", + OFFSET(frame_skip_ratio), AV_OPT_TYPE_INT, {.i64 = 0}, + 0, 1000000, .flags = FLAGS }, + + { "ref_projection", "projection of the reference video", + OFFSET(ref_projection), AV_OPT_TYPE_INT, {.i64 = PROJECTION_EQUIRECT}, + 0, PROJECTION_N - 1, .flags = FLAGS, "projection" }, + + { "e", "equirectangular", 0, AV_OPT_TYPE_CONST, {.i64 = PROJECTION_EQUIRECT}, 0, 0, FLAGS, "projection" }, + { "equirect", "equirectangular", 0, AV_OPT_TYPE_CONST, {.i64 = PROJECTION_EQUIRECT}, 0, 0, FLAGS, "projection" }, + { "c3x2", "cubemap 3x2", 0, AV_OPT_TYPE_CONST, {.i64 = PROJECTION_CUBEMAP32}, 0, 0, FLAGS, "projection" }, + { "c2x3", "cubemap 2x3", 0, AV_OPT_TYPE_CONST, {.i64 = PROJECTION_CUBEMAP23}, 0, 0, FLAGS, "projection" }, + { "barrel", "barrel facebook's 360 format", 0, AV_OPT_TYPE_CONST, {.i64 = PROJECTION_BARREL}, 0, 0, FLAGS, "projection" }, + { "barrelsplit", "barrel split facebook's 360 format", 0, AV_OPT_TYPE_CONST, {.i64 = PROJECTION_BARREL_SPLIT}, 0, 0, FLAGS, "projection" }, + + { "main_projection", "projection of the main video", + OFFSET(main_projection), AV_OPT_TYPE_INT, {.i64 = PROJECTION_N}, + 0, PROJECTION_N, .flags = FLAGS, "projection" }, + + { "ref_stereo", "stereo format of the reference video", + OFFSET(ref_stereo_format), AV_OPT_TYPE_INT, {.i64 = STEREO_FORMAT_MONO}, + 0, STEREO_FORMAT_N - 1, .flags = FLAGS, "stereo_format" }, + + { "mono", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = STEREO_FORMAT_MONO }, 0, 0, FLAGS, "stereo_format" }, + { "tb", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = STEREO_FORMAT_TB }, 0, 0, FLAGS, "stereo_format" }, + { "lr", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = STEREO_FORMAT_LR }, 0, 0, FLAGS, "stereo_format" }, + + { "main_stereo", "stereo format of main video", + OFFSET(main_stereo_format), AV_OPT_TYPE_INT, {.i64 = STEREO_FORMAT_N}, + 0, STEREO_FORMAT_N, .flags = FLAGS, "stereo_format" }, + + { "ref_pad", + "Expansion (padding) coefficient for each cube face of the reference video", + OFFSET(ref_pad), AV_OPT_TYPE_FLOAT, {.dbl = .0f}, 0, 10, .flags = FLAGS }, + + { "main_pad", + "Expansion (padding) coeffiecient for each cube face of the main video", + OFFSET(main_pad), AV_OPT_TYPE_FLOAT, {.dbl = .0f}, 0, 10, .flags = FLAGS }, + + { "use_tape", + "Specifies if the tape based SSIM 360 algorithm must be used independent of the input video types", + OFFSET(use_tape), AV_OPT_TYPE_INT, {.i64 = 0}, + 0, 1, .flags = FLAGS }, + + { "heatmap_path", + "Heatmap data for view-based evaluation. For heatmap file format, please refer to EntSphericalVideoHeatmapData.", + OFFSET(heatmap_path), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, .flags = FLAGS }, + + { "default_heatmap_width", + "Default heatmap dimension. Will be used when dimension is not specified in heatmap data.", + OFFSET(default_heatmap_w), AV_OPT_TYPE_INT, {.i64 = 32}, 1, 4096, .flags = FLAGS }, + + { "default_heatmap_height", + "Default heatmap dimension. Will be used when dimension is not specified in heatmap data.", + OFFSET(default_heatmap_h), AV_OPT_TYPE_INT, {.i64 = 16}, 1, 4096, .flags = FLAGS }, + + { NULL } +}; + +FRAMESYNC_DEFINE_CLASS(ssim360, SSIM360Context, fs); + +static void set_meta(AVDictionary **metadata, const char *key, char comp, float d) +{ + char value[128]; + snprintf(value, sizeof(value), "%0.2f", d); + if (comp) { + char key2[128]; + snprintf(key2, sizeof(key2), "%s%c", key, comp); + av_dict_set(metadata, key2, value, 0); + } else { + av_dict_set(metadata, key, value, 0); + } +} + +static void map_uninit(Map2D *map) +{ + for (int i = 0; i < map->h; i++) + av_freep(&map->value[i]); + + av_freep(&map->value); +} + +static int map_init(Map2D *map, int w, int h) +{ + map->h = h; + map->w = w; + + map->value = av_mallocz_array(h, sizeof(*map->value)); + if (!map->value) + return AVERROR(ENOMEM); + + for (int i = 0; i < h; i++) { + map->value[i] = av_mallocz_array(w, sizeof(*map->value[i])); + if (!map->value[i]) { + map_uninit(map); + return AVERROR(ENOMEM); + } + } + + return 0; +} + +static void map_list_free(HeatmapList **pl) +{ + HeatmapList *l = *pl; + + while (l) { + HeatmapList *next = l->next; + map_uninit(&l->map); + av_freep(&l); + l = next; + } + + *pl = NULL; +} + +static int map_alloc(HeatmapList **pl, int w, int h) +{ + HeatmapList *l; + int ret; + + l = av_mallocz(sizeof(*l)); + if (!l) + return AVERROR(ENOMEM); + + ret = map_init(&l->map, w, h); + if (ret < 0) { + av_freep(&l); + return ret; + } + + *pl = l; + return 0; +} + +static void +ssim360_4x4xn_16bit(const uint8_t *main8, ptrdiff_t main_stride, + const uint8_t *ref8, ptrdiff_t ref_stride, + int64_t (*sums)[4], int width) +{ + const uint16_t *main16 = (const uint16_t *)main8; + const uint16_t *ref16 = (const uint16_t *)ref8; + + main_stride >>= 1; + ref_stride >>= 1; + + for (int z = 0; z < width; z++) { + uint64_t s1 = 0, s2 = 0, ss = 0, s12 = 0; + + for (int y = 0; y < 4; y++) { + for (int x = 0; x < 4; x++) { + unsigned a = main16[x + y * main_stride]; + unsigned b = ref16[x + y * ref_stride]; + + s1 += a; + s2 += b; + ss += a*a; + ss += b*b; + s12 += a*b; + } + } + + sums[z][0] = s1; + sums[z][1] = s2; + sums[z][2] = ss; + sums[z][3] = s12; + main16 += 4; + ref16 += 4; + } +} + +static void +ssim360_4x4xn_8bit(const uint8_t *main, ptrdiff_t main_stride, + const uint8_t *ref, ptrdiff_t ref_stride, + int (*sums)[4], int width) +{ + for (int z = 0; z < width; z++) { + uint32_t s1 = 0, s2 = 0, ss = 0, s12 = 0; + + for (int y = 0; y < 4; y++) { + for (int x = 0; x < 4; x++) { + int a = main[x + y * main_stride]; + int b = ref[x + y * ref_stride]; + + s1 += a; + s2 += b; + ss += a*a; + ss += b*b; + s12 += a*b; + } + } + + sums[z][0] = s1; + sums[z][1] = s2; + sums[z][2] = ss; + sums[z][3] = s12; + main += 4; + ref += 4; + } +} + +static float ssim360_end1x(int64_t s1, int64_t s2, int64_t ss, int64_t s12, int max) +{ + int64_t ssim_c1 = (int64_t)(.01 * .01 * max * max * 64 + .5); + int64_t ssim_c2 = (int64_t)(.03 * .03 * max * max * 64 * 63 + .5); + + int64_t fs1 = s1; + int64_t fs2 = s2; + int64_t fss = ss; + int64_t fs12 = s12; + int64_t vars = fss * 64 - fs1 * fs1 - fs2 * fs2; + int64_t covar = fs12 * 64 - fs1 * fs2; + + return (float)(2 * fs1 * fs2 + ssim_c1) * (float)(2 * covar + ssim_c2) + / ((float)(fs1 * fs1 + fs2 * fs2 + ssim_c1) * (float)(vars + ssim_c2)); +} + +static float ssim360_end1(int s1, int s2, int ss, int s12) +{ + static const int ssim_c1 = (int)(.01*.01*255*255*64 + .5); + static const int ssim_c2 = (int)(.03*.03*255*255*64*63 + .5); + + int fs1 = s1; + int fs2 = s2; + int fss = ss; + int fs12 = s12; + int vars = fss * 64 - fs1 * fs1 - fs2 * fs2; + int covar = fs12 * 64 - fs1 * fs2; + + return (float)(2 * fs1 * fs2 + ssim_c1) * (float)(2 * covar + ssim_c2) + / ((float)(fs1 * fs1 + fs2 * fs2 + ssim_c1) * (float)(vars + ssim_c2)); +} + +static double +ssim360_endn_16bit(const int64_t (*sum0)[4], const int64_t (*sum1)[4], + int width, int max, + double *density_map, int map_width, double *total_weight) +{ + double ssim360 = 0.0, weight; + + for (int i = 0; i < width; i++) { + weight = density_map ? density_map[(int) ((0.5 + i) / width * map_width)] : 1.0; + ssim360 += weight * ssim360_end1x( + sum0[i][0] + sum0[i + 1][0] + sum1[i][0] + sum1[i + 1][0], + sum0[i][1] + sum0[i + 1][1] + sum1[i][1] + sum1[i + 1][1], + sum0[i][2] + sum0[i + 1][2] + sum1[i][2] + sum1[i + 1][2], + sum0[i][3] + sum0[i + 1][3] + sum1[i][3] + sum1[i + 1][3], + max); + *total_weight += weight; + } + return ssim360; +} + +static double +ssim360_endn_8bit(const int (*sum0)[4], const int (*sum1)[4], int width, + double *density_map, int map_width, double *total_weight) +{ + double ssim360 = 0.0, weight; + + for (int i = 0; i < width; i++) { + weight = density_map ? density_map[(int) ((0.5 + i) / width * map_width)] : 1.0; + ssim360 += weight * ssim360_end1( + sum0[i][0] + sum0[i + 1][0] + sum1[i][0] + sum1[i + 1][0], + sum0[i][1] + sum0[i + 1][1] + sum1[i][1] + sum1[i + 1][1], + sum0[i][2] + sum0[i + 1][2] + sum1[i][2] + sum1[i + 1][2], + sum0[i][3] + sum0[i + 1][3] + sum1[i][3] + sum1[i + 1][3]); + *total_weight += weight; + } + return ssim360; +} + +static double +ssim360_plane_16bit(uint8_t *main, int main_stride, + uint8_t *ref, int ref_stride, + int width, int height, void *temp, + int max, Map2D density) +{ + int z = 0; + double ssim360 = 0.0; + int64_t (*sum0)[4] = temp; + int64_t (*sum1)[4] = sum0 + (width >> 2) + 3; + double total_weight = 0.0; + + width >>= 2; + height >>= 2; + + for (int y = 1; y < height; y++) { + for (; z <= y; z++) { + FFSWAP(void*, sum0, sum1); + ssim360_4x4xn_16bit(&main[4 * z * main_stride], main_stride, + &ref[4 * z * ref_stride], ref_stride, + sum0, width); + } + ssim360 += ssim360_endn_16bit( + (const int64_t (*)[4])sum0, (const int64_t (*)[4])sum1, + width - 1, max, + density.value ? density.value[(int) ((z - 1.0) / height * density.h)] : NULL, density.w, + &total_weight); + } + + return (double) (ssim360 / total_weight); +} + +static double +ssim360_plane_8bit(uint8_t *main, int main_stride, + uint8_t *ref, int ref_stride, + int width, int height, void *temp, + int max, Map2D density) +{ + int z = 0; + double ssim360 = 0.0; + int (*sum0)[4] = temp; + int (*sum1)[4] = sum0 + (width >> 2) + 3; + double total_weight = 0.0; + + width >>= 2; + height >>= 2; + + for (int y = 1; y < height; y++) { + for (; z <= y; z++) { + FFSWAP(void*, sum0, sum1); + ssim360_4x4xn_8bit( + &main[4 * z * main_stride], main_stride, + &ref[4 * z * ref_stride], ref_stride, + sum0, width); + } + ssim360 += ssim360_endn_8bit( + (const int (*)[4])sum0, (const int (*)[4])sum1, width - 1, + density.value ? density.value[(int) ((z - 1.0) / height * density.h)] : NULL, density.w, + &total_weight); + } + + return (double) (ssim360 / total_weight); +} + +static double ssim360_db(double ssim360, double weight) +{ + return 10 * log10(weight / (weight - ssim360)); +} + +static int get_bilinear_sample(const uint8_t *data, BilinearMap *m, int max_value) +{ + static const int fixed_point_half = 1 << (FIXED_POINT_PRECISION - 1); + static const int inv_byte_mask = (-1) << 8; + + int tl, tr, bl, br, v; + + if (max_value & inv_byte_mask) { + uint16_t *data16 = (uint16_t *)data; + tl = data16[m->tli]; + tr = data16[m->tri]; + bl = data16[m->bli]; + br = data16[m->bri]; + } else { + tl = data[m->tli]; + tr = data[m->tri]; + bl = data[m->bli]; + br = data[m->bri]; + } + + v = m->tlf * tl + + m->trf * tr + + m->blf * bl + + m->brf * br; + + // Round by half, and revert the fixed-point offset + return ((v + fixed_point_half) >> FIXED_POINT_PRECISION) & max_value; +} + +static void +ssim360_4x4x2_tape(const uint8_t *main, BilinearMap *main_maps, + const uint8_t *ref, BilinearMap *ref_maps, + int offset_y, int max_value, int (*sums)[4]) +{ + int offset_x = 0; + + // Two blocks along the width + for (int z = 0; z < 2; z++) { + int s1 = 0, s2 = 0, ss = 0, s12 = 0; + + // 4 pixel block from (offset_x, offset_y) + for (int y = offset_y; y < offset_y + 4; y++) { + int y_stride = y << 3; + for (int x = offset_x; x < offset_x + 4; x++) { + int map_index = x + y_stride; + int a = get_bilinear_sample(main, main_maps + map_index, max_value); + int b = get_bilinear_sample(ref, ref_maps + map_index, max_value); + + s1 += a; + s2 += b; + ss += a*a; + ss += b*b; + s12 += a*b; + } + } + + sums[z][0] = s1; + sums[z][1] = s2; + sums[z][2] = ss; + sums[z][3] = s12; + + offset_x += 4; + } +} + +static float get_radius_between_negative_and_positive_pi(float theta) +{ + int floor_theta_by_2pi, floor_theta_by_pi; + + // Convert theta to range [0, 2*pi] + floor_theta_by_2pi = (int)(theta / (2.0f * M_PI_F)) - (theta < 0.0f); + theta -= 2.0f * M_PI_F * floor_theta_by_2pi; + + // Convert theta to range [-pi, pi] + floor_theta_by_pi = theta / M_PI_F; + theta -= 2.0f * M_PI_F * floor_theta_by_pi; + return FFMIN(M_PI_F, FFMAX(-M_PI_F, theta)); +} + +static float get_heat(HeatmapList *heatmaps, float angular_resoluation, float norm_tape_pos) +{ + float pitch, yaw, norm_pitch, norm_yaw; + int w, h; + + if (!heatmaps) + return 1.0f; + + pitch = asinf(norm_tape_pos*2); + yaw = M_PI_2_F * pitch / angular_resoluation; + yaw = get_radius_between_negative_and_positive_pi(yaw); + + // normalize into [0,1] + norm_pitch = 1.0f - (pitch / M_PI_F + 0.5f); + norm_yaw = yaw / 2.0f / M_PI_F + 0.5f; + + // get heat on map + w = FFMIN(heatmaps->map.w - 1, FFMAX(0, heatmaps->map.w * norm_yaw)); + h = FFMIN(heatmaps->map.h - 1, FFMAX(0, heatmaps->map.h * norm_pitch)); + return heatmaps->map.value[h][w]; +} + +static double +ssim360_tape(uint8_t *main, BilinearMap *main_maps, + uint8_t *ref, BilinearMap *ref_maps, + int tape_length, int max_value, void *temp, + double *ssim360_hist, double *ssim360_hist_net, + float angular_resolution, HeatmapList *heatmaps) +{ + int horizontal_block_count = 2; + int vertical_block_count = tape_length >> 2; + + int z = 0, y; + // Since the tape will be very long and we need to average over all 8x8 blocks, use double + double ssim360 = 0.0; + double sum_weight = 0.0; + + int (*sum0)[4] = temp; + int (*sum1)[4] = sum0 + horizontal_block_count + 3; + + for (y = 1; y < vertical_block_count; y++) { + int fs1, fs2, fss, fs12, hist_index; + float norm_tape_pos, weight; + double sample_ssim360; + + for (; z <= y; z++) { + FFSWAP(void*, sum0, sum1); + ssim360_4x4x2_tape(main, main_maps, ref, ref_maps, z*4, max_value, sum0); + } + + // Given we have only one 8x8 block, following sums fit within 26 bits even for 10bit videos + fs1 = sum0[0][0] + sum0[1][0] + sum1[0][0] + sum1[1][0]; + fs2 = sum0[0][1] + sum0[1][1] + sum1[0][1] + sum1[1][1]; + fss = sum0[0][2] + sum0[1][2] + sum1[0][2] + sum1[1][2]; + fs12 = sum0[0][3] + sum0[1][3] + sum1[0][3] + sum1[1][3]; + + if (max_value > 255) { + // Since we need high precision to multiply fss / fs12 by 64, use double + double ssim_c1_d = .01*.01*64*max_value*max_value; + double ssim_c2_d = .03*.03*64*63*max_value*max_value; + + double vars = 64. * fss - 1. * fs1 * fs1 - 1. * fs2 * fs2; + double covar = 64. * fs12 - 1.*fs1 * fs2; + sample_ssim360 = (2. * fs1 * fs2 + ssim_c1_d) * (2. * covar + ssim_c2_d) + / ((1. * fs1 * fs1 + 1. * fs2 * fs2 + ssim_c1_d) * (1. * vars + ssim_c2_d)); + } else { + static const int ssim_c1 = (int)(.01*.01*255*255*64 + .5); + static const int ssim_c2 = (int)(.03*.03*255*255*64*63 + .5); + + int vars = fss * 64 - fs1 * fs1 - fs2 * fs2; + int covar = fs12 * 64 - fs1 * fs2; + sample_ssim360 = (double)(2 * fs1 * fs2 + ssim_c1) * (double)(2 * covar + ssim_c2) + / ((double)(fs1 * fs1 + fs2 * fs2 + ssim_c1) * (double)(vars + ssim_c2)); + } + + hist_index = (int)(sample_ssim360 * ((double)SSIM360_HIST_SIZE - .5)); + hist_index = av_clip(hist_index, 0, SSIM360_HIST_SIZE - 1); + + norm_tape_pos = (y - 0.5f) / (vertical_block_count - 1.0f) - 0.5f; + // weight from an input heatmap if available, otherwise weight = 1.0 + weight = get_heat(heatmaps, angular_resolution, norm_tape_pos); + ssim360_hist[hist_index] += weight; + *ssim360_hist_net += weight; + + ssim360 += (sample_ssim360 * weight); + sum_weight += weight; + } + + return ssim360 / sum_weight; +} + +static void compute_bilinear_map(SampleParams *p, BilinearMap *m, float x, float y) +{ + float fixed_point_scale = (float)(1 << FIXED_POINT_PRECISION); + + // All operations in here will fit in the 22 bit mantissa of floating point, + // since the fixed point precision is well under 22 bits + float x_image = av_clipf(x * p->x_image_range, 0, p->x_image_range) + p->x_image_offset; + float y_image = av_clipf(y * p->y_image_range, 0, p->y_image_range) + p->y_image_offset; + + int x_floor = x_image; + int y_floor = y_image; + float x_diff = x_image - x_floor; + float y_diff = y_image - y_floor; + + int x_ceil = x_floor + (x_diff > 1e-6); + int y_ceil = y_floor + (y_diff > 1e-6); + float x_inv_diff = 1.0f - x_diff; + float y_inv_diff = 1.0f - y_diff; + + // Indices of the 4 samples from source frame + m->tli = x_floor + y_floor * p->stride; + m->tri = x_ceil + y_floor * p->stride; + m->bli = x_floor + y_ceil * p->stride; + m->bri = x_ceil + y_ceil * p->stride; + + // Scale to be applied to each of the 4 samples from source frame + m->tlf = x_inv_diff * y_inv_diff * fixed_point_scale; + m->trf = x_diff * y_inv_diff * fixed_point_scale; + m->blf = x_inv_diff * y_diff * fixed_point_scale; + m->brf = x_diff * y_diff * fixed_point_scale; +} + +static void get_equirect_map(float phi, float theta, float *x, float *y) +{ + *x = 0.5f + theta / (2.0f * M_PI_F); + // y increases downwards + *y = 0.5f - phi / M_PI_F; +} + +static void get_barrel_map(float phi, float theta, float *x, float *y) +{ + float abs_phi = FFABS(phi); + + if (abs_phi <= M_PI_4_F) { + // Equirect region + *x = 0.8f * (0.5f + theta / BARREL_THETA_RANGE); + // y increases downwards + *y = 0.5f - phi / BARREL_PHI_RANGE; + } else { + // Radial ratio on a unit circle = cot(abs_phi) / (expansion_cefficient). + // Using cos(abs_phi)/sin(abs_phi) explicitly to avoid division by zero + float radial_ratio = cosf(abs_phi) / (sinf(abs_phi) * DEFAULT_EXPANSION_COEF); + float circle_x = radial_ratio * sinf(theta); + float circle_y = radial_ratio * cosf(theta); + float offset_y = 0.25f; + if (phi < 0) { + // Bottom circle: theta increases clockwise, and front is upward + circle_y *= -1.0f; + offset_y += 0.5f; + } + + *x = 0.8f + 0.1f * (1.0f + circle_x); + *y = offset_y + 0.25f * circle_y; + } +} + +static void get_barrel_split_map(float phi, float theta, float expand_coef, float *x, float *y) +{ + float abs_phi = FFABS(phi); + + // Front Face [-PI/2, PI/2] -> [0,1]. + // Back Face [PI/2, PI] and [-PI, -PI/2] -> [1, 2] + float radian_pi_theta = theta / M_PI_F + 0.5f; + int vFace; + + if (radian_pi_theta < 0.0f) + radian_pi_theta += 2.0f; + + // Front face at top (= 0), back face at bottom (= 1). + vFace = radian_pi_theta >= 1.0f; + + if (abs_phi <= M_PI_4_F) { + // Equirect region + *x = 2.0f / 3.0f * (0.5f + (radian_pi_theta - vFace - 0.5f) / expand_coef); + // y increases downwards + *y = 0.25f + 0.5f * vFace - phi / (M_PI_F * expand_coef); + } else { + // Radial ratio on a unit circle = cot(abs_phi) / (expansion_cefficient). + // Using cos(abs_phi)/sin(abs_phi) explicitly to avoid division by zero + float radial_ratio = cosf(abs_phi) / (sinf(abs_phi) * expand_coef); + float circle_x = radial_ratio * sinf(theta); + float circle_y = radial_ratio * cosf(theta); + float offset_y = 0.25f; + + if (vFace == 1) { + // Back Face: Flip + circle_x *= -1.0f; + circle_y = (circle_y >= 0.0f) ? (1 - circle_y) : (-1 - circle_y); + offset_y += 0.5f; + + // Bottom circle: theta increases clockwise + if (phi < 0) + circle_y *= -1.0f; + } else { + // Front Face + // Bottom circle: theta increases clockwise + if (phi < 0) + circle_y *= -1.0f; + } + + *x = 2.0f / 3.0f + 0.5f / 3.0f * (1.0f + circle_x); + *y = offset_y + 0.25f * circle_y / expand_coef; // y direction of expand_coeff (margin) + } +} + +// Returns cube face, and provided face_x & face_y will range from [0, 1] +static int get_cubemap_face_map(float axis_vec_x, float axis_vec_y, float axis_vec_z, float *face_x, float *face_y) +{ + // To check if phi, theta hits the top / bottom faces, we check the hit point of + // the axis vector on planes y = 1 and y = -1, and see if x & z are within [-1, 1] + + // 0.577 < 1 / sqrt(3), which is less than the smallest sin(phi) falling on top/bottom faces + // This angle check will save computation from unnecessarily checking the top/bottom faces + if (FFABS(axis_vec_y) > 0.577f) { + float x_hit = axis_vec_x / FFABS(axis_vec_y); + float z_hit = axis_vec_z / axis_vec_y; + + if (FFABS(x_hit) <= 1.f && FFABS(z_hit) <= 1.f) { + *face_x = x_hit; + // y increases downwards + *face_y = z_hit; + return axis_vec_y > 0 ? TOP : BOTTOM; + } + } + + // Check for left / right faces + if (FFABS(axis_vec_x) > 0.577f) { + float z_hit = -axis_vec_z / axis_vec_x; + float y_hit = axis_vec_y / FFABS(axis_vec_x); + + if (FFABS(z_hit) <= 1.f && FFABS(y_hit) <= 1.f) { + *face_x = z_hit; + // y increases downwards + *face_y = -y_hit; + return axis_vec_x > 0 ? RIGHT : LEFT; + } + } + + // Front / back faces + *face_x = axis_vec_x / axis_vec_z; + // y increases downwards + *face_y = -axis_vec_y / FFABS(axis_vec_z); + + return axis_vec_z > 0 ? FRONT : BACK; +} + +static void get_cubemap32_map(float phi, float theta, float *x, float *y) +{ + // face_projection_map maps each cube face to an index representing the face on the projection + // The indices 0->5 for cubemap 32 goes as: + // [0, 1, 2] as row 1, left to right + // [3, 4, 5] as row 2, left to right + static const int face_projection_map[] = { + [RIGHT] = 0, [LEFT] = 1, [TOP] = 2, + [BOTTOM] = 3, [FRONT] = 4, [BACK] = 5, + }; + + float axis_vec_x = cosf(phi) * sinf(theta); + float axis_vec_y = sinf(phi); + float axis_vec_z = cosf(phi) * cosf(theta); + float face_x = 0, face_y = 0; + int face_index = get_cubemap_face_map(axis_vec_x, axis_vec_y, axis_vec_z, &face_x, &face_y); + + float x_offset = 1.f / 3.f * (face_projection_map[face_index] % 3); + float y_offset = .5f * (face_projection_map[face_index] / 3); + + *x = x_offset + (face_x / DEFAULT_EXPANSION_COEF + 1.f) / 6.f; + *y = y_offset + (face_y / DEFAULT_EXPANSION_COEF + 1.f) / 4.f; +} + +static void get_rotated_cubemap_map(float phi, float theta, float expand_coef, float *x, float *y) +{ + // face_projection_map maps each cube face to an index representing the face on the projection + // The indices 0->5 for rotated cubemap goes as: + // [0, 1] as row 1, left to right + // [2, 3] as row 2, left to right + // [4, 5] as row 3, left to right + static const int face_projection_map[] = { + [LEFT] = 0, [TOP] = 1, + [FRONT] = 2, [BACK] = 3, + [RIGHT] = 4, [BOTTOM] = 5, + }; + + float axis_yaw_vec_x, axis_yaw_vec_y, axis_yaw_vec_z; + float axis_pitch_vec_z, axis_pitch_vec_y; + float x_offset, y_offset; + float face_x = 0, face_y = 0; + int face_index; + + // Unrotate the cube and fix the face map: + // First undo the 45 degree yaw + theta += M_PI_4_F; + + // Now we are looking at the middle of an edge. So convert to axis vector & undo the pitch + axis_yaw_vec_x = cosf(phi) * sinf(theta); + axis_yaw_vec_y = sinf(phi); + axis_yaw_vec_z = cosf(phi) * cosf(theta); + + // The pitch axis is along +x, and has value of -45 degree. So, only y and z components change + axis_pitch_vec_z = (axis_yaw_vec_z - axis_yaw_vec_y) / M_SQRT2_F; + axis_pitch_vec_y = (axis_yaw_vec_y + axis_yaw_vec_z) / M_SQRT2_F; + + face_index = get_cubemap_face_map(axis_yaw_vec_x, axis_pitch_vec_y, axis_pitch_vec_z, &face_x, &face_y); + + // Correct for the orientation of the axes on the faces + if (face_index == LEFT || face_index == FRONT || face_index == RIGHT) { + // x increases downwards & y increases towards left + float upright_y = face_y; + face_y = face_x; + face_x = -upright_y; + } else if (face_index == TOP || face_index == BOTTOM) { + // turn the face upside-down for top and bottom + face_x *= -1.f; + face_y *= -1.f; + } + + x_offset = .5f * (face_projection_map[face_index] & 1); + y_offset = 1.f / 3.f * (face_projection_map[face_index] >> 1); + + *x = x_offset + (face_x / expand_coef + 1.f) / 4.f; + *y = y_offset + (face_y / expand_coef + 1.f) / 6.f; +} + +static void get_projected_map(float phi, float theta, SampleParams *p, BilinearMap *m) +{ + float x = 0, y = 0; + switch(p->projection) { +// TODO: Calculate for CDS + case PROJECTION_CUBEMAP23: + get_rotated_cubemap_map(phi, theta, p->expand_coef, &x, &y); + break; + case PROJECTION_CUBEMAP32: + get_cubemap32_map(phi, theta, &x, &y); + break; + case PROJECTION_BARREL: + get_barrel_map(phi, theta, &x, &y); + break; + case PROJECTION_BARREL_SPLIT: + get_barrel_split_map(phi, theta, p->expand_coef, &x, &y); + break; + // Assume PROJECTION_EQUIRECT as the default + case PROJECTION_EQUIRECT: + default: + get_equirect_map(phi, theta, &x, &y); + break; + } + compute_bilinear_map(p, m, x, y); +} + +static int tape_supports_projection(int projection) +{ + switch(projection) { + case PROJECTION_CUBEMAP23: + case PROJECTION_CUBEMAP32: + case PROJECTION_BARREL: + case PROJECTION_BARREL_SPLIT: + case PROJECTION_EQUIRECT: + return 1; + default: + return 0; + } +} + +static float get_tape_angular_resolution(int projection, float expand_coef, int image_width, int image_height) +{ + // NOTE: The angular resolution of a projected sphere is defined as + // the maximum possible horizontal angle of a pixel on the equator. + // We apply an intentional bias to the horizon as opposed to the meridian, + // since the view direction of most content is rarely closer to the poles + + switch(projection) { +// TODO: Calculate for CDS + case PROJECTION_CUBEMAP23: + // Approximating atanf(pixel_width / (half_edge_width * sqrt2)) = pixel_width / (half_face_width * sqrt2) + return expand_coef / (M_SQRT2_F * image_width / 4.f); + case PROJECTION_CUBEMAP32: + // Approximating atanf(pixel_width / half_face_width) = pixel_width / half_face_width + return DEFAULT_EXPANSION_COEF / (image_width / 6.f); + case PROJECTION_BARREL: + return FFMAX(BARREL_THETA_RANGE / (0.8f * image_width), BARREL_PHI_RANGE / image_height); + case PROJECTION_BARREL_SPLIT: + return FFMAX((expand_coef * M_PI_F) / (2.0f / 3.0f * image_width), + expand_coef * M_PI_2_F / (image_height / 2.0f)); + // Assume PROJECTION_EQUIRECT as the default + case PROJECTION_EQUIRECT: + default: + return FFMAX(2.0f * M_PI_F / image_width, M_PI_F / image_height); + } +} + +static int +generate_eye_tape_map(SSIM360Context *s, + int plane, int eye, + SampleParams *ref_sample_params, + SampleParams *main_sample_params) +{ + int ref_image_width = ref_sample_params->x_image_range + 1; + int ref_image_height = ref_sample_params->y_image_range + 1; + + float angular_resolution = + get_tape_angular_resolution(s->ref_projection, 1.f + s->ref_pad, + ref_image_width, ref_image_height); + + float conversion_factor = M_PI_2_F / (angular_resolution * angular_resolution); + float start_phi = -M_PI_2_F + 4.0f * angular_resolution; + float start_x = conversion_factor * sinf(start_phi); + float end_phi = M_PI_2_F - 3.0f * angular_resolution; + float end_x = conversion_factor * sinf(end_phi); + float x_range = end_x - start_x; + + // Ensure tape length is a multiple of 4, for full SSIM block coverage + int tape_length = s->tape_length[plane] = ((int)ROUNDED_DIV(x_range, 4)) << 2; + + s->ref_tape_map[plane][eye] = av_malloc_array(tape_length * 8, sizeof(BilinearMap)); + s->main_tape_map[plane][eye] = av_malloc_array(tape_length * 8, sizeof(BilinearMap)); + if (!s->ref_tape_map[plane][eye] || !s->main_tape_map[plane][eye]) + return AVERROR(ENOMEM); + + s->angular_resolution[plane][eye] = angular_resolution; + + // For easy memory access, we navigate the tape lengthwise on y + for (int y_index = 0; y_index < tape_length; y_index ++) { + int y_stride = y_index << 3; + + float x = start_x + x_range * (y_index / (tape_length - 1.0f)); + // phi will be in range [-pi/2, pi/2] + float mid_phi = asinf(x / conversion_factor); + + float theta = mid_phi * M_PI_2_F / angular_resolution; + theta = get_radius_between_negative_and_positive_pi(theta); + + for (int x_index = 0; x_index < 8; x_index ++) { + float phi = mid_phi + angular_resolution * (3.0f - x_index); + int tape_index = y_stride + x_index; + get_projected_map(phi, theta, ref_sample_params, &s->ref_tape_map [plane][eye][tape_index]); + get_projected_map(phi, theta, main_sample_params, &s->main_tape_map[plane][eye][tape_index]); + } + } + + return 0; +} + +static int generate_tape_maps(SSIM360Context *s, AVFrame *main, const AVFrame *ref) +{ + // A tape is a long segment with 8 pixels thickness, with the angular center at the middle (below 4th pixel). + // When it takes a full loop around a sphere, it will overlap the starting point at half the width from above. + int ref_stereo_format = s->ref_stereo_format; + int main_stereo_format = s->main_stereo_format; + int are_both_stereo = (main_stereo_format != STEREO_FORMAT_MONO) && (ref_stereo_format != STEREO_FORMAT_MONO); + int min_eye_count = 1 + are_both_stereo; + int ret; + + for (int i = 0; i < s->nb_components; i ++) { + int ref_width = s->ref_planewidth[i]; + int ref_height = s->ref_planeheight[i]; + int main_width = s->main_planewidth[i]; + int main_height = s->main_planeheight[i]; + + int is_ref_LR = (ref_stereo_format == STEREO_FORMAT_LR); + int is_ref_TB = (ref_stereo_format == STEREO_FORMAT_TB); + int is_main_LR = (main_stereo_format == STEREO_FORMAT_LR); + int is_main_TB = (main_stereo_format == STEREO_FORMAT_TB); + + int ref_image_width = is_ref_LR ? ref_width >> 1 : ref_width; + int ref_image_height = is_ref_TB ? ref_height >> 1 : ref_height; + int main_image_width = is_main_LR ? main_width >> 1 : main_width; + int main_image_height = is_main_TB ? main_height >> 1 : main_height; + + for (int eye = 0; eye < min_eye_count; eye ++) { + SampleParams ref_sample_params = { + .stride = ref->linesize[i], + .planewidth = ref_width, + .planeheight = ref_height, + .x_image_range = ref_image_width - 1, + .y_image_range = ref_image_height - 1, + .x_image_offset = is_ref_LR * eye * ref_image_width, + .y_image_offset = is_ref_TB * eye * ref_image_height, + .projection = s->ref_projection, + .expand_coef = 1.f + s->ref_pad, + }; + + SampleParams main_sample_params = { + .stride = main->linesize[i], + .planewidth = main_width, + .planeheight = main_height, + .x_image_range = main_image_width - 1, + .y_image_range = main_image_height - 1, + .x_image_offset = is_main_LR * eye * main_image_width, + .y_image_offset = is_main_TB * eye * main_image_height, + .projection = s->main_projection, + .expand_coef = 1.f + s->main_pad, + }; + + ret = generate_eye_tape_map(s, i, eye, &ref_sample_params, &main_sample_params); + if (ret < 0) + return ret; + } + } + + return 0; +} + +static int do_ssim360(FFFrameSync *fs) +{ + AVFilterContext *ctx = fs->parent; + SSIM360Context *s = ctx->priv; + AVFrame *master, *ref; + AVDictionary **metadata; + double c[4], ssim360v = 0.0, ssim360p50 = 0.0; + int i, ret; + int need_frame_skip = s->nb_net_frames % (s->frame_skip_ratio + 1); + HeatmapList* h_ptr = NULL; + + ret = ff_framesync_dualinput_get(fs, &master, &ref); + if (ret < 0) + return ret; + + s->nb_net_frames++; + + if (need_frame_skip) + return ff_filter_frame(ctx->outputs[0], master); + + metadata = &master->metadata; + + if (s->use_tape && !s->tape_length[0]) { + ret = generate_tape_maps(s, master, ref); + if (ret < 0) + return ret; + } + + for (i = 0; i < s->nb_components; i++) { + if (s->use_tape) { + c[i] = ssim360_tape(master->data[i], s->main_tape_map[i][0], + ref->data[i], s->ref_tape_map [i][0], + s->tape_length[i], s->max, s->temp, + s->ssim360_hist[i], &s->ssim360_hist_net[i], + s->angular_resolution[i][0], s->heatmaps); + + if (s->ref_tape_map[i][1]) { + c[i] += ssim360_tape(master->data[i], s->main_tape_map[i][1], + ref->data[i], s->ref_tape_map[i][1], + s->tape_length[i], s->max, s->temp, + s->ssim360_hist[i], &s->ssim360_hist_net[i], + s->angular_resolution[i][1], s->heatmaps); + c[i] /= 2.f; + } + } else { + c[i] = s->ssim360_plane(master->data[i], master->linesize[i], + ref->data[i], ref->linesize[i], + s->ref_planewidth[i], s->ref_planeheight[i], + s->temp, s->max, s->density); + } + + s->ssim360[i] += c[i]; + ssim360v += s->coefs[i] * c[i]; + } + + s->nb_ssim_frames++; + if (s->heatmaps) { + map_uninit(&s->heatmaps->map); + h_ptr = s->heatmaps; + s->heatmaps = s->heatmaps->next; + av_freep(&h_ptr); + } + s->ssim360_total += ssim360v; + + // Record percentiles from histogram and attach metadata when using tape + if (s->use_tape) { + int i, p, hist_indices[4]; + double hist_weight[4]; + + for (i = 0; i < s->nb_components; i++) { + hist_indices[i] = SSIM360_HIST_SIZE - 1; + hist_weight[i] = 0; + } + + for (p = 0; PERCENTILE_LIST[p] >= 0.0; p ++) { + for (i = 0; i < s->nb_components; i++) { + double target_weight, ssim360p; + + // Target weight = total number of samples above the specified percentile + target_weight = (1. - PERCENTILE_LIST[p]) * s->ssim360_hist_net[i]; + target_weight = FFMAX(target_weight, 1); + while(hist_indices[i] >= 0 && hist_weight[i] < target_weight) { + hist_weight[i] += s->ssim360_hist[i][hist_indices[i]]; + hist_indices[i] --; + } + + ssim360p = (double)(hist_indices[i] + 1) / (double)(SSIM360_HIST_SIZE - 1); + if (PERCENTILE_LIST[p] == 0.5) + ssim360p50 += s->coefs[i] * ssim360p; + s->ssim360_percentile_sum[i][p] += ssim360p; + } + } + + for (i = 0; i < s->nb_components; i++) { + memset(s->ssim360_hist[i], 0, SSIM360_HIST_SIZE * sizeof(double)); + s->ssim360_hist_net[i] = 0; + } + + for (i = 0; i < s->nb_components; i++) { + int cidx = s->is_rgb ? s->rgba_map[i] : i; + set_meta(metadata, "lavfi.ssim360.", s->comps[i], c[cidx]); + } + + // Use p50 as the aggregated value + set_meta(metadata, "lavfi.ssim360.All", 0, ssim360p50); + set_meta(metadata, "lavfi.ssim360.dB", 0, ssim360_db(ssim360p50, 1.0)); + + if (s->stats_file) { + fprintf(s->stats_file, "n:%"PRId64" ", s->nb_ssim_frames); + + for (i = 0; i < s->nb_components; i++) { + int cidx = s->is_rgb ? s->rgba_map[i] : i; + fprintf(s->stats_file, "%c:%f ", s->comps[i], c[cidx]); + } + + fprintf(s->stats_file, "All:%f (%f)\n", ssim360p50, ssim360_db(ssim360p50, 1.0)); + } + } + + return ff_filter_frame(ctx->outputs[0], master); +} + +static av_cold int init(AVFilterContext *ctx) +{ + SSIM360Context *s = ctx->priv; + + if (s->stats_file_str) { + if (!strcmp(s->stats_file_str, "-")) { + s->stats_file = stdout; + } else { + s->stats_file = fopen(s->stats_file_str, "w"); + if (!s->stats_file) { + int err = AVERROR(errno); + char buf[128]; + av_strerror(err, buf, sizeof(buf)); + av_log(ctx, AV_LOG_ERROR, "Could not open stats file %s: %s\n", + s->stats_file_str, buf); + return err; + } + } + } + + s->fs.on_event = do_ssim360; + return 0; +} + +static int query_formats(AVFilterContext *ctx) +{ + static const enum AVPixelFormat pix_fmts[] = { + AV_PIX_FMT_GRAY8, + AV_PIX_FMT_YUV420P, AV_PIX_FMT_YUV422P, AV_PIX_FMT_YUV444P, + AV_PIX_FMT_YUV440P, AV_PIX_FMT_YUV411P, AV_PIX_FMT_YUV410P, + AV_PIX_FMT_YUVJ411P, AV_PIX_FMT_YUVJ420P, AV_PIX_FMT_YUVJ422P, + AV_PIX_FMT_YUVJ440P, AV_PIX_FMT_YUVJ444P, + AV_PIX_FMT_GBRP, +#define PF(suf) AV_PIX_FMT_YUV420##suf, AV_PIX_FMT_YUV422##suf, AV_PIX_FMT_YUV444##suf, AV_PIX_FMT_GBR##suf + PF(P9), PF(P10), PF(P12), PF(P14), PF(P16), + AV_PIX_FMT_NONE + }; + + AVFilterFormats *fmts_list = ff_make_format_list(pix_fmts); + if (!fmts_list) + return AVERROR(ENOMEM); + return ff_set_common_formats(ctx, fmts_list); +} + +static int config_input_main(AVFilterLink *inlink) +{ + const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(inlink->format); + AVFilterContext *ctx = inlink->dst; + SSIM360Context *s = ctx->priv; + + s->main_planeheight[0] = inlink->h; + s->main_planeheight[3] = inlink->h; + s->main_planeheight[1] = AV_CEIL_RSHIFT(inlink->h, desc->log2_chroma_h); + s->main_planeheight[2] = AV_CEIL_RSHIFT(inlink->h, desc->log2_chroma_h); + + s->main_planewidth[0] = inlink->w; + s->main_planewidth[3] = inlink->w; + s->main_planewidth[1] = AV_CEIL_RSHIFT(inlink->w, desc->log2_chroma_w); + s->main_planewidth[2] = AV_CEIL_RSHIFT(inlink->w, desc->log2_chroma_w); + + // If main projection is unindentified, assume it is same as reference + if (s->main_projection == PROJECTION_N) + s->main_projection = s->ref_projection; + + // If main stereo format is unindentified, assume it is same as reference + if (s->main_stereo_format == STEREO_FORMAT_N) + s->main_stereo_format = s->ref_stereo_format; + + return 0; +} + +static void generate_density_map(SSIM360Context *s, int w, int h) +{ + double d, r_square, cos_square; + int ow, oh; + + map_init(&s->density, w, h); + switch (s->ref_stereo_format) { + case STEREO_FORMAT_TB: + h >>= 1; + break; + case STEREO_FORMAT_LR: + w >>= 1; + break; + } + + switch (s->ref_projection) { + case PROJECTION_EQUIRECT: + for (int i = 0; i < h; i++) { + d = cos(((0.5 + i) / h - 0.5) * M_PI); + for (int j = 0; j < w; j++) + s->density.value[i][j] = d; + } + break; + case PROJECTION_CUBEMAP32: + // for one quater of a face + for (int i = 0; i < h / 4; i++) { + for (int j = 0; j < w / 6; j++) { + // r = normalized distance to the face center + r_square = + (0.5 + i) / (h / 2) * (0.5 + i) / (h / 2) + + (0.5 + j) / (w / 3) * (0.5 + j) / (w / 3); + r_square /= DEFAULT_EXPANSION_COEF * DEFAULT_EXPANSION_COEF; + cos_square = 0.25 / (r_square + 0.25); + d = pow(cos_square, 1.5); + + for (int face = 0; face < 6; face++) { + // center of a face + switch (face) { + case 0: + oh = h / 4; + ow = w / 6; + break; + case 1: + oh = h / 4; + ow = w / 6 + w / 3; + break; + case 2: + oh = h / 4; + ow = w / 6 + 2 * w / 3; + break; + case 3: + oh = h / 4 + h / 2; + ow = w / 6; + break; + case 4: + oh = h / 4 + h / 2; + ow = w / 6 + w / 3; + break; + case 5: + oh = h / 4 + h / 2; + ow = w / 6 + 2 * w / 3; + break; + } + s->density.value[oh - 1 - i][ow - 1 - j] = d; + s->density.value[oh - 1 - i][ow + j] = d; + s->density.value[oh + i] [ow - 1 - j] = d; + s->density.value[oh + i] [ow + j] = d; + } + } + } + break; + case PROJECTION_CUBEMAP23: + // for one quater of a face + for (int i = 0; i < h / 6; i++) { + for (int j = 0; j < w / 4; j++) { + // r = normalized distance to the face center + r_square = + (0.5 + i) / (h / 3) * (0.5 + i) / (h / 3) + + (0.5 + j) / (w / 2) * (0.5 + j) / (w / 2); + r_square /= (1.f + s->ref_pad) * (1.f + s->ref_pad); + cos_square = 0.25 / (r_square + 0.25); + d = pow(cos_square, 1.5); + + for (int face = 0; face < 6; face++) { + // center of a face + switch (face) { + case 0: + ow = w / 4; + oh = h / 6; + break; + case 1: + ow = w / 4; + oh = h / 6 + h / 3; + break; + case 2: + ow = w / 4; + oh = h / 6 + 2 * h / 3; + break; + case 3: + ow = w / 4 + w / 2; + oh = h / 6; + break; + case 4: + ow = w / 4 + w / 2; + oh = h / 6 + h / 3; + break; + case 5: + ow = w / 4 + w / 2; + oh = h / 6 + 2 * h / 3; + break; + } + s->density.value[oh - 1 - i][ow - 1 - j] = d; + s->density.value[oh - 1 - i][ow + j] = d; + s->density.value[oh + i] [ow - 1 - j] = d; + s->density.value[oh + i] [ow + j] = d; + } + } + } + break; + case PROJECTION_BARREL: + // side face + for (int i = 0; i < h; i++) { + for (int j = 0; j < w * 4 / 5; j++) { + d = cos(((0.5 + i) / h - 0.5) * DEFAULT_EXPANSION_COEF * M_PI_2); + s->density.value[i][j] = d * d * d; + } + } + // top and bottom + for (int i = 0; i < h; i++) { + for (int j = w * 4 / 5; j < w; j++) { + double dx = DEFAULT_EXPANSION_COEF * (0.5 + j - w * 0.90) / (w * 0.10); + double dx_squared = dx * dx; + + double top_dy = DEFAULT_EXPANSION_COEF * (0.5 + i - h * 0.25) / (h * 0.25); + double top_dy_squared = top_dy * top_dy; + + double bottom_dy = DEFAULT_EXPANSION_COEF * (0.5 + i - h * 0.75) / (h * 0.25); + double bottom_dy_squared = bottom_dy * bottom_dy; + + // normalized distance to the circle center + r_square = (i < h / 2 ? top_dy_squared : bottom_dy_squared) + dx_squared; + if (r_square > 1.0) + continue; + + cos_square = 1.0 / (r_square + 1.0); + d = pow(cos_square, 1.5); + s->density.value[i][j] = d; + } + } + break; + default: + // TODO: SSIM360_v1 + for (int i = 0; i < h; i++) { + for (int j = 0; j < w; j++) + s->density.value[i][j] = 0; + } + } + + switch (s->ref_stereo_format) { + case STEREO_FORMAT_TB: + for (int i = 0; i < h; i++) { + for (int j = 0; j < w; j++) + s->density.value[i + h][j] = s->density.value[i][j]; + } + break; + case STEREO_FORMAT_LR: + for (int i = 0; i < h; i++) { + for (int j = 0; j < w; j++) + s->density.value[i][j + w] = s->density.value[i][j]; + } + } +} + +static int config_input_ref(AVFilterLink *inlink) +{ + const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(inlink->format); + AVFilterContext *ctx = inlink->dst; + SSIM360Context *s = ctx->priv; + int sum = 0; + + s->nb_components = desc->nb_components; + + s->ref_planeheight[0] = inlink->h; + s->ref_planeheight[3] = inlink->h; + s->ref_planeheight[1] = AV_CEIL_RSHIFT(inlink->h, desc->log2_chroma_h); + s->ref_planeheight[2] = AV_CEIL_RSHIFT(inlink->h, desc->log2_chroma_h); + + s->ref_planewidth[0] = inlink->w; + s->ref_planewidth[3] = inlink->w; + s->ref_planewidth[1] = AV_CEIL_RSHIFT(inlink->w, desc->log2_chroma_w); + s->ref_planewidth[2] = AV_CEIL_RSHIFT(inlink->w, desc->log2_chroma_w); + + s->is_rgb = ff_fill_rgba_map(s->rgba_map, inlink->format) >= 0; + s->comps[0] = s->is_rgb ? 'R' : 'Y'; + s->comps[1] = s->is_rgb ? 'G' : 'U'; + s->comps[2] = s->is_rgb ? 'B' : 'V'; + s->comps[3] = 'A'; + + // If chroma computation is disabled, and the format is YUV, skip U & V channels + if (!s->is_rgb && !s->compute_chroma) + s->nb_components = 1; + + s->max = (1 << desc->comp[0].depth) - 1; + + s->ssim360_plane = desc->comp[0].depth > 8 ? ssim360_plane_16bit : ssim360_plane_8bit; + + for (int i = 0; i < s->nb_components; i++) + sum += s->ref_planeheight[i] * s->ref_planewidth[i]; + for (int i = 0; i < s->nb_components; i++) + s->coefs[i] = (double) s->ref_planeheight[i] * s->ref_planewidth[i] / sum; + + return 0; +} + +static int config_output(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + SSIM360Context *s = ctx->priv; + AVFilterLink *mainlink = ctx->inputs[0]; + AVFilterLink *reflink = ctx->inputs[0]; + const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(outlink->format); + int ret; + + // Use tape algorithm if any of frame sizes, projections or stereo format are not equal + if (ctx->inputs[0]->w != ctx->inputs[1]->w || ctx->inputs[0]->h != ctx->inputs[1]->h || + s->ref_projection != s->main_projection || s->ref_stereo_format != s->main_stereo_format) + s->use_tape = 1; + + // Finally, if we have decided to / forced to use tape, check if tape supports both input and output projection + if (s->use_tape && + !(tape_supports_projection(s->main_projection) && + tape_supports_projection(s->ref_projection))) { + av_log(ctx, AV_LOG_ERROR, "Projection is unsupported for the tape based algorithm\n"); + return AVERROR(EINVAL); + } + + if (s->use_tape) { + // s->temp will be allocated for the tape width = 8. The tape is long downwards + s->temp = av_malloc_array((2 * 8 + 12), sizeof(*s->temp)); + if (!s->temp) + return AVERROR(ENOMEM); + + memset(s->ssim360_percentile_sum, 0, sizeof(s->ssim360_percentile_sum)); + + for (int i = 0; i < s->nb_components; i++) { + s->ssim360_hist[i] = av_mallocz_array(SSIM360_HIST_SIZE, sizeof(*s->ssim360_hist)); + if (!s->ssim360_hist[i]) + return AVERROR(ENOMEM); + } + } else { + s->temp = av_malloc_array((2 * reflink->w + 12), sizeof(*s->temp) * (1 + (desc->comp[0].depth > 8))); + if (!s->temp) + return AVERROR(ENOMEM); + + if (s->density.value == NULL) + generate_density_map(s, reflink->w, reflink->h); + } + + ret = ff_framesync_init_dualinput(&s->fs, ctx); + if (ret < 0) + return ret; + + outlink->w = mainlink->w; + outlink->h = mainlink->h; + outlink->time_base = mainlink->time_base; + outlink->sample_aspect_ratio = mainlink->sample_aspect_ratio; + outlink->frame_rate = mainlink->frame_rate; + + s->fs.opt_shortest = 1; + s->fs.opt_repeatlast = 1; + + ret = ff_framesync_configure(&s->fs); + if (ret < 0) + return ret; + + return 0; +} + +static int load_heatmaps(HeatmapList **proot, FILE *heatmap_file, int w, int h) +{ + HeatmapList *root = NULL; + HeatmapList **next = &root; + char line[8192]; + int i, ret; + + fgets(line, sizeof(line), heatmap_file); // video id line + + while (fgets(line, sizeof(line), heatmap_file) != NULL) { + HeatmapList *cur; + char *saveptr, *val; + + av_strtok(line, ",", &saveptr); // first value is frame id + + ret = map_alloc(next, w, h); + if (ret < 0) + goto fail; + + cur = *next; + next = &cur->next; + + i = 0; + while ((val = av_strtok(NULL, ",", &saveptr))) { + int y = i / w; + int x = i % w; + + if (y >= h) { + ret = AVERROR(EINVAL); + goto fail; + } + + cur->map.value[y][x] = atof(val); + i++; + } + } + + *proot = root; + + return 0; +fail: + map_list_free(&root); + return ret; +} + +static int activate(AVFilterContext *ctx) +{ + SSIM360Context *s = ctx->priv; + + if (s->use_tape && s->heatmap_path && !s->heatmaps) { + FILE *heatmap_file; + int ret; + + + heatmap_file = av_fopen_utf8(s->heatmap_path, "r"); + if (!heatmap_file) { + av_log(ctx, AV_LOG_ERROR, "cannot open heatmap file %s\n", s->heatmap_path); + return AVERROR(EINVAL); + } + ret = load_heatmaps(&s->heatmaps, heatmap_file, + s->default_heatmap_w, s->default_heatmap_h); + if (ret < 0) + return ret; + } + + return ff_framesync_activate(&s->fs); +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + SSIM360Context *s = ctx->priv; + + if (s->nb_ssim_frames > 0) { + char buf[256]; + buf[0] = 0; + // Log average SSIM360 values + for (int i = 0; i < s->nb_components; i++) { + int c = s->is_rgb ? s->rgba_map[i] : i; + av_strlcatf(buf, sizeof(buf), " %c:%f (%f)", s->comps[i], s->ssim360[c] / s->nb_ssim_frames, + ssim360_db(s->ssim360[c], s->nb_ssim_frames)); + } + av_log(ctx, AV_LOG_INFO, "SSIM360%s All:%f (%f)\n", buf, + s->ssim360_total / s->nb_ssim_frames, ssim360_db(s->ssim360_total, s->nb_ssim_frames)); + + // Log percentiles from histogram when using tape + if (s->use_tape) { + for (int p = 0; PERCENTILE_LIST[p] >= 0.0; p++) { + buf[0] = 0; + for (int i = 0; i < s->nb_components; i++) { + int c = s->is_rgb ? s->rgba_map[i] : i; + double ssim360p = s->ssim360_percentile_sum[i][p] / (double)(s->nb_ssim_frames); + av_strlcatf(buf, sizeof(buf), " %c:%f (%f)", s->comps[c], ssim360p, ssim360_db(ssim360p, 1)); + } + av_log(ctx, AV_LOG_INFO, "SSIM360_p%d%s\n", (int)(PERCENTILE_LIST[p] * 100.), buf); + } + } + } + + // free density map + map_uninit(&s->density); + + map_list_free(&s->heatmaps); + + for (int i = 0; i < s->nb_components; i++) { + for (int eye = 0; eye < 2; eye++) { + av_freep(&s->ref_tape_map[i][eye]); + av_freep(&s->main_tape_map[i][eye]); + } + av_freep(&s->ssim360_hist[i]); + } + + ff_framesync_uninit(&s->fs); + + if (s->stats_file && s->stats_file != stdout) + fclose(s->stats_file); + + av_freep(&s->temp); +} + +static const AVFilterPad ssim360_inputs[] = { + { + .name = "main", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_input_main, + }, + { + .name = "reference", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_input_ref, + }, + { NULL } +}; + +static const AVFilterPad ssim360_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_output, + }, + { NULL } +}; + +AVFilter ff_vf_ssim360 = { + .name = "ssim360", + .description = NULL_IF_CONFIG_SMALL("Calculate the SSIM between two 360 video streams."), + .preinit = ssim360_framesync_preinit, + .init = init, + .uninit = uninit, + .query_formats = query_formats, + .activate = activate, + .priv_size = sizeof(SSIM360Context), + .priv_class = &ssim360_class, + .inputs = ssim360_inputs, + .outputs = ssim360_outputs, +}; From patchwork Mon Aug 9 10:29:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 29377 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:6c0f:0:0:0:0:0 with SMTP id a15csp2736204ioh; Mon, 9 Aug 2021 03:29:41 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx+qS2+NgdPMEDu7PgybBFU0HkRf8D3Meg5B0MvESt4gjYVn/gjpFnacQujECR+39y/Cebu X-Received: by 2002:a17:907:1c08:: with SMTP id nc8mr22484678ejc.514.1628504981200; Mon, 09 Aug 2021 03:29:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1628504981; cv=none; d=google.com; s=arc-20160816; b=o5ljy+J3ZeHJqCwgKB7w3vht+EZwrUtBI9b8xkNpdb+cSei8U8nv48E/RJxpLgcM+i SZ8FFqYyFI7YcSVLwuLSd22wGcI5B7okJpstv3B1NyQPPo5shaxDKJmXDV2g0RMcrxI1 u+8BEM308M8Pg4oZO1CuNzhCNyC23zQVMZ0wf/6Hpm+oyP9LP25Ty0e9XPt8guPl0SOF 6f30j6wvcBds5cjrtUEanC8jUmbTABQM4hNEEbAHv6ilRJ6rRzp/iMgrTgqsQ8SnUeh1 m31KVE3LBlfws8eLyYa63evJc9XOj9iA4MsdwOwCZhTh67ot35tbEVesL0V/mXO0aSIp tNKg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:delivered-to; bh=4zf3cgYGGtpXlbWXuVMMs1jeaZQaCd9IgGwrQacoIwA=; b=MxU3TrC1C1iQnxPPNLKaRyKIZRXdpA4InNP0c6o05o4546PdqkHG4HejxYw9s+JcJU lmgG4UFRs2kDA7oAlbCvH5nbwx3/u7RvuIUawlX0nIzmkcwJx/2giEPfEUJscC0eNz/u bsqrBVyxEUiFXbGqs9v+a+EjnxtOW4kzz6+caoKiCBP0A0DipJ7rCmniWKRt4Fdq/FG4 2sosV/CpUBQ0qP06w35PbMxRwtmSGmmW5IUVr5LX3siiwOgaSAw8hsJYWxsQ6yh2+afy mTYW8qUVQsLGj6UKK519yRT8gmtur8Emp8UOXRo1RwJdbty1wKzX/8HLDMsH/FzjgVS3 i1kw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id r7si3129792ejz.374.2021.08.09.03.29.40; Mon, 09 Aug 2021 03:29:41 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8BDB668A0F1; Mon, 9 Aug 2021 13:29:36 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail0.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3E9D16897E8 for ; Mon, 9 Aug 2021 13:29:29 +0300 (EEST) Received: from localhost (localhost [IPv6:::1]) by mail0.khirnov.net (Postfix) with ESMTP id ED181240692 for ; Mon, 9 Aug 2021 12:29:28 +0200 (CEST) Received: from mail0.khirnov.net ([IPv6:::1]) by localhost (mail0.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id GimjAADnzJSL for ; Mon, 9 Aug 2021 12:29:27 +0200 (CEST) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail0.khirnov.net (Postfix) with ESMTPS id B99EB2404C8 for ; Mon, 9 Aug 2021 12:29:27 +0200 (CEST) Received: by libav.khirnov.net (Postfix, from userid 1000) id 88F853A0F56; Mon, 9 Aug 2021 12:29:27 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Mon, 9 Aug 2021 12:29:19 +0200 Message-Id: <20210809102919.387-2-anton@khirnov.net> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210809102919.387-1-anton@khirnov.net> References: <20210809102919.387-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/2] FATE: add tests for v360/ssim360 filters X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: pwmvLhU08A3r --- libavfilter/Makefile | 3 +- tests/Makefile | 4 +- tests/fate-run.sh | 6 + tests/fate/filter-video.mak | 14 ++ tests/ref/fate/filter-spherical-barrel | 192 +++++++++++++++++++++++++ tests/ref/fate/filter-spherical-c3x2 | 192 +++++++++++++++++++++++++ tools/Makefile | 1 + tools/spherical_compare.c | 148 +++++++++++++++++++ 8 files changed, 558 insertions(+), 2 deletions(-) create mode 100644 tests/ref/fate/filter-spherical-barrel create mode 100644 tests/ref/fate/filter-spherical-c3x2 create mode 100644 tools/spherical_compare.c diff --git a/libavfilter/Makefile b/libavfilter/Makefile index b0348ccfa3..27dd0c4b47 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -559,7 +559,8 @@ SKIPHEADERS-$(CONFIG_VULKAN) += vulkan.h OBJS-$(CONFIG_LIBGLSLANG) += glslang.o -TOOLS = graph2dot +TOOLS = graph2dot \ + spherical_compare TESTPROGS = drawutils filtfmts formats integral TOOLS-$(CONFIG_LIBZMQ) += zmqsend diff --git a/tests/Makefile b/tests/Makefile index e42e66d81b..aba31b59d0 100644 --- a/tests/Makefile +++ b/tests/Makefile @@ -208,7 +208,8 @@ FATE_EXTERN-$(CONFIG_FFMPEG) += $(FATE_SAMPLES_AVCONV) $(FATE_SAMPLES_FFMPEG) FATE_EXTERN-$(CONFIG_FFPROBE) += $(FATE_SAMPLES_FFPROBE) FATE_SAMPLES_FFMPEG_FFPROBE += $(FATE_SAMPLES_FFMPEG_FFPROBE-yes) FATE_EXTERN-$(call ALLYES, FFMPEG FFPROBE) += $(FATE_SAMPLES_FFMPEG_FFPROBE) -FATE_EXTERN += $(FATE_EXTERN-yes) $(FATE_SAMPLES_FASTSTART) +FATE_EXTERN += $(FATE_EXTERN-yes) $(FATE_SAMPLES_FASTSTART) \ + $(FATE_SAMPLES_SPHERICAL_COMPARE) FATE += $(FATE-yes) @@ -222,6 +223,7 @@ $(FATE_FFPROBE) $(FATE_FFMPEG_FFPROBE) $(FATE_SAMPLES_FFPROBE) $(FATE_SAMPLES_FF $(FATE_SAMPLES_FASTSTART): tools/qt-faststart$(EXESUF) $(FATE_SAMPLES_DUMP_DATA): tools/venc_data_dump$(EXESUF) $(FATE_SAMPLES_SCALE_SLICE): tools/scale_slice_test$(EXESUF) +$(FATE_SAMPLES_SPHERICAL_COMPARE): tools/spherical_compare$(EXESUF) ifdef SAMPLES FATE += $(FATE_EXTERN) diff --git a/tests/fate-run.sh b/tests/fate-run.sh index cd16f5fcff..afdbb4d2ec 100755 --- a/tests/fate-run.sh +++ b/tests/fate-run.sh @@ -507,6 +507,12 @@ venc_data(){ run tools/venc_data_dump${EXECSUF} ${file} ${stream} ${frames} ${threads} ${thread_type} } +spherical_compare(){ + file=$1 + filterchain=$2 + run tools/spherical_compare${EXECSUF} ${file} ${filterchain} +} + null(){ : } diff --git a/tests/fate/filter-video.mak b/tests/fate/filter-video.mak index 16f24a8422..a28c3f7aec 100644 --- a/tests/fate/filter-video.mak +++ b/tests/fate/filter-video.mak @@ -845,6 +845,20 @@ FATE_FILTER_SAMPLES-$(call ALLYES, MOV_DEMUXER H264_DECODER AAC_FIXED_DECODER PC fate-filter-meta-4560-rotate0: tests/data/file4560-override2rotate0.mov fate-filter-meta-4560-rotate0: CMD = framecrc -auto_conversion_filters -flags +bitexact -c:a aac_fixed -i $(TARGET_PATH)/tests/data/file4560-override2rotate0.mov +SPHERICAL_DEPS = MATROSKA_DEMUXER VP8_DECODER V360_FILTER SSIM360_FILTER +SPHERICAL_SAMPLE = $(TARGET_SAMPLES)/spherical/Worlds_First_Live_360_Rocket_Launch-_Orbital_ATK_CRS-7_cut.mkv + +FATE_FILTER_SPHERICAL += fate-filter-spherical-c3x2 +fate-filter-spherical-c3x2: CMD = spherical_compare $(SPHERICAL_SAMPLE) \ + "split[in][ref];[in]v360=input=e:output=c3x2[main];[main][ref]ssim360=main_projection=c3x2:ref_projection=e" + +FATE_FILTER_SPHERICAL += fate-filter-spherical-barrel +fate-filter-spherical-barrel: CMD = spherical_compare $(SPHERICAL_SAMPLE) \ + "split[in][ref];[in]v360=input=e:output=barrel[main];[main][ref]ssim360=main_projection=barrel:ref_projection=e" + +FATE_SAMPLES_SPHERICAL_COMPARE-$(call ALLYES, $(SPHERICAL_DEPS)) += $(FATE_FILTER_SPHERICAL) +FATE_SAMPLES_SPHERICAL_COMPARE += $(FATE_SAMPLES_SPHERICAL_COMPARE-yes) + REFCMP_DEPS = FFMPEG LAVFI_INDEV TESTSRC2_FILTER AVGBLUR_FILTER METADATA_FILTER FATE_FILTER-$(call ALLYES, $(REFCMP_DEPS) PSNR_FILTER) += fate-filter-refcmp-psnr-rgb diff --git a/tests/ref/fate/filter-spherical-barrel b/tests/ref/fate/filter-spherical-barrel new file mode 100644 index 0000000000..a2ff075a26 --- /dev/null +++ b/tests/ref/fate/filter-spherical-barrel @@ -0,0 +1,192 @@ +frame 0 +lavfi.ssim360.Y=0.97 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=1.00 +lavfi.ssim360.dB=25.19 +frame 1 +lavfi.ssim360.Y=0.97 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=1.00 +lavfi.ssim360.dB=25.04 +frame 2 +lavfi.ssim360.Y=0.97 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=1.00 +lavfi.ssim360.dB=24.95 +frame 3 +lavfi.ssim360.Y=0.97 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=1.00 +lavfi.ssim360.dB=24.88 +frame 4 +lavfi.ssim360.Y=0.97 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=1.00 +lavfi.ssim360.dB=24.80 +frame 5 +lavfi.ssim360.Y=0.97 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=1.00 +lavfi.ssim360.dB=24.75 +frame 6 +lavfi.ssim360.Y=0.97 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=1.00 +lavfi.ssim360.dB=24.70 +frame 7 +lavfi.ssim360.Y=0.97 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=1.00 +lavfi.ssim360.dB=24.63 +frame 8 +lavfi.ssim360.Y=0.97 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=1.00 +lavfi.ssim360.dB=24.58 +frame 9 +lavfi.ssim360.Y=0.97 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=1.00 +lavfi.ssim360.dB=24.54 +frame 10 +lavfi.ssim360.Y=0.97 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=1.00 +lavfi.ssim360.dB=24.51 +frame 11 +lavfi.ssim360.Y=0.97 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=1.00 +lavfi.ssim360.dB=24.49 +frame 12 +lavfi.ssim360.Y=0.97 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=1.00 +lavfi.ssim360.dB=24.46 +frame 13 +lavfi.ssim360.Y=0.97 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=1.00 +lavfi.ssim360.dB=24.42 +frame 14 +lavfi.ssim360.Y=0.97 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=1.00 +lavfi.ssim360.dB=24.41 +frame 15 +lavfi.ssim360.Y=0.96 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=1.00 +lavfi.ssim360.dB=24.39 +frame 16 +lavfi.ssim360.Y=0.97 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=1.00 +lavfi.ssim360.dB=24.33 +frame 17 +lavfi.ssim360.Y=0.96 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=1.00 +lavfi.ssim360.dB=24.30 +frame 18 +lavfi.ssim360.Y=0.96 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=1.00 +lavfi.ssim360.dB=24.27 +frame 19 +lavfi.ssim360.Y=0.96 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=1.00 +lavfi.ssim360.dB=24.25 +frame 20 +lavfi.ssim360.Y=0.96 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=1.00 +lavfi.ssim360.dB=24.22 +frame 21 +lavfi.ssim360.Y=0.96 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=1.00 +lavfi.ssim360.dB=24.19 +frame 22 +lavfi.ssim360.Y=0.96 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=1.00 +lavfi.ssim360.dB=24.16 +frame 23 +lavfi.ssim360.Y=0.96 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=1.00 +lavfi.ssim360.dB=24.14 +frame 24 +lavfi.ssim360.Y=0.96 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=1.00 +lavfi.ssim360.dB=24.08 +frame 25 +lavfi.ssim360.Y=0.96 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=1.00 +lavfi.ssim360.dB=24.02 +frame 26 +lavfi.ssim360.Y=0.96 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=1.00 +lavfi.ssim360.dB=24.01 +frame 27 +lavfi.ssim360.Y=0.96 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=1.00 +lavfi.ssim360.dB=23.98 +frame 28 +lavfi.ssim360.Y=0.96 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=1.00 +lavfi.ssim360.dB=23.96 +frame 29 +lavfi.ssim360.Y=0.96 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=1.00 +lavfi.ssim360.dB=23.95 +frame 30 +lavfi.ssim360.Y=0.96 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=1.00 +lavfi.ssim360.dB=23.93 +frame 31 +lavfi.ssim360.Y=0.96 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=1.00 +lavfi.ssim360.dB=23.91 diff --git a/tests/ref/fate/filter-spherical-c3x2 b/tests/ref/fate/filter-spherical-c3x2 new file mode 100644 index 0000000000..7c3b269f40 --- /dev/null +++ b/tests/ref/fate/filter-spherical-c3x2 @@ -0,0 +1,192 @@ +frame 0 +lavfi.ssim360.Y=0.84 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=0.99 +lavfi.ssim360.dB=18.61 +frame 1 +lavfi.ssim360.Y=0.84 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=0.98 +lavfi.ssim360.dB=18.21 +frame 2 +lavfi.ssim360.Y=0.84 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=0.98 +lavfi.ssim360.dB=18.03 +frame 3 +lavfi.ssim360.Y=0.83 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=0.98 +lavfi.ssim360.dB=17.90 +frame 4 +lavfi.ssim360.Y=0.83 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=0.98 +lavfi.ssim360.dB=17.72 +frame 5 +lavfi.ssim360.Y=0.83 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=0.98 +lavfi.ssim360.dB=17.62 +frame 6 +lavfi.ssim360.Y=0.83 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=0.98 +lavfi.ssim360.dB=17.52 +frame 7 +lavfi.ssim360.Y=0.83 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=0.98 +lavfi.ssim360.dB=17.36 +frame 8 +lavfi.ssim360.Y=0.83 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=0.98 +lavfi.ssim360.dB=17.32 +frame 9 +lavfi.ssim360.Y=0.82 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=0.98 +lavfi.ssim360.dB=17.21 +frame 10 +lavfi.ssim360.Y=0.82 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=0.98 +lavfi.ssim360.dB=17.16 +frame 11 +lavfi.ssim360.Y=0.82 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=0.98 +lavfi.ssim360.dB=17.09 +frame 12 +lavfi.ssim360.Y=0.82 +lavfi.ssim360.U=1.00 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=0.98 +lavfi.ssim360.dB=17.04 +frame 13 +lavfi.ssim360.Y=0.82 +lavfi.ssim360.U=0.99 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=0.98 +lavfi.ssim360.dB=16.96 +frame 14 +lavfi.ssim360.Y=0.82 +lavfi.ssim360.U=0.99 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=0.98 +lavfi.ssim360.dB=16.95 +frame 15 +lavfi.ssim360.Y=0.82 +lavfi.ssim360.U=0.99 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=0.98 +lavfi.ssim360.dB=16.88 +frame 16 +lavfi.ssim360.Y=0.82 +lavfi.ssim360.U=0.99 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=0.98 +lavfi.ssim360.dB=16.81 +frame 17 +lavfi.ssim360.Y=0.82 +lavfi.ssim360.U=0.99 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=0.98 +lavfi.ssim360.dB=16.77 +frame 18 +lavfi.ssim360.Y=0.82 +lavfi.ssim360.U=0.99 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=0.98 +lavfi.ssim360.dB=16.69 +frame 19 +lavfi.ssim360.Y=0.82 +lavfi.ssim360.U=0.99 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=0.98 +lavfi.ssim360.dB=16.68 +frame 20 +lavfi.ssim360.Y=0.82 +lavfi.ssim360.U=0.99 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=0.98 +lavfi.ssim360.dB=16.65 +frame 21 +lavfi.ssim360.Y=0.82 +lavfi.ssim360.U=0.99 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=0.98 +lavfi.ssim360.dB=16.59 +frame 22 +lavfi.ssim360.Y=0.81 +lavfi.ssim360.U=0.99 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=0.98 +lavfi.ssim360.dB=16.56 +frame 23 +lavfi.ssim360.Y=0.82 +lavfi.ssim360.U=0.99 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=0.98 +lavfi.ssim360.dB=16.53 +frame 24 +lavfi.ssim360.Y=0.81 +lavfi.ssim360.U=0.99 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=0.98 +lavfi.ssim360.dB=16.51 +frame 25 +lavfi.ssim360.Y=0.81 +lavfi.ssim360.U=0.99 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=0.98 +lavfi.ssim360.dB=16.47 +frame 26 +lavfi.ssim360.Y=0.81 +lavfi.ssim360.U=0.99 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=0.98 +lavfi.ssim360.dB=16.45 +frame 27 +lavfi.ssim360.Y=0.81 +lavfi.ssim360.U=0.99 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=0.98 +lavfi.ssim360.dB=16.40 +frame 28 +lavfi.ssim360.Y=0.81 +lavfi.ssim360.U=0.99 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=0.98 +lavfi.ssim360.dB=16.34 +frame 29 +lavfi.ssim360.Y=0.81 +lavfi.ssim360.U=0.99 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=0.98 +lavfi.ssim360.dB=16.31 +frame 30 +lavfi.ssim360.Y=0.81 +lavfi.ssim360.U=0.99 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=0.98 +lavfi.ssim360.dB=16.24 +frame 31 +lavfi.ssim360.Y=0.81 +lavfi.ssim360.U=0.99 +lavfi.ssim360.V=1.00 +lavfi.ssim360.All=0.98 +lavfi.ssim360.dB=16.20 diff --git a/tools/Makefile b/tools/Makefile index f4d1327b9f..3cd252d712 100644 --- a/tools/Makefile +++ b/tools/Makefile @@ -19,6 +19,7 @@ tools/target_io_dem_fuzzer.o: tools/target_dem_fuzzer.c tools/venc_data_dump$(EXESUF): tools/decode_simple.o tools/scale_slice_test$(EXESUF): tools/decode_simple.o +tools/spherical_compare$(EXESUF): tools/decode_simple.o OUTDIRS += tools diff --git a/tools/spherical_compare.c b/tools/spherical_compare.c new file mode 100644 index 0000000000..bcd7858636 --- /dev/null +++ b/tools/spherical_compare.c @@ -0,0 +1,148 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include +#include +#include + +#include "decode_simple.h" + +#include "libavutil/avassert.h" +#include "libavutil/avstring.h" +#include "libavutil/common.h" +#include "libavutil/dict.h" +#include "libavutil/error.h" +#include "libavutil/pixdesc.h" + +#include "libavformat/avformat.h" + +#include "libavcodec/avcodec.h" + +#include "libavfilter/avfilter.h" +#include "libavfilter/buffersink.h" +#include "libavfilter/buffersrc.h" + +typedef struct PrivData { + const char *filterchain; + AVFilterGraph *fg; + AVFilterContext *src; + AVFilterContext *sink; + AVFrame *frame; + + uint64_t nb_frames; +} PrivData; + +static int process_frame(DecodeContext *dc, AVFrame *frame) +{ + PrivData *pd = dc->opaque; + int ret; + + if (!pd->fg) { + AVFilterInOut *inputs, *outputs; + char filterchain[1024]; + + if (!frame) + return 0; + + snprintf(filterchain, sizeof(filterchain), + "buffer@src=width=%d:height=%d:pix_fmt=%s:time_base=%d/%d," + "%s,buffersink@sink", + frame->width, frame->height, + av_get_pix_fmt_name(frame->format), + dc->stream->time_base.num, dc->stream->time_base.den, + pd->filterchain); + + pd->fg = avfilter_graph_alloc(); + if (!pd->fg) + return AVERROR(ENOMEM); + + ret = avfilter_graph_parse2(pd->fg, filterchain, &inputs, &outputs); + if (ret < 0) + return ret; + + av_assert0(!inputs && !outputs); + + pd->src = avfilter_graph_get_filter(pd->fg, "buffer@src"); + pd->sink = avfilter_graph_get_filter(pd->fg, "buffersink@sink"); + av_assert0(pd->src && pd->sink); + + ret = avfilter_graph_config(pd->fg, pd->fg); + if (ret < 0) + return ret; + + pd->frame = av_frame_alloc(); + if (!pd->frame) + return AVERROR(ENOMEM); + } + + ret = av_buffersrc_write_frame(pd->src, frame); + if (ret < 0) + return ret; + + while (ret >= 0) { + AVDictionaryEntry *t = NULL; + + ret = av_buffersink_get_frame(pd->sink, pd->frame); + if ((frame && ret == AVERROR(EAGAIN)) || + (!frame && ret == AVERROR_EOF)) + return 0; + else if (ret < 0) + return ret; + + fprintf(stdout, "frame %"PRIu64"\n", pd->nb_frames++); + while ((t = av_dict_get(pd->frame->metadata, "lavfi.ssim360", t, AV_DICT_IGNORE_SUFFIX))) + fprintf(stdout, "%s=%s\n", t->key, t->value); + } + + return 0; +} + +int main(int argc, char **argv) +{ + PrivData pd; + DecodeContext dc; + + const char *filename, *fc; + int ret = 0; + + if (argc <= 2) { + fprintf(stderr, "Usage: %s \n", argv[0]); + return 0; + } + + filename = argv[1]; + fc = argv[2]; + + memset(&pd, 0, sizeof(pd)); + pd.filterchain = fc; + + ret = ds_open(&dc, filename, 0); + if (ret < 0) + goto finish; + + dc.process_frame = process_frame; + dc.opaque = &pd; + + ret = ds_run(&dc); + +finish: + avfilter_graph_free(&pd.fg); + av_frame_free(&pd.frame); + ds_free(&dc); + return ret; +}