From patchwork Fri Jan 15 05:06:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Boris Baracaldo X-Patchwork-Id: 24957 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 80D0344B694 for ; Fri, 15 Jan 2021 07:06:25 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 55ACA68084A; Fri, 15 Jan 2021 07:06:25 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D8C95680282 for ; Fri, 15 Jan 2021 07:06:17 +0200 (EET) Received: from pps.filterd (m0089730.ppops.net [127.0.0.1]) by m0089730.ppops.net (8.16.0.43/8.16.0.43) with SMTP id 10F54YoV003643 for ; Thu, 14 Jan 2021 21:06:16 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : subject : date : message-id : references : in-reply-to : content-type : mime-version; s=facebook; bh=0o+shbbAdY+IQVGYDg73UQXNdsmW1hd1e11cmUQxuH8=; b=YfQQe3HtjqMh7FpqeS9q8N82OKBvFD6/XabKwRGlFArfqT3jFY0740GmfG89PEbPob06 Pg3pI2+QW6+SVHz8cQpIR+sU6roTV5F22tSBGoMXhMQjRKBenJVDPsIXPOdYbnOvdxaD q/qK3KnoTClx2fOsl4yE2IB7q7Ma3rgXAlI= Received: from maileast.thefacebook.com ([163.114.130.16]) by m0089730.ppops.net with ESMTP id 361fpqxuxu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Thu, 14 Jan 2021 21:06:15 -0800 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (100.104.31.183) by o365-in.thefacebook.com (100.104.35.173) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1979.3; Thu, 14 Jan 2021 21:06:14 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=QOT957BgxHAyOqoVRS5LpIOgUYJI7P9aCX7MxXCpxMaPM14HWG2XGDXMgKv1kPWToVy1UIsf8fS2aUNf4F+eEKCZNAZgC2pFQeYDwOhsnI108iWvxgpZ8qTzgz39BesvgvvE8aZuh+ASuvcBeJ6mOYb06lmKZwiAdQ1dIfmAVqGq4/MIT125VpmDgQOt8fRHcxuQv+PHKYa4kWdnlzYJhsakzf4pNfZHwgi/Bk72sjHFCuD3FVpOunMVIH/Ocj88Nx89M9A/kf0LBpkv2re0cmtZyAJdl27y7LgRX0u6nUe8UU+jMddnrPzNCryosIn2h6Cqj9TtkgxiGzjgfidWXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VdKqW31SZNE8w9ned88Pk2LmyJtsh/YyWQYd/q12KgA=; b=DnUJ26vX48N7NwdhJjLTg80HIuhenynXGVfyhnewFNIS68jRtJLiGPrrw2B13DkBfaEed3uvOtPxT1C0zDVLms8dtrEGpatdVa1UUUhTTSfXAgH7mdd3mGYV1SwRnRqEnEcLq80TJi7Ubx4z4HHGzBft2/TzLIFzOkAZs1bLQGujRB3n2YmMd9NQuxKrMpIPMbU0pS+5wKYf/cStPGAeMwGjplfJziO5q1LzM60BGEmoL+A+wL8tldLLicKPJ6Tyrf1HsDYl1FDPYSaPIIIR7yqhFaBc/kIYvwcQ/lYKsEqbjUwHjDKX/THasy+YZ/mTJnN2AWsLPBymLUXwgmB1QQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=fb.com; dmarc=pass action=none header.from=fb.com; dkim=pass header.d=fb.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.onmicrosoft.com; s=selector2-fb-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VdKqW31SZNE8w9ned88Pk2LmyJtsh/YyWQYd/q12KgA=; b=bL1232TFoOvafPoCshH/Z4gR+S/zZ93dEma8UgnNlz8imq5+qYz3UEXjBgTpO7gfX2MTKxV0I+1AsCwwFSJZoUpoZiAIDT5CLZUujHcPbaU15Om8ObSQJKA4GrrVRvBe9q8BnFkTZxDChiH5SeFAQooJETnyj2U2NrJXm19hYBE= Received: from MN2PR15MB2605.namprd15.prod.outlook.com (2603:10b6:208:126::10) by MN2PR15MB2784.namprd15.prod.outlook.com (2603:10b6:208:12f::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3763.9; Fri, 15 Jan 2021 05:06:13 +0000 Received: from MN2PR15MB2605.namprd15.prod.outlook.com ([fe80::dd7a:ed9c:147a:3b47]) by MN2PR15MB2605.namprd15.prod.outlook.com ([fe80::dd7a:ed9c:147a:3b47%5]) with mapi id 15.20.3742.012; Fri, 15 Jan 2021 05:06:13 +0000 From: Boris Baracaldo To: FFmpeg development discussions and patches Thread-Topic: [PATCH] avfilter: Added siti filter Thread-Index: AQHW6vsduDITvYWV10uY4UQN7Vd3+KooIXIY Date: Fri, 15 Jan 2021 05:06:12 +0000 Message-ID: References: <20210115045832.76405-1-borbarak@fb.com> In-Reply-To: <20210115045832.76405-1-borbarak@fb.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: ffmpeg.org; dkim=none (message not signed) header.d=none;ffmpeg.org; dmarc=none action=none header.from=fb.com; x-originating-ip: [2620:10d:c091:480::1:fc1d] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: d59108db-7d10-41e3-5154-08d8b91347a2 x-ms-traffictypediagnostic: MN2PR15MB2784: x-microsoft-antispam-prvs: x-fb-source: Internal x-ms-oob-tlc-oobclassifiers: OLM:3513; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: jgIkyCxWZeg6ev9lX3b2W4kzINfXjy6NQFMfDL5jLAE7nQtm9iyFuCf5W/t+NLzOjcikPiw27poo09YSIsDdNTLqeu2IxknzRaYSdaLwGbV4HOMKaZZE5hd0EKrTeNaXPNVO2EOkeweudQjvmSfxmsPIpsUy23TDL+beG5ZhHwjqzzCk9HdEbJ/IPfYR/QwUvKYL5s0t7PrIaQBnblDa/vT+jvg/YNGaxE9xRCIMyrHr6Wn4Tw1aGDmN+Pvcsic20secV4xztAJgwDova5ckHQ2KymDaFltKCYBlFQf3JulTm0Aa7N+BXTxWcy4K8Z0euh/VjzCmQDWDUwmWgEhmfnjyDJ4tB+8OEA4u62Q89sCSQW6zC/No8zf0pTUdw7vSnR7jyU/vPppG7+Pq8msmuq82RsPTKsRZnt+ulX3aVxFmSsw36dbXiUrRgVouNoeZzM7V7e3P5a2l5hedL4aSAn91bTX1APAoK5IB1SNLeWBrFAnKiKWlD4z79tHkh9HgIvgSey6GnOdBOOH3X14aenEW3EDoz48EihmlCvhPPCk= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR15MB2605.namprd15.prod.outlook.com; PTR:; CAT:NONE; SFS:(136003)(376002)(366004)(396003)(39860400002)(346002)(55016002)(6506007)(19627405001)(316002)(186003)(8936002)(64756008)(6916009)(66946007)(76116006)(8676002)(5660300002)(83380400001)(478600001)(83080400002)(9686003)(52536014)(2906002)(86362001)(66556008)(33656002)(66476007)(66446008)(71200400001)(30864003)(7696005)(2004002); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: =?iso-8859-1?q?5/z7IiG2U7S0qHV8oDQ+qOLIx?= =?iso-8859-1?q?JEM/n3Jw2T1UJp6sia7TIiOQwk3rgzx3W88V42bmfsoeze3wEtUhrZxQq+mo?= =?iso-8859-1?q?PC2iUTYTVDBe0DIK4xzMZQLQQgbNxipuBog0f+19XxSNx8Ax8A4frLEN197p?= =?iso-8859-1?q?/SKXaK9/bpqwd7HlR37xkv4r25RdjWGBEVB+OSpdhdGhSUyWXPO7Oh1adUfk?= =?iso-8859-1?q?thikK2HdIHCp/fVYf0+1pkuuwAW1UOYgHFByg5A+r529f5xLyTrTD6lQ+miK?= =?iso-8859-1?q?T9TM/OgTQEVeNOir7BtuybonM4rx1LJcJLkc9MwdX6jQBoqXuohIu0IpCsRg?= =?iso-8859-1?q?9RhbLf8XWDH9uc0jYJEZo+OcCc/0g4wqKINLx2k7+tflp46AD3sqW9EW/JK1?= =?iso-8859-1?q?Q0i+8ChhfyIkJDDaOatH7Z04Nqg8aBpN/d0DNN3zIqoJFLrchYv6Nhb85SPC?= =?iso-8859-1?q?D9CzNY26nGdws89zXlfVM49gKkw7QPTvl/ExtoQxMZcimLMfgp7XzGTjrjlv?= =?iso-8859-1?q?AdMQZdJp6HinhAxMcxGiKDIIXY9lD5WJ0RJECKeoOuEzWPYHJer2xVbEKfEG?= =?iso-8859-1?q?L6JMlJTMKaz7ie/IUrXL6YK6n4EoiGjTOOr6CaCBbsSvd+XBMxJkSME5vqRH?= =?iso-8859-1?q?MjXoQSolvoWnSkG88JbjOl+KhuV+l4qJit26xe9xF7ZvFjKb34kdkXchC9gE?= =?iso-8859-1?q?YjOhl3O6at8tcD9tDZBYkfT5CmWmHBmOXMs95L1IRWg6aMeZWhlLNkt7QNaB?= =?iso-8859-1?q?mAMB2Atmm70zV7517rIGZlIPg+UAEYHJ6JKko6iO2WVvyb4lSzgT3duHV2Ar?= =?iso-8859-1?q?maWgRbyLOcivQ4QIzwWYywu22ZcR/n5+v+8y7+5qLrGr2gnMkoj5VdGOhuuH?= =?iso-8859-1?q?dYXOlwxvsOIA6HedjYUxq+zSYz/0Phd2H+yVy11zLiA53W7IJli1Ew5RUjyt?= =?iso-8859-1?q?d+5N7ir+fb6/x4TLa4Vsdv9RGeFAd9+xXdMF8x/S0HeVNTR/pn6EqfuOEJ4/?= =?iso-8859-1?q?/gfZECFDiKsiSHjSp7Msb89SyGxcRq8MCg3xzIzAndb3yr7tb6RyhU9wQZyr?= =?iso-8859-1?q?w=3D=3D?= x-ms-exchange-transport-forked: True X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR15MB2605.namprd15.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: d59108db-7d10-41e3-5154-08d8b91347a2 X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Jan 2021 05:06:12.9412 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: oIkXxkTR7qkGetRPrbtELuXSHgCE1KTe9oFofHdO4/fl6blzl6NTphFTHyAkQwAD X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR15MB2784 X-OriginatorOrg: fb.com X-Proofpoint-UnRewURL: 2 URL's were un-rewritten MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-15_01:2021-01-15, 2021-01-15 signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 lowpriorityscore=0 impostorscore=0 suspectscore=0 phishscore=0 malwarescore=0 adultscore=0 mlxlogscore=999 spamscore=0 bulkscore=0 clxscore=1011 priorityscore=1501 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101150028 X-FB-Internal: deliver X-Content-Filtered-By: Mailman/MimeDel 2.1.20 Subject: [FFmpeg-devel] [PATCH] avfilter: Added siti filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Calculate Spatial Info (SI) and Temporal Info (TI) scores for a video, as defined in ITU-T P.910: Subjective video quality assessment methods for multimedia applications. --- Changelog | 1 + doc/filters.texi | 25 ++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/version.h | 2 +- libavfilter/vf_siti.c | 359 +++++++++++++++++++++++++++++++++++++++++++++++ 6 files changed, 388 insertions(+), 1 deletion(-) create mode 100644 libavfilter/vf_siti.c -- 2.13.5 diff --git a/Changelog b/Changelog index dcb80e0ed9..9ca79a12aa 100644 --- a/Changelog +++ b/Changelog @@ -55,6 +55,7 @@ version : - asuperpass and asuperstop filter - shufflepixels filter - tmidequalizer filter +- siti filter version 4.3: diff --git a/doc/filters.texi b/doc/filters.texi index 813e35c2f9..eb1f23128c 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -18158,6 +18158,31 @@ ffmpeg -i input1.mkv -i input2.mkv -filter_complex "[0:v][1:v] signature=nb_inpu @end itemize +@anchor{siti} +@section siti + +Calculate Spatial Info (SI) and Temporal Info (TI) scores for a video, as defined +in ITU-T P.910: Subjective video quality assessment methods for multimedia +applications. Available PDF at @url{https://www.itu.int/rec/T-REC-P.910-199909-S/en }. +Per frame metrics can be written into a file in csv format. + +It accepts the following option: + +@table @option +@item stats_file +Set the path to the file where per frame SI and TI metrics will be written. If no file +is specified, only summary statistics will be printed to the console. +@end table + +@subsection Examples +@itemize +@item +To calculate SI/TI metrics and store per frame data to stats.csv: +@example +ffmpeg -i input.mp4 -vf siti=stats_file='siti.csv' -f null - +@end example +@end itemize + @anchor{smartblur} @section smartblur diff --git a/libavfilter/Makefile b/libavfilter/Makefile index ad1046d526..5514536463 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -413,6 +413,7 @@ OBJS-$(CONFIG_SMARTBLUR_FILTER) += vf_smartblur.o OBJS-$(CONFIG_SOBEL_FILTER) += vf_convolution.o OBJS-$(CONFIG_SOBEL_OPENCL_FILTER) += vf_convolution_opencl.o opencl.o \ opencl/convolution.o +OBJS-$(CONFIG_SITI_FILTER) += vf_siti.o OBJS-$(CONFIG_SPLIT_FILTER) += split.o OBJS-$(CONFIG_SPP_FILTER) += vf_spp.o qp_table.o OBJS-$(CONFIG_SR_FILTER) += vf_sr.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index ce317dfa1c..0aef9fdcef 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -393,6 +393,7 @@ extern AVFilter ff_vf_signature; extern AVFilter ff_vf_smartblur; extern AVFilter ff_vf_sobel; extern AVFilter ff_vf_sobel_opencl; +extern AVFilter ff_vf_siti; extern AVFilter ff_vf_split; extern AVFilter ff_vf_spp; extern AVFilter ff_vf_sr; diff --git a/libavfilter/version.h b/libavfilter/version.h index fbe2ed62b2..2136235e54 100644 --- a/libavfilter/version.h +++ b/libavfilter/version.h @@ -30,7 +30,7 @@ #include "libavutil/version.h" #define LIBAVFILTER_VERSION_MAJOR 7 -#define LIBAVFILTER_VERSION_MINOR 95 +#define LIBAVFILTER_VERSION_MINOR 96 #define LIBAVFILTER_VERSION_MICRO 100 diff --git a/libavfilter/vf_siti.c b/libavfilter/vf_siti.c new file mode 100644 index 0000000000..e8b6142ae7 --- /dev/null +++ b/libavfilter/vf_siti.c @@ -0,0 +1,359 @@ +/* + * Copyright (c) 2002 A'rpi + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License along + * with FFmpeg; if not, write to the Free Software Foundation, Inc., + * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + */ + +/** + * @file + * Calculate Spatial Info (SI) and Temporal Info (TI) scores + */ + +#include + +#include "libavutil/imgutils.h" +#include "libavutil/internal.h" +#include "libavutil/opt.h" + +#include "avfilter.h" +#include "formats.h" +#include "internal.h" +#include "video.h" + +static const int X_FILTER[9] = { + 1, 0, -1, + 2, 0, -2, + 1, 0, -1 +}; + +static const int Y_FILTER[9] = { + 1, 2, 1, + 0, 0, 0, + -1, -2, -1 +}; + +typedef struct SiTiContext { + const AVClass *class; + int pixel_depth; + int width, height; + int nb_frames; + unsigned char *prev_frame; + double max_si; + double max_ti; + double min_si; + double min_ti; + double sum_si; + double sum_ti; + FILE *stats_file; + char *stats_file_str; + int full_range; +} SiTiContext; + +static int query_formats(AVFilterContext *ctx) +{ + static const enum AVPixelFormat pix_fmts[] = { + AV_PIX_FMT_YUV420P, AV_PIX_FMT_YUV422P, + AV_PIX_FMT_YUVJ420P, AV_PIX_FMT_YUVJ422P, + AV_PIX_FMT_YUV420P10, AV_PIX_FMT_YUV422P10, + AV_PIX_FMT_NONE + }; + + AVFilterFormats *fmts_list = ff_make_format_list(pix_fmts); + if (!fmts_list) + return AVERROR(ENOMEM); + return ff_set_common_formats(ctx, fmts_list); +} + +static av_cold int init(AVFilterContext *ctx) +{ + // User options but no input data + SiTiContext *s = ctx->priv; + s->max_si = 0; + if (s->stats_file_str) { + s->stats_file = fopen(s->stats_file_str, "w"); + if (!s->stats_file) { + int err = AVERROR(errno); + char buf[128]; + av_strerror(err, buf, sizeof(buf)); + av_log(ctx, AV_LOG_ERROR, "Could not open stats file %s: %s\n", + s->stats_file_str, buf); + return err; + } + fprintf(s->stats_file, "Frame,SI,TI\n"); + } + + return 0; +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + SiTiContext *s = ctx->priv; + + double avg_si = s->sum_si / s->nb_frames; + double avg_ti = s->sum_ti / s->nb_frames; + av_log(ctx, AV_LOG_INFO, + "Summary:\nTotal frames: %d\n\n" + "Spatial Information:\nAverage: %f\nMax: %f\nMin: %f\n\n" + "Temporal Information:\nAverage: %f\nMax: %f\nMin: %f\n", + s->nb_frames, avg_si, s->max_si, s->min_si, avg_ti, s->max_ti, s->min_ti + ); + + if (s->stats_file && s->stats_file != stdout) + fclose(s->stats_file); +} + +static int config_input(AVFilterLink *inlink) +{ + // Video input data avilable + AVFilterContext *ctx = inlink->dst; + SiTiContext *s = ctx->priv; + int max_pixsteps[4]; + + const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(inlink->format); + av_image_fill_max_pixsteps(max_pixsteps, NULL, desc); + + s->pixel_depth = max_pixsteps[0]; + s->width = inlink->w; + s->height = inlink->h; + size_t pixel_sz = s->pixel_depth==1? (size_t) sizeof(uint8_t) : (size_t) sizeof(uint16_t); + size_t data_sz = (size_t) s->width * pixel_sz * s->height; + s->prev_frame = av_malloc(data_sz); + + return 0; +} + +// Get frame data handling 8 and 10 bit formats +static uint16_t get_frame_data(const unsigned char* src, int pixel_depth, int index) +{ + const uint16_t *src16 = (const uint16_t *)src; + if (pixel_depth == 2) + { + return src16[index]; + } + return (uint16_t) src[index]; +} + +// Set frame data handling 8 and 10 bit formats +static void set_frame_data(unsigned char* dst, int pixel_depth, int index, uint16_t data) +{ + uint16_t *dst16 = (uint16_t *)dst; + if (pixel_depth == 2) + { + dst16[index] = data; + } + else + { + dst[index] = (uint8_t) data; + } +} + +// Determine whether the video is in full or limited range. If not defined, assume limited. +static int is_full_range(AVFrame* frame) +{ + if (frame->color_range == AVCOL_RANGE_UNSPECIFIED || frame->color_range == AVCOL_RANGE_NB) + { + // If color range not specified, fallback to pixel format + return frame->format == AV_PIX_FMT_YUVJ420P || frame->format == AV_PIX_FMT_YUVJ422P; + } + return frame->color_range == AVCOL_RANGE_JPEG; +} + +// Check frame's color range and convert to full range if needed +static uint16_t convert_full_range(uint16_t y, SiTiContext *s) +{ + if (s->full_range == 1) + { + return y; + } + + // For 8 bits, limited range goes from 16 to 235, for 10 bits the range is multiplied by 4 + double factor = s->pixel_depth == 1? 1 : 4; + double shift = 16 * factor; + double limit_upper = 235 * factor - shift; + double full_upper = 256 * factor - 1; + double limit_y = fmin(fmax(y - shift, 0), limit_upper); + return (uint16_t) (full_upper * limit_y / limit_upper); +} + +// Applies sobel convolution +static void convolve_sobel(const unsigned char* src, double* dst, int linesize, SiTiContext *s) +{ + int filter_width = 3; + int filter_size = filter_width * filter_width; + for (int j=1; jheight-1; j++) + { + for (int i=1; iwidth-1; i++) + { + double x_conv_sum = 0, y_conv_sum = 0; + for (int k=0; kpixel_depth) + (i + ki); + uint16_t data = convert_full_range(get_frame_data(src, s->pixel_depth, index), s); + x_conv_sum += data * X_FILTER[k]; + y_conv_sum += data * Y_FILTER[k]; + } + double gradient = sqrt(x_conv_sum * x_conv_sum + y_conv_sum * y_conv_sum); + // Dst matrix is smaller than src since we ignore edges that can't be convolved + dst[(j - 1) * (s->width - 2) + (i - 1)] = gradient; + } + } +} + +// Calculate pixel difference between current and previous frame, and update previous +static void calculate_motion(const unsigned char* curr, double* motion_matrix, + int linesize, SiTiContext *s) +{ + for (int j=0; jheight; j++) + { + for (int i=0; iwidth; i++) + { + double motion = 0; + int curr_index = j * (linesize / s->pixel_depth) + i; + int prev_index = j * s->width + i; + uint16_t curr_data = convert_full_range(get_frame_data(curr, s->pixel_depth, curr_index), s); + + if (s->nb_frames > 1) + { + // Previous frame is already converted to full range + motion = curr_data - get_frame_data(s->prev_frame, s->pixel_depth, prev_index); + } + set_frame_data(s->prev_frame, s->pixel_depth, prev_index, curr_data); + motion_matrix[j * s->width + i] = motion; + } + } +} + +static double std_deviation(double* img_metrics, int width, int height) +{ + double size = height * width; + + double mean_sum = 0; + for (int j=0; jdst; + SiTiContext *s = ctx->priv; + + // Gradient matrix will not include the input frame's edges + size_t gradient_data_sz = (size_t) (s->width - 2) * sizeof(double) * (s->height - 2); + double *gradient_matrix = av_malloc(gradient_data_sz); + size_t motion_data_sz = (size_t) s->width * sizeof(double) * s->height; + double *motion_matrix = av_malloc(motion_data_sz); + if (!gradient_matrix || !motion_matrix) { + av_frame_free(&frame); + return AVERROR(ENOMEM); + } + + s->full_range = is_full_range(frame); + s->nb_frames++; + + // Calculate si and ti + convolve_sobel(frame->data[0], gradient_matrix, frame->linesize[0], s); + calculate_motion(frame->data[0], motion_matrix, frame->linesize[0], s); + double si = std_deviation(gradient_matrix, s->width - 2, s->height - 2); + double ti = std_deviation(motion_matrix, s->width, s->height); + + // Calculate statistics + s->max_si = fmax(si, s->max_si); + s->max_ti = fmax(ti, s->max_ti); + s->sum_si += si; + s->sum_ti += ti; + s->min_si = s->nb_frames == 1? si : fmin(si, s->min_si); + s->min_ti = s->nb_frames == 1? ti : fmin(ti, s->min_ti); + + // Set si ti information in frame metadata + set_meta(&frame->metadata, "lavfi.siti.si", si); + set_meta(&frame->metadata, "lavfi.siti.ti", ti); + + // Print per frame csv data to file + if (s->stats_file) + { + fprintf(s->stats_file, "%d,%f,%f\n", s->nb_frames, si, ti); + } + + av_free(gradient_matrix); + return ff_filter_frame(inlink->dst->outputs[0], frame); +} + +#define OFFSET(x) offsetof(SiTiContext, x) +#define FLAGS AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_FILTERING_PARAM + +static const AVOption siti_options[] = { + {"stats_file", "Set file where to store per-frame si-ti scores", OFFSET(stats_file_str), AV_OPT_TYPE_STRING, {.str=NULL}, 0, 0, FLAGS }, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(siti); + +static const AVFilterPad avfilter_vf_siti_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_input, + .filter_frame = filter_frame, + }, + { NULL } +}; + +static const AVFilterPad avfilter_vf_siti_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO + }, + { NULL } +}; + +AVFilter ff_vf_siti = { + .name = "siti", + .description = NULL_IF_CONFIG_SMALL("Calculate spatial info (SI)."), + .priv_size = sizeof(SiTiContext), + .priv_class = &siti_class, + .init = init, + .uninit = uninit, + .query_formats = query_formats, + .inputs = avfilter_vf_siti_inputs, + .outputs = avfilter_vf_siti_outputs, +};