From patchwork Fri Dec 16 08:19:00 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Paul B Mahol X-Patchwork-Id: 1829 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.65.86 with SMTP id o83csp1200682vsa; Fri, 16 Dec 2016 00:19:12 -0800 (PST) X-Received: by 10.28.165.131 with SMTP id o125mr1529554wme.9.1481876352126; Fri, 16 Dec 2016 00:19:12 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id cm8si5991974wjb.52.2016.12.16.00.19.11; Fri, 16 Dec 2016 00:19:12 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9AFE7689BB8; Fri, 16 Dec 2016 10:19:02 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm0-f41.google.com (mail-wm0-f41.google.com [74.125.82.41]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E28D6680CCB for ; Fri, 16 Dec 2016 10:18:55 +0200 (EET) Received: by mail-wm0-f41.google.com with SMTP id t79so22613383wmt.0 for ; Fri, 16 Dec 2016 00:19:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=yY9ccp+6dD4LZZv/k4uX5XOwBfPikNlh6gepghYGWuU=; b=O+jiZa/fzx/0FGYqLs9+BCNE7ViEDvvu+VINqyZiGxc0IYRt+VShbFMJHK1Hs+5AB/ 4H4Fm24HoEdm7gSSVG8m73DT1d4qOYF83cw1SdAsrYxxnxZ8hIZcgcnktNgzHoHEX8XP fJCCjibChdmWyAe3Qa19nl5M3SSggFf5wWO4SdWnpB4giLyzQlKzehlqBRjRmuRRlm1h B3NcSLOYUYDApQvOOjvmwPWuHOtlOMiExYyMoIrMktHrgS2imdSqynzjmc73XiZdgFET WtjwfPAeXoiqFfAbeccWQxBLM39oXNJvH2aaLhAWPLHW4x35lVYDZPGQ3Yl6fHtyUQf2 yjig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=yY9ccp+6dD4LZZv/k4uX5XOwBfPikNlh6gepghYGWuU=; b=LgNhEuJhKfvjPJhCXGzc1cO1Aba8ereJP9YUKRakPDXtZiWMng0HhF+xyQAWj+s+js Xk+NtmHfigPRx/+3BIet7fDv3vkf2vYf//dn7LKVmU8WpbSC0z+T/vRdRX/HdQNiEzHE sit/0JPdlI78cGVIc5kZp65jUV7rjbG3ZlZZYIAXV4vDB8SRJ55XrVI21KfdVWDgT180 TzznAjnB50mAmtZg2qGMoeLUwl6Lr3Fy1Qp3hMfqvPTezXJFWzqNFCObOSmvpM8we1A0 0CWhnDh/lsMvo+NPJj+coprKFBuAKsaXgnGoelw/yiKEAhEd8Ujqbh+xJqkRWVIISIj6 FwVA== X-Gm-Message-State: AIkVDXIi6pPHjq8Ywv6pt4Y+BHRrWwT4Nr0VrUNC9N/+Aobh7LQ0uTt5q6Lf5I6dkiQcogs3j/pAp4aNt6niPA== X-Received: by 10.28.24.74 with SMTP id 71mr1590071wmy.74.1481876341504; Fri, 16 Dec 2016 00:19:01 -0800 (PST) MIME-Version: 1.0 Received: by 10.28.48.12 with HTTP; Fri, 16 Dec 2016 00:19:00 -0800 (PST) In-Reply-To: <20161215234933.GF4749@nb4> References: <20161215234933.GF4749@nb4> From: Paul B Mahol Date: Fri, 16 Dec 2016 09:19:00 +0100 Message-ID: To: FFmpeg development discussions and patches Subject: Re: [FFmpeg-devel] [PATCH] avfilter/vf_ssim: add 9 and 10 bit depth support X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" On 12/16/16, Michael Niedermayer wrote: > On Thu, Dec 15, 2016 at 10:39:37PM +0100, Paul B Mahol wrote: >> Hi, >> >> patch attached > >> vf_ssim.c | 130 >> ++++++++++++++++++++++++++++++++++++++++++++++++----- >> x86/vf_ssim_init.c | 2 >> 2 files changed, 120 insertions(+), 12 deletions(-) >> 7b60c3cadb5319fb93f3875942b50ab9f9b89e1e >> 0001-avfilter-vf_ssim-add-9-and-10-bit-depth-suppport.patch >> From ebf2f89fd191ed661c503726106e46b7c81d6cba Mon Sep 17 00:00:00 2001 >> From: Paul B Mahol >> Date: Thu, 15 Dec 2016 22:33:42 +0100 >> Subject: [PATCH] avfilter/vf_ssim: add 9 and 10 bit depth suppport >> >> Signed-off-by: Paul B Mahol >> --- >> libavfilter/vf_ssim.c | 130 >> +++++++++++++++++++++++++++++++++++++---- >> libavfilter/x86/vf_ssim_init.c | 2 +- >> 2 files changed, 120 insertions(+), 12 deletions(-) > > this produces some build warnings > > CC libavfilter/vf_ssim.o > libavfilter/vf_ssim.c: In function ‘ssim_4x4xn_16bit’: > libavfilter/vf_ssim.c:100:21: warning: ‘main’ is usually a function > [-Wmain] > libavfilter/vf_ssim.c: In function ‘ssim_plane_16bit’: > libavfilter/vf_ssim.c:246:9: warning: passing argument 1 of > ‘ssim_endn_16bit’ from incompatible pointer type [enabled by default] > libavfilter/vf_ssim.c:195:14: note: expected ‘const int64_t (*)[4]’ but > argument is of type ‘int64_t (*)[4]’ > libavfilter/vf_ssim.c:246:9: warning: passing argument 2 of > ‘ssim_endn_16bit’ from incompatible pointer type [enabled by default] > libavfilter/vf_ssim.c:195:14: note: expected ‘const int64_t (*)[4]’ but > argument is of type ‘int64_t (*)[4]’ > CC libavfilter/x86/vf_ssim_init.o > libavfilter/x86/vf_ssim_init.c: In function ‘ff_ssim_init_x86’: > libavfilter/x86/vf_ssim_init.c:40:28: warning: assignment from incompatible > pointer type [enabled by default] > > [...] > -- > Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB > > I know you won't believe me, but the highest form of Human Excellence is > to question oneself and others. -- Socrates > Better patch attached. From 1900cb2b4e6732bb0c1e0a98d0a28fc85aa87f9a Mon Sep 17 00:00:00 2001 From: Paul B Mahol Date: Thu, 15 Dec 2016 22:33:42 +0100 Subject: [PATCH] avfilter/vf_ssim: add 9 and 10 bit depth suppport Signed-off-by: Paul B Mahol --- libavfilter/vf_ssim.c | 128 +++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 117 insertions(+), 11 deletions(-) diff --git a/libavfilter/vf_ssim.c b/libavfilter/vf_ssim.c index dd8f264..c6c25ab 100644 --- a/libavfilter/vf_ssim.c +++ b/libavfilter/vf_ssim.c @@ -51,6 +51,7 @@ typedef struct SSIMContext { FILE *stats_file; char *stats_file_str; int nb_components; + int max; uint64_t nb_frames; double ssim[4], ssim_total; char comps[4]; @@ -60,6 +61,11 @@ typedef struct SSIMContext { int planeheight[4]; int *temp; int is_rgb; + float (*ssim_plane)(SSIMDSPContext *dsp, + uint8_t *main, int main_stride, + uint8_t *ref, int ref_stride, + int width, int height, void *temp, + int max); SSIMDSPContext dsp; } SSIMContext; @@ -87,9 +93,45 @@ static void set_meta(AVDictionary **metadata, const char *key, char comp, float } } -static void ssim_4x4xn(const uint8_t *main, ptrdiff_t main_stride, - const uint8_t *ref, ptrdiff_t ref_stride, - int (*sums)[4], int width) +static void ssim_4x4xn_16bit(const uint8_t *main8, ptrdiff_t main_stride, + const uint8_t *ref8, ptrdiff_t ref_stride, + int64_t (*sums)[4], int width) +{ + const uint16_t *main16 = (const uint16_t *)main8; + const uint16_t *ref16 = (const uint16_t *)ref8; + int x, y, z; + + main_stride >>= 1; + ref_stride >>= 1; + + for (z = 0; z < width; z++) { + uint64_t s1 = 0, s2 = 0, ss = 0, s12 = 0; + + for (y = 0; y < 4; y++) { + for (x = 0; x < 4; x++) { + int a = main16[x + y * main_stride]; + int b = ref16[x + y * ref_stride]; + + s1 += a; + s2 += b; + ss += a*a; + ss += b*b; + s12 += a*b; + } + } + + sums[z][0] = s1; + sums[z][1] = s2; + sums[z][2] = ss; + sums[z][3] = s12; + main16 += 4; + ref16 += 4; + } +} + +static void ssim_4x4xn_8bit(const uint8_t *main, ptrdiff_t main_stride, + const uint8_t *ref, ptrdiff_t ref_stride, + int (*sums)[4], int width) { int x, y, z; @@ -118,6 +160,22 @@ static void ssim_4x4xn(const uint8_t *main, ptrdiff_t main_stride, } } +static float ssim_end1x(int64_t s1, int64_t s2, int64_t ss, int64_t s12, int max) +{ + int64_t ssim_c1 = (int64_t)(.01*.01*max*max*64 + .5); + int64_t ssim_c2 = (int64_t)(.03*.03*max*max*64*63 + .5); + + int64_t fs1 = s1; + int64_t fs2 = s2; + int64_t fss = ss; + int64_t fs12 = s12; + int64_t vars = fss * 64 - fs1 * fs1 - fs2 * fs2; + int64_t covar = fs12 * 64 - fs1 * fs2; + + return (float)(2 * fs1 * fs2 + ssim_c1) * (float)(2 * covar + ssim_c2) + / ((float)(fs1 * fs1 + fs2 * fs2 + ssim_c1) * (float)(vars + ssim_c2)); +} + static float ssim_end1(int s1, int s2, int ss, int s12) { static const int ssim_c1 = (int)(.01*.01*255*255*64 + .5); @@ -134,7 +192,21 @@ static float ssim_end1(int s1, int s2, int ss, int s12) / ((float)(fs1 * fs1 + fs2 * fs2 + ssim_c1) * (float)(vars + ssim_c2)); } -static float ssim_endn(const int (*sum0)[4], const int (*sum1)[4], int width) +static float ssim_endn_16bit(const int64_t (*sum0)[4], const int64_t (*sum1)[4], int width, int max) +{ + float ssim = 0.0; + int i; + + for (i = 0; i < width; i++) + ssim += ssim_end1x(sum0[i][0] + sum0[i + 1][0] + sum1[i][0] + sum1[i + 1][0], + sum0[i][1] + sum0[i + 1][1] + sum1[i][1] + sum1[i + 1][1], + sum0[i][2] + sum0[i + 1][2] + sum1[i][2] + sum1[i + 1][2], + sum0[i][3] + sum0[i + 1][3] + sum1[i][3] + sum1[i + 1][3], + max); + return ssim; +} + +static float ssim_endn_8bit(const int (*sum0)[4], const int (*sum1)[4], int width) { float ssim = 0.0; int i; @@ -147,10 +219,39 @@ static float ssim_endn(const int (*sum0)[4], const int (*sum1)[4], int width) return ssim; } +static float ssim_plane_16bit(SSIMDSPContext *dsp, + uint8_t *main, int main_stride, + uint8_t *ref, int ref_stride, + int width, int height, void *temp, + int max) +{ + int z = 0, y; + float ssim = 0.0; + int64_t (*sum0)[4] = temp; + int64_t (*sum1)[4] = sum0 + (width >> 2) + 3; + + width >>= 2; + height >>= 2; + + for (y = 1; y < height; y++) { + for (; z <= y; z++) { + FFSWAP(void*, sum0, sum1); + ssim_4x4xn_16bit(&main[4 * z * main_stride], main_stride, + &ref[4 * z * ref_stride], ref_stride, + sum0, width); + } + + ssim += ssim_endn_16bit((const int64_t (*)[4])sum0, (const int64_t (*)[4])sum1, width - 1, max); + } + + return ssim / ((height - 1) * (width - 1)); +} + static float ssim_plane(SSIMDSPContext *dsp, uint8_t *main, int main_stride, uint8_t *ref, int ref_stride, - int width, int height, void *temp) + int width, int height, void *temp, + int max) { int z = 0, y; float ssim = 0.0; @@ -190,9 +291,10 @@ static AVFrame *do_ssim(AVFilterContext *ctx, AVFrame *main, s->nb_frames++; for (i = 0; i < s->nb_components; i++) { - c[i] = ssim_plane(&s->dsp, main->data[i], main->linesize[i], - ref->data[i], ref->linesize[i], - s->planewidth[i], s->planeheight[i], s->temp); + c[i] = s->ssim_plane(&s->dsp, main->data[i], main->linesize[i], + ref->data[i], ref->linesize[i], + s->planewidth[i], s->planeheight[i], s->temp, + s->max); ssimv += s->coefs[i] * c[i]; s->ssim[i] += c[i]; } @@ -254,6 +356,8 @@ static int query_formats(AVFilterContext *ctx) AV_PIX_FMT_YUVJ411P, AV_PIX_FMT_YUVJ420P, AV_PIX_FMT_YUVJ422P, AV_PIX_FMT_YUVJ440P, AV_PIX_FMT_YUVJ444P, AV_PIX_FMT_GBRP, +#define PF(suf) AV_PIX_FMT_YUV420##suf, AV_PIX_FMT_YUV422##suf, AV_PIX_FMT_YUV444##suf + PF(P9), PF(P10), AV_PIX_FMT_NONE }; @@ -297,12 +401,14 @@ static int config_input_ref(AVFilterLink *inlink) for (i = 0; i < s->nb_components; i++) s->coefs[i] = (double) s->planeheight[i] * s->planewidth[i] / sum; - s->temp = av_malloc((2 * inlink->w + 12) * sizeof(*s->temp)); + s->temp = av_malloc_array((2 * inlink->w + 12), sizeof(*s->temp) * (1 + (desc->comp[0].depth > 8))); if (!s->temp) return AVERROR(ENOMEM); + s->max = (1 << desc->comp[0].depth) - 1; - s->dsp.ssim_4x4_line = ssim_4x4xn; - s->dsp.ssim_end_line = ssim_endn; + s->ssim_plane = desc->comp[0].depth > 8 ? ssim_plane_16bit : ssim_plane; + s->dsp.ssim_4x4_line = ssim_4x4xn_8bit; + s->dsp.ssim_end_line = ssim_endn_8bit; if (ARCH_X86) ff_ssim_init_x86(&s->dsp); -- 2.5.0