From patchwork Fri Mar 31 16:49:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chema Gonzalez X-Patchwork-Id: 40963 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4645:b0:e3:3194:9d20 with SMTP id eb5csp870393pzb; Fri, 31 Mar 2023 09:49:56 -0700 (PDT) X-Google-Smtp-Source: AKy350bHEzqJfuuC6X2qJWopzaylBkRzzCLuS8GYMy9uXMOh/0jyJ1qGpi2QUCLCty4689C21Mef X-Received: by 2002:a17:906:f6cd:b0:92f:b8d0:746c with SMTP id jo13-20020a170906f6cd00b0092fb8d0746cmr27265936ejb.20.1680281396713; Fri, 31 Mar 2023 09:49:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1680281396; cv=none; d=google.com; s=arc-20160816; b=n3utxofu40J5jUQRcUxbssglpY+c6Cs/Q1KkmrAv+XljFbllG6ppjQj5B2fvO1E1aT qnWhZoRys9WVkekM742sIzX+9+ISJAEddu+zj53NAF6h1U1CVl8aI3yyCRspL3ThXizi x3HO6p6QDXjJv/InwkWURdI9AmjZrukEBYXwwRYyZEeLUw+RmlfSECfmPD31zu+aaHNH q651OzRHF5fjuPH/a/RJrc7jEWy/0a3kDDje9g/DXT+v9L5znI9n4JQhDD1yOM48DtQk iycbycedaVsNNCQmnyLTxdtZMRjdpg7eb/1lU7A68aXXIgmdk2e3C8rlj1QnHOAijGOw 3M8w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=FNfvLKcsAXuaO9cuU2bwB9Dp5hJ9FNWsfFBhkuvAHkI=; b=ETfyKLFlil5xVgxBjhC/ZtnBEyU+yh1+i/k9grEtbJNLusBRYYpH6QHi6UCFQzm6pF NbEjVklZDJHw1JZ45dGY1+BN0oGqVesbkL8298H0TMSxcVpIQo93a+ERoKFJe6YcSiXl isaK7Lhaa3W6NbogSoomFVHI/mdyKN0hU7n9mWk/LeSaTEDrJYeYkcdWF6mh5PKqn9Fg VSD/L6YTJXO0yJeVNvUF0G56XWGfhFHZMkkMhr0ngRCVKhDwsRaGRAMVx1F20o92pGwP 9F4auZUpVbRhonFxrHvxAVCvM07pNYNyUMMv0dMRb8RofrcXft8vzRFlN5V/IXcFzrKP xn/w== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=XGwsQhfB; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id f7-20020a170906738700b0092a3709e877si2566331ejl.679.2023.03.31.09.49.56; Fri, 31 Mar 2023 09:49:56 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=XGwsQhfB; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id DAC1568C292; Fri, 31 Mar 2023 19:49:39 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f50.google.com (mail-pj1-f50.google.com [209.85.216.50]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1607068C1E4 for ; Fri, 31 Mar 2023 19:49:33 +0300 (EEST) Received: by mail-pj1-f50.google.com with SMTP id o6-20020a17090a9f8600b0023f32869993so26029743pjp.1 for ; Fri, 31 Mar 2023 09:49:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1680281371; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9Zp/+K4jbL6yi0zO6NWXi1wqOrx6GWa8r1w8haAKATM=; b=XGwsQhfB5y8OPabZ6M4tyLkxFPi2FTy+Hb+uHhSJmH/iGes5Jfw/6skTvquiQi8lSX 5rJ3rtVS8JhQqfUyZbx+PyPOqXEnrz8OmivtaBWPGzQwnKuVhrqEnm2M/XLp2r1OZzNF YF6cxB7k1G0uxo8Z2J77iXvkpVGZ/bAecSNcYiRPUckdrUnCWC/jXvWEojjC6HWrK8z7 qMi7FbZDUgL67oqaZm69zn50ze2TG+Xp7twmrB1XJaHVNbGRp+sxyuE8QO3qXR+lCqhz CiJNt0bh+VT0QCaj/GcEykgpp/D97L+mU52O3+5Pjg/fWzRVgzWJVKACrYWEPHZXOn2z vBYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680281371; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9Zp/+K4jbL6yi0zO6NWXi1wqOrx6GWa8r1w8haAKATM=; b=KYonTFcx4nXAz7hKrY/emiKyBYmM34QVCsAmmHw8wK7WDgl1K7TAl3mJwJkGLaGWmZ pN8/Ao1SwNUE6YmxEBo/UiH2JL6iPD8ap+wASJFA9/4J+mficiy551TxDmndmcODvWxG 6jmBxiKxWxzXS/GNr8VPxkyaWf3jlHH0MY3+53qk2hIrsY6uITIEvYe5VPQvb3Eyz/Sn 9ycmEMZ7zxORMpiqSUZpiN6ETDmPhJqszJGTrC9S6qwwfvEcq5cMsdimVActpSnzsD0R YeNRkLlBdWDD0hksFcEKIvF60kC9kbpZYYyaRLpKihhIzfuSa7JybDSPctTTJF0R+Ekv fyGg== X-Gm-Message-State: AAQBX9fWFCEgqj3sIKJhOR0mZ2+dKh8ZG0sNBshCgs1xAFRrJ1ypVKDI mdUYicOMOPJBgzxQW5MzNpU4RfYSs64= X-Received: by 2002:a17:903:22c6:b0:1a2:8924:2230 with SMTP id y6-20020a17090322c600b001a289242230mr7172386plg.27.1680281370940; Fri, 31 Mar 2023 09:49:30 -0700 (PDT) Received: from wilferico.thefacebook.com ([2620:10d:c090:400::5:b0e]) by smtp.gmail.com with ESMTPSA id w7-20020a170902d70700b0019aa6bf4450sm1772117ply.188.2023.03.31.09.49.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Mar 2023 09:49:30 -0700 (PDT) From: Chema Gonzalez To: ffmpeg-devel@ffmpeg.org Date: Fri, 31 Mar 2023 09:49:22 -0700 Message-Id: <20230331164923.4051693-3-chemag@gmail.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230331164923.4051693-1-chemag@gmail.com> References: <20230331164923.4051693-1-chemag@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/3] lavfi/vf_ssim: add warning when color ranges differ X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Chema Gonzalez Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: zonEKGQa/r+P The SSIM filter uses the pixel values without considering the color ranges. This is incorrect. Patch adds a warning so at least the user knows it. Let's see an example. (1) Let's get FR and LR versions of the same image. ``` $ ffmpeg -y -i /tmp/lena.490x490.ppm -vf scale="out_range=full" -pix_fmt yuv420p /tmp/lena.full.y4m $ xxd /tmp/lena.full.y4m |head 00000000: 5955 5634 4d50 4547 3220 5734 3930 2048 YUV4MPEG2 W490 H 00000010: 3439 3020 4632 353a 3120 4970 2041 303a 490 F25:1 Ip A0: 00000020: 3020 4334 3230 6a70 6567 2058 5953 4353 0 C420jpeg XYSCS 00000030: 533d 3432 304a 5045 4720 5843 4f4c 4f52 S=420JPEG XCOLOR 00000040: 5241 4e47 453d 4655 4c4c 0a46 5241 4d45 RANGE=FULL.FRAME 00000050: 0a72 7271 7070 706f 6f6e 6d6d 6c6d 6d6d .rrqpppoonmmlmmm 00000060: 6c6e 6e6d 6d6e 6e6e 6d6c 6d6d 6d6d 6d6d lnnmmnnnmlmmmmmm 00000070: 6d6e 6d6b 6c6d 6e6e 6d6c 6d6d 6e6e 6f6f mnmklmnnmlmmnnoo 00000080: 6f6f 6e6e 6e6e 6f70 7172 7375 7676 7370 oonnnnopqrsuvvsp 00000090: 6d69 6662 5e59 534d 4845 3d35 302e 2d2c mifb^YSMHE=50.-, ``` ``` $ ffmpeg -y -i /tmp/lena.490x490.ppm -vf scale="out_range=limited" -pix_fmt yuv420p /tmp/lena.limited.y4m $ xxd /tmp/lena.limited.y4m | head 00000000: 5955 5634 4d50 4547 3220 5734 3930 2048 YUV4MPEG2 W490 H 00000010: 3439 3020 4632 353a 3120 4970 2041 303a 490 F25:1 Ip A0: 00000020: 3020 4334 3230 6a70 6567 2058 5953 4353 0 C420jpeg XYSCS 00000030: 533d 3432 304a 5045 4720 5843 4f4c 4f52 S=420JPEG XCOLOR 00000040: 5241 4e47 453d 4c49 4d49 5445 440a 4652 RANGE=LIMITED.FR 00000050: 414d 450a 7272 7170 7070 6f6f 6e6e 6e6d AME.rrqpppoonnnm 00000060: 6e6e 6e6d 6f6e 6e6e 6e6e 6e6e 6d6e 6e6e nnnmonnnnnnnmnnn 00000070: 6e6e 6e6e 6f6e 6c6d 6e6f 6e6e 6d6e 6e6f nnnnonlmnonnmnno 00000080: 6f6f 6f6f 6f6f 6f6f 6f6f 7071 7273 7576 oooooooooopqrsuv 00000090: 7673 706e 6a68 6461 5c57 524e 4b44 3d39 vspnjhda\WRNKD=9 ``` Note that the 2x images are the same. Only difference is the range, and the precision issues related to range conversion. (2) Let's calculate the SSIM score: ``` $ ./ffmpeg -filter_threads 1 -filter_complex_threads 1 -i /tmp/lena.full.y4m -i /tmp/lena.limited.y4m -lavfi "ssim" -f null - ... [Parsed_ssim_0 @ 0x360ab00] SSIM Y:0.942347 (12.391801) U:0.995808 (23.776062) V:0.996104 (24.093747) All:0.960217 (14.003012) ``` As we are comparing an image with itself, we expect "Y: 1" as the luma SSIM. Issue here is that the SSIM filter just uses the pixel values, ignoring the color ranges. Proposed solution is to add a warning. ``` $ ./ffmpeg -filter_threads 1 -filter_complex_threads 1 -i /tmp/foo.full.y4m -i /tmp/foo.limited.y4m -lavfi "ssim" -f null - ... [Parsed_ssim_0 @ 0x3766280] Warning: master and reference frames use different color ranges (pc != tv) ... [Parsed_ssim_0 @ 0x3766280] SSIM Y:0.000000 (0.000000) U:0.000000 (0.000000) V:0.000000 (0.000000) All:0.000000 (0.000000) ``` Tested: Ran fate. ``` $ make fate -j ... TEST seek-lavf-ppmpipe TEST seek-lavf-pgmpipe TEST seek-lavf-mxf_opatom ``` --- libavfilter/vf_ssim.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/libavfilter/vf_ssim.c b/libavfilter/vf_ssim.c index 1933b9b82d..c3a05182fb 100644 --- a/libavfilter/vf_ssim.c +++ b/libavfilter/vf_ssim.c @@ -358,6 +358,13 @@ static int do_ssim(FFFrameSync *fs) td.planeheight[n] = s->planeheight[n]; } + if (master->color_range != ref->color_range) { + av_log(ctx, AV_LOG_WARNING, "Warning: master and reference " + "frames use different color ranges (%s != %s)\n", + av_color_range_name(master->color_range), + av_color_range_name(ref->color_range)); + } + ff_filter_execute(ctx, s->ssim_plane, &td, NULL, FFMIN((s->planeheight[1] + 3) >> 2, s->nb_threads));