diff mbox series

[FFmpeg-devel,2/3] lavfi/vf_ssim: add warning when color ranges differ

Message ID 20230331164923.4051693-3-chemag@gmail.com
State New
Headers show
Series add warnings to quality metrics on color range mistmatch | expand

Checks

Context Check Description
andriy/make_x86 success Make finished
andriy/make_fate_x86 success Make fate finished

Commit Message

Chema Gonzalez March 31, 2023, 4:49 p.m. UTC 
The SSIM filter uses the pixel values without considering
the color ranges. This is incorrect. Patch adds a warning
so at least the user knows it.

Let's see an example.

(1) Let's get FR and LR versions of the same image.

```
$ ffmpeg -y -i /tmp/lena.490x490.ppm -vf scale="out_range=full" -pix_fmt yuv420p /tmp/lena.full.y4m
$ xxd /tmp/lena.full.y4m |head
00000000: 5955 5634 4d50 4547 3220 5734 3930 2048  YUV4MPEG2 W490 H
00000010: 3439 3020 4632 353a 3120 4970 2041 303a  490 F25:1 Ip A0:
00000020: 3020 4334 3230 6a70 6567 2058 5953 4353  0 C420jpeg XYSCS
00000030: 533d 3432 304a 5045 4720 5843 4f4c 4f52  S=420JPEG XCOLOR
00000040: 5241 4e47 453d 4655 4c4c 0a46 5241 4d45  RANGE=FULL.FRAME
00000050: 0a72 7271 7070 706f 6f6e 6d6d 6c6d 6d6d  .rrqpppoonmmlmmm
00000060: 6c6e 6e6d 6d6e 6e6e 6d6c 6d6d 6d6d 6d6d  lnnmmnnnmlmmmmmm
00000070: 6d6e 6d6b 6c6d 6e6e 6d6c 6d6d 6e6e 6f6f  mnmklmnnmlmmnnoo
00000080: 6f6f 6e6e 6e6e 6f70 7172 7375 7676 7370  oonnnnopqrsuvvsp
00000090: 6d69 6662 5e59 534d 4845 3d35 302e 2d2c  mifb^YSMHE=50.-,
```

```
$ ffmpeg -y -i /tmp/lena.490x490.ppm -vf scale="out_range=limited" -pix_fmt yuv420p /tmp/lena.limited.y4m
$ xxd /tmp/lena.limited.y4m | head
00000000: 5955 5634 4d50 4547 3220 5734 3930 2048  YUV4MPEG2 W490 H
00000010: 3439 3020 4632 353a 3120 4970 2041 303a  490 F25:1 Ip A0:
00000020: 3020 4334 3230 6a70 6567 2058 5953 4353  0 C420jpeg XYSCS
00000030: 533d 3432 304a 5045 4720 5843 4f4c 4f52  S=420JPEG XCOLOR
00000040: 5241 4e47 453d 4c49 4d49 5445 440a 4652  RANGE=LIMITED.FR
00000050: 414d 450a 7272 7170 7070 6f6f 6e6e 6e6d  AME.rrqpppoonnnm
00000060: 6e6e 6e6d 6f6e 6e6e 6e6e 6e6e 6d6e 6e6e  nnnmonnnnnnnmnnn
00000070: 6e6e 6e6e 6f6e 6c6d 6e6f 6e6e 6d6e 6e6f  nnnnonlmnonnmnno
00000080: 6f6f 6f6f 6f6f 6f6f 6f6f 7071 7273 7576  oooooooooopqrsuv
00000090: 7673 706e 6a68 6461 5c57 524e 4b44 3d39  vspnjhda\WRNKD=9
```

Note that the 2x images are the same. Only difference is the range,
and the precision issues related to range conversion.

(2) Let's calculate the SSIM score:
```
$ ./ffmpeg -filter_threads 1 -filter_complex_threads 1 -i /tmp/lena.full.y4m -i /tmp/lena.limited.y4m -lavfi "ssim" -f null -
...
[Parsed_ssim_0 @ 0x360ab00] SSIM Y:0.942347 (12.391801) U:0.995808 (23.776062) V:0.996104 (24.093747) All:0.960217 (14.003012)
```

As we are comparing an image with itself, we expect "Y: 1" as the
luma SSIM. Issue here is that the SSIM filter just uses the pixel
values, ignoring the color ranges.

Proposed solution is to add a warning.
```
$ ./ffmpeg -filter_threads 1 -filter_complex_threads 1 -i /tmp/foo.full.y4m -i /tmp/foo.limited.y4m -lavfi "ssim" -f null -
...
[Parsed_ssim_0 @ 0x3766280] Warning: master and reference frames use different color ranges (pc != tv)
...
[Parsed_ssim_0 @ 0x3766280] SSIM Y:0.000000 (0.000000) U:0.000000 (0.000000) V:0.000000 (0.000000) All:0.000000 (0.000000)
```

Tested:

Ran fate.
```
$ make fate -j
...
TEST    seek-lavf-ppmpipe
TEST    seek-lavf-pgmpipe
TEST    seek-lavf-mxf_opatom
```
---
 libavfilter/vf_ssim.c | 7 +++++++
 1 file changed, 7 insertions(+)
diff mbox series

Patch

diff --git a/libavfilter/vf_ssim.c b/libavfilter/vf_ssim.c
index 1933b9b82d..c3a05182fb 100644
--- a/libavfilter/vf_ssim.c
+++ b/libavfilter/vf_ssim.c
@@ -358,6 +358,13 @@  static int do_ssim(FFFrameSync *fs)
         td.planeheight[n] = s->planeheight[n];
     }
 
+    if (master->color_range != ref->color_range) {
+        av_log(ctx, AV_LOG_WARNING, "Warning: master and reference "
+               "frames use different color ranges (%s != %s)\n",
+               av_color_range_name(master->color_range),
+               av_color_range_name(ref->color_range));
+    }
+
     ff_filter_execute(ctx, s->ssim_plane, &td, NULL,
                       FFMIN((s->planeheight[1] + 3) >> 2, s->nb_threads));