diff mbox series

[FFmpeg-devel,2/2] fate/video: add DXV3 HQ tests

Message ID 20240124194506.15281-2-connorbworley@gmail.com
State New
Headers show
Series [FFmpeg-devel,1/2] fate/video: add DXV3 DXT1 encoding test | expand

Checks

Context Check Description
yinshiyou/make_loongarch64 success Make finished
yinshiyou/make_fate_loongarch64 fail Make fate failed
andriy/make_x86 success Make finished
andriy/make_fate_x86 fail Make fate failed

Commit Message

Connor Worley Jan. 24, 2024, 7:45 p.m. UTC 
Adds tests to cover decoding YCoCg DXV3 formats with and without alpha

Samples:
https://connorworley.com/fate-suite/dxv/dxv3-hqna.mov
https://connorworley.com/fate-suite/dxv/dxv3-hqwa.mov
Signed-off-by: Connor Worley <connorbworley@gmail.com>
---
 tests/fate/video.mak     | 6 ++++++
 tests/ref/fate/dxv3-ycg6 | 6 ++++++
 tests/ref/fate/dxv3-yg10 | 6 ++++++
 3 files changed, 18 insertions(+)
 create mode 100644 tests/ref/fate/dxv3-ycg6
 create mode 100644 tests/ref/fate/dxv3-yg10

Comments

Connor Worley Jan. 28, 2024, 9:31 p.m. UTC | #1 
I'd like to get this series merged before doing any further DXV work. Is
anyone able to help with uploading the linked samples?

On Wed, Jan 24, 2024 at 11:45 AM Connor Worley <connorbworley@gmail.com>
wrote:

> Adds tests to cover decoding YCoCg DXV3 formats with and without alpha
>
> Samples:
> https://connorworley.com/fate-suite/dxv/dxv3-hqna.mov
> https://connorworley.com/fate-suite/dxv/dxv3-hqwa.mov
> Signed-off-by: Connor Worley <connorbworley@gmail.com>
> ---
>  tests/fate/video.mak     | 6 ++++++
>  tests/ref/fate/dxv3-ycg6 | 6 ++++++
>  tests/ref/fate/dxv3-yg10 | 6 ++++++
>  3 files changed, 18 insertions(+)
>  create mode 100644 tests/ref/fate/dxv3-ycg6
>  create mode 100644 tests/ref/fate/dxv3-yg10
>
> diff --git a/tests/fate/video.mak b/tests/fate/video.mak
> index fbc8a1a682..8f51a42077 100644
> --- a/tests/fate/video.mak
> +++ b/tests/fate/video.mak
> @@ -152,6 +152,12 @@ fate-dxv3-dxt1: CMD = framecrc -i
> $(TARGET_SAMPLES)/dxv/dxv3-nqna.mov
>  FATE_DXV += fate-dxv3-dxt5
>  fate-dxv3-dxt5: CMD = framecrc -i $(TARGET_SAMPLES)/dxv/dxv3-nqwa.mov
>
> +FATE_DXV += fate-dxv3-ycg6
> +fate-dxv3-ycg6: CMD = framecrc -i $(TARGET_SAMPLES)/dxv/dxv3-hqna.mov
> +
> +FATE_DXV += fate-dxv3-yg10
> +fate-dxv3-yg10: CMD = framecrc -i $(TARGET_SAMPLES)/dxv/dxv3-hqwa.mov
> +
>  FATE_VIDEO-$(call FRAMECRC, MOV, DXV) += $(FATE_DXV)
>  fate-dxv: $(FATE_DXV)
>
> diff --git a/tests/ref/fate/dxv3-ycg6 b/tests/ref/fate/dxv3-ycg6
> new file mode 100644
> index 0000000000..63c614fbaf
> --- /dev/null
> +++ b/tests/ref/fate/dxv3-ycg6
> @@ -0,0 +1,6 @@
> +#tb 0: 1/30
> +#media_type 0: video
> +#codec_id 0: rawvideo
> +#dimensions 0: 1920x1080
> +#sar 0: 0/1
> +0,          0,          0,        1,  3110400, 0x1e979f00
> diff --git a/tests/ref/fate/dxv3-yg10 b/tests/ref/fate/dxv3-yg10
> new file mode 100644
> index 0000000000..30e122e4d5
> --- /dev/null
> +++ b/tests/ref/fate/dxv3-yg10
> @@ -0,0 +1,6 @@
> +#tb 0: 1/30
> +#media_type 0: video
> +#codec_id 0: rawvideo
> +#dimensions 0: 1920x1080
> +#sar 0: 0/1
> +0,          0,          0,        1,  5184000, 0xe5f0d3da
> --
> 2.40.1
>
>
Vittorio Giovara Jan. 29, 2024, 9:23 a.m. UTC | #2 
On Sun, Jan 28, 2024 at 10:32 PM Connor Worley <connorbworley@gmail.com>
wrote:

> I'd like to get this series merged before doing any further DXV work. Is
> anyone able to help with uploading the linked samples?
>

I unfortunately don't have access to the FATE samples server, can you mail
samples-request@ffmpeg.org?
Andreas Rheinhardt Jan. 29, 2024, 9:58 a.m. UTC | #3 
Connor Worley:
> Adds tests to cover decoding YCoCg DXV3 formats with and without alpha
> 
> Samples:
> https://connorworley.com/fate-suite/dxv/dxv3-hqna.mov
> https://connorworley.com/fate-suite/dxv/dxv3-hqwa.mov
> Signed-off-by: Connor Worley <connorbworley@gmail.com>
> ---
>  tests/fate/video.mak     | 6 ++++++
>  tests/ref/fate/dxv3-ycg6 | 6 ++++++
>  tests/ref/fate/dxv3-yg10 | 6 ++++++
>  3 files changed, 18 insertions(+)
>  create mode 100644 tests/ref/fate/dxv3-ycg6
>  create mode 100644 tests/ref/fate/dxv3-yg10
> 
> diff --git a/tests/fate/video.mak b/tests/fate/video.mak
> index fbc8a1a682..8f51a42077 100644
> --- a/tests/fate/video.mak
> +++ b/tests/fate/video.mak
> @@ -152,6 +152,12 @@ fate-dxv3-dxt1: CMD = framecrc -i $(TARGET_SAMPLES)/dxv/dxv3-nqna.mov
>  FATE_DXV += fate-dxv3-dxt5
>  fate-dxv3-dxt5: CMD = framecrc -i $(TARGET_SAMPLES)/dxv/dxv3-nqwa.mov
>  
> +FATE_DXV += fate-dxv3-ycg6
> +fate-dxv3-ycg6: CMD = framecrc -i $(TARGET_SAMPLES)/dxv/dxv3-hqna.mov
> +
> +FATE_DXV += fate-dxv3-yg10
> +fate-dxv3-yg10: CMD = framecrc -i $(TARGET_SAMPLES)/dxv/dxv3-hqwa.mov
> +
>  FATE_VIDEO-$(call FRAMECRC, MOV, DXV) += $(FATE_DXV)
>  fate-dxv: $(FATE_DXV)
>  
> diff --git a/tests/ref/fate/dxv3-ycg6 b/tests/ref/fate/dxv3-ycg6
> new file mode 100644
> index 0000000000..63c614fbaf
> --- /dev/null
> +++ b/tests/ref/fate/dxv3-ycg6
> @@ -0,0 +1,6 @@
> +#tb 0: 1/30
> +#media_type 0: video
> +#codec_id 0: rawvideo
> +#dimensions 0: 1920x1080
> +#sar 0: 0/1
> +0,          0,          0,        1,  3110400, 0x1e979f00
> diff --git a/tests/ref/fate/dxv3-yg10 b/tests/ref/fate/dxv3-yg10
> new file mode 100644
> index 0000000000..30e122e4d5
> --- /dev/null
> +++ b/tests/ref/fate/dxv3-yg10
> @@ -0,0 +1,6 @@
> +#tb 0: 1/30
> +#media_type 0: video
> +#codec_id 0: rawvideo
> +#dimensions 0: 1920x1080

Why not use a smaller resolution?

> +#sar 0: 0/1
> +0,          0,          0,        1,  5184000, 0xe5f0d3da
Connor Worley Jan. 29, 2024, 6:37 p.m. UTC | #4 
I could generate smaller samples if preferred. I opted to match the
existing NQ ones.

On Mon, Jan 29, 2024 at 1:56 AM Andreas Rheinhardt <
andreas.rheinhardt@outlook.com> wrote:

> Connor Worley:
> > Adds tests to cover decoding YCoCg DXV3 formats with and without alpha
> >
> > Samples:
> > https://connorworley.com/fate-suite/dxv/dxv3-hqna.mov
> > https://connorworley.com/fate-suite/dxv/dxv3-hqwa.mov
> > Signed-off-by: Connor Worley <connorbworley@gmail.com>
> > ---
> >  tests/fate/video.mak     | 6 ++++++
> >  tests/ref/fate/dxv3-ycg6 | 6 ++++++
> >  tests/ref/fate/dxv3-yg10 | 6 ++++++
> >  3 files changed, 18 insertions(+)
> >  create mode 100644 tests/ref/fate/dxv3-ycg6
> >  create mode 100644 tests/ref/fate/dxv3-yg10
> >
> > diff --git a/tests/fate/video.mak b/tests/fate/video.mak
> > index fbc8a1a682..8f51a42077 100644
> > --- a/tests/fate/video.mak
> > +++ b/tests/fate/video.mak
> > @@ -152,6 +152,12 @@ fate-dxv3-dxt1: CMD = framecrc -i
> $(TARGET_SAMPLES)/dxv/dxv3-nqna.mov
> >  FATE_DXV += fate-dxv3-dxt5
> >  fate-dxv3-dxt5: CMD = framecrc -i $(TARGET_SAMPLES)/dxv/dxv3-nqwa.mov
> >
> > +FATE_DXV += fate-dxv3-ycg6
> > +fate-dxv3-ycg6: CMD = framecrc -i $(TARGET_SAMPLES)/dxv/dxv3-hqna.mov
> > +
> > +FATE_DXV += fate-dxv3-yg10
> > +fate-dxv3-yg10: CMD = framecrc -i $(TARGET_SAMPLES)/dxv/dxv3-hqwa.mov
> > +
> >  FATE_VIDEO-$(call FRAMECRC, MOV, DXV) += $(FATE_DXV)
> >  fate-dxv: $(FATE_DXV)
> >
> > diff --git a/tests/ref/fate/dxv3-ycg6 b/tests/ref/fate/dxv3-ycg6
> > new file mode 100644
> > index 0000000000..63c614fbaf
> > --- /dev/null
> > +++ b/tests/ref/fate/dxv3-ycg6
> > @@ -0,0 +1,6 @@
> > +#tb 0: 1/30
> > +#media_type 0: video
> > +#codec_id 0: rawvideo
> > +#dimensions 0: 1920x1080
> > +#sar 0: 0/1
> > +0,          0,          0,        1,  3110400, 0x1e979f00
> > diff --git a/tests/ref/fate/dxv3-yg10 b/tests/ref/fate/dxv3-yg10
> > new file mode 100644
> > index 0000000000..30e122e4d5
> > --- /dev/null
> > +++ b/tests/ref/fate/dxv3-yg10
> > @@ -0,0 +1,6 @@
> > +#tb 0: 1/30
> > +#media_type 0: video
> > +#codec_id 0: rawvideo
> > +#dimensions 0: 1920x1080
>
> Why not use a smaller resolution?
>
> > +#sar 0: 0/1
> > +0,          0,          0,        1,  5184000, 0xe5f0d3da
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>
Connor Worley Jan. 30, 2024, 7:18 p.m. UTC | #5 
Samples have been added, and this series now passes FATE
Andreas Rheinhardt Jan. 30, 2024, 7:31 p.m. UTC | #6 
Connor Worley:
> Samples have been added, and this series now passes FATE

Way too big.

- Andreas
Connor Worley Jan. 30, 2024, 8:53 p.m. UTC | #7 
In that case, it is probably worth replacing the existing "normal quality"
samples as they're even larger.
dxv3-nqwa.mov does not adequately exercise the code fixed in
https://patchwork.ffmpeg.org/project/ffmpeg/patch/20240130062626.98273-1-connorbworley@gmail.com/
I'm not sure what the process for removing samples from fate-suite is.
Thilo Borgmann Jan. 30, 2024, 9:08 p.m. UTC | #8 
Am 30.01.24 um 21:53 schrieb Connor Worley:
> In that case, it is probably worth replacing the existing "normal quality"
> samples as they're even larger.
> dxv3-nqwa.mov does not adequately exercise the code fixed in
> https://patchwork.ffmpeg.org/project/ffmpeg/patch/20240130062626.98273-1-connorbworley@gmail.com/
> I'm not sure what the process for removing samples from fate-suite is.
> 

The new ones are around 75K, not too big IMO.

Can the new samples cover the same use-cases like the old ones as well?
If we don't loose the old coverage, I can delete the unnecessary ones.

-Thilo
Connor Worley Jan. 30, 2024, 9:14 p.m. UTC | #9 
Each sample covers a different texture format, but it may be possible to
generate smaller versions for each case with the same or better coverage.
Andreas Rheinhardt Jan. 30, 2024, 9:25 p.m. UTC | #10 
Connor Worley:
> In that case, it is probably worth replacing the existing "normal quality"
> samples as they're even larger.
> dxv3-nqwa.mov does not adequately exercise the code fixed in
> https://patchwork.ffmpeg.org/project/ffmpeg/patch/20240130062626.98273-1-connorbworley@gmail.com/
> I'm not sure what the process for removing samples from fate-suite is.
> 

We never remove samples from the fate suite if they have been used in a
commit (i.e. if tests with them have been committed). This allows to
checkout any commit and run fate on it. So replacing the old files by
smaller ones is unfortunately impossible.
But the new ones can be replaced; just upload new, smaller ones.

- Andreas
Connor Worley Jan. 30, 2024, 9:57 p.m. UTC | #11 
OK, attached are some significantly smaller HQ samples that produce the
same coverage.

5b11c393dda223b5dd198ad2f2576fe2  fate-suite/dxv/dxv3-hqna.mov
3af30eaba2b6ec68a0f5b2c136f4dab6  fate-suite/dxv/dxv3-hqwa.mov
Thilo Borgmann Feb. 1, 2024, 10:58 a.m. UTC | #12 
Am 30.01.24 um 22:57 schrieb Connor Worley:
> OK, attached are some significantly smaller HQ samples that produce the
> same coverage.
> 
> 5b11c393dda223b5dd198ad2f2576fe2  fate-suite/dxv/dxv3-hqna.mov
> 3af30eaba2b6ec68a0f5b2c136f4dab6  fate-suite/dxv/dxv3-hqwa.mov

5b11c393dda223b5dd198ad2f2576fe2  download/dxv3-hqna.mov
3af30eaba2b6ec68a0f5b2c136f4dab6  download/dxv3-hqwa.mov

Uploaded.

Thanks,
Thilo
diff mbox series

Patch

diff --git a/tests/fate/video.mak b/tests/fate/video.mak
index fbc8a1a682..8f51a42077 100644
--- a/tests/fate/video.mak
+++ b/tests/fate/video.mak
@@ -152,6 +152,12 @@  fate-dxv3-dxt1: CMD = framecrc -i $(TARGET_SAMPLES)/dxv/dxv3-nqna.mov
 FATE_DXV += fate-dxv3-dxt5
 fate-dxv3-dxt5: CMD = framecrc -i $(TARGET_SAMPLES)/dxv/dxv3-nqwa.mov
 
+FATE_DXV += fate-dxv3-ycg6
+fate-dxv3-ycg6: CMD = framecrc -i $(TARGET_SAMPLES)/dxv/dxv3-hqna.mov
+
+FATE_DXV += fate-dxv3-yg10
+fate-dxv3-yg10: CMD = framecrc -i $(TARGET_SAMPLES)/dxv/dxv3-hqwa.mov
+
 FATE_VIDEO-$(call FRAMECRC, MOV, DXV) += $(FATE_DXV)
 fate-dxv: $(FATE_DXV)
 
diff --git a/tests/ref/fate/dxv3-ycg6 b/tests/ref/fate/dxv3-ycg6
new file mode 100644
index 0000000000..63c614fbaf
--- /dev/null
+++ b/tests/ref/fate/dxv3-ycg6
@@ -0,0 +1,6 @@ 
+#tb 0: 1/30
+#media_type 0: video
+#codec_id 0: rawvideo
+#dimensions 0: 1920x1080
+#sar 0: 0/1
+0,          0,          0,        1,  3110400, 0x1e979f00
diff --git a/tests/ref/fate/dxv3-yg10 b/tests/ref/fate/dxv3-yg10
new file mode 100644
index 0000000000..30e122e4d5
--- /dev/null
+++ b/tests/ref/fate/dxv3-yg10
@@ -0,0 +1,6 @@ 
+#tb 0: 1/30
+#media_type 0: video
+#codec_id 0: rawvideo
+#dimensions 0: 1920x1080
+#sar 0: 0/1
+0,          0,          0,        1,  5184000, 0xe5f0d3da