diff mbox

[FFmpeg-devel] af_hdcd: more FATE tests

Message ID 20161218074226.22778-1-pburt0@gmail.com
State Superseded
Headers show

Commit Message

Burt P Dec. 18, 2016, 7:42 a.m. UTC 
Additional/Modified FATE tests improve code coverage from 63.7% to 98.1%.

Changed fate-suite sample files:
* filter/hdcd-encoding-errors.flac (1.3M) replaced by
  filter/hdcd-encoding-errors2.flac (140K)
* filter/hdcd-mix.flac (2.2M) added
* filter/hdcd-fake20bit.flac (168K) added
* filter/hdcd.flac (910K) removed, although it was a nice tune and
  the samples that replace it are terrible sounds.
Net change +290K.

Signed-off-by: Burt P <pburt0@gmail.com>
---
 tests/fate/filter-audio.mak | 86 ++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 74 insertions(+), 12 deletions(-)

Comments

Hendrik Leppkes Dec. 18, 2016, 9:30 a.m. UTC | #1 
On Sun, Dec 18, 2016 at 8:42 AM, Burt P <pburt0@gmail.com> wrote:
> Additional/Modified FATE tests improve code coverage from 63.7% to 98.1%.
>
> Changed fate-suite sample files:
> * filter/hdcd-encoding-errors.flac (1.3M) replaced by
>   filter/hdcd-encoding-errors2.flac (140K)
> * filter/hdcd-mix.flac (2.2M) added
> * filter/hdcd-fake20bit.flac (168K) added
> * filter/hdcd.flac (910K) removed, although it was a nice tune and
>   the samples that replace it are terrible sounds.
> Net change +290K.
>

We cannot delete samples because that would break old versions of FATE
(which is vital for bisecting etc), so knowing and accounting for
that, would that change any of these choices to add/remove samples?
Because the net change goes to around +2.5M without any removals.

- Hendrik
Burt P Dec. 18, 2016, 11:05 a.m. UTC | #2 
On Sun, Dec 18, 2016 at 3:30 AM, Hendrik Leppkes <h.leppkes@gmail.com> wrote:
> On Sun, Dec 18, 2016 at 8:42 AM, Burt P <pburt0@gmail.com> wrote:
>> Additional/Modified FATE tests improve code coverage from 63.7% to 98.1%.
>>
>> Changed fate-suite sample files:
>> * filter/hdcd-encoding-errors.flac (1.3M) replaced by
>>   filter/hdcd-encoding-errors2.flac (140K)
>> * filter/hdcd-mix.flac (2.2M) added
>> * filter/hdcd-fake20bit.flac (168K) added
>> * filter/hdcd.flac (910K) removed, although it was a nice tune and
>>   the samples that replace it are terrible sounds.
>> Net change +290K.
>>
>
> We cannot delete samples because that would break old versions of FATE
> (which is vital for bisecting etc), so knowing and accounting for
> that, would that change any of these choices to add/remove samples?
> Because the net change goes to around +2.5M without any removals.
>

Well, hdcd-encoding-errors2.flac could simply be named
hdcd-encoding-errors.flac and it should not affect the test that uses it,
even in an old version. There only needs to be a non-zero number of
errors, 2 is as good as 4.

The sample hdcd.flac is a nice sound, but it isn't a very interesting
example of HDCD for testing. hdcd-mix.flac is a concatenation of three
different samples, each one interesting in a different way, and it does
quite a bit for code coverage. I will, however, try to make it smaller.
diff mbox

Patch

diff --git a/tests/fate/filter-audio.mak b/tests/fate/filter-audio.mak
index 9c6f7cd..a55e3c9 100644
--- a/tests/fate/filter-audio.mak
+++ b/tests/fate/filter-audio.mak
@@ -255,17 +255,55 @@  fate-filter-volume: CMD = md5 -i $(SRC) -af aperms=random,volume=precision=fixed
 fate-filter-volume: CMP = oneline
 fate-filter-volume: REF = 4d6ba75ef3e32d305d066b9bc771d6f4
 
-FATE_AFILTER_SAMPLES-$(call FILTERDEMDECENCMUX, HDCD, FLAC, FLAC, PCM_S24LE, PCM_S24LE) += fate-filter-hdcd
-fate-filter-hdcd: SRC = $(TARGET_SAMPLES)/filter/hdcd.flac
-fate-filter-hdcd: CMD = md5 -i $(SRC) -af hdcd -f s24le
-fate-filter-hdcd: CMP = oneline
-fate-filter-hdcd: REF = 5db465a58d2fd0d06ca944b883b33476
-
-FATE_AFILTER_SAMPLES-$(call FILTERDEMDECENCMUX, HDCD, FLAC, FLAC, PCM_S24LE, PCM_S24LE) += fate-filter-hdcd-analyze
-fate-filter-hdcd-analyze: SRC = $(TARGET_SAMPLES)/filter/hdcd.flac
-fate-filter-hdcd-analyze: CMD = md5 -i $(SRC) -af hdcd=analyze_mode=pe -f s24le
-fate-filter-hdcd-analyze: CMP = oneline
-fate-filter-hdcd-analyze: REF = 6e39dc4629c1e84321c0d8bc069b42f6
+# hdcd-mix.flac is a mix of three different sources which are interesting for various reasons:
+# first 5 seconds uses packet format A and max LLE of -7.0db
+# second 5 seconds uses packet format B and has a gain mismatch between channels
+# last 10 seconds is not HDCD but has a coincidental HDCD packet, it needs to be 10 seconds because it also tests the cdt expiration
+FATE_AFILTER_SAMPLES-$(call FILTERDEMDECENCMUX, HDCD, FLAC, FLAC, PCM_S24LE, PCM_S24LE) += fate-filter-hdcd-mix
+fate-filter-hdcd-mix: SRC = $(TARGET_SAMPLES)/filter/hdcd-mix.flac
+fate-filter-hdcd-mix: CMD = md5 -i $(SRC) -af hdcd -f s24le
+fate-filter-hdcd-mix: CMP = oneline
+fate-filter-hdcd-mix: REF = 6a3cf7f920f419477ada264cc63b40da
+
+# output will be different because of the gain mismatch in the second and third parts
+FATE_AFILTER_SAMPLES-$(call FILTERDEMDECENCMUX, HDCD, FLAC, FLAC, PCM_S24LE, PCM_S24LE) += fate-filter-hdcd-mix-psoff
+fate-filter-hdcd-mix-psoff: SRC = $(TARGET_SAMPLES)/filter/hdcd-mix.flac
+fate-filter-hdcd-mix-psoff: CMD = md5 -i $(SRC) -af hdcd=process_stereo=false -f s24le
+fate-filter-hdcd-mix-psoff: CMP = oneline
+fate-filter-hdcd-mix-psoff: REF = b841866d5730852256ca57564c55e0ef
+
+# test the different analyze modes
+FATE_AFILTER_SAMPLES-$(call FILTERDEMDECENCMUX, HDCD, FLAC, FLAC, PCM_S24LE, PCM_S24LE) += fate-filter-hdcd-analyze-pe
+fate-filter-hdcd-analyze-pe: SRC = $(TARGET_SAMPLES)/filter/hdcd-mix.flac
+fate-filter-hdcd-analyze-pe: CMD = md5 -i $(SRC) -af hdcd=analyze_mode=pe -f s24le
+fate-filter-hdcd-analyze-pe: CMP = oneline
+fate-filter-hdcd-analyze-pe: REF = 9ddd10dfea756160456e25dd96a752b8
+FATE_AFILTER_SAMPLES-$(call FILTERDEMDECENCMUX, HDCD, FLAC, FLAC, PCM_S24LE, PCM_S24LE) += fate-filter-hdcd-analyze-lle
+fate-filter-hdcd-analyze-lle: SRC = $(TARGET_SAMPLES)/filter/hdcd-mix.flac
+fate-filter-hdcd-analyze-lle: CMD = md5 -i $(SRC) -af hdcd=analyze_mode=lle -f s24le
+fate-filter-hdcd-analyze-lle: CMP = oneline
+fate-filter-hdcd-analyze-lle: REF = be353f79d3e653d658a6e6e99d7655c8
+FATE_AFILTER_SAMPLES-$(call FILTERDEMDECENCMUX, HDCD, FLAC, FLAC, PCM_S24LE, PCM_S24LE) += fate-filter-hdcd-analyze-cdt
+fate-filter-hdcd-analyze-cdt: SRC = $(TARGET_SAMPLES)/filter/hdcd-mix.flac
+fate-filter-hdcd-analyze-cdt: CMD = md5 -i $(SRC) -af hdcd=analyze_mode=cdt -f s24le
+fate-filter-hdcd-analyze-cdt: CMP = oneline
+fate-filter-hdcd-analyze-cdt: REF = d828abe932e0d2bfc914eaa23c15b7f6
+FATE_AFILTER_SAMPLES-$(call FILTERDEMDECENCMUX, HDCD, FLAC, FLAC, PCM_S24LE, PCM_S24LE) += fate-filter-hdcd-analyze-tgm
+fate-filter-hdcd-analyze-tgm: SRC = $(TARGET_SAMPLES)/filter/hdcd-mix.flac
+fate-filter-hdcd-analyze-tgm: CMD = md5 -i $(SRC) -af hdcd=analyze_mode=tgm -f s24le
+fate-filter-hdcd-analyze-tgm: CMP = oneline
+fate-filter-hdcd-analyze-tgm: REF = 407ed0dc8c6fd17cc8c0b53a8b2b0e34
+# the two additional analyze modes from libhdcd
+FATE_AFILTER_SAMPLES-$(call FILTERDEMDECENCMUX, HDCD, FLAC, FLAC, PCM_S24LE, PCM_S24LE) += fate-filter-hdcd-analyze-ltgm
+fate-filter-hdcd-analyze-ltgm: SRC = $(TARGET_SAMPLES)/filter/hdcd-mix.flac
+fate-filter-hdcd-analyze-ltgm: CMD = md5 -i $(SRC) -af hdcd=analyze_mode=lle:process_stereo=false -f s24le
+fate-filter-hdcd-analyze-ltgm: CMP = oneline
+fate-filter-hdcd-analyze-ltgm: REF = 276a354519a15d1c83c5ca394fe5dffc
+FATE_AFILTER_SAMPLES-$(call FILTERDEMDECENCMUX, HDCD, FLAC, FLAC, PCM_S24LE, PCM_S24LE) += fate-filter-hdcd-analyze-pel
+fate-filter-hdcd-analyze-pel: SRC = $(TARGET_SAMPLES)/filter/hdcd-mix.flac
+fate-filter-hdcd-analyze-pel: CMD = md5 -i $(SRC) -af hdcd=analyze_mode=pe:force_pe=true -f s24le
+fate-filter-hdcd-analyze-pel: CMP = oneline
+fate-filter-hdcd-analyze-pel: REF = 959b10db640f0eceddaed6743b529ae8
 
 FATE_AFILTER_SAMPLES-$(call FILTERDEMDECENCMUX, HDCD, FLAC, FLAC, PCM_S24LE, PCM_S24LE) += fate-filter-hdcd-false-positive
 fate-filter-hdcd-false-positive: SRC = $(TARGET_SAMPLES)/filter/hdcd-false-positive.flac
@@ -274,11 +312,35 @@  fate-filter-hdcd-false-positive: CMP = grep
 fate-filter-hdcd-false-positive: REF = HDCD detected: no
 
 FATE_AFILTER_SAMPLES-$(call FILTERDEMDECENCMUX, HDCD, FLAC, FLAC, PCM_S24LE, PCM_S24LE) += fate-filter-hdcd-detect-errors
-fate-filter-hdcd-detect-errors: SRC = $(TARGET_SAMPLES)/filter/hdcd-encoding-errors.flac
+fate-filter-hdcd-detect-errors: SRC = $(TARGET_SAMPLES)/filter/hdcd-encoding-errors2.flac
 fate-filter-hdcd-detect-errors: CMD = md5 -i $(SRC) -af hdcd -f s24le
 fate-filter-hdcd-detect-errors: CMP = grep
 fate-filter-hdcd-detect-errors: REF = detectable errors: [1-9]
 
+# 20bit HDCD
+FATE_AFILTER_SAMPLES-$(call FILTERDEMDECENCMUX, HDCD, FLAC, FLAC, PCM_S32LE, PCM_S32LE) += fate-filter-hdcd-20bit
+fate-filter-hdcd-20bit: SRC = $(TARGET_SAMPLES)/filter/hdcd-fake20bit.flac
+fate-filter-hdcd-20bit: CMD = md5 -i $(SRC) -af hdcd=bits_per_sample=20 -f s32le
+fate-filter-hdcd-20bit: CMP = oneline
+fate-filter-hdcd-20bit: REF = 365ded883a4a92483b15b69babc81390
+
+# non-hdcd tests of different input formats for code coverage
+FATE_AFILTER_SAMPLES-$(call FILTERDEMDECENCMUX, HDCD, WAV, PCM_S16LE, PCM_S24LE, PCM_S24LE) += fate-filter-hdcd-mono
+fate-filter-hdcd-mono: SRC = $(TARGET_SAMPLES)/audiomatch/tones_44100_mono.wav
+fate-filter-hdcd-mono: CMD = md5 -i $(SRC) -af hdcd -f s24le
+fate-filter-hdcd-mono: CMP = oneline
+fate-filter-hdcd-mono: REF = f51b114b20728e6a463a9491c643d166
+FATE_AFILTER_SAMPLES-$(call FILTERDEMDECENCMUX, HDCD, WV, WAVPACK, PCM_S32LE, PCM_S32LE) += fate-filter-hdcd-s16p
+fate-filter-hdcd-s16p: SRC = $(TARGET_SAMPLES)/wavpack/lossless/16bit-partial.wv
+fate-filter-hdcd-s16p: CMD = md5 -i $(SRC) -af hdcd -f s32le
+fate-filter-hdcd-s16p: CMP = oneline
+fate-filter-hdcd-s16p: REF = 4e767f436b891ac59810a8b2b1d7e96b
+FATE_AFILTER_SAMPLES-$(call FILTERDEMDECENCMUX, HDCD, WV, WAVPACK, PCM_S32LE, PCM_S32LE) += fate-filter-hdcd-s32p
+fate-filter-hdcd-s32p: SRC = $(TARGET_SAMPLES)/wavpack/lossless/24bit-partial.wv
+fate-filter-hdcd-s32p: CMD = md5 -i $(SRC) -af hdcd -f s32le
+fate-filter-hdcd-s32p: CMP = oneline
+fate-filter-hdcd-s32p: REF = 0c5513e83eedaa10ab6fac9ddc173cf5
+
 FATE_AFILTER-yes += fate-filter-formats
 fate-filter-formats: libavfilter/tests/formats$(EXESUF)
 fate-filter-formats: CMD = run libavfilter/tests/formats