[FFmpeg-devel,PATCHv2,2/2] fate: use do_md5sum instead of the md5 protocol for most md5 fate tests

Submitted by Marton Balint on June 17, 2017, 6:11 p.m.

Details

Message ID 20170617181110.29258-1-cus@passwd.hu
State New
Headers show

Commit Message

Marton Balint June 17, 2017, 6:11 p.m.
The md5 protocol has no seek support, but some tests use seeks. This changes
the fate tests to actually create the output files and calculate the md5 on the
written files, which also makes the tests independent of the size of the output
buffers and output buffering in general.

A new md5pipe fate test method is also introduced to keep the old functionality
for tests where using a non-seekable output was intentional, and matroska md5
tests are changed to use that.

Signed-off-by: Marton Balint <cus@passwd.hu>
---
 tests/fate-run.sh                                 | 9 ++++++++-
 tests/fate/filter-audio.mak                       | 4 ++--
 tests/fate/matroska.mak                           | 2 +-
 tests/fate/subtitles.mak                          | 2 +-
 tests/fate/wavpack.mak                            | 4 ++--
 tests/ref/fate/mapchan-6ch-extract-2-downmix-mono | 2 +-
 tests/ref/fate/mapchan-silent-mono                | 2 +-
 tests/ref/fate/mpeg4-bsf-unpack-bframes           | 2 +-
 tests/ref/fate/time_base                          | 2 +-
 tests/ref/fate/v410enc                            | 2 +-
 10 files changed, 19 insertions(+), 12 deletions(-)

Comments

Marton Balint June 18, 2017, 7:54 p.m.
On Sat, 17 Jun 2017, Marton Balint wrote:

> The md5 protocol has no seek support, but some tests use seeks. This changes
> the fate tests to actually create the output files and calculate the md5 on the
> written files, which also makes the tests independent of the size of the output
> buffers and output buffering in general.
>
> A new md5pipe fate test method is also introduced to keep the old functionality
> for tests where using a non-seekable output was intentional, and matroska md5
> tests are changed to use that.

Anybody, more comments? I intend to apply this soon.

Thanks,
Marton
James Almer June 18, 2017, 8:04 p.m.
On 6/17/2017 3:11 PM, Marton Balint wrote:
> The md5 protocol has no seek support, but some tests use seeks. This changes
> the fate tests to actually create the output files and calculate the md5 on the
> written files, which also makes the tests independent of the size of the output
> buffers and output buffering in general.
> 
> A new md5pipe fate test method is also introduced to keep the old functionality
> for tests where using a non-seekable output was intentional, and matroska md5
> tests are changed to use that.
> 
> Signed-off-by: Marton Balint <cus@passwd.hu>
> ---
>  tests/fate-run.sh                                 | 9 ++++++++-
>  tests/fate/filter-audio.mak                       | 4 ++--
>  tests/fate/matroska.mak                           | 2 +-
>  tests/fate/subtitles.mak                          | 2 +-
>  tests/fate/wavpack.mak                            | 4 ++--
>  tests/ref/fate/mapchan-6ch-extract-2-downmix-mono | 2 +-
>  tests/ref/fate/mapchan-silent-mono                | 2 +-
>  tests/ref/fate/mpeg4-bsf-unpack-bframes           | 2 +-
>  tests/ref/fate/time_base                          | 2 +-
>  tests/ref/fate/v410enc                            | 2 +-
>  10 files changed, 19 insertions(+), 12 deletions(-)
> 
> diff --git a/tests/fate-run.sh b/tests/fate-run.sh
> index 931150b5a6..74eca69d4d 100755
> --- a/tests/fate-run.sh
> +++ b/tests/fate-run.sh
> @@ -142,10 +142,17 @@ crc(){
>      ffmpeg "$@" -f crc -
>  }
>  
> -md5(){
> +md5pipe(){
>      ffmpeg "$@" md5:
>  }
>  
> +md5(){
> +    encfile="${outdir}/${test}.out"
> +    cleanfiles=$encfile

Shouldn't this be cleanfiles="$cleanfiles $encfile"? I see some other
functions don't bother to append stuff to cleanfiles and overwrite it
instead, so not sure.

Should be ok otherwise. Thanks for keeping the pipe functionality for
the matroska tests.
Marton Balint June 18, 2017, 9:52 p.m.
On Sun, 18 Jun 2017, James Almer wrote:

> On 6/17/2017 3:11 PM, Marton Balint wrote:
>> The md5 protocol has no seek support, but some tests use seeks. This changes
>> the fate tests to actually create the output files and calculate the md5 on the
>> written files, which also makes the tests independent of the size of the output
>> buffers and output buffering in general.
>> 
>> A new md5pipe fate test method is also introduced to keep the old functionality
>> for tests where using a non-seekable output was intentional, and matroska md5
>> tests are changed to use that.
>> 
>> Signed-off-by: Marton Balint <cus@passwd.hu>
>> ---
>>  tests/fate-run.sh                                 | 9 ++++++++-
>>  tests/fate/filter-audio.mak                       | 4 ++--
>>  tests/fate/matroska.mak                           | 2 +-
>>  tests/fate/subtitles.mak                          | 2 +-
>>  tests/fate/wavpack.mak                            | 4 ++--
>>  tests/ref/fate/mapchan-6ch-extract-2-downmix-mono | 2 +-
>>  tests/ref/fate/mapchan-silent-mono                | 2 +-
>>  tests/ref/fate/mpeg4-bsf-unpack-bframes           | 2 +-
>>  tests/ref/fate/time_base                          | 2 +-
>>  tests/ref/fate/v410enc                            | 2 +-
>>  10 files changed, 19 insertions(+), 12 deletions(-)
>> 
>> diff --git a/tests/fate-run.sh b/tests/fate-run.sh
>> index 931150b5a6..74eca69d4d 100755
>> --- a/tests/fate-run.sh
>> +++ b/tests/fate-run.sh
>> @@ -142,10 +142,17 @@ crc(){
>>      ffmpeg "$@" -f crc -
>>  }
>> 
>> -md5(){
>> +md5pipe(){
>>      ffmpeg "$@" md5:
>>  }
>> 
>> +md5(){
>> +    encfile="${outdir}/${test}.out"
>> +    cleanfiles=$encfile
>
> Shouldn't this be cleanfiles="$cleanfiles $encfile"? I see some other
> functions don't bother to append stuff to cleanfiles and overwrite it
> instead, so not sure.

Probably does not matter, maybe appending it is more future proof, if 
somebody wants to invoke it from another function. I changed it locally.

>
> Should be ok otherwise. Thanks for keeping the pipe functionality for
> the matroska tests.

Applied, thanks.

Regards,
Marton

Patch hide | download patch | download mbox

diff --git a/tests/fate-run.sh b/tests/fate-run.sh
index 931150b5a6..74eca69d4d 100755
--- a/tests/fate-run.sh
+++ b/tests/fate-run.sh
@@ -142,10 +142,17 @@  crc(){
     ffmpeg "$@" -f crc -
 }
 
-md5(){
+md5pipe(){
     ffmpeg "$@" md5:
 }
 
+md5(){
+    encfile="${outdir}/${test}.out"
+    cleanfiles=$encfile
+    ffmpeg "$@" $encfile
+    do_md5sum $encfile | awk '{print $1}'
+}
+
 pcm(){
     ffmpeg "$@" -vn -f s16le -
 }
diff --git a/tests/fate/filter-audio.mak b/tests/fate/filter-audio.mak
index 5d15b31e0b..8030641ed6 100644
--- a/tests/fate/filter-audio.mak
+++ b/tests/fate/filter-audio.mak
@@ -215,7 +215,7 @@  fate-filter-channelmap-one-int: SRC = $(TARGET_PATH)/tests/data/asynth-44100-6.w
 fate-filter-channelmap-one-int: tests/data/asynth-44100-6.wav
 fate-filter-channelmap-one-int: CMD = md5 -i $(SRC) -filter_complex_script $(TARGET_PATH)/tests/data/filtergraphs/channelmap_one_int -f wav -fflags +bitexact
 fate-filter-channelmap-one-int: CMP = oneline
-fate-filter-channelmap-one-int: REF = 428b8f9fac6d57147069b97335019ef5
+fate-filter-channelmap-one-int: REF = 8cfe553d65ed4696756d8c1b824fcdd3
 
 FATE_FILTER_CHANNELMAP += fate-filter-channelmap-one-str
 fate-filter-channelmap-one-str: tests/data/filtergraphs/channelmap_one_str
@@ -223,7 +223,7 @@  fate-filter-channelmap-one-str: SRC = $(TARGET_PATH)/tests/data/asynth-44100-2.w
 fate-filter-channelmap-one-str: tests/data/asynth-44100-2.wav
 fate-filter-channelmap-one-str: CMD = md5 -i $(SRC) -filter_complex_script $(TARGET_PATH)/tests/data/filtergraphs/channelmap_one_str -f wav -fflags +bitexact
 fate-filter-channelmap-one-str: CMP = oneline
-fate-filter-channelmap-one-str: REF = e788890db6a11c2fb29d7c4229072d49
+fate-filter-channelmap-one-str: REF = 0ea3052e482c95d5d3bd9da6dac1b5fa
 
 FATE_AFILTER-$(call FILTERDEMDECENCMUX, CHANNELMAP, WAV, PCM_S16LE, PCM_S16LE, WAV) += $(FATE_FILTER_CHANNELMAP)
 
diff --git a/tests/fate/matroska.mak b/tests/fate/matroska.mak
index 48b710c26c..2747496e1e 100644
--- a/tests/fate/matroska.mak
+++ b/tests/fate/matroska.mak
@@ -2,7 +2,7 @@ 
 # properties in remuxing (-c:v copy)
 # It also tests automatic insertion of the vp9_superframe bitstream filter
 FATE_MATROSKA-$(call DEMMUX, MATROSKA, MATROSKA) += fate-matroska-remux
-fate-matroska-remux: CMD = md5 -i $(TARGET_SAMPLES)/vp9-test-vectors/vp90-2-2pass-akiyo.webm -color_trc 4 -c:v copy -fflags +bitexact -strict -2 -f matroska
+fate-matroska-remux: CMD = md5pipe -i $(TARGET_SAMPLES)/vp9-test-vectors/vp90-2-2pass-akiyo.webm -color_trc 4 -c:v copy -fflags +bitexact -strict -2 -f matroska
 fate-matroska-remux: CMP = oneline
 fate-matroska-remux: REF = 1ed49a4f2b6790357fac268938357353
 
diff --git a/tests/fate/subtitles.mak b/tests/fate/subtitles.mak
index fc2b192e5d..11cbb82b24 100644
--- a/tests/fate/subtitles.mak
+++ b/tests/fate/subtitles.mak
@@ -14,7 +14,7 @@  FATE_SUBTITLES_ASS-$(CONFIG_ASS_DEMUXER) += fate-sub-ssa-to-ass-remux
 fate-sub-ssa-to-ass-remux: CMD = fmtstdout ass -i $(TARGET_SAMPLES)/sub/a9-misc.ssa -c copy
 
 FATE_SUBTITLES-$(call ALLYES, ASS_DEMUXER, MATROSKA_MUXER) += fate-binsub-mksenc
-fate-binsub-mksenc: CMD = md5 -i $(TARGET_SAMPLES)/sub/1ededcbd7b.ass -c copy -f matroska -flags +bitexact -fflags +bitexact
+fate-binsub-mksenc: CMD = md5pipe -i $(TARGET_SAMPLES)/sub/1ededcbd7b.ass -c copy -f matroska -flags +bitexact -fflags +bitexact
 
 FATE_SUBTITLES_ASS-$(call DEMDEC, JACOSUB, JACOSUB) += fate-sub-jacosub
 fate-sub-jacosub: CMD = fmtstdout ass -i $(TARGET_SAMPLES)/sub/JACOsub_capability_tester.jss
diff --git a/tests/fate/wavpack.mak b/tests/fate/wavpack.mak
index 32ae3f6100..e4e8d47128 100644
--- a/tests/fate/wavpack.mak
+++ b/tests/fate/wavpack.mak
@@ -89,12 +89,12 @@  FATE_WAVPACK-$(call DEMDEC, MATROSKA, WAVPACK) += fate-wavpack-matroskamode
 fate-wavpack-matroskamode: CMD = md5 -i $(TARGET_SAMPLES)/wavpack/special/matroska_mode.mka -f s16le
 
 FATE_WAVPACK-$(call DEMMUX, WV, MATROSKA) += fate-wavpack-matroska_mux-mono
-fate-wavpack-matroska_mux-mono: CMD = md5 -i $(TARGET_SAMPLES)/wavpack/num_channels/mono_16bit_int.wv -c copy -fflags +bitexact -f matroska
+fate-wavpack-matroska_mux-mono: CMD = md5pipe -i $(TARGET_SAMPLES)/wavpack/num_channels/mono_16bit_int.wv -c copy -fflags +bitexact -f matroska
 fate-wavpack-matroska_mux-mono: CMP = oneline
 fate-wavpack-matroska_mux-mono: REF = 11773e2a518edc788475f3880d849230
 
 FATE_WAVPACK-$(call DEMMUX, WV, MATROSKA) += fate-wavpack-matroska_mux-61
-fate-wavpack-matroska_mux-61: CMD = md5 -i $(TARGET_SAMPLES)/wavpack/num_channels/eva_2.22_6.1_16bit-partial.wv -c copy -fflags +bitexact -f matroska
+fate-wavpack-matroska_mux-61: CMD = md5pipe -i $(TARGET_SAMPLES)/wavpack/num_channels/eva_2.22_6.1_16bit-partial.wv -c copy -fflags +bitexact -f matroska
 fate-wavpack-matroska_mux-61: CMP = oneline
 fate-wavpack-matroska_mux-61: REF = 9641abdf596c10c2e21bd9b026d4bade
 
diff --git a/tests/ref/fate/mapchan-6ch-extract-2-downmix-mono b/tests/ref/fate/mapchan-6ch-extract-2-downmix-mono
index a47c85cd6b..9ac4dba261 100644
--- a/tests/ref/fate/mapchan-6ch-extract-2-downmix-mono
+++ b/tests/ref/fate/mapchan-6ch-extract-2-downmix-mono
@@ -1 +1 @@ 
-2727e5418eb5b8f89954f140d9f3a19a
+6ededcfc9f0b26d200a7295205e87663
diff --git a/tests/ref/fate/mapchan-silent-mono b/tests/ref/fate/mapchan-silent-mono
index f720c318f4..93f519e0ee 100644
--- a/tests/ref/fate/mapchan-silent-mono
+++ b/tests/ref/fate/mapchan-silent-mono
@@ -1 +1 @@ 
-16dc73c037897dff9b1d3e3d8561dc73
+b6b034fffa8639e64d79be84f16d9ebe
diff --git a/tests/ref/fate/mpeg4-bsf-unpack-bframes b/tests/ref/fate/mpeg4-bsf-unpack-bframes
index 162d436625..703ae3a8e8 100644
--- a/tests/ref/fate/mpeg4-bsf-unpack-bframes
+++ b/tests/ref/fate/mpeg4-bsf-unpack-bframes
@@ -1 +1 @@ 
-c9535e459c2ee4ead6d84b93bc7e9f46
+a0c4ba21ab9d68966b8fce7cc848e2c6
diff --git a/tests/ref/fate/time_base b/tests/ref/fate/time_base
index eec4b21aa5..7923556b35 100644
--- a/tests/ref/fate/time_base
+++ b/tests/ref/fate/time_base
@@ -1 +1 @@ 
-ef9896ce045941607782e895aea930b6
+d26a35b141551b36c5b8bd716451cfcb
diff --git a/tests/ref/fate/v410enc b/tests/ref/fate/v410enc
index 139da7b875..9fddf5a35c 100644
--- a/tests/ref/fate/v410enc
+++ b/tests/ref/fate/v410enc
@@ -1 +1 @@ 
-5fd2d9a7b3311f5c19dbdd647bb9eae6
+465bcc7477104a8295f47b35f1b987df