diff mbox series

[FFmpeg-devel,2/5] avcodec/fft_template: Remove unused fixed-point cosine tables

Message ID 20210106231308.2952217-2-andreas.rheinhardt@gmail.com
State Superseded
Headers show
Series [FFmpeg-devel,1/5] avcodec/tableprint: Don't include mem_internal.h
Related show

Checks

Context Check Description
andriy/x86_make success Make finished
andriy/x86_make_fate success Make fate finished
andriy/PPC64_make success Make finished
andriy/PPC64_make_fate success Make fate finished

Commit Message

Andreas Rheinhardt Jan. 6, 2021, 11:13 p.m. UTC
There are three types of FFTs: floating-point, 32-bit fixed-point and
16-bit fixed-point. The latter has exactly one user: The fixed-point
AC-3-encoder; the cosine tables used by it use up to seven bits. The
tables corresponding to eight to seventeen bits are unused, as are the
FFT functions for these bits.

Therefore this commit removes these tables and functions. This is
especially beneficial when using hardcoded tables as they take up more
than 255 KiB. But even without it one saves said unused functions as
well as entries in corresponding tables (this also saves relocations).

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
---
Thee changes to ARM assembly are honstely untested. I hope someone can
test them. Btw: It seems that the ARM assembly code wouldn't be able to
deal with an FFT with more than 16 bits (no function for this has been
defined), which only worked because no one ever used that many bits with
the fixed-point FFT.

 libavcodec/arm/fft_fixed_neon.S | 18 ------------------
 libavcodec/cos_tablegen.c       |  4 ++--
 libavcodec/fft.h                |  4 +++-
 libavcodec/fft_fixed.c          |  1 +
 libavcodec/fft_template.c       | 31 +++++++++++++++++++++++--------
 tests/fate/fft.mak              |  8 ++++++--
 6 files changed, 35 insertions(+), 31 deletions(-)

Comments

Lynne Jan. 7, 2021, 7:54 p.m. UTC | #1
Jan 7, 2021, 00:13 by andreas.rheinhardt@gmail.com:

> There are three types of FFTs: floating-point, 32-bit fixed-point and
> 16-bit fixed-point. The latter has exactly one user: The fixed-point
> AC-3-encoder; the cosine tables used by it use up to seven bits. The
> tables corresponding to eight to seventeen bits are unused, as are the
> FFT functions for these bits.
>
> Therefore this commit removes these tables and functions. This is
> especially beneficial when using hardcoded tables as they take up more
> than 255 KiB. But even without it one saves said unused functions as
> well as entries in corresponding tables (this also saves relocations).
>
> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
> ---
> Thee changes to ARM assembly are honstely untested. I hope someone can
> test them. Btw: It seems that the ARM assembly code wouldn't be able to
> deal with an FFT with more than 16 bits (no function for this has been
> defined), which only worked because no one ever used that many bits with
> the fixed-point FFT.
>
>  libavcodec/arm/fft_fixed_neon.S | 18 ------------------
>  libavcodec/cos_tablegen.c       |  4 ++--
>  libavcodec/fft.h                |  4 +++-
>  libavcodec/fft_fixed.c          |  1 +
>  libavcodec/fft_template.c       | 31 +++++++++++++++++++++++--------
>  tests/fate/fft.mak              |  8 ++++++--
>  6 files changed, 35 insertions(+), 31 deletions(-)
>

The whole comment was hard to make sense of, since you keep mixing
fixed-point FFT precision bits (16 and 32) and FFT length (confusingly also in bits).
I'd rather have a blank comment with just the code or just no references to
the 16-bit fixed-point FFT.

LGTM. Thankfully nothing of the eldritchian fixed-point FFT monstrosity
is exposed to API users, so as long as FATE passes on ARM, should
be okay.
Michael Niedermayer Jan. 10, 2021, 12:40 a.m. UTC | #2
On Thu, Jan 07, 2021 at 12:13:05AM +0100, Andreas Rheinhardt wrote:
> There are three types of FFTs: floating-point, 32-bit fixed-point and
> 16-bit fixed-point. The latter has exactly one user: The fixed-point
> AC-3-encoder; the cosine tables used by it use up to seven bits. The
> tables corresponding to eight to seventeen bits are unused, as are the
> FFT functions for these bits.
> 
> Therefore this commit removes these tables and functions. This is
> especially beneficial when using hardcoded tables as they take up more
> than 255 KiB. But even without it one saves said unused functions as
> well as entries in corresponding tables (this also saves relocations).
> 
> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
> ---
> Thee changes to ARM assembly are honstely untested. I hope someone can
> test them. Btw: It seems that the ARM assembly code wouldn't be able to
> deal with an FFT with more than 16 bits (no function for this has been
> defined), which only worked because no one ever used that many bits with
> the fixed-point FFT.
> 
>  libavcodec/arm/fft_fixed_neon.S | 18 ------------------
>  libavcodec/cos_tablegen.c       |  4 ++--
>  libavcodec/fft.h                |  4 +++-
>  libavcodec/fft_fixed.c          |  1 +
>  libavcodec/fft_template.c       | 31 +++++++++++++++++++++++--------
>  tests/fate/fft.mak              |  8 ++++++--
>  6 files changed, 35 insertions(+), 31 deletions(-)

make -j32 libavcodec/tests/fft-fixed && libavcodec/tests/fft-fixed
Segmentation fault (core dumped)

(if you cant repro say so and ill rebuild with debug symbols ...)

thx
[...]
Andreas Rheinhardt Jan. 10, 2021, 12:56 a.m. UTC | #3
Michael Niedermayer:
> On Thu, Jan 07, 2021 at 12:13:05AM +0100, Andreas Rheinhardt wrote:
>> There are three types of FFTs: floating-point, 32-bit fixed-point and
>> 16-bit fixed-point. The latter has exactly one user: The fixed-point
>> AC-3-encoder; the cosine tables used by it use up to seven bits. The
>> tables corresponding to eight to seventeen bits are unused, as are the
>> FFT functions for these bits.
>>
>> Therefore this commit removes these tables and functions. This is
>> especially beneficial when using hardcoded tables as they take up moreFirst,
>> than 255 KiB. But even without it one saves said unused functions as
>> well as entries in corresponding tables (this also saves relocations).
>>
>> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
>> ---
>> Thee changes to ARM assembly are honstely untested. I hope someone can
>> test them. Btw: It seems that the ARM assembly code wouldn't be able to
>> deal with an FFT with more than 16 bits (no function for this has been
>> defined), which only worked because no one ever used that many bits with
>> the fixed-point FFT.
>>
>>  libavcodec/arm/fft_fixed_neon.S | 18 ------------------
>>  libavcodec/cos_tablegen.c       |  4 ++--
>>  libavcodec/fft.h                |  4 +++-
>>  libavcodec/fft_fixed.c          |  1 +
>>  libavcodec/fft_template.c       | 31 +++++++++++++++++++++++--------
>>  tests/fate/fft.mak              |  8 ++++++--
>>  6 files changed, 35 insertions(+), 31 deletions(-)
> 
> make -j32 libavcodec/tests/fft-fixed && libavcodec/tests/fft-fixed
> Segmentation fault (core dumped)
> 
> (if you cant repro say so and ill rebuild with debug symbols ...)
> 
> thx
> [...]
> 
1. Lynne has an alternative patchset that makes the only user of
fft_fixed use fft_fixed_32 instead, so this is not important any more.
2. Are you testing the ARM assembly code (for which I ask for a test) or
not? If not, then this surprises me. Did you apply the changes to
fft.mak (some of the tests have been removed as they tested
functionality that was unused (apart from the tests) and has therefore
been removed).

- Andreas
Michael Niedermayer Jan. 10, 2021, 1:31 p.m. UTC | #4
On Sun, Jan 10, 2021 at 01:56:21AM +0100, Andreas Rheinhardt wrote:
> Michael Niedermayer:
> > On Thu, Jan 07, 2021 at 12:13:05AM +0100, Andreas Rheinhardt wrote:
> >> There are three types of FFTs: floating-point, 32-bit fixed-point and
> >> 16-bit fixed-point. The latter has exactly one user: The fixed-point
> >> AC-3-encoder; the cosine tables used by it use up to seven bits. The
> >> tables corresponding to eight to seventeen bits are unused, as are the
> >> FFT functions for these bits.
> >>
> >> Therefore this commit removes these tables and functions. This is
> >> especially beneficial when using hardcoded tables as they take up moreFirst,
> >> than 255 KiB. But even without it one saves said unused functions as
> >> well as entries in corresponding tables (this also saves relocations).
> >>
> >> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
> >> ---
> >> Thee changes to ARM assembly are honstely untested. I hope someone can
> >> test them. Btw: It seems that the ARM assembly code wouldn't be able to
> >> deal with an FFT with more than 16 bits (no function for this has been
> >> defined), which only worked because no one ever used that many bits with
> >> the fixed-point FFT.
> >>
> >>  libavcodec/arm/fft_fixed_neon.S | 18 ------------------
> >>  libavcodec/cos_tablegen.c       |  4 ++--
> >>  libavcodec/fft.h                |  4 +++-
> >>  libavcodec/fft_fixed.c          |  1 +
> >>  libavcodec/fft_template.c       | 31 +++++++++++++++++++++++--------
> >>  tests/fate/fft.mak              |  8 ++++++--
> >>  6 files changed, 35 insertions(+), 31 deletions(-)
> > 
> > make -j32 libavcodec/tests/fft-fixed && libavcodec/tests/fft-fixed
> > Segmentation fault (core dumped)
> > 
> > (if you cant repro say so and ill rebuild with debug symbols ...)
> > 
> > thx
> > [...]
> > 
> 1. Lynne has an alternative patchset that makes the only user of
> fft_fixed use fft_fixed_32 instead, so this is not important any more.

> 2. Are you testing the ARM assembly code (for which I ask for a test) or

x86-64

> not? If not, then this surprises me. Did you apply the changes to
> fft.mak (some of the tests have been removed as they tested
> functionality that was unused (apart from the tests) and has therefore
> been removed).

i applied the changes from this patchset up to and including the patch
and also did a make distclean
FFT 512 test
Checking...
==18069== Jump to the invalid address stated on the next line
==18069==    at 0x0: ???
==18069==    by 0x10F5F9: main (fft.c:529)
==18069==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==18069== 
==18069== 
==18069== Process terminating with default action of signal 11 (SIGSEGV)
==18069==  Bad permissions for mapped region at address 0x0
==18069==    at 0x0: ???
==18069==    by 0x10F5F9: main (fft.c:529)

commit 6c532480712d395f5973063adcefce62fc75f2e1 (HEAD)
Author: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Date:   Thu Jan 7 00:13:05 2021 +0100

    avcodec/fft_template: Remove unused fixed-point cosine tables
    
    There are three types of FFTs: floating-point, 32-bit fixed-point and
    16-bit fixed-point. The latter has exactly one user: The fixed-point
    AC-3-encoder; the cosine tables used by it use up to seven bits. The
    tables corresponding to eight to seventeen bits are unused, as are the
    FFT functions for these bits.
    
    Therefore this commit removes these tables and functions. This is
    especially beneficial when using hardcoded tables as they take up more
    than 255 KiB. But even without it one saves said unused functions as
    well as entries in corresponding tables (this also saves relocations).
    
    Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
    Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

 libavcodec/arm/fft_fixed_neon.S | 18 ------------------
 libavcodec/cos_tablegen.c       |  4 ++--
 libavcodec/fft.h                |  4 +++-
 libavcodec/fft_fixed.c          |  1 +
 libavcodec/fft_template.c       | 31 +++++++++++++++++++++++--------
 tests/fate/fft.mak              |  8 ++++++--
 6 files changed, 35 insertions(+), 31 deletions(-)

commit c592684681700a7d8b41e75a11104f8c1bdd13d9
Author: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Date:   Thu Jan 7 00:13:04 2021 +0100

    avcodec/tableprint: Don't include mem_internal.h
    
    tableprint.h does not declare anything as aligned; it just prints
    DECLARE_ALIGNED. So it can be removed; in fact, it needs to be removed,
    because mem_internal.h includes config.h which leads to warnings when
    building with hardcoded tables enabled because of redefinitions of
    CONFIG_HARDCODED_TABLES.
    
    (Furthermore, config.h is only valid for the target, not the host,
    so HAVE_LOCAL_ALIGNED might even be wrong here.)
    
    Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
    Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

 libavcodec/tableprint.h | 1 -
 1 file changed, 1 deletion(-)

commit 91e1625db15fe8853ceedca9eed14307aaa514c7 (origin/master, origin/HEAD, refs/bisect/good-91e1625db15fe8853ceedca9eed14307aaa514c7)

[...]
Andreas Rheinhardt Jan. 10, 2021, 9:40 p.m. UTC | #5
Michael Niedermayer:
> On Sun, Jan 10, 2021 at 01:56:21AM +0100, Andreas Rheinhardt wrote:
>> Michael Niedermayer:
>>> On Thu, Jan 07, 2021 at 12:13:05AM +0100, Andreas Rheinhardt wrote:
>>>> There are three types of FFTs: floating-point, 32-bit fixed-point and
>>>> 16-bit fixed-point. The latter has exactly one user: The fixed-point
>>>> AC-3-encoder; the cosine tables used by it use up to seven bits. The
>>>> tables corresponding to eight to seventeen bits are unused, as are the
>>>> FFT functions for these bits.
>>>>
>>>> Therefore this commit removes these tables and functions. This is
>>>> especially beneficial when using hardcoded tables as they take up moreFirst,
>>>> than 255 KiB. But even without it one saves said unused functions as
>>>> well as entries in corresponding tables (this also saves relocations).
>>>>
>>>> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
>>>> ---
>>>> Thee changes to ARM assembly are honstely untested. I hope someone can
>>>> test them. Btw: It seems that the ARM assembly code wouldn't be able to
>>>> deal with an FFT with more than 16 bits (no function for this has been
>>>> defined), which only worked because no one ever used that many bits with
>>>> the fixed-point FFT.
>>>>
>>>>  libavcodec/arm/fft_fixed_neon.S | 18 ------------------
>>>>  libavcodec/cos_tablegen.c       |  4 ++--
>>>>  libavcodec/fft.h                |  4 +++-
>>>>  libavcodec/fft_fixed.c          |  1 +
>>>>  libavcodec/fft_template.c       | 31 +++++++++++++++++++++++--------
>>>>  tests/fate/fft.mak              |  8 ++++++--
>>>>  6 files changed, 35 insertions(+), 31 deletions(-)
>>>
>>> make -j32 libavcodec/tests/fft-fixed && libavcodec/tests/fft-fixed
>>> Segmentation fault (core dumped)
>>>
>>> (if you cant repro say so and ill rebuild with debug symbols ...)
>>>
>>> thx
>>> [...]
>>>
>> 1. Lynne has an alternative patchset that makes the only user of
>> fft_fixed use fft_fixed_32 instead, so this is not important any more.
> 
>> 2. Are you testing the ARM assembly code (for which I ask for a test) or
> 
> x86-64
> 
>> not? If not, then this surprises me. Did you apply the changes to
>> fft.mak (some of the tests have been removed as they tested
>> functionality that was unused (apart from the tests) and has therefore
>> been removed).
> 
> i applied the changes from this patchset up to and including the patch
> and also did a make distclean
> FFT 512 test

Now I see what's the problem. You are not running the fft-tests from the
FATE-suite (where I disabled unused and newly unsupported tests);
instead you are directly using the underlying tests in libavcodec/tests
and these tests default to nbits = 9, which is unsupported for fft-fixed
with this patch applied. ff_fft_init_fixed is correctly erroring out,
yet said error code is ignored in the test's fft_init (therefore testing
an unsupported nbits already leads to segfaults on master (try
libavcodec/tests/fft -n 20)). So the error checking in the tests needs
to be improved (there are other unchecked allocations, too).
Furthermore, if this patch were to be applied, one should set the
default number of bits for fft-fixed to something supported; but that
point is moot given that this patch has been superseded by Lynne's (who
wants to nuke fft-fixed).

- Andreas
diff mbox series

Patch

diff --git a/libavcodec/arm/fft_fixed_neon.S b/libavcodec/arm/fft_fixed_neon.S
index 2651607544..c94da56f80 100644
--- a/libavcodec/arm/fft_fixed_neon.S
+++ b/libavcodec/arm/fft_fixed_neon.S
@@ -223,15 +223,6 @@  endfunc
         def_fft    32,    16,     8
         def_fft    64,    32,    16
         def_fft   128,    64,    32
-        def_fft   256,   128,    64
-        def_fft   512,   256,   128
-        def_fft  1024,   512,   256
-        def_fft  2048,  1024,   512
-        def_fft  4096,  2048,  1024
-        def_fft  8192,  4096,  2048
-        def_fft 16384,  8192,  4096
-        def_fft 32768, 16384,  8192
-        def_fft 65536, 32768, 16384
 
 function ff_fft_fixed_calc_neon, export=1
         ldr             r2,  [r0]
@@ -249,13 +240,4 @@  const   fft_fixed_tab_neon, relocate=1
         .word fft32_neon
         .word fft64_neon
         .word fft128_neon
-        .word fft256_neon
-        .word fft512_neon
-        .word fft1024_neon
-        .word fft2048_neon
-        .word fft4096_neon
-        .word fft8192_neon
-        .word fft16384_neon
-        .word fft32768_neon
-        .word fft65536_neon
 endconst
diff --git a/libavcodec/cos_tablegen.c b/libavcodec/cos_tablegen.c
index 7206aad5dd..5929c29e1a 100644
--- a/libavcodec/cos_tablegen.c
+++ b/libavcodec/cos_tablegen.c
@@ -26,7 +26,6 @@ 
 
 #include "libavutil/mathematics.h"
 
-#define BITS 17
 #define FLOATFMT "%.18e"
 #define FIXEDFMT "%6d"
 
@@ -56,12 +55,13 @@  int main(int argc, char *argv[])
     int i, j;
     int do_sin = argc > 1 && !strcmp(argv[1], "sin");
     int fixed  = argc > 1 &&  strstr(argv[1], "fixed");
+    int bits   = fixed ? 7 : 17;
     double (*func)(double) = do_sin ? sin : cos;
 
     printf("/* This file was automatically generated. */\n");
     printf("#define FFT_FLOAT %d\n", !fixed);
     printf("#include \"libavcodec/%s\"\n", do_sin ? "rdft.h" : "fft.h");
-    for (i = 4; i <= BITS; i++) {
+    for (i = 4; i <= bits; i++) {
         int m = 1 << i;
         double freq = 2*M_PI/m;
         printf("%s(%i) = {\n   ", do_sin ? "SINTABLE" : "COSTABLE", m);
diff --git a/libavcodec/fft.h b/libavcodec/fft.h
index 5f67b61f06..fedc0c5ef0 100644
--- a/libavcodec/fft.h
+++ b/libavcodec/fft.h
@@ -127,6 +127,7 @@  extern COSTABLE(16);
 extern COSTABLE(32);
 extern COSTABLE(64);
 extern COSTABLE(128);
+#if FFT_FLOAT
 extern COSTABLE(256);
 extern COSTABLE(512);
 extern COSTABLE(1024);
@@ -137,7 +138,8 @@  extern COSTABLE(16384);
 extern COSTABLE(32768);
 extern COSTABLE(65536);
 extern COSTABLE(131072);
-extern COSTABLE_CONST FFTSample* const FFT_NAME(ff_cos_tabs)[18];
+#endif /* FFT_FLOAT */
+extern COSTABLE_CONST FFTSample* const FFT_NAME(ff_cos_tabs)[];
 
 #define ff_init_ff_cos_tabs FFT_NAME(ff_init_ff_cos_tabs)
 
diff --git a/libavcodec/fft_fixed.c b/libavcodec/fft_fixed.c
index 3d3bd2fca6..52d225ee09 100644
--- a/libavcodec/fft_fixed.c
+++ b/libavcodec/fft_fixed.c
@@ -18,4 +18,5 @@ 
 
 #define FFT_FLOAT 0
 #define FFT_FIXED_32 0
+#define MAX_BITS 7
 #include "fft_template.c"
diff --git a/libavcodec/fft_template.c b/libavcodec/fft_template.c
index 8825e39f79..7a7d51a6b4 100644
--- a/libavcodec/fft_template.c
+++ b/libavcodec/fft_template.c
@@ -33,6 +33,10 @@ 
 #include "fft.h"
 #include "fft-internal.h"
 
+#ifndef MAX_BITS
+#define MAX_BITS 17
+#endif
+
 #if FFT_FIXED_32
 #include "fft_table.h"
 #else /* FFT_FIXED_32 */
@@ -43,6 +47,7 @@  COSTABLE(16);
 COSTABLE(32);
 COSTABLE(64);
 COSTABLE(128);
+#if FFT_FLOAT
 COSTABLE(256);
 COSTABLE(512);
 COSTABLE(1024);
@@ -53,6 +58,7 @@  COSTABLE(16384);
 COSTABLE(32768);
 COSTABLE(65536);
 COSTABLE(131072);
+#endif /* FFT_FLOAT */
 
 static av_cold void init_ff_cos_tabs(int index)
 {
@@ -81,6 +87,7 @@  INIT_FF_COS_TABS_FUNC(4, 16)
 INIT_FF_COS_TABS_FUNC(5, 32)
 INIT_FF_COS_TABS_FUNC(6, 64)
 INIT_FF_COS_TABS_FUNC(7, 128)
+#if FFT_FLOAT
 INIT_FF_COS_TABS_FUNC(8, 256)
 INIT_FF_COS_TABS_FUNC(9, 512)
 INIT_FF_COS_TABS_FUNC(10, 1024)
@@ -91,6 +98,7 @@  INIT_FF_COS_TABS_FUNC(14, 16384)
 INIT_FF_COS_TABS_FUNC(15, 32768)
 INIT_FF_COS_TABS_FUNC(16, 65536)
 INIT_FF_COS_TABS_FUNC(17, 131072)
+#endif /* FFT_FLOAT */
 
 static CosTabsInitOnce cos_tabs_init_once[] = {
     { NULL },
@@ -101,6 +109,7 @@  static CosTabsInitOnce cos_tabs_init_once[] = {
     { init_ff_cos_tabs_32, AV_ONCE_INIT },
     { init_ff_cos_tabs_64, AV_ONCE_INIT },
     { init_ff_cos_tabs_128, AV_ONCE_INIT },
+#if FFT_FLOAT
     { init_ff_cos_tabs_256, AV_ONCE_INIT },
     { init_ff_cos_tabs_512, AV_ONCE_INIT },
     { init_ff_cos_tabs_1024, AV_ONCE_INIT },
@@ -111,6 +120,7 @@  static CosTabsInitOnce cos_tabs_init_once[] = {
     { init_ff_cos_tabs_32768, AV_ONCE_INIT },
     { init_ff_cos_tabs_65536, AV_ONCE_INIT },
     { init_ff_cos_tabs_131072, AV_ONCE_INIT },
+#endif /* FFT_FLOAT */
 };
 
 #endif
@@ -120,6 +130,7 @@  COSTABLE_CONST FFTSample * const FFT_NAME(ff_cos_tabs)[] = {
     FFT_NAME(ff_cos_32),
     FFT_NAME(ff_cos_64),
     FFT_NAME(ff_cos_128),
+#if FFT_FLOAT
     FFT_NAME(ff_cos_256),
     FFT_NAME(ff_cos_512),
     FFT_NAME(ff_cos_1024),
@@ -130,6 +141,7 @@  COSTABLE_CONST FFTSample * const FFT_NAME(ff_cos_tabs)[] = {
     FFT_NAME(ff_cos_32768),
     FFT_NAME(ff_cos_65536),
     FFT_NAME(ff_cos_131072),
+#endif /* FFT_FLOAT */
 };
 
 #endif /* FFT_FIXED_32 */
@@ -200,7 +212,7 @@  av_cold int ff_fft_init(FFTContext *s, int nbits, int inverse)
     s->revtab = NULL;
     s->revtab32 = NULL;
 
-    if (nbits < 2 || nbits > 17)
+    if (nbits < 2 || nbits > MAX_BITS)
         goto fail;
     s->nbits = nbits;
     n = 1 << nbits;
@@ -537,11 +549,6 @@  static void name(FFTComplex *z, const FFTSample *wre, unsigned int n)\
 }
 
 PASS(pass)
-#if !CONFIG_SMALL
-#undef BUTTERFLIES
-#define BUTTERFLIES BUTTERFLIES_BIG
-PASS(pass_big)
-#endif
 
 #define DECL_FFT(n,n2,n4)\
 static void fft##n(FFTComplex *z)\
@@ -603,9 +610,13 @@  DECL_FFT(16,8,4)
 DECL_FFT(32,16,8)
 DECL_FFT(64,32,16)
 DECL_FFT(128,64,32)
+#if FFT_FLOAT
 DECL_FFT(256,128,64)
 DECL_FFT(512,256,128)
 #if !CONFIG_SMALL
+#undef BUTTERFLIES
+#define BUTTERFLIES BUTTERFLIES_BIG
+PASS(pass_big)
 #define pass pass_big
 #endif
 DECL_FFT(1024,512,256)
@@ -616,10 +627,14 @@  DECL_FFT(16384,8192,4096)
 DECL_FFT(32768,16384,8192)
 DECL_FFT(65536,32768,16384)
 DECL_FFT(131072,65536,32768)
+#endif /* FFT_FLOAT */
 
 static void (* const fft_dispatch[])(FFTComplex*) = {
-    fft4, fft8, fft16, fft32, fft64, fft128, fft256, fft512, fft1024,
-    fft2048, fft4096, fft8192, fft16384, fft32768, fft65536, fft131072
+    fft4, fft8, fft16, fft32, fft64, fft128,
+#if FFT_FLOAT
+    fft256, fft512, fft1024, fft2048, fft4096,
+    fft8192, fft16384, fft32768, fft65536, fft131072
+#endif /* FFT_FLOAT */
 };
 
 static void fft_calc_c(FFTContext *s, FFTComplex *z)
diff --git a/tests/fate/fft.mak b/tests/fate/fft.mak
index 5da6e687ec..3eb8450d94 100644
--- a/tests/fate/fft.mak
+++ b/tests/fate/fft.mak
@@ -28,15 +28,19 @@  $(FATE_FFT_ALL): CMD = run libavcodec/tests/fft$(EXESUF) $(CPUFLAGS:%=-c%) $(ARG
 
 define DEF_FFT_FIXED
 FATE_FFT_FIXED-$(CONFIG_FFT)   += fate-fft-fixed-$(1)  fate-ifft-fixed-$(1)
-FATE_MDCT_FIXED-$(CONFIG_MDCT) += fate-mdct-fixed-$(1) fate-imdct-fixed-$(1)
 
 fate-fft-fixed-$(1):   ARGS = -n$(1)
 fate-ifft-fixed-$(1):  ARGS = -n$(1) -i
+endef
+define DEF_MDCT_FIXED
+FATE_MDCT_FIXED-$(CONFIG_MDCT) += fate-mdct-fixed-$(1) fate-imdct-fixed-$(1)
+
 fate-mdct-fixed-$(1):  ARGS = -n$(1) -m
 fate-imdct-fixed-$(1): ARGS = -n$(1) -m -i
 endef
 
-$(foreach N, 4 5 6 7 8 9 10 11 12, $(eval $(call DEF_FFT_FIXED,$(N))))
+$(foreach N, 4 5 6 7,     $(eval $(call DEF_FFT_FIXED,$(N))))
+$(foreach N, 4 5 6 7 8 9, $(eval $(call DEF_MDCT_FIXED,$(N))))
 
 fate-fft-fixed: $(FATE_FFT_FIXED-yes)
 fate-mdct-fixed: $(FATE_MDCT_FIXED-yes)