mbox series

[FFmpeg-devel,v2,000/162] VLC, esp. init_vlc patches

Message ID 20201120072116.818090-1-andreas.rheinhardt@gmail.com
Headers show
Series VLC, esp. init_vlc patches | expand

Message

Andreas Rheinhardt Nov. 20, 2020, 7:18 a.m. UTC 
This is the second version of my init_vlc patchset [1]. Given the
feedback I received for the last version I stressed thread-safety of
init functions of decoders. As a consequence, the amount of decoders
with FF_CODEC_CAP_INIT_THREADSAFE increased by 25 (some of them were
already thread-safe, just the flag was missing). I am not finished yet*,
but I don't see a reason to postpone sending this patchset already.

It now saves 146KiB from the binary (and much more when using hardcoded
tables).

[1]: https://ffmpeg.org/pipermail/ffmpeg-devel/2020-November/272127.html

Two patches of this patchset (#55 and #161) change MIPS code; this is
untested as I don't have a MIPS. Would be nice if someone could test it. 

*: AAC needs much more work (e.g. these decoders claim to be
init-threadsafe, but aren't).

Andreas Rheinhardt (162):
  avcodec/bitstream: Add second function to create VLCs
  avcodec/tscc2: Reduce the size of the tables used to initialize VLCs
  avcodec/tscc2: Combine tables for initializing VLCs
  avcodec/bitstream: Allow static VLC tables to be bigger than needed
  avcodec/tscc2: Mark tscc2 decoder as init-threadsafe
  avcodec/tscc2: Make VLC tables static
  avcodec/cllc: Improve creating VLCs
  avcodec/smacker: Improve creating Huffman VLC tables
  avcodec/smacker: Mark decoders as init-threadsafe
  avcodec/on2avc: Use smaller tables for VLCs
  avcodec/on2avcdata: Combine tables for codebooks
  avcodec/on2avc: Apply offset when initializing VLC
  avcodec/imc: Make Huffman tables smaller
  avcodec/imc: Avoid offsets table when creating VLCs
  avcodec/clearvideo: Avoid code tables for initializing VLCs
  avcodec/clearvideo: Avoid code duplication when initializing VLCs
  avcodec/clearvideo: Avoid huge VLC length tables
  avcodec/clearvideo: Improve handling of VLC escape values
  avcodec/clearvideo: Inline constants
  avcodec/clearvideo: Use minimal max_depth in get_vlc2()
  avcodec/clearvideo: Apply VLC offset during init
  avcodec/clearvideo: Make VLC tables static
  avcodec/wnv1: Make array for initializing VLC smaller
  avcodec/wnv1: Apply offset during init, not later every time
  avcodec/wnv1: Make decoder init-threadsafe
  avcodec/cook: Make tables to initialize VLCs smaller
  avcodec/cook: Apply offset when initializing VLC table
  avcodec/cook: Avoid big length tables for VLC initialization
  avcodec/cook: Inline constants
  avcodec/rv10: Make initializing static RV10 VLCs thread-safe
  avcodec/rv10: Reduce the size of the tables used to initialize VLCs
  avcodec/rv10: Reduce number of exceptions when reading VLC value
  avcodec/rv10: Use symbol table more effectively
  avcodec/rv10: Make VLC tables smaller
  avcodec/rv10: Simplify handling of skip VLC entries
  avcodec/mimic: Reduce size of tables used to initialize VLCs
  avcodec/mimic: Cleanup generically upon init failure
  avcodec/mimic: Mark decoder as init-threadsafe
  avcodec/mimic: Make VLC static
  avcodec/motionpixels: Only create VLC iff it is going to be used
  avcodec/motionpixels: Be more strict when parsing Huffman trees
  avcodec/motionpixels: Don't check for complete VLC
  avcodec/motionpixels: Use symbols table
  avcodec/motionpixels: Simplify creating VLC tables
  avcodec/motionpixels: Make decoder init-threadsafe
  avcodec/mpc7: Reduce size of tables used to initialize VLCs
  avcodec/mpc7: Apply offsets when creating VLCs
  avcodec/mpc7: Avoid offsets table when creating VLCs
  avcodec/mpc8: Reduce size of tables used to initialize VLCs
  avcodec/mpc8: Apply offsets when initializing VLCs
  avcodec/mpc8: Reduce the size of the length tables to initialize VLCs
  avcodec/mpc8: Reduce the size of some VLCs
  avcodec/mpc8: Avoid code duplication when initializing VLCs
  avcodec/mpegaudiodsp: Combine initializing float and int tables
  avcodec/fft_template, fft_init_table: Make ff_fft_init() thread-safe
  avcodec/imc: Make imc/iac decoders init-threadsafe
  avcodec/mpegaudiodsp: Make ff_mpadsp_init() thread-safe
  avcodec/mpegaudiodsp: Make initializing synth windows thread-safe
  avcodec/mpc7: Make decoder init-threadsafe
  avcodec/mpc8: Make decoder init-threadsafe
  avcodec/mobiclip: Avoid redundant codes table to initialize VLCs
  avcodec/mobiclip: Avoid code duplication when initializing VLCs
  avcodec/mobiclip: Make decoder init-threadsafe
  avcodec/mobiclip: Reindentation
  avcodec/qdm2: Make tables used to initialize VLCs smaller
  avcodec/qdm2: Apply offsets when initializing VLCs
  avcodec/qdm2: Initialize array of VLCs in a loop
  avcodec/qdm2: Avoid offsets table when initializing VLCs
  avcodec/qdm2: Make decoder init-threadsafe
  avcodec/qdm2: Remove outdated comment
  avcodec/rv40vlc2: Make VLC smaller
  avcodec/rv40: Make the tables used to initialize VLCs smaller
  avcodec/rv40: Make better use of VLC symbols table
  avcodec/rv40: Avoid offset table when initializing static VLCs
  avcodec/rv40: Avoid code duplication when initializing VLCs
  avcodec/mss4: Simplify creating VLC tables
  avcodec/mss4: Don't duplicate standard JPEG tables
  avcodec/mss4: Mark decoder as init-threadsafe
  avcodec/mss4: Make VLCs static
  avcodec/sheervideo: Add av_cold to build_vlc()
  avcodec/sheervideo: Simplify creating VLC table
  avcodec/g2meet: Use least max_depth in get_vlc2()
  avcodec/mjpegdec: Simplify creating VLC table
  avcodec/indeo2: Make tables used to initialize VLCs smaller
  avcodec/indeo2: Make decoder thread-safe
  avcodec/mpeg4video: Make tables used to initialize VLCs smaller
  avcodec/mpeg4videodec: Make studio VLCs static
  avcodec/truemotion2: Simplify creating VLC table
  avcodec/truemotion2: Mark decoder as init-threadsafe
  avcodec/atrac3plus: Simplify creating VLCs
  avcodec/atrac3plus: Make tables used to initialize VLCs smaller
  avcodec/atrac3plus: Simplify getting offset of VLC in VLC_TYPE buf
  avcodec/atrac3plus: Combine symbols tables
  avcodec/atrac3plus: Combine codebooks into one array
  avcodec/atrac3plus: Run-length encode length tables to make them
    smaller
  avcodec/vc1: Already offset subblock VLC tables during init
  avcodec/vc1: Use symbols table for VLC for Block-level Transform Type
  avcodec/vc1: Use a symbols table to decode interlaced MV vector VLCs
  avcodec/vc1_block, msmpeg4dec: Don't check for errors for complete
    VLCs
  avcodec/vc1_block: Don't use 0 for NULL
  avcodec/vc1: Use symbols table to decode AC coefficients
  avcodec/vc1_block: Use local variables inside macro
  avcodec/vc1: Use symbols table to decode MV differentials
  avcodec/vc1: Use symbols table to parse macroblock mode
  avcodec/vc1: Don't check for errors for complete VLC
  avcodec/vc1: Make tables used to initialize VLCs smaller
  avcodec/vc1data: Remove declaration of inexistent array
  avcodec/vc1: Make ff_vc1_init_common() thread-safe
  avcodec/vc1: Avoid code duplication when initializing VLCs
  avcodec/vc1: Reindentation
  avcodec/rv34: Avoid offsets table for initialization of static VLCs
  avcodec/rv34: Make initializing static VLC tables thread-safe
  avcodec/atrac9dec: Don't confuse max_depth of VLC with max codelength
  avcodec/atrac9dec: Use least max_depth in calls to get_vlc2()
  avcodec/atrac9dec: Don't use unnecessarily large VLC tables
  avcodec/atrac9dec: Make tables used to initialize VLCs smaller
  avcodec/atrac9tab: Unify tables used to initialize VLCs
  avcodec/atrac9dec: Don't create VLCs that are never used
  avcodec/atrac9dec: Make VLCs static
  avcodec/atrac9dec: Replace av_free() by av_freep() in close function
  avcodec/atrac9tab: Add missing static to internal table
  avcodec/vp3: Apply VLC offset during init
  avcodec/vp3: Make tables used to initialize VLCs smaller
  avcodec/vp3: Use symbols table for VP3 motion vectors
  avcodec/vp3: Remove code duplication when initializing Theora VLCs
  avcodec/intrax8: Replace always-false check by assert
  avcodec/intrax8: Remove VLC offsets table
  avcodec/intrax8: Avoid code duplication when initializing VLCs
  avcodec/intrax8: Reduce the size of tables used to initialize VLCs
  avcodec/intrax8: Make ff_intrax8_common_init() thread-safe
  avcodec/atrac3: Use symbols table
  avcodec/hqx: Inline constants
  avcodec/hqx: Don't check for errors for complete VLCs
  avcodec/utvideodec: Avoid implicit qsort when creating Huffman tables
  avcodec/magicyuv: Optimize creating Huffman tables
  avcodec/qdmc: Don't check for errors for complete VLCs
  avcodec/qdmc: Make tables used to initialize VLCs smaller
  avcodec/qdmc: Apply offset when creating VLC
  avcodec/qdmc: Avoid code duplication when initializing VLC
  avcodec/aacdec_template: Avoid code duplication when initializing VLCs
  avcodec/aacdec_template: Use VLC symbols table
  avcodec/aactab: Fix conflicting types for ff_aac_codebook*
  avcodec/aac*: Make initializing ff_aac_pow*sf_tab thread-safe
  avcodec/imm4: Don't use too big VLC table
  avcodec/imm4: Make better use of symbols table
  avcodec/imm4: Reduce the size of tables used to initialize VLCs
  avcodec/imm4: Inline constants
  avcodec/mv30: Don't check for errors for complete VLCs
  avcodec/mv30: Reduce the size of tables used to initialize VLCs
  avcodec/mv30: Inline constants
  avcodec/mpegaudiodec_template: Don't use unnecessarily many VLC bits
  avcodec/mpegaudiodec_template: Avoid VLC size tables
  avcodec/mpegaudiotab: Avoid unused entry in table
  avcodec/mpegaudiodec: Reduce the size of tables used to initialize
    VLCs
  avcodec/mpegaudiodec: Combine tables used to initialize VLCs
  avcodec/mpegaudiodec: Make decoders init-threadsafe
  avcodec/mpegaudiodec_template: Apply shift during init
  avcodec/mpegaudiodec: Share fixed and floating point data and init
    code
  avcodec/mpegaudio_tablegen: Avoid write-only buffers
  avcodec/mpegaudio_tablegen: Don't inappropriately use static array
  avcodec/mpegaudiodec: Hardcode tables to save space
  avcodec/mpegaudio_tablegen: Make exponential LUT shared

 configure                                 |    2 +-
 libavcodec/Makefile                       |    7 +-
 libavcodec/aacdec_template.c              |   47 +-
 libavcodec/aacenc.c                       |   12 +-
 libavcodec/aactab.c                       |   52 +
 libavcodec/aactab.h                       |   52 +-
 libavcodec/atrac3.c                       |   18 +-
 libavcodec/atrac3data.h                   |  100 +-
 libavcodec/atrac3plus.c                   |  159 +-
 libavcodec/atrac3plus_data.h              | 1674 ++---
 libavcodec/atrac9dec.c                    |  126 +-
 libavcodec/atrac9tab.h                    | 2128 +++---
 libavcodec/bitstream.c                    |  138 +-
 libavcodec/clearvideo.c                   |  286 +-
 libavcodec/clearvideodata.h               | 2446 ++-----
 libavcodec/cllc.c                         |   14 +-
 libavcodec/cook.c                         |   45 +-
 libavcodec/cookdata.h                     |  626 +-
 libavcodec/fft_init_table.c               |   24 +-
 libavcodec/fft_table.h                    |    2 +-
 libavcodec/fft_template.c                 |   12 +-
 libavcodec/g2meet.c                       |    4 +-
 libavcodec/hqx.c                          |    6 +-
 libavcodec/hqx.h                          |    1 +
 libavcodec/hqxvlc.c                       |    2 +-
 libavcodec/imc.c                          |   38 +-
 libavcodec/imcdata.h                      |   66 +-
 libavcodec/imm4.c                         |  102 +-
 libavcodec/indeo2.c                       |   16 +-
 libavcodec/indeo2data.h                   |   67 +-
 libavcodec/intrax8.c                      |  100 +-
 libavcodec/intrax8huf.h                   | 1674 ++---
 libavcodec/ituh263dec.c                   |    2 +-
 libavcodec/magicyuv.c                     |   39 +-
 libavcodec/mimic.c                        |   82 +-
 libavcodec/mips/compute_antialias_fixed.h |    3 +-
 libavcodec/mips/compute_antialias_float.h |    2 +-
 libavcodec/mips/fft_mips.c                |    4 +-
 libavcodec/mjpegdec.c                     |   23 +-
 libavcodec/mobiclip.c                     |  254 +-
 libavcodec/motionpixels.c                 |   45 +-
 libavcodec/motionpixels_tablegen.h        |    3 +-
 libavcodec/mpc.c                          |    5 -
 libavcodec/mpc.h                          |    1 -
 libavcodec/mpc7.c                         |   73 +-
 libavcodec/mpc7data.h                     |  160 +-
 libavcodec/mpc8.c                         |  170 +-
 libavcodec/mpc8huff.h                     |  678 +-
 libavcodec/mpeg4data.h                    |  150 +-
 libavcodec/mpeg4video.h                   |   12 +-
 libavcodec/mpeg4videodec.c                |   93 +-
 libavcodec/mpegaudio_tablegen.c           |    3 +-
 libavcodec/mpegaudio_tablegen.h           |   40 +-
 libavcodec/mpegaudiodata.h                |   39 +
 libavcodec/mpegaudiodec_common.c          |  482 ++
 libavcodec/mpegaudiodec_common_tablegen.c |   40 +
 libavcodec/mpegaudiodec_common_tablegen.h |   72 +
 libavcodec/mpegaudiodec_fixed.c           |   22 +-
 libavcodec/mpegaudiodec_float.c           |   34 +-
 libavcodec/mpegaudiodec_template.c        |  197 +-
 libavcodec/mpegaudiodectab.h              |  615 --
 libavcodec/mpegaudiodsp.c                 |   57 +-
 libavcodec/mpegaudiodsp.h                 |    8 +-
 libavcodec/mpegaudiodsp_template.c        |   50 +-
 libavcodec/msmpeg4dec.c                   |    5 -
 libavcodec/mss4.c                         |  130 +-
 libavcodec/mv30.c                         |   15 +-
 libavcodec/on2avc.c                       |   29 +-
 libavcodec/on2avcdata.c                   | 8116 ++++++---------------
 libavcodec/on2avcdata.h                   |   10 +-
 libavcodec/qdm2.c                         |   19 +-
 libavcodec/qdm2_tablegen.h                |  165 +-
 libavcodec/qdm2data.h                     |  248 +-
 libavcodec/qdmc.c                         |  143 +-
 libavcodec/rv10.c                         |  255 +-
 libavcodec/rv34.c                         |   73 +-
 libavcodec/rv40.c                         |   85 +-
 libavcodec/rv40vlc2.h                     | 1139 ++-
 libavcodec/sheervideo.c                   |   15 +-
 libavcodec/smacker.c                      |   49 +-
 libavcodec/truemotion2.c                  |   26 +-
 libavcodec/tscc2.c                        |   79 +-
 libavcodec/tscc2data.h                    | 1269 +---
 libavcodec/utvideodec.c                   |   42 +-
 libavcodec/vc1.c                          |  424 +-
 libavcodec/vc1_block.c                    |  165 +-
 libavcodec/vc1acdata.h                    |  174 -
 libavcodec/vc1data.c                      | 1851 ++---
 libavcodec/vc1data.h                      |  156 +-
 libavcodec/vlc.h                          |   49 +
 libavcodec/vp3.c                          |  103 +-
 libavcodec/vp3data.h                      | 3147 ++------
 libavcodec/vp4data.h                      | 1536 ++--
 libavcodec/wnv1.c                         |   32 +-
 libavcodec/x86/mpegaudiodsp.c             |    9 +-
 95 files changed, 11525 insertions(+), 21537 deletions(-)
 create mode 100644 libavcodec/mpegaudiodec_common.c
 create mode 100644 libavcodec/mpegaudiodec_common_tablegen.c
 create mode 100644 libavcodec/mpegaudiodec_common_tablegen.h
 delete mode 100644 libavcodec/mpegaudiodectab.h

Comments

Andreas Rheinhardt Nov. 23, 2020, 10:54 p.m. UTC | #1 
Andreas Rheinhardt:
> This is the second version of my init_vlc patchset [1]. Given the
> feedback I received for the last version I stressed thread-safety of
> init functions of decoders. As a consequence, the amount of decoders
> with FF_CODEC_CAP_INIT_THREADSAFE increased by 25 (some of them were
> already thread-safe, just the flag was missing). I am not finished yet*,
> but I don't see a reason to postpone sending this patchset already.
> 
> It now saves 146KiB from the binary (and much more when using hardcoded
> tables).
> 
> [1]: https://ffmpeg.org/pipermail/ffmpeg-devel/2020-November/272127.html
> 
> Two patches of this patchset (#55 and #161) change MIPS code; this is
> untested as I don't have a MIPS. Would be nice if someone could test it. 
> 
> *: AAC needs much more work (e.g. these decoders claim to be
> init-threadsafe, but aren't).
> 
Will apply the patches that don't depend on ff_init_vlc_from_lengths()
tomorrow, namely:
avcodec/mpegaudiodec_template: Don't use unnecessarily many VLC bits
avcodec/mv30: Don't check for errors for complete VLCs
avcodec/imm4: Don't use too big VLC table
avcodec/aac*: Make initializing ff_aac_pow*sf_tab thread-safe
avcodec/aactab: Fix conflicting types for ff_aac_codebook*
avcodec/qdmc: Don't check for errors for complete VLCs
avcodec/hqx: Don't check for errors for complete VLCs
avcodec/hqx: Inline constants
avcodec/intrax8: Make ff_intrax8_common_init() thread-safe
avcodec/intrax8: Replace always-false check by assert
avcodec/atrac9tab: Add missing static to internal table
avcodec/atrac9dec: Replace av_free() by av_freep() in close function
avcodec/atrac9dec: Don't use unnecessarily large VLC tables
avcodec/atrac9dec: Use least max_depth in calls to get_vlc2()
avcodec/atrac9dec: Don't confuse max_depth of VLC with max codelength
avcodec/truemotion2: Mark decoder as init-threadsafe
avcodec/g2meet: Use least max_depth in get_vlc2()
avcodec/sheervideo: Add av_cold to build_vlc()
avcodec/mss4: Mark decoder as init-threadsafe
avcodec/rv40vlc2: Make VLC smaller
avcodec/qdm2: Remove outdated comment
avcodec/mpegaudiodsp: Make initializing synth windows thread-safe
avcodec/mpegaudiodsp: Make ff_mpadsp_init() thread-safe
avcodec/fft_template, fft_init_table: Make ff_fft_init() thread-safe
avcodec/mpegaudiodsp: Combine initializing float and int tables
avcodec/motionpixels: Make decoder init-threadsafe
avcodec/motionpixels: Use symbols table
avcodec/motionpixels: Don't check for errors for complete VLC
avcodec/motionpixels: Be more strict when parsing Huffman trees
avcodec/motionpixels: Only create VLC iff it is going to be used
avcodec/mimic: Mark decoder as init-threadsafe
avcodec/mimic: Cleanup generically upon init failure
avcodec/rv10: Make initializing static RV10 VLCs thread-safe
avcodec/smacker: Mark decoders as init-threadsafe
avcodec/tscc2: Mark tscc2 decoder as init-threadsafe

- Andreas
Andreas Rheinhardt Dec. 6, 2020, 7:04 p.m. UTC | #2 
Andreas Rheinhardt:
> This is the second version of my init_vlc patchset [1]. Given the
> feedback I received for the last version I stressed thread-safety of
> init functions of decoders. As a consequence, the amount of decoders
> with FF_CODEC_CAP_INIT_THREADSAFE increased by 25 (some of them were
> already thread-safe, just the flag was missing). I am not finished yet*,
> but I don't see a reason to postpone sending this patchset already.
> 
> It now saves 146KiB from the binary (and much more when using hardcoded
> tables).
> 
> [1]: https://ffmpeg.org/pipermail/ffmpeg-devel/2020-November/272127.html
> 
> Two patches of this patchset (#55 and #161) change MIPS code; this is
> untested as I don't have a MIPS. Would be nice if someone could test it. 
> 
> *: AAC needs much more work (e.g. these decoders claim to be
> init-threadsafe, but aren't).
> 

Will apply this set tomorrow (with the exception of VC-1, where I think
I missed a slight optimization) unless there are objections.

- Andreas