mbox series

[FFmpeg-devel,0/5] 22.2 channel layout support for AAC decoding

Message ID 20200616210253.7169-1-jeebjp@gmail.com
Headers show
Series 22.2 channel layout support for AAC decoding
Related show

Message

Jan Ekström June 16, 2020, 9:02 p.m. UTC
Now that we actually have sample(s) for channel_config 13, it was possible
to see at which various points the AAC decoder fails attempting to parse
the bit stream.

I have not yet found or generated a channel layout test sample so I could
fully validate the channel map output by the AAC decoder, but at least
7/24 seem to be correct looking at the decoded WAV output in Audacity 
(front left/right, center, top and bottom front left/right seem to be
correct based on the sample linked in #8714).

Fixes #8714

Jan Ekström (5):
  avutil/channel_layout: add 22.2 layout
  avcodec/mpeg4audio: add newer channel_coding mappings
  avcodec/aacdectab: add mapping for 22.2
  avcodec/aacdec_template: mark second LFE element as LFE2
  avcodec/aacdec_template: add support for 22.2 / channel_config 13

 doc/APIchanges               |  5 ++++
 libavcodec/aacdec_template.c | 55 ++++++++++++++++++++++++++++++++++--
 libavcodec/aacdectab.h       | 23 +++++++++++++--
 libavcodec/mpeg4audio.c      | 17 +++++++++--
 libavcodec/mpeg4audio.h      |  2 +-
 libavutil/channel_layout.c   |  6 ++++
 libavutil/channel_layout.h   |  6 ++++
 libavutil/version.h          |  2 +-
 8 files changed, 108 insertions(+), 8 deletions(-)

Comments

Lynne July 29, 2020, 11:51 a.m. UTC | #1
Jul 29, 2020, 13:39 by jeebjp@gmail.com:

> Now that we actually have sample(s) for channel_config 13, it was possible
> to see at which various points the AAC decoder fails attempting to parse
> the bit stream.
>
> As the feature `-request_channel_layout 9223372036854775808` (AV_CH_LAYOUT_NATIVE)
> seems to work, the channel order has been validated both visually with
> Audacity, as well as by logging the post-reorder element ordering:
>
> (what specific CPE/SCE/LFE elements mean is documented in the
> `avcodec/aacdectab: add mapping for 22.2` commit):
>
> Setting layout map list after reorder...
> tag 0 = { pos: (null) (3), syn_elem: CPE, elem_id: 1 }
> tag 1 = { pos: FC (4), syn_elem: SCE, elem_id: 0 }
> tag 2 = { pos: LFE (8), syn_elem: LFE, elem_id: 0 }
> tag 3 = { pos: (null) (30), syn_elem: CPE, elem_id: 3 }
> tag 4 = { pos: (null) (c0), syn_elem: CPE, elem_id: 0 }
> tag 5 = { pos: BC (100), syn_elem: SCE, elem_id: 1 }
> tag 6 = { pos: LFE2 (800000000), syn_elem: LFE, elem_id: 1 }
> tag 7 = { pos: (null) (600), syn_elem: CPE, elem_id: 2 }
> tag 8 = { pos: (null) (5000), syn_elem: CPE, elem_id: 4 }
> tag 9 = { pos: TFC (2000), syn_elem: SCE, elem_id: 2 }
> tag 10 = { pos: TC (800), syn_elem: SCE, elem_id: 3 }
> tag 11 = { pos: (null) (28000), syn_elem: CPE, elem_id: 6 }
> tag 12 = { pos: (null) (3000000000), syn_elem: CPE, elem_id: 5 }
> tag 13 = { pos: TBC (10000), syn_elem: SCE, elem_id: 4 }
> tag 14 = { pos: BFC (8000000000), syn_elem: SCE, elem_id: 5 }
> tag 15 = { pos: (null) (14000000000), syn_elem: CPE, elem_id: 7 }
>
> swresample seems to give out the correct channels for 5.1 and stereo
> downmix (checked with ffmpeg.c by utilizing `-channel_layout` 5.1 and stereo),
> although the default downmix logic seems to lower the volume somewhat.
>
> Fixes #8714
>
> Jan Ekström (5):
>  avutil/channel_layout: add 22.2 layout
>  avcodec/mpeg4audio: add newer channel_coding mappings
>  avcodec/aacdectab: add mapping for 22.2
>  avcodec/aacdec_template: mark second LFE element as LFE2
>  avcodec/aacdec_template: add support for 22.2 / channel_config 13
>
>  doc/APIchanges               |  5 ++
>  libavcodec/aacdec_template.c | 98 ++++++++++++++++++++++++++++++++----
>  libavcodec/aacdectab.h       | 23 ++++++++-
>  libavcodec/mpeg4audio.c      | 17 ++++++-
>  libavcodec/mpeg4audio.h      |  2 +-
>  libavutil/channel_layout.c   |  6 +++
>  libavutil/channel_layout.h   |  6 +++
>  libavutil/version.h          |  2 +-
>  8 files changed, 142 insertions(+), 17 deletions(-)
>

Patchset looks good to me.
Jan Ekström July 30, 2020, 3:27 p.m. UTC | #2
On Wed, Jul 29, 2020 at 2:51 PM Lynne <dev@lynne.ee> wrote:
>
> Jul 29, 2020, 13:39 by jeebjp@gmail.com:
>
> > Now that we actually have sample(s) for channel_config 13, it was possible
> > to see at which various points the AAC decoder fails attempting to parse
> > the bit stream.
> >
> > As the feature `-request_channel_layout 9223372036854775808` (AV_CH_LAYOUT_NATIVE)
> > seems to work, the channel order has been validated both visually with
> > Audacity, as well as by logging the post-reorder element ordering:
> >
> > (what specific CPE/SCE/LFE elements mean is documented in the
> > `avcodec/aacdectab: add mapping for 22.2` commit):
> >
> > Setting layout map list after reorder...
> > tag 0 = { pos: (null) (3), syn_elem: CPE, elem_id: 1 }
> > tag 1 = { pos: FC (4), syn_elem: SCE, elem_id: 0 }
> > tag 2 = { pos: LFE (8), syn_elem: LFE, elem_id: 0 }
> > tag 3 = { pos: (null) (30), syn_elem: CPE, elem_id: 3 }
> > tag 4 = { pos: (null) (c0), syn_elem: CPE, elem_id: 0 }
> > tag 5 = { pos: BC (100), syn_elem: SCE, elem_id: 1 }
> > tag 6 = { pos: LFE2 (800000000), syn_elem: LFE, elem_id: 1 }
> > tag 7 = { pos: (null) (600), syn_elem: CPE, elem_id: 2 }
> > tag 8 = { pos: (null) (5000), syn_elem: CPE, elem_id: 4 }
> > tag 9 = { pos: TFC (2000), syn_elem: SCE, elem_id: 2 }
> > tag 10 = { pos: TC (800), syn_elem: SCE, elem_id: 3 }
> > tag 11 = { pos: (null) (28000), syn_elem: CPE, elem_id: 6 }
> > tag 12 = { pos: (null) (3000000000), syn_elem: CPE, elem_id: 5 }
> > tag 13 = { pos: TBC (10000), syn_elem: SCE, elem_id: 4 }
> > tag 14 = { pos: BFC (8000000000), syn_elem: SCE, elem_id: 5 }
> > tag 15 = { pos: (null) (14000000000), syn_elem: CPE, elem_id: 7 }
> >
> > swresample seems to give out the correct channels for 5.1 and stereo
> > downmix (checked with ffmpeg.c by utilizing `-channel_layout` 5.1 and stereo),
> > although the default downmix logic seems to lower the volume somewhat.
> >
> > Fixes #8714
> >
> > Jan Ekström (5):
> >  avutil/channel_layout: add 22.2 layout
> >  avcodec/mpeg4audio: add newer channel_coding mappings
> >  avcodec/aacdectab: add mapping for 22.2
> >  avcodec/aacdec_template: mark second LFE element as LFE2
> >  avcodec/aacdec_template: add support for 22.2 / channel_config 13
> >
> >  doc/APIchanges               |  5 ++
> >  libavcodec/aacdec_template.c | 98 ++++++++++++++++++++++++++++++++----
> >  libavcodec/aacdectab.h       | 23 ++++++++-
> >  libavcodec/mpeg4audio.c      | 17 ++++++-
> >  libavcodec/mpeg4audio.h      |  2 +-
> >  libavutil/channel_layout.c   |  6 +++
> >  libavutil/channel_layout.h   |  6 +++
> >  libavutil/version.h          |  2 +-
> >  8 files changed, 142 insertions(+), 17 deletions(-)
> >
>
> Patchset looks good to me.

Thanks for having a look.

Took quite a bit of time to gather confidence that the channel order
was correct, but given the logging of coding elements as well as
visual comparisons with Audacity I would note that the decoder is now
pushing the correct channel order as specified by the relevant
specifications.

Unless there are additional comments, I will push this a bit later in
the evening, and then proceed to seeing if I can get some FATE
samples/tests made as well as finishing up the debug logging utilized
in the cover letter as I think logging the reordered coding elements
is quite useful when debugging (although it seems to get logged quite
often with LATM AAC, so probably needs some kind of "log only when
things change" kind of thing).

Best regards,
Jan