diff mbox series

[FFmpeg-devel,2/2] aacdec: correctly skip padding at the start of frames and during seeking

Message ID NhyzMEL--3-9@lynne.ee
State New
Headers show
Series [FFmpeg-devel,1/2] decode: add ff_decode_skip_samples function | expand

Checks

Context Check Description
andriy/configure_x86 warning Failed to apply patch
yinshiyou/configure_loongarch64 warning Failed to apply patch

Commit Message

Lynne Oct. 30, 2023, 5:10 a.m. UTC 
Presently, our AAC decoder didn't skip any samples, unless the side data
contained skip information.

This uses the newly added ff_decoder_skip_samples function to skip samples
on two levels: base, and additional.
Base is the inherent decoder delay. Most containers already specify it,
hence it is overwritten in that case.
Additional is any extra samples that have to be skipped, in AAC's case,
it's the delay introduced by SBR.

Patch attached.

Comments

Jean-Baptiste Kempf Oct. 30, 2023, 7:38 a.m. UTC | #1 
Hello,

Does this fix #2325 #9667?

jb

On Mon, 30 Oct 2023, at 06:10, Lynne wrote:
> Presently, our AAC decoder didn't skip any samples, unless the side data
> contained skip information.
>
> This uses the newly added ff_decoder_skip_samples function to skip samples
> on two levels: base, and additional.
> Base is the inherent decoder delay. Most containers already specify it,
> hence it is overwritten in that case.
> Additional is any extra samples that have to be skipped, in AAC's case,
> it's the delay introduced by SBR.
>
> Patch attached.
>
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>
> Attachments:
> * 0002-aacdec-correctly-skip-padding-at-the-start-of-frames.patch
Lynne Oct. 30, 2023, 5:03 p.m. UTC | #2 
Oct 30, 2023, 08:38 by jb@videolan.org:

> Hello,
>
> Does this fix #2325 #9667?
>

It fully fixes the following:
 - Decoding of MP4/M4A HE-AAC streams is delayed by 3009 samples
 - Decoding of standalone ADTS AAC files generated by libavcodec's AAC encoder is delayed by 1024 samples

It partially fixes the following:
 - Standalone ADTS AAC files generated by fdk-aac is delayed by 2048 samples (we cut 1024 with this patch)
- Standalone ADTS AAC files generated by apple's encoders is delayed by 2112 samples (we cut 1024 with this patch)

We cannot fix raw ADTS AAC in any way - with this patch, the decoder cuts off exactly the amount of samples demanded by the standard - 1024 for AAC-LC, and 4034 for HE-AAC. If the encoder adds more than this, it's the encoder's fault.

Does it fix "MP4 AAC Audio is delayed by 2ms when converted to PCM"?
Well, first of all, the title is wrong, 00:00:00.02 seconds is not 2 milliseconds, it is 20 milliseconds.
Second of all, it was not broken before? We strip off exactly what the MP4 editlist tells us to.
If the encoder or muxer's editlist is incorrect - we're going to cut off the wrong amount.
We also parse both editlists and itunes SMPB.

Does it fix "HE-AAC (not in mp4) decode samples off by one sample-time"
Yes, through this mechanism, I add a sample back in, which I forgot to do for this patch, but I've fixed it locally, and we're in sync:

Before: https://files.lynne.ee/aache_before.png
After: https://files.lynne.ee/aache_after.png
Original: https://files.lynne.ee/aache_orig.png
diff mbox series

Patch

From 82120460459cf90c330ff11fe0ccf4b954736b5c Mon Sep 17 00:00:00 2001
From: Lynne <dev@lynne.ee>
Date: Mon, 30 Oct 2023 05:40:22 +0100
Subject: [PATCH 2/2] aacdec: correctly skip padding at the start of frames and
 during seeking

Presently, our AAC decoder didn't skip any samples, unless the side data
contained skip information.

This uses the newly added ff_decoder_skip_samples function to skip samples
on two levels: base, and additional.
Base is the inherent decoder delay. Most containers already specify it,
hence it is overwritten in that case.
Additional is any extra samples that have to be skipped, in AAC's case,
it's the delay introduced by SBR.
---
 libavcodec/aac.h             |  1 +
 libavcodec/aacdec_template.c | 24 +++++++++++++++++++-----
 2 files changed, 20 insertions(+), 5 deletions(-)

diff --git a/libavcodec/aac.h b/libavcodec/aac.h
index 285d3b7482..06ae2222cf 100644
--- a/libavcodec/aac.h
+++ b/libavcodec/aac.h
@@ -298,6 +298,7 @@  struct AACContext {
     AVCodecContext *avctx;
     AVFrame *frame;
 
+    int skip_samples;
     int is_saved;                 ///< Set if elements have stored overlap from previous frame.
     DynamicRangeControl che_drc;
 
diff --git a/libavcodec/aacdec_template.c b/libavcodec/aacdec_template.c
index 954399f86b..ab3037c0d5 100644
--- a/libavcodec/aacdec_template.c
+++ b/libavcodec/aacdec_template.c
@@ -532,6 +532,8 @@  static void flush(AVCodecContext *avctx)
             }
         }
     }
+
+    ac->skip_samples = 1;
 }
 
 /**
@@ -1251,6 +1253,7 @@  static av_cold int aac_decode_init(AVCodecContext *avctx)
         return AVERROR(ENOMEM);
     }
 
+    ac->skip_samples = 1;
     ac->random_state = 0x1f2e3d4c;
 
 #define MDCT_INIT(s, fn, len, sval)                                            \
@@ -2419,7 +2422,7 @@  static int decode_dynamic_range(DynamicRangeControl *che_drc,
 
 static int decode_fill(AACContext *ac, GetBitContext *gb, int len) {
     uint8_t buf[256];
-    int i, major, minor;
+    int i;
 
     if (len < 13+7*8)
         goto unknown;
@@ -2433,10 +2436,6 @@  static int decode_fill(AACContext *ac, GetBitContext *gb, int len) {
     if (ac->avctx->debug & FF_DEBUG_PICT_INFO)
         av_log(ac->avctx, AV_LOG_DEBUG, "FILL:%s\n", buf);
 
-    if (sscanf(buf, "libfaac %d.%d", &major, &minor) == 2){
-        ac->avctx->internal->skip_samples = 1024;
-    }
-
 unknown:
     skip_bits_long(gb, len);
 
@@ -3403,6 +3402,21 @@  static int aac_decode_frame(AVCodecContext *avctx, AVFrame *frame,
         if (buf[buf_offset])
             break;
 
+    if (ac->skip_samples) {
+        int additional = 0;
+
+        if (ac->oc[1].m4ac.sbr)
+            additional = 3010;
+
+        err = ff_decode_skip_samples(avctx, frame,
+                                     ac->oc[1].m4ac.frame_length_short ? 960 : 1024,
+                                     additional);
+        if (err < 0)
+            return err;
+
+        ac->skip_samples = 0;
+    }
+
     return buf_size > buf_offset ? buf_consumed : buf_size;
 }
 
-- 
2.42.0