From patchwork Wed Jan 19 20:38:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Martin_Storsj=C3=B6?= X-Patchwork-Id: 33664 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp5149746iog; Wed, 19 Jan 2022 12:39:01 -0800 (PST) X-Google-Smtp-Source: ABdhPJxAhPUe6CTJ2S6hoSYSsRsTUpZ7KU05MlNK7dcG6Pdl3XRe4nirRXm6dIPjWIhofKXQiGpk X-Received: by 2002:a05:6402:1a53:: with SMTP id bf19mr27195655edb.21.1642624740940; Wed, 19 Jan 2022 12:39:00 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1642624740; cv=none; d=google.com; s=arc-20160816; b=mp2FqqJebFbWcNbDDUTCV5OsgIKRBdPIsWa356FwWyvbXsFo8OoQHbaFZEaG0kabrp bEaQuY4i7enMkQrsQcT5XS/6GSGt98yrZVDVfZcWi/vUT8+30f1hv484zCmAsFyF3/oi h5iyC/pu9F/dN67JmryB45/gUbIubeKGiwwGWFF4ShQOuOasLiJRq4IXhQfl2DYLqyKR mPcTIZ60OyigXCcS5GiD+ipsZD9zVgcmvtHkJqen4wNxd4AEUemL7rGnY+htPYuDpL6n Dud0JoUyEBMl/x3nb0JsQh6yHoKqOHsAvYfC5JPBBc3hRmYZoDo1rNgYMvh6hbFGe95h 2qEQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=OJH56zut8cQrsp3XKDT1sTmLuk29Yim2lYwcKNVzmtM=; b=QLrBpz6gLy1MlFWIv+slmgCw314YiTggjS8xihwrKsmiO5Bg98K9N+/p3BdsPUURQS fhQp5bqWT8s2epSdb+TeUV2oUrvGION7bXkwUzNQNlwkzi2HEkfHJw7VsqTQ+ZxOAHgS xjmR+HTPDedIgdHZU4qEOsppFpZgtK4+2GS01nww3EoQbkduPBabYoxw5l8h+DJgQ9ff vhZUD33shb8kExRVvZ/zUq2Z1NF7chq3MYw0oTCAFKREw/VrueVwGVqKIFPxRvUMDqy7 UP3kToEdhns0pTsa497njGMKDnj3wo+aPZK2njibqrNjmVfPiPdnDrOQVEjcTrgGA5vg pe7w== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@martin-st.20210112.gappssmtp.com header.s=20210112 header.b=rIbULCbn; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id sc19si479827ejc.814.2022.01.19.12.39.00; Wed, 19 Jan 2022 12:39:00 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@martin-st.20210112.gappssmtp.com header.s=20210112 header.b=rIbULCbn; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id CF73A68B187; Wed, 19 Jan 2022 22:38:47 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f47.google.com (mail-wm1-f47.google.com [209.85.128.47]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D83A268B180 for ; Wed, 19 Jan 2022 22:38:40 +0200 (EET) Received: by mail-wm1-f47.google.com with SMTP id w26so7682358wmi.0 for ; Wed, 19 Jan 2022 12:38:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=martin-st.20210112.gappssmtp.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=4dMTFodMlVVVM+nQAT504S+g2hULqW976GjNoK2fK6Y=; b=rIbULCbnSFVy5LTqAkmTYBqrncTlpfPVf/29Ic5uY7Wb7Z+nuYVfUyXLEGVhF5Zn/1 3LweFZX3jYONZlQjJQDTUPmAv1u/TEazlQJjZwgBuNfAI/QjMgQmr/R4LVW8PsglpRn0 Pen7gVGv6TIXifJmssrJv25YlSJ/6MzD53wkOtiKawzCjRhGCPM1NVuoKr5o43dIV8g1 ecC0ATnPErITYBFKY0Yblx6kASTv1bMl71/8fnq6yaqcGN7HIcBFvhWbtzrdQ91U2Rso WjwC4IEwgWVsHcM0Pr2owLM4MGrHOm9rYtrzaNvBLnAs5UIWxg69SiaEIIx0uA91jWG6 g5gA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4dMTFodMlVVVM+nQAT504S+g2hULqW976GjNoK2fK6Y=; b=KDD46aoU78NpPpEOdY3mNwaoeNPaozOO7hYevNeMSvX7YNYYjLtyiEyIAS8BzLP9XG PBVHubmvDnZY5OapJcOYJOqIXgKHI9+XO32izEcicXGh+MTcJrDuU60lGMUgEbA1Rexf 9yag9qMXZYe6U7HMouPHEykUlWfgLc7veyRVX/pMbcoDmGyNcbw7pY8QPxl9B/IfPkBq KjSbXSrhuKo6HM1kHtwTTfEV9wHbk7sdgJ74G4wLU+uB1PM9lf02nkjpPs91JknWyvv8 830noWDFC4OEVHjmelAdCNw9wO68s1HXc569QEgfi7pvpJlfehqwrJoduN4xfUBAi17N VOjg== X-Gm-Message-State: AOAM5329ty+QWb/3gIczzF5eQyOccxs6JlEEzzgOmlC4wWPdkzlPdyXU l4x/UoBdCpV81qQYsqI6RjYaLf0kjjoy9bP5 X-Received: by 2002:adf:e44f:: with SMTP id t15mr22474453wrm.0.1642624720236; Wed, 19 Jan 2022 12:38:40 -0800 (PST) Received: from localhost (host-97-187.parnet.fi. [77.234.97.187]) by smtp.gmail.com with ESMTPSA id s9sm503779wmh.35.2022.01.19.12.38.39 for (version=TLS1 cipher=AES128-SHA bits=128/128); Wed, 19 Jan 2022 12:38:39 -0800 (PST) From: =?utf-8?q?Martin_Storsj=C3=B6?= To: ffmpeg-devel@ffmpeg.org Date: Wed, 19 Jan 2022 22:38:37 +0200 Message-Id: <20220119203837.9047-2-martin@martin.st> X-Mailer: git-send-email 2.32.0 (Apple Git-132) In-Reply-To: <20220119203837.9047-1-martin@martin.st> References: <20220119203837.9047-1-martin@martin.st> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/2] libfdk-aacdec: Flush delayed samples at the end X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: U0uqJUcI7XmC Also trim off delay samples at the start instead of adjusting pts to compensate for them; this avoids unwanted offsets if working with raw samples without considering their pts. --- libavcodec/libfdk-aacdec.c | 80 +++++++++++++++++++++++++++++++------- 1 file changed, 65 insertions(+), 15 deletions(-) diff --git a/libavcodec/libfdk-aacdec.c b/libavcodec/libfdk-aacdec.c index 93b52023b0..d560e313ca 100644 --- a/libavcodec/libfdk-aacdec.c +++ b/libavcodec/libfdk-aacdec.c @@ -58,7 +58,11 @@ typedef struct FDKAACDecContext { int drc_cut; int album_mode; int level_limit; - int output_delay; +#if FDKDEC_VER_AT_LEAST(2, 5) // 2.5.10 + int output_delay_set; + int flush_samples; + int delay_samples; +#endif } FDKAACDecContext; @@ -123,7 +127,12 @@ static int get_stream_info(AVCodecContext *avctx) avctx->sample_rate = info->sampleRate; avctx->frame_size = info->frameSize; #if FDKDEC_VER_AT_LEAST(2, 5) // 2.5.10 - s->output_delay = info->outputDelay; + if (!s->output_delay_set && info->outputDelay) { + // Set this only once. + s->flush_samples = info->outputDelay; + s->delay_samples = info->outputDelay; + s->output_delay_set = 1; + } #endif for (i = 0; i < info->numChannels; i++) { @@ -367,14 +376,31 @@ static int fdk_aac_decode_frame(AVCodecContext *avctx, void *data, int ret; AAC_DECODER_ERROR err; UINT valid = avpkt->size; + UINT flags = 0; + int input_offset = 0; - err = aacDecoder_Fill(s->handle, &avpkt->data, &avpkt->size, &valid); - if (err != AAC_DEC_OK) { - av_log(avctx, AV_LOG_ERROR, "aacDecoder_Fill() failed: %x\n", err); - return AVERROR_INVALIDDATA; + if (avpkt->size) { + err = aacDecoder_Fill(s->handle, &avpkt->data, &avpkt->size, &valid); + if (err != AAC_DEC_OK) { + av_log(avctx, AV_LOG_ERROR, "aacDecoder_Fill() failed: %x\n", err); + return AVERROR_INVALIDDATA; + } + } else { +#if FDKDEC_VER_AT_LEAST(2, 5) // 2.5.10 + /* Handle decoder draining */ + if (s->flush_samples > 0) { + flags |= AACDEC_FLUSH; + } else { + return AVERROR_EOF; + } +#else + return AVERROR_EOF; +#endif } - err = aacDecoder_DecodeFrame(s->handle, (INT_PCM *) s->decoder_buffer, s->decoder_buffer_size / sizeof(INT_PCM), 0); + err = aacDecoder_DecodeFrame(s->handle, (INT_PCM *) s->decoder_buffer, + s->decoder_buffer_size / sizeof(INT_PCM), + flags); if (err == AAC_DEC_NOT_ENOUGH_BITS) { ret = avpkt->size - valid; goto end; @@ -390,16 +416,36 @@ static int fdk_aac_decode_frame(AVCodecContext *avctx, void *data, goto end; frame->nb_samples = avctx->frame_size; +#if FDKDEC_VER_AT_LEAST(2, 5) // 2.5.10 + if (flags & AACDEC_FLUSH) { + // Only return the right amount of samples at the end; if calling the + // decoder with AACDEC_FLUSH, it will keep returning frames indefinitely. + frame->nb_samples = FFMIN(s->flush_samples, frame->nb_samples); + av_log(s, AV_LOG_DEBUG, "Returning %d/%d delayed samples.\n", + frame->nb_samples, s->flush_samples); + s->flush_samples -= frame->nb_samples; + } else { + // Trim off samples from the start to compensate for extra decoder + // delay. We could also just adjust the pts, but this avoids + // including the extra samples in the output altogether. + if (s->delay_samples) { + int drop_samples = FFMIN(s->delay_samples, frame->nb_samples); + av_log(s, AV_LOG_DEBUG, "Dropping %d/%d delayed samples.\n", + drop_samples, s->delay_samples); + s->delay_samples -= drop_samples; + frame->nb_samples -= drop_samples; + input_offset = drop_samples * avctx->channels; + if (frame->nb_samples <= 0) + return 0; + } + } +#endif + if ((ret = ff_get_buffer(avctx, frame, 0)) < 0) goto end; - if (frame->pts != AV_NOPTS_VALUE) - frame->pts -= av_rescale_q(s->output_delay, - (AVRational){1, avctx->sample_rate}, - avctx->time_base); - - memcpy(frame->extended_data[0], s->decoder_buffer, - avctx->channels * avctx->frame_size * + memcpy(frame->extended_data[0], s->decoder_buffer + input_offset, + avctx->channels * frame->nb_samples * av_get_bytes_per_sample(avctx->sample_fmt)); *got_frame_ptr = 1; @@ -432,7 +478,11 @@ const AVCodec ff_libfdk_aac_decoder = { .decode = fdk_aac_decode_frame, .close = fdk_aac_decode_close, .flush = fdk_aac_decode_flush, - .capabilities = AV_CODEC_CAP_DR1 | AV_CODEC_CAP_CHANNEL_CONF, + .capabilities = AV_CODEC_CAP_DR1 | AV_CODEC_CAP_CHANNEL_CONF +#if FDKDEC_VER_AT_LEAST(2, 5) // 2.5.10 + | AV_CODEC_CAP_DELAY +#endif + , .priv_class = &fdk_aac_dec_class, .caps_internal = FF_CODEC_CAP_INIT_THREADSAFE | FF_CODEC_CAP_INIT_CLEANUP,