From patchwork Tue Dec 12 11:20:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gyan Doshi X-Patchwork-Id: 45088 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:1225:b0:181:818d:5e7f with SMTP id v37csp3520202pzf; Tue, 12 Dec 2023 03:20:43 -0800 (PST) X-Google-Smtp-Source: AGHT+IGtvv0IznDG7tofKfbpX6X8FlGepvAAfDZEI6r101TUAOlezKC/AuYNz3dSY/TBidCgUX8d X-Received: by 2002:a17:906:ba86:b0:a1e:1afc:4594 with SMTP id cu6-20020a170906ba8600b00a1e1afc4594mr6819357ejd.0.1702380043592; Tue, 12 Dec 2023 03:20:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702380043; cv=none; d=google.com; s=arc-20160816; b=avOdnYYPoolIHqIRT43CzSAmHU1b+t87KWZ/XeUlVfjwfQU8bnwH+5l5IkXlvUwEHc 2/d1M5a1zPTv0rrcYt0fWSTVDezK9bLzKO64SXj3RCeuHnQtD2SQH4zhmkjwp6zIgTY1 ks1QpNPeWuX2hJsyX36a4CTpfvHY+CTk3635glUUo8XVo761LyK7bmnDh8k5SK+b52nG IpFm9XJE2K+DLUynGU39HxdYFuIsuZr33hks80HsVGOL8RyI++/bByEaW8Q+4bQ8i2IT gEzbBUlsYIJu/Av8quro9vrM9xdN5GdtmKRhGNA332+i4PWV5yfLJTjXdePtDic1hoJ3 gjLw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :delivered-to; bh=7pQh2XLydeIvTvi6uRJbkoJkOxoJh1wSyl3LjLLhaUI=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=vDA0nim+dspBDdlesvoPu3J0Xn/sW1uMGLTBEVrjItlQ6kwh1DKdwb7DA2fLJHlmZ6 Rtr4yqWJQZO1XjP1EtjdOaHto7yE3JNdihsSZb9BuryZVKbBLkKuqxpnxW74XAdy3fEZ pUJYUgldzFZkvAT10xu5QDBpOgsLU6bSPdXMM9evp1nx6RfbaU3ewALK+RpBUN6PDSTe rX5iJ4YRp3XKSx5dyhPmp6c+ry/qFIFWBLs0f/UWKv9oeA8o7EjZq0nZp1rhE0SNv5T5 4Xw+tfmxgpzIClQBqyDV0yz72XAuVAcn5GP9Abbyg+4hGZMhRPu5k/wh0UuMYWf2Yz8V wOxg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id ox4-20020a170907100400b00a1b6ed51cddsi4493316ejb.86.2023.12.12.03.20.40; Tue, 12 Dec 2023 03:20:43 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7A00D68D158; Tue, 12 Dec 2023 13:20:36 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mout-p-102.mailbox.org (mout-p-102.mailbox.org [80.241.56.152]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4649668D00C for ; Tue, 12 Dec 2023 13:20:28 +0200 (EET) Received: from smtp202.mailbox.org (smtp202.mailbox.org [IPv6:2001:67c:2050:b231:465::202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-p-102.mailbox.org (Postfix) with ESMTPS id 4SqGPP6WSKz9sks for ; Tue, 12 Dec 2023 12:20:25 +0100 (CET) From: Gyan Doshi To: ffmpeg-devel@ffmpeg.org Date: Tue, 12 Dec 2023 16:50:08 +0530 Message-Id: <20231212112008.2500-1-ffmpeg@gyani.pro> MIME-Version: 1.0 X-Rspamd-Queue-Id: 4SqGPP6WSKz9sks Subject: [FFmpeg-devel] [PATCH] swr/swresample: avoid reapplication of firstpts X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: b3nn7T3EAsVy During a resampling operation where 1) user has specified first_pts 2) SWR_FLAG_RESAMPLE is not set initially (directly or otherwise) 3) first_pts has been fulfilled (always using hard compensation) then upon first encountering a delay where a soft compensation is required, swr_set_compensation will lead to another init of swr which will reset outpts to the specified firstpts thus leading to an output frame having its pts = firstpts. When the next input frame is received, swr will see a large delay and inject silence from firstpts to the current frame's pts. This can lead to severe desync and in worst case, loss of audio playback. Parameter initcount added to keep track of swr_init calls of the swrcontext and to ultimately avoid enforcing firstpts again. Fixes #4131. --- libswresample/swresample.c | 7 +++++-- libswresample/swresample_internal.h | 1 + libswresample/version.h | 2 +- 3 files changed, 7 insertions(+), 3 deletions(-) diff --git a/libswresample/swresample.c b/libswresample/swresample.c index f2a9b40474..008af3610f 100644 --- a/libswresample/swresample.c +++ b/libswresample/swresample.c @@ -375,8 +375,9 @@ av_cold int swr_init(struct SwrContext *s){ if (s->firstpts_in_samples != AV_NOPTS_VALUE) { if (!s->async && s->min_compensation >= FLT_MAX/2) s->async = 1; - s->firstpts = - s->outpts = s->firstpts_in_samples * s->out_sample_rate; + if (!s->initcount) + s->firstpts = + s->outpts = s->firstpts_in_samples * s->out_sample_rate; } else s->firstpts = AV_NOPTS_VALUE; @@ -458,6 +459,7 @@ av_assert0(s->out.ch_count); if(!s->resample && !s->rematrix && !s->channel_map && !s->dither.method){ s->full_convert = swri_audio_convert_alloc(s->out_sample_fmt, s-> in_sample_fmt, s-> in.ch_count, NULL, 0); + s->initcount++; return 0; } @@ -510,6 +512,7 @@ av_assert0(s->out.ch_count); goto fail; } + s->initcount++; return 0; fail: swr_close(s); diff --git a/libswresample/swresample_internal.h b/libswresample/swresample_internal.h index ad902d73fa..71d0c60e9f 100644 --- a/libswresample/swresample_internal.h +++ b/libswresample/swresample_internal.h @@ -145,6 +145,7 @@ struct SwrContext { float max_soft_compensation; ///< swr maximum soft compensation in seconds over soft_compensation_duration float async; ///< swr simple 1 parameter async, similar to ffmpegs -async int64_t firstpts_in_samples; ///< swr first pts in samples + int initcount; ///< swr no. of times the swrcontext has been inited int resample_first; ///< 1 if resampling must come first, 0 if rematrixing int rematrix; ///< flag to indicate if rematrixing is needed (basically if input and output layouts mismatch) diff --git a/libswresample/version.h b/libswresample/version.h index 46a4e2fc62..fe4469ae6b 100644 --- a/libswresample/version.h +++ b/libswresample/version.h @@ -31,7 +31,7 @@ #include "version_major.h" #define LIBSWRESAMPLE_VERSION_MINOR 13 -#define LIBSWRESAMPLE_VERSION_MICRO 100 +#define LIBSWRESAMPLE_VERSION_MICRO 101 #define LIBSWRESAMPLE_VERSION_INT AV_VERSION_INT(LIBSWRESAMPLE_VERSION_MAJOR, \ LIBSWRESAMPLE_VERSION_MINOR, \