From patchwork Sat Dec 16 10:14:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gyan Doshi X-Patchwork-Id: 45175 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:1225:b0:181:818d:5e7f with SMTP id v37csp6219272pzf; Sat, 16 Dec 2023 02:14:46 -0800 (PST) X-Google-Smtp-Source: AGHT+IEQATddF2M/14nU5BoPcSNzdYUMtLLiX6mUWIbnIgh/+cOJpmcABGwY0MskVlbDcysSFixV X-Received: by 2002:a17:906:344d:b0:a19:a19b:78ba with SMTP id d13-20020a170906344d00b00a19a19b78bamr6892845ejb.125.1702721686299; Sat, 16 Dec 2023 02:14:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702721686; cv=none; d=google.com; s=arc-20160816; b=ymVLTg83r0N/djH2ahYvmXKk6sWz+HpXRbgtWwUILilaYdE6rHKAQD4rUoQowmkpp5 O8ky5L5+uFP4npu8hhj98KNOrPloOBLAqYDqnujJH1sGgKXjIMyyT797s5YE2CyTsVXy j1rlRxePYRkITkDmjM+rXV4/h+rqE4xxZ59D5elBKE2T6Oz4nuFkCjdxMnLyRAUlIjQH w6L9fo6UvhoLti97w8MTMx/RzGTjGFD3BiPvFjNtIhXB0nFUklxefrPYlPn2/g1byIWC MfuBaw2EJvc5iM/W0rpw7Vp64por5YEVaqTcMMXPdBfhS9gF/4ejZYKUZxi8mFSinZf9 TYmw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :delivered-to; bh=W3J9EIAyD7Ti7gGx8Wg5GBdpHp2s6R4qSXsVVGL1n6o=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=R1XZwLg21ZUYHSDm3yK5+TsqeZ8iqcl4sXnMJPqWyUriiCNotGui1cBCi2ISYB7KvG AGzJy4Tt5S9q0Xj1/oX/vTYHHw5WQoBDWS0Egm4EbsHlAuPul4KnFfR+ARjs4RH1cVr9 rT6ueTO1Q8PX8py3Kew78uZesHkZdrkxUj0r+RMV1loUtuIvVYO1XX+n2odq4RZSVcmz VHBXpyXpbore1pem+cfGI917qzxMWSY8lJzfGvTNO199ZxKOHZ3ccEznuhGwyvz9fu5J C/QmQ9aa4VZjjLfBaVLwzj180K1DedYvpcUYezAuoZJlIQVoUSx3FnZ/M07ZsPkKfmvo cUnQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id d19-20020a170906345300b00a1b77b5fc0bsi7515239ejb.225.2023.12.16.02.14.45; Sat, 16 Dec 2023 02:14:46 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1B9FD68D0A5; Sat, 16 Dec 2023 12:14:42 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mout-p-103.mailbox.org (mout-p-103.mailbox.org [80.241.56.161]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B399F68D035 for ; Sat, 16 Dec 2023 12:14:35 +0200 (EET) Received: from smtp202.mailbox.org (smtp202.mailbox.org [IPv6:2001:67c:2050:b231:465::202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-p-103.mailbox.org (Postfix) with ESMTPS id 4SshlX4JS0z9sTD for ; Sat, 16 Dec 2023 11:14:32 +0100 (CET) From: Gyan Doshi To: ffmpeg-devel@ffmpeg.org Date: Sat, 16 Dec 2023 15:44:14 +0530 Message-Id: <20231216101414.12607-1-ffmpeg@gyani.pro> MIME-Version: 1.0 X-Rspamd-Queue-Id: 4SshlX4JS0z9sTD Subject: [FFmpeg-devel] [PATCH v3] swr/swresample: avoid reapplication of firstpts X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: s5/2LUT3FOr7 During a resampling operation where 1) user has specified first_pts 2) SWR_FLAG_RESAMPLE is not set initially (directly or otherwise) 3) first_pts has been fulfilled (always using hard compensation) then upon first encountering a delay where a soft compensation is required, swr_set_compensation will lead to another init of swr which will reset outpts to the specified firstpts thus leading to an output frame having its pts = firstpts. When the next input frame is received, swr will see a large delay and inject silence from firstpts to the current frame's pts. This can lead to severe desync and in worst case, loss of audio playback. Parameter firstpts initialized to AV_NOPTS_VALUE in swr_alloc and then checked in swr_init to avoid resetting outpts, thus avoiding reapplication of firstpts. Fixes #4131. --- Added fate test libswresample/options.c | 1 + libswresample/swresample.c | 5 +++-- tests/fate/libswresample.mak | 3 +++ tests/ref/fate/swr-async-firstpts | 24 ++++++++++++++++++++++++ 4 files changed, 31 insertions(+), 2 deletions(-) create mode 100644 tests/ref/fate/swr-async-firstpts diff --git a/libswresample/options.c b/libswresample/options.c index fb109fdbab..d8cf85c053 100644 --- a/libswresample/options.c +++ b/libswresample/options.c @@ -171,6 +171,7 @@ av_cold struct SwrContext *swr_alloc(void){ if(s){ s->av_class= &av_class; av_opt_set_defaults(s); + s->firstpts = AV_NOPTS_VALUE; } return s; } diff --git a/libswresample/swresample.c b/libswresample/swresample.c index f2a9b40474..1cf83a803f 100644 --- a/libswresample/swresample.c +++ b/libswresample/swresample.c @@ -375,8 +375,9 @@ av_cold int swr_init(struct SwrContext *s){ if (s->firstpts_in_samples != AV_NOPTS_VALUE) { if (!s->async && s->min_compensation >= FLT_MAX/2) s->async = 1; - s->firstpts = - s->outpts = s->firstpts_in_samples * s->out_sample_rate; + if (s->firstpts == AV_NOPTS_VALUE) + s->firstpts = + s->outpts = s->firstpts_in_samples * s->out_sample_rate; } else s->firstpts = AV_NOPTS_VALUE; diff --git a/tests/fate/libswresample.mak b/tests/fate/libswresample.mak index f2108016af..0d29f76024 100644 --- a/tests/fate/libswresample.mak +++ b/tests/fate/libswresample.mak @@ -1082,6 +1082,9 @@ $(call CROSS_TEST,$(SAMPLERATES_LITE),ARESAMPLE_EXACT_LIN_ASYNC,s32p,s32le,s16) $(call CROSS_TEST,$(SAMPLERATES_LITE),ARESAMPLE_EXACT_LIN_ASYNC,fltp,f32le,s16) $(call CROSS_TEST,$(SAMPLERATES_LITE),ARESAMPLE_EXACT_LIN_ASYNC,dblp,f64le,s16) +FATE_SWR_RESAMPLE-$(call FILTERDEMDEC, ARESAMPLE ASETPTS ATRIM SINE, , PCM_S16LE, LAVFI_INDEV) += fate-swr-async-firstpts +fate-swr-async-firstpts: CMD = framecrc -auto_conversion_filters -copyts -f lavfi -i "sine=r=1000:samples_per_frame=100,asetpts=PTS+S+S*floor(ld(1)/4)+st(1\,ld(1)+1)*0,atrim=end=2" -filter:a aresample=async=300:first_pts=0 + FATE_SWR_RESAMPLE-$(call FILTERDEMDECENCMUX, ARESAMPLE, WAV, PCM_S16LE, PCM_S16LE, WAV) += $(FATE_SWR_RESAMPLE) fate-swr-resample: $(FATE_SWR_RESAMPLE-yes) FATE_SWR += $(FATE_SWR_RESAMPLE-yes) diff --git a/tests/ref/fate/swr-async-firstpts b/tests/ref/fate/swr-async-firstpts new file mode 100644 index 0000000000..3f6b290bab --- /dev/null +++ b/tests/ref/fate/swr-async-firstpts @@ -0,0 +1,24 @@ +#tb 0: 1/1000 +#media_type 0: audio +#codec_id 0: pcm_s16le +#sample_rate 0: 1000 +#channel_layout_name 0: mono +0, 0, 0, 132, 264, 0xc2981f45 +0, 132, 132, 68, 136, 0xe78e468d +0, 200, 200, 100, 200, 0xd55c67d0 +0, 300, 300, 100, 200, 0xd55c67d0 +0, 400, 400, 100, 200, 0xd55c67d0 +0, 500, 500, 93, 186, 0x85ca5db4 +0, 593, 593, 110, 220, 0xa2655d0b +0, 703, 703, 108, 216, 0x95cb6f01 +0, 811, 811, 108, 216, 0xf35668b8 +0, 919, 919, 149, 298, 0xc273245f +0, 1068, 1068, 136, 272, 0xedeb6e0a +0, 1204, 1204, 98, 196, 0xea18668e +0, 1302, 1302, 98, 196, 0x412861e7 +0, 1400, 1400, 98, 196, 0x7ec361b2 +0, 1498, 1498, 110, 220, 0xf3ae6a6a +0, 1608, 1608, 108, 216, 0xab2f6c93 +0, 1716, 1716, 107, 214, 0x50de6eb9 +0, 1823, 1823, 106, 212, 0x67b8656d +0, 1929, 1929, 18, 36, 0x2b7911c6