From patchwork Fri Feb 22 19:43:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Jan_Ekstr=C3=B6m?= X-Patchwork-Id: 12141 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 990EF449415 for ; Fri, 22 Feb 2019 21:43:28 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 76082689D32; Fri, 22 Feb 2019 21:43:28 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lj1-f193.google.com (mail-lj1-f193.google.com [209.85.208.193]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7B35C6809C1 for ; Fri, 22 Feb 2019 21:43:22 +0200 (EET) Received: by mail-lj1-f193.google.com with SMTP id j13-v6so2621299ljc.2 for ; Fri, 22 Feb 2019 11:43:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :content-transfer-encoding; bh=w1TSWwPVIzqKY9oNnHwMGRdi/mW/Fm5CefQhiyRC7NQ=; b=vf3XqsY1JTuDKKgI//Ue9PyBa/hEp5T2d8e4074Y+gKP4UA28dekT/ieeaxTFyvk/2 rYy96bH4yJH4SHsvvOBli7zK6elTkB2sFDDqXFcSw0p59iGetJ9Z+uX1qtRcSEDC2FuB FrE6We8dNLpfXBt4U6vYJCqykltkqru4CGJaywoiVK20yrey+kw6AWKOFEkJEuXS5tit OO37UBCvvGkcHTlP40KCaElomUdn14QRvvxRpajAGe8kPqo5til6IruO44W1Z0HYnpGs CiYw4sm6jUf+wwXyOP0Q9vCnGCRMG9s7XddSSYr9232AziPbrkzY7xvjRw2G0xbGm9Kj djbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:content-transfer-encoding; bh=w1TSWwPVIzqKY9oNnHwMGRdi/mW/Fm5CefQhiyRC7NQ=; b=ApuvAqqwwrri8rEReRluvINaGrycVQ3icOsNJ+3f+D80kwHHKXYoyDQ42Rh8FHrrt0 cma8hcjAXOM6V1eGSB6bAxGVrqjSo+MY0R9f11Xg2aryJQo4cLyiPTB3E0xjd3awZNq+ i4wuNSuUdOLKjATlcS1svF/Jv3Izk6bAr2+vJAdWTVhknpWJInKjqB3UUtAjMtxPJjFT LppuAnAAG2tAGQDdvPhkQPrwovJ7Ipju0zP8U6DJr/dc9T06u9HZ7FsRcLqTeTewHot1 +tKZOlZpR/LN1EVqpiTG/xzK3Aj6w3DHa/wxhre7CDwer4of55NpcDIoSBTqPuqsQQZz Y9HA== X-Gm-Message-State: AHQUAubj2QhVMaogYSX93+AyZJa31049GzbV53+ccsZZIJsa544vmcpq R9idfEmPV8PwG01VsFXkRp1wbVyX9q4v0YPlWnBkf+l0 X-Google-Smtp-Source: AHgI3IaKhTJArg2abuClAK0PyOm2khyNrBvlVxmIGH82gpoQf4aQZ/ncBd75da21CSUIbuot4JqN5X8tlksk2vgjWAA= X-Received: by 2002:a2e:9e46:: with SMTP id g6-v6mr3291923ljk.52.1550864601050; Fri, 22 Feb 2019 11:43:21 -0800 (PST) MIME-Version: 1.0 References: <20190222131142.GT3501@michaelspb> In-Reply-To: <20190222131142.GT3501@michaelspb> From: =?UTF-8?B?SmFuIEVrc3Ryw7Zt?= Date: Fri, 22 Feb 2019 21:43:08 +0200 Message-ID: To: FFmpeg development discussions and patches Subject: Re: [FFmpeg-devel] [PATCH] ffmpeg_filter: initialize sub2video.end_pts together with last_pts X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" On Fri, Feb 22, 2019 at 3:11 PM Michael Niedermayer wrote: > > On Thu, Feb 21, 2019 at 01:16:00PM +0200, Jan Ekström wrote: > > This fixes buffering of samples which causes sudden ballooning of > > memory usage in case of no subtitle samples coming in for a while if > > the filter chain had been re-initialized. > > > > You can also see messages a la: > > "Error while add the frame to buffer source(Invalid argument)." > > disappearing after filter chain re-initializations. > > > > Passes fate-sub2video. > > > > Example (memory usage before patch around 700+ MiB, after around 150MiB) : > > /usr/bin/time -v ffmpeg -v verbose \ > > -i "https://megumin.fushizen.eu/samples/2019-01-18-audio_reconfig_causes_buffer_growth.ts" > > \ > > -filter_complex > > '[0:v:0]yadif=deint=interlaced[yadif_out];[yadif_out][0:s:0]overlay=eof_action=pass:repeatlast=0[overlay_out];[overlay_out]scale=1024:-2[video_out];[0:a:0]aresample=48000:async=1,aformat=channel_layouts=stereo[filtered_audio]' > > \ > > -map "[video_out]" \ > > -c:v mpeg4 \ > > -b:v 750k \ > > -map "[filtered_audio]" \ > > -c:a aac \ > > -b:a 192k \ > > "test.mp4" > > > > Best regards, > > Jan > > > ffmpeg_filter.c | 1 + > > 1 file changed, 1 insertion(+) > > 5f88558fef759023173b4c8efe157aa30fc9e337 0001-ffmpeg_filter-initialize-sub2video.end_pts-together-.patch > > From 9c824c36c972aca19f2747437c8edc71b6c0886c Mon Sep 17 00:00:00 2001 > > From: =?UTF-8?q?Jan=20Ekstr=C3=B6m?= > > Date: Thu, 20 Feb 2019 20:54:11 +0200 > > Subject: [PATCH] ffmpeg_filter: initialize sub2video.end_pts together with > > last_pts > > Breaks: (video stream is empty after this) when a duration is used > ./ffmpeg -i in.mkv -filter_complex '[0:s:1]scale=800:600' -t 15 -qscale 2 test.avi > > both video and subtitles appear in the first 5 seconds > > input is a pgs in mkv file. > If you cannot reproduce with a random file then say so and ill try to > turn the file i have into a small testcase i can share > > thanks > I can replicate this. Old behavior is kept if end_pts is initialized to its previous implicit initialization value of zero. I initialized this originally as INT64_MIN because that way it was matching, and there were some comments about negative PTS values in the commit that added the other value's initialization. In my testing earlier I did find both values working for the use cases I tested with, so I will send out a patch to switch around to zero initialization for now, as the main part for not causing possibly infinite buffering in lavfi is that when sub2video_heartbeat gets called after a filter chain (re-)initialization sub2video.end_pts is less than INT64_MAX. This might be an issue with something expecting timestamps that start from zero for some sort of "vsync", though, as the heartbeats are nicely sent out and we are getting actual AVFrames pushed into the filter chain. The major difference seems to be that after X frames failed requests start appearing in the filter chain, and the current sub2video AVFrame gets fed into the filter chain again. Additionally, the ffmpeg.c vsync logic seems to not duplicate or drop frames when starting from INT64_MIN. My debugging av_log code for sub2video can be found at https://github.com/jeeb/ffmpeg/commit/f1750f489be345f74525f7a2ad89e40c4d7e0493 . Jan Excerpt by git diff --no-index --word-diff: diff --git a/pushing_log_before.log b/pushing_log_int64_min.log index c163c36c37..c521380b37 100644 --- a/pushing_log_before.log +++ b/pushing_log_int64_min.log @@ -1,86 +1,74 @@ [mpegts @ [-0x2fe5b00]-]{+0x263eb00]+} sub2video: using 1920x1080 canvas sub2video: Heartbeat from stream 1 (audio, PID: 4352) stream at PTS -1 (last_pts: -9223372036854775808, end_pts: [-0).-]{+-9223372036854775808).+} sub2video: Update for stream 2, (pts: [-0,-]{+-9223372036854775808,+} end_pts: 9223372036854775807, num_rects: 0) sub2video: Pushing a 1920x1080 frame to 'graph 0 input from stream 0:2' with PTS [-0-]{+-9223372036854775808+} sub2video: Heartbeat from stream 1 (audio, PID: 4352) stream at PTS 449 (last_pts: [-0,-]{+-9223372036854775808,+} end_pts: 9223372036854775807). sub2video: Heartbeat from stream 1 (audio, PID: 4352) stream at PTS 899 (last_pts: [-0,-]{+-9223372036854775808,+} end_pts: 9223372036854775807). sub2video: Heartbeat from stream 1 (audio, PID: 4352) stream at PTS 1349 (last_pts: [-0,-]{+-9223372036854775808,+} end_pts: 9223372036854775807). sub2video: Heartbeat from stream 1 (audio, PID: 4352) stream at PTS 1799 (last_pts: [-0,-]{+-9223372036854775808,+} end_pts: 9223372036854775807). sub2video: Heartbeat from stream 1 (audio, PID: 4352) stream at PTS 2249 (last_pts: [-0,-]{+-9223372036854775808,+} end_pts: 9223372036854775807). sub2video: Heartbeat from stream 1 (audio, PID: 4352) stream at PTS 2699 (last_pts: [-0,-]{+-9223372036854775808,+} end_pts: 9223372036854775807). sub2video: Heartbeat from stream 1 (audio, PID: 4352) stream at PTS 3149 (last_pts: [-0,-]{+-9223372036854775808,+} end_pts: 9223372036854775807). sub2video: Heartbeat from stream 1 (audio, PID: 4352) stream at PTS 3599 (last_pts: [-0,-]{+-9223372036854775808,+} end_pts: 9223372036854775807). sub2video: Heartbeat from stream 1 (audio, PID: 4352) stream at PTS 4049 (last_pts: [-0,-]{+-9223372036854775808,+} end_pts: 9223372036854775807). sub2video: Heartbeat from stream 1 (audio, PID: 4352) stream at PTS 4499 (last_pts: [-0,-]{+-9223372036854775808,+} end_pts: 9223372036854775807). sub2video: Heartbeat from stream 1 (audio, PID: 4352) stream at PTS 4949 (last_pts: [-0,-]{+-9223372036854775808,+} end_pts: 9223372036854775807). sub2video: Heartbeat from stream 1 (audio, PID: 4352) stream at PTS 5399 (last_pts: [-0,-]{+-9223372036854775808,+} end_pts: 9223372036854775807). sub2video: Heartbeat from stream 1 (audio, PID: 4352) stream at PTS 5849 (last_pts: [-0,-]{+-9223372036854775808,+} end_pts: 9223372036854775807). sub2video: Heartbeat[-found 1 failed requests. Feeding filter chain again-] [-sub2video: Pushing a 1920x1080 frame to 'graph 0 input from stream 0:2' with PTS 5849-] [-sub2video: Heartbeat-] from stream 1 (audio, PID: 4352) stream at PTS 6299 (last_pts: [-5849,-]{+-9223372036854775808,+} end_pts: 9223372036854775807). sub2video: Heartbeat from stream 1 (audio, PID: 4352) stream at PTS 6749 (last_pts: [-5849,-]{+-9223372036854775808,+} end_pts: 9223372036854775807). sub2video: Heartbeat from stream 1 (audio, PID: 4352) stream at PTS 7199 (last_pts: [-5849,-]{+-9223372036854775808,+} end_pts: 9223372036854775807). sub2video: Heartbeat from stream 1 (audio, PID: 4352) stream at PTS 7649 (last_pts: [-5849,-]{+-9223372036854775808,+} end_pts: 9223372036854775807). sub2video: Heartbeat from stream 1 (audio, PID: 4352) stream at PTS 8099 (last_pts: [-5849,-]{+-9223372036854775808,+} end_pts: 9223372036854775807). _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel