[FFmpeg-devel] ffmpeg_filter: initialize sub2video.end_pts together with last_pts

Submitted by Jan Ekström on Feb. 21, 2019, 11:16 a.m.

Details

Message ID CAEu79Sa+1baujxQ2HZChxr4SCfYQD-i8Sxg62iXJJ1RvRTfn6w@mail.gmail.com
State New
Headers show

Commit Message

Jan Ekström Feb. 21, 2019, 11:16 a.m.
This fixes buffering of samples which causes sudden ballooning of
memory usage in case of no subtitle samples coming in for a while if
the filter chain had been re-initialized.

You can also see messages a la:
"Error while add the frame to buffer source(Invalid argument)."
disappearing after filter chain re-initializations.

Passes fate-sub2video.

Example (memory usage before patch around 700+ MiB, after around 150MiB) :
/usr/bin/time -v ffmpeg -v verbose \
  -i "https://megumin.fushizen.eu/samples/2019-01-18-audio_reconfig_causes_buffer_growth.ts"
\
  -filter_complex
'[0:v:0]yadif=deint=interlaced[yadif_out];[yadif_out][0:s:0]overlay=eof_action=pass:repeatlast=0[overlay_out];[overlay_out]scale=1024:-2[video_out];[0:a:0]aresample=48000:async=1,aformat=channel_layouts=stereo[filtered_audio]'
\
  -map "[video_out]" \
    -c:v mpeg4 \
    -b:v 750k \
  -map "[filtered_audio]" \
    -c:a aac \
    -b:a 192k \
  "test.mp4"

Best regards,
Jan

Comments

Nicolas George Feb. 21, 2019, 11:30 a.m.
Jan Ekström (12019-02-21):
> This fixes buffering of samples which causes sudden ballooning of
> memory usage in case of no subtitle samples coming in for a while if
> the filter chain had been re-initialized.
> 
> You can also see messages a la:
> "Error while add the frame to buffer source(Invalid argument)."
> disappearing after filter chain re-initializations.
> 
> Passes fate-sub2video.
> 
> Example (memory usage before patch around 700+ MiB, after around 150MiB) :
> /usr/bin/time -v ffmpeg -v verbose \
>   -i "https://megumin.fushizen.eu/samples/2019-01-18-audio_reconfig_causes_buffer_growth.ts"
> \
>   -filter_complex
> '[0:v:0]yadif=deint=interlaced[yadif_out];[yadif_out][0:s:0]overlay=eof_action=pass:repeatlast=0[overlay_out];[overlay_out]scale=1024:-2[video_out];[0:a:0]aresample=48000:async=1,aformat=channel_layouts=stereo[filtered_audio]'
> \
>   -map "[video_out]" \
>     -c:v mpeg4 \
>     -b:v 750k \
>   -map "[filtered_audio]" \
>     -c:a aac \
>     -b:a 192k \
>   "test.mp4"

Looks good, thanks.

Regards,
Jan Ekström Feb. 21, 2019, 6:20 p.m.
On Thu, Feb 21, 2019 at 1:30 PM Nicolas George <george@nsup.org> wrote:
>
> Jan Ekström (12019-02-21):
> > This fixes buffering of samples which causes sudden ballooning of
> > memory usage in case of no subtitle samples coming in for a while if
> > the filter chain had been re-initialized.
> >
> > You can also see messages a la:
> > "Error while add the frame to buffer source(Invalid argument)."
> > disappearing after filter chain re-initializations.
> >
> > Passes fate-sub2video.
> >
> > Example (memory usage before patch around 700+ MiB, after around 150MiB) :
> > /usr/bin/time -v ffmpeg -v verbose \
> >   -i "https://megumin.fushizen.eu/samples/2019-01-18-audio_reconfig_causes_buffer_growth.ts"
> > \
> >   -filter_complex
> > '[0:v:0]yadif=deint=interlaced[yadif_out];[yadif_out][0:s:0]overlay=eof_action=pass:repeatlast=0[overlay_out];[overlay_out]scale=1024:-2[video_out];[0:a:0]aresample=48000:async=1,aformat=channel_layouts=stereo[filtered_audio]'
> > \
> >   -map "[video_out]" \
> >     -c:v mpeg4 \
> >     -b:v 750k \
> >   -map "[filtered_audio]" \
> >     -c:a aac \
> >     -b:a 192k \
> >   "test.mp4"
>
> Looks good, thanks.
>
> Regards,

Applied. Thanks for the review.

Jan
Michael Niedermayer Feb. 22, 2019, 1:11 p.m.
On Thu, Feb 21, 2019 at 01:16:00PM +0200, Jan Ekström wrote:
> This fixes buffering of samples which causes sudden ballooning of
> memory usage in case of no subtitle samples coming in for a while if
> the filter chain had been re-initialized.
> 
> You can also see messages a la:
> "Error while add the frame to buffer source(Invalid argument)."
> disappearing after filter chain re-initializations.
> 
> Passes fate-sub2video.
> 
> Example (memory usage before patch around 700+ MiB, after around 150MiB) :
> /usr/bin/time -v ffmpeg -v verbose \
>   -i "https://megumin.fushizen.eu/samples/2019-01-18-audio_reconfig_causes_buffer_growth.ts"
> \
>   -filter_complex
> '[0:v:0]yadif=deint=interlaced[yadif_out];[yadif_out][0:s:0]overlay=eof_action=pass:repeatlast=0[overlay_out];[overlay_out]scale=1024:-2[video_out];[0:a:0]aresample=48000:async=1,aformat=channel_layouts=stereo[filtered_audio]'
> \
>   -map "[video_out]" \
>     -c:v mpeg4 \
>     -b:v 750k \
>   -map "[filtered_audio]" \
>     -c:a aac \
>     -b:a 192k \
>   "test.mp4"
> 
> Best regards,
> Jan

>  ffmpeg_filter.c |    1 +
>  1 file changed, 1 insertion(+)
> 5f88558fef759023173b4c8efe157aa30fc9e337  0001-ffmpeg_filter-initialize-sub2video.end_pts-together-.patch
> From 9c824c36c972aca19f2747437c8edc71b6c0886c Mon Sep 17 00:00:00 2001
> From: =?UTF-8?q?Jan=20Ekstr=C3=B6m?= <jeebjp@gmail.com>
> Date: Thu, 20 Feb 2019 20:54:11 +0200
> Subject: [PATCH] ffmpeg_filter: initialize sub2video.end_pts together with
>  last_pts

Breaks: (video stream is empty after this) when a duration is used
./ffmpeg -i in.mkv  -filter_complex '[0:s:1]scale=800:600' -t 15 -qscale 2 test.avi

both video and subtitles appear in the first 5 seconds

input is a pgs in mkv file.
If you cannot reproduce with a random file then say so and ill try to
turn the file i have into a small testcase i can share

thanks

[...]

Patch hide | download patch | download mbox

From 9c824c36c972aca19f2747437c8edc71b6c0886c Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jan=20Ekstr=C3=B6m?= <jeebjp@gmail.com>
Date: Thu, 20 Feb 2019 20:54:11 +0200
Subject: [PATCH] ffmpeg_filter: initialize sub2video.end_pts together with
 last_pts

This way re-initializations properly update end_pts, enabling
sub2video_heartbeat to call sub2video_update as expected to re-init
the sub2video AVFrame's contents and to feed a frame into the filter
chain.

This then fixes memory usage ballooning due to framesync waiting
for secondary input in case of no actual subtitle samples being present
for a while in source after a re-init occurs.
---
 fftools/ffmpeg_filter.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c
index 8c0ff99dd9..72838de1e2 100644
--- a/fftools/ffmpeg_filter.c
+++ b/fftools/ffmpeg_filter.c
@@ -739,6 +739,7 @@  static int sub2video_prepare(InputStream *ist, InputFilter *ifilter)
     if (!ist->sub2video.frame)
         return AVERROR(ENOMEM);
     ist->sub2video.last_pts = INT64_MIN;
+    ist->sub2video.end_pts  = INT64_MIN;
     return 0;
 }
 
-- 
2.20.1