diff mbox

[FFmpeg-devel] avformat/hlsenc: temp_file usage for master playlist and vtt playlist

Message ID 6c70f542-4e16-31f7-95e1-c59b8361d5a2@vivanet.hu
State Accepted
Commit 098ab932579ea6d81e0b928f09b425fdd00a2890
Headers show

Commit Message

Bodecs Bela June 24, 2019, 4:01 p.m. UTC
Dear All,

currently master playlist and subtitle playlist creation does not use
temporary files even when temp_file flag is set. Most of the use cases
it is not a problem because master playlist creation happens once on the
beginning of the whole process. But if master playlist is periodically
re-created because of master_pl_refresh_rate is set, non-atomic playlist
creation may cause problems in case of live streaming. This poblem (i.e 
non-atomic playlist
creation) may also apply for subtitle playlist (vtt) creation in live 
streaming.
This patch correct this behavior by adding missing functionality.

please review this patch.

thank you in advance,

best regards,

Bela
From 04e70ba586646b927e1b05a9df3860a635871603 Mon Sep 17 00:00:00 2001
From: Bela Bodecs <bodecsb@vivanet.hu>
Date: Mon, 24 Jun 2019 17:41:49 +0200
Subject: [PATCH] avformat/hlsenc: temp_file usage for master playlist and vtt
 playlist

currently master playlist and subtitle playlist creation does not use
temporary files even when temp_file flag is set. Most of the use cases
it is not a problem because master playlist creation happens once on the
beginning of the whole process. But if master playlist is periodically
re-created because of master_pl_refresh_rate is set, non-atomic playlist
creation may cause problems in case of live streaming. This patch
correct this behavior by adding this functionality.


Signed-off-by: Bela Bodecs <bodecsb@vivanet.hu>
---
 doc/muxers.texi      |  6 +++++-
 libavformat/hlsenc.c | 30 +++++++++++++++++++++---------
 2 files changed, 26 insertions(+), 10 deletions(-)

Comments

Bodecs Bela June 27, 2019, 8:36 p.m. UTC | #1
ping

2019.06.24. 18:01 keltezéssel, Bodecs Bela írta:
> Dear All,
>
> currently master playlist and subtitle playlist creation does not use
> temporary files even when temp_file flag is set. Most of the use cases
> it is not a problem because master playlist creation happens once on the
> beginning of the whole process. But if master playlist is periodically
> re-created because of master_pl_refresh_rate is set, non-atomic playlist
> creation may cause problems in case of live streaming. This poblem 
> (i.e non-atomic playlist
> creation) may also apply for subtitle playlist (vtt) creation in live 
> streaming.
> This patch correct this behavior by adding missing functionality.
>
> please review this patch.
>
> thank you in advance,
>
> best regards,
>
> Bela
>
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
Liu Steven June 27, 2019, 11:03 p.m. UTC | #2
> 在 2019年6月28日,04:36,Bodecs Bela <bodecsb@vivanet.hu> 写道:
> 
> ping

Wait, i’m testing it.
> 
> 2019.06.24. 18:01 keltezéssel, Bodecs Bela írta:
>> Dear All,
>> 
>> currently master playlist and subtitle playlist creation does not use
>> temporary files even when temp_file flag is set. Most of the use cases
>> it is not a problem because master playlist creation happens once on the
>> beginning of the whole process. But if master playlist is periodically
>> re-created because of master_pl_refresh_rate is set, non-atomic playlist
>> creation may cause problems in case of live streaming. This poblem (i.e non-atomic playlist
>> creation) may also apply for subtitle playlist (vtt) creation in live streaming.
>> This patch correct this behavior by adding missing functionality.
>> 
>> please review this patch.
>> 
>> thank you in advance,
>> 
>> best regards,
>> 
>> Bela
>> 
>> 
>> _______________________________________________
>> ffmpeg-devel mailing list
>> ffmpeg-devel@ffmpeg.org
>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>> 
>> To unsubscribe, visit link above, or email
>> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Thanks
Steven
Liu Steven June 28, 2019, 6:01 a.m. UTC | #3
> 在 2019年6月28日,上午4:36,Bodecs Bela <bodecsb@vivanet.hu> 写道:
> 
> ping
> 
> 2019.06.24. 18:01 keltezéssel, Bodecs Bela írta:
>> Dear All,
>> 
>> currently master playlist and subtitle playlist creation does not use
>> temporary files even when temp_file flag is set. Most of the use cases
>> it is not a problem because master playlist creation happens once on the
>> beginning of the whole process. But if master playlist is periodically
>> re-created because of master_pl_refresh_rate is set, non-atomic playlist
>> creation may cause problems in case of live streaming. This poblem (i.e non-atomic playlist
>> creation) may also apply for subtitle playlist (vtt) creation in live streaming.
>> This patch correct this behavior by adding missing functionality.
>> 
>> please review this patch.
>> 
>> thank you in advance,
>> 
>> best regards,
>> 
>> Bela
>> 
>> 
>> _______________________________________________
>> ffmpeg-devel mailing list
>> ffmpeg-devel@ffmpeg.org
>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>> 
>> To unsubscribe, visit link above, or email
>> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe”.


pushed


Thanks
Steven
diff mbox

Patch

diff --git a/doc/muxers.texi b/doc/muxers.texi
index 6c5b4bb637..d969e39fff 100644
--- a/doc/muxers.texi
+++ b/doc/muxers.texi
@@ -893,7 +893,11 @@  This will produce segments like this:
 @item temp_file
 Write segment data to filename.tmp and rename to filename only once the segment is complete. A webserver
 serving up segments can be configured to reject requests to *.tmp to prevent access to in-progress segments
-before they have been added to the m3u8 playlist.
+before they have been added to the m3u8 playlist. This flag also affects how m3u8 playlist files are created.
+If this flag is set, all playlist files will written into temporary file and renamed after they are complete, similarly as segments are handled.
+But playlists with @code{file} protocol and with type (@code{hls_playlist_type}) other than @code{vod}
+are always written into temporary file regardles of this flag. Master playlist files (@code{master_pl_name}), if any, with @code{file} protocol,
+are always written into temporary file regardles of this flag if @code{master_pl_publish_rate} value is other than zero.
 
 @end table
 
diff --git a/libavformat/hlsenc.c b/libavformat/hlsenc.c
index 9f5eee5491..eaeafcbb6b 100644
--- a/libavformat/hlsenc.c
+++ b/libavformat/hlsenc.c
@@ -1260,8 +1260,12 @@  static int create_master_playlist(AVFormatContext *s,
     AVDictionary *options = NULL;
     unsigned int i, j;
     int m3u8_name_size, ret, bandwidth;
-    char *m3u8_rel_name, *ccgroup;
+    char *m3u8_rel_name = NULL, *ccgroup;
     ClosedCaptionsStream *ccs;
+    const char *proto = avio_find_protocol_name(hls->master_m3u8_url);
+    int is_file_proto = proto && !strcmp(proto, "file");
+    int use_temp_file = is_file_proto && ((hls->flags & HLS_TEMP_FILE) || hls->master_publish_rate);
+    char temp_filename[1024];
 
     input_vs->m3u8_created = 1;
     if (!hls->master_m3u8_created) {
@@ -1277,12 +1281,12 @@  static int create_master_playlist(AVFormatContext *s,
     }
 
     set_http_options(s, &options, hls);
-
-    ret = hlsenc_io_open(s, &hls->m3u8_out, hls->master_m3u8_url, &options);
+    snprintf(temp_filename, sizeof(temp_filename), use_temp_file ? "%s.tmp" : "%s", hls->master_m3u8_url);
+    ret = hlsenc_io_open(s, &hls->m3u8_out, temp_filename, &options);
     av_dict_free(&options);
     if (ret < 0) {
         av_log(NULL, AV_LOG_ERROR, "Failed to open master play list file '%s'\n",
-                hls->master_m3u8_url);
+                temp_filename);
         goto fail;
     }
 
@@ -1413,7 +1417,10 @@  fail:
     if(ret >=0)
         hls->master_m3u8_created = 1;
     av_freep(&m3u8_rel_name);
-    hlsenc_io_close(s, &hls->m3u8_out, hls->master_m3u8_url);
+    hlsenc_io_close(s, &hls->m3u8_out, temp_filename);
+    if (use_temp_file)
+        ff_rename(temp_filename, hls->master_m3u8_url, s);
+
     return ret;
 }
 
@@ -1424,6 +1431,7 @@  static int hls_window(AVFormatContext *s, int last, VariantStream *vs)
     int target_duration = 0;
     int ret = 0;
     char temp_filename[1024];
+    char temp_vtt_filename[1024];
     int64_t sequence = FFMAX(hls->start_sequence, vs->sequence - vs->nb_entries);
     const char *proto = avio_find_protocol_name(vs->m3u8_name);
     int is_file_proto = proto && !strcmp(proto, "file");
@@ -1505,8 +1513,9 @@  static int hls_window(AVFormatContext *s, int last, VariantStream *vs)
     if (last && (hls->flags & HLS_OMIT_ENDLIST)==0)
         ff_hls_write_end_list(hls->m3u8_out);
 
-    if( vs->vtt_m3u8_name ) {
-        if ((ret = hlsenc_io_open(s, &hls->sub_m3u8_out, vs->vtt_m3u8_name, &options)) < 0) {
+    if (vs->vtt_m3u8_name) {
+        snprintf(temp_vtt_filename, sizeof(temp_vtt_filename), use_temp_file ? "%s.tmp" : "%s", vs->vtt_m3u8_name);
+        if ((ret = hlsenc_io_open(s, &hls->sub_m3u8_out, temp_vtt_filename, &options)) < 0) {
             if (hls->ignore_io_errors)
                 ret = 0;
             goto fail;
@@ -1531,8 +1540,11 @@  fail:
     av_dict_free(&options);
     hlsenc_io_close(s, &hls->m3u8_out, temp_filename);
     hlsenc_io_close(s, &hls->sub_m3u8_out, vs->vtt_m3u8_name);
-    if (use_temp_file)
+    if (use_temp_file) {
         ff_rename(temp_filename, vs->m3u8_name, s);
+        if (vs->vtt_m3u8_name)
+            ff_rename(temp_vtt_filename, vs->vtt_m3u8_name, s);
+    }
     if (ret >= 0 && hls->master_pl_name)
         if (create_master_playlist(s, vs) < 0)
             av_log(s, AV_LOG_WARNING, "Master playlist creation failed\n");
@@ -2991,7 +3003,7 @@  static const AVOption options[] = {
     {"hls_fmp4_init_filename", "set fragment mp4 file init filename", OFFSET(fmp4_init_filename),   AV_OPT_TYPE_STRING, {.str = "init.mp4"},            0,       0,         E},
     {"hls_flags",     "set flags affecting HLS playlist and media file generation", OFFSET(flags), AV_OPT_TYPE_FLAGS, {.i64 = 0 }, 0, UINT_MAX, E, "flags"},
     {"single_file",   "generate a single media file indexed with byte ranges", 0, AV_OPT_TYPE_CONST, {.i64 = HLS_SINGLE_FILE }, 0, UINT_MAX,   E, "flags"},
-    {"temp_file", "write segment to temporary file and rename when complete", 0, AV_OPT_TYPE_CONST, {.i64 = HLS_TEMP_FILE }, 0, UINT_MAX,   E, "flags"},
+    {"temp_file", "write segment and playlist to temporary file and rename when complete", 0, AV_OPT_TYPE_CONST, {.i64 = HLS_TEMP_FILE }, 0, UINT_MAX,   E, "flags"},
     {"delete_segments", "delete segment files that are no longer part of the playlist", 0, AV_OPT_TYPE_CONST, {.i64 = HLS_DELETE_SEGMENTS }, 0, UINT_MAX,   E, "flags"},
     {"round_durations", "round durations in m3u8 to whole numbers", 0, AV_OPT_TYPE_CONST, {.i64 = HLS_ROUND_DURATIONS }, 0, UINT_MAX,   E, "flags"},
     {"discont_start", "start the playlist with a discontinuity tag", 0, AV_OPT_TYPE_CONST, {.i64 = HLS_DISCONT_START }, 0, UINT_MAX,   E, "flags"},