From patchwork Mon Jun 24 16:01:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bodecs Bela X-Patchwork-Id: 13690 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 43EEF448509 for ; Mon, 24 Jun 2019 19:01:34 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1FD8868A96C; Mon, 24 Jun 2019 19:01:34 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail.vivacom.hu (mail.vivacom.hu [185.92.116.29]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2DDB16899F9 for ; Mon, 24 Jun 2019 19:01:27 +0300 (EEST) Received: from localhost (localhost [127.0.0.1]) by mail.vivacom.hu (Postfix) with ESMTP id 9D870881EB for ; Mon, 24 Jun 2019 18:01:26 +0200 (CEST) X-Virus-Scanned: amavisd-new at example.com Received: from mail.vivacom.hu ([127.0.0.1]) by localhost (mail.vivacom.intra [127.0.0.1]) (amavisd-new, port 10024) with LMTP id EpZcYZP-XqL0 for ; Mon, 24 Jun 2019 18:01:23 +0200 (CEST) Received: from [192.168.3.8] (5402FC4C.dsl.pool.telekom.hu [84.2.252.76]) by mail.vivacom.hu (Postfix) with ESMTPA id BC1CE87F17 for ; Mon, 24 Jun 2019 18:01:23 +0200 (CEST) To: FFmpeg development discussions and patches From: Bodecs Bela Message-ID: <6c70f542-4e16-31f7-95e1-c59b8361d5a2@vivanet.hu> Date: Mon, 24 Jun 2019 18:01:22 +0200 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.7.2 MIME-Version: 1.0 Content-Language: hu Subject: [FFmpeg-devel] [PATCH] avformat/hlsenc: temp_file usage for master playlist and vtt playlist X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Dear All, currently master playlist and subtitle playlist creation does not use temporary files even when temp_file flag is set. Most of the use cases it is not a problem because master playlist creation happens once on the beginning of the whole process. But if master playlist is periodically re-created because of master_pl_refresh_rate is set, non-atomic playlist creation may cause problems in case of live streaming. This poblem (i.e non-atomic playlist creation) may also apply for subtitle playlist (vtt) creation in live streaming. This patch correct this behavior by adding missing functionality. please review this patch. thank you in advance, best regards, Bela From 04e70ba586646b927e1b05a9df3860a635871603 Mon Sep 17 00:00:00 2001 From: Bela Bodecs Date: Mon, 24 Jun 2019 17:41:49 +0200 Subject: [PATCH] avformat/hlsenc: temp_file usage for master playlist and vtt playlist currently master playlist and subtitle playlist creation does not use temporary files even when temp_file flag is set. Most of the use cases it is not a problem because master playlist creation happens once on the beginning of the whole process. But if master playlist is periodically re-created because of master_pl_refresh_rate is set, non-atomic playlist creation may cause problems in case of live streaming. This patch correct this behavior by adding this functionality. Signed-off-by: Bela Bodecs --- doc/muxers.texi | 6 +++++- libavformat/hlsenc.c | 30 +++++++++++++++++++++--------- 2 files changed, 26 insertions(+), 10 deletions(-) diff --git a/doc/muxers.texi b/doc/muxers.texi index 6c5b4bb637..d969e39fff 100644 --- a/doc/muxers.texi +++ b/doc/muxers.texi @@ -893,7 +893,11 @@ This will produce segments like this: @item temp_file Write segment data to filename.tmp and rename to filename only once the segment is complete. A webserver serving up segments can be configured to reject requests to *.tmp to prevent access to in-progress segments -before they have been added to the m3u8 playlist. +before they have been added to the m3u8 playlist. This flag also affects how m3u8 playlist files are created. +If this flag is set, all playlist files will written into temporary file and renamed after they are complete, similarly as segments are handled. +But playlists with @code{file} protocol and with type (@code{hls_playlist_type}) other than @code{vod} +are always written into temporary file regardles of this flag. Master playlist files (@code{master_pl_name}), if any, with @code{file} protocol, +are always written into temporary file regardles of this flag if @code{master_pl_publish_rate} value is other than zero. @end table diff --git a/libavformat/hlsenc.c b/libavformat/hlsenc.c index 9f5eee5491..eaeafcbb6b 100644 --- a/libavformat/hlsenc.c +++ b/libavformat/hlsenc.c @@ -1260,8 +1260,12 @@ static int create_master_playlist(AVFormatContext *s, AVDictionary *options = NULL; unsigned int i, j; int m3u8_name_size, ret, bandwidth; - char *m3u8_rel_name, *ccgroup; + char *m3u8_rel_name = NULL, *ccgroup; ClosedCaptionsStream *ccs; + const char *proto = avio_find_protocol_name(hls->master_m3u8_url); + int is_file_proto = proto && !strcmp(proto, "file"); + int use_temp_file = is_file_proto && ((hls->flags & HLS_TEMP_FILE) || hls->master_publish_rate); + char temp_filename[1024]; input_vs->m3u8_created = 1; if (!hls->master_m3u8_created) { @@ -1277,12 +1281,12 @@ static int create_master_playlist(AVFormatContext *s, } set_http_options(s, &options, hls); - - ret = hlsenc_io_open(s, &hls->m3u8_out, hls->master_m3u8_url, &options); + snprintf(temp_filename, sizeof(temp_filename), use_temp_file ? "%s.tmp" : "%s", hls->master_m3u8_url); + ret = hlsenc_io_open(s, &hls->m3u8_out, temp_filename, &options); av_dict_free(&options); if (ret < 0) { av_log(NULL, AV_LOG_ERROR, "Failed to open master play list file '%s'\n", - hls->master_m3u8_url); + temp_filename); goto fail; } @@ -1413,7 +1417,10 @@ fail: if(ret >=0) hls->master_m3u8_created = 1; av_freep(&m3u8_rel_name); - hlsenc_io_close(s, &hls->m3u8_out, hls->master_m3u8_url); + hlsenc_io_close(s, &hls->m3u8_out, temp_filename); + if (use_temp_file) + ff_rename(temp_filename, hls->master_m3u8_url, s); + return ret; } @@ -1424,6 +1431,7 @@ static int hls_window(AVFormatContext *s, int last, VariantStream *vs) int target_duration = 0; int ret = 0; char temp_filename[1024]; + char temp_vtt_filename[1024]; int64_t sequence = FFMAX(hls->start_sequence, vs->sequence - vs->nb_entries); const char *proto = avio_find_protocol_name(vs->m3u8_name); int is_file_proto = proto && !strcmp(proto, "file"); @@ -1505,8 +1513,9 @@ static int hls_window(AVFormatContext *s, int last, VariantStream *vs) if (last && (hls->flags & HLS_OMIT_ENDLIST)==0) ff_hls_write_end_list(hls->m3u8_out); - if( vs->vtt_m3u8_name ) { - if ((ret = hlsenc_io_open(s, &hls->sub_m3u8_out, vs->vtt_m3u8_name, &options)) < 0) { + if (vs->vtt_m3u8_name) { + snprintf(temp_vtt_filename, sizeof(temp_vtt_filename), use_temp_file ? "%s.tmp" : "%s", vs->vtt_m3u8_name); + if ((ret = hlsenc_io_open(s, &hls->sub_m3u8_out, temp_vtt_filename, &options)) < 0) { if (hls->ignore_io_errors) ret = 0; goto fail; @@ -1531,8 +1540,11 @@ fail: av_dict_free(&options); hlsenc_io_close(s, &hls->m3u8_out, temp_filename); hlsenc_io_close(s, &hls->sub_m3u8_out, vs->vtt_m3u8_name); - if (use_temp_file) + if (use_temp_file) { ff_rename(temp_filename, vs->m3u8_name, s); + if (vs->vtt_m3u8_name) + ff_rename(temp_vtt_filename, vs->vtt_m3u8_name, s); + } if (ret >= 0 && hls->master_pl_name) if (create_master_playlist(s, vs) < 0) av_log(s, AV_LOG_WARNING, "Master playlist creation failed\n"); @@ -2991,7 +3003,7 @@ static const AVOption options[] = { {"hls_fmp4_init_filename", "set fragment mp4 file init filename", OFFSET(fmp4_init_filename), AV_OPT_TYPE_STRING, {.str = "init.mp4"}, 0, 0, E}, {"hls_flags", "set flags affecting HLS playlist and media file generation", OFFSET(flags), AV_OPT_TYPE_FLAGS, {.i64 = 0 }, 0, UINT_MAX, E, "flags"}, {"single_file", "generate a single media file indexed with byte ranges", 0, AV_OPT_TYPE_CONST, {.i64 = HLS_SINGLE_FILE }, 0, UINT_MAX, E, "flags"}, - {"temp_file", "write segment to temporary file and rename when complete", 0, AV_OPT_TYPE_CONST, {.i64 = HLS_TEMP_FILE }, 0, UINT_MAX, E, "flags"}, + {"temp_file", "write segment and playlist to temporary file and rename when complete", 0, AV_OPT_TYPE_CONST, {.i64 = HLS_TEMP_FILE }, 0, UINT_MAX, E, "flags"}, {"delete_segments", "delete segment files that are no longer part of the playlist", 0, AV_OPT_TYPE_CONST, {.i64 = HLS_DELETE_SEGMENTS }, 0, UINT_MAX, E, "flags"}, {"round_durations", "round durations in m3u8 to whole numbers", 0, AV_OPT_TYPE_CONST, {.i64 = HLS_ROUND_DURATIONS }, 0, UINT_MAX, E, "flags"}, {"discont_start", "start the playlist with a discontinuity tag", 0, AV_OPT_TYPE_CONST, {.i64 = HLS_DISCONT_START }, 0, UINT_MAX, E, "flags"},