From patchwork Tue Jan 3 08:11:42 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Bodecs Bela X-Patchwork-Id: 2031 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.89.21 with SMTP id n21csp4266945vsb; Tue, 3 Jan 2017 00:12:03 -0800 (PST) X-Received: by 10.28.212.131 with SMTP id l125mr49701764wmg.117.1483431122999; Tue, 03 Jan 2017 00:12:02 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id u197si72585784wmu.25.2017.01.03.00.12.02; Tue, 03 Jan 2017 00:12:02 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3919A689CD1; Tue, 3 Jan 2017 10:11:55 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail.vivacom.hu (mail.vivacom.hu [217.173.41.231]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 77633689A3B for ; Tue, 3 Jan 2017 10:11:48 +0200 (EET) Received: from localhost (localhost [127.0.0.1]) by mail.vivacom.hu (Postfix) with ESMTP id B686386B27 for ; Tue, 3 Jan 2017 09:11:51 +0100 (CET) X-Virus-Scanned: amavisd-new at example.com Received: from mail.vivacom.hu ([127.0.0.1]) by localhost (mail.vivacom.intra [127.0.0.1]) (amavisd-new, port 10024) with LMTP id bhQOp3PW-hWk for ; Tue, 3 Jan 2017 09:11:49 +0100 (CET) Received: from [192.168.100.137] (mail.officeline.hu [217.173.32.91]) by mail.vivacom.hu (Postfix) with ESMTPA id 0246986E78 for ; Tue, 3 Jan 2017 09:11:48 +0100 (CET) References: <211d6df8-d1d0-35a7-d493-648c65ec39b1@vivanet.hu> To: FFmpeg development discussions and patches From: Bodecs Bela Message-ID: Date: Tue, 3 Jan 2017 09:11:42 +0100 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: Subject: Re: [FFmpeg-devel] [PATCH] avformat/hlsenc: actual segment file size and duration in segment filenames X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" 2017.01.03. 2:34 keltezéssel, Steven Liu írta: > 2017-01-03 8:22 GMT+08:00 Steven Liu : > >> >> 2017-01-03 2:58 GMT+08:00 Bodecs Bela : >> >>> Dear All, >>> >>> this patch makes it possible to put actual segment file size (measured >>> in bytes) and/or duration (calculated in microseconds) into segment >>> filenames. This feature is useful when post-processing live streaming >>> access log files. New behaviour works only when -use_localtime option >>> is set and second_level_segment_size or/and >>> second_level_segment_duration new hls_flags are specified. %%s is the >>> placeholder for size and %%t for duration in hls_segment_filename >>> option. Fix sized trailing zeropadding also works eg. %%09s or %%023t. >>> >>> A command to test new features: >>> ./ffmpeg -loglevel info -y -f lavfi -i color=c=red:size=640x480:r=25 -f >>> lavfi -i sine=f=440:b=4:r=44100 -c:v mpeg2video -g 25 -acodec aac >>> -cutoff 20000 -ac 2 -ar 44100 -ab 192k -f hls -hls_time 3 -hls_list_size >>> 5 -hls_flags >>> second_level_segment_index+second_level_segment_size+second_ >>> level_segment_duration >>> -use_localtime 1 -use_localtime_mkdir 1 -hls_segment_filename >>> "segment_%Y%m%d%H%M%S_%%04d_%%08s_%%013t.ts" stream.m3u8 >>> >>> this will produce segments like this: >>> segment_20170102194334_0003_00122200_0000003000000.ts >>> segment_20170102194334_0004_00120072_0000003000000.ts >>> etc. >>> >>> >>> thank you in advance, >>> >>> Bela Bodecs >>> >>> _______________________________________________ >>> ffmpeg-devel mailing list >>> ffmpeg-devel@ffmpeg.org >>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel >>> >>> >> you should add document for the new option. >> > in +static int replace_int_data_in_filename(char *buf, int buf_size, const > char *filename, char placeholder, int64_t number) > > + c = *p; // > will add comment? > > in @@ -388,6 +443,38 @@ static int hls_append_segment(struct > AVFormatContext *s, HLSContext *hls, double > > + if ((hls->flags & (HLS_SECOND_LEVEL_SEGMENT_SIZE | > HLS_SECOND_LEVEL_SEGMENT_DURATION)) && > strlen(hls->current_segment_final_filename_fmt)) { > This is one line and the line is too long, maybe split to or three line be > better, The other too long line should be split. I have shortened and splitted the lines and removed empty comment sign. > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel From 31c9826fc8c0b68c87c94feea268cedacfb8ac9d Mon Sep 17 00:00:00 2001 From: Bela Bodecs Date: Mon, 2 Jan 2017 18:41:31 +0100 Subject: [PATCH] avformat/hlsenc: size and duration in segment filenames This patch makes it possible to put actual segment file size (measured in bytes) and/or duration (calculated in microseconds) into segment filenames. This feature is useful when post-processing live streaming access log files. New behaviour works only when -use_localtime option is set and second_level_segment_size or/and second_level_segment_duration new hls_flags are specified. %%s is the placeholder for size and %%t for duration in hls_segment_filename option. Fix sized trailing zeropadding also works eg. %%09s or %%023t. A command to test new features: ./ffmpeg -loglevel info -y -f lavfi -i color=c=red:size=640x480:r=25 -f lavfi -i sine=f=440:b=4:r=44100 -c:v mpeg2video -g 25 -acodec aac -cutoff 20000 -ac 2 -ar 44100 -ab 192k -f hls -hls_time 3 -hls_list_size 5 -hls_flags second_level_segment_index+second_level_segment_size+second_level_segment_duration -use_localtime 1 -use_localtime_mkdir 1 -hls_segment_filename "segment_%Y%m%d%H%M%S_%%04d_%%08s_%%013t.ts" stream.m3u8 Signed-off-by: Bela Bodecs --- libavformat/hlsenc.c | 174 ++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 166 insertions(+), 8 deletions(-) diff --git a/libavformat/hlsenc.c b/libavformat/hlsenc.c index 591ab73..4975d2a 100644 --- a/libavformat/hlsenc.c +++ b/libavformat/hlsenc.c @@ -67,6 +67,8 @@ typedef enum HLSFlags { HLS_APPEND_LIST = (1 << 6), HLS_PROGRAM_DATE_TIME = (1 << 7), HLS_SECOND_LEVEL_SEGMENT_INDEX = (1 << 8), // include segment index in segment filenames when use_localtime e.g.: %%03d + HLS_SECOND_LEVEL_SEGMENT_DURATION = (1 << 9), // include segment duration (microsec) in segment filenames when use_localtime e.g.: %%09t + HLS_SECOND_LEVEL_SEGMENT_SIZE = (1 << 10), // include segment size (bytes) in segment filenames when use_localtime e.g.: %%014s } HLSFlags; typedef enum { @@ -134,6 +136,7 @@ typedef struct HLSContext { char *method; double initial_prog_date_time; + char current_segment_final_filename_fmt[1024]; // when renaming segments } HLSContext; static int mkdir_p(const char *path) { @@ -169,6 +172,58 @@ static int mkdir_p(const char *path) { return ret; } +static int replace_int_data_in_filename(char *buf, int buf_size, const char *filename, char placeholder, int64_t number) +{ + const char *p; + char *q, buf1[20], c; + int nd, len, addchar_count; + int found_count = 0; + + q = buf; + p = filename; + for (;;) { + c = *p; + if (c == '\0') + break; + if (c == '%' && *(p+1) == '%') // %% + addchar_count = 2; + else if (c == '%' && (av_isdigit(*(p+1)) || *(p+1) == placeholder)) { + nd = 0; + addchar_count = 1; + while (av_isdigit(*(p + addchar_count))) { + nd = nd * 10 + *(p + addchar_count) - '0'; + addchar_count++; + } + + if (*(p + addchar_count) == placeholder) { + len = snprintf(buf1, sizeof(buf1), "%0*"PRId64, (number < 0) ? nd : nd++, number); + if (len < 1) // returned error or empty buf1 + goto fail; + if ((q - buf + len) > buf_size - 1) + goto fail; + memcpy(q, buf1, len); + q += len; + p += (addchar_count + 1); + addchar_count = 0; + found_count++; + } + + } else + addchar_count = 1; + + while (addchar_count--) + if ((q - buf) < buf_size - 1) + *q++ = *p++; + else + goto fail; + } + *q = '\0'; + return found_count; +fail: + *q = '\0'; + return -1; +} + static int hls_delete_old_segments(HLSContext *hls) { HLSSegment *segment, *previous_segment = NULL; @@ -388,6 +443,45 @@ static int hls_append_segment(struct AVFormatContext *s, HLSContext *hls, double if (!en) return AVERROR(ENOMEM); + if ((hls->flags & (HLS_SECOND_LEVEL_SEGMENT_SIZE | HLS_SECOND_LEVEL_SEGMENT_DURATION)) && + strlen(hls->current_segment_final_filename_fmt)) { + char * old_filename = av_strdup(hls->avf->filename); // %%s will be %s after strftime + av_strlcpy(hls->avf->filename, hls->current_segment_final_filename_fmt, sizeof(hls->avf->filename)); + if (hls->flags & HLS_SECOND_LEVEL_SEGMENT_SIZE) { + char * filename = av_strdup(hls->avf->filename); // %%s will be %s after strftime + if (!filename) + return AVERROR(ENOMEM); + if (replace_int_data_in_filename(hls->avf->filename, sizeof(hls->avf->filename), + filename, 's', pos + size) < 1) { + av_log(hls, AV_LOG_ERROR, + "Invalid second level segment filename template '%s', you can try to remove second_level_segment_size flag\n", + filename); + av_free(filename); + av_free(old_filename); + return AVERROR(EINVAL); + } + av_free(filename); + } + if (hls->flags & HLS_SECOND_LEVEL_SEGMENT_DURATION) { + char * filename = av_strdup(hls->avf->filename); // %%t will be %t after strftime + if (!filename) + return AVERROR(ENOMEM); + if (replace_int_data_in_filename(hls->avf->filename, sizeof(hls->avf->filename), + filename, 't', (int64_t)round(1000000 * duration)) < 1) { + av_log(hls, AV_LOG_ERROR, + "Invalid second level segment filename template '%s', you can try to remove second_level_segment_time flag\n", + filename); + av_free(filename); + av_free(old_filename); + return AVERROR(EINVAL); + } + av_free(filename); + } + ff_rename(old_filename, hls->avf->filename, hls); + av_free(old_filename); + } + + filename = av_basename(hls->avf->filename); if (hls->use_localtime_mkdir) { @@ -709,15 +803,45 @@ static int hls_start(AVFormatContext *s) char * filename = av_strdup(oc->filename); // %%d will be %d after strftime if (!filename) return AVERROR(ENOMEM); - if (av_get_frame_filename2(oc->filename, sizeof(oc->filename), - filename, c->wrap ? c->sequence % c->wrap : c->sequence, - AV_FRAME_FILENAME_FLAGS_MULTIPLE) < 0) { - av_log(c, AV_LOG_ERROR, "Invalid second level segment filename template '%s', you can try to remove second_level_segment_index flag\n", filename); + if (replace_int_data_in_filename(oc->filename, sizeof(oc->filename), + filename, 'd', c->wrap ? c->sequence % c->wrap : c->sequence) < 1) { + av_log(c, AV_LOG_ERROR, + "Invalid second level segment filename template '%s', you can try to remove second_level_segment_index flag\n", + filename); av_free(filename); return AVERROR(EINVAL); } av_free(filename); } + if (c->flags & (HLS_SECOND_LEVEL_SEGMENT_SIZE | HLS_SECOND_LEVEL_SEGMENT_DURATION)) { + av_strlcpy(c->current_segment_final_filename_fmt, oc->filename, sizeof(c->current_segment_final_filename_fmt)); + if (c->flags & HLS_SECOND_LEVEL_SEGMENT_SIZE) { + char * filename = av_strdup(oc->filename); // %%s will be %s after strftime + if (!filename) + return AVERROR(ENOMEM); + if (replace_int_data_in_filename(oc->filename, sizeof(oc->filename), filename, 's', 0) < 1) { + av_log(c, AV_LOG_ERROR, + "Invalid second level segment filename template '%s', you can try to remove second_level_segment_size flag\n", + filename); + av_free(filename); + return AVERROR(EINVAL); + } + av_free(filename); + } + if (c->flags & HLS_SECOND_LEVEL_SEGMENT_DURATION) { + char * filename = av_strdup(oc->filename); // %%t will be %t after strftime + if (!filename) + return AVERROR(ENOMEM); + if (replace_int_data_in_filename(oc->filename, sizeof(oc->filename), filename, 't', 0) < 1) { + av_log(c, AV_LOG_ERROR, + "Invalid second level segment filename template '%s', you can try to remove second_level_segment_time flag\n", + filename); + av_free(filename); + return AVERROR(EINVAL); + } + av_free(filename); + } + } if (c->use_localtime_mkdir) { const char *dir; char *fn_copy = av_strdup(oc->filename); @@ -832,6 +956,7 @@ static int hls_write_header(AVFormatContext *s) hls->sequence = hls->start_sequence; hls->recording_time = (hls->init_time ? hls->init_time : hls->time) * AV_TIME_BASE; hls->start_pts = AV_NOPTS_VALUE; + hls->current_segment_final_filename_fmt[0] = '\0'; if (hls->flags & HLS_PROGRAM_DATE_TIME) { time_t now0; @@ -906,10 +1031,41 @@ static int hls_write_header(AVFormatContext *s) av_strlcat(hls->basename, pattern, basename_size); } } - if (!hls->use_localtime && (hls->flags & HLS_SECOND_LEVEL_SEGMENT_INDEX)) { - av_log(hls, AV_LOG_ERROR, "second_level_segment_index hls_flag requires use_localtime to be true\n"); - ret = AVERROR(EINVAL); - goto fail; + if (!hls->use_localtime) { + if (hls->flags & HLS_SECOND_LEVEL_SEGMENT_DURATION) { + av_log(hls, AV_LOG_ERROR, + "second_level_segment_duration hls_flag requires use_localtime to be true\n"); + ret = AVERROR(EINVAL); + goto fail; + } + if (hls->flags & HLS_SECOND_LEVEL_SEGMENT_SIZE) { + av_log(hls, AV_LOG_ERROR, + "second_level_segment_size hls_flag requires use_localtime to be true\n"); + ret = AVERROR(EINVAL); + goto fail; + } + if (hls->flags & HLS_SECOND_LEVEL_SEGMENT_INDEX) { + av_log(hls, AV_LOG_ERROR, + "second_level_segment_index hls_flag requires use_localtime to be true\n"); + ret = AVERROR(EINVAL); + goto fail; + } + } else { + const char *proto = avio_find_protocol_name(hls->basename); + int segment_renaming_ok = proto && !strcmp(proto, "file"); + + if ((hls->flags & HLS_SECOND_LEVEL_SEGMENT_DURATION) && !segment_renaming_ok) { + av_log(hls, AV_LOG_ERROR, + "second_level_segment_duration hls_flag works only with file protocol segment names\n"); + ret = AVERROR(EINVAL); + goto fail; + } + if ((hls->flags & HLS_SECOND_LEVEL_SEGMENT_SIZE) && !segment_renaming_ok) { + av_log(hls, AV_LOG_ERROR, + "second_level_segment_size hls_flag works only with file protocol segment names\n"); + ret = AVERROR(EINVAL); + goto fail; + } } if(hls->has_subtitle) { @@ -1167,6 +1323,8 @@ static const AVOption options[] = { {"append_list", "append the new segments into old hls segment list", 0, AV_OPT_TYPE_CONST, {.i64 = HLS_APPEND_LIST }, 0, UINT_MAX, E, "flags"}, {"program_date_time", "add EXT-X-PROGRAM-DATE-TIME", 0, AV_OPT_TYPE_CONST, {.i64 = HLS_PROGRAM_DATE_TIME }, 0, UINT_MAX, E, "flags"}, {"second_level_segment_index", "include segment index in segment filenames when use_localtime", 0, AV_OPT_TYPE_CONST, {.i64 = HLS_SECOND_LEVEL_SEGMENT_INDEX }, 0, UINT_MAX, E, "flags"}, + {"second_level_segment_duration", "include segment duration in segment filenames when use_localtime", 0, AV_OPT_TYPE_CONST, {.i64 = HLS_SECOND_LEVEL_SEGMENT_DURATION }, 0, UINT_MAX, E, "flags"}, + {"second_level_segment_size", "include segment size in segment filenames when use_localtime", 0, AV_OPT_TYPE_CONST, {.i64 = HLS_SECOND_LEVEL_SEGMENT_SIZE }, 0, UINT_MAX, E, "flags"}, {"use_localtime", "set filename expansion with strftime at segment creation", OFFSET(use_localtime), AV_OPT_TYPE_BOOL, {.i64 = 0 }, 0, 1, E }, {"use_localtime_mkdir", "create last directory component in strftime-generated filename", OFFSET(use_localtime_mkdir), AV_OPT_TYPE_BOOL, {.i64 = 0 }, 0, 1, E }, {"hls_playlist_type", "set the HLS playlist type", OFFSET(pl_type), AV_OPT_TYPE_INT, {.i64 = PLAYLIST_TYPE_NONE }, 0, PLAYLIST_TYPE_NB-1, E, "pl_type" }, -- 2.5.3.windows.1