From patchwork Sun Aug 23 12:23:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongcheng Zhong X-Patchwork-Id: 21840 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id C04F9449B15 for ; Sun, 23 Aug 2020 15:24:20 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A691E6897CD; Sun, 23 Aug 2020 15:24:20 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from smtp181.sjtu.edu.cn (smtp181.sjtu.edu.cn [202.120.2.181]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2D01668973D for ; Sun, 23 Aug 2020 15:24:13 +0300 (EEST) Received: from proxy02.sjtu.edu.cn (smtp188.sjtu.edu.cn [202.120.2.188]) by smtp181.sjtu.edu.cn (Postfix) with ESMTPS id 9AACD1008CBCD; Sun, 23 Aug 2020 20:24:10 +0800 (CST) Received: from localhost (localhost.localdomain [127.0.0.1]) by proxy02.sjtu.edu.cn (Postfix) with ESMTP id 644202008CB41; Sun, 23 Aug 2020 20:24:10 +0800 (CST) X-Virus-Scanned: amavisd-new at Received: from proxy02.sjtu.edu.cn ([127.0.0.1]) by localhost (proxy02.sjtu.edu.cn [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id dgw9WxW3nc9y; Sun, 23 Aug 2020 20:24:10 +0800 (CST) Received: from localhost.localdomain (unknown [10.162.157.92]) (Authenticated sender: sj.hc_Zhong@sjtu.edu.cn) by proxy02.sjtu.edu.cn (Postfix) with ESMTPSA id AF59D200A4B00; Sun, 23 Aug 2020 20:24:05 +0800 (CST) From: Hongcheng Zhong To: ffmpeg-devel@ffmpeg.org Date: Sun, 23 Aug 2020 20:23:51 +0800 Message-Id: <20200823122355.188611-3-sj.hc_Zhong@sjtu.edu.cn> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200823122355.188611-1-sj.hc_Zhong@sjtu.edu.cn> References: <20200823122355.188611-1-sj.hc_Zhong@sjtu.edu.cn> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] [GSoC v3 3/7] avformat/hls: use abr to switch streams X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: spartazhc Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: spartazhc When abr is enable, it will take over the task to call http to download segments, and will return a switch-request for hls to switch streams. For reason not to waste segments that have been downloaded, switch will become effective after old segments is used out. Abr cannot work with http_persistent option, and currently use http_multiple. Signed-off-by: spartazhc v1 fixed: 1. fix memory leak v2 fixed: 1. check malloc result and return AVERROR(ENOMEM) 2. define ABR_THROUGHPUT_FIFO_LEN instead of hardcode v3 fixed: 1. hls should switch variant other than playlist, two structure switch_info and switch_task are added to support arbitrary switch request in different manifest 2. bug fix in throughput update 3. added switch_step to support http_multiple option 4. fix several segment faults --- doc/demuxers.texi | 3 + libavformat/hls.c | 405 +++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 402 insertions(+), 6 deletions(-) diff --git a/doc/demuxers.texi b/doc/demuxers.texi index 3c15ab9eee..4cdbd95962 100644 --- a/doc/demuxers.texi +++ b/doc/demuxers.texi @@ -321,6 +321,9 @@ available in a metadata key named "variant_bitrate". It accepts the following options: @table @option +@item abr +enable abr to switch streams. + @item live_start_index segment index to start live streams at (negative values are from the end). diff --git a/libavformat/hls.c b/libavformat/hls.c index 3ab07f1b3f..4e760f8e8a 100644 --- a/libavformat/hls.c +++ b/libavformat/hls.c @@ -47,6 +47,7 @@ #define MPEG_TIME_BASE 90000 #define MPEG_TIME_BASE_Q (AVRational){1, MPEG_TIME_BASE} +#define ABR_THROUGHPUT_FIFO_LEN 20 /* * An apple http stream consists of a playlist with media segment files, * played sequentially. There may be several playlists with the same @@ -65,6 +66,13 @@ enum KeyType { KEY_SAMPLE_AES }; +enum SwitchType { + SWITCH_VIDEO, + SWITCH_AUDIO, + SWITCH_VIDEO_AUDIO, + SWITCH_TYPES +}; + struct segment { int64_t duration; int64_t url_offset; @@ -189,6 +197,26 @@ struct variant { char subtitles_group[MAX_FIELD_LEN]; }; +struct throughput { + int n_throughputs; + + /* throughputs are in kbps */ + float throughput_fifo[ABR_THROUGHPUT_FIFO_LEN]; + int head; + int tail; +}; + +struct switch_task { + enum SwitchType type; + int64_t switch_timestamp; +}; + +struct switch_info { + int pls_index; + int64_t first_timestamp; + int64_t delta_timestamp; +}; + typedef struct HLSContext { AVClass *class; AVFormatContext *ctx; @@ -213,8 +241,58 @@ typedef struct HLSContext { int http_multiple; int http_seekable; AVIOContext *playlist_pb; + + int abr; + struct throughput *throughputs; + struct switch_task *switch_tasks; + struct switch_info *switch_info; + int switch_inited; + int can_switch; + int switch_request; + int switch_step; + int cur_var; } HLSContext; +static struct segment *next_segment(struct playlist *pls); +static int open_input(HLSContext *c, struct playlist *pls, struct segment *seg, AVIOContext **in); +static int select_cur_seq_no(HLSContext *c, struct playlist *pls); +static AVRational get_timebase(struct playlist *pls); + +static struct segment *next2_segment(struct playlist *pls) +{ + int n = pls->cur_seq_no - pls->start_seq_no + 2; + if (n >= pls->n_segments) + return NULL; + return pls->segments[n]; +} + +static int playlist_type_full(struct playlist *pls) +{ + if (pls->n_main_streams == 1) { + return (enum SwitchType)pls->main_streams[0]->codecpar->codec_type; + } else { + return SWITCH_VIDEO_AUDIO; + } +} + +static int playlist_type_simple(struct playlist *pls) +{ + int type = playlist_type_full(pls); + if (type == SWITCH_VIDEO_AUDIO) + type = SWITCH_VIDEO; + return type; +} + +static int is_pls_switch_to(HLSContext *c, int index) { + if (c->switch_request == -1) + return 0; + for (int i = 0; i < c->variants[c->switch_request]->n_playlists; i++) { + if (c->variants[c->switch_request]->playlists[i]->index == index) + return 1; + } + return 0; +} + static void free_segment_dynarray(struct segment **segments, int n_segments) { int i; @@ -616,6 +694,38 @@ static int open_url_keepalive(AVFormatContext *s, AVIOContext **pb, #endif } +static int update_throughputs(struct throughput *thr, float time, int pb_size) +{ + if (pb_size <= 0 || time <= 0) + return AVERROR(EINVAL); + if (thr->n_throughputs < ABR_THROUGHPUT_FIFO_LEN) { + ++thr->n_throughputs; + } else { + thr->head = (thr->head + 1) % ABR_THROUGHPUT_FIFO_LEN; + } + thr->throughput_fifo[thr->tail] = (float)(pb_size) / time; + thr->tail = (thr->tail + 1) % ABR_THROUGHPUT_FIFO_LEN; + return 0; +} + +static int64_t get_switch_timestamp(HLSContext *c, struct playlist *pls) +{ + int64_t first_timestamp, pos; + int type; + int n = pls->cur_seq_no + c->switch_step; + if (n >= pls->n_segments) + return -1; + + type = playlist_type_simple(pls); + first_timestamp = c->switch_info[type].first_timestamp; + pos = first_timestamp == AV_NOPTS_VALUE ? 0 : first_timestamp; + + for (int i = 0; i < n; i++) { + pos += pls->segments[i]->duration; + } + return pos; +} + static int open_url(AVFormatContext *s, AVIOContext **pb, const char *url, AVDictionary *opts, AVDictionary *opts2, int *is_http_out) { @@ -631,6 +741,9 @@ static int open_url(AVFormatContext *s, AVIOContext **pb, const char *url, } else if (av_strstart(url, "data", NULL)) { if (url[4] == '+' || url[4] == ':') proto_name = avio_find_protocol_name(url + 5); + } else if (av_strstart(url, "ffabr", NULL)) { + if (url[5] == '+' || url[5] == ':') + proto_name = avio_find_protocol_name(url + 6); } if (!proto_name) @@ -661,6 +774,8 @@ static int open_url(AVFormatContext *s, AVIOContext **pb, const char *url, ; else if (av_strstart(url, "data", NULL) && !strncmp(proto_name, url + 5, strlen(proto_name)) && url[5 + strlen(proto_name)] == ':') ; + else if (av_strstart(url, "ffabr", NULL) && !strncmp(proto_name, url + 6, strlen(proto_name)) && url[6 + strlen(proto_name)] == ':') + ; else if (strcmp(proto_name, "file") || !strncmp(url, "file,", 5)) return AVERROR_INVALIDDATA; @@ -682,6 +797,69 @@ static int open_url(AVFormatContext *s, AVIOContext **pb, const char *url, } else { ret = s->io_open(s, pb, url, AVIO_FLAG_READ, &tmp); } + if (c->abr && ret >= 0) { + AVDictionary *abr_ret = NULL; + AVDictionaryEntry *en = NULL; + struct segment *seg; + int pb_size, switch_request; + enum SwitchType type; + av_opt_get_dict_val(*pb, "abr-metadata", AV_OPT_SEARCH_CHILDREN, &abr_ret); + if (abr_ret) { + en = av_dict_get(abr_ret, "download_time", NULL, 0); + if (en) { + pb_size = avio_size(*pb); + update_throughputs(c->throughputs, strtol(en->value, NULL, 10) / 1000.0, pb_size); + av_log(s, AV_LOG_VERBOSE, "[abr] time=%.2fms, size=%.2fkb\n", + strtol(en->value, NULL, 10) / 1000.0, pb_size / 1000.0); + } + en = av_dict_get(abr_ret, "switch_request", NULL, 0); + if (en) { + switch_request = strtol(en->value, NULL, 10); + if (switch_request != -1) + av_log(s, AV_LOG_INFO, "[abr] switch request: %s\n", en->value); + } + en = av_dict_get(abr_ret, "type", NULL, 0); + if (en) { + type = strtol(en->value, NULL, 10); + } + if (switch_request != -1) { + struct variant *var = c->variants[switch_request]; + c->switch_request = switch_request; + c->can_switch = 0; + for (int i = 0; i < var->n_playlists; i++) { + struct playlist *pls = var->playlists[i]; + int64_t switch_timestamp; + pls->cur_seq_no = select_cur_seq_no(c, pls); + // if pls has same type to the segment just downloaded, switch should be delayed + if (type == SWITCH_VIDEO_AUDIO || playlist_type_full(pls) == type) { + pls->cur_seq_no++; + } + if (c->switch_step == 2) { + seg = next2_segment(pls); + } else { + seg = next_segment(pls); + } + switch_timestamp = get_switch_timestamp(c, pls); + if (!seg || switch_timestamp == -1) { + c->switch_request = -1; + av_log(s, AV_LOG_INFO, "[abr] no more segment need to switch\n"); + } else { + int ptype; + ptype = playlist_type_simple(pls); + c->switch_tasks[pls->index].type = ptype; + c->switch_tasks[pls->index].switch_timestamp = switch_timestamp - c->switch_info[ptype].delta_timestamp * 1.5; + av_log(s, AV_LOG_INFO, "[abr] pls%d, switch type: %d timestamp: %ld\n", + pls->index, c->switch_tasks[pls->index].type, c->switch_tasks[pls->index].switch_timestamp); + if (c->switch_step == 2) { + pls->input_next_requested = 1; + ret = open_input(c, pls, seg, &pls->input_next); + } + } + } + } + av_dict_free(&abr_ret); + } + } if (ret >= 0) { // update cookies on http response with setcookies. char *new_cookies = NULL; @@ -1211,6 +1389,38 @@ static void intercept_id3(struct playlist *pls, uint8_t *buf, pls->is_id3_timestamped = (pls->id3_mpegts_timestamp != AV_NOPTS_VALUE); } +static int abrinfo_to_dict(HLSContext *c, enum SwitchType type, char **abr_info) +{ + struct throughput *thr = c->throughputs; + char buffer[MAX_URL_SIZE]; + int size, i, j; + size = snprintf(buffer, sizeof(buffer), "format=hls:"); + size += snprintf(buffer + size, sizeof(buffer) - size, "cur_var=%d:", c->cur_var); + size += snprintf(buffer + size, sizeof(buffer) - size, "type=%d:", type); + size += snprintf(buffer + size, sizeof(buffer) - size, "can_switch=%d:", c->can_switch); + size += snprintf(buffer + size, sizeof(buffer) - size, "n_variants=%d:", c->n_variants); + for (i = 0; i < c->n_variants; i++) { + struct variant *v = c->variants[i]; + size += snprintf(buffer + size, sizeof(buffer) - size, "variant_bitrate%d=%d:", i, v->bandwidth); + } + size += snprintf(buffer + size, sizeof(buffer) - size, "n_throughputs=%d:", thr->n_throughputs); + if (thr->n_throughputs > 0) { + i = thr->head; + j = 0; + do { + size += snprintf(buffer + size, sizeof(buffer) - size, "throughputs%d=%.2f:", j, thr->throughput_fifo[i]); + j++; + i = (i + 1) % ABR_THROUGHPUT_FIFO_LEN; + } while (i != thr->tail); + } + *abr_info = av_malloc(size); + if (!abr_info) + return AVERROR(ENOMEM); + snprintf(*abr_info, size, "%s", buffer); + av_log(c, AV_LOG_VERBOSE, "[abr] abr_info: %s\n", *abr_info); + return 0; +} + static int open_input(HLSContext *c, struct playlist *pls, struct segment *seg, AVIOContext **in) { AVDictionary *opts = NULL; @@ -1230,8 +1440,20 @@ static int open_input(HLSContext *c, struct playlist *pls, struct segment *seg, av_log(pls->parent, AV_LOG_VERBOSE, "HLS request for url '%s', offset %"PRId64", playlist %d\n", seg->url, seg->url_offset, pls->index); + if (c->abr) { + char *abr_opts; + abrinfo_to_dict(c, playlist_type_full(pls), &abr_opts); + av_dict_set(&opts, "abr-params", abr_opts, 0); + av_free(abr_opts); + } if (seg->key_type == KEY_NONE) { - ret = open_url(pls->parent, in, seg->url, c->avio_opts, opts, &is_http); + char url_abr[MAX_URL_SIZE]; + if (c->abr) { + snprintf(url_abr, sizeof(url_abr), "ffabr:%s", seg->url); // + or : tbd + ret = open_url(pls->parent, in, url_abr, c->avio_opts, opts, &is_http); + } else { + ret = open_url(pls->parent, in, seg->url, c->avio_opts, opts, &is_http); + } } else if (seg->key_type == KEY_AES_128) { char iv[33], key[33], url[MAX_URL_SIZE]; if (strcmp(seg->key, pls->key_url)) { @@ -1427,7 +1649,15 @@ restart: /* Check that the playlist is still needed before opening a new * segment. */ v->needed = playlist_needed(v); - + if (c->abr) { + if (c->switch_request == -1) + ; + else if (v->needed && !is_pls_switch_to(c, v->index) && !v->input_next_requested) { + av_log(v->parent, AV_LOG_VERBOSE, "read_data: needed but not download playlist %d ('%s')\n", + v->index, v->url); + return AVERROR_EOF; + } + } if (!v->needed) { av_log(v->parent, AV_LOG_INFO, "No longer receiving playlist %d ('%s')\n", v->index, v->url); @@ -1517,12 +1747,22 @@ reload: int r = av_opt_get(v->input, "http_version", AV_OPT_SEARCH_CHILDREN, &http_version_opt); if (r >= 0) { c->http_multiple = (!strncmp((const char *)http_version_opt, "1.1", 3) || !strncmp((const char *)http_version_opt, "2.0", 3)); + c->switch_step = c->http_multiple + 1; av_freep(&http_version_opt); + } else { + c->switch_step = 1; } + } else { + c->switch_step = c->http_multiple + 1; } seg = next_segment(v); - if (c->http_multiple == 1 && !v->input_next_requested && + + // when get switch_request, old stream should stop downloading next seg + if (c->abr && c->switch_inited && c->switch_request != -1 && c->http_multiple == 1 && !is_pls_switch_to(c, v->index) + && ((c->cur_timestamp + seg->duration * c->switch_step) >= c->switch_tasks[v->index].switch_timestamp)) + av_log(v->parent, AV_LOG_VERBOSE, "old playlist %d should stop downloading next seg\n", v->index); + else if (c->http_multiple == 1 && !v->input_next_requested && seg && seg->key_type == KEY_NONE && av_strstart(seg->url, "http", NULL)) { ret = open_input(c, v, seg, &v->input_next); if (ret < 0) { @@ -1555,7 +1795,7 @@ reload: return ret; } - if (c->http_persistent && + if (c->http_persistent && !c->abr && seg->key_type == KEY_NONE && av_strstart(seg->url, "http", NULL)) { v->input_read_done = 1; } else { @@ -1833,7 +2073,11 @@ static int hls_close(AVFormatContext *s) free_playlist_list(c); free_variant_list(c); free_rendition_list(c); - + if (c->abr) { + av_free(c->throughputs); + av_free(c->switch_tasks); + av_free(c->switch_info); + } av_dict_free(&c->avio_opts); ff_format_io_close(c->ctx, &c->playlist_pb); @@ -1852,6 +2096,19 @@ static int hls_read_header(AVFormatContext *s) c->first_packet = 1; c->first_timestamp = AV_NOPTS_VALUE; c->cur_timestamp = AV_NOPTS_VALUE; + c->cur_var = -1; + c->switch_request = -1; + c->switch_step = 1; + c->can_switch = -1; + + if (c->abr) { + c->http_persistent = 0; + c->throughputs = av_mallocz(sizeof(struct throughput)); + if (!c->throughputs) { + ret = AVERROR(ENOMEM); + goto fail; + } + } if ((ret = save_avio_options(s)) < 0) goto fail; @@ -2042,6 +2299,37 @@ static int hls_read_header(AVFormatContext *s) add_metadata_from_renditions(s, pls, AVMEDIA_TYPE_SUBTITLE); } + if (c->abr) { + c->switch_tasks = av_malloc(c->n_playlists * sizeof(struct switch_task)); + if (!c->switch_tasks) { + ret = AVERROR(ENOMEM); + goto fail; + } + c->switch_info = av_malloc((SWITCH_TYPES - 1) * sizeof(struct switch_info)); + if (!c->switch_info) { + ret = AVERROR(ENOMEM); + goto fail; + } + for (i = 0; i < SWITCH_TYPES - 1; i++) { + c->switch_info[i].pls_index = -1; + c->switch_info[i].first_timestamp = AV_NOPTS_VALUE; + c->switch_info[i].delta_timestamp = AV_NOPTS_VALUE; + } + for (i = 0; i < c->n_playlists; i++) { + struct playlist *pls = c->playlists[i]; + int type; + + c->switch_tasks[i].switch_timestamp = AV_NOPTS_VALUE; + type = playlist_type_simple(pls); + if (type == SWITCH_AUDIO) { + if (c->switch_info[SWITCH_AUDIO].pls_index == -1) + c->switch_info[SWITCH_AUDIO].pls_index = pls->index; + } else { + if (c->switch_info[SWITCH_VIDEO].pls_index == -1) + c->switch_info[SWITCH_VIDEO].pls_index = pls->index; + } + } + } update_noheader_flag(s); return 0; @@ -2050,12 +2338,71 @@ fail: return ret; } +static void change_discard_flags(struct playlist *pls, enum SwitchType type, int flag) +{ + int process_all = 0; + if (type == SWITCH_VIDEO_AUDIO) + process_all = 1; + for (int j = 0; j < pls->n_main_streams; j++) { + if (process_all || pls->main_streams[j]->codecpar->codec_type == (enum AVMediaType)type) + pls->main_streams[j]->discard = flag; + } +} + +static int find_current_variant(HLSContext *c) +{ + int i, j, needed; + for (i = 0; i < c->n_variants; i++) { + needed = 1; + for (j = 0; j < c->variants[i]->n_playlists; j++) { + struct playlist *pls = c->variants[i]->playlists[j]; + if (!playlist_needed(pls)) { + needed = 0; + } + } + if (needed) + return i; + } + return -1; +} + static int recheck_discard_flags(AVFormatContext *s, int first) { HLSContext *c = s->priv_data; int i, changed = 0; int cur_needed; + if (c->abr && c->switch_request != -1) { + struct variant *var = c->variants[c->switch_request]; + int no_switch_task = 1; + for (int i = 0; i < var->n_playlists; i++) { + struct playlist *pls = var->playlists[i]; + if (c->switch_tasks[pls->index].switch_timestamp != AV_NOPTS_VALUE + && c->cur_timestamp >= c->switch_tasks[pls->index].switch_timestamp) { + av_log(s, AV_LOG_INFO, "[switch point] cur_timestamp:%ld\n", c->cur_timestamp); + change_discard_flags(pls, c->switch_tasks[pls->index].type, AVDISCARD_DEFAULT); + c->switch_tasks[pls->index].switch_timestamp = AV_NOPTS_VALUE; + + for (int j = 0; j < c->n_playlists; j++) { + struct playlist *cpls = c->playlists[j]; + if (cpls == pls) + continue; + change_discard_flags(cpls, c->switch_tasks[pls->index].type, AVDISCARD_ALL); + } + } + } + + for (int k = 0; k < c->n_playlists; ++k) { + if (c->switch_tasks[k].switch_timestamp != AV_NOPTS_VALUE) { + no_switch_task = 0; + } + } + if (no_switch_task) { + c->can_switch = 1; + c->cur_var = c->switch_request; + } + } + /* Check if any new streams are needed */ for (i = 0; i < c->n_playlists; i++) { struct playlist *pls = c->playlists[i]; @@ -2070,6 +2417,11 @@ static int recheck_discard_flags(AVFormatContext *s, int first) changed = 1; pls->cur_seq_no = select_cur_seq_no(c, pls); pls->pb.eof_reached = 0; + if (c->abr) { + pls->cur_seq_no++; + avio_flush(&pls->pb); + avformat_flush(pls->ctx); + } if (c->cur_timestamp != AV_NOPTS_VALUE) { /* catch up */ pls->seek_timestamp = c->cur_timestamp; @@ -2077,7 +2429,7 @@ static int recheck_discard_flags(AVFormatContext *s, int first) pls->seek_stream_index = -1; } av_log(s, AV_LOG_INFO, "Now receiving playlist %d, segment %d\n", i, pls->cur_seq_no); - } else if (first && !cur_needed && pls->needed) { + } else if ((first || c->abr) && !cur_needed && pls->needed) { ff_format_io_close(pls->parent, &pls->input); pls->input_read_done = 0; ff_format_io_close(pls->parent, &pls->input_next); @@ -2087,6 +2439,13 @@ static int recheck_discard_flags(AVFormatContext *s, int first) av_log(s, AV_LOG_INFO, "No longer receiving playlist %d\n", i); } } + + if (c->abr && changed && c->cur_var == -1) { + c->cur_var = find_current_variant(c); + av_log(s, AV_LOG_INFO, "Find current variant %d\n", c->cur_var); + if (c->cur_var != -1 && c->can_switch == -1) + c->can_switch = 1; + } return changed; } @@ -2159,7 +2518,39 @@ static int hls_read_packet(AVFormatContext *s, AVPacket *pkt) /* audio elementary streams are id3 timestamped */ fill_timing_for_id3_timestamped_stream(pls); } + if (c->abr && !c->switch_inited && pls->pkt.dts != AV_NOPTS_VALUE) { + for (int j = 0; j < SWITCH_TYPES - 1; j++) { + if (c->switch_info[j].first_timestamp != AV_NOPTS_VALUE + && c->switch_info[j].delta_timestamp == AV_NOPTS_VALUE + && c->switch_info[j].pls_index == pls->index) { + c->switch_info[j].delta_timestamp = + av_rescale_q(pls->pkt.dts, get_timebase(pls), AV_TIME_BASE_Q) + - c->switch_info[j].first_timestamp; + av_log(s, AV_LOG_VERBOSE, "[abr_delta] pls%d dts=%ld\n", pls->index, pls->pkt.dts); + } + if (c->switch_info[j].first_timestamp == AV_NOPTS_VALUE + && c->switch_info[j].pls_index == pls->index) { + c->switch_info[j].first_timestamp = av_rescale_q(pls->pkt.dts, + get_timebase(pls), AV_TIME_BASE_Q); + av_log(s, AV_LOG_VERBOSE, "[abr_first] pls%d dts=%ld\n", pls->index, pls->pkt.dts); + } + } + if ((c->switch_info[SWITCH_AUDIO].pls_index == -1 || + c->switch_info[SWITCH_AUDIO].delta_timestamp != AV_NOPTS_VALUE) + && (c->switch_info[SWITCH_VIDEO].pls_index == -1 || + c->switch_info[SWITCH_VIDEO].delta_timestamp != AV_NOPTS_VALUE)) { + c->switch_inited = 1; + if (c->switch_info[SWITCH_VIDEO].pls_index != -1) + av_log(s, AV_LOG_VERBOSE, "[abr_init] video:first=%ld, delta=%ld\n", + c->switch_info[SWITCH_VIDEO].first_timestamp, + c->switch_info[SWITCH_VIDEO].delta_timestamp); + if (c->switch_info[SWITCH_AUDIO].pls_index != -1) + av_log(s, AV_LOG_VERBOSE, "[abr_init] audio:first=%ld, delta=%ld\n", + c->switch_info[SWITCH_AUDIO].first_timestamp, + c->switch_info[SWITCH_AUDIO].delta_timestamp); + } + } if (c->first_timestamp == AV_NOPTS_VALUE && pls->pkt.dts != AV_NOPTS_VALUE) c->first_timestamp = av_rescale_q(pls->pkt.dts, @@ -2367,6 +2758,8 @@ static int hls_probe(const AVProbeData *p) #define OFFSET(x) offsetof(HLSContext, x) #define FLAGS AV_OPT_FLAG_DECODING_PARAM static const AVOption hls_options[] = { + {"abr", "enable abr to switch streams", + OFFSET(abr), AV_OPT_TYPE_BOOL, {.i64 = 0}, 0, 1, FLAGS }, {"live_start_index", "segment index to start live streams at (negative values are from the end)", OFFSET(live_start_index), AV_OPT_TYPE_INT, {.i64 = -3}, INT_MIN, INT_MAX, FLAGS}, {"allowed_extensions", "List of file extensions that hls is allowed to access",