From patchwork Sun Dec 3 14:34:45 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rainer Hochecker X-Patchwork-Id: 6524 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.161.94 with SMTP id m30csp3384561jah; Sun, 3 Dec 2017 06:35:25 -0800 (PST) X-Google-Smtp-Source: AGs4zMaX9ruLccEbFpfwlWGrhJrXCj0UI+t35XUoSuPYDYAQnWVy50B1M46/w++oO5A40Nh9gIIE X-Received: by 10.223.161.75 with SMTP id r11mr10215955wrr.121.1512311724962; Sun, 03 Dec 2017 06:35:24 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1512311724; cv=none; d=google.com; s=arc-20160816; b=ujSVcj0C+23OhB6Z3phpfMv0Tk3iRMBrVsp64Z981A98g7dtJslmzvO6y44j7MmM7u aYYF+gppj6BE3PrFrReAXZ3GYGrnqHrXp8V3P2TezShRemQn/OxcHhiFQ+IsHqluZAj6 D0YFYC3lk2SmFdFDBc110syE7/PtC7yvFkh+ENFa4OyfGHiYyYzugAKJkOD3DB6p/4v4 IJyjwPNYapDH7cxBuyi9YqN69Eu84bfo3S5O39pnkb0E9ZRwkYXrPSEmrux5fmiEU9jO vKxVfhd4TPMfRbST15cKAqcc5BGaA9rBvRUAwFcEJvudSSa+pY37Z+LP2KOD8X1Wdz1R Da5Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:delivered-to:arc-authentication-results; bh=B4hHDf3+BTB/L2+IYcMkEeHpNzXXMqiicXnfpsBQFjs=; b=VsSJN/sLlLTctEquWKFk5r1Jc7BlKBDR3h0DdbwzY3xnxTWZWQDGJNWsciCGx3HXqX RiA867CVJXMqZn2DoWDuQKiXtinGmRwUmniiYHM7mbSJNwCP4DkVXSK8d89Zx48Ud8Eq z+fXP2+FK80W/RptQ5iTsMhtzgdmxsNgLdSyVshFwXtdOBg8NWDWMZd/Gnq9Y6AQBRRS pAaWTyAAWDuqjn741AyL4aZKKsa01ZA8d4sRX400yY/2X6DEih+5tWpavIkX1HK/LyWT 7vr8VOdm636Ej0VVJsIBEaUslbCoFgx8tGOjADjqkhPlEwQlz5NIpS9BVNq7f7IWEeUz 5sjw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 124si3669123wms.194.2017.12.03.06.35.24; Sun, 03 Dec 2017 06:35:24 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6B81168A08C; Sun, 3 Dec 2017 16:35:19 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mout.kundenserver.de (mout.kundenserver.de [212.227.17.24]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id DF94468A043 for ; Sun, 3 Dec 2017 16:35:13 +0200 (EET) Received: from localhost.localdomain ([89.247.207.216]) by mrelayeu.kundenserver.de (mreue101 [212.227.15.183]) with ESMTPSA (Nemesis) id 0LsQ2q-1f5vwL3Sps-011xGw; Sun, 03 Dec 2017 15:35:17 +0100 From: Rainer Hochecker To: ffmpeg-devel@ffmpeg.org Date: Sun, 3 Dec 2017 15:34:45 +0100 Message-Id: <20171203143447.41590-2-fernetmenta@online.de> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20171203143447.41590-1-fernetmenta@online.de> References: <0489f119587681fc5a2b15fdc8afdd2d@mail.onse.fi> <20171203143447.41590-1-fernetmenta@online.de> X-Provags-ID: V03:K0:pNhqzi3n6mWcRLpzFkV80rwPCWBxDaYJ++a7Mg8OcbIyHrKMUtb 6DlbXbdEZQhcL1rnyfDZKf0CUlC+RF5uqz4FgPKHBZNpBJ2SHymX1Uz+50zX91xv3HyZXTG SVedyDnaxBg4XUfHfKtKxpNg6ij7Uq6hlod1r97QCoc/TcFKtngwv6Ial2wtJC/Fybsgdg5 PCGexT2T1JrwHgLKr05+w== X-UI-Out-Filterresults: notjunk:1; V01:K0:2Www+zHZ4oE=:HknleduD5yg+KJ9ZMMHHi6 2Y0NGA7Q8YCDfalSzaqkvmEMM3W8CKIlLUOQoeuWzzmmykAfedkEKnXv8dtpJxFfnf2y9C6+2 Wv+YwBXSPqAE3ddLd+JsJeA40nxjYp9VSnSNZ8eEOG3tCmDY7Oa5B98Rz0TQGktxX4jGGiLrc pgGwZtKNKslha8X/qhdZ+5sxGCOQXbd3K/4VGMz2Pocg2cSpHMt/33UBSu05BpIGwGEspSSKi WdXUP5l21lF1PzxsfT0IXu0jHrJzV3n1vveoq9tLEiR71eylVbdf8xxqK2M0aUVlVO4ircTFE 5nVpCz+BpA8H290KBnh5Ry0XclC7YG1pLJA8oMvMUSAMbKRQprZsteezqr0pbLcLTpueAay9P 65vAt+nQDdmfOgHv2rgmRgJI7vwKdF2hvL+sdjAuY+7s0Sf+gL+x6sKaacaWOQ2c5zOMSXLQR uQIQvsffZ86PwPJNPCl9tggd9v5ZTcO2UHmdE2RR9mncfN4LZQdkKA3gKvANc5cd25zVsryrH 1m/UkxGjbYbVE10hBUEzFNH9qnKE8b+Gg3NFMCkQhyr+MmaOXX3hrAHcebIf+nUGzNbSFUv3j Po8YOIAn4PfFpQlfycN32gl3WK38MA5zGNmshfKqaZBrdUhkPSL5bce5MPUaDfvCihkuMheaw GCMUKamEZrKyrIxmMfLnWQtmq0a7S91qYjNIHr/Yj4Lp6hferrVnE3aw0m6U2Ij0R/O8epJMv LiZXj30zCSWNz3P/qZuP6tzpc3pcIkHdClRLDuoYPBW2qH70t1TrJmCyeBw= Subject: [FFmpeg-devel] [PATCH] hls demuxer: add option to defer parsing of variants X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Rainer Hochecker MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" fixed mem leak poined out by Steven --- doc/demuxers.texi | 5 + libavformat/hls.c | 304 ++++++++++++++++++++++++++++++++++++------------------ 2 files changed, 209 insertions(+), 100 deletions(-) diff --git a/doc/demuxers.texi b/doc/demuxers.texi index 73dc0feec1..634b122e10 100644 --- a/doc/demuxers.texi +++ b/doc/demuxers.texi @@ -316,6 +316,11 @@ segment index to start live streams at (negative values are from the end). @item max_reload Maximum number of times a insufficient list is attempted to be reloaded. Default value is 1000. + +@item load_all_variants +If 0, only the first variant/playlist is loaded on open. All other variants +get disabled and can be enabled by setting discard option in program. +Default value is 1. @end table @section image2 diff --git a/libavformat/hls.c b/libavformat/hls.c index 786934af03..c42e0b0f95 100644 --- a/libavformat/hls.c +++ b/libavformat/hls.c @@ -112,6 +112,7 @@ struct playlist { int n_segments; struct segment **segments; int needed, cur_needed; + int parsed; int cur_seq_no; int64_t cur_seg_offset; int64_t last_load_time; @@ -206,6 +207,7 @@ typedef struct HLSContext { int strict_std_compliance; char *allowed_extensions; int max_reload; + int load_all_variants; } HLSContext; static int read_chomp_line(AVIOContext *s, char *buf, int maxlen) @@ -314,6 +316,9 @@ static struct playlist *new_playlist(HLSContext *c, const char *url, pls->is_id3_timestamped = -1; pls->id3_mpegts_timestamp = AV_NOPTS_VALUE; + pls->index = c->n_playlists; + pls->parsed = 0; + pls->needed = 0; dynarray_add(&c->playlists, &c->n_playlists, pls); return pls; } @@ -721,6 +726,7 @@ static int parse_playlist(HLSContext *c, const char *url, free_segment_list(pls); pls->finished = 0; pls->type = PLS_TYPE_UNSPECIFIED; + pls->parsed = 1; } while (!avio_feof(in)) { read_chomp_line(in, line, sizeof(line)); @@ -1377,23 +1383,41 @@ reload: static void add_renditions_to_variant(HLSContext *c, struct variant *var, enum AVMediaType type, const char *group_id) { - int i; + int i, j; + int found; for (i = 0; i < c->n_renditions; i++) { struct rendition *rend = c->renditions[i]; if (rend->type == type && !strcmp(rend->group_id, group_id)) { - if (rend->playlist) + if (rend->playlist) { /* rendition is an external playlist * => add the playlist to the variant */ - dynarray_add(&var->playlists, &var->n_playlists, rend->playlist); - else + found = 0; + for (j = 0; j < var->n_playlists; j++) { + if (var->playlists[j] == rend->playlist) { + found = 1; + break; + } + } + if (!found) + dynarray_add(&var->playlists, &var->n_playlists, rend->playlist); + } else { /* rendition is part of the variant main Media Playlist * => add the rendition to the main Media Playlist */ - dynarray_add(&var->playlists[0]->renditions, - &var->playlists[0]->n_renditions, - rend); + found = 0; + for (j = 0; j < var->playlists[0]->n_renditions; j++) { + if (var->playlists[0]->renditions[j] == rend) { + found = 1; + break; + } + } + if (!found) + dynarray_add(&var->playlists[0]->renditions, + &var->playlists[0]->n_renditions, + rend); + } } } } @@ -1631,6 +1655,124 @@ static int hls_close(AVFormatContext *s) return 0; } +static int init_playlist(HLSContext *c, struct playlist *pls) +{ + AVInputFormat *in_fmt = NULL; + int highest_cur_seq_no = 0; + int ret; + int i; + + if (!(pls->ctx = avformat_alloc_context())) { + return AVERROR(ENOMEM); + } + + if (pls->n_segments == 0) + return 0; + + pls->needed = 1; + pls->parent = c->ctx; + + /* + * If this is a live stream and this playlist looks like it is one segment + * behind, try to sync it up so that every substream starts at the same + * time position (so e.g. avformat_find_stream_info() will see packets from + * all active streams within the first few seconds). This is not very generic, + * though, as the sequence numbers are technically independent. + */ + highest_cur_seq_no = 0; + for (i = 0; i < c->n_playlists; i++) { + struct playlist *pls = c->playlists[i]; + if (!pls->parsed) + continue; + if (pls->cur_seq_no > highest_cur_seq_no) + highest_cur_seq_no = pls->cur_seq_no; + } + if (!pls->finished && pls->cur_seq_no == highest_cur_seq_no - 1 && + highest_cur_seq_no < pls->start_seq_no + pls->n_segments) { + pls->cur_seq_no = highest_cur_seq_no; + } + + pls->read_buffer = av_malloc(INITIAL_BUFFER_SIZE); + if (!pls->read_buffer){ + ret = AVERROR(ENOMEM); + avformat_free_context(pls->ctx); + pls->ctx = NULL; + return ret; + } + ffio_init_context(&pls->pb, pls->read_buffer, INITIAL_BUFFER_SIZE, 0, pls, + read_data, NULL, NULL); + pls->pb.seekable = 0; + ret = av_probe_input_buffer(&pls->pb, &in_fmt, pls->segments[0]->url, + NULL, 0, 0); + if (ret < 0) { + /* Free the ctx - it isn't initialized properly at this point, + * so avformat_close_input shouldn't be called. If + * avformat_open_input fails below, it frees and zeros the + * context, so it doesn't need any special treatment like this. */ + av_log(c->ctx, AV_LOG_ERROR, "Error when loading first segment '%s'\n", pls->segments[0]->url); + av_free(pls->read_buffer); + pls->read_buffer = NULL; + avformat_free_context(pls->ctx); + pls->ctx = NULL; + return ret; + } + pls->ctx->pb = &pls->pb; + pls->ctx->io_open = nested_io_open; + pls->ctx->flags |= c->ctx->flags & ~AVFMT_FLAG_CUSTOM_IO; + + if ((ret = ff_copy_whiteblacklists(pls->ctx, c->ctx)) < 0) + return ret; + + ret = avformat_open_input(&pls->ctx, pls->segments[0]->url, in_fmt, NULL); + if (ret < 0) { + av_log(c->ctx, AV_LOG_ERROR, "Error opening playlist %s", pls->segments[0]->url); + avformat_free_context(pls->ctx); + pls->ctx = NULL; + return ret; + } + + if (pls->id3_deferred_extra && pls->ctx->nb_streams == 1) { + ff_id3v2_parse_apic(pls->ctx, &pls->id3_deferred_extra); + avformat_queue_attached_pictures(pls->ctx); + ff_id3v2_free_extra_meta(&pls->id3_deferred_extra); + pls->id3_deferred_extra = NULL; + } + + if (pls->is_id3_timestamped == -1) + av_log(c->ctx, AV_LOG_WARNING, "No expected HTTP requests have been made\n"); + + /* + * For ID3 timestamped raw audio streams we need to detect the packet + * durations to calculate timestamps in fill_timing_for_id3_timestamped_stream(), + * but for other streams we can rely on our user calling avformat_find_stream_info() + * on us if they want to. + */ + if (pls->is_id3_timestamped) { + ret = avformat_find_stream_info(pls->ctx, NULL); + if (ret < 0) { + avformat_free_context(pls->ctx); + pls->ctx = NULL; + return ret; + } + } + + pls->has_noheader_flag = !!(pls->ctx->ctx_flags & AVFMTCTX_NOHEADER); + + /* Create new AVStreams for each stream in this playlist */ + ret = update_streams_from_subdemuxer(c->ctx, pls); + if (ret < 0) { + avformat_free_context(pls->ctx); + pls->ctx = NULL; + return ret; + } + + add_metadata_from_renditions(c->ctx, pls, AVMEDIA_TYPE_AUDIO); + add_metadata_from_renditions(c->ctx, pls, AVMEDIA_TYPE_VIDEO); + add_metadata_from_renditions(c->ctx, pls, AVMEDIA_TYPE_SUBTITLE); + + return 0; +} + static int hls_read_header(AVFormatContext *s) { void *u = (s->flags & AVFMT_FLAG_CUSTOM_IO) ? NULL : s->pb; @@ -1663,6 +1805,9 @@ static int hls_read_header(AVFormatContext *s) if ((ret = parse_playlist(c, s->filename, NULL, s->pb)) < 0) goto fail; + /* first playlist was created, set it to parsed */ + c->variants[0]->playlists[0]->parsed = 1; + if ((ret = save_avio_options(s)) < 0) goto fail; @@ -1675,8 +1820,15 @@ static int hls_read_header(AVFormatContext *s) goto fail; } /* If the playlist only contained playlists (Master Playlist), - * parse each individual playlist. */ - if (c->n_playlists > 1 || c->playlists[0]->n_segments == 0) { + * parse all individual playlists. + * If option load_all_variants is false, load only first variant */ + if (!c->load_all_variants && c->n_variants > 1) { + for (i = 0; i < c->variants[0]->n_playlists; i++) { + struct playlist *pls = c->variants[0]->playlists[i]; + if ((ret = parse_playlist(c, pls->url, pls, NULL)) < 0) + goto fail; + } + } else if (c->n_playlists > 1 || c->playlists[0]->n_segments == 0) { for (i = 0; i < c->n_playlists; i++) { struct playlist *pls = c->playlists[i]; if ((ret = parse_playlist(c, pls->url, pls, NULL)) < 0) @@ -1720,13 +1872,17 @@ static int hls_read_header(AVFormatContext *s) if (!program) goto fail; av_dict_set_int(&program->metadata, "variant_bitrate", v->bandwidth, 0); + + /* start with the first variant and disable all others */ + if (i > 0 && !c->load_all_variants) + program->discard = AVDISCARD_ALL; } /* Select the starting segments */ for (i = 0; i < c->n_playlists; i++) { struct playlist *pls = c->playlists[i]; - if (pls->n_segments == 0) + if (pls->n_segments == 0 && !pls->parsed) continue; pls->cur_seq_no = select_cur_seq_no(c, pls); @@ -1736,97 +1892,9 @@ static int hls_read_header(AVFormatContext *s) /* Open the demuxer for each playlist */ for (i = 0; i < c->n_playlists; i++) { struct playlist *pls = c->playlists[i]; - AVInputFormat *in_fmt = NULL; - - if (!(pls->ctx = avformat_alloc_context())) { - ret = AVERROR(ENOMEM); - goto fail; - } - - if (pls->n_segments == 0) - continue; - pls->index = i; - pls->needed = 1; - pls->parent = s; - - /* - * If this is a live stream and this playlist looks like it is one segment - * behind, try to sync it up so that every substream starts at the same - * time position (so e.g. avformat_find_stream_info() will see packets from - * all active streams within the first few seconds). This is not very generic, - * though, as the sequence numbers are technically independent. - */ - if (!pls->finished && pls->cur_seq_no == highest_cur_seq_no - 1 && - highest_cur_seq_no < pls->start_seq_no + pls->n_segments) { - pls->cur_seq_no = highest_cur_seq_no; - } - - pls->read_buffer = av_malloc(INITIAL_BUFFER_SIZE); - if (!pls->read_buffer){ - ret = AVERROR(ENOMEM); - avformat_free_context(pls->ctx); - pls->ctx = NULL; - goto fail; - } - ffio_init_context(&pls->pb, pls->read_buffer, INITIAL_BUFFER_SIZE, 0, pls, - read_data, NULL, NULL); - pls->pb.seekable = 0; - ret = av_probe_input_buffer(&pls->pb, &in_fmt, pls->segments[0]->url, - NULL, 0, 0); - if (ret < 0) { - /* Free the ctx - it isn't initialized properly at this point, - * so avformat_close_input shouldn't be called. If - * avformat_open_input fails below, it frees and zeros the - * context, so it doesn't need any special treatment like this. */ - av_log(s, AV_LOG_ERROR, "Error when loading first segment '%s'\n", pls->segments[0]->url); - avformat_free_context(pls->ctx); - pls->ctx = NULL; - goto fail; - } - pls->ctx->pb = &pls->pb; - pls->ctx->io_open = nested_io_open; - pls->ctx->flags |= s->flags & ~AVFMT_FLAG_CUSTOM_IO; - - if ((ret = ff_copy_whiteblacklists(pls->ctx, s)) < 0) - goto fail; - - ret = avformat_open_input(&pls->ctx, pls->segments[0]->url, in_fmt, NULL); - if (ret < 0) - goto fail; - - if (pls->id3_deferred_extra && pls->ctx->nb_streams == 1) { - ff_id3v2_parse_apic(pls->ctx, &pls->id3_deferred_extra); - avformat_queue_attached_pictures(pls->ctx); - ff_id3v2_free_extra_meta(&pls->id3_deferred_extra); - pls->id3_deferred_extra = NULL; - } - - if (pls->is_id3_timestamped == -1) - av_log(s, AV_LOG_WARNING, "No expected HTTP requests have been made\n"); - - /* - * For ID3 timestamped raw audio streams we need to detect the packet - * durations to calculate timestamps in fill_timing_for_id3_timestamped_stream(), - * but for other streams we can rely on our user calling avformat_find_stream_info() - * on us if they want to. - */ - if (pls->is_id3_timestamped) { - ret = avformat_find_stream_info(pls->ctx, NULL); - if (ret < 0) - goto fail; - } - - pls->has_noheader_flag = !!(pls->ctx->ctx_flags & AVFMTCTX_NOHEADER); - - /* Create new AVStreams for each stream in this playlist */ - ret = update_streams_from_subdemuxer(s, pls); - if (ret < 0) - goto fail; - - add_metadata_from_renditions(s, pls, AVMEDIA_TYPE_AUDIO); - add_metadata_from_renditions(s, pls, AVMEDIA_TYPE_VIDEO); - add_metadata_from_renditions(s, pls, AVMEDIA_TYPE_SUBTITLE); + if (pls->parsed) + init_playlist(c, pls); } update_noheader_flag(s); @@ -1877,6 +1945,36 @@ static int recheck_discard_flags(AVFormatContext *s, int first) return changed; } +static void recheck_discard_programs(AVFormatContext *s) +{ + HLSContext *c = s->priv_data; + int i, j; + + for (i = 0; i < c->n_variants; i++) { + struct variant *var = c->variants[i]; + AVProgram *program = s->programs[i]; + + if (program->discard >= AVDISCARD_ALL) + continue; + + for (j = 0; j < c->variants[i]->n_playlists; j++) { + struct playlist *pls = c->variants[i]->playlists[j]; + + if (!pls->parsed) { + if (parse_playlist(c, pls->url, pls, NULL) < 0) + continue; + if (var->audio_group[0]) + add_renditions_to_variant(c, var, AVMEDIA_TYPE_AUDIO, var->audio_group); + if (var->video_group[0]) + add_renditions_to_variant(c, var, AVMEDIA_TYPE_VIDEO, var->video_group); + if (var->subtitles_group[0]) + add_renditions_to_variant(c, var, AVMEDIA_TYPE_SUBTITLE, var->subtitles_group); + init_playlist(c, pls); + } + } + } +} + static void fill_timing_for_id3_timestamped_stream(struct playlist *pls) { if (pls->id3_offset >= 0) { @@ -1924,6 +2022,8 @@ static int hls_read_packet(AVFormatContext *s, AVPacket *pkt) HLSContext *c = s->priv_data; int ret, i, minplaylist = -1; + recheck_discard_programs(s); + recheck_discard_flags(s, c->first_packet); c->first_packet = 0; @@ -2101,6 +2201,8 @@ static int hls_read_seek(AVFormatContext *s, int stream_index, for (i = 0; i < c->n_playlists; i++) { /* Reset reading */ struct playlist *pls = c->playlists[i]; + if (!pls->parsed) + continue; if (pls->input) ff_format_io_close(pls->parent, &pls->input); av_packet_unref(&pls->pkt); @@ -2157,6 +2259,8 @@ static const AVOption hls_options[] = { INT_MIN, INT_MAX, FLAGS}, {"max_reload", "Maximum number of times a insufficient list is attempted to be reloaded", OFFSET(max_reload), AV_OPT_TYPE_INT, {.i64 = 1000}, 0, INT_MAX, FLAGS}, + {"load_all_variants", "if > 0 all playlists of all variants are opened at startup", + OFFSET(load_all_variants), AV_OPT_TYPE_BOOL, {.i64 = 1}, 0, 1, FLAGS}, {NULL} };