From patchwork Tue Jan 23 05:21:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dixit, Vishwanath" X-Patchwork-Id: 7396 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.156.27 with SMTP id q27csp3844572jak; Mon, 22 Jan 2018 21:21:49 -0800 (PST) X-Google-Smtp-Source: AH8x227Mez87nMeT4efbN5jGRRkzvJHi7X95c6EZ0ApBszIUR+gWhKBV3GSqqhUtbw9BQdOjbgrE X-Received: by 10.223.179.69 with SMTP id k5mr1060386wrd.15.1516684909276; Mon, 22 Jan 2018 21:21:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1516684909; cv=none; d=google.com; s=arc-20160816; b=nj3HdcsFhlTFn+5PMjS/SGcXp6TmPdCTJ5tZX4WUsnT18HvxNN4nyXWrHtIY7trUaw R9khJ8/6gBFh5X6gg7Ml+c+6W05wjiCv7pn0iG+8x1YJd5e8HvztOw1ko4owUa7Fh/Fs jIF4hNefbdFsAx1XrEpimtT1/nQpQW5AR6urHEW87khBWg5OLoybmVT4XkmG9vx4SzHJ GnzyOD6cO+utPveAjNDAWc653uE901wyIEtSx0u5D2gML95UGL9H6j8CiqXYhr+PqwKd 1o4DDUQ7ZujjfENU/lXKyceF/p4n6XnfyJOV4r92D11WEOuxvGJVYMvXNEPIgSYRSmzc A65g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:feedback-id:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to :arc-authentication-results; bh=xjoyjuNyaqwOumdm5bW/56tNQgdzJYZmqVR+niHRYko=; b=IYw7+4obLq7Yf5uHhuEShSxWY2bkx00BCOqiahdC0Q2V0WNmSVGig3B1lx8NKImS6+ NpsctNkJdcsRaAyuaeNH6WQV09iUo5qxw99414iy4r/Hq754+5UIBX+WaIWNZwY9kxUr 20SNW7/vZhSg3+PQPnFbU71CYyJC6xRtvjPt/JmrZAaApOOjTK7HkyHV+G/8t/U0727+ sc/B/TSnbHBltChbsMtBty+4qa0VYEg0wK/7DPPlpZHNck1xp+YmDK0jeSIQQzqWpucb JDfx35TDdf0/WQePNPFyptJkf6G8OsPOb8hxjm6sY1dpzppgfT/ifp0tVjTJExNdkRHL Aq6Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@smtpservice.net header.s=m78bu0.a1-4.dyn header.b=FKm+ddBh; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=NONE dis=NONE) header.from=akamai.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id o2si4743401wmg.208.2018.01.22.21.21.48; Mon, 22 Jan 2018 21:21:49 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@smtpservice.net header.s=m78bu0.a1-4.dyn header.b=FKm+ddBh; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=NONE dis=NONE) header.from=akamai.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id ACB0B689AAC; Tue, 23 Jan 2018 07:21:44 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from a2i831.smtp2go.com (a2i831.smtp2go.com [103.47.207.63]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5D1CD68924F for ; Tue, 23 Jan 2018 07:21:38 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=smtpservice.net; s=m78bu0.a1-4.dyn; x=1516685805; h=Feedback-ID: X-Smtpcorp-Track:Message-Id:Date:Subject:To:From:Reply-To:Sender: List-Unsubscribe; bh=tIGPY+9AceUUVAMYR7fVRMcNxX/Arcc181lUlwz1+dY=; b=FKm+ddBh Wi9MI3soLJyKVv34aVZHks6EANLyKmyW+HrsLWgySTaYH1QQinbmNzhN17qlIxJfVKPetLW/npsSW JTmdsQm3tyMpKliktGO3Nj7zzSjHCgNMVRI1GRzSidp7ZJVUNaF6uwrfoSQjEOZ3mj4KgJT2dJzxg A22RSr1nq1k1G4ZyKYGcqCyUY6eLoNAjnlD4o94Bk4BRmavoZt7zSwK4nt+M49/6Apowv0J7UlzG9 2eRrO2EyQ+yMWQlh3VxWr71+FC/qJptUJpPVvFHae3K+a/cvQIZkoa4zoqqGjcaQaNfuYdwSCSedB 6uI60bbvCF8HzbMafU6O82q9gQ==; From: vdixit@akamai.com To: ffmpeg-devel@ffmpeg.org Date: Tue, 23 Jan 2018 10:51:20 +0530 Message-Id: <1516684880-26191-1-git-send-email-vdixit@akamai.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <04D2576D-2D89-4128-9091-9D4A71B0C7DF@akamai.com> References: <04D2576D-2D89-4128-9091-9D4A71B0C7DF@akamai.com> X-Smtpcorp-Track: 1-dr1Z4ba5eDKT.ZoZc9IKvg Feedback-ID: 337386m:337386asVRLGB:337386sWFAJT_szW:SMTPCORP X-Report-Abuse: Please forward a copy of this message, including all headers, to Subject: [FFmpeg-devel] [PATCH v3 1/1] avformat/hlsenc: closed caption tags in the master playlist X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Vishwanath Dixit MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Vishwanath Dixit --- doc/muxers.texi | 37 +++++++++++ libavformat/dashenc.c | 2 +- libavformat/hlsenc.c | 155 +++++++++++++++++++++++++++++++++++++++++++++- libavformat/hlsplaylist.c | 5 +- libavformat/hlsplaylist.h | 2 +- 5 files changed, 196 insertions(+), 5 deletions(-) diff --git a/doc/muxers.texi b/doc/muxers.texi index b060c4f..d9a5cc0 100644 --- a/doc/muxers.texi +++ b/doc/muxers.texi @@ -901,6 +901,43 @@ and they are mapped to the two video only variant streams with audio group names By default, a single hls variant containing all the encoded streams is created. +@item cc_stream_map +Map string which specifies different closed captions groups and their +attributes. The closed captions stream groups are separated by space. +Expected string format is like this +"ccgroup:,instreamid:,language: ....". +'ccgroup' and 'instreamid' are mandatory attributes. 'language' is an optional +attribute. +The closed captions groups configured using this option are mapped to different +variant streams by providing the same 'ccgroup' name in the +@code{var_stream_map} string. If @code{var_stream_map} is not set, then the +first available ccgroup in @code{cc_stream_map} is mapped to the output variant +stream. The examples for these two use cases are given below. + +@example +ffmpeg -re -i in.ts -b:v 1000k -b:a 64k -a53cc 1 -f hls \ + -cc_stream_map "ccgroup:cc,instreamid:CC1,language:en" \ + -master_pl_name master.m3u8 \ + http://example.com/live/out.m3u8 +@end example +This example adds @code{#EXT-X-MEDIA} tag with @code{TYPE=CLOSED-CAPTIONS} in +the master playlist with group name 'cc', langauge 'en' (english) and +INSTREAM-ID 'CC1'. Also, it adds @code{CLOSED-CAPTIONS} attribute with group +name 'cc' for the output variant stream. +@example +ffmpeg -re -i in.ts -b:v:0 1000k -b:v:1 256k -b:a:0 64k -b:a:1 32k \ + -a53cc:0 1 -a53cc:1 1\ + -map 0:v -map 0:a -map 0:v -map 0:a -f hls \ + -cc_stream_map "ccgroup:cc,instreamid:CC1,language:en ccgroup:cc,instreamid:CC2,language:sp" \ + -var_stream_map "v:0,a:0,ccgroup:cc v:1,a:1,ccgroup:cc" \ + -master_pl_name master.m3u8 \ + http://example.com/live/out_%v.m3u8 +@end example +This example adds two @code{#EXT-X-MEDIA} tags with @code{TYPE=CLOSED-CAPTIONS} in +the master playlist for the INSTREAM-IDs 'CC1' and 'CC2'. Also, it adds +@code{CLOSED-CAPTIONS} attribute with group name 'cc' for the two output variant +streams. + @item master_pl_name Create HLS master playlist with the given name. diff --git a/libavformat/dashenc.c b/libavformat/dashenc.c index 39d0afe..5ece100 100644 --- a/libavformat/dashenc.c +++ b/libavformat/dashenc.c @@ -820,7 +820,7 @@ static int write_manifest(AVFormatContext *s, int final) stream_bitrate += max_audio_bitrate; } get_hls_playlist_name(playlist_file, sizeof(playlist_file), NULL, i); - ff_hls_write_stream_info(st, out, stream_bitrate, playlist_file, agroup, NULL); + ff_hls_write_stream_info(st, out, stream_bitrate, playlist_file, agroup, NULL, NULL); } avio_close(out); if (use_rename) diff --git a/libavformat/hlsenc.c b/libavformat/hlsenc.c index 42e437f..aab21f2 100644 --- a/libavformat/hlsenc.c +++ b/libavformat/hlsenc.c @@ -152,9 +152,16 @@ typedef struct VariantStream { unsigned int nb_streams; int m3u8_created; /* status of media play-list creation */ char *agroup; /* audio group name */ + char *ccgroup; /* closed caption group name */ char *baseurl; } VariantStream; +typedef struct ClosedCaptionsStream { + char *ccgroup; /* closed caption group name */ + char *instreamid; /* closed captions INSTREAM-ID */ + char *language; /* closed captions langauge */ +} ClosedCaptionsStream; + typedef struct HLSContext { const AVClass *class; // Class for private options. int64_t start_sequence; @@ -203,11 +210,14 @@ typedef struct HLSContext { VariantStream *var_streams; unsigned int nb_varstreams; + ClosedCaptionsStream *cc_streams; + unsigned int nb_ccstreams; int master_m3u8_created; /* status of master play-list creation */ char *master_m3u8_url; /* URL of the master m3u8 file */ int version; /* HLS version */ char *var_stream_map; /* user specified variant stream map string */ + char *cc_stream_map; /* user specified closed caption streams map string */ char *master_pl_name; unsigned int master_publish_rate; int http_persistent; @@ -1167,7 +1177,8 @@ static int create_master_playlist(AVFormatContext *s, AVDictionary *options = NULL; unsigned int i, j; int m3u8_name_size, ret, bandwidth; - char *m3u8_rel_name; + char *m3u8_rel_name, *ccgroup; + ClosedCaptionsStream *ccs; input_vs->m3u8_created = 1; if (!hls->master_m3u8_created) { @@ -1194,6 +1205,16 @@ static int create_master_playlist(AVFormatContext *s, ff_hls_write_playlist_version(hls->m3u8_out, hls->version); + for (i = 0; i < hls->nb_ccstreams; i++) { + ccs = &(hls->cc_streams[i]); + avio_printf(hls->m3u8_out, "#EXT-X-MEDIA:TYPE=CLOSED-CAPTIONS"); + avio_printf(hls->m3u8_out, ",GROUP-ID=\"%s\"", ccs->ccgroup); + avio_printf(hls->m3u8_out, ",NAME=\"%s\"", ccs->instreamid); + if (ccs->language) + avio_printf(hls->m3u8_out, ",LANGUAGE=\"%s\"", ccs->language); + avio_printf(hls->m3u8_out, ",INSTREAM-ID=\"%s\"\n", ccs->instreamid); + } + /* For audio only variant streams add #EXT-X-MEDIA tag with attributes*/ for (i = 0; i < hls->nb_varstreams; i++) { vs = &(hls->var_streams[i]); @@ -1278,8 +1299,23 @@ static int create_master_playlist(AVFormatContext *s, bandwidth += aud_st->codecpar->bit_rate; bandwidth += bandwidth / 10; + ccgroup = NULL; + if (vid_st && vs->ccgroup) { + /* check if this group name is available in the cc map string */ + for (j = 0; j < hls->nb_ccstreams; j++) { + ccs = &(hls->cc_streams[j]); + if (!av_strcasecmp(ccs->ccgroup, vs->ccgroup)) { + ccgroup = vs->ccgroup; + break; + } + } + if (j == hls->nb_ccstreams) + av_log(NULL, AV_LOG_WARNING, "mapping ccgroup %s not found\n", + vs->ccgroup); + } + ff_hls_write_stream_info(vid_st, hls->m3u8_out, bandwidth, m3u8_rel_name, - aud_st ? vs->agroup : NULL, vs->codec_attr); + aud_st ? vs->agroup : NULL, vs->codec_attr, ccgroup); av_freep(&m3u8_rel_name); } @@ -1766,6 +1802,11 @@ static int parse_variant_stream_mapstring(AVFormatContext *s) if (!vs->agroup) return AVERROR(ENOMEM); continue; + } else if (av_strstart(keyval, "ccgroup:", &val)) { + vs->ccgroup = av_strdup(val); + if (!vs->ccgroup) + return AVERROR(ENOMEM); + continue; } else if (av_strstart(keyval, "v:", &val)) { codec_type = AVMEDIA_TYPE_VIDEO; } else if (av_strstart(keyval, "a:", &val)) { @@ -1796,9 +1837,94 @@ static int parse_variant_stream_mapstring(AVFormatContext *s) return 0; } +static int parse_cc_stream_mapstring(AVFormatContext *s) +{ + HLSContext *hls = s->priv_data; + int nb_ccstreams; + char *p, *q, *saveptr1, *saveptr2, *ccstr, *keyval; + const char *val; + ClosedCaptionsStream *ccs; + + p = av_strdup(hls->cc_stream_map); + q = p; + while(av_strtok(q, " \t", &saveptr1)) { + q = NULL; + hls->nb_ccstreams++; + } + av_freep(&p); + + hls->cc_streams = av_mallocz(sizeof(*hls->cc_streams) * hls->nb_ccstreams); + if (!hls->cc_streams) + return AVERROR(ENOMEM); + + p = hls->cc_stream_map; + nb_ccstreams = 0; + while (ccstr = av_strtok(p, " \t", &saveptr1)) { + p = NULL; + + if (nb_ccstreams < hls->nb_ccstreams) + ccs = &(hls->cc_streams[nb_ccstreams++]); + else + return AVERROR(EINVAL); + + while (keyval = av_strtok(ccstr, ",", &saveptr2)) { + ccstr = NULL; + + if (av_strstart(keyval, "ccgroup:", &val)) { + ccs->ccgroup = av_strdup(val); + if (!ccs->ccgroup) + return AVERROR(ENOMEM); + } else if (av_strstart(keyval, "instreamid:", &val)) { + ccs->instreamid = av_strdup(val); + if (!ccs->instreamid) + return AVERROR(ENOMEM); + } else if (av_strstart(keyval, "language:", &val)) { + ccs->language = av_strdup(val); + if (!ccs->language) + return AVERROR(ENOMEM); + } else { + av_log(s, AV_LOG_ERROR, "Invalid keyval %s\n", keyval); + return AVERROR(EINVAL); + } + } + + if (!ccs->ccgroup || !ccs->instreamid) { + av_log(s, AV_LOG_ERROR, "Insufficient parameters in cc stream map string\n"); + return AVERROR(EINVAL); + } + + if (av_strstart(ccs->instreamid, "CC", &val)) { + if(atoi(val) < 1 || atoi(val) > 4) { + av_log(s, AV_LOG_ERROR, "Invalid instream ID CC index %d in %s, range 1-4\n", + atoi(val), ccs->instreamid); + return AVERROR(EINVAL); + } + } else if (av_strstart(ccs->instreamid, "SERVICE", &val)) { + if(atoi(val) < 1 || atoi(val) > 63) { + av_log(s, AV_LOG_ERROR, "Invalid instream ID SERVICE index %d in %s, range 1-63 \n", + atoi(val), ccs->instreamid); + return AVERROR(EINVAL); + } + } else { + av_log(s, AV_LOG_ERROR, "Invalid instream ID %s, supported are CCn or SERIVICEn\n", + ccs->instreamid); + return AVERROR(EINVAL); + } + } + + return 0; +} + static int update_variant_stream_info(AVFormatContext *s) { HLSContext *hls = s->priv_data; unsigned int i; + int ret = 0; + + if (hls->cc_stream_map) { + ret = parse_cc_stream_mapstring(s); + if (ret < 0) + return ret; + } if (hls->var_stream_map) { return parse_variant_stream_mapstring(s); @@ -1816,6 +1942,13 @@ static int update_variant_stream_info(AVFormatContext *s) { if (!hls->var_streams[0].streams) return AVERROR(ENOMEM); + //by default, the first available ccgroup is mapped to the variant stream + if (hls->nb_ccstreams) { + hls->var_streams[0].ccgroup = av_strdup(hls->cc_streams[0].ccgroup); + if (!hls->var_streams[0].ccgroup) + return AVERROR(ENOMEM); + } + for (i = 0; i < s->nb_streams; i++) hls->var_streams[0].streams[i] = s->streams[i]; } @@ -2192,13 +2325,22 @@ failed: av_freep(&vs->m3u8_name); av_freep(&vs->streams); av_freep(&vs->agroup); + av_freep(&vs->ccgroup); av_freep(&vs->baseurl); } + for (i = 0; i < hls->nb_ccstreams; i++) { + ClosedCaptionsStream *ccs = &hls->cc_streams[i]; + av_freep(&ccs->ccgroup); + av_freep(&ccs->instreamid); + av_freep(&ccs->language); + } + ff_format_io_close(s, &hls->m3u8_out); ff_format_io_close(s, &hls->sub_m3u8_out); av_freep(&hls->key_basename); av_freep(&hls->var_streams); + av_freep(&hls->cc_streams); av_freep(&hls->master_m3u8_url); return 0; } @@ -2535,13 +2677,21 @@ fail: av_freep(&vs->vtt_m3u8_name); av_freep(&vs->streams); av_freep(&vs->agroup); + av_freep(&vs->ccgroup); av_freep(&vs->baseurl); if (vs->avf) avformat_free_context(vs->avf); if (vs->vtt_avf) avformat_free_context(vs->vtt_avf); } + for (i = 0; i < hls->nb_ccstreams; i++) { + ClosedCaptionsStream *ccs = &hls->cc_streams[i]; + av_freep(&ccs->ccgroup); + av_freep(&ccs->instreamid); + av_freep(&ccs->language); + } av_freep(&hls->var_streams); + av_freep(&hls->cc_streams); av_freep(&hls->master_m3u8_url); } @@ -2601,6 +2751,7 @@ static const AVOption options[] = { {"datetime", "current datetime as YYYYMMDDhhmmss", 0, AV_OPT_TYPE_CONST, {.i64 = HLS_START_SEQUENCE_AS_FORMATTED_DATETIME }, INT_MIN, INT_MAX, E, "start_sequence_source_type" }, {"http_user_agent", "override User-Agent field in HTTP header", OFFSET(user_agent), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, E}, {"var_stream_map", "Variant stream map string", OFFSET(var_stream_map), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, E}, + {"cc_stream_map", "Closed captions stream map string", OFFSET(cc_stream_map), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, E}, {"master_pl_name", "Create HLS master playlist with this name", OFFSET(master_pl_name), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, E}, {"master_pl_publish_rate", "Publish master play list every after this many segment intervals", OFFSET(master_publish_rate), AV_OPT_TYPE_INT, {.i64 = 0}, 0, UINT_MAX, E}, {"http_persistent", "Use persistent HTTP connections", OFFSET(http_persistent), AV_OPT_TYPE_BOOL, {.i64 = 0 }, 0, 1, E }, diff --git a/libavformat/hlsplaylist.c b/libavformat/hlsplaylist.c index b8a3a14..efcbff0 100644 --- a/libavformat/hlsplaylist.c +++ b/libavformat/hlsplaylist.c @@ -47,7 +47,8 @@ void ff_hls_write_audio_rendition(AVIOContext *out, char *agroup, void ff_hls_write_stream_info(AVStream *st, AVIOContext *out, int bandwidth, char *filename, char *agroup, - char *codecs) { + char *codecs, char *ccgroup) { + if (!out || !filename) return; @@ -65,6 +66,8 @@ void ff_hls_write_stream_info(AVStream *st, AVIOContext *out, avio_printf(out, ",CODECS=\"%s\"", codecs); if (agroup && strlen(agroup) > 0) avio_printf(out, ",AUDIO=\"group_%s\"", agroup); + if (ccgroup && strlen(ccgroup) > 0) + avio_printf(out, ",CLOSED-CAPTIONS=\"%s\"", ccgroup); avio_printf(out, "\n%s\n\n", filename); } diff --git a/libavformat/hlsplaylist.h b/libavformat/hlsplaylist.h index 7532e37..5054b01 100644 --- a/libavformat/hlsplaylist.h +++ b/libavformat/hlsplaylist.h @@ -41,7 +41,7 @@ void ff_hls_write_audio_rendition(AVIOContext *out, char *agroup, char *filename, int name_id, int is_default); void ff_hls_write_stream_info(AVStream *st, AVIOContext *out, int bandwidth, char *filename, char *agroup, - char *codecs); + char *codecs, char *ccgroup); void ff_hls_write_playlist_header(AVIOContext *out, int version, int allowcache, int target_duration, int64_t sequence, uint32_t playlist_type);