From patchwork Tue Jan 9 08:45:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dixit, Vishwanath" X-Patchwork-Id: 7229 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.78.2 with SMTP id r2csp3667258jaa; Tue, 9 Jan 2018 00:46:06 -0800 (PST) X-Google-Smtp-Source: ACJfBov6m6meOTxQUmLqjermWtKcrCIASYuZNIa1uPMiXO++rGXWuhX4QvtSgnAQ08DMNZNqmTp8 X-Received: by 10.28.15.201 with SMTP id 192mr11998467wmp.88.1515487566210; Tue, 09 Jan 2018 00:46:06 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1515487566; cv=none; d=google.com; s=arc-20160816; b=Bklw+b9cezPS8dOktxdzr/l2eimZUrx4LCdDUzY9jYLejffUhKFViPqvEJf1MosVHZ fvzTZs444h9Y61u7gYjJIRrYdYw83S+TGXIDJfcCL8UDcYabNXSjy4scIp52viGCL6/R QjKLSbheExiGccmMf+gUA5oUpyV6WstL9lK3GMPolT0WXV6KZKqIkz7eBhJRHsiw+uMD HDeoK3JDV4BMZ9WvI6vg10gf7R24tFsLfPTJ5Z+Bi+ySO2ElDCSi42364rgbwziEfbrG pOX679ikuHhcjl+FXl/OKxj5zjTdAnJnkhsA3P0yddPj5cFTH9D/6Yn9FHxqQVVBZ4Q+ +jTA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:feedback-id:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to :arc-authentication-results; bh=9zYGopQhZTIIZgy5Gawja0D2dk/ljjX2r/9yEJdJAu4=; b=WDDgKM8exrU37DxVPN0Y3zp0a0K3UhXkYIRcn1LXmIpk3c7PtgXDEzSkM4UB3I4o5M a7qCFGTzxfE3tmBXkB13oAyc6kL6f6eX3ghYhPjbpufV1F+mFPTc5IZHYeK27qGISvxW 8cnrc6tfIPlrIwrVnRnbMtjbbGeFxl1MXZbCIQZcA6ayR7ratMR/MQhSnHEtgRo0JDJ8 +KqfhpAgID29sdEFj4nNrduavtAxzOV7+s27lzEmL+4RMBYtkU5W2NCTJjJNrtifYL+x Jh37fIiwcktSUAv5REAjlhWv9dUhfj/c5eOv6LaaJ7XJTvltHvikIBMdzRVbZpeV2uVi 8/nA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@smtpservice.net header.s=m78bu0.a1-4.dyn header.b=3rBWoZin; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=NONE dis=NONE) header.from=akamai.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id o2si10220096wro.368.2018.01.09.00.46.05; Tue, 09 Jan 2018 00:46:06 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@smtpservice.net header.s=m78bu0.a1-4.dyn header.b=3rBWoZin; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=NONE dis=NONE) header.from=akamai.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id EF031689F69; Tue, 9 Jan 2018 10:46:02 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from a2i831.smtp2go.com (a2i831.smtp2go.com [103.47.207.63]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 337BF689C16 for ; Tue, 9 Jan 2018 10:45:56 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=smtpservice.net; s=m78bu0.a1-4.dyn; x=1515488462; h=Feedback-ID: X-Smtpcorp-Track:Message-Id:Date:Subject:To:From:Reply-To:Sender: List-Unsubscribe; bh=p2oAGRQHe6c41oguvYa3t+IK3DaJ2r4IrpDwGa8BkI4=; b=3rBWoZin 8KzMHTsu6hTBv1dmeRc98ssSPNuNdAcU+kUsZvUTpFudNq+J6RQg1GNpednJmgecIcsIi1ZXPOvwC 99jGej8leMPctHk05apHPHZTO4lkz9Hq/A2HmVy3/LmiOix6E17d5iWcQCzgav3/bC/sp2NZodYI8 bnTN4vurXLIo8tBrxIRqAu/hHg9j1c1u/CEc4cQN8GiuALWuLbsxXm46Fwp4rdEzjUOSZ1ljgayk0 xzEGWQmWzqaOU3V8stchfcDOM+N3RDTMi8RMAKsR4hgISgADKNImj1tC6smwA1MZ3OgwZgJm6x9bm +cU9rWYANXphKXY/fAqkzayPLg==; From: vdixit@akamai.com To: ffmpeg-devel@ffmpeg.org Date: Tue, 9 Jan 2018 14:15:41 +0530 Message-Id: <1515487541-13767-1-git-send-email-vdixit@akamai.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: References: X-Smtpcorp-Track: 1-YpbPRyIJO0QD.VuEAKLQQE Feedback-ID: 337386m:337386asVRLGB:337386svOmvxRJyg:SMTPCORP X-Report-Abuse: Please forward a copy of this message, including all headers, to Subject: [FFmpeg-devel] [PATCH v2 1/1] avformat/hlsenc: closed caption tags in the master playlist X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Vishwanath Dixit MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Vishwanath Dixit --- doc/muxers.texi | 37 +++++++++++ libavformat/dashenc.c | 2 +- libavformat/hlsenc.c | 155 +++++++++++++++++++++++++++++++++++++++++++++- libavformat/hlsplaylist.c | 5 +- libavformat/hlsplaylist.h | 3 +- 5 files changed, 197 insertions(+), 5 deletions(-) diff --git a/doc/muxers.texi b/doc/muxers.texi index b060c4f..d9a5cc0 100644 --- a/doc/muxers.texi +++ b/doc/muxers.texi @@ -901,6 +901,43 @@ and they are mapped to the two video only variant streams with audio group names By default, a single hls variant containing all the encoded streams is created. +@item cc_stream_map +Map string which specifies different closed captions groups and their +attributes. The closed captions stream groups are separated by space. +Expected string format is like this +"ccgroup:,instreamid:,language: ....". +'ccgroup' and 'instreamid' are mandatory attributes. 'language' is an optional +attribute. +The closed captions groups configured using this option are mapped to different +variant streams by providing the same 'ccgroup' name in the +@code{var_stream_map} string. If @code{var_stream_map} is not set, then the +first available ccgroup in @code{cc_stream_map} is mapped to the output variant +stream. The examples for these two use cases are given below. + +@example +ffmpeg -re -i in.ts -b:v 1000k -b:a 64k -a53cc 1 -f hls \ + -cc_stream_map "ccgroup:cc,instreamid:CC1,language:en" \ + -master_pl_name master.m3u8 \ + http://example.com/live/out.m3u8 +@end example +This example adds @code{#EXT-X-MEDIA} tag with @code{TYPE=CLOSED-CAPTIONS} in +the master playlist with group name 'cc', langauge 'en' (english) and +INSTREAM-ID 'CC1'. Also, it adds @code{CLOSED-CAPTIONS} attribute with group +name 'cc' for the output variant stream. +@example +ffmpeg -re -i in.ts -b:v:0 1000k -b:v:1 256k -b:a:0 64k -b:a:1 32k \ + -a53cc:0 1 -a53cc:1 1\ + -map 0:v -map 0:a -map 0:v -map 0:a -f hls \ + -cc_stream_map "ccgroup:cc,instreamid:CC1,language:en ccgroup:cc,instreamid:CC2,language:sp" \ + -var_stream_map "v:0,a:0,ccgroup:cc v:1,a:1,ccgroup:cc" \ + -master_pl_name master.m3u8 \ + http://example.com/live/out_%v.m3u8 +@end example +This example adds two @code{#EXT-X-MEDIA} tags with @code{TYPE=CLOSED-CAPTIONS} in +the master playlist for the INSTREAM-IDs 'CC1' and 'CC2'. Also, it adds +@code{CLOSED-CAPTIONS} attribute with group name 'cc' for the two output variant +streams. + @item master_pl_name Create HLS master playlist with the given name. diff --git a/libavformat/dashenc.c b/libavformat/dashenc.c index 3345b89..39d0afe 100644 --- a/libavformat/dashenc.c +++ b/libavformat/dashenc.c @@ -820,7 +820,7 @@ static int write_manifest(AVFormatContext *s, int final) stream_bitrate += max_audio_bitrate; } get_hls_playlist_name(playlist_file, sizeof(playlist_file), NULL, i); - ff_hls_write_stream_info(st, out, stream_bitrate, playlist_file, agroup); + ff_hls_write_stream_info(st, out, stream_bitrate, playlist_file, agroup, NULL); } avio_close(out); if (use_rename) diff --git a/libavformat/hlsenc.c b/libavformat/hlsenc.c index e36120c..4e4b287 100644 --- a/libavformat/hlsenc.c +++ b/libavformat/hlsenc.c @@ -145,9 +145,16 @@ typedef struct VariantStream { unsigned int nb_streams; int m3u8_created; /* status of media play-list creation */ char *agroup; /* audio group name */ + char *ccgroup; /* closed caption group name */ char *baseurl; } VariantStream; +typedef struct ClosedCaptionsStream { + char *ccgroup; /* closed caption group name */ + char *instreamid; /* closed captions INSTREAM-ID */ + char *language; /* closed captions langauge */ +} ClosedCaptionsStream; + typedef struct HLSContext { const AVClass *class; // Class for private options. int64_t start_sequence; @@ -196,11 +203,14 @@ typedef struct HLSContext { VariantStream *var_streams; unsigned int nb_varstreams; + ClosedCaptionsStream *cc_streams; + unsigned int nb_ccstreams; int master_m3u8_created; /* status of master play-list creation */ char *master_m3u8_url; /* URL of the master m3u8 file */ int version; /* HLS version */ char *var_stream_map; /* user specified variant stream map string */ + char *cc_stream_map; /* user specified closed caption streams map string */ char *master_pl_name; unsigned int master_publish_rate; int http_persistent; @@ -1115,7 +1125,8 @@ static int create_master_playlist(AVFormatContext *s, AVDictionary *options = NULL; unsigned int i, j; int m3u8_name_size, ret, bandwidth; - char *m3u8_rel_name; + char *m3u8_rel_name, *ccgroup; + ClosedCaptionsStream *ccs; input_vs->m3u8_created = 1; if (!hls->master_m3u8_created) { @@ -1142,6 +1153,16 @@ static int create_master_playlist(AVFormatContext *s, ff_hls_write_playlist_version(hls->m3u8_out, hls->version); + for (i = 0; i < hls->nb_ccstreams; i++) { + ccs = &(hls->cc_streams[i]); + avio_printf(hls->m3u8_out, "#EXT-X-MEDIA:TYPE=CLOSED-CAPTIONS"); + avio_printf(hls->m3u8_out, ",GROUP-ID=\"%s\"", ccs->ccgroup); + avio_printf(hls->m3u8_out, ",NAME=\"%s\"", ccs->instreamid); + if (ccs->language) + avio_printf(hls->m3u8_out, ",LANGUAGE=\"%s\"", ccs->language); + avio_printf(hls->m3u8_out, ",INSTREAM-ID=\"%s\"\n", ccs->instreamid); + } + /* For audio only variant streams add #EXT-X-MEDIA tag with attributes*/ for (i = 0; i < hls->nb_varstreams; i++) { vs = &(hls->var_streams[i]); @@ -1226,8 +1247,23 @@ static int create_master_playlist(AVFormatContext *s, bandwidth += aud_st->codecpar->bit_rate; bandwidth += bandwidth / 10; + ccgroup = NULL; + if (vid_st && vs->ccgroup) { + /* check if this group name is available in the cc map string */ + for (j = 0; j < hls->nb_ccstreams; j++) { + ccs = &(hls->cc_streams[j]); + if (!av_strcasecmp(ccs->ccgroup, vs->ccgroup)) { + ccgroup = vs->ccgroup; + break; + } + } + if (j == hls->nb_ccstreams) + av_log(NULL, AV_LOG_WARNING, "mapping ccgroup %s not found\n", + vs->ccgroup); + } + ff_hls_write_stream_info(vid_st, hls->m3u8_out, bandwidth, m3u8_rel_name, - aud_st ? vs->agroup : NULL); + aud_st ? vs->agroup : NULL, ccgroup); av_freep(&m3u8_rel_name); } @@ -1714,6 +1750,11 @@ static int parse_variant_stream_mapstring(AVFormatContext *s) if (!vs->agroup) return AVERROR(ENOMEM); continue; + } else if (av_strstart(keyval, "ccgroup:", &val)) { + vs->ccgroup = av_strdup(val); + if (!vs->ccgroup) + return AVERROR(ENOMEM); + continue; } else if (av_strstart(keyval, "v:", &val)) { codec_type = AVMEDIA_TYPE_VIDEO; } else if (av_strstart(keyval, "a:", &val)) { @@ -1744,9 +1785,94 @@ static int parse_variant_stream_mapstring(AVFormatContext *s) return 0; } +static int parse_cc_stream_mapstring(AVFormatContext *s) +{ + HLSContext *hls = s->priv_data; + int nb_ccstreams; + char *p, *q, *saveptr1, *saveptr2, *ccstr, *keyval; + const char *val; + ClosedCaptionsStream *ccs; + + p = av_strdup(hls->cc_stream_map); + q = p; + while(av_strtok(q, " \t", &saveptr1)) { + q = NULL; + hls->nb_ccstreams++; + } + av_freep(&p); + + hls->cc_streams = av_mallocz(sizeof(*hls->cc_streams) * hls->nb_ccstreams); + if (!hls->cc_streams) + return AVERROR(ENOMEM); + + p = hls->cc_stream_map; + nb_ccstreams = 0; + while (ccstr = av_strtok(p, " \t", &saveptr1)) { + p = NULL; + + if (nb_ccstreams < hls->nb_ccstreams) + ccs = &(hls->cc_streams[nb_ccstreams++]); + else + return AVERROR(EINVAL); + + while (keyval = av_strtok(ccstr, ",", &saveptr2)) { + ccstr = NULL; + + if (av_strstart(keyval, "ccgroup:", &val)) { + ccs->ccgroup = av_strdup(val); + if (!ccs->ccgroup) + return AVERROR(ENOMEM); + } else if (av_strstart(keyval, "instreamid:", &val)) { + ccs->instreamid = av_strdup(val); + if (!ccs->instreamid) + return AVERROR(ENOMEM); + } else if (av_strstart(keyval, "language:", &val)) { + ccs->language = av_strdup(val); + if (!ccs->language) + return AVERROR(ENOMEM); + } else { + av_log(s, AV_LOG_ERROR, "Invalid keyval %s\n", keyval); + return AVERROR(EINVAL); + } + } + + if (!ccs->ccgroup || !ccs->instreamid) { + av_log(s, AV_LOG_ERROR, "Insufficient parameters in cc stream map string\n"); + return AVERROR(EINVAL); + } + + if (av_strstart(ccs->instreamid, "CC", &val)) { + if(atoi(val) < 1 || atoi(val) > 4) { + av_log(s, AV_LOG_ERROR, "Invalid instream ID CC index %d in %s, range 1-4\n", + atoi(val), ccs->instreamid); + return AVERROR(EINVAL); + } + } else if (av_strstart(ccs->instreamid, "SERVICE", &val)) { + if(atoi(val) < 1 || atoi(val) > 63) { + av_log(s, AV_LOG_ERROR, "Invalid instream ID SERVICE index %d in %s, range 1-63 \n", + atoi(val), ccs->instreamid); + return AVERROR(EINVAL); + } + } else { + av_log(s, AV_LOG_ERROR, "Invalid instream ID %s, supported are CCn or SERIVICEn\n", + ccs->instreamid); + return AVERROR(EINVAL); + } + } + + return 0; +} + static int update_variant_stream_info(AVFormatContext *s) { HLSContext *hls = s->priv_data; unsigned int i; + int ret = 0; + + if (hls->cc_stream_map) { + ret = parse_cc_stream_mapstring(s); + if (ret < 0) + return ret; + } if (hls->var_stream_map) { return parse_variant_stream_mapstring(s); @@ -1764,6 +1890,13 @@ static int update_variant_stream_info(AVFormatContext *s) { if (!hls->var_streams[0].streams) return AVERROR(ENOMEM); + //by default, the first available ccgroup is mapped to the variant stream + if (hls->nb_ccstreams) { + hls->var_streams[0].ccgroup = av_strdup(hls->cc_streams[0].ccgroup); + if (!hls->var_streams[0].ccgroup) + return AVERROR(ENOMEM); + } + for (i = 0; i < s->nb_streams; i++) hls->var_streams[0].streams[i] = s->streams[i]; } @@ -2127,13 +2260,22 @@ failed: av_freep(&vs->m3u8_name); av_freep(&vs->streams); av_freep(&vs->agroup); + av_freep(&vs->ccgroup); av_freep(&vs->baseurl); } + for (i = 0; i < hls->nb_ccstreams; i++) { + ClosedCaptionsStream *ccs = &hls->cc_streams[i]; + av_freep(&ccs->ccgroup); + av_freep(&ccs->instreamid); + av_freep(&ccs->language); + } + ff_format_io_close(s, &hls->m3u8_out); ff_format_io_close(s, &hls->sub_m3u8_out); av_freep(&hls->key_basename); av_freep(&hls->var_streams); + av_freep(&hls->cc_streams); av_freep(&hls->master_m3u8_url); return 0; } @@ -2470,13 +2612,21 @@ fail: av_freep(&vs->vtt_m3u8_name); av_freep(&vs->streams); av_freep(&vs->agroup); + av_freep(&vs->ccgroup); av_freep(&vs->baseurl); if (vs->avf) avformat_free_context(vs->avf); if (vs->vtt_avf) avformat_free_context(vs->vtt_avf); } + for (i = 0; i < hls->nb_ccstreams; i++) { + ClosedCaptionsStream *ccs = &hls->cc_streams[i]; + av_freep(&ccs->ccgroup); + av_freep(&ccs->instreamid); + av_freep(&ccs->language); + } av_freep(&hls->var_streams); + av_freep(&hls->cc_streams); av_freep(&hls->master_m3u8_url); } @@ -2536,6 +2686,7 @@ static const AVOption options[] = { {"datetime", "current datetime as YYYYMMDDhhmmss", 0, AV_OPT_TYPE_CONST, {.i64 = HLS_START_SEQUENCE_AS_FORMATTED_DATETIME }, INT_MIN, INT_MAX, E, "start_sequence_source_type" }, {"http_user_agent", "override User-Agent field in HTTP header", OFFSET(user_agent), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, E}, {"var_stream_map", "Variant stream map string", OFFSET(var_stream_map), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, E}, + {"cc_stream_map", "Closed captions stream map string", OFFSET(cc_stream_map), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, E}, {"master_pl_name", "Create HLS master playlist with this name", OFFSET(master_pl_name), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, E}, {"master_pl_publish_rate", "Publish master play list every after this many segment intervals", OFFSET(master_publish_rate), AV_OPT_TYPE_INT, {.i64 = 0}, 0, UINT_MAX, E}, {"http_persistent", "Use persistent HTTP connections", OFFSET(http_persistent), AV_OPT_TYPE_BOOL, {.i64 = 0 }, 0, 1, E }, diff --git a/libavformat/hlsplaylist.c b/libavformat/hlsplaylist.c index 098dc89..e797f14 100644 --- a/libavformat/hlsplaylist.c +++ b/libavformat/hlsplaylist.c @@ -46,7 +46,8 @@ void ff_hls_write_audio_rendition(AVIOContext *out, char *agroup, } void ff_hls_write_stream_info(AVStream *st, AVIOContext *out, - int bandwidth, char *filename, char *agroup) { + int bandwidth, char *filename, char *agroup, + char *ccgroup) { if (!out || !filename) return; @@ -62,6 +63,8 @@ void ff_hls_write_stream_info(AVStream *st, AVIOContext *out, st->codecpar->height); if (agroup && strlen(agroup) > 0) avio_printf(out, ",AUDIO=\"group_%s\"", agroup); + if (ccgroup && strlen(ccgroup) > 0) + avio_printf(out, ",CLOSED-CAPTIONS=\"%s\"", ccgroup); avio_printf(out, "\n%s\n\n", filename); } diff --git a/libavformat/hlsplaylist.h b/libavformat/hlsplaylist.h index 9969315..29c5123 100644 --- a/libavformat/hlsplaylist.h +++ b/libavformat/hlsplaylist.h @@ -40,7 +40,8 @@ void ff_hls_write_playlist_version(AVIOContext *out, int version); void ff_hls_write_audio_rendition(AVIOContext *out, char *agroup, char *filename, int name_id, int is_default); void ff_hls_write_stream_info(AVStream *st, AVIOContext *out, - int bandwidth, char *filename, char *agroup); + int bandwidth, char *filename, char *agroup, + char *ccgroup); void ff_hls_write_playlist_header(AVIOContext *out, int version, int allowcache, int target_duration, int64_t sequence, uint32_t playlist_type);