From patchwork Sun Mar 29 11:13:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lance Wang X-Patchwork-Id: 18483 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 59946448E87 for ; Sun, 29 Mar 2020 14:13:58 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 43A7C68B2A6; Sun, 29 Mar 2020 14:13:58 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pg1-f176.google.com (mail-pg1-f176.google.com [209.85.215.176]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2712C68B0AD for ; Sun, 29 Mar 2020 14:13:51 +0300 (EEST) Received: by mail-pg1-f176.google.com with SMTP id l14so100290pgb.1 for ; Sun, 29 Mar 2020 04:13:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=q0p2NKKlxWfaQXhNI6+y/5K3CbtkxLmjD2b/9ZtVBzo=; b=c6HdJoUFMvIWv/anJJRaLIUhnJplbU9u70Zm9akOkAcBUq+0irdIGPv25gCkVdwLlG RHamKR4pCRrMw2B095+dADKhYXK+MQUq3cAaNbCWFsvJwereVFWK6ejwJstklE06DitK XoYmfU8Sx4zLzD8HrAw2Fq8HZ/gqUI/leh4ACSzOjxlz1dvCg05uh9ST/pyh7LP+Hoa1 9rWDlw2d4UOGUm5s82ced+RUHsmo81RnfbrfozrvGmmr4wC5YCIRYM2HOh483+A1b/ra yRREE4+4VSJdCwo8WgF2i+IgjJtDJrO9+xLKKIRviVCT0GHstywmR427H3SpiKM3PUss fq0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=q0p2NKKlxWfaQXhNI6+y/5K3CbtkxLmjD2b/9ZtVBzo=; b=ri4tEe+TAP76Ek0C8czAzhMBSDnUSgdIDw2LsW0m4lW+wOfQdruqvxREjGgmn/Xq34 wWSiXLZ7U44ztd+xeBzEYBPvSnBK2gHmbU1c+ZerilNvYnvOEMw1eabLs1b3Vq/TSNyE wi+oEkdcA9LOw2gYGEjpW5cOiCCUwRLDk0bZr+ni1UQh84wOSPvpcP1doZyc6zretOqr WTzfdrlO1nDtMPrL87Bn+a1q1goY8EqxjeKbzpckbt458bnaTcDC2Fbyonbb/lRHqsZ0 VlsNUWh8F9eFr/nKK/nXs78jxKhl01XEsZLJcd/IP68dw/gU9agmI1O7sqycY/aeAc18 mhsg== X-Gm-Message-State: ANhLgQ117vnTOuF0Jur6jbcytw9g3PqSzUVUk15vaAvDldpSmoLD6PlZ DcNQ9qOoEYkjanAecSaxIG45H+bN X-Google-Smtp-Source: ADFU+vvGNv9wi35BhBqmiMiIM8NPIn6cVDjBuqM4piBG2hLtexb0Z1BJxcm9J9q5tv6k52ZJjOYf7w== X-Received: by 2002:a65:68cb:: with SMTP id k11mr7963306pgt.78.1585480429048; Sun, 29 Mar 2020 04:13:49 -0700 (PDT) Received: from vpn.localdomain ([47.90.99.151]) by smtp.gmail.com with ESMTPSA id p1sm7900069pfq.114.2020.03.29.04.13.47 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 29 Mar 2020 04:13:48 -0700 (PDT) From: lance.lmwang@gmail.com To: ffmpeg-devel@ffmpeg.org Date: Sun, 29 Mar 2020 19:13:44 +0800 Message-Id: <20200329111344.31907-1-lance.lmwang@gmail.com> X-Mailer: git-send-email 2.9.5 In-Reply-To: <20200326135700.11167-3-lance.lmwang@gmail.com> References: <20200326135700.11167-3-lance.lmwang@gmail.com> Subject: [FFmpeg-devel] [PATCH v2 3/4] avformat: add subtitle support in master playlist m3u8 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Limin Wang MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Limin Wang Test with the following command for the webvtt subtitle: $ ./ffmpeg -y -i input_with_subtitle.mkv \ -b:v:0 5250k -c:v h264 -pix_fmt yuv420p -profile:v main -level 4.1 \ -b:a:0 256k \ -c:s webvtt -c:a mp2 -ar 48000 -ac 2 -map 0:v -map 0:a:0 -map 0:s:0 \ -f hls -var_stream_map "v:0,a:0,s:0,sgroup:subtitle" \ -master_pl_name master.m3u8 -t 300 -hls_time 10 -hls_init_time 4 -hls_list_size \ 10 -master_pl_publish_rate 10 -hls_flags \ delete_segments+discont_start+split_by_time ./tmp/video.m3u8 Check the master m3u8: $ cat tmp/master.m3u8 #EXTM3U #EXT-X-VERSION:3 #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subtitle",NAME="subtitle_0",DEFAULT=YES,URI="video_vtt.m3u8" #EXT-X-STREAM-INF:BANDWIDTH=6056600,RESOLUTION=1280x720,CODECS="avc1.4d4829,mp4a.40.33",SUBTITLES="subtitle" video.m3u8 Check the result by convert to mkv: $ ./ffmpeg -strict experimental -i ./tmp/master.m3u8 -c:v copy -c:a mp2 -c:s srt ./test.mkv Signed-off-by: Limin Wang --- doc/muxers.texi | 15 +++++++++++++++ libavformat/dashenc.c | 2 +- libavformat/hlsenc.c | 26 ++++++++++++++++++++++++-- libavformat/hlsplaylist.c | 17 ++++++++++++++++- libavformat/hlsplaylist.h | 4 +++- 5 files changed, 59 insertions(+), 5 deletions(-) diff --git a/doc/muxers.texi b/doc/muxers.texi index d304181..5b40691 100644 --- a/doc/muxers.texi +++ b/doc/muxers.texi @@ -1062,6 +1062,21 @@ have and language is named ENG, the other audio language is named CHN. By default, a single hls variant containing all the encoded streams is created. +@example +ffmpeg -y -i input_with_subtitle.mkv \ + -b:v:0 5250k -c:v h264 -pix_fmt yuv420p -profile:v main -level 4.1 \ + -b:a:0 256k \ + -c:s webvtt -c:a mp2 -ar 48000 -ac 2 -map 0:v -map 0:a:0 -map 0:s:0 \ + -f hls -var_stream_map "v:0,a:0,s:0,sgroup:subtitle" \ + -master_pl_name master.m3u8 -t 300 -hls_time 10 -hls_init_time 4 -hls_list_size \ + 10 -master_pl_publish_rate 10 -hls_flags \ + delete_segments+discont_start+split_by_time ./tmp/video.m3u8 +@end example + +This example adds @code{#EXT-X-MEDIA} tag with @code{TYPE=SUBTITLES} in +the master playlist with webvtt subtitle group name 'subtitle'. Please make sure +the input file has one text subtitle stream at least. + @item cc_stream_map Map string which specifies different closed captions groups and their attributes. The closed captions stream groups are separated by space. diff --git a/libavformat/dashenc.c b/libavformat/dashenc.c index 94d4639..d1fe90b 100644 --- a/libavformat/dashenc.c +++ b/libavformat/dashenc.c @@ -1311,7 +1311,7 @@ static int write_manifest(AVFormatContext *s, int final) get_hls_playlist_name(playlist_file, sizeof(playlist_file), NULL, i); ff_hls_write_stream_info(st, c->m3u8_out, stream_bitrate, playlist_file, agroup, - codec_str_ptr, NULL); + codec_str_ptr, NULL, NULL); } dashenc_io_close(s, &c->m3u8_out, temp_filename); if (use_rename) diff --git a/libavformat/hlsenc.c b/libavformat/hlsenc.c index a0a3a46..d7b9c0e 100644 --- a/libavformat/hlsenc.c +++ b/libavformat/hlsenc.c @@ -164,6 +164,7 @@ typedef struct VariantStream { int is_default; /* default status of audio group */ char *language; /* audio lauguage name */ char *agroup; /* audio group name */ + char *sgroup; /* subtitle group name */ char *ccgroup; /* closed caption group name */ char *baseurl; char *varname; // variant name @@ -1289,7 +1290,9 @@ static int create_master_playlist(AVFormatContext *s, unsigned int i, j; int ret, bandwidth; const char *m3u8_rel_name = NULL; + const char *vtt_m3u8_rel_name = NULL; char *ccgroup; + char *sgroup = NULL; ClosedCaptionsStream *ccs; const char *proto = avio_find_protocol_name(hls->master_m3u8_url); int is_file_proto = proto && !strcmp(proto, "file"); @@ -1412,13 +1415,24 @@ static int create_master_playlist(AVFormatContext *s, vs->ccgroup); } + if (vid_st && vs->sgroup) { + sgroup = vs->sgroup; + vtt_m3u8_rel_name = get_relative_url(hls->master_m3u8_url, vs->vtt_m3u8_name); + if (!vtt_m3u8_rel_name) { + av_log(s, AV_LOG_WARNING, "Unable to find relative subtitle URL\n"); + break; + } + + ff_hls_write_subtitle_rendition(hls->m3u8_out, sgroup, vtt_m3u8_rel_name, vs->language, i, hls->has_default_key ? vs->is_default : 1); + } + if (!hls->has_default_key || !hls->has_video_m3u8) { ff_hls_write_stream_info(vid_st, hls->m3u8_out, bandwidth, m3u8_rel_name, - aud_st ? vs->agroup : NULL, vs->codec_attr, ccgroup); + aud_st ? vs->agroup : NULL, vs->codec_attr, ccgroup, sgroup); } else { if (vid_st) { ff_hls_write_stream_info(vid_st, hls->m3u8_out, bandwidth, m3u8_rel_name, - aud_st ? vs->agroup : NULL, vs->codec_attr, ccgroup); + aud_st ? vs->agroup : NULL, vs->codec_attr, ccgroup, sgroup); } } } @@ -1893,6 +1907,7 @@ static int parse_variant_stream_mapstring(AVFormatContext *s) * practical usage) * * agroup: is key to specify audio group. A string can be given as value. + * sgroup: is key to specify subtitle group. A string can be given as value. */ p = av_strdup(hls->var_stream_map); if (!p) @@ -1960,6 +1975,12 @@ static int parse_variant_stream_mapstring(AVFormatContext *s) if (!vs->agroup) return AVERROR(ENOMEM); continue; + } else if (av_strstart(keyval, "sgroup:", &val)) { + av_free(vs->sgroup); + vs->sgroup = av_strdup(val); + if (!vs->sgroup) + return AVERROR(ENOMEM); + continue; } else if (av_strstart(keyval, "ccgroup:", &val)) { av_free(vs->ccgroup); vs->ccgroup = av_strdup(val); @@ -2516,6 +2537,7 @@ static void hls_free_variant_streams(struct HLSContext *hls) av_freep(&vs->m3u8_name); av_freep(&vs->streams); av_freep(&vs->agroup); + av_freep(&vs->sgroup); av_freep(&vs->language); av_freep(&vs->ccgroup); av_freep(&vs->baseurl); diff --git a/libavformat/hlsplaylist.c b/libavformat/hlsplaylist.c index 5624449..43f9d28 100644 --- a/libavformat/hlsplaylist.c +++ b/libavformat/hlsplaylist.c @@ -48,9 +48,22 @@ void ff_hls_write_audio_rendition(AVIOContext *out, char *agroup, avio_printf(out, "URI=\"%s\"\n", filename); } +void ff_hls_write_subtitle_rendition(AVIOContext *out, char *sgroup, + const char *filename, char *language, int name_id, int is_default) { + if (!out || !filename) + return; + + avio_printf(out, "#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID=\"%s\"", sgroup); + avio_printf(out, ",NAME=\"subtitle_%d\",DEFAULT=%s,", name_id, is_default ? "YES" : "NO"); + if (language) { + avio_printf(out, "LANGUAGE=\"%s\",", language); + } + avio_printf(out, "URI=\"%s\"\n", filename); +} + void ff_hls_write_stream_info(AVStream *st, AVIOContext *out, int bandwidth, const char *filename, char *agroup, - char *codecs, char *ccgroup) { + char *codecs, char *ccgroup, char *sgroup) { if (!out || !filename) return; @@ -71,6 +84,8 @@ void ff_hls_write_stream_info(AVStream *st, AVIOContext *out, avio_printf(out, ",AUDIO=\"group_%s\"", agroup); if (ccgroup && ccgroup[0]) avio_printf(out, ",CLOSED-CAPTIONS=\"%s\"", ccgroup); + if (sgroup && sgroup[0]) + avio_printf(out, ",SUBTITLES=\"%s\"", sgroup); avio_printf(out, "\n%s\n\n", filename); } diff --git a/libavformat/hlsplaylist.h b/libavformat/hlsplaylist.h index a8d29d6..a124bdc 100644 --- a/libavformat/hlsplaylist.h +++ b/libavformat/hlsplaylist.h @@ -39,9 +39,11 @@ typedef enum { void ff_hls_write_playlist_version(AVIOContext *out, int version); void ff_hls_write_audio_rendition(AVIOContext *out, char *agroup, const char *filename, char *language, int name_id, int is_default); +void ff_hls_write_subtitle_rendition(AVIOContext *out, char *sgroup, + const char *filename, char *language, int name_id, int is_default); void ff_hls_write_stream_info(AVStream *st, AVIOContext *out, int bandwidth, const char *filename, char *agroup, - char *codecs, char *ccgroup); + char *codecs, char *ccgroup, char *sgroup); void ff_hls_write_playlist_header(AVIOContext *out, int version, int allowcache, int target_duration, int64_t sequence, uint32_t playlist_type, int iframe_mode);