[FFmpeg-devel] libavformat/webvttenc: Allow (but discard) additional streams

Submitted by Soft Works on Sept. 4, 2019, 8:43 p.m.

Details

Message ID MN2PR13MB275255C8B4A6B06598E3158DBAB80@MN2PR13MB2752.namprd13.prod.outlook.com
State New
Headers show

Commit Message

Soft Works Sept. 4, 2019, 8:43 p.m.
This allows having a video stream as reference stream  when using the segment muxer
---
 libavformat/webvttenc.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

Comments

Carl Eugen Hoyos Sept. 4, 2019, 10:24 p.m.
Am Mi., 4. Sept. 2019 um 22:43 Uhr schrieb Soft Works <softworkz@hotmail.com>:
>
>
> This allows having a video stream as reference stream  when using the segment muxer

Please add the command line that needs this patch.

Carl Eugen
Soft Works Sept. 4, 2019, 10:37 p.m.
> -----Original Message-----

> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of

> Carl Eugen Hoyos

> 

> Am Mi., 4. Sept. 2019 um 22:43 Uhr schrieb Soft Works

> <softworkz@hotmail.com>:

> >

> >

> > This allows having a video stream as reference stream  when using the

> > segment muxer

> 

> Please add the command line that needs this patch.

> 

> Carl Eugen


The short version would be this:
------------------
Allows mapping a video stream as reference stream for creating vtt segments:
ffmpeg -i INPUT -map 0:3 -c:0 webvtt -map 0:0 -c:v:0 copy -f segment -segment_format webvtt -segment_time 6 -write_empty_segments 1 -y "sub_segment3%d.vtt"
------------------

Is this sufficient? Otherwise, here's the long story:

THE REQUIREMENT
I need to do HLS segmentation for video/audio while additionally creating one or more subtitle playlists and segments. Very simplified it’s like this:

ffmpeg -i INPUT
  -map 0:0 -map 0:2 -c:v:0 copy -c:a:0 copy -f segment -segment_format mpegts -segment_time 6 -y "segment%d.ts"
  -map 0:3 -c:0 webvtt -f segment -segment_format webvtt -segment_time 6 -write_empty_segments 1 -y "sub_segment3%d.vtt"


THE PROBLEM
The mpegts segments are being created in a constant and regular way. But not necessearily in the case of subtitles: When hitting a time range without any subtitle event, the segmenter stops generating subtitle segments, even when write_empty_segments is configured.
It’s not that write_empty_segments would not be working, though: As soon as a new subtitle event reaches the muxer, it catches up by creating the missed empty segments all at once. This might be OK for precreating content for HLS/VOD scenarios but it’s a behavioral bug when you rely on having accurate results while the process is running.

THE CAUSE
That’s rather trivial: When there are no subtitle events, there are no packets arriving at the segment muxer (for vtt) and there’s probably nothiing that the segment muxer could do about it.

A REASONABLE APPROACH?
After some rather ugly ideas I found that the best way to bring some more action into the vtt segmenter might be to map the video stream in addition to the subtitle stream – not with the purpose of outputting the video – just to ensure regular activity in the segment muxer.
Also, the segment muxer has implementation for having a ‚reference_stream‘ and the video stream is supposed to be this anyway.

Remains one problem: The WebVTT muxer (libavformat/webvttenc.c) errors when there’s more than a single stream. ("Exactly one WebVTT stream is needed.) 
And that’s what this commit does: Removed the single-stream limit from the webvtt muxer and simply discard packets from other streams instead of failing.

softworkz

Patch hide | download patch | download mbox

diff --git a/libavformat/webvttenc.c b/libavformat/webvttenc.c
index 61b7f54622..be85c03130 100644
--- a/libavformat/webvttenc.c
+++ b/libavformat/webvttenc.c
@@ -49,8 +49,8 @@  static int webvtt_write_header(AVFormatContext *ctx)
     AVCodecParameters *par = ctx->streams[0]->codecpar;
     AVIOContext *pb = ctx->pb;
 
-    if (ctx->nb_streams != 1 || par->codec_id != AV_CODEC_ID_WEBVTT) {
-        av_log(ctx, AV_LOG_ERROR, "Exactly one WebVTT stream is needed.\n");
+    if (par->codec_id != AV_CODEC_ID_WEBVTT) {
+        av_log(ctx, AV_LOG_ERROR, "First stream must be WebVTT.\n");
         return AVERROR(EINVAL);
     }
 
@@ -68,6 +68,9 @@  static int webvtt_write_packet(AVFormatContext *ctx, AVPacket *pkt)
     int id_size, settings_size;
     uint8_t *id, *settings;
 
+    if (pkt->stream_index != 0)
+        return 0;
+
     avio_printf(pb, "\n");
 
     id = av_packet_get_side_data(pkt, AV_PKT_DATA_WEBVTT_IDENTIFIER,