diff mbox series

[FFmpeg-devel] added parameter to dash encoder for start available time

Message ID 20210318011727.517496-1-jerome@percipient.ai
State New
Headers show
Series [FFmpeg-devel] added parameter to dash encoder for start available time
Related show

Checks

Context Check Description
andriy/configure warning Failed to apply patch
andriy/configure warning Failed to apply patch

Commit Message

Jerome Berclaz March 18, 2021, 1:17 a.m. UTC
---
 libavformat/dashenc.c | 28 ++++++++++++++++++++--------
 1 file changed, 20 insertions(+), 8 deletions(-)

Comments

Jan Ekström March 18, 2021, 2:42 p.m. UTC | #1
On Thu, Mar 18, 2021 at 3:49 AM Jerome Berclaz <jerome@percipient.ai> wrote:
>
> ---
>  libavformat/dashenc.c | 28 ++++++++++++++++++++--------
>  1 file changed, 20 insertions(+), 8 deletions(-)

So right now we do have an option in ffmpeg.c called itsoffset. It
lets you configure the input offset in time (seconds - such as
"1337.123" or time format - such as "00:00:01.123"). The only problem
with it right now is that unless you set -copyts it will not get
applied for anything else than subtitles, as the offset gets applied
before and is not taken into account in the "start from PTS 0"
calculation for video/audio streams. I have been meaning to do some
git blaming and asking the people still around who added that logic,
if that was really meant - since as far as I can tell even people such
as Martin actually recommended me this option - probably expecting for
it to work.

If we just make itsoffset usable without copyts (just take it into
mention in the zero'ification process), I think all these options in
the output modules would effectively become unnecessary, since your
DTS/PTS will start from nonzero positive value depending on the
itsoffset.

Best regards,
Jan
Gyan Doshi March 18, 2021, 3:01 p.m. UTC | #2
On 2021-03-18 20:12, Jan Ekström wrote:
> So right now we do have an option in ffmpeg.c called itsoffset. It
> lets you configure the input offset in time (seconds - such as
> "1337.123" or time format - such as "00:00:01.123"). The only problem
> with it right now is that unless you set -copyts it will not get
> applied for anything else than subtitles, as the offset gets applied
> before and is not taken into account in the "start from PTS 0"
> calculation for video/audio streams.

Not the case.

     ffmpeg -f lavfi -itsoffset 3 -i nullsrc=r=1:d=3 -vf showinfo -f null -

gives

[Parsed_showinfo_0 @ 0000019cc1b88540] n:   0 pts:      3 pts_time:3   ...
[Parsed_showinfo_0 @ 0000019cc1b88540] n:   1 pts:      4 pts_time:4   ...
[Parsed_showinfo_0 @ 0000019cc1b88540] n:   2 pts:      5 pts_time:5   ...

I've used it many times to adjust sync among streams.

Regards,
Gyan
Jan Ekström March 18, 2021, 3:07 p.m. UTC | #3
On Thu, Mar 18, 2021, 17:01 Gyan Doshi <ffmpeg@gyani.pro> wrote:

>
>
> On 2021-03-18 20:12, Jan Ekström wrote:
> > So right now we do have an option in ffmpeg.c called itsoffset. It
> > lets you configure the input offset in time (seconds - such as
> > "1337.123" or time format - such as "00:00:01.123"). The only problem
> > with it right now is that unless you set -copyts it will not get
> > applied for anything else than subtitles, as the offset gets applied
> > before and is not taken into account in the "start from PTS 0"
> > calculation for video/audio streams.
>
> Not the case.
>
>      ffmpeg -f lavfi -itsoffset 3 -i nullsrc=r=1:d=3 -vf showinfo -f null -
>
> gives
>
> [Parsed_showinfo_0 @ 0000019cc1b88540] n:   0 pts:      3 pts_time:3   ...
> [Parsed_showinfo_0 @ 0000019cc1b88540] n:   1 pts:      4 pts_time:4   ...
> [Parsed_showinfo_0 @ 0000019cc1b88540] n:   2 pts:      5 pts_time:5   ...
>
> I've used it many times to adjust sync among streams.
>
> Regards,
> Gyan
>

Very interesting!

I will have to retest, I just recall that the last time I tried it got
applied when there were multiple streams in an input when the start_time
based adjustment was done.

Jan

>
Jan Ekström March 18, 2021, 9:52 p.m. UTC | #4
On Thu, Mar 18, 2021 at 5:07 PM Jan Ekström <jeebjp@gmail.com> wrote:
>
> On Thu, Mar 18, 2021, 17:01 Gyan Doshi <ffmpeg@gyani.pro> wrote:
>>
>>
>>
>> On 2021-03-18 20:12, Jan Ekström wrote:
>> > So right now we do have an option in ffmpeg.c called itsoffset. It
>> > lets you configure the input offset in time (seconds - such as
>> > "1337.123" or time format - such as "00:00:01.123"). The only problem
>> > with it right now is that unless you set -copyts it will not get
>> > applied for anything else than subtitles, as the offset gets applied
>> > before and is not taken into account in the "start from PTS 0"
>> > calculation for video/audio streams.
>>
>> Not the case.
>>
>>      ffmpeg -f lavfi -itsoffset 3 -i nullsrc=r=1:d=3 -vf showinfo -f null -
>>
>> gives
>>
>> [Parsed_showinfo_0 @ 0000019cc1b88540] n:   0 pts:      3 pts_time:3   ...
>> [Parsed_showinfo_0 @ 0000019cc1b88540] n:   1 pts:      4 pts_time:4   ...
>> [Parsed_showinfo_0 @ 0000019cc1b88540] n:   2 pts:      5 pts_time:5   ...
>>
>> I've used it many times to adjust sync among streams.
>>
>> Regards,
>> Gyan
>
>
> Very interesting!
>
> I will have to retest, I just recall that the last time I tried it got applied when there were multiple streams in an input when the start_time based adjustment was done.

Right, so taking in a Matroska file it works fine, but when taking in
an MPEG-TS file with a start_time it not only transforms (start_time +
input_ts_offset) to (input_ts_offset), but just to 0 :) .

Example:
ffmpeg -v verbose -debug_ts -itsoffset "1616090389" -i
'https://megumin.fushizen.eu/samples/switchingaudio.ts' -c copy -bsf:a
aac_adtstoasc -movflags dash+frag_discont+delay_moov -vframes 5 -f mp4
- > test.mp4

Probed start_time:
  Duration: 00:00:15.25, start: 19516.478644, bitrate: 23823 kb/s

And how ffmpeg.c then adjusted it:
demuxer -> ist_index:2 type:audio next_dts:NOPTS next_dts_time:NOPTS
next_pts:NOPTS next_pts_time:NOPTS pkt_pts:1756483078
pkt_pts_time:19516.5 pkt_dts:1756483078 pkt_dts_time:19516.5
off:1616070872521356 off_time:1.61607e+09
demuxer+ffmpeg -> ist_index:2 type:audio pkt_pts:0 pkt_pts_time:0
pkt_dts:0 pkt_dts_time:0 off:-19516478644 off_time:-19516.5

Thus, ffmpeg.c is not only adjusting the timestamp to start at zero
with the start_time, but also takes the itsoffset off as well.

Best regards,
Jan
Gyan Doshi March 19, 2021, 4:19 a.m. UTC | #5
On 2021-03-19 03:22, Jan Ekström wrote:
> On Thu, Mar 18, 2021 at 5:07 PM Jan Ekström <jeebjp@gmail.com> wrote:
>> On Thu, Mar 18, 2021, 17:01 Gyan Doshi <ffmpeg@gyani.pro> wrote:
>>>
>>>
>>> On 2021-03-18 20:12, Jan Ekström wrote:
>>>> So right now we do have an option in ffmpeg.c called itsoffset. It
>>>> lets you configure the input offset in time (seconds - such as
>>>> "1337.123" or time format - such as "00:00:01.123"). The only problem
>>>> with it right now is that unless you set -copyts it will not get
>>>> applied for anything else than subtitles, as the offset gets applied
>>>> before and is not taken into account in the "start from PTS 0"
>>>> calculation for video/audio streams.
>>> Not the case.
>>>
>>>       ffmpeg -f lavfi -itsoffset 3 -i nullsrc=r=1:d=3 -vf showinfo -f null -
>>>
>>> gives
>>>
>>> [Parsed_showinfo_0 @ 0000019cc1b88540] n:   0 pts:      3 pts_time:3   ...
>>> [Parsed_showinfo_0 @ 0000019cc1b88540] n:   1 pts:      4 pts_time:4   ...
>>> [Parsed_showinfo_0 @ 0000019cc1b88540] n:   2 pts:      5 pts_time:5   ...
>>>
>>> I've used it many times to adjust sync among streams.
>>>
>>> Regards,
>>> Gyan
>>
>> Very interesting!
>>
>> I will have to retest, I just recall that the last time I tried it got applied when there were multiple streams in an input when the start_time based adjustment was done.
> Right, so taking in a Matroska file it works fine, but when taking in
> an MPEG-TS file with a start_time it not only transforms (start_time +
> input_ts_offset) to (input_ts_offset), but just to 0 :) .
>
> Example:
> ffmpeg -v verbose -debug_ts -itsoffset "1616090389" -i
> 'https://megumin.fushizen.eu/samples/switchingaudio.ts' -c copy -bsf:a
> aac_adtstoasc -movflags dash+frag_discont+delay_moov -vframes 5 -f mp4
> - > test.mp4
>
> Probed start_time:
>    Duration: 00:00:15.25, start: 19516.478644, bitrate: 23823 kb/s
>
> And how ffmpeg.c then adjusted it:
> demuxer -> ist_index:2 type:audio next_dts:NOPTS next_dts_time:NOPTS
> next_pts:NOPTS next_pts_time:NOPTS pkt_pts:1756483078
> pkt_pts_time:19516.5 pkt_dts:1756483078 pkt_dts_time:19516.5
> off:1616070872521356 off_time:1.61607e+09
> demuxer+ffmpeg -> ist_index:2 type:audio pkt_pts:0 pkt_pts_time:0
> pkt_dts:0 pkt_dts_time:0 off:-19516478644 off_time:-19516.5
>
> Thus, ffmpeg.c is not only adjusting the timestamp to start at zero
> with the start_time, but also takes the itsoffset off as well.

This affects formats with AVFMT_TS_DISCONT flag, ffmpeg checks for a 
discontinuity and readjusts the file offset to make output timestamps 
smooth.

Add -dts_delta_threshold 1616090389 alongside itsoffset to prevent this 
readjustment. Perhaps, both the itsoffset and ts_scale adjustments 
should be moved to after the discontinuity checks.

Regards,
Gyan
Jerome Berclaz March 19, 2021, 6:27 p.m. UTC | #6
Hi all,

Thanks for the comments on my patch. I should have given more explanations,
but this is my first time using git-send-email and I wasn't able to add
text to the patch.

The reason for the patch occurred while trying to live stream a Dash feed
with low latency. I did not use a timeline, hence having a precise start
time is crucial, otherwise the player will try to download chunks at the
wrong time. In the current implementation (branch 4.2) the start time is
set to the system time, which may not correspond exactly to the live feed
start time. This new parameter allows the user to pass the exact value for
the start time.

I've seen that the dash encoder has been refactored in branch 4.3, so I
don't know if this change is relevant anymore. The patch was intended for
branch 4.2.

Thanks,
Jerome

On Thu, Mar 18, 2021 at 9:20 PM Gyan Doshi <ffmpeg@gyani.pro> wrote:

>
>
> On 2021-03-19 03:22, Jan Ekström wrote:
> > On Thu, Mar 18, 2021 at 5:07 PM Jan Ekström <jeebjp@gmail.com> wrote:
> >> On Thu, Mar 18, 2021, 17:01 Gyan Doshi <ffmpeg@gyani.pro> wrote:
> >>>
> >>>
> >>> On 2021-03-18 20:12, Jan Ekström wrote:
> >>>> So right now we do have an option in ffmpeg.c called itsoffset. It
> >>>> lets you configure the input offset in time (seconds - such as
> >>>> "1337.123" or time format - such as "00:00:01.123"). The only problem
> >>>> with it right now is that unless you set -copyts it will not get
> >>>> applied for anything else than subtitles, as the offset gets applied
> >>>> before and is not taken into account in the "start from PTS 0"
> >>>> calculation for video/audio streams.
> >>> Not the case.
> >>>
> >>>       ffmpeg -f lavfi -itsoffset 3 -i nullsrc=r=1:d=3 -vf showinfo -f
> null -
> >>>
> >>> gives
> >>>
> >>> [Parsed_showinfo_0 @ 0000019cc1b88540] n:   0 pts:      3 pts_time:3
>  ...
> >>> [Parsed_showinfo_0 @ 0000019cc1b88540] n:   1 pts:      4 pts_time:4
>  ...
> >>> [Parsed_showinfo_0 @ 0000019cc1b88540] n:   2 pts:      5 pts_time:5
>  ...
> >>>
> >>> I've used it many times to adjust sync among streams.
> >>>
> >>> Regards,
> >>> Gyan
> >>
> >> Very interesting!
> >>
> >> I will have to retest, I just recall that the last time I tried it got
> applied when there were multiple streams in an input when the start_time
> based adjustment was done.
> > Right, so taking in a Matroska file it works fine, but when taking in
> > an MPEG-TS file with a start_time it not only transforms (start_time +
> > input_ts_offset) to (input_ts_offset), but just to 0 :) .
> >
> > Example:
> > ffmpeg -v verbose -debug_ts -itsoffset "1616090389" -i
> > 'https://megumin.fushizen.eu/samples/switchingaudio.ts' -c copy -bsf:a
> > aac_adtstoasc -movflags dash+frag_discont+delay_moov -vframes 5 -f mp4
> > - > test.mp4
> >
> > Probed start_time:
> >    Duration: 00:00:15.25, start: 19516.478644, bitrate: 23823 kb/s
> >
> > And how ffmpeg.c then adjusted it:
> > demuxer -> ist_index:2 type:audio next_dts:NOPTS next_dts_time:NOPTS
> > next_pts:NOPTS next_pts_time:NOPTS pkt_pts:1756483078
> > pkt_pts_time:19516.5 pkt_dts:1756483078 pkt_dts_time:19516.5
> > off:1616070872521356 off_time:1.61607e+09
> > demuxer+ffmpeg -> ist_index:2 type:audio pkt_pts:0 pkt_pts_time:0
> > pkt_dts:0 pkt_dts_time:0 off:-19516478644 off_time:-19516.5
> >
> > Thus, ffmpeg.c is not only adjusting the timestamp to start at zero
> > with the start_time, but also takes the itsoffset off as well.
>
> This affects formats with AVFMT_TS_DISCONT flag, ffmpeg checks for a
> discontinuity and readjusts the file offset to make output timestamps
> smooth.
>
> Add -dts_delta_threshold 1616090389 alongside itsoffset to prevent this
> readjustment. Perhaps, both the itsoffset and ts_scale adjustments
> should be moved to after the discontinuity checks.
>
> Regards,
> Gyan
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
diff mbox series

Patch

diff --git a/libavformat/dashenc.c b/libavformat/dashenc.c
index 24d43c34ea..81855ca8d0 100644
--- a/libavformat/dashenc.c
+++ b/libavformat/dashenc.c
@@ -149,6 +149,7 @@  typedef struct DASHContext {
     int master_publish_rate;
     int nr_of_streams_to_flush;
     int nr_of_streams_flushed;
+    int64_t start_time_ms;
 } DASHContext;
 
 static struct codec_string {
@@ -725,13 +726,11 @@  static void write_time(AVIOContext *out, int64_t time)
     avio_printf(out, "%d.%dS", seconds, fractions / (AV_TIME_BASE / 10));
 }
 
-static void format_date_now(char *buf, int size)
+static void format_date(uint64_t epoch_ms, char *buf, int size)
 {
     struct tm *ptm, tmbuf;
-    int64_t time_us = av_gettime();
-    int64_t time_ms = time_us / 1000;
-    const time_t time_s = time_ms / 1000;
-    int millisec = time_ms - (time_s * 1000);
+    const time_t time_s = epoch_ms / 1000;
+    int millisec = epoch_ms - (time_s * 1000);
     ptm = gmtime_r(&time_s, &tmbuf);
     if (ptm) {
         int len;
@@ -744,6 +743,12 @@  static void format_date_now(char *buf, int size)
     }
 }
 
+static void format_date_now(char *buf, int size)
+{
+    int64_t time_us = av_gettime();
+    format_date( time_us / 1000, buf, size);
+}
+
 static int write_adaptation_set(AVFormatContext *s, AVIOContext *out, int as_index,
                                 int final)
 {
@@ -1712,10 +1717,16 @@  static int dash_write_packet(AVFormatContext *s, AVPacket *pkt)
     os->last_pts = pkt->pts;
 
     if (!c->availability_start_time[0]) {
-        int64_t start_time_us = av_gettime();
-        c->start_time_s = start_time_us / 1000000;
-        format_date_now(c->availability_start_time,
+        if (c->start_time_ms) {
+            c->start_time_s = c->start_time_ms / 1000;
+            format_date(c->start_time_ms, c->availability_start_time,
                         sizeof(c->availability_start_time));
+        } else {
+            int64_t start_time_us = av_gettime();
+            c->start_time_s = start_time_us / 1000000;
+            format_date_now(c->availability_start_time,
+                            sizeof(c->availability_start_time));
+        }
     }
 
     if (!os->availability_time_offset && pkt->duration) {
@@ -1922,6 +1933,7 @@  static const AVOption options[] = {
     { "ignore_io_errors", "Ignore IO errors during open and write. Useful for long-duration runs with network output", OFFSET(ignore_io_errors), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, E },
     { "lhls", "Enable Low-latency HLS(Experimental). Adds #EXT-X-PREFETCH tag with current segment's URI", OFFSET(lhls), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, E },
     { "master_m3u8_publish_rate", "Publish master playlist every after this many segment intervals", OFFSET(master_publish_rate), AV_OPT_TYPE_INT, {.i64 = 0}, 0, UINT_MAX, E},
+    { "start_time_ms", "stream start time in epoch milliseconds", OFFSET(start_time_ms), AV_OPT_TYPE_INT64, {.i64 = 0}, 0, UINT64_MAX, E, "ms"},
     { NULL },
 };