| Message ID | 20210608190343.74970-1-kevin@aiera.com |
|---|---|
| State | Accepted |
| Commit | 7d1464721ebf5c90308e48cb81ad52dbada47c18 |
| Headers | show |
| Series | [FFmpeg-devel] dashenc: Write out DASH manifest immediately in streaming mode | expand |
| Context | Check | Description |
|---|---|---|
| andriy/x86_make | success | Make finished |
| andriy/x86_make_fate | success | Make fate finished |
| andriy/PPC64_make | success | Make finished |
| andriy/PPC64_make_fate | success | Make fate finished |
On 08.06.2021 21:03, Kevin LaFlamme wrote: > When streaming mode is enabled with fMP4/CMAF for DASH output, the > segment files are available to read by players as soon as the first byte > is written instead of only after the file is fully written. The DASH > manifest currently only gets written when the final write to the segment > file occurs. This means that players cannot stream the first segment > while it is being written. Is this really a good idea? Imagine the files are being served by a http server. The server will stat() the file, and then send that size to the client in the HTTP headers. If now a Client reads that file way early, it will receive an incomplete version of the file. HTTP servers don't typically are capable of streaming files that are actively being written to. Reading a file while it's being written to is also not sensible possible or behaves in potentially unexpected ways, depending on the OS. > When -lhls is enabled with MP4 segments the HLS manifest is written > immediately to advertise the in-flight segments. This change adds the > same behavior for the DASH manifest so players can stream it > immediately.
For streaming mode with fragmented MP4 the intention is to have the client read a partial file since it’s broken up into sequential boxes that are playable independently. This doesn’t change the segment files themselves or how they are written, it just writes a full index file ahead of time. Even currently, the manifest gets written after the first segment is written and the player can immediately attempt to start reading the second segment before it’s fully written. This is the same thing that the “-lhls” is doing in the block below, writing out the HLS manifest immediately so that it contains X-EXT-PREFETCH fields for the partial segment files. Currently, if you have a low latency DASH stream with “-ldash” and “-streaming” and the player joins in the middle of the stream everything works and it starts playing from the middle of the current segment. This means even with 10 second segments you can have latency < 1sec of the live edge. However, this doesn’t work for the very first segment because the manifest isn’t available to the player yet. I have a low-latency player where I’m seeing this issue right now and this addresses the problem, and seems consistent with the “-lhls” handling below, but happy to make changes if there is something else I’m missing. Kevin LaFlamme Director of Engineering (Front End) 774.265.0382 (m) aiera.com On Jun 8, 2021, 3:10 PM -0400, Timo Rothenpieler <timo@rothenpieler.org>, wrote: > On 08.06.2021 21:03, Kevin LaFlamme wrote: > > When streaming mode is enabled with fMP4/CMAF for DASH output, the > > segment files are available to read by players as soon as the first byte > > is written instead of only after the file is fully written. The DASH > > manifest currently only gets written when the final write to the segment > > file occurs. This means that players cannot stream the first segment > > while it is being written. > > Is this really a good idea? > Imagine the files are being served by a http server. > The server will stat() the file, and then send that size to the client > in the HTTP headers. > If now a Client reads that file way early, it will receive an incomplete > version of the file. HTTP servers don't typically are capable of > streaming files that are actively being written to. > > Reading a file while it's being written to is also not sensible possible > or behaves in potentially unexpected ways, depending on the OS. > > > When -lhls is enabled with MP4 segments the HLS manifest is written > > immediately to advertise the in-flight segments. This change adds the > > same behavior for the DASH manifest so players can stream it > > immediately. > > > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
On 08.06.2021 21:24, Kevin LaFlamme wrote: > For streaming mode with fragmented MP4 the intention is to have the client read a partial file since it’s broken up into sequential boxes that are playable independently. This doesn’t change the segment files themselves or how they are written, it just writes a full index file ahead of time. > > Even currently, the manifest gets written after the first segment is written and the player can immediately attempt to start reading the second segment before it’s fully written. > > This is the same thing that the “-lhls” is doing in the block below, writing out the HLS manifest immediately so that it contains X-EXT-PREFETCH fields for the partial segment files. > > Currently, if you have a low latency DASH stream with “-ldash” and “-streaming” and the player joins in the middle of the stream everything works and it starts playing from the middle of the current segment. This means even with 10 second segments you can have latency < 1sec of the live edge. However, this doesn’t work for the very first segment because the manifest isn’t available to the player yet. > > I have a low-latency player where I’m seeing this issue right now and this addresses the problem, and seems consistent with the “-lhls” handling below, but happy to make changes if there is something else I’m missing. Yes, I'm aware that that's how it's supposed to work in theory. But does that work with any HTTP server? Do they really manage to stream the file while it's being written to? Or does this need special software to serve the file? And in turn, will it break for a user just pointing apache/nginx at such a list?
To serve it in a truly streaming way does require some special purpose server or configuration, but the `@availabilityTimeComplete` field that is written into the manifest is supposed to indicate to the client making the request whether or not the segment file is complete at the time it becomes available. In streaming mode, this is set to false so the client should expect that it can get a partial segment between the available start time and the segment duration. So as each box is written to the file, it should be safe for the HTTP server to send the partial segment file that is currently on disk. And the DASH spec specifically indicates whether or not to expect a partial file for the segment being requested. If the client can’t handle a partial file and is still making an attempt to request it before it’s fully available (which is calculable from the information in the manifest), the client wouldn’t be respecting the DASH spec. Kevin LaFlamme Director of Engineering (Front End) 774.265.0382 (m) aiera.com On Jun 8, 2021, 3:34 PM -0400, Timo Rothenpieler <timo@rothenpieler.org>, wrote: > On 08.06.2021 21:24, Kevin LaFlamme wrote: > > For streaming mode with fragmented MP4 the intention is to have the client read a partial file since it’s broken up into sequential boxes that are playable independently. This doesn’t change the segment files themselves or how they are written, it just writes a full index file ahead of time. > > > > Even currently, the manifest gets written after the first segment is written and the player can immediately attempt to start reading the second segment before it’s fully written. > > > > This is the same thing that the “-lhls” is doing in the block below, writing out the HLS manifest immediately so that it contains X-EXT-PREFETCH fields for the partial segment files. > > > > Currently, if you have a low latency DASH stream with “-ldash” and “-streaming” and the player joins in the middle of the stream everything works and it starts playing from the middle of the current segment. This means even with 10 second segments you can have latency < 1sec of the live edge. However, this doesn’t work for the very first segment because the manifest isn’t available to the player yet. > > > > I have a low-latency player where I’m seeing this issue right now and this addresses the problem, and seems consistent with the “-lhls” handling below, but happy to make changes if there is something else I’m missing. > > Yes, I'm aware that that's how it's supposed to work in theory. > But does that work with any HTTP server? > Do they really manage to stream the file while it's being written to? > Or does this need special software to serve the file? And in turn, will > it break for a user just pointing apache/nginx at such a list? > > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
I just realized there’s probably a more straightforward explanation: The scenario you brought up is already happening for all segments after the first one, and this changes just makes the first one do the same thing. The following scenario happens even without this change: Say you have a stream with "-streaming" and "-use_template" set and a segment duration of 10 seconds. While the first segment is being written there is no manifest, but then the first segment and manifest are written and the segment segment starts to be written. At 15 seconds in a client requests the manifest. It wants to be 2 seconds behind the live edge (13 second seek position) so it calculates which segment name it needs, which is the second one, and makes a request for it even though it's only partially written. And this could currently happen for any segment index > 1. With the proposed change this same thing just now happens on the first segment as well. I'm definitely happy to try a different approach to fix the underlying issue if I'm misunderstanding something or this reasoning is incorrect, but this "seems" right to me from my understanding of the spec and the behaviors I've witnessed so far. Kevin LaFlamme Director of Engineering (Front End) 774.265.0382 (m) aiera.com On Jun 8, 2021, 4:03 PM -0400, Kevin LaFlamme <kevin@aiera.com>, wrote: > To serve it in a truly streaming way does require some special purpose server or configuration, but the `@availabilityTimeComplete` field that is written into the manifest is supposed to indicate to the client making the request whether or not the segment file is complete at the time it becomes available. In streaming mode, this is set to false so the client should expect that it can get a partial segment between the available start time and the segment duration. > > So as each box is written to the file, it should be safe for the HTTP server to send the partial segment file that is currently on disk. And the DASH spec specifically indicates whether or not to expect a partial file for the segment being requested. If the client can’t handle a partial file and is still making an attempt to request it before it’s fully available (which is calculable from the information in the manifest), the client wouldn’t be respecting the DASH spec. > > Kevin LaFlamme > Director of Engineering (Front End) > 774.265.0382 (m) > aiera.com > On Jun 8, 2021, 3:34 PM -0400, Timo Rothenpieler <timo@rothenpieler.org>, wrote: > > On 08.06.2021 21:24, Kevin LaFlamme wrote: > > > For streaming mode with fragmented MP4 the intention is to have the client read a partial file since it’s broken up into sequential boxes that are playable independently. This doesn’t change the segment files themselves or how they are written, it just writes a full index file ahead of time. > > > > > > Even currently, the manifest gets written after the first segment is written and the player can immediately attempt to start reading the second segment before it’s fully written. > > > > > > This is the same thing that the “-lhls” is doing in the block below, writing out the HLS manifest immediately so that it contains X-EXT-PREFETCH fields for the partial segment files. > > > > > > Currently, if you have a low latency DASH stream with “-ldash” and “-streaming” and the player joins in the middle of the stream everything works and it starts playing from the middle of the current segment. This means even with 10 second segments you can have latency < 1sec of the live edge. However, this doesn’t work for the very first segment because the manifest isn’t available to the player yet. > > > > > > I have a low-latency player where I’m seeing this issue right now and this addresses the problem, and seems consistent with the “-lhls” handling below, but happy to make changes if there is something else I’m missing. > > > > Yes, I'm aware that that's how it's supposed to work in theory. > > But does that work with any HTTP server? > > Do they really manage to stream the file while it's being written to? > > Or does this need special software to serve the file? And in turn, will > > it break for a user just pointing apache/nginx at such a list? > > > > > > _______________________________________________ > > ffmpeg-devel mailing list > > ffmpeg-devel@ffmpeg.org > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > > > To unsubscribe, visit link above, or email > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
Does my last explanation make sense or if not could you point me to where this reasoning is incorrect? Kevin LaFlamme Director of Engineering (Front End) 774.265.0382 (m) aiera.com On Jun 8, 2021, 8:52 PM -0400, Kevin LaFlamme <kevin@aiera.com>, wrote: > I just realized there’s probably a more straightforward explanation: > > The scenario you brought up is already happening for all segments after > the first one, and this changes just makes the first one do the same > thing. > > The following scenario happens even without this change: > > Say you have a stream with "-streaming" and "-use_template" set and a > segment duration of 10 seconds. While the first segment is being > written there is no manifest, but then the first segment and manifest > are written and the segment segment starts to be written. At 15 seconds > in a client requests the manifest. It wants to be 2 seconds behind the > live edge (13 second seek position) so it calculates which segment name > it needs, which is the second one, and makes a request for it even > though it's only partially written. And this could currently happen for > any segment index > 1. > > With the proposed change this same thing just now happens on the first > segment as well. > > I'm definitely happy to try a different approach to fix the underlying > issue if I'm misunderstanding something or this reasoning is incorrect, > but this "seems" right to me from my understanding of the spec and the > behaviors I've witnessed so far. > > Kevin LaFlamme > Director of Engineering (Front End) > 774.265.0382 (m) > aiera.com > On Jun 8, 2021, 4:03 PM -0400, Kevin LaFlamme <kevin@aiera.com>, wrote: > > To serve it in a truly streaming way does require some special purpose server or configuration, but the `@availabilityTimeComplete` field that is written into the manifest is supposed to indicate to the client making the request whether or not the segment file is complete at the time it becomes available. In streaming mode, this is set to false so the client should expect that it can get a partial segment between the available start time and the segment duration. > > > > So as each box is written to the file, it should be safe for the HTTP server to send the partial segment file that is currently on disk. And the DASH spec specifically indicates whether or not to expect a partial file for the segment being requested. If the client can’t handle a partial file and is still making an attempt to request it before it’s fully available (which is calculable from the information in the manifest), the client wouldn’t be respecting the DASH spec. > > > > Kevin LaFlamme > > Director of Engineering (Front End) > > 774.265.0382 (m) > > aiera.com > > On Jun 8, 2021, 3:34 PM -0400, Timo Rothenpieler <timo@rothenpieler.org>, wrote: > > > On 08.06.2021 21:24, Kevin LaFlamme wrote: > > > > For streaming mode with fragmented MP4 the intention is to have the client read a partial file since it’s broken up into sequential boxes that are playable independently. This doesn’t change the segment files themselves or how they are written, it just writes a full index file ahead of time. > > > > > > > > Even currently, the manifest gets written after the first segment is written and the player can immediately attempt to start reading the second segment before it’s fully written. > > > > > > > > This is the same thing that the “-lhls” is doing in the block below, writing out the HLS manifest immediately so that it contains X-EXT-PREFETCH fields for the partial segment files. > > > > > > > > Currently, if you have a low latency DASH stream with “-ldash” and “-streaming” and the player joins in the middle of the stream everything works and it starts playing from the middle of the current segment. This means even with 10 second segments you can have latency < 1sec of the live edge. However, this doesn’t work for the very first segment because the manifest isn’t available to the player yet. > > > > > > > > I have a low-latency player where I’m seeing this issue right now and this addresses the problem, and seems consistent with the “-lhls” handling below, but happy to make changes if there is something else I’m missing. > > > > > > Yes, I'm aware that that's how it's supposed to work in theory. > > > But does that work with any HTTP server? > > > Do they really manage to stream the file while it's being written to? > > > Or does this need special software to serve the file? And in turn, will > > > it break for a user just pointing apache/nginx at such a list? > > > > > > > > > _______________________________________________ > > > ffmpeg-devel mailing list > > > ffmpeg-devel@ffmpeg.org > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > > > > > To unsubscribe, visit link above, or email > > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
On 6/11/21, 9:31 PM, "Kevin LaFlamme" <kevin@aiera.com> wrote: >Does my last explanation make sense or if not could you point me to >where this reasoning is incorrect? Your reasoning is perfectly fine. Since I haven't heard any objections to your reply, I have pushed this patch. Thanks. > >Kevin LaFlamme >Director of Engineering (Front End) >774.265.0382 (m) >aiera.com >On Jun 8, 2021, 8:52 PM -0400, Kevin LaFlamme <kevin@aiera.com>, wrote: >> I just realized there’s probably a more straightforward explanation: >> >> The scenario you brought up is already happening for all segments after >> the first one, and this changes just makes the first one do the same >> thing. >> >> The following scenario happens even without this change: >> >> Say you have a stream with "-streaming" and "-use_template" set and a >> segment duration of 10 seconds. While the first segment is being >> written there is no manifest, but then the first segment and manifest >> are written and the segment segment starts to be written. At 15 seconds >> in a client requests the manifest. It wants to be 2 seconds behind the >> live edge (13 second seek position) so it calculates which segment name >> it needs, which is the second one, and makes a request for it even >> though it's only partially written. And this could currently happen for >> any segment index > 1. >> >> With the proposed change this same thing just now happens on the first >> segment as well. >> >> I'm definitely happy to try a different approach to fix the underlying >> issue if I'm misunderstanding something or this reasoning is incorrect, >> but this "seems" right to me from my understanding of the spec and the >> behaviors I've witnessed so far. >> >> Kevin LaFlamme >> Director of Engineering (Front End) >> 774.265.0382 (m) >> aiera.com >> On Jun 8, 2021, 4:03 PM -0400, Kevin LaFlamme <kevin@aiera.com>, wrote: >> > To serve it in a truly streaming way does require some special purpose server or configuration, but >the `@availabilityTimeComplete` field that is written into the manifest is supposed to indicate to the >client making the request whether or not the segment file is complete at the time it becomes available. >In streaming mode, this is set to false so the client should expect that it can get a partial segment >between the available start time and the segment duration. >> > >> > So as each box is written to the file, it should be safe for the HTTP server to send the partial >segment file that is currently on disk. And the DASH spec specifically indicates whether or not to expect >a partial file for the segment being requested. If the client can’t handle a partial file and is still making >an attempt to request it before it’s fully available (which is calculable from the information in the >manifest), the client wouldn’t be respecting the DASH spec. >> > >> > Kevin LaFlamme >> > Director of Engineering (Front End) >> > 774.265.0382 (m) >> > aiera.com >> > On Jun 8, 2021, 3:34 PM -0400, Timo Rothenpieler <timo@rothenpieler.org>, wrote: >> > > On 08.06.2021 21:24, Kevin LaFlamme wrote: >> > > > For streaming mode with fragmented MP4 the intention is to have the client read a partial file >since it’s broken up into sequential boxes that are playable independently. This doesn’t change the >segment files themselves or how they are written, it just writes a full index file ahead of time. >> > > > >> > > > Even currently, the manifest gets written after the first segment is written and the player can >immediately attempt to start reading the second segment before it’s fully written. >> > > > >> > > > This is the same thing that the “-lhls” is doing in the block below, writing out the HLS manifest >immediately so that it contains X-EXT-PREFETCH fields for the partial segment files. >> > > > >> > > > Currently, if you have a low latency DASH stream with “-ldash” and “-streaming” and the player >joins in the middle of the stream everything works and it starts playing from the middle of the current >segment. This means even with 10 second segments you can have latency < 1sec of the live edge. >However, this doesn’t work for the very first segment because the manifest isn’t available to the player >yet. >> > > > >> > > > I have a low-latency player where I’m seeing this issue right now and this addresses the >problem, and seems consistent with the “-lhls” handling below, but happy to make changes if there is >something else I’m missing. >> > > >> > > Yes, I'm aware that that's how it's supposed to work in theory. >> > > But does that work with any HTTP server? >> > > Do they really manage to stream the file while it's being written to? This feature works with most of the popular CDNs. And yes, they do manage a file as it is being written into. >> > > Or does this need special software to serve the file? And in turn, will >> > > it break for a user just pointing apache/nginx at such a list? To start with, -streaming feature in dashenc, will not work with a standard apache/nginx. This feature needs a http server that handles a streaming. This change is just a bugfix to -streaming feature. regards, Karthick >> > > >> > > >> > > _______________________________________________ >> > > ffmpeg-devel mailing list >> > > ffmpeg-devel@ffmpeg.org >> > > >> > > To unsubscribe, visit link above, or email >> > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
diff --git a/libavformat/dashenc.c b/libavformat/dashenc.c index 810ab22416..ea3c76b802 100644 --- a/libavformat/dashenc.c +++ b/libavformat/dashenc.c @@ -2271,6 +2271,14 @@ static int dash_write_packet(AVFormatContext *s, AVPacket *pkt) if (ret < 0) { return handle_io_open_error(s, ret, os->temp_path); } + + // in streaming mode, the segments are available for playing + // before fully written but the manifest is needed so that + // clients and discover the segment filenames. + if (c->streaming && os->segment_type == SEGMENT_TYPE_MP4) { + write_manifest(s, 0); + } + if (c->lhls) { char *prefetch_url = use_rename ? NULL : os->filename; write_hls_media_playlist(os, s, pkt->stream_index, 0, prefetch_url);