| Message ID | 20200218161336.1804-3-jamrial@gmail.com |
|---|---|
| State | Superseded |
| Headers | show |
| Series | [FFmpeg-devel,1/3] avformat/dashenc: write the styp box when the first frame of a segment is ready | expand |
| Context | Check | Description |
|---|---|---|
| andriy/ffmpeg-patchwork | success | Make fate finished |
On 2/18/20 9:43 PM, James Almer wrote: > Signed-off-by: James Almer <jamrial@gmail.com> > --- > libavformat/dashenc.c | 5 +++++ > 1 file changed, 5 insertions(+) > > diff --git a/libavformat/dashenc.c b/libavformat/dashenc.c > index b910cc22d0..045d2f4df6 100644 > --- a/libavformat/dashenc.c > +++ b/libavformat/dashenc.c > @@ -1395,6 +1395,11 @@ static int dash_init(AVFormatContext *s) > c->frag_type = FRAG_TYPE_EVERY_FRAME; > } > > + if (c->ldash && !c->write_prft) { > + av_log(s, AV_LOG_INFO, "Enabling Producer Reference Time element for Low Latency mode\n"); > + c->write_prft = 1; > + } > + PRFT elements has a significant bitrate overhead, especially in streaming mode when each frame is a moof fragment. In terms of percentage of stream's bitrate this overhead will be a significant % for lower bitrate streams(such as audio streams). For any application which does not need PRFT this is an unnecessary wastage of bits. Hence, I would advise against enabling PRFT without user control. > if (c->write_prft && !c->utc_timing_url) { > av_log(s, AV_LOG_WARNING, "Producer Reference Time element option will be ignored as utc_timing_url is not set\n"); > c->write_prft = 0;
Am 19.02.20 um 06:18 schrieb Jeyapal, Karthick: > > On 2/18/20 9:43 PM, James Almer wrote: >> Signed-off-by: James Almer <jamrial@gmail.com> >> --- >> libavformat/dashenc.c | 5 +++++ >> 1 file changed, 5 insertions(+) >> >> diff --git a/libavformat/dashenc.c b/libavformat/dashenc.c >> index b910cc22d0..045d2f4df6 100644 >> --- a/libavformat/dashenc.c >> +++ b/libavformat/dashenc.c >> @@ -1395,6 +1395,11 @@ static int dash_init(AVFormatContext *s) >> c->frag_type = FRAG_TYPE_EVERY_FRAME; >> } >> >> + if (c->ldash && !c->write_prft) { >> + av_log(s, AV_LOG_INFO, "Enabling Producer Reference Time element for Low Latency mode\n"); >> + c->write_prft = 1; >> + } >> + > PRFT elements has a significant bitrate overhead, especially in streaming mode when each frame is a moof fragment. > In terms of percentage of stream's bitrate this overhead will be a significant % for lower bitrate streams(such as audio streams). > For any application which does not need PRFT this is an unnecessary wastage of bits. > Hence, I would advise against enabling PRFT without user control. Latest to-become spec for low latency mode declares it mandatory [1]. I see your point, though. What significance would this actually have, can you provide some numbers / examples? If that turns out to be actually significant, I don't know if we would prefer an override option to disable it and produce non-conformant manifests or live with the overhead. -Thilo [1] https://dashif.org/docs/DASH-IF-IOP-CR-Low-Latency-Live-Community-Review-Dec-2019.pdf
On 2/19/20 4:21 PM, Thilo Borgmann wrote: > Am 19.02.20 um 06:18 schrieb Jeyapal, Karthick: >> >> On 2/18/20 9:43 PM, James Almer wrote: >>> Signed-off-by: James Almer <jamrial@gmail.com> >>> --- >>> libavformat/dashenc.c | 5 +++++ >>> 1 file changed, 5 insertions(+) >>> >>> diff --git a/libavformat/dashenc.c b/libavformat/dashenc.c >>> index b910cc22d0..045d2f4df6 100644 >>> --- a/libavformat/dashenc.c >>> +++ b/libavformat/dashenc.c >>> @@ -1395,6 +1395,11 @@ static int dash_init(AVFormatContext *s) >>> c->frag_type = FRAG_TYPE_EVERY_FRAME; >>> } >>> >>> + if (c->ldash && !c->write_prft) { >>> + av_log(s, AV_LOG_INFO, "Enabling Producer Reference Time element for Low Latency mode\n"); >>> + c->write_prft = 1; >>> + } >>> + >> PRFT elements has a significant bitrate overhead, especially in streaming mode when each frame is a moof fragment. >> In terms of percentage of stream's bitrate this overhead will be a significant % for lower bitrate streams(such as audio streams). >> For any application which does not need PRFT this is an unnecessary wastage of bits. >> Hence, I would advise against enabling PRFT without user control. > > Latest to-become spec for low latency mode declares it mandatory [1]. I see. Now I understand the motive behind this change. > > I see your point, though. What significance would this actually have, can you provide some numbers / examples? Sorry. I worked on this bitrate overhead optimizations around a year back. Hence, I don’t have the numbers with and without PRFT handy. But I do have the final overhead numbers (without PRFT) for an audio stream. CMAF Muxer overhead (for an AAC-LC codec) by Sampling frequency 16000 Hz - 14 Kbps 24000 Hz - 20 Kbps 32000 Hz - 28 Kbps 48000 Hz - 40 Kbps At 48KHz, the overhead due to CMAF was 40 Kbps which was significant by itself. My random guess is that PRFT would add another 10Kbps - 20Kbps. But I could be wrong here, as I don’t remember exactly. > If that turns out to be actually significant, I don't know if we would prefer an override option to disable it and produce non-conformant manifests or live with the overhead. Yes. Having an option to control this behavior would be useful. > > -Thilo > > [1] https://dashif.org/docs/DASH-IF-IOP-CR-Low-Latency-Live-Community-Review-Dec-2019.pdf > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
On 2/19/2020 8:50 AM, Jeyapal, Karthick wrote: > > On 2/19/20 4:21 PM, Thilo Borgmann wrote: >> Am 19.02.20 um 06:18 schrieb Jeyapal, Karthick: >>> >>> On 2/18/20 9:43 PM, James Almer wrote: >>>> Signed-off-by: James Almer <jamrial@gmail.com> >>>> --- >>>> libavformat/dashenc.c | 5 +++++ >>>> 1 file changed, 5 insertions(+) >>>> >>>> diff --git a/libavformat/dashenc.c b/libavformat/dashenc.c >>>> index b910cc22d0..045d2f4df6 100644 >>>> --- a/libavformat/dashenc.c >>>> +++ b/libavformat/dashenc.c >>>> @@ -1395,6 +1395,11 @@ static int dash_init(AVFormatContext *s) >>>> c->frag_type = FRAG_TYPE_EVERY_FRAME; >>>> } >>>> >>>> + if (c->ldash && !c->write_prft) { >>>> + av_log(s, AV_LOG_INFO, "Enabling Producer Reference Time element for Low Latency mode\n"); >>>> + c->write_prft = 1; >>>> + } >>>> + >>> PRFT elements has a significant bitrate overhead, especially in streaming mode when each frame is a moof fragment. >>> In terms of percentage of stream's bitrate this overhead will be a significant % for lower bitrate streams(such as audio streams). >>> For any application which does not need PRFT this is an unnecessary wastage of bits. >>> Hence, I would advise against enabling PRFT without user control. >> >> Latest to-become spec for low latency mode declares it mandatory [1]. > I see. Now I understand the motive behind this change. >> >> I see your point, though. What significance would this actually have, can you provide some numbers / examples? > Sorry. I worked on this bitrate overhead optimizations around a year back. Hence, I don’t have the numbers with and without PRFT handy. > But I do have the final overhead numbers (without PRFT) for an audio stream. > CMAF Muxer overhead (for an AAC-LC codec) by Sampling frequency > 16000 Hz - 14 Kbps > 24000 Hz - 20 Kbps > 32000 Hz - 28 Kbps > 48000 Hz - 40 Kbps > > At 48KHz, the overhead due to CMAF was 40 Kbps which was significant by itself. > My random guess is that PRFT would add another 10Kbps - 20Kbps. But I could be wrong here, as I don’t remember exactly. prft is a 32 byte box per dash segment, and segment duration can be configured. A 1 second long segment for a 96kbps 44kHz audio stream with a single moof/mdat pair inside is about 12kb. 32 bytes aren't going to affect it. The real overhead is in the CMAF fragmentation/segmentation. Each moof box can be in the hundreds of bytes depending on frame count. The more moof/mdat pairs (AKA CMAF Chunks) are used, the bigger the overhead. Before my recent changes the dash muxer would always make one per frame in streaming mode, which was excessive, especially for audio. But now you can customize it in various ways. > >> If that turns out to be actually significant, I don't know if we would prefer an override option to disable it and produce non-conformant manifests or live with the overhead. > Yes. Having an option to control this behavior would be useful. >> >> -Thilo >> >> [1] https://dashif.org/docs/DASH-IF-IOP-CR-Low-Latency-Live-Community-Review-Dec-2019.pdf >> _______________________________________________ >> ffmpeg-devel mailing list >> ffmpeg-devel@ffmpeg.org >> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel >> >> To unsubscribe, visit link above, or email >> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >
Am 19.02.20 um 14:35 schrieb James Almer: > On 2/19/2020 8:50 AM, Jeyapal, Karthick wrote: >> >> On 2/19/20 4:21 PM, Thilo Borgmann wrote: >>> Am 19.02.20 um 06:18 schrieb Jeyapal, Karthick: >>>> >>>> On 2/18/20 9:43 PM, James Almer wrote: >>>>> Signed-off-by: James Almer <jamrial@gmail.com> >>>>> --- >>>>> libavformat/dashenc.c | 5 +++++ >>>>> 1 file changed, 5 insertions(+) >>>>> >>>>> diff --git a/libavformat/dashenc.c b/libavformat/dashenc.c >>>>> index b910cc22d0..045d2f4df6 100644 >>>>> --- a/libavformat/dashenc.c >>>>> +++ b/libavformat/dashenc.c >>>>> @@ -1395,6 +1395,11 @@ static int dash_init(AVFormatContext *s) >>>>> c->frag_type = FRAG_TYPE_EVERY_FRAME; >>>>> } >>>>> >>>>> + if (c->ldash && !c->write_prft) { >>>>> + av_log(s, AV_LOG_INFO, "Enabling Producer Reference Time element for Low Latency mode\n"); >>>>> + c->write_prft = 1; >>>>> + } >>>>> + >>>> PRFT elements has a significant bitrate overhead, especially in streaming mode when each frame is a moof fragment. >>>> In terms of percentage of stream's bitrate this overhead will be a significant % for lower bitrate streams(such as audio streams). >>>> For any application which does not need PRFT this is an unnecessary wastage of bits. >>>> Hence, I would advise against enabling PRFT without user control. >>> >>> Latest to-become spec for low latency mode declares it mandatory [1]. >> I see. Now I understand the motive behind this change. >>> >>> I see your point, though. What significance would this actually have, can you provide some numbers / examples? >> Sorry. I worked on this bitrate overhead optimizations around a year back. Hence, I don’t have the numbers with and without PRFT handy. >> But I do have the final overhead numbers (without PRFT) for an audio stream. >> CMAF Muxer overhead (for an AAC-LC codec) by Sampling frequency >> 16000 Hz - 14 Kbps >> 24000 Hz - 20 Kbps >> 32000 Hz - 28 Kbps >> 48000 Hz - 40 Kbps >> >> At 48KHz, the overhead due to CMAF was 40 Kbps which was significant by itself. >> My random guess is that PRFT would add another 10Kbps - 20Kbps. But I could be wrong here, as I don’t remember exactly. > > prft is a 32 byte box per dash segment, and segment duration can be > configured. A 1 second long segment for a 96kbps 44kHz audio stream with > a single moof/mdat pair inside is about 12kb. 32 bytes aren't going to > affect it. > > The real overhead is in the CMAF fragmentation/segmentation. Each moof > box can be in the hundreds of bytes depending on frame count. The more > moof/mdat pairs (AKA CMAF Chunks) are used, the bigger the overhead. > Before my recent changes the dash muxer would always make one per frame > in streaming mode, which was excessive, especially for audio. But now > you can customize it in various ways. > >> >>> If that turns out to be actually significant, I don't know if we would prefer an override option to disable it and produce non-conformant manifests or live with the overhead. >> Yes. Having an option to control this behavior would be useful. The overhead being 32 bytes per segment seems negligible then. 3/3 LGTM -Thilo
On 2/19/20 7:05 PM, James Almer wrote: > On 2/19/2020 8:50 AM, Jeyapal, Karthick wrote: >> >> On 2/19/20 4:21 PM, Thilo Borgmann wrote: >>> Am 19.02.20 um 06:18 schrieb Jeyapal, Karthick: >>>> >>>> On 2/18/20 9:43 PM, James Almer wrote: >>>>> Signed-off-by: James Almer <jamrial@gmail.com> >>>>> --- >>>>> libavformat/dashenc.c | 5 +++++ >>>>> 1 file changed, 5 insertions(+) >>>>> >>>>> diff --git a/libavformat/dashenc.c b/libavformat/dashenc.c >>>>> index b910cc22d0..045d2f4df6 100644 >>>>> --- a/libavformat/dashenc.c >>>>> +++ b/libavformat/dashenc.c >>>>> @@ -1395,6 +1395,11 @@ static int dash_init(AVFormatContext *s) >>>>> c->frag_type = FRAG_TYPE_EVERY_FRAME; >>>>> } >>>>> >>>>> + if (c->ldash && !c->write_prft) { >>>>> + av_log(s, AV_LOG_INFO, "Enabling Producer Reference Time element for Low Latency mode\n"); >>>>> + c->write_prft = 1; >>>>> + } >>>>> + >>>> PRFT elements has a significant bitrate overhead, especially in streaming mode when each frame is a moof fragment. >>>> In terms of percentage of stream's bitrate this overhead will be a significant % for lower bitrate streams(such as audio streams). >>>> For any application which does not need PRFT this is an unnecessary wastage of bits. >>>> Hence, I would advise against enabling PRFT without user control. >>> >>> Latest to-become spec for low latency mode declares it mandatory [1]. >> I see. Now I understand the motive behind this change. >>> >>> I see your point, though. What significance would this actually have, can you provide some numbers / examples? >> Sorry. I worked on this bitrate overhead optimizations around a year back. Hence, I don’t have the numbers with and without PRFT handy. >> But I do have the final overhead numbers (without PRFT) for an audio stream. >> CMAF Muxer overhead (for an AAC-LC codec) by Sampling frequency >> 16000 Hz - 14 Kbps >> 24000 Hz - 20 Kbps >> 32000 Hz - 28 Kbps >> 48000 Hz - 40 Kbps >> >> At 48KHz, the overhead due to CMAF was 40 Kbps which was significant by itself. >> My random guess is that PRFT would add another 10Kbps - 20Kbps. But I could be wrong here, as I don’t remember exactly. > > prft is a 32 byte box per dash segment, and segment duration can be > configured. A 1 second long segment for a 96kbps 44kHz audio stream with > a single moof/mdat pair inside is about 12kb. 32 bytes aren't going to > affect it. Thanks for your clarification. If the prft is created only once per segment, then it is not a big overhead. I had encountered a case where pfrt was getting created once per fragment, and hence was worried. > > The real overhead is in the CMAF fragmentation/segmentation. Each moof > box can be in the hundreds of bytes depending on frame count. The more > moof/mdat pairs (AKA CMAF Chunks) are used, the bigger the overhead. > Before my recent changes the dash muxer would always make one per frame > in streaming mode, which was excessive, especially for audio. But now > you can customize it in various ways. > >> >>> If that turns out to be actually significant, I don't know if we would prefer an override option to disable it and produce non-conformant manifests or live with the overhead. >> Yes. Having an option to control this behavior would be useful. >>> >>> -Thilo >>> >>> [1] https://dashif.org/docs/DASH-IF-IOP-CR-Low-Latency-Live-Community-Review-Dec-2019.pdf >>> _______________________________________________ >>> ffmpeg-devel mailing list >>> ffmpeg-devel@ffmpeg.org >>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel >>> >>> To unsubscribe, visit link above, or email >>> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >> >> _______________________________________________ >> ffmpeg-devel mailing list >> ffmpeg-devel@ffmpeg.org >> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel >> >> To unsubscribe, visit link above, or email >> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >> > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
On 2/19/2020 9:33 PM, Jeyapal, Karthick wrote: > > On 2/19/20 7:05 PM, James Almer wrote: >> On 2/19/2020 8:50 AM, Jeyapal, Karthick wrote: >>> >>> On 2/19/20 4:21 PM, Thilo Borgmann wrote: >>>> Am 19.02.20 um 06:18 schrieb Jeyapal, Karthick: >>>>> >>>>> On 2/18/20 9:43 PM, James Almer wrote: >>>>>> Signed-off-by: James Almer <jamrial@gmail.com> >>>>>> --- >>>>>> libavformat/dashenc.c | 5 +++++ >>>>>> 1 file changed, 5 insertions(+) >>>>>> >>>>>> diff --git a/libavformat/dashenc.c b/libavformat/dashenc.c >>>>>> index b910cc22d0..045d2f4df6 100644 >>>>>> --- a/libavformat/dashenc.c >>>>>> +++ b/libavformat/dashenc.c >>>>>> @@ -1395,6 +1395,11 @@ static int dash_init(AVFormatContext *s) >>>>>> c->frag_type = FRAG_TYPE_EVERY_FRAME; >>>>>> } >>>>>> >>>>>> + if (c->ldash && !c->write_prft) { >>>>>> + av_log(s, AV_LOG_INFO, "Enabling Producer Reference Time element for Low Latency mode\n"); >>>>>> + c->write_prft = 1; >>>>>> + } >>>>>> + >>>>> PRFT elements has a significant bitrate overhead, especially in streaming mode when each frame is a moof fragment. >>>>> In terms of percentage of stream's bitrate this overhead will be a significant % for lower bitrate streams(such as audio streams). >>>>> For any application which does not need PRFT this is an unnecessary wastage of bits. >>>>> Hence, I would advise against enabling PRFT without user control. >>>> >>>> Latest to-become spec for low latency mode declares it mandatory [1]. >>> I see. Now I understand the motive behind this change. >>>> >>>> I see your point, though. What significance would this actually have, can you provide some numbers / examples? >>> Sorry. I worked on this bitrate overhead optimizations around a year back. Hence, I don’t have the numbers with and without PRFT handy. >>> But I do have the final overhead numbers (without PRFT) for an audio stream. >>> CMAF Muxer overhead (for an AAC-LC codec) by Sampling frequency >>> 16000 Hz - 14 Kbps >>> 24000 Hz - 20 Kbps >>> 32000 Hz - 28 Kbps >>> 48000 Hz - 40 Kbps >>> >>> At 48KHz, the overhead due to CMAF was 40 Kbps which was significant by itself. >>> My random guess is that PRFT would add another 10Kbps - 20Kbps. But I could be wrong here, as I don’t remember exactly. >> >> prft is a 32 byte box per dash segment, and segment duration can be >> configured. A 1 second long segment for a 96kbps 44kHz audio stream with >> a single moof/mdat pair inside is about 12kb. 32 bytes aren't going to >> affect it. > Thanks for your clarification. If the prft is created only once per segment, then it is not a big overhead. > I had encountered a case where pfrt was getting created once per fragment, and hence was worried. Actually, you're right, it's one per fragment. My mistake. But you can control both fragment count per segment and frame count per fragment in the dash muxer now, and for low latency dash one fragment per segment of about 1 second each is recommended for audio streams. >> >> The real overhead is in the CMAF fragmentation/segmentation. Each moof >> box can be in the hundreds of bytes depending on frame count. The more >> moof/mdat pairs (AKA CMAF Chunks) are used, the bigger the overhead. >> Before my recent changes the dash muxer would always make one per frame >> in streaming mode, which was excessive, especially for audio. But now >> you can customize it in various ways. >> >>> >>>> If that turns out to be actually significant, I don't know if we would prefer an override option to disable it and produce non-conformant manifests or live with the overhead. >>> Yes. Having an option to control this behavior would be useful. >>>> >>>> -Thilo >>>> >>>> [1] https://dashif.org/docs/DASH-IF-IOP-CR-Low-Latency-Live-Community-Review-Dec-2019.pdf >>>> _______________________________________________ >>>> ffmpeg-devel mailing list >>>> ffmpeg-devel@ffmpeg.org >>>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel >>>> >>>> To unsubscribe, visit link above, or email >>>> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >>> >>> _______________________________________________ >>> ffmpeg-devel mailing list >>> ffmpeg-devel@ffmpeg.org >>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel >>> >>> To unsubscribe, visit link above, or email >>> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >>> >> >> _______________________________________________ >> ffmpeg-devel mailing list >> ffmpeg-devel@ffmpeg.org >> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel >> >> To unsubscribe, visit link above, or email >> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >
On 2/20/20 7:19 AM, James Almer wrote: > On 2/19/2020 9:33 PM, Jeyapal, Karthick wrote: >> >> On 2/19/20 7:05 PM, James Almer wrote: >>> On 2/19/2020 8:50 AM, Jeyapal, Karthick wrote: >>>> >>>> On 2/19/20 4:21 PM, Thilo Borgmann wrote: >>>>> Am 19.02.20 um 06:18 schrieb Jeyapal, Karthick: >>>>>> >>>>>> On 2/18/20 9:43 PM, James Almer wrote: >>>>>>> Signed-off-by: James Almer <jamrial@gmail.com> >>>>>>> --- >>>>>>> libavformat/dashenc.c | 5 +++++ >>>>>>> 1 file changed, 5 insertions(+) >>>>>>> >>>>>>> diff --git a/libavformat/dashenc.c b/libavformat/dashenc.c >>>>>>> index b910cc22d0..045d2f4df6 100644 >>>>>>> --- a/libavformat/dashenc.c >>>>>>> +++ b/libavformat/dashenc.c >>>>>>> @@ -1395,6 +1395,11 @@ static int dash_init(AVFormatContext *s) >>>>>>> c->frag_type = FRAG_TYPE_EVERY_FRAME; >>>>>>> } >>>>>>> >>>>>>> + if (c->ldash && !c->write_prft) { >>>>>>> + av_log(s, AV_LOG_INFO, "Enabling Producer Reference Time element for Low Latency mode\n"); >>>>>>> + c->write_prft = 1; >>>>>>> + } >>>>>>> + >>>>>> PRFT elements has a significant bitrate overhead, especially in streaming mode when each frame is a moof fragment. >>>>>> In terms of percentage of stream's bitrate this overhead will be a significant % for lower bitrate streams(such as audio streams). >>>>>> For any application which does not need PRFT this is an unnecessary wastage of bits. >>>>>> Hence, I would advise against enabling PRFT without user control. >>>>> >>>>> Latest to-become spec for low latency mode declares it mandatory [1]. >>>> I see. Now I understand the motive behind this change. >>>>> >>>>> I see your point, though. What significance would this actually have, can you provide some numbers / examples? >>>> Sorry. I worked on this bitrate overhead optimizations around a year back. Hence, I don’t have the numbers with and without PRFT handy. >>>> But I do have the final overhead numbers (without PRFT) for an audio stream. >>>> CMAF Muxer overhead (for an AAC-LC codec) by Sampling frequency >>>> 16000 Hz - 14 Kbps >>>> 24000 Hz - 20 Kbps >>>> 32000 Hz - 28 Kbps >>>> 48000 Hz - 40 Kbps >>>> >>>> At 48KHz, the overhead due to CMAF was 40 Kbps which was significant by itself. >>>> My random guess is that PRFT would add another 10Kbps - 20Kbps. But I could be wrong here, as I don’t remember exactly. >>> >>> prft is a 32 byte box per dash segment, and segment duration can be >>> configured. A 1 second long segment for a 96kbps 44kHz audio stream with >>> a single moof/mdat pair inside is about 12kb. 32 bytes aren't going to >>> affect it. >> Thanks for your clarification. If the prft is created only once per segment, then it is not a big overhead. >> I had encountered a case where pfrt was getting created once per fragment, and hence was worried. > > Actually, you're right, it's one per fragment. My mistake. But you can > control both fragment count per segment and frame count per fragment in > the dash muxer now, and for low latency dash one fragment per segment of > about 1 second each is recommended for audio streams. If that is the case, I would like to point out that not everybody might choose 1 second per fragment. Anyone interested in reduced latency and stability would go for 1 frame per fragment. In our tests(in production environments), 1 frame per fragment provides much smoother and stable behavior at low latency streaming than higher fragment sizes. And in such a case PRFT will add 12Kbps overhead for a 48Khz AAC-LC stream. Hence, I suggest we keep this behavior configurable. > >>> >>> The real overhead is in the CMAF fragmentation/segmentation. Each moof >>> box can be in the hundreds of bytes depending on frame count. The more >>> moof/mdat pairs (AKA CMAF Chunks) are used, the bigger the overhead. >>> Before my recent changes the dash muxer would always make one per frame >>> in streaming mode, which was excessive, especially for audio. But now >>> you can customize it in various ways. >>> >>>> >>>>> If that turns out to be actually significant, I don't know if we would prefer an override option to disable it and produce non-conformant manifests or live with the overhead. >>>> Yes. Having an option to control this behavior would be useful. >>>>> >>>>> -Thilo >>>>> >>>>> [1] https://dashif.org/docs/DASH-IF-IOP-CR-Low-Latency-Live-Community-Review-Dec-2019.pdf >>>>> _______________________________________________ >>>>> ffmpeg-devel mailing list >>>>> ffmpeg-devel@ffmpeg.org >>>>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel >>>>> >>>>> To unsubscribe, visit link above, or email >>>>> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >>>> >>>> _______________________________________________ >>>> ffmpeg-devel mailing list >>>> ffmpeg-devel@ffmpeg.org >>>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel >>>> >>>> To unsubscribe, visit link above, or email >>>> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >>>> >>> >>> _______________________________________________ >>> ffmpeg-devel mailing list >>> ffmpeg-devel@ffmpeg.org >>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel >>> >>> To unsubscribe, visit link above, or email >>> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >> >> _______________________________________________ >> ffmpeg-devel mailing list >> ffmpeg-devel@ffmpeg.org >> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel >> >> To unsubscribe, visit link above, or email >> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >> > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
On 2/19/2020 11:24 PM, Jeyapal, Karthick wrote: > > On 2/20/20 7:19 AM, James Almer wrote: >> On 2/19/2020 9:33 PM, Jeyapal, Karthick wrote: >>> >>> On 2/19/20 7:05 PM, James Almer wrote: >>>> On 2/19/2020 8:50 AM, Jeyapal, Karthick wrote: >>>>> >>>>> On 2/19/20 4:21 PM, Thilo Borgmann wrote: >>>>>> Am 19.02.20 um 06:18 schrieb Jeyapal, Karthick: >>>>>>> >>>>>>> On 2/18/20 9:43 PM, James Almer wrote: >>>>>>>> Signed-off-by: James Almer <jamrial@gmail.com> >>>>>>>> --- >>>>>>>> libavformat/dashenc.c | 5 +++++ >>>>>>>> 1 file changed, 5 insertions(+) >>>>>>>> >>>>>>>> diff --git a/libavformat/dashenc.c b/libavformat/dashenc.c >>>>>>>> index b910cc22d0..045d2f4df6 100644 >>>>>>>> --- a/libavformat/dashenc.c >>>>>>>> +++ b/libavformat/dashenc.c >>>>>>>> @@ -1395,6 +1395,11 @@ static int dash_init(AVFormatContext *s) >>>>>>>> c->frag_type = FRAG_TYPE_EVERY_FRAME; >>>>>>>> } >>>>>>>> >>>>>>>> + if (c->ldash && !c->write_prft) { >>>>>>>> + av_log(s, AV_LOG_INFO, "Enabling Producer Reference Time element for Low Latency mode\n"); >>>>>>>> + c->write_prft = 1; >>>>>>>> + } >>>>>>>> + >>>>>>> PRFT elements has a significant bitrate overhead, especially in streaming mode when each frame is a moof fragment. >>>>>>> In terms of percentage of stream's bitrate this overhead will be a significant % for lower bitrate streams(such as audio streams). >>>>>>> For any application which does not need PRFT this is an unnecessary wastage of bits. >>>>>>> Hence, I would advise against enabling PRFT without user control. >>>>>> >>>>>> Latest to-become spec for low latency mode declares it mandatory [1]. >>>>> I see. Now I understand the motive behind this change. >>>>>> >>>>>> I see your point, though. What significance would this actually have, can you provide some numbers / examples? >>>>> Sorry. I worked on this bitrate overhead optimizations around a year back. Hence, I don’t have the numbers with and without PRFT handy. >>>>> But I do have the final overhead numbers (without PRFT) for an audio stream. >>>>> CMAF Muxer overhead (for an AAC-LC codec) by Sampling frequency >>>>> 16000 Hz - 14 Kbps >>>>> 24000 Hz - 20 Kbps >>>>> 32000 Hz - 28 Kbps >>>>> 48000 Hz - 40 Kbps >>>>> >>>>> At 48KHz, the overhead due to CMAF was 40 Kbps which was significant by itself. >>>>> My random guess is that PRFT would add another 10Kbps - 20Kbps. But I could be wrong here, as I don’t remember exactly. >>>> >>>> prft is a 32 byte box per dash segment, and segment duration can be >>>> configured. A 1 second long segment for a 96kbps 44kHz audio stream with >>>> a single moof/mdat pair inside is about 12kb. 32 bytes aren't going to >>>> affect it. >>> Thanks for your clarification. If the prft is created only once per segment, then it is not a big overhead. >>> I had encountered a case where pfrt was getting created once per fragment, and hence was worried. >> >> Actually, you're right, it's one per fragment. My mistake. But you can >> control both fragment count per segment and frame count per fragment in >> the dash muxer now, and for low latency dash one fragment per segment of >> about 1 second each is recommended for audio streams. > If that is the case, I would like to point out that not everybody might choose 1 second per fragment. > Anyone interested in reduced latency and stability would go for 1 frame per fragment. > In our tests(in production environments), 1 frame per fragment provides much smoother and stable behavior at low latency streaming than higher fragment sizes. > And in such a case PRFT will add 12Kbps overhead for a 48Khz AAC-LC stream. Hence, I suggest we keep this behavior configurable. One audio frame per fragment is a considerable overhead by itself just by the excess of moof atoms. But in any case, i guess i can skip this patch for the time being, while i look for a good way to for example configure prft in a per adaptation set basis, or just enable it for video streams. >> >>>> >>>> The real overhead is in the CMAF fragmentation/segmentation. Each moof >>>> box can be in the hundreds of bytes depending on frame count. The more >>>> moof/mdat pairs (AKA CMAF Chunks) are used, the bigger the overhead. >>>> Before my recent changes the dash muxer would always make one per frame >>>> in streaming mode, which was excessive, especially for audio. But now >>>> you can customize it in various ways. >>>> >>>>> >>>>>> If that turns out to be actually significant, I don't know if we would prefer an override option to disable it and produce non-conformant manifests or live with the overhead. >>>>> Yes. Having an option to control this behavior would be useful. >>>>>> >>>>>> -Thilo >>>>>> >>>>>> [1] https://dashif.org/docs/DASH-IF-IOP-CR-Low-Latency-Live-Community-Review-Dec-2019.pdf
Am 20.02.20 um 04:24 schrieb James Almer: > On 2/19/2020 11:24 PM, Jeyapal, Karthick wrote: >> >> On 2/20/20 7:19 AM, James Almer wrote: >>> On 2/19/2020 9:33 PM, Jeyapal, Karthick wrote: >>>> >>>> On 2/19/20 7:05 PM, James Almer wrote: >>>>> On 2/19/2020 8:50 AM, Jeyapal, Karthick wrote: >>>>>> >>>>>> On 2/19/20 4:21 PM, Thilo Borgmann wrote: >>>>>>> Am 19.02.20 um 06:18 schrieb Jeyapal, Karthick: >>>>>>>> >>>>>>>> On 2/18/20 9:43 PM, James Almer wrote: >>>>>>>>> Signed-off-by: James Almer <jamrial@gmail.com> >>>>>>>>> --- >>>>>>>>> libavformat/dashenc.c | 5 +++++ >>>>>>>>> 1 file changed, 5 insertions(+) >>>>>>>>> >>>>>>>>> diff --git a/libavformat/dashenc.c b/libavformat/dashenc.c >>>>>>>>> index b910cc22d0..045d2f4df6 100644 >>>>>>>>> --- a/libavformat/dashenc.c >>>>>>>>> +++ b/libavformat/dashenc.c >>>>>>>>> @@ -1395,6 +1395,11 @@ static int dash_init(AVFormatContext *s) >>>>>>>>> c->frag_type = FRAG_TYPE_EVERY_FRAME; >>>>>>>>> } >>>>>>>>> >>>>>>>>> + if (c->ldash && !c->write_prft) { >>>>>>>>> + av_log(s, AV_LOG_INFO, "Enabling Producer Reference Time element for Low Latency mode\n"); >>>>>>>>> + c->write_prft = 1; >>>>>>>>> + } >>>>>>>>> + >>>>>>>> PRFT elements has a significant bitrate overhead, especially in streaming mode when each frame is a moof fragment. >>>>>>>> In terms of percentage of stream's bitrate this overhead will be a significant % for lower bitrate streams(such as audio streams). >>>>>>>> For any application which does not need PRFT this is an unnecessary wastage of bits. >>>>>>>> Hence, I would advise against enabling PRFT without user control. >>>>>>> >>>>>>> Latest to-become spec for low latency mode declares it mandatory [1]. >>>>>> I see. Now I understand the motive behind this change. >>>>>>> >>>>>>> I see your point, though. What significance would this actually have, can you provide some numbers / examples? >>>>>> Sorry. I worked on this bitrate overhead optimizations around a year back. Hence, I don’t have the numbers with and without PRFT handy. >>>>>> But I do have the final overhead numbers (without PRFT) for an audio stream. >>>>>> CMAF Muxer overhead (for an AAC-LC codec) by Sampling frequency >>>>>> 16000 Hz - 14 Kbps >>>>>> 24000 Hz - 20 Kbps >>>>>> 32000 Hz - 28 Kbps >>>>>> 48000 Hz - 40 Kbps >>>>>> >>>>>> At 48KHz, the overhead due to CMAF was 40 Kbps which was significant by itself. >>>>>> My random guess is that PRFT would add another 10Kbps - 20Kbps. But I could be wrong here, as I don’t remember exactly. >>>>> >>>>> prft is a 32 byte box per dash segment, and segment duration can be >>>>> configured. A 1 second long segment for a 96kbps 44kHz audio stream with >>>>> a single moof/mdat pair inside is about 12kb. 32 bytes aren't going to >>>>> affect it. >>>> Thanks for your clarification. If the prft is created only once per segment, then it is not a big overhead. >>>> I had encountered a case where pfrt was getting created once per fragment, and hence was worried. >>> >>> Actually, you're right, it's one per fragment. My mistake. But you can >>> control both fragment count per segment and frame count per fragment in >>> the dash muxer now, and for low latency dash one fragment per segment of >>> about 1 second each is recommended for audio streams. >> If that is the case, I would like to point out that not everybody might choose 1 second per fragment. >> Anyone interested in reduced latency and stability would go for 1 frame per fragment. >> In our tests(in production environments), 1 frame per fragment provides much smoother and stable behavior at low latency streaming than higher fragment sizes. >> And in such a case PRFT will add 12Kbps overhead for a 48Khz AAC-LC stream. Hence, I suggest we keep this behavior configurable. > > One audio frame per fragment is a considerable overhead by itself just > by the excess of moof atoms. > But in any case, i guess i can skip this > patch for the time being, while i look for a good way to for example > configure prft in a per adaptation set basis, or just enable it for > video streams. I don't think so. We should not fail the spec on default because of one possible use case. Since we seem to have found that this can be a significant overhead, we should add an override option to violate the spec (and skip prft) - but in that case also print a warning. -Thilo > >>> >>>>> >>>>> The real overhead is in the CMAF fragmentation/segmentation. Each moof >>>>> box can be in the hundreds of bytes depending on frame count. The more >>>>> moof/mdat pairs (AKA CMAF Chunks) are used, the bigger the overhead. >>>>> Before my recent changes the dash muxer would always make one per frame >>>>> in streaming mode, which was excessive, especially for audio. But now >>>>> you can customize it in various ways. >>>>> >>>>>> >>>>>>> If that turns out to be actually significant, I don't know if we would prefer an override option to disable it and produce non-conformant manifests or live with the overhead. >>>>>> Yes. Having an option to control this behavior would be useful. >>>>>>> >>>>>>> -Thilo >>>>>>> >>>>>>> [1] https://dashif.org/docs/DASH-IF-IOP-CR-Low-Latency-Live-Community-Review-Dec-2019.pdf > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >
On 2/20/2020 7:38 AM, Thilo Borgmann wrote: > Am 20.02.20 um 04:24 schrieb James Almer: >> On 2/19/2020 11:24 PM, Jeyapal, Karthick wrote: >>> >>> On 2/20/20 7:19 AM, James Almer wrote: >>>> On 2/19/2020 9:33 PM, Jeyapal, Karthick wrote: >>>>> >>>>> On 2/19/20 7:05 PM, James Almer wrote: >>>>>> On 2/19/2020 8:50 AM, Jeyapal, Karthick wrote: >>>>>>> >>>>>>> On 2/19/20 4:21 PM, Thilo Borgmann wrote: >>>>>>>> Am 19.02.20 um 06:18 schrieb Jeyapal, Karthick: >>>>>>>>> >>>>>>>>> On 2/18/20 9:43 PM, James Almer wrote: >>>>>>>>>> Signed-off-by: James Almer <jamrial@gmail.com> >>>>>>>>>> --- >>>>>>>>>> libavformat/dashenc.c | 5 +++++ >>>>>>>>>> 1 file changed, 5 insertions(+) >>>>>>>>>> >>>>>>>>>> diff --git a/libavformat/dashenc.c b/libavformat/dashenc.c >>>>>>>>>> index b910cc22d0..045d2f4df6 100644 >>>>>>>>>> --- a/libavformat/dashenc.c >>>>>>>>>> +++ b/libavformat/dashenc.c >>>>>>>>>> @@ -1395,6 +1395,11 @@ static int dash_init(AVFormatContext *s) >>>>>>>>>> c->frag_type = FRAG_TYPE_EVERY_FRAME; >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> + if (c->ldash && !c->write_prft) { >>>>>>>>>> + av_log(s, AV_LOG_INFO, "Enabling Producer Reference Time element for Low Latency mode\n"); >>>>>>>>>> + c->write_prft = 1; >>>>>>>>>> + } >>>>>>>>>> + >>>>>>>>> PRFT elements has a significant bitrate overhead, especially in streaming mode when each frame is a moof fragment. >>>>>>>>> In terms of percentage of stream's bitrate this overhead will be a significant % for lower bitrate streams(such as audio streams). >>>>>>>>> For any application which does not need PRFT this is an unnecessary wastage of bits. >>>>>>>>> Hence, I would advise against enabling PRFT without user control. >>>>>>>> >>>>>>>> Latest to-become spec for low latency mode declares it mandatory [1]. >>>>>>> I see. Now I understand the motive behind this change. >>>>>>>> >>>>>>>> I see your point, though. What significance would this actually have, can you provide some numbers / examples? >>>>>>> Sorry. I worked on this bitrate overhead optimizations around a year back. Hence, I don’t have the numbers with and without PRFT handy. >>>>>>> But I do have the final overhead numbers (without PRFT) for an audio stream. >>>>>>> CMAF Muxer overhead (for an AAC-LC codec) by Sampling frequency >>>>>>> 16000 Hz - 14 Kbps >>>>>>> 24000 Hz - 20 Kbps >>>>>>> 32000 Hz - 28 Kbps >>>>>>> 48000 Hz - 40 Kbps >>>>>>> >>>>>>> At 48KHz, the overhead due to CMAF was 40 Kbps which was significant by itself. >>>>>>> My random guess is that PRFT would add another 10Kbps - 20Kbps. But I could be wrong here, as I don’t remember exactly. >>>>>> >>>>>> prft is a 32 byte box per dash segment, and segment duration can be >>>>>> configured. A 1 second long segment for a 96kbps 44kHz audio stream with >>>>>> a single moof/mdat pair inside is about 12kb. 32 bytes aren't going to >>>>>> affect it. >>>>> Thanks for your clarification. If the prft is created only once per segment, then it is not a big overhead. >>>>> I had encountered a case where pfrt was getting created once per fragment, and hence was worried. >>>> >>>> Actually, you're right, it's one per fragment. My mistake. But you can >>>> control both fragment count per segment and frame count per fragment in >>>> the dash muxer now, and for low latency dash one fragment per segment of >>>> about 1 second each is recommended for audio streams. >>> If that is the case, I would like to point out that not everybody might choose 1 second per fragment. >>> Anyone interested in reduced latency and stability would go for 1 frame per fragment. >>> In our tests(in production environments), 1 frame per fragment provides much smoother and stable behavior at low latency streaming than higher fragment sizes. >>> And in such a case PRFT will add 12Kbps overhead for a 48Khz AAC-LC stream. Hence, I suggest we keep this behavior configurable. >> >> One audio frame per fragment is a considerable overhead by itself just >> by the excess of moof atoms. > >> But in any case, i guess i can skip this >> patch for the time being, while i look for a good way to for example >> configure prft in a per adaptation set basis, or just enable it for >> video streams. > > I don't think so. We should not fail the spec on default because of one possible use case. > > Since we seem to have found that this can be a significant overhead, we should add an override option to violate the spec (and skip prft) - but in that case also print a warning. We're already printing a warning when prft isn't written in ldash mode. This patch merely attempts to enable it, and can fail to do it if no utc_timing url was provided. Also, my suggestion above to try a way to only enable prft for some adaptation sets, or only for video, is a good solution since the spec only requires at least one stream to have a prft, and not all of them. Also, how about aborting if no prft is written in ldash mode when strict_std_compliance is >= FF_COMPLIANCE_STRICT? > > -Thilo > > >> >>>> >>>>>> >>>>>> The real overhead is in the CMAF fragmentation/segmentation. Each moof >>>>>> box can be in the hundreds of bytes depending on frame count. The more >>>>>> moof/mdat pairs (AKA CMAF Chunks) are used, the bigger the overhead. >>>>>> Before my recent changes the dash muxer would always make one per frame >>>>>> in streaming mode, which was excessive, especially for audio. But now >>>>>> you can customize it in various ways. >>>>>> >>>>>>> >>>>>>>> If that turns out to be actually significant, I don't know if we would prefer an override option to disable it and produce non-conformant manifests or live with the overhead. >>>>>>> Yes. Having an option to control this behavior would be useful. >>>>>>>> >>>>>>>> -Thilo >>>>>>>> >>>>>>>> [1] https://dashif.org/docs/DASH-IF-IOP-CR-Low-Latency-Live-Community-Review-Dec-2019.pdf >> >> _______________________________________________ >> ffmpeg-devel mailing list >> ffmpeg-devel@ffmpeg.org >> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel >> >> To unsubscribe, visit link above, or email >> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >> > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >
diff --git a/libavformat/dashenc.c b/libavformat/dashenc.c index b910cc22d0..045d2f4df6 100644 --- a/libavformat/dashenc.c +++ b/libavformat/dashenc.c @@ -1395,6 +1395,11 @@ static int dash_init(AVFormatContext *s) c->frag_type = FRAG_TYPE_EVERY_FRAME; } + if (c->ldash && !c->write_prft) { + av_log(s, AV_LOG_INFO, "Enabling Producer Reference Time element for Low Latency mode\n"); + c->write_prft = 1; + } + if (c->write_prft && !c->utc_timing_url) { av_log(s, AV_LOG_WARNING, "Producer Reference Time element option will be ignored as utc_timing_url is not set\n"); c->write_prft = 0;
Signed-off-by: James Almer <jamrial@gmail.com> --- libavformat/dashenc.c | 5 +++++ 1 file changed, 5 insertions(+)