diff mbox

[FFmpeg-devel] avformat/hlsenc: Dont write stream info for agroup

Message ID 1516266587-4368-1-git-send-email-redmcg@redmandi.dyndns.org
State New
Headers show

Commit Message

Brendan McGrath Jan. 18, 2018, 9:09 a.m. UTC
When using an 'agroup' within var_stream_map - the audio stream is
being added to the master playlist file as both an audio rendition
and an individual stream (with an AUDIO reference back to itself).

This patch ensures an audio rendition does not also appear within
the stream info list.

What follows is an example of the command to create this issue and
the contents of the master playlist before and after this patch is
applied:

ffmpeg -i in.ts -b:a:0 128k -b:v:0 1800k -b:v:1 1024k -map 0:a \
-map 0:v -map 0:v -f hls -var_stream_map "a:0,agroup:audio_0 "\
"v:0,agroup:audio_0 v:1,agroup:audio_0" -master_pl_name \
tv_hls.m3u8 tv_hls_%v.m3u8

Before:
 #EXTM3U
 #EXT-X-VERSION:3
 #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="group_audio_0",NAME="audio_0",DEFAULT=YES,URI="tv_hls_0.m3u8"
 #EXT-X-STREAM-INF:BANDWIDTH=140800,AUDIO="group_audio_0"
 tv_hls_0.m3u8

 #EXT-X-STREAM-INF:BANDWIDTH=2120800,RESOLUTION=1920x1080,AUDIO="group_audio_0"
 tv_hls_1.m3u8

 #EXT-X-STREAM-INF:BANDWIDTH=1267200,RESOLUTION=1920x1080,AUDIO="group_audio_0"
 tv_hls_2.m3u8

After:
 #EXTM3U
 #EXT-X-VERSION:3
 #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="group_audio_0",NAME="audio_0",DEFAULT=YES,URI="tv_hls_0.m3u8"
 #EXT-X-STREAM-INF:BANDWIDTH=2120800,RESOLUTION=1920x1080,AUDIO="group_audio_0"
 tv_hls_1.m3u8

 #EXT-X-STREAM-INF:BANDWIDTH=1267200,RESOLUTION=1920x1080,AUDIO="group_audio_0"
 tv_hls_2.m3u8

Signed-off-by: Brendan McGrath <redmcg@redmandi.dyndns.org>
---

Pre-patch - the hls stream I was testing would not play on the iOS devices I was testing with.

With the patch applied - they now play the stream

 libavformat/hlsenc.c | 3 +++
 1 file changed, 3 insertions(+)

Comments

Steven Liu Jan. 18, 2018, 9:43 a.m. UTC | #1
> On 18 Jan 2018, at 17:09, Brendan McGrath <redmcg@redmandi.dyndns.org> wrote:
> 
> When using an 'agroup' within var_stream_map - the audio stream is
> being added to the master playlist file as both an audio rendition
> and an individual stream (with an AUDIO reference back to itself).
https://patchwork.ffmpeg.org/patch/7229/

Can this patch pass your test case?
> 
> This patch ensures an audio rendition does not also appear within
> the stream info list.
> 
> What follows is an example of the command to create this issue and
> the contents of the master playlist before and after this patch is
> applied:
> 
> ffmpeg -i in.ts -b:a:0 128k -b:v:0 1800k -b:v:1 1024k -map 0:a \
> -map 0:v -map 0:v -f hls -var_stream_map "a:0,agroup:audio_0 "\
> "v:0,agroup:audio_0 v:1,agroup:audio_0" -master_pl_name \
> tv_hls.m3u8 tv_hls_%v.m3u8
> 
> Before:
> #EXTM3U
> #EXT-X-VERSION:3
> #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="group_audio_0",NAME="audio_0",DEFAULT=YES,URI="tv_hls_0.m3u8"
> #EXT-X-STREAM-INF:BANDWIDTH=140800,AUDIO="group_audio_0"
> tv_hls_0.m3u8
> 
> #EXT-X-STREAM-INF:BANDWIDTH=2120800,RESOLUTION=1920x1080,AUDIO="group_audio_0"
> tv_hls_1.m3u8
> 
> #EXT-X-STREAM-INF:BANDWIDTH=1267200,RESOLUTION=1920x1080,AUDIO="group_audio_0"
> tv_hls_2.m3u8
> 
> After:
> #EXTM3U
> #EXT-X-VERSION:3
> #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="group_audio_0",NAME="audio_0",DEFAULT=YES,URI="tv_hls_0.m3u8"
> #EXT-X-STREAM-INF:BANDWIDTH=2120800,RESOLUTION=1920x1080,AUDIO="group_audio_0"
> tv_hls_1.m3u8
> 
> #EXT-X-STREAM-INF:BANDWIDTH=1267200,RESOLUTION=1920x1080,AUDIO="group_audio_0"
> tv_hls_2.m3u8
> 
> Signed-off-by: Brendan McGrath <redmcg@redmandi.dyndns.org>
> ---
> 
> Pre-patch - the hls stream I was testing would not play on the iOS devices I was testing with.
> 
> With the patch applied - they now play the stream
> 
> libavformat/hlsenc.c | 3 +++
> 1 file changed, 3 insertions(+)
> 
> diff --git a/libavformat/hlsenc.c b/libavformat/hlsenc.c
> index e36120c..a75853b 100644
> --- a/libavformat/hlsenc.c
> +++ b/libavformat/hlsenc.c
> @@ -1172,6 +1172,9 @@ static int create_master_playlist(AVFormatContext *s,
>     for (i = 0; i < hls->nb_varstreams; i++) {
>         vs = &(hls->var_streams[i]);
> 
> +        if (!vs->has_video && !vs->has_subtitle && vs->agroup)
> +            continue;
> +
>         m3u8_name_size = strlen(vs->m3u8_name) + 1;
>         m3u8_rel_name = av_malloc(m3u8_name_size);
>         if (!m3u8_rel_name) {
> -- 
> 2.7.4
> 
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Dixit, Vishwanath Jan. 18, 2018, 10:59 a.m. UTC | #2
On 1/18/18 2:39 PM, Brendan McGrath wrote:
> When using an 'agroup' within var_stream_map - the audio stream is

> being added to the master playlist file as both an audio rendition

> and an individual stream (with an AUDIO reference back to itself).

>

> This patch ensures an audio rendition does not also appear within

> the stream info list.

>

> What follows is an example of the command to create this issue and

> the contents of the master playlist before and after this patch is

> applied:

>

> ffmpeg -i in.ts -b:a:0 128k -b:v:0 1800k -b:v:1 1024k -map 0:a \

> -map 0:v -map 0:v -f hls -var_stream_map "a:0,agroup:audio_0 "\

> "v:0,agroup:audio_0 v:1,agroup:audio_0" -master_pl_name \

> tv_hls.m3u8 tv_hls_%v.m3u8

>

> Before:

>  #EXTM3U

>  #EXT-X-VERSION:3

>  #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="group_audio_0",NAME="audio_0",DEFAULT=YES,URI="tv_hls_0.m3u8"

>  #EXT-X-STREAM-INF:BANDWIDTH=140800,AUDIO="group_audio_0"

>  tv_hls_0.m3u8

>

>  #EXT-X-STREAM-INF:BANDWIDTH=2120800,RESOLUTION=1920x1080,AUDIO="group_audio_0"

>  tv_hls_1.m3u8

>

>  #EXT-X-STREAM-INF:BANDWIDTH=1267200,RESOLUTION=1920x1080,AUDIO="group_audio_0"

>  tv_hls_2.m3u8

>

> After:

>  #EXTM3U

>  #EXT-X-VERSION:3

>  #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="group_audio_0",NAME="audio_0",DEFAULT=YES,URI="tv_hls_0.m3u8"

>  #EXT-X-STREAM-INF:BANDWIDTH=2120800,RESOLUTION=1920x1080,AUDIO="group_audio_0"

>  tv_hls_1.m3u8

>

>  #EXT-X-STREAM-INF:BANDWIDTH=1267200,RESOLUTION=1920x1080,AUDIO="group_audio_0"

>  tv_hls_2.m3u8

>

> Signed-off-by: Brendan McGrath <redmcg@redmandi.dyndns.org>

> ---

>

Some use cases may need audio only variant streams. Ex: when bandwidth is too low. Also, I think the playback issue you are seeing, is because of AUDIO referencing back to itself, but, not because of audio only variant stream. So, instead of completely removing the audio only variant streams, you can just remove the self-referencing attribute (AUDIO=) from the #EXT-X-STREAM-INF tag’s attribute list, for the audio only variant streams.
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="group_audio_0",NAME="audio_0",DEFAULT=YES,URI="tv_hls_0.m3u8"
#EXT-X-STREAM-INF:BANDWIDTH=140800
tv_hls_0.m3u8

#EXT-X-STREAM-INF:BANDWIDTH=2120800,RESOLUTION=1920x1080,AUDIO="group_audio_0"
tv_hls_1.m3u8

#EXT-X-STREAM-INF:BANDWIDTH=1267200,RESOLUTION=1920x1080,AUDIO="group_audio_0"
tv_hls_2.m3u8


> Pre-patch - the hls stream I was testing would not play on the iOS devices I was testing with.

>

> With the patch applied - they now play the stream

>

>  libavformat/hlsenc.c | 3 +++

>  1 file changed, 3 insertions(+)

>

> diff --git a/libavformat/hlsenc.c b/libavformat/hlsenc.c

> index e36120c..a75853b 100644

> --- a/libavformat/hlsenc.c

> +++ b/libavformat/hlsenc.c

> @@ -1172,6 +1172,9 @@ static int create_master_playlist(AVFormatContext *s,

>      for (i = 0; i < hls->nb_varstreams; i++) {

>          vs = &(hls->var_streams[i]);

>  

> +        if (!vs->has_video && !vs->has_subtitle && vs->agroup)

> +            continue;

> +

>          m3u8_name_size = strlen(vs->m3u8_name) + 1;

>          m3u8_rel_name = av_malloc(m3u8_name_size);

>          if (!m3u8_rel_name) {
Brendan McGrath Jan. 18, 2018, 1:23 p.m. UTC | #3
I tried your suggestion and it still didn't work. So I took another look 
at the spec and it looks like what was originally there was correct (as 
seen in the below example):
https://tools.ietf.org/html/rfc8216#section-8.6

In that example - english-audio.m3u8 appears in both a MEDIA and 
STREAM-INF entry with the STREAM-INF entry including an AUDIO tag that 
references back to itself.

So I believe this patch can be dropped.

What I did discover was that it worked when I added a CODECS tag to the 
STREAM-INF variant of the audio stream. The spec just says:
Every EXT-X-STREAM-INF tag SHOULD include a CODECS attribute

So I guess this is a bug in iOS.

On 18/01/18 21:59, Dixit, Vishwanath wrote:
>
> On 1/18/18 2:39 PM, Brendan McGrath wrote:
>> When using an 'agroup' within var_stream_map - the audio stream is
>> being added to the master playlist file as both an audio rendition
>> and an individual stream (with an AUDIO reference back to itself).
>>
>> This patch ensures an audio rendition does not also appear within
>> the stream info list.
>>
>> What follows is an example of the command to create this issue and
>> the contents of the master playlist before and after this patch is
>> applied:
>>
>> ffmpeg -i in.ts -b:a:0 128k -b:v:0 1800k -b:v:1 1024k -map 0:a \
>> -map 0:v -map 0:v -f hls -var_stream_map "a:0,agroup:audio_0 "\
>> "v:0,agroup:audio_0 v:1,agroup:audio_0" -master_pl_name \
>> tv_hls.m3u8 tv_hls_%v.m3u8
>>
>> Before:
>>   #EXTM3U
>>   #EXT-X-VERSION:3
>>   #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="group_audio_0",NAME="audio_0",DEFAULT=YES,URI="tv_hls_0.m3u8"
>>   #EXT-X-STREAM-INF:BANDWIDTH=140800,AUDIO="group_audio_0"
>>   tv_hls_0.m3u8
>>
>>   #EXT-X-STREAM-INF:BANDWIDTH=2120800,RESOLUTION=1920x1080,AUDIO="group_audio_0"
>>   tv_hls_1.m3u8
>>
>>   #EXT-X-STREAM-INF:BANDWIDTH=1267200,RESOLUTION=1920x1080,AUDIO="group_audio_0"
>>   tv_hls_2.m3u8
>>
>> After:
>>   #EXTM3U
>>   #EXT-X-VERSION:3
>>   #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="group_audio_0",NAME="audio_0",DEFAULT=YES,URI="tv_hls_0.m3u8"
>>   #EXT-X-STREAM-INF:BANDWIDTH=2120800,RESOLUTION=1920x1080,AUDIO="group_audio_0"
>>   tv_hls_1.m3u8
>>
>>   #EXT-X-STREAM-INF:BANDWIDTH=1267200,RESOLUTION=1920x1080,AUDIO="group_audio_0"
>>   tv_hls_2.m3u8
>>
>> Signed-off-by: Brendan McGrath <redmcg@redmandi.dyndns.org>
>> ---
>>
> Some use cases may need audio only variant streams. Ex: when bandwidth is too low. Also, I think the playback issue you are seeing, is because of AUDIO referencing back to itself, but, not because of audio only variant stream. So, instead of completely removing the audio only variant streams, you can just remove the self-referencing attribute (AUDIO=) from the #EXT-X-STREAM-INF tag’s attribute list, for the audio only variant streams.
> #EXTM3U
> #EXT-X-VERSION:3
> #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="group_audio_0",NAME="audio_0",DEFAULT=YES,URI="tv_hls_0.m3u8"
> #EXT-X-STREAM-INF:BANDWIDTH=140800
> tv_hls_0.m3u8
>
> #EXT-X-STREAM-INF:BANDWIDTH=2120800,RESOLUTION=1920x1080,AUDIO="group_audio_0"
> tv_hls_1.m3u8
>
> #EXT-X-STREAM-INF:BANDWIDTH=1267200,RESOLUTION=1920x1080,AUDIO="group_audio_0"
> tv_hls_2.m3u8
>
>
>> Pre-patch - the hls stream I was testing would not play on the iOS devices I was testing with.
>>
>> With the patch applied - they now play the stream
>>
>>   libavformat/hlsenc.c | 3 +++
>>   1 file changed, 3 insertions(+)
>>
>> diff --git a/libavformat/hlsenc.c b/libavformat/hlsenc.c
>> index e36120c..a75853b 100644
>> --- a/libavformat/hlsenc.c
>> +++ b/libavformat/hlsenc.c
>> @@ -1172,6 +1172,9 @@ static int create_master_playlist(AVFormatContext *s,
>>       for (i = 0; i < hls->nb_varstreams; i++) {
>>           vs = &(hls->var_streams[i]);
>>   
>> +        if (!vs->has_video && !vs->has_subtitle && vs->agroup)
>> +            continue;
>> +
>>           m3u8_name_size = strlen(vs->m3u8_name) + 1;
>>           m3u8_rel_name = av_malloc(m3u8_name_size);
>>           if (!m3u8_rel_name) {
>
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Jeyapal, Karthick Jan. 18, 2018, 1:41 p.m. UTC | #4
On 1/18/18 6:53 PM, Brendan McGrath wrote:
> I tried your suggestion and it still didn't work. So I took another look at the spec and it looks like what was originally there was correct (as seen in the below example):

> https://tools.ietf.org/html/rfc8216#section-8.6

>

> In that example - english-audio.m3u8 appears in both a MEDIA and STREAM-INF entry with the STREAM-INF entry including an AUDIO tag that references back to itself.

>

> So I believe this patch can be dropped.

>

> What I did discover was that it worked when I added a CODECS tag to the STREAM-INF variant of the audio stream. The spec just says:

> Every EXT-X-STREAM-INF tag SHOULD include a CODECS attribute

>

> So I guess this is a bug in iOS.

I agree. But, there is a patch available https://patchwork.ffmpeg.org/patch/7000/ to add CODECS tag. 
You could check if applying that patch solves your issue. 
>

> On 18/01/18 21:59, Dixit, Vishwanath wrote:

>>

>> On 1/18/18 2:39 PM, Brendan McGrath wrote:

>>> When using an 'agroup' within var_stream_map - the audio stream is

>>> being added to the master playlist file as both an audio rendition

>>> and an individual stream (with an AUDIO reference back to itself).

>>>

>>> This patch ensures an audio rendition does not also appear within

>>> the stream info list.

>>>

>>> What follows is an example of the command to create this issue and

>>> the contents of the master playlist before and after this patch is

>>> applied:

>>>

>>> ffmpeg -i in.ts -b:a:0 128k -b:v:0 1800k -b:v:1 1024k -map 0:a \

>>> -map 0:v -map 0:v -f hls -var_stream_map "a:0,agroup:audio_0 "\

>>> "v:0,agroup:audio_0 v:1,agroup:audio_0" -master_pl_name \

>>> tv_hls.m3u8 tv_hls_%v.m3u8

>>>

>>> Before:

>>>   #EXTM3U

>>>   #EXT-X-VERSION:3

>>>   #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="group_audio_0",NAME="audio_0",DEFAULT=YES,URI="tv_hls_0.m3u8"

>>>   #EXT-X-STREAM-INF:BANDWIDTH=140800,AUDIO="group_audio_0"

>>>   tv_hls_0.m3u8

>>>

>>>   #EXT-X-STREAM-INF:BANDWIDTH=2120800,RESOLUTION=1920x1080,AUDIO="group_audio_0"

>>>   tv_hls_1.m3u8

>>>

>>>   #EXT-X-STREAM-INF:BANDWIDTH=1267200,RESOLUTION=1920x1080,AUDIO="group_audio_0"

>>>   tv_hls_2.m3u8

>>>

>>> After:

>>>   #EXTM3U

>>>   #EXT-X-VERSION:3

>>>   #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="group_audio_0",NAME="audio_0",DEFAULT=YES,URI="tv_hls_0.m3u8"

>>>   #EXT-X-STREAM-INF:BANDWIDTH=2120800,RESOLUTION=1920x1080,AUDIO="group_audio_0"

>>>   tv_hls_1.m3u8

>>>

>>>   #EXT-X-STREAM-INF:BANDWIDTH=1267200,RESOLUTION=1920x1080,AUDIO="group_audio_0"

>>>   tv_hls_2.m3u8

>>>

>>> Signed-off-by: Brendan McGrath <redmcg@redmandi.dyndns.org>

>>> ---

>>>

>> Some use cases may need audio only variant streams. Ex: when bandwidth is too low. Also, I think the playback issue you are seeing, is because of AUDIO referencing back to itself, but, not because of audio only variant stream. So, instead of completely removing the audio only variant streams, you can just remove the self-referencing attribute (AUDIO=) from the #EXT-X-STREAM-INF tag’s attribute list, for the audio only variant streams.

>> #EXTM3U

>> #EXT-X-VERSION:3

>> #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="group_audio_0",NAME="audio_0",DEFAULT=YES,URI="tv_hls_0.m3u8"

>> #EXT-X-STREAM-INF:BANDWIDTH=140800

>> tv_hls_0.m3u8

>>

>> #EXT-X-STREAM-INF:BANDWIDTH=2120800,RESOLUTION=1920x1080,AUDIO="group_audio_0"

>> tv_hls_1.m3u8

>>

>> #EXT-X-STREAM-INF:BANDWIDTH=1267200,RESOLUTION=1920x1080,AUDIO="group_audio_0"

>> tv_hls_2.m3u8

>>

>>

>>> Pre-patch - the hls stream I was testing would not play on the iOS devices I was testing with.

>>>

>>> With the patch applied - they now play the stream

>>>

>>>   libavformat/hlsenc.c | 3 +++

>>>   1 file changed, 3 insertions(+)

>>>

>>> diff --git a/libavformat/hlsenc.c b/libavformat/hlsenc.c

>>> index e36120c..a75853b 100644

>>> --- a/libavformat/hlsenc.c

>>> +++ b/libavformat/hlsenc.c

>>> @@ -1172,6 +1172,9 @@ static int create_master_playlist(AVFormatContext *s,

>>>       for (i = 0; i < hls->nb_varstreams; i++) {

>>>           vs = &(hls->var_streams[i]);

>>>   +        if (!vs->has_video && !vs->has_subtitle && vs->agroup)

>>> +            continue;

>>> +

>>>           m3u8_name_size = strlen(vs->m3u8_name) + 1;

>>>           m3u8_rel_name = av_malloc(m3u8_name_size);

>>>           if (!m3u8_rel_name) { 

>>

>>

>> _______________________________________________

>> ffmpeg-devel mailing list

>> ffmpeg-devel@ffmpeg.org

>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel 

>

>

> _______________________________________________

> ffmpeg-devel mailing list

> ffmpeg-devel@ffmpeg.org

> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Brendan McGrath Jan. 19, 2018, 3:12 a.m. UTC | #5
Thanks Karthick - I can confirm that your patch does solve the issue.

However - I noticed your patch does not include HEVC. I believe this 
format is now supported in HLS (and that several players including iOS 
11 now provide support).

On 19/01/18 00:41, Jeyapal, Karthick wrote:
> On 1/18/18 6:53 PM, Brendan McGrath wrote:
>> I tried your suggestion and it still didn't work. So I took another look at the spec and it looks like what was originally there was correct (as seen in the below example):
>> https://tools.ietf.org/html/rfc8216#section-8.6
>>
>> In that example - english-audio.m3u8 appears in both a MEDIA and STREAM-INF entry with the STREAM-INF entry including an AUDIO tag that references back to itself.
>>
>> So I believe this patch can be dropped.
>>
>> What I did discover was that it worked when I added a CODECS tag to the STREAM-INF variant of the audio stream. The spec just says:
>> Every EXT-X-STREAM-INF tag SHOULD include a CODECS attribute
>>
>> So I guess this is a bug in iOS.
> I agree. But, there is a patch availablehttps://patchwork.ffmpeg.org/patch/7000/  to add CODECS tag.
> You could check if applying that patch solves your issue.
>> On 18/01/18 21:59, Dixit, Vishwanath wrote:
>>> On 1/18/18 2:39 PM, Brendan McGrath wrote:
>>>> When using an 'agroup' within var_stream_map - the audio stream is
>>>> being added to the master playlist file as both an audio rendition
>>>> and an individual stream (with an AUDIO reference back to itself).
>>>>
>>>> This patch ensures an audio rendition does not also appear within
>>>> the stream info list.
>>>>
>>>> What follows is an example of the command to create this issue and
>>>> the contents of the master playlist before and after this patch is
>>>> applied:
>>>>
>>>> ffmpeg -i in.ts -b:a:0 128k -b:v:0 1800k -b:v:1 1024k -map 0:a \
>>>> -map 0:v -map 0:v -f hls -var_stream_map "a:0,agroup:audio_0 "\
>>>> "v:0,agroup:audio_0 v:1,agroup:audio_0" -master_pl_name \
>>>> tv_hls.m3u8 tv_hls_%v.m3u8
>>>>
>>>> Before:
>>>>    #EXTM3U
>>>>    #EXT-X-VERSION:3
>>>>    #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="group_audio_0",NAME="audio_0",DEFAULT=YES,URI="tv_hls_0.m3u8"
>>>>    #EXT-X-STREAM-INF:BANDWIDTH=140800,AUDIO="group_audio_0"
>>>>    tv_hls_0.m3u8
>>>>
>>>>    #EXT-X-STREAM-INF:BANDWIDTH=2120800,RESOLUTION=1920x1080,AUDIO="group_audio_0"
>>>>    tv_hls_1.m3u8
>>>>
>>>>    #EXT-X-STREAM-INF:BANDWIDTH=1267200,RESOLUTION=1920x1080,AUDIO="group_audio_0"
>>>>    tv_hls_2.m3u8
>>>>
>>>> After:
>>>>    #EXTM3U
>>>>    #EXT-X-VERSION:3
>>>>    #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="group_audio_0",NAME="audio_0",DEFAULT=YES,URI="tv_hls_0.m3u8"
>>>>    #EXT-X-STREAM-INF:BANDWIDTH=2120800,RESOLUTION=1920x1080,AUDIO="group_audio_0"
>>>>    tv_hls_1.m3u8
>>>>
>>>>    #EXT-X-STREAM-INF:BANDWIDTH=1267200,RESOLUTION=1920x1080,AUDIO="group_audio_0"
>>>>    tv_hls_2.m3u8
>>>>
>>>> Signed-off-by: Brendan McGrath<redmcg@redmandi.dyndns.org>
>>>> ---
>>>>
>>> Some use cases may need audio only variant streams. Ex: when bandwidth is too low. Also, I think the playback issue you are seeing, is because of AUDIO referencing back to itself, but, not because of audio only variant stream. So, instead of completely removing the audio only variant streams, you can just remove the self-referencing attribute (AUDIO=) from the #EXT-X-STREAM-INF tag’s attribute list, for the audio only variant streams.
>>> #EXTM3U
>>> #EXT-X-VERSION:3
>>> #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="group_audio_0",NAME="audio_0",DEFAULT=YES,URI="tv_hls_0.m3u8"
>>> #EXT-X-STREAM-INF:BANDWIDTH=140800
>>> tv_hls_0.m3u8
>>>
>>> #EXT-X-STREAM-INF:BANDWIDTH=2120800,RESOLUTION=1920x1080,AUDIO="group_audio_0"
>>> tv_hls_1.m3u8
>>>
>>> #EXT-X-STREAM-INF:BANDWIDTH=1267200,RESOLUTION=1920x1080,AUDIO="group_audio_0"
>>> tv_hls_2.m3u8
>>>
>>>
>>>> Pre-patch - the hls stream I was testing would not play on the iOS devices I was testing with.
>>>>
>>>> With the patch applied - they now play the stream
>>>>
>>>>    libavformat/hlsenc.c | 3 +++
>>>>    1 file changed, 3 insertions(+)
>>>>
>>>> diff --git a/libavformat/hlsenc.c b/libavformat/hlsenc.c
>>>> index e36120c..a75853b 100644
>>>> --- a/libavformat/hlsenc.c
>>>> +++ b/libavformat/hlsenc.c
>>>> @@ -1172,6 +1172,9 @@ static int create_master_playlist(AVFormatContext *s,
>>>>        for (i = 0; i < hls->nb_varstreams; i++) {
>>>>            vs = &(hls->var_streams[i]);
>>>>    +        if (!vs->has_video && !vs->has_subtitle && vs->agroup)
>>>> +            continue;
>>>> +
>>>>            m3u8_name_size = strlen(vs->m3u8_name) + 1;
>>>>            m3u8_rel_name = av_malloc(m3u8_name_size);
>>>>            if (!m3u8_rel_name) {
>>> _______________________________________________
>>> ffmpeg-devel mailing list
>>> ffmpeg-devel@ffmpeg.org
>>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel  
>> _______________________________________________
>> ffmpeg-devel mailing list
>> ffmpeg-devel@ffmpeg.org
>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel  
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Jeyapal, Karthick Jan. 19, 2018, 3:18 a.m. UTC | #6
On 1/19/18 8:42 AM, Brendan McGrath wrote:
> Thanks Karthick - I can confirm that your patch does solve the issue.

Thanks for the confirmation.
>

> However - I noticed your patch does not include HEVC. I believe this format is now supported in HLS (and that several players including iOS 11 now provide support).

Yes, I just supported H264 since I was working on it. But that patch is modular enough to easily extend CODECS tag support for HEVC. 
Maybe somebody can add the HEVC support in CODECS tag later :)
>

> On 19/01/18 00:41, Jeyapal, Karthick wrote:

>> On 1/18/18 6:53 PM, Brendan McGrath wrote:

>>> I tried your suggestion and it still didn't work. So I took another look at the spec and it looks like what was originally there was correct (as seen in the below example):

>>> https://tools.ietf.org/html/rfc8216#section-8.6

>>>

>>> In that example - english-audio.m3u8 appears in both a MEDIA and STREAM-INF entry with the STREAM-INF entry including an AUDIO tag that references back to itself.

>>>

>>> So I believe this patch can be dropped.

>>>

>>> What I did discover was that it worked when I added a CODECS tag to the STREAM-INF variant of the audio stream. The spec just says:

>>> Every EXT-X-STREAM-INF tag SHOULD include a CODECS attribute

>>>

>>> So I guess this is a bug in iOS.

>> I agree. But, there is a patch available https://patchwork.ffmpeg.org/patch/7000/ to add CODECS tag. 

>> You could check if applying that patch solves your issue. 

>>> On 18/01/18 21:59, Dixit, Vishwanath wrote:

>>>> On 1/18/18 2:39 PM, Brendan McGrath wrote:

>>>>> When using an 'agroup' within var_stream_map - the audio stream is

>>>>> being added to the master playlist file as both an audio rendition

>>>>> and an individual stream (with an AUDIO reference back to itself).

>>>>>

>>>>> This patch ensures an audio rendition does not also appear within

>>>>> the stream info list.

>>>>>

>>>>> What follows is an example of the command to create this issue and

>>>>> the contents of the master playlist before and after this patch is

>>>>> applied:

>>>>>

>>>>> ffmpeg -i in.ts -b:a:0 128k -b:v:0 1800k -b:v:1 1024k -map 0:a \

>>>>> -map 0:v -map 0:v -f hls -var_stream_map "a:0,agroup:audio_0 "\

>>>>> "v:0,agroup:audio_0 v:1,agroup:audio_0" -master_pl_name \

>>>>> tv_hls.m3u8 tv_hls_%v.m3u8

>>>>>

>>>>> Before:

>>>>>   #EXTM3U

>>>>>   #EXT-X-VERSION:3

>>>>>   #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="group_audio_0",NAME="audio_0",DEFAULT=YES,URI="tv_hls_0.m3u8"

>>>>>   #EXT-X-STREAM-INF:BANDWIDTH=140800,AUDIO="group_audio_0"

>>>>>   tv_hls_0.m3u8

>>>>>

>>>>>   #EXT-X-STREAM-INF:BANDWIDTH=2120800,RESOLUTION=1920x1080,AUDIO="group_audio_0"

>>>>>   tv_hls_1.m3u8

>>>>>

>>>>>   #EXT-X-STREAM-INF:BANDWIDTH=1267200,RESOLUTION=1920x1080,AUDIO="group_audio_0"

>>>>>   tv_hls_2.m3u8

>>>>>

>>>>> After:

>>>>>   #EXTM3U

>>>>>   #EXT-X-VERSION:3

>>>>>   #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="group_audio_0",NAME="audio_0",DEFAULT=YES,URI="tv_hls_0.m3u8"

>>>>>   #EXT-X-STREAM-INF:BANDWIDTH=2120800,RESOLUTION=1920x1080,AUDIO="group_audio_0"

>>>>>   tv_hls_1.m3u8

>>>>>

>>>>>   #EXT-X-STREAM-INF:BANDWIDTH=1267200,RESOLUTION=1920x1080,AUDIO="group_audio_0"

>>>>>   tv_hls_2.m3u8

>>>>>

>>>>> Signed-off-by: Brendan McGrath <redmcg@redmandi.dyndns.org>

>>>>> ---

>>>>>

>>>> Some use cases may need audio only variant streams. Ex: when bandwidth is too low. Also, I think the playback issue you are seeing, is because of AUDIO referencing back to itself, but, not because of audio only variant stream. So, instead of completely removing the audio only variant streams, you can just remove the self-referencing attribute (AUDIO=) from the #EXT-X-STREAM-INF tag’s attribute list, for the audio only variant streams.

>>>> #EXTM3U

>>>> #EXT-X-VERSION:3

>>>> #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="group_audio_0",NAME="audio_0",DEFAULT=YES,URI="tv_hls_0.m3u8"

>>>> #EXT-X-STREAM-INF:BANDWIDTH=140800

>>>> tv_hls_0.m3u8

>>>>

>>>> #EXT-X-STREAM-INF:BANDWIDTH=2120800,RESOLUTION=1920x1080,AUDIO="group_audio_0"

>>>> tv_hls_1.m3u8

>>>>

>>>> #EXT-X-STREAM-INF:BANDWIDTH=1267200,RESOLUTION=1920x1080,AUDIO="group_audio_0"

>>>> tv_hls_2.m3u8

>>>>

>>>>

>>>>> Pre-patch - the hls stream I was testing would not play on the iOS devices I was testing with.

>>>>>

>>>>> With the patch applied - they now play the stream

>>>>>

>>>>>   libavformat/hlsenc.c | 3 +++

>>>>>   1 file changed, 3 insertions(+)

>>>>>

>>>>> diff --git a/libavformat/hlsenc.c b/libavformat/hlsenc.c

>>>>> index e36120c..a75853b 100644

>>>>> --- a/libavformat/hlsenc.c

>>>>> +++ b/libavformat/hlsenc.c

>>>>> @@ -1172,6 +1172,9 @@ static int create_master_playlist(AVFormatContext *s,

>>>>>       for (i = 0; i < hls->nb_varstreams; i++) {

>>>>>           vs = &(hls->var_streams[i]);

>>>>>   +        if (!vs->has_video && !vs->has_subtitle && vs->agroup)

>>>>> +            continue;

>>>>> +

>>>>>           m3u8_name_size = strlen(vs->m3u8_name) + 1;

>>>>>           m3u8_rel_name = av_malloc(m3u8_name_size);

>>>>>           if (!m3u8_rel_name) { 

>>>> _______________________________________________

>>>> ffmpeg-devel mailing list

>>>> ffmpeg-devel@ffmpeg.org

>>>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel 

>>> _______________________________________________

>>> ffmpeg-devel mailing list

>>> ffmpeg-devel@ffmpeg.org

>>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel 

>>

>> _______________________________________________

>> ffmpeg-devel mailing list

>> ffmpeg-devel@ffmpeg.org

>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

>

>
Steven Liu Jan. 19, 2018, 3:31 a.m. UTC | #7
> On 19 Jan 2018, at 11:18, Jeyapal, Karthick <kjeyapal@akamai.com> wrote:
> 
> 
> 
> On 1/19/18 8:42 AM, Brendan McGrath wrote:
>> Thanks Karthick - I can confirm that your patch does solve the issue.
> Thanks for the confirmation.
>> 
>> However - I noticed your patch does not include HEVC. I believe this format is now supported in HLS (and that several players including iOS 11 now provide support).
> Yes, I just supported H264 since I was working on it. But that patch is modular enough to easily extend CODECS tag support for HEVC. 
> Maybe somebody can add the HEVC support in CODECS tag later :)
Can you add it into the patch, so that will be perfect :D
>> 
>> On 19/01/18 00:41, Jeyapal, Karthick wrote:
>>> On 1/18/18 6:53 PM, Brendan McGrath wrote:
>>>> I tried your suggestion and it still didn't work. So I took another look at the spec and it looks like what was originally there was correct (as seen in the below example):
>>>> https://tools.ietf.org/html/rfc8216#section-8.6
>>>> 
>>>> In that example - english-audio.m3u8 appears in both a MEDIA and STREAM-INF entry with the STREAM-INF entry including an AUDIO tag that references back to itself.
>>>> 
>>>> So I believe this patch can be dropped.
>>>> 
>>>> What I did discover was that it worked when I added a CODECS tag to the STREAM-INF variant of the audio stream. The spec just says:
>>>> Every EXT-X-STREAM-INF tag SHOULD include a CODECS attribute
>>>> 
>>>> So I guess this is a bug in iOS.
>>> I agree. But, there is a patch available https://patchwork.ffmpeg.org/patch/7000/ to add CODECS tag. 
>>> You could check if applying that patch solves your issue. 
>>>> On 18/01/18 21:59, Dixit, Vishwanath wrote:
>>>>> On 1/18/18 2:39 PM, Brendan McGrath wrote:
>>>>>> When using an 'agroup' within var_stream_map - the audio stream is
>>>>>> being added to the master playlist file as both an audio rendition
>>>>>> and an individual stream (with an AUDIO reference back to itself).
>>>>>> 
>>>>>> This patch ensures an audio rendition does not also appear within
>>>>>> the stream info list.
>>>>>> 
>>>>>> What follows is an example of the command to create this issue and
>>>>>> the contents of the master playlist before and after this patch is
>>>>>> applied:
>>>>>> 
>>>>>> ffmpeg -i in.ts -b:a:0 128k -b:v:0 1800k -b:v:1 1024k -map 0:a \
>>>>>> -map 0:v -map 0:v -f hls -var_stream_map "a:0,agroup:audio_0 "\
>>>>>> "v:0,agroup:audio_0 v:1,agroup:audio_0" -master_pl_name \
>>>>>> tv_hls.m3u8 tv_hls_%v.m3u8
>>>>>> 
>>>>>> Before:
>>>>>>  #EXTM3U
>>>>>>  #EXT-X-VERSION:3
>>>>>>  #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="group_audio_0",NAME="audio_0",DEFAULT=YES,URI="tv_hls_0.m3u8"
>>>>>>  #EXT-X-STREAM-INF:BANDWIDTH=140800,AUDIO="group_audio_0"
>>>>>>  tv_hls_0.m3u8
>>>>>> 
>>>>>>  #EXT-X-STREAM-INF:BANDWIDTH=2120800,RESOLUTION=1920x1080,AUDIO="group_audio_0"
>>>>>>  tv_hls_1.m3u8
>>>>>> 
>>>>>>  #EXT-X-STREAM-INF:BANDWIDTH=1267200,RESOLUTION=1920x1080,AUDIO="group_audio_0"
>>>>>>  tv_hls_2.m3u8
>>>>>> 
>>>>>> After:
>>>>>>  #EXTM3U
>>>>>>  #EXT-X-VERSION:3
>>>>>>  #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="group_audio_0",NAME="audio_0",DEFAULT=YES,URI="tv_hls_0.m3u8"
>>>>>>  #EXT-X-STREAM-INF:BANDWIDTH=2120800,RESOLUTION=1920x1080,AUDIO="group_audio_0"
>>>>>>  tv_hls_1.m3u8
>>>>>> 
>>>>>>  #EXT-X-STREAM-INF:BANDWIDTH=1267200,RESOLUTION=1920x1080,AUDIO="group_audio_0"
>>>>>>  tv_hls_2.m3u8
>>>>>> 
>>>>>> Signed-off-by: Brendan McGrath <redmcg@redmandi.dyndns.org>
>>>>>> ---
>>>>>> 
>>>>> Some use cases may need audio only variant streams. Ex: when bandwidth is too low. Also, I think the playback issue you are seeing, is because of AUDIO referencing back to itself, but, not because of audio only variant stream. So, instead of completely removing the audio only variant streams, you can just remove the self-referencing attribute (AUDIO=) from the #EXT-X-STREAM-INF tag’s attribute list, for the audio only variant streams.
>>>>> #EXTM3U
>>>>> #EXT-X-VERSION:3
>>>>> #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="group_audio_0",NAME="audio_0",DEFAULT=YES,URI="tv_hls_0.m3u8"
>>>>> #EXT-X-STREAM-INF:BANDWIDTH=140800
>>>>> tv_hls_0.m3u8
>>>>> 
>>>>> #EXT-X-STREAM-INF:BANDWIDTH=2120800,RESOLUTION=1920x1080,AUDIO="group_audio_0"
>>>>> tv_hls_1.m3u8
>>>>> 
>>>>> #EXT-X-STREAM-INF:BANDWIDTH=1267200,RESOLUTION=1920x1080,AUDIO="group_audio_0"
>>>>> tv_hls_2.m3u8
>>>>> 
>>>>> 
>>>>>> Pre-patch - the hls stream I was testing would not play on the iOS devices I was testing with.
>>>>>> 
>>>>>> With the patch applied - they now play the stream
>>>>>> 
>>>>>>  libavformat/hlsenc.c | 3 +++
>>>>>>  1 file changed, 3 insertions(+)
>>>>>> 
>>>>>> diff --git a/libavformat/hlsenc.c b/libavformat/hlsenc.c
>>>>>> index e36120c..a75853b 100644
>>>>>> --- a/libavformat/hlsenc.c
>>>>>> +++ b/libavformat/hlsenc.c
>>>>>> @@ -1172,6 +1172,9 @@ static int create_master_playlist(AVFormatContext *s,
>>>>>>      for (i = 0; i < hls->nb_varstreams; i++) {
>>>>>>          vs = &(hls->var_streams[i]);
>>>>>>  +        if (!vs->has_video && !vs->has_subtitle && vs->agroup)
>>>>>> +            continue;
>>>>>> +
>>>>>>          m3u8_name_size = strlen(vs->m3u8_name) + 1;
>>>>>>          m3u8_rel_name = av_malloc(m3u8_name_size);
>>>>>>          if (!m3u8_rel_name) { 
>>>>> _______________________________________________
>>>>> ffmpeg-devel mailing list
>>>>> ffmpeg-devel@ffmpeg.org
>>>>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel 
>>>> _______________________________________________
>>>> ffmpeg-devel mailing list
>>>> ffmpeg-devel@ffmpeg.org
>>>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel 
>>> 
>>> _______________________________________________
>>> ffmpeg-devel mailing list
>>> ffmpeg-devel@ffmpeg.org
>>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>> 
>> 
> 
> 
> 
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
diff mbox

Patch

diff --git a/libavformat/hlsenc.c b/libavformat/hlsenc.c
index e36120c..a75853b 100644
--- a/libavformat/hlsenc.c
+++ b/libavformat/hlsenc.c
@@ -1172,6 +1172,9 @@  static int create_master_playlist(AVFormatContext *s,
     for (i = 0; i < hls->nb_varstreams; i++) {
         vs = &(hls->var_streams[i]);
 
+        if (!vs->has_video && !vs->has_subtitle && vs->agroup)
+            continue;
+
         m3u8_name_size = strlen(vs->m3u8_name) + 1;
         m3u8_rel_name = av_malloc(m3u8_name_size);
         if (!m3u8_rel_name) {