mbox series

[FFmpeg-devel,v7,0/4] add ARIB caption decoder using libaribcaption

Message ID 20230223101718.562202-1-aimingoff@pc.nifty.jp
Headers show
Series add ARIB caption decoder using libaribcaption | expand

Message

TADANO Tokumei Feb. 23, 2023, 10:17 a.m. UTC
Ping with rebased patch set.
Some chages are added to reflect review results from outside of this ML.

---
This patch set add another ARIB caption decoder using libaribcaption
external library: https://github.com/xqq/libaribcaption
The library decodes subtitles of ISDB-based TV broadcasting.
It is not only for Japanese ARIB STD-B24 caption, but also for
Brazilian ABNT NBR 15606-1 and Philippines version of ISDB-T.

Unlike libaribb24, it supports 3 types of subtitle outputs:
* text: plain text
* ass: ASS formatted text
* bitmap: bitmap image

Default subtitle type is ASS as same as libaribb24.
Advantages compared with libaribb24 on ASS subtitle are:
* Subtitle positioning.
* Multi-rect subtitle: some captions are displayed at different
  position at a time.
* More stability and reproducibility.

Subtitle type can be changed by `-sub_type` option.
You can see ARIB caption with ffplay tool:
  ffplay -sub_type bitmap MPEG.TS

Sample files exist under:
  https://streams.videolan.org/streams/ts/ARIB/
Some of them are encrypted and some don't contain ARIB caption data.
Good samples for ARIB caption are:
  brazil/07-25_20-33-35_UCV - HD_.ts
  japan/channel2[137]_clear.ts
  japan/osaka_15.ts

---
v7: reflect review results from outside of this ML
  - rename -ass_workaround option to -ass_single_rect and default to
    false (latest mpv supports multiple ASS rectangle)
  - add -canvas_size option to specify frame size to render bitmap subtitle
  - add -caption_encoding option to specify encoding of subtitle text
  - change behavior of profile C ARIB caption
  - some cosmetic changes
v6: reflect review results from outside of this ML
  - check clut table overflow
  - do not adjust vertical position of ruby for ASS format
v5: reflect review comments from Mao Hata
  - reset correct variable in aribcaption_close()
  - add aribcaption_close() to some places in aribcaption_init()
  - check if av_strdup() returns NULL in set_ass_header()
v4: reflect review results from outside of this ML.
  - resize bitmap subtitle image to display fonts with correct aspect ratio
  - multiple font families can be specified by '-font' option
  - remove 'rendering_backend' option
  - add document
  - minor bug fixes
v3: combine former 1/4 and 2/4 due to the patchwork shows build error.
  - fix help option content which incorrectly separated to 2 lines in 2/4.
  - amend commit message of 4/4.

TADANO Tokumei (4):
  lavc/libaribcaption.c: add ARIB caption decoder using libaribcaption
  lavc/codec_desc.c: remove AV_CODEC_PROP_TEXT_SUB property
  lavf/mpegts.c: set some properties for ARIB caption
  doc/decoders.texi: add document of aribcaption decoder

 configure                   |    4 +
 doc/decoders.texi           |  150 +++++
 libavcodec/Makefile         |    1 +
 libavcodec/allcodecs.c      |    1 +
 libavcodec/codec_desc.c     |    1 -
 libavcodec/libaribb24.c     |    1 +
 libavcodec/libaribcaption.c | 1171 +++++++++++++++++++++++++++++++++++
 libavformat/mpegts.c        |    6 +-
 8 files changed, 1333 insertions(+), 2 deletions(-)
 create mode 100644 libavcodec/libaribcaption.c

Comments

rcombs Feb. 28, 2023, 12:07 a.m. UTC | #1
> On Feb 23, 2023, at 04:17, TADANO Tokumei <aimingoff@pc.nifty.jp> wrote:
> 
> Ping with rebased patch set.
> Some chages are added to reflect review results from outside of this ML.
> 
> ---
> This patch set add another ARIB caption decoder using libaribcaption
> external library: https://github.com/xqq/libaribcaption
> The library decodes subtitles of ISDB-based TV broadcasting.
> It is not only for Japanese ARIB STD-B24 caption, but also for
> Brazilian ABNT NBR 15606-1 and Philippines version of ISDB-T.
> 
> Unlike libaribb24, it supports 3 types of subtitle outputs:
> * text: plain text
> * ass: ASS formatted text
> * bitmap: bitmap image
> 
> Default subtitle type is ASS as same as libaribb24.
> Advantages compared with libaribb24 on ASS subtitle are:
> * Subtitle positioning.
> * Multi-rect subtitle: some captions are displayed at different
>  position at a time.
> * More stability and reproducibility.
> 
> Subtitle type can be changed by `-sub_type` option.
> You can see ARIB caption with ffplay tool:
>  ffplay -sub_type bitmap MPEG.TS
> 
> Sample files exist under:
>  https://streams.videolan.org/streams/ts/ARIB/
> Some of them are encrypted and some don't contain ARIB caption data.
> Good samples for ARIB caption are:
>  brazil/07-25_20-33-35_UCV - HD_.ts
>  japan/channel2[137]_clear.ts
>  japan/osaka_15.ts
> 
> ---
> v7: reflect review results from outside of this ML
>  - rename -ass_workaround option to -ass_single_rect and default to
>    false (latest mpv supports multiple ASS rectangle)
>  - add -canvas_size option to specify frame size to render bitmap subtitle
>  - add -caption_encoding option to specify encoding of subtitle text
>  - change behavior of profile C ARIB caption
>  - some cosmetic changes
> v6: reflect review results from outside of this ML
>  - check clut table overflow
>  - do not adjust vertical position of ruby for ASS format
> v5: reflect review comments from Mao Hata
>  - reset correct variable in aribcaption_close()
>  - add aribcaption_close() to some places in aribcaption_init()
>  - check if av_strdup() returns NULL in set_ass_header()
> v4: reflect review results from outside of this ML.
>  - resize bitmap subtitle image to display fonts with correct aspect ratio
>  - multiple font families can be specified by '-font' option
>  - remove 'rendering_backend' option
>  - add document
>  - minor bug fixes
> v3: combine former 1/4 and 2/4 due to the patchwork shows build error.
>  - fix help option content which incorrectly separated to 2 lines in 2/4.
>  - amend commit message of 4/4.
> 
> TADANO Tokumei (4):
>  lavc/libaribcaption.c: add ARIB caption decoder using libaribcaption
>  lavc/codec_desc.c: remove AV_CODEC_PROP_TEXT_SUB property
>  lavf/mpegts.c: set some properties for ARIB caption
>  doc/decoders.texi: add document of aribcaption decoder
> 
> configure                   |    4 +
> doc/decoders.texi           |  150 +++++
> libavcodec/Makefile         |    1 +
> libavcodec/allcodecs.c      |    1 +
> libavcodec/codec_desc.c     |    1 -
> libavcodec/libaribb24.c     |    1 +
> libavcodec/libaribcaption.c | 1171 +++++++++++++++++++++++++++++++++++
> libavformat/mpegts.c        |    6 +-
> 8 files changed, 1333 insertions(+), 2 deletions(-)
> create mode 100644 libavcodec/libaribcaption.c
> 
> -- 
> 2.30.2
> 
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

This set looks reasonable to me at this point; the one thing blocking it is the issue with assenc.c muxing for AVSubtitles with multiple rects, which needs a decision on this question: http://ffmpeg.org/pipermail/ffmpeg-devel/2023-February/306850.html

Once that's resolved, I'm happy to merge this.
TADANO Tokumei March 3, 2023, 9:58 a.m. UTC | #2
On 2023/02/28 9:07, Ridley Combs wrote:
> 
>> On Feb 23, 2023, at 04:17, TADANO Tokumei <aimingoff@pc.nifty.jp> wrote:
>>
>> Ping with rebased patch set.
>> Some chages are added to reflect review results from outside of this ML.
>>
>> ---
>> This patch set add another ARIB caption decoder using libaribcaption
>> external library: https://github.com/xqq/libaribcaption
>> The library decodes subtitles of ISDB-based TV broadcasting.
>> It is not only for Japanese ARIB STD-B24 caption, but also for
>> Brazilian ABNT NBR 15606-1 and Philippines version of ISDB-T.
>>
>> Unlike libaribb24, it supports 3 types of subtitle outputs:
>> * text: plain text
>> * ass: ASS formatted text
>> * bitmap: bitmap image
>>
>> Default subtitle type is ASS as same as libaribb24.
>> Advantages compared with libaribb24 on ASS subtitle are:
>> * Subtitle positioning.
>> * Multi-rect subtitle: some captions are displayed at different
>>   position at a time.
>> * More stability and reproducibility.
>>
>> Subtitle type can be changed by `-sub_type` option.
>> You can see ARIB caption with ffplay tool:
>>   ffplay -sub_type bitmap MPEG.TS
>>
>> Sample files exist under:
>>   https://streams.videolan.org/streams/ts/ARIB/
>> Some of them are encrypted and some don't contain ARIB caption data.
>> Good samples for ARIB caption are:
>>   brazil/07-25_20-33-35_UCV - HD_.ts
>>   japan/channel2[137]_clear.ts
>>   japan/osaka_15.ts
>>
>> ---
>> v7: reflect review results from outside of this ML
>>   - rename -ass_workaround option to -ass_single_rect and default to
>>     false (latest mpv supports multiple ASS rectangle)
>>   - add -canvas_size option to specify frame size to render bitmap subtitle
>>   - add -caption_encoding option to specify encoding of subtitle text
>>   - change behavior of profile C ARIB caption
>>   - some cosmetic changes
>> v6: reflect review results from outside of this ML
>>   - check clut table overflow
>>   - do not adjust vertical position of ruby for ASS format
>> v5: reflect review comments from Mao Hata
>>   - reset correct variable in aribcaption_close()
>>   - add aribcaption_close() to some places in aribcaption_init()
>>   - check if av_strdup() returns NULL in set_ass_header()
>> v4: reflect review results from outside of this ML.
>>   - resize bitmap subtitle image to display fonts with correct aspect ratio
>>   - multiple font families can be specified by '-font' option
>>   - remove 'rendering_backend' option
>>   - add document
>>   - minor bug fixes
>> v3: combine former 1/4 and 2/4 due to the patchwork shows build error.
>>   - fix help option content which incorrectly separated to 2 lines in 2/4.
>>   - amend commit message of 4/4.
>>
>> TADANO Tokumei (4):
>>   lavc/libaribcaption.c: add ARIB caption decoder using libaribcaption
>>   lavc/codec_desc.c: remove AV_CODEC_PROP_TEXT_SUB property
>>   lavf/mpegts.c: set some properties for ARIB caption
>>   doc/decoders.texi: add document of aribcaption decoder
>>
>> configure                   |    4 +
>> doc/decoders.texi           |  150 +++++
>> libavcodec/Makefile         |    1 +
>> libavcodec/allcodecs.c      |    1 +
>> libavcodec/codec_desc.c     |    1 -
>> libavcodec/libaribb24.c     |    1 +
>> libavcodec/libaribcaption.c | 1171 +++++++++++++++++++++++++++++++++++
>> libavformat/mpegts.c        |    6 +-
>> 8 files changed, 1333 insertions(+), 2 deletions(-)
>> create mode 100644 libavcodec/libaribcaption.c
>>
>> -- 
>> 2.30.2
>>
>> _______________________________________________
>> ffmpeg-devel mailing list
>> ffmpeg-devel@ffmpeg.org
>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>
>> To unsubscribe, visit link above, or email
>> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> 
> This set looks reasonable to me at this point; the one thing blocking it is the issue with assenc.c muxing for AVSubtitles with multiple rects, which needs a decision on this question: http://ffmpeg.org/pipermail/ffmpeg-devel/2023-February/306850.html

I think this patch set can be applied independently from your patch.
If you want to export ARIB captions to ASS format, specifying
`-ass_single_rect true` is sufficient as a tentative solution.
Of course positioning information would be lost, but is valid ASS format.
Your patch has worth to improve the output, but it can be resolved later.
  
> Once that's resolved, I'm happy to merge this.
> 
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
TADANO Tokumei March 23, 2023, 9:10 a.m. UTC | #3
On 2023/02/23 19:17, TADANO Tokumei wrote:
> Ping with rebased patch set.
> Some chages are added to reflect review results from outside of this ML.
> 
> ---
> This patch set add another ARIB caption decoder using libaribcaption
> external library: https://github.com/xqq/libaribcaption
> The library decodes subtitles of ISDB-based TV broadcasting.
> It is not only for Japanese ARIB STD-B24 caption, but also for
> Brazilian ABNT NBR 15606-1 and Philippines version of ISDB-T.
> 
> Unlike libaribb24, it supports 3 types of subtitle outputs:
> * text: plain text
> * ass: ASS formatted text
> * bitmap: bitmap image
> 
> Default subtitle type is ASS as same as libaribb24.
> Advantages compared with libaribb24 on ASS subtitle are:
> * Subtitle positioning.
> * Multi-rect subtitle: some captions are displayed at different
>    position at a time.
> * More stability and reproducibility.
> 
> Subtitle type can be changed by `-sub_type` option.
> You can see ARIB caption with ffplay tool:
>    ffplay -sub_type bitmap MPEG.TS
> 
> Sample files exist under:
>    https://streams.videolan.org/streams/ts/ARIB/
> Some of them are encrypted and some don't contain ARIB caption data.
> Good samples for ARIB caption are:
>    brazil/07-25_20-33-35_UCV - HD_.ts
>    japan/channel2[137]_clear.ts
>    japan/osaka_15.ts
> 
> ---
> v7: reflect review results from outside of this ML
>    - rename -ass_workaround option to -ass_single_rect and default to
>      false (latest mpv supports multiple ASS rectangle)
>    - add -canvas_size option to specify frame size to render bitmap subtitle
>    - add -caption_encoding option to specify encoding of subtitle text
>    - change behavior of profile C ARIB caption
>    - some cosmetic changes
> v6: reflect review results from outside of this ML
>    - check clut table overflow
>    - do not adjust vertical position of ruby for ASS format
> v5: reflect review comments from Mao Hata
>    - reset correct variable in aribcaption_close()
>    - add aribcaption_close() to some places in aribcaption_init()
>    - check if av_strdup() returns NULL in set_ass_header()
> v4: reflect review results from outside of this ML.
>    - resize bitmap subtitle image to display fonts with correct aspect ratio
>    - multiple font families can be specified by '-font' option
>    - remove 'rendering_backend' option
>    - add document
>    - minor bug fixes
> v3: combine former 1/4 and 2/4 due to the patchwork shows build error.
>    - fix help option content which incorrectly separated to 2 lines in 2/4.
>    - amend commit message of 4/4.
> 
> TADANO Tokumei (4):
>    lavc/libaribcaption.c: add ARIB caption decoder using libaribcaption
>    lavc/codec_desc.c: remove AV_CODEC_PROP_TEXT_SUB property
>    lavf/mpegts.c: set some properties for ARIB caption
>    doc/decoders.texi: add document of aribcaption decoder
> 
>   configure                   |    4 +
>   doc/decoders.texi           |  150 +++++
>   libavcodec/Makefile         |    1 +
>   libavcodec/allcodecs.c      |    1 +
>   libavcodec/codec_desc.c     |    1 -
>   libavcodec/libaribb24.c     |    1 +
>   libavcodec/libaribcaption.c | 1171 +++++++++++++++++++++++++++++++++++
>   libavformat/mpegts.c        |    6 +-
>   8 files changed, 1333 insertions(+), 2 deletions(-)
>   create mode 100644 libavcodec/libaribcaption.c
> 

ping!
Are there any other comments?

If nobody has objections, could anyone push the patch set?