mbox series

[FFmpeg-devel,v3,0/2] TTML in MP4, part 1

Message ID 20210726134104.42737-1-jeebjp@gmail.com
Headers show
Series TTML in MP4, part 1 | expand

Message

Jan Ekström July 26, 2021, 1:41 p.m. UTC 
To keep it simpler this first part includes only non-fragmented use
cases if the built-in sample squashing is utilized. Contains the basic
mapping code as well as the function that writes out the squashed documents.

Works with both subtitle-only output as well as with other streams.

In a separate patch set I will add additional complexity on top
by introducing fragmentation matching based on the other streams in
the mux (since most things that ingest fragmented MP4 with subtitles
want the fragment time ranges to match between the video/audio streams
and subtitles in the mux).

Changes from v2:

* Switched to the specified handler type (`subt`, which is the ISOBMFF subtitle
  handler type and should be utilized for XMLSubtitleSampleEntry) and
  SubtitleMediaHeaderBox (`sthd`, an empty all-zeroes FullBox) with MP4 style
  TTML. I did not notice this until now since most implementations seem to be
  fine with just the sample entry being correct.

  ISMV style TTML was getting the correct boxes written, `text` and `nmhd`.
* Switched the `dfxp` tag check from track->mode == MODE_MP4 to != MODE_ISM,
  as that matches better; After all, `dfxp` is only supposed to be utilized in
  that specific context.
* A trailing semicolon was removed in mov_write_ttml_document_from_queue.
* Applied changes based on Andreas' review:
** Using AVERROR_PATCHWELCOME when erroring out that fragmentation is not yet
   supported.
** movenc_ttml::mov_init_ttml_writer now returns error ASAP if the lavf context
   allocation fails making the context always a valid pointer after that in the
   function. As mov_init_ttml_writer handles its own clean-up, the error is then
   propagated right up in ff_mov_generate_squashed_ttml_packet as well.
** ffio_free_dyn_buf is now being utilized to free the muxer's dynamic buffer
   in movenc_ttml.
** movenc_ttml::mov_write_ttml_document_from_queue now utilizes a passed AVPacket
   instead of allocating its own.

Jan

Jan Ekström (2):
  avformat/ttml: split TTML paragraph based or not check into header
  avformat/movenc: add support for TTML muxing

 libavformat/Makefile             |   2 +-
 libavformat/isom.h               |   3 +
 libavformat/movenc.c             | 179 ++++++++++++++++++++++++++++++-
 libavformat/movenc.h             |   5 +
 libavformat/movenc_ttml.c        | 178 ++++++++++++++++++++++++++++++
 libavformat/movenc_ttml.h        |  31 ++++++
 libavformat/ttmlenc.c            |   9 +-
 libavformat/ttmlenc.h            |  39 +++++++
 tests/fate/subtitles.mak         |   4 +
 tests/ref/fate/sub-ttml-mp4-dfxp |  44 ++++++++
 tests/ref/fate/sub-ttml-mp4-stpp |  44 ++++++++
 11 files changed, 528 insertions(+), 10 deletions(-)
 create mode 100644 libavformat/movenc_ttml.c
 create mode 100644 libavformat/movenc_ttml.h
 create mode 100644 libavformat/ttmlenc.h
 create mode 100644 tests/ref/fate/sub-ttml-mp4-dfxp
 create mode 100644 tests/ref/fate/sub-ttml-mp4-stpp