mbox

[FFmpeg-devel,v5,00/12] Subtitle Filtering

Message ID MN2PR04MB598163F7245146B8734BEF02BAD89@MN2PR04MB5981.namprd04.prod.outlook.com
Headers show

Message

Soft Works Sept. 12, 2021, 3:21 a.m. UTC 
v5 Update:

- Merge AVSubtitle into AVFrame
- Move FATE test adjustments to corresponding commit
- Move documentation updates to corresponding filter commits
- Remove mediatype parameter from av_frame_get_buffer2

(still need some advice for splitting the commits in a way that 
every single one will pass FATE)

v4 Update:

- Re-Sending due to Patchwork having failed to parse my patchset
  There seems to be a bug in Patchwork when parallel processing is 
  enabled. 
  This time, I'll send the e-mails slowly, one after another.

v3 Update:

- Reworked, revised, rebased, reorganized, refactored
- No more prototype/test-style code
- Additional subtitle filters added
- Filter documentation added
- Adjusted FATE tests


This patchset is about introducing filtering support for subtitles.

The current sub2video "hack" implementation is slow, ineffective and 
limited in capabilities.

=> This patchset introduces true subtitle filtering

These new filters are included:

- overlay_graphicsubs (VS -> V)
  Overlay graphic subtitles onto a video stream

- graphicsub2video {S -> V)
  Converts graphic subtitles to (transparent) video frames
  
- overlay_textsubs {VS -> V)
  Overlay text subtitles onto a video stream.
  
- textsubs2video {S -> V)
  Converts text subtitles to video frames
  
- textmod {S -> S)
  Modify subtitle text in a number of ways

- stripstyles {S -> S)
  Remove all inline styles from subtitle events

Regards,
softworkz


softworkz (12):
  avutil/frame: Subtitle Filtering - Add AVMediaType property to AVFrame
  global: Merge AVSubtitle into AVFrame
  fftools/play,probe: Adjust for subtitle format type change
  fftools/ffmpeg: Replace sub2video with subtitle frame filtering
  avfilter/subtitles: Add subtitles.c
  avfilter/avfilter: Handle subtitle frames
  avfilter/sbuffer: Add sbuffersrv and sbuffersink filters
  avfilter/overlay_graphicsubs: Add overlay_graphicsubs and
    graphicsub2video filters
  avfilter/overlay_textsubs: Add overlay_textsubs and textsubs2video
    filters
  avfilter/textmod: Add textmod filter
  avcodec/ass_split: Extend ass dialog parsing
  avfilter/stripstyles: Add stripstyles filter

 configure                                 |   4 +-
 doc/filters.texi                          | 261 ++++++++
 fftools/ffmpeg.c                          | 435 ++++++-------
 fftools/ffmpeg.h                          |  14 +-
 fftools/ffmpeg_filter.c                   | 198 ++++--
 fftools/ffmpeg_hw.c                       |   2 +-
 fftools/ffmpeg_opt.c                      |   3 +-
 fftools/ffplay.c                          |  30 +-
 fftools/ffprobe.c                         |  49 +-
 libavcodec/ass.c                          |  16 +-
 libavcodec/ass.h                          |   8 +-
 libavcodec/ass_split.c                    |  12 +-
 libavcodec/ass_split.h                    |   2 +
 libavcodec/assdec.c                       |  18 +-
 libavcodec/assenc.c                       |  10 +-
 libavcodec/avcodec.c                      |  19 -
 libavcodec/avcodec.h                      |  81 +--
 libavcodec/ccaption_dec.c                 |  28 +-
 libavcodec/codec.h                        |   4 +-
 libavcodec/codec_desc.h                   |   4 +-
 libavcodec/decode.c                       |  23 +-
 libavcodec/dvbsubdec.c                    |  48 +-
 libavcodec/dvbsubenc.c                    |  72 +--
 libavcodec/dvdsubdec.c                    | 122 ++--
 libavcodec/dvdsubenc.c                    |  40 +-
 libavcodec/encode.c                       |   6 +-
 libavcodec/jacosubdec.c                   |   4 +-
 libavcodec/libzvbi-teletextdec.c          |  16 +-
 libavcodec/microdvddec.c                  |   4 +-
 libavcodec/movtextdec.c                   |   4 +-
 libavcodec/movtextenc.c                   |  10 +-
 libavcodec/mpl2dec.c                      |   4 +-
 libavcodec/pgssubdec.c                    |  68 +-
 libavcodec/realtextdec.c                  |   4 +-
 libavcodec/samidec.c                      |   4 +-
 libavcodec/srtdec.c                       |   4 +-
 libavcodec/srtenc.c                       |  14 +-
 libavcodec/subviewerdec.c                 |   4 +-
 libavcodec/textdec.c                      |   4 +-
 libavcodec/ttmlenc.c                      |  10 +-
 libavcodec/utils.c                        |  11 +
 libavcodec/webvttdec.c                    |   4 +-
 libavcodec/webvttenc.c                    |  10 +-
 libavcodec/xsubdec.c                      |  58 +-
 libavcodec/xsubenc.c                      |  48 +-
 libavfilter/Makefile                      |   9 +
 libavfilter/allfilters.c                  |  16 +-
 libavfilter/avfilter.c                    |  26 +-
 libavfilter/avfiltergraph.c               |   5 +
 libavfilter/buffersink.c                  |  63 ++
 libavfilter/buffersink.h                  |  15 +
 libavfilter/buffersrc.c                   |  72 +++
 libavfilter/buffersrc.h                   |   1 +
 libavfilter/f_interleave.c                |   3 +
 libavfilter/formats.c                     |  14 +
 libavfilter/formats.h                     |   3 +
 libavfilter/internal.h                    |   1 +
 libavfilter/sf_stripstyles.c              | 215 +++++++
 libavfilter/sf_textmod.c                  | 381 +++++++++++
 libavfilter/subtitles.c                   |  61 ++
 libavfilter/subtitles.h                   |  44 ++
 libavfilter/version.h                     |   2 +-
 libavfilter/vf_overlay_graphicsubs.c      | 731 ++++++++++++++++++++++
 libavfilter/vf_overlay_textsubs.c         | 633 +++++++++++++++++++
 libavfilter/vf_subtitles.c                |  28 +-
 libavformat/utils.c                       |   5 +-
 libavutil/Makefile                        |   2 +
 libavutil/frame.c                         | 126 +++-
 libavutil/frame.h                         |  51 +-
 libavutil/subfmt.c                        |  52 ++
 libavutil/subfmt.h                        |  94 +++
 libavutil/version.h                       |   2 +-
 tests/ref/fate/filter-overlay-dvdsub-2397 | 181 +++---
 tests/ref/fate/sub-dvb                    | 162 ++---
 tests/ref/fate/sub2video                  | 178 ++----
 tests/ref/fate/sub2video_basic            |  93 ++-
 tests/ref/fate/sub2video_time_limited     |   6 +-
 77 files changed, 3914 insertions(+), 1155 deletions(-)
 create mode 100644 libavfilter/sf_stripstyles.c
 create mode 100644 libavfilter/sf_textmod.c
 create mode 100644 libavfilter/subtitles.c
 create mode 100644 libavfilter/subtitles.h
 create mode 100644 libavfilter/vf_overlay_graphicsubs.c
 create mode 100644 libavfilter/vf_overlay_textsubs.c
 create mode 100644 libavutil/subfmt.c
 create mode 100644 libavutil/subfmt.h