mbox series

[FFmpeg-devel,RFC,0/5] Properly handle CEA-708 caption data when transcoding

Message ID 20230317200941.3936-1-dheitmueller@ltnglobal.com
Headers show
Series Properly handle CEA-708 caption data when transcoding | expand

Message

Devin Heitmueller March 17, 2023, 8:09 p.m. UTC
This patch series is intended to address long-standing known issues where
CEA-708 caption data is either duplicated or corrupted (i.e. 50% of the data
is lost).  We employ an intermediate queue, which is available to filters that
change the frame rate of the video, which stashes the 608/708 data and
reconstructs properly formed 708 tuples on the output side (injecting
appropriate padding as needed).

The common functions are implemented in libavutil, but I am open to
suggestions.  While all the users in this patch series are within
libavfilter, there are use cases where this functionality would
be useful within libavformat (which is why libavutil was chosen).

Comments/feedback are welcomed.

Thanks,

Devin

Devin Heitmueller (5):
  ccfifo: Properly handle CEA-708 captions through framerate conversion
  vf_fps: properly preserve CEA-708 captions
  yadif: Properly preserve CEA-708 closed captions
  tinterlace: Properly preserve CEA-708 closed captions
  vf_ccrepack: Add new filter to repack CEA-708 side data

 doc/filters.texi            |  10 ++
 libavfilter/Makefile        |   1 +
 libavfilter/allfilters.c    |   1 +
 libavfilter/tinterlace.h    |   2 +
 libavfilter/vf_bwdif.c      |   7 ++
 libavfilter/vf_ccrepack.c   |  95 ++++++++++++++++++
 libavfilter/vf_fps.c        |   9 +-
 libavfilter/vf_tinterlace.c |  10 ++
 libavfilter/vf_yadif.c      |   6 ++
 libavfilter/vf_yadif_cuda.c |   8 ++
 libavfilter/yadif.h         |   2 +
 libavfilter/yadif_common.c  |   5 +
 libavutil/Makefile          |   2 +
 libavutil/ccfifo.c          | 192 ++++++++++++++++++++++++++++++++++++
 libavutil/ccfifo.h          |  85 ++++++++++++++++
 15 files changed, 434 insertions(+), 1 deletion(-)
 create mode 100644 libavfilter/vf_ccrepack.c
 create mode 100644 libavutil/ccfifo.c
 create mode 100644 libavutil/ccfifo.h

Comments

BUCCIANTINI Francesco - ADECCO March 17, 2023, 11:23 p.m. UTC | #1
This is indeed very welcome.
Being able to re-encode from 25i to 50p while preserving subtitles has been something I've been trying to do for a while, so seeing this patch for CEA-708 is definitely a nice thing.

I have a question, though: will something similar be done for OP-47 subtitles too?
Currently they suffer the same issue when a re-encode changes the frame rate... :(

On Mar 17, 2023 21:10, Devin Heitmueller <devin.heitmueller@ltnglobal.com> wrote:
ATTENZIONE: mail esterna. Se il messaggio è sospetto, invialo come allegato a bedatasmart@skytv.it<mailto:bedatasmart@skytv.it>


This patch series is intended to address long-standing known issues where
CEA-708 caption data is either duplicated or corrupted (i.e. 50% of the data
is lost).  We employ an intermediate queue, which is available to filters that
change the frame rate of the video, which stashes the 608/708 data and
reconstructs properly formed 708 tuples on the output side (injecting
appropriate padding as needed).

The common functions are implemented in libavutil, but I am open to
suggestions.  While all the users in this patch series are within
libavfilter, there are use cases where this functionality would
be useful within libavformat (which is why libavutil was chosen).

Comments/feedback are welcomed.

Thanks,

Devin

Devin Heitmueller (5):
  ccfifo: Properly handle CEA-708 captions through framerate conversion
  vf_fps: properly preserve CEA-708 captions
  yadif: Properly preserve CEA-708 closed captions
  tinterlace: Properly preserve CEA-708 closed captions
  vf_ccrepack: Add new filter to repack CEA-708 side data

 doc/filters.texi            |  10 ++
 libavfilter/Makefile        |   1 +
 libavfilter/allfilters.c    |   1 +
 libavfilter/tinterlace.h    |   2 +
 libavfilter/vf_bwdif.c      |   7 ++
 libavfilter/vf_ccrepack.c   |  95 ++++++++++++++++++
 libavfilter/vf_fps.c        |   9 +-
 libavfilter/vf_tinterlace.c |  10 ++
 libavfilter/vf_yadif.c      |   6 ++
 libavfilter/vf_yadif_cuda.c |   8 ++
 libavfilter/yadif.h         |   2 +
 libavfilter/yadif_common.c  |   5 +
 libavutil/Makefile          |   2 +
 libavutil/ccfifo.c          | 192 ++++++++++++++++++++++++++++++++++++
 libavutil/ccfifo.h          |  85 ++++++++++++++++
 15 files changed, 434 insertions(+), 1 deletion(-)
 create mode 100644 libavfilter/vf_ccrepack.c
 create mode 100644 libavutil/ccfifo.c
 create mode 100644 libavutil/ccfifo.h

--
2.35.1.655.ga68dfadae5
Devin Heitmueller March 18, 2023, 11:16 p.m. UTC | #2
On Fri, Mar 17, 2023 at 7:23 PM BUCCIANTINI Francesco - ADECCO
<francesco.bucciantini.adecco@skytv.it> wrote:
>
> This is indeed very welcome.
> Being able to re-encode from 25i to 50p while preserving subtitles has been something I've been trying to do for a while, so seeing this patch for CEA-708 is definitely a nice thing.
>
> I have a question, though: will something similar be done for OP-47 subtitles too?
> Currently they suffer the same issue when a re-encode changes the frame rate... :(

I don't have any plans to work on OP-47, and I'm not sure if anyone
else does.  While I've done a bit of work with it in the SDI domain,
I'm not super familiar with how it's treated by ffmpeg, or whether a
similar approach is appropriate for dealing with it.  I can say that
the FIFO here is highly specific to the CEA-708 standard, as we have
to disassemble and reassemble the payloads to meet the specification
(i.e. not applicable to OP-47).

Devin