diff mbox series

[FFmpeg-devel,2/2] avcodec: add an AVCodecContext flag to export A53/SCTE20/DVD CC side data on demand

Message ID 1651994221-11660-2-git-send-email-lance.lmwang@gmail.com
State New
Headers show
Series [FFmpeg-devel,1/2] avfilter/src_movie: add dec_opts for the opened file | expand

Checks

Context Check Description
yinshiyou/make_loongarch64 success Make finished
yinshiyou/make_fate_loongarch64 success Make fate finished
andriy/make_x86 success Make finished
andriy/make_fate_x86 success Make fate finished

Commit Message

Lance Wang May 8, 2022, 7:17 a.m. UTC 
From: Limin Wang <lance.lmwang@gmail.com>

some samples include both A53 and SCTE20 data. Before the commit, both
of the will be exported, so the CC data will be repeated or garbarge
as they're using the same frame side data. If you know your samples include
only one of them, You can export by +a53cc+scte20.

After the commit, the default will not export MPEG2 A53/SCTE20/DVD CC side data,
please export on demand.

Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
---
 doc/codecs.texi            | 10 ++++++++++
 libavcodec/avcodec.h       | 16 +++++++++++++++-
 libavcodec/mpeg12dec.c     |  6 +++---
 libavcodec/options_table.h |  3 +++
 libavcodec/version.h       |  2 +-
 tests/fate/ffmpeg.mak      |  2 +-
 tests/fate/subtitles.mak   |  6 +++---
 7 files changed, 36 insertions(+), 9 deletions(-)

Comments

Anton Khirnov May 11, 2022, noon UTC | #1 
Quoting lance.lmwang@gmail.com (2022-05-08 09:17:01)
> From: Limin Wang <lance.lmwang@gmail.com>
> 
> some samples include both A53 and SCTE20 data. Before the commit, both
> of the will be exported, so the CC data will be repeated or garbarge

Why would it be garbage? That sounds like a bug.
Why can't we just export both?
Lance Wang May 11, 2022, 1:30 p.m. UTC | #2 
On Wed, May 11, 2022 at 02:00:10PM +0200, Anton Khirnov wrote:
> Quoting lance.lmwang@gmail.com (2022-05-08 09:17:01)
> > From: Limin Wang <lance.lmwang@gmail.com>
> > 
> > some samples include both A53 and SCTE20 data. Before the commit, both
> > of the will be exported, so the CC data will be repeated or garbarge
> 
> Why would it be garbage? That sounds like a bug.
> Why can't we just export both?
for A53/SCTE20/DVDCC are exprted by A53_CC side data and they're sharing
the same a53_buf_ref to store the data, so if stream contains CC wrapped
as ATSC A53 packets + the same data wrapped as SCTE-20 packets, the CC
data will repeated. We can consider to add a new side data type if necessary,
but it's preferable to export A53_CC even it's SCTE20, so we can't export
both by A53_CC still.

related topic for it:
https://trac.ffmpeg.org/ticket/9724
https://patchwork.ffmpeg.org/project/ffmpeg/patch/20220427084949.73931-1-4rusxg@gmail.com/
https://patchwork.ffmpeg.org/project/ffmpeg/patch/20180306222512.20124-1-ffmpeg@tmm1.net/


> 
> -- 
> Anton Khirnov
Anton Khirnov May 19, 2022, 9:12 a.m. UTC | #3 
Quoting lance.lmwang@gmail.com (2022-05-11 15:30:37)
> On Wed, May 11, 2022 at 02:00:10PM +0200, Anton Khirnov wrote:
> > Quoting lance.lmwang@gmail.com (2022-05-08 09:17:01)
> > > From: Limin Wang <lance.lmwang@gmail.com>
> > > 
> > > some samples include both A53 and SCTE20 data. Before the commit, both
> > > of the will be exported, so the CC data will be repeated or garbarge
> > 
> > Why would it be garbage? That sounds like a bug.
> > Why can't we just export both?
> for A53/SCTE20/DVDCC are exprted by A53_CC side data and they're sharing
> the same a53_buf_ref to store the data, so if stream contains CC wrapped
> as ATSC A53 packets + the same data wrapped as SCTE-20 packets, the CC
> data will repeated. We can consider to add a new side data type if necessary,
> but it's preferable to export A53_CC even it's SCTE20, so we can't export
> both by A53_CC still.

Does this apply to anything other than mpeg2? If not, it should be a
codec-private option rather than a new global flag.
Lance Wang May 19, 2022, 12:31 p.m. UTC | #4 
On Thu, May 19, 2022 at 11:12:58AM +0200, Anton Khirnov wrote:
> Quoting lance.lmwang@gmail.com (2022-05-11 15:30:37)
> > On Wed, May 11, 2022 at 02:00:10PM +0200, Anton Khirnov wrote:
> > > Quoting lance.lmwang@gmail.com (2022-05-08 09:17:01)
> > > > From: Limin Wang <lance.lmwang@gmail.com>
> > > > 
> > > > some samples include both A53 and SCTE20 data. Before the commit, both
> > > > of the will be exported, so the CC data will be repeated or garbarge
> > > 
> > > Why would it be garbage? That sounds like a bug.
> > > Why can't we just export both?
> > for A53/SCTE20/DVDCC are exprted by A53_CC side data and they're sharing
> > the same a53_buf_ref to store the data, so if stream contains CC wrapped
> > as ATSC A53 packets + the same data wrapped as SCTE-20 packets, the CC
> > data will repeated. We can consider to add a new side data type if necessary,
> > but it's preferable to export A53_CC even it's SCTE20, so we can't export
> > both by A53_CC still.
> 
> Does this apply to anything other than mpeg2? If not, it should be a
> codec-private option rather than a new global flag.

Yes, it's for mpeg2 only. At first, I think export_side_data option is 
proper place to use. But if export_side_data are considered to use for
global, then I'll consider to add a mpeg2-private option for it.

> 
> -- 
> Anton Khirnov
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
diff mbox series

Patch

diff --git a/doc/codecs.texi b/doc/codecs.texi
index 5e10020900..4cced983b9 100644
--- a/doc/codecs.texi
+++ b/doc/codecs.texi
@@ -662,6 +662,16 @@  for codecs that support it. At present, those are H.264 and VP9.
 @item film_grain
 Export film grain parameters through frame side data (see @code{AV_FRAME_DATA_FILM_GRAIN_PARAMS}).
 Supported at present by AV1 decoders.
+@item a53cc
+Export A53 CC through frame side data (see @code{AV_FRAME_DATA_A53_CC})
+for codecs that support it.
+@item scte20cc
+Export SCTE20 CC through frame side data (see @code{AV_FRAME_DATA_A53_CC})
+for codecs that support it.
+@item dvdcc
+Export DVD CC through frame side data (see @code{AV_FRAME_DATA_A53_CC})
+for codecs that support it.
+
 @end table
 
 @item threads @var{integer} (@emph{decoding/encoding,video})
diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index 4dae23d06e..25fd4de2fe 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -360,7 +360,21 @@  typedef struct RcOverride{
  * Do not apply film grain, export it instead.
  */
 #define AV_CODEC_EXPORT_DATA_FILM_GRAIN (1 << 3)
-
+/**
+ * Decoding only.
+ * Export A53 CC through frame side data
+ */
+#define AV_CODEC_EXPORT_DATA_A53_CC (1 << 4)
+/**
+ * Decoding only.
+ * Export SCTE20 CC through frame side data
+ */
+#define AV_CODEC_EXPORT_DATA_SCTE20_CC (1 << 5)
+/**
+ * Decoding only.
+ * Export DVD CC through frame side data
+ */
+#define AV_CODEC_EXPORT_DATA_DVD_CC (1 << 6)
 /**
  * The decoder will keep a reference to the frame and may reuse it later.
  */
diff --git a/libavcodec/mpeg12dec.c b/libavcodec/mpeg12dec.c
index e9bde48f7a..032cb8f9b1 100644
--- a/libavcodec/mpeg12dec.c
+++ b/libavcodec/mpeg12dec.c
@@ -2203,7 +2203,7 @@  static int mpeg_decode_a53_cc(AVCodecContext *avctx,
 {
     Mpeg1Context *s1 = avctx->priv_data;
 
-    if (buf_size >= 6 &&
+    if (buf_size >= 6 && (avctx->export_side_data & AV_CODEC_EXPORT_DATA_A53_CC) &&
         p[0] == 'G' && p[1] == 'A' && p[2] == '9' && p[3] == '4' &&
         p[4] == 3 && (p[5] & 0x40)) {
         /* extract A53 Part 4 CC data */
@@ -2224,7 +2224,7 @@  static int mpeg_decode_a53_cc(AVCodecContext *avctx,
             avctx->properties |= FF_CODEC_PROPERTY_CLOSED_CAPTIONS;
         }
         return 1;
-    } else if (buf_size >= 2 &&
+    } else if (buf_size >= 2 && (avctx->export_side_data & AV_CODEC_EXPORT_DATA_SCTE20_CC) &&
                p[0] == 0x03 && (p[1]&0x7f) == 0x01) {
         /* extract SCTE-20 CC data */
         GetBitContext gb;
@@ -2269,7 +2269,7 @@  static int mpeg_decode_a53_cc(AVCodecContext *avctx,
             avctx->properties |= FF_CODEC_PROPERTY_CLOSED_CAPTIONS;
         }
         return 1;
-    } else if (buf_size >= 11 &&
+    } else if (buf_size >= 11 && (avctx->export_side_data & AV_CODEC_EXPORT_DATA_DVD_CC) &&
                p[0] == 'C' && p[1] == 'C' && p[2] == 0x01 && p[3] == 0xf8) {
         /* extract DVD CC data
          *
diff --git a/libavcodec/options_table.h b/libavcodec/options_table.h
index e72b4d12b6..3c6db07459 100644
--- a/libavcodec/options_table.h
+++ b/libavcodec/options_table.h
@@ -88,6 +88,9 @@  static const AVOption avcodec_options[] = {
 {"prft", "export Producer Reference Time through packet side data", 0, AV_OPT_TYPE_CONST, {.i64 = AV_CODEC_EXPORT_DATA_PRFT}, INT_MIN, INT_MAX, A|V|S|E, "export_side_data"},
 {"venc_params", "export video encoding parameters through frame side data", 0, AV_OPT_TYPE_CONST, {.i64 = AV_CODEC_EXPORT_DATA_VIDEO_ENC_PARAMS}, INT_MIN, INT_MAX, V|D, "export_side_data"},
 {"film_grain", "export film grain parameters through frame side data", 0, AV_OPT_TYPE_CONST, {.i64 = AV_CODEC_EXPORT_DATA_FILM_GRAIN}, INT_MIN, INT_MAX, V|D, "export_side_data"},
+{"a53cc", "export A53 CC through frame side data", 0, AV_OPT_TYPE_CONST, {.i64 = AV_CODEC_EXPORT_DATA_A53_CC}, INT_MIN, INT_MAX, V|D, "export_side_data"},
+{"scte20cc", "export SCTE20 CC through frame side data", 0, AV_OPT_TYPE_CONST, {.i64 = AV_CODEC_EXPORT_DATA_SCTE20_CC}, INT_MIN, INT_MAX, V|D, "export_side_data"},
+{"dvdcc", "export DVD CC through frame side data", 0, AV_OPT_TYPE_CONST, {.i64 = AV_CODEC_EXPORT_DATA_DVD_CC}, INT_MIN, INT_MAX, V|D, "export_side_data"},
 {"time_base", NULL, OFFSET(time_base), AV_OPT_TYPE_RATIONAL, {.dbl = 0}, 0, INT_MAX},
 {"g", "set the group of picture (GOP) size", OFFSET(gop_size), AV_OPT_TYPE_INT, {.i64 = 12 }, INT_MIN, INT_MAX, V|E},
 {"ar", "set audio sampling rate (in Hz)", OFFSET(sample_rate), AV_OPT_TYPE_INT, {.i64 = DEFAULT }, 0, INT_MAX, A|D|E},
diff --git a/libavcodec/version.h b/libavcodec/version.h
index 735c8b813c..87b7284a95 100644
--- a/libavcodec/version.h
+++ b/libavcodec/version.h
@@ -29,7 +29,7 @@ 
 
 #include "version_major.h"
 
-#define LIBAVCODEC_VERSION_MINOR  27
+#define LIBAVCODEC_VERSION_MINOR  28
 #define LIBAVCODEC_VERSION_MICRO 100
 
 #define LIBAVCODEC_VERSION_INT  AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \
diff --git a/tests/fate/ffmpeg.mak b/tests/fate/ffmpeg.mak
index 9d14a96e13..a403aa27a1 100644
--- a/tests/fate/ffmpeg.mak
+++ b/tests/fate/ffmpeg.mak
@@ -109,7 +109,7 @@  FATE_SAMPLES_FFMPEG-$(call ALLYES, LAVFI_INDEV MOVIE_FILTER FILE_PROTOCOL \
                            += fate-ffmpeg-fix_sub_duration
 fate-ffmpeg-fix_sub_duration: CMD = fmtstdout srt -fix_sub_duration \
   -real_time 1 -f lavfi \
-  -i "movie=$(TARGET_SAMPLES)/sub/Closedcaption_rollup.m2v[out0+subcc]"
+  -i "movie=$(TARGET_SAMPLES)/sub/Closedcaption_rollup.m2v:dec_opts=export_side_data=a53cc[out0+subcc]"
 
 FATE_STREAMCOPY-$(call ALLYES, EAC3_DEMUXER MOV_MUXER) += fate-copy-trac3074
 fate-copy-trac3074: $(SAMPLES)/eac3/csi_miami_stereo_128_spx.eac3
diff --git a/tests/fate/subtitles.mak b/tests/fate/subtitles.mak
index bc464edce6..bb1dc7434e 100644
--- a/tests/fate/subtitles.mak
+++ b/tests/fate/subtitles.mak
@@ -2,13 +2,13 @@  FATE_SUBTITLES_ASS-$(call ALLYES, AQTITLE_DEMUXER TEXT_DECODER ICONV) += fate-su
 fate-sub-aqtitle: CMD = fmtstdout ass -sub_charenc windows-1250 -i $(TARGET_SAMPLES)/sub/AQTitle_capability_tester.aqt
 
 FATE_SUBTITLES_ASS-$(call ALLYES, AVDEVICE LAVFI_INDEV CCAPTION_DECODER MOVIE_FILTER MPEGTS_DEMUXER) += fate-sub-cc
-fate-sub-cc: CMD = fmtstdout ass -f lavfi -i "movie=$(TARGET_SAMPLES)/sub/Closedcaption_rollup.m2v[out0+subcc]"
+fate-sub-cc: CMD = fmtstdout ass -f lavfi -i "movie=$(TARGET_SAMPLES)/sub/Closedcaption_rollup.m2v:dec_opts=export_side_data=a53cc[out0+subcc]"
 
 FATE_SUBTITLES_ASS-$(call ALLYES, AVDEVICE LAVFI_INDEV CCAPTION_DECODER MOVIE_FILTER MPEGTS_DEMUXER) += fate-sub-cc-realtime
-fate-sub-cc-realtime: CMD = fmtstdout ass -real_time 1 -f lavfi -i "movie=$(TARGET_SAMPLES)/sub/Closedcaption_rollup.m2v[out0+subcc]"
+fate-sub-cc-realtime: CMD = fmtstdout ass -real_time 1 -f lavfi -i "movie=$(TARGET_SAMPLES)/sub/Closedcaption_rollup.m2v:dec_opts=export_side_data=a53cc[out0+subcc]"
 
 FATE_SUBTITLES_ASS-$(call ALLYES, AVDEVICE LAVFI_INDEV CCAPTION_DECODER MOVIE_FILTER MPEGTS_DEMUXER) += fate-sub-cc-scte20
-fate-sub-cc-scte20: CMD = fmtstdout ass -f lavfi -i "movie=$(TARGET_SAMPLES)/sub/scte20.ts[out0+subcc]"
+fate-sub-cc-scte20: CMD = fmtstdout ass -f lavfi -i "movie=$(TARGET_SAMPLES)/sub/scte20.ts:dec_opts=export_side_data=scte20cc[out0+subcc]"
 
 FATE_SUBTITLES_ASS-$(call DEMDEC, ASS, ASS) += fate-sub-ass-to-ass-transcode
 fate-sub-ass-to-ass-transcode: CMD = fmtstdout ass -i $(TARGET_SAMPLES)/sub/1ededcbd7b.ass