[FFmpeg-devel] mpeg12enc: Use Closed Captions if available

Submitted by Mathieu Duponchelle on Feb. 7, 2019, 7:16 p.m.

Details

Message ID 20190207191659.435-1-mathieu@centricular.com
State New
Headers show

Commit Message

Mathieu Duponchelle Feb. 7, 2019, 7:16 p.m.
---
 doc/encoders.texi      |  3 +++
 libavcodec/mpeg12enc.c | 31 +++++++++++++++++++++++++++++++
 libavcodec/mpegvideo.h |  2 ++
 3 files changed, 36 insertions(+)

Comments

Carl Eugen Hoyos Feb. 7, 2019, 7:48 p.m.
2019-02-07 20:16 GMT+01:00, Mathieu Duponchelle <mathieu@centricular.com>:
> ---
>  doc/encoders.texi      |  3 +++
>  libavcodec/mpeg12enc.c | 31 +++++++++++++++++++++++++++++++
>  libavcodec/mpegvideo.h |  2 ++
>  3 files changed, 36 insertions(+)
>
> diff --git a/doc/encoders.texi b/doc/encoders.texi
> index e86ae69cc5..378a2ca8eb 100644
> --- a/doc/encoders.texi
> +++ b/doc/encoders.texi
> @@ -2574,6 +2574,9 @@ Specifies the video_format written into the sequence
> display extension
>  indicating the source of the video pictures. The default is
> @samp{unspecified},
>  can be @samp{component}, @samp{pal}, @samp{ntsc}, @samp{secam} or
> @samp{mac}.
>  For maximum compatibility, use @samp{component}.
> +@item a53cc @var{boolean}
> +Import closed captions (which must be ATSC compatible format) into output.

> +Only the mpeg2 and h264 decoders provide these.

Sorry for the late comment:
This is not a helpful sentence imo, many features are not provided
by all parts of FFmpeg and it is (too) difficult to keep such lists up-to-date.
Which other decoder are you thinking about?

>  Default is 1 (on).

>  @end table
>
>  @section png
> diff --git a/libavcodec/mpeg12enc.c b/libavcodec/mpeg12enc.c
> index d0b458e34b..0e2ab6da47 100644
> --- a/libavcodec/mpeg12enc.c
> +++ b/libavcodec/mpeg12enc.c
> @@ -61,6 +61,8 @@ static uint32_t mpeg1_chr_dc_uni[512];
>  static uint8_t mpeg1_index_run[2][64];
>  static int8_t  mpeg1_max_level[2][64];
>
> +#define A53_MAX_CC_COUNT 0x1f
> +
>  static av_cold void init_uni_ac_vlc(RLTable *rl, uint8_t *uni_ac_vlc_len)
>  {
>      int i;
> @@ -544,6 +546,35 @@ void ff_mpeg1_encode_picture_header(MpegEncContext *s,
> int picture_number)
>          }
>      }
>
> +    if (s->codec_id == AV_CODEC_ID_MPEG2VIDEO && s->a53_cc) {
> +        side_data = av_frame_get_side_data(s->current_picture_ptr->f,
> +            AV_FRAME_DATA_A53_CC);
> +        if (side_data) {
> +            if (side_data->size <= A53_MAX_CC_COUNT) {
> +                int i = 0;
> +
> +                put_header (s, USER_START_CODE);
> +
> +                put_bits(&s->pb, 8, 'G');                   //
> user_identifier
> +                put_bits(&s->pb, 8, 'A');
> +                put_bits(&s->pb, 8, '9');
> +                put_bits(&s->pb, 8, '4');
> +                put_bits(&s->pb, 8, 3);                     //
> user_data_type_code
> +                put_bits(&s->pb, 8,

> +                    ((side_data->size / 3) & A53_MAX_CC_COUNT) | 0x40); //
> flags, cc_count

If you decide to send another version, please remove the
superfluous brackets.

Carl Eugen
Mathieu Duponchelle Feb. 7, 2019, 7:58 p.m.
On 2/7/19 8:48 PM, Carl Eugen Hoyos wrote:
> +Only the mpeg2 and h264 decoders provide these.
> Sorry for the late comment:
> This is not a helpful sentence imo, many features are not provided
> by all parts of FFmpeg and it is (too) difficult to keep such lists up-to-date.
> Which other decoder are you thinking about?

This was simply copy pasted from the "libx264, libx264rgb" section
(currently l2272), I'll remove it

> If you decide to send another version, please remove the
> superfluous brackets.

Sure
Gyan Feb. 7, 2019, 8:07 p.m.
On 08-02-2019 01:18 AM, Carl Eugen Hoyos wrote:
> 2019-02-07 20:16 GMT+01:00, Mathieu Duponchelle <mathieu@centricular.com>:
>> ---
>>   doc/encoders.texi      |  3 +++
>>   libavcodec/mpeg12enc.c | 31 +++++++++++++++++++++++++++++++
>>   libavcodec/mpegvideo.h |  2 ++
>>   3 files changed, 36 insertions(+)
>>
>> diff --git a/doc/encoders.texi b/doc/encoders.texi
>> index e86ae69cc5..378a2ca8eb 100644
>> --- a/doc/encoders.texi
>> +++ b/doc/encoders.texi
>> @@ -2574,6 +2574,9 @@ Specifies the video_format written into the sequence
>> display extension
>>   indicating the source of the video pictures. The default is
>> @samp{unspecified},
>>   can be @samp{component}, @samp{pal}, @samp{ntsc}, @samp{secam} or
>> @samp{mac}.
>>   For maximum compatibility, use @samp{component}.
>> +@item a53cc @var{boolean}
>> +Import closed captions (which must be ATSC compatible format) into output.
>> +Only the mpeg2 and h264 decoders provide these.
> Sorry for the late comment:
> This is not a helpful sentence imo, many features are not provided
> by all parts of FFmpeg and it is (too) difficult to keep such lists up-to-date.
> Which other decoder are you thinking about?
>
Let it remain. It is informative, and an extremely short list. Can be 
updated as needed.

Gyan
Carl Eugen Hoyos Feb. 7, 2019, 8:09 p.m.
2019-02-07 21:07 GMT+01:00, Gyan <ffmpeg@gyani.pro>:
>
>
> On 08-02-2019 01:18 AM, Carl Eugen Hoyos wrote:
>> 2019-02-07 20:16 GMT+01:00, Mathieu Duponchelle <mathieu@centricular.com>:
>>> ---
>>>   doc/encoders.texi      |  3 +++
>>>   libavcodec/mpeg12enc.c | 31 +++++++++++++++++++++++++++++++
>>>   libavcodec/mpegvideo.h |  2 ++
>>>   3 files changed, 36 insertions(+)
>>>
>>> diff --git a/doc/encoders.texi b/doc/encoders.texi
>>> index e86ae69cc5..378a2ca8eb 100644
>>> --- a/doc/encoders.texi
>>> +++ b/doc/encoders.texi
>>> @@ -2574,6 +2574,9 @@ Specifies the video_format written into the
>>> sequence
>>> display extension
>>>   indicating the source of the video pictures. The default is
>>> @samp{unspecified},
>>>   can be @samp{component}, @samp{pal}, @samp{ntsc}, @samp{secam} or
>>> @samp{mac}.
>>>   For maximum compatibility, use @samp{component}.
>>> +@item a53cc @var{boolean}
>>> +Import closed captions (which must be ATSC compatible format) into
>>> output.
>>> +Only the mpeg2 and h264 decoders provide these.
>> Sorry for the late comment:
>> This is not a helpful sentence imo, many features are not provided
>> by all parts of FFmpeg and it is (too) difficult to keep such lists
>> up-to-date.
>> Which other decoder are you thinking about?
>>
> Let it remain. It is informative

How is it informative?
Which other decoders are you thinking about?

Carl Eugen
Gyan Feb. 7, 2019, 9:15 p.m.
On 08-02-2019 01:39 AM, Carl Eugen Hoyos wrote:
> 2019-02-07 21:07 GMT+01:00, Gyan <ffmpeg@gyani.pro>:
>>
>> On 08-02-2019 01:18 AM, Carl Eugen Hoyos wrote:
>>> 2019-02-07 20:16 GMT+01:00, Mathieu Duponchelle <mathieu@centricular.com>:
>>>> ---
>>>>    doc/encoders.texi      |  3 +++
>>>>    libavcodec/mpeg12enc.c | 31 +++++++++++++++++++++++++++++++
>>>>    libavcodec/mpegvideo.h |  2 ++
>>>>    3 files changed, 36 insertions(+)
>>>>
>>>> diff --git a/doc/encoders.texi b/doc/encoders.texi
>>>> index e86ae69cc5..378a2ca8eb 100644
>>>> --- a/doc/encoders.texi
>>>> +++ b/doc/encoders.texi
>>>> @@ -2574,6 +2574,9 @@ Specifies the video_format written into the
>>>> sequence
>>>> display extension
>>>>    indicating the source of the video pictures. The default is
>>>> @samp{unspecified},
>>>>    can be @samp{component}, @samp{pal}, @samp{ntsc}, @samp{secam} or
>>>> @samp{mac}.
>>>>    For maximum compatibility, use @samp{component}.
>>>> +@item a53cc @var{boolean}
>>>> +Import closed captions (which must be ATSC compatible format) into
>>>> output.
>>>> +Only the mpeg2 and h264 decoders provide these.
>>> Sorry for the late comment:
>>> This is not a helpful sentence imo, many features are not provided
>>> by all parts of FFmpeg and it is (too) difficult to keep such lists
>>> up-to-date.
>>> Which other decoder are you thinking about?
>>>
>> Let it remain. It is informative
> How is it informative?
> Which other decoders are you thinking about?

I don't see other h264 decoders, as used within ffmpeg, exporting CC 
side-data, e.g. OpenH264 or QSV.

(I do see native hevc doing so, as well as the Decklink indev , but not 
mentioned here.)

Full coverage is difficult, but that shouldn't be used as a reason to 
suppress any coverage.

Gyan
Carl Eugen Hoyos Feb. 7, 2019, 9:48 p.m.
2019-02-07 22:15 GMT+01:00, Gyan <ffmpeg@gyani.pro>:

> I don't see other h264 decoders, as used within ffmpeg, exporting CC
> side-data, e.g. OpenH264 or QSV.

Do they provide it?

> (I do see native hevc doing so, as well as the Decklink indev , but not
> mentioned here.)

Seems like a good reason to remove the useless information.

Carl Eugen
Mathieu Duponchelle Feb. 7, 2019, 10:34 p.m.
On 2/7/19 10:48 PM, Carl Eugen Hoyos wrote:
> Seems like a good reason to remove the useless information.

Done in my latest patch for what it's worth, I also don't think it's very important
information, plus CC can also be provided by the motivated user through the
new_side_data() API anyway :)
Gyan Feb. 8, 2019, 5:12 a.m.
On 08-02-2019 03:18 AM, Carl Eugen Hoyos wrote:
> 2019-02-07 22:15 GMT+01:00, Gyan <ffmpeg@gyani.pro>:
>
>> I don't see other h264 decoders, as used within ffmpeg, exporting CC
>> side-data, e.g. OpenH264 or QSV.
> Do they provide it?

QSV offers a method to retrieve it but our wrapper doesn't.

>> (I do see native hevc doing so, as well as the Decklink indev , but not
>> mentioned here.)
> Seems like a good reason to remove the useless information.
>
Just means a bit more diligence needed before writing docs, not skipping it.

Gyan

Patch hide | download patch | download mbox

diff --git a/doc/encoders.texi b/doc/encoders.texi
index e86ae69cc5..378a2ca8eb 100644
--- a/doc/encoders.texi
+++ b/doc/encoders.texi
@@ -2574,6 +2574,9 @@  Specifies the video_format written into the sequence display extension
 indicating the source of the video pictures. The default is @samp{unspecified},
 can be @samp{component}, @samp{pal}, @samp{ntsc}, @samp{secam} or @samp{mac}.
 For maximum compatibility, use @samp{component}.
+@item a53cc @var{boolean}
+Import closed captions (which must be ATSC compatible format) into output.
+Only the mpeg2 and h264 decoders provide these. Default is 1 (on).
 @end table
 
 @section png
diff --git a/libavcodec/mpeg12enc.c b/libavcodec/mpeg12enc.c
index d0b458e34b..0e2ab6da47 100644
--- a/libavcodec/mpeg12enc.c
+++ b/libavcodec/mpeg12enc.c
@@ -61,6 +61,8 @@  static uint32_t mpeg1_chr_dc_uni[512];
 static uint8_t mpeg1_index_run[2][64];
 static int8_t  mpeg1_max_level[2][64];
 
+#define A53_MAX_CC_COUNT 0x1f
+
 static av_cold void init_uni_ac_vlc(RLTable *rl, uint8_t *uni_ac_vlc_len)
 {
     int i;
@@ -544,6 +546,35 @@  void ff_mpeg1_encode_picture_header(MpegEncContext *s, int picture_number)
         }
     }
 
+    if (s->codec_id == AV_CODEC_ID_MPEG2VIDEO && s->a53_cc) {
+        side_data = av_frame_get_side_data(s->current_picture_ptr->f,
+            AV_FRAME_DATA_A53_CC);
+        if (side_data) {
+            if (side_data->size <= A53_MAX_CC_COUNT) {
+                int i = 0;
+
+                put_header (s, USER_START_CODE);
+
+                put_bits(&s->pb, 8, 'G');                   // user_identifier
+                put_bits(&s->pb, 8, 'A');
+                put_bits(&s->pb, 8, '9');
+                put_bits(&s->pb, 8, '4');
+                put_bits(&s->pb, 8, 3);                     // user_data_type_code
+                put_bits(&s->pb, 8,
+                    ((side_data->size / 3) & A53_MAX_CC_COUNT) | 0x40); // flags, cc_count
+                put_bits(&s->pb, 8, 0xff);                  // em_data
+
+                for (i = 0; i < side_data->size; i++)
+                    put_bits(&s->pb, 8, side_data->data[i]);
+
+                put_bits(&s->pb, 8, 0xff);                  // marker_bits
+            } else {
+                av_log(s->avctx, AV_LOG_WARNING,
+                    "Warning Closed Caption size can not exceed 93 bytes\n");
+            }
+        }
+    }
+
     s->mb_y = 0;
     ff_mpeg1_encode_slice_header(s);
 }
diff --git a/libavcodec/mpegvideo.h b/libavcodec/mpegvideo.h
index bbc6b5646a..3e52f98390 100644
--- a/libavcodec/mpegvideo.h
+++ b/libavcodec/mpegvideo.h
@@ -455,6 +455,7 @@  typedef struct MpegEncContext {
     /* MPEG-2-specific - I wished not to have to support this mess. */
     int progressive_sequence;
     int mpeg_f_code[2][2];
+    int a53_cc;
 
     // picture structure defines are loaded from mpegutils.h
     int picture_structure;
@@ -663,6 +664,7 @@  FF_MPV_OPT_CMP_FUNC, \
 {"ps", "RTP payload size in bytes",                             FF_MPV_OFFSET(rtp_payload_size), AV_OPT_TYPE_INT, {.i64 = 0 }, INT_MIN, INT_MAX, FF_MPV_OPT_FLAGS }, \
 {"mepc", "Motion estimation bitrate penalty compensation (1.0 = 256)", FF_MPV_OFFSET(me_penalty_compensation), AV_OPT_TYPE_INT, {.i64 = 256 }, INT_MIN, INT_MAX, FF_MPV_OPT_FLAGS }, \
 {"mepre", "pre motion estimation", FF_MPV_OFFSET(me_pre), AV_OPT_TYPE_INT, {.i64 = 0 }, INT_MIN, INT_MAX, FF_MPV_OPT_FLAGS }, \
+{ "a53cc", "Use A53 Closed Captions (if available)", FF_MPV_OFFSET(a53_cc), AV_OPT_TYPE_BOOL, {.i64 = 1}, 0, 1, FF_MPV_OPT_FLAGS }, \
 
 extern const AVOption ff_mpv_generic_options[];