diff mbox series

[FFmpeg-devel,v4,1/6] ccfifo: Properly handle CEA-708 captions through framerate conversion

Message ID 1682699871-22331-2-git-send-email-dheitmueller@ltnglobal.com
State New
Headers show
Series Add support for Closed Caption FIFO | expand

Checks

Context Check Description
andriy/make_x86 success Make finished
andriy/make_fate_x86 success Make fate finished

Commit Message

Devin Heitmueller April 28, 2023, 4:37 p.m. UTC 
When transcoding video that contains 708 closed captions, the
caption data is tied to the frames as side data.  Simply dropping
or adding frames to change the framerate will result in loss of
data, so the caption data needs to be preserved and reformatted.

For example, without this patch converting 720p59 to 1080i59
would result in loss of 50% of the caption bytes, resulting in
garbled 608 captions and 708 probably wouldn't render at all.
Further, the frames that are there will have an illegal
cc_count for the target framerate, so some decoders may ignore
the packets entirely.

Extract the 608 and 708 tuples and insert them onto queues.  Then
after dropping/adding frames, re-write the tuples back into the
resulting frames at the appropriate rate given the target
framerate.  This includes both having the correct cc_count as
well as clocking out the 608 pairs at the appropriate rate.

Thanks for Lance Wang <lance.lmwang@gmail.com> for providing
review/feedback.

Signed-off-by: Devin Heitmueller <dheitmueller@ltnglobal.com>
---
 libavfilter/Makefile |   1 +
 libavfilter/ccfifo.c | 240 +++++++++++++++++++++++++++++++++++++++++++++++++++
 libavfilter/ccfifo.h |  94 ++++++++++++++++++++
 3 files changed, 335 insertions(+)
 create mode 100644 libavfilter/ccfifo.c
 create mode 100644 libavfilter/ccfifo.h

Comments

Anton Khirnov May 3, 2023, 9:20 a.m. UTC | #1 
Quoting Devin Heitmueller (2023-04-28 18:37:46)
> +void ff_ccfifo_freep(AVCCFifo **ccf)
> +{
> +    if (ccf && *ccf) {

Don't check for ccf, it makes no sense to call this function with
ccf==NULL, so silently ignoring it can hide bugs.

> +        AVCCFifo *tmp = *ccf;
> +        if (tmp->cc_608_fifo)
> +            av_fifo_freep2(&tmp->cc_608_fifo);

Useless checks, av_fifo_freep2 can be called on &NULL.

> +        if (tmp->cc_708_fifo)
> +            av_fifo_freep2(&tmp->cc_708_fifo);
> +        av_freep(*ccf);
> +    }
> +}
> +
> +AVCCFifo *ff_ccfifo_alloc(AVRational *framerate, void *log_ctx)

We typically pass AVRational directly, not as pointers. That also makes
it clear that the passed value is not modified.

> +{
> +    AVCCFifo *ccf;
> +    int i;
> +
> +    ccf = av_mallocz(sizeof(*ccf));
> +    if (!ccf)
> +        return NULL;
> +
> +    if (!(ccf->cc_708_fifo = av_fifo_alloc2(MAX_CC_ELEMENTS, CC_BYTES_PER_ENTRY, 0)))
> +        goto error;
> +
> +    if (!(ccf->cc_608_fifo = av_fifo_alloc2(MAX_CC_ELEMENTS, CC_BYTES_PER_ENTRY, 0)))
> +        goto error;
> +
> +    /* Based on the target FPS, figure out the expected cc_count and number of
> +       608 tuples per packet.  See ANSI/CTA-708-E Sec 4.3.6.1. */
> +    for (i = 0; i < FF_ARRAY_ELEMS(cc_lookup_vals); i++) {
> +        if (framerate->num == cc_lookup_vals[i].num &&
> +            framerate->den == cc_lookup_vals[i].den) {
> +            ccf->expected_cc_count = cc_lookup_vals[i].cc_count;
> +            ccf->expected_608 = cc_lookup_vals[i].num_608;
> +            break;
> +        }
> +    }
> +
> +    if (ccf->expected_608 == 0) {
> +        /* We didn't find an output frame we support.  We'll let the call succeed
> +           and the FIFO to be allocated, but the extract/inject functions will simply
> +           leave everything the way it is */
> +        av_log(ccf->log_ctx, AV_LOG_WARNING, "cc_fifo cannot transcode captions fps=%d/%d\n",
> +               framerate->num, framerate->den);

Won't this result in spurious warnings for users who are just converting
framerates and don't have any captions. If so it should probably be
printed the first time we actually see caption side data.

> +        ccf->passthrough = 1;
> +    }
> +
> +    return ccf;
> +
> +error:
> +    ff_ccfifo_freep(&ccf);
> +    return NULL;
> +}
> +
> +int ff_ccfifo_injectbytes(AVCCFifo *ccf, uint8_t **data, size_t *len)
> +{
> +    char *cc_data;
> +    int cc_filled = 0;
> +    int i;
> +
> +    if (!ccf)
> +        return AVERROR(EINVAL);

For all the extract/inject functions: it should be invalid to call them
with a NULL context, so this should be an av_assert0() or not be here at
all.

> +
> +    if (ccf->passthrough) {
> +        *data = NULL;
> +        *len = 0;
> +        return 0;
> +    }
> +
> +    cc_data = av_mallocz(ccf->expected_cc_count * CC_BYTES_PER_ENTRY);

This buffer size is constant for a given AVCCFifo object, so perhaps
ff_ccfifo_alloc() could return required buffer size and this function
could write into a user-provided buffer and avoid constant dynamic
allocation.

> +    if (!cc_data) {
> +        return AVERROR(ENOMEM);
> +    }
> +
> +    for (i = 0; i < ccf->expected_608; i++) {
> +        if (av_fifo_can_read(ccf->cc_608_fifo) >= CC_BYTES_PER_ENTRY) {
> +            av_fifo_read(ccf->cc_608_fifo, &cc_data[cc_filled * CC_BYTES_PER_ENTRY],
> +                         CC_BYTES_PER_ENTRY);

This looks wrong, as fifo operations are in elements, and your
elements are already CC_BYTES_PER_ENTRY. So every read actually writes
CC_BYTES_PER_ENTRY * CC_BYTES_PER_ENTRY bytes to cc_data.

I think you can also do this as a single av_fifo_read() call without a
loop.
Devin Heitmueller May 3, 2023, 1:15 p.m. UTC | #2 
Hi Anton,

Thanks for your feedback.  Comments inline:

On Wed, May 3, 2023 at 5:20 AM Anton Khirnov <anton@khirnov.net> wrote:
>
> Quoting Devin Heitmueller (2023-04-28 18:37:46)
> > +void ff_ccfifo_freep(AVCCFifo **ccf)
> > +{
> > +    if (ccf && *ccf) {
>
> Don't check for ccf, it makes no sense to call this function with
> ccf==NULL, so silently ignoring it can hide bugs.

Ok.

> > +        AVCCFifo *tmp = *ccf;
> > +        if (tmp->cc_608_fifo)
> > +            av_fifo_freep2(&tmp->cc_608_fifo);
>
> Useless checks, av_fifo_freep2 can be called on &NULL.

Ok.

> > +        if (tmp->cc_708_fifo)
> > +            av_fifo_freep2(&tmp->cc_708_fifo);
> > +        av_freep(*ccf);
> > +    }
> > +}
> > +
> > +AVCCFifo *ff_ccfifo_alloc(AVRational *framerate, void *log_ctx)
>
> We typically pass AVRational directly, not as pointers. That also makes
> it clear that the passed value is not modified.

Ok.

> > +{
> > +    AVCCFifo *ccf;
> > +    int i;
> > +
> > +    ccf = av_mallocz(sizeof(*ccf));
> > +    if (!ccf)
> > +        return NULL;
> > +
> > +    if (!(ccf->cc_708_fifo = av_fifo_alloc2(MAX_CC_ELEMENTS, CC_BYTES_PER_ENTRY, 0)))
> > +        goto error;
> > +
> > +    if (!(ccf->cc_608_fifo = av_fifo_alloc2(MAX_CC_ELEMENTS, CC_BYTES_PER_ENTRY, 0)))
> > +        goto error;
> > +
> > +    /* Based on the target FPS, figure out the expected cc_count and number of
> > +       608 tuples per packet.  See ANSI/CTA-708-E Sec 4.3.6.1. */
> > +    for (i = 0; i < FF_ARRAY_ELEMS(cc_lookup_vals); i++) {
> > +        if (framerate->num == cc_lookup_vals[i].num &&
> > +            framerate->den == cc_lookup_vals[i].den) {
> > +            ccf->expected_cc_count = cc_lookup_vals[i].cc_count;
> > +            ccf->expected_608 = cc_lookup_vals[i].num_608;
> > +            break;
> > +        }
> > +    }
> > +
> > +    if (ccf->expected_608 == 0) {
> > +        /* We didn't find an output frame we support.  We'll let the call succeed
> > +           and the FIFO to be allocated, but the extract/inject functions will simply
> > +           leave everything the way it is */
> > +        av_log(ccf->log_ctx, AV_LOG_WARNING, "cc_fifo cannot transcode captions fps=%d/%d\n",
> > +               framerate->num, framerate->den);
>
> Won't this result in spurious warnings for users who are just converting
> framerates and don't have any captions. If so it should probably be
> printed the first time we actually see caption side data.

Yeah, I see your point.  Let me investigate further and see what I can do here.

> > +        ccf->passthrough = 1;
> > +    }
> > +
> > +    return ccf;
> > +
> > +error:
> > +    ff_ccfifo_freep(&ccf);
> > +    return NULL;
> > +}
> > +
> > +int ff_ccfifo_injectbytes(AVCCFifo *ccf, uint8_t **data, size_t *len)
> > +{
> > +    char *cc_data;
> > +    int cc_filled = 0;
> > +    int i;
> > +
> > +    if (!ccf)
> > +        return AVERROR(EINVAL);
>
> For all the extract/inject functions: it should be invalid to call them
> with a NULL context, so this should be an av_assert0() or not be here at
> all.

Ok.

> > +
> > +    if (ccf->passthrough) {
> > +        *data = NULL;
> > +        *len = 0;
> > +        return 0;
> > +    }
> > +
> > +    cc_data = av_mallocz(ccf->expected_cc_count * CC_BYTES_PER_ENTRY);
>
> This buffer size is constant for a given AVCCFifo object, so perhaps
> ff_ccfifo_alloc() could return required buffer size and this function
> could write into a user-provided buffer and avoid constant dynamic
> allocation.

So I had previously considered the approach you suggested, but decided
against it.  This is because while the AVCCFifo today always returns
the same number of bytes, there are cases where cc_count could vary on
a per frame basis (and thus the buffer size differs).  In particular
with 30i with 3:2 pulldown the cc_count alternates between 20 and 30.
See CTA-708-E R-2018 Sec 4.6.3.1.

Having the queue return a properly sized buffer avoids situations
where you make a call to get the size and then make a separate call to
fill the buffer (where the size might not be correct).

I don't particularly love the approach, given it means callers have to
memcpy() the result into the final buffer and then free the buffer
created by the call to inject.  But it seemed like the better approach
given the issue described above.

I guess because the API is private and we don't support that feature
today, we could do it as you described and then change the API later.
Suggestions welcome.

> > +    if (!cc_data) {
> > +        return AVERROR(ENOMEM);
> > +    }
> > +
> > +    for (i = 0; i < ccf->expected_608; i++) {
> > +        if (av_fifo_can_read(ccf->cc_608_fifo) >= CC_BYTES_PER_ENTRY) {
> > +            av_fifo_read(ccf->cc_608_fifo, &cc_data[cc_filled * CC_BYTES_PER_ENTRY],
> > +                         CC_BYTES_PER_ENTRY);
>
> This looks wrong, as fifo operations are in elements, and your
> elements are already CC_BYTES_PER_ENTRY. So every read actually writes
> CC_BYTES_PER_ENTRY * CC_BYTES_PER_ENTRY bytes to cc_data.
>
> I think you can also do this as a single av_fifo_read() call without a
> loop.

I'll review the logic above based on your comments and if appropriate
rework the loop.

Thanks,

Devin
diff mbox series

Patch

diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 71e198b..628ade8 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -14,6 +14,7 @@  OBJS = allfilters.o                                                     \
        buffersink.o                                                     \
        buffersrc.o                                                      \
        colorspace.o                                                     \
+       ccfifo.o                                                         \
        drawutils.o                                                      \
        fifo.o                                                           \
        formats.o                                                        \
diff --git a/libavfilter/ccfifo.c b/libavfilter/ccfifo.c
new file mode 100644
index 0000000..05a77dd
--- /dev/null
+++ b/libavfilter/ccfifo.c
@@ -0,0 +1,240 @@ 
+/*
+ * CEA-708 Closed Captioning FIFO
+ * Copyright (c) 2023 LTN Global Communications
+ *
+ * Author: Devin Heitmueller <dheitmueller@ltnglobal.com>
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "ccfifo.h"
+
+struct AVCCFifo {
+    AVFifo *cc_608_fifo;
+    AVFifo *cc_708_fifo;
+    int expected_cc_count;
+    int expected_608;
+    int cc_detected;
+    int passthrough;
+    void *log_ctx;
+};
+
+#define MAX_CC_ELEMENTS 128
+#define CC_BYTES_PER_ENTRY 3
+
+struct cc_lookup {
+    int num;
+    int den;
+    int cc_count;
+    int num_608;
+};
+
+const static struct cc_lookup cc_lookup_vals[] = {
+    { 15, 1, 40, 4 },
+    { 24, 1, 25, 3 },
+    { 24000, 1001, 25, 3 },
+    { 30, 1, 20, 2 },
+    { 30000, 1001, 20, 2},
+    { 60, 1, 10, 1 },
+    { 60000, 1001, 10, 1},
+};
+
+void ff_ccfifo_freep(AVCCFifo **ccf)
+{
+    if (ccf && *ccf) {
+        AVCCFifo *tmp = *ccf;
+        if (tmp->cc_608_fifo)
+            av_fifo_freep2(&tmp->cc_608_fifo);
+        if (tmp->cc_708_fifo)
+            av_fifo_freep2(&tmp->cc_708_fifo);
+        av_freep(*ccf);
+    }
+}
+
+AVCCFifo *ff_ccfifo_alloc(AVRational *framerate, void *log_ctx)
+{
+    AVCCFifo *ccf;
+    int i;
+
+    ccf = av_mallocz(sizeof(*ccf));
+    if (!ccf)
+        return NULL;
+
+    if (!(ccf->cc_708_fifo = av_fifo_alloc2(MAX_CC_ELEMENTS, CC_BYTES_PER_ENTRY, 0)))
+        goto error;
+
+    if (!(ccf->cc_608_fifo = av_fifo_alloc2(MAX_CC_ELEMENTS, CC_BYTES_PER_ENTRY, 0)))
+        goto error;
+
+    /* Based on the target FPS, figure out the expected cc_count and number of
+       608 tuples per packet.  See ANSI/CTA-708-E Sec 4.3.6.1. */
+    for (i = 0; i < FF_ARRAY_ELEMS(cc_lookup_vals); i++) {
+        if (framerate->num == cc_lookup_vals[i].num &&
+            framerate->den == cc_lookup_vals[i].den) {
+            ccf->expected_cc_count = cc_lookup_vals[i].cc_count;
+            ccf->expected_608 = cc_lookup_vals[i].num_608;
+            break;
+        }
+    }
+
+    if (ccf->expected_608 == 0) {
+        /* We didn't find an output frame we support.  We'll let the call succeed
+           and the FIFO to be allocated, but the extract/inject functions will simply
+           leave everything the way it is */
+        av_log(ccf->log_ctx, AV_LOG_WARNING, "cc_fifo cannot transcode captions fps=%d/%d\n",
+               framerate->num, framerate->den);
+        ccf->passthrough = 1;
+    }
+
+    return ccf;
+
+error:
+    ff_ccfifo_freep(&ccf);
+    return NULL;
+}
+
+int ff_ccfifo_injectbytes(AVCCFifo *ccf, uint8_t **data, size_t *len)
+{
+    char *cc_data;
+    int cc_filled = 0;
+    int i;
+
+    if (!ccf)
+        return AVERROR(EINVAL);
+
+    if (ccf->passthrough) {
+        *data = NULL;
+        *len = 0;
+        return 0;
+    }
+
+    cc_data = av_mallocz(ccf->expected_cc_count * CC_BYTES_PER_ENTRY);
+    if (!cc_data) {
+        return AVERROR(ENOMEM);
+    }
+
+    for (i = 0; i < ccf->expected_608; i++) {
+        if (av_fifo_can_read(ccf->cc_608_fifo) >= CC_BYTES_PER_ENTRY) {
+            av_fifo_read(ccf->cc_608_fifo, &cc_data[cc_filled * CC_BYTES_PER_ENTRY],
+                         CC_BYTES_PER_ENTRY);
+            cc_filled++;
+        } else {
+            break;
+        }
+    }
+
+    /* Insert any available data from the 708 FIFO */
+    while (cc_filled < ccf->expected_cc_count) {
+        if (av_fifo_can_read(ccf->cc_708_fifo) >= CC_BYTES_PER_ENTRY) {
+            av_fifo_read(ccf->cc_708_fifo, &cc_data[cc_filled * CC_BYTES_PER_ENTRY],
+                         CC_BYTES_PER_ENTRY);
+            cc_filled++;
+        } else {
+            break;
+        }
+    }
+
+    /* Insert 708 padding into any remaining fields */
+    while (cc_filled < ccf->expected_cc_count) {
+        cc_data[cc_filled * CC_BYTES_PER_ENTRY]     = 0xfa;
+        cc_data[cc_filled * CC_BYTES_PER_ENTRY + 1] = 0x00;
+        cc_data[cc_filled * CC_BYTES_PER_ENTRY + 2] = 0x00;
+        cc_filled++;
+    }
+
+    *data = cc_data;
+    *len = ccf->expected_cc_count * CC_BYTES_PER_ENTRY;
+    return 0;
+}
+
+int ff_ccfifo_inject(AVCCFifo *ccf, AVFrame *frame)
+{
+    AVFrameSideData *sd;
+    uint8_t *cc_data;
+    size_t cc_size;
+    int ret;
+
+    if (!ccf)
+        return AVERROR(EINVAL);
+
+    if (ccf->passthrough == 1 || ccf->cc_detected == 0 || ccf->expected_cc_count == 0)
+        return 0;
+
+    ret = ff_ccfifo_injectbytes(ccf, &cc_data, &cc_size);
+    if (ret == 0) {
+        sd = av_frame_new_side_data(frame, AV_FRAME_DATA_A53_CC, cc_size);
+        if (!sd) {
+            av_freep(&cc_data);
+            return AVERROR(ENOMEM);
+        }
+        memcpy(sd->data, cc_data, cc_size);
+        av_freep(&cc_data);
+    }
+
+    return 0;
+}
+
+int ff_ccfifo_extractbytes(AVCCFifo *ccf, uint8_t *cc_bytes, size_t len)
+{
+    int cc_count = len / CC_BYTES_PER_ENTRY;
+
+    if (!ccf)
+        return AVERROR(EINVAL);
+
+    if (ccf->passthrough == 1)
+        return 0;
+
+    ccf->cc_detected = 1;
+
+    for (int i = 0; i < cc_count; i++) {
+        /* See ANSI/CTA-708-E Sec 4.3, Table 3 */
+        uint8_t cc_valid = (cc_bytes[CC_BYTES_PER_ENTRY*i] & 0x04) >> 2;
+        uint8_t cc_type = cc_bytes[CC_BYTES_PER_ENTRY*i] & 0x03;
+        if (cc_type == 0x00 || cc_type == 0x01) {
+            av_fifo_write(ccf->cc_608_fifo, &cc_bytes[CC_BYTES_PER_ENTRY*i],
+                          CC_BYTES_PER_ENTRY);
+        } else if (cc_valid && (cc_type == 0x02 || cc_type == 0x03)) {
+            av_fifo_write(ccf->cc_708_fifo, &cc_bytes[CC_BYTES_PER_ENTRY*i],
+                          CC_BYTES_PER_ENTRY);
+        }
+    }
+    return 0;
+}
+
+int ff_ccfifo_extract(AVCCFifo *ccf, AVFrame *frame)
+{
+    if (!ccf)
+        return AVERROR(EINVAL);
+
+    if (ccf->passthrough == 1)
+        return 0;
+
+    /* Read the A53 side data, discard padding, and put 608/708 into
+       queues so we can ensure they get into the output frames at
+       the correct rate... */
+    if (ccf->expected_cc_count > 0) {
+        AVFrameSideData *side_data = av_frame_get_side_data(frame, AV_FRAME_DATA_A53_CC);
+        if (side_data) {
+            ff_ccfifo_extractbytes(ccf, side_data->data, side_data->size);
+
+            /* Remove the side data, as we will re-create it on the
+               output as needed */
+            av_frame_remove_side_data(frame, AV_FRAME_DATA_A53_CC);
+        }
+    }
+    return 0;
+}
diff --git a/libavfilter/ccfifo.h b/libavfilter/ccfifo.h
new file mode 100644
index 0000000..baa4ae1
--- /dev/null
+++ b/libavfilter/ccfifo.h
@@ -0,0 +1,94 @@ 
+/*
+ * CEA-708 Closed Captioning FIFO
+ * Copyright (c) 2023 LTN Global Communications
+ *
+ * Author: Devin Heitmueller <dheitmueller@ltnglobal.com>
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+/**
+ * @file
+ * CC FIFO Buffer
+ */
+
+#ifndef AVFILTER_CCFIFO_H
+#define AVFILTER_CCFIFO_H
+
+#include "libavutil/avutil.h"
+#include "libavutil/frame.h"
+#include "libavutil/fifo.h"
+
+typedef struct AVCCFifo AVCCFifo;
+
+/**
+ * Allocate an AVCCFifo.
+ *
+ * @param framerate   output framerate
+ * @param log_ctx     used for any av_log() calls
+ * @return            newly allocated AVCCFifo, or NULL on error
+ */
+AVCCFifo *ff_ccfifo_alloc(AVRational *framerate, void *log_ctx);
+
+/**
+ * Free an AVCCFifo
+ *
+ * @param ccf Pointer to the pointer to the AVCCFifo which should be freed
+ * @note `*ptr = NULL` is safe and leads to no action.
+ */
+void ff_ccfifo_freep(AVCCFifo **ccf);
+
+
+/**
+ * Extract CC data from an AVFrame
+ *
+ * Extract CC bytes from the AVFrame, insert them into our queue, and
+ * remove the side data from the AVFrame.  The side data is removed
+ * as it will be re-inserted at the appropriate rate later in the
+ * filter.
+ *
+ * @param af          AVCCFifo to write to
+ * @param frame       AVFrame with the video frame to operate on
+ * @return            Zero on success, or negative AVERROR
+ *                    code on failure.
+ */
+int ff_ccfifo_extract(AVCCFifo *ccf, AVFrame *frame);
+
+/**
+ *Just like ff_ccfifo_extract(), but takes the raw bytes instead of an AVFrame
+ */
+int ff_ccfifo_extractbytes(AVCCFifo *ccf, uint8_t *data, size_t len);
+
+/**
+ * Insert CC data from the FIFO into an AVFrame (as side data)
+ *
+ * Dequeue the appropriate number of CC tuples based on the
+ * frame rate, and insert them into the AVFrame
+ *
+ * @param af          AVCCFifo to read from
+ * @param frame       AVFrame with the video frame to operate on
+ * @return            Zero on success, or negative AVERROR
+ *                    code on failure.
+ */
+int ff_ccfifo_inject(AVCCFifo *ccf, AVFrame *frame);
+
+/* Just like ff_ccfifo_inject(), but takes the raw bytes instead of an AVFrame
+ * Note: the caller is responsible for calling av_freep() against the value
+ * returned in the "data" argument */
+int ff_ccfifo_injectbytes(AVCCFifo *ccf, uint8_t **data, size_t *len);
+
+#endif /* AVFILTER_CCFIFO_H */