From patchwork Fri May 5 19:09:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 41496 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:dca6:b0:f3:34fa:f187 with SMTP id ky38csp685491pzb; Fri, 5 May 2023 11:13:44 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4phVaW79vt5PqRPHP+8CaZ2nBcc89meZDQuwRNl9/ols9npJzvQe2oNKKKV0W1K4ZjK8/d X-Received: by 2002:a17:907:a40a:b0:94f:1ac0:850b with SMTP id sg10-20020a170907a40a00b0094f1ac0850bmr2109857ejc.71.1683310424612; Fri, 05 May 2023 11:13:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683310424; cv=none; d=google.com; s=arc-20160816; b=WSGdmgMVxfwJeMQahi7MhD0j7JeeCC/KYGq/IWhIOe3SZ5/VNHDICcMGCAntN1SUG1 WVbIpXCIJVz7k2EH55zCin7HZtXhAJUVStatuEHbFBTDJm2IAXEv7wrlD6LO6Uepgbns V5mSRcjagoekCunNarS+wXJU4SNRMT+tSNKIH8Tl3txuxsOApQfiMXEJGRnPdB+XeR9j Cz1KTCk0EtSAm7RvkennjSi2XGPJJOp/lFyFDuy6gysqnvZhgOUFReatQK8Obr/dYMfk pKl5+5eQC/ZgWMDfD3lX2k5nBkAUB1WiwSsgtviLsBv6ozcm1qIL4vF6ss1Q9TaPl7v7 F3ow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=dPSqqH3tIFZ3JX0PPsPKqkSuGDJ8Sa6+hulsiyBuueQ=; b=zBLifHsi0zqQKWY7lg1xoNimskxY88EFZSvnPU0LjZht3ckkhyD9rE9zWboEyN9DBY 8vh57cLCnKUoMbmnZHSvblPca9xqj8W+45Fdf5JXy/MgdrzjV/A3R8O8QrxAvBlREQIT F74pvKdx7KZuIREg0qPuPlYsO8nTmmFepmp7V/kcuep+/Ef0U9a6f+8r5PN5nDuBSxoj tZeY7O0F5nucv1q7LUOdD2IP0Xkrzumpri1FM9qLG9hLYDmkE5nb1Ws++h2f2Idq3LuS qgVpdrWSZJCMbMYVHJosoNKcoA7pUXe7hiKvd/5Ip11Kxd1yxho4JQi6MiiFA+PRgjoP ClBg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b="aB3hD/ft"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id n9-20020a170906688900b0096599ff921bsi1379892ejr.331.2023.05.05.11.13.44; Fri, 05 May 2023 11:13:44 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b="aB3hD/ft"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7D83F68C17C; Fri, 5 May 2023 21:13:30 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qt1-f180.google.com (mail-qt1-f180.google.com [209.85.160.180]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id F210168C110 for ; Fri, 5 May 2023 21:13:22 +0300 (EEST) Received: by mail-qt1-f180.google.com with SMTP id d75a77b69052e-3ef38864360so21572071cf.2 for ; Fri, 05 May 2023 11:13:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20221208.gappssmtp.com; s=20221208; t=1683310401; x=1685902401; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=KJs68WU/pG18xrAFwzbunbbIqZADJN/evyHtohSusN0=; b=aB3hD/ftir0Ayp0GuNnXao+mKrOMT7wwr8y4zSakbplcl5uawd9CDS7H85nZx4O38x npOsni1VXx8myetVDgStyaIvjIeJJMV6NxvEDVU3wHL08Yj/K30gNYGJLMCvK3IqMaZE Mbo72LBx4Utt5jrheIwmUr7dlX9eeKBijY8qkaQO+r7ZrIx9k2CMB+v7JyNni7VE0qVz eETKiimaWxRjLNg0W6Ua5F/FqHxRLI0y00WrvGpRshpvx+8nipMnpvJwF2bCdfl2NCxm wULTv1AmKtrCVPo3NnKZ+VfqQwgNnuFdXVaHAjuhS4oIL125fwwZZMVe+W8oYm/9KYjD ZNsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683310401; x=1685902401; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=KJs68WU/pG18xrAFwzbunbbIqZADJN/evyHtohSusN0=; b=T9gPYFYvvcSbEYvrpj9KOXP07O37izt6scy/5tEl8THbEelWagMT1QHOohYPngWgAH 1Ut8OZsi0U3zv4jTLjhy8WEnLWgPagw+pl97Io/zXsdvP2cBdECMxBaY2lwZQ3MgVXCP USPrDdnGBs+3SlOhVroGWre7wJ4dfM8Flg48ZZxwrMjYrjM3W+MESuvukAH7QhqRWzMc a1gfy9mUNQzwxXlIg03V1UbQFVAAbeDKa1t3t23vuraWz/J7dNi3ybYCkrD7Nt2lBEUw dfE1aQByJ27J0opROVptoMQqehJglgYWmlb/7QF8lndyWbH/K28630MTPHZb5YSQZsWi FNgw== X-Gm-Message-State: AC+VfDyzkGTWYTbs07vuWSAofYj/rB6TfYbpXTHPuXqEWEb4KiRxpAdI AsdwCms+D6lk3KnbowQ4tslpOEfxBTmYrpccbPM= X-Received: by 2002:a05:622a:188f:b0:3ee:d076:78dd with SMTP id v15-20020a05622a188f00b003eed07678ddmr3429107qtc.12.1683310401271; Fri, 05 May 2023 11:13:21 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id d1-20020a05620a166100b0074e26158cf2sm747289qko.93.2023.05.05.11.13.20 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 May 2023 11:13:20 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 5 May 2023 15:09:02 -0400 Message-Id: <1683313747-3775-2-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1683313747-3775-1-git-send-email-dheitmueller@ltnglobal.com> References: <1683313747-3775-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [PATCH v6 1/6] ccfifo: Properly handle CEA-708 captions through framerate conversion X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: r+YfSHlamG0/ When transcoding video that contains 708 closed captions, the caption data is tied to the frames as side data. Simply dropping or adding frames to change the framerate will result in loss of data, so the caption data needs to be preserved and reformatted. For example, without this patch converting 720p59 to 1080i59 would result in loss of 50% of the caption bytes, resulting in garbled 608 captions and 708 probably wouldn't render at all. Further, the frames that are there will have an illegal cc_count for the target framerate, so some decoders may ignore the packets entirely. Extract the 608 and 708 tuples and insert them onto queues. Then after dropping/adding frames, re-write the tuples back into the resulting frames at the appropriate rate given the target framerate. This includes both having the correct cc_count as well as clocking out the 608 pairs at the appropriate rate. Thanks to Lance Wang , Anton Khirnov , and Michael Niedermayer for providing review/feedback. Signed-off-by: Devin Heitmueller --- libavfilter/Makefile | 1 + libavfilter/ccfifo.c | 224 +++++++++++++++++++++++++++++++++++++++++++++++++++ libavfilter/ccfifo.h | 110 +++++++++++++++++++++++++ 3 files changed, 335 insertions(+) create mode 100644 libavfilter/ccfifo.c create mode 100644 libavfilter/ccfifo.h diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 482aeaf..68c8f14 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -14,6 +14,7 @@ OBJS = allfilters.o \ buffersink.o \ buffersrc.o \ colorspace.o \ + ccfifo.o \ drawutils.o \ fifo.o \ formats.o \ diff --git a/libavfilter/ccfifo.c b/libavfilter/ccfifo.c new file mode 100644 index 0000000..5fb68ce --- /dev/null +++ b/libavfilter/ccfifo.c @@ -0,0 +1,224 @@ +/* + * CEA-708 Closed Captioning FIFO + * Copyright (c) 2023 LTN Global Communications + * + * Author: Devin Heitmueller + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "ccfifo.h" + +struct AVCCFifo { + AVFifo *cc_608_fifo; + AVFifo *cc_708_fifo; + AVRational framerate; + int expected_cc_count; + int expected_608; + int cc_detected; + int passthrough; + int passthrough_warning; + void *log_ctx; +}; + +#define MAX_CC_ELEMENTS 128 +#define CC_BYTES_PER_ENTRY 3 + +struct cc_lookup { + int num; + int den; + int cc_count; + int num_608; +}; + +const static struct cc_lookup cc_lookup_vals[] = { + { 15, 1, 40, 4 }, + { 24, 1, 25, 3 }, + { 24000, 1001, 25, 3 }, + { 30, 1, 20, 2 }, + { 30000, 1001, 20, 2}, + { 60, 1, 10, 1 }, + { 60000, 1001, 10, 1}, +}; + +void ff_ccfifo_freep(AVCCFifo **ccf) +{ + AVCCFifo *tmp = *ccf; + if (tmp) { + av_fifo_freep2(&tmp->cc_608_fifo); + av_fifo_freep2(&tmp->cc_708_fifo); + } + av_freep(ccf); +} + +AVCCFifo *ff_ccfifo_alloc(AVRational framerate, void *log_ctx) +{ + AVCCFifo *ccf; + int i; + + ccf = av_mallocz(sizeof(*ccf)); + if (!ccf) + return NULL; + + ccf->log_ctx = log_ctx; + ccf->framerate = framerate; + + if (!(ccf->cc_708_fifo = av_fifo_alloc2(MAX_CC_ELEMENTS, CC_BYTES_PER_ENTRY, 0))) + goto error; + + if (!(ccf->cc_608_fifo = av_fifo_alloc2(MAX_CC_ELEMENTS, CC_BYTES_PER_ENTRY, 0))) + goto error; + + /* Based on the target FPS, figure out the expected cc_count and number of + 608 tuples per packet. See ANSI/CTA-708-E Sec 4.3.6.1. */ + for (i = 0; i < FF_ARRAY_ELEMS(cc_lookup_vals); i++) { + if (framerate.num == cc_lookup_vals[i].num && + framerate.den == cc_lookup_vals[i].den) { + ccf->expected_cc_count = cc_lookup_vals[i].cc_count; + ccf->expected_608 = cc_lookup_vals[i].num_608; + break; + } + } + + if (ccf->expected_608 == 0) { + /* We didn't find an output frame we support. We'll let the call succeed + and the FIFO to be allocated, but the extract/inject functions will simply + leave everything the way it is */ + ccf->passthrough = 1; + } + + return ccf; + +error: + ff_ccfifo_freep(&ccf); + return NULL; +} + +int ff_ccfifo_getoutputsize(AVCCFifo *ccf) +{ + return ccf->expected_cc_count * CC_BYTES_PER_ENTRY; +} + +int ff_ccfifo_ccdetected(AVCCFifo *ccf) +{ + return ccf->cc_detected; +} + +int ff_ccfifo_injectbytes(AVCCFifo *ccf, uint8_t *cc_data, size_t len) +{ + int cc_608_tuples = 0; + int cc_708_tuples = 0; + int cc_filled = 0; + + if (ccf->passthrough) { + return 0; + } + + if (len < ff_ccfifo_getoutputsize(ccf)) { + return AVERROR(EINVAL); + } + + /* Insert any available data from the 608 FIFO */ + if (ccf->expected_608 <= av_fifo_can_read(ccf->cc_608_fifo)) + cc_608_tuples = ccf->expected_608; + else + cc_608_tuples = av_fifo_can_read(ccf->cc_608_fifo); + av_fifo_read(ccf->cc_608_fifo, cc_data, cc_608_tuples); + cc_filled += cc_608_tuples; + + /* Insert any available data from the 708 FIFO */ + if ((ccf->expected_cc_count - cc_filled) <= av_fifo_can_read(ccf->cc_708_fifo)) + cc_708_tuples = ccf->expected_cc_count - cc_filled; + else + cc_708_tuples = av_fifo_can_read(ccf->cc_708_fifo); + av_fifo_read(ccf->cc_708_fifo, &cc_data[cc_filled * CC_BYTES_PER_ENTRY], cc_708_tuples); + cc_filled += cc_708_tuples; + + /* Insert 708 padding into any remaining fields */ + while (cc_filled < ccf->expected_cc_count) { + cc_data[cc_filled * CC_BYTES_PER_ENTRY] = 0xfa; + cc_data[cc_filled * CC_BYTES_PER_ENTRY + 1] = 0x00; + cc_data[cc_filled * CC_BYTES_PER_ENTRY + 2] = 0x00; + cc_filled++; + } + + return 0; +} + +int ff_ccfifo_inject(AVCCFifo *ccf, AVFrame *frame) +{ + AVFrameSideData *sd; + int ret; + + if (ccf->passthrough == 1 || ccf->cc_detected == 0) + return 0; + + sd = av_frame_new_side_data(frame, AV_FRAME_DATA_A53_CC, + ff_ccfifo_getoutputsize(ccf)); + if (sd) { + ret = ff_ccfifo_injectbytes(ccf, sd->data, sd->size); + if (ret < 0) { + av_frame_remove_side_data(frame, AV_FRAME_DATA_A53_CC); + return ret; + } + } + + return 0; +} + +int ff_ccfifo_extractbytes(AVCCFifo *ccf, uint8_t *cc_bytes, size_t len) +{ + int cc_count = len / CC_BYTES_PER_ENTRY; + + if (ccf->passthrough == 1) { + av_log_once(ccf->log_ctx, AV_LOG_WARNING, AV_LOG_DEBUG, &ccf->passthrough_warning, + "cc_fifo cannot transcode captions fps=%d/%d\n", + ccf->framerate.num, ccf->framerate.den); + return 0; + } + + ccf->cc_detected = 1; + + for (int i = 0; i < cc_count; i++) { + /* See ANSI/CTA-708-E Sec 4.3, Table 3 */ + uint8_t cc_valid = (cc_bytes[CC_BYTES_PER_ENTRY*i] & 0x04) >> 2; + uint8_t cc_type = cc_bytes[CC_BYTES_PER_ENTRY*i] & 0x03; + if (cc_type == 0x00 || cc_type == 0x01) { + av_fifo_write(ccf->cc_608_fifo, &cc_bytes[CC_BYTES_PER_ENTRY*i], 1); + } else if (cc_valid && (cc_type == 0x02 || cc_type == 0x03)) { + av_fifo_write(ccf->cc_708_fifo, &cc_bytes[CC_BYTES_PER_ENTRY*i], 1); + } + } + return 0; +} + +/* Read the A53 side data, discard padding, and put 608/708 into + queues so we can ensure they get into the output frames at + the correct rate... */ +int ff_ccfifo_extract(AVCCFifo *ccf, AVFrame *frame) +{ + AVFrameSideData *side_data = av_frame_get_side_data(frame, AV_FRAME_DATA_A53_CC); + if (side_data) { + ff_ccfifo_extractbytes(ccf, side_data->data, side_data->size); + + /* Remove the side data, as we will re-create it on the + output as needed */ + if (!ccf->passthrough) + av_frame_remove_side_data(frame, AV_FRAME_DATA_A53_CC); + } + return 0; +} diff --git a/libavfilter/ccfifo.h b/libavfilter/ccfifo.h new file mode 100644 index 0000000..44c9245 --- /dev/null +++ b/libavfilter/ccfifo.h @@ -0,0 +1,110 @@ +/* + * CEA-708 Closed Captioning FIFO + * Copyright (c) 2023 LTN Global Communications + * + * Author: Devin Heitmueller + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * CC FIFO Buffer + */ + +#ifndef AVFILTER_CCFIFO_H +#define AVFILTER_CCFIFO_H + +#include "libavutil/avutil.h" +#include "libavutil/frame.h" +#include "libavutil/fifo.h" + +typedef struct AVCCFifo AVCCFifo; + +/** + * Allocate an AVCCFifo. + * + * @param framerate output framerate + * @param log_ctx used for any av_log() calls + * @return newly allocated AVCCFifo, or NULL on error + */ +AVCCFifo *ff_ccfifo_alloc(AVRational framerate, void *log_ctx); + +/** + * Free an AVCCFifo + * + * @param ccf Pointer to the pointer to the AVCCFifo which should be freed + * @note `*ptr = NULL` is safe and leads to no action. + */ +void ff_ccfifo_freep(AVCCFifo **ccf); + + +/** + * Extract CC data from an AVFrame + * + * Extract CC bytes from the AVFrame, insert them into our queue, and + * remove the side data from the AVFrame. The side data is removed + * as it will be re-inserted at the appropriate rate later in the + * filter. + * + * @param af AVCCFifo to write to + * @param frame AVFrame with the video frame to operate on + * @return Zero on success, or negative AVERROR + * code on failure. + */ +int ff_ccfifo_extract(AVCCFifo *ccf, AVFrame *frame); + +/** + *Just like ff_ccfifo_extract(), but takes the raw bytes instead of an AVFrame + */ +int ff_ccfifo_extractbytes(AVCCFifo *ccf, uint8_t *data, size_t len); + +/** + * Provide the size in bytes of an output buffer to allocate + * + * Ask for how many bytes the output will contain, so the caller can allocate + * an appropriately sized buffer and pass it to ff_ccfifo_injectbytes() + * + */ +int ff_ccfifo_getoutputsize(AVCCFifo *ccf); + +/** + * Insert CC data from the FIFO into an AVFrame (as side data) + * + * Dequeue the appropriate number of CC tuples based on the + * frame rate, and insert them into the AVFrame + * + * @param af AVCCFifo to read from + * @param frame AVFrame with the video frame to operate on + * @return Zero on success, or negative AVERROR + * code on failure. + */ +int ff_ccfifo_inject(AVCCFifo *ccf, AVFrame *frame); + +/** + * Just like ff_ccfifo_inject(), but takes the raw bytes to insert the CC data + * int rather than an AVFrame + */ +int ff_ccfifo_injectbytes(AVCCFifo *ccf, uint8_t *data, size_t len); + +/** + * Returns 1 if captions have been found as a prior call + * to ff_ccfifo_extract() or ff_ccfifo_extractbytes() + */ +int ff_ccfifo_ccdetected(AVCCFifo *ccf); + +#endif /* AVFILTER_CCFIFO_H */ From patchwork Fri May 5 19:09:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 41501 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:dca6:b0:f3:34fa:f187 with SMTP id ky38csp690541pzb; Fri, 5 May 2023 11:22:02 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ68az6ba7+OLreASemMkUy2d9BjB9okq6bfWoFR2JfG1chZsh3QDTJATHXBzTLUZg3/PLdp X-Received: by 2002:a17:907:26c6:b0:91f:b13f:a028 with SMTP id bp6-20020a17090726c600b0091fb13fa028mr1774701ejc.34.1683310922336; Fri, 05 May 2023 11:22:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683310922; cv=none; d=google.com; s=arc-20160816; b=JxcZMYiAPSUGMm1dV+Um05LP63t1X1/NWOOP0ANMKCK5cQVT/9loy7oktBsFwuU1KW KEzXpUBbPWyI9xIIMkVAzMK03+BOdPfmPOSroZdiU9Du+HRvDCeEnVolGUOWZuLFyxOM xPJ1XRf0IYoeUEcTkPn/lL09YW6IDuBGbMXGxaaSLg995uq5kz/7Pm3NKP/lNqNdj+mF XlLJKpS23MFdN8W5mcO99EsiUYXs05prJl7qqb7veCH9ATdDA00aVV1JDbuwZyfW2+mF JLTZbczg4q/1IP1Rio1Wl7IziP7PibhsYdByRte7aR2phevyHT/fkQz1bGtObCTGnBsf uY8A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=i/Ql5yBOaLTnnMeO3UnuNKhMMHjT1qC21ceeShMd/aE=; b=0oqwbb5c+WTBTW2EUfT5VospQyfrdCnh7Y1b6LqmkNhAHt9n8+agQAm6hjzePW5xVq YlddZ0pzTR/+aifOtZb5oXKusIaAyGs5EJGU8mMN3i39/0yMfKvZOy5B9X8Er86B3ZSu fI+xuKoT1C2lerrRVJtfKQOwqok37F+qfmMZEz7E2nxDL/71bW6JEtq6zIwBJg+eBs7w v/Nvabr+WexFr7zlh7GtEqA2OEEboLuJia3AsznPSqmmZZvDr+8kbjd0u47+mxyrH1qQ n5nyCCs5JohBmb4oTLv08x0XjZQQYzgSA64mw+KyJ+QvB+JoFwk01iAJkbyRg06U5BOf xN+A== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=GQwQgrzs; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id g23-20020a1709061e1700b009659c34e845si1557255ejj.166.2023.05.05.11.22.00; Fri, 05 May 2023 11:22:02 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=GQwQgrzs; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5BA8468C186; Fri, 5 May 2023 21:21:49 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ua1-f51.google.com (mail-ua1-f51.google.com [209.85.222.51]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C523668BFAF for ; Fri, 5 May 2023 21:21:42 +0300 (EEST) Received: by mail-ua1-f51.google.com with SMTP id a1e0cc1a2514c-77d049b9040so15391829241.1 for ; Fri, 05 May 2023 11:21:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20221208.gappssmtp.com; s=20221208; t=1683310901; x=1685902901; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=VhC2/eK2+zuscWyj1kOpnDgubrv9ynrNIh/81ZaAl1g=; b=GQwQgrzsiyKP9ZDarKBeoLApRi1giLlXDvBDP2KPIIoWYQlxBBtgenMlsKl51G7j6Q yzp7fuJpdc1OKG8pfAl1KR9/Onx2XYT5HUNlDx7sNuk2q5TVD1LyiU6jh7Di9RVktKie 3PY0KTazJxOzS2NQElb2DyVpHus1fpliWBZ1bVL9QxSBGeVbShoFUZvC71sQB4WXptEq gV+bNIz6vmfKaLN3LFWSZcW6OL6jK2EcvDnDQ/QDJJyAZNZz/h1L81kPxHGX9pKzGaN/ AI2e9s51y5XsCGWdeKv2Q/b5A975z11UAgQtoBuR+bjB7JrxFk07ZuDT+NxALZlLXV7D a11Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683310901; x=1685902901; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=VhC2/eK2+zuscWyj1kOpnDgubrv9ynrNIh/81ZaAl1g=; b=ZenXzqYHscS8jMmH3pbE/oeren5ue6RzusZK9H2b49b9yl9h+24XMv/xeJZUIgUjQy 9Z0+NM2FSXfyP1tg9EyPwJKQDt5HrUBP6PO+t7N4ga8Utcn1b1xxqrmcu2ZqdUKhQgo0 v1lnEcGQHZlhol2KTaXckTi6+41BP/uCgzmMTB8SjaaA6W9ul0JgdogipUklmGSssW2q q4ODhpbc6Quxunclk2ZZdoIuC5Jbywl3WwRLPFQcsQlEpjSGJqS8MVp04EmIf+vYBd8H 6+b9dVZ0tHVHTAc7P9PSkwRhoiOXvWX25bTi9isZNpZQMG4Rh7LVHQ276GyaFW9+YMFX GvOg== X-Gm-Message-State: AC+VfDyaSejQf8Iuf7GVJv1DmDLOTeTLHEUt69gq8gFGaqtGOTo0Kjte MCjm7RAvAuE6jDlqoN8KqXhuerz/4wOGvK2zHPo= X-Received: by 2002:ac8:5848:0:b0:3f2:11db:9658 with SMTP id h8-20020ac85848000000b003f211db9658mr14230868qth.2.1683310401753; Fri, 05 May 2023 11:13:21 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id d1-20020a05620a166100b0074e26158cf2sm747289qko.93.2023.05.05.11.13.21 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 May 2023 11:13:21 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 5 May 2023 15:09:03 -0400 Message-Id: <1683313747-3775-3-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1683313747-3775-1-git-send-email-dheitmueller@ltnglobal.com> References: <1683313747-3775-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [PATCH v6 2/6] vf_fps: properly preserve CEA-708 captions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: CaCvqE8gE1cX The existing implementation made an attempt to remove duplicate captions if increasing the framerate, but made no attempt to handle reducing the framerate, nor did it rewrite the caption payloads to have the appropriate cc_count (e.g. the cc_count needs to change from 20 to 10 when going from 1080i59 to 720p59 and vice-versa). Make use of the new ccfifo mechanism to ensure that caption data is properly preserved. Signed-off-by: Devin Heitmueller --- libavfilter/vf_fps.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/libavfilter/vf_fps.c b/libavfilter/vf_fps.c index 051d278..824e7a1 100644 --- a/libavfilter/vf_fps.c +++ b/libavfilter/vf_fps.c @@ -34,6 +34,7 @@ #include "libavutil/mathematics.h" #include "libavutil/opt.h" #include "avfilter.h" +#include "ccfifo.h" #include "filters.h" #include "internal.h" @@ -85,6 +86,7 @@ typedef struct FPSContext { AVFrame *frames[2]; ///< buffered frames int frames_count; ///< number of buffered frames + AVCCFifo *cc_fifo; ///< closed captions int64_t next_pts; ///< pts of the next frame to output @@ -165,6 +167,7 @@ static av_cold void uninit(AVFilterContext *ctx) frame = shift_frame(ctx, s); av_frame_free(&frame); } + ff_ccfifo_freep(&s->cc_fifo); av_log(ctx, AV_LOG_VERBOSE, "%d frames in, %d frames out; %d frames dropped, " "%d frames duplicated.\n", s->frames_in, s->frames_out, s->drop, s->dup); @@ -210,6 +213,11 @@ static int config_props(AVFilterLink* outlink) s->in_pts_off, s->out_pts_off, s->start_time); } + if (!(s->cc_fifo = ff_ccfifo_alloc(outlink->frame_rate, ctx))) { + av_log(ctx, AV_LOG_ERROR, "Failure to setup CC FIFO queue\n"); + return AVERROR(ENOMEM); + } + av_log(ctx, AV_LOG_VERBOSE, "fps=%d/%d\n", outlink->frame_rate.num, outlink->frame_rate.den); return 0; @@ -242,6 +250,7 @@ static int read_frame(AVFilterContext *ctx, FPSContext *s, AVFilterLink *inlink, av_log(ctx, AV_LOG_DEBUG, "Read frame with in pts %"PRId64", out pts %"PRId64"\n", in_pts, frame->pts); + ff_ccfifo_extract(s->cc_fifo, frame); s->frames[s->frames_count++] = frame; s->frames_in++; @@ -289,7 +298,7 @@ static int write_frame(AVFilterContext *ctx, FPSContext *s, AVFilterLink *outlin if (!frame) return AVERROR(ENOMEM); // Make sure Closed Captions will not be duplicated - av_frame_remove_side_data(s->frames[0], AV_FRAME_DATA_A53_CC); + ff_ccfifo_inject(s->cc_fifo, frame); frame->pts = s->next_pts++; frame->duration = 1; From patchwork Fri May 5 19:09:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 41497 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:dca6:b0:f3:34fa:f187 with SMTP id ky38csp685606pzb; Fri, 5 May 2023 11:13:54 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6JmBlLvPt2eOHlcRnCdh8CRAvKwTEFJL2e+lr5lfvuouwpcx1yhtnX+4MAjgoZsnHyPpTY X-Received: by 2002:a17:907:c17:b0:965:ae3a:52c8 with SMTP id ga23-20020a1709070c1700b00965ae3a52c8mr3350295ejc.32.1683310434069; Fri, 05 May 2023 11:13:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683310434; cv=none; d=google.com; s=arc-20160816; b=UQu1aQ5Dba6GxL/BcgXTskfY2EnWxWy4TL2SQDoEmqoLY3BW9lvWqgOYlS47A/P6WA ykni5ACiEZcpI1DDN2h1/xuWTIOPj3cmX14PjQomkt63y651ZZGzoi9F2aBDSfO9yR8n im2yWPgGphSo/+i4fQtqADH8Vdm2aiJ7z4TgnAT9aIdduTJ9vBm69uP5dR6bymcKe330 oUuatqC9ne8Q1hYttAx5XrsgWZ0uCYRzfrYE0aAufmdcsVd7ngz//T0Vlwiozw8o2auq OJSVScttdR++7zmkC1Jb+2pQlwk8IkrI9Zhli+1kcxh8yHw/pR+hr9jG4Jr1OK4nImTL Rr1w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=bcDlT4bS2F//ovCPqsEsS4Ld/G2EA1Xjznx7V/m2BfU=; b=nSher9GKh73u7eKCU/C4hcPAlg8GuMGSixjt1jGwqqDqZdXg6TQCwdro/KgOE6yVFQ mylrFxxJAXz6nGpdf4rAnJBHzYehogz2y1h0PR7rNg7Mv/T7x3VjHyA6GdvLxkR+q52P cDocQ7uZGdnzXR8TkZPFoVDq8F2mCgRedcXMVWsYawh6T3o3zKvepr8mgMTT44Qj6NUP meLpX3I9WvEBBHOgMPC3l1Iq9A1OBTcRQrBdSHGSF2RwulCLXdQb9fp32DvO/oM3oVH4 oDFu1yQYqFAlOUMAdquuQ/Py2VBLv6MwunYnZIXbuz8Gvo1hTZlB82hT/yr6oeoUTVQT /AUQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=0LEUIPwV; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id gv44-20020a1709072bec00b00965fac6722esi453631ejc.171.2023.05.05.11.13.53; Fri, 05 May 2023 11:13:54 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=0LEUIPwV; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9EBDF68C18D; Fri, 5 May 2023 21:13:31 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qk1-f170.google.com (mail-qk1-f170.google.com [209.85.222.170]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E2AC068BF52 for ; Fri, 5 May 2023 21:13:23 +0300 (EEST) Received: by mail-qk1-f170.google.com with SMTP id af79cd13be357-74fb8677a36so108177785a.0 for ; Fri, 05 May 2023 11:13:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20221208.gappssmtp.com; s=20221208; t=1683310402; x=1685902402; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=1icKU5ZG9Ueq6yYspjCh0KLlMAJMos3aHnhNpRwpoGA=; b=0LEUIPwV7jhn99p3H9fK6+pJbv0mrkDfi/oEuut9Pq951CFM9x/4+vScPW7P8PmI4o n6c0An+gfdCzln8CGzmcXEaTJEI31JgIiDjSGFtlGvxFsnqY9kuO/kaRZ2P/93nrXAAw +fH4UrOsIGks09dfCy1pBl6f8AiVANn7JpUI7dja0xY7X4SaHcAAIF+ewCgE7s33vsR+ qpf6JM0Z0Ka3PsYEGarbIVq8Lh1qTUeenOdnsHVnkbzsccpuO9inaCUb1UD6Ao1pDMVl +zzK9xIlEULGcbEu4p85ccV0aQZfcwQgDUFJHAngNDj7y6w9qtSPpgI0C8WNcsIW69Mz G5hA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683310402; x=1685902402; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=1icKU5ZG9Ueq6yYspjCh0KLlMAJMos3aHnhNpRwpoGA=; b=V8+9SahIFT9JwtxiDIkyu2Tk5ITGhEh1Zbkdd9ILY+MrbACU/E2fN+5SD8VdnNeeAf Exxm/UAD/HkKnSQERsKfy2YB5/CjQyUvcu0tqjwNQRFUqC1VjK6vAq4XNhWZrys/C8I6 7yrLU+YEqi1w8QslzzO9UBf0p2I84TrApAOEbFc0r7/dOVwQsthwzdYnuLvfW7Utp54V efvWQN3Z9yjL8cKdQrvydNKZSFFuLL285FDZjEluHcgNnbN1PAtscoHRFvcZXzVhQEjE aqexzv1DpyvFoQfjlwLR41QYYvAf9e2QUGqNZaBfKAZ+28PsMtxaaGosfdMW6U+uwhqv C5wA== X-Gm-Message-State: AC+VfDyYKY+7VobUyNiSHBdoYPmWJRCUF5n/az0g4wFxWj4tje64oGcv QAtRzi9BIdKL3VjZRV6shH8+qQb3r+3HLW1PM7k= X-Received: by 2002:a05:6214:5007:b0:61b:743c:34f6 with SMTP id jo7-20020a056214500700b0061b743c34f6mr3790188qvb.30.1683310402418; Fri, 05 May 2023 11:13:22 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id d1-20020a05620a166100b0074e26158cf2sm747289qko.93.2023.05.05.11.13.21 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 May 2023 11:13:21 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 5 May 2023 15:09:04 -0400 Message-Id: <1683313747-3775-4-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1683313747-3775-1-git-send-email-dheitmueller@ltnglobal.com> References: <1683313747-3775-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [PATCH v6 3/6] yadif: Properly preserve CEA-708 closed captions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: ceQnEN77AelF Various deinterlacing modes have the effect of doubling the framerate, and we need to ensure that the caption data isn't duplicated (or else you get double captions on-screen). Use the new ccfifo mechanism for yadif (and yadif_cuda and bwdif since they use the same yadif core) so that CEA-708 data is properly preserved through this filter. Signed-off-by: Devin Heitmueller --- libavfilter/vf_bwdif.c | 8 ++++++++ libavfilter/vf_yadif.c | 8 ++++++++ libavfilter/vf_yadif_cuda.c | 9 +++++++++ libavfilter/yadif.h | 2 ++ libavfilter/yadif_common.c | 5 +++++ 5 files changed, 32 insertions(+) diff --git a/libavfilter/vf_bwdif.c b/libavfilter/vf_bwdif.c index 34e8c5e..51e1e02 100644 --- a/libavfilter/vf_bwdif.c +++ b/libavfilter/vf_bwdif.c @@ -297,6 +297,7 @@ static av_cold void uninit(AVFilterContext *ctx) av_frame_free(&yadif->prev); av_frame_free(&yadif->cur ); av_frame_free(&yadif->next); + ff_ccfifo_freep(&yadif->cc_fifo); } static const enum AVPixelFormat pix_fmts[] = { @@ -332,6 +333,13 @@ static int config_props(AVFilterLink *link) if(yadif->mode&1) link->frame_rate = av_mul_q(link->src->inputs[0]->frame_rate, (AVRational){2,1}); + else + link->frame_rate = ctx->inputs[0]->frame_rate; + + if (!(yadif->cc_fifo = ff_ccfifo_alloc(link->frame_rate, ctx))) { + av_log(ctx, AV_LOG_ERROR, "Failure to setup CC FIFO queue\n"); + return AVERROR(ENOMEM); + } if (link->w < 3 || link->h < 4) { av_log(ctx, AV_LOG_ERROR, "Video of less than 3 columns or 4 lines is not supported\n"); diff --git a/libavfilter/vf_yadif.c b/libavfilter/vf_yadif.c index 1be02de..f77f811 100644 --- a/libavfilter/vf_yadif.c +++ b/libavfilter/vf_yadif.c @@ -261,6 +261,7 @@ static av_cold void uninit(AVFilterContext *ctx) av_frame_free(&yadif->prev); av_frame_free(&yadif->cur ); av_frame_free(&yadif->next); + ff_ccfifo_freep(&yadif->cc_fifo); } static const enum AVPixelFormat pix_fmts[] = { @@ -293,6 +294,13 @@ static int config_output(AVFilterLink *outlink) if(s->mode & 1) outlink->frame_rate = av_mul_q(ctx->inputs[0]->frame_rate, (AVRational){2, 1}); + else + outlink->frame_rate = ctx->inputs[0]->frame_rate; + + if (!(s->cc_fifo = ff_ccfifo_alloc(outlink->frame_rate, ctx))) { + av_log(ctx, AV_LOG_ERROR, "Failure to setup CC FIFO queue\n"); + return AVERROR(ENOMEM); + } if (outlink->w < 3 || outlink->h < 3) { av_log(ctx, AV_LOG_ERROR, "Video of less than 3 columns or lines is not supported\n"); diff --git a/libavfilter/vf_yadif_cuda.c b/libavfilter/vf_yadif_cuda.c index 685b8a2..f3f0b56 100644 --- a/libavfilter/vf_yadif_cuda.c +++ b/libavfilter/vf_yadif_cuda.c @@ -205,6 +205,7 @@ static av_cold void deint_cuda_uninit(AVFilterContext *ctx) av_frame_free(&y->prev); av_frame_free(&y->cur); av_frame_free(&y->next); + ff_cc_fifo_freep(&y->cc_fifo); av_buffer_unref(&s->device_ref); s->hwctx = NULL; @@ -291,6 +292,14 @@ static int config_output(AVFilterLink *link) if(y->mode & 1) link->frame_rate = av_mul_q(ctx->inputs[0]->frame_rate, (AVRational){2, 1}); + else + link->frame_rate = ctx->inputs[0]->frame_rate; + + if (!(s->cc_fifo = ff_cc_fifo_alloc(link->frame_rate, ctx))) { + av_log(ctx, AV_LOG_ERROR, "Failure to setup CC FIFO queue\n"); + ret = AVERROR(ENOMEM); + goto exit; + } if (link->w < 3 || link->h < 3) { av_log(ctx, AV_LOG_ERROR, "Video of less than 3 columns or lines is not supported\n"); diff --git a/libavfilter/yadif.h b/libavfilter/yadif.h index c928911..1077576 100644 --- a/libavfilter/yadif.h +++ b/libavfilter/yadif.h @@ -22,6 +22,7 @@ #include "libavutil/opt.h" #include "libavutil/pixdesc.h" #include "avfilter.h" +#include "ccfifo.h" enum YADIFMode { YADIF_MODE_SEND_FRAME = 0, ///< send 1 frame for each frame @@ -76,6 +77,7 @@ typedef struct YADIFContext { int eof; uint8_t *temp_line; int temp_line_size; + AVCCFifo *cc_fifo; /* * An algorithm that treats first and/or last fields in a sequence diff --git a/libavfilter/yadif_common.c b/libavfilter/yadif_common.c index a10cf7a..676853e 100644 --- a/libavfilter/yadif_common.c +++ b/libavfilter/yadif_common.c @@ -60,6 +60,8 @@ static int return_frame(AVFilterContext *ctx, int is_second) yadif->out->pts = AV_NOPTS_VALUE; } } + + ff_ccfifo_inject(yadif->cc_fifo, yadif->out); ret = ff_filter_frame(ctx->outputs[0], yadif->out); yadif->frame_pending = (yadif->mode&1) && !is_second; @@ -96,6 +98,8 @@ int ff_yadif_filter_frame(AVFilterLink *link, AVFrame *frame) av_assert0(frame); + ff_ccfifo_extract(yadif->cc_fifo, frame); + if (yadif->frame_pending) return_frame(ctx, 1); @@ -137,6 +141,7 @@ int ff_yadif_filter_frame(AVFilterLink *link, AVFrame *frame) if (!yadif->out) return AVERROR(ENOMEM); + ff_ccfifo_inject(yadif->cc_fifo, yadif->out); av_frame_free(&yadif->prev); if (yadif->out->pts != AV_NOPTS_VALUE) yadif->out->pts *= 2; From patchwork Fri May 5 19:09:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 41499 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:dca6:b0:f3:34fa:f187 with SMTP id ky38csp685956pzb; Fri, 5 May 2023 11:14:28 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6o7UsadkhOtAdF/M2y4deSRt1SvWtCOFPhdSdly1SlB9KJue5REMQU4ZbN01LSNSLgTdF1 X-Received: by 2002:a05:6402:4d9:b0:50b:c8bf:67b1 with SMTP id n25-20020a05640204d900b0050bc8bf67b1mr2275832edw.10.1683310468565; Fri, 05 May 2023 11:14:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683310468; cv=none; d=google.com; s=arc-20160816; b=cLRSmHySgSqbOTkk7qsSO1F7RkDT2ZIyrW51f3vlnHiOJEidSTqYjsfOEQVe8VBvkb PkmT44XjiE8i1HEc1zqBuxR7ZLHZNd2UEdMvWZe0ZYzfRgox1i2DNQkI1oE9PGYvwGj8 IbWGJhAqtLSk5V/JjKOawJhC49gBWZO/0o5hFEmfmCSk19lzRK2/ocXZh3CA7pxoo3br Nrrq49v6HsAVsUJvkDeykgCtEKJtBqen0lgV2G05zonJWn4krMCUDwVabNXQcwGoRB5s wRxEU5e9xmr77I/DDFgyPrkaeGTgJRjPZrSpGtFw26RMB0SXaEmc88IOFOFDX76WbOfS ZKHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=uVIMZrlwzOov84q7mWtcL5k6rqUaJvoojSkCsz4keoI=; b=M+w0s1rtosBG3Ydbbd/DM+LSxSDaxzW6nVYo/GsWO54gzUB45Y4phgqz3fMpwgcWkt cJwyHAk6+XAswEwIcfYb8xi99WNIpWNmuXqcfQkP+7mdELW2bbYUgPjl1idNY5Xxx5eM O2E7N8Po6MFZiYhcqjCpF20FTQr7NmGRYOSUSA9fEzZQ4LgAKGWQX3xiNuuxC5e1SfxU xyM0xcfiPf8PXYY9LyyRYO1Jr8jcxymi/tadC0KGMYSmjIzSpm8h0ksW5bMWFboukjYd E+v3RwnZZVN0Ya6UPfPAY0muQlxSzrtyMd2b4l1C+vpit3CinAGVeA9oM03/dKS2UNhl 54GA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=CJdGHbxu; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id b17-20020aa7d491000000b005089ebf6cb7si4927324edr.252.2023.05.05.11.14.04; Fri, 05 May 2023 11:14:28 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=CJdGHbxu; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 915AC68C15F; Fri, 5 May 2023 21:13:32 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qk1-f179.google.com (mail-qk1-f179.google.com [209.85.222.179]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6957E68C176 for ; Fri, 5 May 2023 21:13:24 +0300 (EEST) Received: by mail-qk1-f179.google.com with SMTP id af79cd13be357-74e12e93384so105054085a.3 for ; Fri, 05 May 2023 11:13:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20221208.gappssmtp.com; s=20221208; t=1683310403; x=1685902403; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=dHdpTVoMhyj4hLIW3Nkvvd7cp7Pxmqk2Fi3QmMqSDvk=; b=CJdGHbxueQ2w0ZjNAADQFNPZJNx3fNjcYAL5siPWFcKjSFOMleN68ciNXxgq3zx7Jy Uqj45J1BIgfGEJGWu3Sr7NEaveMDvRrppdbb4EOcMc214AK48Ia11MeJmmRXINWDW+Qr epM4XmLuyvxqHYUCBPcBifdtE+g0KPl+CVDPnfJJSrZjLeGLzXOoiEzskWEwqArvlh9p vcYW5ejAsdjVuQKNnvZivk6OaUXmnEe35JQ4gjEDThMO1pLpxy1DCr0hZ4+1IM8iFbL6 ewqbzEg191BQy1UuYoJIsKKpfsVvuPMo4oMcs4hpGpG6bVKxNMUuTRgtXXMzysTWmuPW Ea0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683310403; x=1685902403; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=dHdpTVoMhyj4hLIW3Nkvvd7cp7Pxmqk2Fi3QmMqSDvk=; b=IdHrLcw67amHmnc2i+HqE2dr0CMVjMg8oGkGFnpLLfVGtL88+tHxTSxLyIj7RAiYDA WbuyPSjOr6LOuwz3+8tUT41ZuJCDSbZljFNfEVzpIHLIZkmZ9kD617/cZqgdG18nH0TS d1xAmvEAr7/TER14ra7Tg6Vbqqvdt0swjSiw0Xbijhb0o/LGIltMWeOCF/8YZLcKdOnD gHv/lmrlesLxfyUankfJ3JwmHPyKz6irb3hll+WwsfFuXhQEFjBglmzzVMCPMWwdRMoN 6yS4YmWqcSpQbr8ZeGSKLs2p+qdwdzbFXDfUYEvsNQbD/9ztSd10d4YVqlB3b3mPJOnS O9EA== X-Gm-Message-State: AC+VfDwaUevZcYtZ43q3RnX1ROajIvR2ehT2cWyqKXrb0Vts5EGNkWnE extXIGQDNUbDdT5Y3/1tBlcfeJtzT6re8JervvU= X-Received: by 2002:a05:6214:301a:b0:5e8:11a0:ae17 with SMTP id ke26-20020a056214301a00b005e811a0ae17mr2745084qvb.43.1683310402985; Fri, 05 May 2023 11:13:22 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id d1-20020a05620a166100b0074e26158cf2sm747289qko.93.2023.05.05.11.13.22 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 May 2023 11:13:22 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 5 May 2023 15:09:05 -0400 Message-Id: <1683313747-3775-5-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1683313747-3775-1-git-send-email-dheitmueller@ltnglobal.com> References: <1683313747-3775-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [PATCH v6 4/6] tinterlace: Properly preserve CEA-708 closed captions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: UnLiQz8HhsYh Because the interlacing filter halves the effective framerate, we need to ensure that no CEA-708 data is lost as frames are merged. Make use of the new ccfifo mechanism to ensure that caption data is properly preserved as frames pass through the filter. Thanks to Thomas Mundt for review and noticing a couple of missed codepaths for injection on output. Thanks to Lance Wang for pointing out a memory leak. Signed-off-by: Devin Heitmueller --- libavfilter/tinterlace.h | 2 ++ libavfilter/vf_tinterlace.c | 11 +++++++++++ 2 files changed, 13 insertions(+) diff --git a/libavfilter/tinterlace.h b/libavfilter/tinterlace.h index 37b6c10..9f5ce7e 100644 --- a/libavfilter/tinterlace.h +++ b/libavfilter/tinterlace.h @@ -32,6 +32,7 @@ #include "libavutil/pixdesc.h" #include "drawutils.h" #include "avfilter.h" +#include "ccfifo.h" #define TINTERLACE_FLAG_VLPF 01 #define TINTERLACE_FLAG_CVLPF 2 @@ -77,6 +78,7 @@ typedef struct TInterlaceContext { const AVPixFmtDescriptor *csp; void (*lowpass_line)(uint8_t *dstp, ptrdiff_t width, const uint8_t *srcp, ptrdiff_t mref, ptrdiff_t pref, int clip_max); + AVCCFifo *cc_fifo; } TInterlaceContext; void ff_tinterlace_init_x86(TInterlaceContext *interlace); diff --git a/libavfilter/vf_tinterlace.c b/libavfilter/vf_tinterlace.c index 0326292..8716a94 100644 --- a/libavfilter/vf_tinterlace.c +++ b/libavfilter/vf_tinterlace.c @@ -203,6 +203,7 @@ static av_cold void uninit(AVFilterContext *ctx) av_frame_free(&tinterlace->next); av_freep(&tinterlace->black_data[0][0]); av_freep(&tinterlace->black_data[1][0]); + ff_ccfifo_freep(&tinterlace->cc_fifo); } static int config_out_props(AVFilterLink *outlink) @@ -291,6 +292,11 @@ static int config_out_props(AVFilterLink *outlink) #endif } + if (!(tinterlace->cc_fifo = ff_ccfifo_alloc(outlink->frame_rate, ctx))) { + av_log(ctx, AV_LOG_ERROR, "Failure to setup CC FIFO queue\n"); + return AVERROR(ENOMEM); + } + av_log(ctx, AV_LOG_VERBOSE, "mode:%d filter:%s h:%d -> h:%d\n", tinterlace->mode, (tinterlace->flags & TINTERLACE_FLAG_CVLPF) ? "complex" : (tinterlace->flags & TINTERLACE_FLAG_VLPF) ? "linear" : "off", @@ -375,6 +381,8 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) tinterlace->cur = tinterlace->next; tinterlace->next = picref; + ff_ccfifo_extract(tinterlace->cc_fifo, picref); + cur = tinterlace->cur; next = tinterlace->next; /* we need at least two frames */ @@ -451,6 +459,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) if (!out) return AVERROR(ENOMEM); out->pts /= 2; // adjust pts to new framerate + ff_ccfifo_inject(tinterlace->cc_fifo, out); ret = ff_filter_frame(outlink, out); return ret; } @@ -486,6 +495,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) out->pts = cur->pts*2; out->pts = av_rescale_q(out->pts, tinterlace->preout_time_base, outlink->time_base); + ff_ccfifo_inject(tinterlace->cc_fifo, out); if ((ret = ff_filter_frame(outlink, out)) < 0) return ret; @@ -521,6 +531,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) out->pts = av_rescale_q(out->pts, tinterlace->preout_time_base, outlink->time_base); out->duration = av_rescale_q(1, av_inv_q(outlink->frame_rate), outlink->time_base); + ff_ccfifo_inject(tinterlace->cc_fifo, out); ret = ff_filter_frame(outlink, out); return ret; From patchwork Fri May 5 19:09:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 41500 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:dca6:b0:f3:34fa:f187 with SMTP id ky38csp690457pzb; Fri, 5 May 2023 11:21:52 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7XwlO716jr72fSmiQIaGZgZ5j2RiyVBbi20eTpbg2ge3lsezuajhldwxR+Om033WuYMV0O X-Received: by 2002:a17:907:74c:b0:953:37eb:7727 with SMTP id xc12-20020a170907074c00b0095337eb7727mr2134978ejb.43.1683310912187; Fri, 05 May 2023 11:21:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683310912; cv=none; d=google.com; s=arc-20160816; b=j4Cy84NLeCHilzTX/f58KZ08GB0ZpS0q9H7/9kPREZNBhuNw37UU2wlXgwv06siTlE X01QgfBkmxKm7JrmUVxR+oI9OdXpR0AMqdzKWhyNlcU2nUR1oahdpfRJu+r7RJ1755Wl qTm4B7pLA/UhjlWoiVLR4zI66kmchNR2tssdCU6v4WUhVXYemJQqNfhgVrgiKJgXLCsT vN6dra3kM0Oqq41O1D18HfqWTLrBDXG/D5EzlHp2VzjZiCnOjj2yj6Bc8KiC9dps4oiS 9OPweO8Bcwc+dvs4baj6M3h5wRrsc4I90J2rV/OwmzVY5AvBXZsOqUkSDKsmGODL0KvD rUhg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=OyBub1Xg+VFvNiP/+n0WtRAVGO+tM6fGawsnRuaLl90=; b=YVw8UukKJh97ktuq9+DaWJ2D24Smdq4Vg4TQ5gnr8aoZe7oBFmbUdF9Y1aliB1C66+ 44brvcHWuLjlPGy9RUbG8y92oYXqdUMDlnpumD1Cbbk4vgWwhrbpzkDk7GHpCO+K4gUn 4UmV83hmMQG7nml7R/bz+w0bZJICu3rFnwYRYH9iCep0XF7yjWKxxP8YQ+p+Lozz3ikJ x43jajelYTkR4FcS2w+6HKN5RW3wDfddy8D/0Kgndo+2+2QsiBHi5RtldKzR4L4RO2l7 N7muY2YLdnCar0DLHIpLUSaGHyQJGE+tJUNwf+INdIwlvBc+JHaFeAvckXjI/dFyLU/q jR7Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=J0EtjJtR; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id kl9-20020a170907994900b0094ae88a6e68si1759787ejc.743.2023.05.05.11.21.51; Fri, 05 May 2023 11:21:52 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=J0EtjJtR; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 36FDE68BFF3; Fri, 5 May 2023 21:21:48 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qk1-f178.google.com (mail-qk1-f178.google.com [209.85.222.178]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A6D8368BFAF for ; Fri, 5 May 2023 21:21:41 +0300 (EEST) Received: by mail-qk1-f178.google.com with SMTP id af79cd13be357-7516d97a63aso190784385a.1 for ; Fri, 05 May 2023 11:21:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20221208.gappssmtp.com; s=20221208; t=1683310900; x=1685902900; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=VK83yqMTe/l2pg95fYViHCLfcPSlu7eURCol8Hq+x0E=; b=J0EtjJtRG9OKJTfjXaBOj8gwLjN8F1OUHOQmenO90/k/h4nmMs9Mu0xAeRs1DwkJY1 aOqZMS/Q1rOhuqaIKgFW18RPUQUub14nyux9q111WcrUCCQMO7qYbP8UGHapJl87Xuba a68ZmCtrpLjNKms9qrk1kfQlzNrtgDaofPL1p5EwWh0J3+ryW2jlJ77G68TLjPs8TNiR 0lwcxs06Xb4WlYFx79v9GmqalDVf91q5GguFnAaTNi8LGFILWWo+Hnlf0TNV6QzQLzDR AkjFWL1qB39vq0FP2mM5TUxiii/g18qtnStqCKUkmwbwR+pvjrQz8r2wffewDD/KUHJp l7SQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683310900; x=1685902900; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=VK83yqMTe/l2pg95fYViHCLfcPSlu7eURCol8Hq+x0E=; b=hAcuf9rPKdbvpsUBFI8cTUBlXK0dV7qqokg+PZKy1s1v/HjTF6kn1knt7cullZU/z7 KH8UgFpZARmkZ84qF6q1aS9Y03MG9do4l4XQOnK2bjZtxBtf+CTsMDCJgqhLvmkSYkkk 5D8uw21N3rTsZ0p2kSLILJZbneZWG37EIdvZ09AwbT3nSiUumj6DOAXMQOhw211uIApW HgR7PKtvDeniCpbBi87AeWr2yEbbt0XFk4SHf/M8Z7UxKyDTlFCe/ScIt3OAYWbqYpgW 0O+rXXYXUbiK0aRqoEGBYU3u3ojTndYMYdaUmBFT+3CTvjxo7Ayl6DZqPMukbEBio41G /CvQ== X-Gm-Message-State: AC+VfDxefnjAeJ+k04lEAG4RFV2UNOCco8N4zVHgEV4VTZrxRhVo9h3i y41cHa1tYpvQFV6qT2cV43sXQBvok557T6ty9io= X-Received: by 2002:a05:622a:13cc:b0:3ef:2c3a:faa3 with SMTP id p12-20020a05622a13cc00b003ef2c3afaa3mr4372037qtk.17.1683310403569; Fri, 05 May 2023 11:13:23 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id d1-20020a05620a166100b0074e26158cf2sm747289qko.93.2023.05.05.11.13.23 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 May 2023 11:13:23 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 5 May 2023 15:09:06 -0400 Message-Id: <1683313747-3775-6-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1683313747-3775-1-git-send-email-dheitmueller@ltnglobal.com> References: <1683313747-3775-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [PATCH v6 5/6] vf_ccrepack: Add new filter to repack CEA-708 side data X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: McOp4HeRjmyy THis filter can correct certain issues seen from upstream sources where the cc_count is not properly set or the CEA-608 tuples are not at the start of the payload as expected. Make use of the ccfifo to extract and immediately repack the CEA-708 side data, thereby removing any extra padding and ensuring the 608 tuples are at the front of the payload. Signed-off-by: Devin Heitmueller --- doc/filters.texi | 10 +++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/vf_ccrepack.c | 102 ++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 114 insertions(+) create mode 100644 libavfilter/vf_ccrepack.c diff --git a/doc/filters.texi b/doc/filters.texi index 9a0fe9c..d425081 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -9070,6 +9070,16 @@ Only deinterlace frames marked as interlaced. The default value is @code{all}. @end table +@section ccrepack + +Repack CEA-708 closed captioning side data + +This filter fixes various issues seen with commerical encoders +related to upstream malformed CEA-708 payloads, specifically +incorrect number of tuples (wrong cc_count for the target FPS), +and incorrect ordering of tuples (i.e. the CEA-608 tuples are not at +the first entries in the payload). + @section cas Apply Contrast Adaptive Sharpen filter to video stream. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 68c8f14..7c2195f 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -214,6 +214,7 @@ OBJS-$(CONFIG_BOXBLUR_OPENCL_FILTER) += vf_avgblur_opencl.o opencl.o \ opencl/avgblur.o boxblur.o OBJS-$(CONFIG_BWDIF_FILTER) += vf_bwdif.o yadif_common.o OBJS-$(CONFIG_CAS_FILTER) += vf_cas.o +OBJS-$(CONFIG_CCREPACK_FILTER) += vf_ccrepack.o OBJS-$(CONFIG_CHROMABER_VULKAN_FILTER) += vf_chromaber_vulkan.o vulkan.o vulkan_filter.o OBJS-$(CONFIG_CHROMAHOLD_FILTER) += vf_chromakey.o OBJS-$(CONFIG_CHROMAKEY_FILTER) += vf_chromakey.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 6994124..ee2ac17 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -198,6 +198,7 @@ extern const AVFilter ff_vf_boxblur; extern const AVFilter ff_vf_boxblur_opencl; extern const AVFilter ff_vf_bwdif; extern const AVFilter ff_vf_cas; +extern const AVFilter ff_vf_ccrepack; extern const AVFilter ff_vf_chromaber_vulkan; extern const AVFilter ff_vf_chromahold; extern const AVFilter ff_vf_chromakey; diff --git a/libavfilter/vf_ccrepack.c b/libavfilter/vf_ccrepack.c new file mode 100644 index 0000000..e3fd67f --- /dev/null +++ b/libavfilter/vf_ccrepack.c @@ -0,0 +1,102 @@ +/* + * CEA-708 Closed Caption Repacker + * Copyright (c) 2023 LTN Global Communications + * + * Author: Devin Heitmueller + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/* + * Repackage CEA-708 arrays, which deals with incorrect cc_count for a given + * output framerate, and incorrect 708 padding. + * + * See CEA CEA-10-A "EIA-708-B Implementation Guidance", Section 26.5 + * "Grouping DTVCC Data Within user_data() Structure" + */ + +#include "avfilter.h" +#include "internal.h" +#include "ccfifo.h" +#include "libavutil/opt.h" + +typedef struct CCRepackContext +{ + const AVClass *class; + AVCCFifo *cc_fifo; +} CCRepackContext; + +static const AVOption ccrepack_options[] = { + { NULL } +}; + +AVFILTER_DEFINE_CLASS(ccrepack); + +static int config_input(AVFilterLink *link) +{ + CCRepackContext *ctx = link->dst->priv; + + if (!(ctx->cc_fifo = ff_ccfifo_alloc(link->frame_rate, ctx))) { + av_log(ctx, AV_LOG_ERROR, "Failure to setup CC FIFO queue\n"); + return AVERROR(ENOMEM); + } + + return 0; +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *frame) +{ + CCRepackContext *ctx = inlink->dst->priv; + AVFilterLink *outlink = inlink->dst->outputs[0]; + + ff_ccfifo_extract(ctx->cc_fifo, frame); + ff_ccfifo_inject(ctx->cc_fifo, frame); + + return ff_filter_frame(outlink, frame); +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + CCRepackContext *s = ctx->priv; + ff_ccfifo_freep(&s->cc_fifo); +} + +static const AVFilterPad avfilter_vf_ccrepack_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .filter_frame = filter_frame, + .config_props = config_input, + }, +}; + +static const AVFilterPad avfilter_vf_ccrepack_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + }, +}; + +AVFilter ff_vf_ccrepack = { + .name = "ccrepack", + .description = NULL_IF_CONFIG_SMALL("Repack CEA-708 closed caption metadata"), + .uninit = uninit, + .priv_size = sizeof(CCRepackContext), + .priv_class = &ccrepack_class, + FILTER_INPUTS(avfilter_vf_ccrepack_inputs), + FILTER_OUTPUTS(avfilter_vf_ccrepack_outputs), +}; From patchwork Fri May 5 19:09:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 41498 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:dca6:b0:f3:34fa:f187 with SMTP id ky38csp685837pzb; Fri, 5 May 2023 11:14:15 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4Xy82KlScC7hap2JwA5fY4rfKq9uV66viWnnMCE9Iqynn1oQoq5R6hQO3fVET5N+LaOk4/ X-Received: by 2002:a17:907:724e:b0:94a:6f1d:54df with SMTP id ds14-20020a170907724e00b0094a6f1d54dfmr2246351ejc.67.1683310454811; Fri, 05 May 2023 11:14:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683310454; cv=none; d=google.com; s=arc-20160816; b=tLzji0/btQXgzysfwdohfxetsgYKM8BvWM0EhnwiyOSYAPXOixK1+iDud5AWU+krlQ oWhKgH1RJhlr14xZrY/vK5R/PnZiY9943MQy07Mo2LwYbkDi857LrghISihBtnw/c1Ac HPXQJtTMLDK85GqIj/GIQF16A+cKZFyt6sYA6E9anIL8BYS+35lzQvKMSE7AbNHvFsz3 GFsN6dqLB+lGvH+v7QPf9KlMOF6bL8ooRdyL/UKmZnt7A0rq/RLavKCRIFjJkisA86z9 8LcLpu/bbwhIqxeg+bPnSZR3j7bJm3huxoOtlKeK1ygPifPHTyAyM8+OU1UcJysBlfuh TkfQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=RhfARq+TsdbvsHUpS9ShGbmqkLo8aYDj8RqzOhYYibg=; b=e8FRysTdKzrXpuZ5Edq5qnlfvAsOy8DSXeSSBhoB4KXg9PdWCOuNkn5LKuBexaP3g9 nFHhXfNCLPkJpNPEO0UuVQVMBA6tJa7JNuXAxD5HpjaxuyZWnMtGyqQVVUS5X0D8HWRX eUn86r86k/RrULwx4TWwkRXU36Rzx0kWx21ktm26qBoU17Q0dNgLytaHyEob3swusg5H VR7tHQXiVIoKnSxi+9uLmZXtN9uls2qUxcQiAVsQcPruX5pZFYr5Mb9cBaTPUwOaXlil OOmI02qmI0zTE/TgFH/folsSNCgf3S9bNWp4zn3YkgpSPhQajIfwFSjZ/ha5oJtke1Ep VUhg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=ZztL1uCb; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id lc16-20020a170906f91000b0094f8f6a912bsi1461619ejb.416.2023.05.05.11.14.14; Fri, 05 May 2023 11:14:14 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=ZztL1uCb; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8D2BE68C19B; Fri, 5 May 2023 21:13:33 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qt1-f175.google.com (mail-qt1-f175.google.com [209.85.160.175]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9FA9868C17E for ; Fri, 5 May 2023 21:13:25 +0300 (EEST) Received: by mail-qt1-f175.google.com with SMTP id d75a77b69052e-3ef33a83ff1so9868101cf.1 for ; Fri, 05 May 2023 11:13:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20221208.gappssmtp.com; s=20221208; t=1683310404; x=1685902404; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=KWa1d9LQ33UJDbR44HgPCk5kbWfV1BdRusHOBdIv6go=; b=ZztL1uCbqk5J0Zhixn/tjDD0UmGYvPvL5K2DLdd61Vns6my3YAc91RZb09G8mSzAam Pw+q0P0dmYfDdqr9598hBGTV7qUcw7i1qFiLRUPhumtugcDsfRYMb+jzgUkszCNRyze+ Eo40JapfjPbdg4VTARKbJH1h4m6Lz2dO0DUs1lzEE7O4s+7m7OO5VLKJG3cJbjwtVUA+ otrJNevmBOq5YCC6km5C1P47ipHrsfxJwkvp9GoqTE4M0upLy1rtf+h0VlzSEVlnvHkS 2IT+Xl5cOn/y2odfd0zECOHo3FpDqLsWqwxExhmv5tsJ6qwLqt7a8N/xbRLWMWNkpTzy BF3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683310404; x=1685902404; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=KWa1d9LQ33UJDbR44HgPCk5kbWfV1BdRusHOBdIv6go=; b=F1PSA12uY2Fqs4P+XH2A9GjEnoQ/cUEP2u5NzgKNQCxUiuhqBFPwj1ypqltTZ9CtsT MgGIYr8+We+Cpuz//ZDfy3dLR6wOKIG88OW6SNJT34iyORAv8KyqKJ4R30CMbbLxws9s fQ/VtWOo6eDcou9v6dVTzjQO4IZr12sBegT1RiPjaUjm1+9fQWbP4jtXHVILuAMbJQ6t NVYjhOXyGk8avcnEOMcw31xZcb3C7dC4e7jsmmPCWepWHAyZuxhsRVwnI3FUYqaneMi+ ldE0YJkoIdZlj81RaitXyJdXnfrd9uOYlC4m6FnnFHPVCvbKh2EnN9YLruqUkzJRFT/o bd2g== X-Gm-Message-State: AC+VfDwnMGirjsSX1cnxrk6cijUA806Uc3SchOFR9JJjF5RDf2y0lzTs K8iCKa5uMPRP9Fb9aDhRmLkguxrdTkrrvcI7yHI= X-Received: by 2002:a05:622a:14cc:b0:3ef:336f:56c6 with SMTP id u12-20020a05622a14cc00b003ef336f56c6mr4321542qtx.17.1683310404209; Fri, 05 May 2023 11:13:24 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id d1-20020a05620a166100b0074e26158cf2sm747289qko.93.2023.05.05.11.13.23 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 May 2023 11:13:23 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 5 May 2023 15:09:07 -0400 Message-Id: <1683313747-3775-7-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1683313747-3775-1-git-send-email-dheitmueller@ltnglobal.com> References: <1683313747-3775-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [PATCH v6 6/6] decklink_enc: add support for playout of 608 captions in MOV files X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: BWCcmAEcGG8y Unlike other cases where the closed captions are embedded in the video stream as MPEG-2 userdata or H.264 SEI data, with MOV files the captions are often found on a separate "e608" subtitle track. Add support for playout of such files, leveraging the new ccfifo mechanism to ensure that they are embedded into VANC at the correct rate (since e608 packets often contain batches of multiple 608 pairs). Note this patch includes a new file named libavdevice/ccfifo.c, which allows the ccfifo functionality in libavfilter to be reused even if doing shared builds. This is the same approach used for log2_tab.c. Signed-off-by: Devin Heitmueller --- libavdevice/Makefile | 1 + libavdevice/ccfifo.c | 24 ++++++++++++++++ libavdevice/decklink_common.h | 3 ++ libavdevice/decklink_enc.cpp | 65 +++++++++++++++++++++++++++++++++++++++++++ libavdevice/decklink_enc_c.c | 2 +- 5 files changed, 94 insertions(+), 1 deletion(-) create mode 100644 libavdevice/ccfifo.c diff --git a/libavdevice/Makefile b/libavdevice/Makefile index 8a62822..c304492 100644 --- a/libavdevice/Makefile +++ b/libavdevice/Makefile @@ -57,6 +57,7 @@ OBJS-$(CONFIG_LIBDC1394_INDEV) += libdc1394.o # Objects duplicated from other libraries for shared builds SHLIBOBJS-$(CONFIG_DECKLINK_INDEV) += reverse.o +SHLIBOBJS-$(CONFIG_DECKLINK_OUTDEV) += ccfifo.o # Windows resource file SHLIBOBJS-$(HAVE_GNU_WINDRES) += avdeviceres.o diff --git a/libavdevice/ccfifo.c b/libavdevice/ccfifo.c new file mode 100644 index 0000000..9007094 --- /dev/null +++ b/libavdevice/ccfifo.c @@ -0,0 +1,24 @@ +/* + * CEA-708 Closed Captioning FIFO + * Copyright (c) 2023 LTN Global Communications + * + * Author: Devin Heitmueller + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavfilter/ccfifo.c" diff --git a/libavdevice/decklink_common.h b/libavdevice/decklink_common.h index 088e165..0d33f94 100644 --- a/libavdevice/decklink_common.h +++ b/libavdevice/decklink_common.h @@ -31,6 +31,7 @@ extern "C" { #include "libavcodec/packet_internal.h" +#include "libavfilter/ccfifo.h" } #include "libavutil/thread.h" #include "decklink_common_c.h" @@ -112,6 +113,8 @@ struct decklink_ctx { /* Capture buffer queue */ AVPacketQueue queue; + AVCCFifo *cc_fifo; ///< closed captions + /* Streams present */ int audio; int video; diff --git a/libavdevice/decklink_enc.cpp b/libavdevice/decklink_enc.cpp index 9f1a8df..22aa80b 100644 --- a/libavdevice/decklink_enc.cpp +++ b/libavdevice/decklink_enc.cpp @@ -326,6 +326,25 @@ static int create_s337_payload(AVPacket *pkt, uint8_t **outbuf, int *outsize) return 0; } +static int decklink_setup_subtitle(AVFormatContext *avctx, AVStream *st) +{ + int ret = -1; + + switch(st->codecpar->codec_id) { +#if CONFIG_LIBKLVANC + case AV_CODEC_ID_EIA_608: + /* No special setup required */ + ret = 0; + break; +#endif + default: + av_log(avctx, AV_LOG_ERROR, "Unsupported subtitle codec specified\n"); + break; + } + + return ret; +} + av_cold int ff_decklink_write_trailer(AVFormatContext *avctx) { struct decklink_cctx *cctx = (struct decklink_cctx *)avctx->priv_data; @@ -352,6 +371,7 @@ av_cold int ff_decklink_write_trailer(AVFormatContext *avctx) klvanc_context_destroy(ctx->vanc_ctx); #endif + ff_ccfifo_freep(&ctx->cc_fifo); av_freep(&cctx->ctx); return 0; @@ -503,6 +523,21 @@ out: free(afd_words); } +/* Parse any EIA-608 subtitles sitting on the queue, and write packet side data + that will later be handled by construct_cc... */ +static void parse_608subs(AVFormatContext *avctx, struct decklink_ctx *ctx, AVPacket *pkt) +{ + size_t cc_size = ff_ccfifo_getoutputsize(ctx->cc_fifo); + uint8_t *cc_data; + + if (!ff_ccfifo_ccdetected(ctx->cc_fifo)) + return; + + cc_data = av_packet_new_side_data(pkt, AV_PKT_DATA_A53_CC, cc_size); + if (cc_data) + ff_ccfifo_injectbytes(ctx->cc_fifo, cc_data, cc_size); +} + static int decklink_construct_vanc(AVFormatContext *avctx, struct decklink_ctx *ctx, AVPacket *pkt, decklink_frame *frame, AVStream *st) @@ -513,6 +548,7 @@ static int decklink_construct_vanc(AVFormatContext *avctx, struct decklink_ctx * if (!ctx->supports_vanc) return 0; + parse_608subs(avctx, ctx, pkt); construct_cc(avctx, ctx, pkt, &vanc_lines); construct_afd(avctx, ctx, pkt, &vanc_lines, st); @@ -704,6 +740,16 @@ static int decklink_write_audio_packet(AVFormatContext *avctx, AVPacket *pkt) return ret; } +static int decklink_write_subtitle_packet(AVFormatContext *avctx, AVPacket *pkt) +{ + struct decklink_cctx *cctx = (struct decklink_cctx *)avctx->priv_data; + struct decklink_ctx *ctx = (struct decklink_ctx *)cctx->ctx; + + ff_ccfifo_extractbytes(ctx->cc_fifo, pkt->data, pkt->size); + + return 0; +} + extern "C" { av_cold int ff_decklink_write_header(AVFormatContext *avctx) @@ -768,12 +814,29 @@ av_cold int ff_decklink_write_header(AVFormatContext *avctx) } else if (c->codec_type == AVMEDIA_TYPE_VIDEO) { if (decklink_setup_video(avctx, st)) goto error; + } else if (c->codec_type == AVMEDIA_TYPE_SUBTITLE) { + if (decklink_setup_subtitle(avctx, st)) + goto error; } else { av_log(avctx, AV_LOG_ERROR, "Unsupported stream type.\n"); goto error; } } + for (n = 0; n < avctx->nb_streams; n++) { + AVStream *st = avctx->streams[n]; + AVCodecParameters *c = st->codecpar; + + if(c->codec_type == AVMEDIA_TYPE_SUBTITLE) + avpriv_set_pts_info(st, 64, ctx->bmd_tb_num, ctx->bmd_tb_den); + } + + if (!(ctx->cc_fifo = ff_ccfifo_alloc(av_make_q(ctx->bmd_tb_den, ctx->bmd_tb_num), avctx))) { + av_log(ctx, AV_LOG_ERROR, "Failure to setup CC FIFO queue\n"); + ret = AVERROR(ENOMEM); + goto error; + } + return 0; error: @@ -789,6 +852,8 @@ int ff_decklink_write_packet(AVFormatContext *avctx, AVPacket *pkt) return decklink_write_video_packet(avctx, pkt); else if (st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO) return decklink_write_audio_packet(avctx, pkt); + else if (st->codecpar->codec_type == AVMEDIA_TYPE_SUBTITLE) + return decklink_write_subtitle_packet(avctx, pkt); return AVERROR(EIO); } diff --git a/libavdevice/decklink_enc_c.c b/libavdevice/decklink_enc_c.c index f7e3150..0a3984b 100644 --- a/libavdevice/decklink_enc_c.c +++ b/libavdevice/decklink_enc_c.c @@ -77,7 +77,7 @@ const FFOutputFormat ff_decklink_muxer = { .p.long_name = NULL_IF_CONFIG_SMALL("Blackmagic DeckLink output"), .p.audio_codec = AV_CODEC_ID_PCM_S16LE, .p.video_codec = AV_CODEC_ID_WRAPPED_AVFRAME, - .p.subtitle_codec = AV_CODEC_ID_NONE, + .p.subtitle_codec = AV_CODEC_ID_EIA_608, .p.flags = AVFMT_NOFILE, .p.priv_class = &decklink_muxer_class, .get_device_list = ff_decklink_list_output_devices,