From patchwork Thu May 4 22:02:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 41471 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:dca6:b0:f3:34fa:f187 with SMTP id ky38csp12560pzb; Thu, 4 May 2023 14:07:10 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ77FLucBbB8xF8t27iv07sK4R7dw1W9hl6ZPDqRT+u+giwBv+37XdRcAz6ghv+VD7tPKW/V X-Received: by 2002:a17:907:1b1a:b0:94e:6a24:9463 with SMTP id mp26-20020a1709071b1a00b0094e6a249463mr231967ejc.28.1683234429588; Thu, 04 May 2023 14:07:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683234429; cv=none; d=google.com; s=arc-20160816; b=CgXdcEDOi+KAysTdbcbQo1m4U4C02gLbv2+u/mLzY13KmFwt7ttT0dN0WvxAt9rjq9 ERkVxWT0jOBbF83xvvdytrraxVDnT9RpC2beT800eoC4O2n2H8Ig3nARhErdJr/l13dX z/29usSKaKr9c1htbdNPJXfPnd5IjiG/rHtiE1MWd/7IaptkXwyowKBup12UMzh4dvQY u1hwIVAoHPTZ5soVHEDacTVBbXZCvtn338rJxxsLxEKTnVTOt8nwqX43fbJWnJyq1Onj rlRIiQoo6s1vT+CdTCEKOAtVgKNHipknoP183AN49ACDx8X6qxpWFfe5RVVxo3cIlbKK T+sQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=PyaCM6K80713d8R0MrRYZKpV4s+apUf6A1KX5WLGKmY=; b=Ws6zKIqMjJQGeT0h0+KRSeAOrFYh7ZnURXL+S0zPDCfflvQmHpTgo8+sYH++di9SXn gEHp4lMNO7qVPftT0hTTYTDGv8CH3TVoNxSa7ukRZEZHmoWUWM61q+vUeiONxtb6GZdh MYFhaY/gsYdcOEBN+IsuG8PC8tpIhUxSMA14+pFRNr/1cko+4Re5H+cR0Ac0Q8fvhNbC RfMt99FoPFlqKHw8xHY1YbxYMO7lbvHB6Bz0cJEHSHLoHM7OkbWCdbUYnmSJdxHpdbi+ txcF7+pO4oE32GqOKh9hgMKy0O6Gcq5ebH1KpQs//DL0tKUZOUkDGwt7etz3SDGdttFG DIdA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=Ap8f8CrN; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id h23-20020a1709067cd700b00965c55d9e9csi32151ejp.670.2023.05.04.14.07.06; Thu, 04 May 2023 14:07:09 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=Ap8f8CrN; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 280D368C114; Fri, 5 May 2023 00:06:46 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qt1-f177.google.com (mail-qt1-f177.google.com [209.85.160.177]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2A53C68BF32 for ; Fri, 5 May 2023 00:06:38 +0300 (EEST) Received: by mail-qt1-f177.google.com with SMTP id d75a77b69052e-3ef657f5702so9325691cf.3 for ; Thu, 04 May 2023 14:06:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20221208.gappssmtp.com; s=20221208; t=1683234396; x=1685826396; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=tyPHGzpCmp1hdJYtS97OMfjG4RuGZSNij4Tyx+P3upk=; b=Ap8f8CrNWDhpnsKolaRvbKuX3M49SdBs5TQtdGZbasOYAezqIswxF9m9k5A2ZBevsU RaQt66cCjnDBWpg5wHGQye7zCgFC7kn3v40dQ0NizVqFKJIDiXQkCNnzVQqeenOM6Ktk PAvsi5jW4yAcuLL7rxDxAMa3cDIMMNIRf3bopaVJ2fSKRwKpBw7x1YQYvS/nrKOXzPax Bt14AL2WgfBKXDhMmU7UBB+a9pV141m6Mq0S1A9BjTuOP+8gCBzUzB//UOcO3B58/d0/ r3X5BROmrkg3uhWUtxbOJsGcv/7NwmFtYXuU3gpsw1SL0KIrzQqiZqenA+nHW8HTfKqu hOVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683234396; x=1685826396; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=tyPHGzpCmp1hdJYtS97OMfjG4RuGZSNij4Tyx+P3upk=; b=QltbNZaoUmnK+B0aCLyg732qy3dhFPE7AV7UgvAGwErsZ+1xW8oUYUN06qGnHRZNiC rDnz9/Gz0v4PgNihtO6ibZjBl4Zrv948O593TruhogvroTgnT+SZ4G4p0sgDXK6rGSrr XSAOIC1HUpe9KMUuNvQi/FIVvcEjIsmYcAzeMqdgi2DtjRETa71KjgfxaWC8bEXJEGLR TQPpSxUHAJE61zGqzv9pvG/MugS3R6RhkHZg+h2qmRk0+vugGxlxG6GabhzRRPsS+0tg nDUKbkn4eYDZxkAv09AVrHdfcExbXIx+iIngs3OpWN/ZN6jijz+/F15ybAwwwOU8mBQQ 6fFA== X-Gm-Message-State: AC+VfDybR6bKq2eQ4YupK4/SWKpm8bd/j4duyUF0MaVwXT0o/631/Jbb Nku1edC0jP1yuIj2RExJE+sMwn5n2e9m9dvBnWc= X-Received: by 2002:ac8:57c8:0:b0:3ef:3ff5:ce28 with SMTP id w8-20020ac857c8000000b003ef3ff5ce28mr7462278qta.39.1683234396420; Thu, 04 May 2023 14:06:36 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id 4-20020a05620a070400b0074e21c3bc8asm67996qkc.126.2023.05.04.14.06.35 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 04 May 2023 14:06:35 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Thu, 4 May 2023 18:02:15 -0400 Message-Id: <1683237740-32743-2-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1683237740-32743-1-git-send-email-dheitmueller@ltnglobal.com> References: <1683237740-32743-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [PATCH v5 1/6] ccfifo: Properly handle CEA-708 captions through framerate conversion X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 9v+HGnjzAzeZ When transcoding video that contains 708 closed captions, the caption data is tied to the frames as side data. Simply dropping or adding frames to change the framerate will result in loss of data, so the caption data needs to be preserved and reformatted. For example, without this patch converting 720p59 to 1080i59 would result in loss of 50% of the caption bytes, resulting in garbled 608 captions and 708 probably wouldn't render at all. Further, the frames that are there will have an illegal cc_count for the target framerate, so some decoders may ignore the packets entirely. Extract the 608 and 708 tuples and insert them onto queues. Then after dropping/adding frames, re-write the tuples back into the resulting frames at the appropriate rate given the target framerate. This includes both having the correct cc_count as well as clocking out the 608 pairs at the appropriate rate. Thanks for Lance Wang and Anton Khirnov for providing review/feedback. Signed-off-by: Devin Heitmueller --- libavfilter/Makefile | 1 + libavfilter/ccfifo.c | 222 +++++++++++++++++++++++++++++++++++++++++++++++++++ libavfilter/ccfifo.h | 110 +++++++++++++++++++++++++ 3 files changed, 333 insertions(+) create mode 100644 libavfilter/ccfifo.c create mode 100644 libavfilter/ccfifo.h diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 482aeaf..68c8f14 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -14,6 +14,7 @@ OBJS = allfilters.o \ buffersink.o \ buffersrc.o \ colorspace.o \ + ccfifo.o \ drawutils.o \ fifo.o \ formats.o \ diff --git a/libavfilter/ccfifo.c b/libavfilter/ccfifo.c new file mode 100644 index 0000000..357f764 --- /dev/null +++ b/libavfilter/ccfifo.c @@ -0,0 +1,222 @@ +/* + * CEA-708 Closed Captioning FIFO + * Copyright (c) 2023 LTN Global Communications + * + * Author: Devin Heitmueller + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "ccfifo.h" + +struct AVCCFifo { + AVFifo *cc_608_fifo; + AVFifo *cc_708_fifo; + AVRational framerate; + int expected_cc_count; + int expected_608; + int cc_detected; + int passthrough; + int passthrough_warning; + void *log_ctx; +}; + +#define MAX_CC_ELEMENTS 128 +#define CC_BYTES_PER_ENTRY 3 + +struct cc_lookup { + int num; + int den; + int cc_count; + int num_608; +}; + +const static struct cc_lookup cc_lookup_vals[] = { + { 15, 1, 40, 4 }, + { 24, 1, 25, 3 }, + { 24000, 1001, 25, 3 }, + { 30, 1, 20, 2 }, + { 30000, 1001, 20, 2}, + { 60, 1, 10, 1 }, + { 60000, 1001, 10, 1}, +}; + +void ff_ccfifo_freep(AVCCFifo **ccf) +{ + AVCCFifo *tmp = *ccf; + av_fifo_freep2(&tmp->cc_608_fifo); + av_fifo_freep2(&tmp->cc_708_fifo); + av_freep(*ccf); +} + +AVCCFifo *ff_ccfifo_alloc(AVRational framerate, void *log_ctx) +{ + AVCCFifo *ccf; + int i; + + ccf = av_mallocz(sizeof(*ccf)); + if (!ccf) + return NULL; + + ccf->log_ctx = log_ctx; + ccf->framerate = framerate; + + if (!(ccf->cc_708_fifo = av_fifo_alloc2(MAX_CC_ELEMENTS, CC_BYTES_PER_ENTRY, 0))) + goto error; + + if (!(ccf->cc_608_fifo = av_fifo_alloc2(MAX_CC_ELEMENTS, CC_BYTES_PER_ENTRY, 0))) + goto error; + + /* Based on the target FPS, figure out the expected cc_count and number of + 608 tuples per packet. See ANSI/CTA-708-E Sec 4.3.6.1. */ + for (i = 0; i < FF_ARRAY_ELEMS(cc_lookup_vals); i++) { + if (framerate.num == cc_lookup_vals[i].num && + framerate.den == cc_lookup_vals[i].den) { + ccf->expected_cc_count = cc_lookup_vals[i].cc_count; + ccf->expected_608 = cc_lookup_vals[i].num_608; + break; + } + } + + if (ccf->expected_608 == 0) { + /* We didn't find an output frame we support. We'll let the call succeed + and the FIFO to be allocated, but the extract/inject functions will simply + leave everything the way it is */ + ccf->passthrough = 1; + } + + return ccf; + +error: + ff_ccfifo_freep(&ccf); + return NULL; +} + +int ff_ccfifo_getoutputsize(AVCCFifo *ccf) +{ + return ccf->expected_cc_count * CC_BYTES_PER_ENTRY; +} + +int ff_ccfifo_ccdetected(AVCCFifo *ccf) +{ + return ccf->cc_detected; +} + +int ff_ccfifo_injectbytes(AVCCFifo *ccf, uint8_t *cc_data, size_t len) +{ + int cc_608_tuples = 0; + int cc_708_tuples = 0; + int cc_filled = 0; + + if (ccf->passthrough) { + return 0; + } + + if (len < ff_ccfifo_getoutputsize(ccf)) { + return AVERROR(EINVAL); + } + + /* Insert any available data from the 608 FIFO */ + if (ccf->expected_608 <= av_fifo_can_read(ccf->cc_608_fifo)) + cc_608_tuples = ccf->expected_608; + else + cc_608_tuples = av_fifo_can_read(ccf->cc_608_fifo); + av_fifo_read(ccf->cc_608_fifo, cc_data, cc_608_tuples); + cc_filled += cc_608_tuples; + + /* Insert any available data from the 708 FIFO */ + if ((ccf->expected_cc_count - cc_filled) <= av_fifo_can_read(ccf->cc_708_fifo)) + cc_708_tuples = ccf->expected_cc_count - cc_filled; + else + cc_708_tuples = av_fifo_can_read(ccf->cc_708_fifo); + av_fifo_read(ccf->cc_708_fifo, &cc_data[cc_filled * CC_BYTES_PER_ENTRY], cc_708_tuples); + cc_filled += cc_708_tuples; + + /* Insert 708 padding into any remaining fields */ + while (cc_filled < ccf->expected_cc_count) { + cc_data[cc_filled * CC_BYTES_PER_ENTRY] = 0xfa; + cc_data[cc_filled * CC_BYTES_PER_ENTRY + 1] = 0x00; + cc_data[cc_filled * CC_BYTES_PER_ENTRY + 2] = 0x00; + cc_filled++; + } + + return 0; +} + +int ff_ccfifo_inject(AVCCFifo *ccf, AVFrame *frame) +{ + AVFrameSideData *sd; + int ret; + + if (ccf->passthrough == 1 || ccf->cc_detected == 0) + return 0; + + sd = av_frame_new_side_data(frame, AV_FRAME_DATA_A53_CC, + ff_ccfifo_getoutputsize(ccf)); + if (sd) { + ret = ff_ccfifo_injectbytes(ccf, sd->data, sd->size); + if (ret < 0) { + av_frame_remove_side_data(frame, AV_FRAME_DATA_A53_CC); + return AVERROR(ENOMEM); + } + } + + return 0; +} + +int ff_ccfifo_extractbytes(AVCCFifo *ccf, uint8_t *cc_bytes, size_t len) +{ + int cc_count = len / CC_BYTES_PER_ENTRY; + + if (ccf->passthrough == 1) { + av_log_once(ccf->log_ctx, AV_LOG_WARNING, AV_LOG_DEBUG, &ccf->passthrough_warning, + "cc_fifo cannot transcode captions fps=%d/%d\n", + ccf->framerate.num, ccf->framerate.den); + return 0; + } + + ccf->cc_detected = 1; + + for (int i = 0; i < cc_count; i++) { + /* See ANSI/CTA-708-E Sec 4.3, Table 3 */ + uint8_t cc_valid = (cc_bytes[CC_BYTES_PER_ENTRY*i] & 0x04) >> 2; + uint8_t cc_type = cc_bytes[CC_BYTES_PER_ENTRY*i] & 0x03; + if (cc_type == 0x00 || cc_type == 0x01) { + av_fifo_write(ccf->cc_608_fifo, &cc_bytes[CC_BYTES_PER_ENTRY*i], 1); + } else if (cc_valid && (cc_type == 0x02 || cc_type == 0x03)) { + av_fifo_write(ccf->cc_708_fifo, &cc_bytes[CC_BYTES_PER_ENTRY*i], 1); + } + } + return 0; +} + +/* Read the A53 side data, discard padding, and put 608/708 into + queues so we can ensure they get into the output frames at + the correct rate... */ +int ff_ccfifo_extract(AVCCFifo *ccf, AVFrame *frame) +{ + AVFrameSideData *side_data = av_frame_get_side_data(frame, AV_FRAME_DATA_A53_CC); + if (side_data) { + ff_ccfifo_extractbytes(ccf, side_data->data, side_data->size); + + /* Remove the side data, as we will re-create it on the + output as needed */ + if (!ccf->passthrough) + av_frame_remove_side_data(frame, AV_FRAME_DATA_A53_CC); + } + return 0; +} diff --git a/libavfilter/ccfifo.h b/libavfilter/ccfifo.h new file mode 100644 index 0000000..44c9245 --- /dev/null +++ b/libavfilter/ccfifo.h @@ -0,0 +1,110 @@ +/* + * CEA-708 Closed Captioning FIFO + * Copyright (c) 2023 LTN Global Communications + * + * Author: Devin Heitmueller + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * CC FIFO Buffer + */ + +#ifndef AVFILTER_CCFIFO_H +#define AVFILTER_CCFIFO_H + +#include "libavutil/avutil.h" +#include "libavutil/frame.h" +#include "libavutil/fifo.h" + +typedef struct AVCCFifo AVCCFifo; + +/** + * Allocate an AVCCFifo. + * + * @param framerate output framerate + * @param log_ctx used for any av_log() calls + * @return newly allocated AVCCFifo, or NULL on error + */ +AVCCFifo *ff_ccfifo_alloc(AVRational framerate, void *log_ctx); + +/** + * Free an AVCCFifo + * + * @param ccf Pointer to the pointer to the AVCCFifo which should be freed + * @note `*ptr = NULL` is safe and leads to no action. + */ +void ff_ccfifo_freep(AVCCFifo **ccf); + + +/** + * Extract CC data from an AVFrame + * + * Extract CC bytes from the AVFrame, insert them into our queue, and + * remove the side data from the AVFrame. The side data is removed + * as it will be re-inserted at the appropriate rate later in the + * filter. + * + * @param af AVCCFifo to write to + * @param frame AVFrame with the video frame to operate on + * @return Zero on success, or negative AVERROR + * code on failure. + */ +int ff_ccfifo_extract(AVCCFifo *ccf, AVFrame *frame); + +/** + *Just like ff_ccfifo_extract(), but takes the raw bytes instead of an AVFrame + */ +int ff_ccfifo_extractbytes(AVCCFifo *ccf, uint8_t *data, size_t len); + +/** + * Provide the size in bytes of an output buffer to allocate + * + * Ask for how many bytes the output will contain, so the caller can allocate + * an appropriately sized buffer and pass it to ff_ccfifo_injectbytes() + * + */ +int ff_ccfifo_getoutputsize(AVCCFifo *ccf); + +/** + * Insert CC data from the FIFO into an AVFrame (as side data) + * + * Dequeue the appropriate number of CC tuples based on the + * frame rate, and insert them into the AVFrame + * + * @param af AVCCFifo to read from + * @param frame AVFrame with the video frame to operate on + * @return Zero on success, or negative AVERROR + * code on failure. + */ +int ff_ccfifo_inject(AVCCFifo *ccf, AVFrame *frame); + +/** + * Just like ff_ccfifo_inject(), but takes the raw bytes to insert the CC data + * int rather than an AVFrame + */ +int ff_ccfifo_injectbytes(AVCCFifo *ccf, uint8_t *data, size_t len); + +/** + * Returns 1 if captions have been found as a prior call + * to ff_ccfifo_extract() or ff_ccfifo_extractbytes() + */ +int ff_ccfifo_ccdetected(AVCCFifo *ccf); + +#endif /* AVFILTER_CCFIFO_H */ From patchwork Thu May 4 22:02:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 41472 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:dca6:b0:f3:34fa:f187 with SMTP id ky38csp12684pzb; Thu, 4 May 2023 14:07:24 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5/mrUHHYDvF1nqAFU6YRYGJfwGgdXk8tCUy2wW2j1i7FVoviZOewoUVMSTCA2ThfrA3Y4H X-Received: by 2002:a17:907:268e:b0:94f:9b6a:b5c4 with SMTP id bn14-20020a170907268e00b0094f9b6ab5c4mr127148ejc.41.1683234444264; Thu, 04 May 2023 14:07:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683234444; cv=none; d=google.com; s=arc-20160816; b=U1kbQ75sfO8w7IcMcvBJqAyNNB9noA16Lc5BGyzUWVTE6teTYAfsq/pZls6gpb1nD/ s7LcsjLLbKVw6CV0VbyKexhr2a5t95KBK7WdI/ps8OtSJPjVdbq1OSwlEPuD4fVvnCuH pLBduVTrWNodVOXVei2+VtJe3pRUjO/Xc2Br8Xr5xmsOv0Ji6QPMejSeVa/Wwh51iKjk ztXtxMHbEp/Sk4zICUaYiayV4p/BtLPmekXHNRsP0O1/7pn5CufVUWq3/vi4pf2194V0 XcE4EmbLHqaBwubjMChAABKiF3NKmygGuL2f4zBkotv8QshuHDg+cMfIFM+nCyMMxvzP 6zNg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=i/Ql5yBOaLTnnMeO3UnuNKhMMHjT1qC21ceeShMd/aE=; b=Ku6od8aRcJMHbdERQmcsJUtHdtHQtcPv4kqnecvVtTRAEpFzGt17LuXD41GsGb4IwA xRTYAyrs7B+axDh0YrsyM0cLVdrYOyDoJTYUT9AFzGmiQooXruGsqfuppNojoVPVdMri ssyzbhYpkC/R3YzJqpzHbBYFbaPz+S83QQfr4Pppctf4evSPFePkh8eBiIeT1wXc9Spo DGCOIBuuRtZnjkoGLyc0rh9W55IVO5xNJgZSYoK5hFG4V/Y5x7IpdoFPTnIfpXjq9Brp KFf6v0oWLZ5pYhMTSdWrZUzZQiIEVoSx0mbPsFOfb28b26uOhCkfJaJ5yk/0WL7gv00a zUbQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=qfpW1c02; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id bj12-20020a170906b04c00b009535dac8577si27002ejb.626.2023.05.04.14.07.19; Thu, 04 May 2023 14:07:24 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=qfpW1c02; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4967B68C122; Fri, 5 May 2023 00:06:47 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qt1-f169.google.com (mail-qt1-f169.google.com [209.85.160.169]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 62BA968BF32 for ; Fri, 5 May 2023 00:06:38 +0300 (EEST) Received: by mail-qt1-f169.google.com with SMTP id d75a77b69052e-3ef35d44612so9425721cf.1 for ; Thu, 04 May 2023 14:06:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20221208.gappssmtp.com; s=20221208; t=1683234397; x=1685826397; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=VhC2/eK2+zuscWyj1kOpnDgubrv9ynrNIh/81ZaAl1g=; b=qfpW1c02NPafU5Fg2S95iWW+fgPFwKU8SOjm01vmddLj6wk/XOXOf7Kw+7tEshsBBS XnNWku9HH2tbgb9DylFdxtCTPZKzA3V7O2r9hHL9xxZ8Z8UjWVJj67TkzpXD2g9imFpw xUBiR3ryTyMrO1rWMXve9M3CA7bVQ/hWTdt+51AD+1Fk2GerNhlr39ZNR7MvJ7PODq75 8GUkC8/CJGks43XJoed/gFzXremDSy7a2/Uidk/SdkhiHFVuO2hhJYRUvJ/Kffj5APMh x+C8+coco3T+Bkn9iENptVqgDjy84UNzxOK8AQt3Iel1M49oVjLcI0gehrXVbXf1TZI8 I3ug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683234397; x=1685826397; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=VhC2/eK2+zuscWyj1kOpnDgubrv9ynrNIh/81ZaAl1g=; b=gBWwDqPYGKjGQ7Sjgr5YDa67u2GDAitXqrfkITJgMRmxIvISr9kzmcW8IJ1S13zzee 3PHE+KqJTCfCBTKNJ/FdldX9ql7rLR3Xs1b9x5UbW7IckCw12O5dL19XrlFTwGtquKt8 EpcSrq5/mUeC9h4OZw5vmEEnr2pLEXUkSle3IXHzufTSPydOAcVWBiaf0Ao/1E8Lvexo v93E7dfW5619L83lW9Z4o2oB47Pd3YtXcEKAggrReQIhZGOUm/rVK3v6MXcB7tmq5Rtu ST6+FWWIOI/pW1sASqXFgNxhmFXzce1OLhIbe0QujrBH2XpvKrh4UDW5CorbW7Hp3yFV KINQ== X-Gm-Message-State: AC+VfDzJLuwqShotYTqBZ3MhxfdyakoMa0yEVO1T1szitNusgh/wrS9m ggq1gx28n2tp7YNtZJ2ptBiePtsfKQiWyHaG9NY= X-Received: by 2002:ac8:5a8e:0:b0:3f0:ae16:685c with SMTP id c14-20020ac85a8e000000b003f0ae16685cmr7267087qtc.61.1683234396966; Thu, 04 May 2023 14:06:36 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id 4-20020a05620a070400b0074e21c3bc8asm67996qkc.126.2023.05.04.14.06.36 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 04 May 2023 14:06:36 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Thu, 4 May 2023 18:02:16 -0400 Message-Id: <1683237740-32743-3-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1683237740-32743-1-git-send-email-dheitmueller@ltnglobal.com> References: <1683237740-32743-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [PATCH v5 2/6] vf_fps: properly preserve CEA-708 captions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: bM+2uJolhySS The existing implementation made an attempt to remove duplicate captions if increasing the framerate, but made no attempt to handle reducing the framerate, nor did it rewrite the caption payloads to have the appropriate cc_count (e.g. the cc_count needs to change from 20 to 10 when going from 1080i59 to 720p59 and vice-versa). Make use of the new ccfifo mechanism to ensure that caption data is properly preserved. Signed-off-by: Devin Heitmueller --- libavfilter/vf_fps.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/libavfilter/vf_fps.c b/libavfilter/vf_fps.c index 051d278..824e7a1 100644 --- a/libavfilter/vf_fps.c +++ b/libavfilter/vf_fps.c @@ -34,6 +34,7 @@ #include "libavutil/mathematics.h" #include "libavutil/opt.h" #include "avfilter.h" +#include "ccfifo.h" #include "filters.h" #include "internal.h" @@ -85,6 +86,7 @@ typedef struct FPSContext { AVFrame *frames[2]; ///< buffered frames int frames_count; ///< number of buffered frames + AVCCFifo *cc_fifo; ///< closed captions int64_t next_pts; ///< pts of the next frame to output @@ -165,6 +167,7 @@ static av_cold void uninit(AVFilterContext *ctx) frame = shift_frame(ctx, s); av_frame_free(&frame); } + ff_ccfifo_freep(&s->cc_fifo); av_log(ctx, AV_LOG_VERBOSE, "%d frames in, %d frames out; %d frames dropped, " "%d frames duplicated.\n", s->frames_in, s->frames_out, s->drop, s->dup); @@ -210,6 +213,11 @@ static int config_props(AVFilterLink* outlink) s->in_pts_off, s->out_pts_off, s->start_time); } + if (!(s->cc_fifo = ff_ccfifo_alloc(outlink->frame_rate, ctx))) { + av_log(ctx, AV_LOG_ERROR, "Failure to setup CC FIFO queue\n"); + return AVERROR(ENOMEM); + } + av_log(ctx, AV_LOG_VERBOSE, "fps=%d/%d\n", outlink->frame_rate.num, outlink->frame_rate.den); return 0; @@ -242,6 +250,7 @@ static int read_frame(AVFilterContext *ctx, FPSContext *s, AVFilterLink *inlink, av_log(ctx, AV_LOG_DEBUG, "Read frame with in pts %"PRId64", out pts %"PRId64"\n", in_pts, frame->pts); + ff_ccfifo_extract(s->cc_fifo, frame); s->frames[s->frames_count++] = frame; s->frames_in++; @@ -289,7 +298,7 @@ static int write_frame(AVFilterContext *ctx, FPSContext *s, AVFilterLink *outlin if (!frame) return AVERROR(ENOMEM); // Make sure Closed Captions will not be duplicated - av_frame_remove_side_data(s->frames[0], AV_FRAME_DATA_A53_CC); + ff_ccfifo_inject(s->cc_fifo, frame); frame->pts = s->next_pts++; frame->duration = 1; From patchwork Thu May 4 22:02:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 41473 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:dca6:b0:f3:34fa:f187 with SMTP id ky38csp12778pzb; Thu, 4 May 2023 14:07:37 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7NaghDZeKv1w/QLsw1hYc5RcIx4uLu8lbotZ8JS+JxG5c6wSRxhi3SaXff9fzTSiBlgdMP X-Received: by 2002:a17:907:868d:b0:948:d1af:3a11 with SMTP id qa13-20020a170907868d00b00948d1af3a11mr169608ejc.50.1683234457530; Thu, 04 May 2023 14:07:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683234457; cv=none; d=google.com; s=arc-20160816; b=F0OFKlBwXsD6MOLWqKiyNwzky0mj4Jd+0VeLPD+I5JLD2McrCkbHJMgzXrcA4LeIja TvAsbWhTRszSfAHzqF6nbXjIH+OGKlb7XVF5wbQvLA2LspvIbzPugSs/QmNnUVe1cY8q PYGVhoGlMCVpRDZ+no0NP4p8TekeZ914wzOnqPW6/ECdO5RrzrLZZgrKimPtXboyg2D6 LUUObEuG8hGRYvSU2V05ogbm+8diMsAmGkc0rlWAwTSmRcOcRHvSf5/K4eSNi0lez8r6 dYULbBs8FwkHOwwMS1ysvC+P/1j5k8PbFxPT0WN7znH+HdPwsueaC0EBJ/+5DQyywoFt fFoA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=bcDlT4bS2F//ovCPqsEsS4Ld/G2EA1Xjznx7V/m2BfU=; b=t1pCHoXguyi15CFAZCJH+G+Kvc6foabClvUMr9LNk4EAUZ5auBDP9n76UkEOZn4xUp dwsR52Xz8bISIyWlbRPmRoPwTxZksh1csAjmv+uDAD56s/K7G/YmhWUUsit5MOtWjbBp qpaY8QabWiuDK0ZCL25iIXUl2uppTHDeapZBJ/7hXJiWT5EcWp4UNgVtF8hPra5gtnEh HH72scx4j8QfIGJFW298OimEvm74kVdZ2a7DWpKW6rg7B/yFcbLRDjFwbSlaCHBEN65x 95Oq1Y9fxcTJe8KRUzgdinQtRTgySPF0MlqBKmidz9yNkB14ppRxJgbuVa7x3ccTdRG7 npvQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=yfucjDtL; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id os2-20020a170906af6200b0094f6bd17e83si35454ejb.449.2023.05.04.14.07.31; Thu, 04 May 2023 14:07:37 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=yfucjDtL; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3AAD968C126; Fri, 5 May 2023 00:06:48 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qt1-f179.google.com (mail-qt1-f179.google.com [209.85.160.179]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id BD99D68B36D for ; Fri, 5 May 2023 00:06:39 +0300 (EEST) Received: by mail-qt1-f179.google.com with SMTP id d75a77b69052e-3f364b1149eso4626801cf.0 for ; Thu, 04 May 2023 14:06:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20221208.gappssmtp.com; s=20221208; t=1683234398; x=1685826398; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=1icKU5ZG9Ueq6yYspjCh0KLlMAJMos3aHnhNpRwpoGA=; b=yfucjDtLCQR/kKwtGM/wXpFQoRLDrc8GDVDyKt8scsKhRLxvO2OBkYiCFRLgetFIXl S7RFB1ZOGbH9LCxtWcOKoaE4+eTbqYvwleae5k2lBY1ANggk6a/8CbESp/7Sous9AEtH sfmq5nv7dpeB8beimY+cNpA66UMwGSTKmbK00HuiJrkXm4uqK8Hyq7Kbvk/vDCnYNERj B3pg+j9ePXXmm5hLBFA+cQDYtS6CERlWJxzjAreMK98KWoo87VfMb6b0Bunjgo/f944B yAAe5f7iEBi8+oOui1w5k1esRNZC8Sn3fYYEm1aEBho4ixEVxt+wUlTBWM1fMDi6RLNs /PbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683234398; x=1685826398; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=1icKU5ZG9Ueq6yYspjCh0KLlMAJMos3aHnhNpRwpoGA=; b=NWwidrEOmILOPGo7y1v0iRPo2GF8na6JpG7QmUHwI0chXB4LgTyPLi+sIhukUPQK54 SxUXUBKI7vYmZOdT380gnZbWXfu4tm6U1PjSxnE88y3LXdpGvnAJEvXs8QyWjXBNbFAD U3yekgOwQCEbw4rWx1qVKswlgVtNaPwcuodeqK4PwUFnaDk/XZRhQiJ84MMW8vbR4TY+ XgGl4tn/8DjBjiMgqY/WTa1BIiGojaTUTt6jFaJsAW6Spe6FXeYjU8pJ/gIPp6jbIcq5 95chdjPn5R/caAFnNXNKx+hTtDFXL4sWRejowP2UscIR2sg0qKnUHDwe/gyPEKoF2iI9 w4yA== X-Gm-Message-State: AC+VfDxZoz3vq4RKi4WkK1JipIBc3Gjzp+vH1n0Aeu7ahhWOfnfaofhp FI0Ryug4I+DZpMYY05agaQXdyJzvNMv+Du73Tvk= X-Received: by 2002:a05:622a:58c:b0:3ed:a2f5:f27c with SMTP id c12-20020a05622a058c00b003eda2f5f27cmr6342989qtb.12.1683234397629; Thu, 04 May 2023 14:06:37 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id 4-20020a05620a070400b0074e21c3bc8asm67996qkc.126.2023.05.04.14.06.37 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 04 May 2023 14:06:37 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Thu, 4 May 2023 18:02:17 -0400 Message-Id: <1683237740-32743-4-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1683237740-32743-1-git-send-email-dheitmueller@ltnglobal.com> References: <1683237740-32743-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [PATCH v5 3/6] yadif: Properly preserve CEA-708 closed captions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: PP7YbSlo5rxs Various deinterlacing modes have the effect of doubling the framerate, and we need to ensure that the caption data isn't duplicated (or else you get double captions on-screen). Use the new ccfifo mechanism for yadif (and yadif_cuda and bwdif since they use the same yadif core) so that CEA-708 data is properly preserved through this filter. Signed-off-by: Devin Heitmueller --- libavfilter/vf_bwdif.c | 8 ++++++++ libavfilter/vf_yadif.c | 8 ++++++++ libavfilter/vf_yadif_cuda.c | 9 +++++++++ libavfilter/yadif.h | 2 ++ libavfilter/yadif_common.c | 5 +++++ 5 files changed, 32 insertions(+) diff --git a/libavfilter/vf_bwdif.c b/libavfilter/vf_bwdif.c index 34e8c5e..51e1e02 100644 --- a/libavfilter/vf_bwdif.c +++ b/libavfilter/vf_bwdif.c @@ -297,6 +297,7 @@ static av_cold void uninit(AVFilterContext *ctx) av_frame_free(&yadif->prev); av_frame_free(&yadif->cur ); av_frame_free(&yadif->next); + ff_ccfifo_freep(&yadif->cc_fifo); } static const enum AVPixelFormat pix_fmts[] = { @@ -332,6 +333,13 @@ static int config_props(AVFilterLink *link) if(yadif->mode&1) link->frame_rate = av_mul_q(link->src->inputs[0]->frame_rate, (AVRational){2,1}); + else + link->frame_rate = ctx->inputs[0]->frame_rate; + + if (!(yadif->cc_fifo = ff_ccfifo_alloc(link->frame_rate, ctx))) { + av_log(ctx, AV_LOG_ERROR, "Failure to setup CC FIFO queue\n"); + return AVERROR(ENOMEM); + } if (link->w < 3 || link->h < 4) { av_log(ctx, AV_LOG_ERROR, "Video of less than 3 columns or 4 lines is not supported\n"); diff --git a/libavfilter/vf_yadif.c b/libavfilter/vf_yadif.c index 1be02de..f77f811 100644 --- a/libavfilter/vf_yadif.c +++ b/libavfilter/vf_yadif.c @@ -261,6 +261,7 @@ static av_cold void uninit(AVFilterContext *ctx) av_frame_free(&yadif->prev); av_frame_free(&yadif->cur ); av_frame_free(&yadif->next); + ff_ccfifo_freep(&yadif->cc_fifo); } static const enum AVPixelFormat pix_fmts[] = { @@ -293,6 +294,13 @@ static int config_output(AVFilterLink *outlink) if(s->mode & 1) outlink->frame_rate = av_mul_q(ctx->inputs[0]->frame_rate, (AVRational){2, 1}); + else + outlink->frame_rate = ctx->inputs[0]->frame_rate; + + if (!(s->cc_fifo = ff_ccfifo_alloc(outlink->frame_rate, ctx))) { + av_log(ctx, AV_LOG_ERROR, "Failure to setup CC FIFO queue\n"); + return AVERROR(ENOMEM); + } if (outlink->w < 3 || outlink->h < 3) { av_log(ctx, AV_LOG_ERROR, "Video of less than 3 columns or lines is not supported\n"); diff --git a/libavfilter/vf_yadif_cuda.c b/libavfilter/vf_yadif_cuda.c index 685b8a2..f3f0b56 100644 --- a/libavfilter/vf_yadif_cuda.c +++ b/libavfilter/vf_yadif_cuda.c @@ -205,6 +205,7 @@ static av_cold void deint_cuda_uninit(AVFilterContext *ctx) av_frame_free(&y->prev); av_frame_free(&y->cur); av_frame_free(&y->next); + ff_cc_fifo_freep(&y->cc_fifo); av_buffer_unref(&s->device_ref); s->hwctx = NULL; @@ -291,6 +292,14 @@ static int config_output(AVFilterLink *link) if(y->mode & 1) link->frame_rate = av_mul_q(ctx->inputs[0]->frame_rate, (AVRational){2, 1}); + else + link->frame_rate = ctx->inputs[0]->frame_rate; + + if (!(s->cc_fifo = ff_cc_fifo_alloc(link->frame_rate, ctx))) { + av_log(ctx, AV_LOG_ERROR, "Failure to setup CC FIFO queue\n"); + ret = AVERROR(ENOMEM); + goto exit; + } if (link->w < 3 || link->h < 3) { av_log(ctx, AV_LOG_ERROR, "Video of less than 3 columns or lines is not supported\n"); diff --git a/libavfilter/yadif.h b/libavfilter/yadif.h index c928911..1077576 100644 --- a/libavfilter/yadif.h +++ b/libavfilter/yadif.h @@ -22,6 +22,7 @@ #include "libavutil/opt.h" #include "libavutil/pixdesc.h" #include "avfilter.h" +#include "ccfifo.h" enum YADIFMode { YADIF_MODE_SEND_FRAME = 0, ///< send 1 frame for each frame @@ -76,6 +77,7 @@ typedef struct YADIFContext { int eof; uint8_t *temp_line; int temp_line_size; + AVCCFifo *cc_fifo; /* * An algorithm that treats first and/or last fields in a sequence diff --git a/libavfilter/yadif_common.c b/libavfilter/yadif_common.c index a10cf7a..676853e 100644 --- a/libavfilter/yadif_common.c +++ b/libavfilter/yadif_common.c @@ -60,6 +60,8 @@ static int return_frame(AVFilterContext *ctx, int is_second) yadif->out->pts = AV_NOPTS_VALUE; } } + + ff_ccfifo_inject(yadif->cc_fifo, yadif->out); ret = ff_filter_frame(ctx->outputs[0], yadif->out); yadif->frame_pending = (yadif->mode&1) && !is_second; @@ -96,6 +98,8 @@ int ff_yadif_filter_frame(AVFilterLink *link, AVFrame *frame) av_assert0(frame); + ff_ccfifo_extract(yadif->cc_fifo, frame); + if (yadif->frame_pending) return_frame(ctx, 1); @@ -137,6 +141,7 @@ int ff_yadif_filter_frame(AVFilterLink *link, AVFrame *frame) if (!yadif->out) return AVERROR(ENOMEM); + ff_ccfifo_inject(yadif->cc_fifo, yadif->out); av_frame_free(&yadif->prev); if (yadif->out->pts != AV_NOPTS_VALUE) yadif->out->pts *= 2; From patchwork Thu May 4 22:02:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 41474 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:dca6:b0:f3:34fa:f187 with SMTP id ky38csp12913pzb; Thu, 4 May 2023 14:07:53 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5wzfhym73dxKqL669JGxx70bICAceJYPARlBnjzgVsCmowtJD75IYgT1y7es31hwxU9z4o X-Received: by 2002:a05:6402:1504:b0:50b:cf07:ad0 with SMTP id f4-20020a056402150400b0050bcf070ad0mr2362393edw.37.1683234473591; Thu, 04 May 2023 14:07:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683234473; cv=none; d=google.com; s=arc-20160816; b=i+VNfPHkThzoHgW+v5zMEAMF+s2r4UPjnTnR8MqTkKaUsnj5yB+Gs7vKm5V/zFKjat zudYCD7fAhZrtvlj97L8wD+XJ+qVWKslylnFLubQMkkISZkTuaEPLOFQtbYpBpmQ+so9 fdtaunhlmuaWWQ3pe9D9g5byV118mGIBhwfOINIxEpRFdUiP1wpWwyuDWuFYDZcn3opG 0li/5cC0kiYjHocwC7K38ZVk15XAtmZ94RkU7mVj3S1xqx5MWyx7rHLi617dfvuGNIeH uIx2zDsVfSjMSVSO6JmVkbiaj2ELZJET33TyrkrWPAb5k3lWkzA3pIVi+LfI2x5qmAAN nb8Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=uVIMZrlwzOov84q7mWtcL5k6rqUaJvoojSkCsz4keoI=; b=N7xEgkWQrwevpUOtxbyhCj2uiFR7r2cPIAsmLsefciU6cIFBj86nAdt4/WFz2Z1qHL b+hJHQxSHewxQvNewM8W22z4rKhMTP5Ldnsd5co5DEc8OLSIsJsPyfS0OiTvM9z8MnYy 7vMw46S1x4b5GgFSq0BKQsZ4Gv+q9Z7RwLz9ZxIn3lKJB/by5b5N1vs5UozzXyqxcuqG aXBtfnq8EQJejFJjibkNveRF3Ii5Ihd1M15mAMuxc0TOkjcDD8E+RRGV0cvG7QzjgBPF XIsZodkBnny7O177GXNvoqJBsWCmIdRLcgBHOPP7HK6DZ6xahLTyJND8UJxngCvNpn5H cuAw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=B4JBCh7p; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id d11-20020a50fb0b000000b00504ae1b658dsi3601374edq.490.2023.05.04.14.07.41; Thu, 04 May 2023 14:07:53 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=B4JBCh7p; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 260FB68C12D; Fri, 5 May 2023 00:06:49 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qk1-f176.google.com (mail-qk1-f176.google.com [209.85.222.176]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B149268C102 for ; Fri, 5 May 2023 00:06:40 +0300 (EEST) Received: by mail-qk1-f176.google.com with SMTP id af79cd13be357-74e07c2ee30so48656485a.1 for ; Thu, 04 May 2023 14:06:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20221208.gappssmtp.com; s=20221208; t=1683234399; x=1685826399; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=dHdpTVoMhyj4hLIW3Nkvvd7cp7Pxmqk2Fi3QmMqSDvk=; b=B4JBCh7phvuM2T7b2egmibrDoFO0apdJEjUJWyfWRqRJNG2hYMayxKt6ZtcGd+M515 Lqa8zjzpGdcgXjsSzcvLH7Qy7ZzBnNlwg59rKs/xvcEUwvgRPPJ9DLNjDKMaFgKdGIoU 9WeyetxBpWhJBdgpDuPAXI0JJK+wtnumaxIfzHoyRhG/2ETQ2p627R90lGz9/HCA83bl 58BK+NrA3xhoIelrL+KpFXN+0uuu66SQsxWFgyNSGX+O3xtzOgMUd7CYBBGyrTqYYU1B RxNMdTvqvBwPzDETcy5GASi0JGbuaIV5oDqyWEhy1TC/utbZdozMl0dxU2qo1Rho1j7C jqdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683234399; x=1685826399; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=dHdpTVoMhyj4hLIW3Nkvvd7cp7Pxmqk2Fi3QmMqSDvk=; b=kzvOGti6+2c/xglXo3GQVtBIBVYD1ec1fVQAmlmSlX0i7cv2CD0T/aMQWuyOEK/eHE ktRyeNFrtlfHqsWGvxR0kBB6huNren+wPfpAetDB06Z5ubuvH/4KkxJjdjhBpEQs0Egu jdKUEGRzgoiIMzBEFlOrER2F9ROcZZqfKvq7WqqhL/RZ382ymiRxUH911Fs0GfidSa8S EEMe4DO7ItEMaGhU0Pr6J6zaCe93ospdPmrN0NjyDB31KFzRhTuhm0q7zF4WZrBxj05R HEcRR71CoLABCwQYfl9syNbPbDX36SNbsMDv1ynYDELR9ho2aIIu/DPkZQu6DVR5+eTE 8QfQ== X-Gm-Message-State: AC+VfDyZ/M+QyG0s4wa85HmQHsCws6M1Qzrw66xozWKho+W6b24RJeo4 npv3mYUagz8tTHwdsEhPxBRm25ulEPZCK810WlQ= X-Received: by 2002:a05:6214:2689:b0:5cc:277c:b5e with SMTP id gm9-20020a056214268900b005cc277c0b5emr17533780qvb.33.1683234398935; Thu, 04 May 2023 14:06:38 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id 4-20020a05620a070400b0074e21c3bc8asm67996qkc.126.2023.05.04.14.06.38 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 04 May 2023 14:06:38 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Thu, 4 May 2023 18:02:18 -0400 Message-Id: <1683237740-32743-5-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1683237740-32743-1-git-send-email-dheitmueller@ltnglobal.com> References: <1683237740-32743-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [PATCH v5 4/6] tinterlace: Properly preserve CEA-708 closed captions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: JCL3ctCYjR6E Because the interlacing filter halves the effective framerate, we need to ensure that no CEA-708 data is lost as frames are merged. Make use of the new ccfifo mechanism to ensure that caption data is properly preserved as frames pass through the filter. Thanks to Thomas Mundt for review and noticing a couple of missed codepaths for injection on output. Thanks to Lance Wang for pointing out a memory leak. Signed-off-by: Devin Heitmueller --- libavfilter/tinterlace.h | 2 ++ libavfilter/vf_tinterlace.c | 11 +++++++++++ 2 files changed, 13 insertions(+) diff --git a/libavfilter/tinterlace.h b/libavfilter/tinterlace.h index 37b6c10..9f5ce7e 100644 --- a/libavfilter/tinterlace.h +++ b/libavfilter/tinterlace.h @@ -32,6 +32,7 @@ #include "libavutil/pixdesc.h" #include "drawutils.h" #include "avfilter.h" +#include "ccfifo.h" #define TINTERLACE_FLAG_VLPF 01 #define TINTERLACE_FLAG_CVLPF 2 @@ -77,6 +78,7 @@ typedef struct TInterlaceContext { const AVPixFmtDescriptor *csp; void (*lowpass_line)(uint8_t *dstp, ptrdiff_t width, const uint8_t *srcp, ptrdiff_t mref, ptrdiff_t pref, int clip_max); + AVCCFifo *cc_fifo; } TInterlaceContext; void ff_tinterlace_init_x86(TInterlaceContext *interlace); diff --git a/libavfilter/vf_tinterlace.c b/libavfilter/vf_tinterlace.c index 0326292..8716a94 100644 --- a/libavfilter/vf_tinterlace.c +++ b/libavfilter/vf_tinterlace.c @@ -203,6 +203,7 @@ static av_cold void uninit(AVFilterContext *ctx) av_frame_free(&tinterlace->next); av_freep(&tinterlace->black_data[0][0]); av_freep(&tinterlace->black_data[1][0]); + ff_ccfifo_freep(&tinterlace->cc_fifo); } static int config_out_props(AVFilterLink *outlink) @@ -291,6 +292,11 @@ static int config_out_props(AVFilterLink *outlink) #endif } + if (!(tinterlace->cc_fifo = ff_ccfifo_alloc(outlink->frame_rate, ctx))) { + av_log(ctx, AV_LOG_ERROR, "Failure to setup CC FIFO queue\n"); + return AVERROR(ENOMEM); + } + av_log(ctx, AV_LOG_VERBOSE, "mode:%d filter:%s h:%d -> h:%d\n", tinterlace->mode, (tinterlace->flags & TINTERLACE_FLAG_CVLPF) ? "complex" : (tinterlace->flags & TINTERLACE_FLAG_VLPF) ? "linear" : "off", @@ -375,6 +381,8 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) tinterlace->cur = tinterlace->next; tinterlace->next = picref; + ff_ccfifo_extract(tinterlace->cc_fifo, picref); + cur = tinterlace->cur; next = tinterlace->next; /* we need at least two frames */ @@ -451,6 +459,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) if (!out) return AVERROR(ENOMEM); out->pts /= 2; // adjust pts to new framerate + ff_ccfifo_inject(tinterlace->cc_fifo, out); ret = ff_filter_frame(outlink, out); return ret; } @@ -486,6 +495,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) out->pts = cur->pts*2; out->pts = av_rescale_q(out->pts, tinterlace->preout_time_base, outlink->time_base); + ff_ccfifo_inject(tinterlace->cc_fifo, out); if ((ret = ff_filter_frame(outlink, out)) < 0) return ret; @@ -521,6 +531,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) out->pts = av_rescale_q(out->pts, tinterlace->preout_time_base, outlink->time_base); out->duration = av_rescale_q(1, av_inv_q(outlink->frame_rate), outlink->time_base); + ff_ccfifo_inject(tinterlace->cc_fifo, out); ret = ff_filter_frame(outlink, out); return ret; From patchwork Thu May 4 22:02:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 41475 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:dca6:b0:f3:34fa:f187 with SMTP id ky38csp12922pzb; Thu, 4 May 2023 14:07:54 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ71Dna4hXBsXzypNeJwsEJsREtSkhcA+BrK1b9nkGM4TeMErlpny6bT0lWn4aSSR4blNJoD X-Received: by 2002:a17:907:368e:b0:965:b493:59e0 with SMTP id bi14-20020a170907368e00b00965b49359e0mr198245ejc.15.1683234474018; Thu, 04 May 2023 14:07:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683234474; cv=none; d=google.com; s=arc-20160816; b=mM0DW/n2GuR+oRDDoHzigyvx3FSYx9TUL4rLXijaCx4xb5F9NmDxx108lEBEvNTwI0 ZuTgdYTIJZQTw3eCBjQYoaUC/DpFEDwe/2ainpb94TBdODHyr1U1FzVCoc10efJInZxQ vinQUBQBhwAJH/cDpr7Ew0s8661GRbtMKmXChMHiqmPJhyAELzhJqMg+7FyrZb9S9iIS FHr1oxdUgHcSVicO2Bud2y1t+tsRlPdW39jzD5Fcckmd+OtN/KA9ykI/k28fkG52psJM xxmGQW5JyHJVxuIvNiQWbdRGoGz4fMsSR8NjKkR6SVEGcoZcwnae8GRmTzM2+7yK6OBA aUNw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=OyBub1Xg+VFvNiP/+n0WtRAVGO+tM6fGawsnRuaLl90=; b=qTqzz7jTD6mmSHjFbuNRgG3y2JCy8a3nPtMi67GirA2Wc5YhAYWn6sxkxkkPwn9yx/ e4S4r0m5QRY88FiBa6pPDiFCkrGwvERhw1NhQ0ryzjVYluYUcoqpoemJ2rxFsmqnBXUN lBuVkQW/jqif8eHSR/MPuRYTH5k+YavL8qwcFVVADHVCp0VBnlyFxxzA1x/vxRBRk8L3 KefmwqJbla7aXE/iHFFgXbkSlOlFHmqerQ+0yrlyzGG6ZSMNA7CKBzkT32OH493U93X3 n+UzMNfhmJsfi8pUyQ1BbqnLAp8wnDdgGERsv+sIvsk/dNEBl4CNkAoTs8u4+6vdFwE1 i5bQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b="Tiai/amR"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id k26-20020a170906681a00b00965a7a7c949si46808ejr.364.2023.05.04.14.07.53; Thu, 04 May 2023 14:07:54 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b="Tiai/amR"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2501168C138; Fri, 5 May 2023 00:06:50 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qk1-f172.google.com (mail-qk1-f172.google.com [209.85.222.172]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0860768C105 for ; Fri, 5 May 2023 00:06:41 +0300 (EEST) Received: by mail-qk1-f172.google.com with SMTP id af79cd13be357-74e1745356dso49025085a.0 for ; Thu, 04 May 2023 14:06:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20221208.gappssmtp.com; s=20221208; t=1683234399; x=1685826399; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=VK83yqMTe/l2pg95fYViHCLfcPSlu7eURCol8Hq+x0E=; b=Tiai/amRdqlRoRxAjwImXDv4xS/1bX9FuKBu3OLgwfsiOpSjICC94vgz9VhbYhiC9U TwP8SWSAKP4dEv/UiFV7wm9iga3cSszsr3uoJfTEXn9bh1aH8Ca6fUF/v4idHuHfDynA R819xpkrMcJPydjMcRwpaRw02Jn9oPztJOjLlO/ZW4sgFw1SodO1/gU1lQZORIun9ykt ayAMyCaUMa/ERWpo1Y88yRj6tT97QILJzy5aG6QHVkD3FhP2ehOqeJqFv5FBAWPDNb+l jL0G9BtaMlX2AzlealTWAYhT5ny166vQCIBRL8Dv7y+I3qDT0p3968qm/1vjK0RcuOmu wKYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683234399; x=1685826399; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=VK83yqMTe/l2pg95fYViHCLfcPSlu7eURCol8Hq+x0E=; b=H7Njxcj89Ozlug2trfGMOITfXmjvj42CTKgthSrtwh3wMb9QVntIkHKCRJ83O1hMTb jfVXzNOZF2pAITaLt4N0054BD0+Yc9VTrQRWBdX4HcgiPucoSgmIL+yeK0UjmKF2/laC TIkFt69VmWVPTRHmIeXzS6nC8BLU5KQkXqj3m0s8RT0C+ly43Fyq/4FYVHXHEy8tTkaL A3GWbadMjTD4+Rrm6jf/IL92khxCgLMXMcbJsXJwHFcOGa+ylXVBRiPeyffme/3sRups yPvQBFqlAkRbWb1J8UAZklOa2nOn3PYa74w1o5dOAGolAprO2XnnQi8lXslm2fCh+St0 +x6A== X-Gm-Message-State: AC+VfDxxOpYNW9oTBNJROJn6CUZHyiIxJwKB5KPVGb4Eb0ECeXdxzw3l gIcP/M9NQO/QQa7Map10JKIkqvdtj0X/WBVigmo= X-Received: by 2002:a05:6214:29ca:b0:61b:5a60:329c with SMTP id gh10-20020a05621429ca00b0061b5a60329cmr16123503qvb.39.1683234399618; Thu, 04 May 2023 14:06:39 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id 4-20020a05620a070400b0074e21c3bc8asm67996qkc.126.2023.05.04.14.06.39 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 04 May 2023 14:06:39 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Thu, 4 May 2023 18:02:19 -0400 Message-Id: <1683237740-32743-6-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1683237740-32743-1-git-send-email-dheitmueller@ltnglobal.com> References: <1683237740-32743-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [PATCH v5 5/6] vf_ccrepack: Add new filter to repack CEA-708 side data X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: ha0WiE7skn8j THis filter can correct certain issues seen from upstream sources where the cc_count is not properly set or the CEA-608 tuples are not at the start of the payload as expected. Make use of the ccfifo to extract and immediately repack the CEA-708 side data, thereby removing any extra padding and ensuring the 608 tuples are at the front of the payload. Signed-off-by: Devin Heitmueller --- doc/filters.texi | 10 +++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/vf_ccrepack.c | 102 ++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 114 insertions(+) create mode 100644 libavfilter/vf_ccrepack.c diff --git a/doc/filters.texi b/doc/filters.texi index 9a0fe9c..d425081 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -9070,6 +9070,16 @@ Only deinterlace frames marked as interlaced. The default value is @code{all}. @end table +@section ccrepack + +Repack CEA-708 closed captioning side data + +This filter fixes various issues seen with commerical encoders +related to upstream malformed CEA-708 payloads, specifically +incorrect number of tuples (wrong cc_count for the target FPS), +and incorrect ordering of tuples (i.e. the CEA-608 tuples are not at +the first entries in the payload). + @section cas Apply Contrast Adaptive Sharpen filter to video stream. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 68c8f14..7c2195f 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -214,6 +214,7 @@ OBJS-$(CONFIG_BOXBLUR_OPENCL_FILTER) += vf_avgblur_opencl.o opencl.o \ opencl/avgblur.o boxblur.o OBJS-$(CONFIG_BWDIF_FILTER) += vf_bwdif.o yadif_common.o OBJS-$(CONFIG_CAS_FILTER) += vf_cas.o +OBJS-$(CONFIG_CCREPACK_FILTER) += vf_ccrepack.o OBJS-$(CONFIG_CHROMABER_VULKAN_FILTER) += vf_chromaber_vulkan.o vulkan.o vulkan_filter.o OBJS-$(CONFIG_CHROMAHOLD_FILTER) += vf_chromakey.o OBJS-$(CONFIG_CHROMAKEY_FILTER) += vf_chromakey.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 6994124..ee2ac17 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -198,6 +198,7 @@ extern const AVFilter ff_vf_boxblur; extern const AVFilter ff_vf_boxblur_opencl; extern const AVFilter ff_vf_bwdif; extern const AVFilter ff_vf_cas; +extern const AVFilter ff_vf_ccrepack; extern const AVFilter ff_vf_chromaber_vulkan; extern const AVFilter ff_vf_chromahold; extern const AVFilter ff_vf_chromakey; diff --git a/libavfilter/vf_ccrepack.c b/libavfilter/vf_ccrepack.c new file mode 100644 index 0000000..e3fd67f --- /dev/null +++ b/libavfilter/vf_ccrepack.c @@ -0,0 +1,102 @@ +/* + * CEA-708 Closed Caption Repacker + * Copyright (c) 2023 LTN Global Communications + * + * Author: Devin Heitmueller + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/* + * Repackage CEA-708 arrays, which deals with incorrect cc_count for a given + * output framerate, and incorrect 708 padding. + * + * See CEA CEA-10-A "EIA-708-B Implementation Guidance", Section 26.5 + * "Grouping DTVCC Data Within user_data() Structure" + */ + +#include "avfilter.h" +#include "internal.h" +#include "ccfifo.h" +#include "libavutil/opt.h" + +typedef struct CCRepackContext +{ + const AVClass *class; + AVCCFifo *cc_fifo; +} CCRepackContext; + +static const AVOption ccrepack_options[] = { + { NULL } +}; + +AVFILTER_DEFINE_CLASS(ccrepack); + +static int config_input(AVFilterLink *link) +{ + CCRepackContext *ctx = link->dst->priv; + + if (!(ctx->cc_fifo = ff_ccfifo_alloc(link->frame_rate, ctx))) { + av_log(ctx, AV_LOG_ERROR, "Failure to setup CC FIFO queue\n"); + return AVERROR(ENOMEM); + } + + return 0; +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *frame) +{ + CCRepackContext *ctx = inlink->dst->priv; + AVFilterLink *outlink = inlink->dst->outputs[0]; + + ff_ccfifo_extract(ctx->cc_fifo, frame); + ff_ccfifo_inject(ctx->cc_fifo, frame); + + return ff_filter_frame(outlink, frame); +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + CCRepackContext *s = ctx->priv; + ff_ccfifo_freep(&s->cc_fifo); +} + +static const AVFilterPad avfilter_vf_ccrepack_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .filter_frame = filter_frame, + .config_props = config_input, + }, +}; + +static const AVFilterPad avfilter_vf_ccrepack_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + }, +}; + +AVFilter ff_vf_ccrepack = { + .name = "ccrepack", + .description = NULL_IF_CONFIG_SMALL("Repack CEA-708 closed caption metadata"), + .uninit = uninit, + .priv_size = sizeof(CCRepackContext), + .priv_class = &ccrepack_class, + FILTER_INPUTS(avfilter_vf_ccrepack_inputs), + FILTER_OUTPUTS(avfilter_vf_ccrepack_outputs), +}; From patchwork Thu May 4 22:02:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 41476 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:dca6:b0:f3:34fa:f187 with SMTP id ky38csp13009pzb; Thu, 4 May 2023 14:08:04 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4AmKMyLEoukpp4vA/l2qxZk3klP0Mz2Mr3iU0EILd3tHVeOtNISDjfgKsSScYySJiggVkf X-Received: by 2002:a17:906:4fd1:b0:889:5686:486a with SMTP id i17-20020a1709064fd100b008895686486amr195413ejw.30.1683234483908; Thu, 04 May 2023 14:08:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683234483; cv=none; d=google.com; s=arc-20160816; b=gPupBFKA+DJjMD0NBz76tUkBx9eB4Jm+icws3oAXVJWdoboaHlhCBLcerDxFWDXJc3 a/PP18eZwbG+c3yJQesZ8kouC0UqFJt+fg17RF9ci+7Dy45Pj3nTlZWe6ANl+1ffSDks E6X2JBF2EOwJZHubcAQRi6j8RC40a14ZGFHi/4IHMbtkD000mXfgIjhkDM8pZS5T55Om s60qB6WGT0r+ojVOUcVVgTLK10xqrz0gKq8e90TQ1rS3WvTy7s6FGRJwO4x9oNCJnRNN 8vZrjsLX7yIdUyKM7J1tdUCzlsq/rb6fVNU/7GHfQnO0bGafv4ovTd2Ot88MQPTmO4Xo crHg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=RhfARq+TsdbvsHUpS9ShGbmqkLo8aYDj8RqzOhYYibg=; b=Z8NODByxfycCLO/KI2tMwA4QDNsT3yXJuAruACKPoCuaButVc8fQgs6p7unLZRyM3w q53mLT7f8Uoqfe5OBmUwIbX6K4CDP0D1cWB7slEchVdQFVXfCoBScMiWBfhPPgujWa0+ Yz3L2IKkZZ2ao9EXLNH9RJNJQe9pGb0iztYKg6hAtDJ8F4YJyPY8no7fFl+He37cYSf8 XS57X21USBDrRk0jyC+Gi8cWxjCiYMBDuhgXrc91G3cLzys8uCIzisA5hZsmDrv1ZI+F r0XiwN1d2DM4ANaXoA/xx6bAzgCLIudqv4DCg/4XRyP6tvNzbhDPxDudE7mIapaVHb8I Zaug== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b="OZ3+/Gqp"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id ec15-20020a170906b6cf00b009604f4a3fe4si54951ejb.53.2023.05.04.14.08.03; Thu, 04 May 2023 14:08:03 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b="OZ3+/Gqp"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 193D668C13E; Fri, 5 May 2023 00:06:51 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qk1-f179.google.com (mail-qk1-f179.google.com [209.85.222.179]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B28D868C10D for ; Fri, 5 May 2023 00:06:41 +0300 (EEST) Received: by mail-qk1-f179.google.com with SMTP id af79cd13be357-74fc1452fbdso79025385a.2 for ; Thu, 04 May 2023 14:06:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20221208.gappssmtp.com; s=20221208; t=1683234400; x=1685826400; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=KWa1d9LQ33UJDbR44HgPCk5kbWfV1BdRusHOBdIv6go=; b=OZ3+/Gqpjj7B6RIPS4Dag8OXarEgBeEmj+B3SFzma1ggJ4Hd2k+MYXnDyZ1bXTF0wg wav4Bno0sQ0ziOD6Y6XcIz81RetmnPmGC9HZHXsZBskBaZacmvt5u2QY1OeKZ2V9ZBGa XV9Q8H+/WADRfr02PCfNOygFaAOeT23hJLJR/heCpwo7OLBH5T9GgE4Gd9sA1abk1Z9Z ZavPCcNCh71XspDsSoYrmMyoFtIwNhPPCfzRuX1UQ7BHMW8nM7BvDcN5QnpaAL2aGySw 0vSOf1YeTBKkEPfArsBS+z/HcwmS0Q+0IPadktauwgAtivEeCtBlMKH2+euDjf9rdK+a 6T/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683234400; x=1685826400; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=KWa1d9LQ33UJDbR44HgPCk5kbWfV1BdRusHOBdIv6go=; b=aw6Z746iiT+15HtzFBRLb7zGK5lTID4hw0M3w3hf+Wfe34vPi/9wZQoy4sEpv01qFc Dksa1mpM27fJkv2wP/0SD8Iqa4B8Jnm2C5bp/zQ5J3zCBEM+2ODv8cxCIiWKDusXeBXR IA+WVmrWyQwbJ2Z4PGH9NJDKlOBnqGQl/xN2gtzkffnJJdtdLJTCAOO1+mENEG5l/Bv1 haSq6vasJLNZ7J6PFbN4jPXxMSebvfDsmExx84juE+ff3v59rrQ4QbDwD47n0WazCpeg yD5ieIjmoSTLA22EoYoe4Wo9fTIx6WBeOOTNehMRjap5FCMKoNQykyhBPG004qmoJNAl tNNw== X-Gm-Message-State: AC+VfDwlwUmuEWZfOwRHUFPZ7YqCe46fodnrplgX4RCpmNji/xdvPYdq 7/psTKJw8HdQV0dBnrPFHIKM+oQSvoQOBdRwotc= X-Received: by 2002:a05:6214:e66:b0:616:516e:f3ea with SMTP id jz6-20020a0562140e6600b00616516ef3eamr18116643qvb.51.1683234400295; Thu, 04 May 2023 14:06:40 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id 4-20020a05620a070400b0074e21c3bc8asm67996qkc.126.2023.05.04.14.06.39 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 04 May 2023 14:06:39 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Thu, 4 May 2023 18:02:20 -0400 Message-Id: <1683237740-32743-7-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1683237740-32743-1-git-send-email-dheitmueller@ltnglobal.com> References: <1683237740-32743-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [PATCH v5 6/6] decklink_enc: add support for playout of 608 captions in MOV files X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: rVhiNTmjl32R Unlike other cases where the closed captions are embedded in the video stream as MPEG-2 userdata or H.264 SEI data, with MOV files the captions are often found on a separate "e608" subtitle track. Add support for playout of such files, leveraging the new ccfifo mechanism to ensure that they are embedded into VANC at the correct rate (since e608 packets often contain batches of multiple 608 pairs). Note this patch includes a new file named libavdevice/ccfifo.c, which allows the ccfifo functionality in libavfilter to be reused even if doing shared builds. This is the same approach used for log2_tab.c. Signed-off-by: Devin Heitmueller --- libavdevice/Makefile | 1 + libavdevice/ccfifo.c | 24 ++++++++++++++++ libavdevice/decklink_common.h | 3 ++ libavdevice/decklink_enc.cpp | 65 +++++++++++++++++++++++++++++++++++++++++++ libavdevice/decklink_enc_c.c | 2 +- 5 files changed, 94 insertions(+), 1 deletion(-) create mode 100644 libavdevice/ccfifo.c diff --git a/libavdevice/Makefile b/libavdevice/Makefile index 8a62822..c304492 100644 --- a/libavdevice/Makefile +++ b/libavdevice/Makefile @@ -57,6 +57,7 @@ OBJS-$(CONFIG_LIBDC1394_INDEV) += libdc1394.o # Objects duplicated from other libraries for shared builds SHLIBOBJS-$(CONFIG_DECKLINK_INDEV) += reverse.o +SHLIBOBJS-$(CONFIG_DECKLINK_OUTDEV) += ccfifo.o # Windows resource file SHLIBOBJS-$(HAVE_GNU_WINDRES) += avdeviceres.o diff --git a/libavdevice/ccfifo.c b/libavdevice/ccfifo.c new file mode 100644 index 0000000..9007094 --- /dev/null +++ b/libavdevice/ccfifo.c @@ -0,0 +1,24 @@ +/* + * CEA-708 Closed Captioning FIFO + * Copyright (c) 2023 LTN Global Communications + * + * Author: Devin Heitmueller + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavfilter/ccfifo.c" diff --git a/libavdevice/decklink_common.h b/libavdevice/decklink_common.h index 088e165..0d33f94 100644 --- a/libavdevice/decklink_common.h +++ b/libavdevice/decklink_common.h @@ -31,6 +31,7 @@ extern "C" { #include "libavcodec/packet_internal.h" +#include "libavfilter/ccfifo.h" } #include "libavutil/thread.h" #include "decklink_common_c.h" @@ -112,6 +113,8 @@ struct decklink_ctx { /* Capture buffer queue */ AVPacketQueue queue; + AVCCFifo *cc_fifo; ///< closed captions + /* Streams present */ int audio; int video; diff --git a/libavdevice/decklink_enc.cpp b/libavdevice/decklink_enc.cpp index 9f1a8df..22aa80b 100644 --- a/libavdevice/decklink_enc.cpp +++ b/libavdevice/decklink_enc.cpp @@ -326,6 +326,25 @@ static int create_s337_payload(AVPacket *pkt, uint8_t **outbuf, int *outsize) return 0; } +static int decklink_setup_subtitle(AVFormatContext *avctx, AVStream *st) +{ + int ret = -1; + + switch(st->codecpar->codec_id) { +#if CONFIG_LIBKLVANC + case AV_CODEC_ID_EIA_608: + /* No special setup required */ + ret = 0; + break; +#endif + default: + av_log(avctx, AV_LOG_ERROR, "Unsupported subtitle codec specified\n"); + break; + } + + return ret; +} + av_cold int ff_decklink_write_trailer(AVFormatContext *avctx) { struct decklink_cctx *cctx = (struct decklink_cctx *)avctx->priv_data; @@ -352,6 +371,7 @@ av_cold int ff_decklink_write_trailer(AVFormatContext *avctx) klvanc_context_destroy(ctx->vanc_ctx); #endif + ff_ccfifo_freep(&ctx->cc_fifo); av_freep(&cctx->ctx); return 0; @@ -503,6 +523,21 @@ out: free(afd_words); } +/* Parse any EIA-608 subtitles sitting on the queue, and write packet side data + that will later be handled by construct_cc... */ +static void parse_608subs(AVFormatContext *avctx, struct decklink_ctx *ctx, AVPacket *pkt) +{ + size_t cc_size = ff_ccfifo_getoutputsize(ctx->cc_fifo); + uint8_t *cc_data; + + if (!ff_ccfifo_ccdetected(ctx->cc_fifo)) + return; + + cc_data = av_packet_new_side_data(pkt, AV_PKT_DATA_A53_CC, cc_size); + if (cc_data) + ff_ccfifo_injectbytes(ctx->cc_fifo, cc_data, cc_size); +} + static int decklink_construct_vanc(AVFormatContext *avctx, struct decklink_ctx *ctx, AVPacket *pkt, decklink_frame *frame, AVStream *st) @@ -513,6 +548,7 @@ static int decklink_construct_vanc(AVFormatContext *avctx, struct decklink_ctx * if (!ctx->supports_vanc) return 0; + parse_608subs(avctx, ctx, pkt); construct_cc(avctx, ctx, pkt, &vanc_lines); construct_afd(avctx, ctx, pkt, &vanc_lines, st); @@ -704,6 +740,16 @@ static int decklink_write_audio_packet(AVFormatContext *avctx, AVPacket *pkt) return ret; } +static int decklink_write_subtitle_packet(AVFormatContext *avctx, AVPacket *pkt) +{ + struct decklink_cctx *cctx = (struct decklink_cctx *)avctx->priv_data; + struct decklink_ctx *ctx = (struct decklink_ctx *)cctx->ctx; + + ff_ccfifo_extractbytes(ctx->cc_fifo, pkt->data, pkt->size); + + return 0; +} + extern "C" { av_cold int ff_decklink_write_header(AVFormatContext *avctx) @@ -768,12 +814,29 @@ av_cold int ff_decklink_write_header(AVFormatContext *avctx) } else if (c->codec_type == AVMEDIA_TYPE_VIDEO) { if (decklink_setup_video(avctx, st)) goto error; + } else if (c->codec_type == AVMEDIA_TYPE_SUBTITLE) { + if (decklink_setup_subtitle(avctx, st)) + goto error; } else { av_log(avctx, AV_LOG_ERROR, "Unsupported stream type.\n"); goto error; } } + for (n = 0; n < avctx->nb_streams; n++) { + AVStream *st = avctx->streams[n]; + AVCodecParameters *c = st->codecpar; + + if(c->codec_type == AVMEDIA_TYPE_SUBTITLE) + avpriv_set_pts_info(st, 64, ctx->bmd_tb_num, ctx->bmd_tb_den); + } + + if (!(ctx->cc_fifo = ff_ccfifo_alloc(av_make_q(ctx->bmd_tb_den, ctx->bmd_tb_num), avctx))) { + av_log(ctx, AV_LOG_ERROR, "Failure to setup CC FIFO queue\n"); + ret = AVERROR(ENOMEM); + goto error; + } + return 0; error: @@ -789,6 +852,8 @@ int ff_decklink_write_packet(AVFormatContext *avctx, AVPacket *pkt) return decklink_write_video_packet(avctx, pkt); else if (st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO) return decklink_write_audio_packet(avctx, pkt); + else if (st->codecpar->codec_type == AVMEDIA_TYPE_SUBTITLE) + return decklink_write_subtitle_packet(avctx, pkt); return AVERROR(EIO); } diff --git a/libavdevice/decklink_enc_c.c b/libavdevice/decklink_enc_c.c index f7e3150..0a3984b 100644 --- a/libavdevice/decklink_enc_c.c +++ b/libavdevice/decklink_enc_c.c @@ -77,7 +77,7 @@ const FFOutputFormat ff_decklink_muxer = { .p.long_name = NULL_IF_CONFIG_SMALL("Blackmagic DeckLink output"), .p.audio_codec = AV_CODEC_ID_PCM_S16LE, .p.video_codec = AV_CODEC_ID_WRAPPED_AVFRAME, - .p.subtitle_codec = AV_CODEC_ID_NONE, + .p.subtitle_codec = AV_CODEC_ID_EIA_608, .p.flags = AVFMT_NOFILE, .p.priv_class = &decklink_muxer_class, .get_device_list = ff_decklink_list_output_devices,