From patchwork Mon Mar 27 16:47:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 40876 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:7a30:b0:df:834d:2c1a with SMTP id t48csp1485194pzh; Mon, 27 Mar 2023 09:47:31 -0700 (PDT) X-Google-Smtp-Source: AKy350bO1FfRGd0NtgHkADN+VlmP6RxiUfQfrwartwRcNC3Z5zRrz/cu1nWiofhjLQP4Qdf2/U6k X-Received: by 2002:a05:6402:1a48:b0:501:d3a1:9a4a with SMTP id bf8-20020a0564021a4800b00501d3a19a4amr13087021edb.19.1679935650881; Mon, 27 Mar 2023 09:47:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1679935650; cv=none; d=google.com; s=arc-20160816; b=fj2eSZYoWoCA7ItI63m6ZOwnEZrL2sjQZk1CZh5YhDfQy2L0YBri72pk1NIjbFeKNu DgSQAySSlehKNSo+M6+vlIRbL6Ame9toCksdogvRH4qkfNG72t/F9q7/yldzo8uV+PNu blGk7do3JIo5ZfGpGdrTemcAIde4iuRNOhmRvSgNiMqtTdSy2l5wi1oai/Xl5uofVisf 6wz2WvpSCykg400c2NgEoMQ4p59b1pL0sdFsVTRgFKZRozxR05bq98bV/2Mht8S9c4Vr 3KIdC5DwNSfzgFp9IrXuzLsvTSJSMNlI5B6WimwdySYf5sjVk9JGs4HOZBwh0pSLHLaK ptqw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=JAyyp/M04VCroqIpwAsGkhVc1IbgA5i2pAMSzA+muA8=; b=XVT6AmbWZrsbIlLkMZhJXnV7Z1cCXIMJ/ENAP9VCvqGnT61pM0MEgme9lQNt52lRJU KhRq/VTbHMfo7xBOtCoM2Vr7DQopxMHKkkDiGgPAdO4dOjAAPLAAmAcG4XxAnbZn/Jb3 ZnYPhhoXhDcRJSOQLSNcfeMLdl34G8dcRTm2dn4dFvrvWkGswfWgYBUA7u1AaILt9XZT ruTxVbQj1r3aOKUQUYEmbDpxCtdjQz4BKsqmvz9hLWOX95d116ULC8Z1ulkrsjQtZZp/ RykYWBaU8oebN+nwqD39Ym0JWVwZCN9lNDEI4X1ZHScWGUn+k+jMH6mr8ieCjmKOua7X y9oA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=u9nlj25E; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id r1-20020aa7c141000000b004fe948c73f3si27624305edp.573.2023.03.27.09.47.30; Mon, 27 Mar 2023 09:47:30 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=u9nlj25E; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3A29068CB16; Mon, 27 Mar 2023 19:47:18 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qv1-f48.google.com (mail-qv1-f48.google.com [209.85.219.48]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 022FF68C570 for ; Mon, 27 Mar 2023 19:47:11 +0300 (EEST) Received: by mail-qv1-f48.google.com with SMTP id jl13so7129476qvb.10 for ; Mon, 27 Mar 2023 09:47:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20210112.gappssmtp.com; s=20210112; t=1679935629; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TKbzjzNB88QYSpsBtD8A1K8p5xmRfrPqyFPkhdj6UqY=; b=u9nlj25EY7Mo9nYGPw8lipTe2LP1QYKW5y5SYldm5dZYaVZ2timGhOeJ5Or3mdD69F 4QEqA+rtkJXYkztXh3xx/NqHpnQXOc/LAxxt0ecmo8EzIUbku4+mbb0AiXoNj71usP0i xPFeD4J2dTDTODuI6r1x6+8o5eh0O5fPv9gLMQ4njV0oKuMvJRovzBG5fa32mRZR2XNJ c4cIIUWROULJzJpNTvleb4KXW5hgOhJkfkqAsxHyzmU4UBzCE7CF2AgqR0HT6fR6Dcy1 t9bcHhjdOWZdA74vtfZVrUWSUfEOucD4CR093h+Fin/yxDiaEUaYxA9mwi/VJeYRJGnJ aCuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679935629; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TKbzjzNB88QYSpsBtD8A1K8p5xmRfrPqyFPkhdj6UqY=; b=OjtFSlNnpeW7w9OvOphYCdeJHo27gt+2UJeu4rrUyAWMMQXngpnL7kTocwnn1iW9Fj XUYslLhI8Z2HySgkUMGK0NfC6PHuF3NQpxWlFafouSdXT9hIzaU2G21UDelYA9Xs8Oj6 Zrc0ch6BmqXl5qz0l6nO4nYxudiFKhebzn5oVQwxBpQFTMCXztmL1GFw5xfpcd+v3Wac QWK2srz/xfEk3O+WYmwhyOyWeGez4yXftENHa5XXzpbGNHXmf4h2RvPBube/yTb9mBwi Xf9CUvjlZj7c+t5QMV9xClRQUxBEBBqkJ1+E8bv37LE5sLer75chqB0WzT1QWWB+++BA u3/w== X-Gm-Message-State: AAQBX9f7JNsVYzjQs98pM+DQ+xSIx3pjG8Gpnq9/Yq41jQxiHAcD8lgn z22rCbIMVmNHVXMOtmHMAbRBGVC9mP1tVFt2nt0= X-Received: by 2002:a05:6214:2506:b0:56f:37a:4562 with SMTP id gf6-20020a056214250600b0056f037a4562mr21909459qvb.13.1679935629453; Mon, 27 Mar 2023 09:47:09 -0700 (PDT) Received: from klab-nyc-kernellabs2.localdomain (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id lf10-20020a0562142cca00b005dd8b9345e9sm3093100qvb.129.2023.03.27.09.47.08 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 27 Mar 2023 09:47:09 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Mon, 27 Mar 2023 12:47:00 -0400 Message-Id: <20230327164704.12962-2-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 2.35.1.655.ga68dfadae5 In-Reply-To: <20230327164704.12962-1-dheitmueller@ltnglobal.com> References: <20230327164704.12962-1-dheitmueller@ltnglobal.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/5] ccfifo: Properly handle CEA-708 captions through framerate conversion X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: +HxS6Vh6hFXW When transcoding video that contains 708 closed captions, the caption data is tied to the frames as side data. Simply dropping or adding frames to change the framerate will result in loss of data, so the caption data needs to be preserved and reformatted. For example, without this patch converting 720p59 to 1080i59 would result in loss of 50% of the caption bytes, resulting in garbled 608 captions and 708 probably wouldn't render at all. Further, the frames that are there will have an illegal cc_count for the target framerate, so some decoders may ignore the packets entirely. Extract the 608 and 708 tuples and insert them onto queues. Then after dropping/adding frames, re-write the tuples back into the resulting frames at the appropriate rate given the target framerate. This includes both having the correct cc_count as well as clocking out the 608 pairs at the appropriate rate. Signed-off-by: Devin Heitmueller --- libavutil/Makefile | 2 + libavutil/ccfifo.c | 191 +++++++++++++++++++++++++++++++++++++++++++++ libavutil/ccfifo.h | 85 ++++++++++++++++++++ 3 files changed, 278 insertions(+) create mode 100644 libavutil/ccfifo.c create mode 100644 libavutil/ccfifo.h diff --git a/libavutil/Makefile b/libavutil/Makefile index dc9012f9a8..b6115859fa 100644 --- a/libavutil/Makefile +++ b/libavutil/Makefile @@ -17,6 +17,7 @@ HEADERS = adler32.h \ buffer.h \ cast5.h \ camellia.h \ + ccfifo.h \ channel_layout.h \ common.h \ cpu.h \ @@ -113,6 +114,7 @@ OBJS = adler32.o \ buffer.o \ cast5.o \ camellia.o \ + ccfifo.o \ channel_layout.o \ cpu.o \ crc.o \ diff --git a/libavutil/ccfifo.c b/libavutil/ccfifo.c new file mode 100644 index 0000000000..5db4149f3b --- /dev/null +++ b/libavutil/ccfifo.c @@ -0,0 +1,191 @@ +/* + * CEA-708 Closed Captioning FIFO + * Copyright (c) 2023 LTN Global Communications + * + * Author: Devin Heitmueller + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "ccfifo.h" + +struct AVCCFifo { + AVFifo *cc_608_fifo; + AVFifo *cc_708_fifo; + int expected_cc_count; + int expected_608; + int cc_detected; + void *log_ctx; +}; + +#define MAX_CC_ELEMENTS 128 +#define CC_BYTES_PER_ENTRY 3 + +struct cc_lookup { + int num; + int den; + int cc_count; + int num_608; +}; + +const static struct cc_lookup cc_lookup_vals[] = { + { 15, 1, 40, 4 }, + { 24, 1, 25, 3 }, + { 24000, 1001, 25, 3 }, + { 30, 1, 20, 2 }, + { 30000, 1001, 20, 2}, + { 60, 1, 10, 1 }, + { 60000, 1001, 10, 1}, +}; + +void av_ccfifo_freep(AVCCFifo **ccf) +{ + if (ccf && *ccf) { + AVCCFifo *tmp = *ccf; + if (tmp->cc_608_fifo) + av_fifo_freep2(&tmp->cc_608_fifo); + if (tmp->cc_708_fifo) + av_fifo_freep2(&tmp->cc_708_fifo); + av_freep(*ccf); + } +} + +AVCCFifo *av_ccfifo_alloc(AVRational *framerate, void *log_ctx) +{ + AVCCFifo *ccf; + int i; + + ccf = av_mallocz(sizeof(*ccf)); + if (!ccf) + return NULL; + + if (!(ccf->cc_708_fifo = av_fifo_alloc2(MAX_CC_ELEMENTS, CC_BYTES_PER_ENTRY, 0))) + goto error; + + if (!(ccf->cc_608_fifo = av_fifo_alloc2(MAX_CC_ELEMENTS, CC_BYTES_PER_ENTRY, 0))) + goto error; + + /* Based on the target FPS, figure out the expected cc_count and number of + 608 tuples per packet. See ANSI/CTA-708-E Sec 4.3.6.1. */ + for (i = 0; i < (sizeof(cc_lookup_vals) / sizeof(struct cc_lookup)); i++) { + if (framerate->num == cc_lookup_vals[i].num && + framerate->den == cc_lookup_vals[i].den) { + ccf->expected_cc_count = cc_lookup_vals[i].cc_count; + ccf->expected_608 = cc_lookup_vals[i].num_608; + break; + } + } + + if (ccf->expected_608 == 0) { + av_log(ccf->log_ctx, AV_LOG_WARNING, "cc_fifo cannot transcode captions fps=%d/%d\n", + framerate->num, framerate->den); + return NULL; + } + + return ccf; + +error: + av_ccfifo_freep(&ccf); + return NULL; +} + +int av_ccfifo_inject(AVCCFifo *ccf, AVFrame *frame) +{ + AVFrameSideData *sd; + int cc_filled = 0; + int i; + + if (!ccf) + return 0; + + if (ccf->cc_detected == 0 || ccf->expected_cc_count == 0) + return 0; + + sd = av_frame_new_side_data(frame, AV_FRAME_DATA_A53_CC, + ccf->expected_cc_count * CC_BYTES_PER_ENTRY); + if (!sd) + return 0; + + for (i = 0; i < ccf->expected_608; i++) { + if (av_fifo_can_read(ccf->cc_608_fifo) >= CC_BYTES_PER_ENTRY) { + av_fifo_read(ccf->cc_608_fifo, &sd->data[cc_filled * CC_BYTES_PER_ENTRY], + CC_BYTES_PER_ENTRY); + cc_filled++; + } else { + break; + } + } + + /* Insert any available data from the 708 FIFO */ + while (cc_filled < ccf->expected_cc_count) { + if (av_fifo_can_read(ccf->cc_708_fifo) >= CC_BYTES_PER_ENTRY) { + av_fifo_read(ccf->cc_708_fifo, &sd->data[cc_filled * CC_BYTES_PER_ENTRY], + CC_BYTES_PER_ENTRY); + cc_filled++; + } else { + break; + } + } + + /* Insert 708 padding into any remaining fields */ + while (cc_filled < ccf->expected_cc_count) { + sd->data[cc_filled * CC_BYTES_PER_ENTRY] = 0xfa; + sd->data[cc_filled * CC_BYTES_PER_ENTRY + 1] = 0x00; + sd->data[cc_filled * CC_BYTES_PER_ENTRY + 2] = 0x00; + cc_filled++; + } + + return 0; +} + +int av_ccfifo_extract(AVCCFifo *ccf, AVFrame *frame) +{ + int i; + + if (!ccf) + return 0; + + /* Read the A53 side data, discard padding, and put 608/708 into + queues so we can ensure they get into the output frames at + the correct rate... */ + if (ccf->expected_cc_count > 0) { + AVFrameSideData *side_data = av_frame_get_side_data(frame, AV_FRAME_DATA_A53_CC); + if (side_data) { + uint8_t *cc_bytes = side_data->data; + int cc_count = side_data->size / CC_BYTES_PER_ENTRY; + ccf->cc_detected = 1; + + for (i = 0; i < cc_count; i++) { + /* See ANSI/CTA-708-E Sec 4.3, Table 3 */ + uint8_t cc_valid = (cc_bytes[CC_BYTES_PER_ENTRY*i] & 0x04) >> 2; + uint8_t cc_type = cc_bytes[CC_BYTES_PER_ENTRY*i] & 0x03; + if (cc_type == 0x00 || cc_type == 0x01) { + av_fifo_write(ccf->cc_608_fifo, &cc_bytes[CC_BYTES_PER_ENTRY*i], + CC_BYTES_PER_ENTRY); + } else if (cc_valid && (cc_type == 0x02 || cc_type == 0x03)) { + av_fifo_write(ccf->cc_708_fifo, &cc_bytes[CC_BYTES_PER_ENTRY*i], + CC_BYTES_PER_ENTRY); + } + } + + /* Remove the side data, as we will re-create it on the + output as needed */ + av_frame_remove_side_data(frame, AV_FRAME_DATA_A53_CC); + } + } + return 0; +} diff --git a/libavutil/ccfifo.h b/libavutil/ccfifo.h new file mode 100644 index 0000000000..ad48c0a39c --- /dev/null +++ b/libavutil/ccfifo.h @@ -0,0 +1,85 @@ +/* + * CEA-708 Closed Captioning FIFO + * Copyright (c) 2023 LTN Global Communications + * + * Author: Devin Heitmueller + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * CC FIFO Buffer + */ + +#ifndef AVUTIL_CCFIFO_H +#define AVUTIL_CCFIFO_H + +#include "avutil.h" +#include "frame.h" +#include "fifo.h" + +typedef struct AVCCFifo AVCCFifo; + +/** + * Allocate an AVCCFifo. + * + * @param sample_fmt sample format + * @param channels number of channels + * @param nb_samples initial allocation size, in samples + * @return newly allocated AVCCFifo, or NULL on error + */ +AVCCFifo *av_ccfifo_alloc(AVRational *framerate, void *log_ctx); + +/** + * Free an AVCCFifo + * + * @param ccf Pointer to the pointer to the AVCCFifo which should be freed + * @note `*ptr = NULL` is safe and leads to no action. + */ +void av_ccfifo_freep(AVCCFifo **ccf); + + +/** + * Read a frame into a CC Fifo + * + * Extract CC bytes from the AVFrame, insert them into our queue, and + * remove the side data from the AVFrame. The side data is removed + * as it will be re-inserted at the appropriate rate later in the + * filter. + * + * @param af AVCCFifo to write to + * @param frame AVFrame with the video frame to operate on + * @return Zero on success, or negative AVERROR + * code on failure. + */ +int av_ccfifo_extract(AVCCFifo *af, AVFrame *frame); + +/** + * Insert CC data from the FIFO into an AVFrame (as side data) + * + * Dequeue the appropriate number of CC tuples based on the + * frame rate, and insert them into the AVFrame + * + * @param af AVCCFifo to read from + * @param frame AVFrame with the video frame to operate on + * @return Zero on success, or negative AVERROR + * code on failure. + */ +int av_ccfifo_inject(AVCCFifo *af, AVFrame *frame); + +#endif /* AVUTIL_CCFIFO_H */ From patchwork Mon Mar 27 16:47:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 40877 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:7a30:b0:df:834d:2c1a with SMTP id t48csp1485315pzh; Mon, 27 Mar 2023 09:47:40 -0700 (PDT) X-Google-Smtp-Source: AKy350aKyq13eEb4XafMO5Q9B2jEkk4XQVofyy0RnJeTk+cgjQ7U7Gf8cOwp9A06M/ywWFRDZtnz X-Received: by 2002:aa7:c7d4:0:b0:500:40b5:193a with SMTP id o20-20020aa7c7d4000000b0050040b5193amr12945737eds.17.1679935659821; Mon, 27 Mar 2023 09:47:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1679935659; cv=none; d=google.com; s=arc-20160816; b=EhQSBSZtnS9WCk8ywqH27YfnLF699p1N0tnQT1ehUCH0s6jPO/QOUaFfsr9QCucwoa 41tCzbeQ4njS5oQFBRKlYvAI7UV1bikZEopZ59wy+rpRWWH8chyD9GFoEDrSt+syx1oZ b8qDI84gXRnptlkuLHbhrFWNoeiI7r6Xuc3srz6HyGdt/+JgNSKEiKAkMQYlSe9bVDVl W4GU+vKuYrTpd414iXo6cqSoWayFofPigrksEhnbZLJgwLiBdVUMaR2r4KYMmvi1L8DS laUtKOIpP+iSDxT6vCyIfmyPukvY0JPaVUow58qVpR/BESZiIMVqcRE8kv5boFh7argl j0MA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=hcxQZapIfB9EBl7f3v+zeTIdJ8VNYml2c3EUUJOHx7A=; b=O8dNfEqG4PSzeGBVD5/FgTMy5wuMGxS4xvk0cAtrqDae6Z4R9SRF747+HextkSilgE Pyg9JpYOvTEQxhHN00Cs5bWYzNegwitg+F1+mvOEq4cSS+3Cf8ZgxuRs7KtV26YDTgud y0iarlvbqwHafbYHNHRa21FE527psAskLZpdnDlkafFRoemoo+kag+2DSMudTjxPUALt fYJX+Xt0ARLIwlc8w/9rq7gM1y3C+r6w07gqpSuTORw4giZgiZ8lV7VpI9JGynLjRbQR U84vmGLTApidZyScsjhAM+LDNysMorSqp8l1GN5ftJuOAwXzIZxrLZNzwBusQsuRqGPq AgcQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=rKwc40Pr; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id bu2-20020a170906a14200b008d7fab1c562si21248428ejb.115.2023.03.27.09.47.39; Mon, 27 Mar 2023 09:47:39 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=rKwc40Pr; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9AA9C68CB27; Mon, 27 Mar 2023 19:47:19 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qv1-f41.google.com (mail-qv1-f41.google.com [209.85.219.41]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3959E68C570 for ; Mon, 27 Mar 2023 19:47:11 +0300 (EEST) Received: by mail-qv1-f41.google.com with SMTP id m6so7299690qvq.0 for ; Mon, 27 Mar 2023 09:47:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20210112.gappssmtp.com; s=20210112; t=1679935630; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0bCQOPrmDxAb+8BhhkfSXibGnT/xgDHTuqXZh1Sn7iE=; b=rKwc40Pr0AW2sbvcSKkrP8FDkyoLhe1os2iDkGkF+jtEmacTmnN2ot7Ll//Zx6WD6v 819OebJlFmHm7bNmZNpE0FoRI2V+NRPMjqE4IWv71V2ZmwTEKrGySlMOMNYuFFiLtu0w +PWTk9XC9PI+uPoYKL3JIRbpE/18bw62140Alcqp3xeEDa2aNE+y0upg9908S96QDnx4 38RTGSsM5bTq+Qb5VJYlNS36IygOZv/GV49gEBr/iyYmEMjBjIz3mIVnmBtjL/E/RNYC dPOE7pIpCspDdNL8Bzj8hLQMBZ7+Ssg/Ctoq200vYdCUCBsHPFsC+8nicXAXj+5OhO/c R58Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679935630; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0bCQOPrmDxAb+8BhhkfSXibGnT/xgDHTuqXZh1Sn7iE=; b=C+acU2Hr6xR4SBj99V028flzcOA5ULbyLgslrlkaDerWiqpPdy7Je2y8WFfrAkK91W vZzCNnPAxz48ozCdsPIpO5sB5oqCicBVenBG8HZIuIzYRCt3oReSAJZ3TdwHCLbIBzAJ TWsGvKgkrmlGfbc3/i6O3dbzeWj+va7SVSUjKCF+CZroGLUeEK6cYLbNzFq6gcGLI8xP aF8lWnNEV3uAJesXqMuiVGyv+USLK7gN3xBauYJAfTGSd2ijnWJLTXcbNGFVbj3plPtj BHxeJxkdI/mExBuc2Cq1fRWzIMdODkpcAQdAsChJDNlc+j0suvpgNNxDs02vpwqScIPB KJ0g== X-Gm-Message-State: AAQBX9fDsJ/Lt8E1lJBSNN8E8KK16HM/AvgwTO56uJ+8Voo67gtzGxLO gSTVCy5e7zEG54AFNURnfz5INxKF/g6iCzf2tBY= X-Received: by 2002:ad4:5fcb:0:b0:56b:340c:ee1a with SMTP id jq11-20020ad45fcb000000b0056b340cee1amr20283141qvb.49.1679935629908; Mon, 27 Mar 2023 09:47:09 -0700 (PDT) Received: from klab-nyc-kernellabs2.localdomain (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id lf10-20020a0562142cca00b005dd8b9345e9sm3093100qvb.129.2023.03.27.09.47.09 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 27 Mar 2023 09:47:09 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Mon, 27 Mar 2023 12:47:01 -0400 Message-Id: <20230327164704.12962-3-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 2.35.1.655.ga68dfadae5 In-Reply-To: <20230327164704.12962-1-dheitmueller@ltnglobal.com> References: <20230327164704.12962-1-dheitmueller@ltnglobal.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/5] vf_fps: properly preserve CEA-708 captions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: WrP/OpzXM77p The existing implementation made an attempt to remove duplicate captions if increasing the framerate, but made no attempt to handle reducing the framerate, nor did it rewrite the caption payloads to have the appropriate cc_count (e.g. the cc_count needs to change from 20 to 10 when going from 1080i59 to 720p59 and vice-versa). Make use of the new ccfifo mechanism to ensure that caption data is properly preserved. Signed-off-by: Devin Heitmueller --- libavfilter/vf_fps.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/libavfilter/vf_fps.c b/libavfilter/vf_fps.c index 051d278f54..d2fab09c66 100644 --- a/libavfilter/vf_fps.c +++ b/libavfilter/vf_fps.c @@ -30,6 +30,7 @@ #include #include "libavutil/avassert.h" +#include "libavutil/ccfifo.h" #include "libavutil/eval.h" #include "libavutil/mathematics.h" #include "libavutil/opt.h" @@ -85,6 +86,7 @@ typedef struct FPSContext { AVFrame *frames[2]; ///< buffered frames int frames_count; ///< number of buffered frames + AVCCFifo *cc_fifo; ///< closed captions int64_t next_pts; ///< pts of the next frame to output @@ -165,6 +167,7 @@ static av_cold void uninit(AVFilterContext *ctx) frame = shift_frame(ctx, s); av_frame_free(&frame); } + av_ccfifo_freep(&s->cc_fifo); av_log(ctx, AV_LOG_VERBOSE, "%d frames in, %d frames out; %d frames dropped, " "%d frames duplicated.\n", s->frames_in, s->frames_out, s->drop, s->dup); @@ -210,6 +213,9 @@ static int config_props(AVFilterLink* outlink) s->in_pts_off, s->out_pts_off, s->start_time); } + if (!(s->cc_fifo = av_ccfifo_alloc(&outlink->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue. Captions will be passed through\n"); + av_log(ctx, AV_LOG_VERBOSE, "fps=%d/%d\n", outlink->frame_rate.num, outlink->frame_rate.den); return 0; @@ -242,6 +248,7 @@ static int read_frame(AVFilterContext *ctx, FPSContext *s, AVFilterLink *inlink, av_log(ctx, AV_LOG_DEBUG, "Read frame with in pts %"PRId64", out pts %"PRId64"\n", in_pts, frame->pts); + av_ccfifo_extract(s->cc_fifo, frame); s->frames[s->frames_count++] = frame; s->frames_in++; @@ -289,7 +296,7 @@ static int write_frame(AVFilterContext *ctx, FPSContext *s, AVFilterLink *outlin if (!frame) return AVERROR(ENOMEM); // Make sure Closed Captions will not be duplicated - av_frame_remove_side_data(s->frames[0], AV_FRAME_DATA_A53_CC); + av_ccfifo_inject(s->cc_fifo, frame); frame->pts = s->next_pts++; frame->duration = 1; From patchwork Mon Mar 27 16:47:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 40878 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:7a30:b0:df:834d:2c1a with SMTP id t48csp1485394pzh; Mon, 27 Mar 2023 09:47:48 -0700 (PDT) X-Google-Smtp-Source: AKy350bBthz5JNfR6jCK5qdaDweUQ5e/BD5e9AmIwyRF3qGFbA+PijIkKv75JypK+EG1Prt3D1Q9 X-Received: by 2002:a17:907:3f97:b0:93f:505b:9742 with SMTP id hr23-20020a1709073f9700b0093f505b9742mr10405297ejc.65.1679935668760; Mon, 27 Mar 2023 09:47:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1679935668; cv=none; d=google.com; s=arc-20160816; b=H4IA/eAd89jDLHBAVSlZ7j55W+AMuO8Nqm+ifL4XBe78x/kAlh4UZCzWQ6lGBcEuHc to+zBJp2F1bAQyFrgOFI7LLBy5PW4xPA/wjyXClJTmNx6gI4RSolfKHcR+38WLFPvhLE iWtS6YF38B+hqRmvpyfIEdkoqkrjPPpMQvA/ziJ0P6I1xcfMR3kOgDdMlBvlD6bQOk/S ZT3ye6Ta/t6nRN7RW1eM4ZhfntKDd7fQM6BEGy5QltGudooJw2eGA7KEu/YmWZW2Lo+7 XTJnwXgUeCfxzlsNx9cbeM9gXhAINgCbCzMMtqQzxhAY84mfXErDxpKBY+20GDDmge6x KquQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=LleGLIeaVuphbWJAHVTqE1glkwbDP0agSYSnJDoAGu0=; b=dbBdXy5J9XPS13X+Zn+0gD+5fVZk7PwrzL4l2bJxW+0eW+t3ckegkD22FNoCvyngbn 4bE/rszfICFUL8Py1CKEjuvIJkPmblgKa2xGVOTze5vOdgTBssiW/HR/WTz+ypwAQm2x 9jkH0KchY5bCBeTiLOz1TdwPtHsyTHFY166ia9oOBEqFdWaFbtQ//BJYYHHkXPUJGLnd bEIOPrFWQlOjvDW5iYIIoA++TG6fWEhs685+h8KYIPSVRn6hqdmtCLNIknnaICfWvdBN SAzltE0LLP44ZgPfX7Fqt/0/EPo4v4BxHtV9Oa7WLH6b1K9eCsNN/nK7F0AGKMEAt7wh JTAQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=6CxYDycz; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id xo21-20020a170907bb9500b009332ffa5891si14089575ejc.136.2023.03.27.09.47.48; Mon, 27 Mar 2023 09:47:48 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=6CxYDycz; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9CC9068C99F; Mon, 27 Mar 2023 19:47:20 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qv1-f41.google.com (mail-qv1-f41.google.com [209.85.219.41]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E008A68C99C for ; Mon, 27 Mar 2023 19:47:11 +0300 (EEST) Received: by mail-qv1-f41.google.com with SMTP id q88so7085347qvq.13 for ; Mon, 27 Mar 2023 09:47:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20210112.gappssmtp.com; s=20210112; t=1679935630; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rpYUnmwUkr8rWM0d+Fcnd4ripFUMXxsU8NwTIFpmeU8=; b=6CxYDyczTgnElQKk+JHpRxmaGeOf7p89MIn12ZikUddqSCyoFAc2FuJM/XGB685PWB 1DNhVMzJGkaRpML7QihWSZqNUHHRpIC5X979TZMn0RQ8Z3XD8oUeWrkTg5k55J077+kc yX/5wzm1bbYMxaT1mJOlJaMM+cczX1G9GTu8oTRism7xG/NRhPXU63OPmL0XQIaZRAk+ Z0FpAnW5WGKpp5uevSGpDQekHTYUidfJHIU/Q7Nbs2HKp13xVngZeNpAGJ8AEE3eHbaV bVhNWbU46Qt3KLosrF/0UUQNv6CVaN55+a3hv9r0ppu74IO2AzKq+lAX/n4EWSK2YJyV MaVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679935630; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rpYUnmwUkr8rWM0d+Fcnd4ripFUMXxsU8NwTIFpmeU8=; b=VfOv1MxtK0/dKGUiqPDeHc0IMWi3IIiGuWNAT4rRZUjWaMDnVuk//gstBIiCGidYER KjaL+hSyjllA2YsoCT1kO2VW2QC/0G8TbpMpmAqsif/OWsEkUDRJjFC1Cmjq0OwhLU/b JoyYkSj1Chde3oRDM+wn/GY0UnopyX50VyuY7+MBsJ2ZXfDHq3xtiexOeynbH28jpqXL +DmuVOAPmnxJvs5dK9InQ1kbJQdPGxHkwVqc7kwJGPCtkL1KQ4YEhvBNoqefitqHfIZj 00GCp7UqIfU8vFNCmQlmVXhRCT/o6T9aSg1A2UPJRhYOMYDNuY3EhhktJY2RElklXNjR VyXQ== X-Gm-Message-State: AAQBX9dp5uV+maN9qDb/ZYtMSIIsilRC8lRGj0nPMz9x0TLYEbDrc5Zq KhH//mdLXP8Y1AOObZWBsWZBvNO2t/7UMGFHG+Q= X-Received: by 2002:a05:6214:2428:b0:56e:fe7b:8bc2 with SMTP id gy8-20020a056214242800b0056efe7b8bc2mr24070150qvb.32.1679935630483; Mon, 27 Mar 2023 09:47:10 -0700 (PDT) Received: from klab-nyc-kernellabs2.localdomain (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id lf10-20020a0562142cca00b005dd8b9345e9sm3093100qvb.129.2023.03.27.09.47.10 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 27 Mar 2023 09:47:10 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Mon, 27 Mar 2023 12:47:02 -0400 Message-Id: <20230327164704.12962-4-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 2.35.1.655.ga68dfadae5 In-Reply-To: <20230327164704.12962-1-dheitmueller@ltnglobal.com> References: <20230327164704.12962-1-dheitmueller@ltnglobal.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 3/5] yadif: Properly preserve CEA-708 closed captions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 8pkYYrv66DkA Various deinterlacing modes have the effect of doubling the framerate, and we need to ensure that the caption data isn't duplicated (or else you get double captions on-screen). Use the new ccfifo mechanism for yadif (and yadif_cuda and bwdif since they use the same yadif core) so that CEA-708 data is properly preserved through this filter. Signed-off-by: Devin Heitmueller --- libavfilter/vf_bwdif.c | 7 +++++++ libavfilter/vf_yadif.c | 6 ++++++ libavfilter/vf_yadif_cuda.c | 8 ++++++++ libavfilter/yadif.h | 2 ++ libavfilter/yadif_common.c | 5 +++++ 5 files changed, 28 insertions(+) diff --git a/libavfilter/vf_bwdif.c b/libavfilter/vf_bwdif.c index 34e8c5e234..c625b68914 100644 --- a/libavfilter/vf_bwdif.c +++ b/libavfilter/vf_bwdif.c @@ -297,6 +297,7 @@ static av_cold void uninit(AVFilterContext *ctx) av_frame_free(&yadif->prev); av_frame_free(&yadif->cur ); av_frame_free(&yadif->next); + av_ccfifo_freep(&yadif->cc_fifo); } static const enum AVPixelFormat pix_fmts[] = { @@ -332,6 +333,12 @@ static int config_props(AVFilterLink *link) if(yadif->mode&1) link->frame_rate = av_mul_q(link->src->inputs[0]->frame_rate, (AVRational){2,1}); + else + link->frame_rate = ctx->inputs[0]->frame_rate; + + if (!(yadif->cc_fifo = av_ccfifo_alloc(&link->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue. Captions will be passed through\n"); + if (link->w < 3 || link->h < 4) { av_log(ctx, AV_LOG_ERROR, "Video of less than 3 columns or 4 lines is not supported\n"); diff --git a/libavfilter/vf_yadif.c b/libavfilter/vf_yadif.c index 1be02de1a9..b51d21f6ff 100644 --- a/libavfilter/vf_yadif.c +++ b/libavfilter/vf_yadif.c @@ -261,6 +261,7 @@ static av_cold void uninit(AVFilterContext *ctx) av_frame_free(&yadif->prev); av_frame_free(&yadif->cur ); av_frame_free(&yadif->next); + av_ccfifo_freep(&yadif->cc_fifo); } static const enum AVPixelFormat pix_fmts[] = { @@ -293,6 +294,11 @@ static int config_output(AVFilterLink *outlink) if(s->mode & 1) outlink->frame_rate = av_mul_q(ctx->inputs[0]->frame_rate, (AVRational){2, 1}); + else + outlink->frame_rate = ctx->inputs[0]->frame_rate; + + if (!(s->cc_fifo = av_ccfifo_alloc(&outlink->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue. Captions will be passed through\n"); if (outlink->w < 3 || outlink->h < 3) { av_log(ctx, AV_LOG_ERROR, "Video of less than 3 columns or lines is not supported\n"); diff --git a/libavfilter/vf_yadif_cuda.c b/libavfilter/vf_yadif_cuda.c index 685b8a2035..d96ec391a0 100644 --- a/libavfilter/vf_yadif_cuda.c +++ b/libavfilter/vf_yadif_cuda.c @@ -206,6 +206,9 @@ static av_cold void deint_cuda_uninit(AVFilterContext *ctx) av_frame_free(&y->cur); av_frame_free(&y->next); + if (yadif->cc_fifo) + av_cc_fifo_free(yadif->cc_fifo); + av_buffer_unref(&s->device_ref); s->hwctx = NULL; av_buffer_unref(&s->input_frames_ref); @@ -291,6 +294,11 @@ static int config_output(AVFilterLink *link) if(y->mode & 1) link->frame_rate = av_mul_q(ctx->inputs[0]->frame_rate, (AVRational){2, 1}); + else + outlink->frame_rate = ctx->inputs[0]->frame_rate; + + if (!(s->cc_fifo = av_cc_fifo_alloc(&outlink->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue. Captions will be passed through\n"); if (link->w < 3 || link->h < 3) { av_log(ctx, AV_LOG_ERROR, "Video of less than 3 columns or lines is not supported\n"); diff --git a/libavfilter/yadif.h b/libavfilter/yadif.h index c928911b35..ccd4304860 100644 --- a/libavfilter/yadif.h +++ b/libavfilter/yadif.h @@ -21,6 +21,7 @@ #include "libavutil/opt.h" #include "libavutil/pixdesc.h" +#include "libavutil/ccfifo.h" #include "avfilter.h" enum YADIFMode { @@ -76,6 +77,7 @@ typedef struct YADIFContext { int eof; uint8_t *temp_line; int temp_line_size; + AVCCFifo *cc_fifo; /* * An algorithm that treats first and/or last fields in a sequence diff --git a/libavfilter/yadif_common.c b/libavfilter/yadif_common.c index a10cf7a17f..24cc72ab5f 100644 --- a/libavfilter/yadif_common.c +++ b/libavfilter/yadif_common.c @@ -60,6 +60,8 @@ static int return_frame(AVFilterContext *ctx, int is_second) yadif->out->pts = AV_NOPTS_VALUE; } } + + av_ccfifo_inject(yadif->cc_fifo, yadif->out); ret = ff_filter_frame(ctx->outputs[0], yadif->out); yadif->frame_pending = (yadif->mode&1) && !is_second; @@ -96,6 +98,8 @@ int ff_yadif_filter_frame(AVFilterLink *link, AVFrame *frame) av_assert0(frame); + av_ccfifo_extract(yadif->cc_fifo, frame); + if (yadif->frame_pending) return_frame(ctx, 1); @@ -137,6 +141,7 @@ int ff_yadif_filter_frame(AVFilterLink *link, AVFrame *frame) if (!yadif->out) return AVERROR(ENOMEM); + av_ccfifo_inject(yadif->cc_fifo, yadif->out); av_frame_free(&yadif->prev); if (yadif->out->pts != AV_NOPTS_VALUE) yadif->out->pts *= 2; From patchwork Mon Mar 27 16:47:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 40879 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:7a30:b0:df:834d:2c1a with SMTP id t48csp1485520pzh; Mon, 27 Mar 2023 09:47:57 -0700 (PDT) X-Google-Smtp-Source: AKy350ZAF/VpVYyx9RiMF4EMzsKC36wIaOtvj+RiUB5nT0TE+E8doL5BtsibCt/z2C+2hvbmWSNF X-Received: by 2002:a17:907:c608:b0:92b:e1ff:be4e with SMTP id ud8-20020a170907c60800b0092be1ffbe4emr14012750ejc.1.1679935677565; Mon, 27 Mar 2023 09:47:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1679935677; cv=none; d=google.com; s=arc-20160816; b=sI/wXWcQIRSR8BmCPbzDCCCwoAZcdURQnS8Agf+84cJJTeheNv8gEevRa7fpL/z1R3 pAjjPtIdM7gC8BdgyOxYp0NDLGmJ8isKKroHLVReKZbbj4PlXfSmK2RPIUUO7yfosPzs TX4PLzy6UdE5DMBm3YjPs4m9J2GgwvJrIIe0pnEzoCDq3ZT3X15YNKQfHHGRMsB2GOOa h/qCk9XKGHnn0rsDLi73w79oGNUWCkoZSvPi44+j0AdQcGDQe1QtMkHRahwIHjwzI31p IJtssbdh0XwFobsaN2jbz8noAd+64ouORxX2u4mY2zV5vWneO2ycFJMbj+MlXshaghXJ 6YhQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=7hHfzJO6vyBRj9dk7sPHVZQAgh3KLg9bCg0MOrvMnq8=; b=AwdhANIsr4YxeRVe4OKdZxHO4DnCCfFmPj5tXQNZp7MMwPqEwFXNUCSTBe+G/Fia4i 6EFZ9dJ5hnDFu8bMbCZRV711W9Nc6u7zz443nH2LLbsDWVv1NNHhbONRCIu83OYXFz2/ Ur7wTw2E7fFyJPiDvxfuFBEugdJR/IQnJWjM5fyPXPEwbFGv2mMsLgcO6+tR4u1c/F8x 32ZTanM9I6uo5j2ConAwcwlGd0d6KJtLXcue1A3HkSZVq7pyLW79fSnsvYCH0ZUgsc3q S+LpQTtHkGX4nnRAzx59IeYyqkLHSNQyDfWkBX4v6LcGa8lvmdhCo9fkfyGnkiDKVLBL A3mg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=bK2i4gvx; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id w17-20020a170906131100b009332b99a1d0si22499697ejb.857.2023.03.27.09.47.57; Mon, 27 Mar 2023 09:47:57 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=bK2i4gvx; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id BFC0768CB51; Mon, 27 Mar 2023 19:47:21 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qv1-f41.google.com (mail-qv1-f41.google.com [209.85.219.41]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 84DFE68C9B9 for ; Mon, 27 Mar 2023 19:47:12 +0300 (EEST) Received: by mail-qv1-f41.google.com with SMTP id l7so7189694qvh.5 for ; Mon, 27 Mar 2023 09:47:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20210112.gappssmtp.com; s=20210112; t=1679935631; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=iwdsU5mMAz/VMxFzInA2ah2sSGhpvxB7qxJWVBiiXwI=; b=bK2i4gvxYJyNIEed95yXgJ6CjbO3N+M21Kk2uAEGktyygojRzlQdjzIXp+Sdv4VtBC UmFwABu302x3m1i8B9TanrtdqSQpfdtET63nAvBmHMHuby9fNirDXZ3wqWQPsBZchDC2 qW3KcCjCATh5X7CiJvkJwgLwME6kwNNfB0NfBDTQXyxKpVv9dFYMx9Llw+VHGkRh8Igj y7DurhpbFbNDfCfzWGft11t0+eS/r3OKTeup2soancnLUnvfkMYfQrkVo1Vl0NF9dU8N WxYL6tAhOa9H5sue3d0ybm4pUileGtVljlA6k9Tb+eldVMkTXqNxETPzWQpvGKcDzVsI zPEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679935631; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iwdsU5mMAz/VMxFzInA2ah2sSGhpvxB7qxJWVBiiXwI=; b=4T9CVbH+1FdmnaTuDLqUa+VEBVfWZWVN4yxEJVwExj7tnxkxE0xPVSqd0Fvxqj5rUi Ie4StVeb+/zlN5LpjpY3aK87kQgEHWZ77NMXQyh/LYr0PMRwuQ0NfVILovqEn3Y1v+sj Oda+TJ4R1MxTCr5x4fV7v6mCLHdHuZ21/WkJA7rK1YqsGrNFt/F76CwwhL9Arve04S3g yTln0bma4QXElWkysHC2GgOgRdKRZT5WmgmYHBF0K6CPpj5od7JMzofgH8W2LwQ2F6sc 0pgW0ttKrG+Lix8DVgZaVoT+w0o/YR2SzpJEH1gmY8KhRvV2/jDAD+aKfEO7tj2eW9q5 w4ag== X-Gm-Message-State: AAQBX9e2OUKQ0OLMB+R2788PXMZWTHgDzr42DCuKpOfoKQ0vC5V4YEI/ /xRtXXNSRNVyyNjJEWX0Gs42S4o5FjcHb9+IPOA= X-Received: by 2002:a05:6214:2506:b0:5c7:31af:2ea2 with SMTP id gf6-20020a056214250600b005c731af2ea2mr24328298qvb.12.1679935631137; Mon, 27 Mar 2023 09:47:11 -0700 (PDT) Received: from klab-nyc-kernellabs2.localdomain (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id lf10-20020a0562142cca00b005dd8b9345e9sm3093100qvb.129.2023.03.27.09.47.10 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 27 Mar 2023 09:47:10 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Mon, 27 Mar 2023 12:47:03 -0400 Message-Id: <20230327164704.12962-5-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 2.35.1.655.ga68dfadae5 In-Reply-To: <20230327164704.12962-1-dheitmueller@ltnglobal.com> References: <20230327164704.12962-1-dheitmueller@ltnglobal.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 4/5] tinterlace: Properly preserve CEA-708 closed captions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 1nt/cHTMS6bR Because the interlacing filter halves the effective framerate, we need to ensure that no CEA-708 data is lost as frames are merged. Make use of the new ccfifo mechanism to ensure that caption data is properly preserved as frames pass through the filter. Thanks to Thomas Mundt for review and noticing a couple of missed codepaths for injection on output. Signed-off-by: Devin Heitmueller --- libavfilter/tinterlace.h | 2 ++ libavfilter/vf_tinterlace.c | 8 ++++++++ 2 files changed, 10 insertions(+) diff --git a/libavfilter/tinterlace.h b/libavfilter/tinterlace.h index 37b6c10c08..30574c2ebf 100644 --- a/libavfilter/tinterlace.h +++ b/libavfilter/tinterlace.h @@ -30,6 +30,7 @@ #include "libavutil/bswap.h" #include "libavutil/opt.h" #include "libavutil/pixdesc.h" +#include "libavutil/ccfifo.h" #include "drawutils.h" #include "avfilter.h" @@ -77,6 +78,7 @@ typedef struct TInterlaceContext { const AVPixFmtDescriptor *csp; void (*lowpass_line)(uint8_t *dstp, ptrdiff_t width, const uint8_t *srcp, ptrdiff_t mref, ptrdiff_t pref, int clip_max); + AVCCFifo *cc_fifo; } TInterlaceContext; void ff_tinterlace_init_x86(TInterlaceContext *interlace); diff --git a/libavfilter/vf_tinterlace.c b/libavfilter/vf_tinterlace.c index 032629279a..226983c645 100644 --- a/libavfilter/vf_tinterlace.c +++ b/libavfilter/vf_tinterlace.c @@ -291,6 +291,9 @@ static int config_out_props(AVFilterLink *outlink) #endif } + if (!(tinterlace->cc_fifo = av_ccfifo_alloc(&outlink->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue. Captions will be passed through\n"); + av_log(ctx, AV_LOG_VERBOSE, "mode:%d filter:%s h:%d -> h:%d\n", tinterlace->mode, (tinterlace->flags & TINTERLACE_FLAG_CVLPF) ? "complex" : (tinterlace->flags & TINTERLACE_FLAG_VLPF) ? "linear" : "off", @@ -375,6 +378,8 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) tinterlace->cur = tinterlace->next; tinterlace->next = picref; + av_ccfifo_extract(tinterlace->cc_fifo, picref); + cur = tinterlace->cur; next = tinterlace->next; /* we need at least two frames */ @@ -451,6 +456,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) if (!out) return AVERROR(ENOMEM); out->pts /= 2; // adjust pts to new framerate + av_ccfifo_inject(tinterlace->cc_fifo, out); ret = ff_filter_frame(outlink, out); return ret; } @@ -486,6 +492,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) out->pts = cur->pts*2; out->pts = av_rescale_q(out->pts, tinterlace->preout_time_base, outlink->time_base); + av_ccfifo_inject(tinterlace->cc_fifo, out); if ((ret = ff_filter_frame(outlink, out)) < 0) return ret; @@ -521,6 +528,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) out->pts = av_rescale_q(out->pts, tinterlace->preout_time_base, outlink->time_base); out->duration = av_rescale_q(1, av_inv_q(outlink->frame_rate), outlink->time_base); + av_ccfifo_inject(tinterlace->cc_fifo, out); ret = ff_filter_frame(outlink, out); return ret; From patchwork Mon Mar 27 16:47:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 40880 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:7a30:b0:df:834d:2c1a with SMTP id t48csp1485660pzh; Mon, 27 Mar 2023 09:48:07 -0700 (PDT) X-Google-Smtp-Source: AK7set+MA0MeP3rivL0xCawcBGS1Hvm8EOwMC0Hhk8LRJeqQecApAxL3ZjGnqlK677zlYUIZYmlj X-Received: by 2002:a17:906:9b8f:b0:8af:22b4:99d2 with SMTP id dd15-20020a1709069b8f00b008af22b499d2mr21406985ejc.5.1679935686762; Mon, 27 Mar 2023 09:48:06 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1679935686; cv=none; d=google.com; s=arc-20160816; b=lA7GdpLhY/bQ/OCQ5mIDfy2quFHMHR+pIPY6Ymi+V5BJSyxEvb8s9HNsrgyuf8Tmk3 MMHIuTQgoBI8ZhYB530DytchAtA4+RrcbuWPraPN7t03fyaLoCEOslYMWVmGUdxeoh0t pHHlfiBniDdc/xEf2Q+tavtfPfYkq0YmX+rgWY4d4f0ptFpzMZzx46wPa9U+LSGRD9gG 3lMJ8yUGTIgmB0SSHgLeL+2nQ/N9nuyNvFG38VrfRUE7nIwk3bW7pQXurZfwdm0/qImF g2AzRWQvK/exkhWeCZSSXs7rAsVgZ45XGg5yZZcCm07ybmyI8XZ8Ujvqir8X1kOh53QM sMOw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=ka7FO+T1xh5XCZDnkghWnmEeoABBdiy1WdoWK2AwX1I=; b=qofTxOywmAxdSNJHWGiarf6Wsv2OcSCt2W+VY+eAuEJKJZfdlygtV0AvbdMwCOyu7X OBA37lNvVBvNB6q/RcJrazViTmKpKU+9ssWZrZ/Xsj5yTQHMYWrC9FQKjieYN7JW07F4 6kvgft7KbICev1KhGk1sVgdIJ+Ka0A2monER9nLUvoOitWoYbBpeABkH9TOGfdJMJQFU uVAsxaLI8qxPsQmqk2eg/168zraJXDDuHUPZnH0ad1km6Jb0IoVPoIPaxL3ab+0E2xyP N+cFqMIqV7nrQLkebMb1+7hd9fs8D8zVcMz1svP+3JPhEz1FDvNMBA4q21LQrPwAL56C uANg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=JXC95GY3; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id hr19-20020a1709073f9300b0093549a153f9si25296502ejc.912.2023.03.27.09.48.06; Mon, 27 Mar 2023 09:48:06 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=JXC95GY3; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1BEB168C600; Mon, 27 Mar 2023 19:47:23 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qv1-f44.google.com (mail-qv1-f44.google.com [209.85.219.44]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 33ABB68CB09 for ; Mon, 27 Mar 2023 19:47:13 +0300 (EEST) Received: by mail-qv1-f44.google.com with SMTP id jl13so7129558qvb.10 for ; Mon, 27 Mar 2023 09:47:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20210112.gappssmtp.com; s=20210112; t=1679935632; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=83dUY2NI/sip5h+TztFvCBw7Jk59lj0M3jhHRBSYSDg=; b=JXC95GY33e11qZZVqFPrGdRcTgEQeWEON8bDgWWV3eF+OMljaBu5BxjXxf3xSwFpu/ 45CIS8soAOoVs3Vfy0iN+jPNmIIjJZPON5ggOC5ZU4p0LuihjQJ74PjdGaGJOVkHv+bF 7kxGfm1/nLw6VPXfq5S+32p6bygXv+0aGaFg1MDolFTGQZTcmeEQkAfsMglm6GN3q+9l bzbI+IARazFlIht2n50W+NtR0JqdjBIVATQgKgk4uWo+fUzv9lQm9T3hCo4u6ZnuOfBG 1Uobqe9/oxigvJ4Wyqyeen+57z59e+PcZhOazMzncnlVEptqcA0ZfftEuxXiUfRgSqtH 0EhA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679935632; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=83dUY2NI/sip5h+TztFvCBw7Jk59lj0M3jhHRBSYSDg=; b=DHECHsghXfs6Uy+KajTUi95F5fYWO2+PfX3pNWre0YeRlSh+t+8D1W3/dFwxNXEGe5 FmG4aGrZJjYSSLeq1hRkf8KhQJALXXQawTtR1aUsSY45VSyK0Idf/S8dD+dhREuMY73J haWETr0IWpsTBaoiIZWEzNMItV4iBH9OhCXiRTu1Gk11ekpMMvCZSy+QAE98khx6dCBf mJp2/N0PVNGD8/T+2NFJWACu/zPo4JOLT566kGXFp3gH2M2jMLu5TBtPlq/GNcNdeh44 P2+bCGLR+TaPXTiL9zYQMLAvcHRvoInXmRRoDVtA1GxoU5LTOYTlPMBkUlgFj1sm+Nis s9MA== X-Gm-Message-State: AAQBX9fJ2ObQRRK5FJciMMJDMhMSR7NrrluBWDq9VFzukhPkG5CxRy7z msrlP48kSFUotNRB+Cs6j91S8GmrqgD5dWCIJ34= X-Received: by 2002:a05:6214:dc6:b0:5cd:3326:780 with SMTP id 6-20020a0562140dc600b005cd33260780mr24974399qvt.38.1679935631762; Mon, 27 Mar 2023 09:47:11 -0700 (PDT) Received: from klab-nyc-kernellabs2.localdomain (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id lf10-20020a0562142cca00b005dd8b9345e9sm3093100qvb.129.2023.03.27.09.47.11 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 27 Mar 2023 09:47:11 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Mon, 27 Mar 2023 12:47:04 -0400 Message-Id: <20230327164704.12962-6-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 2.35.1.655.ga68dfadae5 In-Reply-To: <20230327164704.12962-1-dheitmueller@ltnglobal.com> References: <20230327164704.12962-1-dheitmueller@ltnglobal.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 5/5] vf_ccrepack: Add new filter to repack CEA-708 side data X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 4e0gvx2WW8iz THis filter can correct certain issues seen from upstream sources where the cc_count is not properly set or the CEA-608 tuples are not at the start of the payload as expected. Make use of the ccfifo to extract and immediately repack the CEA-708 side data, thereby removing any extra padding and ensuring the 608 tuples are at the front of the payload. Signed-off-by: Devin Heitmueller --- doc/filters.texi | 10 +++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/vf_ccrepack.c | 95 +++++++++++++++++++++++++++++++++++++++ 4 files changed, 107 insertions(+) create mode 100644 libavfilter/vf_ccrepack.c diff --git a/doc/filters.texi b/doc/filters.texi index 4449998ba4..d042b034f8 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -8901,6 +8901,16 @@ Only deinterlace frames marked as interlaced. The default value is @code{all}. @end table +@section ccrepack + +Repack CEA-708 closed captioning side data + +This filter fixes various issues seen with commerical encoders +related to upstream malformed CEA-708 payloads, specifically +incorrect number of tuples (wrong cc_count for the target FPS), +and incorrect ordering of tuples (i.e. the CEA-608 tuples are not at +the first entries in the payload). + @section cas Apply Contrast Adaptive Sharpen filter to video stream. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 71e198bbf9..6b0da0f34f 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -211,6 +211,7 @@ OBJS-$(CONFIG_BOXBLUR_OPENCL_FILTER) += vf_avgblur_opencl.o opencl.o \ opencl/avgblur.o boxblur.o OBJS-$(CONFIG_BWDIF_FILTER) += vf_bwdif.o yadif_common.o OBJS-$(CONFIG_CAS_FILTER) += vf_cas.o +OBJS-$(CONFIG_CCREPACK_FILTER) += vf_ccrepack.o OBJS-$(CONFIG_CHROMABER_VULKAN_FILTER) += vf_chromaber_vulkan.o vulkan.o vulkan_filter.o OBJS-$(CONFIG_CHROMAHOLD_FILTER) += vf_chromakey.o OBJS-$(CONFIG_CHROMAKEY_FILTER) += vf_chromakey.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index d7db46c2af..b38550b358 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -196,6 +196,7 @@ extern const AVFilter ff_vf_boxblur; extern const AVFilter ff_vf_boxblur_opencl; extern const AVFilter ff_vf_bwdif; extern const AVFilter ff_vf_cas; +extern const AVFilter ff_vf_ccrepack; extern const AVFilter ff_vf_chromaber_vulkan; extern const AVFilter ff_vf_chromahold; extern const AVFilter ff_vf_chromakey; diff --git a/libavfilter/vf_ccrepack.c b/libavfilter/vf_ccrepack.c new file mode 100644 index 0000000000..ab8473e7c1 --- /dev/null +++ b/libavfilter/vf_ccrepack.c @@ -0,0 +1,95 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/* + * Repackage CEA-708 arrays, which deals with incorrect cc_count for a given + * output framerate, and incorrect 708 padding. + * + * See CEA CEA-10-A "EIA-708-B Implementation Guidance", Section 26.5 + * "Grouping DTVCC Data Within user_data() Structure" + */ + +#include "avfilter.h" +#include "internal.h" +#include "libavutil/opt.h" +#include "libavutil/ccfifo.h" + +typedef struct CCRepackContext +{ + const AVClass *class; + AVCCFifo *cc_fifo; +} CCRepackContext; + +static const AVOption ccrepack_options[] = { + { NULL } +}; + +AVFILTER_DEFINE_CLASS(ccrepack); + +static int config_input(AVFilterLink *link) +{ + CCRepackContext *ctx = link->dst->priv; + + if (!(ctx->cc_fifo = av_ccfifo_alloc(&link->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue. Captions will be passed through\n"); + + return 0; +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *frame) +{ + CCRepackContext *ctx = inlink->dst->priv; + AVFilterLink *outlink = inlink->dst->outputs[0]; + + av_ccfifo_extract(ctx->cc_fifo, frame); + av_ccfifo_inject(ctx->cc_fifo, frame); + + return ff_filter_frame(outlink, frame); +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + CCRepackContext *s = ctx->priv; + av_ccfifo_freep(&s->cc_fifo); +} + +static const AVFilterPad avfilter_vf_ccrepack_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .filter_frame = filter_frame, + .config_props = config_input, + }, +}; + +static const AVFilterPad avfilter_vf_ccrepack_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + }, +}; + +AVFilter ff_vf_ccrepack = { + .name = "ccrepack", + .description = NULL_IF_CONFIG_SMALL("Repack CEA-708 closed caption metadata"), + .uninit = uninit, + .priv_size = sizeof(CCRepackContext), + .priv_class = &ccrepack_class, + FILTER_INPUTS(avfilter_vf_ccrepack_inputs), + FILTER_OUTPUTS(avfilter_vf_ccrepack_outputs), +};