From patchwork Fri Apr 28 16:37:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 41401 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:dca6:b0:f3:34fa:f187 with SMTP id ky38csp1439164pzb; Fri, 28 Apr 2023 08:42:36 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5ZKoa8KRZLEPETCHxg9xY0wpBZoejUxFJeWqd4QrPYWpxMHTPvFkhJdfV6xSls15OlhheN X-Received: by 2002:a50:fe8f:0:b0:506:c1a6:2771 with SMTP id d15-20020a50fe8f000000b00506c1a62771mr4514275edt.39.1682696555850; Fri, 28 Apr 2023 08:42:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682696555; cv=none; d=google.com; s=arc-20160816; b=tSKA9Qatf3h/nkaPvMezTOuEuu7x0TE21JlNWBsdUxttLTfYivPYlaWPcTYYD/UB5y JXi+rJemMyBC/mKbM9b9JNP7eJkpVpyvrVyLNHDdtwAFeebCYeWIc4NIu+W5S9YOGl6X 5xPqRaXyMX8cOLeAuG9kcDdQ/HODkgrQzKjBfUWEwLmU5Ps3gvVCs21baaOd35iLQ8kf YYaU6ykrMrX8QNrL6RCoiRGnRt/3mGDO/y6QZ4r2y8NFT9x4qiSF3q3v4paOSZk8O53E PlwWwFH3fiFu0FiyLAOscQCQIiAq3X4nHtgUUjDIwZRUVcm39LDGSimBVnzUgkcT9wvX qsSg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=niAqHOsP86ZlU19izeqJBgOYMinZq705WMG80Cd89TE=; b=EBsdhzxE0BW7Ym/WsgJDQogBtFEBLpvyKAd84q+Bnv1oZKu7r/QlC0t5JvSxhMzcC1 ab1CgA9OBm4YElHmXGkYPDLl44ockY9mcvfDspM5zz1QTkH5QLNets8vZC5uZtpLwmZD IdSD4qFI+kxeeJ87vymu++o6i/yETjoOOQjiMukXUvvdpYtjdQ8l+vi/FjhWxjyLbUGz UguJtwCaiOekXCdn2yd3xxcQllNv/Y7Eyltn79Z9BPUhzP4nOnmgmV2yLV/j+yzG4eLs AiHuHGl3lmvsfgyTAZ+xeO94kbZjbgAU6KNsei7kITtaaEnClGgphVwNfDYcFSOgn1iL wlZQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=D3mIbSVn; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id p18-20020a05640210d200b00506a7d9c0fdsi16795497edu.133.2023.04.28.08.42.35; Fri, 28 Apr 2023 08:42:35 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=D3mIbSVn; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5747968C007; Fri, 28 Apr 2023 18:42:19 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ua1-f45.google.com (mail-ua1-f45.google.com [209.85.222.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CBCE968BFE0 for ; Fri, 28 Apr 2023 18:42:12 +0300 (EEST) Received: by mail-ua1-f45.google.com with SMTP id a1e0cc1a2514c-77297c876aaso3163651241.0 for ; Fri, 28 Apr 2023 08:42:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20221208.gappssmtp.com; s=20221208; t=1682696531; x=1685288531; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=rShEXe66M0nR2zqmVJ0/mtYAr8RUfnYX8Urz51c0c5U=; b=D3mIbSVn/07ad+u8TzwpzbJaxBGcgGO8Rlao+nY3z3ilkFOEH9AMXw9HpPIkoMTrx2 7AnL5ZINOIuFjZ/k++3/MO1EOnAUZxG15/xwlQDDYX47bH0t/CIFr2eaoOMEQp6/jsTk o7BTuxfnIx759F3kEmkPDjDfQ2sFEFKblJ+q7z3lTqe/fh4OFn3rlZk4a86q3ekPe5RQ Tdio3Ava6nlsHROTCC2IewSBGwXQ1myC3HADE6Qo1xbDbpCZEnraxWZhHAOQ1zuuycPC w5ssX63Idk8eMJ8PB+77Ww2LypXAz26IbcCp51LsVLanhBjrsWKzN/QlFE82sS77iBFb EX0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682696531; x=1685288531; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=rShEXe66M0nR2zqmVJ0/mtYAr8RUfnYX8Urz51c0c5U=; b=eDFY5hEKpTXeZxqHlcZeQO7ooOZd/822LQiAYg8TEPISRs/Fsl/vxnQcAhiNKUaoPM aXFYXqodh7BNMI307Idz/lYsq2QV9A9qfsOnNtwS+xPA6gSnl0nXzRplqET5mdfJZ2Wx xYMBrOkt5CKZvc/KQcnJ9qrot5oxxpVEwrc4fekDqhflbUJdbXgfR72ppnsiy5jtEEST 2ukjpesCvRTAjukBkbazGzXwJfaBymjq04XIC5O19y5HbCrAK2v7YwCZzjloRc6xrWaR lWwhRN58IX/9pEJfmBBzhF6Ye5vexb5w4vnC+EDC00/guN3MtgKcNOp8NAMmmozMbOF2 7Mpg== X-Gm-Message-State: AC+VfDw2KRTPtJRpzfe6K1l+5Gpx+MR++W/NRzpFCSdYvBQ5IllVqqSa oDYH7ymcgZM4ScSvmJxrtfh3Ou+egSANZulkL20= X-Received: by 2002:a1f:c1d3:0:b0:43f:e858:e948 with SMTP id r202-20020a1fc1d3000000b0043fe858e948mr2300273vkf.6.1682696530821; Fri, 28 Apr 2023 08:42:10 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id do54-20020a05620a2b3600b0074e13ed6ee9sm6733873qkb.132.2023.04.28.08.42.10 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 28 Apr 2023 08:42:10 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 28 Apr 2023 12:37:46 -0400 Message-Id: <1682699871-22331-2-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1682699871-22331-1-git-send-email-dheitmueller@ltnglobal.com> References: <1682699871-22331-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [PATCH v4 1/6] ccfifo: Properly handle CEA-708 captions through framerate conversion X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 8TJ1VfAWdymq When transcoding video that contains 708 closed captions, the caption data is tied to the frames as side data. Simply dropping or adding frames to change the framerate will result in loss of data, so the caption data needs to be preserved and reformatted. For example, without this patch converting 720p59 to 1080i59 would result in loss of 50% of the caption bytes, resulting in garbled 608 captions and 708 probably wouldn't render at all. Further, the frames that are there will have an illegal cc_count for the target framerate, so some decoders may ignore the packets entirely. Extract the 608 and 708 tuples and insert them onto queues. Then after dropping/adding frames, re-write the tuples back into the resulting frames at the appropriate rate given the target framerate. This includes both having the correct cc_count as well as clocking out the 608 pairs at the appropriate rate. Thanks for Lance Wang for providing review/feedback. Signed-off-by: Devin Heitmueller --- libavfilter/Makefile | 1 + libavfilter/ccfifo.c | 240 +++++++++++++++++++++++++++++++++++++++++++++++++++ libavfilter/ccfifo.h | 94 ++++++++++++++++++++ 3 files changed, 335 insertions(+) create mode 100644 libavfilter/ccfifo.c create mode 100644 libavfilter/ccfifo.h diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 71e198b..628ade8 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -14,6 +14,7 @@ OBJS = allfilters.o \ buffersink.o \ buffersrc.o \ colorspace.o \ + ccfifo.o \ drawutils.o \ fifo.o \ formats.o \ diff --git a/libavfilter/ccfifo.c b/libavfilter/ccfifo.c new file mode 100644 index 0000000..05a77dd --- /dev/null +++ b/libavfilter/ccfifo.c @@ -0,0 +1,240 @@ +/* + * CEA-708 Closed Captioning FIFO + * Copyright (c) 2023 LTN Global Communications + * + * Author: Devin Heitmueller + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "ccfifo.h" + +struct AVCCFifo { + AVFifo *cc_608_fifo; + AVFifo *cc_708_fifo; + int expected_cc_count; + int expected_608; + int cc_detected; + int passthrough; + void *log_ctx; +}; + +#define MAX_CC_ELEMENTS 128 +#define CC_BYTES_PER_ENTRY 3 + +struct cc_lookup { + int num; + int den; + int cc_count; + int num_608; +}; + +const static struct cc_lookup cc_lookup_vals[] = { + { 15, 1, 40, 4 }, + { 24, 1, 25, 3 }, + { 24000, 1001, 25, 3 }, + { 30, 1, 20, 2 }, + { 30000, 1001, 20, 2}, + { 60, 1, 10, 1 }, + { 60000, 1001, 10, 1}, +}; + +void ff_ccfifo_freep(AVCCFifo **ccf) +{ + if (ccf && *ccf) { + AVCCFifo *tmp = *ccf; + if (tmp->cc_608_fifo) + av_fifo_freep2(&tmp->cc_608_fifo); + if (tmp->cc_708_fifo) + av_fifo_freep2(&tmp->cc_708_fifo); + av_freep(*ccf); + } +} + +AVCCFifo *ff_ccfifo_alloc(AVRational *framerate, void *log_ctx) +{ + AVCCFifo *ccf; + int i; + + ccf = av_mallocz(sizeof(*ccf)); + if (!ccf) + return NULL; + + if (!(ccf->cc_708_fifo = av_fifo_alloc2(MAX_CC_ELEMENTS, CC_BYTES_PER_ENTRY, 0))) + goto error; + + if (!(ccf->cc_608_fifo = av_fifo_alloc2(MAX_CC_ELEMENTS, CC_BYTES_PER_ENTRY, 0))) + goto error; + + /* Based on the target FPS, figure out the expected cc_count and number of + 608 tuples per packet. See ANSI/CTA-708-E Sec 4.3.6.1. */ + for (i = 0; i < FF_ARRAY_ELEMS(cc_lookup_vals); i++) { + if (framerate->num == cc_lookup_vals[i].num && + framerate->den == cc_lookup_vals[i].den) { + ccf->expected_cc_count = cc_lookup_vals[i].cc_count; + ccf->expected_608 = cc_lookup_vals[i].num_608; + break; + } + } + + if (ccf->expected_608 == 0) { + /* We didn't find an output frame we support. We'll let the call succeed + and the FIFO to be allocated, but the extract/inject functions will simply + leave everything the way it is */ + av_log(ccf->log_ctx, AV_LOG_WARNING, "cc_fifo cannot transcode captions fps=%d/%d\n", + framerate->num, framerate->den); + ccf->passthrough = 1; + } + + return ccf; + +error: + ff_ccfifo_freep(&ccf); + return NULL; +} + +int ff_ccfifo_injectbytes(AVCCFifo *ccf, uint8_t **data, size_t *len) +{ + char *cc_data; + int cc_filled = 0; + int i; + + if (!ccf) + return AVERROR(EINVAL); + + if (ccf->passthrough) { + *data = NULL; + *len = 0; + return 0; + } + + cc_data = av_mallocz(ccf->expected_cc_count * CC_BYTES_PER_ENTRY); + if (!cc_data) { + return AVERROR(ENOMEM); + } + + for (i = 0; i < ccf->expected_608; i++) { + if (av_fifo_can_read(ccf->cc_608_fifo) >= CC_BYTES_PER_ENTRY) { + av_fifo_read(ccf->cc_608_fifo, &cc_data[cc_filled * CC_BYTES_PER_ENTRY], + CC_BYTES_PER_ENTRY); + cc_filled++; + } else { + break; + } + } + + /* Insert any available data from the 708 FIFO */ + while (cc_filled < ccf->expected_cc_count) { + if (av_fifo_can_read(ccf->cc_708_fifo) >= CC_BYTES_PER_ENTRY) { + av_fifo_read(ccf->cc_708_fifo, &cc_data[cc_filled * CC_BYTES_PER_ENTRY], + CC_BYTES_PER_ENTRY); + cc_filled++; + } else { + break; + } + } + + /* Insert 708 padding into any remaining fields */ + while (cc_filled < ccf->expected_cc_count) { + cc_data[cc_filled * CC_BYTES_PER_ENTRY] = 0xfa; + cc_data[cc_filled * CC_BYTES_PER_ENTRY + 1] = 0x00; + cc_data[cc_filled * CC_BYTES_PER_ENTRY + 2] = 0x00; + cc_filled++; + } + + *data = cc_data; + *len = ccf->expected_cc_count * CC_BYTES_PER_ENTRY; + return 0; +} + +int ff_ccfifo_inject(AVCCFifo *ccf, AVFrame *frame) +{ + AVFrameSideData *sd; + uint8_t *cc_data; + size_t cc_size; + int ret; + + if (!ccf) + return AVERROR(EINVAL); + + if (ccf->passthrough == 1 || ccf->cc_detected == 0 || ccf->expected_cc_count == 0) + return 0; + + ret = ff_ccfifo_injectbytes(ccf, &cc_data, &cc_size); + if (ret == 0) { + sd = av_frame_new_side_data(frame, AV_FRAME_DATA_A53_CC, cc_size); + if (!sd) { + av_freep(&cc_data); + return AVERROR(ENOMEM); + } + memcpy(sd->data, cc_data, cc_size); + av_freep(&cc_data); + } + + return 0; +} + +int ff_ccfifo_extractbytes(AVCCFifo *ccf, uint8_t *cc_bytes, size_t len) +{ + int cc_count = len / CC_BYTES_PER_ENTRY; + + if (!ccf) + return AVERROR(EINVAL); + + if (ccf->passthrough == 1) + return 0; + + ccf->cc_detected = 1; + + for (int i = 0; i < cc_count; i++) { + /* See ANSI/CTA-708-E Sec 4.3, Table 3 */ + uint8_t cc_valid = (cc_bytes[CC_BYTES_PER_ENTRY*i] & 0x04) >> 2; + uint8_t cc_type = cc_bytes[CC_BYTES_PER_ENTRY*i] & 0x03; + if (cc_type == 0x00 || cc_type == 0x01) { + av_fifo_write(ccf->cc_608_fifo, &cc_bytes[CC_BYTES_PER_ENTRY*i], + CC_BYTES_PER_ENTRY); + } else if (cc_valid && (cc_type == 0x02 || cc_type == 0x03)) { + av_fifo_write(ccf->cc_708_fifo, &cc_bytes[CC_BYTES_PER_ENTRY*i], + CC_BYTES_PER_ENTRY); + } + } + return 0; +} + +int ff_ccfifo_extract(AVCCFifo *ccf, AVFrame *frame) +{ + if (!ccf) + return AVERROR(EINVAL); + + if (ccf->passthrough == 1) + return 0; + + /* Read the A53 side data, discard padding, and put 608/708 into + queues so we can ensure they get into the output frames at + the correct rate... */ + if (ccf->expected_cc_count > 0) { + AVFrameSideData *side_data = av_frame_get_side_data(frame, AV_FRAME_DATA_A53_CC); + if (side_data) { + ff_ccfifo_extractbytes(ccf, side_data->data, side_data->size); + + /* Remove the side data, as we will re-create it on the + output as needed */ + av_frame_remove_side_data(frame, AV_FRAME_DATA_A53_CC); + } + } + return 0; +} diff --git a/libavfilter/ccfifo.h b/libavfilter/ccfifo.h new file mode 100644 index 0000000..baa4ae1 --- /dev/null +++ b/libavfilter/ccfifo.h @@ -0,0 +1,94 @@ +/* + * CEA-708 Closed Captioning FIFO + * Copyright (c) 2023 LTN Global Communications + * + * Author: Devin Heitmueller + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * CC FIFO Buffer + */ + +#ifndef AVFILTER_CCFIFO_H +#define AVFILTER_CCFIFO_H + +#include "libavutil/avutil.h" +#include "libavutil/frame.h" +#include "libavutil/fifo.h" + +typedef struct AVCCFifo AVCCFifo; + +/** + * Allocate an AVCCFifo. + * + * @param framerate output framerate + * @param log_ctx used for any av_log() calls + * @return newly allocated AVCCFifo, or NULL on error + */ +AVCCFifo *ff_ccfifo_alloc(AVRational *framerate, void *log_ctx); + +/** + * Free an AVCCFifo + * + * @param ccf Pointer to the pointer to the AVCCFifo which should be freed + * @note `*ptr = NULL` is safe and leads to no action. + */ +void ff_ccfifo_freep(AVCCFifo **ccf); + + +/** + * Extract CC data from an AVFrame + * + * Extract CC bytes from the AVFrame, insert them into our queue, and + * remove the side data from the AVFrame. The side data is removed + * as it will be re-inserted at the appropriate rate later in the + * filter. + * + * @param af AVCCFifo to write to + * @param frame AVFrame with the video frame to operate on + * @return Zero on success, or negative AVERROR + * code on failure. + */ +int ff_ccfifo_extract(AVCCFifo *ccf, AVFrame *frame); + +/** + *Just like ff_ccfifo_extract(), but takes the raw bytes instead of an AVFrame + */ +int ff_ccfifo_extractbytes(AVCCFifo *ccf, uint8_t *data, size_t len); + +/** + * Insert CC data from the FIFO into an AVFrame (as side data) + * + * Dequeue the appropriate number of CC tuples based on the + * frame rate, and insert them into the AVFrame + * + * @param af AVCCFifo to read from + * @param frame AVFrame with the video frame to operate on + * @return Zero on success, or negative AVERROR + * code on failure. + */ +int ff_ccfifo_inject(AVCCFifo *ccf, AVFrame *frame); + +/* Just like ff_ccfifo_inject(), but takes the raw bytes instead of an AVFrame + * Note: the caller is responsible for calling av_freep() against the value + * returned in the "data" argument */ +int ff_ccfifo_injectbytes(AVCCFifo *ccf, uint8_t **data, size_t *len); + +#endif /* AVFILTER_CCFIFO_H */ From patchwork Fri Apr 28 16:37:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 41402 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:dca6:b0:f3:34fa:f187 with SMTP id ky38csp1439276pzb; Fri, 28 Apr 2023 08:42:46 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ61abgBav9NHb9cb8ymJC3d+bI9P5d52pZhoUNzUQTWIwThcCxGJV/j1Cl4n/pwxBsyNQsB X-Received: by 2002:a17:907:9807:b0:932:cec7:6801 with SMTP id ji7-20020a170907980700b00932cec76801mr5340708ejc.54.1682696565689; Fri, 28 Apr 2023 08:42:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682696565; cv=none; d=google.com; s=arc-20160816; b=YT+pFPHd+cpuauY0FAVt9ZCyDmvDSOYbDysQ4i+kSb0b0p+EBUJiKzMZQzk6lJXzR+ oGRwcBgbRCTzs+VJkEEKKAbjTyJG3fkHAbHF83jPvYvAv/08sNTQNj8VPS2AiDrcpT5L KwrPgRZmcHkiYuNvJFwACLpmpu5aVfeukeQLIU5dTgRpRzntMcPYF6gqLSM0wqSPA0st dgVo6ZTb3BAQkLAAMU+imd85DBreD4wDAyU9HC3fn+9epc4a7z10tK0+RNB4P3Q867Vh DS+b8rVLb8ELVIjfbarKn63Nj0zYmvBoS8veMrV+SkDYArwxG68hjJsNSVx1nWxZL+vw QayA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=VOADUg+jf0A2UhiTf6eb+09K5P8onfTsS0iCCddcbpU=; b=PreSosqQLolIFqD3U0NQeNPLFC3YGF56d12eDSnBtMeIvX5Z0aA8wOkIDHW2Nkf+BL v+WeMdMC+0KSWknfRnIgJSXc3cY4Wk8U9ezLGqmpjMxUhuHGWlGMojnSRyFTa+xQXy4F AzojdLo8tdm3KYTtJQkzb8NYmMIOpIw2Ddymka1NqQ97Gbk4ld4YJ3Eq0yE1VEMsRcum Frdg5Ekv2yUjsKrgkN/nQfMR8fFDMnv48QRVS/MkwwSi66PmYAmpZiBAggCCrsAqfXoF r6MdkGXZBA1xPflFLn/ruqvtlYMHV5LCMZ20HUrqSf6vWNMvcq0dBKQzqGVGeyy6/v+G DshA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=e09Iv9AM; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id kt9-20020a170906aac900b0094f364bfe04si15631075ejb.478.2023.04.28.08.42.45; Fri, 28 Apr 2023 08:42:45 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=e09Iv9AM; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id BE27C68BF86; Fri, 28 Apr 2023 18:42:20 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qk1-f178.google.com (mail-qk1-f178.google.com [209.85.222.178]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CF4A268BFED for ; Fri, 28 Apr 2023 18:42:12 +0300 (EEST) Received: by mail-qk1-f178.google.com with SMTP id af79cd13be357-74adf6adac6so981574785a.0 for ; Fri, 28 Apr 2023 08:42:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20221208.gappssmtp.com; s=20221208; t=1682696531; x=1685288531; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=DzQy+uTBYt4op9TDj0lkU4GTw46/UXsWTEH2+pLpR7Y=; b=e09Iv9AM4Nj/7j/8eH+Vv5m7Nm3OmYu01qShDGxZNnZSVvH8hrPAWR3G+badMDG2r+ EvEVgq+s30RWGGu6sCzssKHKYLvhyR+WXDi7IZ96tI9I16YcNt0B9PRQ5X0kTeq7xYKr 0T0K6PozSnb4qxC8If58372q+WEvWpvwFJ03h6kClpCiA4kHXjey5Sk+axmWRGp2vzkw SZ3rMg2tr0m18zga1C+b2npRHIIQWZStiPEjmZymAgafDnmtYdRg0csGDtfB6JPexMIF l9W2I2WbfP5hJJxvuMWziZDI7yqlt7YjcnBOreqOASwPGxYTdvlLt67q7LvUdoNBNCAB VlGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682696531; x=1685288531; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=DzQy+uTBYt4op9TDj0lkU4GTw46/UXsWTEH2+pLpR7Y=; b=Rvfn/Exd8i2mB//i6I80gn0AJQWOUSnktRuKsCe11wEiHUpyxy3vlF3QQmoOymwktO lzOqStVrhA/iOI/24MFXyO73WSGWFE765dVTpgfNckfHKsezdRSCuSB4r18BA64uAZKt gm6yDxPzr5oh+W2BFOxshmyNsJOiukA/m7ZrQ/fEg2HCFM/ciC0nM2uastxLJbu6vc3I Wi3f93w39yHy+j/JMHX/a/bHzx9+EyGr6FpI4K+0APpSxntFckHO6u43UfjarW1TW3Qu jCIoNFy7Z/5yqMAeR0OsgjJfIsd8iXibJwHYr+JnI/bHfAED4EL8fKsVUGKyWGfLGifO HLIQ== X-Gm-Message-State: AC+VfDzAjkZ6j7bvCwkhHcw4/xOrJhG+2ACmjWi0QEwbh2GgAVU1ReL+ NKdjVNaIkF3IyC6NF5wVU7Re7FrpTQrSEx52lqE= X-Received: by 2002:a05:6214:2523:b0:56c:13cc:d21f with SMTP id gg3-20020a056214252300b0056c13ccd21fmr9053356qvb.50.1682696531413; Fri, 28 Apr 2023 08:42:11 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id do54-20020a05620a2b3600b0074e13ed6ee9sm6733873qkb.132.2023.04.28.08.42.10 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 28 Apr 2023 08:42:11 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 28 Apr 2023 12:37:47 -0400 Message-Id: <1682699871-22331-3-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1682699871-22331-1-git-send-email-dheitmueller@ltnglobal.com> References: <1682699871-22331-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [PATCH v4 2/6] vf_fps: properly preserve CEA-708 captions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: iNjNNcrlMHaE The existing implementation made an attempt to remove duplicate captions if increasing the framerate, but made no attempt to handle reducing the framerate, nor did it rewrite the caption payloads to have the appropriate cc_count (e.g. the cc_count needs to change from 20 to 10 when going from 1080i59 to 720p59 and vice-versa). Make use of the new ccfifo mechanism to ensure that caption data is properly preserved. Signed-off-by: Devin Heitmueller --- libavfilter/vf_fps.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/libavfilter/vf_fps.c b/libavfilter/vf_fps.c index 051d278..19c727e 100644 --- a/libavfilter/vf_fps.c +++ b/libavfilter/vf_fps.c @@ -34,6 +34,7 @@ #include "libavutil/mathematics.h" #include "libavutil/opt.h" #include "avfilter.h" +#include "ccfifo.h" #include "filters.h" #include "internal.h" @@ -85,6 +86,7 @@ typedef struct FPSContext { AVFrame *frames[2]; ///< buffered frames int frames_count; ///< number of buffered frames + AVCCFifo *cc_fifo; ///< closed captions int64_t next_pts; ///< pts of the next frame to output @@ -165,6 +167,7 @@ static av_cold void uninit(AVFilterContext *ctx) frame = shift_frame(ctx, s); av_frame_free(&frame); } + ff_ccfifo_freep(&s->cc_fifo); av_log(ctx, AV_LOG_VERBOSE, "%d frames in, %d frames out; %d frames dropped, " "%d frames duplicated.\n", s->frames_in, s->frames_out, s->drop, s->dup); @@ -210,6 +213,9 @@ static int config_props(AVFilterLink* outlink) s->in_pts_off, s->out_pts_off, s->start_time); } + if (!(s->cc_fifo = ff_ccfifo_alloc(&outlink->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue\n"); + av_log(ctx, AV_LOG_VERBOSE, "fps=%d/%d\n", outlink->frame_rate.num, outlink->frame_rate.den); return 0; @@ -242,6 +248,7 @@ static int read_frame(AVFilterContext *ctx, FPSContext *s, AVFilterLink *inlink, av_log(ctx, AV_LOG_DEBUG, "Read frame with in pts %"PRId64", out pts %"PRId64"\n", in_pts, frame->pts); + ff_ccfifo_extract(s->cc_fifo, frame); s->frames[s->frames_count++] = frame; s->frames_in++; @@ -289,7 +296,7 @@ static int write_frame(AVFilterContext *ctx, FPSContext *s, AVFilterLink *outlin if (!frame) return AVERROR(ENOMEM); // Make sure Closed Captions will not be duplicated - av_frame_remove_side_data(s->frames[0], AV_FRAME_DATA_A53_CC); + ff_ccfifo_inject(s->cc_fifo, frame); frame->pts = s->next_pts++; frame->duration = 1; From patchwork Fri Apr 28 16:37:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 41403 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:dca6:b0:f3:34fa:f187 with SMTP id ky38csp1439340pzb; Fri, 28 Apr 2023 08:42:55 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7RNo53M+DODpYWqtbNZKM/j0bunPmO+fuC2idFSSS+8RmQNLtemZs6UbRyRcJGOBVPQp6k X-Received: by 2002:a17:907:5c9:b0:94e:8431:4767 with SMTP id wg9-20020a17090705c900b0094e84314767mr6355258ejb.38.1682696575128; Fri, 28 Apr 2023 08:42:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682696575; cv=none; d=google.com; s=arc-20160816; b=fC9p8IIhbKSb5aGFHibOYzCbbwDNh9Pr/s8Cp+m6zMiuoIA8dZAuD6zV1xHf1PROTZ hy89deKCMjFcAzAiC6zHDH37dPsWkDh/iebGYjOzZdbr4g/hYATO4AXBqOxLtrMWcB/B 6Ko9Hoa5F/emt4Zj0+Ydukto49WJL76VrNyX6csbrW+g4tuJWyrP25uNUKI/NLvsXX1N pT7D30YbFBAugPYdXrjIO/IFHqUWEujoQkyGTs94VYltGi7ARx0WFJD/xnZmk79mh4UZ QgWpwhSc9BqESYqooTyA6wfrjX80iyOABOz9Isp1pgMzcpFzLJGqL/6G4zAJvOaKdD2B couA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=rKeMC79WFZB/sYH2r4Le/E17h5eU+ZJHYpIpoaXM6SE=; b=BV3eO4v47src7GnF+W952IIN8fDDyc74qa+g5CxSXKxXkfU2qZ6UaZVeONfw4c/Tfb SlMlxUqMtTafd6AidY+zQLOCBleMa10meLvNQRiMJnvzIOi/FvtVANJOHmDk7XqalTFy ZIJgvXswFGjV+Qh5UghLuy/+jbUk+VolkafhvseQGH8mKBL9z7J0ummaHFi8jPSqJgIB TpVT+iWrO1V2Pd+YTKq9NwRP2LbMuTEtVOvJgUMeoxnwBf+AaQ32HotsCMgAgZ46Bkx4 57cgdavqgyD0k4/BgFQI5znek5GhdHE5DGSsrMRB2EwHO/GB7SpjT7MVuAnRdv/bXX50 QJtw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=bmvCSMH4; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id dt2-20020a170906b78200b0094f5aa7285asi14880007ejb.338.2023.04.28.08.42.54; Fri, 28 Apr 2023 08:42:55 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=bmvCSMH4; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 01D8B68C027; Fri, 28 Apr 2023 18:42:22 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qt1-f180.google.com (mail-qt1-f180.google.com [209.85.160.180]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A14F168B36D for ; Fri, 28 Apr 2023 18:42:13 +0300 (EEST) Received: by mail-qt1-f180.google.com with SMTP id d75a77b69052e-3ef6e84945dso33094071cf.2 for ; Fri, 28 Apr 2023 08:42:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20221208.gappssmtp.com; s=20221208; t=1682696532; x=1685288532; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=pvqNWkBJ53NGFYsifR1bhNht3LeZ7tMeBZoA5fvqVf8=; b=bmvCSMH4I5XwSzvL5zfw4aIzH0GLAcfvDxBzaamAXbPOxgiZlNx2ur7A7swjmy0W5C i3K76RECGaGkBpu9sj1FxmkrBsNzQRbF25DMOixPwm/5w/ogvYReN2xj1V3uwYvmmvGO e/oTFPck+d9FybH6yPNfyBL7TnbSLKgwo+jjXvDHUpx7c70RwncQVu/sgSwu/lGMaJSC gs5qC6SJ75C0lSLpfxRIrJcqDG7vzs2vsRhULnT06LmULE74Lsg1uOCAsfw1g4w1+KrS dEnrrKscTxwERDE1Kw9xs99bREWVteQZ+BScb+0cUasTvP4NioAUsKIzaiGfyxRjV4wi 7V5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682696532; x=1685288532; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=pvqNWkBJ53NGFYsifR1bhNht3LeZ7tMeBZoA5fvqVf8=; b=J/3y2H/x4uP6v0pfb+IXMhl/LWL4QEeiNHm9BcQS48eGaXH2br0waQM6SqqfiLWygO SJp9n2XFsrWqsxyD8BQDN5JP4LNJrEzEaQ/jb/xWqhd3RS3grBKpuoXy7oHQ4i60dF8o qG/+yJ3xOTm2daAdanlFmlZGaTks30phPZTM/ZEtbCHlSdFNYYGl/3Y7anZJUADBHIYa tumQ9QcOKfjuSY+wVBNN1KhbBr4oJNUsal9Kvh7WY/63Salk3AIbaNBQ7zc5Lpwl+JZh ITxhoubfPNLWvQjAXxyCEaV9rEiC2I0/TR8IlTUOg6kJsYQZDEXCWEkR6d67AgFKVy1d 5PIA== X-Gm-Message-State: AC+VfDz6wEXg/fKxZP8vmOrOLTNYiZlwnu4M5D7myZnuuzoclOLoFqUy CF/gayp9Z6yyQB/sFw6QIDRLKcbsG6U3UIDuRjY= X-Received: by 2002:a05:622a:4f:b0:3bf:e43f:6992 with SMTP id y15-20020a05622a004f00b003bfe43f6992mr9245541qtw.57.1682696531965; Fri, 28 Apr 2023 08:42:11 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id do54-20020a05620a2b3600b0074e13ed6ee9sm6733873qkb.132.2023.04.28.08.42.11 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 28 Apr 2023 08:42:11 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 28 Apr 2023 12:37:48 -0400 Message-Id: <1682699871-22331-4-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1682699871-22331-1-git-send-email-dheitmueller@ltnglobal.com> References: <1682699871-22331-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [PATCH v4 3/6] yadif: Properly preserve CEA-708 closed captions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: xG+MRi81Kerr Various deinterlacing modes have the effect of doubling the framerate, and we need to ensure that the caption data isn't duplicated (or else you get double captions on-screen). Use the new ccfifo mechanism for yadif (and yadif_cuda and bwdif since they use the same yadif core) so that CEA-708 data is properly preserved through this filter. Signed-off-by: Devin Heitmueller --- libavfilter/vf_bwdif.c | 7 +++++++ libavfilter/vf_yadif.c | 6 ++++++ libavfilter/vf_yadif_cuda.c | 8 ++++++++ libavfilter/yadif.h | 2 ++ libavfilter/yadif_common.c | 5 +++++ 5 files changed, 28 insertions(+) diff --git a/libavfilter/vf_bwdif.c b/libavfilter/vf_bwdif.c index 34e8c5e..fb46fdc 100644 --- a/libavfilter/vf_bwdif.c +++ b/libavfilter/vf_bwdif.c @@ -297,6 +297,7 @@ static av_cold void uninit(AVFilterContext *ctx) av_frame_free(&yadif->prev); av_frame_free(&yadif->cur ); av_frame_free(&yadif->next); + ff_ccfifo_freep(&yadif->cc_fifo); } static const enum AVPixelFormat pix_fmts[] = { @@ -332,6 +333,12 @@ static int config_props(AVFilterLink *link) if(yadif->mode&1) link->frame_rate = av_mul_q(link->src->inputs[0]->frame_rate, (AVRational){2,1}); + else + link->frame_rate = ctx->inputs[0]->frame_rate; + + if (!(yadif->cc_fifo = ff_ccfifo_alloc(&link->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue\n"); + if (link->w < 3 || link->h < 4) { av_log(ctx, AV_LOG_ERROR, "Video of less than 3 columns or 4 lines is not supported\n"); diff --git a/libavfilter/vf_yadif.c b/libavfilter/vf_yadif.c index 1be02de..d3b6900 100644 --- a/libavfilter/vf_yadif.c +++ b/libavfilter/vf_yadif.c @@ -261,6 +261,7 @@ static av_cold void uninit(AVFilterContext *ctx) av_frame_free(&yadif->prev); av_frame_free(&yadif->cur ); av_frame_free(&yadif->next); + ff_ccfifo_freep(&yadif->cc_fifo); } static const enum AVPixelFormat pix_fmts[] = { @@ -293,6 +294,11 @@ static int config_output(AVFilterLink *outlink) if(s->mode & 1) outlink->frame_rate = av_mul_q(ctx->inputs[0]->frame_rate, (AVRational){2, 1}); + else + outlink->frame_rate = ctx->inputs[0]->frame_rate; + + if (!(s->cc_fifo = ff_ccfifo_alloc(&outlink->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue\n"); if (outlink->w < 3 || outlink->h < 3) { av_log(ctx, AV_LOG_ERROR, "Video of less than 3 columns or lines is not supported\n"); diff --git a/libavfilter/vf_yadif_cuda.c b/libavfilter/vf_yadif_cuda.c index 685b8a2..e6773ff 100644 --- a/libavfilter/vf_yadif_cuda.c +++ b/libavfilter/vf_yadif_cuda.c @@ -206,6 +206,9 @@ static av_cold void deint_cuda_uninit(AVFilterContext *ctx) av_frame_free(&y->cur); av_frame_free(&y->next); + if (yadif->cc_fifo) + ff_cc_fifo_free(yadif->cc_fifo); + av_buffer_unref(&s->device_ref); s->hwctx = NULL; av_buffer_unref(&s->input_frames_ref); @@ -291,6 +294,11 @@ static int config_output(AVFilterLink *link) if(y->mode & 1) link->frame_rate = av_mul_q(ctx->inputs[0]->frame_rate, (AVRational){2, 1}); + else + outlink->frame_rate = ctx->inputs[0]->frame_rate; + + if (!(s->cc_fifo = ff_cc_fifo_alloc(&outlink->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue\n"); if (link->w < 3 || link->h < 3) { av_log(ctx, AV_LOG_ERROR, "Video of less than 3 columns or lines is not supported\n"); diff --git a/libavfilter/yadif.h b/libavfilter/yadif.h index c928911..1077576 100644 --- a/libavfilter/yadif.h +++ b/libavfilter/yadif.h @@ -22,6 +22,7 @@ #include "libavutil/opt.h" #include "libavutil/pixdesc.h" #include "avfilter.h" +#include "ccfifo.h" enum YADIFMode { YADIF_MODE_SEND_FRAME = 0, ///< send 1 frame for each frame @@ -76,6 +77,7 @@ typedef struct YADIFContext { int eof; uint8_t *temp_line; int temp_line_size; + AVCCFifo *cc_fifo; /* * An algorithm that treats first and/or last fields in a sequence diff --git a/libavfilter/yadif_common.c b/libavfilter/yadif_common.c index a10cf7a..676853e 100644 --- a/libavfilter/yadif_common.c +++ b/libavfilter/yadif_common.c @@ -60,6 +60,8 @@ static int return_frame(AVFilterContext *ctx, int is_second) yadif->out->pts = AV_NOPTS_VALUE; } } + + ff_ccfifo_inject(yadif->cc_fifo, yadif->out); ret = ff_filter_frame(ctx->outputs[0], yadif->out); yadif->frame_pending = (yadif->mode&1) && !is_second; @@ -96,6 +98,8 @@ int ff_yadif_filter_frame(AVFilterLink *link, AVFrame *frame) av_assert0(frame); + ff_ccfifo_extract(yadif->cc_fifo, frame); + if (yadif->frame_pending) return_frame(ctx, 1); @@ -137,6 +141,7 @@ int ff_yadif_filter_frame(AVFilterLink *link, AVFrame *frame) if (!yadif->out) return AVERROR(ENOMEM); + ff_ccfifo_inject(yadif->cc_fifo, yadif->out); av_frame_free(&yadif->prev); if (yadif->out->pts != AV_NOPTS_VALUE) yadif->out->pts *= 2; From patchwork Fri Apr 28 16:37:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 41404 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:dca6:b0:f3:34fa:f187 with SMTP id ky38csp1439440pzb; Fri, 28 Apr 2023 08:43:04 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6s8ZDUz/OTQOKAWy/Xb9Xki9ngFv4tbV/UqwYYXPgbFPG8VX/a5jtbcH1N4DB0kfKRTGfj X-Received: by 2002:a17:907:1b16:b0:93b:866:bafe with SMTP id mp22-20020a1709071b1600b0093b0866bafemr5267646ejc.56.1682696584326; Fri, 28 Apr 2023 08:43:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682696584; cv=none; d=google.com; s=arc-20160816; b=ipbhbjpUyqKn8skw3ZxVRU84o5jNpXqwbYagthfNBzjPl7+iKvVlYewMez5wSOCXcC 4yO5tIBq9uYjr17EbL1T5cp4QoFu7aqB2w1XwsPReo4HLTI5rzTe3LSotnZ01PIZoiXf hTD70XH6Het26/N7RkzdqnQ+iCQKBlxGzeS7Rlb/stA0MIwRCTH99rR4s0sR3PnEo7Tk dw1LSj0Bn79xsBLoMgO7HjDpFLhECS2+mUmgiRsgnAHAmEByXEUfjtuzV3BXNSc6PF5k MHRuQA2HNwqia4KkahK6hpq5o9WBvSz8MyNzJaUSIgVWs3mUP4cGzDhS0+a98gZCqRIT ow+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=prUwPdBiPZ/rFndzbZiCULfuFs1ZNdnNHg4hfWlZUgA=; b=qMvvvDVBUdUIrtTR1Y91TPVwx2IG71M2z6wYGijiRVnsAq94FEtT8FjjIVOKINCMy9 W6VvQWXiyzgPvAS26zsvjls5PLUmuTGq25nesBZ4LfEGbr6i86wWUXb719+h14qpunbc fMhSkdmrnNow1LOY0T6pbqWkk/BGaC/pePBbbXplssBWU7LA6ugnV34t++R7w9s1WLJA HvxbdUki1oOI7NShrZNbEO1HZsYaN9Sj+8IUqCm4r2kwbI+C6qV2YkAA8QacEtvrFXgd SgIzC2U5tJxSt8VKBVmkMiJKDtfox+0k2iw9JQ9pGN6iI1iJxgk70ZbaZkh5idb+1wSp +c4Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=kWsP+JSJ; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id sg37-20020a170907a42500b0095ec54a22bcsi7177509ejc.106.2023.04.28.08.43.03; Fri, 28 Apr 2023 08:43:04 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=kWsP+JSJ; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0B16168C023; Fri, 28 Apr 2023 18:42:23 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qk1-f172.google.com (mail-qk1-f172.google.com [209.85.222.172]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 46BA568C006 for ; Fri, 28 Apr 2023 18:42:14 +0300 (EEST) Received: by mail-qk1-f172.google.com with SMTP id af79cd13be357-75131c2997bso668479185a.1 for ; Fri, 28 Apr 2023 08:42:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20221208.gappssmtp.com; s=20221208; t=1682696533; x=1685288533; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=RECp0FoM08xJycNye3PsgZXhNbgE1KGRGsbU9jKw71w=; b=kWsP+JSJT1U3wTjuTG8MugAuf92Cyi2IvrKNXOFf9Wg4QNpkhtIth8Hbo+s252ETOG FTfu9TatJ5DwGTciPlwOuVYoIrVI8z7NfvDZo52d/2SgNFO/aSB3+UMOCAsIvuJZGDPW qfKVh0menZSZO4s15u6+S9ncUbAlq7FTAPFRSnz8jbSdXTRZx+jFCEne1QOTU6ybZDr/ +AqQZs5KmlLMedDeqF/VGwX5qJoL+yCHJXDjrCwvou4kU4aaxC+SMlgySb6hHIMqmEY7 zivJZW/9InU/3geRq3FqEinN9lJ1wGV44yWAhVjy/hy9vWHJ1Ul1eqk3T6HW7UPkBFDb ygKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682696533; x=1685288533; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=RECp0FoM08xJycNye3PsgZXhNbgE1KGRGsbU9jKw71w=; b=PcQ93axZXQkrC74DaFiJihEjYAGxjhuRHZzkG3X6k3+ue6X71fYgC2PKhOjrPHlU09 cJcd7ZhMM/jU9auq6uZlQkIyGZOeCKQaEtBWtLvP1gARdlJaiWOwP1BDir0wKMqqGfUP vCjqwVdRvv49d8I0kJYzijhtN9B348rHDFpY1JkVVMXq9tSk/DLgUjdssAE2yHyXK+Ga z3CqLD0UyJSsEyPsBfuBt7ZCFY7R//AoUb1sktVxFA0vAinNCIaVfP+5YYUDhjA/mPKl 74V6rjAhEOfCv4Ftspj/hyJ5//xhvvyCwCet/0LLf7AD+i00NcsjuzRbuKcExADTczVH cZ5w== X-Gm-Message-State: AC+VfDxpeiuOrxyXJMx+B9/xrCrtELFdHKaZa0YL7FtFWnU4WSX295vX adxMRKGXsEBb1RKnFTK7pLjtiJOzrEmJCfzeI90= X-Received: by 2002:a05:622a:170a:b0:3ef:2f81:7865 with SMTP id h10-20020a05622a170a00b003ef2f817865mr17984055qtk.33.1682696532709; Fri, 28 Apr 2023 08:42:12 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id do54-20020a05620a2b3600b0074e13ed6ee9sm6733873qkb.132.2023.04.28.08.42.12 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 28 Apr 2023 08:42:12 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 28 Apr 2023 12:37:49 -0400 Message-Id: <1682699871-22331-5-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1682699871-22331-1-git-send-email-dheitmueller@ltnglobal.com> References: <1682699871-22331-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [PATCH v4 4/6] tinterlace: Properly preserve CEA-708 closed captions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: M1wajQOuyO18 Because the interlacing filter halves the effective framerate, we need to ensure that no CEA-708 data is lost as frames are merged. Make use of the new ccfifo mechanism to ensure that caption data is properly preserved as frames pass through the filter. Thanks to Thomas Mundt for review and noticing a couple of missed codepaths for injection on output. Thanks to Lance Wang for pointing out a memory leak. Signed-off-by: Devin Heitmueller --- libavfilter/tinterlace.h | 2 ++ libavfilter/vf_tinterlace.c | 9 +++++++++ 2 files changed, 11 insertions(+) diff --git a/libavfilter/tinterlace.h b/libavfilter/tinterlace.h index 37b6c10..9f5ce7e 100644 --- a/libavfilter/tinterlace.h +++ b/libavfilter/tinterlace.h @@ -32,6 +32,7 @@ #include "libavutil/pixdesc.h" #include "drawutils.h" #include "avfilter.h" +#include "ccfifo.h" #define TINTERLACE_FLAG_VLPF 01 #define TINTERLACE_FLAG_CVLPF 2 @@ -77,6 +78,7 @@ typedef struct TInterlaceContext { const AVPixFmtDescriptor *csp; void (*lowpass_line)(uint8_t *dstp, ptrdiff_t width, const uint8_t *srcp, ptrdiff_t mref, ptrdiff_t pref, int clip_max); + AVCCFifo *cc_fifo; } TInterlaceContext; void ff_tinterlace_init_x86(TInterlaceContext *interlace); diff --git a/libavfilter/vf_tinterlace.c b/libavfilter/vf_tinterlace.c index 0326292..309bf83 100644 --- a/libavfilter/vf_tinterlace.c +++ b/libavfilter/vf_tinterlace.c @@ -203,6 +203,7 @@ static av_cold void uninit(AVFilterContext *ctx) av_frame_free(&tinterlace->next); av_freep(&tinterlace->black_data[0][0]); av_freep(&tinterlace->black_data[1][0]); + ff_ccfifo_freep(&tinterlace->cc_fifo); } static int config_out_props(AVFilterLink *outlink) @@ -291,6 +292,9 @@ static int config_out_props(AVFilterLink *outlink) #endif } + if (!(tinterlace->cc_fifo = ff_ccfifo_alloc(&outlink->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue\n"); + av_log(ctx, AV_LOG_VERBOSE, "mode:%d filter:%s h:%d -> h:%d\n", tinterlace->mode, (tinterlace->flags & TINTERLACE_FLAG_CVLPF) ? "complex" : (tinterlace->flags & TINTERLACE_FLAG_VLPF) ? "linear" : "off", @@ -375,6 +379,8 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) tinterlace->cur = tinterlace->next; tinterlace->next = picref; + ff_ccfifo_extract(tinterlace->cc_fifo, picref); + cur = tinterlace->cur; next = tinterlace->next; /* we need at least two frames */ @@ -451,6 +457,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) if (!out) return AVERROR(ENOMEM); out->pts /= 2; // adjust pts to new framerate + ff_ccfifo_inject(tinterlace->cc_fifo, out); ret = ff_filter_frame(outlink, out); return ret; } @@ -486,6 +493,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) out->pts = cur->pts*2; out->pts = av_rescale_q(out->pts, tinterlace->preout_time_base, outlink->time_base); + ff_ccfifo_inject(tinterlace->cc_fifo, out); if ((ret = ff_filter_frame(outlink, out)) < 0) return ret; @@ -521,6 +529,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) out->pts = av_rescale_q(out->pts, tinterlace->preout_time_base, outlink->time_base); out->duration = av_rescale_q(1, av_inv_q(outlink->frame_rate), outlink->time_base); + ff_ccfifo_inject(tinterlace->cc_fifo, out); ret = ff_filter_frame(outlink, out); return ret; From patchwork Fri Apr 28 16:37:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 41405 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:dca6:b0:f3:34fa:f187 with SMTP id ky38csp1439599pzb; Fri, 28 Apr 2023 08:43:15 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5Gn/8+FWKgk/tgjnvtG2r9edTB93x/2y8lMgJDawGtBxyywi41VqbdCMgi7UySZyvCqPW0 X-Received: by 2002:aa7:d84e:0:b0:504:9b63:4756 with SMTP id f14-20020aa7d84e000000b005049b634756mr4331678eds.18.1682696595381; Fri, 28 Apr 2023 08:43:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682696595; cv=none; d=google.com; s=arc-20160816; b=XIA4zeyh3dgOdOci6Loy+8NPe00/kvOlDBYYueT7lEO0Q7nt9h/RCWUCYUPMaoCu6A 5C796cR9E17cdj+q2r2rOF8MOmX4dm7cNpRdnJmqOVdy7jY203qvyv9KkqYPEQdEU6op XbCyv2ETj+1kPPCrbtlRr4tTHLYxj4wOrpVspAnqgpkVtE5L0qUDxovASgcfh/IMdh77 E1KZyKjeuDV7V1SLTrfHfcycTDqZ8ZcAEidClXiEBjNVXcgPRROkrn72m4Bh7rpggvDY mDmkc9BDaYax2Y1Ip1oKXJYJloBQlF40mLVjZ67+QGf4wmaqNjTBUQGWecmsTkl81yaZ mEmQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=beyuSkoef37HEDL6B2uLL/ASunEApeEFgzPiP39XNS4=; b=TMIdOafdGETT7fxabvo9SytSgiv8T1ECUgK88U1BZQGMvBMB36IFuimkIEntASk+HY AwtEsVlbTaT2PbclYcevRDY88ot8rR2jaHOn+mGQK63EctzoNBn18Kmk/+lXGTxMF6ra kiu4PWBqHSqt9UazDMDO9vstJKba+BUAgvLXwjzh0SAZTFKsvSnrsf+stbsqn1nZY1Db IC9eX2v8QS5vsQ0ZEYsEI+Zsftzki9Xxf4gzzl8m2hwvc9mdXwrn0NIlxZClruTJkbxE 3kgEC2zFKxWKx87JrDoH7sVq4H+EcXandFuMjQO07NTcwlJAEQ8pr9htj7ceTze119Kd eCQA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=M7LPOj38; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id q24-20020a056402041800b00504b01ceb0esi15613024edv.363.2023.04.28.08.43.13; Fri, 28 Apr 2023 08:43:15 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=M7LPOj38; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2CE3568008D; Fri, 28 Apr 2023 18:42:24 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qk1-f176.google.com (mail-qk1-f176.google.com [209.85.222.176]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id AA65468C00B for ; Fri, 28 Apr 2023 18:42:14 +0300 (EEST) Received: by mail-qk1-f176.google.com with SMTP id af79cd13be357-75131c2997bso668482585a.1 for ; Fri, 28 Apr 2023 08:42:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20221208.gappssmtp.com; s=20221208; t=1682696533; x=1685288533; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=Mg25YWkcljHBWleOyVU0CTB4AQOYtmnIgL1a55heBkI=; b=M7LPOj38DRLIMsR1pcdEfLpH23ORU1B5Sbic97uXtzCSDS+2Zfo+y5DJb2hN1F9kfw zfq2zOhr0hr9Ksne+hGSHkaFZ5k4naFiwoqSdzMGiyf3bLp56mPHORV6aULWLFkoyN4W 6xDTK7pwS4VMhH1/9V+boBZbu7Y/dGE0t02encSoxs86R6bxaiIoCmKZ8dAQzCkfWZNx +OxipA2ulGft860xgBgxU1C8EG7c8ELrpoCgOKpbBWoXFC6QmMVxunYGOQrUkS9/iVa4 b17CbSfRVxSbm89vvHLVuBsBU+aGNH39DA6F12OKbYQpR1Uyu5wZYQGelMfc8hD+T0+M 1/Hw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682696533; x=1685288533; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Mg25YWkcljHBWleOyVU0CTB4AQOYtmnIgL1a55heBkI=; b=VKlwcQIRDmpJs+pNOardTjyg5plrUFoJ4HPp6agu7iIREFSRiW9YIbHpVNZ06koh7P OcoCe42wdio10j8LHEL1h7gJ1Io+37kq1gljtXik2yW/7VoZLI430Lw+I6xzSXPT9ba1 5p2NQh2I6nQ/AIT2+EQ+QapDiZjif/UFyJ4Ysq9pe+PSpgafU3P/3MybY6IT5QRb7+lH sjlpOpYyEj/Y+jud03BL9FGfmULGTuZFmtNZzBZHnFeug7YitLWXZrvNEAccvahDbqRF SvT7hKIOyVunEF6B6XqJ+2cJRkYQ/7Ckj77YEBTDiKB1RDBWXS/YDOsI8BUtAlHckRJH 6CAQ== X-Gm-Message-State: AC+VfDx7LnynhHS8Y/z/yOr7qq8KqIbJwYxNowZ76hh5lnIZjXZFB4re mZBUva2EvaPSKegDknuLW5+OASvnTX8jjjVYkps= X-Received: by 2002:ac8:7f8e:0:b0:3ef:379c:71b9 with SMTP id z14-20020ac87f8e000000b003ef379c71b9mr17472703qtj.30.1682696533187; Fri, 28 Apr 2023 08:42:13 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id do54-20020a05620a2b3600b0074e13ed6ee9sm6733873qkb.132.2023.04.28.08.42.12 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 28 Apr 2023 08:42:12 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 28 Apr 2023 12:37:50 -0400 Message-Id: <1682699871-22331-6-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1682699871-22331-1-git-send-email-dheitmueller@ltnglobal.com> References: <1682699871-22331-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [PATCH v4 5/6] vf_ccrepack: Add new filter to repack CEA-708 side data X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: i/XW9XSX8Xqh THis filter can correct certain issues seen from upstream sources where the cc_count is not properly set or the CEA-608 tuples are not at the start of the payload as expected. Make use of the ccfifo to extract and immediately repack the CEA-708 side data, thereby removing any extra padding and ensuring the 608 tuples are at the front of the payload. Signed-off-by: Devin Heitmueller --- doc/filters.texi | 10 +++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/vf_ccrepack.c | 100 ++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 112 insertions(+) create mode 100644 libavfilter/vf_ccrepack.c diff --git a/doc/filters.texi b/doc/filters.texi index 5dde799..9f67a00 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -8936,6 +8936,16 @@ Only deinterlace frames marked as interlaced. The default value is @code{all}. @end table +@section ccrepack + +Repack CEA-708 closed captioning side data + +This filter fixes various issues seen with commerical encoders +related to upstream malformed CEA-708 payloads, specifically +incorrect number of tuples (wrong cc_count for the target FPS), +and incorrect ordering of tuples (i.e. the CEA-608 tuples are not at +the first entries in the payload). + @section cas Apply Contrast Adaptive Sharpen filter to video stream. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 628ade8..3d0b3ad 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -212,6 +212,7 @@ OBJS-$(CONFIG_BOXBLUR_OPENCL_FILTER) += vf_avgblur_opencl.o opencl.o \ opencl/avgblur.o boxblur.o OBJS-$(CONFIG_BWDIF_FILTER) += vf_bwdif.o yadif_common.o OBJS-$(CONFIG_CAS_FILTER) += vf_cas.o +OBJS-$(CONFIG_CCREPACK_FILTER) += vf_ccrepack.o OBJS-$(CONFIG_CHROMABER_VULKAN_FILTER) += vf_chromaber_vulkan.o vulkan.o vulkan_filter.o OBJS-$(CONFIG_CHROMAHOLD_FILTER) += vf_chromakey.o OBJS-$(CONFIG_CHROMAKEY_FILTER) += vf_chromakey.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index d7db46c..b38550b 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -196,6 +196,7 @@ extern const AVFilter ff_vf_boxblur; extern const AVFilter ff_vf_boxblur_opencl; extern const AVFilter ff_vf_bwdif; extern const AVFilter ff_vf_cas; +extern const AVFilter ff_vf_ccrepack; extern const AVFilter ff_vf_chromaber_vulkan; extern const AVFilter ff_vf_chromahold; extern const AVFilter ff_vf_chromakey; diff --git a/libavfilter/vf_ccrepack.c b/libavfilter/vf_ccrepack.c new file mode 100644 index 0000000..fb9120c --- /dev/null +++ b/libavfilter/vf_ccrepack.c @@ -0,0 +1,100 @@ +/* + * CEA-708 Closed Caption Repacker + * Copyright (c) 2023 LTN Global Communications + * + * Author: Devin Heitmueller + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/* + * Repackage CEA-708 arrays, which deals with incorrect cc_count for a given + * output framerate, and incorrect 708 padding. + * + * See CEA CEA-10-A "EIA-708-B Implementation Guidance", Section 26.5 + * "Grouping DTVCC Data Within user_data() Structure" + */ + +#include "avfilter.h" +#include "internal.h" +#include "ccfifo.h" +#include "libavutil/opt.h" + +typedef struct CCRepackContext +{ + const AVClass *class; + AVCCFifo *cc_fifo; +} CCRepackContext; + +static const AVOption ccrepack_options[] = { + { NULL } +}; + +AVFILTER_DEFINE_CLASS(ccrepack); + +static int config_input(AVFilterLink *link) +{ + CCRepackContext *ctx = link->dst->priv; + + if (!(ctx->cc_fifo = ff_ccfifo_alloc(&link->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue\n"); + + return 0; +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *frame) +{ + CCRepackContext *ctx = inlink->dst->priv; + AVFilterLink *outlink = inlink->dst->outputs[0]; + + ff_ccfifo_extract(ctx->cc_fifo, frame); + ff_ccfifo_inject(ctx->cc_fifo, frame); + + return ff_filter_frame(outlink, frame); +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + CCRepackContext *s = ctx->priv; + ff_ccfifo_freep(&s->cc_fifo); +} + +static const AVFilterPad avfilter_vf_ccrepack_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .filter_frame = filter_frame, + .config_props = config_input, + }, +}; + +static const AVFilterPad avfilter_vf_ccrepack_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + }, +}; + +AVFilter ff_vf_ccrepack = { + .name = "ccrepack", + .description = NULL_IF_CONFIG_SMALL("Repack CEA-708 closed caption metadata"), + .uninit = uninit, + .priv_size = sizeof(CCRepackContext), + .priv_class = &ccrepack_class, + FILTER_INPUTS(avfilter_vf_ccrepack_inputs), + FILTER_OUTPUTS(avfilter_vf_ccrepack_outputs), +}; From patchwork Fri Apr 28 16:37:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 41406 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:dca6:b0:f3:34fa:f187 with SMTP id ky38csp1439703pzb; Fri, 28 Apr 2023 08:43:24 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5dC0Dy9Iz9BZ+YTwAPQqMfmFR5wM1KUn8glcJevhWgEjGLwGQl9L2rm8iFbSWMjBwL8G0q X-Received: by 2002:a05:6402:104f:b0:502:233e:af49 with SMTP id e15-20020a056402104f00b00502233eaf49mr5689563edu.4.1682696603883; Fri, 28 Apr 2023 08:43:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682696603; cv=none; d=google.com; s=arc-20160816; b=TTg4UV/gSVgG665ETZxL7JErXOycRhK9TJMdkgV4KY8IlgcKUH/kYDk+2PZsYqFyNJ 9GZtvYGJ3uKlTgBAtE26pxtAXpo5o2BEYK9T1cN+zfMZFKLCS2ho3iRgUi61iUNe6Qw5 s1ojPgyI2wnpmOYIExVe8MOjoZ9s82Imj/1zghbTUvxwNndScY7eC5rOkVihTlSZ1n/6 b8aNFJTqaog5JoNdGSLK+GSAIpEIiA0oaG6m3M4MZ8Y3JWpmaRVvOc+Iy6IeNgod9iW9 OGcT8yCwo2K0q/9N3fRDKx0ZYIu92WGNHhuEh079iCWYnp+f2BYxkClf5CNAzI9Mu/Ns YLDQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=5aiRIs6qzegI3WmCoWmA+9ZkyJN9CDLrgSjbk/mCPaU=; b=hp9jhjPVrfl5ogiBgfhdnxwVrm1/lP+23ThA1DYd0qCGN/vuQzan+y2ns4TOAAY2xN F6IGEwXuXm7EN0RevtL0YuWZdCKdj41NLv3eD/nkJ9KxhmCOpDu6EaKWUKh+/S7u6cBR dOTyCUVJ5+h+VNslgwXdw9GMK3si5WdjzhVhkcNNnin2+MfWVAsIGhKFZC/wM9TYb+PZ 8aKZxuSFsfTp3p+1kGlZVQ7msSKQJHdrWetVkE/yGnuVvNPjWXFFcEvhaV7a2l6QSp3i rLu8LyyVMs/AfmQJbQXIeexzcHNFEt/gTzuP9zLy03cGcJhLiqR53AwS+iOCkimLQcNY m4+g== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=RcY2NkBs; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id i26-20020aa7c71a000000b00506883845c3si11808087edq.623.2023.04.28.08.43.23; Fri, 28 Apr 2023 08:43:23 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=RcY2NkBs; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2D6CF68C041; Fri, 28 Apr 2023 18:42:25 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qt1-f175.google.com (mail-qt1-f175.google.com [209.85.160.175]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 692D368B36D for ; Fri, 28 Apr 2023 18:42:15 +0300 (EEST) Received: by mail-qt1-f175.google.com with SMTP id d75a77b69052e-3ef38bea86aso49245961cf.3 for ; Fri, 28 Apr 2023 08:42:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20221208.gappssmtp.com; s=20221208; t=1682696534; x=1685288534; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=n+PsPSiO/Acwk3mWvbSUS4VIIm7twte22bpMeYfIdRo=; b=RcY2NkBsPLNyVqz6vxzV9Gb2qvFuYGHSPpUv1DB6Sk/c2NdQ2rBSjJr7GYSAmyy77L Lm0yiyNor4qZ7OPuO/SouYAA3wQkDmtqM8TO9BIVoY1SbQ34UX7rdmDAzuI92T6yKKmq TNZJdlhmNnI0ekZLPLjc0HGOXfytL3JXeDN46zmdlY8z7KjFmqfXFUTBi68k3poLeOTT zCt2Q7I9h/hva9PPT9pxjYLCYIILL2HzhAe5XOGNJRY0+EX2K4oICV0Pk3zO8fFiR7CA mEwWLlTAtgTLQPFdLCIVd1gBuuAlLHJo0a0EHZUxk29shJ1M2Ol1WcZALRL1N5LZ19Ko R/aA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682696534; x=1685288534; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=n+PsPSiO/Acwk3mWvbSUS4VIIm7twte22bpMeYfIdRo=; b=HJ9cFwynqWxEYYQtZOsuXBkG9ZimlWGayuSCLMiz07eAKt2uQScciivuo42BEWW32S xuE5vy+pq0naLOXtP1vN+jkUvYpCUzmupt9gnhlTizi44+L6ZxuREwTEw4k6O1HMU0cA fTHOycMfjZqN6SW2ZseWpqAybH0msvWMYSpmdLySkZt6JQYa6STM9s9X1zwZw3l9l5jN ZQKipg1tauK0o0awOvezsy+lZAnbZn0E4ONVEAGBiB2r49LNtavzcOUWLcBwjNHy2n19 AC0uL04lMmF0t6pbGtDWvGS1QfBy1Ix8UF1trqs4bXEHZA625jaC6YRtsrDuBBF1h/nF zopQ== X-Gm-Message-State: AC+VfDwF9Ts4m/CBWTV0u6MKIfYocCtdKs913chitO23/qKIXuBhMUwK dg+KM5JRGkiJxzwrYTY3RHIKSc9zt0cZQXv/22o= X-Received: by 2002:a05:622a:f:b0:3e3:937b:2a64 with SMTP id x15-20020a05622a000f00b003e3937b2a64mr10348710qtw.18.1682696533931; Fri, 28 Apr 2023 08:42:13 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id do54-20020a05620a2b3600b0074e13ed6ee9sm6733873qkb.132.2023.04.28.08.42.13 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 28 Apr 2023 08:42:13 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 28 Apr 2023 12:37:51 -0400 Message-Id: <1682699871-22331-7-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1682699871-22331-1-git-send-email-dheitmueller@ltnglobal.com> References: <1682699871-22331-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [PATCH v4 6/6] decklink_enc: add support for playout of 608 captions in MOV files X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: +AHSi/ZNOOha Unlike other cases where the closed captions are embedded in the video stream as MPEG-2 userdata or H.264 SEI data, with MOV files the captions are often found on a separate "e608" subtitle track. Add support for playout of such files, leveraging the new ccfifo mechanism to ensure that they are embedded into VANC at the correct rate (since e608 packets often contain batches of multiple 608 pairs). Note this patch includes a new file named libavdevice/ccfifo.c, which allows the ccfifo functionality in libavfilter to be reused even if doing shared builds. This is the same approach used for log2_tab.c. Signed-off-by: Devin Heitmueller Signed-off-by: Devin Heitmueller --- libavdevice/Makefile | 1 + libavdevice/ccfifo.c | 24 ++++++++++++++++ libavdevice/decklink_common.h | 3 ++ libavdevice/decklink_enc.cpp | 66 +++++++++++++++++++++++++++++++++++++++++++ libavdevice/decklink_enc_c.c | 2 +- 5 files changed, 95 insertions(+), 1 deletion(-) create mode 100644 libavdevice/ccfifo.c diff --git a/libavdevice/Makefile b/libavdevice/Makefile index 8a62822..c304492 100644 --- a/libavdevice/Makefile +++ b/libavdevice/Makefile @@ -57,6 +57,7 @@ OBJS-$(CONFIG_LIBDC1394_INDEV) += libdc1394.o # Objects duplicated from other libraries for shared builds SHLIBOBJS-$(CONFIG_DECKLINK_INDEV) += reverse.o +SHLIBOBJS-$(CONFIG_DECKLINK_OUTDEV) += ccfifo.o # Windows resource file SHLIBOBJS-$(HAVE_GNU_WINDRES) += avdeviceres.o diff --git a/libavdevice/ccfifo.c b/libavdevice/ccfifo.c new file mode 100644 index 0000000..9007094 --- /dev/null +++ b/libavdevice/ccfifo.c @@ -0,0 +1,24 @@ +/* + * CEA-708 Closed Captioning FIFO + * Copyright (c) 2023 LTN Global Communications + * + * Author: Devin Heitmueller + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavfilter/ccfifo.c" diff --git a/libavdevice/decklink_common.h b/libavdevice/decklink_common.h index 088e165..0d33f94 100644 --- a/libavdevice/decklink_common.h +++ b/libavdevice/decklink_common.h @@ -31,6 +31,7 @@ extern "C" { #include "libavcodec/packet_internal.h" +#include "libavfilter/ccfifo.h" } #include "libavutil/thread.h" #include "decklink_common_c.h" @@ -112,6 +113,8 @@ struct decklink_ctx { /* Capture buffer queue */ AVPacketQueue queue; + AVCCFifo *cc_fifo; ///< closed captions + /* Streams present */ int audio; int video; diff --git a/libavdevice/decklink_enc.cpp b/libavdevice/decklink_enc.cpp index 9f1a8df..616e9c7 100644 --- a/libavdevice/decklink_enc.cpp +++ b/libavdevice/decklink_enc.cpp @@ -326,6 +326,25 @@ static int create_s337_payload(AVPacket *pkt, uint8_t **outbuf, int *outsize) return 0; } +static int decklink_setup_subtitle(AVFormatContext *avctx, AVStream *st) +{ + int ret = -1; + + switch(st->codecpar->codec_id) { +#if CONFIG_LIBKLVANC + case AV_CODEC_ID_EIA_608: + /* No special setup required */ + ret = 0; + break; +#endif + default: + av_log(avctx, AV_LOG_ERROR, "Unsupported subtitle codec specified\n"); + break; + } + + return ret; +} + av_cold int ff_decklink_write_trailer(AVFormatContext *avctx) { struct decklink_cctx *cctx = (struct decklink_cctx *)avctx->priv_data; @@ -352,6 +371,7 @@ av_cold int ff_decklink_write_trailer(AVFormatContext *avctx) klvanc_context_destroy(ctx->vanc_ctx); #endif + ff_ccfifo_freep(&ctx->cc_fifo); av_freep(&cctx->ctx); return 0; @@ -503,6 +523,23 @@ out: free(afd_words); } +/* Parse any EIA-608 subtitles sitting on the queue, and write packet side data + that will later be handled by construct_cc... */ +static void parse_608subs(AVFormatContext *avctx, struct decklink_ctx *ctx, AVPacket *pkt) +{ + uint8_t *cc_data; + size_t cc_size; + int ret; + + ret = ff_ccfifo_injectbytes(ctx->cc_fifo, &cc_data, &cc_size); + if (ret == 0) { + uint8_t *cc_buf = av_packet_new_side_data(pkt, AV_PKT_DATA_A53_CC, cc_size); + if (cc_buf) + memcpy(cc_buf, cc_data, cc_size); + av_freep(&cc_data); + } +} + static int decklink_construct_vanc(AVFormatContext *avctx, struct decklink_ctx *ctx, AVPacket *pkt, decklink_frame *frame, AVStream *st) @@ -513,6 +550,7 @@ static int decklink_construct_vanc(AVFormatContext *avctx, struct decklink_ctx * if (!ctx->supports_vanc) return 0; + parse_608subs(avctx, ctx, pkt); construct_cc(avctx, ctx, pkt, &vanc_lines); construct_afd(avctx, ctx, pkt, &vanc_lines, st); @@ -704,12 +742,23 @@ static int decklink_write_audio_packet(AVFormatContext *avctx, AVPacket *pkt) return ret; } +static int decklink_write_subtitle_packet(AVFormatContext *avctx, AVPacket *pkt) +{ + struct decklink_cctx *cctx = (struct decklink_cctx *)avctx->priv_data; + struct decklink_ctx *ctx = (struct decklink_ctx *)cctx->ctx; + + ff_ccfifo_extractbytes(ctx->cc_fifo, pkt->data, pkt->size); + + return 0; +} + extern "C" { av_cold int ff_decklink_write_header(AVFormatContext *avctx) { struct decklink_cctx *cctx = (struct decklink_cctx *)avctx->priv_data; struct decklink_ctx *ctx; + AVRational frame_rate; unsigned int n; int ret; @@ -768,12 +817,27 @@ av_cold int ff_decklink_write_header(AVFormatContext *avctx) } else if (c->codec_type == AVMEDIA_TYPE_VIDEO) { if (decklink_setup_video(avctx, st)) goto error; + } else if (c->codec_type == AVMEDIA_TYPE_SUBTITLE) { + if (decklink_setup_subtitle(avctx, st)) + goto error; } else { av_log(avctx, AV_LOG_ERROR, "Unsupported stream type.\n"); goto error; } } + for (n = 0; n < avctx->nb_streams; n++) { + AVStream *st = avctx->streams[n]; + AVCodecParameters *c = st->codecpar; + + if(c->codec_type == AVMEDIA_TYPE_SUBTITLE) + avpriv_set_pts_info(st, 64, ctx->bmd_tb_num, ctx->bmd_tb_den); + } + + frame_rate = av_make_q(ctx->bmd_tb_den, ctx->bmd_tb_num); + if (!(ctx->cc_fifo = ff_ccfifo_alloc(&frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue\n"); + return 0; error: @@ -789,6 +853,8 @@ int ff_decklink_write_packet(AVFormatContext *avctx, AVPacket *pkt) return decklink_write_video_packet(avctx, pkt); else if (st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO) return decklink_write_audio_packet(avctx, pkt); + else if (st->codecpar->codec_type == AVMEDIA_TYPE_SUBTITLE) + return decklink_write_subtitle_packet(avctx, pkt); return AVERROR(EIO); } diff --git a/libavdevice/decklink_enc_c.c b/libavdevice/decklink_enc_c.c index f7e3150..0a3984b 100644 --- a/libavdevice/decklink_enc_c.c +++ b/libavdevice/decklink_enc_c.c @@ -77,7 +77,7 @@ const FFOutputFormat ff_decklink_muxer = { .p.long_name = NULL_IF_CONFIG_SMALL("Blackmagic DeckLink output"), .p.audio_codec = AV_CODEC_ID_PCM_S16LE, .p.video_codec = AV_CODEC_ID_WRAPPED_AVFRAME, - .p.subtitle_codec = AV_CODEC_ID_NONE, + .p.subtitle_codec = AV_CODEC_ID_EIA_608, .p.flags = AVFMT_NOFILE, .p.priv_class = &decklink_muxer_class, .get_device_list = ff_decklink_list_output_devices,