From patchwork Fri Apr 7 21:58:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 41018 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4645:b0:e3:3194:9d20 with SMTP id eb5csp627564pzb; Fri, 7 Apr 2023 14:03:24 -0700 (PDT) X-Google-Smtp-Source: AKy350Ypg99Ei22ZF7iOGOM5iEFmEKTN8IvNbRq0CLrbN6UDQhFthU2MroZYLr4XzOwCjMmcXPnF X-Received: by 2002:a50:ee09:0:b0:4fc:61a5:b9c8 with SMTP id g9-20020a50ee09000000b004fc61a5b9c8mr3579855eds.11.1680901403752; Fri, 07 Apr 2023 14:03:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1680901403; cv=none; d=google.com; s=arc-20160816; b=nOv/JAWcpvo4AjiD87PlYULw06YXyvTFEMX9OzSliZbJs7z/cVX+rAgyKH6Idg8Iqg Oe2EIe09LwB03JsdtyfqE4fZkqhF4un5yPmDjLkVF8PsxDNlgTM2wg0MwwyeTEjJohvj Sz8mlN4sG5Yx66gvjKjI/8VkN1qMmFmiVgcMS30onN2SkVdqySvV1ezT9CAfefPNHwPa RYPfFBuHwufzaTbu29qOpm8BHGtWD9jroTn3TX7E64lXwQeu5M6nCz2gv+jlBWFsz0OF XzXIE1IVUYYVsXkOFaVXzLp4skbFY64ys8tc25kXpb9f3c3svgsrbUkY+w55H6gjSzlR OLjQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=dcvx6eO9jL21q5fQTjvDaEowzk03GFU0T6n6Nfh8qCA=; b=sXY35GHynXL3eYwsydGHdCazHbLspiZmN41QIs74jbkJZESs1ll3Jj4DENBf3Z9w9g btpqUttW8L9H7YpQ3ZR/AwM3WC1jrI0UMspTDUo3PrD3U9uuLiLKdgMDH/tLUv5uoQuI JfKy9n2cHzX2HiHGiBwR9dgk7R+H18Uty0JTi7vtB37CamWLpbiUmZY9FeCrd/dnn1h2 ejpUQo//0iFlryWr8zfQH/Jcg2pFTU5hDleSP+/cnExHFawD6n4L6GKnEAtYHbcjyXVC KFKfDBknIhBrv6RK/4XhpJ8hKdmr/z+ObihIK6eN9d1RezqcKPvScsKlXrr97X60kPPt bwhg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b="75j9K/p3"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id x3-20020aa7d6c3000000b005026e488c81si3912252edr.520.2023.04.07.14.03.23; Fri, 07 Apr 2023 14:03:23 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b="75j9K/p3"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0BC4C68BA80; Sat, 8 Apr 2023 00:03:12 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qv1-f46.google.com (mail-qv1-f46.google.com [209.85.219.46]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 03150689D6E for ; Sat, 8 Apr 2023 00:03:04 +0300 (EEST) Received: by mail-qv1-f46.google.com with SMTP id ld14so1890101qvb.13 for ; Fri, 07 Apr 2023 14:03:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20210112.gappssmtp.com; s=20210112; t=1680901382; x=1683493382; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=3mYy3pXPNwQYo5kXs1m9XoTJCX42kBUun+8X6hucmvU=; b=75j9K/p3gYBQoPymxQ7HEOsoeWlA8NCtG5ENXCmLcZi7YA/p6U9PxMMFsWB0klcc0r Y6QZu3HxRLMu3M7uZmVMizU2eAcAoPX8QfaKYTXSVMujWxc+994nWDL34wvREeuOTZtz okbQPIP5D1DlzUOGf+iJo+KiN/Ag56gOACAZph1gtxvtaeMR05z9ips6Y69+5KxSNPyk ATWP2gAmcR3bWDDLxokAz06A2K/fcvuGJOh0z1+G4hUt0HKFP7H1Qcw2jL7WPAt3z5LN Q4RWiXYYz5CHdsG7Aejtzowm5Nk2n7eMkiON0rb5zyqu9OObWhD0xhWIiSxWnmF9vnqP 9Wyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680901382; x=1683493382; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=3mYy3pXPNwQYo5kXs1m9XoTJCX42kBUun+8X6hucmvU=; b=R0vYJeJsusxsnH0GvFybMlEcLefHnd6zm06+luWzwzwlLRNmOKixG9nqRHDuNNTL+f xLEbwvjMJJIuU+zucKz+ZrwuvDdFr6NrCgPJqrMv0imTr6zHUik2MtPqBME4UEpVlhsY nl3DmN4mcyydlKJh0+E4Amdwkel4l6BIGlNBEE+AGyJSqc32914N/JfVuqKuH3Rvg9/R uUIKyEzAp47M5vc9Gxe25ilHei3l+Zf8nG+FHi2nyVeUeqAkNhTzXy8dh7BR5CfymGDf 0kak9JuUAuT/ObwxTZNghXWK1mFilQ7AmgM+l2bnMDwlXrbCDcRVcsfB7sJfXtNVRcI3 0iBQ== X-Gm-Message-State: AAQBX9deAs/9/gNqac9jTqG2rBHzfR0r5KeQzLpF2jhtub6D36vUTiOG 450BErc0gYMWzqWXeHrCEQQukx/wc8l08yQBQbw= X-Received: by 2002:ad4:5962:0:b0:56b:7ec7:b158 with SMTP id eq2-20020ad45962000000b0056b7ec7b158mr5231961qvb.36.1680901381976; Fri, 07 Apr 2023 14:03:01 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id z13-20020ad4414d000000b005dd8b9345d8sm1405143qvp.112.2023.04.07.14.03.01 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 07 Apr 2023 14:03:01 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 7 Apr 2023 17:58:25 -0400 Message-Id: <1680904709-25951-2-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1680904709-25951-1-git-send-email-dheitmueller@ltnglobal.com> References: <1680904709-25951-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [PATCH v2 1/5] ccfifo: Properly handle CEA-708 captions through framerate conversion X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: uj5aWJqzwSN4 When transcoding video that contains 708 closed captions, the caption data is tied to the frames as side data. Simply dropping or adding frames to change the framerate will result in loss of data, so the caption data needs to be preserved and reformatted. For example, without this patch converting 720p59 to 1080i59 would result in loss of 50% of the caption bytes, resulting in garbled 608 captions and 708 probably wouldn't render at all. Further, the frames that are there will have an illegal cc_count for the target framerate, so some decoders may ignore the packets entirely. Extract the 608 and 708 tuples and insert them onto queues. Then after dropping/adding frames, re-write the tuples back into the resulting frames at the appropriate rate given the target framerate. This includes both having the correct cc_count as well as clocking out the 608 pairs at the appropriate rate. Signed-off-by: Devin Heitmueller --- libavfilter/Makefile | 1 + libavfilter/ccfifo.c | 191 +++++++++++++++++++++++++++++++++++++++++++++++++++ libavfilter/ccfifo.h | 85 +++++++++++++++++++++++ 3 files changed, 277 insertions(+) create mode 100644 libavfilter/ccfifo.c create mode 100644 libavfilter/ccfifo.h diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 71e198b..628ade8 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -14,6 +14,7 @@ OBJS = allfilters.o \ buffersink.o \ buffersrc.o \ colorspace.o \ + ccfifo.o \ drawutils.o \ fifo.o \ formats.o \ diff --git a/libavfilter/ccfifo.c b/libavfilter/ccfifo.c new file mode 100644 index 0000000..5db4149 --- /dev/null +++ b/libavfilter/ccfifo.c @@ -0,0 +1,191 @@ +/* + * CEA-708 Closed Captioning FIFO + * Copyright (c) 2023 LTN Global Communications + * + * Author: Devin Heitmueller + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "ccfifo.h" + +struct AVCCFifo { + AVFifo *cc_608_fifo; + AVFifo *cc_708_fifo; + int expected_cc_count; + int expected_608; + int cc_detected; + void *log_ctx; +}; + +#define MAX_CC_ELEMENTS 128 +#define CC_BYTES_PER_ENTRY 3 + +struct cc_lookup { + int num; + int den; + int cc_count; + int num_608; +}; + +const static struct cc_lookup cc_lookup_vals[] = { + { 15, 1, 40, 4 }, + { 24, 1, 25, 3 }, + { 24000, 1001, 25, 3 }, + { 30, 1, 20, 2 }, + { 30000, 1001, 20, 2}, + { 60, 1, 10, 1 }, + { 60000, 1001, 10, 1}, +}; + +void av_ccfifo_freep(AVCCFifo **ccf) +{ + if (ccf && *ccf) { + AVCCFifo *tmp = *ccf; + if (tmp->cc_608_fifo) + av_fifo_freep2(&tmp->cc_608_fifo); + if (tmp->cc_708_fifo) + av_fifo_freep2(&tmp->cc_708_fifo); + av_freep(*ccf); + } +} + +AVCCFifo *av_ccfifo_alloc(AVRational *framerate, void *log_ctx) +{ + AVCCFifo *ccf; + int i; + + ccf = av_mallocz(sizeof(*ccf)); + if (!ccf) + return NULL; + + if (!(ccf->cc_708_fifo = av_fifo_alloc2(MAX_CC_ELEMENTS, CC_BYTES_PER_ENTRY, 0))) + goto error; + + if (!(ccf->cc_608_fifo = av_fifo_alloc2(MAX_CC_ELEMENTS, CC_BYTES_PER_ENTRY, 0))) + goto error; + + /* Based on the target FPS, figure out the expected cc_count and number of + 608 tuples per packet. See ANSI/CTA-708-E Sec 4.3.6.1. */ + for (i = 0; i < (sizeof(cc_lookup_vals) / sizeof(struct cc_lookup)); i++) { + if (framerate->num == cc_lookup_vals[i].num && + framerate->den == cc_lookup_vals[i].den) { + ccf->expected_cc_count = cc_lookup_vals[i].cc_count; + ccf->expected_608 = cc_lookup_vals[i].num_608; + break; + } + } + + if (ccf->expected_608 == 0) { + av_log(ccf->log_ctx, AV_LOG_WARNING, "cc_fifo cannot transcode captions fps=%d/%d\n", + framerate->num, framerate->den); + return NULL; + } + + return ccf; + +error: + av_ccfifo_freep(&ccf); + return NULL; +} + +int av_ccfifo_inject(AVCCFifo *ccf, AVFrame *frame) +{ + AVFrameSideData *sd; + int cc_filled = 0; + int i; + + if (!ccf) + return 0; + + if (ccf->cc_detected == 0 || ccf->expected_cc_count == 0) + return 0; + + sd = av_frame_new_side_data(frame, AV_FRAME_DATA_A53_CC, + ccf->expected_cc_count * CC_BYTES_PER_ENTRY); + if (!sd) + return 0; + + for (i = 0; i < ccf->expected_608; i++) { + if (av_fifo_can_read(ccf->cc_608_fifo) >= CC_BYTES_PER_ENTRY) { + av_fifo_read(ccf->cc_608_fifo, &sd->data[cc_filled * CC_BYTES_PER_ENTRY], + CC_BYTES_PER_ENTRY); + cc_filled++; + } else { + break; + } + } + + /* Insert any available data from the 708 FIFO */ + while (cc_filled < ccf->expected_cc_count) { + if (av_fifo_can_read(ccf->cc_708_fifo) >= CC_BYTES_PER_ENTRY) { + av_fifo_read(ccf->cc_708_fifo, &sd->data[cc_filled * CC_BYTES_PER_ENTRY], + CC_BYTES_PER_ENTRY); + cc_filled++; + } else { + break; + } + } + + /* Insert 708 padding into any remaining fields */ + while (cc_filled < ccf->expected_cc_count) { + sd->data[cc_filled * CC_BYTES_PER_ENTRY] = 0xfa; + sd->data[cc_filled * CC_BYTES_PER_ENTRY + 1] = 0x00; + sd->data[cc_filled * CC_BYTES_PER_ENTRY + 2] = 0x00; + cc_filled++; + } + + return 0; +} + +int av_ccfifo_extract(AVCCFifo *ccf, AVFrame *frame) +{ + int i; + + if (!ccf) + return 0; + + /* Read the A53 side data, discard padding, and put 608/708 into + queues so we can ensure they get into the output frames at + the correct rate... */ + if (ccf->expected_cc_count > 0) { + AVFrameSideData *side_data = av_frame_get_side_data(frame, AV_FRAME_DATA_A53_CC); + if (side_data) { + uint8_t *cc_bytes = side_data->data; + int cc_count = side_data->size / CC_BYTES_PER_ENTRY; + ccf->cc_detected = 1; + + for (i = 0; i < cc_count; i++) { + /* See ANSI/CTA-708-E Sec 4.3, Table 3 */ + uint8_t cc_valid = (cc_bytes[CC_BYTES_PER_ENTRY*i] & 0x04) >> 2; + uint8_t cc_type = cc_bytes[CC_BYTES_PER_ENTRY*i] & 0x03; + if (cc_type == 0x00 || cc_type == 0x01) { + av_fifo_write(ccf->cc_608_fifo, &cc_bytes[CC_BYTES_PER_ENTRY*i], + CC_BYTES_PER_ENTRY); + } else if (cc_valid && (cc_type == 0x02 || cc_type == 0x03)) { + av_fifo_write(ccf->cc_708_fifo, &cc_bytes[CC_BYTES_PER_ENTRY*i], + CC_BYTES_PER_ENTRY); + } + } + + /* Remove the side data, as we will re-create it on the + output as needed */ + av_frame_remove_side_data(frame, AV_FRAME_DATA_A53_CC); + } + } + return 0; +} diff --git a/libavfilter/ccfifo.h b/libavfilter/ccfifo.h new file mode 100644 index 0000000..66403be --- /dev/null +++ b/libavfilter/ccfifo.h @@ -0,0 +1,85 @@ +/* + * CEA-708 Closed Captioning FIFO + * Copyright (c) 2023 LTN Global Communications + * + * Author: Devin Heitmueller + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * CC FIFO Buffer + */ + +#ifndef AVUTIL_CCFIFO_H +#define AVUTIL_CCFIFO_H + +#include "libavutil/avutil.h" +#include "libavutil/frame.h" +#include "libavutil/fifo.h" + +typedef struct AVCCFifo AVCCFifo; + +/** + * Allocate an AVCCFifo. + * + * @param sample_fmt sample format + * @param channels number of channels + * @param nb_samples initial allocation size, in samples + * @return newly allocated AVCCFifo, or NULL on error + */ +AVCCFifo *av_ccfifo_alloc(AVRational *framerate, void *log_ctx); + +/** + * Free an AVCCFifo + * + * @param ccf Pointer to the pointer to the AVCCFifo which should be freed + * @note `*ptr = NULL` is safe and leads to no action. + */ +void av_ccfifo_freep(AVCCFifo **ccf); + + +/** + * Read a frame into a CC Fifo + * + * Extract CC bytes from the AVFrame, insert them into our queue, and + * remove the side data from the AVFrame. The side data is removed + * as it will be re-inserted at the appropriate rate later in the + * filter. + * + * @param af AVCCFifo to write to + * @param frame AVFrame with the video frame to operate on + * @return Zero on success, or negative AVERROR + * code on failure. + */ +int av_ccfifo_extract(AVCCFifo *af, AVFrame *frame); + +/** + * Insert CC data from the FIFO into an AVFrame (as side data) + * + * Dequeue the appropriate number of CC tuples based on the + * frame rate, and insert them into the AVFrame + * + * @param af AVCCFifo to read from + * @param frame AVFrame with the video frame to operate on + * @return Zero on success, or negative AVERROR + * code on failure. + */ +int av_ccfifo_inject(AVCCFifo *af, AVFrame *frame); + +#endif /* AVUTIL_CCFIFO_H */ From patchwork Fri Apr 7 21:58:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 41019 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4645:b0:e3:3194:9d20 with SMTP id eb5csp627635pzb; Fri, 7 Apr 2023 14:03:33 -0700 (PDT) X-Google-Smtp-Source: AKy350bGe7yu/jjh6YEbKVO5r5Dtj9h67R2ccFDUHPjxX36AupLFgQJqv771+aNPnukNr0yEsn/4 X-Received: by 2002:a17:906:58d4:b0:93e:6f40:d141 with SMTP id e20-20020a17090658d400b0093e6f40d141mr926078ejs.40.1680901412971; Fri, 07 Apr 2023 14:03:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1680901412; cv=none; d=google.com; s=arc-20160816; b=VPf81+OOjsofWnxwN95eUFkUDbzC8toXEF1ZeabslHbQt219r1sHTK+StmCERectVe 4l99b81+4w9E4SvHZo6omXAQgIOv0aBoc6HBw7yu8MaN5EID3HEd6FQS2PRevf/H68Sy CQBT0tlOgi3h5SoV/UP3bClcn+DSQb3vqwRGUZEV2JFc2H84oAXNd/T2svski18d7MpH PfR4OuzMPKKY04eYjdN1FZA2WJEhyxxloYFIY6bFV8r0dGrLYydB9WJ4/RxDUD6sR1EO KYx6HTvCm/zn+7gtGAiOgIgVEiIjZ/brFR9caJ5/xJ3KeDMX2KHFXxXTZG8RgESZQ5Vd noKg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=SvPLUPcTKt6EqD8wDZKUsiX6ezuEXBo10ZcT1+v7U8s=; b=0MN/6gfVFRtfJ7JFBb9Wz9WHG3oTA6wTTfT+IqLKPGiiOC5IG6mMIPpa17Lm4Gn118 v+rVbM70KDA/fJ6CA9/HmnVNEF7O0msubL6Z1QVPOin/AEkWrHkU8ef/T7ybK9kPkDHY 0QjQrE9PqYPrijAllD6Lko3JpayTHm2ZtD7yVuUe2z1Hm1tG1cRs7wZDb1oz6QXc3Qi1 zFABMZorFUlRj2UZEN6PzkYwEqZaDSi6vbKJrSPMdZpW4jbZQFDL7ZPjBRJ4Y1IuWZuG EIRBaqI/DPii+EnK+vlqfLRqQ18eCCx+OeXa0vGMboRoRDT9nLMkzHiqtnLtwCDT7pOC +QfA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=tDyxlCYm; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id gg11-20020a170906e28b00b00947eadd521fsi3574852ejb.333.2023.04.07.14.03.32; Fri, 07 Apr 2023 14:03:32 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=tDyxlCYm; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1517F68B815; Sat, 8 Apr 2023 00:03:13 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qv1-f46.google.com (mail-qv1-f46.google.com [209.85.219.46]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5E57A689D6E for ; Sat, 8 Apr 2023 00:03:04 +0300 (EEST) Received: by mail-qv1-f46.google.com with SMTP id l1so3204265qvv.4 for ; Fri, 07 Apr 2023 14:03:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20210112.gappssmtp.com; s=20210112; t=1680901383; x=1683493383; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=y3zUV4HoAAcSZDj0fY9VeI4retV5G+49wX4qL3f2y4w=; b=tDyxlCYmUm+CugSMujgozLV0M7iMLoDeK5aLScDf7bGLIcoqlc65m718aFMnxjS5TL 9Eus6+jDOv0Kuozsm2E/LLCmi91FgQ9n6Dz1L/fDh7i+HsqWsdaJsOjayojmSj2NVHOb KBKEhl9La6GzRfExwtfBB3LS3Z45UfM04164+klszWBKafduk4Rr8wtTbMMqFD/OJubj 1nvGKAfhrI1KzgxYORHMegmzN2Uc3Xaw7CKd+bbt1pwTxJGnJthgQc0GI/118IjlpPaG vZ/bas/gz+bTnBcs0OJG1mJW0IOGZQ/QCa1pLtlQp9+4rJcLQTR+0sSIOwUIadg3ky/x 9kug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680901383; x=1683493383; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=y3zUV4HoAAcSZDj0fY9VeI4retV5G+49wX4qL3f2y4w=; b=66MSNiV+ixQkwKcgkA3FoQxFhdGiHkOhdCqhWhvppTynWQZzmP+Mx7SA5O/p5n8hVK jzIXkEfx8pXQFrf+wYZOBcatv6QXglbBXp8eBRA4O3C5s9RhZGichGxrIn7LN0ycBA62 D3Yx7z+VZoTktIew7SB+3B0LkO5gAuqU4Hh7Z5LuPNJraxbAIodG0Q01On0Ud9MITGLy AsyYB3bsf2RQdpwX5NxNo0NyMqOpHnCgmkm1Tqr28rHp6Zzd0DauZNOUKcNtEk7llK/Z WUTgwCFSKf7W/45NAwopDES5x+uVShnJ1TNSr7+Gh3BZpY1ZskVF9JL9cI+68nnQnHy4 CYtg== X-Gm-Message-State: AAQBX9eGZhnj+aAZrseGY7y2sIQyRjgckNtI+zwu8JDTWgyXQMxXw55g UFs5tX/iiOD/eTkz1ERNuzTxaNmZm9p99OyqES8= X-Received: by 2002:a05:6214:c4d:b0:5c1:b700:2caa with SMTP id r13-20020a0562140c4d00b005c1b7002caamr6419682qvj.26.1680901383006; Fri, 07 Apr 2023 14:03:03 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id z13-20020ad4414d000000b005dd8b9345d8sm1405143qvp.112.2023.04.07.14.03.02 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 07 Apr 2023 14:03:02 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 7 Apr 2023 17:58:26 -0400 Message-Id: <1680904709-25951-3-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1680904709-25951-1-git-send-email-dheitmueller@ltnglobal.com> References: <1680904709-25951-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [PATCH v2 2/5] vf_fps: properly preserve CEA-708 captions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: nyEgUR8KkC5D The existing implementation made an attempt to remove duplicate captions if increasing the framerate, but made no attempt to handle reducing the framerate, nor did it rewrite the caption payloads to have the appropriate cc_count (e.g. the cc_count needs to change from 20 to 10 when going from 1080i59 to 720p59 and vice-versa). Make use of the new ccfifo mechanism to ensure that caption data is properly preserved. Signed-off-by: Devin Heitmueller --- libavfilter/vf_fps.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/libavfilter/vf_fps.c b/libavfilter/vf_fps.c index 051d278..3f18bfc 100644 --- a/libavfilter/vf_fps.c +++ b/libavfilter/vf_fps.c @@ -34,6 +34,7 @@ #include "libavutil/mathematics.h" #include "libavutil/opt.h" #include "avfilter.h" +#include "ccfifo.h" #include "filters.h" #include "internal.h" @@ -85,6 +86,7 @@ typedef struct FPSContext { AVFrame *frames[2]; ///< buffered frames int frames_count; ///< number of buffered frames + AVCCFifo *cc_fifo; ///< closed captions int64_t next_pts; ///< pts of the next frame to output @@ -165,6 +167,7 @@ static av_cold void uninit(AVFilterContext *ctx) frame = shift_frame(ctx, s); av_frame_free(&frame); } + av_ccfifo_freep(&s->cc_fifo); av_log(ctx, AV_LOG_VERBOSE, "%d frames in, %d frames out; %d frames dropped, " "%d frames duplicated.\n", s->frames_in, s->frames_out, s->drop, s->dup); @@ -210,6 +213,9 @@ static int config_props(AVFilterLink* outlink) s->in_pts_off, s->out_pts_off, s->start_time); } + if (!(s->cc_fifo = av_ccfifo_alloc(&outlink->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue. Captions will be passed through\n"); + av_log(ctx, AV_LOG_VERBOSE, "fps=%d/%d\n", outlink->frame_rate.num, outlink->frame_rate.den); return 0; @@ -242,6 +248,7 @@ static int read_frame(AVFilterContext *ctx, FPSContext *s, AVFilterLink *inlink, av_log(ctx, AV_LOG_DEBUG, "Read frame with in pts %"PRId64", out pts %"PRId64"\n", in_pts, frame->pts); + av_ccfifo_extract(s->cc_fifo, frame); s->frames[s->frames_count++] = frame; s->frames_in++; @@ -289,7 +296,7 @@ static int write_frame(AVFilterContext *ctx, FPSContext *s, AVFilterLink *outlin if (!frame) return AVERROR(ENOMEM); // Make sure Closed Captions will not be duplicated - av_frame_remove_side_data(s->frames[0], AV_FRAME_DATA_A53_CC); + av_ccfifo_inject(s->cc_fifo, frame); frame->pts = s->next_pts++; frame->duration = 1; From patchwork Fri Apr 7 21:58:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 41020 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4645:b0:e3:3194:9d20 with SMTP id eb5csp627725pzb; Fri, 7 Apr 2023 14:03:41 -0700 (PDT) X-Google-Smtp-Source: AKy350bfDKGwlQ2o5wyBBf0pATRHUH9jT1HiuaV0pnDIcyWUKsVj7SIeWc0/0+vxqc8jYCxduR6H X-Received: by 2002:a17:906:6b1a:b0:947:9142:e521 with SMTP id q26-20020a1709066b1a00b009479142e521mr474490ejr.35.1680901421750; Fri, 07 Apr 2023 14:03:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1680901421; cv=none; d=google.com; s=arc-20160816; b=T0LYJ64D2NbWcx50TQrOfpUsNsGZwhKmoOc5RBbcI8Zbwh5cNUBRGw87edJyPdmdmP u5AvfTZigpscQdoTCZobafoPrCDqrL6bDIguEfHDf2OAKcAAFIR0svYSs5W0ryeUEPRG Xz4adfnwhWPfBDbcX34UBlLtSQ9cHUafGqhJO0KaG1kYQqXMAutWxBs3IfrhtbpT1+N4 W0h2p6lwswqQRzStSHB6N2mNDzS1znKE9bumN9DDF+HyQL4xG0kkRvuv3+13FxwvoRgt FfE80kvyNnQ1U8ons5A5omGKy3KG/5E2dfqU+5YO0HckcSWfAs6FB+QmDP76zAlz9kPO +z3g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=v6aWwe1NkgGWn56drD9h4n7iByPrMeisI4wQJAQc4DI=; b=w6iCPGU2d9CPCWliIV2y03oWj8+be8TNEU71yvEAO6KFAxJ435tkqPP+7ILp71CjtN 6aswHdHJ0WdkSkRq3s+6G9hz7Bq6kkHihDKj5d3HmA/OCiCWob88qn9ImjXeACdiqwh2 uhNrTEfBNJyQv488uFAG0cMIUoTp0W0oAZXAa8Kd3AcNifDZhoD7PWSTKl5XJIJKdh9a 6ehKJpfHB6wzHaicncJ5rqStpnPjurlSOTqmuQ7HsX+Tp2HOheQTOnO2JFus2h1/Km1u 9S8ErmLmZz44JJ2IHBRiPpbwOMBBD48jrq4pnQEB1eXWKBmUgbeHqknZGqE25Y7AVlQC +xlQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=BB6RqYLY; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id e25-20020a170906845900b0093d1c4a53aesi3995454ejy.240.2023.04.07.14.03.41; Fri, 07 Apr 2023 14:03:41 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=BB6RqYLY; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E8FB668BA9C; Sat, 8 Apr 2023 00:03:13 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qv1-f48.google.com (mail-qv1-f48.google.com [209.85.219.48]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 17D95688313 for ; Sat, 8 Apr 2023 00:03:05 +0300 (EEST) Received: by mail-qv1-f48.google.com with SMTP id lz6so50824qvb.6 for ; Fri, 07 Apr 2023 14:03:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20210112.gappssmtp.com; s=20210112; t=1680901383; x=1683493383; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=vGePmGvavry5nM6OKP9lTZufL8URRnAHBeAGpU+6OKM=; b=BB6RqYLYDwhHivk1uTGnQ4Z54F5jkZp+fAf0uykzRzOMVDSqH/Lt3gIpn1pVPeowbC 44sKZIAflBzNSSTl9uwufxAcxOU0OTOfV5EqzVRnKG1LECje1bI+KZOX1Q36VrYrxyIt qhqyUh74WwlL0HeeRDQpKqpO6Ya8rO1lnBZcW6dQFYsarhH6iRdU2L43Ff6Ilo04IXCg MyXnSC8p2eH9lLYTZjTFBPk+y3uqUG/ZQBbr9ZY7dBEfQPr98nq9tPHeZta5q546Mkkz 5wmg4cvZ2/DOhtcxzhvoFN6U8UFFATb0J0z0zdBc0ukiRE34DgbJlRQcD2OxOZLs1QRv apeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680901383; x=1683493383; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=vGePmGvavry5nM6OKP9lTZufL8URRnAHBeAGpU+6OKM=; b=o3VgV53caaJfcDa/afJkt/t8hiP9K0SmcqpGK4DoY6yjOq/DahP0IB/gPz91eEi5hf eWe/t1Z6XHLzMKmSsdMc0hxaRPbDYb+3DS6EZCaUMQDKIuZx7A7id31/5sjusSkhzG/I +3At5vyUYKvjiyCqvEmToIlDw6DtX6iu+76w8qPnkfN7FkyRYAHNcdElK/bQAz0kJgNL +uXLT5vC/WsA99xlBpW2fyKeMudhh4Eg/xYbOLDP1LJ3Ga77pCWPGyzEAVdHHLgw661z iBNlMt6uk97+28SV52GFF5FNf0WLOnG3bCfNCka5hZgAWiiCMswRdoREA0uubffrH4uk koLw== X-Gm-Message-State: AAQBX9dlaz6iuxV3io0+A0qRN17Eial/iIOEtgiBOssug5GF+CZeyUsE 68il4mgTgWlKGQRVkjXqiypfeWV1sgfRylkb66o= X-Received: by 2002:ad4:5cec:0:b0:5e0:3825:9ad9 with SMTP id iv12-20020ad45cec000000b005e038259ad9mr990809qvb.2.1680901383629; Fri, 07 Apr 2023 14:03:03 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id z13-20020ad4414d000000b005dd8b9345d8sm1405143qvp.112.2023.04.07.14.03.03 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 07 Apr 2023 14:03:03 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 7 Apr 2023 17:58:27 -0400 Message-Id: <1680904709-25951-4-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1680904709-25951-1-git-send-email-dheitmueller@ltnglobal.com> References: <1680904709-25951-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [PATCH v2 3/5] yadif: Properly preserve CEA-708 closed captions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: iCu2OJG5qiOu Various deinterlacing modes have the effect of doubling the framerate, and we need to ensure that the caption data isn't duplicated (or else you get double captions on-screen). Use the new ccfifo mechanism for yadif (and yadif_cuda and bwdif since they use the same yadif core) so that CEA-708 data is properly preserved through this filter. Signed-off-by: Devin Heitmueller --- libavfilter/vf_bwdif.c | 7 +++++++ libavfilter/vf_yadif.c | 6 ++++++ libavfilter/vf_yadif_cuda.c | 8 ++++++++ libavfilter/yadif.h | 2 ++ libavfilter/yadif_common.c | 5 +++++ 5 files changed, 28 insertions(+) diff --git a/libavfilter/vf_bwdif.c b/libavfilter/vf_bwdif.c index 34e8c5e..c625b68 100644 --- a/libavfilter/vf_bwdif.c +++ b/libavfilter/vf_bwdif.c @@ -297,6 +297,7 @@ static av_cold void uninit(AVFilterContext *ctx) av_frame_free(&yadif->prev); av_frame_free(&yadif->cur ); av_frame_free(&yadif->next); + av_ccfifo_freep(&yadif->cc_fifo); } static const enum AVPixelFormat pix_fmts[] = { @@ -332,6 +333,12 @@ static int config_props(AVFilterLink *link) if(yadif->mode&1) link->frame_rate = av_mul_q(link->src->inputs[0]->frame_rate, (AVRational){2,1}); + else + link->frame_rate = ctx->inputs[0]->frame_rate; + + if (!(yadif->cc_fifo = av_ccfifo_alloc(&link->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue. Captions will be passed through\n"); + if (link->w < 3 || link->h < 4) { av_log(ctx, AV_LOG_ERROR, "Video of less than 3 columns or 4 lines is not supported\n"); diff --git a/libavfilter/vf_yadif.c b/libavfilter/vf_yadif.c index 1be02de..b51d21f 100644 --- a/libavfilter/vf_yadif.c +++ b/libavfilter/vf_yadif.c @@ -261,6 +261,7 @@ static av_cold void uninit(AVFilterContext *ctx) av_frame_free(&yadif->prev); av_frame_free(&yadif->cur ); av_frame_free(&yadif->next); + av_ccfifo_freep(&yadif->cc_fifo); } static const enum AVPixelFormat pix_fmts[] = { @@ -293,6 +294,11 @@ static int config_output(AVFilterLink *outlink) if(s->mode & 1) outlink->frame_rate = av_mul_q(ctx->inputs[0]->frame_rate, (AVRational){2, 1}); + else + outlink->frame_rate = ctx->inputs[0]->frame_rate; + + if (!(s->cc_fifo = av_ccfifo_alloc(&outlink->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue. Captions will be passed through\n"); if (outlink->w < 3 || outlink->h < 3) { av_log(ctx, AV_LOG_ERROR, "Video of less than 3 columns or lines is not supported\n"); diff --git a/libavfilter/vf_yadif_cuda.c b/libavfilter/vf_yadif_cuda.c index 685b8a2..d96ec39 100644 --- a/libavfilter/vf_yadif_cuda.c +++ b/libavfilter/vf_yadif_cuda.c @@ -206,6 +206,9 @@ static av_cold void deint_cuda_uninit(AVFilterContext *ctx) av_frame_free(&y->cur); av_frame_free(&y->next); + if (yadif->cc_fifo) + av_cc_fifo_free(yadif->cc_fifo); + av_buffer_unref(&s->device_ref); s->hwctx = NULL; av_buffer_unref(&s->input_frames_ref); @@ -291,6 +294,11 @@ static int config_output(AVFilterLink *link) if(y->mode & 1) link->frame_rate = av_mul_q(ctx->inputs[0]->frame_rate, (AVRational){2, 1}); + else + outlink->frame_rate = ctx->inputs[0]->frame_rate; + + if (!(s->cc_fifo = av_cc_fifo_alloc(&outlink->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue. Captions will be passed through\n"); if (link->w < 3 || link->h < 3) { av_log(ctx, AV_LOG_ERROR, "Video of less than 3 columns or lines is not supported\n"); diff --git a/libavfilter/yadif.h b/libavfilter/yadif.h index c928911..1077576 100644 --- a/libavfilter/yadif.h +++ b/libavfilter/yadif.h @@ -22,6 +22,7 @@ #include "libavutil/opt.h" #include "libavutil/pixdesc.h" #include "avfilter.h" +#include "ccfifo.h" enum YADIFMode { YADIF_MODE_SEND_FRAME = 0, ///< send 1 frame for each frame @@ -76,6 +77,7 @@ typedef struct YADIFContext { int eof; uint8_t *temp_line; int temp_line_size; + AVCCFifo *cc_fifo; /* * An algorithm that treats first and/or last fields in a sequence diff --git a/libavfilter/yadif_common.c b/libavfilter/yadif_common.c index a10cf7a..24cc72a 100644 --- a/libavfilter/yadif_common.c +++ b/libavfilter/yadif_common.c @@ -60,6 +60,8 @@ static int return_frame(AVFilterContext *ctx, int is_second) yadif->out->pts = AV_NOPTS_VALUE; } } + + av_ccfifo_inject(yadif->cc_fifo, yadif->out); ret = ff_filter_frame(ctx->outputs[0], yadif->out); yadif->frame_pending = (yadif->mode&1) && !is_second; @@ -96,6 +98,8 @@ int ff_yadif_filter_frame(AVFilterLink *link, AVFrame *frame) av_assert0(frame); + av_ccfifo_extract(yadif->cc_fifo, frame); + if (yadif->frame_pending) return_frame(ctx, 1); @@ -137,6 +141,7 @@ int ff_yadif_filter_frame(AVFilterLink *link, AVFrame *frame) if (!yadif->out) return AVERROR(ENOMEM); + av_ccfifo_inject(yadif->cc_fifo, yadif->out); av_frame_free(&yadif->prev); if (yadif->out->pts != AV_NOPTS_VALUE) yadif->out->pts *= 2; From patchwork Fri Apr 7 21:58:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 41017 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4645:b0:e3:3194:9d20 with SMTP id eb5csp627832pzb; Fri, 7 Apr 2023 14:03:51 -0700 (PDT) X-Google-Smtp-Source: AKy350aHbKO0BLBsfOR+KBbPl0VI6U+lCnTC/JAjdMyzVQfnRuSoA7unX7VOO6KsN7VRYW4+vukg X-Received: by 2002:a17:906:9bf5:b0:8b0:ad0b:7ab8 with SMTP id de53-20020a1709069bf500b008b0ad0b7ab8mr609958ejc.14.1680901430994; Fri, 07 Apr 2023 14:03:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1680901430; cv=none; d=google.com; s=arc-20160816; b=eqhEudfPjbztHXeP20LoTcMnQ9pDmLhMgpqYstAY17+d3Lmkl/IA7DVuC09TXiwZvN P+PNoEgF6qFQP4VGbmAKAcpczDhUs8e5eaJPmgK0qYgfYa2etgbeEnUDIcu7sjNg89FW KQRX+iYju3lwtDm+1P/XTGaEY7jzBSBMEWU4U/EDG4+CxnD+78pxLP2t5IW5Qg1VjLBX FlkoSzYTX9oEHpiaqCqkTNU0N3sHIrswAWKEr7kTQq7X8UW7mAIJok/chiCduXvV8FO+ PbzqL2TTnjH1S0kpalKwCk9CsQbmnYPRZTHgjKjr2phoCuVo4Krl4BcCvwlL0eEGjPPO qdqA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=o3gYJwzzWZW9ND3pJnmcEXkpItJyKfeYKhyvWadQJLI=; b=0yUZHQ/d8IamF90psxByt047Q8rJJaUZWjKD1bk+CQ9ntsKJpNmaGNxUFdczZmJc9m mKIjUyYyzTK3sQueFPLr23MYk78+l7tSr9fFeZQUwQGJRXAJhIPUtPS4NJyt8Vgmwjen T0TQDwM14FRt/QVlI66m+R5ShV05I3dQdf146o2BVAkdL89I+RuNRFGpLYE7X8tlh0g5 JYmKd0ioBbWDjj8qLLYcCbreb4Cy909TDS0SgeGWV+Hxbmovew36X8aX1ZBfPUgQV1Ch ObTFD9iYkkcaPpIHFM6pH34TQ9TIYZrcwZ5kslfn9sOd3nU5mGDdKvxps9KKy6Gydw/D jjHQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=b9+JWZAz; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id k23-20020a170906681700b008f100fd5096si4080480ejr.887.2023.04.07.14.03.50; Fri, 07 Apr 2023 14:03:50 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=b9+JWZAz; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id DAD3868BAEC; Sat, 8 Apr 2023 00:03:14 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qv1-f43.google.com (mail-qv1-f43.google.com [209.85.219.43]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A79AE68B6CA for ; Sat, 8 Apr 2023 00:03:05 +0300 (EEST) Received: by mail-qv1-f43.google.com with SMTP id kj14so4638qvb.12 for ; Fri, 07 Apr 2023 14:03:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20210112.gappssmtp.com; s=20210112; t=1680901384; x=1683493384; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=rRpMBNpUWem3BV21BZG3HkzeATIVq7Xa7cR2Qq7/quo=; b=b9+JWZAzyw4urCUMOBkC2w/kkQsrutUXVb+q1ItC0IKrryDxk5WO3NxAWF74mRgyl3 350U5GJr52/WNTUpiAFfmWZEfoZ5svtB3N2r7hBu6WekG5iJqlriGMeh6s7c8x5oT/Wd X6n3pOc7gsR2/ImdbVpNPIZ8nDIPc1xoVh5J2R983V15fqsOJ7sW0K5yLhnSQBtjuBsv aZi2RRbGLckDnw7ZBJoTpEmwcmZGI8A6RsY8+AqcmiWIQADYXVlneis5yQ+8dvgQxgMW ofK9x/VrJ7RUyme8vU8VcevB0sH3DdfK75bsM0JFenH2WVkckVRhFK8kE+Ot9JVOAlCl Vy1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680901384; x=1683493384; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=rRpMBNpUWem3BV21BZG3HkzeATIVq7Xa7cR2Qq7/quo=; b=eQ8aWEzcDFif2b2+wBl05Y/4Gh/V4qe1lnxodW6EnJIxasjJeoyQLtnBJttY/rQqdv GEflkRUHFy7b3FLkD93F1yONwaky847BBt96YHdI2RwINGfer3JbQzWxfeVT2n6mm1QC hH6qb7kJkPk/L0Ab5YDvCiCHyisUVjDvH8z5btSLM4m39sbyJS5S/K5MAl9gwo5WabMS O/3hOi8Qy8fw8Gjf4GINWCAVcFWT0jzaL+pvyAePD4A02DnukKr9WMnd+f+1wF6+IQxd BkXbYJN+RRlCOfdYKPu33pbjtt7rwOjgsvDEP/K/Cd3kLcK/xEEjVCKu215DjO/0080t wlvA== X-Gm-Message-State: AAQBX9ehcTi3xEG7Q8FWvh40NlzyyOEdm6SBQ5yRBPfWPO2q9vHovqMd 7UfkusHt3smZUeQanu2bwxLEQceAepBg0jAK0+M= X-Received: by 2002:a05:6214:1316:b0:5cb:ab2e:b15c with SMTP id pn22-20020a056214131600b005cbab2eb15cmr7556145qvb.30.1680901384230; Fri, 07 Apr 2023 14:03:04 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id z13-20020ad4414d000000b005dd8b9345d8sm1405143qvp.112.2023.04.07.14.03.03 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 07 Apr 2023 14:03:03 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 7 Apr 2023 17:58:28 -0400 Message-Id: <1680904709-25951-5-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1680904709-25951-1-git-send-email-dheitmueller@ltnglobal.com> References: <1680904709-25951-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [PATCH v2 4/5] tinterlace: Properly preserve CEA-708 closed captions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: OdPY1oQkyigm Because the interlacing filter halves the effective framerate, we need to ensure that no CEA-708 data is lost as frames are merged. Make use of the new ccfifo mechanism to ensure that caption data is properly preserved as frames pass through the filter. Thanks to Thomas Mundt for review and noticing a couple of missed codepaths for injection on output. Signed-off-by: Devin Heitmueller --- libavfilter/tinterlace.h | 2 ++ libavfilter/vf_tinterlace.c | 8 ++++++++ 2 files changed, 10 insertions(+) diff --git a/libavfilter/tinterlace.h b/libavfilter/tinterlace.h index 37b6c10..9f5ce7e 100644 --- a/libavfilter/tinterlace.h +++ b/libavfilter/tinterlace.h @@ -32,6 +32,7 @@ #include "libavutil/pixdesc.h" #include "drawutils.h" #include "avfilter.h" +#include "ccfifo.h" #define TINTERLACE_FLAG_VLPF 01 #define TINTERLACE_FLAG_CVLPF 2 @@ -77,6 +78,7 @@ typedef struct TInterlaceContext { const AVPixFmtDescriptor *csp; void (*lowpass_line)(uint8_t *dstp, ptrdiff_t width, const uint8_t *srcp, ptrdiff_t mref, ptrdiff_t pref, int clip_max); + AVCCFifo *cc_fifo; } TInterlaceContext; void ff_tinterlace_init_x86(TInterlaceContext *interlace); diff --git a/libavfilter/vf_tinterlace.c b/libavfilter/vf_tinterlace.c index 0326292..226983c 100644 --- a/libavfilter/vf_tinterlace.c +++ b/libavfilter/vf_tinterlace.c @@ -291,6 +291,9 @@ static int config_out_props(AVFilterLink *outlink) #endif } + if (!(tinterlace->cc_fifo = av_ccfifo_alloc(&outlink->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue. Captions will be passed through\n"); + av_log(ctx, AV_LOG_VERBOSE, "mode:%d filter:%s h:%d -> h:%d\n", tinterlace->mode, (tinterlace->flags & TINTERLACE_FLAG_CVLPF) ? "complex" : (tinterlace->flags & TINTERLACE_FLAG_VLPF) ? "linear" : "off", @@ -375,6 +378,8 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) tinterlace->cur = tinterlace->next; tinterlace->next = picref; + av_ccfifo_extract(tinterlace->cc_fifo, picref); + cur = tinterlace->cur; next = tinterlace->next; /* we need at least two frames */ @@ -451,6 +456,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) if (!out) return AVERROR(ENOMEM); out->pts /= 2; // adjust pts to new framerate + av_ccfifo_inject(tinterlace->cc_fifo, out); ret = ff_filter_frame(outlink, out); return ret; } @@ -486,6 +492,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) out->pts = cur->pts*2; out->pts = av_rescale_q(out->pts, tinterlace->preout_time_base, outlink->time_base); + av_ccfifo_inject(tinterlace->cc_fifo, out); if ((ret = ff_filter_frame(outlink, out)) < 0) return ret; @@ -521,6 +528,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) out->pts = av_rescale_q(out->pts, tinterlace->preout_time_base, outlink->time_base); out->duration = av_rescale_q(1, av_inv_q(outlink->frame_rate), outlink->time_base); + av_ccfifo_inject(tinterlace->cc_fifo, out); ret = ff_filter_frame(outlink, out); return ret; From patchwork Fri Apr 7 21:58:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 41021 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4645:b0:e3:3194:9d20 with SMTP id eb5csp627906pzb; Fri, 7 Apr 2023 14:04:00 -0700 (PDT) X-Google-Smtp-Source: AKy350aLP++RHKbalNZpozfpgtDa4k9fXfigL9DYWvpSqb3TgrNZ7Kxiy1oVSuDidPHv6zYIwB+Y X-Received: by 2002:a05:6402:1209:b0:4fd:2140:5cc6 with SMTP id c9-20020a056402120900b004fd21405cc6mr3565974edw.17.1680901440524; Fri, 07 Apr 2023 14:04:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1680901440; cv=none; d=google.com; s=arc-20160816; b=F3RuKtNpFTYT9IPET4LsAJ3Mt3CdE1DakLYHlhoOR8rpyZrlgAcFbN+R6+EiwdUOnm pc0A3y+dM+rf6Rcl5sE0H+5YLhWXfi2c25UTY/YPOdDgao7Or8AHmjbCk4GPdsSZY9J1 p3x5uOrEpfc9NzofbOv2aJ75me/9jwOMmfO/zCFwfhA39bAp5Bk0u42y03W+rQkTL4gV cnFIIE4e5UzfbmPOPUHq3UtSWqfZX1cMUJVPnNMOVXQ1qVaAvsI7Xf2S+JxFUC6iOPtp YNPbkRIaG3+cKCwHYfIUSgUEzyl9fuCAyG1AMqMuq3dkcusXCTQpnFgmJkkwdHULOW+1 Rn2Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=STvH3ArHLJ03FlKa4m0BZMSQH31SSpGPx3kAYX/Ymw0=; b=j2kNGW8EZc31oawx4DJpDdAac5h7sPDktx+fsHUFNyQx2ugNjyheS6YaK6HbvX381b bBX5rCpY0L33Ccqi3b9Ung75zBtg0A/GEhBLVICU9BpR02HuCORXMnYq/hCltjZ00FNK y4zM4IwK0XyoJFhq6tfuBoCR4bsTbuAu7bqi4LRWqaUVKBTrETUG8P0d9H93kXnzmDJ2 57XmtCKAkgwPER+tNByAUsAeSgUP0LQtRblF2AenHWtCjjZn4uwHqSK4j+T6WLoZWDP4 EbTEGjGLGbDmH9nO5GfKeR7iJQkC/HCiRfN61C9yfJAhVDqUEr4YEvhKYfINQ92RTV6B AgcA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b="WPgAHkS/"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a6-20020aa7d906000000b004fd2a2c6ff8si3712885edr.467.2023.04.07.14.04.00; Fri, 07 Apr 2023 14:04:00 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b="WPgAHkS/"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 03C3068BAF5; Sat, 8 Apr 2023 00:03:16 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qv1-f45.google.com (mail-qv1-f45.google.com [209.85.219.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 377D868B6CA for ; Sat, 8 Apr 2023 00:03:06 +0300 (EEST) Received: by mail-qv1-f45.google.com with SMTP id on15so17702780qvb.7 for ; Fri, 07 Apr 2023 14:03:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20210112.gappssmtp.com; s=20210112; t=1680901385; x=1683493385; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=AM+Fy6OO8ELpLskiZx+Qxyrf7Skt058OW7NUjniSMes=; b=WPgAHkS/+t8Q5MRzaIg4TQu4ABxiAQFNHi1g7zCc1ndhsTkF/+tDilvuNGnDpzNZVt m0AVSfIhMC6FiDFu/FFBsHD9/mz3LE8rhVgVzNUNNKVnI5K+UQWTIALYpRoV+NBcWAXD A32qfxfStL+BdEwu1iYXcUmg3KTKhiF9g1xj0DSVh2JZymCeqoGQl9aOz8clWxr06wBr grMWvgGeuqnrriJdsHu0F314dYyDl+PsO2At6GD69J0sYWtC0HvRjJFiulfnnfZvf0UB KYlwlC+MmQvSadAWPThtr2qgXYOA6DOE1VDYJM0P3tDpyZddUyBctuCGwC+/RLSHdrDm l+zQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680901385; x=1683493385; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=AM+Fy6OO8ELpLskiZx+Qxyrf7Skt058OW7NUjniSMes=; b=6XnUzutaMm7K2YjIAj9zQgd4DVW8eXC3v/4V9M45xApC5KmkKKIjx7EINCWfpxWTrg EOe2fZ2DrPCLucxhAvDxDotJ8gQwJZuHg9ko+1LAlLJCjy+lPjows3fTLPtSjgMlRtv3 bxXrTmr1w2ecEKpgIa01usFWKX8wdsDWt5qr2UDGPZAEpoiyGknS0R7+D1BCnHJYMQS8 8OYiVT6l1AFB3iyUwQNwVxHumkJUY8V4BGYuVQZy4vBMX039KnRj/GJrOPsb/+Zci8tG 5/VzNMXis+AYWrOgfRIMIbzJ3zHuI8GUS0JTL/WZ2O3+lTPpkqmZKLxEShIPhej9i/ML SHFg== X-Gm-Message-State: AAQBX9fj1qQDb+qQsfwbapgwGa/eRbfvZOOjYReo39ZZ36MxGEVjOHqn LStzG8J9v8CPlVB9fZNe0Ebfta6xWGhDgNFKego= X-Received: by 2002:a05:6214:b6b:b0:56c:37a:58b2 with SMTP id ey11-20020a0562140b6b00b0056c037a58b2mr6898629qvb.15.1680901384775; Fri, 07 Apr 2023 14:03:04 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id z13-20020ad4414d000000b005dd8b9345d8sm1405143qvp.112.2023.04.07.14.03.04 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 07 Apr 2023 14:03:04 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 7 Apr 2023 17:58:29 -0400 Message-Id: <1680904709-25951-6-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1680904709-25951-1-git-send-email-dheitmueller@ltnglobal.com> References: <1680904709-25951-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [PATCH v2 5/5] vf_ccrepack: Add new filter to repack CEA-708 side data X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 4r/X4VXLBNtP THis filter can correct certain issues seen from upstream sources where the cc_count is not properly set or the CEA-608 tuples are not at the start of the payload as expected. Make use of the ccfifo to extract and immediately repack the CEA-708 side data, thereby removing any extra padding and ensuring the 608 tuples are at the front of the payload. Signed-off-by: Devin Heitmueller --- doc/filters.texi | 10 +++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/vf_ccrepack.c | 100 ++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 112 insertions(+) create mode 100644 libavfilter/vf_ccrepack.c diff --git a/doc/filters.texi b/doc/filters.texi index 120fe66..e72e73d 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -8930,6 +8930,16 @@ Only deinterlace frames marked as interlaced. The default value is @code{all}. @end table +@section ccrepack + +Repack CEA-708 closed captioning side data + +This filter fixes various issues seen with commerical encoders +related to upstream malformed CEA-708 payloads, specifically +incorrect number of tuples (wrong cc_count for the target FPS), +and incorrect ordering of tuples (i.e. the CEA-608 tuples are not at +the first entries in the payload). + @section cas Apply Contrast Adaptive Sharpen filter to video stream. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 628ade8..3d0b3ad 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -212,6 +212,7 @@ OBJS-$(CONFIG_BOXBLUR_OPENCL_FILTER) += vf_avgblur_opencl.o opencl.o \ opencl/avgblur.o boxblur.o OBJS-$(CONFIG_BWDIF_FILTER) += vf_bwdif.o yadif_common.o OBJS-$(CONFIG_CAS_FILTER) += vf_cas.o +OBJS-$(CONFIG_CCREPACK_FILTER) += vf_ccrepack.o OBJS-$(CONFIG_CHROMABER_VULKAN_FILTER) += vf_chromaber_vulkan.o vulkan.o vulkan_filter.o OBJS-$(CONFIG_CHROMAHOLD_FILTER) += vf_chromakey.o OBJS-$(CONFIG_CHROMAKEY_FILTER) += vf_chromakey.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index d7db46c..b38550b 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -196,6 +196,7 @@ extern const AVFilter ff_vf_boxblur; extern const AVFilter ff_vf_boxblur_opencl; extern const AVFilter ff_vf_bwdif; extern const AVFilter ff_vf_cas; +extern const AVFilter ff_vf_ccrepack; extern const AVFilter ff_vf_chromaber_vulkan; extern const AVFilter ff_vf_chromahold; extern const AVFilter ff_vf_chromakey; diff --git a/libavfilter/vf_ccrepack.c b/libavfilter/vf_ccrepack.c new file mode 100644 index 0000000..063c360 --- /dev/null +++ b/libavfilter/vf_ccrepack.c @@ -0,0 +1,100 @@ +/* + * CEA-708 Closed Caption Repacker + * Copyright (c) 2023 LTN Global Communications + * + * Author: Devin Heitmueller + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/* + * Repackage CEA-708 arrays, which deals with incorrect cc_count for a given + * output framerate, and incorrect 708 padding. + * + * See CEA CEA-10-A "EIA-708-B Implementation Guidance", Section 26.5 + * "Grouping DTVCC Data Within user_data() Structure" + */ + +#include "avfilter.h" +#include "internal.h" +#include "ccfifo.h" +#include "libavutil/opt.h" + +typedef struct CCRepackContext +{ + const AVClass *class; + AVCCFifo *cc_fifo; +} CCRepackContext; + +static const AVOption ccrepack_options[] = { + { NULL } +}; + +AVFILTER_DEFINE_CLASS(ccrepack); + +static int config_input(AVFilterLink *link) +{ + CCRepackContext *ctx = link->dst->priv; + + if (!(ctx->cc_fifo = av_ccfifo_alloc(&link->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue. Captions will be passed through\n"); + + return 0; +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *frame) +{ + CCRepackContext *ctx = inlink->dst->priv; + AVFilterLink *outlink = inlink->dst->outputs[0]; + + av_ccfifo_extract(ctx->cc_fifo, frame); + av_ccfifo_inject(ctx->cc_fifo, frame); + + return ff_filter_frame(outlink, frame); +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + CCRepackContext *s = ctx->priv; + av_ccfifo_freep(&s->cc_fifo); +} + +static const AVFilterPad avfilter_vf_ccrepack_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .filter_frame = filter_frame, + .config_props = config_input, + }, +}; + +static const AVFilterPad avfilter_vf_ccrepack_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + }, +}; + +AVFilter ff_vf_ccrepack = { + .name = "ccrepack", + .description = NULL_IF_CONFIG_SMALL("Repack CEA-708 closed caption metadata"), + .uninit = uninit, + .priv_size = sizeof(CCRepackContext), + .priv_class = &ccrepack_class, + FILTER_INPUTS(avfilter_vf_ccrepack_inputs), + FILTER_OUTPUTS(avfilter_vf_ccrepack_outputs), +};