From patchwork Fri Mar 17 20:09:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 40714 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:d046:b0:cd:afd7:272c with SMTP id hv6csp14826pzb; Fri, 17 Mar 2023 13:10:23 -0700 (PDT) X-Google-Smtp-Source: AK7set+45rgxlCM0hrp25XTNaEhwPmRnK+thd5kXuO/FgynqIHcO0+s5kmUu2X/bxITTsRcD/35X X-Received: by 2002:a17:906:b349:b0:878:6b39:6d2a with SMTP id cd9-20020a170906b34900b008786b396d2amr704291ejb.46.1679083822984; Fri, 17 Mar 2023 13:10:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1679083822; cv=none; d=google.com; s=arc-20160816; b=pIZhcJp17vtaQ6+S+9igSV19PaI7Mwvy72FOBP9d+N8dr/CgRwrh/ZEE4bBED5OcLT OkfZcJjAy8En0i8+ViB85g5uy40HqpDPaKtNbKF52CHjNzbUCGJbTb/H1etjoGvkgUF/ WmpBPRW1Bs7e7GCTAzxxFp2uHD+8ly+xWx2w8Xop/W6+97I5NiRzsGYxLcaC2oWd4wbh pwjoK8Vx/kdMQEPhm1r0WR5+Bu7xbbS75CcCo10lmZ0Sa0ybrN8YINcKCCzzsbmcCdO5 lyRI0pzyAGDIXZmH3c6BohPZ9S4dLaM959jY0GqVUV9Fl4ey2WB+cKClY56OnG8JoRBm zsUg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=dNpMxOsiXB37ff7q//nIpDU70bqWFQVgSreNwokrBfk=; b=FnnDuEnAKRkq7KjHOsqQZPTJGhtSv3LcXu5tJNDWXQQZ69gx3CyKTvMze59zb5snIN ndu7CplPr81o4tWEA0x44wP+WdLuR8ca6Fi9srG7Ea55gzEvIfVXNKDOSle/7tnfztip yXhhPE3tsJB9DhaNqhQC/2StgcrSGOdkUbatCoTzEv0oSxA+00issbyw7WRAb29Vi7oP /ULO9smT9enwXa9oyt+K2ZyPdsH+WhYURrwNBtqqhFphVSwzYQFx2oi48oWxbQvum6fp 4we4/QnllUJvM7Gi8qr2uRdCA1ddI9QzdBWTI6afK5J7z2q4hFsdOO5L8oK9ggbzYGMH /O5g== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=KvQEQkf5; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id e10-20020a170906844a00b0093065cf57absi2454274ejy.367.2023.03.17.13.10.21; Fri, 17 Mar 2023 13:10:22 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=KvQEQkf5; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 43BC168C25D; Fri, 17 Mar 2023 22:09:56 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qt1-f170.google.com (mail-qt1-f170.google.com [209.85.160.170]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 714FA68C154 for ; Fri, 17 Mar 2023 22:09:48 +0200 (EET) Received: by mail-qt1-f170.google.com with SMTP id t9so6939831qtx.8 for ; Fri, 17 Mar 2023 13:09:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20210112.gappssmtp.com; s=20210112; t=1679083787; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Bk+ORmg8byGfBpHsnXhu945d54blgWlky1zb/u6hCvU=; b=KvQEQkf5bhEcq/iaP4rGRMCRzOMnd7u6NY1Tq8d4SdVXVKT5Nz9Rp2SdAWy/1HgZ8z GRCgjIr5UJq9n3iHcyQyqfZgBvAPfIPDmQAJS7JzcAPIYJliWvmCm41n+lL5UD04aKDn kDhVJ5jBHqPdcnccZ6mxIpxYMtXHSIMqt3XuDL7mdU1h2Bmx7uhadCshfTWOOGpMYSwY /elB8KzjDRqTXdCUkvTd79GUx5yZ1vCER33f3o/LQHtTHzSUPMFG1ZmVEyjFIqzLhqTu eNeCUBoVdDUEqHc0+JmgqSgqMAOS2Pcglv26z1QV3hBgIbyJT6PjkAIRMwhcIJslkfXg 7BKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679083787; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Bk+ORmg8byGfBpHsnXhu945d54blgWlky1zb/u6hCvU=; b=LvAzVWAGXJ/y3NuU2R85v3/tHxo5wSBDWh+wKfjAyWCRChq2GOA3Qw0ZX2H7LEh8xK qazzlQEXQ6ZoLMLuuP2uiUmtb7SPkZ6sOXhOElUSZEkVaCzbqj+F0ITLDS6fjA+BYBM6 4LrhJ4KQOUIU/WBwZJFr4YDZe5EN1xo/hjFIw2yQ1OidiBaGmqsWNW9NPEyUWLZe7H3A h5s+zE6d7nUaQhRx2bXGkolYW1YqsWc70V9X2NTa1H8Nyv3PczQkNPXiUIRguh16+K1L nfL1jYyEA94z8jpxG+996ewCJZHp847QcZ/4uyEbDL6SYt+QTo5ne1+VZVNGbR0J++sp 1ypg== X-Gm-Message-State: AO0yUKVKs6x5zDHd002uxC9asuruYekOiNe++2hmMIe+ZHbcFumki2Jl nOCXebQCPXIUdXyipcgZhUbkJN+HCD4mFI6Ud4s= X-Received: by 2002:ac8:5c96:0:b0:3b9:b5c5:ebb1 with SMTP id r22-20020ac85c96000000b003b9b5c5ebb1mr14502629qta.9.1679083786991; Fri, 17 Mar 2023 13:09:46 -0700 (PDT) Received: from klab-nyc-kernellabs2.localdomain (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id j5-20020ac86645000000b003bd0e7ff466sm2065685qtp.7.2023.03.17.13.09.46 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 17 Mar 2023 13:09:46 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 17 Mar 2023 16:09:37 -0400 Message-Id: <20230317200941.3936-2-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 2.35.1.655.ga68dfadae5 In-Reply-To: <20230317200941.3936-1-dheitmueller@ltnglobal.com> References: <20230317200941.3936-1-dheitmueller@ltnglobal.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [RFC PATCH 1/5] ccfifo: Properly handle CEA-708 captions through framerate conversion X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: NzH2oAExMXKE When transcoding video that contains 708 closed captions, the caption data is tied to the frames as side data. Simply dropping or adding frames to change the framerate will result in loss of data, so the caption data needs to be preserved and reformatted. For example, without this patch converting 720p59 to 1080i59 would result in loss of 50% of the caption bytes, resulting in garbled 608 captions and 708 probably wouldn't render at all. Further, the frames that are there will have an illegal cc_count for the target framerate, so some decoders may ignore the packets entirely. Extract the 608 and 708 tuples and insert them onto queues. Then after dropping/adding frames, re-write the tuples back into the resulting frames at the appropriate rate given the target framerate. This includes both having the correct cc_count as well as clocking out the 608 pairs at the appropriate rate. Signed-off-by: Devin Heitmueller --- libavutil/Makefile | 2 + libavutil/ccfifo.c | 192 +++++++++++++++++++++++++++++++++++++++++++++ libavutil/ccfifo.h | 85 ++++++++++++++++++++ 3 files changed, 279 insertions(+) create mode 100644 libavutil/ccfifo.c create mode 100644 libavutil/ccfifo.h diff --git a/libavutil/Makefile b/libavutil/Makefile index dc9012f9a8..b6115859fa 100644 --- a/libavutil/Makefile +++ b/libavutil/Makefile @@ -17,6 +17,7 @@ HEADERS = adler32.h \ buffer.h \ cast5.h \ camellia.h \ + ccfifo.h \ channel_layout.h \ common.h \ cpu.h \ @@ -113,6 +114,7 @@ OBJS = adler32.o \ buffer.o \ cast5.o \ camellia.o \ + ccfifo.o \ channel_layout.o \ cpu.o \ crc.o \ diff --git a/libavutil/ccfifo.c b/libavutil/ccfifo.c new file mode 100644 index 0000000000..98ee365eee --- /dev/null +++ b/libavutil/ccfifo.c @@ -0,0 +1,192 @@ +/* + * CEA-708 Closed Captioning FIFO + * Copyright (c) 2023 LTN Global Communications + * + * Author: Devin Heitmueller + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "ccfifo.h" + +struct AVCCFifo { + AVFifo *cc_608_fifo; + AVFifo *cc_708_fifo; + int expected_cc_count; + int expected_608; + int cc_detected; + void *log_ctx; +}; + +#define MAX_CC_ELEMENTS 128 +#define CC_BYTES_PER_ENTRY 3 + +struct cc_lookup { + int num; + int den; + int cc_count; + int num_608; +}; + +const static struct cc_lookup cc_lookup_vals[] = { + { 15, 1, 40, 4 }, + { 24, 1, 25, 3 }, + { 24000, 1001, 25, 3 }, + { 30, 1, 20, 2 }, + { 30000, 1001, 20, 2}, + { 60, 1, 10, 1 }, + { 60000, 1001, 10, 1}, +}; + +void av_ccfifo_freep(AVCCFifo **ccf) +{ + if (ccf && *ccf) { + AVCCFifo *tmp = *ccf; + if (tmp->cc_608_fifo) + av_fifo_freep2(&tmp->cc_608_fifo); + if (tmp->cc_708_fifo) + av_fifo_freep2(&tmp->cc_708_fifo); + av_freep(*ccf); + } +} + +AVCCFifo *av_ccfifo_alloc(AVRational *framerate, void *log_ctx) +{ + AVCCFifo *ccf; + int i; + + ccf = av_mallocz(sizeof(*ccf)); + if (!ccf) + return NULL; + + if (!(ccf->cc_708_fifo = av_fifo_alloc2(MAX_CC_ELEMENTS, CC_BYTES_PER_ENTRY, 0))) + goto error; + + if (!(ccf->cc_608_fifo = av_fifo_alloc2(MAX_CC_ELEMENTS, CC_BYTES_PER_ENTRY, 0))) + goto error; + + printf("cc_fifo framerate=%d/%d\n", framerate->num, framerate->den); + /* Based on the target FPS, figure out the expected cc_count and number of + 608 tuples per packet. See ANSI/CTA-708-E Sec 4.3.6.1. */ + for (i = 0; i < (sizeof(cc_lookup_vals) / sizeof(struct cc_lookup)); i++) { + if (framerate->num == cc_lookup_vals[i].num && + framerate->den == cc_lookup_vals[i].den) { + ccf->expected_cc_count = cc_lookup_vals[i].cc_count; + ccf->expected_608 = cc_lookup_vals[i].num_608; + break; + } + } + + if (ccf->expected_608 == 0) { + av_log(ccf->log_ctx, AV_LOG_WARNING, "cc_fifo cannot transcode captions fps=%d/%d\n", + framerate->num, framerate->den); + return NULL; + } + + return ccf; + +error: + av_ccfifo_freep(&ccf); + return NULL; +} + +int av_ccfifo_inject(AVCCFifo *ccf, AVFrame *frame) +{ + AVFrameSideData *sd; + int cc_filled = 0; + int i; + + if (!ccf) + return 0; + + if (ccf->cc_detected == 0 || ccf->expected_cc_count == 0) + return 0; + + sd = av_frame_new_side_data(frame, AV_FRAME_DATA_A53_CC, + ccf->expected_cc_count * CC_BYTES_PER_ENTRY); + if (!sd) + return 0; + + for (i = 0; i < ccf->expected_608; i++) { + if (av_fifo_can_read(ccf->cc_608_fifo) >= CC_BYTES_PER_ENTRY) { + av_fifo_read(ccf->cc_608_fifo, &sd->data[cc_filled * CC_BYTES_PER_ENTRY], + CC_BYTES_PER_ENTRY); + cc_filled++; + } else { + break; + } + } + + /* Insert any available data from the 708 FIFO */ + while (cc_filled < ccf->expected_cc_count) { + if (av_fifo_can_read(ccf->cc_708_fifo) >= CC_BYTES_PER_ENTRY) { + av_fifo_read(ccf->cc_708_fifo, &sd->data[cc_filled * CC_BYTES_PER_ENTRY], + CC_BYTES_PER_ENTRY); + cc_filled++; + } else { + break; + } + } + + /* Insert 708 padding into any remaining fields */ + while (cc_filled < ccf->expected_cc_count) { + sd->data[cc_filled * CC_BYTES_PER_ENTRY] = 0xfa; + sd->data[cc_filled * CC_BYTES_PER_ENTRY + 1] = 0x00; + sd->data[cc_filled * CC_BYTES_PER_ENTRY + 2] = 0x00; + cc_filled++; + } + printf("enqueue cc_filled=%d\n", cc_filled); + return 0; +} + +int av_ccfifo_extract(AVCCFifo *ccf, AVFrame *frame) +{ + int i; + + if (!ccf) + return 0; + + /* Read the A53 side data, discard padding, and put 608/708 into + queues so we can ensure they get into the output frames at + the correct rate... */ + if (ccf->expected_cc_count > 0) { + AVFrameSideData *side_data = av_frame_get_side_data(frame, AV_FRAME_DATA_A53_CC); + if (side_data) { + uint8_t *cc_bytes = side_data->data; + int cc_count = side_data->size / CC_BYTES_PER_ENTRY; + ccf->cc_detected = 1; + + for (i = 0; i < cc_count; i++) { + /* See ANSI/CTA-708-E Sec 4.3, Table 3 */ + uint8_t cc_valid = (cc_bytes[CC_BYTES_PER_ENTRY*i] & 0x04) >> 2; + uint8_t cc_type = cc_bytes[CC_BYTES_PER_ENTRY*i] & 0x03; + if (cc_type == 0x00 || cc_type == 0x01) { + av_fifo_write(ccf->cc_608_fifo, &cc_bytes[CC_BYTES_PER_ENTRY*i], + CC_BYTES_PER_ENTRY); + } else if (cc_valid && (cc_type == 0x02 || cc_type == 0x03)) { + av_fifo_write(ccf->cc_708_fifo, &cc_bytes[CC_BYTES_PER_ENTRY*i], + CC_BYTES_PER_ENTRY); + } + } + printf("dequeue cc_filled=%d\n", cc_count); + /* Remove the side data, as we will re-create it on the + output as needed */ + av_frame_remove_side_data(frame, AV_FRAME_DATA_A53_CC); + } + } + return 0; +} diff --git a/libavutil/ccfifo.h b/libavutil/ccfifo.h new file mode 100644 index 0000000000..ad48c0a39c --- /dev/null +++ b/libavutil/ccfifo.h @@ -0,0 +1,85 @@ +/* + * CEA-708 Closed Captioning FIFO + * Copyright (c) 2023 LTN Global Communications + * + * Author: Devin Heitmueller + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * CC FIFO Buffer + */ + +#ifndef AVUTIL_CCFIFO_H +#define AVUTIL_CCFIFO_H + +#include "avutil.h" +#include "frame.h" +#include "fifo.h" + +typedef struct AVCCFifo AVCCFifo; + +/** + * Allocate an AVCCFifo. + * + * @param sample_fmt sample format + * @param channels number of channels + * @param nb_samples initial allocation size, in samples + * @return newly allocated AVCCFifo, or NULL on error + */ +AVCCFifo *av_ccfifo_alloc(AVRational *framerate, void *log_ctx); + +/** + * Free an AVCCFifo + * + * @param ccf Pointer to the pointer to the AVCCFifo which should be freed + * @note `*ptr = NULL` is safe and leads to no action. + */ +void av_ccfifo_freep(AVCCFifo **ccf); + + +/** + * Read a frame into a CC Fifo + * + * Extract CC bytes from the AVFrame, insert them into our queue, and + * remove the side data from the AVFrame. The side data is removed + * as it will be re-inserted at the appropriate rate later in the + * filter. + * + * @param af AVCCFifo to write to + * @param frame AVFrame with the video frame to operate on + * @return Zero on success, or negative AVERROR + * code on failure. + */ +int av_ccfifo_extract(AVCCFifo *af, AVFrame *frame); + +/** + * Insert CC data from the FIFO into an AVFrame (as side data) + * + * Dequeue the appropriate number of CC tuples based on the + * frame rate, and insert them into the AVFrame + * + * @param af AVCCFifo to read from + * @param frame AVFrame with the video frame to operate on + * @return Zero on success, or negative AVERROR + * code on failure. + */ +int av_ccfifo_inject(AVCCFifo *af, AVFrame *frame); + +#endif /* AVUTIL_CCFIFO_H */ From patchwork Fri Mar 17 20:09:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 40715 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:d046:b0:cd:afd7:272c with SMTP id hv6csp14844pzb; Fri, 17 Mar 2023 13:10:24 -0700 (PDT) X-Google-Smtp-Source: AK7set99hV8eON0q9MMQBJfcuS/Dxu/D1xYmJjs26YWQJJ3h+Y8Hm79OJaoqGre/Y8v0B91INLZ9 X-Received: by 2002:aa7:d504:0:b0:4af:601e:6039 with SMTP id y4-20020aa7d504000000b004af601e6039mr4314608edq.22.1679083824318; Fri, 17 Mar 2023 13:10:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1679083824; cv=none; d=google.com; s=arc-20160816; b=goSnDhFxQRkK5u47xBcmx9D60EeZuX4vnpopuJKCKax3Rxaj/ie46S1om1FaRO5J7O gXLKVdaTdQbQ+nH0UmTwp9gnS1/RUT5vFhAth4d3zFmeBwP5ogDJ7MpIBxQVFjQfarqb FzTDe1Y6f7f5sCOtnuwTK2LDkbJunsob4N2oARdyitozMkLjn8DTgd4Wpg0NFvN3dds0 wU5yj9dq4YMSTcQcgOZ4lLCMKQNl97ejNUKoMyxsY9Via7Nj8FwjEyFTB1lTfvhJhyJ+ Noy+IoreBlAVFwBgc2ovET7Xenw/792jMjR4id2NWgDmwVdc/qtICVhYqd3J5jZ9NjJd t3Vg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=hcxQZapIfB9EBl7f3v+zeTIdJ8VNYml2c3EUUJOHx7A=; b=zjQkdrbZ+j4RKa38yVVQEiHaHkHEw7YFDAh7LZY2yjQ07kTtd+X/PQIJNBuwqBQHjN MkXXeYMb2Ijmn+346APZ0m/xTC/eC1safcxqf8j050zf2B6x6hxg9UXrsarXrDyyApOC EmMO4A64Lq+rRGyjg6BbIFXvDuOiV+S9wtTej+LQuSPpdRYDBc67ElnIR4WadgaMoUFC KjA5DdDiy5DirbsTWBVe/2YXUpss5uEuotxPYXzs/vRlPxDSVyUuadHATDbbkSqjCj99 roKheI7VhkOzjrtdklzt45dj/SXEoWpVIt1M+s9IWL87GEhzn6kf6z4JiUQcwO+TtNsp WvnQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=vpyhBzYs; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id ay21-20020a056402203500b004fd1ee1380fsi3229813edb.612.2023.03.17.13.10.23; Fri, 17 Mar 2023 13:10:24 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=vpyhBzYs; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4088168C269; Fri, 17 Mar 2023 22:09:57 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qt1-f182.google.com (mail-qt1-f182.google.com [209.85.160.182]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1722168C226 for ; Fri, 17 Mar 2023 22:09:49 +0200 (EET) Received: by mail-qt1-f182.google.com with SMTP id x1so6940430qtr.7 for ; Fri, 17 Mar 2023 13:09:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20210112.gappssmtp.com; s=20210112; t=1679083787; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0bCQOPrmDxAb+8BhhkfSXibGnT/xgDHTuqXZh1Sn7iE=; b=vpyhBzYs6c0VVbg3alkEp+7WDvZFZdOxqvg4sXdC0jXpDydZ+4AV8DooiCREMT/Csj hDBAw1R+hg6/tYKrz/g+hfV1Uj70uiU7Gri4ahRMGHrMQrDnyJWtHJtUMpXgFyCWQEH+ VYJtHQRH53fBsTftq7Q1Cmz83lwUGTn5MwbEdQp3XHRcjevNCi0KmeG3Aal5k4R4r7jm qMsJkRsEOdQ4SGW+2h3NKAGTkYVsOUuRbHFNIxwGnL4bYqtZaGpG73fSc0SrCCCDVjuS 0Wptc7m/BJ3jQJpa5vf46ZvyXRMScDlzQN01lRtdriiO1VYyYiNM0sYTDStkm632Ydnq tQtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679083787; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0bCQOPrmDxAb+8BhhkfSXibGnT/xgDHTuqXZh1Sn7iE=; b=6npNRr/4uW9bMnp16WUJ3wnVSbXpLzzwcf0NiEZL2PRZjrdMxQmJRJJSSfqQ9ZWf8F ICxZW532/8jVHBqGV11NXiJa9DUSUpflwTcwbC8NL2IeMGxc26vyftVJTBc146G9k8HO cbTulCfNzuceSRDgiatVQKDAFQyjt+QBuW5QqVfoSdOadtiJESifTs02H2WQL1we9/mb 18BZumjytOTazDWOJnBLX6Xl6WvNoxHj5S60Fn9I9uDTH3HvEXyoX5wtydcBcNSrMOGA 5Gf4BCBX7fvJ6xjpYXKmLdM819oAjzx/oCZfSE+BeUiOxC8RrEpZ4L7JkZ9XxKF9w0Jf v3RQ== X-Gm-Message-State: AO0yUKXOBKjL7bcWHYYS/Ais3VUfR8ghudpfXKJCC0zvdtO5D+/Z1iOE We7MGxKHmeFZTjpKds/qNcRXMwYr4fRe4giCU0Q= X-Received: by 2002:a05:622a:44f:b0:3b8:340b:1aab with SMTP id o15-20020a05622a044f00b003b8340b1aabmr13690600qtx.25.1679083787639; Fri, 17 Mar 2023 13:09:47 -0700 (PDT) Received: from klab-nyc-kernellabs2.localdomain (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id j5-20020ac86645000000b003bd0e7ff466sm2065685qtp.7.2023.03.17.13.09.47 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 17 Mar 2023 13:09:47 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 17 Mar 2023 16:09:38 -0400 Message-Id: <20230317200941.3936-3-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 2.35.1.655.ga68dfadae5 In-Reply-To: <20230317200941.3936-1-dheitmueller@ltnglobal.com> References: <20230317200941.3936-1-dheitmueller@ltnglobal.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [RFC PATCH 2/5] vf_fps: properly preserve CEA-708 captions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Tzkbqp3aGqUf The existing implementation made an attempt to remove duplicate captions if increasing the framerate, but made no attempt to handle reducing the framerate, nor did it rewrite the caption payloads to have the appropriate cc_count (e.g. the cc_count needs to change from 20 to 10 when going from 1080i59 to 720p59 and vice-versa). Make use of the new ccfifo mechanism to ensure that caption data is properly preserved. Signed-off-by: Devin Heitmueller --- libavfilter/vf_fps.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/libavfilter/vf_fps.c b/libavfilter/vf_fps.c index 051d278f54..d2fab09c66 100644 --- a/libavfilter/vf_fps.c +++ b/libavfilter/vf_fps.c @@ -30,6 +30,7 @@ #include #include "libavutil/avassert.h" +#include "libavutil/ccfifo.h" #include "libavutil/eval.h" #include "libavutil/mathematics.h" #include "libavutil/opt.h" @@ -85,6 +86,7 @@ typedef struct FPSContext { AVFrame *frames[2]; ///< buffered frames int frames_count; ///< number of buffered frames + AVCCFifo *cc_fifo; ///< closed captions int64_t next_pts; ///< pts of the next frame to output @@ -165,6 +167,7 @@ static av_cold void uninit(AVFilterContext *ctx) frame = shift_frame(ctx, s); av_frame_free(&frame); } + av_ccfifo_freep(&s->cc_fifo); av_log(ctx, AV_LOG_VERBOSE, "%d frames in, %d frames out; %d frames dropped, " "%d frames duplicated.\n", s->frames_in, s->frames_out, s->drop, s->dup); @@ -210,6 +213,9 @@ static int config_props(AVFilterLink* outlink) s->in_pts_off, s->out_pts_off, s->start_time); } + if (!(s->cc_fifo = av_ccfifo_alloc(&outlink->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue. Captions will be passed through\n"); + av_log(ctx, AV_LOG_VERBOSE, "fps=%d/%d\n", outlink->frame_rate.num, outlink->frame_rate.den); return 0; @@ -242,6 +248,7 @@ static int read_frame(AVFilterContext *ctx, FPSContext *s, AVFilterLink *inlink, av_log(ctx, AV_LOG_DEBUG, "Read frame with in pts %"PRId64", out pts %"PRId64"\n", in_pts, frame->pts); + av_ccfifo_extract(s->cc_fifo, frame); s->frames[s->frames_count++] = frame; s->frames_in++; @@ -289,7 +296,7 @@ static int write_frame(AVFilterContext *ctx, FPSContext *s, AVFilterLink *outlin if (!frame) return AVERROR(ENOMEM); // Make sure Closed Captions will not be duplicated - av_frame_remove_side_data(s->frames[0], AV_FRAME_DATA_A53_CC); + av_ccfifo_inject(s->cc_fifo, frame); frame->pts = s->next_pts++; frame->duration = 1; From patchwork Fri Mar 17 20:09:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 40716 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:d046:b0:cd:afd7:272c with SMTP id hv6csp15331pzb; Fri, 17 Mar 2023 13:11:11 -0700 (PDT) X-Google-Smtp-Source: AK7set+PJOR0dJtUhimUAey6LJbY9XuAFCk5x7q8gsvpV+SEkMXjCtuL9S8KpqTw7ppkIQZ49NDf X-Received: by 2002:a17:906:36d4:b0:932:d2bb:507 with SMTP id b20-20020a17090636d400b00932d2bb0507mr688005ejc.24.1679083870944; Fri, 17 Mar 2023 13:11:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1679083870; cv=none; d=google.com; s=arc-20160816; b=hyYHS314PD6bw9i5TvzfdbM6tNzw0Xe2NbTXhesiZoFg83YdHdPlc2b2MaOjpMLM7V nMwNEQz7FBHt0MX2aD+a0Wyo2AeI5RcCzZ2cVEw03dxJqI/zodr8+nmKwGENBMu+RBKU JmyEO73iCW/1D/VS2CQKNkZrxbSdsF1Weh8XQsXQsKnQNkTUwhzRGJAEW40bmz34ak7q rPmvPTj+gUTMHn8EItG6pFroSJEn9mjlLLdobGAwyRIcrjaHE/tvXA4bqIM6DGnaMmrD quLhgdcshW6vgjwfeiaAnnwD/iXB/LEzhmzg41g2cHxnAhUik3el6068Tvd6oF9Mue2/ 6vXw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=66S4oqhr3CX2BaIXqPjuQpjJH5nc87h5h1njzf+OkqE=; b=xr70pK3oZmPNtmjR9N+hJbkLfzo9Jt7E6XV0MkeEudwVO1vcjxCEIz2aJoevNhZI7f VvivKPF9kv8eBFyUtVdrttF8Nw8ALTm6BRlbML4pM10oz/3rNsChxgQd9v6ElkmrL5tU 6AeVtFeQKMWWup/1buwYyAe+qbrhcU9cFt6z9Lf7oeX7D5aUWCDE69MvXSHvOF8JKJTG mpQgywPqnAXRocqQ5b5YcIGArR6KZvqsIZScEPpA/8ToCK0eFyB615ICeC34Yo2ttihH 5dbDy73AsRCRaQhoEel+f7pqm830PqhZEfyHlFGFhXmCclovYtEnOMlJGHAZAYsZg+Jg 2s0Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=Yj0akOxZ; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id y24-20020a170906071800b009317f50118fsi2633434ejb.621.2023.03.17.13.10.45; Fri, 17 Mar 2023 13:11:10 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=Yj0akOxZ; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5B94668C26F; Fri, 17 Mar 2023 22:09:58 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qt1-f173.google.com (mail-qt1-f173.google.com [209.85.160.173]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A8D3C68C24F for ; Fri, 17 Mar 2023 22:09:49 +0200 (EET) Received: by mail-qt1-f173.google.com with SMTP id i24so6958270qtm.6 for ; Fri, 17 Mar 2023 13:09:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20210112.gappssmtp.com; s=20210112; t=1679083788; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XEjn1HFG6pYOq83kHK5HbwpVqXMIOBSsp9h3emu4OCY=; b=Yj0akOxZ4sI7Q5CG8srKrDybxKrW6rrlrZrrUtJcegGMgkI4u4xvaezC6ok3BSiez6 xDqaugDb6PZwroRHpsDeP3RncDc7GJrpEeT9QkmRKAQrkBjz36IS0G7O5r+pUYYCZnrc 5HXIHWDk3ic8bPxiBihocPEBR8eg3sS5Vfbq8Kip4qWYa+ZwRlDF2nxY79pj001keZD9 HpJv/iLSqx/aEsQ1MNPCB1QSp8mS+7944Vf/pXNxouwU3kq2xaipm/uzE4tSciz0noaK o6fZgs6SPrWoexnk9RRoYuOTikB4kt4rjbBeWJMxGVLqglLeIlKxm6yZlPjGhK88uZMF lMog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679083788; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XEjn1HFG6pYOq83kHK5HbwpVqXMIOBSsp9h3emu4OCY=; b=NQfSiiKCn3V674C2GOPxIg1ARXdIa2kUkKw+2M1TONXTkWn7WyIxkq37LpONHoDZts o16Cn5DCDcuyBMs/G08TIS+tsQ6PMKmiWHRHjDbucLNMo+WwiYs26raVe2p8dXtI+Al1 AbfC10vdKSBuHhe/hPQye2ruLl5FxHJROL/RxZmK2EQpQBN7RlkSUJgdVOPpcZsPqShb Acvl4EsE7nNjfXdJGtYZSMABmaDcdBlfOzDzpaAVvPSA6DZyymoy8HcSAKOpEXxJP1rX ovpjYwnzemsv/4D/qFva0WelOPB3L/g49EhsaFfYmRH8uWrENDjPqH2YAKpnhNNiHqEJ oYuw== X-Gm-Message-State: AO0yUKX59jSAjxsv8dXmcQ7uE42PyTaypoNOsbvYk0fj0VTdop1LzEgk KRmNNaw5cHGarsO5OJGoWxnMQL8DJB7YzpuaHIo= X-Received: by 2002:a05:622a:4e:b0:3b9:b112:e86b with SMTP id y14-20020a05622a004e00b003b9b112e86bmr17347253qtw.13.1679083788285; Fri, 17 Mar 2023 13:09:48 -0700 (PDT) Received: from klab-nyc-kernellabs2.localdomain (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id j5-20020ac86645000000b003bd0e7ff466sm2065685qtp.7.2023.03.17.13.09.47 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 17 Mar 2023 13:09:47 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 17 Mar 2023 16:09:39 -0400 Message-Id: <20230317200941.3936-4-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 2.35.1.655.ga68dfadae5 In-Reply-To: <20230317200941.3936-1-dheitmueller@ltnglobal.com> References: <20230317200941.3936-1-dheitmueller@ltnglobal.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [RFC PATCH 3/5] yadif: Properly preserve CEA-708 closed captions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: ngcJjOc1ZcFp Various deinterlacing modes have the effect of doubling the framerate, and we need to ensure that the caption data isn't duplicated (or else you get double captions on-screen). Use the new ccfifo mechanism for yadif (and yadif_cuda and bwdif since they use the same yadif core) so that CEA-708 data is properly preserved through this filter. Signed-off-by: Devin Heitmueller --- libavfilter/vf_bwdif.c | 7 +++++++ libavfilter/vf_yadif.c | 6 ++++++ libavfilter/vf_yadif_cuda.c | 8 ++++++++ libavfilter/yadif.h | 2 ++ libavfilter/yadif_common.c | 5 +++++ 5 files changed, 28 insertions(+) diff --git a/libavfilter/vf_bwdif.c b/libavfilter/vf_bwdif.c index 65c617ebb3..de937d83cf 100644 --- a/libavfilter/vf_bwdif.c +++ b/libavfilter/vf_bwdif.c @@ -297,6 +297,7 @@ static av_cold void uninit(AVFilterContext *ctx) av_frame_free(&yadif->prev); av_frame_free(&yadif->cur ); av_frame_free(&yadif->next); + av_ccfifo_freep(&yadif->cc_fifo); } static const enum AVPixelFormat pix_fmts[] = { @@ -332,6 +333,12 @@ static int config_props(AVFilterLink *link) if(yadif->mode&1) link->frame_rate = av_mul_q(link->src->inputs[0]->frame_rate, (AVRational){2,1}); + else + link->frame_rate = ctx->inputs[0]->frame_rate; + + if (!(yadif->cc_fifo = av_ccfifo_alloc(&link->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue. Captions will be passed through\n"); + if (link->w < 3 || link->h < 4) { av_log(ctx, AV_LOG_ERROR, "Video of less than 3 columns or 4 lines is not supported\n"); diff --git a/libavfilter/vf_yadif.c b/libavfilter/vf_yadif.c index 1be02de1a9..b51d21f6ff 100644 --- a/libavfilter/vf_yadif.c +++ b/libavfilter/vf_yadif.c @@ -261,6 +261,7 @@ static av_cold void uninit(AVFilterContext *ctx) av_frame_free(&yadif->prev); av_frame_free(&yadif->cur ); av_frame_free(&yadif->next); + av_ccfifo_freep(&yadif->cc_fifo); } static const enum AVPixelFormat pix_fmts[] = { @@ -293,6 +294,11 @@ static int config_output(AVFilterLink *outlink) if(s->mode & 1) outlink->frame_rate = av_mul_q(ctx->inputs[0]->frame_rate, (AVRational){2, 1}); + else + outlink->frame_rate = ctx->inputs[0]->frame_rate; + + if (!(s->cc_fifo = av_ccfifo_alloc(&outlink->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue. Captions will be passed through\n"); if (outlink->w < 3 || outlink->h < 3) { av_log(ctx, AV_LOG_ERROR, "Video of less than 3 columns or lines is not supported\n"); diff --git a/libavfilter/vf_yadif_cuda.c b/libavfilter/vf_yadif_cuda.c index 685b8a2035..d96ec391a0 100644 --- a/libavfilter/vf_yadif_cuda.c +++ b/libavfilter/vf_yadif_cuda.c @@ -206,6 +206,9 @@ static av_cold void deint_cuda_uninit(AVFilterContext *ctx) av_frame_free(&y->cur); av_frame_free(&y->next); + if (yadif->cc_fifo) + av_cc_fifo_free(yadif->cc_fifo); + av_buffer_unref(&s->device_ref); s->hwctx = NULL; av_buffer_unref(&s->input_frames_ref); @@ -291,6 +294,11 @@ static int config_output(AVFilterLink *link) if(y->mode & 1) link->frame_rate = av_mul_q(ctx->inputs[0]->frame_rate, (AVRational){2, 1}); + else + outlink->frame_rate = ctx->inputs[0]->frame_rate; + + if (!(s->cc_fifo = av_cc_fifo_alloc(&outlink->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue. Captions will be passed through\n"); if (link->w < 3 || link->h < 3) { av_log(ctx, AV_LOG_ERROR, "Video of less than 3 columns or lines is not supported\n"); diff --git a/libavfilter/yadif.h b/libavfilter/yadif.h index c928911b35..ccd4304860 100644 --- a/libavfilter/yadif.h +++ b/libavfilter/yadif.h @@ -21,6 +21,7 @@ #include "libavutil/opt.h" #include "libavutil/pixdesc.h" +#include "libavutil/ccfifo.h" #include "avfilter.h" enum YADIFMode { @@ -76,6 +77,7 @@ typedef struct YADIFContext { int eof; uint8_t *temp_line; int temp_line_size; + AVCCFifo *cc_fifo; /* * An algorithm that treats first and/or last fields in a sequence diff --git a/libavfilter/yadif_common.c b/libavfilter/yadif_common.c index a10cf7a17f..f95b8c10d3 100644 --- a/libavfilter/yadif_common.c +++ b/libavfilter/yadif_common.c @@ -43,6 +43,7 @@ static int return_frame(AVFilterContext *ctx, int is_second) return AVERROR(ENOMEM); av_frame_copy_props(yadif->out, yadif->cur); + av_ccfifo_inject(yadif->cc_fifo, yadif->out); yadif->out->interlaced_frame = 0; if (yadif->current_field == YADIF_FIELD_BACK_END) yadif->current_field = YADIF_FIELD_END; @@ -96,6 +97,8 @@ int ff_yadif_filter_frame(AVFilterLink *link, AVFrame *frame) av_assert0(frame); + av_ccfifo_extract(yadif->cc_fifo, frame); + if (yadif->frame_pending) return_frame(ctx, 1); @@ -137,6 +140,7 @@ int ff_yadif_filter_frame(AVFilterLink *link, AVFrame *frame) if (!yadif->out) return AVERROR(ENOMEM); + av_ccfifo_inject(yadif->cc_fifo, yadif->out); av_frame_free(&yadif->prev); if (yadif->out->pts != AV_NOPTS_VALUE) yadif->out->pts *= 2; @@ -148,6 +152,7 @@ int ff_yadif_filter_frame(AVFilterLink *link, AVFrame *frame) return AVERROR(ENOMEM); av_frame_copy_props(yadif->out, yadif->cur); + av_ccfifo_inject(yadif->cc_fifo, yadif->out); yadif->out->interlaced_frame = 0; if (yadif->out->pts != AV_NOPTS_VALUE) From patchwork Fri Mar 17 20:09:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 40717 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:d046:b0:cd:afd7:272c with SMTP id hv6csp15367pzb; Fri, 17 Mar 2023 13:11:13 -0700 (PDT) X-Google-Smtp-Source: AK7set9GmBvnVgCIIiyAgD2/qW0H2AlCArfXqaR13ZYvBB52mIY/mOB+0kA7gk3EL+BHbRelr1Fg X-Received: by 2002:a50:fe81:0:b0:4fb:5a36:1539 with SMTP id d1-20020a50fe81000000b004fb5a361539mr3846539edt.25.1679083873744; Fri, 17 Mar 2023 13:11:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1679083873; cv=none; d=google.com; s=arc-20160816; b=l781EvkYcp8O3mwrElllCn7zCE5CM6Rn10h6KJXsT6XmuR6i8ZME4LwWmHxkY9srFt PzMgKqeOB71k0zkUXwwzM5wbNtTa9hhW7XA8wAkg1f8WwyKb6mLyU97jhdUkHnyXrMzF K0X7KnKuVXPJHOUP6vA9CQyYDgTg8ztzeXoiYXzdFRGFXZXwC5JSqq3y66MfYwcSXF10 57buBniUtC4iSl4SBV9CpD/pW19PbZVY4bCOdZOGnBOefrJSHmdZUyCWDN+c+OLxmd73 ICLbvSgEw62OPRUJpVLLHkv8enwsqW/jB2l4GOum5jNT8FrCjCGWxN+7TiierZlsLk4s FVGw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=BPYRTCjp1JkAXKfV+urP/css97O5YWfiz6PCjBB+Z64=; b=E2yc6XdrO4oQ54/fLVtOxXmerJ2ZV2TGRnPnhWqPM9UaTcIfAmlMw96jk3MxEBGCwt DtMxvECTlhZsHXNNXl6B+U3XN9JsaQf9IFTUz57jV6qmQ+fT69VNPJxr4XZxrnA7hFm7 BM8FxEHzST8Kl3s4kOzGHnPsfOHDMlTQtEbHtauECrl1zyER0Zx7s/mCHuGp0y4xlv1G 5gx1OP8PGD5rZTp5zJ7sa1v7dmNRwmL2coUWM8rN4gP3Gf1siowTq5qjZ3WhthQuQFW3 JRVvLGhfpI8WG4I90xT//NoeRWZX+LsQN918ytAxtpQmao4IKZ1rv2KprAb1lSkO7gPC W6mA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=xZfaDC4h; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id n15-20020aa7db4f000000b004fa39a6af22si3445837edt.445.2023.03.17.13.10.51; Fri, 17 Mar 2023 13:11:13 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=xZfaDC4h; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 664D768C277; Fri, 17 Mar 2023 22:09:59 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qt1-f181.google.com (mail-qt1-f181.google.com [209.85.160.181]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6868568C257 for ; Fri, 17 Mar 2023 22:09:50 +0200 (EET) Received: by mail-qt1-f181.google.com with SMTP id i24so6958304qtm.6 for ; Fri, 17 Mar 2023 13:09:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20210112.gappssmtp.com; s=20210112; t=1679083789; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qWr2BIavd2S5WylSPqSwJnA/Bfz+SM9WD8XI0N1PtRo=; b=xZfaDC4hb5np7RI5JEhBl4gzi4cJnjQlPYLOntmvBCEkzZPfHv/OGCeuc8Bd9iexl7 UAeMed2a5BkpzSejDu4MbA604q9dKQ+45ISdGbnRJtLcQjEzZnDCX5Fo4T7Co0Su9cQp 47Zawgc5N5s/gjck1pINjFG3Sf6Edl1JOagYYaRQ2diuxlU3Z3FrhdUbjmU1fXl/cGtZ gVauoBP+a1IFDKPhOaNXt9j76ePVLqRMEdwzxV/KEy+5TaV9DhNC2uh6TH8nm4cnSog3 QOVNLT/x7n46M+wfP6aFvWmjMAwi6cMD77aBWsTLfPE5aJSryXhHgDIDhq5E1pI5ZTry 3xbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679083789; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qWr2BIavd2S5WylSPqSwJnA/Bfz+SM9WD8XI0N1PtRo=; b=dAFp/nzn5/5AcJF2aSyBoIn+49UKZ4otKHplaWSUXPlMGAQrAiFuyNYY14/9Q2oXSw tNoeK+Fsilpc5cuG2K7eVAUCO8G3dMztW7Sn4Fo1m3olREIIuarLI+0R/oTQYzH1BjRN +/AExenw05yR9E2Q/0+kgRVOuva3CEYwq2E5/POKCiWPoUz03arPuNoTFCxOqIFa/AgE tZMAqKQ8rWqtVHbxQngjlQmtuzWwpWEKhMiuDilXxj9g1KESuPYejRzQx2+jxJcfgUqG eOO8Km92hc28UT7NCSLTo1cQVRmKNoHeuz+TXtg1/aeJtm8/Sj6Urni1UIoYSN3OLlIO RHwA== X-Gm-Message-State: AO0yUKUQsviWSGoG0QCDrU+q/pAgwOQRfP5iM8FF6FjZgdKExLZD789j 6iLVXU43mfWAZRyRBVVQ7Yc6qQv7RydrVlgy3v0= X-Received: by 2002:a05:622a:86:b0:3dc:38e:8f5 with SMTP id o6-20020a05622a008600b003dc038e08f5mr3034845qtw.49.1679083788981; Fri, 17 Mar 2023 13:09:48 -0700 (PDT) Received: from klab-nyc-kernellabs2.localdomain (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id j5-20020ac86645000000b003bd0e7ff466sm2065685qtp.7.2023.03.17.13.09.48 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 17 Mar 2023 13:09:48 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 17 Mar 2023 16:09:40 -0400 Message-Id: <20230317200941.3936-5-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 2.35.1.655.ga68dfadae5 In-Reply-To: <20230317200941.3936-1-dheitmueller@ltnglobal.com> References: <20230317200941.3936-1-dheitmueller@ltnglobal.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [RFC PATCH 4/5] tinterlace: Properly preserve CEA-708 closed captions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: MQrDtuSFO96I Because the interlacing filter halves the effective framerate, we need to ensure that no CEA-708 data is lost as frames are merged. Make use of the new ccfifo mechanism to ensure that caption data is properly preserved as frames pass through the filter. Signed-off-by: Devin Heitmueller --- libavfilter/tinterlace.h | 2 ++ libavfilter/vf_tinterlace.c | 10 ++++++++++ 2 files changed, 12 insertions(+) diff --git a/libavfilter/tinterlace.h b/libavfilter/tinterlace.h index 37b6c10c08..30574c2ebf 100644 --- a/libavfilter/tinterlace.h +++ b/libavfilter/tinterlace.h @@ -30,6 +30,7 @@ #include "libavutil/bswap.h" #include "libavutil/opt.h" #include "libavutil/pixdesc.h" +#include "libavutil/ccfifo.h" #include "drawutils.h" #include "avfilter.h" @@ -77,6 +78,7 @@ typedef struct TInterlaceContext { const AVPixFmtDescriptor *csp; void (*lowpass_line)(uint8_t *dstp, ptrdiff_t width, const uint8_t *srcp, ptrdiff_t mref, ptrdiff_t pref, int clip_max); + AVCCFifo *cc_fifo; } TInterlaceContext; void ff_tinterlace_init_x86(TInterlaceContext *interlace); diff --git a/libavfilter/vf_tinterlace.c b/libavfilter/vf_tinterlace.c index 032629279a..260386a889 100644 --- a/libavfilter/vf_tinterlace.c +++ b/libavfilter/vf_tinterlace.c @@ -291,6 +291,9 @@ static int config_out_props(AVFilterLink *outlink) #endif } + if (!(tinterlace->cc_fifo = av_ccfifo_alloc(&outlink->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue. Captions will be passed through\n"); + av_log(ctx, AV_LOG_VERBOSE, "mode:%d filter:%s h:%d -> h:%d\n", tinterlace->mode, (tinterlace->flags & TINTERLACE_FLAG_CVLPF) ? "complex" : (tinterlace->flags & TINTERLACE_FLAG_VLPF) ? "linear" : "off", @@ -375,6 +378,8 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) tinterlace->cur = tinterlace->next; tinterlace->next = picref; + av_ccfifo_extract(tinterlace->cc_fifo, picref); + cur = tinterlace->cur; next = tinterlace->next; /* we need at least two frames */ @@ -390,6 +395,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) if (!out) return AVERROR(ENOMEM); av_frame_copy_props(out, cur); + av_ccfifo_inject(tinterlace->cc_fifo, out); out->height = outlink->h; out->interlaced_frame = 1; out->top_field_first = 1; @@ -423,6 +429,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) if (!out) return AVERROR(ENOMEM); av_frame_copy_props(out, cur); + av_ccfifo_inject(tinterlace->cc_fifo, out); out->height = outlink->h; out->sample_aspect_ratio = av_mul_q(cur->sample_aspect_ratio, av_make_q(2, 1)); @@ -459,6 +466,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) if (!out) return AVERROR(ENOMEM); av_frame_copy_props(out, cur); + av_ccfifo_inject(tinterlace->cc_fifo, out); out->interlaced_frame = 1; out->top_field_first = tff; @@ -481,6 +489,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) out = av_frame_clone(cur); if (!out) return AVERROR(ENOMEM); + av_ccfifo_inject(tinterlace->cc_fifo, out); out->interlaced_frame = 1; if (cur->pts != AV_NOPTS_VALUE) out->pts = cur->pts*2; @@ -495,6 +504,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) if (!out) return AVERROR(ENOMEM); av_frame_copy_props(out, next); + av_ccfifo_inject(tinterlace->cc_fifo, out); out->interlaced_frame = 1; out->top_field_first = !tff; From patchwork Fri Mar 17 20:09:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 40718 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:d046:b0:cd:afd7:272c with SMTP id hv6csp15433pzb; Fri, 17 Mar 2023 13:11:19 -0700 (PDT) X-Google-Smtp-Source: AK7set/aV6a7Tf5DXwnxkRN01SJZuipB6vYTdwrWOtm00eAm06f7FguvDfgfxqhhm/1v5aSU598R X-Received: by 2002:a17:906:37d3:b0:92f:d1ec:a7d7 with SMTP id o19-20020a17090637d300b0092fd1eca7d7mr604126ejc.15.1679083878982; Fri, 17 Mar 2023 13:11:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1679083878; cv=none; d=google.com; s=arc-20160816; b=meCEpIuV1EGhBPuR7b0apYNVtpTSVzRBV5JQwpNj3s9WhvFGjiJEO/36/MpB9xkXlx A3qHSeMw2OeJzMOYfS1N/hvEjEMyO+3GbDqen/YpJbEWZfzgDVifEFQqUZGpne0JVzrn KnX4fRy1BLlYXKtNdKpGo60bvKHToDjJrcK/C1TcDNszYpZSxOFu3hHZtdkVLbN5lr+N AK/QoFZlvuINcWyu6XMpI/hebT1761bFLIwJshY/sud6RmfnoBMFj1791cPIvXelvxkc cBqN9VHMUqAFWeFXDSJCtPBSjk4R2fxWjImOdJTmgRAeMRlbJOoAnECnABPWCKtmgBG3 gd7A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=PaSny6V6LTBsumHzr+0L9Rc93Nxz0cdvTUF4iHBZR2Q=; b=Ry48/ZXjJppxzs5/n/+hps1X354FsmkOadsQm//ejBt5uyfuqt8EwtQqYX3CN8B8AS XKUkdDKputLTrEFyRbh+LpSG+HK8H4TLGlIYSRE0D9wcD9ElNGMqKDN2IWpXr1xURPf0 mmI3JDTXg2SkgYdG9Z2AVqqlyHyaypCTE+pUuPChzesUq+XsyQb3XHtw31QphyDc+LdB fyMwNVHxSHVPIYwWjt90It/BlD5KaWT8cYUmQeuXtns161QgVnkFOYgHA29FH/wotxdA 1SPspqkfCzRGdWgWcREJh47oRNwuZp9uaygpAMKDEWS5/ePzN7vHK0q9vEmdHVYBzXV7 SQig== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=c8Lc+K3o; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id x11-20020a170906148b00b008c8f5756068si3125255ejc.766.2023.03.17.13.10.58; Fri, 17 Mar 2023 13:11:18 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=c8Lc+K3o; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6119568C280; Fri, 17 Mar 2023 22:10:00 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qt1-f179.google.com (mail-qt1-f179.google.com [209.85.160.179]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 33C4868C24D for ; Fri, 17 Mar 2023 22:09:51 +0200 (EET) Received: by mail-qt1-f179.google.com with SMTP id fy17so4374841qtb.2 for ; Fri, 17 Mar 2023 13:09:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20210112.gappssmtp.com; s=20210112; t=1679083790; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=aci2xh9xWgYUWVbVC1jrQ/1QbeRYXIlHza+nQaMFd9U=; b=c8Lc+K3o8fnbZaT3gWIxCxGPegWd8Ifwx9hosd+gFen9ulrj4BHWbXgHDz7m7nslje r1eK2igT3QSy+sJPUpUVK0CXMNdJpAW+X2IAyEBO4jVY88NqTQlTN8m0Urq7TD91l2oN WwYphSXsZRWh47Xl5d4aurVCGuSIQ63yDBjBwurg8QBfu7wrMs1XasLVdp5CvuyEW2zU G3AGhAVtxH4VA19ZbY1YOLFGbzv9/0J3qU2/Ppsz1opA6Gn8ds4ryGHL90LCONcSxLCP 2k2XuiSMB3EjLcoZ8QaDzgnFwk/lHQQ0S31HfdU6AMkwOXAG9cWTghpSf79gTgX1Z9NG os+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679083790; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=aci2xh9xWgYUWVbVC1jrQ/1QbeRYXIlHza+nQaMFd9U=; b=JTSjFx1ZpSoM3sQQk8j5RPV5YY8fPFS167Tugt8aeJNPHAAm2L3HNGy2xxkTyOaQJp nRZKdhRMmOhi+odYhxvXSC2nAy6jb5mnd1InX0Z4pM427MZXl/rhyPtbMy5TakXzfWzy kt2gCKsEuSJh4SWkMamzR/iYnl5vBwsEc+V7QtTKw1dP6sUrwrCofSVkD2Idtghc23G+ QUbfLI105GResUS2EfhTI3CvGicTyZpXTmhA7hOHmx50sA7Y4nk3S9fLamYSigbf+ZR1 wgd+N/7Fu6TVh+iSIR49r6ylzKWHlExPUqokIaoWHSStgTmrAXmHdhiH5j3ojnp9/wZw pzFA== X-Gm-Message-State: AO0yUKUf8aMgoQMY0Ezo9uLdDr2txyquXicvdY9Ax6epu927CycP1Kg1 gYg6Y3drrAeVFj8ISOMah7/474tC3JnMWtubXbY= X-Received: by 2002:a05:622a:1a9d:b0:3bf:c3e8:c19b with SMTP id s29-20020a05622a1a9d00b003bfc3e8c19bmr15868865qtc.22.1679083789750; Fri, 17 Mar 2023 13:09:49 -0700 (PDT) Received: from klab-nyc-kernellabs2.localdomain (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id j5-20020ac86645000000b003bd0e7ff466sm2065685qtp.7.2023.03.17.13.09.49 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 17 Mar 2023 13:09:49 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 17 Mar 2023 16:09:41 -0400 Message-Id: <20230317200941.3936-6-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 2.35.1.655.ga68dfadae5 In-Reply-To: <20230317200941.3936-1-dheitmueller@ltnglobal.com> References: <20230317200941.3936-1-dheitmueller@ltnglobal.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [RFC PATCH 5/5] vf_ccrepack: Add new filter to repack CEA-708 side data X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: GvKlcz9ZZJ3c THis filter can correct certain issues seen from upstream sources where the cc_count is not properly set or the CEA-608 tuples are not at the start of the payload as expected. Make use of the ccfifo to extract and immediately repack the CEA-708 side data, thereby removing any extra padding and ensuring the 608 tuples are at the front of the payload. Signed-off-by: Devin Heitmueller --- doc/filters.texi | 10 +++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/vf_ccrepack.c | 95 +++++++++++++++++++++++++++++++++++++++ 4 files changed, 107 insertions(+) create mode 100644 libavfilter/vf_ccrepack.c diff --git a/doc/filters.texi b/doc/filters.texi index 77b594f69c..a84fef0e4d 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -8904,6 +8904,16 @@ Only deinterlace frames marked as interlaced. The default value is @code{all}. @end table +@section ccrepack + +Repack CEA-708 closed captioning side data + +This filter fixes various issues seen with commerical encoders +related to upstream malformed CEA-708 payloads, specifically +incorrect number of tuples (wrong cc_count for the target FPS), +and incorrect ordering of tuples (i.e. the CEA-608 tuples are not at +the first entries in the payload). + @section cas Apply Contrast Adaptive Sharpen filter to video stream. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index b3d3d981dd..eed2460ffc 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -211,6 +211,7 @@ OBJS-$(CONFIG_BOXBLUR_OPENCL_FILTER) += vf_avgblur_opencl.o opencl.o \ opencl/avgblur.o boxblur.o OBJS-$(CONFIG_BWDIF_FILTER) += vf_bwdif.o yadif_common.o OBJS-$(CONFIG_CAS_FILTER) += vf_cas.o +OBJS-$(CONFIG_CCREPACK_FILTER) += vf_ccrepack.o OBJS-$(CONFIG_CHROMABER_VULKAN_FILTER) += vf_chromaber_vulkan.o vulkan.o vulkan_filter.o OBJS-$(CONFIG_CHROMAHOLD_FILTER) += vf_chromakey.o OBJS-$(CONFIG_CHROMAKEY_FILTER) += vf_chromakey.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index d7db46c2af..b38550b358 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -196,6 +196,7 @@ extern const AVFilter ff_vf_boxblur; extern const AVFilter ff_vf_boxblur_opencl; extern const AVFilter ff_vf_bwdif; extern const AVFilter ff_vf_cas; +extern const AVFilter ff_vf_ccrepack; extern const AVFilter ff_vf_chromaber_vulkan; extern const AVFilter ff_vf_chromahold; extern const AVFilter ff_vf_chromakey; diff --git a/libavfilter/vf_ccrepack.c b/libavfilter/vf_ccrepack.c new file mode 100644 index 0000000000..ab8473e7c1 --- /dev/null +++ b/libavfilter/vf_ccrepack.c @@ -0,0 +1,95 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/* + * Repackage CEA-708 arrays, which deals with incorrect cc_count for a given + * output framerate, and incorrect 708 padding. + * + * See CEA CEA-10-A "EIA-708-B Implementation Guidance", Section 26.5 + * "Grouping DTVCC Data Within user_data() Structure" + */ + +#include "avfilter.h" +#include "internal.h" +#include "libavutil/opt.h" +#include "libavutil/ccfifo.h" + +typedef struct CCRepackContext +{ + const AVClass *class; + AVCCFifo *cc_fifo; +} CCRepackContext; + +static const AVOption ccrepack_options[] = { + { NULL } +}; + +AVFILTER_DEFINE_CLASS(ccrepack); + +static int config_input(AVFilterLink *link) +{ + CCRepackContext *ctx = link->dst->priv; + + if (!(ctx->cc_fifo = av_ccfifo_alloc(&link->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue. Captions will be passed through\n"); + + return 0; +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *frame) +{ + CCRepackContext *ctx = inlink->dst->priv; + AVFilterLink *outlink = inlink->dst->outputs[0]; + + av_ccfifo_extract(ctx->cc_fifo, frame); + av_ccfifo_inject(ctx->cc_fifo, frame); + + return ff_filter_frame(outlink, frame); +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + CCRepackContext *s = ctx->priv; + av_ccfifo_freep(&s->cc_fifo); +} + +static const AVFilterPad avfilter_vf_ccrepack_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .filter_frame = filter_frame, + .config_props = config_input, + }, +}; + +static const AVFilterPad avfilter_vf_ccrepack_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + }, +}; + +AVFilter ff_vf_ccrepack = { + .name = "ccrepack", + .description = NULL_IF_CONFIG_SMALL("Repack CEA-708 closed caption metadata"), + .uninit = uninit, + .priv_size = sizeof(CCRepackContext), + .priv_class = &ccrepack_class, + FILTER_INPUTS(avfilter_vf_ccrepack_inputs), + FILTER_OUTPUTS(avfilter_vf_ccrepack_outputs), +};