From patchwork Fri Apr 21 14:24:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 41288 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4645:b0:e3:3194:9d20 with SMTP id eb5csp1312711pzb; Fri, 21 Apr 2023 06:29:43 -0700 (PDT) X-Google-Smtp-Source: AKy350aNuPbcZH4vwFTawqz9jPTKRssh3jsm+KY42Pdg+XTCUNChdmKh58fWIJPqXJNxhOKfEvmj X-Received: by 2002:a17:906:f6c8:b0:947:5acb:920c with SMTP id jo8-20020a170906f6c800b009475acb920cmr2268252ejb.34.1682083783415; Fri, 21 Apr 2023 06:29:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682083783; cv=none; d=google.com; s=arc-20160816; b=iKxryW2rDY7iXupdfJzziYa8x5VED47eUGoh1nHdN4Qo92lKFGMUbYc2adX7OlsQMY 7oYwdUUBSu9IH/v//ls+XD1xtSavGLDfG4gS9w9gtyz8QAA+TYhUbxJE3qnypGZDVXxJ pu07ByguG4e/+fjvIyXJxGCouLyy3xLV19HjclOopw63wu0isok3UM1OiQO5CiiB42es Erxzb41KKAYCUdFF2Vpg5zTDfPCz1ch2gVhMU1UZv9Na882UZkf5pGmPlXgy7pJkp5Fe GweIULYYjhSFHkCBD4agVmygX5VugctANmr4Q0Mizd6B0emSktXTUQnEaDjasD0fGMaS kIxQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=ozFLlRgmkT8IfCTAont0giMcDz6boWrqG7U2DDujcG8=; b=V8J2cT1w9lxHdgDb+cI+1nj9hMhGHjzC3mP9Pk2sT9pN2u5VjehVFvghDSPdTnK5k0 dXHZC+PpVlOTuowOIBp3mcxarBqsxMaqwBPfgeh2ZKglz74PTkTp1ohNXc4kCNXjZ6VV d6CFlUM76RXHkzsgDETy5K+zi+dJMymh0WQznwHWrtEFecqrFOvt4WEbuMuLNKCrJS8A ZsWGZFrAfNjd4k7QMwJ1ok1blF7swdQrRCUZb+FBioFs7Gfztqrv9FOS5CBccDsPCdFa GqB3T+uD1u/TsFjOXSei3cyBS6RH7+X70CvoyLKz51cRJjQTujx4mbww/wpoR0gZQK6q vzuw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=La1gi9E3; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id h10-20020a1709063b4a00b0094eed7f1000si3192237ejf.763.2023.04.21.06.29.43; Fri, 21 Apr 2023 06:29:43 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=La1gi9E3; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D734368BED3; Fri, 21 Apr 2023 16:29:28 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qt1-f171.google.com (mail-qt1-f171.google.com [209.85.160.171]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1521E68BED3 for ; Fri, 21 Apr 2023 16:29:21 +0300 (EEST) Received: by mail-qt1-f171.google.com with SMTP id d75a77b69052e-3ef112cac17so9259261cf.0 for ; Fri, 21 Apr 2023 06:29:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20221208.gappssmtp.com; s=20221208; t=1682083759; x=1684675759; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=xSFbumH3pSrTwxRr/qs0PMjllIZ1s3S6/lLCOx2wa0c=; b=La1gi9E3gKdLL9XRw3jFEz3DttCn1mgjfOTqtlifx6COliZzH1Gvk2LfB5p16ggz1K JURM8rLfRoeIsES7v1CCCRkDWO1yi/LRnNikf8w7SIkcLd91JD+/0ef6WPS88shaNH4N ubcwa8rCTwxIXUB6sq0dGUv3+OScMqmczwKli17c/yIQaAE9Cn667mBMJxtWOBma83L1 Clr2mNi02VFlZ09ad1stJQ1jpclNMmkaE66+mkeiXpVZ2otGudriQ3C2CMxLrPKZdaOu TAMYfGdCTnr5XG74FavFpcXDvZnPlNVW+5QyVvOact42Xjc8aPhovLMPk0tuvnN+Ejnk pibw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682083759; x=1684675759; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=xSFbumH3pSrTwxRr/qs0PMjllIZ1s3S6/lLCOx2wa0c=; b=kxmzBKBl5jZwTTTIJqP/hCKc1/dtscYnQ9FisUuo57HPnAV/5kpisEzmDT7NgVj4oV VxQjcT1GSnUn0fZTQF+Y305CsfUI4Pqzd6abFl0ZtU3soXybQRDW6bYFyrYL02QpV3H6 dcxd7bnoZHHF1HcGpZkkNL0PUOSbbHX+fySntHhGozdRF19grRMDF7QlH6ARQeS8DAf4 BK0DPkpZj1/ZB7oYlh8jJo8dKcjGPphJz4EC0f7nyUufPfy5koQooiR1NM7E9DCMSzbv K2sepeHcxLLpAuIGAn7YfiJLxjM3bzp/95CsE3z+W+eKrBmSAT6BvU3S85pvT5BnMiO0 3iJw== X-Gm-Message-State: AAQBX9dM+4Zn55aDK5iHGxVSqJOgal9gUyM3eJXwtTTtySH8r8lUm+h2 Y1SwdmzZP3StWEzUGdpShhe6cLSbPzh4sDGVSjo= X-Received: by 2002:a05:622a:4ca:b0:3ef:415f:197 with SMTP id q10-20020a05622a04ca00b003ef415f0197mr7041284qtx.24.1682083759521; Fri, 21 Apr 2023 06:29:19 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id b142-20020ae9eb94000000b0074abc456dddsm1305626qkg.134.2023.04.21.06.29.19 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Apr 2023 06:29:19 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 21 Apr 2023 10:24:51 -0400 Message-Id: <1682087095-582-2-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1682087095-582-1-git-send-email-dheitmueller@ltnglobal.com> References: <1682087095-582-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [PATCH v3 1/5] ccfifo: Properly handle CEA-708 captions through framerate conversion X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: lfURNj/mgm/l When transcoding video that contains 708 closed captions, the caption data is tied to the frames as side data. Simply dropping or adding frames to change the framerate will result in loss of data, so the caption data needs to be preserved and reformatted. For example, without this patch converting 720p59 to 1080i59 would result in loss of 50% of the caption bytes, resulting in garbled 608 captions and 708 probably wouldn't render at all. Further, the frames that are there will have an illegal cc_count for the target framerate, so some decoders may ignore the packets entirely. Extract the 608 and 708 tuples and insert them onto queues. Then after dropping/adding frames, re-write the tuples back into the resulting frames at the appropriate rate given the target framerate. This includes both having the correct cc_count as well as clocking out the 608 pairs at the appropriate rate. Signed-off-by: Devin Heitmueller --- libavfilter/Makefile | 1 + libavfilter/ccfifo.c | 191 +++++++++++++++++++++++++++++++++++++++++++++++++++ libavfilter/ccfifo.h | 85 +++++++++++++++++++++++ 3 files changed, 277 insertions(+) create mode 100644 libavfilter/ccfifo.c create mode 100644 libavfilter/ccfifo.h diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 71e198b..628ade8 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -14,6 +14,7 @@ OBJS = allfilters.o \ buffersink.o \ buffersrc.o \ colorspace.o \ + ccfifo.o \ drawutils.o \ fifo.o \ formats.o \ diff --git a/libavfilter/ccfifo.c b/libavfilter/ccfifo.c new file mode 100644 index 0000000..3f8be57 --- /dev/null +++ b/libavfilter/ccfifo.c @@ -0,0 +1,191 @@ +/* + * CEA-708 Closed Captioning FIFO + * Copyright (c) 2023 LTN Global Communications + * + * Author: Devin Heitmueller + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "ccfifo.h" + +struct AVCCFifo { + AVFifo *cc_608_fifo; + AVFifo *cc_708_fifo; + int expected_cc_count; + int expected_608; + int cc_detected; + void *log_ctx; +}; + +#define MAX_CC_ELEMENTS 128 +#define CC_BYTES_PER_ENTRY 3 + +struct cc_lookup { + int num; + int den; + int cc_count; + int num_608; +}; + +const static struct cc_lookup cc_lookup_vals[] = { + { 15, 1, 40, 4 }, + { 24, 1, 25, 3 }, + { 24000, 1001, 25, 3 }, + { 30, 1, 20, 2 }, + { 30000, 1001, 20, 2}, + { 60, 1, 10, 1 }, + { 60000, 1001, 10, 1}, +}; + +void ff_ccfifo_freep(AVCCFifo **ccf) +{ + if (ccf && *ccf) { + AVCCFifo *tmp = *ccf; + if (tmp->cc_608_fifo) + av_fifo_freep2(&tmp->cc_608_fifo); + if (tmp->cc_708_fifo) + av_fifo_freep2(&tmp->cc_708_fifo); + av_freep(*ccf); + } +} + +AVCCFifo *ff_ccfifo_alloc(AVRational *framerate, void *log_ctx) +{ + AVCCFifo *ccf; + int i; + + ccf = av_mallocz(sizeof(*ccf)); + if (!ccf) + return NULL; + + if (!(ccf->cc_708_fifo = av_fifo_alloc2(MAX_CC_ELEMENTS, CC_BYTES_PER_ENTRY, 0))) + goto error; + + if (!(ccf->cc_608_fifo = av_fifo_alloc2(MAX_CC_ELEMENTS, CC_BYTES_PER_ENTRY, 0))) + goto error; + + /* Based on the target FPS, figure out the expected cc_count and number of + 608 tuples per packet. See ANSI/CTA-708-E Sec 4.3.6.1. */ + for (i = 0; i < (sizeof(cc_lookup_vals) / sizeof(struct cc_lookup)); i++) { + if (framerate->num == cc_lookup_vals[i].num && + framerate->den == cc_lookup_vals[i].den) { + ccf->expected_cc_count = cc_lookup_vals[i].cc_count; + ccf->expected_608 = cc_lookup_vals[i].num_608; + break; + } + } + + if (ccf->expected_608 == 0) { + av_log(ccf->log_ctx, AV_LOG_WARNING, "cc_fifo cannot transcode captions fps=%d/%d\n", + framerate->num, framerate->den); + return NULL; + } + + return ccf; + +error: + ff_ccfifo_freep(&ccf); + return NULL; +} + +int ff_ccfifo_inject(AVCCFifo *ccf, AVFrame *frame) +{ + AVFrameSideData *sd; + int cc_filled = 0; + int i; + + if (!ccf) + return 0; + + if (ccf->cc_detected == 0 || ccf->expected_cc_count == 0) + return 0; + + sd = av_frame_new_side_data(frame, AV_FRAME_DATA_A53_CC, + ccf->expected_cc_count * CC_BYTES_PER_ENTRY); + if (!sd) + return 0; + + for (i = 0; i < ccf->expected_608; i++) { + if (av_fifo_can_read(ccf->cc_608_fifo) >= CC_BYTES_PER_ENTRY) { + av_fifo_read(ccf->cc_608_fifo, &sd->data[cc_filled * CC_BYTES_PER_ENTRY], + CC_BYTES_PER_ENTRY); + cc_filled++; + } else { + break; + } + } + + /* Insert any available data from the 708 FIFO */ + while (cc_filled < ccf->expected_cc_count) { + if (av_fifo_can_read(ccf->cc_708_fifo) >= CC_BYTES_PER_ENTRY) { + av_fifo_read(ccf->cc_708_fifo, &sd->data[cc_filled * CC_BYTES_PER_ENTRY], + CC_BYTES_PER_ENTRY); + cc_filled++; + } else { + break; + } + } + + /* Insert 708 padding into any remaining fields */ + while (cc_filled < ccf->expected_cc_count) { + sd->data[cc_filled * CC_BYTES_PER_ENTRY] = 0xfa; + sd->data[cc_filled * CC_BYTES_PER_ENTRY + 1] = 0x00; + sd->data[cc_filled * CC_BYTES_PER_ENTRY + 2] = 0x00; + cc_filled++; + } + + return 0; +} + +int ff_ccfifo_extract(AVCCFifo *ccf, AVFrame *frame) +{ + int i; + + if (!ccf) + return 0; + + /* Read the A53 side data, discard padding, and put 608/708 into + queues so we can ensure they get into the output frames at + the correct rate... */ + if (ccf->expected_cc_count > 0) { + AVFrameSideData *side_data = av_frame_get_side_data(frame, AV_FRAME_DATA_A53_CC); + if (side_data) { + uint8_t *cc_bytes = side_data->data; + int cc_count = side_data->size / CC_BYTES_PER_ENTRY; + ccf->cc_detected = 1; + + for (i = 0; i < cc_count; i++) { + /* See ANSI/CTA-708-E Sec 4.3, Table 3 */ + uint8_t cc_valid = (cc_bytes[CC_BYTES_PER_ENTRY*i] & 0x04) >> 2; + uint8_t cc_type = cc_bytes[CC_BYTES_PER_ENTRY*i] & 0x03; + if (cc_type == 0x00 || cc_type == 0x01) { + av_fifo_write(ccf->cc_608_fifo, &cc_bytes[CC_BYTES_PER_ENTRY*i], + CC_BYTES_PER_ENTRY); + } else if (cc_valid && (cc_type == 0x02 || cc_type == 0x03)) { + av_fifo_write(ccf->cc_708_fifo, &cc_bytes[CC_BYTES_PER_ENTRY*i], + CC_BYTES_PER_ENTRY); + } + } + + /* Remove the side data, as we will re-create it on the + output as needed */ + av_frame_remove_side_data(frame, AV_FRAME_DATA_A53_CC); + } + } + return 0; +} diff --git a/libavfilter/ccfifo.h b/libavfilter/ccfifo.h new file mode 100644 index 0000000..70e8b9b --- /dev/null +++ b/libavfilter/ccfifo.h @@ -0,0 +1,85 @@ +/* + * CEA-708 Closed Captioning FIFO + * Copyright (c) 2023 LTN Global Communications + * + * Author: Devin Heitmueller + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * CC FIFO Buffer + */ + +#ifndef AVUTIL_CCFIFO_H +#define AVUTIL_CCFIFO_H + +#include "libavutil/avutil.h" +#include "libavutil/frame.h" +#include "libavutil/fifo.h" + +typedef struct AVCCFifo AVCCFifo; + +/** + * Allocate an AVCCFifo. + * + * @param sample_fmt sample format + * @param channels number of channels + * @param nb_samples initial allocation size, in samples + * @return newly allocated AVCCFifo, or NULL on error + */ +AVCCFifo *ff_ccfifo_alloc(AVRational *framerate, void *log_ctx); + +/** + * Free an AVCCFifo + * + * @param ccf Pointer to the pointer to the AVCCFifo which should be freed + * @note `*ptr = NULL` is safe and leads to no action. + */ +void ff_ccfifo_freep(AVCCFifo **ccf); + + +/** + * Read a frame into a CC Fifo + * + * Extract CC bytes from the AVFrame, insert them into our queue, and + * remove the side data from the AVFrame. The side data is removed + * as it will be re-inserted at the appropriate rate later in the + * filter. + * + * @param af AVCCFifo to write to + * @param frame AVFrame with the video frame to operate on + * @return Zero on success, or negative AVERROR + * code on failure. + */ +int ff_ccfifo_extract(AVCCFifo *af, AVFrame *frame); + +/** + * Insert CC data from the FIFO into an AVFrame (as side data) + * + * Dequeue the appropriate number of CC tuples based on the + * frame rate, and insert them into the AVFrame + * + * @param af AVCCFifo to read from + * @param frame AVFrame with the video frame to operate on + * @return Zero on success, or negative AVERROR + * code on failure. + */ +int ff_ccfifo_inject(AVCCFifo *af, AVFrame *frame); + +#endif /* AVUTIL_CCFIFO_H */ From patchwork Fri Apr 21 14:24:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 41289 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4645:b0:e3:3194:9d20 with SMTP id eb5csp1312805pzb; Fri, 21 Apr 2023 06:29:52 -0700 (PDT) X-Google-Smtp-Source: AKy350b5OWmH+n0HCL+J3JatjfLMS74kEWdZzgGjO0M5b38BbedL+G8WSeo1FzGJyxdmHH8mYQcs X-Received: by 2002:aa7:c592:0:b0:506:c288:118 with SMTP id g18-20020aa7c592000000b00506c2880118mr5055822edq.39.1682083792699; Fri, 21 Apr 2023 06:29:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682083792; cv=none; d=google.com; s=arc-20160816; b=oqjlvLctUIbny61XLiuPh1Dv4iC70mEcwcT49+IQ3NnhXsCJ31VF/0ASpV8sVypFi/ R7CN5eZDxErIENFbOa0cxAtCPXMI5wpmOiMxsd1UE+pCHHAos0+R7CGj1io2TDTY1AM3 qOAe8HhbVFLmplFkQahKM7zo1tOpFFSAaqtv6XfL/LulT+LFu1Edc+UWBiRsvPE4YkqT ta/NcI5S4so90dK3ELFSk85uJD4IBx1TsYRWMYGERqiLFY35P5mEUGB6X1FMdE2fYwve wMz+j1zAnQW0YFNE5jOzzPr8AQPkEbUSr79uoGdzm3lpfiFpTeKw8W394RfmWKxj+pKu Ppdw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=RVrpmOrowsLwtM0TRquoTPXS2zxKfF8tREpA8M7y79E=; b=ARuXSLWKY6O2S7RgfbWyrQxHiZzvVztBoOTNLaj6AHLLQrZzr8q9353eAVW/V6Pn/I KYeJX5HOeJZgo1eQuH6mHRb/PqJq1Pr95VCAWxTeg70ntiqYS7T5qxxtFgGnwWuk/Vjl tAyht40/MCzuyATIxsjuQHUNE/ZI/NcnA1j+pToTYsUJvFeditGMag8HzJ+I8DPXP/8y 9LmOy2nXVKndT+hpgdj1i+I79i8rqw0uEdmL61yT87yxCHUQQwTh6Yix9LCfLz9NXQ52 96edS3KheLxXau7snkPZf3RW6TqS8FbGXZ5CTMVWR3VGOqwcRX6bS42bW43ySZx0Dv0A NECQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=26KJBV1b; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id q11-20020aa7da8b000000b005084710b840si1277884eds.661.2023.04.21.06.29.52; Fri, 21 Apr 2023 06:29:52 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=26KJBV1b; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0904E68BF89; Fri, 21 Apr 2023 16:29:30 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qk1-f173.google.com (mail-qk1-f173.google.com [209.85.222.173]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 145E26808A9 for ; Fri, 21 Apr 2023 16:29:22 +0300 (EEST) Received: by mail-qk1-f173.google.com with SMTP id af79cd13be357-74dd7f52f18so588714085a.0 for ; Fri, 21 Apr 2023 06:29:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20221208.gappssmtp.com; s=20221208; t=1682083760; x=1684675760; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=nW8mqH+MItVKYQuVteI+1QrqXRzb+1QWctuI6goP5+c=; b=26KJBV1bA6OxthyArFdt0PbCggFmhBGT1BDb1Qf8ZvHuImJ5FjJRBknQix/ZMFgtwF RCCiFePsXu/8UjUyTKLtszbkB8AnH7n5gEllmtHqOSHczw60Fz9Kyi44kove8TNl8kUx OqZkfxA9Us7moxx4Njf7k05lg8zMV48qtoD/n+1rKIH01R18VDASoM5oinDiiB5mlebp pyk/KltYOG0Hbg09BwUHUAbfJmHNplE4h2FKpbVyTZYvsWJlV6Q+3bl1Xe4JrMnA0qyb u83j7hL4DCJ5jAGPgwVlZllyQAv+5Ex79+jFRm7vSp2Gl7z/5eo+dE5twJGzp3o0IHRc Tmug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682083760; x=1684675760; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=nW8mqH+MItVKYQuVteI+1QrqXRzb+1QWctuI6goP5+c=; b=XGkrF5C1WHKRtgFeNv0+TzJU9TapoTbB8l+mp7ljCiBkYVys/RK7TqdunwnsuCXrek hKoCsXk87OgmL5BsKBjm7SFGgg4PbutduwU0TY/LrYRe5UYKAkNBHEKjiDYlWuFenMz3 dACR2a3fOCLhfo+C9dx4XCnqRaTb8EpDZsbMqciGN67Z3drvSMmfvHAeIGPEZCXcU6ZA kqT45OF1rqA6aaRD+XvPkimMLruWgdoKG9H2J7ew0iPnZrOGC7k+G+LogtqDS2NpZHtA EuI+z1OQac+JIX0nkdEEIzLxCOQrQtM6P+HsUYol8LBCpYr8iRdlY75JE874rLlIYzb0 aMNw== X-Gm-Message-State: AAQBX9crvUSPZYqbfJZFK5iMLyC8qMotFoL0XhfJrN4aYE2slzLnp3iQ A1GAJiQ8/mQUbkbb3RSa7BTaOV6kZ01Y64+OJ+c= X-Received: by 2002:ad4:5aa7:0:b0:5dd:5c8d:866f with SMTP id u7-20020ad45aa7000000b005dd5c8d866fmr11027268qvg.23.1682083760259; Fri, 21 Apr 2023 06:29:20 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id b142-20020ae9eb94000000b0074abc456dddsm1305626qkg.134.2023.04.21.06.29.19 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Apr 2023 06:29:19 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 21 Apr 2023 10:24:52 -0400 Message-Id: <1682087095-582-3-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1682087095-582-1-git-send-email-dheitmueller@ltnglobal.com> References: <1682087095-582-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [PATCH v3 2/5] vf_fps: properly preserve CEA-708 captions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: TxkMhDiytZgf The existing implementation made an attempt to remove duplicate captions if increasing the framerate, but made no attempt to handle reducing the framerate, nor did it rewrite the caption payloads to have the appropriate cc_count (e.g. the cc_count needs to change from 20 to 10 when going from 1080i59 to 720p59 and vice-versa). Make use of the new ccfifo mechanism to ensure that caption data is properly preserved. Signed-off-by: Devin Heitmueller --- libavfilter/vf_fps.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/libavfilter/vf_fps.c b/libavfilter/vf_fps.c index 051d278..06e7441 100644 --- a/libavfilter/vf_fps.c +++ b/libavfilter/vf_fps.c @@ -34,6 +34,7 @@ #include "libavutil/mathematics.h" #include "libavutil/opt.h" #include "avfilter.h" +#include "ccfifo.h" #include "filters.h" #include "internal.h" @@ -85,6 +86,7 @@ typedef struct FPSContext { AVFrame *frames[2]; ///< buffered frames int frames_count; ///< number of buffered frames + AVCCFifo *cc_fifo; ///< closed captions int64_t next_pts; ///< pts of the next frame to output @@ -165,6 +167,7 @@ static av_cold void uninit(AVFilterContext *ctx) frame = shift_frame(ctx, s); av_frame_free(&frame); } + ff_ccfifo_freep(&s->cc_fifo); av_log(ctx, AV_LOG_VERBOSE, "%d frames in, %d frames out; %d frames dropped, " "%d frames duplicated.\n", s->frames_in, s->frames_out, s->drop, s->dup); @@ -210,6 +213,9 @@ static int config_props(AVFilterLink* outlink) s->in_pts_off, s->out_pts_off, s->start_time); } + if (!(s->cc_fifo = ff_ccfifo_alloc(&outlink->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue. Captions will be passed through\n"); + av_log(ctx, AV_LOG_VERBOSE, "fps=%d/%d\n", outlink->frame_rate.num, outlink->frame_rate.den); return 0; @@ -242,6 +248,7 @@ static int read_frame(AVFilterContext *ctx, FPSContext *s, AVFilterLink *inlink, av_log(ctx, AV_LOG_DEBUG, "Read frame with in pts %"PRId64", out pts %"PRId64"\n", in_pts, frame->pts); + ff_ccfifo_extract(s->cc_fifo, frame); s->frames[s->frames_count++] = frame; s->frames_in++; @@ -289,7 +296,7 @@ static int write_frame(AVFilterContext *ctx, FPSContext *s, AVFilterLink *outlin if (!frame) return AVERROR(ENOMEM); // Make sure Closed Captions will not be duplicated - av_frame_remove_side_data(s->frames[0], AV_FRAME_DATA_A53_CC); + ff_ccfifo_inject(s->cc_fifo, frame); frame->pts = s->next_pts++; frame->duration = 1; From patchwork Fri Apr 21 14:24:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 41290 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4645:b0:e3:3194:9d20 with SMTP id eb5csp1312904pzb; Fri, 21 Apr 2023 06:30:01 -0700 (PDT) X-Google-Smtp-Source: AKy350ayJwuJtYtlaue2EtxNMX+KSxsr60xwNMKb1caERJuGrnFu/zCNwJwJ4rqtGF4+6NgxMnJi X-Received: by 2002:aa7:cb59:0:b0:504:aeb5:89c3 with SMTP id w25-20020aa7cb59000000b00504aeb589c3mr4608946edt.5.1682083801465; Fri, 21 Apr 2023 06:30:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682083801; cv=none; d=google.com; s=arc-20160816; b=hqlq6L3V98L9rFruputrMqvPNHKOQMvUzoNiAITGNxqALBbfDvXHnM3jrXuz65yerB /GiZI5lg36O+t7hwlLbyX0zMqkWuve5rb5kdzO/bUqev7NXzTGC7FCL62gEcpjWPTNZe TTOeNjVku3Zk13NQojocA30JWaBkAhw+xM0d1p+o2EpHDseaWCigOoBHtlArc+RR0K4i Fk2Z9O/LzQ6aaGK0hcmGbd4hqsCGfAX3H6KO8j6v7eD/QTOiNEpS8ON1n59WJ5d2AVy4 ZTACWSoswo4zITluR74igagi1BkrkbOAxBOqXfk9xh3jRwh1JZ3WlN2G/y2jPRTRcqi4 Z0HA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=+TZOZ+mDqtIajLdIRI1BJ5qAGHbzGQYj9pTpg1mTt2Q=; b=pk3BZEnrU2b4UFFXXnMuybXyBgjSu8jyYMht2tUWtzrdyzLoSquhPzULiF6FIrrzX3 19XH5s1u6WwsvylFv2U1gO8zoJ1ZfyBfcBYVT+GbGzchjmZgQb/+Xlf9To6ITHWnmQcb HghF8kItKyzRRV6CmXrNO4HeyhwHS0I+rZpFOBmRiUxh4nk0EkBrodtYRKHnHpbGrqKb R4I+ja3rDyifV5RtZgNG6qDQrHj0CwB508XjpK9kV0Can+E6eiznA8FpKW9AhXct4NUw z+YbZJqZUa5KGon6cXnkuo6GUwETto8D9WjrMiJioHvgBcr5kKU54zUH2e9fNxOFYTCC aXxw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=BVeDhuU7; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id u7-20020a17090657c700b00953291871e5si3545022ejr.624.2023.04.21.06.30.01; Fri, 21 Apr 2023 06:30:01 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=BVeDhuU7; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0275668BF86; Fri, 21 Apr 2023 16:29:31 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qt1-f177.google.com (mail-qt1-f177.google.com [209.85.160.177]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 383686808A9 for ; Fri, 21 Apr 2023 16:29:22 +0300 (EEST) Received: by mail-qt1-f177.google.com with SMTP id d75a77b69052e-3ef31206cbdso9997731cf.0 for ; Fri, 21 Apr 2023 06:29:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20221208.gappssmtp.com; s=20221208; t=1682083761; x=1684675761; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=262VCUW0q7S3KOzwVUwaeJx27npcOtt/zI9mUKDLmmw=; b=BVeDhuU7X+H05dC4tphSyJ/1FDdUR5D9wkUAKhoGULli4diGn3dQK5cbVr89nj79gO FEjaZ5yt4FAUAwKvk5Qk9fOT86YxC5IghPwEzg2/BSMGaoxL4cuiBeVXmHyAcXAI3qZe XepiZsdFv/foGRaDNkv5ufteUoNrMLWcflKSKfmAFd66KF9ziGmDIlD9eAG10DgIc4lK Ea208tWIBgp2+JBYOMZ+koMTl8W5MNPsw+GJYnESqZbzpck2UhXmhcl57ZIDV0gMIJW8 no9Wv34b/S+FtkCuUPuadFUJMZjYT7FF51UEY4eD94Kae56srU+pcJys2TV/2OvkqCtP S0qw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682083761; x=1684675761; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=262VCUW0q7S3KOzwVUwaeJx27npcOtt/zI9mUKDLmmw=; b=MAAPmjFoPJwEVuU0aDHF3+Q6cuzkdl9sKREk2zyaLqdSzMs9rG0VY+1APYQV9QwFRn OsNPc8Teq89Ec8ADPKh+pFGGoHfViSui5mp4CV48IjpTWy+WrLEXAK82vNQPBMkrYVF6 TMf6vdebwHoomPI878UlVrP5B6gYAkgmsFRym8ZRUOgDKA1RtRGtuzHLyOi/6uzQqY6n 4ImmGYdsNfeGd4Sgb1NL2l2PUAZimZyKkf8x7jei3H/7C2PG9Z/HfxKBYDGJqztov7jR rAnaQyVyWfTjmfQZZlVuzd3LLGB54uPI3Dqy0oPmcedZvS2I1wahJIt0K6rVfo+3LPnF 00fA== X-Gm-Message-State: AAQBX9dcMYE681gtv/i09CN6SR7vIHc5+8+aTBU8hrlc9g2h1p+0O4In 1ymvrKxVnU624guvYN6KLGC2cSTkqCayCIf5J9U= X-Received: by 2002:ac8:5cd1:0:b0:3e4:ebc7:c07d with SMTP id s17-20020ac85cd1000000b003e4ebc7c07dmr8683739qta.63.1682083760793; Fri, 21 Apr 2023 06:29:20 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id b142-20020ae9eb94000000b0074abc456dddsm1305626qkg.134.2023.04.21.06.29.20 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Apr 2023 06:29:20 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 21 Apr 2023 10:24:53 -0400 Message-Id: <1682087095-582-4-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1682087095-582-1-git-send-email-dheitmueller@ltnglobal.com> References: <1682087095-582-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [PATCH v3 3/5] yadif: Properly preserve CEA-708 closed captions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: +GhanDuK4WN0 Various deinterlacing modes have the effect of doubling the framerate, and we need to ensure that the caption data isn't duplicated (or else you get double captions on-screen). Use the new ccfifo mechanism for yadif (and yadif_cuda and bwdif since they use the same yadif core) so that CEA-708 data is properly preserved through this filter. Signed-off-by: Devin Heitmueller --- libavfilter/vf_bwdif.c | 7 +++++++ libavfilter/vf_yadif.c | 6 ++++++ libavfilter/vf_yadif_cuda.c | 8 ++++++++ libavfilter/yadif.h | 2 ++ libavfilter/yadif_common.c | 5 +++++ 5 files changed, 28 insertions(+) diff --git a/libavfilter/vf_bwdif.c b/libavfilter/vf_bwdif.c index 34e8c5e..a97a468 100644 --- a/libavfilter/vf_bwdif.c +++ b/libavfilter/vf_bwdif.c @@ -297,6 +297,7 @@ static av_cold void uninit(AVFilterContext *ctx) av_frame_free(&yadif->prev); av_frame_free(&yadif->cur ); av_frame_free(&yadif->next); + ff_ccfifo_freep(&yadif->cc_fifo); } static const enum AVPixelFormat pix_fmts[] = { @@ -332,6 +333,12 @@ static int config_props(AVFilterLink *link) if(yadif->mode&1) link->frame_rate = av_mul_q(link->src->inputs[0]->frame_rate, (AVRational){2,1}); + else + link->frame_rate = ctx->inputs[0]->frame_rate; + + if (!(yadif->cc_fifo = ff_ccfifo_alloc(&link->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue. Captions will be passed through\n"); + if (link->w < 3 || link->h < 4) { av_log(ctx, AV_LOG_ERROR, "Video of less than 3 columns or 4 lines is not supported\n"); diff --git a/libavfilter/vf_yadif.c b/libavfilter/vf_yadif.c index 1be02de..36ecc0d 100644 --- a/libavfilter/vf_yadif.c +++ b/libavfilter/vf_yadif.c @@ -261,6 +261,7 @@ static av_cold void uninit(AVFilterContext *ctx) av_frame_free(&yadif->prev); av_frame_free(&yadif->cur ); av_frame_free(&yadif->next); + ff_ccfifo_freep(&yadif->cc_fifo); } static const enum AVPixelFormat pix_fmts[] = { @@ -293,6 +294,11 @@ static int config_output(AVFilterLink *outlink) if(s->mode & 1) outlink->frame_rate = av_mul_q(ctx->inputs[0]->frame_rate, (AVRational){2, 1}); + else + outlink->frame_rate = ctx->inputs[0]->frame_rate; + + if (!(s->cc_fifo = ff_ccfifo_alloc(&outlink->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue. Captions will be passed through\n"); if (outlink->w < 3 || outlink->h < 3) { av_log(ctx, AV_LOG_ERROR, "Video of less than 3 columns or lines is not supported\n"); diff --git a/libavfilter/vf_yadif_cuda.c b/libavfilter/vf_yadif_cuda.c index 685b8a2..c6cbdca 100644 --- a/libavfilter/vf_yadif_cuda.c +++ b/libavfilter/vf_yadif_cuda.c @@ -206,6 +206,9 @@ static av_cold void deint_cuda_uninit(AVFilterContext *ctx) av_frame_free(&y->cur); av_frame_free(&y->next); + if (yadif->cc_fifo) + ff_cc_fifo_free(yadif->cc_fifo); + av_buffer_unref(&s->device_ref); s->hwctx = NULL; av_buffer_unref(&s->input_frames_ref); @@ -291,6 +294,11 @@ static int config_output(AVFilterLink *link) if(y->mode & 1) link->frame_rate = av_mul_q(ctx->inputs[0]->frame_rate, (AVRational){2, 1}); + else + outlink->frame_rate = ctx->inputs[0]->frame_rate; + + if (!(s->cc_fifo = ff_cc_fifo_alloc(&outlink->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue. Captions will be passed through\n"); if (link->w < 3 || link->h < 3) { av_log(ctx, AV_LOG_ERROR, "Video of less than 3 columns or lines is not supported\n"); diff --git a/libavfilter/yadif.h b/libavfilter/yadif.h index c928911..1077576 100644 --- a/libavfilter/yadif.h +++ b/libavfilter/yadif.h @@ -22,6 +22,7 @@ #include "libavutil/opt.h" #include "libavutil/pixdesc.h" #include "avfilter.h" +#include "ccfifo.h" enum YADIFMode { YADIF_MODE_SEND_FRAME = 0, ///< send 1 frame for each frame @@ -76,6 +77,7 @@ typedef struct YADIFContext { int eof; uint8_t *temp_line; int temp_line_size; + AVCCFifo *cc_fifo; /* * An algorithm that treats first and/or last fields in a sequence diff --git a/libavfilter/yadif_common.c b/libavfilter/yadif_common.c index a10cf7a..676853e 100644 --- a/libavfilter/yadif_common.c +++ b/libavfilter/yadif_common.c @@ -60,6 +60,8 @@ static int return_frame(AVFilterContext *ctx, int is_second) yadif->out->pts = AV_NOPTS_VALUE; } } + + ff_ccfifo_inject(yadif->cc_fifo, yadif->out); ret = ff_filter_frame(ctx->outputs[0], yadif->out); yadif->frame_pending = (yadif->mode&1) && !is_second; @@ -96,6 +98,8 @@ int ff_yadif_filter_frame(AVFilterLink *link, AVFrame *frame) av_assert0(frame); + ff_ccfifo_extract(yadif->cc_fifo, frame); + if (yadif->frame_pending) return_frame(ctx, 1); @@ -137,6 +141,7 @@ int ff_yadif_filter_frame(AVFilterLink *link, AVFrame *frame) if (!yadif->out) return AVERROR(ENOMEM); + ff_ccfifo_inject(yadif->cc_fifo, yadif->out); av_frame_free(&yadif->prev); if (yadif->out->pts != AV_NOPTS_VALUE) yadif->out->pts *= 2; From patchwork Fri Apr 21 14:24:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 41291 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4645:b0:e3:3194:9d20 with SMTP id eb5csp1313014pzb; Fri, 21 Apr 2023 06:30:11 -0700 (PDT) X-Google-Smtp-Source: AKy350ZcWIaTEtCHt5PSmjwQk42QCT1/q9FfaK0ey8w/TB2YiGHYV0h0/xio4zP2puxEXX03Xp7g X-Received: by 2002:a17:906:a101:b0:94f:558b:ed7f with SMTP id t1-20020a170906a10100b0094f558bed7fmr2437948ejy.18.1682083811316; Fri, 21 Apr 2023 06:30:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682083811; cv=none; d=google.com; s=arc-20160816; b=PndZ8uIuRn+5QCj63DdlJFqVKdIkIs3wKfC6XNRWAcdkOTXP5+6wY2Qt2J5w0z+y/r Jkw8EcdA7b5H4RFO8yDQUvVMQ3N3PHtRPJHsVaPHqn1FDe0h72YClFdpqBDnse9Yjz6U Idy8L6iIwbUQ5fEy+xRtTCkDBGUvnF7Dn+DjGw3rnOmmPmRvYcJzrC4pz9EMX7DIRtuH 3ODdTIw5Zjerj/ADyt8awCqPFhuW/YPCi77ahskSYvhS0r2EZ5Z0gRGlFCZosLGntGKn PneZCneopVxWGrayhka0T5zcOU7medmVhPf7esivzBNEAXwt+cs0tYQctvbo3i4nb1VX H7Hg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=dFCcj6hVB53vAI7kiSLbvmUo7BAthYFa+X/Benc0VOo=; b=IcR6C1NrVL2ruIhivl2Wcwjt9r+JxvPFRDbT0njRdQqLNQeynJPSQMPHmUD2cw2Mpm Cf+gyaG6qqi22hPeNGSydj1DS4N2Qx0VTloU2CMygfhFzRm0Zml9iwILgOwHFNVasmvS 2Xdzl87DD/+h6JxeRhZKV25xK08eXAL00MMGBOJWmpVN8eQsOp0/LjE39sfES27rlIsj UwNG9LJHtz3FrvU3ICjm8bEv+MWJcElzKmOPCRUgKTnXCh6Hl7cW6Wht04NEU4BUAwxG M6BUNBkQstg1rkgOZM9UQWNb9j+LFLWPxF7VtgzHocUoDDvoC5El2EOELm65XFW+ZpTz yRMQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b="juxUQJ/u"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id g18-20020a50ee12000000b00506a2c48e17si3497986eds.472.2023.04.21.06.30.10; Fri, 21 Apr 2023 06:30:11 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b="juxUQJ/u"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E8E7868BF90; Fri, 21 Apr 2023 16:29:31 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qt1-f173.google.com (mail-qt1-f173.google.com [209.85.160.173]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0C34668BF6F for ; Fri, 21 Apr 2023 16:29:23 +0300 (EEST) Received: by mail-qt1-f173.google.com with SMTP id d75a77b69052e-3ee339e8c2fso9550871cf.0 for ; Fri, 21 Apr 2023 06:29:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20221208.gappssmtp.com; s=20221208; t=1682083761; x=1684675761; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=n70T5p2X9pc1P+DSrkL+aOq9Z+kNMFaijosn6juOX3E=; b=juxUQJ/u453dPcF4PH+7JYtzO/wb3t5z6QO7H56/wzUsm6CqmEhSDM9pWCIhJiH4WQ BTMQEBfZB4ms2AUzWBdNLhNIT56oumaLMpU0Yzdl0wxZXbOyi/vFMf7KSa1BQnyEOUUP z14n+F/FyjT7Qq0XOHH6hsmLfRB4fH142r+8sM0yOOF/t2Yge2ckEYzFdVptPoRLdFgg 0GzhIemhOSYRVjBxVy90zdy4FoXGBt182wSlSnbBb+7NcBjpuDgvVV5rxn18elwMowpC SE7i8TNX/28YL3FfHfTaqfg1Bo44tfMuF9duEcQx295N7H0jBKhSiP0hXYTjKZSb1UKe cjhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682083761; x=1684675761; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=n70T5p2X9pc1P+DSrkL+aOq9Z+kNMFaijosn6juOX3E=; b=KzeBds9ss9ypL/V7tZGKfYogAro00blkeVnwSWER0/EwUgv/UukhW1i2uiI/zWjoB3 YWboSzrrUx8NStECWUaqWK9/SmOMhrrJeACDkkcSb49LiZdsLxBXx8hWJMKLpahpixkI fdUuVBd/L14Yj6+mz96JI4igApmdiybuLLBa9KTxDbj1Qd1gJeOXr5CPAFX/yKwCzNKl vbsj0yRYfzNGx6rHsUcq9r8J3UVWLvcrcAWCA8D/Zv1iE5mhnRsgBlKRjLYUAl+MuibJ cnCbeOpgCgiqChQUk3nrtYsiuX2klcVZx+Jb01ex/m9AoKJuFOI+RCaqAnjYh7kW2P9I ODIw== X-Gm-Message-State: AAQBX9fLII6blUEqu2j8oZ5mTJZB4eZSPK3/vRx+tb1AfJnWeNQ4WmKv aGnB9eNYm+O+KBHHfCXUesTtOszZ2sYK035JjyQ= X-Received: by 2002:ac8:5ac3:0:b0:3e3:8035:7bf5 with SMTP id d3-20020ac85ac3000000b003e380357bf5mr7984034qtd.20.1682083761595; Fri, 21 Apr 2023 06:29:21 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id b142-20020ae9eb94000000b0074abc456dddsm1305626qkg.134.2023.04.21.06.29.20 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Apr 2023 06:29:21 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 21 Apr 2023 10:24:54 -0400 Message-Id: <1682087095-582-5-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1682087095-582-1-git-send-email-dheitmueller@ltnglobal.com> References: <1682087095-582-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [PATCH v3 4/5] tinterlace: Properly preserve CEA-708 closed captions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: pa7SBbGewK5X Because the interlacing filter halves the effective framerate, we need to ensure that no CEA-708 data is lost as frames are merged. Make use of the new ccfifo mechanism to ensure that caption data is properly preserved as frames pass through the filter. Thanks to Thomas Mundt for review and noticing a couple of missed codepaths for injection on output. Thanks to Lance Wang for pointing out a memory leak. Signed-off-by: Devin Heitmueller --- libavfilter/tinterlace.h | 2 ++ libavfilter/vf_tinterlace.c | 9 +++++++++ 2 files changed, 11 insertions(+) diff --git a/libavfilter/tinterlace.h b/libavfilter/tinterlace.h index 37b6c10..9f5ce7e 100644 --- a/libavfilter/tinterlace.h +++ b/libavfilter/tinterlace.h @@ -32,6 +32,7 @@ #include "libavutil/pixdesc.h" #include "drawutils.h" #include "avfilter.h" +#include "ccfifo.h" #define TINTERLACE_FLAG_VLPF 01 #define TINTERLACE_FLAG_CVLPF 2 @@ -77,6 +78,7 @@ typedef struct TInterlaceContext { const AVPixFmtDescriptor *csp; void (*lowpass_line)(uint8_t *dstp, ptrdiff_t width, const uint8_t *srcp, ptrdiff_t mref, ptrdiff_t pref, int clip_max); + AVCCFifo *cc_fifo; } TInterlaceContext; void ff_tinterlace_init_x86(TInterlaceContext *interlace); diff --git a/libavfilter/vf_tinterlace.c b/libavfilter/vf_tinterlace.c index 0326292..090c7a7 100644 --- a/libavfilter/vf_tinterlace.c +++ b/libavfilter/vf_tinterlace.c @@ -203,6 +203,7 @@ static av_cold void uninit(AVFilterContext *ctx) av_frame_free(&tinterlace->next); av_freep(&tinterlace->black_data[0][0]); av_freep(&tinterlace->black_data[1][0]); + ff_ccfifo_freep(&tinterlace->cc_fifo); } static int config_out_props(AVFilterLink *outlink) @@ -291,6 +292,9 @@ static int config_out_props(AVFilterLink *outlink) #endif } + if (!(tinterlace->cc_fifo = ff_ccfifo_alloc(&outlink->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue. Captions will be passed through\n"); + av_log(ctx, AV_LOG_VERBOSE, "mode:%d filter:%s h:%d -> h:%d\n", tinterlace->mode, (tinterlace->flags & TINTERLACE_FLAG_CVLPF) ? "complex" : (tinterlace->flags & TINTERLACE_FLAG_VLPF) ? "linear" : "off", @@ -375,6 +379,8 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) tinterlace->cur = tinterlace->next; tinterlace->next = picref; + ff_ccfifo_extract(tinterlace->cc_fifo, picref); + cur = tinterlace->cur; next = tinterlace->next; /* we need at least two frames */ @@ -451,6 +457,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) if (!out) return AVERROR(ENOMEM); out->pts /= 2; // adjust pts to new framerate + ff_ccfifo_inject(tinterlace->cc_fifo, out); ret = ff_filter_frame(outlink, out); return ret; } @@ -486,6 +493,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) out->pts = cur->pts*2; out->pts = av_rescale_q(out->pts, tinterlace->preout_time_base, outlink->time_base); + ff_ccfifo_inject(tinterlace->cc_fifo, out); if ((ret = ff_filter_frame(outlink, out)) < 0) return ret; @@ -521,6 +529,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) out->pts = av_rescale_q(out->pts, tinterlace->preout_time_base, outlink->time_base); out->duration = av_rescale_q(1, av_inv_q(outlink->frame_rate), outlink->time_base); + ff_ccfifo_inject(tinterlace->cc_fifo, out); ret = ff_filter_frame(outlink, out); return ret; From patchwork Fri Apr 21 14:24:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 41292 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4645:b0:e3:3194:9d20 with SMTP id eb5csp1313132pzb; Fri, 21 Apr 2023 06:30:21 -0700 (PDT) X-Google-Smtp-Source: AKy350bj/yd8rKwQAYMbRX6++/JBm58rvm2mrnXspbf1LvmuWgkR80+eFiCo2byARMwIitWk+aM6 X-Received: by 2002:a17:907:2907:b0:953:93ab:50cf with SMTP id eq7-20020a170907290700b0095393ab50cfmr1783438ejc.70.1682083821024; Fri, 21 Apr 2023 06:30:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682083821; cv=none; d=google.com; s=arc-20160816; b=wKPNMt/5uJduxymBj9prhQPHYIVhHOdXvhacySlVwLqyGvrf/jBuVCSeD9jdOHdskh lKYI32juUdkrkLO33XHWLmbEc9+g/K5V+hamRql/7JxaqMHQdM9cMdafOimvytlRHBPT 1svzuVDc09D5+gsp8g7CmYEeKeSZ7e75vxQ722rkInzqNlQyB5R8V9MWbS/nJCHYAeeO Uv5DUrjZPSrwCZ5foz0JL8P7Lx2nUAby3OeoZXWFgydaD3UUdac4YlbXOwPPFEr4ECNb XJ83DqXRf3NpX3KIsyArHGVNo4jsXxJrrTmBf0nvgOJ3IqZXTgn2D8bzBSv6vGhsg+De LTzA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=x0vhOF7IeZmd7pWgNpvl1/S06ZipS/XiAJbsyVRM+fo=; b=jDoYl9ykRx+BAMr8JPinqsdxO+c9bIMn/VKV8JBmWXAAUheizICGitBKGj2vgfarhj 3NC25F0onGrPew5WKkFPrRu0voFAlcpX9FWwgwEcEb7GIAEO1NhmjVYNJ/0un0hyVuu3 nU2esU4DxI6h+4aY/rLzhY4Uj/OpWly4ZHhpIR+v1Mbo/Prc9U3/jdVhQxnmGbCgU/zA LrzPn27e3/7ANJppUzvLVpMa/RNmGWw0jfefgiixgqepJuQ4DQFdbu6klhHJ6dTCnbhA BPVJia48kp4Nf4nq0MyG4yDqBOuY12KmLy0eyJUr2RL62PasAoTgjeUEqmQLh9QecIk2 /Q1A== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=Rsq4UZtY; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id v22-20020a056402185600b0050697d54502si3573021edy.545.2023.04.21.06.30.20; Fri, 21 Apr 2023 06:30:21 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=Rsq4UZtY; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0504468BF96; Fri, 21 Apr 2023 16:29:33 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qk1-f171.google.com (mail-qk1-f171.google.com [209.85.222.171]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 99E1E68BF7F for ; Fri, 21 Apr 2023 16:29:23 +0300 (EEST) Received: by mail-qk1-f171.google.com with SMTP id af79cd13be357-74e3cade11fso20082785a.2 for ; Fri, 21 Apr 2023 06:29:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20221208.gappssmtp.com; s=20221208; t=1682083762; x=1684675762; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=gpkbLj99EbwL/CCVwH9gJNe35uA7okrY35JCwbgLoXk=; b=Rsq4UZtYvxSXlujlP6kZ3R47Ydx+F4vby3LG0NQfa2MYN0ZX3HerCYA5Awi8NSyoh4 uV8ieoCcZwk+ws27rsmyLpx0CjUsKFBRttDjzUUsvFBLhFt4w4cay9eq859Yt6HResVg qQuOc3gmh8NRqyV9sap9BsJTHH4a4diS0ZIyGF2wPMnAapWwrx2Fc+ge3tMc2HKk/MuS ZXjm6hQ6AIPe72k6aDcGjnYmZZ71npq/W5NmEq1EGAmcn3jJKQY6jeEepK2Yz6c8YbdV e+WZGlgKeHkgWM9tn/5AYxIMDG5ithmAscO9Ce07tEx2IoZAknC2K61HOqll3cdGs5gu +OLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682083762; x=1684675762; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=gpkbLj99EbwL/CCVwH9gJNe35uA7okrY35JCwbgLoXk=; b=LefUXbLn/KlWS4IEODw6xyjKR9Fb/0uhoDAKfZ+4XF6Zls0nKHuatyR3tXVREiocI1 rpcu0HtWZS+fE7G6bD6Yii4HVksebz8HrxNXLijR86Xeuodp4QV5UlRCKK3tjpZcCnTV Bk8f8Z+9iInQLLClpiMQVpsCfuZAQgiBjYdWPZpbofNvhk8s7QbJuZMA0JuOgOTJTKIf iDBRNi9WAVXl/oWQL8+MfXFDgVtAdQsG0/WRdGT5mgU+hJA8t4o3xIRthoArW7YMkAcV qIi/Ii+fE6tX0MGxzE9uwJOnCDYC+FZotUY2AwV+hPTEQA4AxgFr6YfdlTvzvDebt0gq +CfA== X-Gm-Message-State: AAQBX9cGfSsxyR8G8UmLOGQj+vJ5jIHrSZk0wlD9LCh0ZCuwd7T9qrNp BkWNYBh/PAzzH3Xvob2zGQiJ9FgYfEoLpEmH+AM= X-Received: by 2002:ac8:5786:0:b0:3e3:8e1a:c313 with SMTP id v6-20020ac85786000000b003e38e1ac313mr8129517qta.61.1682083762157; Fri, 21 Apr 2023 06:29:22 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id b142-20020ae9eb94000000b0074abc456dddsm1305626qkg.134.2023.04.21.06.29.21 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Apr 2023 06:29:21 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 21 Apr 2023 10:24:55 -0400 Message-Id: <1682087095-582-6-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1682087095-582-1-git-send-email-dheitmueller@ltnglobal.com> References: <1682087095-582-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [PATCH v3 5/5] vf_ccrepack: Add new filter to repack CEA-708 side data X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: kT8Zv3S0qrxs THis filter can correct certain issues seen from upstream sources where the cc_count is not properly set or the CEA-608 tuples are not at the start of the payload as expected. Make use of the ccfifo to extract and immediately repack the CEA-708 side data, thereby removing any extra padding and ensuring the 608 tuples are at the front of the payload. Signed-off-by: Devin Heitmueller --- doc/filters.texi | 10 +++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/vf_ccrepack.c | 100 ++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 112 insertions(+) create mode 100644 libavfilter/vf_ccrepack.c diff --git a/doc/filters.texi b/doc/filters.texi index 120fe66..e72e73d 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -8930,6 +8930,16 @@ Only deinterlace frames marked as interlaced. The default value is @code{all}. @end table +@section ccrepack + +Repack CEA-708 closed captioning side data + +This filter fixes various issues seen with commerical encoders +related to upstream malformed CEA-708 payloads, specifically +incorrect number of tuples (wrong cc_count for the target FPS), +and incorrect ordering of tuples (i.e. the CEA-608 tuples are not at +the first entries in the payload). + @section cas Apply Contrast Adaptive Sharpen filter to video stream. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 628ade8..3d0b3ad 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -212,6 +212,7 @@ OBJS-$(CONFIG_BOXBLUR_OPENCL_FILTER) += vf_avgblur_opencl.o opencl.o \ opencl/avgblur.o boxblur.o OBJS-$(CONFIG_BWDIF_FILTER) += vf_bwdif.o yadif_common.o OBJS-$(CONFIG_CAS_FILTER) += vf_cas.o +OBJS-$(CONFIG_CCREPACK_FILTER) += vf_ccrepack.o OBJS-$(CONFIG_CHROMABER_VULKAN_FILTER) += vf_chromaber_vulkan.o vulkan.o vulkan_filter.o OBJS-$(CONFIG_CHROMAHOLD_FILTER) += vf_chromakey.o OBJS-$(CONFIG_CHROMAKEY_FILTER) += vf_chromakey.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index d7db46c..b38550b 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -196,6 +196,7 @@ extern const AVFilter ff_vf_boxblur; extern const AVFilter ff_vf_boxblur_opencl; extern const AVFilter ff_vf_bwdif; extern const AVFilter ff_vf_cas; +extern const AVFilter ff_vf_ccrepack; extern const AVFilter ff_vf_chromaber_vulkan; extern const AVFilter ff_vf_chromahold; extern const AVFilter ff_vf_chromakey; diff --git a/libavfilter/vf_ccrepack.c b/libavfilter/vf_ccrepack.c new file mode 100644 index 0000000..c6819b8 --- /dev/null +++ b/libavfilter/vf_ccrepack.c @@ -0,0 +1,100 @@ +/* + * CEA-708 Closed Caption Repacker + * Copyright (c) 2023 LTN Global Communications + * + * Author: Devin Heitmueller + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/* + * Repackage CEA-708 arrays, which deals with incorrect cc_count for a given + * output framerate, and incorrect 708 padding. + * + * See CEA CEA-10-A "EIA-708-B Implementation Guidance", Section 26.5 + * "Grouping DTVCC Data Within user_data() Structure" + */ + +#include "avfilter.h" +#include "internal.h" +#include "ccfifo.h" +#include "libavutil/opt.h" + +typedef struct CCRepackContext +{ + const AVClass *class; + AVCCFifo *cc_fifo; +} CCRepackContext; + +static const AVOption ccrepack_options[] = { + { NULL } +}; + +AVFILTER_DEFINE_CLASS(ccrepack); + +static int config_input(AVFilterLink *link) +{ + CCRepackContext *ctx = link->dst->priv; + + if (!(ctx->cc_fifo = ff_ccfifo_alloc(&link->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue. Captions will be passed through\n"); + + return 0; +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *frame) +{ + CCRepackContext *ctx = inlink->dst->priv; + AVFilterLink *outlink = inlink->dst->outputs[0]; + + ff_ccfifo_extract(ctx->cc_fifo, frame); + ff_ccfifo_inject(ctx->cc_fifo, frame); + + return ff_filter_frame(outlink, frame); +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + CCRepackContext *s = ctx->priv; + ff_ccfifo_freep(&s->cc_fifo); +} + +static const AVFilterPad avfilter_vf_ccrepack_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .filter_frame = filter_frame, + .config_props = config_input, + }, +}; + +static const AVFilterPad avfilter_vf_ccrepack_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + }, +}; + +AVFilter ff_vf_ccrepack = { + .name = "ccrepack", + .description = NULL_IF_CONFIG_SMALL("Repack CEA-708 closed caption metadata"), + .uninit = uninit, + .priv_size = sizeof(CCRepackContext), + .priv_class = &ccrepack_class, + FILTER_INPUTS(avfilter_vf_ccrepack_inputs), + FILTER_OUTPUTS(avfilter_vf_ccrepack_outputs), +};