From patchwork Fri Mar 17 20:09:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 40718 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:d046:b0:cd:afd7:272c with SMTP id hv6csp15433pzb; Fri, 17 Mar 2023 13:11:19 -0700 (PDT) X-Google-Smtp-Source: AK7set/aV6a7Tf5DXwnxkRN01SJZuipB6vYTdwrWOtm00eAm06f7FguvDfgfxqhhm/1v5aSU598R X-Received: by 2002:a17:906:37d3:b0:92f:d1ec:a7d7 with SMTP id o19-20020a17090637d300b0092fd1eca7d7mr604126ejc.15.1679083878982; Fri, 17 Mar 2023 13:11:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1679083878; cv=none; d=google.com; s=arc-20160816; b=meCEpIuV1EGhBPuR7b0apYNVtpTSVzRBV5JQwpNj3s9WhvFGjiJEO/36/MpB9xkXlx A3qHSeMw2OeJzMOYfS1N/hvEjEMyO+3GbDqen/YpJbEWZfzgDVifEFQqUZGpne0JVzrn KnX4fRy1BLlYXKtNdKpGo60bvKHToDjJrcK/C1TcDNszYpZSxOFu3hHZtdkVLbN5lr+N AK/QoFZlvuINcWyu6XMpI/hebT1761bFLIwJshY/sud6RmfnoBMFj1791cPIvXelvxkc cBqN9VHMUqAFWeFXDSJCtPBSjk4R2fxWjImOdJTmgRAeMRlbJOoAnECnABPWCKtmgBG3 gd7A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=PaSny6V6LTBsumHzr+0L9Rc93Nxz0cdvTUF4iHBZR2Q=; b=Ry48/ZXjJppxzs5/n/+hps1X354FsmkOadsQm//ejBt5uyfuqt8EwtQqYX3CN8B8AS XKUkdDKputLTrEFyRbh+LpSG+HK8H4TLGlIYSRE0D9wcD9ElNGMqKDN2IWpXr1xURPf0 mmI3JDTXg2SkgYdG9Z2AVqqlyHyaypCTE+pUuPChzesUq+XsyQb3XHtw31QphyDc+LdB fyMwNVHxSHVPIYwWjt90It/BlD5KaWT8cYUmQeuXtns161QgVnkFOYgHA29FH/wotxdA 1SPspqkfCzRGdWgWcREJh47oRNwuZp9uaygpAMKDEWS5/ePzN7vHK0q9vEmdHVYBzXV7 SQig== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=c8Lc+K3o; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id x11-20020a170906148b00b008c8f5756068si3125255ejc.766.2023.03.17.13.10.58; Fri, 17 Mar 2023 13:11:18 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=c8Lc+K3o; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6119568C280; Fri, 17 Mar 2023 22:10:00 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qt1-f179.google.com (mail-qt1-f179.google.com [209.85.160.179]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 33C4868C24D for ; Fri, 17 Mar 2023 22:09:51 +0200 (EET) Received: by mail-qt1-f179.google.com with SMTP id fy17so4374841qtb.2 for ; Fri, 17 Mar 2023 13:09:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20210112.gappssmtp.com; s=20210112; t=1679083790; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=aci2xh9xWgYUWVbVC1jrQ/1QbeRYXIlHza+nQaMFd9U=; b=c8Lc+K3o8fnbZaT3gWIxCxGPegWd8Ifwx9hosd+gFen9ulrj4BHWbXgHDz7m7nslje r1eK2igT3QSy+sJPUpUVK0CXMNdJpAW+X2IAyEBO4jVY88NqTQlTN8m0Urq7TD91l2oN WwYphSXsZRWh47Xl5d4aurVCGuSIQ63yDBjBwurg8QBfu7wrMs1XasLVdp5CvuyEW2zU G3AGhAVtxH4VA19ZbY1YOLFGbzv9/0J3qU2/Ppsz1opA6Gn8ds4ryGHL90LCONcSxLCP 2k2XuiSMB3EjLcoZ8QaDzgnFwk/lHQQ0S31HfdU6AMkwOXAG9cWTghpSf79gTgX1Z9NG os+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679083790; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=aci2xh9xWgYUWVbVC1jrQ/1QbeRYXIlHza+nQaMFd9U=; b=JTSjFx1ZpSoM3sQQk8j5RPV5YY8fPFS167Tugt8aeJNPHAAm2L3HNGy2xxkTyOaQJp nRZKdhRMmOhi+odYhxvXSC2nAy6jb5mnd1InX0Z4pM427MZXl/rhyPtbMy5TakXzfWzy kt2gCKsEuSJh4SWkMamzR/iYnl5vBwsEc+V7QtTKw1dP6sUrwrCofSVkD2Idtghc23G+ QUbfLI105GResUS2EfhTI3CvGicTyZpXTmhA7hOHmx50sA7Y4nk3S9fLamYSigbf+ZR1 wgd+N/7Fu6TVh+iSIR49r6ylzKWHlExPUqokIaoWHSStgTmrAXmHdhiH5j3ojnp9/wZw pzFA== X-Gm-Message-State: AO0yUKUf8aMgoQMY0Ezo9uLdDr2txyquXicvdY9Ax6epu927CycP1Kg1 gYg6Y3drrAeVFj8ISOMah7/474tC3JnMWtubXbY= X-Received: by 2002:a05:622a:1a9d:b0:3bf:c3e8:c19b with SMTP id s29-20020a05622a1a9d00b003bfc3e8c19bmr15868865qtc.22.1679083789750; Fri, 17 Mar 2023 13:09:49 -0700 (PDT) Received: from klab-nyc-kernellabs2.localdomain (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id j5-20020ac86645000000b003bd0e7ff466sm2065685qtp.7.2023.03.17.13.09.49 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 17 Mar 2023 13:09:49 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 17 Mar 2023 16:09:41 -0400 Message-Id: <20230317200941.3936-6-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 2.35.1.655.ga68dfadae5 In-Reply-To: <20230317200941.3936-1-dheitmueller@ltnglobal.com> References: <20230317200941.3936-1-dheitmueller@ltnglobal.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [RFC PATCH 5/5] vf_ccrepack: Add new filter to repack CEA-708 side data X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: GvKlcz9ZZJ3c THis filter can correct certain issues seen from upstream sources where the cc_count is not properly set or the CEA-608 tuples are not at the start of the payload as expected. Make use of the ccfifo to extract and immediately repack the CEA-708 side data, thereby removing any extra padding and ensuring the 608 tuples are at the front of the payload. Signed-off-by: Devin Heitmueller --- doc/filters.texi | 10 +++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/vf_ccrepack.c | 95 +++++++++++++++++++++++++++++++++++++++ 4 files changed, 107 insertions(+) create mode 100644 libavfilter/vf_ccrepack.c diff --git a/doc/filters.texi b/doc/filters.texi index 77b594f69c..a84fef0e4d 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -8904,6 +8904,16 @@ Only deinterlace frames marked as interlaced. The default value is @code{all}. @end table +@section ccrepack + +Repack CEA-708 closed captioning side data + +This filter fixes various issues seen with commerical encoders +related to upstream malformed CEA-708 payloads, specifically +incorrect number of tuples (wrong cc_count for the target FPS), +and incorrect ordering of tuples (i.e. the CEA-608 tuples are not at +the first entries in the payload). + @section cas Apply Contrast Adaptive Sharpen filter to video stream. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index b3d3d981dd..eed2460ffc 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -211,6 +211,7 @@ OBJS-$(CONFIG_BOXBLUR_OPENCL_FILTER) += vf_avgblur_opencl.o opencl.o \ opencl/avgblur.o boxblur.o OBJS-$(CONFIG_BWDIF_FILTER) += vf_bwdif.o yadif_common.o OBJS-$(CONFIG_CAS_FILTER) += vf_cas.o +OBJS-$(CONFIG_CCREPACK_FILTER) += vf_ccrepack.o OBJS-$(CONFIG_CHROMABER_VULKAN_FILTER) += vf_chromaber_vulkan.o vulkan.o vulkan_filter.o OBJS-$(CONFIG_CHROMAHOLD_FILTER) += vf_chromakey.o OBJS-$(CONFIG_CHROMAKEY_FILTER) += vf_chromakey.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index d7db46c2af..b38550b358 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -196,6 +196,7 @@ extern const AVFilter ff_vf_boxblur; extern const AVFilter ff_vf_boxblur_opencl; extern const AVFilter ff_vf_bwdif; extern const AVFilter ff_vf_cas; +extern const AVFilter ff_vf_ccrepack; extern const AVFilter ff_vf_chromaber_vulkan; extern const AVFilter ff_vf_chromahold; extern const AVFilter ff_vf_chromakey; diff --git a/libavfilter/vf_ccrepack.c b/libavfilter/vf_ccrepack.c new file mode 100644 index 0000000000..ab8473e7c1 --- /dev/null +++ b/libavfilter/vf_ccrepack.c @@ -0,0 +1,95 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/* + * Repackage CEA-708 arrays, which deals with incorrect cc_count for a given + * output framerate, and incorrect 708 padding. + * + * See CEA CEA-10-A "EIA-708-B Implementation Guidance", Section 26.5 + * "Grouping DTVCC Data Within user_data() Structure" + */ + +#include "avfilter.h" +#include "internal.h" +#include "libavutil/opt.h" +#include "libavutil/ccfifo.h" + +typedef struct CCRepackContext +{ + const AVClass *class; + AVCCFifo *cc_fifo; +} CCRepackContext; + +static const AVOption ccrepack_options[] = { + { NULL } +}; + +AVFILTER_DEFINE_CLASS(ccrepack); + +static int config_input(AVFilterLink *link) +{ + CCRepackContext *ctx = link->dst->priv; + + if (!(ctx->cc_fifo = av_ccfifo_alloc(&link->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue. Captions will be passed through\n"); + + return 0; +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *frame) +{ + CCRepackContext *ctx = inlink->dst->priv; + AVFilterLink *outlink = inlink->dst->outputs[0]; + + av_ccfifo_extract(ctx->cc_fifo, frame); + av_ccfifo_inject(ctx->cc_fifo, frame); + + return ff_filter_frame(outlink, frame); +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + CCRepackContext *s = ctx->priv; + av_ccfifo_freep(&s->cc_fifo); +} + +static const AVFilterPad avfilter_vf_ccrepack_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .filter_frame = filter_frame, + .config_props = config_input, + }, +}; + +static const AVFilterPad avfilter_vf_ccrepack_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + }, +}; + +AVFilter ff_vf_ccrepack = { + .name = "ccrepack", + .description = NULL_IF_CONFIG_SMALL("Repack CEA-708 closed caption metadata"), + .uninit = uninit, + .priv_size = sizeof(CCRepackContext), + .priv_class = &ccrepack_class, + FILTER_INPUTS(avfilter_vf_ccrepack_inputs), + FILTER_OUTPUTS(avfilter_vf_ccrepack_outputs), +};