From patchwork Sat Jan 6 02:14:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marth64 X-Patchwork-Id: 45498 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:1093:b0:199:52ba:a72d with SMTP id w19csp451066pze; Fri, 5 Jan 2024 18:15:44 -0800 (PST) X-Google-Smtp-Source: AGHT+IEFMW5f1+D6Ph72ziEpMDvKqlmrY8pSLAdQPJ57J50M5EB5erdtsN+KE2cQB1vxziT0+7He X-Received: by 2002:a2e:884b:0:b0:2cc:e877:2d5a with SMTP id z11-20020a2e884b000000b002cce8772d5amr109151ljj.27.1704507344507; Fri, 05 Jan 2024 18:15:44 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1704507344; cv=none; d=google.com; s=arc-20160816; b=Io+OfozBbkcpAD5xV+Myq6qnIRHcx8JA4oU5pQdPXjNUPHuZdgHTL2X9wF01GylyPf 3mwj5ghGq+plJjJo/EZ4b2tErgDxAigc9ioJs3MdPWEvbKLnfzR1h1AEXZzBfmLcurhX veU9r0oYNrYwRnxh9dG6vyxOclAi9ZSkLMjMYwTAdMfRsBufHEdsP8jV1utYvQ6gNH/G 1tS0Hi0fEfCpfXBfKuxWk1WXtzvbK9O6tfXxJjI+TF06xr3bC3zYe7hqAa298Z4KF1go dDWr32NJt6od0lAwazNdR4P60StkCr5Lel9hdDN2HMcQMBUmIr6ubYyMg01h9aBu1FRc Vd8w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=8RIIWNKylzbIF1X3wOZmBY7V4pkU2QilntWFmowL7Uc=; fh=PlWMzmI9LD2qGS7ipLrQl8z0iaQTLQLHzoGuXcBzpCg=; b=l5S6+14gPpgP/Mh1+51CdQPBlbUYepkW6dqprjyup8t5KAqMvVA8fLu4R5MU27HHiO WG05BoHKnMlEVAGAAtSAZOLNatFaH7fmVuoNTKt6eLzqQ8ZQUvWrHb3gZ8PjGoszHEZS w0CCWgdvsChCzljncrxIK2q5NeaWTPG7NuNhBCdesPIQFsRsZQt+lewOOCpOMUw0kb+u AZx6pfSA+VHdu6iMuwrmMhbdrH70M6gMWSiA3kz6y1v9Hq2NqJM+A9HxIrUBll+AvVzc tuKJLIKo0hzQRB2L5f0qQc7/AORdYIMOFKFfczDHoHWDOcpT/hWTIWMYKA28jyLy8LWR R7OQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@proxyid.net header.s=google header.b=eGCQdlcV; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id cb18-20020a0564020b7200b00557621916a3si25016edb.13.2024.01.05.18.15.44; Fri, 05 Jan 2024 18:15:44 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@proxyid.net header.s=google header.b=eGCQdlcV; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 10E6068C463; Sat, 6 Jan 2024 04:15:41 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-il1-f225.google.com (mail-il1-f225.google.com [209.85.166.225]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7B0C968CBC3 for ; Sat, 6 Jan 2024 04:15:34 +0200 (EET) Received: by mail-il1-f225.google.com with SMTP id e9e14a558f8ab-3606f57e08aso996925ab.3 for ; Fri, 05 Jan 2024 18:15:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=proxyid.net; s=google; t=1704507333; x=1705112133; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=kMT2elAlN6TpONQVVYCLiN9bil5uioYc8UNQb1O6TP4=; b=eGCQdlcVv7W7MjyzY6MwPuUQN4HbJNb2Nzbi9xwlna+xx1kXFrmFmID4JCJdRIsvua E7uRYXB9UgRx+KEcJ0EaHCOipHiqiHcfx6pMQZkusWxDUNuhseYE/Gz14to/YwMfgPUw A4MkY4lg96g3ubOiHOy5uwZWniwHLTsfhjzPZMQEsZ+0bBKIDlZbE38qPJff5O2aI4Dm mRoB4iR5v4jmkvB5Eun9D6e+P4TtIrMdrZ1x/nsDNaS8U5arajd9X+YfPlu1Fg6fV1tx pZYUh882RWYlc4hqUXpj/R85NHKnWoPXt2qifsxtizX1dfUIPqa+7ppAbDLWycRaNfaN DeKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704507333; x=1705112133; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=kMT2elAlN6TpONQVVYCLiN9bil5uioYc8UNQb1O6TP4=; b=Fb+g+QesKpiEKap7Q7cvFQq44Bx0HsUJG9dAn8K+LCL1tX7I4mRYFcqz0uV2sgJpe6 uZe7qtqU0eQ3oek0Y97o5/Am5d2RXSX14ffvT2zUxbnQRVIXkXaObkxLfz8Q+xvKDT0d A1A60GlKlTn8lbw+utjD5rjPdwz11EcpGyOn0dfEsCQ8nIpn7wuVNBaqXGQPp+FSwSAI AcQeYKOC2uPynCBCj89SKUmYPYYugBvch474bsB2HF+rNjwxnELzruUkKYaCMnTSBCMV GpAciYEXsu7AWMqWQ7DahBmtXzneBOJioVR3Y32LFjhBUtBEGkPvZGq+tOhNZWpGP8oB WF6w== X-Gm-Message-State: AOJu0YxMPz9txZovJHVrzqWa7UF80rYvCebJXpvlqGndp6+Gnx/huHRI P5JX2EADzE0IAFgAvvOgOxdO4eqQ8idRSgNizSb774pi6p2L7iHXLOTVD0D5pFKDig== X-Received: by 2002:a05:6e02:178d:b0:35f:f8fc:1112 with SMTP id y13-20020a056e02178d00b0035ff8fc1112mr588632ilu.31.1704507332970; Fri, 05 Jan 2024 18:15:32 -0800 (PST) Received: from wsx-cc1-001.. (c-76-141-0-17.hsd1.il.comcast.net. [76.141.0.17]) by smtp-relay.gmail.com with ESMTPS id n15-20020a92d9cf000000b0035ff597673csm109030ilq.19.2024.01.05.18.15.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 05 Jan 2024 18:15:32 -0800 (PST) X-Relaying-Domain: proxyid.net From: Marth64 To: ffmpeg-devel@ffmpeg.org Date: Fri, 5 Jan 2024 20:14:58 -0600 Message-Id: <20240106021457.1483354-1-marth64@proxyid.net> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] libavformat: add RCWT closed caption muxer X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Marth64 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Ghi9f+znAtZj Signed-off-by: Marth64 Raw Captions With Time (RCWT) is a format native to ccextractor, a commonly used open source tool for processing 608/708 closed caption (CC) sources. It can be used to archive the original, raw CC bitstream and to produce a source file file for later CC processing or conversion. As a result, it also allows for interopability with ccextractor for processing CC data extracted via ffmpeg. The format is simple to parse and can be used to retain all lines and variants of CC. A free specification of RCWT can be found here: https://github.com/CCExtractor/ccextractor/blob/master/docs/BINARY_FILE_FORMAT.TXT This muxer implements the specification as of 01/05/2024, which has been stable and unchanged for 10 years as of this writing. This muxer will have some nuances from the way that ccextractor muxes RCWT. No compatibility issues when processing the output with ccextractor have been observed as a result of this so far, but mileage may vary and outputs will not be a bit-exact match. Specifically, the differences are: (1) This muxer will identify as "FF" as the writing program identifier, so as to be honest about the output's origin. (2) ffmpeg's MPEG-1/2, H264, HEVC, etc. decoders extract closed captioning data differently than ccextractor from embedded SEI/user data. For example, DVD captioning bytes will be translated to ATSC A53 format. This allows ffmpeg to handle 608/708 in a consistant way downstream. This is a lossless conversion and the meaningful data is retained. (3) This muxer will not alter the extracted data except to remove invalid packets in between valid CC blocks. On the other hand, ccextractor will by default remove mid-stream padding, and add padding at the end of the stream (in order to convey the end time of the source video). --- libavformat/Makefile | 1 + libavformat/allformats.c | 1 + libavformat/rcwtenc.c | 203 +++++++++++++++++++++++++++++++++++++++ tests/fate/subtitles.mak | 3 + tests/ref/fate/sub-rcwt | 1 + 5 files changed, 209 insertions(+) create mode 100644 libavformat/rcwtenc.c create mode 100644 tests/ref/fate/sub-rcwt diff --git a/libavformat/Makefile b/libavformat/Makefile index 45dba53044..03c2c70e67 100644 --- a/libavformat/Makefile +++ b/libavformat/Makefile @@ -489,6 +489,7 @@ OBJS-$(CONFIG_QOA_DEMUXER) += qoadec.o OBJS-$(CONFIG_R3D_DEMUXER) += r3d.o OBJS-$(CONFIG_RAWVIDEO_DEMUXER) += rawvideodec.o OBJS-$(CONFIG_RAWVIDEO_MUXER) += rawenc.o +OBJS-$(CONFIG_RCWT_MUXER) += rcwtenc.o subtitles.o OBJS-$(CONFIG_REALTEXT_DEMUXER) += realtextdec.o subtitles.o OBJS-$(CONFIG_REDSPARK_DEMUXER) += redspark.o OBJS-$(CONFIG_RKA_DEMUXER) += rka.o apetag.o img2.o diff --git a/libavformat/allformats.c b/libavformat/allformats.c index dc2acf575c..fb14f15739 100644 --- a/libavformat/allformats.c +++ b/libavformat/allformats.c @@ -388,6 +388,7 @@ extern const AVInputFormat ff_qoa_demuxer; extern const AVInputFormat ff_r3d_demuxer; extern const AVInputFormat ff_rawvideo_demuxer; extern const FFOutputFormat ff_rawvideo_muxer; +extern const FFOutputFormat ff_rcwt_muxer; extern const AVInputFormat ff_realtext_demuxer; extern const AVInputFormat ff_redspark_demuxer; extern const AVInputFormat ff_rka_demuxer; diff --git a/libavformat/rcwtenc.c b/libavformat/rcwtenc.c new file mode 100644 index 0000000000..f70a80b175 --- /dev/null +++ b/libavformat/rcwtenc.c @@ -0,0 +1,203 @@ +/* + * Raw Captions With Time (RCWT) muxer + * Author: Marth64 + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/* + * Raw Captions With Time (RCWT) is a format native to ccextractor, a commonly + * used open source tool for processing 608/708 closed caption (CC) sources. + * It can be used to archive the original, raw CC bitstream and to produce + * a source file file for later CC processing or conversion. As a result, + * it also allows for interopability with ccextractor for processing CC data + * extracted via ffmpeg. The format is simple to parse and can be used + * to retain all lines and variants of CC. + * + * A free specification of RCWT can be found here: + * https://github.com/CCExtractor/ccextractor/blob/master/docs/BINARY_FILE_FORMAT.TXT + * This muxer implements the specification as of 01/05/2024, which has + * been stable and unchanged for 10 years as of this writing. + * + * This muxer will have some nuances from the way that ccextractor muxes RCWT. + * No compatibility issues when processing the output with ccextractor + * have been observed as a result of this so far, but mileage may vary + * and outputs will not be a bit-exact match. + * + * Specifically, the differences are: + * (1) This muxer will identify as "FF" as the writing program identifier, so + * as to be honest about the output's origin. + * (2) ffmpeg's MPEG-1/2, H264, HEVC, etc. decoders extract closed captioning + * data differently than ccextractor from embedded SEI/user data. + * For example, DVD captioning bytes will be translated to ATSC A53 format. + * This allows ffmpeg to handle 608/708 in a consistant way downstream. + * This is a lossless conversion and the meaningful data is retained. + * (3) This muxer will not alter the extracted data except to remove invalid + * packets in between valid CC blocks. On the other hand, ccextractor + * will by default remove mid-stream padding, and add padding at the end + * of the stream (in order to convey the end time of the source video). + */ + +#include "avformat.h" +#include "internal.h" +#include "mux.h" +#include "libavutil/log.h" +#include "libavutil/intreadwrite.h" + +#define RCWT_CLUSTER_MAX_BLOCKS 65535 +#define RCWT_BLOCK_SIZE 3 * sizeof(uint8_t) + +typedef struct RCWTContext { + int cluster_nb_blocks; + int cluster_pos; + int64_t cluster_pts; + uint8_t cluster_buf[RCWT_CLUSTER_MAX_BLOCKS * RCWT_BLOCK_SIZE]; +} RCWTContext; + +static void rcwt_init_cluster(AVFormatContext *avf) +{ + RCWTContext *rcwt = avf->priv_data; + + rcwt->cluster_nb_blocks = 0; + rcwt->cluster_pos = 0; + rcwt->cluster_pts = AV_NOPTS_VALUE; + memset(rcwt->cluster_buf, 0, sizeof(rcwt->cluster_buf)); +} + +static void rcwt_flush_cluster(AVFormatContext *avf) +{ + RCWTContext *rcwt = avf->priv_data; + + if (rcwt->cluster_nb_blocks > 0) { + avio_wl64(avf->pb, rcwt->cluster_pts); + avio_wl16(avf->pb, rcwt->cluster_nb_blocks); + avio_write(avf->pb, rcwt->cluster_buf, + (rcwt->cluster_nb_blocks * RCWT_BLOCK_SIZE)); + } + + rcwt_init_cluster(avf); +} + +static int rcwt_write_header(AVFormatContext *avf) +{ + if (avf->nb_streams != 1 + || avf->streams[0]->codecpar->codec_type != AVMEDIA_TYPE_SUBTITLE + || avf->streams[0]->codecpar->codec_id != AV_CODEC_ID_EIA_608) { + av_log(avf, AV_LOG_ERROR, + "RCWT supports only one CC (608/708) stream\n"); + return AVERROR(EINVAL); + } + + avpriv_set_pts_info(avf->streams[0], 64, 1, 1000); + + /* magic number */ + avio_wb16(avf->pb, 0xCCCC); + avio_w8(avf->pb, 0xED); + + /* program version (identify as ffmpeg) */ + avio_wb16(avf->pb, 0xFF00); + avio_w8(avf->pb, 0x60); + + /* format version, only version 0.001 supported for now */ + avio_wb16(avf->pb, 0x0001); + + /* reserved */ + avio_wb16(avf->pb, 0x000); + avio_w8(avf->pb, 0x00); + + rcwt_init_cluster(avf); + + return 0; +} + +static int rcwt_write_packet(AVFormatContext *avf, AVPacket *pkt) +{ + RCWTContext *rcwt = avf->priv_data; + + int in_block = 0; + int nb_block_bytes = 0; + + if (pkt->size == 0) + return 0; + + /* new PTS, new cluster */ + if (pkt->pts != rcwt->cluster_pts) { + rcwt_flush_cluster(avf); + rcwt->cluster_pts = pkt->pts; + } + + if (pkt->pts == AV_NOPTS_VALUE) { + av_log(avf, AV_LOG_WARNING, "Ignoring CC packet with no PTS\n"); + return 0; + } + + for (int i = 0; i < pkt->size; i++) { + uint8_t cc_valid; + uint8_t cc_type; + + if (rcwt->cluster_nb_blocks == RCWT_CLUSTER_MAX_BLOCKS) { + av_log(avf, AV_LOG_WARNING, + "Starting new cluster due to size\n"); + rcwt_flush_cluster(avf); + } + + cc_valid = (pkt->data[i] & 0x04) >> 2; + cc_type = pkt->data[i] & 0x03; + + if (!in_block && !(cc_valid || cc_type == 3)) + continue; + + memcpy(&rcwt->cluster_buf[rcwt->cluster_pos], + &pkt->data[i], sizeof(uint8_t)); + rcwt->cluster_pos++; + + if (!in_block) { + in_block = 1; + nb_block_bytes = 1; + continue; + } + + nb_block_bytes++; + + if (nb_block_bytes == RCWT_BLOCK_SIZE) { + in_block = 0; + nb_block_bytes = 0; + rcwt->cluster_nb_blocks++; + } + } + + return 0; +} + +static int rcwt_write_trailer(AVFormatContext *avf) +{ + rcwt_flush_cluster(avf); + + return 0; +} + +const FFOutputFormat ff_rcwt_muxer = { + .p.name = "rcwt", + .p.long_name = NULL_IF_CONFIG_SMALL("Raw Captions With Time"), + .p.extensions = "bin", + .p.flags = AVFMT_GLOBALHEADER | AVFMT_VARIABLE_FPS | AVFMT_TS_NONSTRICT, + .p.subtitle_codec = AV_CODEC_ID_EIA_608, + .priv_data_size = sizeof(RCWTContext), + .write_header = rcwt_write_header, + .write_packet = rcwt_write_packet, + .write_trailer = rcwt_write_trailer +}; diff --git a/tests/fate/subtitles.mak b/tests/fate/subtitles.mak index 59595b9cc1..d7edd31e85 100644 --- a/tests/fate/subtitles.mak +++ b/tests/fate/subtitles.mak @@ -118,6 +118,9 @@ fate-sub-scc: CMD = fmtstdout ass -ss 57 -i $(TARGET_SAMPLES)/sub/witch.scc FATE_SUBTITLES-$(call DEMMUX, SCC, SCC) += fate-sub-scc-remux fate-sub-scc-remux: CMD = fmtstdout scc -i $(TARGET_SAMPLES)/sub/witch.scc -ss 4:00 -map 0 -c copy +FATE_SUBTITLES-$(call DEMMUX, SCC, RCWT) += fate-sub-rcwt +fate-sub-rcwt: CMD = md5 -i $(TARGET_SAMPLES)/sub/witch.scc -map 0 -c copy -f rcwt + FATE_SUBTITLES-$(call ALLYES, MPEGTS_DEMUXER DVBSUB_DECODER DVBSUB_ENCODER) += fate-sub-dvb fate-sub-dvb: CMD = framecrc -i $(TARGET_SAMPLES)/sub/dvbsubtest_filter.ts -map s:0 -c dvbsub diff --git a/tests/ref/fate/sub-rcwt b/tests/ref/fate/sub-rcwt new file mode 100644 index 0000000000..722cbe1c5b --- /dev/null +++ b/tests/ref/fate/sub-rcwt @@ -0,0 +1 @@ +d86f179094a5752d68aa97d82cf887b0