From patchwork Mon Dec 13 23:40:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 32467 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp6141291iog; Mon, 13 Dec 2021 15:43:22 -0800 (PST) X-Google-Smtp-Source: ABdhPJxNQn8WImpC0LNVVBpIvc4rUCTfySeU/Gm9GyzoupE17QFtb8ybK9kakZccPqWT49Y1YDIn X-Received: by 2002:a05:6402:5cc:: with SMTP id n12mr2754190edx.246.1639439002631; Mon, 13 Dec 2021 15:43:22 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id b13si28408654ede.393.2021.12.13.15.43.22; Mon, 13 Dec 2021 15:43:22 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b="VWusKAm/"; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7529B68B0E7; Tue, 14 Dec 2021 01:40:55 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12olkn2087.outbound.protection.outlook.com [40.92.21.87]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 8A9D168B0D7 for ; Tue, 14 Dec 2021 01:40:52 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KFx63UTIbfitqEwdBta0S+06MLFxk9fSfb5kWjhsJ+AUVlKXtSV5fQUFSjB+mXDKU3x+cj3g2yMuYIBCLRMV9C35neiG6c7oUNxmkRti2aRjxdFxRVvoPCaHFZD1//JDxkfYe/aOS+aQNbJGpRur71HgapbQTQHzivPpE2Ck+XW6ULWvYtLJ2AZmgn0kJ+2MHr2GsBIAZkNZ9XS4VX7oaNWRgppiAf1cf6iEJlPOiTH4dPxaVtlLh6Hw1GLTBX9DK2WImm2SUHJJ8YfcQzh6BFZFLeOAzsOKIJFN/Kqithp3Nxmvq4qb2ZKpaPfOLid0vUgFDSBZvW0B0QJZ+mpptw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ntlEjMnOpwwB/8Ro9QyQKlyjm6VZYlu033LdyGk/WkY=; b=nejVQE8KL8MC4cVgEWsmsGuNr6h6dYydXZYON4mbrXpQf01QiDGrmAzGzo1M+2di2TJIrvlq/P8lHKdxPNB3LDZYPTysMlTPoTUxcZDUFhgzDs/IKQy0TBW2f1yS66cBriRvbr4TZQmITxFM9MDwWNqC5O8geiXPTdkcfjA02DcvWOlqvP+I0yLt7g7/EoWSBmOCJrQwM6erL+0s2BPcW3C/6QTvBR+Jy124AuxUBDoa8p1BS+uJBlZzcBlg+zz6J9yXJfpgBJqsTcDgyZks2HlppQLaBqOBBy97STZ+csNAPU25jBMQ/hb45lsB1ss7ASuE7TjZuWcByZV5eptQpA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ntlEjMnOpwwB/8Ro9QyQKlyjm6VZYlu033LdyGk/WkY=; b=VWusKAm/MKbsJnLlx7v5EaPDyIanUto4pef4mCinMe2hlO7V8eORmCEXCllCLrmt8Y/i8Y8eKcWtjbvcCPsUcxMmygy+zGNTGkDCy0OnpMToS/N1ek4TGKERSgeTL3XlNiC4VDW9IedS2wwLCxYlkKf9qFqw/71jz56m8d3bigJnMnckFSiRS0HYkj96zK0yJJtib+5u85G8tTzHSfQAtFxegFwTJnqvkT6IzLEXLDxptEGbSsNtg30BRz4rf4Gn4G0RNUYZJE602Mye1OaSnxAqz/pLjIoFf8xxUupy/lSOE0S6TADyPr1IENVGRLsmN76M/8iR70ug7fNBeq4rCw== Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::20) by DM8P223MB0397.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4778.17; Mon, 13 Dec 2021 23:40:51 +0000 Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775]) by DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775%7]) with mapi id 15.20.4778.018; Mon, 13 Dec 2021 23:40:51 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v24 16/21] avfilter/textmod: Add textmod, censor and show_speaker filters Thread-Index: AQHX8HrcIUrVnVM1OUW9B+zvjeHSXw== Date: Mon, 13 Dec 2021 23:40:51 +0000 Message-ID: References: <1ef42eb01b3627bbad539abd46206f320bb39697.1639438462.git.softworkz@hotmail.com> <4148d77a82d4b7e229fc81ce9bc1bc06904533f1.1639438462.git.softworkz@hotmail.com> <2ac798924ee984010d2e85f00bc992d9aa106171.1639438462.git.softworkz@hotmail.com> <300ea7188b331ca5e00dd7e04c03df8aba2a99c0.1639438462.git.softworkz@hotmail.com> <9474726b26f0d3e8f353d07caed2fbd33cdf473a.1639438462.git.softworkz@hotmail.com> <424522c96b5cdf6b03faa6bdceb1d25449463429.1639438462.git.softworkz@hotmail.com> <90caff245055005e6a4a890b3665da55aa853b31.1639438462.git.softworkz@hotmail.com> <0ebcf3ed2adb02d3ffb243a499b434c3bdd976a7.1639438462.git.softworkz@hotmail.com> <4b5c1c1185075fd6e5d8ef70e07fdce6f896d699.1639438462.git.softworkz@hotmail.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [Ef0NmRZl0o4ppgPxSwNzB+Vi1Ez9YoUI] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: a2766cc5-1ea8-47cc-3a63-08d9be91ff6a x-ms-traffictypediagnostic: DM8P223MB0397:EE_ x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: yzIuFUqDhF5DoQ5A1gfHO79hfbXVIUZST/igDJl9MPwoODLWmPsta9hJDP0CP7o4AhO7hNSolrRopnsMoBjagQuvCNhvMIecoMh+9lhOPNcb9rXJ0g9NXd8+dPaSdU0KHtPGw1rXfrElqBmdk1uWBJtLYBVk0+UDHyz2IdW1+HrC7fExhKJJK6w9BTH5YP+xTLXCX0Bu8DVJoBnPnp1NlX2kCJq69EmWBQt9luyz33HK6qat284DgSZujaMWz7Ds8UGuZQmr8R/7j2MqTlXOfvaV0NbpKERzG6MQPjXQ60bNpAYwrBG5WnjC+Cgp+lO8AlSKur2gbqxqhoWoHKEHDy5iuqhjRZU9gQe9pvYh8QC/mRNUSyFZ90Mp6hQxEFQ5xomS1ZIwz4Jr1f/GF36XGYupA+f0ZLwnRG8lLxWsT69nFhLL/QnKzdmKYiSeSh3xKieHHBR5noeEdeWK9+O8zgLc8Ojw6B4+nt/5gcAUQJFKmcmZ4bCPkPDMHa8zspULTg1KM4gM0QJ4itiUBkZZ+qFS059PQtY3oHsQMcbtrLQQgppc815rq2u4V7A6WgisXVWQc8jmKDNPdqhZs8tebZ1OqvFfXh1UHxcwpiqGMejfO4MPHwir4rAXjcl2sGBh/N6P3lt0mGdi+FWjRjQM+J5p0ql3BQRQ3pPckzc7zmE= x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?q?5skaPIOcTXh1+IW9jU6i+D4uB30s?= =?utf-8?q?WFlDj4W0argxb/9x5+kgdUjYt5xBu2va95iGF3tnvo2YCyLfPQgAIzO1Ai7gco6Ke?= =?utf-8?q?7nhrGrNASAXDQGXIYu2ppSjzlbW68kbe1BohT2UkFkgKAp7KA8NyrIUz2b4+U8D+Y?= =?utf-8?q?4YOXuB2JkaSs/YI+W6KnJImcijgmFYChW0JGNIC3kvM88EziDKOEjJSGtBPnpW7AE?= =?utf-8?q?MIWGxiB7vQV31lV3U1ySNhq2NZKoXV8nBntkqg7cPYVMeMFl1v8RrtZoFWHom6auV?= =?utf-8?q?eRLH72YcocyfDScJ0GamHNjJWjiK1EbAT9NBTtpH+EfYNitrTSKQ5OwIo04u4TPLc?= =?utf-8?q?awHWWCm465HxlbWzZHUz5YNKzYVqb5UOmPfCwpKKACGN2eOykDQiZxrsFr+h5u5bl?= =?utf-8?q?uK6RHq2cW86O2PLH5aQB1JsKhoVqDz857RRk3Kp2fFqrDoC5fqmbbCnl57rnv2klD?= =?utf-8?q?vQ2NtR0z0lTQTir8HxR+61GzWVmbCLCSyIVMXfhd1S3HxVJN/TIr04imm3hf2IisL?= =?utf-8?q?Qjz2mNElhf9h422XaHK7WlhyVWFUq0PVns3e4faN68Ttpl2+d9UvY1UZzKw7+AlyU?= =?utf-8?q?ygFC1MRSigCwJM71BOEmHdFVDrkJ/+zURHsN//xxns9Ub/iLAM2PID8lN+uGNt8jl?= =?utf-8?q?8R0ISV0dp7vQhteAJ6OUWHMn1qPztwqTOTO1lFIDFRW/X3O3nJ9Eu7QmzqOjVpnwi?= =?utf-8?q?lVFfY1RozLjl2S9cDNO70YEUucr4aFKftvnzb2ny4Fa/kqzvRImfyHW1Az55rwpUP?= =?utf-8?q?Dlm2VDvf/9EUIOGzoBvQ7v6nnvvYIDI7Z0xlQoZ3qWkPlnu2NQONh7ThAnLCDS+oQ?= =?utf-8?q?keIsTxHt+CLtA8Mn+lfucDcaSOMrS9YZxAfOo06ipXdPn2nAW96YSfjAQ+n+1eBvc?= =?utf-8?q?fw/YGdAA8dW+MfuhGFgQExd6PcQx7Ramo5FdSK9qP9Kg=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-1ff67.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: a2766cc5-1ea8-47cc-3a63-08d9be91ff6a X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Dec 2021 23:40:51.3924 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8P223MB0397 Subject: [FFmpeg-devel] [PATCH v24 16/21] avfilter/textmod: Add textmod, censor and show_speaker filters X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: gyUULGbg9ri1 s- textmod {S -> S) Modify subtitle text in a number of ways - censor {S -> S) Censor subtitles using a word list - show_speaker {S -> S) Prepend speaker names from ASS subtitles to the visible text lines Signed-off-by: softworkz --- doc/filters.texi | 206 ++++++++++++ libavfilter/Makefile | 5 + libavfilter/allfilters.c | 3 + libavfilter/sf_textmod.c | 697 +++++++++++++++++++++++++++++++++++++++ 4 files changed, 911 insertions(+) create mode 100644 libavfilter/sf_textmod.c diff --git a/doc/filters.texi b/doc/filters.texi index 5e00716083..6ef95c4926 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -25720,6 +25720,145 @@ existing filters using @code{--disable-filters}. Below is a description of the currently available subtitle filters. + +@section censor + +Censor selected words in text subtitles. + +Inputs: +@itemize +@item 0: Subtitles[TEXT] +@end itemize + +Outputs: +@itemize +@item 0: Subtitles[TEXT] +@end itemize + +It accepts the following parameters: + +@table @option +@item mode +The censoring mode to apply. + +Supported censoring modes are: + +@table @var +@item 0, keep_first_last +Replace all characters with the 'censor_char' except the first and the last character of a word. +For words with less than 4 characters, the last character will be replaced as well. +For words with less than 3 characters, the first character will be replaced as well. +@item 1, keep_first +Replace all characters with the 'censor_char' except the first character of a word. +For words with less than 3 characters, the first character will be replaced as well. +@item 2, all +Replace all characters with the 'censor_char'. +@end table + +@item words +A list of words to censor, separated by 'separator'. + +@item words_file +Specify a file from which to load the contents for the 'words' parameter. + +@item censor_char +Single character used as replacement for censoring. + +@item separator +Delimiter character for words. Used with replace_words and remove_words- Must be a single character. +The default is '.'. + +@end table + +@subsection Examples + +@itemize +@item +Censor a few given words with a pound character. +@example +ffmpeg -i "http://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.mkv" -filter_complex "[0:1]censor=words='diss,louder,hope,beam,word':censor_char='#'" -map 0 -y output.mkv +@end example +@end itemize + + +@section textmod + +Modify subtitle text in a number of ways. + +Inputs: +@itemize +@item 0: Subtitles[TEXT] +@end itemize + +Outputs: +@itemize +@item 0: Subtitles[TEXT] +@end itemize + +It accepts the following parameters: + +@table @option +@item mode +The kind of text modification to apply + +Supported operation modes are: + +@table @var +@item 0, leet +Convert subtitle text to 'leet speak'. It's primarily useful for testing as the modification will be visible with almost all text lines. +@item 1, to_upper +Change all text to upper case. Might improve readability. +@item 2, to_lower +Change all text to lower case. +@item 3, replace_chars +Replace one or more characters. Requires the find and replace parameters to be specified. +Both need to be equal in length. +The first char in find is replaced by the first char in replace, same for all subsequent chars. +@item 4, remove_chars +Remove certain characters. Requires the find parameter to be specified. +All chars in the find parameter string will be removed from all subtitle text. +@item 5, replace_words +Replace one or more words. Requires the find and replace parameters to be specified. Multiple words must be separated by the delimiter char specified vie the separator parameter (default: ','). +The number of words in the find and replace parameters needs to be equal. +The first word in find is replaced by the first word in replace, same for all subsequent words +@item 6, remove_words +Remove certain words. Requires the find parameter to be specified. Multiple words must be separated by the delimiter char specified vie the separator parameter (default: ','). +All words in the find parameter string will be removed from all subtitle text. +@end table + +@item find +Required for replace_chars, remove_chars, replace_words and remove_words. + +@item find_file +Specify a file from which to load the contents for the 'find' parameter. + +@item replace +Required for replace_chars and replace_words. + +@item replace_file +Specify a file from which to load the contents for the 'replace' parameter. + +@item separator +Delimiter character for words. Used with replace_words and remove_words- Must be a single character. +The default is '.'. + +@end table + +@subsection Examples + +@itemize +@item +Change all characters to upper case while keeping all styles and animations: +@example +ffmpeg -i "https://streams.videolan.org/ffmpeg/mkv_subtitles.mkv" -filter_complex "[0:s]textmod=mode=to_upper" -map 0 -y out.mkv +@end example +@item +Remove a set of symbol characters for am improved and smoother visual apperance: +@example +ffmpeg -i "https://streams.videolan.org/ffmpeg/mkv_subtitles.mkv" -filter_complex "[0:s]textmod=mode=remove_chars:find='$&#@*§'" -map 0 -y out.mkv +@end example +@end itemize + @section graphicsub2video Renders graphic subtitles as video frames. @@ -25887,6 +26026,73 @@ ffmpeg -i "http://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex. @end example @end itemize +@section showspeaker + +Prepend speaker names to subtitle lines (when available). + +Subtitles in ASS/SSA format are often including the names of the persons +or character for each subtitle line. The showspeaker filter adds those names +to the actual subtitle text to make it visible on playback. + +Inputs: +@itemize +@item 0: Subtitles[TEXT] +@end itemize + +Outputs: +@itemize +@item 0: Subtitles[TEXT] +@end itemize + +It accepts the following parameters: + +@table @option +@item format +The format for prepending speaker names. Default is 'square_brackets'. + +Supported operation modes are: + +@table @var +@item 0, square_brackets +Enclose the speaker name in square brackets, followed by space ('[speaker] text'). +@item 1, round_brackets +Enclose the speaker name in round brackets, followed by space ('(speaker) text'). +@item 2, colon +Separate the speaker name with a colon and space ('speaker: text'). +@item 3, plain +Separate the speaker name with a space only ('speaker text'). +@end table + +@item line_break +Set thís parameter to insert a line break between speaker name and text instead of the space character. + +@item style +Allows to set a specific style for the speaker name text. + +This can be either a named style that exists in the ass subtitle header script (e.g. 'Default') or an ass style override code. +Example: @{\\c&HDD0000&\\be1\\i1\\bord10@} +This sets the color to blue, enables edge blurring, italic font and a border of size 10. + +The behavior is as follows: + +- When the style parameter is not provided, the filter will find the first position in the event string that is actual text. + The speaker name will be inserted at this position. This allows to have the speaker name shown in the same style like the + regular text, in case the string would start with a sequence of style codes. +- When the style parameter is provided, everything will be prepended to the original text: + Style Code or Style name >> Speaker Name >> Style Reset Code >> Original Text + +@end table + +@subsection Examples + +@itemize +@item +Prepend speaker names with blue text, smooth edges and blend/overlay that onto the video. +@example +ffmpeg -i INPUT -filter_complex "showspeaker=format=colon:style='@{\\c&HDD0000&\\be1@}',[0:v]overlay_textsubs" +@end example +@end itemize + @section textsub2video Converts text subtitles to video frames. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index fe16fc147a..d47bdfa8f6 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -555,6 +555,11 @@ OBJS-$(CONFIG_YUVTESTSRC_FILTER) += vsrc_testsrc.o OBJS-$(CONFIG_NULLSINK_FILTER) += vsink_nullsink.o +# subtitle filters +OBJS-$(CONFIG_CENSOR_FILTER) += sf_textmod.o +OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o +OBJS-$(CONFIG_TEXTMOD_FILTER) += sf_textmod.o + # multimedia filters OBJS-$(CONFIG_ABITSCOPE_FILTER) += avf_abitscope.o OBJS-$(CONFIG_ADRAWGRAPH_FILTER) += f_drawgraph.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index b3a4bf24aa..cc8f43c925 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -546,6 +546,9 @@ extern const AVFilter ff_avf_showvolume; extern const AVFilter ff_avf_showwaves; extern const AVFilter ff_avf_showwavespic; extern const AVFilter ff_vaf_spectrumsynth; +extern const AVFilter ff_sf_censor; +extern const AVFilter ff_sf_showspeaker; +extern const AVFilter ff_sf_textmod; extern const AVFilter ff_svf_graphicsub2video; extern const AVFilter ff_svf_textsub2video; diff --git a/libavfilter/sf_textmod.c b/libavfilter/sf_textmod.c new file mode 100644 index 0000000000..d73f329b87 --- /dev/null +++ b/libavfilter/sf_textmod.c @@ -0,0 +1,697 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * text subtitle filter which allows to modify subtitle text in several ways + */ + +#include + +#include "libavutil/opt.h" +#include "internal.h" +#include "libavutil/ass_split_internal.h" +#include "libavutil/bprint.h" +#include "libavutil/file.h" + +static const char* leet_src = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"; +static const char* leet_dst = "abcd3f6#1jklmn0pq257uvwxyzAB(D3F6#1JKLMN0PQ257UVWXYZ"; + +enum TextModFilterType { + TM_TEXTMOD, + TM_CENSOR, + TM_SHOW_SPEAKER, +}; + +enum TextModOperation { + OP_LEET, + OP_TO_UPPER, + OP_TO_LOWER, + OP_REPLACE_CHARS, + OP_REMOVE_CHARS, + OP_REPLACE_WORDS, + OP_REMOVE_WORDS, + NB_OPS, +}; + +enum CensorMode { + CM_KEEP_FIRST_LAST, + CM_KEEP_FIRST, + CM_ALL, +}; + +enum ShowSpeakerMode { + SM_SQUARE_BRACKETS, + SM_ROUND_BRACKETS, + SM_COLON, + SM_PLAIN, +}; + +typedef struct TextModContext { + const AVClass *class; + enum AVSubtitleType format; + enum TextModFilterType filter_type; + enum TextModOperation operation; + enum CensorMode censor_mode; + enum ShowSpeakerMode speaker_mode; + char *find; + char *find_file; + char *style; + char *replace; + char *replace_file; + char *separator; + char *censor_char; + char **find_list; + int line_break; + int nb_find_list; + char **replace_list; + int nb_replace_list; +} TextModContext; + +static char **split_string(char *source, int *nb_elems, const char *delim) +{ + char **list = NULL; + char *temp = NULL; + char *ptr = av_strtok(source, delim, &temp); + + while (ptr) { + if (strlen(ptr)) { + av_dynarray_add(&list, nb_elems, ptr); + if (!list) + return NULL; + } + + ptr = av_strtok(NULL, delim, &temp); + } + + if (!list) + return NULL; + + for (int i = 0; i < *nb_elems; i++) { + list[i] = av_strdup(list[i]); + if (!list[i]) { + for (int n = 0; n < i; n++) + av_free(list[n]); + av_free(list); + return NULL; + } + } + + return list; +} + +static int load_text_from_file(AVFilterContext *ctx, const char *file_name, char **text, char separator) +{ + int err; + uint8_t *textbuf; + char *tmp; + size_t textbuf_size; + int offset = 0; + + if ((err = av_file_map(file_name, &textbuf, &textbuf_size, 0, ctx)) < 0) { + av_log(ctx, AV_LOG_ERROR, "The text file '%s' could not be read or is empty\n", file_name); + return err; + } + + if (textbuf_size > 1 && + (textbuf[0] == 0xFF && textbuf[1] == 0xFE + || textbuf[0] == 0xFE && textbuf[1] == 0xFF)) { + av_log(ctx, AV_LOG_ERROR, "UTF text files are not supported. File: %s\n", file_name); + return AVERROR(EINVAL); + } + + if (textbuf_size > 2 && textbuf[0] == 0xEF && textbuf[1] == 0xBB && textbuf[2] == 0xBF) + offset = 3; // UTF-8 + + if (textbuf_size > SIZE_MAX - 1 || !((tmp = av_strndup((char *)textbuf + offset, textbuf_size - offset)))) { + av_file_unmap(textbuf, textbuf_size); + return AVERROR(ENOMEM); + } + + av_file_unmap(textbuf, textbuf_size); + + for (size_t i = 0; i < strlen(tmp); i++) { + switch (tmp[i]) { + case '\n': + case '\r': + case '\f': + case '\v': + tmp[i] = separator; + } + } + + *text = tmp; + + return 0; +} + +static int load_files(AVFilterContext *ctx) +{ + TextModContext *s = ctx->priv; + int ret; + + if (!s->separator || strlen(s->separator) != 1) { + av_log(ctx, AV_LOG_ERROR, "A single character needs to be specified for the separator parameter.\n"); + return AVERROR(EINVAL); + } + + if (s->find_file && strlen(s->find_file)) { + ret = load_text_from_file(ctx, s->find_file, &s->find, s->separator[0]); + if (ret < 0 ) + return ret; + } + + if (s->replace_file && strlen(s->replace_file)) { + ret = load_text_from_file(ctx, s->replace_file, &s->replace, s->separator[0]); + if (ret < 0 ) + return ret; + } + + return 0; +} + +static int init_censor(AVFilterContext *ctx) +{ + TextModContext *s = ctx->priv; + int ret; + + s->filter_type = TM_CENSOR; + s->operation = OP_REPLACE_WORDS; + + ret = load_files(ctx); + if (ret < 0 ) + return ret; + + if (!s->find || !strlen(s->find)) { + av_log(ctx, AV_LOG_ERROR, "Either the 'words' or the 'words_file' parameter needs to be specified\n"); + return AVERROR(EINVAL); + } + + if (!s->censor_char || strlen(s->censor_char) != 1) { + av_log(ctx, AV_LOG_ERROR, "A single character needs to be specified for the censor_char parameter\n"); + return AVERROR(EINVAL); + } + + s->find_list = split_string(s->find, &s->nb_find_list, s->separator); + if (!s->find_list) + return AVERROR(ENOMEM); + + s->replace_list = av_calloc(s->nb_find_list, sizeof(char *)); + if (!s->replace_list) + return AVERROR(ENOMEM); + + for (int i = 0; i < s->nb_find_list; i++) { + size_t len, start = 0, end; + char *item = av_strdup(s->find_list[i]); + if (!item) + return AVERROR(ENOMEM); + + len = end = strlen(item); + + switch (s->censor_mode) { + case CM_KEEP_FIRST_LAST: + + if (len > 2) + start = 1; + if (len > 3) + end--; + + break; + case CM_KEEP_FIRST: + + if (len > 2) + start = 1; + + break; + } + + for (size_t n = start; n < end; n++) + item[n] = s->censor_char[0]; + + s->replace_list[i] = item; + } + + return 0; +} + +static int init_showspeaker(AVFilterContext *ctx) +{ + TextModContext *s = ctx->priv; + s->filter_type = TM_SHOW_SPEAKER; + + return 0; +} + +static int init(AVFilterContext *ctx) +{ + TextModContext *s = ctx->priv; + int ret; + + ret = load_files(ctx); + if (ret < 0 ) + return ret; + + switch (s->operation) { + case OP_REPLACE_CHARS: + case OP_REMOVE_CHARS: + case OP_REPLACE_WORDS: + case OP_REMOVE_WORDS: + if (!s->find || !strlen(s->find)) { + av_log(ctx, AV_LOG_ERROR, "Selected mode requires the 'find' parameter to be specified\n"); + return AVERROR(EINVAL); + } + break; + } + + switch (s->operation) { + case OP_REPLACE_CHARS: + case OP_REPLACE_WORDS: + if (!s->replace || !strlen(s->replace)) { + av_log(ctx, AV_LOG_ERROR, "Selected mode requires the 'replace' parameter to be specified\n"); + return AVERROR(EINVAL); + } + break; + } + + if (s->operation == OP_REPLACE_CHARS && strlen(s->find) != strlen(s->replace)) { + av_log(ctx, AV_LOG_ERROR, "Selected mode requires the 'find' and 'replace' parameters to have the same length\n"); + return AVERROR(EINVAL); + } + + if (s->operation == OP_REPLACE_WORDS || s->operation == OP_REMOVE_WORDS) { + if (!s->separator || strlen(s->separator) != 1) { + av_log(ctx, AV_LOG_ERROR, "Selected mode requires a single separator char to be specified\n"); + return AVERROR(EINVAL); + } + + s->find_list = split_string(s->find, &s->nb_find_list, s->separator); + if (!s->find_list) + return AVERROR(ENOMEM); + + if (s->operation == OP_REPLACE_WORDS) { + + s->replace_list = split_string(s->replace, &s->nb_replace_list, s->separator); + if (!s->replace_list) + return AVERROR(ENOMEM); + + if (s->nb_find_list != s->nb_replace_list) { + av_log(ctx, AV_LOG_ERROR, "The number of words in 'find' and 'replace' needs to be equal\n"); + return AVERROR(EINVAL); + } + } + } + + return 0; +} + +static void uninit(AVFilterContext *ctx) +{ + TextModContext *s = ctx->priv; + + for (int i = 0; i < s->nb_find_list; i++) + av_freep(&s->find_list[i]); + + s->nb_find_list = 0; + av_freep(&s->find_list); + + for (int i = 0; i < s->nb_replace_list; i++) + av_freep(&s->replace_list[i]); + + s->nb_replace_list = 0; + av_freep(&s->replace_list); +} + +static char *process_text(TextModContext *s, char *text) +{ + const char *char_src = s->find; + const char *char_dst = s->replace; + char *result = NULL; + int escape_level = 0, k = 0; + + switch (s->operation) { + case OP_LEET: + case OP_REPLACE_CHARS: + + if (s->operation == OP_LEET) { + char_src = leet_src; + char_dst = leet_dst; + } + + result = av_strdup(text); + if (!result) + return NULL; + + for (size_t n = 0; n < strlen(result); n++) { + if (result[n] == '{') + escape_level++; + + if (!escape_level) { + size_t len = strlen(char_src); + for (size_t t = 0; t < len; t++) { + if (result[n] == char_src[t]) { + result[n] = char_dst[t]; + break; + } + } + } + + if (result[n] == '}') + escape_level--; + } + + break; + case OP_TO_UPPER: + case OP_TO_LOWER: + + result = av_strdup(text); + if (!result) + return NULL; + + for (size_t n = 0; n < strlen(result); n++) { + if (result[n] == '{') + escape_level++; + if (!escape_level) + result[n] = s->operation == OP_TO_LOWER ? av_tolower(result[n]) : av_toupper(result[n]); + if (result[n] == '}') + escape_level--; + } + + break; + case OP_REMOVE_CHARS: + + result = av_strdup(text); + if (!result) + return NULL; + + for (size_t n = 0; n < strlen(result); n++) { + int skip_char = 0; + + if (result[n] == '{') + escape_level++; + + if (!escape_level) { + size_t len = strlen(char_src); + for (size_t t = 0; t < len; t++) { + if (result[n] == char_src[t]) { + skip_char = 1; + break; + } + } + } + + if (!skip_char) + result[k++] = result[n]; + + if (result[n] == '}') + escape_level--; + } + + result[k] = 0; + + break; + case OP_REPLACE_WORDS: + case OP_REMOVE_WORDS: + + result = av_strdup(text); + if (!result) + return NULL; + + for (int n = 0; n < s->nb_find_list; n++) { + char *tmp = result; + const char *replace = (s->operation == OP_REPLACE_WORDS) ? s->replace_list[n] : ""; + + result = av_strireplace(result, s->find_list[n], replace); + if (!result) + return NULL; + + av_free(tmp); + } + + break; + } + + return result; +} + +static char *process_dialog_show_speaker(TextModContext *s, char *ass_line) +{ + ASSDialog *dialog = avpriv_ass_split_dialog(NULL, ass_line); + int escape_level = 0; + unsigned pos = 0, len; + char *result, *text; + AVBPrint pbuf; + + if (!dialog) + return NULL; + + text = process_text(s, dialog->text); + if (!text) + return NULL; + + if (!dialog->name || !strlen(dialog->name) || !dialog->text || !strlen(dialog->text)) + return av_strdup(ass_line); + + // Find insertion point in case the line starts with style codes + len = (unsigned)strlen(dialog->text); + for (unsigned i = 0; i < len; i++) { + + if (dialog->text[i] == '{') + escape_level++; + + if (dialog->text[i] == '}') + escape_level--; + + if (escape_level == 0) { + pos = i; + break; + } + } + + if (s->style && strlen(s->style)) + // When a style is specified reset the insertion point + // (always add speaker plus style at the start in that case) + pos = 0; + + if (pos >= len - 1) + return av_strdup(ass_line); + + av_bprint_init(&pbuf, 1, AV_BPRINT_SIZE_UNLIMITED); + + if (pos > 0) { + av_bprint_append_data(&pbuf, dialog->text, pos); + } + + if (s->style && strlen(s->style)) { + if (s->style[0] == '{') + // Assume complete and valid style code, e.g. {\c&HFF0000&} + av_bprintf(&pbuf, "%s", s->style); + else + // Otherwise it must be a style name + av_bprintf(&pbuf, "{\\r%s}", s->style); + } + + switch (s->speaker_mode) { + case SM_SQUARE_BRACKETS: + av_bprintf(&pbuf, "[%s]", dialog->name); + break; + case SM_ROUND_BRACKETS: + av_bprintf(&pbuf, "(%s)", dialog->name); + break; + case SM_COLON: + av_bprintf(&pbuf, "%s:", dialog->name); + break; + case SM_PLAIN: + av_bprintf(&pbuf, "%s", dialog->name); + break; + } + + if (s->style && strlen(s->style)) { + // Reset line style + if (dialog->style && strlen(dialog->style) && !av_strcasecmp(dialog->style, "default")) + av_bprintf(&pbuf, "{\\r%s}", dialog->style); + else + av_bprintf(&pbuf, "{\\r}"); + } + + if (s->line_break) + av_bprintf(&pbuf, "\\N"); + else + av_bprintf(&pbuf, " "); + + av_bprint_append_data(&pbuf, dialog->text + pos, len - pos); + + av_bprint_finalize(&pbuf, &text); + + result = avpriv_ass_get_dialog(dialog->readorder, dialog->layer, dialog->style, dialog->name, text); + + av_free(text); + avpriv_ass_free_dialog(&dialog); + return result; +} + +static char *process_dialog(TextModContext *s, char *ass_line) +{ + ASSDialog *dialog; + char *result, *text; + + if (s->filter_type == TM_SHOW_SPEAKER) + return process_dialog_show_speaker(s, ass_line); + + dialog = avpriv_ass_split_dialog(NULL, ass_line); + if (!dialog) + return NULL; + + text = process_text(s, dialog->text); + if (!text) + return NULL; + + result = avpriv_ass_get_dialog(dialog->readorder, dialog->layer, dialog->style, dialog->name, text); + + av_free(text); + avpriv_ass_free_dialog(&dialog); + return result; +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *frame) +{ + TextModContext *s = inlink->dst->priv; + AVFilterLink *outlink = inlink->dst->outputs[0]; + int ret; + + outlink->format = inlink->format; + + ret = av_frame_make_writable(frame); + if (ret < 0) + return ret; + + for (unsigned i = 0; i < frame->num_subtitle_areas; i++) { + + AVSubtitleArea *area = frame->subtitle_areas[i]; + + if (area->ass) { + char *tmp = area->ass; + area->ass = process_dialog(s, area->ass); + av_free(tmp); + if (!area->ass) + return AVERROR(ENOMEM); + } + } + + return ff_filter_frame(outlink, frame); +} + +#define OFFSET(x) offsetof(TextModContext, x) +#define FLAGS (AV_OPT_FLAG_SUBTITLE_PARAM | AV_OPT_FLAG_FILTERING_PARAM) + +static const AVOption textmod_options[] = { + { "mode", "set operation mode", OFFSET(operation), AV_OPT_TYPE_INT, {.i64=OP_LEET}, OP_LEET, NB_OPS-1, FLAGS, "mode" }, + { "leet", "convert text to 'leet speak'", 0, AV_OPT_TYPE_CONST, {.i64=OP_LEET}, 0, 0, FLAGS, "mode" }, + { "to_upper", "change to upper case", 0, AV_OPT_TYPE_CONST, {.i64=OP_TO_UPPER}, 0, 0, FLAGS, "mode" }, + { "to_lower", "change to lower case", 0, AV_OPT_TYPE_CONST, {.i64=OP_TO_LOWER}, 0, 0, FLAGS, "mode" }, + { "replace_chars", "replace characters", 0, AV_OPT_TYPE_CONST, {.i64=OP_REPLACE_CHARS}, 0, 0, FLAGS, "mode" }, + { "remove_chars", "remove characters", 0, AV_OPT_TYPE_CONST, {.i64=OP_REMOVE_CHARS}, 0, 0, FLAGS, "mode" }, + { "replace_words", "replace words", 0, AV_OPT_TYPE_CONST, {.i64=OP_REPLACE_WORDS}, 0, 0, FLAGS, "mode" }, + { "remove_words", "remove words", 0, AV_OPT_TYPE_CONST, {.i64=OP_REMOVE_WORDS}, 0, 0, FLAGS, "mode" }, + { "find", "chars/words to find or remove", OFFSET(find), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, FLAGS, NULL }, + { "find_file", "load find param from file", OFFSET(find_file), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, FLAGS, NULL }, + { "replace", "chars/words to replace", OFFSET(replace), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, FLAGS, NULL }, + { "replace_file", "load replace param from file", OFFSET(replace_file), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, FLAGS, NULL }, + { "separator", "word separator", OFFSET(separator), AV_OPT_TYPE_STRING, {.str = ","}, 0, 0, FLAGS, NULL }, + { NULL }, +}; + + +static const AVOption censor_options[] = { + { "mode", "set censoring mode", OFFSET(censor_mode), AV_OPT_TYPE_INT, {.i64=CM_KEEP_FIRST_LAST}, 0, 2, FLAGS, "mode" }, + { "keep_first_last", "censor inner chars", 0, AV_OPT_TYPE_CONST, {.i64=CM_KEEP_FIRST_LAST}, 0, 0, FLAGS, "mode" }, + { "keep_first", "censor all but first char", 0, AV_OPT_TYPE_CONST, {.i64=CM_KEEP_FIRST}, 0, 0, FLAGS, "mode" }, + { "all", "censor all chars", 0, AV_OPT_TYPE_CONST, {.i64=CM_ALL}, 0, 0, FLAGS, "mode" }, + { "words", "list of words to censor", OFFSET(find), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, FLAGS, NULL }, + { "words_file", "path to word list file", OFFSET(find_file), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, FLAGS, NULL }, + { "separator", "word separator", OFFSET(separator), AV_OPT_TYPE_STRING, {.str = ","}, 0, 0, FLAGS, NULL }, + { "censor_char", "replacement character", OFFSET(censor_char), AV_OPT_TYPE_STRING, {.str = "*"}, 0, 0, FLAGS, NULL }, + { NULL }, +}; + +static const AVOption showspeaker_options[] = { + { "format", "speaker name formatting", OFFSET(speaker_mode), AV_OPT_TYPE_INT, {.i64=SM_SQUARE_BRACKETS}, 0, 2, FLAGS, "format" }, + { "square_brackets", "[speaker] text", 0, AV_OPT_TYPE_CONST, {.i64=SM_SQUARE_BRACKETS}, 0, 0, FLAGS, "format" }, + { "round_brackets", "(speaker) text", 0, AV_OPT_TYPE_CONST, {.i64=SM_ROUND_BRACKETS}, 0, 0, FLAGS, "format" }, + { "colon", "speaker: text", 0, AV_OPT_TYPE_CONST, {.i64=SM_COLON}, 0, 0, FLAGS, "format" }, + { "plain", "speaker text", 0, AV_OPT_TYPE_CONST, {.i64=SM_PLAIN}, 0, 0, FLAGS, "format" }, + { "line_break", "insert line break", OFFSET(line_break), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, FLAGS, NULL }, + { "style", "ass type name or style code", OFFSET(style), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, FLAGS, NULL }, + { NULL }, +}; + +AVFILTER_DEFINE_CLASS(textmod); +AVFILTER_DEFINE_CLASS(censor); +AVFILTER_DEFINE_CLASS(showspeaker); + +static const AVFilterPad inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .filter_frame = filter_frame, + }, +}; + +static const AVFilterPad outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + }, +}; + +const AVFilter ff_sf_textmod = { + .name = "textmod", + .description = NULL_IF_CONFIG_SMALL("Modify subtitle text in several ways"), + .init = init, + .uninit = uninit, + .priv_size = sizeof(TextModContext), + .priv_class = &textmod_class, + FILTER_INPUTS(inputs), + FILTER_OUTPUTS(outputs), + FILTER_SINGLE_SUBFMT(AV_SUBTITLE_FMT_ASS), +}; + +const AVFilter ff_sf_censor = { + .name = "censor", + .description = NULL_IF_CONFIG_SMALL("Censor words in subtitle text"), + .init = init_censor, + .uninit = uninit, + .priv_size = sizeof(TextModContext), + .priv_class = &censor_class, + FILTER_INPUTS(inputs), + FILTER_OUTPUTS(outputs), + FILTER_SINGLE_SUBFMT(AV_SUBTITLE_FMT_ASS), +}; + +const AVFilter ff_sf_showspeaker = { + .name = "showspeaker", + .description = NULL_IF_CONFIG_SMALL("Prepend speaker names to text subtitles (when available)"), + .init = init_showspeaker, + .uninit = uninit, + .priv_size = sizeof(TextModContext), + .priv_class = &showspeaker_class, + FILTER_INPUTS(inputs), + FILTER_OUTPUTS(outputs), + FILTER_SINGLE_SUBFMT(AV_SUBTITLE_FMT_ASS), +};