From patchwork Tue Sep 21 23:54:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 30440 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:6506:0:0:0:0:0 with SMTP id z6csp2918837iob; Tue, 21 Sep 2021 16:54:38 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx6KfaDh+kTB1rH6S+gIywHvXU/5yLWmvPkuaGU8T4evGsEiziA0a7R+ErjKe6LcCpvtOw8 X-Received: by 2002:a17:907:760d:: with SMTP id jx13mr35364414ejc.194.1632268478567; Tue, 21 Sep 2021 16:54:38 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 26si554029ejl.744.2021.09.21.16.54.32; Tue, 21 Sep 2021 16:54:38 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=lCTfLntW; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 68EBA68AF33; Wed, 22 Sep 2021 02:54:28 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12olkn2109.outbound.protection.outlook.com [40.92.23.109]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 80D8668A9ED for ; Wed, 22 Sep 2021 02:54:20 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FzYro6VUZz8HhyDIdlK2DQOxYjyM9CvJaRguxj5GBZoY3uIC6e3zUKgu7lic8Avc5H2YEyZc+Yoy2cijEZ3A3lDVlfT2K8Ar4r3FLiD3IDU3UlqT5m6JUzkf9CqOtynW5kRzDIUADRqCzCZLbg6Wr0OT3deVK47Qic+HlyC3S6vXjQniU06FKxmfVUL8quWElPcOUfAlZN6ziwSGRoHwFSN8RWa3LjycKBssP3VkMlWLxjrbCWTu7knPyns6owiOFSZYVSZ5uf+UGYivKVFdOwlx/077/f0vEn1pm0lxD/6l88G5OD12oMhYQEVlw1bhj1R3CWOtveS7lySF/vkNYA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=00lUW9dkzg9TWIG+/IdYmDjIm1l3SGLXhX2MTCdBSMI=; b=UfZRGm/9uYGw7BesNkB04E9xXX6XD1EvR+Z8tNFmItXrZlx+grk4FGeY2q+un0JElYkVnW1JZb/1YC/mYbEJqPdvSf5sM7oskvGAewrLFT+9I8mVfUXuoodm2K2FCWCOXNDKWF1gY/hUOteKT2v1Q9jg3p/W+xKR2C3cjuqQ9bY6TVjwUW/XfGT6zXMre273vRHZg3LGgd+SgaSDMfP+eb3yekRUtIyxhELghb1n4wUgLALBElQqrs1o3VnRo2qOfILCWqgFuF+WyyXfmBf+4fH6Fqq8IpnGfro6PqmhJhwUW4HS7iQFq0VEuumY3NxqTDCVDp5XkVP2e/vrzySMxw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=00lUW9dkzg9TWIG+/IdYmDjIm1l3SGLXhX2MTCdBSMI=; b=lCTfLntWlJRCLX7LX1ZIsr7J7lz34gdTu3zUlSAricbN02fKeMu9ykMWjQ9BsZ2aUMZ1MgQai+UHTxug+8Zzn0EQ+7GLqxzCCxMHi/tD0v0N7mCNY6vEWi1T/EKtJfkM4iXquQaCku0aHt5dTecDv9T2X6uQuNMNrszQaKXjsx1fSKZHDbwhT8mDn0F64avRkF2l9CswIl3TCYgR9EbO5Z9PlUFv0xAwDw8jw/C/Txc3N6+bDk2E3GxV6LgQpR6mEuXmoO94BMEZrZvmvD40LOI+gwT1/s96NBaAKOmccdlQd2+m+G9Lv6Vo95l+533ffwvNWVeD9EpPB/+15lFGEg== Received: from MN2PR04MB5981.namprd04.prod.outlook.com (2603:10b6:208:da::10) by MN2PR04MB5520.namprd04.prod.outlook.com (2603:10b6:208:d7::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Tue, 21 Sep 2021 23:54:16 +0000 Received: from MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb]) by MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb%5]) with mapi id 15.20.4523.019; Tue, 21 Sep 2021 23:54:16 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v8 00/13] Subtitle Filtering Thread-Index: AQMAH2Lc4uYoI7dTrB1UP3HOEkSqog== Date: Tue, 21 Sep 2021 23:54:16 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [bLWVNzd0P4wX9rFYLu5lC4QxUc7Q91Kr] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: c1500cc7-f103-42da-9b3b-08d97d5b1eda x-ms-traffictypediagnostic: MN2PR04MB5520: x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: c8+DE1EIU0oZEl0OvTfNDyBLfc7BTQtyHLbXUSHyhOG98rivaFYWWq59Tt9RX6nfGPzOsZYYveYMa1QPaYI4lO08ZZG7fOnWvaxnTlFEpOXjd2qlCR6ytjHYd2OzKIhaq4gWcr8ZoKCyyl+xtLj+QtBMmp+oQSmoEKufczBrTBCN1ZYYv/MFC+8k0A7iSc495e8ClaOPMC8UgwB8GKp7wNodU1TgXWgFsVBWPirLR5OOjjq7td5/U1ZVcRjGM26Mik0ZWexn6pXWE0tE01dUtygOQuY6mIw4XUA86YAQhiy1/8aYG96RVWUJGSv7+gQx84Fit/j2LJw6YqOrYUsPJFPvomq2pQeEr3DLxkLMhJe0J6R2bwPf9Z9r4/816h7S8rCYkV3UNwyiQvK4WhWK/S9s/cTorvkzfFhN5mjAj0RJT72yjsqI+r4PHhPSU/3K x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: aAn49uQuTV/Wj4bHlYevn2TO433FKLWWqC9DQ39tg8ohFyGh6piyUQIh0Bdlie2k4GXKGb4khd9jGJyGK7J7RfzSVgwlaJZj7rDoKF1BbaLVpcYWJEWLC+CkSI5Q2X3s73vaOjcTPGLArfidiYdRrA== x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-3174-20-msonline-outlook-529c7.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR04MB5981.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: c1500cc7-f103-42da-9b3b-08d97d5b1eda X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Sep 2021 23:54:16.0935 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR04MB5520 Subject: [FFmpeg-devel] [PATCH v8 00/13] Subtitle Filtering X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: RU0YD/gG/u53 v8 Update: - Added new subtitle filters: graphicsub2text, censor, show_speaker (see below) - Added parameters to load words from file for textmod and censor - Copied copyright owners from vf_overlay to vf_overlay_graphicsubs - Remove "convenience" pointers (data) from AVSubtitleArea (not needed, will avoid migration errors) - Minor fixes -------------------------------------------------------------- This patchset introduces filtering support for subtitles. Besides the ground work, this patchset includes a range of new filters for subtitle processing: - graphicsubs2text (S -> S) Convert graphic subtitles to text subtitles via OCR - censor {S -> S) Allows censoring word in subtitles by a configurable word list - show_speaker {S -> S) Prepend the speaker names to text lines (those are often available in ass subs) - split_cc (V -> VS) Splits out Closed-Caption data as a subtitle stream for filtering and encoding - overlay_graphicsubs (VS -> V) Overlay graphic subtitles onto a video stream - graphicsub2video {S -> V) Converts graphic subtitles to (transparent) video frames - overlay_textsubs {VS -> V) Overlay text subtitles onto a video stream. - textsubs2video {S -> V) Converts text subtitles to video frames - textmod {S -> S) Modify subtitle text in a number of ways - stripstyles {S -> S) Remove all inline styles from subtitle events Regards, softworkz softworkz (13): global: Prepare AVFrame for subtitle handling fftools/play,probe: Adjust for subtitle changes avfilter/subtitles: Add subtitles.c for subtitle frame allocation avfilter/avfilter: Handle subtitle frames avfilter/sbuffer: Add sbuffersrc and sbuffersink filters avfilter/overlay_graphicsubs: Add overlay_graphicsubs and graphicsub2video filters fftools/ffmpeg: Replace sub2video with subtitle frame filtering avfilter/overlay_textsubs: Add overlay_textsubs and textsubs2video filters avfilter/textmod: Add textmod, censor and show_speaker filters avcodec/ass_split: Extend ass dialog parsing avfilter/stripstyles: Add stripstyles filter avfilter/split_cc: Add split_cc filter for closed caption handling avfilter/graphicsub2text: Add new graphicsub2text filter (OCR) configure | 5 +- doc/filters.texi | 482 ++++++++++++++ fftools/ffmpeg.c | 460 ++++++-------- fftools/ffmpeg.h | 14 +- fftools/ffmpeg_filter.c | 200 ++++-- fftools/ffmpeg_hw.c | 2 +- fftools/ffmpeg_opt.c | 3 +- fftools/ffplay.c | 79 +-- fftools/ffprobe.c | 49 +- libavcodec/ass_split.c | 12 +- libavcodec/ass_split.h | 2 + libavcodec/avcodec.c | 19 - libavcodec/avcodec.h | 105 ++-- libavcodec/decode.c | 24 +- libavcodec/pgssubdec.c | 1 + libavcodec/utils.c | 11 + libavfilter/Makefile | 13 + libavfilter/allfilters.c | 20 +- libavfilter/avfilter.c | 30 +- libavfilter/avfiltergraph.c | 5 + libavfilter/buffersink.c | 63 ++ libavfilter/buffersink.h | 15 + libavfilter/buffersrc.c | 72 +++ libavfilter/buffersrc.h | 1 + libavfilter/formats.c | 14 + libavfilter/formats.h | 3 + libavfilter/internal.h | 1 + libavfilter/sf_graphicsub2text.c | 326 ++++++++++ libavfilter/sf_split_cc.c | 272 ++++++++ libavfilter/sf_stripstyles.c | 211 +++++++ libavfilter/sf_textmod.c | 727 +++++++++++++++++++++ libavfilter/subtitles.c | 61 ++ libavfilter/subtitles.h | 44 ++ libavfilter/version.h | 2 +- libavfilter/vf_overlay_graphicsubs.c | 730 ++++++++++++++++++++++ libavfilter/vf_overlay_textsubs.c | 615 ++++++++++++++++++ libavfilter/vf_subtitles.c | 27 +- libavformat/utils.c | 1 + libavutil/Makefile | 2 + libavutil/frame.c | 187 +++++- libavutil/frame.h | 93 ++- libavutil/subfmt.c | 219 +++++++ libavutil/subfmt.h | 185 ++++++ tests/ref/fate/filter-overlay-dvdsub-2397 | 181 +++--- tests/ref/fate/sub-dvb | 162 ++--- tests/ref/fate/sub2video | 44 -- tests/ref/fate/sub2video_basic | 93 ++- tests/ref/fate/sub2video_time_limited | 4 +- 48 files changed, 5117 insertions(+), 774 deletions(-) create mode 100644 libavfilter/sf_graphicsub2text.c create mode 100644 libavfilter/sf_split_cc.c create mode 100644 libavfilter/sf_stripstyles.c create mode 100644 libavfilter/sf_textmod.c create mode 100644 libavfilter/subtitles.c create mode 100644 libavfilter/subtitles.h create mode 100644 libavfilter/vf_overlay_graphicsubs.c create mode 100644 libavfilter/vf_overlay_textsubs.c create mode 100644 libavutil/subfmt.c create mode 100644 libavutil/subfmt.h