From patchwork Sun Sep 12 03:21:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 30173 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp2862735iov; Sat, 11 Sep 2021 20:21:44 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwjDZ+bNZS5Trfqkbw9XP5aidB/YVTbwHx8LnHrfKyYHg6Cbsxdsup80tNMH22fbiPH5clx X-Received: by 2002:a17:907:2168:: with SMTP id rl8mr5763377ejb.42.1631416904471; Sat, 11 Sep 2021 20:21:44 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id c24si3273559edn.26.2021.09.11.20.21.43; Sat, 11 Sep 2021 20:21:44 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=WhW5bSo+; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A849868A8A5; Sun, 12 Sep 2021 06:21:39 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10olkn2062.outbound.protection.outlook.com [40.92.41.62]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 481FF689985 for ; Sun, 12 Sep 2021 06:21:32 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TNvh2JSHic59Zf2HMAoBpmWhC2Vz8JgbPiUP9f7vkzTwI+2QXxrmh7VKfXTWT9KBP5dA/+P+Ory9i45X5lgpvIvkPQrdkyhqfq5ReLny9eQO0f9xvWlZBS7Mw0jlDWvYdfSWU1NS88tySj4xznMcgRQ8Dwc2y0ErwI9ADYUKYiUZ2wExwpcJV4AI0PSfbB6AmGJ0ksQJs0Wr+0zD0x5Zh9hFA8BK4hofVCK/jI9mzRmWJ1+6EwecB4lXrGY+uBNjuRhZxsCkK0jHORFsSF+K6xoQApo9PAo6t7BI0rrSqCglItr1f5wUGPTzJiKBovyex9tG3Ha+JzezyPWYGVhn8Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=POHYieuMSV/6QL613yp1z9nC8G5Bm1dGs8AwKKAog6w=; b=aqMVtm6sQvAYiFSNEdhzrJbS2QlITPEvyBeSJMR3Mz/lXo9wxgBylFCbhqhxjhQ+ZDth6ryoxytc/hcq0K7cL/4NLJUlQC2Fqm8SJp2hQgFQUF2N90CUGqLc8jPs5X4iG9sBbLqK7efwhoqvmMVZuaXugPDN/1wk6hjH6rnakHBqp4+ua7hYzy1G9l0hWXjwphoB52uPMlQtxBiNwl9FRhWqjpHll7kIyCbMm3FshQZCrAL+/HzegWk6zc3brBCnFOf8Dq2sE19PZlqhNrtiWFrcTy4YlmCI2rx4A8tk7bX9iirBZcB7UY63i/mHF/SLgwmBtzxUsuMtpA8HU5HUCw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=POHYieuMSV/6QL613yp1z9nC8G5Bm1dGs8AwKKAog6w=; b=WhW5bSo+r8FfXBuAgaJIElYcD+XgKSHHPQUeTRiFJnQ6b+Q/k4vH1xSc/udf0lKu1Y9MhdSAUKkLCY4gxA1LpPuNXEr9PWVR5/W7F/aN2ENuoPtF+VBFCw3JJR4HZKMaeiWAFsfkxvGi6F6LBwid15Z8npgCp8EWNxMn/pAs4uwwdUhe0JX5G4vhStzjtQ7PjyXrpqnZ2nOHdX7siCubNgi1xKkDyNPfuq6BWrgogpZvnGwgYmLIXDrli+pmYJoj2jWwH86ttAyWp6hkerUS0WlR0/g/4hpbG3LbLaLZ1PyQAW+OWZ+xmm/VnnwcK+HelFtZqxoHtLa34PyVMjUKyg== Received: from MN2PR04MB5981.namprd04.prod.outlook.com (2603:10b6:208:da::10) by MN2PR04MB5728.namprd04.prod.outlook.com (2603:10b6:208:3d::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Sun, 12 Sep 2021 03:21:28 +0000 Received: from MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb]) by MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb%5]) with mapi id 15.20.4500.018; Sun, 12 Sep 2021 03:21:28 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v5 00/12] Subtitle Filtering Thread-Index: AQFr7TS1NB+ALXcMFT/CM2pAugkk4g== Date: Sun, 12 Sep 2021 03:21:28 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [qZmtvrEBo8NthhE/skBdjcahuNFdmfvTEQuWsZggYEw=] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 0aaf2334-7119-45eb-e11b-08d9759c68b0 x-ms-traffictypediagnostic: MN2PR04MB5728: x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 1pluxJVlR6HL1/TyMwdlbAIkB0aaR2tTT/kBLXwmJ95+ipi5i7mWJIlR3KsZjztVSIejmcVnGQfrytxU6VMGK0plGI7M+l8oTJMkYc5rtLTxCKKG9p2NntyKZqZ/CnvLkLSDGYCpNouVsQEO+5vjTQYiUo58htDnMMy9Y5pdDJu5feLOIF8ge+moqmi9VS6QebwlfWbVXYVpMTc1u9oe4dupE937ISkYG+wtbjvFFneOMSNiQVR6bh8pVahy13P8CWBkbolOpPbrv/CybkspIXm20Z3vjj4Hj4hDgjpXQWC0FtaNRQ7gc2ialUX90SdWO6j0DdhMRGIc9mOmjxfpblPxMwEtwVqZFMZj5UZ7tEg2qTzrWggObuSSmsJWAe8AVhLawLlq26E7qRY6wdrWcC3J7Z11DMTmiI1Eg1Sc+nLGBmSbwc2KbaNjTep+rxB1 x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: RqDPfOEke0JC+w6LPbMB4MWCw1Zp5RValdxlDYXVlPy5FbtZS7RYulEYVaCcCGZFRTmq2J11G9TX6Rsuxe8noGAAwvRTE6C3yZ2X23SYunyAGbMwJw9S9aL7SV4zel4T1TNEjzDSdQ9meOZRVgGHnA== x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-3174-20-msonline-outlook-529c7.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR04MB5981.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 0aaf2334-7119-45eb-e11b-08d9759c68b0 X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Sep 2021 03:21:28.0526 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR04MB5728 Subject: [FFmpeg-devel] [PATCH v5 00/12] Subtitle Filtering X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: AgpeWgOnk3Fc v5 Update: - Merge AVSubtitle into AVFrame - Move FATE test adjustments to corresponding commit - Move documentation updates to corresponding filter commits - Remove mediatype parameter from av_frame_get_buffer2 (still need some advice for splitting the commits in a way that every single one will pass FATE) v4 Update: - Re-Sending due to Patchwork having failed to parse my patchset There seems to be a bug in Patchwork when parallel processing is enabled. This time, I'll send the e-mails slowly, one after another. v3 Update: - Reworked, revised, rebased, reorganized, refactored - No more prototype/test-style code - Additional subtitle filters added - Filter documentation added - Adjusted FATE tests This patchset is about introducing filtering support for subtitles. The current sub2video "hack" implementation is slow, ineffective and limited in capabilities. => This patchset introduces true subtitle filtering These new filters are included: - overlay_graphicsubs (VS -> V) Overlay graphic subtitles onto a video stream - graphicsub2video {S -> V) Converts graphic subtitles to (transparent) video frames - overlay_textsubs {VS -> V) Overlay text subtitles onto a video stream. - textsubs2video {S -> V) Converts text subtitles to video frames - textmod {S -> S) Modify subtitle text in a number of ways - stripstyles {S -> S) Remove all inline styles from subtitle events Regards, softworkz softworkz (12): avutil/frame: Subtitle Filtering - Add AVMediaType property to AVFrame global: Merge AVSubtitle into AVFrame fftools/play,probe: Adjust for subtitle format type change fftools/ffmpeg: Replace sub2video with subtitle frame filtering avfilter/subtitles: Add subtitles.c avfilter/avfilter: Handle subtitle frames avfilter/sbuffer: Add sbuffersrv and sbuffersink filters avfilter/overlay_graphicsubs: Add overlay_graphicsubs and graphicsub2video filters avfilter/overlay_textsubs: Add overlay_textsubs and textsubs2video filters avfilter/textmod: Add textmod filter avcodec/ass_split: Extend ass dialog parsing avfilter/stripstyles: Add stripstyles filter configure | 4 +- doc/filters.texi | 261 ++++++++ fftools/ffmpeg.c | 435 ++++++------- fftools/ffmpeg.h | 14 +- fftools/ffmpeg_filter.c | 198 ++++-- fftools/ffmpeg_hw.c | 2 +- fftools/ffmpeg_opt.c | 3 +- fftools/ffplay.c | 30 +- fftools/ffprobe.c | 49 +- libavcodec/ass.c | 16 +- libavcodec/ass.h | 8 +- libavcodec/ass_split.c | 12 +- libavcodec/ass_split.h | 2 + libavcodec/assdec.c | 18 +- libavcodec/assenc.c | 10 +- libavcodec/avcodec.c | 19 - libavcodec/avcodec.h | 81 +-- libavcodec/ccaption_dec.c | 28 +- libavcodec/codec.h | 4 +- libavcodec/codec_desc.h | 4 +- libavcodec/decode.c | 23 +- libavcodec/dvbsubdec.c | 48 +- libavcodec/dvbsubenc.c | 72 +-- libavcodec/dvdsubdec.c | 122 ++-- libavcodec/dvdsubenc.c | 40 +- libavcodec/encode.c | 6 +- libavcodec/jacosubdec.c | 4 +- libavcodec/libzvbi-teletextdec.c | 16 +- libavcodec/microdvddec.c | 4 +- libavcodec/movtextdec.c | 4 +- libavcodec/movtextenc.c | 10 +- libavcodec/mpl2dec.c | 4 +- libavcodec/pgssubdec.c | 68 +- libavcodec/realtextdec.c | 4 +- libavcodec/samidec.c | 4 +- libavcodec/srtdec.c | 4 +- libavcodec/srtenc.c | 14 +- libavcodec/subviewerdec.c | 4 +- libavcodec/textdec.c | 4 +- libavcodec/ttmlenc.c | 10 +- libavcodec/utils.c | 11 + libavcodec/webvttdec.c | 4 +- libavcodec/webvttenc.c | 10 +- libavcodec/xsubdec.c | 58 +- libavcodec/xsubenc.c | 48 +- libavfilter/Makefile | 9 + libavfilter/allfilters.c | 16 +- libavfilter/avfilter.c | 26 +- libavfilter/avfiltergraph.c | 5 + libavfilter/buffersink.c | 63 ++ libavfilter/buffersink.h | 15 + libavfilter/buffersrc.c | 72 +++ libavfilter/buffersrc.h | 1 + libavfilter/f_interleave.c | 3 + libavfilter/formats.c | 14 + libavfilter/formats.h | 3 + libavfilter/internal.h | 1 + libavfilter/sf_stripstyles.c | 215 +++++++ libavfilter/sf_textmod.c | 381 +++++++++++ libavfilter/subtitles.c | 61 ++ libavfilter/subtitles.h | 44 ++ libavfilter/version.h | 2 +- libavfilter/vf_overlay_graphicsubs.c | 731 ++++++++++++++++++++++ libavfilter/vf_overlay_textsubs.c | 633 +++++++++++++++++++ libavfilter/vf_subtitles.c | 28 +- libavformat/utils.c | 5 +- libavutil/Makefile | 2 + libavutil/frame.c | 126 +++- libavutil/frame.h | 51 +- libavutil/subfmt.c | 52 ++ libavutil/subfmt.h | 94 +++ libavutil/version.h | 2 +- tests/ref/fate/filter-overlay-dvdsub-2397 | 181 +++--- tests/ref/fate/sub-dvb | 162 ++--- tests/ref/fate/sub2video | 178 ++---- tests/ref/fate/sub2video_basic | 93 ++- tests/ref/fate/sub2video_time_limited | 6 +- 77 files changed, 3914 insertions(+), 1155 deletions(-) create mode 100644 libavfilter/sf_stripstyles.c create mode 100644 libavfilter/sf_textmod.c create mode 100644 libavfilter/subtitles.c create mode 100644 libavfilter/subtitles.h create mode 100644 libavfilter/vf_overlay_graphicsubs.c create mode 100644 libavfilter/vf_overlay_textsubs.c create mode 100644 libavutil/subfmt.c create mode 100644 libavutil/subfmt.h