From patchwork Thu Nov 25 00:47:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 31637 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:d206:0:0:0:0:0 with SMTP id q6csp11413703iob; Wed, 24 Nov 2021 16:48:06 -0800 (PST) X-Google-Smtp-Source: ABdhPJxKHAwXHX2NFEP34FufSmkn69SkbveKLefluqoutWnNvbV8si5QlssJALDedzyGsn+rZIHa X-Received: by 2002:a17:907:774d:: with SMTP id kx13mr27379108ejc.239.1637801286379; Wed, 24 Nov 2021 16:48:06 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id g9si5161355edb.263.2021.11.24.16.48.05; Wed, 24 Nov 2021 16:48:06 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=mFv4wDFC; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id DDB5C68AB7A; Thu, 25 Nov 2021 02:48:00 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10olkn2054.outbound.protection.outlook.com [40.92.40.54]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 23A66689F94 for ; Thu, 25 Nov 2021 02:47:54 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ju6L7IA30XTNggRv93Q6sKkKuWsngQD5WXZEf+aDRPfaHwa0CJhvL4IdnvQntvjYfj7y6uRoHRLeBkzef9Lz/3Wz7Gr9ehfgIsgndbto33d4BDA5R5uIbqSGZMPUgkP7s4dnO9Ey85ES+aekpxTtO4ZVaXLel/k9HNyqSnUZWxwOh3Qah7F7MAzYJC4xT4BxpoyI+VeCdpM18291+C2CxfHMiaC5zTkdYJeDtEeJ0RcQU3Y55QRoChZlhFUBL1qj8fZXPnCMO7Ym578tnwkS7565L8suHoaHTH25zv+Pd3aLU8xJnj9I0GbtQwfA9B5JtfrC2PxIE0zJshUbpdtZFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=5OW1BnZbqmceMv/SERbGxKkfwM47Hvralr/BGKLjGt0=; b=SL7CIooa5JU+MFprDluE5CohBr7xe66+NM7E4iQVCgCuU01zpwoQkNnWEX7tlSxqM2/o7nJ7jy5k3ucUjnCEq09Ul1J3qaH/7PpnVCZE41hppGwhHx/hjwJv2GXOeuKppUle79t25ZQz0zQFLUOp281Vmg3ImaFH1uXmz19Pk40b/CJC8Az6UJGkafboDl8m+E0f3sytadUMKxT55SklRxkRuOis3U4HB58TonpDmml0gE08Pgv7lZeSAhnbJUQ3zf4dIWK+shfGz6YiA8gDFLixGs5Pu5wsGx4K0y5JPL0is71YNw+5h/vDNqcfzVFm0+Vx6SohjFjcnW4EV7OTmg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5OW1BnZbqmceMv/SERbGxKkfwM47Hvralr/BGKLjGt0=; b=mFv4wDFCj+dinTZizeqV/Olw+3fQVA+lnRORy0HCadBiTqACxZ7aQmTULXV9tXYkWfJ1Hrx7rkOsfDOliYnXPmkMs+rvx3BDkFM7Cku+m9DlAItFo52eSixXAsFSXXBH8xTbZX4Mff1k/y+oVM8c2Q6Gvr2NbcRrbPiEylXT5kkEFlKxGQRyUqqAgXkbMCfdz5aAvPOPeNLCRoAcVFAjiSrNaSmBnkVTNNp1FxIAoOKJRdmjllXLzVkXacT7al0kwQpxNloqKxNU/6MujZqrad+eo/Y0+cgXhh4kn4HaXkZ6/ZaVGbrw+42xQKHj8SEf5Ecz9S6it2dmZorGFOGpWw== Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::20) by DM8P223MB0063.NAMP223.PROD.OUTLOOK.COM (2603:10b6:5:316::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4713.19; Thu, 25 Nov 2021 00:47:51 +0000 Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::8c05:4078:21a:53fb]) by DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::8c05:4078:21a:53fb%5]) with mapi id 15.20.4734.020; Thu, 25 Nov 2021 00:47:51 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v15 00/16] *** SUBJECT HERE *** Thread-Index: AQG8RZwpV+QPIiMKcaPrI9wVEkbi3A== Date: Thu, 25 Nov 2021 00:47:51 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [5tD5FfFkzcJE4z7qb4+r94Zd6ByQIIV6] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: e1022d90-6e46-46d1-fb9c-08d9afad359d x-ms-traffictypediagnostic: DM8P223MB0063: x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: DgakHMFIKkqUPx6htsys2ob63m38jzu9aSUaHBBcOWxCG48987I6Wj2O/5Fbkymu67DJfOOMhEW8z3c8r32J4sNyP5ACIhR4RZ5UclqLDiXnuXEZj0sAEHDZstpNvAPIU3WHf+F9gIwuAMT+ywzoLKq2MLS88zvlPl2tgyZW5LDRog60CsfQ4PHIH6qpPJ9KSxJQ7YEA3EQiURk10CwNUzvGPr+vCiV+pNwlKQBl4gWwxp9v8klfyTm2xkaObxNsDwdgR3Ka9JPQU3CdCJDCujkxbSPoFLuCjVIVNnkmCSdqn6BYqz1R+Btj0hSGK9F+V0zeZb3Lvs4U+bWRw3ZPIkmQWq3M3Qll2TDf2fzBDhM+ydJC6kXTM9CJpbfrQHiCi0Nr19GF+gILcg8H7mNqUsxKpvwM3L+s/kKpG/cnTfuQ9OR1I1NlYOSGkpzXQ0TTSdDOPLqtAOuGcdVgLCh+8UKw/JHsG/cO+Cur54rbHLuALgTU7brg7LlnXDuym/qCn/vKT76VNto2x5K/ff2Lrds4BKaqER32jKfdtBniw4f3SMFaPNbdAYTjekC7Qm2MpV3pXumZOc9DEpXa2Jy6MQ== x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: DiQYSptAKYYPsjJg7euQTtOpEU8PrUbp7rrmtu4bLMhl3hPXON9d0lb3adjpyCYEMUl33MQG+LDNGGxA36kcrnqRhIiOAfEyZGBKfidi7pQMmxCHv55pj8C3UgxFKVm6qD+fYiM4M+5Fcn/lzeTcL3kAaB2SPMSSYvHhL53ngO/U139iyzsqsx2bRymUqiMz40+NbsBgSYGC9UPXEQ91rh433FT4IyAFR/68MOkm76QJaH7VOrgqS9P1ENI6M6PWXH0anVhvCQ5o0ueZpkde3zlY/csGR2HZDqhckm/sTEf9LEVrz4dHkYVhJVefKDek44yJjuXShZJUb4qW9pYq/6vxSTE8d5V+LUClyUtyH9JhsMYKZkcqHqQjNCWKVvbi3uZk/S0ncsk1rsIo7QCC0KtcDJbxZ+WFUwp1mroU+x1n41HyRaHcl29MaTVi3VNMk77X8IwqBCWpUrCdvIpg1rBH0pD2WyzZGbC6ZPc7P3sE8S9YyoEeEwcpaY1DVv6sF1aX+Rbk59CcwtnZNAptk4KLy+Honn4xDkhxvTh/1yjAyrsmMOmx1w7ZB0Ciirde2fOb8r1Xh3nKPfgcH2Ys6QFw8m3UNguC932S76rKBQ8FaLWijTv7EBvIqOZcXLZu9M/GGURQR0Y21rzAc+cZCpyyFW2jukL0hLS/NqMA9CmHHlhF+22xooNTQq9+Vrx4WeFCtG4GA+Wp4G06D1uKdw== MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-3912-16-msonline-outlook-1fc6d.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: e1022d90-6e46-46d1-fb9c-08d9afad359d X-MS-Exchange-CrossTenant-originalarrivaltime: 25 Nov 2021 00:47:51.3056 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8P223MB0063 Subject: [FFmpeg-devel] [PATCH v15 00/16] *** SUBJECT HERE *** X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: gDFtQege66U6 Root commit for adding subtitle filtering capabilities. In detail: - Add type (AVMediaType) field to AVFrame Replaces previous way of distinction which was based on checking width and height to determine whether a frame is audio or video - Add subtitle fields to AVFrame - Add new struct AVSubtitleArea, similar to AVSubtitleRect, but different allocation logic. Cannot and must not be used interchangeably, hence the new struct - Move enum AVSubtitleType, AVSubtitle and AVSubtitleRect to avutil - Add public-named members to enum AVSubtitleType (AV_SUBTITLE_FMT_) - Add avcodec_decode_subtitle3 which takes subtitle frames, serving as compatibility shim to legacy subtitle decoding - Add additional methods for conversion between old and new API New in V15 - Rebased to upstream changes - avcodec/subtitles: Migrate subtitle encoders to frame-based API and provide compatibility shim for legacy api - fftools/ffmpeg: Use new frame-based subtitle encoding API - AVSubtitleArea: copy flags field, make params const - graphicsubs2text: Don't emit duplicate frames - graphicsubs2text: Combined OCR output into a single AVSubtitleArea (I have a prototype for detecting text colors and positions, but it's not ready at this point) - splitcc: cleanup local subtitle_header ref - stripstyles: add parameter for ass layer selection - avcodec/subtitles: deferred loading of ass header for text subtitle encoders - verified all example command lines in the docs are working, added somre more Kind regards, softworkz softworkz (16): global: Prepare AVFrame for subtitle handling global: Move ass helper functions to avutil as avpriv_ and extend ass dialog parsing fftools/play,probe: Adjust for subtitle changes avfilter/subtitles: Add subtitles.c for subtitle frame allocation avfilter/avfilter: Handle subtitle frames avfilter/sbuffer: Add sbuffersrc and sbuffersink filters avfilter/overlaygraphicsubs: Add overlaygraphicsubs and graphicsub2video filters fftools/ffmpeg: Replace sub2video with subtitle frame filtering avfilter/overlaytextsubs: Add overlaytextsubs and textsubs2video filters avfilter/textmod: Add textmod, censor and show_speaker filters avfilter/stripstyles: Add stripstyles filter avfilter/splitcc: Add splitcc filter for closed caption handling avfilter/graphicsub2text: Add new graphicsub2text filter (OCR) avfilter/subscale: Add filter for scaling and/or re-arranging graphical subtitles avcodec/subtitles: Migrate subtitle encoders to frame-based API and provide a compatibility shim for the legacy api fftools/ffmpeg: Use new frame-based subtitle encoding API configure | 7 +- doc/filters.texi | 756 +++++++++++++++ fftools/ffmpeg.c | 553 ++++++----- fftools/ffmpeg.h | 15 +- fftools/ffmpeg_filter.c | 217 +++-- fftools/ffmpeg_hw.c | 2 +- fftools/ffmpeg_opt.c | 3 +- fftools/ffplay.c | 102 +- fftools/ffprobe.c | 48 +- libavcodec/Makefile | 56 +- libavcodec/ass.h | 129 +-- libavcodec/assdec.c | 2 +- libavcodec/assenc.c | 83 +- libavcodec/avcodec.c | 19 - libavcodec/avcodec.h | 82 +- libavcodec/ccaption_dec.c | 19 +- libavcodec/decode.c | 53 +- libavcodec/dvbsubenc.c | 85 +- libavcodec/dvdsubenc.c | 89 +- libavcodec/encode.c | 97 +- libavcodec/jacosubdec.c | 2 +- libavcodec/libaribb24.c | 2 +- libavcodec/libzvbi-teletextdec.c | 14 +- libavcodec/microdvddec.c | 7 +- libavcodec/movtextdec.c | 3 +- libavcodec/movtextenc.c | 115 ++- libavcodec/mpl2dec.c | 2 +- libavcodec/pgssubdec.c | 1 + libavcodec/realtextdec.c | 2 +- libavcodec/samidec.c | 2 +- libavcodec/srtdec.c | 2 +- libavcodec/srtenc.c | 104 ++- libavcodec/subviewerdec.c | 2 +- libavcodec/tests/avcodec.c | 2 - libavcodec/textdec.c | 4 +- libavcodec/ttmlenc.c | 95 +- libavcodec/utils.c | 11 + libavcodec/webvttdec.c | 2 +- libavcodec/webvttenc.c | 81 +- libavcodec/xsubenc.c | 65 +- libavfilter/Makefile | 15 + libavfilter/allfilters.c | 13 + libavfilter/avfilter.c | 30 +- libavfilter/avfilter.h | 11 + libavfilter/avfiltergraph.c | 5 + libavfilter/buffersink.c | 63 ++ libavfilter/buffersink.h | 15 + libavfilter/buffersrc.c | 72 ++ libavfilter/buffersrc.h | 1 + libavfilter/formats.c | 22 + libavfilter/formats.h | 3 + libavfilter/internal.h | 19 +- libavfilter/sf_graphicsub2text.c | 354 +++++++ libavfilter/sf_splitcc.c | 316 +++++++ libavfilter/sf_stripstyles.c | 196 ++++ libavfilter/sf_subscale.c | 883 ++++++++++++++++++ libavfilter/sf_textmod.c | 697 ++++++++++++++ libavfilter/subtitles.c | 63 ++ libavfilter/subtitles.h | 44 + libavfilter/vf_overlaygraphicsubs.c | 737 +++++++++++++++ libavfilter/vf_overlaytextsubs.c | 624 +++++++++++++ libavfilter/vf_subtitles.c | 50 +- libavformat/utils.c | 1 + libavutil/Makefile | 4 + {libavcodec => libavutil}/ass.c | 73 +- libavutil/ass_internal.h | 139 +++ {libavcodec => libavutil}/ass_split.c | 30 +- .../ass_split_internal.h | 24 +- libavutil/frame.c | 194 +++- libavutil/frame.h | 93 +- libavutil/subfmt.c | 243 +++++ libavutil/subfmt.h | 185 ++++ tests/ref/fate/filter-overlay-dvdsub-2397 | 181 ++-- tests/ref/fate/sub-dvb | 162 ++-- tests/ref/fate/sub2video | 116 +-- tests/ref/fate/sub2video_basic | 135 +-- tests/ref/fate/sub2video_time_limited | 4 +- 77 files changed, 7371 insertions(+), 1381 deletions(-) create mode 100644 libavfilter/sf_graphicsub2text.c create mode 100644 libavfilter/sf_splitcc.c create mode 100644 libavfilter/sf_stripstyles.c create mode 100644 libavfilter/sf_subscale.c create mode 100644 libavfilter/sf_textmod.c create mode 100644 libavfilter/subtitles.c create mode 100644 libavfilter/subtitles.h create mode 100644 libavfilter/vf_overlaygraphicsubs.c create mode 100644 libavfilter/vf_overlaytextsubs.c rename {libavcodec => libavutil}/ass.c (73%) create mode 100644 libavutil/ass_internal.h rename {libavcodec => libavutil}/ass_split.c (94%) rename libavcodec/ass_split.h => libavutil/ass_split_internal.h (89%) create mode 100644 libavutil/subfmt.c create mode 100644 libavutil/subfmt.h