From patchwork Sat Nov 27 19:37:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 31767 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:d206:0:0:0:0:0 with SMTP id q6csp4018322iob; Sat, 27 Nov 2021 11:37:51 -0800 (PST) X-Google-Smtp-Source: ABdhPJwIo7dFMsLsG7pfAFz0DHrhRuKfyb5/S5+KP23ZbSEba4qvlvTECvYmdjawVlyN0D+SEHrB X-Received: by 2002:a17:906:619:: with SMTP id s25mr48216043ejb.237.1638041871378; Sat, 27 Nov 2021 11:37:51 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id js1si21453714ejc.427.2021.11.27.11.37.50; Sat, 27 Nov 2021 11:37:51 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=HqeKg1r7; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3B92868A940; Sat, 27 Nov 2021 21:37:46 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11olkn2050.outbound.protection.outlook.com [40.92.20.50]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2AF8168A7DE for ; Sat, 27 Nov 2021 21:37:40 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=IECPUdOXEb5Agm6f7Zfkv9KsfVNrgIabvUJrJv6P7HxyYhIPPLr8PmP8Id8BWxCm0E/78bYIBN4oW2gAVnmvdPOOOBbb6pvyw/p7d3QJAkE+aKXvxDWnttf0gbyw6fiExDBGtuPrrIZqZIdn5eODhrv1d6FLFaLm2TASMFp8+UzAGGQ1aAIhIUIsMgvu9Qtvxd0lUicArh3Kc3atS3xWkv2PHbEy4BeWOHBy8MUiHHduaQP3Ln7mJeKsp+AprqqVGIEHAAbFrWb6nyRff9v5BH3iWnAV7g3fdICK2Gfrw8AusRte9id9lje4LcWDYmDBqxQ9xiQvSjXX29lyMjTWEQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=gD4ISxImyHfEvA1J0uc2MzvvdrOxTRJAcaFZsvxmBOQ=; b=AJEFZDwRU5/YYvY06o5HtJg3CIhHtrvD+MvVn3kn+zeKPpAfgbMtZlaIruF1vfn0LNS6xR179RwNM1uaFyqsOqQBtxxOU7lRWqEl5ioU99ScIv11RRD9mLTtw5oM7HAXysNrf9vslDOGSHs8FMHFcUivN/M7nrj8k99n77v1fffMCEUiQ2gQclfj4O83A2Cq5Y9phyXE9r1gimna6fGqOsKZVU17+kCxNrm2imaxB9G/tUq7pZGnOZMax6rt5GweykponpYhRQ+TAtO6JKlLU8k6zdky/r5WE+P95dKaLUXPAw9rspJAashgq6FEOpYR2bqh5iGvXi5P6eYoXHxqWg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gD4ISxImyHfEvA1J0uc2MzvvdrOxTRJAcaFZsvxmBOQ=; b=HqeKg1r73NHLHpvbDr3y2Sr/lXSGgLvmGQthxrxa5ppoJgLoUE/9PLs7GWoaWeqr1CCJ5r9a6kK8cshdo2OsTuJbcw4rbSMG+/QD42bHPohsLAaGZJCa8lzedqvMDeyeKgr/8pkqC/UPZf6sfvEipeKqgxTevX3Ennw9GZppkKkuhA92RZvklH5qYRfoHXQXihMNlKrlkclHjC1DTzOT828OYCDnMsYiJZYFTe6RN215wtS91yhpRxDDlhd/KUVU6f8TYwd56ba8JnGFJ5AcN9TXnU+pXorRp7k/HCHWlcHITQAJktMrCAku871aHFVG/3OefCotH05dPFbiSZw42A== Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::20) by DM8P223MB0398.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:5::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4734.23; Sat, 27 Nov 2021 19:37:36 +0000 Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775]) by DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775%5]) with mapi id 15.20.4734.022; Sat, 27 Nov 2021 19:37:36 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v17 00/19] Subtitle Filtering Thread-Index: AQKdW//cp6OvIempw6WCsMaYkLZLmQ== Date: Sat, 27 Nov 2021 19:37:36 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [h6rWC1owREUa17Qr1QdNFavOGMNRQ6Tm] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 67a82e3f-4d88-4e80-1c29-08d9b1dd5d77 x-ms-traffictypediagnostic: DM8P223MB0398: x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: hgro42iC7L033jN6BH70EKwxD2wsslAqyOJ5BSBuaH7swGyyWSxak9T3jbt1uqA3P5cmt2eV+7ak/IbAAX88ZwTMlLGmz4QG0XuySCmuyFea/KRzSBbn1GaMxGWFB+BVJODHX93A5fxlmBH7M4p19cpUXMNp6Mg13OM0X5kBQKFZ8EQPlL0cj92Uq1S9+LoJ47RJlBZgR4rgJAy9Sc+QbwABIAcVojvoQgogS3L+mDcjjNjy6AkeLwX3G1BhHFJWTUxd5201UlVmO6M6ve/45FZtH6t7o42Wv8S6nN38KvnfCNQrEnhXU5Ndtn/UulHijN3onPyv8rrlwR260Xcq97xYvBleVsIKLqtID9FTRN6ybVjihhzwLd1NFZEH+cLRgHWF3gIXG6TnkzPwL4ExzQ22bJXkqymxqXtfHswwSn66eJLHAqucyanHvdVX87ds0UwQjsnv9HvS5J/N42UVsPBFSS9pMP73igW0YrSgrSlX+qBQYZp7rAa08e0q0OglD5GLsd76rYmyxuHGbZpmNe3l7zwtwio4tdoRbwM/4fS7ZVMKlDHCoP4oeSA+CL8agpBMKudj5CiZkij2DQZYng== x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: FMdxKZqSKfuC/9j23Ljk2IGN18DFbyCmUnC3qD6urmrFcF9heUW2NQDH/Y7Ct6LciQOwvPZKGFgJDESOs6L0IMOYMOjNKKgQhH32GmPnVuA0QvU1Qn9KGwGNR27DxqA39IhExfeZYaFtQt86LDNTLaRAQvmIv5r2jaoPcFEDnzK/6v2MxD9+o87HCWJpsRuC3y4gf4WXgMxHbzq3AXo+XeKX0mR15iGuW8MWZOCHUdrgTfA33vHtMJo2qPLFaPK4KyFB5yIagwRLvtf0uabidT8jkbttGAh4uf+eGcuDEbcqLvCpJuRHklw3k0k5YScJ7ivRAel/2h9c3hNEOlhny+OGNqnaqOu5lKiwhMWodUnfbxozrNNa7rWtt8ipHCLyELa5OFs2idkSsvGyCfxxf6zDRWeM4Tus7oMSR0m+7FL/Edm6QHRkEsIM4+O9SIMz33uK7GlS5PY5BpU5yinkYs+V7gFKdNdIjKMedlyEbxj6OeuCIi2v69ot0P6PK35MbKRbQ+jJ/Lf0rTp0vQ0jthEM4HIaL0Us2u9JimA7PuMb6yCB2DULquQ8+Z0aIjtgzu2sxoJIG2ClA+ZdGU3nf+YjjG9TJyIb9rxQrqkL1/TkO6tReLzlu6Ul9G/fNyEO41yp0r6p0I1t+scptXWSifWfdEEcARjgXNsaBLQ1knA+hcihiBc3zdP3iJ7T198WykxlPYl4WSUYjr5WcxnSIA== MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-3912-16-msonline-outlook-1fc6d.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 67a82e3f-4d88-4e80-1c29-08d9b1dd5d77 X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Nov 2021 19:37:36.2914 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8P223MB0398 Subject: [FFmpeg-devel] [PATCH v17 00/19] Subtitle Filtering X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: WzCgUJVajrqZ New in V17 - Applied almost all suggestions that were made (thanks everybody!) - Split the initial commit, no longer moving the legacy structs (AVSubtitle..) - Use the regular encoding API for subtitles - Updated subtitle encoders to handle packet allocation - Removed compatibility layer for accessing legacy encoders - Updated the compatibility implementation for the legacy subtitle encode api - Reordered commits and merged ffmpeg.c changes into a single commit New in V16 - Added missing reference to subfmt.h iun libzvbi-teletextdec.c - Fixed Fate error in patch 15/16 - Removed all trsiling whitespace in tilers.texi New in V15 - Rebased to upstream changes - avcodec/subtitles: Migrate subtitle encoders to frame-based API and provide compatibility shim for legacy api - fftools/ffmpeg: Use new frame-based subtitle encoding API - AVSubtitleArea: copy flags field, make params const - graphicsubs2text: Don't emit duplicate frames - graphicsubs2text: Combined OCR output into a single AVSubtitleArea (I have a prototype for detecting text colors and positions, but it's not ready at this point) - splitcc: cleanup local subtitle_header ref - stripstyles: add parameter for ass layer selection - avcodec/subtitles: deferred loading of ass header for text subtitle encoders - verified all example command lines in the docs are working, added somre more Kind regards, softworkz softworkz (19): avcodec,avutil: Move enum AVSubtitleType avutil/frame: Prepare AVFrame for subtitle handling avcodec/subtitles: Introduce new frame-based subtitle decoding API avfilter/subtitles: Update vf_subtitles to use new decoding api avcodec,avutil: Move ass helper functions to avutil as avpriv_ and extend ass dialog parsing avcodec/subtitles: Migrate subtitle encoders to frame-based API and provide a compatibility shim for the legacy api fftools/play,probe: Adjust for subtitle changes avfilter/subtitles: Add subtitles.c for subtitle frame allocation avfilter/avfilter: Handle subtitle frames avfilter/sbuffer: Add sbuffersrc and sbuffersink filters avfilter/overlaygraphicsubs: Add overlaygraphicsubs and graphicsub2video filters fftools/ffmpeg: Replace sub2video with subtitle frame filtering and use new frame-based subtitle encoding API avfilter/avfilter: Fix hardcoded input index avfilter/overlaytextsubs: Add overlaytextsubs and textsubs2video filters avfilter/textmod: Add textmod, censor and show_speaker filters avfilter/stripstyles: Add stripstyles filter avfilter/splitcc: Add splitcc filter for closed caption handling avfilter/graphicsub2text: Add new graphicsub2text filter (OCR) avfilter/subscale: Add filter for scaling and/or re-arranging graphical subtitles configure | 7 +- doc/filters.texi | 756 +++++++++++++++ fftools/ffmpeg.c | 584 ++++++------ fftools/ffmpeg.h | 15 +- fftools/ffmpeg_filter.c | 217 +++-- fftools/ffmpeg_hw.c | 2 +- fftools/ffmpeg_opt.c | 3 +- fftools/ffplay.c | 102 +- fftools/ffprobe.c | 48 +- libavcodec/Makefile | 56 +- libavcodec/ass.h | 147 +-- libavcodec/assdec.c | 2 +- libavcodec/assenc.c | 92 +- libavcodec/avcodec.h | 32 +- libavcodec/ccaption_dec.c | 19 +- libavcodec/codec_desc.c | 11 + libavcodec/codec_desc.h | 8 + libavcodec/decode.c | 54 +- libavcodec/dvbsubenc.c | 96 +- libavcodec/dvdsubenc.c | 100 +- libavcodec/encode.c | 63 +- libavcodec/internal.h | 16 + libavcodec/jacosubdec.c | 2 +- libavcodec/libaribb24.c | 2 +- libavcodec/libzvbi-teletextdec.c | 14 +- libavcodec/microdvddec.c | 7 +- libavcodec/movtextdec.c | 3 +- libavcodec/movtextenc.c | 126 ++- libavcodec/mpl2dec.c | 2 +- libavcodec/realtextdec.c | 2 +- libavcodec/samidec.c | 2 +- libavcodec/srtdec.c | 2 +- libavcodec/srtenc.c | 112 ++- libavcodec/subviewerdec.c | 2 +- libavcodec/tests/avcodec.c | 2 - libavcodec/textdec.c | 4 +- libavcodec/ttmlenc.c | 111 ++- libavcodec/utils.c | 182 ++++ libavcodec/webvttdec.c | 2 +- libavcodec/webvttenc.c | 90 +- libavcodec/xsubenc.c | 76 +- libavfilter/Makefile | 15 + libavfilter/allfilters.c | 13 + libavfilter/avfilter.c | 30 +- libavfilter/avfilter.h | 11 + libavfilter/avfiltergraph.c | 5 + libavfilter/buffersink.c | 54 ++ libavfilter/buffersink.h | 7 + libavfilter/buffersrc.c | 72 ++ libavfilter/buffersrc.h | 1 + libavfilter/formats.c | 22 + libavfilter/formats.h | 3 + libavfilter/internal.h | 19 +- libavfilter/sf_graphicsub2text.c | 354 +++++++ libavfilter/sf_splitcc.c | 316 +++++++ libavfilter/sf_stripstyles.c | 196 ++++ libavfilter/sf_subscale.c | 883 ++++++++++++++++++ libavfilter/sf_textmod.c | 697 ++++++++++++++ libavfilter/subtitles.c | 63 ++ libavfilter/subtitles.h | 44 + libavfilter/vf_overlaygraphicsubs.c | 737 +++++++++++++++ libavfilter/vf_overlaytextsubs.c | 633 +++++++++++++ libavfilter/vf_subtitles.c | 54 +- libavutil/Makefile | 4 + {libavcodec => libavutil}/ass.c | 87 +- libavutil/ass_internal.h | 133 +++ {libavcodec => libavutil}/ass_split.c | 30 +- .../ass_split_internal.h | 24 +- libavutil/frame.c | 211 ++++- libavutil/frame.h | 77 +- libavutil/subfmt.c | 50 + libavutil/subfmt.h | 106 +++ tests/ref/fate/filter-overlay-dvdsub-2397 | 181 ++-- tests/ref/fate/sub-dvb | 162 ++-- tests/ref/fate/sub2video | 116 +-- tests/ref/fate/sub2video_basic | 135 +-- tests/ref/fate/sub2video_time_limited | 4 +- 77 files changed, 7361 insertions(+), 1361 deletions(-) create mode 100644 libavfilter/sf_graphicsub2text.c create mode 100644 libavfilter/sf_splitcc.c create mode 100644 libavfilter/sf_stripstyles.c create mode 100644 libavfilter/sf_subscale.c create mode 100644 libavfilter/sf_textmod.c create mode 100644 libavfilter/subtitles.c create mode 100644 libavfilter/subtitles.h create mode 100644 libavfilter/vf_overlaygraphicsubs.c create mode 100644 libavfilter/vf_overlaytextsubs.c rename {libavcodec => libavutil}/ass.c (65%) create mode 100644 libavutil/ass_internal.h rename {libavcodec => libavutil}/ass_split.c (94%) rename libavcodec/ass_split.h => libavutil/ass_split_internal.h (89%) create mode 100644 libavutil/subfmt.c create mode 100644 libavutil/subfmt.h