From patchwork Mon Dec 13 23:40:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 32452 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp6138586iog; Mon, 13 Dec 2021 15:40:39 -0800 (PST) X-Google-Smtp-Source: ABdhPJwoghg8QiFkXJ7PIoP8tagEBebA1DalnRjMyNSSnUNBcx1lJDYPkJ3sKnREuGBGm/eJAskJ X-Received: by 2002:a50:9d48:: with SMTP id j8mr2680494edk.192.1639438839520; Mon, 13 Dec 2021 15:40:39 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id z15si17491642edx.151.2021.12.13.15.40.39; Mon, 13 Dec 2021 15:40:39 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=fPLqABkD; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 90FB468B029; Tue, 14 Dec 2021 01:40:32 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11olkn2032.outbound.protection.outlook.com [40.92.18.32]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 8C67768AD39 for ; Tue, 14 Dec 2021 01:40:25 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ORTT7sKiA8HTS19PwUSd0N8jfy8btWP4anmFt/prMYSdtl6ESveCy5PPgaIPaVny4f3OiG7Kq6qQ8Ip5O09MKhzf/ejNSuu8j/Rq7qkylZLofdRrFP/M39jsoO5fQAlq8l8bJQHwMA61xU78iS7+AD9RWZfmqPZGPjl3En3Way8qwwpONEDvThs1F6ELFDmQEd0BFqxepQq5EfbzN3h2ISuBo5+OE07NwoRuXWyr4TTngI1z7A5lmUJm8utTTup76D7SGo6u0XgK60jZHCy95ckLrN/nax6qfbzbUUbKLuwcb0KMZgJ27uwWliKsL3DtZSPBc0p7U/4yfJ9dvOBeng== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=L/JwRvIuwCyQ8zzIEsYtEJ3V3EVmXCY+mbNEMHVFp7Q=; b=b7Oa715SAqH/QgEltdD1D9ANeDULoGMAHfE7HLaJOsV/vKTtlpz4PFWLEMj1vqLYXuQWUKrAiLswa6R7j+31YO8lEFR2WQY0WjRB+P17xmeuUzKc/pJNRTOgrO2tMtNF7e81Cc83xJ4Xuxto1Cc/EacPq3bUcgLyIBUMQJ7lz0aIpeNxdNqegpj6HT64WJGgnpDLJJTqk57jUi+6iIzoYSWxi54jlt2BtGuqr5GFgazbC7i+H9ApRA/MHHyPwmTnZSDId92p7Ojg2XEGnqKijW/Op1TsuFaXpg01diABGcVZoSjtALiy2Q6IIMIfxR64rWVkl3NpzySOZSzQJMjiHg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=L/JwRvIuwCyQ8zzIEsYtEJ3V3EVmXCY+mbNEMHVFp7Q=; b=fPLqABkDbMjT7mig3fHkLPC69Ha/pRdC/T29lTW+sNXQEv3lrSAAsMw/7BBbAW4nngLX30OiaYCXRrlfwcIkZ1mU2pmgTbMlz0zb6egFlybobO5Wwuns/GG7AMbWgh3latvxUJafJO4LFRxJmo+5Eoi4rTLcNP8vgL337FTzOVCEN7PyeEQ2sCyP4G/GIHpmNhNdIyFU/OiRRxDEbg0B02F4Czj7JES87gVK9fbTXQKryaPyxvS/tkh4W8PC3wwjWnUc++GfG8Mnj/yI/CnXg027JkMIRoXkcy7xaecvnogorjcDtB0gG0mEDnrHgK3hgXRk1IVvjBoP4iKQsMqtRQ== Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::20) by DM8P223MB0303.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4778.12; Mon, 13 Dec 2021 23:40:20 +0000 Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775]) by DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775%7]) with mapi id 15.20.4778.018; Mon, 13 Dec 2021 23:40:20 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v24 01/21] avcodec,avutil: Move enum AVSubtitleType to avutil, add new and deprecate old values Thread-Index: AQHX8HrK2MViD7eCbkafIlYzN4PZag== Date: Mon, 13 Dec 2021 23:40:20 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [gZz+dSmUCKf3FmyOTdYe4L5iQXSrcsOJ] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: d7587541-4fba-4e57-a53e-08d9be91ecfc x-ms-traffictypediagnostic: DM8P223MB0303:EE_ x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 5reF0d4y02Lwqe7kIJ04YvavDUeYrpyfWpMxRaAqTkIJmYgDditu0mgfeD/ozdU2AyterbAQUh9bh0CpXk1FgN2n9TiGzk0p+hFQbzw0owv8lvxb4sTApH276XjBSOOvAq8mz92M3JOmpYLwrDd2/4bFbksAykNzzo+byRE7IKcfALJgzlckRX9nkHqQN2iyD7gZ18R9mbapZ1cPPg4ynhDK5v8rcxp++DIt9cpe5psng7fsokeTCUaxhHJ5gyPwI+52S0Q8FJN+/LyhkfniMsHJxkQieJRw9wmUrxV8RQ0hFy2coqtL15ingsnzybLmj5MPXUvi3UM2vx9Ecf79KRoNYsaMKlzVTR/QskLpRZGSi9DnsHCk0qPcNGlb9WkR/SS2u5nsjeFxsx7tm3g9VYeDwwMXt7xfpjw/5WhRF73nwfxcxaHe1vPJHkK7kWrf2ydhlzsdanvRNNFrf0sgQMtWZkazlT1CJC4EU+Kzp0CzwfQuPEHz8zM5MxJQpA56zgOPKHiemOYz87exRJ/cl1rfkBamnH9zn73/Durpun5deXsU5XCJekqUCosRQklw6NqfqHMW+mkOE+PHxV3zjA== x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?q?sBWUSmUUQFGDAE66+HIGthCJz/2l?= =?utf-8?q?rHG7Ofyfc7ORurgkWDM6pBfAXJcnNnMLxLbC60xPywL83m3He2ECNCo23L8fVIC59?= =?utf-8?q?aeGt67JvQVNCPVBgtBrQdDEOzLASzq+IBrLu6DjdsDElbZU1FPIXs/jSJQnnEBwRh?= =?utf-8?q?8+VAa66vT14vBT25HO1eHlSkKeT36KwlT3hLZvzdzzUv8g77ejQlaDnAD15BBaiQq?= =?utf-8?q?T56NOA49UKQz/tYK+kg4U5pG3eOOfjzBqN/al7dVzpUrUJgl3geSbi+9orfZbKJsP?= =?utf-8?q?uH8LTwrqiDHegGxoax9yY9cngyXqY/P2AqIX6eKMf7eyjGWSj3NrkYes+D3IYApGT?= =?utf-8?q?fsrKzyTAG9EOj9+5Xft0qISWMqhWhDOe9sM0gkV/SHngSe0kY1yXnnZs1UO5h10Ah?= =?utf-8?q?IlA7W8Y/UflebVMCo74agojQrdrRF4vYiZNT1Z38Wh5WXXpbukNvtBWu2c12WSwlv?= =?utf-8?q?m848JrMIqA3KOeoGlezG+wO6zmc/oklXP5Pn889tQzCwg49mh8aklLW4ODzV4EhqJ?= =?utf-8?q?u/b0diNlUCedbOtsmuIP0Dho3uIL8uRSDlKhmsh467elVK3B1EjzL3zr2+cgX2aPG?= =?utf-8?q?QN43jDJKO2IRXv2KeHi2s+XytCfhhVFe8ppGTJ9sauPa4t5CONAAezarPQJSEPeiZ?= =?utf-8?q?MakgR7Z4R90t9jov0ifgvf4EyETBkNVGmp8qS0EhuS9p6JnSYHBsUtRw4A8BCH+5U?= =?utf-8?q?HsQu9UxRMUMMlhXcUqtBP+kZilIiRbkC0kPsHiLIgNm4cL+sLE+iJzAt9Zll3yAww?= =?utf-8?q?F32LlVtPij5qVHyfl4KRDUMoQQkvd8ohH7h4F45oORFssWwyA3PQ5++hi+AMZeANA?= =?utf-8?q?thX5Z2g/RNhwOoV4K6Ng5VLpX78exNuYRnmt1uvavQzzHKgI7yXK/0bORbUVKQVdD?= =?utf-8?q?OMvm6rhWITN1r/Unl5Or59JMek1MB0GL8H2PGhl/QOpA=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-1ff67.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: d7587541-4fba-4e57-a53e-08d9be91ecfc X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Dec 2021 23:40:20.4623 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8P223MB0303 Subject: [FFmpeg-devel] [PATCH v24 01/21] avcodec, avutil: Move enum AVSubtitleType to avutil, add new and deprecate old values X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: MLfeFPJ/i52r Signed-off-by: softworkz --- libavcodec/avcodec.h | 19 +------------ libavutil/Makefile | 1 + libavutil/subfmt.h | 68 ++++++++++++++++++++++++++++++++++++++++++++ libavutil/version.h | 1 + 4 files changed, 71 insertions(+), 18 deletions(-) create mode 100644 libavutil/subfmt.h diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index 7ee8bc2b7c..b05c12e47e 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -35,6 +35,7 @@ #include "libavutil/frame.h" #include "libavutil/log.h" #include "libavutil/pixfmt.h" +#include "libavutil/subfmt.h" #include "libavutil/rational.h" #include "codec.h" @@ -2238,24 +2239,6 @@ typedef struct AVHWAccel { * @} */ -enum AVSubtitleType { - SUBTITLE_NONE, - - SUBTITLE_BITMAP, ///< A bitmap, pict will be set - - /** - * Plain text, the text field must be set by the decoder and is - * authoritative. ass and pict fields may contain approximations. - */ - SUBTITLE_TEXT, - - /** - * Formatted text, the ass field must be set by the decoder and is - * authoritative. pict and text fields may contain approximations. - */ - SUBTITLE_ASS, -}; - #define AV_SUBTITLE_FLAG_FORCED 0x00000001 typedef struct AVSubtitleRect { diff --git a/libavutil/Makefile b/libavutil/Makefile index 529046dbc8..c7843db1e4 100644 --- a/libavutil/Makefile +++ b/libavutil/Makefile @@ -74,6 +74,7 @@ HEADERS = adler32.h \ sha512.h \ spherical.h \ stereo3d.h \ + subfmt.h \ threadmessage.h \ time.h \ timecode.h \ diff --git a/libavutil/subfmt.h b/libavutil/subfmt.h new file mode 100644 index 0000000000..791b45519f --- /dev/null +++ b/libavutil/subfmt.h @@ -0,0 +1,68 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVUTIL_SUBFMT_H +#define AVUTIL_SUBFMT_H + +#include "version.h" + +enum AVSubtitleType { + + /** + * Subtitle format unknown. + */ + AV_SUBTITLE_FMT_NONE = -1, + + /** + * Subtitle format unknown. + */ + AV_SUBTITLE_FMT_UNKNOWN = 0, +#if FF_API_OLD_SUBTITLES + SUBTITLE_NONE = 0, ///< Deprecated, use AV_SUBTITLE_FMT_NONE instead. +#endif + + /** + * Bitmap area in AVSubtitleRect.data, pixfmt AV_PIX_FMT_PAL8. + */ + AV_SUBTITLE_FMT_BITMAP = 1, +#if FF_API_OLD_SUBTITLES + SUBTITLE_BITMAP = 1, ///< Deprecated, use AV_SUBTITLE_FMT_BITMAP instead. +#endif + + /** + * Plain text in AVSubtitleRect.text. + */ + AV_SUBTITLE_FMT_TEXT = 2, +#if FF_API_OLD_SUBTITLES + SUBTITLE_TEXT = 2, ///< Deprecated, use AV_SUBTITLE_FMT_TEXT instead. +#endif + + /** + * Text Formatted as per ASS specification, contained AVSubtitleRect.ass. + */ + AV_SUBTITLE_FMT_ASS = 3, +#if FF_API_OLD_SUBTITLES + SUBTITLE_ASS = 3, ///< Deprecated, use AV_SUBTITLE_FMT_ASS instead. +#endif + + AV_SUBTITLE_FMT_NB, ///< number of subtitle formats, DO NOT USE THIS if you want to link with shared libav* because the number of formats might differ between versions. +}; + +#endif /* AVUTIL_SUBFMT_H */ diff --git a/libavutil/version.h b/libavutil/version.h index 0e7b36865a..cc8caadf50 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -109,6 +109,7 @@ #define FF_API_DECLARE_ALIGNED (LIBAVUTIL_VERSION_MAJOR < 58) #define FF_API_COLORSPACE_NAME (LIBAVUTIL_VERSION_MAJOR < 58) #define FF_API_AV_MALLOCZ_ARRAY (LIBAVUTIL_VERSION_MAJOR < 58) +#define FF_API_OLD_SUBTITLES (LIBAVUTIL_VERSION_MAJOR < 58) /** * @} From patchwork Mon Dec 13 23:40:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 32453 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp6138757iog; Mon, 13 Dec 2021 15:40:49 -0800 (PST) X-Google-Smtp-Source: ABdhPJxiqjVJT7YkNWF5peP3TuSPUxha65pdBzZ7YTYms6LgzDl8JWiVzf7hLluj5jZCDpzxM3mb X-Received: by 2002:a05:6402:26cd:: with SMTP id x13mr2784617edd.11.1639438849147; Mon, 13 Dec 2021 15:40:49 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id n16si17557912edr.511.2021.12.13.15.40.48; Mon, 13 Dec 2021 15:40:49 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=quNU8qh6; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id AEDD768B06C; Tue, 14 Dec 2021 01:40:38 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11olkn2032.outbound.protection.outlook.com [40.92.18.32]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 052F168AED3 for ; Tue, 14 Dec 2021 01:40:31 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VyQBiqQ1bgdgRnFCjtOQgxFcv8LeQVKR3q9bPREChDuwKk6eiaPKd4/z5WjHdelehj1JVDkKg67I96SCYGE5Ts+4Tte6pZgehfkY/JC2KhW5HTfBmPZgSonxtfcA0D1Vau1nfZX/668qAOQVWSAtGpWOjARRw+sVgRv5RrZ/mnXGpak8H+G7Y3i7YNqGfcIcJZtBIyG+Rb24MchHp4rE8Z9xRjeq4z09vt+JtHn/ZAYnQTmtjDVZkj0M+M3ZbLd6aHGaxVWrUbia7Jyht1rAXsZ9L9KIKEZN0iTs07ZPEmrSToaUf+d0cj4/c641pjqozRNrTV/5WbdrSSAbO9q5tA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=sN2pMnhrEu115XwlmsrHoV4GbgnNiC8z/3LT9rTfDYA=; b=DGV09n3y29tJnQFudg469s8bE2hrr8XvGvghgzsB96h9RWPb+Xnq5W8mWLl5ltdYuH4x/2oCuZrF1nMfKikHdEHDoejADDg2v1VreMCTfhomZZ/IgyIWaiBV6wBBgzIw449qSEqdcbIIfEpX78jnLHu4NvVCq8unVMpe8rgiSUZz6R2v2YmzyA7MeAyBRxi2qJPt3v0ETwl7h6+5NetO26yiC4o1qCONz6oIfPBzoyURNyDFZZ4YKQZl92esylcInwUeBoMgDplMtBTQjZOtHgVtrwT5xNdXtDI/eV2I27bAFeJEo40GY/Th0CiiUHzPPa4tchpwNVRiYalUt8XrTw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=sN2pMnhrEu115XwlmsrHoV4GbgnNiC8z/3LT9rTfDYA=; b=quNU8qh6Tt85tV8e+pLwWmdbkpwdIp6OIZlJChD0ALzWN7SoE289weGmNhVWjmCS8G7i1W7u64sB0HUo2TKVrN2A1aw602bMXO0ubaNJs4rsm3LLjL3OMIDpXls4JgdTNmmKThLKMGVNBf4CEsESubkbO7V9IOfL2seeTPwqHBtr+SBGjuytok/FPNKN9kYhNUAof6of23HpHt3Xa02RuOKuniHDShOw7lcDup9UAvUiFZQbmJbyEt9n0mOOqL8sF7pZ4swmBrT6CNoXZrXOyeHZOzgjN6lr4kYJ1vku6yGGmeG6Ddjskk8UeRCuP5GYVC/rNPcZHmY+P8aqdKAE7g== Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::20) by DM8P223MB0303.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4778.12; Mon, 13 Dec 2021 23:40:23 +0000 Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775]) by DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775%7]) with mapi id 15.20.4778.018; Mon, 13 Dec 2021 23:40:23 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v24 02/21] avutil/frame: Prepare AVFrame for subtitle handling Thread-Index: AQHX8HrM1CAAa+CovUyA86MKflLiTQ== Date: Mon, 13 Dec 2021 23:40:23 +0000 Message-ID: References: <1ef42eb01b3627bbad539abd46206f320bb39697.1639438462.git.softworkz@hotmail.com> In-Reply-To: <1ef42eb01b3627bbad539abd46206f320bb39697.1639438462.git.softworkz@hotmail.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [xQLVxTNBDZr22+wPhQ3LV12p88VFEQ7v] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 4e9380f9-47f5-437b-fb25-08d9be91eee5 x-ms-traffictypediagnostic: DM8P223MB0303:EE_ x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: jJL6Sh8mQ4b8Z40/4X1N0gUJ6H+e1SPb4O8xDQxItY8OWnNnzalP/3D4xu4rwFN1L9qdxKN2s4AzBbeOq178hXs/NoxoSTaQPID1Cqyx79l1Zh2n94pUKH6fN83TN/hKC/ROh/83K2eF8hK0TLOE0gsID2imx6yoNYz6YKkwa91G1A02IN5+borr0tGwZt2srFbNSQWoUlXxr+KP1+iKbza1qKP3WFCI+ySK8ohkSAAc+6FjecPZyHRAJOyfec/sdaeA8camfhZ1BASXgM6Jts7xRr94r3gwCa1PcQAp4pgtzk/lQb0cJs9SIzmhWiS2JuUEtqC8RxqjRySTFPSSUdu7LuRZJEE35eE9RGxOGm8Jk6dwGXilLAQqyXf0GxNP8SIne4VD1v4zDb+cA8gk/Wy8iqfme2X7Q3a1II+1N1xp4SsJceJKTCHJqQBSmo/VLW7oFyTAtmULQH7cqWtez7zKXceb4V6OijtDQIBeywdKmdywMZiIIVA4HzmlaWtilyXRbjdQnHt26HAABZhz+4Wlj5x+vsP+UoC6Ly7pVDqYlzp6YY1pp7Vg7bmhhs2wxG1HS1F4cGj72urnSmdj7A== x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?q?OXy2HWC94aUKS4pmWS5lrpwzcrR9?= =?utf-8?q?pWpCYUeW3xcAl3yLanLGpslFlwwud3nMRil9fTXnX6dwW6uWv4D/ZB0E11zupCsaB?= =?utf-8?q?1vmWgPgOFogxuewq+RvpzQsWvP0HmGyUwieQGkZvbPWYQOEoSSRjiUDegiJ+7igZ2?= =?utf-8?q?OE2VOhmTHH71ec1tYf57SI8wMBDTDWuLy7PwYdd9YjZoc5F+zvHN4LQ4FsdUMqAUZ?= =?utf-8?q?ahjyI/j8KuMrnXJFMAgWN8fY9DGoRAItNJJ3JOYwgHiGkG+tGBcqGNVuI5/DIN3sD?= =?utf-8?q?jurhBsm2JBD5lygjBN3o3DMdvbVRbCHPHw96rxJzuFCAOxZn++igkZ+jaXrr/6eck?= =?utf-8?q?D2ORZ8iouLKfuahj5L5qE9WVqB7f6g0/XO1BOGu14YlBcsMltPFBzQPrsmPNH0nHK?= =?utf-8?q?SFNk4OIxiDKMgQ5Ar1+2JuxIjdlZwx9dzXR1MArWubjjddRROlTOUiEFqm72GogQb?= =?utf-8?q?E38ipZjuLFjNBgJrrHMxcFKjqFDo4JC27piBtkXZLz3+p1YMsw5cgDI3eUiy6zW/n?= =?utf-8?q?Cu5jmdZp5JFnx/lAPmuihbD/C7KpEzL1+5TTssvoyLTlwzrgK+5c7S9JzAYHNRcvZ?= =?utf-8?q?J6e4LpN3FjZ8fV5hXsdMnbw69nPhYtTOC+U77Uk6Z3IYcC+DotBC3nc4v6LQLWUgN?= =?utf-8?q?Y4giD90lueVlGG6WvM9PXN7V8QiZaHPJVe8SoP33sHhrQ4FYmJZOERewAZnatOLrT?= =?utf-8?q?McideKhWszudud/PwQp+bMyB89DHh2cN8Xlfh6zfiGD44xHK+gQtFV22UOFtXuJu8?= =?utf-8?q?WudIIvluS1dQkEBbTJAnVbXeDUSw0jn78GMm4b2ljvrS3+MuinUZb2AZcob+4Is8G?= =?utf-8?q?jzDi1rDuo+Sdu0jEbiIjlOzDm9m7oDXyHig1a9Q2HcRfVoxyXrb1/v6PhOPBJUuEn?= =?utf-8?q?mKz9eYbZK6hdzqylOXjt96BfyootYT/KPPZ7hBnU0jEQ=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-1ff67.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 4e9380f9-47f5-437b-fb25-08d9be91eee5 X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Dec 2021 23:40:23.6344 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8P223MB0303 Subject: [FFmpeg-devel] [PATCH v24 02/21] avutil/frame: Prepare AVFrame for subtitle handling X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Lr8VT2IIiri5 Root commit for adding subtitle filtering capabilities. In detail: - Add type (AVMediaType) field to AVFrame Replaces previous way of distinction which was based on checking width and height to determine whether a frame is audio or video - Add subtitle fields to AVFrame - Add new struct AVSubtitleArea, similar to AVSubtitleRect, but different allocation logic. Cannot and must not be used interchangeably, hence the new struct Signed-off-by: softworkz --- libavutil/Makefile | 1 + libavutil/frame.c | 211 ++++++++++++++++++++++++++++++++++++++++----- libavutil/frame.h | 78 ++++++++++++++++- libavutil/subfmt.c | 45 ++++++++++ libavutil/subfmt.h | 47 ++++++++++ 5 files changed, 358 insertions(+), 24 deletions(-) create mode 100644 libavutil/subfmt.c diff --git a/libavutil/Makefile b/libavutil/Makefile index c7843db1e4..7e79936876 100644 --- a/libavutil/Makefile +++ b/libavutil/Makefile @@ -160,6 +160,7 @@ OBJS = adler32.o \ slicethread.o \ spherical.o \ stereo3d.o \ + subfmt.o \ threadmessage.o \ time.o \ timecode.o \ diff --git a/libavutil/frame.c b/libavutil/frame.c index 0912ad9131..62d29ae7d6 100644 --- a/libavutil/frame.c +++ b/libavutil/frame.c @@ -26,6 +26,7 @@ #include "imgutils.h" #include "mem.h" #include "samplefmt.h" +#include "subfmt.h" #include "hwcontext.h" #define CHECK_CHANNELS_CONSISTENCY(frame) \ @@ -50,6 +51,9 @@ const char *av_get_colorspace_name(enum AVColorSpace val) return name[val]; } #endif + +static int frame_copy_subtitles(AVFrame *dst, const AVFrame *src, int copy_data); + static void get_frame_defaults(AVFrame *frame) { memset(frame, 0, sizeof(*frame)); @@ -70,7 +74,12 @@ static void get_frame_defaults(AVFrame *frame) frame->colorspace = AVCOL_SPC_UNSPECIFIED; frame->color_range = AVCOL_RANGE_UNSPECIFIED; frame->chroma_location = AVCHROMA_LOC_UNSPECIFIED; - frame->flags = 0; + frame->subtitle_start_time = 0; + frame->subtitle_end_time = 0; + frame->num_subtitle_areas = 0; + frame->subtitle_areas = NULL; + frame->subtitle_pts = 0; + frame->subtitle_header = NULL; } static void free_side_data(AVFrameSideData **ptr_sd) @@ -240,23 +249,55 @@ static int get_audio_buffer(AVFrame *frame, int align) } +static int get_subtitle_buffer(AVFrame *frame) +{ + // Buffers in AVFrame->buf[] are not used in case of subtitle frames. + // To accomodate with existing code, checking ->buf[0] to determine + // whether a frame is ref-counted or has data, we're adding a 1-byte + // buffer here, which marks the subtitle frame to contain data. + frame->buf[0] = av_buffer_alloc(1); + if (!frame->buf[0]) { + av_frame_unref(frame); + return AVERROR(ENOMEM); + } + + frame->extended_data = frame->data; + + return 0; +} + int av_frame_get_buffer(AVFrame *frame, int align) +{ + if (frame->width > 0 && frame->height > 0) + frame->type = AVMEDIA_TYPE_VIDEO; + else if (frame->nb_samples > 0 && (frame->channel_layout || frame->channels > 0)) + frame->type = AVMEDIA_TYPE_AUDIO; + + return av_frame_get_buffer2(frame, align); +} + +int av_frame_get_buffer2(AVFrame *frame, int align) { if (frame->format < 0) return AVERROR(EINVAL); - if (frame->width > 0 && frame->height > 0) + switch(frame->type) { + case AVMEDIA_TYPE_VIDEO: return get_video_buffer(frame, align); - else if (frame->nb_samples > 0 && (frame->channel_layout || frame->channels > 0)) + case AVMEDIA_TYPE_AUDIO: return get_audio_buffer(frame, align); - - return AVERROR(EINVAL); + case AVMEDIA_TYPE_SUBTITLE: + return get_subtitle_buffer(frame); + default: + return AVERROR(EINVAL); + } } static int frame_copy_props(AVFrame *dst, const AVFrame *src, int force_copy) { int ret, i; + dst->type = src->type; dst->key_frame = src->key_frame; dst->pict_type = src->pict_type; dst->sample_aspect_ratio = src->sample_aspect_ratio; @@ -288,6 +329,12 @@ static int frame_copy_props(AVFrame *dst, const AVFrame *src, int force_copy) dst->colorspace = src->colorspace; dst->color_range = src->color_range; dst->chroma_location = src->chroma_location; + dst->subtitle_start_time = src->subtitle_start_time; + dst->subtitle_end_time = src->subtitle_end_time; + dst->subtitle_pts = src->subtitle_pts; + + if ((ret = av_buffer_replace(&dst->subtitle_header, src->subtitle_header)) < 0) + return ret; av_dict_copy(&dst->metadata, src->metadata, 0); @@ -329,6 +376,7 @@ int av_frame_ref(AVFrame *dst, const AVFrame *src) av_assert1(dst->width == 0 && dst->height == 0); av_assert1(dst->channels == 0); + dst->type = src->type; dst->format = src->format; dst->width = src->width; dst->height = src->height; @@ -342,7 +390,7 @@ int av_frame_ref(AVFrame *dst, const AVFrame *src) /* duplicate the frame data if it's not refcounted */ if (!src->buf[0]) { - ret = av_frame_get_buffer(dst, 0); + ret = av_frame_get_buffer2(dst, 0); if (ret < 0) goto fail; @@ -364,6 +412,10 @@ int av_frame_ref(AVFrame *dst, const AVFrame *src) } } + /* copy subtitle areas */ + if (src->type == AVMEDIA_TYPE_SUBTITLE) + frame_copy_subtitles(dst, src, 0); + if (src->extended_buf) { dst->extended_buf = av_calloc(src->nb_extended_buf, sizeof(*dst->extended_buf)); @@ -434,7 +486,7 @@ AVFrame *av_frame_clone(const AVFrame *src) void av_frame_unref(AVFrame *frame) { - int i; + unsigned i, n; if (!frame) return; @@ -453,6 +505,21 @@ void av_frame_unref(AVFrame *frame) av_buffer_unref(&frame->opaque_ref); av_buffer_unref(&frame->private_ref); + av_buffer_unref(&frame->subtitle_header); + + for (i = 0; i < frame->num_subtitle_areas; i++) { + AVSubtitleArea *area = frame->subtitle_areas[i]; + + for (n = 0; n < FF_ARRAY_ELEMS(area->buf); n++) + av_buffer_unref(&area->buf[n]); + + av_freep(&area->text); + av_freep(&area->ass); + av_free(area); + } + + av_freep(&frame->subtitle_areas); + if (frame->extended_data != frame->data) av_freep(&frame->extended_data); @@ -472,18 +539,28 @@ void av_frame_move_ref(AVFrame *dst, AVFrame *src) int av_frame_is_writable(AVFrame *frame) { - int i, ret = 1; + AVSubtitleArea *area; + int ret = 1; /* assume non-refcounted frames are not writable */ if (!frame->buf[0]) return 0; - for (i = 0; i < FF_ARRAY_ELEMS(frame->buf); i++) + for (unsigned i = 0; i < FF_ARRAY_ELEMS(frame->buf); i++) if (frame->buf[i]) ret &= !!av_buffer_is_writable(frame->buf[i]); - for (i = 0; i < frame->nb_extended_buf; i++) + for (unsigned i = 0; i < frame->nb_extended_buf; i++) ret &= !!av_buffer_is_writable(frame->extended_buf[i]); + for (unsigned i = 0; i < frame->num_subtitle_areas; i++) { + area = frame->subtitle_areas[i]; + if (area) { + for (unsigned n = 0; n < FF_ARRAY_ELEMS(area->buf); n++) + if (area->buf[n]) + ret &= !!av_buffer_is_writable(area->buf[n]); + } + } + return ret; } @@ -499,6 +576,7 @@ int av_frame_make_writable(AVFrame *frame) return 0; memset(&tmp, 0, sizeof(tmp)); + tmp.type = frame->type; tmp.format = frame->format; tmp.width = frame->width; tmp.height = frame->height; @@ -509,7 +587,7 @@ int av_frame_make_writable(AVFrame *frame) if (frame->hw_frames_ctx) ret = av_hwframe_get_buffer(frame->hw_frames_ctx, &tmp, 0); else - ret = av_frame_get_buffer(&tmp, 0); + ret = av_frame_get_buffer2(&tmp, 0); if (ret < 0) return ret; @@ -544,14 +622,22 @@ AVBufferRef *av_frame_get_plane_buffer(AVFrame *frame, int plane) uint8_t *data; int planes, i; - if (frame->nb_samples) { - int channels = frame->channels; - if (!channels) - return NULL; - CHECK_CHANNELS_CONSISTENCY(frame); - planes = av_sample_fmt_is_planar(frame->format) ? channels : 1; - } else + switch(frame->type) { + case AVMEDIA_TYPE_VIDEO: planes = 4; + break; + case AVMEDIA_TYPE_AUDIO: + { + int channels = frame->channels; + if (!channels) + return NULL; + CHECK_CHANNELS_CONSISTENCY(frame); + planes = av_sample_fmt_is_planar(frame->format) ? channels : 1; + break; + } + default: + return NULL; + } if (plane < 0 || plane >= planes || !frame->extended_data[plane]) return NULL; @@ -675,17 +761,98 @@ static int frame_copy_audio(AVFrame *dst, const AVFrame *src) return 0; } +static int av_subtitle_area2area(AVSubtitleArea *dst, const AVSubtitleArea *src, int copy_data) +{ + dst->x = src->x; + dst->y = src->y; + dst->w = src->w; + dst->h = src->h; + dst->nb_colors = src->nb_colors; + dst->type = src->type; + dst->flags = src->flags; + + switch (dst->type) { + case AV_SUBTITLE_FMT_BITMAP: + + for (unsigned i = 0; i < AV_NUM_BUFFER_POINTERS; i++) { + if (src->h > 0 && src->w > 0 && src->buf[i]) { + dst->buf[0] = av_buffer_ref(src->buf[i]); + if (!dst->buf[i]) + return AVERROR(ENOMEM); + + if (copy_data) { + const int ret = av_buffer_make_writable(&dst->buf[i]); + if (ret < 0) + return ret; + } + + dst->linesize[i] = src->linesize[i]; + } + } + + memcpy(&dst->pal[0], &src->pal[0], sizeof(src->pal[0]) * 256); + + break; + case AV_SUBTITLE_FMT_TEXT: + + if (src->text) { + dst->text = av_strdup(src->text); + if (!dst->text) + return AVERROR(ENOMEM); + } + + break; + case AV_SUBTITLE_FMT_ASS: + + if (src->ass) { + dst->ass = av_strdup(src->ass); + if (!dst->ass) + return AVERROR(ENOMEM); + } + + break; + default: + + av_log(NULL, AV_LOG_ERROR, "Subtitle area has invalid format: %d", dst->type); + return AVERROR(ENOMEM); + } + + return 0; +} + +static int frame_copy_subtitles(AVFrame *dst, const AVFrame *src, int copy_data) +{ + dst->format = src->format; + + dst->num_subtitle_areas = src->num_subtitle_areas; + + if (src->num_subtitle_areas) { + dst->subtitle_areas = av_malloc_array(src->num_subtitle_areas, sizeof(AVSubtitleArea *)); + + for (unsigned i = 0; i < src->num_subtitle_areas; i++) { + dst->subtitle_areas[i] = av_mallocz(sizeof(AVSubtitleArea)); + av_subtitle_area2area(dst->subtitle_areas[i], src->subtitle_areas[i], copy_data); + } + } + + return 0; +} + int av_frame_copy(AVFrame *dst, const AVFrame *src) { if (dst->format != src->format || dst->format < 0) return AVERROR(EINVAL); - if (dst->width > 0 && dst->height > 0) + switch(dst->type) { + case AVMEDIA_TYPE_VIDEO: return frame_copy_video(dst, src); - else if (dst->nb_samples > 0 && dst->channels > 0) + case AVMEDIA_TYPE_AUDIO: return frame_copy_audio(dst, src); - - return AVERROR(EINVAL); + case AVMEDIA_TYPE_SUBTITLE: + return frame_copy_subtitles(dst, src, 1); + default: + return AVERROR(EINVAL); + } } void av_frame_remove_side_data(AVFrame *frame, enum AVFrameSideDataType type) diff --git a/libavutil/frame.h b/libavutil/frame.h index 3f295f6b9e..7162a67f92 100644 --- a/libavutil/frame.h +++ b/libavutil/frame.h @@ -34,6 +34,7 @@ #include "rational.h" #include "samplefmt.h" #include "pixfmt.h" +#include "subfmt.h" #include "version.h" @@ -278,7 +279,7 @@ typedef struct AVRegionOfInterest { } AVRegionOfInterest; /** - * This structure describes decoded (raw) audio or video data. + * This structure describes decoded (raw) audio, video or subtitle data. * * AVFrame must be allocated using av_frame_alloc(). Note that this only * allocates the AVFrame itself, the buffers for the data must be managed @@ -392,7 +393,7 @@ typedef struct AVFrame { /** * format of the frame, -1 if unknown or unset * Values correspond to enum AVPixelFormat for video frames, - * enum AVSampleFormat for audio) + * enum AVSampleFormat for audio, AVSubtitleType for subtitles) */ int format; @@ -674,6 +675,47 @@ typedef struct AVFrame { * for the target frame's private_ref field. */ AVBufferRef *private_ref; + + /** + * Media type of the frame (audio, video, subtitles..) + * + * See AVMEDIA_TYPE_xxx + */ + enum AVMediaType type; + + + /** + * Display start time, relative to packet pts, in ms. + */ + uint32_t subtitle_start_time; + + /** + * Display end time, relative to packet pts, in ms. + */ + uint32_t subtitle_end_time; + + /** + * Number of items in the @ref subtitle_areas array. + */ + unsigned num_subtitle_areas; + + /** + * Array of subtitle areas, may be empty. + */ + AVSubtitleArea **subtitle_areas; + + /** + * The display start time, in AV_TIME_BASE. + * + * For subtitle frames, AVFrame.pts is populated from the packet pts value, + * which is not always the same as this value. + */ + int64_t subtitle_pts; + + /** + * Header containing style information for text subtitles. + */ + AVBufferRef *subtitle_header; } AVFrame; @@ -750,6 +792,8 @@ void av_frame_move_ref(AVFrame *dst, AVFrame *src); /** * Allocate new buffer(s) for audio or video data. * + * Note: For subtitle data, use av_frame_get_buffer2 + * * The following fields must be set on frame before calling this function: * - format (pixel format for video, sample format for audio) * - width and height for video @@ -769,9 +813,39 @@ void av_frame_move_ref(AVFrame *dst, AVFrame *src); * recommended to pass 0 here unless you know what you are doing. * * @return 0 on success, a negative AVERROR on error. + * + * @deprecated Use @ref av_frame_get_buffer2 instead and set @ref AVFrame.type + * before calling. */ +attribute_deprecated int av_frame_get_buffer(AVFrame *frame, int align); +/** + * Allocate new buffer(s) for audio, video or subtitle data. + * + * The following fields must be set on frame before calling this function: + * - format (pixel format for video, sample format for audio) + * - width and height for video + * - nb_samples and channel_layout for audio + * - type (AVMediaType) + * + * This function will fill AVFrame.data and AVFrame.buf arrays and, if + * necessary, allocate and fill AVFrame.extended_data and AVFrame.extended_buf. + * For planar formats, one buffer will be allocated for each plane. + * + * @warning: if frame already has been allocated, calling this function will + * leak memory. In addition, undefined behavior can occur in certain + * cases. + * + * @param frame frame in which to store the new buffers. + * @param align Required buffer size alignment. If equal to 0, alignment will be + * chosen automatically for the current CPU. It is highly + * recommended to pass 0 here unless you know what you are doing. + * + * @return 0 on success, a negative AVERROR on error. + */ +int av_frame_get_buffer2(AVFrame *frame, int align); + /** * Check if the frame data is writable. * diff --git a/libavutil/subfmt.c b/libavutil/subfmt.c new file mode 100644 index 0000000000..c72ebe2a43 --- /dev/null +++ b/libavutil/subfmt.c @@ -0,0 +1,45 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include +#include "common.h" +#include "subfmt.h" + +static const char sub_fmt_info[AV_SUBTITLE_FMT_NB][24] = { + [AV_SUBTITLE_FMT_UNKNOWN] = "Unknown subtitle format", + [AV_SUBTITLE_FMT_BITMAP] = "Graphical subtitles", + [AV_SUBTITLE_FMT_TEXT] = "Text subtitles (plain)", + [AV_SUBTITLE_FMT_ASS] = "Text subtitles (ass)", +}; + +const char *av_get_subtitle_fmt_name(enum AVSubtitleType sub_fmt) +{ + if (sub_fmt < 0 || sub_fmt >= AV_SUBTITLE_FMT_NB) + return NULL; + return sub_fmt_info[sub_fmt]; +} + +enum AVSubtitleType av_get_subtitle_fmt(const char *name) +{ + for (int i = 0; i < AV_SUBTITLE_FMT_NB; i++) + if (!strcmp(sub_fmt_info[i], name)) + return i; + return AV_SUBTITLE_FMT_NONE; +} diff --git a/libavutil/subfmt.h b/libavutil/subfmt.h index 791b45519f..3b8a0613b2 100644 --- a/libavutil/subfmt.h +++ b/libavutil/subfmt.h @@ -21,6 +21,9 @@ #ifndef AVUTIL_SUBFMT_H #define AVUTIL_SUBFMT_H +#include + +#include "buffer.h" #include "version.h" enum AVSubtitleType { @@ -65,4 +68,48 @@ enum AVSubtitleType { AV_SUBTITLE_FMT_NB, ///< number of subtitle formats, DO NOT USE THIS if you want to link with shared libav* because the number of formats might differ between versions. }; +typedef struct AVSubtitleArea { +#define AV_NUM_BUFFER_POINTERS 1 + + enum AVSubtitleType type; + int flags; + + int x; ///< top left corner of area. + int y; ///< top left corner of area. + int w; ///< width of area. + int h; ///< height of area. + int nb_colors; ///< number of colors in bitmap palette (@ref pal). + + /** + * Buffers and line sizes for the bitmap of this subtitle. + * + * @{ + */ + AVBufferRef *buf[AV_NUM_BUFFER_POINTERS]; + int linesize[AV_NUM_BUFFER_POINTERS]; + /** + * @} + */ + + uint32_t pal[256]; ///< RGBA palette for the bitmap. + + char *text; ///< 0-terminated plain UTF-8 text + char *ass; ///< 0-terminated ASS/SSA compatible event line. + +} AVSubtitleArea; + +/** + * Return the name of sub_fmt, or NULL if sub_fmt is not + * recognized. + */ +const char *av_get_subtitle_fmt_name(enum AVSubtitleType sub_fmt); + +/** + * Return a subtitle format corresponding to name, or AV_SUBTITLE_FMT_NONE + * on error. + * + * @param name Subtitle format name. + */ +enum AVSubtitleType av_get_subtitle_fmt(const char *name); + #endif /* AVUTIL_SUBFMT_H */ From patchwork Mon Dec 13 23:40:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 32454 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp6138954iog; Mon, 13 Dec 2021 15:40:59 -0800 (PST) X-Google-Smtp-Source: ABdhPJw9crHxFB5hnl8YuJkO4+TY8BcmnPdKnZRE1AlObovmd7HbDjFW0VsgGNEOcFPUwFSHGJSQ X-Received: by 2002:a17:907:7e93:: with SMTP id qb19mr1606077ejc.613.1639438859283; Mon, 13 Dec 2021 15:40:59 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id f15si17251140edt.115.2021.12.13.15.40.58; Mon, 13 Dec 2021 15:40:59 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=XJ1yPxha; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id DE48468B075; Tue, 14 Dec 2021 01:40:39 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11olkn2032.outbound.protection.outlook.com [40.92.18.32]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7FC9768AED3 for ; Tue, 14 Dec 2021 01:40:36 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VCUliygG8s+JbOXfLqV4i2Etm66OK/YrNUhsVHx+qqouJuw6cSqNFEl8eseR8jEm80Swntdq4/vNGq8ZR7FCSVIHVyzb1LD4BmZ9FGFNq0VoWAUITbB6fbtkQUNRqNXCnRO1+wmFg11dl1ZDF+SgOYbXAOT/7G/wmJUvPBWYvqIVwPM/jOBA2d2TZgRgimKmXC/BnsiaInxoxKw2A7M6+UWBh0z7FkFyKn8JuEm+tRldXP+Ds1z8nfq0vwpULePIgq14DDwKdyIaHbynZk50CFYSGNMUobzQg0PgvtYXVpjHPO6rRwtlLwpgKHrO3EleHsNmWBnC5Vxj1JqhFuGVFw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=t6EU6R9g6brQVABHJUoWauKn6GnshJqGxlAC+Db+QJI=; b=lRRag5AT6T0FbacN07eKQBXwDN59QMXRnAB03/NLTRBMWsCINLxCmmMHE0m8piO8CNV9njALola04eKJvMJgbINMtPdptBpWHTPaXtQ2De13FiT9ZFjNaSqEaF0cce5MoziOqRUsdrSkOBauTpGBc8Jaw4//EJsK6Jj0Uf1QOT9xluInquRXtHQd7mE/piTLMNx6BfsukgCYyadiyiRDqTZ7qwy0P2x3mrhqHPwKqYg4PZkuc6CVgNXdrS0DWP7GX3nwBcAI+JY5CcInAA6fb4InnxWBbOxONQp1k+5GdMMI1IxSE6AcCCxFzcMo1X1ofyhjBYBKZmR2tR6WgwBd/w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=t6EU6R9g6brQVABHJUoWauKn6GnshJqGxlAC+Db+QJI=; b=XJ1yPxhaEooJFN3c6JnhYfj/XknCrJU+z+0xlvRcSjZyPdLk+pKjAuMya0A9B543ptsW7mfRtv++fXGIbVdU1yKbLO8Ub1XcjE7NKxCbnM6lswF3Hrn3u59GI+K4zDHPboAo5d4fpXleVaS6SpWOm9sINnBTIcfzx+e1crsrljgE/VRyxGUv9WH2Oqdj3UN3EZ34DMbjT7pGk6UM7KW4VEtBnznsoJfmBRV33o4OWrR+DRiEfl1Tv1bB0DmvsLTJZPtYKLZkx1LMVln7HLMp4nEFA1R0xxJSF/OZ5Zi1hC4MJ5FbaNJCuqQeCBNpaCVA+lAUApuAoxlHSgSMjc5cvg== Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::20) by DM8P223MB0303.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4778.12; Mon, 13 Dec 2021 23:40:25 +0000 Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775]) by DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775%7]) with mapi id 15.20.4778.018; Mon, 13 Dec 2021 23:40:25 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v24 03/21] avcodec/subtitles: Introduce new frame-based subtitle decoding API Thread-Index: AQHX8HrNquDoiA/Vx06gAgZ43c7M4Q== Date: Mon, 13 Dec 2021 23:40:25 +0000 Message-ID: References: <1ef42eb01b3627bbad539abd46206f320bb39697.1639438462.git.softworkz@hotmail.com> <4148d77a82d4b7e229fc81ce9bc1bc06904533f1.1639438462.git.softworkz@hotmail.com> In-Reply-To: <4148d77a82d4b7e229fc81ce9bc1bc06904533f1.1639438462.git.softworkz@hotmail.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [UGNj5lkS5P4M1tFxsnVY+Ey4C1io7xoG] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 1cc04668-c95d-4f75-1948-08d9be91f00c x-ms-traffictypediagnostic: DM8P223MB0303:EE_ x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: XHOHVpMJAq2kf7VTPz4+Ci9PfRbNCE6stJ8kTikkDgInUlL94qLgd9BSyqjSzb/vmkGXwTyxcvNU3dReYGm04Tr1TIGP04Mu60aXt4bvi8rOdgrV5heVDCxON2jlZXb4IZkWe7+TBJeSWhJQ8nc75Wj2QsLQE3c5TranpLQv9Vb/56gSeARXOtURhXXdJBOhQW7cmIPrxVNia3dhwKROSJBeHaHdcX1835BqxwTdWWCVTGCRBEr739w4dfjvxr/o+HsO/uvRZh1T42pCfRZlhyU/PFMy2Ff82+M+fFIDJyDFk7nAfwnIN2WOxklYvQRfjDCZbctIFXK5BalKZmBykmFvTvxDKDZPGbooxKQVNe+jkBzVrJpGkk1x0zbN/31+/i5lafuSa53QRmbPtZH4cC8KeZFku2qIoym8e+SCbkV+ui+a5t+ceqehLeKH74T2XAAOrppGkmlAdJSVQexrMJlTJyIKuDJ4Rr179T7ay9XWKewlytxrLNCmf2efhNd7U9YVRucR/uZTZgkaQtKa+0uVjZf0MUuAJmuuVKWc0hI6rNz80LzzR+zO2eRM85Z09Hn582eXyawUtjHx/VDh4g== x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?q?CfOo6AIS4AzaAq27THHViLf1U+7r?= =?utf-8?q?AKkdlQzhryn0Kgwe2gPkj1QPbzw+4VouQ2hU2kFsgsxH7SjAUvaEHH2XGTcPHS1s4?= =?utf-8?q?GEAKXftnfY1qSXUtR90yBklwwlZ8n1Wn7/tpQxXrtFubVE7D4cd1HvPOEKszEdAPO?= =?utf-8?q?SeM9vrbWxqovttMARGzWxX0olEgoZtwxivo96fXw+A2OLlsqz+rYd10xwMa2FsoDC?= =?utf-8?q?zvBVy83vRxwMD8EO3xSpGFfpkyHUByrobfpuHh1kKXMDda0ChxEHFb0C3lTa2IQdy?= =?utf-8?q?hYaHK6ib5oyZFNs+eWal3sAhU9cU6woKyrVyoGkYB7ziaz4HhYp5Nj6FEnVJ/IHK/?= =?utf-8?q?WG4jZsVJoC8WW+s13BlgP1D49EPGT2tPSiHSt1arREJYoRDp7YzPaQWSIOkl+I4Zb?= =?utf-8?q?ieaR4F/Ioruuj4l+ZjdJR87eM+4URLlHRw3bi6p5eQQehCwhF7brI+KD9im0ko02D?= =?utf-8?q?wFhvAbxHZRkpWbDdNRsNALifh1uqQ7bAJFdgVO8ILm8bMXTSN7nRTwyk996S11apx?= =?utf-8?q?JXD6Hm7p1cN6pcBwK3h6KR2ELEpb8oYk43KzBCVpDo0VuUrtiT8jyA7HPbGZWNfci?= =?utf-8?q?+0ZEMeSRHnSH+LqbqS4tTMeJepXzUQK4YNf4ua2Bt5T2wBwLiCYlsgrGRcvLKIR1z?= =?utf-8?q?zORNBUgmFTZguS4t9slawHH0BbzBOp0EcggtSgxFgKo6oKbDaOKgH3VWRo2v206Iu?= =?utf-8?q?eh0tpenUWm7S77Pc5QMyhtLPneTQA4xBh0jR1ixlgDSaId+0HZItuNnkxMH2OEEUq?= =?utf-8?q?Y2D9x7wUruWL4DjFslTBJcdUtvCm76Auitok+vVsyf6WzFaWy7YhfkruIHgg3lSzF?= =?utf-8?q?FFBN8qf41mDOLSV5KENiKGBhDpOiESWoPsXZUVIrTlw6e93BCwjuBb3vVE4Z6NMov?= =?utf-8?q?6IjyDud90Vh9S7VIn2fxNGX4MXMKli7beD85rEKQ+yPA=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-1ff67.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 1cc04668-c95d-4f75-1948-08d9be91f00c X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Dec 2021 23:40:25.5271 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8P223MB0303 Subject: [FFmpeg-devel] [PATCH v24 03/21] avcodec/subtitles: Introduce new frame-based subtitle decoding API X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: RuaktYOuSHpG - Add avcodec_decode_subtitle3 which takes subtitle frames, serving as compatibility shim to legacy subtitle decoding - Add additional methods for conversion between old and new API Signed-off-by: softworkz --- libavcodec/avcodec.h | 8 +- libavcodec/codec_desc.c | 11 +++ libavcodec/codec_desc.h | 8 ++ libavcodec/decode.c | 56 +++++++++++-- libavcodec/internal.h | 16 ++++ libavcodec/utils.c | 182 ++++++++++++++++++++++++++++++++++++++++ 6 files changed, 272 insertions(+), 9 deletions(-) diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index b05c12e47e..3e734d3003 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -1675,7 +1675,7 @@ typedef struct AVCodecContext { /** * Header containing style information for text subtitles. - * For SUBTITLE_ASS subtitle type, it should contain the whole ASS + * For AV_SUBTITLE_FMT_ASS subtitle type, it should contain the whole ASS * [Script Info] and [V4+ Styles] section, plus the [Events] line and * the Format line following. It shouldn't include any Dialogue line. * - encoding: Set/allocated/freed by user (before avcodec_open2()) @@ -2417,7 +2417,10 @@ int avcodec_close(AVCodecContext *avctx); * Free all allocated data in the given subtitle struct. * * @param sub AVSubtitle to free. + * + * @deprecated Use the regular frame based encode and decode APIs instead. */ +attribute_deprecated void avsubtitle_free(AVSubtitle *sub); /** @@ -2510,7 +2513,10 @@ enum AVChromaLocation avcodec_chroma_pos_to_enum(int xpos, int ypos); * must be freed with avsubtitle_free if *got_sub_ptr is set. * @param[in,out] got_sub_ptr Zero if no subtitle could be decompressed, otherwise, it is nonzero. * @param[in] avpkt The input AVPacket containing the input buffer. + * + * @deprecated Use the new decode API (avcodec_send_packet, avcodec_receive_frame) instead. */ +attribute_deprecated int avcodec_decode_subtitle2(AVCodecContext *avctx, AVSubtitle *sub, int *got_sub_ptr, AVPacket *avpkt); diff --git a/libavcodec/codec_desc.c b/libavcodec/codec_desc.c index 0974ee03de..e48e4532ba 100644 --- a/libavcodec/codec_desc.c +++ b/libavcodec/codec_desc.c @@ -3548,3 +3548,14 @@ enum AVMediaType avcodec_get_type(enum AVCodecID codec_id) const AVCodecDescriptor *desc = avcodec_descriptor_get(codec_id); return desc ? desc->type : AVMEDIA_TYPE_UNKNOWN; } + +enum AVSubtitleType avcodec_descriptor_get_subtitle_format(const AVCodecDescriptor *codec_descriptor) +{ + if(codec_descriptor->props & AV_CODEC_PROP_BITMAP_SUB) + return AV_SUBTITLE_FMT_BITMAP; + + if(codec_descriptor->props & AV_CODEC_PROP_TEXT_SUB) + return AV_SUBTITLE_FMT_ASS; + + return AV_SUBTITLE_FMT_UNKNOWN; +} diff --git a/libavcodec/codec_desc.h b/libavcodec/codec_desc.h index 126b52df47..ba68d24e0e 100644 --- a/libavcodec/codec_desc.h +++ b/libavcodec/codec_desc.h @@ -121,6 +121,14 @@ const AVCodecDescriptor *avcodec_descriptor_next(const AVCodecDescriptor *prev); */ const AVCodecDescriptor *avcodec_descriptor_get_by_name(const char *name); +/** + * Return subtitle format from a codec descriptor + * + * @param codec_descriptor codec descriptor + * @return the subtitle type (e.g. bitmap, text) + */ +enum AVSubtitleType avcodec_descriptor_get_subtitle_format(const AVCodecDescriptor *codec_descriptor); + /** * @} */ diff --git a/libavcodec/decode.c b/libavcodec/decode.c index 52bf5dcd33..7ccf494f9a 100644 --- a/libavcodec/decode.c +++ b/libavcodec/decode.c @@ -576,6 +576,39 @@ static int decode_receive_frame_internal(AVCodecContext *avctx, AVFrame *frame) return ret; } +static int decode_subtitle2_priv(AVCodecContext *avctx, AVSubtitle *sub, + int *got_sub_ptr, AVPacket *avpkt); + +static int decode_subtitle_shim(AVCodecContext *avctx, AVFrame *frame, AVPacket *avpkt) +{ + int ret, got_sub_ptr = 0; + AVSubtitle subtitle = { 0 }; + + if (frame->buf[0]) + return AVERROR(EAGAIN); + + av_frame_unref(frame); + + ret = decode_subtitle2_priv(avctx, &subtitle, &got_sub_ptr, avpkt); + + if (ret >= 0 && got_sub_ptr) { + frame->type = AVMEDIA_TYPE_SUBTITLE; + frame->format = subtitle.format; + ret = av_frame_get_buffer2(frame, 0); + + if (ret >= 0) + ret = ff_frame_put_subtitle(frame, &subtitle); + + frame->width = avctx->width; + frame->height = avctx->height; + frame->pkt_dts = avpkt->dts; + } + + avsubtitle_free(&subtitle); + + return ret; +} + int attribute_align_arg avcodec_send_packet(AVCodecContext *avctx, const AVPacket *avpkt) { AVCodecInternal *avci = avctx->internal; @@ -590,6 +623,9 @@ int attribute_align_arg avcodec_send_packet(AVCodecContext *avctx, const AVPacke if (avpkt && !avpkt->size && avpkt->data) return AVERROR(EINVAL); + if (avctx->codec_type == AVMEDIA_TYPE_SUBTITLE) + return decode_subtitle_shim(avctx, avci->buffer_frame, avpkt); + av_packet_unref(avci->buffer_pkt); if (avpkt && (avpkt->data || avpkt->side_data_elems)) { ret = av_packet_ref(avci->buffer_pkt, avpkt); @@ -651,7 +687,9 @@ int attribute_align_arg avcodec_receive_frame(AVCodecContext *avctx, AVFrame *fr if (avci->buffer_frame->buf[0]) { av_frame_move_ref(frame, avci->buffer_frame); - } else { + } else if (avctx->codec_type == AVMEDIA_TYPE_SUBTITLE) + return AVERROR(EAGAIN); + else { ret = decode_receive_frame_internal(avctx, frame); if (ret < 0) return ret; @@ -802,9 +840,8 @@ static int utf8_check(const uint8_t *str) return 1; } -int avcodec_decode_subtitle2(AVCodecContext *avctx, AVSubtitle *sub, - int *got_sub_ptr, - AVPacket *avpkt) +static int decode_subtitle2_priv(AVCodecContext *avctx, AVSubtitle *sub, + int *got_sub_ptr, AVPacket *avpkt) { int ret = 0; @@ -844,10 +881,7 @@ int avcodec_decode_subtitle2(AVCodecContext *avctx, AVSubtitle *sub, avctx->pkt_timebase, ms); } - if (avctx->codec_descriptor->props & AV_CODEC_PROP_BITMAP_SUB) - sub->format = 0; - else if (avctx->codec_descriptor->props & AV_CODEC_PROP_TEXT_SUB) - sub->format = 1; + sub->format = avcodec_descriptor_get_subtitle_format(avctx->codec_descriptor); for (unsigned i = 0; i < sub->num_rects; i++) { if (avctx->sub_charenc_mode != FF_SUB_CHARENC_MODE_IGNORE && @@ -871,6 +905,12 @@ int avcodec_decode_subtitle2(AVCodecContext *avctx, AVSubtitle *sub, return ret; } +int avcodec_decode_subtitle2(AVCodecContext *avctx, AVSubtitle *sub, + int *got_sub_ptr, AVPacket *avpkt) +{ + return decode_subtitle2_priv(avctx, sub, got_sub_ptr, avpkt); +} + enum AVPixelFormat avcodec_default_get_format(struct AVCodecContext *avctx, const enum AVPixelFormat *fmt) { diff --git a/libavcodec/internal.h b/libavcodec/internal.h index a62f8dbd4e..10443ed2d1 100644 --- a/libavcodec/internal.h +++ b/libavcodec/internal.h @@ -363,6 +363,22 @@ int ff_int_from_list_or_default(void *ctx, const char * val_name, int val, void ff_dvdsub_parse_palette(uint32_t *palette, const char *p); +/** + * Copies subtitle data from AVSubtitle to AVFrame. + * + * @deprecated This is a compatibility method for interoperability with + * the legacy subtitle API. + */ +int ff_frame_put_subtitle(AVFrame* frame, const AVSubtitle* sub); + +/** + * Copies subtitle data from AVFrame to AVSubtitle. + * + * @deprecated This is a compatibility method for interoperability with + * the legacy subtitle API. + */ +int ff_frame_get_subtitle(AVSubtitle* sub, AVFrame* frame); + #if defined(_WIN32) && CONFIG_SHARED && !defined(BUILDING_avcodec) # define av_export_avcodec __declspec(dllimport) #else diff --git a/libavcodec/utils.c b/libavcodec/utils.c index a91a54b0dc..613c580c19 100644 --- a/libavcodec/utils.c +++ b/libavcodec/utils.c @@ -824,6 +824,188 @@ int av_get_audio_frame_duration(AVCodecContext *avctx, int frame_bytes) return FFMAX(0, duration); } +static int subtitle_area2rect(AVSubtitleRect *dst, const AVSubtitleArea *src) +{ + dst->x = src->x; + dst->y = src->y; + dst->w = src->w; + dst->h = src->h; + dst->nb_colors = src->nb_colors; + dst->type = src->type; + dst->flags = src->flags; + + switch (dst->type) { + case AV_SUBTITLE_FMT_BITMAP: + + if (src->h > 0 && src->w > 0 && src->buf[0]) { + uint32_t *pal; + AVBufferRef *buf = src->buf[0]; + dst->data[0] = av_mallocz(buf->size); + memcpy(dst->data[0], buf->data, buf->size); + dst->linesize[0] = src->linesize[0]; + + dst->data[1] = av_mallocz(256 * 4); + pal = (uint32_t *)dst->data[1]; + + for (unsigned i = 0; i < 256; i++) { + pal[i] = src->pal[i]; + } + } + + break; + case AV_SUBTITLE_FMT_TEXT: + + if (src->text) + dst->text = av_strdup(src->text); + else + dst->text = av_strdup(""); + + if (!dst->text) + return AVERROR(ENOMEM); + + break; + case AV_SUBTITLE_FMT_ASS: + + if (src->ass) + dst->ass = av_strdup(src->ass); + else + dst->ass = av_strdup(""); + + if (!dst->ass) + return AVERROR(ENOMEM); + + break; + default: + + av_log(NULL, AV_LOG_ERROR, "Subtitle rect has invalid format: %d", dst->type); + return AVERROR(EINVAL); + } + + return 0; +} + +static int subtitle_rect2area(AVSubtitleArea *dst, const AVSubtitleRect *src) +{ + dst->x = src->x; + dst->y = src->y; + dst->w = src->w; + dst->h = src->h; + dst->nb_colors = src->nb_colors; + dst->type = src->type; + dst->flags = src->flags; + + switch (dst->type) { + case AV_SUBTITLE_FMT_BITMAP: + + if (src->h > 0 && src->w > 0 && src->data[0]) { + AVBufferRef *buf = av_buffer_allocz(src->h * src->linesize[0]); + memcpy(buf->data, src->data[0], buf->size); + + dst->buf[0] = buf; + dst->linesize[0] = src->linesize[0]; + } + + if (src->data[1]) { + uint32_t *pal = (uint32_t *)src->data[1]; + + for (unsigned i = 0; i < 256; i++) { + dst->pal[i] = pal[i]; + } + } + + break; + case AV_SUBTITLE_FMT_TEXT: + + if (src->text) { + dst->text = av_strdup(src->text); + if (!dst->text) + return AVERROR(ENOMEM); + } + + break; + case AV_SUBTITLE_FMT_ASS: + + if (src->ass) { + dst->ass = av_strdup(src->ass); + if (!dst->ass) + return AVERROR(ENOMEM); + } + + break; + default: + + av_log(NULL, AV_LOG_ERROR, "Subtitle area has invalid format: %d", dst->type); + return AVERROR(EINVAL); + } + + return 0; +} + +/** + * Copies subtitle data from AVSubtitle (deprecated) to AVFrame + * + * @note This is a compatibility method for conversion to the legacy API + */ +int ff_frame_put_subtitle(AVFrame *frame, const AVSubtitle *sub) +{ + frame->format = sub->format; + frame->subtitle_start_time = sub->start_display_time; + frame->subtitle_end_time = sub->end_display_time; + frame->subtitle_pts = sub->pts; + + if (sub->num_rects) { + frame->subtitle_areas = av_malloc_array(sub->num_rects, sizeof(AVSubtitleArea*)); + if (!frame->subtitle_areas) + return AVERROR(ENOMEM); + + for (unsigned i = 0; i < sub->num_rects; i++) { + int ret; + frame->subtitle_areas[i] = av_mallocz(sizeof(AVSubtitleArea)); + if (!frame->subtitle_areas[i]) + return AVERROR(ENOMEM); + ret = subtitle_rect2area(frame->subtitle_areas[i], sub->rects[i]); + if (ret < 0) { + frame->num_subtitle_areas = i; + return ret; + } + } + } + + frame->num_subtitle_areas = sub->num_rects; + return 0; +} + +/** + * Copies subtitle data from AVFrame to AVSubtitle (deprecated) + * + * @note This is a compatibility method for conversion to the legacy API + */ +int ff_frame_get_subtitle(AVSubtitle *sub, AVFrame *frame) +{ + sub->start_display_time = frame->subtitle_start_time; + sub->end_display_time = frame->subtitle_end_time; + sub->pts = frame->subtitle_pts; + + if (frame->num_subtitle_areas) { + sub->rects = av_malloc_array(frame->num_subtitle_areas, sizeof(AVSubtitleRect*)); + if (!sub->rects) + return AVERROR(ENOMEM); + + for (unsigned i = 0; i < frame->num_subtitle_areas; i++) { + int ret; + sub->rects[i] = av_mallocz(sizeof(AVSubtitleRect)); + ret = subtitle_area2rect(sub->rects[i], frame->subtitle_areas[i]); + if (ret < 0) { + sub->num_rects = i; + return ret; + } + } + } + + sub->num_rects = frame->num_subtitle_areas; + return 0; +} + int av_get_audio_frame_duration2(AVCodecParameters *par, int frame_bytes) { int duration = get_audio_frame_duration(par->codec_id, par->sample_rate, From patchwork Mon Dec 13 23:40:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 32455 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp6139142iog; Mon, 13 Dec 2021 15:41:09 -0800 (PST) X-Google-Smtp-Source: ABdhPJzjmVUxsfewXhdNB/z4WMCBzt2nfgQDKBsL0sx2wSvJ3vZGmZDswBShStsy695xDbx+H04V X-Received: by 2002:aa7:c846:: with SMTP id g6mr2695201edt.75.1639438869201; Mon, 13 Dec 2021 15:41:09 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id he6si19938189ejc.702.2021.12.13.15.41.08; Mon, 13 Dec 2021 15:41:09 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=DCnMki8A; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id CDBBD68B06D; Tue, 14 Dec 2021 01:40:40 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11olkn2032.outbound.protection.outlook.com [40.92.18.32]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id EE67F68B05B for ; Tue, 14 Dec 2021 01:40:36 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XrlqaUicjvml301141Mrc4iAYBgyQVIEjxCvpEzZXG2MtApPpZ/rly2JbnWaxJjFmo5KfrPRSlhaW1L6VHEwlHqhE6j0tYXVYzY8kRpKWYjeG2HTbnSBar4hjysXBBUVL83pGAXMTbaQO6xuMm8w50gew6q1G4iuYSrSSh/Ueak4oKX5mYrqUjCh83HYuD0GogI4FqaA63yhCh4+MGFGAa/UQ/NkXCABriYdQYFlenMFKgvPAWKqzWTPwziXvP4J4rzOJkGkXBsPEmxemqlaoOx5WE/Jvuyz3eiH9zwnA5qLgmmhsqQ8iH5vQJKhlvfAXj/5K0fyZNbt41K23zVeFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=AoI/EGlNnnc++ofMlMGwlI1pqDtONuV76wSE4f+/6Rs=; b=Gp1Eq3ApkiDen8FCJTybEHO7UsbG87QIdXUP5sMGk3H4kKU/rnjpDcH8Kv09GthUPEeL20aTOKWouh3kudLqmWNMbPVlavT26mrJAvATyWzKPbaaA2EWvrR8+yhcze3sc+tu+xEpGqngOy445X6ecsGQQR6hrAkVafUJEwvOwdkha1BnBCJ413yzXvQkE2ouySV+AcwVMxxvLdWHdCtWa36Sfe/W4NLqeqLbsCsHzaWc/iCMqvgB8YAj+B4mgBRYJVkE8u0zcAgtgHwGH00RjNS2iayFOuK63E+zvVlMLooyuQwESbQWwTIzD/tpJ633D/aDyTgAvj+o+kHQGsWKfg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AoI/EGlNnnc++ofMlMGwlI1pqDtONuV76wSE4f+/6Rs=; b=DCnMki8AIKrkxn9G46CQeiSWBWU1w+2mKpxrbvpEw1MzZr3Tz9zZ8T7tpyBjUl+XfbKj75W2DD1N6CSz6aV12GD0l2wthNtv8OuEaeOxly7i3ebFdWRzsjmMLVoIedeiCk7xoUgmJSbBFUoTxHu+6AGbYo8K+e171Uam2TQ9v3OW9YffMwXoy6WR4QazAgH4VKSqbCACJHsNwYs8cZp8p21vMOxrSHOdAP0bmYmWIvI4Fw3hK8OpgQhG8jGISle8HyKpeTypn2AYZoMqh4uDY7cFeUk8bSY4aa3UY/CdvBT8dUh9Bz8Pre9ZV+8RZZ/W+x3UA4LG5zqPl/n5wuNABw== Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::20) by DM8P223MB0303.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4778.12; Mon, 13 Dec 2021 23:40:27 +0000 Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775]) by DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775%7]) with mapi id 15.20.4778.018; Mon, 13 Dec 2021 23:40:27 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v24 04/21] avfilter/subtitles: Update vf_subtitles to use new decoding api Thread-Index: AQHX8HrOWYsH0y28PU+Dub6vZVeOdQ== Date: Mon, 13 Dec 2021 23:40:27 +0000 Message-ID: References: <1ef42eb01b3627bbad539abd46206f320bb39697.1639438462.git.softworkz@hotmail.com> <4148d77a82d4b7e229fc81ce9bc1bc06904533f1.1639438462.git.softworkz@hotmail.com> <2ac798924ee984010d2e85f00bc992d9aa106171.1639438462.git.softworkz@hotmail.com> In-Reply-To: <2ac798924ee984010d2e85f00bc992d9aa106171.1639438462.git.softworkz@hotmail.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [ATV5SFEvzipgiN0M769QIao6RaqWIGOL] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 08e77961-f5dc-4cd5-3d6b-08d9be91f124 x-ms-traffictypediagnostic: DM8P223MB0303:EE_ x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: TJ4caWznrDG4MFcYzCCM4tn/T8CUu40KJqf3nGaTUiOxtOP7sbcOvzh9uuTci74hSdtAcH/sYNHiaSm1dRBUMMr4w9Ayq5r24Q4clwPxzF6uBVriRIX3m71mMu/YkFKTh/4Kr2li2Luqq00K/e5S25sunqDPwQhMHgl2dUAfLhWePdhUo4S32EVmNWwlm4CyC5sjV4ewAzpeKRADMmCm6x4Is2/ukfehkrAUMdahRi4X7BEuslsdd27BppBOYAC66Q4snX73uIVetXM2QHSekmVCTkjeZXwrCEKIoO7fzOFR92I0kQu50zFZNgsl5J2U8z0496T7ObT1390vcRyFa0t1FPoOvjaZLvFvEbhek6crL1kk7sYODeeZEbqSu1G8+f7ClV2ioDtB4Lc9oD23ubR3wKm8tmspLYRRtRaTEGBWf2yGvlORv5D6Z5jsM2BqBfAVIbEPjYRUbi2J2mqd1onM1OVd2V3/ZcOKAU44kwQucqapcGxMrEBywQwUURvGewLfrvFJ8rySoa8kuMfhs7etRvqpA4KxdnddK9D8ptdYQecEVyjeueFqXD1kD+R86Vvt826zecJrYwdlLcfBYw== x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?q?P1OssGyB1g7+3P+vGO2oot6Xwqvq?= =?utf-8?q?X5HoJrOpqRxCZkjX3FpyU0Rq9wlCPKVMDExCS2Wd0fvfReAbVxlRhATLqRJ885QZS?= =?utf-8?q?nBjM3ObrNj8ePiicGxRL367Dlq4EdH0DQM2Jmr2Z/zoKtNjIODCemHVfPbJ0I5Hk3?= =?utf-8?q?VN5up8D3GLFGdhLM10evz21hxFHffIP4T16zTmieEttfA7bCRa5lmPaiMtQVG3AmM?= =?utf-8?q?m2feU8vtnVJrXTH1cazLMqZPsbSs7esOxNpGGZ30JJ3AJoHakW5M2mT8AFqp1ihfu?= =?utf-8?q?XWhcK2sUbXlTXzbgkzScf5hJXhKBZkgTjYuIlvlwNS+3c04yqOM7osc9uyesD9QVZ?= =?utf-8?q?cvzbc1t8CqOLyAzi/mMvWEdFVO/47XRd9x1r3YNjIuBXgYBr5nyG/3FfB5kN4S+10?= =?utf-8?q?11mWJuFSaf2SynpYvCLMe09B2xQU78YxMEpmPYZzGiCbLBoX95YOEEFut+RAHikuC?= =?utf-8?q?8M1dW9nfQRSQ9vHsCA4CeRpsT39rnThac13vtswSkqaxL1CriG8OnzxKeEL11LAFM?= =?utf-8?q?SpVz04wSRQCGkZpxEH08vRMDDGmPEJUSiyBcOsuxs8zwJMfTHRhdejSRCQJtCjI7l?= =?utf-8?q?EPs/wsCkOVukKpGIj1EwTsABcTj8EQ45d5bkv+Yy9z2V8s15IfPZkFkg+hENiGL5m?= =?utf-8?q?vus7F5mK+5Oy0FgKEjXCTNSV7cyCiKIQauzg1g2gZz9NT6wp+BbLbvDq6UKmnDSfU?= =?utf-8?q?wI2RuJ/+tmUfImYgW9ofKjEeCo/HwMp7oC5Rj+Lcd5wt/scjgkGTzq7tWbMPuzveQ?= =?utf-8?q?Pbsdav6zMBKygbDlebDMX7pZvOCJeZ/W/NIMMhmYHcQXS0JpLwT32lBcBN3vdihwZ?= =?utf-8?q?RwcWp9Aw+TAAajxqQomcUcoajxuer7cBempmdzsDuYGM3eNIfJ4FZuWGxtBiwxKhr?= =?utf-8?q?QQSLIJQiIBcbdIBlL9zDD1UXoUXHRBNQZBYIJV5qq/sA=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-1ff67.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 08e77961-f5dc-4cd5-3d6b-08d9be91f124 X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Dec 2021 23:40:27.4556 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8P223MB0303 Subject: [FFmpeg-devel] [PATCH v24 04/21] avfilter/subtitles: Update vf_subtitles to use new decoding api X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: XLvj164kiGSs Signed-off-by: softworkz --- libavfilter/vf_subtitles.c | 54 +++++++++++++++++++++++++++++--------- 1 file changed, 42 insertions(+), 12 deletions(-) diff --git a/libavfilter/vf_subtitles.c b/libavfilter/vf_subtitles.c index 3fc4eeb63d..66b2082894 100644 --- a/libavfilter/vf_subtitles.c +++ b/libavfilter/vf_subtitles.c @@ -35,14 +35,12 @@ # include "libavformat/avformat.h" #endif #include "libavutil/avstring.h" -#include "libavutil/imgutils.h" #include "libavutil/opt.h" #include "libavutil/parseutils.h" #include "drawutils.h" #include "avfilter.h" #include "internal.h" #include "formats.h" -#include "video.h" typedef struct AssContext { const AVClass *class; @@ -292,6 +290,29 @@ static int attachment_is_font(AVStream * st) return 0; } +static int decode(AVCodecContext *avctx, AVFrame *frame, int *got_frame, AVPacket *pkt) +{ + int ret; + + *got_frame = 0; + + if (pkt) { + ret = avcodec_send_packet(avctx, pkt); + // In particular, we don't expect AVERROR(EAGAIN), because we read all + // decoded frames with avcodec_receive_frame() until done. + if (ret < 0 && ret != AVERROR_EOF) + return ret; + } + + ret = avcodec_receive_frame(avctx, frame); + if (ret < 0 && ret != AVERROR(EAGAIN)) + return ret; + if (ret >= 0) + *got_frame = 1; + + return 0; +} + AVFILTER_DEFINE_CLASS(subtitles); static av_cold int init_subtitles(AVFilterContext *ctx) @@ -306,6 +327,7 @@ static av_cold int init_subtitles(AVFilterContext *ctx) AVStream *st; AVPacket pkt; AssContext *ass = ctx->priv; + enum AVSubtitleType subtitle_format; /* Init libass */ ret = init(ctx); @@ -386,13 +408,17 @@ static av_cold int init_subtitles(AVFilterContext *ctx) ret = AVERROR_DECODER_NOT_FOUND; goto end; } + dec_desc = avcodec_descriptor_get(st->codecpar->codec_id); - if (dec_desc && !(dec_desc->props & AV_CODEC_PROP_TEXT_SUB)) { + subtitle_format = avcodec_descriptor_get_subtitle_format(dec_desc); + + if (subtitle_format != AV_SUBTITLE_FMT_ASS) { av_log(ctx, AV_LOG_ERROR, - "Only text based subtitles are currently supported\n"); - ret = AVERROR_PATCHWELCOME; + "Only text based subtitles are supported by this filter\n"); + ret = AVERROR_INVALIDDATA; goto end; } + if (ass->charenc) av_dict_set(&codec_opts, "sub_charenc", ass->charenc, 0); @@ -448,18 +474,22 @@ static av_cold int init_subtitles(AVFilterContext *ctx) dec_ctx->subtitle_header_size); while (av_read_frame(fmt, &pkt) >= 0) { int i, got_subtitle; - AVSubtitle sub = {0}; + AVFrame *sub = av_frame_alloc(); + if (!sub) { + ret = AVERROR(ENOMEM); + goto end; + } if (pkt.stream_index == sid) { - ret = avcodec_decode_subtitle2(dec_ctx, &sub, &got_subtitle, &pkt); + ret = decode(dec_ctx, sub, &got_subtitle, &pkt); if (ret < 0) { av_log(ctx, AV_LOG_WARNING, "Error decoding: %s (ignored)\n", av_err2str(ret)); } else if (got_subtitle) { - const int64_t start_time = av_rescale_q(sub.pts, AV_TIME_BASE_Q, av_make_q(1, 1000)); - const int64_t duration = sub.end_display_time; - for (i = 0; i < sub.num_rects; i++) { - char *ass_line = sub.rects[i]->ass; + const int64_t start_time = av_rescale_q(sub->subtitle_pts, AV_TIME_BASE_Q, av_make_q(1, 1000)); + const int64_t duration = sub->subtitle_end_time; + for (i = 0; i < sub->num_subtitle_areas; i++) { + char *ass_line = sub->subtitle_areas[i]->ass; if (!ass_line) break; ass_process_chunk(ass->track, ass_line, strlen(ass_line), @@ -468,7 +498,7 @@ static av_cold int init_subtitles(AVFilterContext *ctx) } } av_packet_unref(&pkt); - avsubtitle_free(&sub); + av_frame_free(&sub); } end: From patchwork Mon Dec 13 23:40:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 32456 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp6139310iog; Mon, 13 Dec 2021 15:41:19 -0800 (PST) X-Google-Smtp-Source: ABdhPJyL5+CauKGh1YCMvifH99s+S7LKtB1tkPnpjAhUPUQqfOB9861O87SAPPOlH+pSvRDS+Ahn X-Received: by 2002:a17:906:b785:: with SMTP id dt5mr1597982ejb.515.1639438879409; Mon, 13 Dec 2021 15:41:19 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a14si18014229ejs.205.2021.12.13.15.41.19; Mon, 13 Dec 2021 15:41:19 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=iE5E3ghY; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 04CB668B074; Tue, 14 Dec 2021 01:40:42 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11olkn2032.outbound.protection.outlook.com [40.92.18.32]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7511768AED3 for ; Tue, 14 Dec 2021 01:40:37 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=h0AuFgw+Wb4Iu27lE+49xmya3Ywevz9cKPU8SKPkHeVn3VMyM6DUMfA6zGDJiViEFJyKdA6nG5ZltTelw6D4F3E/zJuDw9CftlSwZtH6fP0ke1673o7hDwL8sCPXvcx90BsRnE3fFgkEA97eczrZyvBDZ0gcEY4AW4NFCmw9gM1c/7cYv+sq7/seZsHbL5PRYIZ5JddXObppfGFV/3UFqW8mW7AnclHM+hT75w6jTy5J2yu92he3mbeXPHrApdjn16ecgwwqXUJljwIExsCUKDW/wejXThO4W2UtayRXxTEPocQwUWRNvOTmLqCx8O2fnJabDzyjOl6NiX8TdsYSMA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=oFfU/sXaUQYhCvKNXsRQzknUJyGEY2gaS+M7UiYUtUs=; b=WQcizIPJH71AdA299s0RGJNXZR2hOgaU+sZ/1IQWAeb682YGOUODJzoITDKnvkTAokkJQWQmtdsDhE1zyZj2oJ+jeVVYFERskLabJ/qDPZY8Ed+Q0VAKW7y7z20dIeqSSxQRz6nYQNr6jDMrwkpuSWf+92ezrvpTVFUrZg5xGMKk7kJBH5pxDYbW8o5VT89yyGuRZkKasNvgQBIuXcIQWWVxSeV4C9nWfo/eFT2xEV+afVQfFOgPIdvr3c54IuwemLUhr0M7U4g/3q9E58cyiMUOqTWW6cloceon2Z7tVJ8+0Jhj15ExfRCu9nPDyfLNnhFnZg/mXnVzqLrCof97Gg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=oFfU/sXaUQYhCvKNXsRQzknUJyGEY2gaS+M7UiYUtUs=; b=iE5E3ghYcF3gYlgOfkcWS5HVr51DM/AiV3AqN1pStCs5QlBbMUonTa7RGwsQ4zrjMUnK80puv/iW1xOU+TvzkQdxk2tBW5xn4RYl/ClL5uVD1Czjp0YQ8r8Sv+S9H0EgHDIEExpriq/Ngtw3xfeeH/dpFO5Ns/g5aUfhK9ebg3ej1WCyxXdWf3FFVaa3rKCq1W/zteG2CzpVzC4TpxHmD9C5xVJLf7Vq0NPjrCov6EhYgQa8unsnenpNhsytL32Ni6U36m1xeAQ5dMys0nT5hPVIWAffiJofE99XGJNEn4VrpFeBNRau2PcBO4FDSnij72K/kbO0etVHC3ODYXqImA== Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::20) by DM8P223MB0303.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4778.12; Mon, 13 Dec 2021 23:40:29 +0000 Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775]) by DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775%7]) with mapi id 15.20.4778.018; Mon, 13 Dec 2021 23:40:29 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v24 05/21] avcodec,avutil: Move ass helper functions to avutil as avpriv_ and extend ass dialog parsing Thread-Index: AQHX8HrQ0Am8VFReNkCcNAdHVALN5A== Date: Mon, 13 Dec 2021 23:40:29 +0000 Message-ID: References: <1ef42eb01b3627bbad539abd46206f320bb39697.1639438462.git.softworkz@hotmail.com> <4148d77a82d4b7e229fc81ce9bc1bc06904533f1.1639438462.git.softworkz@hotmail.com> <2ac798924ee984010d2e85f00bc992d9aa106171.1639438462.git.softworkz@hotmail.com> <300ea7188b331ca5e00dd7e04c03df8aba2a99c0.1639438462.git.softworkz@hotmail.com> In-Reply-To: <300ea7188b331ca5e00dd7e04c03df8aba2a99c0.1639438462.git.softworkz@hotmail.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [ipLzz4ADjpP0JSJ4xXCwHn8zvr27Ktwd] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 25e08374-bf33-4d12-cc75-08d9be91f294 x-ms-traffictypediagnostic: DM8P223MB0303:EE_ x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: tnAmENpHwqjdgC7e0o8tA0Oi3hN0dvYkEr1ZKcwd4/2H/JLHXFgzEVcVozmZdjZGyqBhucBMHUzl7A+VL+hSq1gIs7X086ZCjYHWwYWEbc/RpFw+7kkpQEIRuznGPPOCc6xyABSOCuSc3YEdcArQOxRk+GVHxuf+PO+sMyKQ1XkqTaUKN9GJ029q4TJQNGGk2yGC+F6nW3TQZy9KDP9TEPxv+bILXPvOxVPKaZ5lFh3enrcPfmGZfBBx1g73y6b1DS6UV7ZhWCjIRTsJBwigvi1CR4lGcSfENJv1rvh3Z2RFujPZpoIcRyUzUwIArn0YRWKCr3DsS0kjeHkt7a6IPbq2Rh7gecu98DMET6USOKk1R0sGUwbP1wMPMU4jH8vmDCrGX6an7ZgsHZvErxn8bajyKW94vZDlsxjNp3RbuJui5rr121y80CkIVEn5mJdedPxpCkpLOwXQkA95tNPI5joxlPFTKhkScM/JMOEg1e3+gBb6RgQUQpeTa20srbaahMleIxWZJ4IHGeF7+DnKlJ7oVFJ7Dxh0C30ju2eR5T43NRFw8ZD0iAe+kgynzVBSc/DgD0vXTeWqqK09cYFFUQ== x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?q?BGDWgysg7NNPRb9YJd4A2gDAf8hp?= =?utf-8?q?02nrxblRBhkegWlsgN5i8XB++6i9wbqAPALUzNb4V3yce+5Yp+vRvOOO1+QKcuOed?= =?utf-8?q?+CbpBT/y80B3HJ5W3ZiNpIDP9VElReXHnDaKC0+jx/GJKYg0PXr3IyBiR77pPAsds?= =?utf-8?q?+98wzkbaXs0JNKVuSQnbsUnX2sckvRK15E4oHf5Vd73NoiK/kxghuSatPn8qqhgHo?= =?utf-8?q?OHXJjEyVpXD9TGsg7UjoZYSSdIJ7Nc3ZueaPi9rygszaK9aqKuOqyzrdliHSsuMLl?= =?utf-8?q?+MxLBqMuK2FR5qewsK4kshlnQq63ufp2eMhoQ3uUNCEx8QSypj25owf2GBCvMQoDJ?= =?utf-8?q?o6Wq81IXPhP1rVJNaWmVGNUrInygQzHyJ6NIi+CQqQ7NvStc2IvUAbXjQrDq0dqlo?= =?utf-8?q?wkA8PCuKwakJYR/0Hut0KWIPpgwRTr6WilYrEvty4GdYCnSl7XpeQIfCvdGYBSxLt?= =?utf-8?q?vdTlpFUh/wve9euy55eQx4ni7dWSeraOuXGh1AbD3O6acTPzACIvEQDUgeJEzHy/9?= =?utf-8?q?BhN6gtaGdqBFLGaIiTaXFcX4r85p2A1TEeDEtQuzs4WwN0SWh1qze9w+WScl7e5vX?= =?utf-8?q?eCjjk9eRP3BQgznWGPNysOp3BpCv9EGxfSvmHmAX20wUSwXe2IID9SwGdf505CYhV?= =?utf-8?q?PNYzLGmdda2WTnTcQMiNTNlU5FzTes4hnE1xoAlUWTmkXXZGICmpTYWcXMVwYYtLL?= =?utf-8?q?tZ6HXSWDsuxH8OVTdn2qe1tzmPjbUrEjNUYg6Vdsr8HvgbPJWDLJfwWRjQ6CYVmFU?= =?utf-8?q?jlbOzGlcr8MLxqzv+EB3jfpOEjS72dAw/bVSI2ChvluE05vpgnMcLZK9kUYdo/lrl?= =?utf-8?q?+N9Q6NmlxxJU1Oi08qZtdGl369GTx79J4IWk2jchoVyZXAS43sDYXJ/riXYANKAZz?= =?utf-8?q?YQlOaVzNpvarZsDBiAoGq1KEARKTpbAhgVXeYydDeQrA=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-1ff67.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 25e08374-bf33-4d12-cc75-08d9be91f294 X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Dec 2021 23:40:29.8312 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8P223MB0303 Subject: [FFmpeg-devel] [PATCH v24 05/21] avcodec, avutil: Move ass helper functions to avutil as avpriv_ and extend ass dialog parsing X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: I3LBjv617lVq Also add - hard_space callback (for upcoming fix) - extensible callback (for future extension) Signed-off-by: softworkz --- libavcodec/Makefile | 56 +++---- libavcodec/ass.h | 147 ++++++------------ libavcodec/assdec.c | 2 +- libavcodec/assenc.c | 2 +- libavcodec/ccaption_dec.c | 19 +-- libavcodec/jacosubdec.c | 2 +- libavcodec/libaribb24.c | 2 +- libavcodec/libzvbi-teletextdec.c | 14 +- libavcodec/microdvddec.c | 7 +- libavcodec/movtextdec.c | 3 +- libavcodec/movtextenc.c | 20 +-- libavcodec/mpl2dec.c | 2 +- libavcodec/realtextdec.c | 2 +- libavcodec/samidec.c | 2 +- libavcodec/srtdec.c | 2 +- libavcodec/srtenc.c | 16 +- libavcodec/subviewerdec.c | 2 +- libavcodec/textdec.c | 4 +- libavcodec/ttmlenc.c | 15 +- libavcodec/webvttdec.c | 2 +- libavcodec/webvttenc.c | 16 +- libavutil/Makefile | 2 + {libavcodec => libavutil}/ass.c | 87 ++++------- libavutil/ass_internal.h | 133 ++++++++++++++++ {libavcodec => libavutil}/ass_split.c | 30 ++-- .../ass_split_internal.h | 32 ++-- 26 files changed, 347 insertions(+), 274 deletions(-) rename {libavcodec => libavutil}/ass.c (65%) create mode 100644 libavutil/ass_internal.h rename {libavcodec => libavutil}/ass_split.c (94%) rename libavcodec/ass_split.h => libavutil/ass_split_internal.h (86%) diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 4122a9b144..b12638da5e 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -209,10 +209,10 @@ OBJS-$(CONFIG_APNG_DECODER) += png.o pngdec.o pngdsp.o OBJS-$(CONFIG_APNG_ENCODER) += png.o pngenc.o OBJS-$(CONFIG_ARBC_DECODER) += arbc.o OBJS-$(CONFIG_ARGO_DECODER) += argo.o -OBJS-$(CONFIG_SSA_DECODER) += assdec.o ass.o -OBJS-$(CONFIG_SSA_ENCODER) += assenc.o ass.o -OBJS-$(CONFIG_ASS_DECODER) += assdec.o ass.o -OBJS-$(CONFIG_ASS_ENCODER) += assenc.o ass.o +OBJS-$(CONFIG_SSA_DECODER) += assdec.o +OBJS-$(CONFIG_SSA_ENCODER) += assenc.o +OBJS-$(CONFIG_ASS_DECODER) += assdec.o +OBJS-$(CONFIG_ASS_ENCODER) += assenc.o OBJS-$(CONFIG_ASV1_DECODER) += asvdec.o asv.o mpeg12data.o OBJS-$(CONFIG_ASV1_ENCODER) += asvenc.o asv.o mpeg12data.o OBJS-$(CONFIG_ASV2_DECODER) += asvdec.o asv.o mpeg12data.o @@ -253,7 +253,7 @@ OBJS-$(CONFIG_BRENDER_PIX_DECODER) += brenderpix.o OBJS-$(CONFIG_C93_DECODER) += c93.o OBJS-$(CONFIG_CAVS_DECODER) += cavs.o cavsdec.o cavsdsp.o \ cavsdata.o -OBJS-$(CONFIG_CCAPTION_DECODER) += ccaption_dec.o ass.o +OBJS-$(CONFIG_CCAPTION_DECODER) += ccaption_dec.o OBJS-$(CONFIG_CDGRAPHICS_DECODER) += cdgraphics.o OBJS-$(CONFIG_CDTOONS_DECODER) += cdtoons.o OBJS-$(CONFIG_CDXL_DECODER) += cdxl.o @@ -425,7 +425,7 @@ OBJS-$(CONFIG_INTERPLAY_ACM_DECODER) += interplayacm.o OBJS-$(CONFIG_INTERPLAY_DPCM_DECODER) += dpcm.o OBJS-$(CONFIG_INTERPLAY_VIDEO_DECODER) += interplayvideo.o OBJS-$(CONFIG_IPU_DECODER) += mpeg12dec.o mpeg12.o mpeg12data.o -OBJS-$(CONFIG_JACOSUB_DECODER) += jacosubdec.o ass.o +OBJS-$(CONFIG_JACOSUB_DECODER) += jacosubdec.o OBJS-$(CONFIG_JPEG2000_ENCODER) += j2kenc.o mqcenc.o mqc.o jpeg2000.o \ jpeg2000dwt.o OBJS-$(CONFIG_JPEG2000_DECODER) += jpeg2000dec.o jpeg2000.o jpeg2000dsp.o \ @@ -447,7 +447,7 @@ OBJS-$(CONFIG_MAGICYUV_ENCODER) += magicyuvenc.o OBJS-$(CONFIG_MDEC_DECODER) += mdec.o mpeg12.o mpeg12data.o OBJS-$(CONFIG_METASOUND_DECODER) += metasound.o metasound_data.o \ twinvq.o -OBJS-$(CONFIG_MICRODVD_DECODER) += microdvddec.o ass.o +OBJS-$(CONFIG_MICRODVD_DECODER) += microdvddec.o OBJS-$(CONFIG_MIMIC_DECODER) += mimic.o OBJS-$(CONFIG_MJPEG_DECODER) += mjpegdec.o mjpegdec_common.o OBJS-$(CONFIG_MJPEG_QSV_DECODER) += qsvdec.o @@ -462,8 +462,8 @@ OBJS-$(CONFIG_MLP_ENCODER) += mlpenc.o mlp.o OBJS-$(CONFIG_MMVIDEO_DECODER) += mmvideo.o OBJS-$(CONFIG_MOBICLIP_DECODER) += mobiclip.o OBJS-$(CONFIG_MOTIONPIXELS_DECODER) += motionpixels.o -OBJS-$(CONFIG_MOVTEXT_DECODER) += movtextdec.o ass.o -OBJS-$(CONFIG_MOVTEXT_ENCODER) += movtextenc.o ass_split.o +OBJS-$(CONFIG_MOVTEXT_DECODER) += movtextdec.o +OBJS-$(CONFIG_MOVTEXT_ENCODER) += movtextenc.o OBJS-$(CONFIG_MP1_DECODER) += mpegaudiodec_fixed.o OBJS-$(CONFIG_MP1FLOAT_DECODER) += mpegaudiodec_float.o OBJS-$(CONFIG_MP2_DECODER) += mpegaudiodec_fixed.o @@ -502,7 +502,7 @@ OBJS-$(CONFIG_MPEG4_MEDIACODEC_DECODER) += mediacodecdec.o OBJS-$(CONFIG_MPEG4_OMX_ENCODER) += omx.o OBJS-$(CONFIG_MPEG4_V4L2M2M_DECODER) += v4l2_m2m_dec.o OBJS-$(CONFIG_MPEG4_V4L2M2M_ENCODER) += v4l2_m2m_enc.o -OBJS-$(CONFIG_MPL2_DECODER) += mpl2dec.o ass.o +OBJS-$(CONFIG_MPL2_DECODER) += mpl2dec.o OBJS-$(CONFIG_MSA1_DECODER) += mss3.o OBJS-$(CONFIG_MSCC_DECODER) += mscc.o OBJS-$(CONFIG_MSMPEG4V1_DECODER) += msmpeg4dec.o msmpeg4.o msmpeg4data.o @@ -555,7 +555,7 @@ OBJS-$(CONFIG_PGX_DECODER) += pgxdec.o OBJS-$(CONFIG_PHOTOCD_DECODER) += photocd.o OBJS-$(CONFIG_PICTOR_DECODER) += pictordec.o cga_data.o OBJS-$(CONFIG_PIXLET_DECODER) += pixlet.o -OBJS-$(CONFIG_PJS_DECODER) += textdec.o ass.o +OBJS-$(CONFIG_PJS_DECODER) += textdec.o OBJS-$(CONFIG_PNG_DECODER) += png.o pngdec.o pngdsp.o OBJS-$(CONFIG_PNG_ENCODER) += png.o pngenc.o OBJS-$(CONFIG_PPM_DECODER) += pnmdec.o pnm.o @@ -587,7 +587,7 @@ OBJS-$(CONFIG_RALF_DECODER) += ralf.o OBJS-$(CONFIG_RASC_DECODER) += rasc.o OBJS-$(CONFIG_RAWVIDEO_DECODER) += rawdec.o OBJS-$(CONFIG_RAWVIDEO_ENCODER) += rawenc.o -OBJS-$(CONFIG_REALTEXT_DECODER) += realtextdec.o ass.o +OBJS-$(CONFIG_REALTEXT_DECODER) += realtextdec.o OBJS-$(CONFIG_RL2_DECODER) += rl2.o OBJS-$(CONFIG_ROQ_DECODER) += roqvideodec.o roqvideo.o OBJS-$(CONFIG_ROQ_ENCODER) += roqvideoenc.o roqvideo.o elbg.o @@ -602,7 +602,7 @@ OBJS-$(CONFIG_RV20_DECODER) += rv10.o OBJS-$(CONFIG_RV20_ENCODER) += rv20enc.o OBJS-$(CONFIG_RV30_DECODER) += rv30.o rv34.o rv30dsp.o OBJS-$(CONFIG_RV40_DECODER) += rv40.o rv34.o rv40dsp.o -OBJS-$(CONFIG_SAMI_DECODER) += samidec.o ass.o htmlsubtitles.o +OBJS-$(CONFIG_SAMI_DECODER) += samidec.o htmlsubtitles.o OBJS-$(CONFIG_S302M_DECODER) += s302m.o OBJS-$(CONFIG_S302M_ENCODER) += s302menc.o OBJS-$(CONFIG_SANM_DECODER) += sanm.o @@ -637,13 +637,13 @@ OBJS-$(CONFIG_SPEEDHQ_ENCODER) += speedhq.o mpeg12data.o mpeg12enc.o spe OBJS-$(CONFIG_SPEEX_DECODER) += speexdec.o OBJS-$(CONFIG_SP5X_DECODER) += sp5xdec.o OBJS-$(CONFIG_SRGC_DECODER) += mscc.o -OBJS-$(CONFIG_SRT_DECODER) += srtdec.o ass.o htmlsubtitles.o -OBJS-$(CONFIG_SRT_ENCODER) += srtenc.o ass_split.o -OBJS-$(CONFIG_STL_DECODER) += textdec.o ass.o -OBJS-$(CONFIG_SUBRIP_DECODER) += srtdec.o ass.o htmlsubtitles.o -OBJS-$(CONFIG_SUBRIP_ENCODER) += srtenc.o ass_split.o -OBJS-$(CONFIG_SUBVIEWER1_DECODER) += textdec.o ass.o -OBJS-$(CONFIG_SUBVIEWER_DECODER) += subviewerdec.o ass.o +OBJS-$(CONFIG_SRT_DECODER) += srtdec.o htmlsubtitles.o +OBJS-$(CONFIG_SRT_ENCODER) += srtenc.o +OBJS-$(CONFIG_STL_DECODER) += textdec.o +OBJS-$(CONFIG_SUBRIP_DECODER) += srtdec.o htmlsubtitles.o +OBJS-$(CONFIG_SUBRIP_ENCODER) += srtenc.o +OBJS-$(CONFIG_SUBVIEWER1_DECODER) += textdec.o +OBJS-$(CONFIG_SUBVIEWER_DECODER) += subviewerdec.o OBJS-$(CONFIG_SUNRAST_DECODER) += sunrast.o OBJS-$(CONFIG_SUNRAST_ENCODER) += sunrastenc.o OBJS-$(CONFIG_LIBRSVG_DECODER) += librsvgdec.o @@ -653,8 +653,8 @@ OBJS-$(CONFIG_SVQ1_DECODER) += svq1dec.o svq1.o h263data.o OBJS-$(CONFIG_SVQ1_ENCODER) += svq1enc.o svq1.o h263data.o \ h263.o ituh263enc.o OBJS-$(CONFIG_SVQ3_DECODER) += svq3.o mpegutils.o h264data.o -OBJS-$(CONFIG_TEXT_DECODER) += textdec.o ass.o -OBJS-$(CONFIG_TEXT_ENCODER) += srtenc.o ass_split.o +OBJS-$(CONFIG_TEXT_DECODER) += textdec.o +OBJS-$(CONFIG_TEXT_ENCODER) += srtenc.o OBJS-$(CONFIG_TAK_DECODER) += takdec.o tak.o takdsp.o OBJS-$(CONFIG_TARGA_DECODER) += targa.o OBJS-$(CONFIG_TARGA_ENCODER) += targaenc.o rle.o @@ -674,7 +674,7 @@ OBJS-$(CONFIG_TSCC_DECODER) += tscc.o msrledec.o OBJS-$(CONFIG_TSCC2_DECODER) += tscc2.o OBJS-$(CONFIG_TTA_DECODER) += tta.o ttadata.o ttadsp.o OBJS-$(CONFIG_TTA_ENCODER) += ttaenc.o ttaencdsp.o ttadata.o -OBJS-$(CONFIG_TTML_ENCODER) += ttmlenc.o ass_split.o +OBJS-$(CONFIG_TTML_ENCODER) += ttmlenc.o OBJS-$(CONFIG_TWINVQ_DECODER) += twinvqdec.o twinvq.o metasound_data.o OBJS-$(CONFIG_TXD_DECODER) += txd.o OBJS-$(CONFIG_ULTI_DECODER) += ulti.o @@ -729,15 +729,15 @@ OBJS-$(CONFIG_VP9_MEDIACODEC_DECODER) += mediacodecdec.o OBJS-$(CONFIG_VP9_RKMPP_DECODER) += rkmppdec.o OBJS-$(CONFIG_VP9_VAAPI_ENCODER) += vaapi_encode_vp9.o OBJS-$(CONFIG_VP9_QSV_ENCODER) += qsvenc_vp9.o -OBJS-$(CONFIG_VPLAYER_DECODER) += textdec.o ass.o +OBJS-$(CONFIG_VPLAYER_DECODER) += textdec.o OBJS-$(CONFIG_VP9_V4L2M2M_DECODER) += v4l2_m2m_dec.o OBJS-$(CONFIG_VQA_DECODER) += vqavideo.o OBJS-$(CONFIG_WAVPACK_DECODER) += wavpack.o wavpackdata.o dsd.o OBJS-$(CONFIG_WAVPACK_ENCODER) += wavpackdata.o wavpackenc.o OBJS-$(CONFIG_WCMV_DECODER) += wcmv.o OBJS-$(CONFIG_WEBP_DECODER) += webp.o -OBJS-$(CONFIG_WEBVTT_DECODER) += webvttdec.o ass.o -OBJS-$(CONFIG_WEBVTT_ENCODER) += webvttenc.o ass_split.o +OBJS-$(CONFIG_WEBVTT_DECODER) += webvttdec.o +OBJS-$(CONFIG_WEBVTT_ENCODER) += webvttenc.o OBJS-$(CONFIG_WMALOSSLESS_DECODER) += wmalosslessdec.o wma_common.o OBJS-$(CONFIG_WMAPRO_DECODER) += wmaprodec.o wma.o wma_common.o OBJS-$(CONFIG_WMAV1_DECODER) += wmadec.o wma.o wma_common.o aactab.o @@ -1023,7 +1023,7 @@ OBJS-$(CONFIG_PCM_ALAW_AT_ENCODER) += audiotoolboxenc.o OBJS-$(CONFIG_PCM_MULAW_AT_ENCODER) += audiotoolboxenc.o OBJS-$(CONFIG_LIBAOM_AV1_DECODER) += libaomdec.o OBJS-$(CONFIG_LIBAOM_AV1_ENCODER) += libaomenc.o -OBJS-$(CONFIG_LIBARIBB24_DECODER) += libaribb24.o ass.o +OBJS-$(CONFIG_LIBARIBB24_DECODER) += libaribb24.o OBJS-$(CONFIG_LIBCELT_DECODER) += libcelt_dec.o OBJS-$(CONFIG_LIBCODEC2_DECODER) += libcodec2.o OBJS-$(CONFIG_LIBCODEC2_ENCODER) += libcodec2.o @@ -1074,7 +1074,7 @@ OBJS-$(CONFIG_LIBX265_ENCODER) += libx265.o OBJS-$(CONFIG_LIBXAVS_ENCODER) += libxavs.o OBJS-$(CONFIG_LIBXAVS2_ENCODER) += libxavs2.o OBJS-$(CONFIG_LIBXVID_ENCODER) += libxvid.o -OBJS-$(CONFIG_LIBZVBI_TELETEXT_DECODER) += libzvbi-teletextdec.o ass.o +OBJS-$(CONFIG_LIBZVBI_TELETEXT_DECODER) += libzvbi-teletextdec.o # parsers OBJS-$(CONFIG_AAC_LATM_PARSER) += latm_parser.o diff --git a/libavcodec/ass.h b/libavcodec/ass.h index 2c260e4e78..9c9cb01c8a 100644 --- a/libavcodec/ass.h +++ b/libavcodec/ass.h @@ -1,6 +1,5 @@ /* - * SSA/ASS common functions - * Copyright (c) 2010 Aurelien Jacobs + * Copyright (c) 2021 The FFmpeg Project * * This file is part of FFmpeg. * @@ -23,117 +22,73 @@ #define AVCODEC_ASS_H #include "avcodec.h" -#include "libavutil/bprint.h" - -#define ASS_DEFAULT_PLAYRESX 384 -#define ASS_DEFAULT_PLAYRESY 288 - -/** - * @name Default values for ASS style - * @{ - */ -#define ASS_DEFAULT_FONT "Arial" -#define ASS_DEFAULT_FONT_SIZE 16 -#define ASS_DEFAULT_COLOR 0xffffff -#define ASS_DEFAULT_BACK_COLOR 0 -#define ASS_DEFAULT_BOLD 0 -#define ASS_DEFAULT_ITALIC 0 -#define ASS_DEFAULT_UNDERLINE 0 -#define ASS_DEFAULT_ALIGNMENT 2 -#define ASS_DEFAULT_BORDERSTYLE 1 -/** @} */ +#include "libavutil/ass_internal.h" typedef struct FFASSDecoderContext { int readorder; } FFASSDecoderContext; -/** - * Generate a suitable AVCodecContext.subtitle_header for SUBTITLE_ASS. - * Can specify all fields explicitly - * - * @param avctx pointer to the AVCodecContext - * @param play_res_x subtitle frame width - * @param play_res_y subtitle frame height - * @param font name of the default font face to use - * @param font_size default font size to use - * @param primary_color default text color to use (ABGR) - * @param secondary_color default secondary text color to use (ABGR) - * @param outline_color default outline color to use (ABGR) - * @param back_color default background color to use (ABGR) - * @param bold 1 for bold text, 0 for normal text - * @param italic 1 for italic text, 0 for normal text - * @param underline 1 for underline text, 0 for normal text - * @param border_style 1 for outline, 3 for opaque box - * @param alignment position of the text (left, center, top...), defined after - * the layout of the numpad (1-3 sub, 4-6 mid, 7-9 top) - * @return >= 0 on success otherwise an error code <0 - */ -int ff_ass_subtitle_header_full(AVCodecContext *avctx, +static inline int ff_ass_subtitle_header_full(AVCodecContext *avctx, int play_res_x, int play_res_y, const char *font, int font_size, int primary_color, int secondary_color, int outline_color, int back_color, int bold, int italic, int underline, - int border_style, int alignment); -/** - * Generate a suitable AVCodecContext.subtitle_header for SUBTITLE_ASS. - * - * @param avctx pointer to the AVCodecContext - * @param font name of the default font face to use - * @param font_size default font size to use - * @param color default text color to use (ABGR) - * @param back_color default background color to use (ABGR) - * @param bold 1 for bold text, 0 for normal text - * @param italic 1 for italic text, 0 for normal text - * @param underline 1 for underline text, 0 for normal text - * @param alignment position of the text (left, center, top...), defined after - * the layout of the numpad (1-3 sub, 4-6 mid, 7-9 top) - * @return >= 0 on success otherwise an error code <0 - */ -int ff_ass_subtitle_header(AVCodecContext *avctx, - const char *font, int font_size, - int color, int back_color, - int bold, int italic, int underline, - int border_style, int alignment); + int border_style, int alignment) +{ + avctx->subtitle_header = (uint8_t *)avpriv_ass_get_subtitle_header_full( + play_res_x, play_res_y, font, font_size, + primary_color, secondary_color, outline_color, + back_color, bold,italic,underline,border_style,alignment, + !(avctx->flags & AV_CODEC_FLAG_BITEXACT)); -/** - * Generate a suitable AVCodecContext.subtitle_header for SUBTITLE_ASS - * with default style. - * - * @param avctx pointer to the AVCodecContext - * @return >= 0 on success otherwise an error code <0 - */ -int ff_ass_subtitle_header_default(AVCodecContext *avctx); + if (!avctx->subtitle_header) + return AVERROR(ENOMEM); + avctx->subtitle_header_size = strlen((char *)avctx->subtitle_header); + return 0; +} -/** - * Craft an ASS dialog string. - */ -char *ff_ass_get_dialog(int readorder, int layer, const char *style, - const char *speaker, const char *text); +static inline int ff_ass_subtitle_header_default(AVCodecContext *avctx) +{ + avctx->subtitle_header = (uint8_t *)avpriv_ass_get_subtitle_header_default(!(avctx->flags & AV_CODEC_FLAG_BITEXACT)); + + if (!avctx->subtitle_header) + return AVERROR(ENOMEM); + avctx->subtitle_header_size = strlen((char *)avctx->subtitle_header); + return 0; +} + +static inline void ff_ass_decoder_flush(AVCodecContext *avctx) +{ + FFASSDecoderContext *s = avctx->priv_data; + if (!(avctx->flags2 & AV_CODEC_FLAG2_RO_FLUSH_NOOP)) + s->readorder = 0; +} /** * Add an ASS dialog to a subtitle. */ -int ff_ass_add_rect(AVSubtitle *sub, const char *dialog, +static inline int avpriv_ass_add_rect(AVSubtitle *sub, const char *dialog, int readorder, int layer, const char *style, - const char *speaker); + const char *speaker) +{ + char *ass_str; + AVSubtitleRect **rects; -/** - * Helper to flush a text subtitles decoder making use of the - * FFASSDecoderContext. - */ -void ff_ass_decoder_flush(AVCodecContext *avctx); + rects = av_realloc_array(sub->rects, sub->num_rects+1, sizeof(*sub->rects)); + if (!rects) + return AVERROR(ENOMEM); + sub->rects = rects; + rects[sub->num_rects] = av_mallocz(sizeof(*rects[0])); + if (!rects[sub->num_rects]) + return AVERROR(ENOMEM); + rects[sub->num_rects]->type = SUBTITLE_ASS; + ass_str = avpriv_ass_get_dialog(readorder, layer, style, speaker, dialog); + if (!ass_str) + return AVERROR(ENOMEM); + rects[sub->num_rects]->ass = ass_str; + sub->num_rects++; + return 0; +} -/** - * Escape a text subtitle using ASS syntax into an AVBPrint buffer. - * Newline characters will be escaped to \N. - * - * @param buf pointer to an initialized AVBPrint buffer - * @param p source text - * @param size size of the source text - * @param linebreaks additional newline chars, which will be escaped to \N - * @param keep_ass_markup braces and backslash will not be escaped if set - */ -void ff_ass_bprint_text_event(AVBPrint *buf, const char *p, int size, - const char *linebreaks, int keep_ass_markup); #endif /* AVCODEC_ASS_H */ diff --git a/libavcodec/assdec.c b/libavcodec/assdec.c index 319279490c..7802a44e71 100644 --- a/libavcodec/assdec.c +++ b/libavcodec/assdec.c @@ -22,7 +22,7 @@ #include #include "avcodec.h" -#include "ass.h" +#include "libavutil/ass_internal.h" #include "internal.h" #include "libavutil/internal.h" #include "libavutil/mem.h" diff --git a/libavcodec/assenc.c b/libavcodec/assenc.c index a6d107ded2..b0e475834b 100644 --- a/libavcodec/assenc.c +++ b/libavcodec/assenc.c @@ -22,7 +22,7 @@ #include #include "avcodec.h" -#include "ass.h" +#include "libavutil/ass_internal.h" #include "internal.h" #include "libavutil/avstring.h" #include "libavutil/internal.h" diff --git a/libavcodec/ccaption_dec.c b/libavcodec/ccaption_dec.c index 27c61527f6..27eef75657 100644 --- a/libavcodec/ccaption_dec.c +++ b/libavcodec/ccaption_dec.c @@ -272,15 +272,12 @@ static av_cold int init_decoder(AVCodecContext *avctx) ctx->bg_color = CCCOL_BLACK; ctx->rollup = 2; ctx->cursor_row = 10; - ret = ff_ass_subtitle_header(avctx, "Monospace", + ret = ff_ass_subtitle_header_full(avctx, ASS_DEFAULT_PLAYRESX, ASS_DEFAULT_PLAYRESY, "Monospace", ASS_DEFAULT_FONT_SIZE, - ASS_DEFAULT_COLOR, - ASS_DEFAULT_BACK_COLOR, - ASS_DEFAULT_BOLD, - ASS_DEFAULT_ITALIC, - ASS_DEFAULT_UNDERLINE, - 3, - ASS_DEFAULT_ALIGNMENT); + ASS_DEFAULT_COLOR, ASS_DEFAULT_COLOR, + ASS_DEFAULT_BACK_COLOR, ASS_DEFAULT_BACK_COLOR, + ASS_DEFAULT_BOLD, ASS_DEFAULT_ITALIC, ASS_DEFAULT_UNDERLINE, + 3, ASS_DEFAULT_ALIGNMENT); if (ret < 0) { return ret; } @@ -886,7 +883,7 @@ static int decode(AVCodecContext *avctx, void *data, int *got_sub, AVPacket *avp AV_TIME_BASE_Q, ms_tb); else sub->end_display_time = -1; - ret = ff_ass_add_rect(sub, ctx->buffer[bidx].str, ctx->readorder++, 0, NULL, NULL); + ret = avpriv_ass_add_rect(sub, ctx->buffer[bidx].str, ctx->readorder++, 0, NULL, NULL); if (ret < 0) return ret; ctx->last_real_time = sub->pts; @@ -896,7 +893,7 @@ static int decode(AVCodecContext *avctx, void *data, int *got_sub, AVPacket *avp if (!bptr && !ctx->real_time && ctx->buffer[!ctx->buffer_index].str[0]) { bidx = !ctx->buffer_index; - ret = ff_ass_add_rect(sub, ctx->buffer[bidx].str, ctx->readorder++, 0, NULL, NULL); + ret = avpriv_ass_add_rect(sub, ctx->buffer[bidx].str, ctx->readorder++, 0, NULL, NULL); if (ret < 0) return ret; sub->pts = ctx->buffer_time[1]; @@ -914,7 +911,7 @@ static int decode(AVCodecContext *avctx, void *data, int *got_sub, AVPacket *avp capture_screen(ctx); ctx->buffer_changed = 0; - ret = ff_ass_add_rect(sub, ctx->buffer[bidx].str, ctx->readorder++, 0, NULL, NULL); + ret = avpriv_ass_add_rect(sub, ctx->buffer[bidx].str, ctx->readorder++, 0, NULL, NULL); if (ret < 0) return ret; sub->end_display_time = -1; diff --git a/libavcodec/jacosubdec.c b/libavcodec/jacosubdec.c index 698895a86b..6a53ec3e34 100644 --- a/libavcodec/jacosubdec.c +++ b/libavcodec/jacosubdec.c @@ -183,7 +183,7 @@ static int jacosub_decode_frame(AVCodecContext *avctx, av_bprint_init(&buffer, JSS_MAX_LINESIZE, JSS_MAX_LINESIZE); jacosub_to_ass(avctx, &buffer, ptr); - ret = ff_ass_add_rect(sub, buffer.str, s->readorder++, 0, NULL, NULL); + ret = avpriv_ass_add_rect(sub, buffer.str, s->readorder++, 0, NULL, NULL); av_bprint_finalize(&buffer, NULL); if (ret < 0) return ret; diff --git a/libavcodec/libaribb24.c b/libavcodec/libaribb24.c index 0766c0079d..3fb7e5f16e 100644 --- a/libavcodec/libaribb24.c +++ b/libavcodec/libaribb24.c @@ -273,7 +273,7 @@ next_region: av_log(avctx, AV_LOG_DEBUG, "Styled ASS line: %s\n", buf.str); - ret = ff_ass_add_rect(sub, buf.str, b24->read_order++, + ret = avpriv_ass_add_rect(sub, buf.str, b24->read_order++, 0, NULL, NULL); } diff --git a/libavcodec/libzvbi-teletextdec.c b/libavcodec/libzvbi-teletextdec.c index 1073d6a0bd..bd9edc34d7 100644 --- a/libavcodec/libzvbi-teletextdec.c +++ b/libavcodec/libzvbi-teletextdec.c @@ -152,12 +152,12 @@ static char *create_ass_text(TeletextContext *ctx, const char *text) AVBPrint buf; av_bprint_init(&buf, 0, AV_BPRINT_SIZE_UNLIMITED); - ff_ass_bprint_text_event(&buf, text, strlen(text), "", 0); + avpriv_ass_bprint_text_event(&buf, text, strlen(text), "", 0); if (!av_bprint_is_complete(&buf)) { av_bprint_finalize(&buf, NULL); return NULL; } - dialog = ff_ass_get_dialog(ctx->readorder++, 0, NULL, NULL, buf.str); + dialog = avpriv_ass_get_dialog(ctx->readorder++, 0, NULL, NULL, buf.str); av_bprint_finalize(&buf, NULL); return dialog; } @@ -224,7 +224,7 @@ static int gen_sub_text(TeletextContext *ctx, AVSubtitleRect *sub_rect, vbi_page } av_log(ctx, AV_LOG_DEBUG, "subtext:%s:txetbus\n", sub_rect->ass); } else { - sub_rect->type = SUBTITLE_NONE; + sub_rect->type = AV_SUBTITLE_FMT_NONE; } av_bprint_finalize(&buf, NULL); return 0; @@ -394,7 +394,7 @@ static int gen_sub_ass(TeletextContext *ctx, AVSubtitleRect *sub_rect, vbi_page if (buf.len) { sub_rect->type = SUBTITLE_ASS; - sub_rect->ass = ff_ass_get_dialog(ctx->readorder++, 0, is_subtitle_page ? "Subtitle" : "Teletext", NULL, buf.str); + sub_rect->ass = avpriv_ass_get_dialog(ctx->readorder++, 0, is_subtitle_page ? "Subtitle" : "Teletext", NULL, buf.str); if (!sub_rect->ass) { av_bprint_finalize(&buf, NULL); @@ -402,7 +402,7 @@ static int gen_sub_ass(TeletextContext *ctx, AVSubtitleRect *sub_rect, vbi_page } av_log(ctx, AV_LOG_DEBUG, "subtext:%s:txetbus\n", sub_rect->ass); } else { - sub_rect->type = SUBTITLE_NONE; + sub_rect->type = AV_SUBTITLE_FMT_NONE; } av_bprint_finalize(&buf, NULL); return 0; @@ -462,7 +462,7 @@ static int gen_sub_bitmap(TeletextContext *ctx, AVSubtitleRect *sub_rect, vbi_pa if (vc >= vcend) { av_log(ctx, AV_LOG_DEBUG, "dropping empty page %3x\n", page->pgno); - sub_rect->type = SUBTITLE_NONE; + sub_rect->type = AV_SUBTITLE_FMT_NONE; return 0; } @@ -695,7 +695,7 @@ static int teletext_decode_frame(AVCodecContext *avctx, void *data, int *got_sub sub->num_rects = 0; sub->pts = ctx->pages->pts; - if (ctx->pages->sub_rect->type != SUBTITLE_NONE) { + if (ctx->pages->sub_rect->type != AV_SUBTITLE_FMT_NONE) { sub->rects = av_malloc(sizeof(*sub->rects)); if (sub->rects) { sub->num_rects = 1; diff --git a/libavcodec/microdvddec.c b/libavcodec/microdvddec.c index c45fe043bf..fc09db8997 100644 --- a/libavcodec/microdvddec.c +++ b/libavcodec/microdvddec.c @@ -310,7 +310,7 @@ static int microdvd_decode_frame(AVCodecContext *avctx, } } if (new_line.len) { - int ret = ff_ass_add_rect(sub, new_line.str, s->readorder++, 0, NULL, NULL); + int ret = avpriv_ass_add_rect(sub, new_line.str, s->readorder++, 0, NULL, NULL); av_bprint_finalize(&new_line, NULL); if (ret < 0) return ret; @@ -363,8 +363,9 @@ static int microdvd_init(AVCodecContext *avctx) } } } - return ff_ass_subtitle_header(avctx, font_buf.str, font_size, color, - ASS_DEFAULT_BACK_COLOR, bold, italic, + return ff_ass_subtitle_header_full(avctx, ASS_DEFAULT_PLAYRESX, ASS_DEFAULT_PLAYRESY, + font_buf.str, font_size, color, color, + ASS_DEFAULT_BACK_COLOR, ASS_DEFAULT_BACK_COLOR, bold, italic, underline, ASS_DEFAULT_BORDERSTYLE, alignment); } diff --git a/libavcodec/movtextdec.c b/libavcodec/movtextdec.c index 825632ca9b..9b17bac9ea 100644 --- a/libavcodec/movtextdec.c +++ b/libavcodec/movtextdec.c @@ -22,7 +22,6 @@ #include "avcodec.h" #include "ass.h" #include "libavutil/opt.h" -#include "libavutil/avstring.h" #include "libavutil/common.h" #include "libavutil/bprint.h" #include "libavutil/intreadwrite.h" @@ -554,7 +553,7 @@ static int mov_text_decode_frame(AVCodecContext *avctx, } else text_to_ass(&buf, ptr, end, avctx); - ret = ff_ass_add_rect(sub, buf.str, m->readorder++, 0, NULL, NULL); + ret = avpriv_ass_add_rect(sub, buf.str, m->readorder++, 0, NULL, NULL); av_bprint_finalize(&buf, NULL); if (ret < 0) return ret; diff --git a/libavcodec/movtextenc.c b/libavcodec/movtextenc.c index 221cd76fea..d506ed5c37 100644 --- a/libavcodec/movtextenc.c +++ b/libavcodec/movtextenc.c @@ -26,8 +26,8 @@ #include "libavutil/intreadwrite.h" #include "libavutil/mem.h" #include "libavutil/common.h" -#include "ass_split.h" -#include "ass.h" +#include "libavutil/ass_split_internal.h" +#include "libavutil/ass_internal.h" #include "bytestream.h" #include "internal.h" @@ -167,7 +167,7 @@ static int mov_text_encode_close(AVCodecContext *avctx) { MovTextContext *s = avctx->priv_data; - ff_ass_split_free(s->ass_ctx); + avpriv_ass_split_free(s->ass_ctx); av_freep(&s->style_attributes); av_freep(&s->fonts); av_bprint_finalize(&s->buffer, NULL); @@ -222,7 +222,7 @@ static int encode_sample_description(AVCodecContext *avctx) else s->font_scale_factor = 1; - style = ff_ass_style_get(s->ass_ctx, "Default"); + style = avpriv_ass_style_get(s->ass_ctx, "Default"); if (!style && ass->styles_count) { style = &ass->styles[0]; } @@ -329,7 +329,7 @@ static av_cold int mov_text_encode_init(AVCodecContext *avctx) av_bprint_init(&s->buffer, 0, AV_BPRINT_SIZE_UNLIMITED); - s->ass_ctx = ff_ass_split(avctx->subtitle_header); + s->ass_ctx = avpriv_ass_split(avctx->subtitle_header); if (!s->ass_ctx) return AVERROR_INVALIDDATA; ret = encode_sample_description(avctx); @@ -566,7 +566,7 @@ static void mov_text_ass_style_set(MovTextContext *s, ASSStyle *style) static void mov_text_dialog(MovTextContext *s, ASSDialog *dialog) { - ASSStyle *style = ff_ass_style_get(s->ass_ctx, dialog->style); + ASSStyle *style = avpriv_ass_style_get(s->ass_ctx, dialog->style); s->ass_dialog_style = style; mov_text_ass_style_set(s, style); @@ -580,7 +580,7 @@ static void mov_text_cancel_overrides_cb(void *priv, const char *style_name) if (!style_name || !*style_name) style = s->ass_dialog_style; else - style= ff_ass_style_get(s->ass_ctx, style_name); + style= avpriv_ass_style_get(s->ass_ctx, style_name); mov_text_ass_style_set(s, style); } @@ -652,12 +652,12 @@ static int mov_text_encode_frame(AVCodecContext *avctx, unsigned char *buf, return AVERROR(EINVAL); } - dialog = ff_ass_split_dialog(s->ass_ctx, ass); + dialog = avpriv_ass_split_dialog(s->ass_ctx, ass); if (!dialog) return AVERROR(ENOMEM); mov_text_dialog(s, dialog); - ff_ass_split_override_codes(&mov_text_callbacks, s, dialog->text); - ff_ass_free_dialog(&dialog); + avpriv_ass_split_override_codes(&mov_text_callbacks, s, dialog->text); + avpriv_ass_free_dialog(&dialog); } if (s->buffer.len > UINT16_MAX) diff --git a/libavcodec/mpl2dec.c b/libavcodec/mpl2dec.c index 61e47050ec..fe806b4927 100644 --- a/libavcodec/mpl2dec.c +++ b/libavcodec/mpl2dec.c @@ -74,7 +74,7 @@ static int mpl2_decode_frame(AVCodecContext *avctx, void *data, av_bprint_init(&buf, 0, AV_BPRINT_SIZE_UNLIMITED); if (ptr && avpkt->size > 0 && *ptr && !mpl2_event_to_ass(&buf, ptr)) - ret = ff_ass_add_rect(sub, buf.str, s->readorder++, 0, NULL, NULL); + ret = avpriv_ass_add_rect(sub, buf.str, s->readorder++, 0, NULL, NULL); av_bprint_finalize(&buf, NULL); if (ret < 0) return ret; diff --git a/libavcodec/realtextdec.c b/libavcodec/realtextdec.c index 11b586d493..acd548b6b5 100644 --- a/libavcodec/realtextdec.c +++ b/libavcodec/realtextdec.c @@ -67,7 +67,7 @@ static int realtext_decode_frame(AVCodecContext *avctx, av_bprint_init(&buf, 0, 4096); if (ptr && avpkt->size > 0 && !rt_event_to_ass(&buf, ptr)) - ret = ff_ass_add_rect(sub, buf.str, s->readorder++, 0, NULL, NULL); + ret = avpriv_ass_add_rect(sub, buf.str, s->readorder++, 0, NULL, NULL); av_bprint_finalize(&buf, NULL); if (ret < 0) return ret; diff --git a/libavcodec/samidec.c b/libavcodec/samidec.c index 32d07447b4..1249b629d6 100644 --- a/libavcodec/samidec.c +++ b/libavcodec/samidec.c @@ -144,7 +144,7 @@ static int sami_decode_frame(AVCodecContext *avctx, if (ret < 0) return ret; // TODO: pass escaped sami->encoded_source.str as source - ret = ff_ass_add_rect(sub, sami->full.str, sami->readorder++, 0, NULL, NULL); + ret = avpriv_ass_add_rect(sub, sami->full.str, sami->readorder++, 0, NULL, NULL); if (ret < 0) return ret; } diff --git a/libavcodec/srtdec.c b/libavcodec/srtdec.c index 4f16226b83..847352eb37 100644 --- a/libavcodec/srtdec.c +++ b/libavcodec/srtdec.c @@ -78,7 +78,7 @@ static int srt_decode_frame(AVCodecContext *avctx, ret = srt_to_ass(avctx, &buffer, avpkt->data, x1, y1, x2, y2); if (ret >= 0) - ret = ff_ass_add_rect(sub, buffer.str, s->readorder++, 0, NULL, NULL); + ret = avpriv_ass_add_rect(sub, buffer.str, s->readorder++, 0, NULL, NULL); av_bprint_finalize(&buffer, NULL); if (ret < 0) return ret; diff --git a/libavcodec/srtenc.c b/libavcodec/srtenc.c index 2e3ac55770..a7c5fccefe 100644 --- a/libavcodec/srtenc.c +++ b/libavcodec/srtenc.c @@ -23,8 +23,8 @@ #include "avcodec.h" #include "libavutil/avstring.h" #include "libavutil/bprint.h" -#include "ass_split.h" -#include "ass.h" +#include "libavutil/ass_split_internal.h" +#include "libavutil/ass_internal.h" #include "internal.h" @@ -94,7 +94,7 @@ static void srt_stack_push_pop(SRTContext *s, const char c, int close) static void srt_style_apply(SRTContext *s, const char *style) { - ASSStyle *st = ff_ass_style_get(s->ass_ctx, style); + ASSStyle *st = avpriv_ass_style_get(s->ass_ctx, style); if (st) { int c = st->primary_color & 0xFFFFFF; if (st->font_name && strcmp(st->font_name, ASS_DEFAULT_FONT) || @@ -135,7 +135,7 @@ static av_cold int srt_encode_init(AVCodecContext *avctx) { SRTContext *s = avctx->priv_data; s->avctx = avctx; - s->ass_ctx = ff_ass_split(avctx->subtitle_header); + s->ass_ctx = avpriv_ass_split(avctx->subtitle_header); av_bprint_init(&s->buffer, 0, AV_BPRINT_SIZE_UNLIMITED); return s->ass_ctx ? 0 : AVERROR_INVALIDDATA; } @@ -245,14 +245,14 @@ static int encode_frame(AVCodecContext *avctx, return AVERROR(EINVAL); } - dialog = ff_ass_split_dialog(s->ass_ctx, ass); + dialog = avpriv_ass_split_dialog(s->ass_ctx, ass); if (!dialog) return AVERROR(ENOMEM); s->alignment_applied = 0; if (avctx->codec_id == AV_CODEC_ID_SUBRIP) srt_style_apply(s, dialog->style); - ff_ass_split_override_codes(cb, s, dialog->text); - ff_ass_free_dialog(&dialog); + avpriv_ass_split_override_codes(cb, s, dialog->text); + avpriv_ass_free_dialog(&dialog); } if (!av_bprint_is_complete(&s->buffer)) @@ -284,7 +284,7 @@ static int text_encode_frame(AVCodecContext *avctx, static int srt_encode_close(AVCodecContext *avctx) { SRTContext *s = avctx->priv_data; - ff_ass_split_free(s->ass_ctx); + avpriv_ass_split_free(s->ass_ctx); av_bprint_finalize(&s->buffer, NULL); return 0; } diff --git a/libavcodec/subviewerdec.c b/libavcodec/subviewerdec.c index 5c650d0cde..04e9ae7c09 100644 --- a/libavcodec/subviewerdec.c +++ b/libavcodec/subviewerdec.c @@ -58,7 +58,7 @@ static int subviewer_decode_frame(AVCodecContext *avctx, av_bprint_init(&buf, 0, AV_BPRINT_SIZE_UNLIMITED); if (ptr && avpkt->size > 0 && !subviewer_event_to_ass(&buf, ptr)) - ret = ff_ass_add_rect(sub, buf.str, s->readorder++, 0, NULL, NULL); + ret = avpriv_ass_add_rect(sub, buf.str, s->readorder++, 0, NULL, NULL); av_bprint_finalize(&buf, NULL); if (ret < 0) return ret; diff --git a/libavcodec/textdec.c b/libavcodec/textdec.c index 308553660a..7556deefe8 100644 --- a/libavcodec/textdec.c +++ b/libavcodec/textdec.c @@ -54,8 +54,8 @@ static int text_decode_frame(AVCodecContext *avctx, void *data, av_bprint_init(&buf, 0, AV_BPRINT_SIZE_UNLIMITED); if (ptr && avpkt->size > 0 && *ptr) { - ff_ass_bprint_text_event(&buf, ptr, avpkt->size, text->linebreaks, text->keep_ass_markup); - ret = ff_ass_add_rect(sub, buf.str, text->readorder++, 0, NULL, NULL); + avpriv_ass_bprint_text_event(&buf, ptr, avpkt->size, text->linebreaks, text->keep_ass_markup); + ret = avpriv_ass_add_rect(sub, buf.str, text->readorder++, 0, NULL, NULL); } av_bprint_finalize(&buf, NULL); if (ret < 0) diff --git a/libavcodec/ttmlenc.c b/libavcodec/ttmlenc.c index ad2eddfdd5..083f2dd67a 100644 --- a/libavcodec/ttmlenc.c +++ b/libavcodec/ttmlenc.c @@ -32,8 +32,7 @@ #include "libavutil/avstring.h" #include "libavutil/bprint.h" #include "libavutil/internal.h" -#include "ass_split.h" -#include "ass.h" +#include "libavutil/ass_split_internal.h" #include "ttmlenc.h" typedef struct { @@ -95,7 +94,7 @@ static int ttml_encode_frame(AVCodecContext *avctx, uint8_t *buf, return AVERROR(EINVAL); } - dialog = ff_ass_split_dialog(s->ass_ctx, ass); + dialog = avpriv_ass_split_dialog(s->ass_ctx, ass); if (!dialog) return AVERROR(ENOMEM); @@ -107,7 +106,7 @@ static int ttml_encode_frame(AVCodecContext *avctx, uint8_t *buf, av_bprintf(&s->buffer, "\">"); } - ret = ff_ass_split_override_codes(&ttml_callbacks, s, dialog->text); + ret = avpriv_ass_split_override_codes(&ttml_callbacks, s, dialog->text); if (ret < 0) { int log_level = (ret != AVERROR_INVALIDDATA || avctx->err_recognition & AV_EF_EXPLODE) ? @@ -118,7 +117,7 @@ static int ttml_encode_frame(AVCodecContext *avctx, uint8_t *buf, av_err2str(ret)); if (log_level == AV_LOG_ERROR) { - ff_ass_free_dialog(&dialog); + avpriv_ass_free_dialog(&dialog); return ret; } } @@ -126,7 +125,7 @@ static int ttml_encode_frame(AVCodecContext *avctx, uint8_t *buf, if (dialog->style) av_bprintf(&s->buffer, ""); - ff_ass_free_dialog(&dialog); + avpriv_ass_free_dialog(&dialog); } if (!av_bprint_is_complete(&s->buffer)) @@ -148,7 +147,7 @@ static av_cold int ttml_encode_close(AVCodecContext *avctx) { TTMLContext *s = avctx->priv_data; - ff_ass_split_free(s->ass_ctx); + avpriv_ass_split_free(s->ass_ctx); av_bprint_finalize(&s->buffer, NULL); @@ -372,7 +371,7 @@ static av_cold int ttml_encode_init(AVCodecContext *avctx) av_bprint_init(&s->buffer, 0, AV_BPRINT_SIZE_UNLIMITED); - if (!(s->ass_ctx = ff_ass_split(avctx->subtitle_header))) { + if (!(s->ass_ctx = avpriv_ass_split(avctx->subtitle_header))) { return AVERROR_INVALIDDATA; } diff --git a/libavcodec/webvttdec.c b/libavcodec/webvttdec.c index 0093f328fa..d304edc705 100644 --- a/libavcodec/webvttdec.c +++ b/libavcodec/webvttdec.c @@ -91,7 +91,7 @@ static int webvtt_decode_frame(AVCodecContext *avctx, av_bprint_init(&buf, 0, AV_BPRINT_SIZE_UNLIMITED); if (ptr && avpkt->size > 0 && !webvtt_event_to_ass(&buf, ptr)) - ret = ff_ass_add_rect(sub, buf.str, s->readorder++, 0, NULL, NULL); + ret = avpriv_ass_add_rect(sub, buf.str, s->readorder++, 0, NULL, NULL); av_bprint_finalize(&buf, NULL); if (ret < 0) return ret; diff --git a/libavcodec/webvttenc.c b/libavcodec/webvttenc.c index 89b49e42bf..761099b69a 100644 --- a/libavcodec/webvttenc.c +++ b/libavcodec/webvttenc.c @@ -24,8 +24,8 @@ #include "avcodec.h" #include "libavutil/avstring.h" #include "libavutil/bprint.h" -#include "ass_split.h" -#include "ass.h" +#include "libavutil/ass_split_internal.h" +#include "libavutil/ass_internal.h" #include "internal.h" #define WEBVTT_STACK_SIZE 64 @@ -93,7 +93,7 @@ static void webvtt_stack_push_pop(WebVTTContext *s, const char c, int close) static void webvtt_style_apply(WebVTTContext *s, const char *style) { - ASSStyle *st = ff_ass_style_get(s->ass_ctx, style); + ASSStyle *st = avpriv_ass_style_get(s->ass_ctx, style); if (st) { if (st->bold != ASS_DEFAULT_BOLD) { webvtt_print(s, ""); @@ -172,12 +172,12 @@ static int webvtt_encode_frame(AVCodecContext *avctx, return AVERROR(EINVAL); } - dialog = ff_ass_split_dialog(s->ass_ctx, ass); + dialog = avpriv_ass_split_dialog(s->ass_ctx, ass); if (!dialog) return AVERROR(ENOMEM); webvtt_style_apply(s, dialog->style); - ff_ass_split_override_codes(&webvtt_callbacks, s, dialog->text); - ff_ass_free_dialog(&dialog); + avpriv_ass_split_override_codes(&webvtt_callbacks, s, dialog->text); + avpriv_ass_free_dialog(&dialog); } if (!av_bprint_is_complete(&s->buffer)) @@ -197,7 +197,7 @@ static int webvtt_encode_frame(AVCodecContext *avctx, static int webvtt_encode_close(AVCodecContext *avctx) { WebVTTContext *s = avctx->priv_data; - ff_ass_split_free(s->ass_ctx); + avpriv_ass_split_free(s->ass_ctx); av_bprint_finalize(&s->buffer, NULL); return 0; } @@ -206,7 +206,7 @@ static av_cold int webvtt_encode_init(AVCodecContext *avctx) { WebVTTContext *s = avctx->priv_data; s->avctx = avctx; - s->ass_ctx = ff_ass_split(avctx->subtitle_header); + s->ass_ctx = avpriv_ass_split(avctx->subtitle_header); av_bprint_init(&s->buffer, 0, AV_BPRINT_SIZE_UNLIMITED); return s->ass_ctx ? 0 : AVERROR_INVALIDDATA; } diff --git a/libavutil/Makefile b/libavutil/Makefile index 7e79936876..b81d8e0d35 100644 --- a/libavutil/Makefile +++ b/libavutil/Makefile @@ -101,6 +101,8 @@ BUILT_HEADERS = avconfig.h \ OBJS = adler32.o \ aes.o \ aes_ctr.o \ + ass.o \ + ass_split.o \ audio_fifo.o \ avstring.o \ avsscanf.o \ diff --git a/libavcodec/ass.c b/libavutil/ass.c similarity index 65% rename from libavcodec/ass.c rename to libavutil/ass.c index 907e2d7b88..d494de5fe2 100644 --- a/libavcodec/ass.c +++ b/libavutil/ass.c @@ -19,21 +19,22 @@ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ -#include "avcodec.h" -#include "ass.h" +#include "ass_internal.h" + +#include "subfmt.h" #include "libavutil/avstring.h" #include "libavutil/bprint.h" #include "libavutil/common.h" -int ff_ass_subtitle_header_full(AVCodecContext *avctx, - int play_res_x, int play_res_y, - const char *font, int font_size, - int primary_color, int secondary_color, - int outline_color, int back_color, - int bold, int italic, int underline, - int border_style, int alignment) +char* avpriv_ass_get_subtitle_header_full(int play_res_x, int play_res_y, + const char *font, int font_size, + int primary_color, int secondary_color, + int outline_color, int back_color, + int bold, int italic, int underline, + int border_style, int alignment, + int print_av_version) { - avctx->subtitle_header = av_asprintf( + char* header = av_asprintf( "[Script Info]\r\n" "; Script generated by FFmpeg/Lavc%s\r\n" "ScriptType: v4.00+\r\n" @@ -68,34 +69,31 @@ int ff_ass_subtitle_header_full(AVCodecContext *avctx, "\r\n" "[Events]\r\n" "Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text\r\n", - !(avctx->flags & AV_CODEC_FLAG_BITEXACT) ? AV_STRINGIFY(LIBAVCODEC_VERSION) : "", + print_av_version ? AV_STRINGIFY(LIBAVCODEC_VERSION) : "", play_res_x, play_res_y, font, font_size, primary_color, secondary_color, outline_color, back_color, -bold, -italic, -underline, border_style, alignment); - if (!avctx->subtitle_header) - return AVERROR(ENOMEM); - avctx->subtitle_header_size = strlen(avctx->subtitle_header); - return 0; + return header; } -int ff_ass_subtitle_header(AVCodecContext *avctx, - const char *font, int font_size, +char* avpriv_ass_get_subtitle_header(const char *font, int font_size, int color, int back_color, int bold, int italic, int underline, - int border_style, int alignment) + int border_style, int alignment, + int print_av_version) { - return ff_ass_subtitle_header_full(avctx, - ASS_DEFAULT_PLAYRESX, ASS_DEFAULT_PLAYRESY, - font, font_size, color, color, - back_color, back_color, - bold, italic, underline, - border_style, alignment); + return avpriv_ass_get_subtitle_header_full(ASS_DEFAULT_PLAYRESX, ASS_DEFAULT_PLAYRESY, + font, font_size, color, color, + back_color, back_color, + bold, italic, underline, + border_style, alignment, + print_av_version); } -int ff_ass_subtitle_header_default(AVCodecContext *avctx) +char* avpriv_ass_get_subtitle_header_default(int print_av_version) { - return ff_ass_subtitle_header(avctx, ASS_DEFAULT_FONT, + return avpriv_ass_get_subtitle_header(ASS_DEFAULT_FONT, ASS_DEFAULT_FONT_SIZE, ASS_DEFAULT_COLOR, ASS_DEFAULT_BACK_COLOR, @@ -103,10 +101,11 @@ int ff_ass_subtitle_header_default(AVCodecContext *avctx) ASS_DEFAULT_ITALIC, ASS_DEFAULT_UNDERLINE, ASS_DEFAULT_BORDERSTYLE, - ASS_DEFAULT_ALIGNMENT); + ASS_DEFAULT_ALIGNMENT, + print_av_version); } -char *ff_ass_get_dialog(int readorder, int layer, const char *style, +char *avpriv_ass_get_dialog(int readorder, int layer, const char *style, const char *speaker, const char *text) { return av_asprintf("%d,%d,%s,%s,0,0,0,,%s", @@ -114,37 +113,7 @@ char *ff_ass_get_dialog(int readorder, int layer, const char *style, speaker ? speaker : "", text); } -int ff_ass_add_rect(AVSubtitle *sub, const char *dialog, - int readorder, int layer, const char *style, - const char *speaker) -{ - char *ass_str; - AVSubtitleRect **rects; - - rects = av_realloc_array(sub->rects, sub->num_rects+1, sizeof(*sub->rects)); - if (!rects) - return AVERROR(ENOMEM); - sub->rects = rects; - rects[sub->num_rects] = av_mallocz(sizeof(*rects[0])); - if (!rects[sub->num_rects]) - return AVERROR(ENOMEM); - rects[sub->num_rects]->type = SUBTITLE_ASS; - ass_str = ff_ass_get_dialog(readorder, layer, style, speaker, dialog); - if (!ass_str) - return AVERROR(ENOMEM); - rects[sub->num_rects]->ass = ass_str; - sub->num_rects++; - return 0; -} - -void ff_ass_decoder_flush(AVCodecContext *avctx) -{ - FFASSDecoderContext *s = avctx->priv_data; - if (!(avctx->flags2 & AV_CODEC_FLAG2_RO_FLUSH_NOOP)) - s->readorder = 0; -} - -void ff_ass_bprint_text_event(AVBPrint *buf, const char *p, int size, +void avpriv_ass_bprint_text_event(AVBPrint *buf, const char *p, int size, const char *linebreaks, int keep_ass_markup) { const char *p_end = p + size; diff --git a/libavutil/ass_internal.h b/libavutil/ass_internal.h new file mode 100644 index 0000000000..833766eada --- /dev/null +++ b/libavutil/ass_internal.h @@ -0,0 +1,133 @@ +/* + * SSA/ASS common functions + * Copyright (c) 2010 Aurelien Jacobs + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVUTIL_ASS_INTERNAL_H +#define AVUTIL_ASS_INTERNAL_H + +#include "subfmt.h" +#include "libavutil/bprint.h" + +#define ASS_DEFAULT_PLAYRESX 384 +#define ASS_DEFAULT_PLAYRESY 288 + +/** + * @name Default values for ASS style + * @{ + */ +#define ASS_DEFAULT_FONT "Arial" +#define ASS_DEFAULT_FONT_SIZE 16 +#define ASS_DEFAULT_COLOR 0xffffff +#define ASS_DEFAULT_BACK_COLOR 0 +#define ASS_DEFAULT_BOLD 0 +#define ASS_DEFAULT_ITALIC 0 +#define ASS_DEFAULT_UNDERLINE 0 +#define ASS_DEFAULT_ALIGNMENT 2 +#define ASS_DEFAULT_BORDERSTYLE 1 +/** @} */ + +/** + * Generate a suitable AVCodecContext.subtitle_header for SUBTITLE_ASS. + * Can specify all fields explicitly + * + * @param play_res_x subtitle frame width + * @param play_res_y subtitle frame height + * @param font name of the default font face to use + * @param font_size default font size to use + * @param primary_color default text color to use (ABGR) + * @param secondary_color default secondary text color to use (ABGR) + * @param outline_color default outline color to use (ABGR) + * @param back_color default background color to use (ABGR) + * @param bold 1 for bold text, 0 for normal text + * @param italic 1 for italic text, 0 for normal text + * @param underline 1 for underline text, 0 for normal text + * @param border_style 1 for outline, 3 for opaque box + * @param alignment position of the text (left, center, top...), defined after + * the layout of the numpad (1-3 sub, 4-6 mid, 7-9 top) + * @param print_av_version include library version in header + * @return a string containing the subtitle header that needs + * to be released via av_free() + */ +char* avpriv_ass_get_subtitle_header_full(int play_res_x, int play_res_y, + const char *font, int font_size, + int primary_color, int secondary_color, + int outline_color, int back_color, + int bold, int italic, int underline, + int border_style, int alignment, + int print_av_version); +/** + * Generate a suitable AVCodecContext.subtitle_header for SUBTITLE_ASS. + * + * @param font name of the default font face to use + * @param font_size default font size to use + * @param color default text color to use (ABGR) + * @param back_color default background color to use (ABGR) + * @param bold 1 for bold text, 0 for normal text + * @param italic 1 for italic text, 0 for normal text + * @param underline 1 for underline text, 0 for normal text + * @param border_style 1 for outline, 3 for opaque box + * @param alignment position of the text (left, center, top...), defined after + * the layout of the numpad (1-3 sub, 4-6 mid, 7-9 top) + * @param print_av_version include library version in header + * @return a string containing the subtitle header that needs + * to be released via av_free() + */ +char* avpriv_ass_get_subtitle_header(const char *font, int font_size, + int color, int back_color, + int bold, int italic, int underline, + int border_style, int alignment, + int print_av_version); + +/** + * Generate a suitable AVCodecContext.subtitle_header for SUBTITLE_ASS + * with default style. + * + * @param print_av_version include library version in header + * @return a string containing the subtitle header that needs + * to be released via av_free() + */ +char* avpriv_ass_get_subtitle_header_default(int print_av_version); + +/** + * Craft an ASS dialog string. + */ +char *avpriv_ass_get_dialog(int readorder, int layer, const char *style, + const char *speaker, const char *text); + + +/////** +//// * Helper to flush a text subtitles decoder making use of the +//// * FFASSDecoderContext. +//// */ +////void ff_ass_decoder_flush(AVCodecContext *avctx); + +/** + * Escape a text subtitle using ASS syntax into an AVBPrint buffer. + * Newline characters will be escaped to \N. + * + * @param buf pointer to an initialized AVBPrint buffer + * @param p source text + * @param size size of the source text + * @param linebreaks additional newline chars, which will be escaped to \N + * @param keep_ass_markup braces and backslash will not be escaped if set + */ +void avpriv_ass_bprint_text_event(AVBPrint *buf, const char *p, int size, + const char *linebreaks, int keep_ass_markup); +#endif /* AVUTIL_ASS_INTERNAL_H */ diff --git a/libavcodec/ass_split.c b/libavutil/ass_split.c similarity index 94% rename from libavcodec/ass_split.c rename to libavutil/ass_split.c index 05c5453e53..c5963351fc 100644 --- a/libavcodec/ass_split.c +++ b/libavutil/ass_split.c @@ -22,7 +22,7 @@ #include "libavutil/common.h" #include "libavutil/error.h" #include "libavutil/mem.h" -#include "ass_split.h" +#include "ass_split_internal.h" typedef enum { ASS_STR, @@ -373,7 +373,7 @@ static int ass_split(ASSSplitContext *ctx, const char *buf) return buf ? 0 : AVERROR_INVALIDDATA; } -ASSSplitContext *ff_ass_split(const char *buf) +ASSSplitContext *avpriv_ass_split(const char *buf) { ASSSplitContext *ctx = av_mallocz(sizeof(*ctx)); if (!ctx) @@ -382,7 +382,7 @@ ASSSplitContext *ff_ass_split(const char *buf) buf += 3; ctx->current_section = -1; if (ass_split(ctx, buf) < 0) { - ff_ass_split_free(ctx); + avpriv_ass_split_free(ctx); return NULL; } return ctx; @@ -412,7 +412,7 @@ static void free_section(ASSSplitContext *ctx, const ASSSection *section) av_freep((uint8_t *)&ctx->ass + section->offset); } -void ff_ass_free_dialog(ASSDialog **dialogp) +void avpriv_ass_free_dialog(ASSDialog **dialogp) { ASSDialog *dialog = *dialogp; if (!dialog) @@ -424,7 +424,7 @@ void ff_ass_free_dialog(ASSDialog **dialogp) av_freep(dialogp); } -ASSDialog *ff_ass_split_dialog(ASSSplitContext *ctx, const char *buf) +ASSDialog *avpriv_ass_split_dialog(ASSSplitContext *ctx, const char *buf) { int i; static const ASSFields fields[] = { @@ -451,7 +451,7 @@ ASSDialog *ff_ass_split_dialog(ASSSplitContext *ctx, const char *buf) buf = skip_space(buf); len = last ? strlen(buf) : strcspn(buf, ","); if (len >= INT_MAX) { - ff_ass_free_dialog(&dialog); + avpriv_ass_free_dialog(&dialog); return NULL; } convert_func[type](ptr, buf, len); @@ -461,7 +461,7 @@ ASSDialog *ff_ass_split_dialog(ASSSplitContext *ctx, const char *buf) return dialog; } -void ff_ass_split_free(ASSSplitContext *ctx) +void avpriv_ass_split_free(ASSSplitContext *ctx) { if (ctx) { int i; @@ -474,7 +474,7 @@ void ff_ass_split_free(ASSSplitContext *ctx) } -int ff_ass_split_override_codes(const ASSCodesCallbacks *callbacks, void *priv, +int avpriv_ass_split_override_codes(const ASSCodesCallbacks *callbacks, void *priv, const char *buf) { const char *text = NULL; @@ -497,8 +497,8 @@ int ff_ass_split_override_codes(const ASSCodesCallbacks *callbacks, void *priv, while (*buf == '\\') { char style[2], c[2], sep[2], c_num[2] = "0", tmp[128] = {0}; unsigned int color = 0xFFFFFFFF; - int len, size = -1, an = -1, alpha = -1; - int x1, y1, x2, y2, t1 = -1, t2 = -1; + int len, size = -1, an = -1, alpha = -1, scale = 0; + int x1, y1, x2, y2, t1 = -1, t2 = -1, accel = 1; if (sscanf(buf, "\\%1[bisu]%1[01\\}]%n", style, c, &len) > 1) { int close = c[0] == '0' ? 1 : c[0] == '1' ? 0 : -1; len += close != -1; @@ -546,6 +546,14 @@ int ff_ass_split_override_codes(const ASSCodesCallbacks *callbacks, void *priv, } else if (sscanf(buf, "\\org(%d,%d)%1[\\}]%n", &x1, &y1, sep, &len) > 2) { if (callbacks->origin) callbacks->origin(priv, x1, y1); + } else if (sscanf(buf, "\\t(%d,%d,%1[\\}]%n", &t1, &t2, sep, &len) > 2 || + sscanf(buf, "\\t(%d,%d,%d,%1[\\}]%n", &t1, &t2, &accel, sep, &len) > 3) { + if (callbacks->animate) + callbacks->animate(priv, t1, t2, accel, tmp); + } else if (sscanf(buf, "\\p%1[\\}]%n", sep, &len) > 0 || + sscanf(buf, "\\p%u%1[\\}]%n", &scale, sep, &len) > 1) { + if (callbacks->drawing_mode) + callbacks->drawing_mode(priv, scale); } else { len = strcspn(buf+1, "\\}") + 2; /* skip unknown code */ } @@ -569,7 +577,7 @@ int ff_ass_split_override_codes(const ASSCodesCallbacks *callbacks, void *priv, return 0; } -ASSStyle *ff_ass_style_get(ASSSplitContext *ctx, const char *style) +ASSStyle *avpriv_ass_style_get(ASSSplitContext *ctx, const char *style) { ASS *ass = &ctx->ass; int i; diff --git a/libavcodec/ass_split.h b/libavutil/ass_split_internal.h similarity index 86% rename from libavcodec/ass_split.h rename to libavutil/ass_split_internal.h index a45fb9b8a1..eee49ef0f5 100644 --- a/libavcodec/ass_split.h +++ b/libavutil/ass_split_internal.h @@ -19,8 +19,8 @@ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ -#ifndef AVCODEC_ASS_SPLIT_H -#define AVCODEC_ASS_SPLIT_H +#ifndef AVUTIL_ASS_SPLIT_INTERNAL_H +#define AVUTIL_ASS_SPLIT_INTERNAL_H /** * fields extracted from the [Script Info] section @@ -81,7 +81,7 @@ typedef struct { char *effect; char *text; /**< actual text which will be displayed as a subtitle, can include style override control codes (see - ff_ass_split_override_codes()) */ + avpriv_ass_split_override_codes()) */ } ASSDialog; /** @@ -107,12 +107,12 @@ typedef struct ASSSplitContext ASSSplitContext; * @param buf String containing the ASS formatted data. * @return Newly allocated struct containing split data. */ -ASSSplitContext *ff_ass_split(const char *buf); +ASSSplitContext *avpriv_ass_split(const char *buf); /** - * Free a dialogue obtained from ff_ass_split_dialog(). + * Free a dialogue obtained from avpriv_ass_split_dialog(). */ -void ff_ass_free_dialog(ASSDialog **dialogp); +void avpriv_ass_free_dialog(ASSDialog **dialogp); /** * Split one ASS Dialogue line from a string buffer. @@ -121,14 +121,14 @@ void ff_ass_free_dialog(ASSDialog **dialogp); * @param buf String containing the ASS "Dialogue" line. * @return Pointer to the split ASSDialog. Must be freed with ff_ass_free_dialog() */ -ASSDialog *ff_ass_split_dialog(ASSSplitContext *ctx, const char *buf); +ASSDialog *avpriv_ass_split_dialog(ASSSplitContext *ctx, const char *buf); /** * Free all the memory allocated for an ASSSplitContext. * * @param ctx Context previously initialized by ff_ass_split(). */ -void ff_ass_split_free(ASSSplitContext *ctx); +void avpriv_ass_split_free(ASSSplitContext *ctx); /** @@ -141,6 +141,7 @@ typedef struct { * @{ */ void (*text)(void *priv, const char *text, int len); + void (*hard_space)(void *priv); void (*new_line)(void *priv, int forced); void (*style)(void *priv, char style, int close); void (*color)(void *priv, unsigned int /* color */, unsigned int color_id); @@ -156,7 +157,16 @@ typedef struct { * @{ */ void (*move)(void *priv, int x1, int y1, int x2, int y2, int t1, int t2); + void (*animate)(void *priv, int t1, int t2, int accel, char *style); void (*origin)(void *priv, int x, int y); + void (*drawing_mode)(void *priv, int scale); + /** @} */ + + /** + * @defgroup ass_ext ASS extensible parsing callback + * @{ + */ + void (*ext)(void *priv, int ext_id, const char *text, int p1, int p2); /** @} */ /** @@ -176,7 +186,7 @@ typedef struct { * @param buf The ASS "Dialogue" Text field to split. * @return >= 0 on success otherwise an error code <0 */ -int ff_ass_split_override_codes(const ASSCodesCallbacks *callbacks, void *priv, +int avpriv_ass_split_override_codes(const ASSCodesCallbacks *callbacks, void *priv, const char *buf); /** @@ -186,6 +196,6 @@ int ff_ass_split_override_codes(const ASSCodesCallbacks *callbacks, void *priv, * @param style name of the style to search for. * @return the ASSStyle corresponding to style, or NULL if style can't be found */ -ASSStyle *ff_ass_style_get(ASSSplitContext *ctx, const char *style); +ASSStyle *avpriv_ass_style_get(ASSSplitContext *ctx, const char *style); -#endif /* AVCODEC_ASS_SPLIT_H */ +#endif /* AVUTIL_ASS_SPLIT_INTERNAL_H */ From patchwork Mon Dec 13 23:40:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 32457 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp6139532iog; Mon, 13 Dec 2021 15:41:31 -0800 (PST) X-Google-Smtp-Source: ABdhPJz2MLKaW61HQVKgpMqvTsRho7+4nB2viY/P+u9RaK9DvSYgwoJuA8Z3mc0T74yU9ChVgT55 X-Received: by 2002:a05:6402:4382:: with SMTP id o2mr2699442edc.143.1639438891608; Mon, 13 Dec 2021 15:41:31 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id qa31si16173183ejc.116.2021.12.13.15.41.31; Mon, 13 Dec 2021 15:41:31 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=tmzb00Hl; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id ECD1D68B0A6; Tue, 14 Dec 2021 01:40:42 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11olkn2032.outbound.protection.outlook.com [40.92.18.32]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id AD7C368B074 for ; Tue, 14 Dec 2021 01:40:39 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=T6qyrIwHCytHaFINUfLL/claTX+sEi+9mFUT7SHR5QXS2uxk65NWAsuNOamMNnxs6vNQB/DvNh6U3OMR10X50oIIV5c0m4aX4llL/C7ySMLgKEGFcPiUiWR2eQVFjT1MOvwIiqJjBgTqKyazPz6rqZW7+fst7lL/2mtBXP8dM8A1gEkJ3KNUBBIVTPR2iC0geq8eLQzBv5ym8lru9ndSMgglcq3fjYzBwdGXP/T0oV/GA7aElAD5jYszb43j5IToztbEYP6wgX4Dj9BAiE3Q/OJvWubMifvzTSlWyyzf6ocCOl2BmcpwBOYqfB6iG6Ys0QzCXfXYZK4MFZFTNDcR9w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=rE7F+ymM6cYe9voC+saiM94GQDnE0Tu27BlZ3QvFfCY=; b=ZdeD/YzZrwEPb6gLZfiznLkGhycVB6+EBAzLyzndBPiw4XWiFEAahmW+/TKN/qpCG2HPlo/i0Sw9IFLc253oY46RlfYsVYvB2dOwwUysLXWhvmdiFxUeOoqVT65a4Hg9z2RODhQrk1+h5OXGjZGq68ibVixsQtVBGKUGjkZVd6CqcPrG1A2CjGRA8eDUDC9tVuTxFjLuHjPl0obkeiQStcaLD2GVf5uQgFnGvZCJ/U3sq8tayVAaGVQ+V2x2kwB+NfF7Fsi2291cDqECGYUuquEu4RRr4lD1l+h0hSgt8sbQq6zGYhm3MofGFXnHcMHhWObJHftAQn7B3koUKL9Clw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rE7F+ymM6cYe9voC+saiM94GQDnE0Tu27BlZ3QvFfCY=; b=tmzb00HlAdeKEqS4/w9eMQehkyjNKZbZxSavomXy4Ey8nd9mIAZk5kU8gzkcksSn55eIIzM8Bb8az+glhBGKgOAytrRzb/yqVdusj//2pIJgTZyZeLbfsvNFnJse8VpR+dG+BpEbj5CC9ISePzwJqiK4gRPt0BY1MmAa1JS5GjmX0pXhLCt9YmTk+NsP0lCQwbxb2mKhZxMi4rMb22CnqtbYefFB/j/0kp6vN+H8k+TPDjdBreHpd0VILxdxhYlQUFTIwMXtjVOZrVOIUn9Tf9qZ2rX50LxUl1PfuNRUAn1FWfx0133jF2TW2/i5vCH04jRroXDpwIPsMuytRs1Ydw== Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::20) by DM8P223MB0303.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4778.12; Mon, 13 Dec 2021 23:40:32 +0000 Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775]) by DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775%7]) with mapi id 15.20.4778.018; Mon, 13 Dec 2021 23:40:32 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v24 06/21] avcodec/subtitles: Migrate subtitle encoders to frame-based API and provide a compatibility shim for the legacy api Thread-Index: AQHX8HrRfWDT7g9XYUm4g8bpNizcNw== Date: Mon, 13 Dec 2021 23:40:32 +0000 Message-ID: References: <1ef42eb01b3627bbad539abd46206f320bb39697.1639438462.git.softworkz@hotmail.com> <4148d77a82d4b7e229fc81ce9bc1bc06904533f1.1639438462.git.softworkz@hotmail.com> <2ac798924ee984010d2e85f00bc992d9aa106171.1639438462.git.softworkz@hotmail.com> <300ea7188b331ca5e00dd7e04c03df8aba2a99c0.1639438462.git.softworkz@hotmail.com> <9474726b26f0d3e8f353d07caed2fbd33cdf473a.1639438462.git.softworkz@hotmail.com> In-Reply-To: <9474726b26f0d3e8f353d07caed2fbd33cdf473a.1639438462.git.softworkz@hotmail.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [GKASNtEsalQqngLR3t+lQdO53JYGcyd6] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: e4125d8d-0fa2-41ca-7593-08d9be91f409 x-ms-traffictypediagnostic: DM8P223MB0303:EE_ x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: uelPSMpFYGe5Ibcr1YbwFUH1bpSd4pYGEHxUbMDop7xqWM7WKqdKv3bUJ+QC9DoPXziQ4H5mdlFMI5G0frennyWoTkqxdirCKVv29a2VD50mACVqDHJO+NCHQx7mLRBYygqlmvEHD5T9ioRWtJbb8zEC1yqigEMxszDx+4/0M0wY9+A6MWyHV7pMiOldWPk3VXjPEQ9Q3BcwC1NthkFuQpppRCuaVNF/+orwBkBZ0Tlq25zHFN/mfO/EeZEAAw+W6KIVRJ35e/IUzbkMDyhfO3reNo7BBA3xVCH/wzo2oeAOUWzfIZ4wmvjkPEGgJLbo/21sk91wyR5QN6mWoprIU6tiVBSPRzlh9p9r8F8aMGpPGd7HrU/ALPU9Mz72rsumh0dm5DxTOhvngxoSstkAIzspDn/fCvSK+xGK5PHTQIEHTBrtTQBtM+yP+pbDQm3fKsSYA1z1KvDYudjYL27qJvCFJvunX2Ks0SzG0HlrIRm+quVyZ8wNXtBPzaW/rsRU+80QJ6Z235jJK2LRTnUOhLjW+d1VXNjHEvT4yqY28uyO3Urq1flAvpyWiDIvBCIiiTqQWAitUi9vKMfhk5gwwQ== x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?q?/PximhlBHiNV4GbwvqcpLMfrj0zC?= =?utf-8?q?T150g6wAZZyY4fatMF680IkbIp5krObdWqf4HKP7fRRrz5wf40j4RuYOqQVHFVBI/?= =?utf-8?q?satnhnJFl+EI4PIJjcSjZfzaVMpmdegBcvAccAG2yyzKoK4AkbnwsTaXtx89j2klv?= =?utf-8?q?1Kuj9hbfq1J5yDIt6iIIbiR8Lm1FkIWs53HNQrdwTHTNrQrSwlBzfiFninQvR4Jea?= =?utf-8?q?KFFDVYoQB65Yy1okS8/zVaJh5C17mqvf7j4JT9n93g6Uv45oEy/68XhAkSaMo4wuq?= =?utf-8?q?7MAnkHz7jxPMUO5OwGxWrzj+YkysidEdZE02iYjRbI0B+4fPnX1nzjmlHyiu7M1kO?= =?utf-8?q?5V88RC/DsRiy+exuvrNvPpg5T6ko7VHI/Jcp1lxXq1bBgUDYBWvJrJcDcgRumHQXh?= =?utf-8?q?fEnPSQoz1Hi6E7bCeV3vNw7boSsYDIEL5OCwYas3TbAX7DmR3mJm6pteWwvadP/FI?= =?utf-8?q?q1txuu+TPjwjAbfOf5Nuj9Ib8seSoeX43cMf5G9Ep1/WOviUHOTyQnv26fZ18WCYd?= =?utf-8?q?wiC6JK9YEyCr/9wCejI/7/gFg4TgZUGcc8R6cJbWRps3W1tlrwjeuVNG2JzV6NM8r?= =?utf-8?q?sDL08XE1RyClGI1KEZ8NbDCr5KZ0HACz/2wlajJZr7kp66EsCLIwTjIEAi753UNaL?= =?utf-8?q?TVxRz0RdHzSOT08dXFitdYfif/uhvAv17FJc3tIZcrym5L6EWs0rmGhSFQODz3e+h?= =?utf-8?q?9wmqvtAralGwU2jAguFGumM4/sB1/g7HxqPrzLYE9IQgxJFqFs3+WD7hFk9ndNhHT?= =?utf-8?q?53x/IVJvp8tVhlz5hQjqjtVq5ZP93+54/POLvo10Yk9XxeKegdrlYov8T62m7xlyf?= =?utf-8?q?MTnc+wb08gXtx+kaxJTAIoP2TzJJ/SxPvkzF8+MGUSMWrBKVj7VcczYPG+6UhQl5T?= =?utf-8?q?GvFbo/vobPGV/XOUo9D0HjrDV/O4jtNPLiGcIPaQxKvA=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-1ff67.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: e4125d8d-0fa2-41ca-7593-08d9be91f409 X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Dec 2021 23:40:32.2934 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8P223MB0303 Subject: [FFmpeg-devel] [PATCH v24 06/21] avcodec/subtitles: Migrate subtitle encoders to frame-based API and provide a compatibility shim for the legacy api X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: G/gU/rMxXRPk Signed-off-by: softworkz --- libavcodec/assenc.c | 90 +++++++++++++++++++++-------- libavcodec/avcodec.h | 5 +- libavcodec/dvbsubenc.c | 96 +++++++++++++++++-------------- libavcodec/dvdsubenc.c | 100 +++++++++++++++++++-------------- libavcodec/encode.c | 63 ++++++++++++++++++++- libavcodec/movtextenc.c | 112 +++++++++++++++++++++++++++---------- libavcodec/srtenc.c | 104 ++++++++++++++++++++++------------ libavcodec/tests/avcodec.c | 2 - libavcodec/ttmlenc.c | 98 ++++++++++++++++++++++++-------- libavcodec/webvttenc.c | 82 +++++++++++++++++++-------- libavcodec/xsubenc.c | 87 +++++++++++++++++----------- 11 files changed, 584 insertions(+), 255 deletions(-) diff --git a/libavcodec/assenc.c b/libavcodec/assenc.c index b0e475834b..94601bba68 100644 --- a/libavcodec/assenc.c +++ b/libavcodec/assenc.c @@ -22,48 +22,92 @@ #include #include "avcodec.h" +#include "encode.h" #include "libavutil/ass_internal.h" #include "internal.h" #include "libavutil/avstring.h" #include "libavutil/internal.h" #include "libavutil/mem.h" +static void check_write_header(AVCodecContext* avctx, const AVFrame* frame) +{ + if (avctx->extradata_size) + return; + + if (frame->subtitle_header && frame->subtitle_header->size > 0) { + const char* subtitle_header = (char*)frame->subtitle_header->data; + avctx->extradata_size = strlen(subtitle_header); + avctx->extradata = av_mallocz(frame->subtitle_header->size + 1); + memcpy(avctx->extradata, subtitle_header, avctx->extradata_size); + avctx->extradata[avctx->extradata_size] = 0; + } + + if (!avctx->extradata_size) { + const char* subtitle_header = avpriv_ass_get_subtitle_header_default(0); + if (!subtitle_header) + return; + + avctx->extradata_size = strlen(subtitle_header); + avctx->extradata = av_mallocz(avctx->extradata_size + 1); + memcpy(avctx->extradata, subtitle_header, avctx->extradata_size); + avctx->extradata[avctx->extradata_size] = 0; + av_freep(&subtitle_header); + } +} + static av_cold int ass_encode_init(AVCodecContext *avctx) { - avctx->extradata = av_malloc(avctx->subtitle_header_size + 1); - if (!avctx->extradata) - return AVERROR(ENOMEM); - memcpy(avctx->extradata, avctx->subtitle_header, avctx->subtitle_header_size); - avctx->extradata_size = avctx->subtitle_header_size; - avctx->extradata[avctx->extradata_size] = 0; + if (avctx->subtitle_header_size) { + avctx->extradata = av_malloc(avctx->subtitle_header_size + 1); + if (!avctx->extradata) + return AVERROR(ENOMEM); + memcpy(avctx->extradata, avctx->subtitle_header, avctx->subtitle_header_size); + avctx->extradata_size = avctx->subtitle_header_size; + avctx->extradata[avctx->extradata_size] = 0; + } + return 0; } -static int ass_encode_frame(AVCodecContext *avctx, - unsigned char *buf, int bufsize, - const AVSubtitle *sub) +static int ass_encode_frame(AVCodecContext* avctx, AVPacket* avpkt, + const AVFrame* frame, int* got_packet) { - int i, len, total_len = 0; + int ret; + size_t req_len = 0, total_len = 0; + + check_write_header(avctx, frame); - for (i=0; inum_rects; i++) { - const char *ass = sub->rects[i]->ass; + for (unsigned i = 0; i < frame->num_subtitle_areas; i++) { + const char *ass = frame->subtitle_areas[i]->ass; - if (sub->rects[i]->type != SUBTITLE_ASS) { - av_log(avctx, AV_LOG_ERROR, "Only SUBTITLE_ASS type supported.\n"); + if (frame->subtitle_areas[i]->type != AV_SUBTITLE_FMT_ASS) { + av_log(avctx, AV_LOG_ERROR, "Only AV_SUBTITLE_FMT_ASS type supported.\n"); return AVERROR(EINVAL); } - len = av_strlcpy(buf+total_len, ass, bufsize-total_len); + if (ass) + req_len += strlen(ass); + } - if (len > bufsize-total_len-1) { - av_log(avctx, AV_LOG_ERROR, "Buffer too small for ASS event.\n"); - return AVERROR_BUFFER_TOO_SMALL; - } + ret = ff_get_encode_buffer(avctx, avpkt, req_len + 1, 0); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Error getting output packet.\n"); + return ret; + } - total_len += len; + for (unsigned i = 0; i < frame->num_subtitle_areas; i++) { + const char *ass = frame->subtitle_areas[i]->ass; + + if (ass) { + size_t len = av_strlcpy((char *)avpkt->data + total_len, ass, avpkt->size - total_len); + total_len += len; + } } - return total_len; + avpkt->size = total_len; + *got_packet = total_len > 0; + + return 0; } #if CONFIG_SSA_ENCODER @@ -73,7 +117,7 @@ const AVCodec ff_ssa_encoder = { .type = AVMEDIA_TYPE_SUBTITLE, .id = AV_CODEC_ID_ASS, .init = ass_encode_init, - .encode_sub = ass_encode_frame, + .encode2 = ass_encode_frame, .caps_internal = FF_CODEC_CAP_INIT_THREADSAFE, }; #endif @@ -85,7 +129,7 @@ const AVCodec ff_ass_encoder = { .type = AVMEDIA_TYPE_SUBTITLE, .id = AV_CODEC_ID_ASS, .init = ass_encode_init, - .encode_sub = ass_encode_frame, + .encode2 = ass_encode_frame, .caps_internal = FF_CODEC_CAP_INIT_THREADSAFE, }; #endif diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index 3e734d3003..3f25fda708 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -2997,10 +2997,13 @@ void av_parser_close(AVCodecParserContext *s); * @{ */ + /** + * @deprecated Use @ref avcodec_encode_subtitle2() instead. + */ +attribute_deprecated int avcodec_encode_subtitle(AVCodecContext *avctx, uint8_t *buf, int buf_size, const AVSubtitle *sub); - /** * @} */ diff --git a/libavcodec/dvbsubenc.c b/libavcodec/dvbsubenc.c index 322fc27cb4..3b5a76daa7 100644 --- a/libavcodec/dvbsubenc.c +++ b/libavcodec/dvbsubenc.c @@ -20,6 +20,7 @@ */ #include "avcodec.h" #include "bytestream.h" +#include "encode.h" #include "libavutil/colorspace.h" typedef struct DVBSubtitleContext { @@ -268,21 +269,29 @@ static int dvb_encode_rle8(uint8_t **pq, int buf_size, return len; } -static int dvbsub_encode(AVCodecContext *avctx, uint8_t *outbuf, int buf_size, - const AVSubtitle *h) +static int dvbsub_encode(AVCodecContext* avctx, AVPacket* avpkt, + const AVFrame* frame, int* got_packet) { DVBSubtitleContext *s = avctx->priv_data; uint8_t *q, *pseg_len; int page_id, region_id, clut_id, object_id, i, bpp_index, page_state; - - - q = outbuf; + size_t buf_size; + int ret; page_id = 1; - if (h->num_rects && !h->rects) + if (frame->num_subtitle_areas && !frame->subtitle_areas) return AVERROR(EINVAL); + ret = ff_get_encode_buffer(avctx, avpkt, 1024 * 1024, 0); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Error getting output packet.\n"); + return ret; + } + + buf_size = avpkt->size; + q = avpkt->data; + if (avctx->width > 0 && avctx->height > 0) { if (buf_size < 11) return AVERROR_BUFFER_TOO_SMALL; @@ -301,7 +310,7 @@ static int dvbsub_encode(AVCodecContext *avctx, uint8_t *outbuf, int buf_size, /* page composition segment */ - if (buf_size < 8 + h->num_rects * 6) + if (buf_size < 8 + frame->num_subtitle_areas * 6) return AVERROR_BUFFER_TOO_SMALL; *q++ = 0x0f; /* sync_byte */ *q++ = 0x10; /* segment_type */ @@ -313,30 +322,30 @@ static int dvbsub_encode(AVCodecContext *avctx, uint8_t *outbuf, int buf_size, /* page_version = 0 + page_state */ *q++ = (s->object_version << 4) | (page_state << 2) | 3; - for (region_id = 0; region_id < h->num_rects; region_id++) { + for (region_id = 0; region_id < frame->num_subtitle_areas; region_id++) { *q++ = region_id; *q++ = 0xff; /* reserved */ - bytestream_put_be16(&q, h->rects[region_id]->x); /* left pos */ - bytestream_put_be16(&q, h->rects[region_id]->y); /* top pos */ + bytestream_put_be16(&q, frame->subtitle_areas[region_id]->x); /* left pos */ + bytestream_put_be16(&q, frame->subtitle_areas[region_id]->y); /* top pos */ } bytestream_put_be16(&pseg_len, q - pseg_len - 2); - buf_size -= 8 + h->num_rects * 6; + buf_size -= 8 + frame->num_subtitle_areas * 6; - if (h->num_rects) { - for (clut_id = 0; clut_id < h->num_rects; clut_id++) { - if (buf_size < 6 + h->rects[clut_id]->nb_colors * 6) + if (frame->num_subtitle_areas) { + for (clut_id = 0; clut_id < frame->num_subtitle_areas; clut_id++) { + if (buf_size < 6 + frame->subtitle_areas[clut_id]->nb_colors * 6) return AVERROR_BUFFER_TOO_SMALL; /* CLUT segment */ - if (h->rects[clut_id]->nb_colors <= 4) { + if (frame->subtitle_areas[clut_id]->nb_colors <= 4) { /* 2 bpp, some decoders do not support it correctly */ bpp_index = 0; - } else if (h->rects[clut_id]->nb_colors <= 16) { + } else if (frame->subtitle_areas[clut_id]->nb_colors <= 16) { /* 4 bpp, standard encoding */ bpp_index = 1; - } else if (h->rects[clut_id]->nb_colors <= 256) { + } else if (frame->subtitle_areas[clut_id]->nb_colors <= 256) { /* 8 bpp, standard encoding */ bpp_index = 2; } else { @@ -353,12 +362,12 @@ static int dvbsub_encode(AVCodecContext *avctx, uint8_t *outbuf, int buf_size, *q++ = clut_id; *q++ = (0 << 4) | 0xf; /* version = 0 */ - for(i = 0; i < h->rects[clut_id]->nb_colors; i++) { + for(i = 0; i < frame->subtitle_areas[clut_id]->nb_colors; i++) { *q++ = i; /* clut_entry_id */ *q++ = (1 << (7 - bpp_index)) | (0xf << 1) | 1; /* 2 bits/pixel full range */ { int a, r, g, b; - uint32_t x= ((uint32_t*)h->rects[clut_id]->data[1])[i]; + uint32_t x= ((uint32_t*)frame->subtitle_areas[clut_id]->pal)[i]; a = (x >> 24) & 0xff; r = (x >> 16) & 0xff; g = (x >> 8) & 0xff; @@ -372,22 +381,22 @@ static int dvbsub_encode(AVCodecContext *avctx, uint8_t *outbuf, int buf_size, } bytestream_put_be16(&pseg_len, q - pseg_len - 2); - buf_size -= 6 + h->rects[clut_id]->nb_colors * 6; + buf_size -= 6 + frame->subtitle_areas[clut_id]->nb_colors * 6; } - if (buf_size < h->num_rects * 22) + if (buf_size < frame->num_subtitle_areas * 22) return AVERROR_BUFFER_TOO_SMALL; - for (region_id = 0; region_id < h->num_rects; region_id++) { + for (region_id = 0; region_id < frame->num_subtitle_areas; region_id++) { /* region composition segment */ - if (h->rects[region_id]->nb_colors <= 4) { + if (frame->subtitle_areas[region_id]->nb_colors <= 4) { /* 2 bpp, some decoders do not support it correctly */ bpp_index = 0; - } else if (h->rects[region_id]->nb_colors <= 16) { + } else if (frame->subtitle_areas[region_id]->nb_colors <= 16) { /* 4 bpp, standard encoding */ bpp_index = 1; - } else if (h->rects[region_id]->nb_colors <= 256) { + } else if (frame->subtitle_areas[region_id]->nb_colors <= 256) { /* 8 bpp, standard encoding */ bpp_index = 2; } else { @@ -401,8 +410,8 @@ static int dvbsub_encode(AVCodecContext *avctx, uint8_t *outbuf, int buf_size, q += 2; /* segment length */ *q++ = region_id; *q++ = (s->object_version << 4) | (0 << 3) | 0x07; /* version , no fill */ - bytestream_put_be16(&q, h->rects[region_id]->w); /* region width */ - bytestream_put_be16(&q, h->rects[region_id]->h); /* region height */ + bytestream_put_be16(&q, frame->subtitle_areas[region_id]->w); /* region width */ + bytestream_put_be16(&q, frame->subtitle_areas[region_id]->h); /* region height */ *q++ = ((1 + bpp_index) << 5) | ((1 + bpp_index) << 2) | 0x03; *q++ = region_id; /* clut_id == region_id */ *q++ = 0; /* 8 bit fill colors */ @@ -416,9 +425,9 @@ static int dvbsub_encode(AVCodecContext *avctx, uint8_t *outbuf, int buf_size, bytestream_put_be16(&pseg_len, q - pseg_len - 2); } - buf_size -= h->num_rects * 22; + buf_size -= frame->num_subtitle_areas * 22; - for (object_id = 0; object_id < h->num_rects; object_id++) { + for (object_id = 0; object_id < frame->num_subtitle_areas; object_id++) { int (*dvb_encode_rle)(uint8_t **pq, int buf_size, const uint8_t *bitmap, int linesize, int w, int h); @@ -427,13 +436,13 @@ static int dvbsub_encode(AVCodecContext *avctx, uint8_t *outbuf, int buf_size, return AVERROR_BUFFER_TOO_SMALL; /* bpp_index maths */ - if (h->rects[object_id]->nb_colors <= 4) { + if (frame->subtitle_areas[object_id]->nb_colors <= 4) { /* 2 bpp, some decoders do not support it correctly */ dvb_encode_rle = dvb_encode_rle2; - } else if (h->rects[object_id]->nb_colors <= 16) { + } else if (frame->subtitle_areas[object_id]->nb_colors <= 16) { /* 4 bpp, standard encoding */ dvb_encode_rle = dvb_encode_rle4; - } else if (h->rects[object_id]->nb_colors <= 256) { + } else if (frame->subtitle_areas[object_id]->nb_colors <= 256) { /* 8 bpp, standard encoding */ dvb_encode_rle = dvb_encode_rle8; } else { @@ -463,19 +472,19 @@ static int dvbsub_encode(AVCodecContext *avctx, uint8_t *outbuf, int buf_size, top_ptr = q; ret = dvb_encode_rle(&q, buf_size, - h->rects[object_id]->data[0], - h->rects[object_id]->w * 2, - h->rects[object_id]->w, - h->rects[object_id]->h >> 1); + frame->subtitle_areas[object_id]->buf[0]->data, + frame->subtitle_areas[object_id]->w * 2, + frame->subtitle_areas[object_id]->w, + frame->subtitle_areas[object_id]->h >> 1); if (ret < 0) return ret; buf_size -= ret; bottom_ptr = q; ret = dvb_encode_rle(&q, buf_size, - h->rects[object_id]->data[0] + h->rects[object_id]->w, - h->rects[object_id]->w * 2, - h->rects[object_id]->w, - h->rects[object_id]->h >> 1); + frame->subtitle_areas[object_id]->buf[0]->data + frame->subtitle_areas[object_id]->w, + frame->subtitle_areas[object_id]->w * 2, + frame->subtitle_areas[object_id]->w, + frame->subtitle_areas[object_id]->h >> 1); if (ret < 0) return ret; buf_size -= ret; @@ -502,7 +511,10 @@ static int dvbsub_encode(AVCodecContext *avctx, uint8_t *outbuf, int buf_size, buf_size -= 6; s->object_version = (s->object_version + 1) & 0xf; - return q - outbuf; + avpkt->size = q - avpkt->data; + *got_packet = 1; + + return 0; } const AVCodec ff_dvbsub_encoder = { @@ -511,5 +523,5 @@ const AVCodec ff_dvbsub_encoder = { .type = AVMEDIA_TYPE_SUBTITLE, .id = AV_CODEC_ID_DVB_SUBTITLE, .priv_data_size = sizeof(DVBSubtitleContext), - .encode_sub = dvbsub_encode, + .encode2 = dvbsub_encode, }; diff --git a/libavcodec/dvdsubenc.c b/libavcodec/dvdsubenc.c index ff4fbed39d..4916410fc8 100644 --- a/libavcodec/dvdsubenc.c +++ b/libavcodec/dvdsubenc.c @@ -20,6 +20,7 @@ */ #include "avcodec.h" #include "bytestream.h" +#include "encode.h" #include "internal.h" #include "libavutil/avassert.h" #include "libavutil/bprint.h" @@ -114,15 +115,14 @@ static int color_distance(uint32_t a, uint32_t b) * Count colors used in a rectangle, quantizing alpha and grouping by * nearest global palette entry. */ -static void count_colors(AVCodecContext *avctx, unsigned hits[33], - const AVSubtitleRect *r) +static void count_colors(const AVCodecContext *avctx, unsigned hits[33], + const AVSubtitleArea *r) { DVDSubtitleContext *dvdc = avctx->priv_data; unsigned count[256] = { 0 }; - uint32_t *palette = (uint32_t *)r->data[1]; uint32_t color; int x, y, i, j, match, d, best_d, av_uninit(best_j); - uint8_t *p = r->data[0]; + uint8_t *p = r->buf[0]->data; for (y = 0; y < r->h; y++) { for (x = 0; x < r->w; x++) @@ -132,7 +132,7 @@ static void count_colors(AVCodecContext *avctx, unsigned hits[33], for (i = 0; i < 256; i++) { if (!count[i]) /* avoid useless search */ continue; - color = palette[i]; + color = r->pal[i]; /* 0: transparent, 1-16: semi-transparent, 17-33 opaque */ match = color < 0x33000000 ? 0 : color < 0xCC000000 ? 1 : 17; if (match) { @@ -232,13 +232,13 @@ static void build_color_map(AVCodecContext *avctx, int cmap[], } } -static void copy_rectangle(AVSubtitleRect *dst, AVSubtitleRect *src, int cmap[]) +static void copy_rectangle(AVSubtitleArea*dst, AVSubtitleArea *src, int cmap[]) { int x, y; uint8_t *p, *q; - p = src->data[0]; - q = dst->data[0] + (src->x - dst->x) + + p = src->buf[0]->data; + q = dst->buf[0]->data + (src->x - dst->x) + (src->y - dst->y) * dst->linesize[0]; for (y = 0; y < src->h; y++) { for (x = 0; x < src->w; x++) @@ -248,51 +248,56 @@ static void copy_rectangle(AVSubtitleRect *dst, AVSubtitleRect *src, int cmap[]) } } -static int encode_dvd_subtitles(AVCodecContext *avctx, - uint8_t *outbuf, int outbuf_size, - const AVSubtitle *h) +static int encode_dvd_subtitles(AVCodecContext* avctx, AVPacket* avpkt, + const AVFrame* frame, int* got_packet) { DVDSubtitleContext *dvdc = avctx->priv_data; uint8_t *q, *qq; int offset1, offset2; - int i, rects = h->num_rects, ret; + int ret = 0; + unsigned i, rects = frame->num_subtitle_areas; unsigned global_palette_hits[33] = { 0 }; int cmap[256]; int out_palette[4]; int out_alpha[4]; - AVSubtitleRect vrect; - uint8_t *vrect_data = NULL; + AVSubtitleArea vrect; + uint8_t* vrect_data = NULL, *outbuf; int x2, y2; int forced = 0; + int outbuf_size; - if (rects == 0 || !h->rects) + if (rects == 0) + return 0; + + if (!frame->subtitle_areas) return AVERROR(EINVAL); + for (i = 0; i < rects; i++) - if (h->rects[i]->type != SUBTITLE_BITMAP) { + if (frame->subtitle_areas[i]->type != SUBTITLE_BITMAP) { av_log(avctx, AV_LOG_ERROR, "Bitmap subtitle required\n"); return AVERROR(EINVAL); } /* Mark this subtitle forced if any of the rectangles is forced. */ for (i = 0; i < rects; i++) - if ((h->rects[i]->flags & AV_SUBTITLE_FLAG_FORCED) != 0) { + if ((frame->subtitle_areas[i]->flags & AV_SUBTITLE_FLAG_FORCED) != 0) { forced = 1; break; } - vrect = *h->rects[0]; + vrect = *frame->subtitle_areas[0]; if (rects > 1) { /* DVD subtitles can have only one rectangle: build a virtual rectangle containing all actual rectangles. The data of the rectangles will be copied later, when the palette is decided, because the rectangles may have different palettes. */ - int xmin = h->rects[0]->x, xmax = xmin + h->rects[0]->w; - int ymin = h->rects[0]->y, ymax = ymin + h->rects[0]->h; + int xmin = frame->subtitle_areas[0]->x, xmax = xmin + frame->subtitle_areas[0]->w; + int ymin = frame->subtitle_areas[0]->y, ymax = ymin + frame->subtitle_areas[0]->h; for (i = 1; i < rects; i++) { - xmin = FFMIN(xmin, h->rects[i]->x); - ymin = FFMIN(ymin, h->rects[i]->y); - xmax = FFMAX(xmax, h->rects[i]->x + h->rects[i]->w); - ymax = FFMAX(ymax, h->rects[i]->y + h->rects[i]->h); + xmin = FFMIN(xmin, frame->subtitle_areas[i]->x); + ymin = FFMIN(ymin, frame->subtitle_areas[i]->y); + xmax = FFMAX(xmax, frame->subtitle_areas[i]->x + frame->subtitle_areas[i]->w); + ymax = FFMAX(ymax, frame->subtitle_areas[i]->y + frame->subtitle_areas[i]->h); } vrect.x = xmin; vrect.y = ymin; @@ -304,27 +309,29 @@ static int encode_dvd_subtitles(AVCodecContext *avctx, /* Count pixels outside the virtual rectangle as transparent */ global_palette_hits[0] = vrect.w * vrect.h; for (i = 0; i < rects; i++) - global_palette_hits[0] -= h->rects[i]->w * h->rects[i]->h; + global_palette_hits[0] -= frame->subtitle_areas[i]->w * frame->subtitle_areas[i]->h; } for (i = 0; i < rects; i++) - count_colors(avctx, global_palette_hits, h->rects[i]); + count_colors(avctx, global_palette_hits, frame->subtitle_areas[i]); select_palette(avctx, out_palette, out_alpha, global_palette_hits); if (rects > 1) { - if (!(vrect_data = av_calloc(vrect.w, vrect.h))) + + vrect.buf[0] = av_buffer_allocz((size_t)vrect.w * vrect.h); + if (!vrect.buf[0]) return AVERROR(ENOMEM); - vrect.data [0] = vrect_data; + vrect.linesize[0] = vrect.w; for (i = 0; i < rects; i++) { - build_color_map(avctx, cmap, (uint32_t *)h->rects[i]->data[1], + build_color_map(avctx, cmap, frame->subtitle_areas[i]->pal, out_palette, out_alpha); - copy_rectangle(&vrect, h->rects[i], cmap); + copy_rectangle(&vrect, frame->subtitle_areas[i], cmap); } for (i = 0; i < 4; i++) cmap[i] = i; } else { - build_color_map(avctx, cmap, (uint32_t *)h->rects[0]->data[1], + build_color_map(avctx, cmap, frame->subtitle_areas[0]->pal, out_palette, out_alpha); } @@ -335,6 +342,16 @@ static int encode_dvd_subtitles(AVCodecContext *avctx, out_palette[i], out_alpha[i] >> 4); av_log(avctx, AV_LOG_DEBUG, "\n"); + + ret = ff_get_encode_buffer(avctx, avpkt, 4 + vrect.w * vrect.h / 2 + 17 + 21, 0); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Error getting output packet.\n"); + return ret; + } + + outbuf_size = avpkt->size; + outbuf = avpkt->data; + // encode data block q = outbuf + 4; offset1 = q - outbuf; @@ -344,10 +361,10 @@ static int encode_dvd_subtitles(AVCodecContext *avctx, ret = AVERROR_BUFFER_TOO_SMALL; goto fail; } - dvd_encode_rle(&q, vrect.data[0], vrect.w * 2, + dvd_encode_rle(&q, vrect.buf[0]->data, vrect.w * 2, vrect.w, (vrect.h + 1) >> 1, cmap); offset2 = q - outbuf; - dvd_encode_rle(&q, vrect.data[0] + vrect.w, vrect.w * 2, + dvd_encode_rle(&q, vrect.buf[0]->data + vrect.w, vrect.w * 2, vrect.w, vrect.h >> 1, cmap); if (dvdc->even_rows_fix && (vrect.h & 1)) { @@ -362,7 +379,7 @@ static int encode_dvd_subtitles(AVCodecContext *avctx, bytestream_put_be16(&qq, q - outbuf); // send start display command - bytestream_put_be16(&q, (h->start_display_time*90) >> 10); + bytestream_put_be16(&q, (frame->subtitle_start_time * 90) >> 10); bytestream_put_be16(&q, (q - outbuf) /*- 2 */ + 8 + 12 + 2); *q++ = 0x03; // palette - 4 nibbles *q++ = (out_palette[3] << 4) | out_palette[2]; @@ -394,7 +411,7 @@ static int encode_dvd_subtitles(AVCodecContext *avctx, *q++ = 0xff; // terminating command // send stop display command last - bytestream_put_be16(&q, (h->end_display_time*90) >> 10); + bytestream_put_be16(&q, (frame->subtitle_end_time*90) >> 10); bytestream_put_be16(&q, (q - outbuf) - 2 /*+ 4*/); *q++ = 0x02; // set end *q++ = 0xff; // terminating command @@ -403,7 +420,9 @@ static int encode_dvd_subtitles(AVCodecContext *avctx, bytestream_put_be16(&qq, q - outbuf); av_log(NULL, AV_LOG_DEBUG, "subtitle_packet size=%"PTRDIFF_SPECIFIER"\n", q - outbuf); - ret = q - outbuf; + avpkt->size = q - outbuf; + ret = 0; + *got_packet = 1; fail: av_free(vrect_data); @@ -467,14 +486,13 @@ static int dvdsub_init(AVCodecContext *avctx) return 0; } -static int dvdsub_encode(AVCodecContext *avctx, - unsigned char *buf, int buf_size, - const AVSubtitle *sub) +static int dvdsub_encode(struct AVCodecContext* avctx, struct AVPacket* avpkt, + const struct AVFrame* frame, int* got_packet) { //DVDSubtitleContext *s = avctx->priv_data; int ret; - ret = encode_dvd_subtitles(avctx, buf, buf_size, sub); + ret = encode_dvd_subtitles(avctx, avpkt, frame, got_packet); return ret; } @@ -499,7 +517,7 @@ const AVCodec ff_dvdsub_encoder = { .type = AVMEDIA_TYPE_SUBTITLE, .id = AV_CODEC_ID_DVD_SUBTITLE, .init = dvdsub_init, - .encode_sub = dvdsub_encode, + .encode2 = dvdsub_encode, .priv_class = &dvdsubenc_class, .priv_data_size = sizeof(DVDSubtitleContext), .caps_internal = FF_CODEC_CAP_INIT_THREADSAFE, diff --git a/libavcodec/encode.c b/libavcodec/encode.c index dd25cf999b..8c915cab1d 100644 --- a/libavcodec/encode.c +++ b/libavcodec/encode.c @@ -140,17 +140,76 @@ fail: return ret; } +/** + * \brief + * \param avctx + * \param buf q + * \param buf_size + * \param sub + * \return + */ int avcodec_encode_subtitle(AVCodecContext *avctx, uint8_t *buf, int buf_size, const AVSubtitle *sub) { - int ret; + int ret = 0, got_packet = 0; + AVFrame *frame = NULL; + AVPacket* avpkt = NULL; + if (sub->start_display_time) { av_log(avctx, AV_LOG_ERROR, "start_display_time must be 0.\n"); return -1; } - ret = avctx->codec->encode_sub(avctx, buf, buf_size, sub); + memset(buf, 0, buf_size); + // Create a temporary frame for calling the regular api: + frame = av_frame_alloc(); + if (!frame) { + ret = AVERROR(ENOMEM); + goto exit; + } + + frame->format = sub->format; + frame->type = AVMEDIA_TYPE_SUBTITLE; + ret = av_frame_get_buffer2(frame, 0); + if (ret < 0) + goto exit; + + // Create a temporary packet + avpkt = av_packet_alloc(); + if (!avpkt) { + ret = AVERROR(ENOMEM); + goto exit; + } + + // Copy legacy subtitle data to temp frame + ret = ff_frame_put_subtitle(frame, sub); + if (ret < 0) + goto exit; + + if (frame->subtitle_start_time) { + av_log(avctx, AV_LOG_ERROR, "start_display_time must be 0.\n"); + ret = AVERROR_INVALIDDATA; + goto exit; + } + + ret = avctx->codec->encode2(avctx, avpkt, frame, &got_packet); + avctx->frame_number++; + + if (got_packet) { + if (avpkt->size > buf_size) { + ret = AVERROR_BUFFER_TOO_SMALL; + goto exit; + } + + memcpy(buf, avpkt->data, avpkt->size); + ret = avpkt->size; + } + +exit: + + av_packet_free(&avpkt); + av_frame_free(&frame); return ret; } diff --git a/libavcodec/movtextenc.c b/libavcodec/movtextenc.c index d506ed5c37..97a214b371 100644 --- a/libavcodec/movtextenc.c +++ b/libavcodec/movtextenc.c @@ -29,6 +29,7 @@ #include "libavutil/ass_split_internal.h" #include "libavutil/ass_internal.h" #include "bytestream.h" +#include "encode.h" #include "internal.h" #define STYLE_FLAG_BOLD (1<<0) @@ -73,6 +74,7 @@ typedef struct { AVCodecContext *avctx; ASSSplitContext *ass_ctx; + int is_default_ass_context; ASSStyle *ass_dialog_style; StyleBox *style_attributes; unsigned count; @@ -329,7 +331,7 @@ static av_cold int mov_text_encode_init(AVCodecContext *avctx) av_bprint_init(&s->buffer, 0, AV_BPRINT_SIZE_UNLIMITED); - s->ass_ctx = avpriv_ass_split(avctx->subtitle_header); + s->ass_ctx = avpriv_ass_split((char*)avctx->subtitle_header); if (!s->ass_ctx) return AVERROR_INVALIDDATA; ret = encode_sample_description(avctx); @@ -633,56 +635,108 @@ static const ASSCodesCallbacks mov_text_callbacks = { .end = mov_text_end_cb, }; -static int mov_text_encode_frame(AVCodecContext *avctx, unsigned char *buf, - int bufsize, const AVSubtitle *sub) +static void ensure_ass_context(AVCodecContext* avctx, const AVFrame* frame) +{ + MovTextContext* s = avctx->priv_data; + int ret; + + if (s->ass_ctx && !s->is_default_ass_context) + // We already have a (non-default context) + return; + + if (!frame->num_subtitle_areas) + // Don't need ass context for processing empty subtitle frames + return; + + // The frame has content, so we need to set up a context + if (frame->subtitle_header && frame->subtitle_header->size > 0) { + const char* subtitle_header = (char*)frame->subtitle_header->data; + avpriv_ass_split_free(s->ass_ctx); + s->ass_ctx = avpriv_ass_split(subtitle_header); + s->is_default_ass_context = 0; + } + else if (!s->ass_ctx) { + char* subtitle_header = avpriv_ass_get_subtitle_header_default(0); + if (!subtitle_header) + return; + + s->ass_ctx = avpriv_ass_split(subtitle_header); + s->is_default_ass_context = 1; + av_free(subtitle_header); + } + + if (s->ass_ctx && !avctx->extradata_size) { + ret = encode_sample_description(avctx); + if (ret < 0) + av_log(avctx, AV_LOG_ERROR, "Error during encode_sample_description().\n"); + } +} + +static int mov_text_encode_frame(AVCodecContext *avctx, AVPacket *avpkt, + const AVFrame *frame, int *got_packet) { MovTextContext *s = avctx->priv_data; ASSDialog *dialog; - int i, length; + int i, ret = 0; + size_t j; + uint8_t* buf; + + ensure_ass_context(avctx, frame); + s->text_pos = 0; s->count = 0; s->box_flags = 0; av_bprint_clear(&s->buffer); - for (i = 0; i < sub->num_rects; i++) { - const char *ass = sub->rects[i]->ass; + for (i = 0; i < frame->num_subtitle_areas; i++) { + const char *ass = frame->subtitle_areas[i]->ass; - if (sub->rects[i]->type != SUBTITLE_ASS) { - av_log(avctx, AV_LOG_ERROR, "Only SUBTITLE_ASS type supported.\n"); + if (frame->subtitle_areas[i]->type != AV_SUBTITLE_FMT_ASS) { + av_log(avctx, AV_LOG_ERROR, "Only AV_SUBTITLE_FMT_ASS type supported.\n"); return AVERROR(EINVAL); } - dialog = avpriv_ass_split_dialog(s->ass_ctx, ass); - if (!dialog) - return AVERROR(ENOMEM); - mov_text_dialog(s, dialog); - avpriv_ass_split_override_codes(&mov_text_callbacks, s, dialog->text); - avpriv_ass_free_dialog(&dialog); + if (ass) { + dialog = avpriv_ass_split_dialog(s->ass_ctx, ass); + if (!dialog) + return AVERROR(ENOMEM); + mov_text_dialog(s, dialog); + avpriv_ass_split_override_codes(&mov_text_callbacks, s, dialog->text); + avpriv_ass_free_dialog(&dialog); + + for (j = 0; j < box_count; j++) { + box_types[j].encode(s); + } + } } - if (s->buffer.len > UINT16_MAX) - return AVERROR(ERANGE); - AV_WB16(buf, s->buffer.len); - buf += 2; + if (!av_bprint_is_complete(&s->buffer)) { + return AVERROR(ENOMEM); + } - for (size_t j = 0; j < box_count; j++) - box_types[j].encode(s); + ret = ff_get_encode_buffer(avctx, avpkt, s->buffer.len + 3, 0); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Error getting output packet.\n"); + return ret; + } - if (!av_bprint_is_complete(&s->buffer)) - return AVERROR(ENOMEM); + buf = avpkt->data; - if (!s->buffer.len) - return 0; + AV_WB16(buf, s->buffer.len); + buf += 2; - if (s->buffer.len > bufsize - 3) { + if (s->buffer.len > avpkt->size - 3) { av_log(avctx, AV_LOG_ERROR, "Buffer too small for ASS event.\n"); - return AVERROR_BUFFER_TOO_SMALL; + ret = AVERROR_BUFFER_TOO_SMALL; + goto exit; } memcpy(buf, s->buffer.str, s->buffer.len); - length = s->buffer.len + 2; + avpkt->size = s->buffer.len + 2; + *got_packet = 1; - return length; +exit: + return ret; } #define OFFSET(x) offsetof(MovTextContext, x) @@ -707,7 +761,7 @@ const AVCodec ff_movtext_encoder = { .priv_data_size = sizeof(MovTextContext), .priv_class = &mov_text_encoder_class, .init = mov_text_encode_init, - .encode_sub = mov_text_encode_frame, + .encode2 = mov_text_encode_frame, .close = mov_text_encode_close, .caps_internal = FF_CODEC_CAP_INIT_THREADSAFE | FF_CODEC_CAP_INIT_CLEANUP, }; diff --git a/libavcodec/srtenc.c b/libavcodec/srtenc.c index a7c5fccefe..9b661da54d 100644 --- a/libavcodec/srtenc.c +++ b/libavcodec/srtenc.c @@ -21,6 +21,7 @@ #include #include "avcodec.h" +#include "encode.h" #include "libavutil/avstring.h" #include "libavutil/bprint.h" #include "libavutil/ass_split_internal.h" @@ -33,6 +34,7 @@ typedef struct { AVCodecContext *avctx; ASSSplitContext *ass_ctx; + int is_default_ass_context; AVBPrint buffer; char stack[SRT_STACK_SIZE]; int stack_ptr; @@ -130,14 +132,13 @@ static void srt_style_apply(SRTContext *s, const char *style) } } - static av_cold int srt_encode_init(AVCodecContext *avctx) { SRTContext *s = avctx->priv_data; s->avctx = avctx; - s->ass_ctx = avpriv_ass_split(avctx->subtitle_header); + s->ass_ctx = avpriv_ass_split((char *)avctx->subtitle_header); av_bprint_init(&s->buffer, 0, AV_BPRINT_SIZE_UNLIMITED); - return s->ass_ctx ? 0 : AVERROR_INVALIDDATA; + return 0; } static void srt_text_cb(void *priv, const char *text, int len) @@ -227,58 +228,91 @@ static const ASSCodesCallbacks text_callbacks = { .new_line = srt_new_line_cb, }; -static int encode_frame(AVCodecContext *avctx, - unsigned char *buf, int bufsize, const AVSubtitle *sub, - const ASSCodesCallbacks *cb) +static void ensure_ass_context(SRTContext* s, const AVFrame* frame) +{ + if (s->ass_ctx && !s->is_default_ass_context) + // We already have a (non-default context) + return; + + if (!frame->num_subtitle_areas) + // Don't need ass context for processing empty subtitle frames + return; + + // The frame has content, so we need to set up a context + if (frame->subtitle_header && frame->subtitle_header->size > 0) { + const char* subtitle_header = (char*)frame->subtitle_header->data; + avpriv_ass_split_free(s->ass_ctx); + s->ass_ctx = avpriv_ass_split(subtitle_header); + s->is_default_ass_context = 0; + } + else if (!s->ass_ctx) { + char* subtitle_header = avpriv_ass_get_subtitle_header_default(0); + if (!subtitle_header) + return; + + s->ass_ctx = avpriv_ass_split(subtitle_header); + s->is_default_ass_context = 1; + av_free(subtitle_header); + } +} + +static int encode_frame(AVCodecContext* avctx, AVPacket* avpkt, + const AVFrame* frame, int* got_packet, const ASSCodesCallbacks* cb) { SRTContext *s = avctx->priv_data; ASSDialog *dialog; - int i; + int i, ret; + + ensure_ass_context(s, frame); av_bprint_clear(&s->buffer); - for (i=0; inum_rects; i++) { - const char *ass = sub->rects[i]->ass; + for (i=0; i< frame->num_subtitle_areas; i++) { + const char *ass = frame->subtitle_areas[i]->ass; - if (sub->rects[i]->type != SUBTITLE_ASS) { - av_log(avctx, AV_LOG_ERROR, "Only SUBTITLE_ASS type supported.\n"); + if (frame->subtitle_areas[i]->type != AV_SUBTITLE_FMT_ASS) { + av_log(avctx, AV_LOG_ERROR, "Only AV_SUBTITLE_FMT_ASS type supported.\n"); return AVERROR(EINVAL); } - dialog = avpriv_ass_split_dialog(s->ass_ctx, ass); - if (!dialog) - return AVERROR(ENOMEM); - s->alignment_applied = 0; - if (avctx->codec_id == AV_CODEC_ID_SUBRIP) - srt_style_apply(s, dialog->style); - avpriv_ass_split_override_codes(cb, s, dialog->text); - avpriv_ass_free_dialog(&dialog); + if (ass) { + dialog = avpriv_ass_split_dialog(s->ass_ctx, ass); + if (!dialog) + return AVERROR(ENOMEM); + s->alignment_applied = 0; + if (avctx->codec_id == AV_CODEC_ID_SUBRIP) + srt_style_apply(s, dialog->style); + avpriv_ass_split_override_codes(cb, s, dialog->text); + avpriv_ass_free_dialog(&dialog); + } } if (!av_bprint_is_complete(&s->buffer)) return AVERROR(ENOMEM); - if (!s->buffer.len) - return 0; - if (s->buffer.len > bufsize) { - av_log(avctx, AV_LOG_ERROR, "Buffer too small for ASS event.\n"); - return AVERROR_BUFFER_TOO_SMALL; + ret = ff_get_encode_buffer(avctx, avpkt, s->buffer.len + 1, 0); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Error getting output packet.\n"); + return ret; } - memcpy(buf, s->buffer.str, s->buffer.len); - return s->buffer.len; + memcpy(avpkt->data, s->buffer.str, s->buffer.len); + avpkt->size = s->buffer.len; + *got_packet = 1; + + return 0; } -static int srt_encode_frame(AVCodecContext *avctx, - unsigned char *buf, int bufsize, const AVSubtitle *sub) +static int srt_encode_frame(AVCodecContext* avctx, AVPacket* avpkt, + const AVFrame* frame, int* got_packet) { - return encode_frame(avctx, buf, bufsize, sub, &srt_callbacks); + return encode_frame(avctx, avpkt, frame, got_packet, &srt_callbacks); } -static int text_encode_frame(AVCodecContext *avctx, - unsigned char *buf, int bufsize, const AVSubtitle *sub) +static int text_encode_frame(AVCodecContext* avctx, AVPacket* avpkt, + const AVFrame* frame, int* got_packet) { - return encode_frame(avctx, buf, bufsize, sub, &text_callbacks); + return encode_frame(avctx, avpkt, frame, got_packet, &text_callbacks); } static int srt_encode_close(AVCodecContext *avctx) @@ -298,7 +332,7 @@ const AVCodec ff_srt_encoder = { .id = AV_CODEC_ID_SUBRIP, .priv_data_size = sizeof(SRTContext), .init = srt_encode_init, - .encode_sub = srt_encode_frame, + .encode2 = srt_encode_frame, .close = srt_encode_close, .caps_internal = FF_CODEC_CAP_INIT_THREADSAFE, }; @@ -312,7 +346,7 @@ const AVCodec ff_subrip_encoder = { .id = AV_CODEC_ID_SUBRIP, .priv_data_size = sizeof(SRTContext), .init = srt_encode_init, - .encode_sub = srt_encode_frame, + .encode2 = srt_encode_frame, .close = srt_encode_close, .caps_internal = FF_CODEC_CAP_INIT_THREADSAFE, }; @@ -326,7 +360,7 @@ const AVCodec ff_text_encoder = { .id = AV_CODEC_ID_TEXT, .priv_data_size = sizeof(SRTContext), .init = srt_encode_init, - .encode_sub = text_encode_frame, + .encode2 = text_encode_frame, .close = srt_encode_close, .caps_internal = FF_CODEC_CAP_INIT_THREADSAFE, }; diff --git a/libavcodec/tests/avcodec.c b/libavcodec/tests/avcodec.c index 5d0ff9432c..bd979b2184 100644 --- a/libavcodec/tests/avcodec.c +++ b/libavcodec/tests/avcodec.c @@ -107,8 +107,6 @@ int main(void){ continue; } if (is_encoder) { - if (codec->type == AVMEDIA_TYPE_SUBTITLE ^ !!codec->encode_sub) - ERR("Encoder %s is both subtitle encoder and not subtitle encoder."); if (!!codec->encode_sub + !!codec->encode2 + !!codec->receive_packet != 1) ERR("Encoder %s does not implement exactly one encode API.\n"); if (codec->update_thread_context || codec->update_thread_context_for_user || codec->bsfs) diff --git a/libavcodec/ttmlenc.c b/libavcodec/ttmlenc.c index 083f2dd67a..011b7aa753 100644 --- a/libavcodec/ttmlenc.c +++ b/libavcodec/ttmlenc.c @@ -33,11 +33,17 @@ #include "libavutil/bprint.h" #include "libavutil/internal.h" #include "libavutil/ass_split_internal.h" +#include "libavutil/ass_internal.h" #include "ttmlenc.h" +#include "encode.h" + + typedef struct { AVCodecContext *avctx; ASSSplitContext *ass_ctx; + int is_default_ass_context; + int extradata_written; AVBPrint buffer; } TTMLContext; @@ -76,24 +82,68 @@ static const ASSCodesCallbacks ttml_callbacks = { .new_line = ttml_new_line_cb, }; -static int ttml_encode_frame(AVCodecContext *avctx, uint8_t *buf, - int bufsize, const AVSubtitle *sub) +static int ttml_write_header_content(AVCodecContext* avctx); + +static void ensure_ass_context(AVCodecContext* avctx, const AVFrame* frame) +{ + TTMLContext* s = avctx->priv_data; + int ret; + + if (s->ass_ctx && !s->is_default_ass_context) + // We already have a (non-default context) + return; + + if (!frame->num_subtitle_areas) + // Don't need ass context for processing empty subtitle frames + return; + + // The frame has content, so we need to set up a context + if (frame->subtitle_header && frame->subtitle_header->size > 0) { + const char* subtitle_header = (char*)frame->subtitle_header->data; + avpriv_ass_split_free(s->ass_ctx); + s->ass_ctx = avpriv_ass_split(subtitle_header); + s->is_default_ass_context = 0; + } + else if (!s->ass_ctx) { + char* subtitle_header = avpriv_ass_get_subtitle_header_default(0); + if (!subtitle_header) + return; + + s->ass_ctx = avpriv_ass_split(subtitle_header); + s->is_default_ass_context = 1; + av_free(subtitle_header); + } + + if (s->ass_ctx && !s->extradata_written) { + s->extradata_written = 1; + if ((ret = ttml_write_header_content(avctx)) < 0) { + av_log(avctx, AV_LOG_ERROR, "Error writing header content.\n"); + } + } +} + +static int ttml_encode_frame(AVCodecContext* avctx, AVPacket* avpkt, + const AVFrame* frame, int* got_packet) { TTMLContext *s = avctx->priv_data; ASSDialog *dialog; - int i; + int i, ret; + + ensure_ass_context(avctx, frame); av_bprint_clear(&s->buffer); - for (i=0; inum_rects; i++) { - const char *ass = sub->rects[i]->ass; - int ret; + for (i=0; i< frame->num_subtitle_areas; i++) { + const char *ass = frame->subtitle_areas[i]->ass; - if (sub->rects[i]->type != SUBTITLE_ASS) { - av_log(avctx, AV_LOG_ERROR, "Only SUBTITLE_ASS type supported.\n"); + if (frame->subtitle_areas[i]->type != AV_SUBTITLE_FMT_ASS) { + av_log(avctx, AV_LOG_ERROR, "Only AV_SUBTITLE_FMT_ASS type supported.\n"); return AVERROR(EINVAL); } + if (!ass) + continue; + dialog = avpriv_ass_split_dialog(s->ass_ctx, ass); if (!dialog) return AVERROR(ENOMEM); @@ -130,17 +180,19 @@ static int ttml_encode_frame(AVCodecContext *avctx, uint8_t *buf, if (!av_bprint_is_complete(&s->buffer)) return AVERROR(ENOMEM); - if (!s->buffer.len) - return 0; - - // force null-termination, so in case our destination buffer is - // too small, the return value is larger than bufsize minus null. - if (av_strlcpy(buf, s->buffer.str, bufsize) > bufsize - 1) { - av_log(avctx, AV_LOG_ERROR, "Buffer too small for TTML event.\n"); - return AVERROR_BUFFER_TOO_SMALL; + + ret = ff_get_encode_buffer(avctx, avpkt, s->buffer.len + 3, 0); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Error getting output packet.\n"); + return ret; } - return s->buffer.len; + av_strlcpy(avpkt->data, s->buffer.str, avpkt->size); + + avpkt->size = s->buffer.len; + *got_packet = 1; + + return 0; } static av_cold int ttml_encode_close(AVCodecContext *avctx) @@ -370,13 +422,13 @@ static av_cold int ttml_encode_init(AVCodecContext *avctx) s->avctx = avctx; av_bprint_init(&s->buffer, 0, AV_BPRINT_SIZE_UNLIMITED); + s->ass_ctx = avpriv_ass_split((char*)avctx->subtitle_header); - if (!(s->ass_ctx = avpriv_ass_split(avctx->subtitle_header))) { - return AVERROR_INVALIDDATA; - } + if (s->ass_ctx) { + if (ret = ttml_write_header_content(avctx) < 0) + return ret; - if ((ret = ttml_write_header_content(avctx)) < 0) { - return ret; + s->extradata_written = 1; } return 0; @@ -389,7 +441,7 @@ const AVCodec ff_ttml_encoder = { .id = AV_CODEC_ID_TTML, .priv_data_size = sizeof(TTMLContext), .init = ttml_encode_init, - .encode_sub = ttml_encode_frame, + .encode2 = ttml_encode_frame, .close = ttml_encode_close, .caps_internal = FF_CODEC_CAP_INIT_THREADSAFE | FF_CODEC_CAP_INIT_CLEANUP, }; diff --git a/libavcodec/webvttenc.c b/libavcodec/webvttenc.c index 761099b69a..43dace6575 100644 --- a/libavcodec/webvttenc.c +++ b/libavcodec/webvttenc.c @@ -22,6 +22,7 @@ #include #include "avcodec.h" +#include "encode.h" #include "libavutil/avstring.h" #include "libavutil/bprint.h" #include "libavutil/ass_split_internal.h" @@ -32,6 +33,7 @@ typedef struct { AVCodecContext *avctx; ASSSplitContext *ass_ctx; + int is_default_ass_context; AVBPrint buffer; unsigned timestamp_end; int count; @@ -155,43 +157,77 @@ static const ASSCodesCallbacks webvtt_callbacks = { .end = webvtt_end_cb, }; -static int webvtt_encode_frame(AVCodecContext *avctx, - unsigned char *buf, int bufsize, const AVSubtitle *sub) +static void ensure_ass_context(WebVTTContext* s, const AVFrame* frame) +{ + if (s->ass_ctx && !s->is_default_ass_context) + // We already have a (non-default context) + return; + + if (!frame->num_subtitle_areas) + // Don't need ass context for processing empty subtitle frames + return; + + // The frame has content, so we need to set up a context + if (frame->subtitle_header && frame->subtitle_header->size > 0) { + avpriv_ass_split_free(s->ass_ctx); + const char* subtitle_header = (char*)frame->subtitle_header->data; + s->ass_ctx = avpriv_ass_split(subtitle_header); + s->is_default_ass_context = 0; + } + else if (!s->ass_ctx) { + char* subtitle_header = avpriv_ass_get_subtitle_header_default(0); + if (!subtitle_header) + return; + + s->ass_ctx = avpriv_ass_split(subtitle_header); + s->is_default_ass_context = 1; + av_free(subtitle_header); + } +} + +static int webvtt_encode_frame(AVCodecContext* avctx, AVPacket* avpkt, + const AVFrame* frame, int* got_packet) { WebVTTContext *s = avctx->priv_data; ASSDialog *dialog; - int i; + int ret, i; + + ensure_ass_context(s, frame); av_bprint_clear(&s->buffer); - for (i=0; inum_rects; i++) { - const char *ass = sub->rects[i]->ass; + for (i=0; i< frame->num_subtitle_areas; i++) { + const char *ass = frame->subtitle_areas[i]->ass; - if (sub->rects[i]->type != SUBTITLE_ASS) { - av_log(avctx, AV_LOG_ERROR, "Only SUBTITLE_ASS type supported.\n"); + if (frame->subtitle_areas[i]->type != AV_SUBTITLE_FMT_ASS) { + av_log(avctx, AV_LOG_ERROR, "Only AV_SUBTITLE_FMT_ASS type supported.\n"); return AVERROR(EINVAL); } - dialog = avpriv_ass_split_dialog(s->ass_ctx, ass); - if (!dialog) - return AVERROR(ENOMEM); - webvtt_style_apply(s, dialog->style); - avpriv_ass_split_override_codes(&webvtt_callbacks, s, dialog->text); - avpriv_ass_free_dialog(&dialog); + if (ass) { + dialog = avpriv_ass_split_dialog(s->ass_ctx, ass); + if (!dialog) + return AVERROR(ENOMEM); + webvtt_style_apply(s, dialog->style); + avpriv_ass_split_override_codes(&webvtt_callbacks, s, dialog->text); + avpriv_ass_free_dialog(&dialog); + } } if (!av_bprint_is_complete(&s->buffer)) return AVERROR(ENOMEM); - if (!s->buffer.len) - return 0; - if (s->buffer.len > bufsize) { - av_log(avctx, AV_LOG_ERROR, "Buffer too small for ASS event.\n"); - return AVERROR_BUFFER_TOO_SMALL; + ret = ff_get_encode_buffer(avctx, avpkt, s->buffer.len + 1, 0); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Error getting output packet.\n"); + return ret; } - memcpy(buf, s->buffer.str, s->buffer.len); - return s->buffer.len; + memcpy(avpkt->data, s->buffer.str, s->buffer.len); + avpkt->size = s->buffer.len; + *got_packet = s->buffer.len > 0; + + return 0; } static int webvtt_encode_close(AVCodecContext *avctx) @@ -206,9 +242,9 @@ static av_cold int webvtt_encode_init(AVCodecContext *avctx) { WebVTTContext *s = avctx->priv_data; s->avctx = avctx; - s->ass_ctx = avpriv_ass_split(avctx->subtitle_header); + s->ass_ctx = avpriv_ass_split((char*)avctx->subtitle_header); av_bprint_init(&s->buffer, 0, AV_BPRINT_SIZE_UNLIMITED); - return s->ass_ctx ? 0 : AVERROR_INVALIDDATA; + return 0; } const AVCodec ff_webvtt_encoder = { @@ -218,7 +254,7 @@ const AVCodec ff_webvtt_encoder = { .id = AV_CODEC_ID_WEBVTT, .priv_data_size = sizeof(WebVTTContext), .init = webvtt_encode_init, - .encode_sub = webvtt_encode_frame, + .encode2 = webvtt_encode_frame, .close = webvtt_encode_close, .caps_internal = FF_CODEC_CAP_INIT_THREADSAFE, }; diff --git a/libavcodec/xsubenc.c b/libavcodec/xsubenc.c index 03d0dc2d86..52fc7044da 100644 --- a/libavcodec/xsubenc.c +++ b/libavcodec/xsubenc.c @@ -22,6 +22,7 @@ #include "avcodec.h" #include "bytestream.h" +#include "encode.h" #include "internal.h" #include "put_bits.h" @@ -111,39 +112,55 @@ static int make_tc(uint64_t ms, int *tc) return ms > 99; } -static int xsub_encode(AVCodecContext *avctx, unsigned char *buf, - int bufsize, const AVSubtitle *h) +static int xsub_encode(AVCodecContext* avctx, AVPacket* avpkt, + const AVFrame* frame, int* got_packet) { - uint64_t startTime = h->pts / 1000; // FIXME: need better solution... - uint64_t endTime = startTime + h->end_display_time - h->start_display_time; + const uint64_t startTime = frame->subtitle_pts / 1000; // FIXME: need better solution... + const uint64_t endTime = startTime + frame->subtitle_end_time - frame->subtitle_start_time; int start_tc[4], end_tc[4]; - uint8_t *hdr = buf + 27; // Point behind the timestamp + uint8_t *hdr; uint8_t *rlelenptr; uint16_t width, height; - int i; + int i, ret; PutBitContext pb; + uint8_t* buf; + int64_t req_size; - if (bufsize < 27 + 7*2 + 4*3) { - av_log(avctx, AV_LOG_ERROR, "Buffer too small for XSUB header.\n"); - return AVERROR_BUFFER_TOO_SMALL; + if (!frame->num_subtitle_areas) { + // Don't encode empty sub events + return 0; } + // Estimate size (timestamp 27, header 7*2 + 4*3, padding 10) + req_size = 27 + 7*2 + 4*3 + 10; + req_size += frame->subtitle_areas[0]->linesize[0] * frame->subtitle_areas[0]->h; + req_size += 256; // Palette + + ret = ff_get_encode_buffer(avctx, avpkt, req_size, 0); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Error getting output packet.\n"); + return ret; + } + + buf = avpkt->data; + hdr = avpkt->data + 27; // Point behind the timestamp + // TODO: support multiple rects - if (h->num_rects != 1) - av_log(avctx, AV_LOG_WARNING, "Only single rects supported (%d in subtitle.)\n", h->num_rects); + if (frame->num_subtitle_areas != 1) + av_log(avctx, AV_LOG_WARNING, "Only single rects supported (%d in subtitle.)\n", frame->num_subtitle_areas); // TODO: render text-based subtitles into bitmaps - if (!h->rects[0]->data[0] || !h->rects[0]->data[1]) { + if (!frame->subtitle_areas[0]->buf[0]->data || !frame->subtitle_areas[0]->pal) { av_log(avctx, AV_LOG_WARNING, "No subtitle bitmap available.\n"); return AVERROR(EINVAL); } // TODO: color reduction, similar to dvdsub encoder - if (h->rects[0]->nb_colors > 4) - av_log(avctx, AV_LOG_WARNING, "No more than 4 subtitle colors supported (%d found.)\n", h->rects[0]->nb_colors); + if (frame->subtitle_areas[0]->nb_colors > 4) + av_log(avctx, AV_LOG_WARNING, "No more than 4 subtitle colors supported (%d found.)\n", frame->subtitle_areas[0]->nb_colors); // TODO: Palette swapping if color zero is not transparent - if (((uint32_t *)h->rects[0]->data[1])[0] & 0xff000000) + if (((uint32_t *)frame->subtitle_areas[0]->pal)[0] & 0xff000000) av_log(avctx, AV_LOG_WARNING, "Color index 0 is not transparent. Transparency will be messed up.\n"); if (make_tc(startTime, start_tc) || make_tc(endTime, end_tc)) { @@ -151,7 +168,7 @@ static int xsub_encode(AVCodecContext *avctx, unsigned char *buf, return AVERROR(EINVAL); } - snprintf(buf, 28, + snprintf((char *)avpkt->data, 28, "[%02d:%02d:%02d.%03d-%02d:%02d:%02d.%03d]", start_tc[3], start_tc[2], start_tc[1], start_tc[0], end_tc[3], end_tc[2], end_tc[1], end_tc[0]); @@ -160,45 +177,47 @@ static int xsub_encode(AVCodecContext *avctx, unsigned char *buf, // 2 pixels required on either side of subtitle. // Possibly due to limitations of hardware renderers. // TODO: check if the bitmap is already padded - width = FFALIGN(h->rects[0]->w, 2) + PADDING * 2; - height = FFALIGN(h->rects[0]->h, 2); + width = FFALIGN(frame->subtitle_areas[0]->w, 2) + PADDING * 2; + height = FFALIGN(frame->subtitle_areas[0]->h, 2); bytestream_put_le16(&hdr, width); bytestream_put_le16(&hdr, height); - bytestream_put_le16(&hdr, h->rects[0]->x); - bytestream_put_le16(&hdr, h->rects[0]->y); - bytestream_put_le16(&hdr, h->rects[0]->x + width -1); - bytestream_put_le16(&hdr, h->rects[0]->y + height -1); + bytestream_put_le16(&hdr, frame->subtitle_areas[0]->x); + bytestream_put_le16(&hdr, frame->subtitle_areas[0]->y); + bytestream_put_le16(&hdr, frame->subtitle_areas[0]->x + width -1); + bytestream_put_le16(&hdr, frame->subtitle_areas[0]->y + height -1); rlelenptr = hdr; // Will store length of first field here later. hdr+=2; // Palette for (i=0; i<4; i++) - bytestream_put_be24(&hdr, ((uint32_t *)h->rects[0]->data[1])[i]); + bytestream_put_be24(&hdr, ((uint32_t *)frame->subtitle_areas[0]->pal)[i]); // Bitmap // RLE buffer. Reserve 2 bytes for possible padding after the last row. - init_put_bits(&pb, hdr, bufsize - (hdr - buf) - 2); - if (xsub_encode_rle(&pb, h->rects[0]->data[0], - h->rects[0]->linesize[0] * 2, - h->rects[0]->w, (h->rects[0]->h + 1) >> 1)) + init_put_bits(&pb, hdr, avpkt->size - (hdr - buf) - 2); + if (xsub_encode_rle(&pb, frame->subtitle_areas[0]->buf[0]->data, + frame->subtitle_areas[0]->linesize[0] * 2, + frame->subtitle_areas[0]->w, (frame->subtitle_areas[0]->h + 1) >> 1)) return AVERROR_BUFFER_TOO_SMALL; bytestream_put_le16(&rlelenptr, put_bytes_count(&pb, 0)); // Length of first field - if (xsub_encode_rle(&pb, h->rects[0]->data[0] + h->rects[0]->linesize[0], - h->rects[0]->linesize[0] * 2, - h->rects[0]->w, h->rects[0]->h >> 1)) + if (xsub_encode_rle(&pb, frame->subtitle_areas[0]->buf[0]->data + frame->subtitle_areas[0]->linesize[0], + frame->subtitle_areas[0]->linesize[0] * 2, + frame->subtitle_areas[0]->w, frame->subtitle_areas[0]->h >> 1)) return AVERROR_BUFFER_TOO_SMALL; // Enforce total height to be a multiple of 2 - if (h->rects[0]->h & 1) { - put_xsub_rle(&pb, h->rects[0]->w, PADDING_COLOR); + if (frame->subtitle_areas[0]->h & 1) { + put_xsub_rle(&pb, frame->subtitle_areas[0]->w, PADDING_COLOR); } flush_put_bits(&pb); - return hdr - buf + put_bytes_output(&pb); + avpkt->size = hdr - buf + put_bytes_output(&pb); + *got_packet = 1; + return 0; } static av_cold int xsub_encoder_init(AVCodecContext *avctx) @@ -217,6 +236,6 @@ const AVCodec ff_xsub_encoder = { .type = AVMEDIA_TYPE_SUBTITLE, .id = AV_CODEC_ID_XSUB, .init = xsub_encoder_init, - .encode_sub = xsub_encode, + .encode2 = xsub_encode, .caps_internal = FF_CODEC_CAP_INIT_THREADSAFE, }; From patchwork Mon Dec 13 23:40:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 32458 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp6139734iog; Mon, 13 Dec 2021 15:41:43 -0800 (PST) X-Google-Smtp-Source: ABdhPJzntOZCt6kuvPmeNOAXvgmLOMeA1sazrlY1N40tHHmg1WMZqc62rjHLVe7KbEShrZADfg4o X-Received: by 2002:a17:906:3a4a:: with SMTP id a10mr1586652ejf.253.1639438903038; Mon, 13 Dec 2021 15:41:43 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id gm1si10342759ejc.26.2021.12.13.15.41.42; Mon, 13 Dec 2021 15:41:43 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=XY7D4V0R; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D0D9868B0B0; Tue, 14 Dec 2021 01:40:43 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12olkn2082.outbound.protection.outlook.com [40.92.21.82]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7C31268B06D for ; Tue, 14 Dec 2021 01:40:35 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=nJCRIQAUs65KaCZ/HJ4/q5xd60Jyycm1zFGeaTRjqLXEvSbKoJA2V+bJW3CFmVija0nBe+6MCOZgJiOGgpBozZ0ld1u0k7yywFx76GlvItKc60UxOYXlTYZ9wQmRkG+aI5kkHK+gRS8/0fz38YsFAn4GCoctH553iqMqU5JN7uIheiVY+9ZgdeR+m8AJrw1K4PXgIZc9unCw0pws3LXw4EXAXFEIfz2MfGCAUErWsoeVNBIQSFX7gX6BkELgCXUOW3KDg3tW3b4+Wethg+wyARt9VQDABZEcNmv+MJ1ZVkz8Ll+JjHBwKoT1eNjkR7xgYFSLUik4giv4Sjwov2TQXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=zABAXZwKS+fGKLetN9nU1YP+cWesDOAoqYEA0xTWjtA=; b=AGAwdMMwscv3WFG0O6epG0mo75eK4w3Mp2BIJR4bINjAY34Oppg/b6o9R8zmp4pFc9b7CycukS8hypV4oGn5hpyH/s+vbk2OWL4fz97uloxYF3e42fzZRNp9tbd2OQCgSA1CMM5s7CckrfLsF2EwsgK7DZRkWmvwTUbOXil48GZFBsr2xgkJYj1Ja3YBmRWlTwVQhUqQgB3iMdJdDgx/J8klOGXpUJwJvd+bqCBcZI5lQxtsoQupEgoTbWJK8FJIQs7qSlOIfzQoAQzHzYQ3pVsWsza7puIgNHD0+vqNcI7WTw2Ye6txKRy/txiwqMlHJ8BmX+GFTtlKif6kIjSGTg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zABAXZwKS+fGKLetN9nU1YP+cWesDOAoqYEA0xTWjtA=; b=XY7D4V0RcN+wBhRUaePs6vcbT0GVgRDA7ciTS0wldmUgMXRH5ti0F8NoEKZauM4DCsyZEASN9jVDwb9xmnDRIg+nVrPTEbCfZa3QgRwMMj2LyRqFMN3+1cI9oN2KcVOpPAxDByDHsZcAX4MIlcuH03oJXI5ZS2+UWarEs8vLFH7UjQ9pB9RnvidkkaBQiYepN40+hWCbFqSz4OnM3sr3x1DBWMj0T9jnJEtBaLi01LqUsfSlpHyIpGDAUh4heG0lgggR+v3anUrvqkdBO+tyzwv6exL0yRavQahxNkgP05h7L1vD3li55tAcrkJlsgwvkTA5edUg96rWPap42kyxag== Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::20) by DM8P223MB0397.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4778.17; Mon, 13 Dec 2021 23:40:33 +0000 Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775]) by DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775%7]) with mapi id 15.20.4778.018; Mon, 13 Dec 2021 23:40:33 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v24 07/21] avcodec/subtitles: Replace deprecated enum values Thread-Index: AQHX8HrSeRt0u+riXEeZZkV1Zpb//w== Date: Mon, 13 Dec 2021 23:40:33 +0000 Message-ID: References: <1ef42eb01b3627bbad539abd46206f320bb39697.1639438462.git.softworkz@hotmail.com> <4148d77a82d4b7e229fc81ce9bc1bc06904533f1.1639438462.git.softworkz@hotmail.com> <2ac798924ee984010d2e85f00bc992d9aa106171.1639438462.git.softworkz@hotmail.com> <300ea7188b331ca5e00dd7e04c03df8aba2a99c0.1639438462.git.softworkz@hotmail.com> <9474726b26f0d3e8f353d07caed2fbd33cdf473a.1639438462.git.softworkz@hotmail.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [JYZnl6Yn8cdPUSDRx+iJKZmRTfl+HNsj] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 9d50175f-a268-4d6c-7995-08d9be91f4d0 x-ms-traffictypediagnostic: DM8P223MB0397:EE_ x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: IWS03VzOBmLMmJ9sjcBW82W4l3ibS2G2QOItrn1/TGbp9qGRd9bQXCK2yp0tvd/lk5C2cCH5dPHENrFyQF0yKxRpQxiWTkP259ObrDcBHIkrgoNINghOH0UCYEqHCmYHw869TWrGHRK1do0lqvZPoE6xJBG4RPj/miyWpDWJWur8YNFkSmhxwoG0ucacRqJ7dO3WBP0AFNTM3P34qOXsHzQR00ytDqruDhtSQme5PdM3NosBGjRT7oXZB/QVH8HuhGYBYvXBsHb5sWbqpjcZ0h8cGiT2aqqkS0B9ZDFipQ2Q0M2e7Lacx1HTjhWi6ahOEQHomT606Y4Ed980RE06wom5wgh6/bIBdcj4J0x2wc71X3Snx7N3H1p0IXlryuQ/c+FSEhcIgk2P+QM5KwUkG+52XprOXZSjRkFMjdsckzlxaPNamx9IPLcAwD0vx+4+j2bbKayE61TQzGt4MmS2xqOZvO2WsLxmTNhGZAQmQyofMOwku3d6Dx+xRbQSlG7GK/IHhIZCrpMjD5pQ4CWEy2pDugGoAgEt1gzSvc+TkeSBP4Y5KzWndniH2EntzuMMFUfj86MZ7YA6ShIQJx9CtA== x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?q?Ly5i3mTf0H7jdPuQQoKp3FtM4Pnr?= =?utf-8?q?mc7JzR3g3frI0npjqwvX/fl3hkQLV21LTeKky97U77GiIrtGvZXC/q7RN3VUXdki8?= =?utf-8?q?CXybNbhuqEdNDSZDNZoB7yEvzIf02VH4WHgNqYutvoGjze1U/Ehw4TILQuUueZpiX?= =?utf-8?q?ViBAiw34n4DJ57s635Ax3/mhkyqMhPdNAecg77qBZSluFbZgoZUgWaNy8higue+1k?= =?utf-8?q?DlgSBfhbS/HdJj5IvI8tYnIw9cY89mroiLBbPMbgwX8zcBiIjVjv0ad8EHnFYcdKe?= =?utf-8?q?e+bcn6jdJQ2/YJ3M/fY1E+BfhgvXo+a62CUFU089PcUyAzukFtGgTzLEmaItJl3AJ?= =?utf-8?q?zoS3jj4YZ79v/7AauIbrOAV5vTXNdFdbRWmlUVEGHBDUajs/huRcr3rxazEO6b6t+?= =?utf-8?q?ju9Gfxoo4uqN6hx6dKX9hHzyWeccQWfwATDrPyexdyubOaeNRDEV/byvFO6+j2vX9?= =?utf-8?q?oGr2AKpzbWj6YJ2Nh8xr7aIHNLVKgyGuV5QOMs9l/pBQl+cwsSCyCphN7f0S4KfXI?= =?utf-8?q?pZqns/C7JcpIG+UUX1DaO0YJOylCxaEDFNU+AUhIpcXVbv1JV18He4gVkqlOzyMiR?= =?utf-8?q?9XRopFtwepL0eNN0ZR2b0AquPqnu3LuIZbaELrfV/fvttzP5+BpE8YHHzgHMRNo0M?= =?utf-8?q?Zl/D1xxD5zp9fvdBBDVNWDyYL7bbolum0+ilMMvOok0Zu35Z9ImlCT1RUebgrF8rR?= =?utf-8?q?CGfN6GrKddujikK33WCc8S13KciYlW56IAFriA3lw5ComvUUHiIBRAT0hJbIQoDY/?= =?utf-8?q?H3a+odfCH84QVzBUe4FB+OeJxCmqnA7BvDnuc2OzmgGNQhab/99De2TH+2GDK4n9+?= =?utf-8?q?zx3F/UBVAWo4/BylXvoTZgk/ZT5OLAmZu3o6hXp8hI88ZNczRMSe8sUxaYaxVYSBy?= =?utf-8?q?UPkyHCJnr65I0aUfQi8U3aGk86YnaETm0tUZsoamx4kQ=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-1ff67.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 9d50175f-a268-4d6c-7995-08d9be91f4d0 X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Dec 2021 23:40:33.6245 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8P223MB0397 Subject: [FFmpeg-devel] [PATCH v24 07/21] avcodec/subtitles: Replace deprecated enum values X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: ZZISQm1XpVJI Signed-off-by: softworkz --- libavcodec/ass.h | 2 +- libavcodec/assdec.c | 2 +- libavcodec/dvbsubdec.c | 2 +- libavcodec/dvdsubdec.c | 2 +- libavcodec/dvdsubenc.c | 2 +- libavcodec/pgssubdec.c | 2 +- libavcodec/xsubdec.c | 2 +- 7 files changed, 7 insertions(+), 7 deletions(-) diff --git a/libavcodec/ass.h b/libavcodec/ass.h index 9c9cb01c8a..8e885b3ae2 100644 --- a/libavcodec/ass.h +++ b/libavcodec/ass.h @@ -82,7 +82,7 @@ static inline int avpriv_ass_add_rect(AVSubtitle *sub, const char *dialog, rects[sub->num_rects] = av_mallocz(sizeof(*rects[0])); if (!rects[sub->num_rects]) return AVERROR(ENOMEM); - rects[sub->num_rects]->type = SUBTITLE_ASS; + rects[sub->num_rects]->type = AV_SUBTITLE_FMT_ASS; ass_str = avpriv_ass_get_dialog(readorder, layer, style, speaker, dialog); if (!ass_str) return AVERROR(ENOMEM); diff --git a/libavcodec/assdec.c b/libavcodec/assdec.c index 7802a44e71..fd321e7004 100644 --- a/libavcodec/assdec.c +++ b/libavcodec/assdec.c @@ -54,7 +54,7 @@ static int ass_decode_frame(AVCodecContext *avctx, void *data, int *got_sub_ptr, if (!sub->rects[0]) return AVERROR(ENOMEM); sub->num_rects = 1; - sub->rects[0]->type = SUBTITLE_ASS; + sub->rects[0]->type = AV_SUBTITLE_FMT_ASS; sub->rects[0]->ass = av_strdup(avpkt->data); if (!sub->rects[0]->ass) return AVERROR(ENOMEM); diff --git a/libavcodec/dvbsubdec.c b/libavcodec/dvbsubdec.c index 81ccaf4c57..b13244c803 100644 --- a/libavcodec/dvbsubdec.c +++ b/libavcodec/dvbsubdec.c @@ -795,7 +795,7 @@ static int save_subtitle_set(AVCodecContext *avctx, AVSubtitle *sub, int *got_ou rect->w = region->width; rect->h = region->height; rect->nb_colors = (1 << region->depth); - rect->type = SUBTITLE_BITMAP; + rect->type = AV_SUBTITLE_FMT_BITMAP; rect->linesize[0] = region->width; clut = get_clut(ctx, region->clut); diff --git a/libavcodec/dvdsubdec.c b/libavcodec/dvdsubdec.c index 52259f0730..b39b3d1838 100644 --- a/libavcodec/dvdsubdec.c +++ b/libavcodec/dvdsubdec.c @@ -406,7 +406,7 @@ static int decode_dvd_subtitles(DVDSubContext *ctx, AVSubtitle *sub_header, sub_header->rects[0]->y = y1; sub_header->rects[0]->w = w; sub_header->rects[0]->h = h; - sub_header->rects[0]->type = SUBTITLE_BITMAP; + sub_header->rects[0]->type = AV_SUBTITLE_FMT_BITMAP; sub_header->rects[0]->linesize[0] = w; sub_header->rects[0]->flags = is_menu ? AV_SUBTITLE_FLAG_FORCED : 0; } diff --git a/libavcodec/dvdsubenc.c b/libavcodec/dvdsubenc.c index 4916410fc8..fbab88a992 100644 --- a/libavcodec/dvdsubenc.c +++ b/libavcodec/dvdsubenc.c @@ -273,7 +273,7 @@ static int encode_dvd_subtitles(AVCodecContext* avctx, AVPacket* avpkt, return AVERROR(EINVAL); for (i = 0; i < rects; i++) - if (frame->subtitle_areas[i]->type != SUBTITLE_BITMAP) { + if (frame->subtitle_areas[i]->type != AV_SUBTITLE_FMT_BITMAP) { av_log(avctx, AV_LOG_ERROR, "Bitmap subtitle required\n"); return AVERROR(EINVAL); } diff --git a/libavcodec/pgssubdec.c b/libavcodec/pgssubdec.c index 388639a110..4829babb7c 100644 --- a/libavcodec/pgssubdec.c +++ b/libavcodec/pgssubdec.c @@ -539,7 +539,7 @@ static int display_end_segment(AVCodecContext *avctx, void *data, return AVERROR(ENOMEM); } sub->num_rects++; - sub->rects[i]->type = SUBTITLE_BITMAP; + sub->rects[i]->type = AV_SUBTITLE_FMT_BITMAP; /* Process bitmap */ object = find_object(ctx->presentation.objects[i].id, &ctx->objects); diff --git a/libavcodec/xsubdec.c b/libavcodec/xsubdec.c index 979399bae6..5c9e65f737 100644 --- a/libavcodec/xsubdec.c +++ b/libavcodec/xsubdec.c @@ -107,7 +107,7 @@ static int decode_frame(AVCodecContext *avctx, void *data, int *got_sub_ptr, } sub->rects[0]->x = x; sub->rects[0]->y = y; sub->rects[0]->w = w; sub->rects[0]->h = h; - sub->rects[0]->type = SUBTITLE_BITMAP; + sub->rects[0]->type = AV_SUBTITLE_FMT_BITMAP; sub->rects[0]->linesize[0] = w; sub->rects[0]->data[0] = av_malloc(w * h); sub->rects[0]->nb_colors = 4; From patchwork Mon Dec 13 23:40:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 32459 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp6139904iog; Mon, 13 Dec 2021 15:41:53 -0800 (PST) X-Google-Smtp-Source: ABdhPJxtY5MPFMUbnTjHIW2aQjqKVAB2O2PnmvO1Q0d2zS6iZxP5/RSJbl+7rI0yo+xy3fHHZ6a1 X-Received: by 2002:a17:907:960d:: with SMTP id gb13mr1698716ejc.582.1639438913364; Mon, 13 Dec 2021 15:41:53 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id sb31si28522154ejc.716.2021.12.13.15.41.53; Mon, 13 Dec 2021 15:41:53 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=ektQ2eU8; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E030368B0A5; Tue, 14 Dec 2021 01:40:44 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12olkn2082.outbound.protection.outlook.com [40.92.21.82]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D2D9068B093 for ; Tue, 14 Dec 2021 01:40:40 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=O7roftZqjRFAvn9J/gC0ry51M8Rt+BUo+UrZpSn5qKygVX2q+ujtkEY6PJCF8BZydiTA8M4i+ochKl4euhxnrHcitffxiFajbxOSCX6IL3kocBaucLTktAoZ7ggRVjwQudurnVIqpQ/ExGUXLl2uu4vZ5dNxx5NsmLLiyqvWjY4JasiFV7yE3Ef9PHGpeoRz7PMQ5ttAGTzgXlAmacM0Wi25N1iuC7oCALKr7thD3KfTNNHT59r/CJ/9zG4AbAh0cSwNqP6KOQodRB60Murtj3vqx8b69rtjSMALMY0m1+KuugSHaqdj5gsO7F4N6N/SBf1Rs62fd8nVyJ0p8GgeAQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=agoTNQuVB2Ais+uQPxc+g18TNwhkXmRPdQt0SQn6ozo=; b=bqvcGGEnUNMK3phUuYIf3hN4u96Vtg+H+aE2uG2llUHpKTWWlRH89OfR3pnMcwfEDAGM2N4fhmMbMTW/XxLAUUfWMDhh7DTa6+RFNfnyLCMZHgllJYqPkLlkAyv9YVr2rVbcz2+tgbMbPlWvfOdmRNx9lugj7FMSR9Ae2490IMGPfTmDjTjB1W8p8xKC6qrSiIYcBdRYKj3Owwnw5gaajrTAn9IC4TH4dKRCnjYFqw/ITxmeSOGrC9VGnLGH2N6nvAGk+6vtAIC9jf+S001COxPF73F5pLL+a12mUyOZT7LScJfmpw1le/6oViBFga+mPqxXWDrDdu6l4gulZTG1TA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=agoTNQuVB2Ais+uQPxc+g18TNwhkXmRPdQt0SQn6ozo=; b=ektQ2eU8aGCNii5qkhQnewUn+/ERtPGvheAO5aYNjRPQDCp5lj5VtA2SmGkuQLVjf7agUKbc/NPYBnWdqnO2BB4qcWoa6/APsPuSMOlHA1n2h4edRm70eLGwrhoQgMmCQNgSL3efNYa4MtOl4VaUtGtXftnhpj/rWbeDmRNRwbk3BrAZ5pQJ+u5z+wXDBSSvQKBMLzDW6nLhrRZN3ICQRBm+E9aewQbiMsI7/9GrmQbrVU8Rd1CJUaHFhkRaBlCm3YyWaZc1K4OjN6yFq1wcZSzSLGnp1x6sD3hJcMjAFyyEOHI0eG3lg9w1X/ANi6pHq4TWwGp/j8iKfK55c2D5kw== Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::20) by DM8P223MB0397.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4778.17; Mon, 13 Dec 2021 23:40:35 +0000 Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775]) by DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775%7]) with mapi id 15.20.4778.018; Mon, 13 Dec 2021 23:40:35 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v24 08/21] fftools/play,probe: Adjust for subtitle changes Thread-Index: AQHX8HrTBEH3NXD2SkSZvRgyhBKxHA== Date: Mon, 13 Dec 2021 23:40:35 +0000 Message-ID: References: <1ef42eb01b3627bbad539abd46206f320bb39697.1639438462.git.softworkz@hotmail.com> <4148d77a82d4b7e229fc81ce9bc1bc06904533f1.1639438462.git.softworkz@hotmail.com> <2ac798924ee984010d2e85f00bc992d9aa106171.1639438462.git.softworkz@hotmail.com> <300ea7188b331ca5e00dd7e04c03df8aba2a99c0.1639438462.git.softworkz@hotmail.com> <9474726b26f0d3e8f353d07caed2fbd33cdf473a.1639438462.git.softworkz@hotmail.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [6XkTXvt2RTVzhihbDGUPu4rWw3y7IXIM] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: c858c3b9-a14f-4ca1-e747-08d9be91f626 x-ms-traffictypediagnostic: DM8P223MB0397:EE_ x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 1ZV0V8g/edWtsZqBlTPwDeVxN4ARC5yLqQZf14kf7NQcKeGdBZEVcVVdUjqiy9WhdpcCW9rSLHU/0CBcoNa7Pe9mibH0A7EBdMUf4ZQnPDHzH2n2IE94vGOi5fXggrcs1SGPPdMYb/GfbQrgrm9QMRseOkqs9yL7jL7PrSLFjnRhy7uxMwniDQQGGozDmNhi51DR/bZbhwzU3tP6o8TfdJDIi/HHVMBEMwkHVrOQKeruJyCJUfbsQZvtVh6lGUAAC6sU3IIDqHyrJCcHld6UCTxzW/Ga647GZqBXNvBxYopxeT9nZvhwESXzddQHeHLJdlmpxcsuFKl7ONuCibM3iaccDvs4HdxSxDEb74aQ53L58UVK0tTq8y6RrggirekHtqARiitSYyVYQa9qQyYumlqUW+oUzKrjvnArTkwI63iEzPV/eDEK6LsRdXr+qyjCzVk5QN4v5idyZNcgRSD+GORcIGpnBUV0IZA8PplU+Q/b7QrOO4NXzm7wnDCXZat3Hh+lnWDyzHPy4PggnJlZ9nIR/y51FNvk/DDPfXKpgd3jh/EEpI+VlK88SZ+cn2aEerm5xYmZSKLDX+tQmpwntQ== x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?q?LefxKrvXeGdZXUije1P9Ya9rkGEz?= =?utf-8?q?neSvMmmSsS04Kayj2Ut7TEyEEm71VmFiGdeVuVQwCxS0L247d1KSncfZeMBhWTOXU?= =?utf-8?q?kxunYxRsR/a/NbpIMSTmKn3rDEyGMZbH2TJNPWjfDZWuKgInwD+dbpsa8XVzGbNXy?= =?utf-8?q?JzOYUGuHlv2JZsuY9QblIuVWhxP+/JeikxB3h7Z/jZ9ieJogBnEraUDhlMFlqaQ9P?= =?utf-8?q?EvfpLfs7ZmXJXLaT59+84IL69Wdnv4A/HU1v5lpPhjRKK34VuLx/cP/13uM3gZTwX?= =?utf-8?q?vGumuPJt8OZ7rYSCV3TTDwAQpxjWYGB7duCEECsI9H/WM7kke9YUNeXmEEnELKmFi?= =?utf-8?q?AIsclsaXfX3DCYZWDRWS/jPs+1F6WKeDueo/etHLRXHtphDAdM4OdzDV1vbSm3cC1?= =?utf-8?q?Dipt/ASHZ8TRJ44BVD20TxWLlTAfddtrDwB/GtP+0x5+JunEopCebkCXmILZHuo9s?= =?utf-8?q?2mmfoKQCEjtCiRvqzhCLOyzQpO4bZUqLr6hnw7XCVU+WvzgOi4fOMGM/5u1xc2+Jl?= =?utf-8?q?ON14jOaFQvf85tBHOI4Upoql9Q0LBBlCqfp+Wd16aw41tNmfUyKuQS5A3e3IB+PZm?= =?utf-8?q?WGxIZKx665A9RgsAPiDZQRIibKGtjjX0iCXfi5PeM+X+1Im+VU7KRQ42E8Le4guna?= =?utf-8?q?aoNVNIowGXoGFAa48QPNZVvp85I9J7gIP3WyUnj8IO6ejA9DfLWhGn6bv4UNtfS3c?= =?utf-8?q?EVmQ+kpc9Prt0DV+6pm+vPwkv5KeQnAs3fUPTDCcd5LMshcM4ojFfk3G0bZGI3hGK?= =?utf-8?q?MfythJea2kGuKnuHWPbSUlXx3sMpxM0IgQkbbQOiKgZK5MJ/6mllxmftmIaiUOLgp?= =?utf-8?q?9HrGjWrHoiY2cqk6KvPqmpxuFXSWMEUtAqHx3TwWdTX4gsSplXxB2DXNO/KFVcy1F?= =?utf-8?q?UOwQbqGvXyHH+iFNbTKE8skLZfehxkKUjhf82UkwT/Kw=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-1ff67.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: c858c3b9-a14f-4ca1-e747-08d9be91f626 X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Dec 2021 23:40:35.7790 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8P223MB0397 Subject: [FFmpeg-devel] [PATCH v24 08/21] fftools/play, probe: Adjust for subtitle changes X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: yUrTTB08gpO5 Signed-off-by: softworkz --- fftools/ffplay.c | 102 +++++++++++++++++++++------------------------- fftools/ffprobe.c | 48 ++++++++++++++-------- 2 files changed, 78 insertions(+), 72 deletions(-) diff --git a/fftools/ffplay.c b/fftools/ffplay.c index e7b20be76b..0af32888da 100644 --- a/fftools/ffplay.c +++ b/fftools/ffplay.c @@ -152,7 +152,6 @@ typedef struct Clock { /* Common struct for handling all types of decoded data and allocated render buffers. */ typedef struct Frame { AVFrame *frame; - AVSubtitle sub; int serial; double pts; /* presentation timestamp for the frame */ double duration; /* estimated duration of the frame */ @@ -586,7 +585,7 @@ static int decoder_init(Decoder *d, AVCodecContext *avctx, PacketQueue *queue, S return 0; } -static int decoder_decode_frame(Decoder *d, AVFrame *frame, AVSubtitle *sub) { +static int decoder_decode_frame(Decoder *d, AVFrame *frame) { int ret = AVERROR(EAGAIN); for (;;) { @@ -620,6 +619,9 @@ static int decoder_decode_frame(Decoder *d, AVFrame *frame, AVSubtitle *sub) { } } break; + case AVMEDIA_TYPE_SUBTITLE: + ret = avcodec_receive_frame(d->avctx, frame); + break; } if (ret == AVERROR_EOF) { d->finished = d->pkt_serial; @@ -652,25 +654,11 @@ static int decoder_decode_frame(Decoder *d, AVFrame *frame, AVSubtitle *sub) { av_packet_unref(d->pkt); } while (1); - if (d->avctx->codec_type == AVMEDIA_TYPE_SUBTITLE) { - int got_frame = 0; - ret = avcodec_decode_subtitle2(d->avctx, sub, &got_frame, d->pkt); - if (ret < 0) { - ret = AVERROR(EAGAIN); - } else { - if (got_frame && !d->pkt->data) { - d->packet_pending = 1; - } - ret = got_frame ? 0 : (d->pkt->data ? AVERROR(EAGAIN) : AVERROR_EOF); - } - av_packet_unref(d->pkt); + if (avcodec_send_packet(d->avctx, d->pkt) == AVERROR(EAGAIN)) { + av_log(d->avctx, AV_LOG_ERROR, "Receive_frame and send_packet both returned EAGAIN, which is an API violation.\n"); + d->packet_pending = 1; } else { - if (avcodec_send_packet(d->avctx, d->pkt) == AVERROR(EAGAIN)) { - av_log(d->avctx, AV_LOG_ERROR, "Receive_frame and send_packet both returned EAGAIN, which is an API violation.\n"); - d->packet_pending = 1; - } else { - av_packet_unref(d->pkt); - } + av_packet_unref(d->pkt); } } } @@ -683,7 +671,6 @@ static void decoder_destroy(Decoder *d) { static void frame_queue_unref_item(Frame *vp) { av_frame_unref(vp->frame); - avsubtitle_free(&vp->sub); } static int frame_queue_init(FrameQueue *f, PacketQueue *pktq, int max_size, int keep_last) @@ -981,7 +968,7 @@ static void video_image_display(VideoState *is) if (frame_queue_nb_remaining(&is->subpq) > 0) { sp = frame_queue_peek(&is->subpq); - if (vp->pts >= sp->pts + ((float) sp->sub.start_display_time / 1000)) { + if (vp->pts >= sp->pts + ((float) sp->frame->subtitle_start_time / 1000)) { if (!sp->uploaded) { uint8_t* pixels[4]; int pitch[4]; @@ -993,25 +980,27 @@ static void video_image_display(VideoState *is) if (realloc_texture(&is->sub_texture, SDL_PIXELFORMAT_ARGB8888, sp->width, sp->height, SDL_BLENDMODE_BLEND, 1) < 0) return; - for (i = 0; i < sp->sub.num_rects; i++) { - AVSubtitleRect *sub_rect = sp->sub.rects[i]; + for (i = 0; i < sp->frame->num_subtitle_areas; i++) { + AVSubtitleArea *area = sp->frame->subtitle_areas[i]; + SDL_Rect sdl_rect = { .x = area->x, .y = area->y, .w = area->w, .h = area->h }; - sub_rect->x = av_clip(sub_rect->x, 0, sp->width ); - sub_rect->y = av_clip(sub_rect->y, 0, sp->height); - sub_rect->w = av_clip(sub_rect->w, 0, sp->width - sub_rect->x); - sub_rect->h = av_clip(sub_rect->h, 0, sp->height - sub_rect->y); + area->x = av_clip(area->x, 0, sp->width ); + area->y = av_clip(area->y, 0, sp->height); + area->w = av_clip(area->w, 0, sp->width - area->x); + area->h = av_clip(area->h, 0, sp->height - area->y); is->sub_convert_ctx = sws_getCachedContext(is->sub_convert_ctx, - sub_rect->w, sub_rect->h, AV_PIX_FMT_PAL8, - sub_rect->w, sub_rect->h, AV_PIX_FMT_BGRA, + area->w, area->h, AV_PIX_FMT_PAL8, + area->w, area->h, AV_PIX_FMT_BGRA, 0, NULL, NULL, NULL); if (!is->sub_convert_ctx) { av_log(NULL, AV_LOG_FATAL, "Cannot initialize the conversion context\n"); return; } - if (!SDL_LockTexture(is->sub_texture, (SDL_Rect *)sub_rect, (void **)pixels, pitch)) { - sws_scale(is->sub_convert_ctx, (const uint8_t * const *)sub_rect->data, sub_rect->linesize, - 0, sub_rect->h, pixels, pitch); + if (!SDL_LockTexture(is->sub_texture, &sdl_rect, (void **)pixels, pitch)) { + const uint8_t* data[2] = { area->buf[0]->data, (uint8_t *)&area->pal }; + sws_scale(is->sub_convert_ctx, data, area->linesize, + 0, area->h, pixels, pitch); SDL_UnlockTexture(is->sub_texture); } } @@ -1038,16 +1027,18 @@ static void video_image_display(VideoState *is) #if USE_ONEPASS_SUBTITLE_RENDER SDL_RenderCopy(renderer, is->sub_texture, NULL, &rect); #else - int i; + unsigned i; double xratio = (double)rect.w / (double)sp->width; double yratio = (double)rect.h / (double)sp->height; - for (i = 0; i < sp->sub.num_rects; i++) { - SDL_Rect *sub_rect = (SDL_Rect*)sp->sub.rects[i]; - SDL_Rect target = {.x = rect.x + sub_rect->x * xratio, - .y = rect.y + sub_rect->y * yratio, - .w = sub_rect->w * xratio, - .h = sub_rect->h * yratio}; - SDL_RenderCopy(renderer, is->sub_texture, sub_rect, &target); + for (i = 0; i < sp->frame->num_subtitle_areas; i++) { + AVSubtitleArea *area = sp->frame->subtitle_areas[i]; + SDL_Rect sub_rect = { .x = area->x, .y = area->y, + .w = area->w, .h = area->h}; + SDL_Rect target = {.x = rect.x + sub_rect.x * xratio, + .y = rect.y + sub_rect.y * yratio, + .w = sub_rect.w * xratio, + .h = sub_rect.h * yratio}; + SDL_RenderCopy(renderer, is->sub_texture, &sub_rect, &target); } #endif } @@ -1651,19 +1642,20 @@ retry: sp2 = NULL; if (sp->serial != is->subtitleq.serial - || (is->vidclk.pts > (sp->pts + ((float) sp->sub.end_display_time / 1000))) - || (sp2 && is->vidclk.pts > (sp2->pts + ((float) sp2->sub.start_display_time / 1000)))) + || (is->vidclk.pts > (sp->pts + ((float) sp->frame->subtitle_end_time / 1000))) + || (sp2 && is->vidclk.pts > (sp2->pts + ((float) sp2->frame->subtitle_start_time / 1000)))) { if (sp->uploaded) { int i; - for (i = 0; i < sp->sub.num_rects; i++) { - AVSubtitleRect *sub_rect = sp->sub.rects[i]; + for (i = 0; i < sp->frame->num_subtitle_areas; i++) { + AVSubtitleArea *area = sp->frame->subtitle_areas[i]; + SDL_Rect sdl_rect = { .x = area->x, .y = area->y, .w = area->w, .h = area->h }; uint8_t *pixels; int pitch, j; - if (!SDL_LockTexture(is->sub_texture, (SDL_Rect *)sub_rect, (void **)&pixels, &pitch)) { - for (j = 0; j < sub_rect->h; j++, pixels += pitch) - memset(pixels, 0, sub_rect->w << 2); + if (!SDL_LockTexture(is->sub_texture, &sdl_rect, (void **)&pixels, &pitch)) { + for (j = 0; j < area->h; j++, pixels += pitch) + memset(pixels, 0, area->w << 2); SDL_UnlockTexture(is->sub_texture); } } @@ -1774,7 +1766,7 @@ static int get_video_frame(VideoState *is, AVFrame *frame) { int got_picture; - if ((got_picture = decoder_decode_frame(&is->viddec, frame, NULL)) < 0) + if ((got_picture = decoder_decode_frame(&is->viddec, frame)) < 0) return -1; if (got_picture) { @@ -2048,7 +2040,7 @@ static int audio_thread(void *arg) return AVERROR(ENOMEM); do { - if ((got_frame = decoder_decode_frame(&is->auddec, frame, NULL)) < 0) + if ((got_frame = decoder_decode_frame(&is->auddec, frame)) < 0) goto the_end; if (got_frame) { @@ -2246,14 +2238,14 @@ static int subtitle_thread(void *arg) if (!(sp = frame_queue_peek_writable(&is->subpq))) return 0; - if ((got_subtitle = decoder_decode_frame(&is->subdec, NULL, &sp->sub)) < 0) + if ((got_subtitle = decoder_decode_frame(&is->subdec, sp->frame)) < 0) break; pts = 0; - if (got_subtitle && sp->sub.format == 0) { - if (sp->sub.pts != AV_NOPTS_VALUE) - pts = sp->sub.pts / (double)AV_TIME_BASE; + if (got_subtitle && sp->frame->format == AV_SUBTITLE_FMT_BITMAP) { + if (sp->frame->subtitle_pts != AV_NOPTS_VALUE) + pts = sp->frame->subtitle_pts / (double)AV_TIME_BASE; sp->pts = pts; sp->serial = is->subdec.pkt_serial; sp->width = is->subdec.avctx->width; @@ -2263,7 +2255,7 @@ static int subtitle_thread(void *arg) /* now we can update the picture count */ frame_queue_push(&is->subpq); } else if (got_subtitle) { - avsubtitle_free(&sp->sub); + av_frame_free(&sp->frame); } } return 0; diff --git a/fftools/ffprobe.c b/fftools/ffprobe.c index 0711e02922..367c708e52 100644 --- a/fftools/ffprobe.c +++ b/fftools/ffprobe.c @@ -2208,22 +2208,43 @@ static void show_packet(WriterContext *w, InputFile *ifile, AVPacket *pkt, int p fflush(stdout); } -static void show_subtitle(WriterContext *w, AVSubtitle *sub, AVStream *stream, +static void show_subtitle(WriterContext *w, AVFrame *sub, AVStream *stream, AVFormatContext *fmt_ctx) { AVBPrint pbuf; + const char *s; av_bprint_init(&pbuf, 1, AV_BPRINT_SIZE_UNLIMITED); writer_print_section_header(w, SECTION_ID_SUBTITLE); print_str ("media_type", "subtitle"); - print_ts ("pts", sub->pts); - print_time("pts_time", sub->pts, &AV_TIME_BASE_Q); - print_int ("format", sub->format); - print_int ("start_display_time", sub->start_display_time); - print_int ("end_display_time", sub->end_display_time); - print_int ("num_rects", sub->num_rects); + print_ts ("pts", sub->subtitle_pts); + print_time("pts_time", sub->subtitle_pts, &AV_TIME_BASE_Q); + + // Remain compatible with previous outputs + switch (sub->format) { + case AV_SUBTITLE_FMT_BITMAP: + print_int ("format", 0); + break; + case AV_SUBTITLE_FMT_TEXT: + print_int ("format", 1); + break; + case AV_SUBTITLE_FMT_ASS: + print_int ("format", 1); + break; + default: + print_int ("format", -1); + break; + } + + s = av_get_subtitle_fmt_name(sub->format); + if (s) print_str ("format_str", s); + else print_str_opt("format_str", "unknown"); + + print_int ("start_display_time", sub->subtitle_start_time); + print_int ("end_display_time", sub->subtitle_end_time); + print_int ("num_subtitle_rects", sub->num_subtitle_areas); writer_print_section_footer(w); @@ -2388,7 +2409,6 @@ static av_always_inline int process_frame(WriterContext *w, AVFormatContext *fmt_ctx = ifile->fmt_ctx; AVCodecContext *dec_ctx = ifile->streams[pkt->stream_index].dec_ctx; AVCodecParameters *par = ifile->streams[pkt->stream_index].st->codecpar; - AVSubtitle sub; int ret = 0, got_frame = 0; clear_log(1); @@ -2396,6 +2416,7 @@ static av_always_inline int process_frame(WriterContext *w, switch (par->codec_type) { case AVMEDIA_TYPE_VIDEO: case AVMEDIA_TYPE_AUDIO: + case AVMEDIA_TYPE_SUBTITLE: if (*packet_new) { ret = avcodec_send_packet(dec_ctx, pkt); if (ret == AVERROR(EAGAIN)) { @@ -2414,12 +2435,6 @@ static av_always_inline int process_frame(WriterContext *w, } } break; - - case AVMEDIA_TYPE_SUBTITLE: - if (*packet_new) - ret = avcodec_decode_subtitle2(dec_ctx, &sub, &got_frame, pkt); - *packet_new = 0; - break; default: *packet_new = 0; } @@ -2434,12 +2449,11 @@ static av_always_inline int process_frame(WriterContext *w, nb_streams_frames[pkt->stream_index]++; if (do_show_frames) if (is_sub) - show_subtitle(w, &sub, ifile->streams[pkt->stream_index].st, fmt_ctx); + show_subtitle(w, frame, ifile->streams[pkt->stream_index].st, fmt_ctx); else show_frame(w, frame, ifile->streams[pkt->stream_index].st, fmt_ctx); - if (is_sub) - avsubtitle_free(&sub); } + return got_frame || *packet_new; } From patchwork Mon Dec 13 23:40:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 32460 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp6140062iog; Mon, 13 Dec 2021 15:42:05 -0800 (PST) X-Google-Smtp-Source: ABdhPJyAvZ6/uCuHYjjdDP0Jcx9PXEDRdNWP3JEFIin36Ldz12C/m6gl+/B/X5eDgnX+HMesND/l X-Received: by 2002:aa7:c946:: with SMTP id h6mr2784552edt.190.1639438925450; Mon, 13 Dec 2021 15:42:05 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id qf36si25901852ejc.35.2021.12.13.15.42.05; Mon, 13 Dec 2021 15:42:05 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=eiCxdCGI; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 71ACB68B094; Tue, 14 Dec 2021 01:40:46 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12olkn2082.outbound.protection.outlook.com [40.92.21.82]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 37AE068B097 for ; Tue, 14 Dec 2021 01:40:41 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HSn+bUo6I9n37NhSYPi3cdDTRNDFH8GMn7w65kwStHXyPi6lmu5pzEUQtVNkRtRSiVICTHu1Dat964bXodkLjYC3OKnptYrw9eyjxNax7Pg4kwVuUxLg+oh5whLijaBqB+EPW4aZZMaRSvtcmtEPx3awyarfgLlTcFQ5KexdwPbNWUNnK2BgfPOVDDZZBLj7tY46o7pmoYgBfQKsPlRxNnBFZZRmoK7/cLO44A7Nqgk3WEi7PPdM22DJgvhLGxab7ZeLLK5DuzJSc68r+a1xMjZeP/FFHfwQXyrP421FkYTt8ruylfOUxIoxL0HZ4bo6rOEZhedhJ5QbcOuZW9cmWQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=OFPGP6c/q3W5r6K7OtjtiKLBZ7YZwwGUi/mkkxikzZM=; b=b28i5hL4wRB6L11GpqGPsC185cZbaBTw0MZXL8NrWISNZraQK+ipyvj/SNwrAnlhj1vKLyZ35rv3e0EtsrnMWIUcNUS9+GApNEkwfE/mGCc5PbeDh11zfOVS4Xm1T6I5IEjwibTQH5UbSN4vQMbiJuA0QAFawEKbWjg1EyJJgnuZD+PXRmr8cxBq3iH+M6snJxyZElCvZo9jVpxTW6y2SZ05y7fqftbEPYwBPSTgR0EuAge2TAHNqgdgZHVD8GW8vNgr1WIEzIMS4GRX998+n6dfQzp3NRtZcg43wJGoS4OI9y3W0VayaS21hADt9Uqfab9EkOXLO0d3AZGxmERliw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OFPGP6c/q3W5r6K7OtjtiKLBZ7YZwwGUi/mkkxikzZM=; b=eiCxdCGIg5tm3zqWPC+rl74e9QrdedCmU8hJVLiEapTxJvuaiQXBsmat0T67A0wE71+WA8qVbW4AJ78mytqH9v/PN4t10Lil1n4BnavltHT38rVABXlPLg7YxZqU+nF3qZLCI1JOQTwKYYRVed589sWUSOLZXvUrEAM/5MZbncUit3UgxIQrOA6Q6lrhwJiiVeqRh3P3w8+L13Vj3RTuFe7rcExeOwIOLM05G3YQ+f61gY2EzK+wLzktUB4YaHbB+FCsfiPPds0pAc3I/DpmKmvY6wYXjrPJ3bE2KDzpLJ1LcOY6aW7xKh5124vHymlEbWWYtUDU3EVSD+oLftWxNA== Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::20) by DM8P223MB0397.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4778.17; Mon, 13 Dec 2021 23:40:37 +0000 Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775]) by DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775%7]) with mapi id 15.20.4778.018; Mon, 13 Dec 2021 23:40:37 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v24 09/21] avfilter/subtitles: Add subtitles.c for subtitle frame allocation Thread-Index: AQHX8HrUp3GaD0y7bkeWVrmfrkT1rw== Date: Mon, 13 Dec 2021 23:40:37 +0000 Message-ID: References: <1ef42eb01b3627bbad539abd46206f320bb39697.1639438462.git.softworkz@hotmail.com> <4148d77a82d4b7e229fc81ce9bc1bc06904533f1.1639438462.git.softworkz@hotmail.com> <2ac798924ee984010d2e85f00bc992d9aa106171.1639438462.git.softworkz@hotmail.com> <300ea7188b331ca5e00dd7e04c03df8aba2a99c0.1639438462.git.softworkz@hotmail.com> <9474726b26f0d3e8f353d07caed2fbd33cdf473a.1639438462.git.softworkz@hotmail.com> <424522c96b5cdf6b03faa6bdceb1d25449463429.1639438462.git.softworkz@hotmail.com> In-Reply-To: <424522c96b5cdf6b03faa6bdceb1d25449463429.1639438462.git.softworkz@hotmail.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [u9JThvXlugyYMTt9JjlQrJUz1Qo7/w/p] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: b3a16f15-b06f-4949-0b64-08d9be91f73f x-ms-traffictypediagnostic: DM8P223MB0397:EE_ x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: umvNPp3WNtDbNmmiMuxTa8T7sru6IZoGBuECJT59zlr8mVXv7d4qiob9EyhYjhT4P9Cr7XeJO5v1QiVX7tY63zuGVESlif8BZdDlQ2JP4VQhIDFwxlTw04Elch0eg9C77L42JqnP3MbWW4RiwjXiRUtDEfmm19AoXtZPq/5Xbsaq8B+DZPw6yUFxJDUmf17cxHIp9pQyTzIJQq1pIsPUxvnVgYGjpL9nstsXd6SSureYXYuCmSJh9DRbkO1BID1GGHANp90lwiTExlc5Q2qMuOMYLEUUt6xdFksq8tvPwXE795HS/kru0hp7ONGwZ6HEZeUfKXjx88/E9IuDnSeniy1wMvstu+a1TXgIuSjSg7AosUKrE9yEwQ4Q708zUwelAklRYSs5tsaP7iUVM3GI1qt5q1DpN3mJWaF5uBqfnS9lEHBk3QVNpvPbzGBWOi/c9iYv2clMmqxnBxgn7F57N3gfHT9qPEIzaHKpsn5Rs21fzWEm0UeAJaITqJ7EjMPCrhtWY5D+prBthcR61ixdMnhYgdHMz4+JFQeasGkZlWqSy7MwVFX1LQXBv18t1a+cAYgst5H/XrDJXecyoaxB5Q== x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?q?I0pVvUoLSI7LSxKcZ8YaZBS9VCGL?= =?utf-8?q?bX6MGX/8HwF+KvYWXsnQYmSNQPI+UV/Z+Egge6E4yJpOJvc28ldCGQi7K4MVKnO+K?= =?utf-8?q?VTNPDudW37oyomEqRlkadYOslP/K2YX6ACY3BoXlxbvTzDc1vgU1GCw4mjPeO5ICn?= =?utf-8?q?GxH2zjUw3hLAZKTwNaN1sEApSJyxz95wHwkVrAdAdxifF0WhmO5ue8fusdXJfylFJ?= =?utf-8?q?vlAwG918tY0DkP98A+Ux89V8JWs69ZsKEhnz3+EtftUuqf/uX7gDMOztbojbcx76i?= =?utf-8?q?c35r2ibGL9QPBFDQR5TvtWUvHhQSd9wFR10SUxJiLk/v2/94/wmntp3OPaoyOljRJ?= =?utf-8?q?LECd3AsPkplLxUNi2VpTJtDLh2oeG4E/Nyr4RYz6DgpyFIJSdT1j6ki/eelRpUUgs?= =?utf-8?q?yhTP/ieT4V2phUaAALYqwMoFAc6ne3hPo8J2ktX8JBjtT6tnuedzIOC/TmCm0M5P8?= =?utf-8?q?C5K4R4pBBNT8umPuoAo5MoGJa+Inb3V260K4jM05aQZBJB4gV1M9KZRiR25dQbbkB?= =?utf-8?q?uACB4VNmf1qpVWaBQJuaWY378/udKGLZLAKRcVH2uAmy0KvefriVdTkvoWCxkthmu?= =?utf-8?q?BfWdppprCu36lGzoG9iUUEGnBJdPzu/C59GwukiOrkvXLXn5ThsEe0dGeWalCVBBd?= =?utf-8?q?VfeqXLNiO+ov/vGvQnHW/uRgiBgG9oigBtQY3djmWcj2hYq3zWot3KTbuk3egBfbX?= =?utf-8?q?pVFEQwnXK0zTthXGNaiiVk4CTHSeHnWIDhU3PUvFuRBaywlJ4e3jaRmIR6w8Jz8/X?= =?utf-8?q?UCY5bzOoP8J6A6R+kn5YGkCqXX4PlvJ89uyrjcb24dSaI8VYGHmKXJjcYplUFO/Ev?= =?utf-8?q?s//yJZ0sq4rlJWXHiyWnFO8FGV0afqMaRlMJp7vJwcwyaBymOe6x/CE3JqRtQKtYM?= =?utf-8?q?/vCfyysXR2cgBZZXEESMxuF2f6HsSVelW5XE/nz0BZiQ=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-1ff67.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: b3a16f15-b06f-4949-0b64-08d9be91f73f X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Dec 2021 23:40:37.6657 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8P223MB0397 Subject: [FFmpeg-devel] [PATCH v24 09/21] avfilter/subtitles: Add subtitles.c for subtitle frame allocation X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: QDRzodTMu6T4 Analog to avfilter/video.c and avfilter/audio.c Signed-off-by: softworkz --- libavfilter/Makefile | 1 + libavfilter/avfilter.c | 4 +++ libavfilter/internal.h | 1 + libavfilter/subtitles.c | 63 +++++++++++++++++++++++++++++++++++++++++ libavfilter/subtitles.h | 44 ++++++++++++++++++++++++++++ 5 files changed, 113 insertions(+) create mode 100644 libavfilter/subtitles.c create mode 100644 libavfilter/subtitles.h diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 2fe495df28..a2f61d39f0 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -19,6 +19,7 @@ OBJS = allfilters.o \ framequeue.o \ graphdump.o \ graphparser.o \ + subtitles.o \ video.o \ OBJS-$(HAVE_THREADS) += pthread.o diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c index 7362bcdab5..df5b8f483c 100644 --- a/libavfilter/avfilter.c +++ b/libavfilter/avfilter.c @@ -43,6 +43,7 @@ #include "formats.h" #include "framepool.h" #include "internal.h" +#include "subtitles.h" #include "libavutil/ffversion.h" const char av_filter_ffversion[] = "FFmpeg version " FFMPEG_VERSION; @@ -1475,6 +1476,9 @@ int ff_inlink_make_frame_writable(AVFilterLink *link, AVFrame **rframe) case AVMEDIA_TYPE_AUDIO: out = ff_get_audio_buffer(link, frame->nb_samples); break; + case AVMEDIA_TYPE_SUBTITLE: + out = ff_get_subtitles_buffer(link, link->format); + break; default: return AVERROR(EINVAL); } diff --git a/libavfilter/internal.h b/libavfilter/internal.h index 1099b82b4b..fc09ef574c 100644 --- a/libavfilter/internal.h +++ b/libavfilter/internal.h @@ -90,6 +90,7 @@ struct AVFilterPad { union { AVFrame *(*video)(AVFilterLink *link, int w, int h); AVFrame *(*audio)(AVFilterLink *link, int nb_samples); + AVFrame *(*subtitle)(AVFilterLink *link, int format); } get_buffer; /** diff --git a/libavfilter/subtitles.c b/libavfilter/subtitles.c new file mode 100644 index 0000000000..951bfd612c --- /dev/null +++ b/libavfilter/subtitles.c @@ -0,0 +1,63 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/common.h" + +#include "subtitles.h" +#include "avfilter.h" +#include "internal.h" + + +AVFrame *ff_null_get_subtitles_buffer(AVFilterLink *link, int format) +{ + return ff_get_subtitles_buffer(link->dst->outputs[0], format); +} + +AVFrame *ff_default_get_subtitles_buffer(AVFilterLink *link, int format) +{ + AVFrame *frame; + + frame = av_frame_alloc(); + if (!frame) + return NULL; + + frame->format = format; + frame->type = AVMEDIA_TYPE_SUBTITLE; + + if (av_frame_get_buffer2(frame, 0) < 0) { + av_frame_free(&frame); + return NULL; + } + + return frame; +} + +AVFrame *ff_get_subtitles_buffer(AVFilterLink *link, int format) +{ + AVFrame *ret = NULL; + + if (link->dstpad->get_buffer.subtitle) + ret = link->dstpad->get_buffer.subtitle(link, format); + + if (!ret) + ret = ff_default_get_subtitles_buffer(link, format); + + return ret; +} diff --git a/libavfilter/subtitles.h b/libavfilter/subtitles.h new file mode 100644 index 0000000000..4a9115126e --- /dev/null +++ b/libavfilter/subtitles.h @@ -0,0 +1,44 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVFILTER_SUBTITLES_H +#define AVFILTER_SUBTITLES_H + +#include "avfilter.h" +#include "internal.h" + +/** default handler for get_subtitles_buffer() for subtitle inputs */ +AVFrame *ff_default_get_subtitles_buffer(AVFilterLink *link, int format); + +/** get_subtitles_buffer() handler for filters which simply pass subtitles along */ +AVFrame *ff_null_get_subtitles_buffer(AVFilterLink *link, int format); + +/** + * Request a subtitles frame with a specific set of permissions. + * + * @param link the output link to the filter from which the buffer will + * be requested + * @param format The subtitles format. + * @return A reference to the frame. This must be unreferenced with + * av_frame_free when you are finished with it. +*/ +AVFrame *ff_get_subtitles_buffer(AVFilterLink *link, int format); + +#endif /* AVFILTER_SUBTITLES_H */ From patchwork Mon Dec 13 23:40:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 32461 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp6140254iog; Mon, 13 Dec 2021 15:42:17 -0800 (PST) X-Google-Smtp-Source: ABdhPJx+/D9xHWpcmh2JW7qliNjtf1cQe5l3giG7Y4wBhOJ6Ta4ZrGhVn03wCb4bTHAzRPLBnRYQ X-Received: by 2002:a17:907:7295:: with SMTP id dt21mr1605662ejc.595.1639438937010; Mon, 13 Dec 2021 15:42:17 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id ne38si21477618ejc.97.2021.12.13.15.42.16; Mon, 13 Dec 2021 15:42:17 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=e8NcEJ3H; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 87A1468B0BC; Tue, 14 Dec 2021 01:40:47 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12olkn2082.outbound.protection.outlook.com [40.92.21.82]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 827D868B03C for ; Tue, 14 Dec 2021 01:40:41 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Obr0dwW3Xot9BjUyGrcdjZeyuq5+0lgQHkCwJE8DeD8JGNekL7Li9Nq8GeVf8v0nkwIRF29g2cFfL5ZBXrmI2Jg5RVQTzajAoDlV8+pOFj5vd6ToD+HmW61KEDAGGXIfB3NI3dLzWz4mUbZKi7D5jEFHaplbFjZKXBXfrPbQkfhgLbA0MsaDbPasoq/MKe4B7L6qT6DRM+ZDgn1CdakZqxSw0i/gGmjzQe5fPZSRNMc44IgEye0iTWNKWGIRQDqhea2NeA50kGGYLYhoJg3vgOzjZLUT9msSRpijrRW1BkwMsNkFJpRoVxcC8acEL8pLIEhPnZdHLJJSCpHzOO/GCw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=qgJmS0GpwRFmLIz6mjkSSnEW3v7UTXOe8XP6GYa7hBQ=; b=Ul2hkEUbpbh3SHiJnG/3N0bXUyXNxGSTGe+62/tNAllguwPGhq3RT1fXyzlDnh6kKw+egJ4+zbpt1p9aE4lP0FzXyCdqIwdWj6ALMcd+VrZqQDpLRb0axwoKUEBLQJyjg/4sczdLmYNaOxeBpBKzHBbai4hd10ktXdNInoUQB7TmH29UTTTLImKYqk3sYsb12uctx10EXuqhRBzszDoQRJ3Pqx0PEvU2qogQ7Ghb0bpnADYoWgYhWM8GsT5IVRzW11BjqjkfNrHRyB/xa12iT0pOgHfqMUbHlAale3TZ8zcIIr06JmY/mwTDn4cu3EEjTyQQF0rhiJGQgUBtW1EIdg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qgJmS0GpwRFmLIz6mjkSSnEW3v7UTXOe8XP6GYa7hBQ=; b=e8NcEJ3HYEAknlzwdC4YcaUiXtU0jJfInFfiF5wLxBfqkfZNrqSHnWDLIidxmyo7KO7l4awojqtu4iUKj4bCUcPFR6T6vX68z+ogRTG0a9fRnd4AwQgQ31zu3OH8vwOmujuz0Sn+TAJ6ZP5sAxgd47jz3Qq389Id+nlSnCpb7uHn+MXvFJls7n4jgeSxZHrelnWOiYBBug0OEQEX4sDBmLZrxbKxwdmCv5Ko0AiCLqJ5Ohwpo+MZs8OBxH7Q6A0Ev5xkkAOMuD8Au7frX+7rWLWFweAiCXiDUkaXWJjNwJ2huFgg/e/3LkozfQxe5j3iZMpmAJ+3aieGs96aSR3qXg== Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::20) by DM8P223MB0397.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4778.17; Mon, 13 Dec 2021 23:40:39 +0000 Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775]) by DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775%7]) with mapi id 15.20.4778.018; Mon, 13 Dec 2021 23:40:39 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v24 10/21] avfilter/avfilter: Handle subtitle frames Thread-Index: AQHX8HrVm9OCcf2YD0q6WDAHL2K6Sg== Date: Mon, 13 Dec 2021 23:40:39 +0000 Message-ID: References: <1ef42eb01b3627bbad539abd46206f320bb39697.1639438462.git.softworkz@hotmail.com> <4148d77a82d4b7e229fc81ce9bc1bc06904533f1.1639438462.git.softworkz@hotmail.com> <2ac798924ee984010d2e85f00bc992d9aa106171.1639438462.git.softworkz@hotmail.com> <300ea7188b331ca5e00dd7e04c03df8aba2a99c0.1639438462.git.softworkz@hotmail.com> <9474726b26f0d3e8f353d07caed2fbd33cdf473a.1639438462.git.softworkz@hotmail.com> <424522c96b5cdf6b03faa6bdceb1d25449463429.1639438462.git.softworkz@hotmail.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [1Bcr0o90OXoH8sGlCU3rcmyg6SQDVRgm] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 6a5a26e2-e88f-4c40-bbb2-08d9be91f86e x-ms-traffictypediagnostic: DM8P223MB0397:EE_ x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: HVsT1bTKe2A8CiGH2tZE9ht178jhqHCi0QxA8Or+GAFNT+mD2EqofH9hGEx9acMdsXembSTVxsv1se1uro4ajR3uzjZB6SEGqxYXTHAmo/Pl6Q1xeZY4XPabnr7jGeF0pHeWY9gcAt+5+UutQCEePBL1HhhWQKV8DT4J30YSbmpEUrtNw3FRYK1dFMvNwByK1R/348fnyqhN6u2KbkOmdJsTbEtX+bkIWn7T/yEeOgJ1I/90BdUe1Gr52kl8itbP0e2BUhruIbrdyDTMOWXqqFxO/2YCHIIYFjt1HCFDakIsGD/IR9kU6nyph47xsSpnfOGAj/RRP2d3DdR1Gxv9YIjIrO2rpvwuxTDst9sjOOdR9ng4QT0mHdBfKP5GB3iXsVpZ/UzMR7OovX6WUuuk2Ivlqy4B0YlYnQH6zp2Sx5aLkbWt6yd89mV3NQGgv+drheqDim9Pskixw17zb0xEtMAqxFP+0TB1ADlfNemJhhdfwUUZFX9QiG23k+Juv332TvnCZKo6hRMijAlB7kac3Lcy7XMtJh3yNzfCkCBIWY3tiY05NSFgyWsTfSSKvyb/npYiKTF1qQnq079Z+lSxdw== x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?q?hRxdEbzSeXMyC5CDabpdB+X857Eb?= =?utf-8?q?xMZeSENjrqCyVBSdS25GEpTzI3A0oRFkhwOzTB229i+yrspNq/1X6Sz/Avg/vvCeD?= =?utf-8?q?hJnXGkrja3dCoG+oy4ivK/NtIDrqXMLjZRO9J78U4CM0SFpXlN8gW4mw1lJ6K7+yt?= =?utf-8?q?W2c0u3HhVsdzF5BoJmzNWd5GE47tfzxSTv96Gwz7kxpLtghA8Huq2y5+wFn5kMb6N?= =?utf-8?q?RNMXX19mjJB0RqD6iuavKygvMVTHZ4EXCHrot1xbQlkJK3BqivS1Ccr+yzAn3eLlG?= =?utf-8?q?//BUWEq6s3xuKj1Yj6Ocn5W1+mV7xb6YVLvNM9kigLt21xI1tbhCJ/UVe6TR2spri?= =?utf-8?q?2dqFOElAyFFZKgXNfYymgW6iBiO9i4qh88JMWjSInYdx8TWl9Hi8dKoD0JeRg1BMy?= =?utf-8?q?30EcKqPQEl2+REiaYDN9z8tittM31d/Q8EeZTbsK3WLfMtUR7uD5PopaTITQC+fOG?= =?utf-8?q?339nR7+CXGC1CiEFTQpsxRA3FKEi0JCklXQGFzI8n8Xc7v4u9yZcT6L7NFKyBDCq9?= =?utf-8?q?kVmx6ZeUAndlQ7PO+cnQCcilgZ+r1Jf45zBf6eZr8gb8KKqBQN7moh48Z37uOjogP?= =?utf-8?q?iq3JrJlViRu0eeJ/M1ax1Jj26GF9FukBET34+1GDSpRW5cMGko3AJyLicQjX+gYrq?= =?utf-8?q?OaO0xxbutrrAUY9RyEUQ2sIdyQdf4OkdhkhDSpkMzAs01dJ3Fky0BuTbX+rou9QwC?= =?utf-8?q?AC6Im5WeMShVYfTBS93KH/AsQhOLTH2/o1fdWS01jYXjviCURgloVihYA7XGvXvHe?= =?utf-8?q?zxr4gxEWxLzi3DoyF2hi/B32zmiCW0bfgHdeOPVwqHFx7cghmsbY+EhNpGfwdAfkx?= =?utf-8?q?dODdX3uvQrM/vSOhnw6ZhrxVX0ji9AiTFO1AFnpJJuriW2kMJy6REzXzy6453HMI9?= =?utf-8?q?XCnhiBGsjz9NZh4qwWiFb01FFziUGbzoHc7DOo46aRIQ=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-1ff67.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 6a5a26e2-e88f-4c40-bbb2-08d9be91f86e X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Dec 2021 23:40:39.6062 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8P223MB0397 Subject: [FFmpeg-devel] [PATCH v24 10/21] avfilter/avfilter: Handle subtitle frames X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: UVERsyu7wQ1U Signed-off-by: softworkz --- libavfilter/avfilter.c | 8 +++++--- libavfilter/avfilter.h | 11 +++++++++++ libavfilter/avfiltergraph.c | 5 +++++ libavfilter/formats.c | 22 ++++++++++++++++++++++ libavfilter/formats.h | 3 +++ libavfilter/internal.h | 18 +++++++++++++++--- 6 files changed, 61 insertions(+), 6 deletions(-) diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c index df5b8f483c..75d5e86539 100644 --- a/libavfilter/avfilter.c +++ b/libavfilter/avfilter.c @@ -56,7 +56,8 @@ static void tlog_ref(void *ctx, AVFrame *ref, int end) ref->linesize[0], ref->linesize[1], ref->linesize[2], ref->linesize[3], ref->pts, ref->pkt_pos); - if (ref->width) { + switch(ref->type) { + case AVMEDIA_TYPE_VIDEO: ff_tlog(ctx, " a:%d/%d s:%dx%d i:%c iskey:%d type:%c", ref->sample_aspect_ratio.num, ref->sample_aspect_ratio.den, ref->width, ref->height, @@ -64,12 +65,13 @@ static void tlog_ref(void *ctx, AVFrame *ref, int end) ref->top_field_first ? 'T' : 'B', /* Top / Bottom */ ref->key_frame, av_get_picture_type_char(ref->pict_type)); - } - if (ref->nb_samples) { + break; + case AVMEDIA_TYPE_AUDIO: ff_tlog(ctx, " cl:%"PRId64"d n:%d r:%d", ref->channel_layout, ref->nb_samples, ref->sample_rate); + break; } ff_tlog(ctx, "]%s", end ? "\n" : ""); diff --git a/libavfilter/avfilter.h b/libavfilter/avfilter.h index b105dc3159..9f917deb41 100644 --- a/libavfilter/avfilter.h +++ b/libavfilter/avfilter.h @@ -45,6 +45,7 @@ #include "libavutil/log.h" #include "libavutil/samplefmt.h" #include "libavutil/pixfmt.h" +#include "libavutil/subfmt.h" #include "libavutil/rational.h" #include "libavfilter/version.h" @@ -343,6 +344,12 @@ typedef struct AVFilter { * and outputs use the same sample rate and channel count/layout. */ const enum AVSampleFormat *samples_list; + /** + * Analogous to pixels, but delimited by AV_SUBTITLE_FMT_NONE + * and restricted to filters that only have AVMEDIA_TYPE_SUBTITLE + * inputs and outputs. + */ + const enum AVSubtitleType *subs_list; /** * Equivalent to { pix_fmt, AV_PIX_FMT_NONE } as pixels_list. */ @@ -351,6 +358,10 @@ typedef struct AVFilter { * Equivalent to { sample_fmt, AV_SAMPLE_FMT_NONE } as samples_list. */ enum AVSampleFormat sample_fmt; + /** + * Equivalent to { sub_fmt, AV_SUBTITLE_FMT_NONE } as subs_list. + */ + enum AVSubtitleType sub_fmt; } formats; int priv_size; ///< size of private data to allocate for the filter diff --git a/libavfilter/avfiltergraph.c b/libavfilter/avfiltergraph.c index b8b432e98b..f4987654af 100644 --- a/libavfilter/avfiltergraph.c +++ b/libavfilter/avfiltergraph.c @@ -311,6 +311,8 @@ static int filter_link_check_formats(void *log, AVFilterLink *link, AVFilterForm return ret; break; + case AVMEDIA_TYPE_SUBTITLE: + return 0; default: av_assert0(!"reached"); } @@ -441,6 +443,9 @@ static int query_formats(AVFilterGraph *graph, void *log_ctx) if (!link) continue; + if (link->type == AVMEDIA_TYPE_SUBTITLE) + continue; + neg = ff_filter_get_negotiation(link); av_assert0(neg); for (neg_step = 1; neg_step < neg->nb_mergers; neg_step++) { diff --git a/libavfilter/formats.c b/libavfilter/formats.c index ba62f73248..5c972bb183 100644 --- a/libavfilter/formats.c +++ b/libavfilter/formats.c @@ -19,6 +19,7 @@ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ +#include "libavutil/subfmt.h" #include "libavutil/avassert.h" #include "libavutil/channel_layout.h" #include "libavutil/common.h" @@ -431,6 +432,12 @@ int ff_add_channel_layout(AVFilterChannelLayouts **l, uint64_t channel_layout) return 0; } +int ff_add_subtitle_type(AVFilterFormats **avff, int64_t fmt) +{ + ADD_FORMAT(avff, fmt, ff_formats_unref, int, formats, nb_formats); + return 0; +} + AVFilterFormats *ff_make_formats_list_singleton(int fmt) { int fmts[2] = { fmt, -1 }; @@ -450,6 +457,13 @@ AVFilterFormats *ff_all_formats(enum AVMediaType type) return NULL; fmt++; } + } else if (type == AVMEDIA_TYPE_SUBTITLE) { + if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_BITMAP) < 0) + return NULL; + if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_ASS) < 0) + return NULL; + if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_TEXT) < 0) + return NULL; } return ret; @@ -724,6 +738,10 @@ int ff_default_query_formats(AVFilterContext *ctx) type = AVMEDIA_TYPE_AUDIO; formats = ff_make_format_list(f->formats.samples_list); break; + case FF_FILTER_FORMATS_SUBFMTS_LIST: + type = AVMEDIA_TYPE_SUBTITLE; + formats = ff_make_format_list(f->formats.subs_list); + break; case FF_FILTER_FORMATS_SINGLE_PIXFMT: type = AVMEDIA_TYPE_VIDEO; formats = ff_make_formats_list_singleton(f->formats.pix_fmt); @@ -732,6 +750,10 @@ int ff_default_query_formats(AVFilterContext *ctx) type = AVMEDIA_TYPE_AUDIO; formats = ff_make_formats_list_singleton(f->formats.sample_fmt); break; + case FF_FILTER_FORMATS_SINGLE_SUBFMT: + type = AVMEDIA_TYPE_SUBTITLE; + formats = ff_make_formats_list_singleton(f->formats.sub_fmt); + break; default: av_assert2(!"Unreachable"); /* Intended fallthrough */ diff --git a/libavfilter/formats.h b/libavfilter/formats.h index a884d15213..94754ebd88 100644 --- a/libavfilter/formats.h +++ b/libavfilter/formats.h @@ -180,6 +180,9 @@ int ff_set_common_formats_from_list(AVFilterContext *ctx, const int *fmts); av_warn_unused_result int ff_add_channel_layout(AVFilterChannelLayouts **l, uint64_t channel_layout); +av_warn_unused_result +int ff_add_subtitle_type(AVFilterFormats **avff, int64_t fmt); + /** * Add *ref as a new reference to f. */ diff --git a/libavfilter/internal.h b/libavfilter/internal.h index fc09ef574c..192b0ae196 100644 --- a/libavfilter/internal.h +++ b/libavfilter/internal.h @@ -149,9 +149,11 @@ static av_always_inline int ff_filter_execute(AVFilterContext *ctx, avfilter_act enum FilterFormatsState { /** - * The default value meaning that this filter supports all formats - * and (for audio) sample rates and channel layouts/counts as long - * as these properties agree for all inputs and outputs. + * The default value meaning that this filter supports + * - For video: all formats + * - For audio: all sample rates and channel layouts/counts + * - For subtitles: all subtitle formats + * as long as these properties agree for all inputs and outputs. * This state is only allowed in case all inputs and outputs actually * have the same type. * The union is unused in this state. @@ -162,8 +164,10 @@ enum FilterFormatsState { FF_FILTER_FORMATS_QUERY_FUNC, ///< formats.query active. FF_FILTER_FORMATS_PIXFMT_LIST, ///< formats.pixels_list active. FF_FILTER_FORMATS_SAMPLEFMTS_LIST, ///< formats.samples_list active. + FF_FILTER_FORMATS_SUBFMTS_LIST, ///< formats.subs_list active. FF_FILTER_FORMATS_SINGLE_PIXFMT, ///< formats.pix_fmt active FF_FILTER_FORMATS_SINGLE_SAMPLEFMT, ///< formats.sample_fmt active. + FF_FILTER_FORMATS_SINGLE_SUBFMT, ///< formats.sub_fmt active. }; #define FILTER_QUERY_FUNC(func) \ @@ -175,16 +179,24 @@ enum FilterFormatsState { #define FILTER_SAMPLEFMTS_ARRAY(array) \ .formats.samples_list = array, \ .formats_state = FF_FILTER_FORMATS_SAMPLEFMTS_LIST +#define FILTER_SUBFMTS_ARRAY(array) \ + .formats.subs_list = array, \ + .formats_state = FF_FILTER_FORMATS_SUBFMTS_LIST #define FILTER_PIXFMTS(...) \ FILTER_PIXFMTS_ARRAY(((const enum AVPixelFormat []) { __VA_ARGS__, AV_PIX_FMT_NONE })) #define FILTER_SAMPLEFMTS(...) \ FILTER_SAMPLEFMTS_ARRAY(((const enum AVSampleFormat[]) { __VA_ARGS__, AV_SAMPLE_FMT_NONE })) +#define FILTER_SUBFMTS(...) \ + FILTER_SUBFMTS_ARRAY(((const enum AVSubtitleType[]) { __VA_ARGS__, AV_SUBTITLE_FMT_NONE })) #define FILTER_SINGLE_PIXFMT(pix_fmt_) \ .formats.pix_fmt = pix_fmt_, \ .formats_state = FF_FILTER_FORMATS_SINGLE_PIXFMT #define FILTER_SINGLE_SAMPLEFMT(sample_fmt_) \ .formats.sample_fmt = sample_fmt_, \ .formats_state = FF_FILTER_FORMATS_SINGLE_SAMPLEFMT +#define FILTER_SINGLE_SUBFMT(sub_fmt_) \ + .formats.sub_fmt = sub_fmt_, \ + .formats_state = FF_FILTER_FORMATS_SINGLE_SUBFMT #define FILTER_INOUTPADS(inout, array) \ .inout = array, \ From patchwork Mon Dec 13 23:40:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 32462 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp6140458iog; Mon, 13 Dec 2021 15:42:29 -0800 (PST) X-Google-Smtp-Source: ABdhPJyZiljgLN0sSTQbKq0d2mmVI3xe7k2qgPf1pLpdN6BePq3Yg+17Z8UQ5EnANMxEImyBbpXU X-Received: by 2002:a17:907:6ea6:: with SMTP id sh38mr1607330ejc.585.1639438949635; Mon, 13 Dec 2021 15:42:29 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id n11si17111338edo.578.2021.12.13.15.42.28; Mon, 13 Dec 2021 15:42:29 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=PlVP8j9z; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 99ECC68B0C3; Tue, 14 Dec 2021 01:40:48 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12olkn2082.outbound.protection.outlook.com [40.92.21.82]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CDA9668B0A0 for ; Tue, 14 Dec 2021 01:40:41 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Wt2qkPYE/ZjfMOMxjZgj+WNNV6MW8J6YcgPWh9w2c4AlhtoMHrmmj7w6RWuiDpt932sSs89/t5JgSHXAKU8L+7rpMRH8wnA3u4xD2BBy58OZXvu7NP19QmSH4Dlkr0u553F107+45+aSv3qF8uLl+lf8SDneR3XifYvvCj9+fde4L4uu9lQhHbUjcmsDvTaeWQV8MtC3Ef4eOlyBVloPohnx60ONfmF2G4Iv7iTI8LwmNSMke5utRvgUdg8HvxbyGkzPyu54WPAdcSPq+7s2fkzhQVpuv7a32HwkX48snWILhzFVzKIy/fZlxGGnplh2g6ZqxSxB2tqY7D+mXH0o9A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=jSi6/fKTW8CrEj1Ui4nO859/uABVA3v6MpWqEtcs02w=; b=GZZfK1X/VaKm/TVYN674CTQZb/NHKEIqt4GNLRYexqfqLbBtenHVF2GiIBfa045RSus0Q3oJimWiVBCdfuJfUsGanDWSr6/aKDWiQRpDjvhOMz773rnHpxtnaZcT54KrLkteVn0vWsaLTQqrPbJ3/6MRXAPy9zsEksN1EB9ZH9aTVJyNq8xs//kyaD19oBgIK/Hx5sQ8SVpxDwqJrpRdN31gKR2z4eZzwXdDKzeTNgfYtny95783rpA3jOpOPTm7Ba8kvkwKe6sXWpugYfS/ZMzij0UowFaIgAWex8fqFJqEJVyI+dZj6U71ZbnCk9pGe98/IDmPNlF7GLqtPQyv9Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jSi6/fKTW8CrEj1Ui4nO859/uABVA3v6MpWqEtcs02w=; b=PlVP8j9z3KlLjwMj4n05KDbBpBdio4XLTX2kxfN8ewm9SAhnmYyhSamugD65b01LXlMFWVGh74uO3USW40Bpk33X4PSmhPzDuWgn5hr/tXw9JOUDV6Xp/TzxUqBRqQsCWC+MaQctIXmoVwn19NyutQ7yuOpmX1W9tJFndpdCqJyAIPAcEYKHwN+uyFfs2mHJ/MlfL5WzKmmtrrEW9dyJ+YvARcqEsqSiL+w2dp6eO2fvTZ/B5dOCK4b3B3vT9rCLGpq0kYRF32kszzqqeWkmVGEtoCvEI1CcMegDj3oYz65cumqUT5G3RiyC6MhnQ9ScoBHwy0JbGLeKB6qyJN5Ecg== Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::20) by DM8P223MB0397.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4778.17; Mon, 13 Dec 2021 23:40:40 +0000 Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775]) by DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775%7]) with mapi id 15.20.4778.018; Mon, 13 Dec 2021 23:40:40 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v24 11/21] avfilter/sbuffer: Add sbuffersrc and sbuffersink filters Thread-Index: AQHX8HrWhPOpQCPNJ0u1YaSsiS956A== Date: Mon, 13 Dec 2021 23:40:40 +0000 Message-ID: References: <1ef42eb01b3627bbad539abd46206f320bb39697.1639438462.git.softworkz@hotmail.com> <4148d77a82d4b7e229fc81ce9bc1bc06904533f1.1639438462.git.softworkz@hotmail.com> <2ac798924ee984010d2e85f00bc992d9aa106171.1639438462.git.softworkz@hotmail.com> <300ea7188b331ca5e00dd7e04c03df8aba2a99c0.1639438462.git.softworkz@hotmail.com> <9474726b26f0d3e8f353d07caed2fbd33cdf473a.1639438462.git.softworkz@hotmail.com> <424522c96b5cdf6b03faa6bdceb1d25449463429.1639438462.git.softworkz@hotmail.com> <90caff245055005e6a4a890b3665da55aa853b31.1639438462.git.softworkz@hotmail.com> In-Reply-To: <90caff245055005e6a4a890b3665da55aa853b31.1639438462.git.softworkz@hotmail.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [09JMtovXBLUZVVgl9M7xSQP20uJ7XWId] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 331787c7-1fd6-47f5-33f7-08d9be91f92a x-ms-traffictypediagnostic: DM8P223MB0397:EE_ x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 0g/nQ6t5KxSSeNOkfK9iI49ImRaen/hwpsIZyS8m40g+wipgE4/ISjcPgmNhw6c6gTxtO/4YPYS2gPKCsYbvEUrqVfiMzac58mGETc21OG+Nk3tWUQ4ZT5nqvi5qE+2+bdu2E0cttL5yLYClCPG+TBA/wuNnCTPtja/HBgRM3RADNdqJ9vvFxay1fcMp2LSbhOSHvf53CTdTTqCL1j2XbKINNUZF5jFUB1Yw99MqLen8vSmz6NMUIWz5DM97Yd+07SreUiEFrtlQPgrf3yEDsv6SwfeX11NSEx8oYtg31LdYBzVQxK/ZNqpbHSSxOHok/JfIuJeNv7aAE3kNTLY43DObrtZz3m5qOMVlvDXRJ2CrPxpgOo5aCBGEd8810DaIUudD5HaNFE4GYzGTfII44ovevzuqwvt1PxmInk2RHPBcaZsi+pQhbEbmb22HSzgCVGDjbqlnp5+5g6s9KltuPdqngNPtTCQk/uKGziF0PMgdP6rGkXn6+8Q9qg9r7w3WEHJ3Z+WPNxQ8peKyRmpbVR5EPZxVTcTTpNaIQ5Pu4e3g3b/KbbKXo8QFmi+zS38fRFFfkJ1oidor6QOlSl3h0w== x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?q?s86XJC7wojdn+OtTfwJrT+xjSQ2G?= =?utf-8?q?FrqwX6Jnp9z9kdAoP89xl8btqqISz/7gyDNIH+Z+WlaZYz7sT+bAUOWakV3X8CSgo?= =?utf-8?q?p+dJYtPmao0byccx3xMIAL5c2qUNVRx1lo9M2nwxVNnhNVTJMs2/Gk/+22G7HyVQ0?= =?utf-8?q?gOL8YEAPLF7G36XmGsjYxSUC2YhycDKZTmsyB1YidyEevSm/Srm3TZJZrZ7iMuy08?= =?utf-8?q?MrhKmANCzDVkcBmAvtGyFW4KmNw5CsBbJYMVbCpE/L/wJal+iGAOKeCtKeUrKa9L3?= =?utf-8?q?S/94n233wFe6PGo3Fecpkf7NgsjlLu2bio51+cAUHZpcrpUsYeLP3uOgSDBjlgv/8?= =?utf-8?q?1BR7e+9VU7UTI4qBOWEnz180fJVzl8xSudb7JMZgHwiFv5IndDzfj/r6J4Z/S8AZS?= =?utf-8?q?ilfh7cVPFsWqPSWcZvaLN7Ke4Yhppdwismyc4/aPcA5SNAlQB2D9MHLeuT9vVfMdN?= =?utf-8?q?iNSFzyOu1S0y4c1uO3gSZmLKOjPJrwzPvSzjYJDsHBG6M5TuGMXkebfzCrxBRVNY3?= =?utf-8?q?xQbXVr5+zFwbOhnoSKS8geozSH/EBZV0ctFdbDxnpLOKVMFmBG2c/DW08oLuYmPc8?= =?utf-8?q?HtnGa1Bbs8lJVpcVQ5BeiaEe2bq4RzCjJSPVeNiQjc6aa1CmFbJxUHTIe2cXJYg5N?= =?utf-8?q?I6fpJq5DJLsJO3p5jLT496mJ/qhbaSVOSPfSgWWpweNSF5gqv+0hcPlrdO2fYhcUx?= =?utf-8?q?z+hSNvZno0X0D/neCLokCbzQveJpaoWqOVhmK0H2KOe7CP+zPfwuL8sZjexpLHde6?= =?utf-8?q?OLjMSPe3EzTBc7YrR6BR0ZTd9KocLVTt5uG1x9l8f3NLg81EpP4Sime7uIGlzgRHJ?= =?utf-8?q?rD7EiQXl5TrURFbwzhuxWvERqowLw9xZ8DaYF8W7UL2jQ02iPPxSWE8msQiMLxYav?= =?utf-8?q?F50iQP3hEII2u4oCEOwYk7JiPVPvwILhlUUlJ4P8CQIw=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-1ff67.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 331787c7-1fd6-47f5-33f7-08d9be91f92a X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Dec 2021 23:40:40.8826 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8P223MB0397 Subject: [FFmpeg-devel] [PATCH v24 11/21] avfilter/sbuffer: Add sbuffersrc and sbuffersink filters X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: ZQHyo9aSBm4j Signed-off-by: softworkz --- configure | 2 +- libavfilter/allfilters.c | 2 ++ libavfilter/buffersink.c | 54 ++++++++++++++++++++++++++++++ libavfilter/buffersink.h | 7 ++++ libavfilter/buffersrc.c | 72 ++++++++++++++++++++++++++++++++++++++++ libavfilter/buffersrc.h | 1 + 6 files changed, 137 insertions(+), 1 deletion(-) diff --git a/configure b/configure index a7593ec2db..0d0310cc9d 100755 --- a/configure +++ b/configure @@ -7809,7 +7809,7 @@ print_enabled_components(){ fi done if [ "$name" = "filter_list" ]; then - for c in asrc_abuffer vsrc_buffer asink_abuffer vsink_buffer; do + for c in asrc_abuffer vsrc_buffer ssrc_sbuffer asink_abuffer vsink_buffer ssink_sbuffer; do printf " &ff_%s,\n" $c >> $TMPH done fi diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index ec57a2c49c..f98e1a594c 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -555,8 +555,10 @@ extern const AVFilter ff_avsrc_movie; * being the same while having different 'types'). */ extern const AVFilter ff_asrc_abuffer; extern const AVFilter ff_vsrc_buffer; +extern const AVFilter ff_ssrc_sbuffer; extern const AVFilter ff_asink_abuffer; extern const AVFilter ff_vsink_buffer; +extern const AVFilter ff_ssink_sbuffer; extern const AVFilter ff_af_afifo; extern const AVFilter ff_vf_fifo; diff --git a/libavfilter/buffersink.c b/libavfilter/buffersink.c index c0215669e7..0b268c2fa4 100644 --- a/libavfilter/buffersink.c +++ b/libavfilter/buffersink.c @@ -29,6 +29,8 @@ #include "libavutil/internal.h" #include "libavutil/opt.h" +#include "libavcodec/avcodec.h" + #define FF_INTERNAL_FIELDS 1 #include "framequeue.h" @@ -57,6 +59,10 @@ typedef struct BufferSinkContext { int *sample_rates; ///< list of accepted sample rates int sample_rates_size; + /* only used for subtitles */ + enum AVSubtitleType *subtitle_types; ///< list of accepted subtitle types, must be terminated with -1 + int subtitle_types_size; + AVFrame *peeked_frame; } BufferSinkContext; @@ -305,6 +311,28 @@ static int asink_query_formats(AVFilterContext *ctx) return 0; } +static int ssink_query_formats(AVFilterContext *ctx) +{ + BufferSinkContext *buf = ctx->priv; + AVFilterFormats *formats = NULL; + unsigned i; + int ret; + + CHECK_LIST_SIZE(subtitle_types) + if (buf->subtitle_types_size) { + for (i = 0; i < NB_ITEMS(buf->subtitle_types); i++) + if ((ret = ff_add_subtitle_type(&formats, buf->subtitle_types[i])) < 0) + return ret; + if ((ret = ff_set_common_formats(ctx, formats)) < 0) + return ret; + } else { + if ((ret = ff_default_query_formats(ctx)) < 0) + return ret; + } + + return 0; +} + #define OFFSET(x) offsetof(BufferSinkContext, x) #define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM static const AVOption buffersink_options[] = { @@ -322,9 +350,16 @@ static const AVOption abuffersink_options[] = { { NULL }, }; #undef FLAGS +#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_SUBTITLE_PARAM +static const AVOption sbuffersink_options[] = { + { "subtitle_types", "set the supported subtitle formats", OFFSET(subtitle_types), AV_OPT_TYPE_BINARY, .flags = FLAGS }, + { NULL }, +}; +#undef FLAGS AVFILTER_DEFINE_CLASS(buffersink); AVFILTER_DEFINE_CLASS(abuffersink); +AVFILTER_DEFINE_CLASS(sbuffersink); static const AVFilterPad avfilter_vsink_buffer_inputs[] = { { @@ -363,3 +398,22 @@ const AVFilter ff_asink_abuffer = { .outputs = NULL, FILTER_QUERY_FUNC(asink_query_formats), }; + +static const AVFilterPad avfilter_ssink_sbuffer_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + }, +}; + +const AVFilter ff_ssink_sbuffer = { + .name = "sbuffersink", + .description = NULL_IF_CONFIG_SMALL("Buffer subtitle frames, and make them available to the end of the filter graph."), + .priv_class = &sbuffersink_class, + .priv_size = sizeof(BufferSinkContext), + .init = common_init, + .activate = activate, + FILTER_INPUTS(avfilter_ssink_sbuffer_inputs), + .outputs = NULL, + FILTER_QUERY_FUNC(ssink_query_formats), +}; diff --git a/libavfilter/buffersink.h b/libavfilter/buffersink.h index 69ed0f29a8..11905abdc5 100644 --- a/libavfilter/buffersink.h +++ b/libavfilter/buffersink.h @@ -129,6 +129,13 @@ typedef struct AVABufferSinkParams { */ attribute_deprecated AVABufferSinkParams *av_abuffersink_params_alloc(void); + +/** + * Deprecated and unused struct to use for initializing an sbuffersink context. + */ +typedef struct AVSBufferSinkParams { + const int *subtitle_type; +} AVSBufferSinkParams; #endif /** diff --git a/libavfilter/buffersrc.c b/libavfilter/buffersrc.c index b0611872f1..d2362999a2 100644 --- a/libavfilter/buffersrc.c +++ b/libavfilter/buffersrc.c @@ -39,6 +39,7 @@ #include "formats.h" #include "internal.h" #include "video.h" +#include "libavcodec/avcodec.h" typedef struct BufferSourceContext { const AVClass *class; @@ -63,6 +64,9 @@ typedef struct BufferSourceContext { uint64_t channel_layout; char *channel_layout_str; + /* subtitle only */ + enum AVSubtitleType subtitle_type; + int eof; } BufferSourceContext; @@ -130,6 +134,13 @@ int av_buffersrc_parameters_set(AVFilterContext *ctx, AVBufferSrcParameters *par if (param->channel_layout) s->channel_layout = param->channel_layout; break; + case AVMEDIA_TYPE_SUBTITLE: + s->subtitle_type = param->format; + if (param->width > 0) + s->w = param->width; + if (param->height > 0) + s->h = param->height; + break; default: return AVERROR_BUG; } @@ -197,6 +208,8 @@ int attribute_align_arg av_buffersrc_add_frame_flags(AVFilterContext *ctx, AVFra CHECK_AUDIO_PARAM_CHANGE(ctx, s, frame->sample_rate, frame->channel_layout, frame->channels, frame->format, frame->pts); break; + case AVMEDIA_TYPE_SUBTITLE: + break; default: return AVERROR(EINVAL); } @@ -269,6 +282,7 @@ unsigned av_buffersrc_get_nb_failed_requests(AVFilterContext *buffer_src) #define OFFSET(x) offsetof(BufferSourceContext, x) #define A AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_AUDIO_PARAM #define V AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM +#define S AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_SUBTITLE_PARAM static const AVOption buffer_options[] = { { "width", NULL, OFFSET(w), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, V }, @@ -298,6 +312,16 @@ static const AVOption abuffer_options[] = { AVFILTER_DEFINE_CLASS(abuffer); +static const AVOption sbuffer_options[] = { + { "time_base", NULL, OFFSET(time_base), AV_OPT_TYPE_RATIONAL, { .dbl = 0 }, 0, INT_MAX, S }, + { "subtitle_type", NULL, OFFSET(subtitle_type), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, S }, + { "width", NULL, OFFSET(w), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, V }, + { "height", NULL, OFFSET(h), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, V }, + { NULL }, +}; + +AVFILTER_DEFINE_CLASS(sbuffer); + static av_cold int init_audio(AVFilterContext *ctx) { BufferSourceContext *s = ctx->priv; @@ -347,6 +371,21 @@ static av_cold int init_audio(AVFilterContext *ctx) return ret; } +static av_cold int init_subtitle(AVFilterContext *ctx) +{ + BufferSourceContext *c = ctx->priv; + + if (c->subtitle_type == AV_SUBTITLE_FMT_BITMAP) + av_log(ctx, AV_LOG_VERBOSE, "graphical subtitles - w:%d h:%d tb:%d/%d\n", + c->w, c->h, c->time_base.num, c->time_base.den); + else + av_log(ctx, AV_LOG_VERBOSE, "text subtitles - w:%d h:%d tb:%d/%d\n", + c->w, c->h, c->time_base.num, c->time_base.den); + + return 0; +} + + static av_cold void uninit(AVFilterContext *ctx) { BufferSourceContext *s = ctx->priv; @@ -381,6 +420,11 @@ static int query_formats(AVFilterContext *ctx) if ((ret = ff_set_common_channel_layouts(ctx, channel_layouts)) < 0) return ret; break; + case AVMEDIA_TYPE_SUBTITLE: + if ((ret = ff_add_format (&formats, c->subtitle_type)) < 0 || + (ret = ff_set_common_formats (ctx , formats )) < 0) + return ret; + break; default: return AVERROR(EINVAL); } @@ -408,6 +452,11 @@ static int config_props(AVFilterLink *link) if (!c->channel_layout) c->channel_layout = link->channel_layout; break; + case AVMEDIA_TYPE_SUBTITLE: + link->format = c->subtitle_type; + link->w = c->w; + link->h = c->h; + break; default: return AVERROR(EINVAL); } @@ -472,3 +521,26 @@ const AVFilter ff_asrc_abuffer = { FILTER_QUERY_FUNC(query_formats), .priv_class = &abuffer_class, }; + +static const AVFilterPad ssrc_sbuffer_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .request_frame = request_frame, + .config_props = config_props, + }, +}; + +const AVFilter ff_ssrc_sbuffer = { + .name = "sbuffer", + .description = NULL_IF_CONFIG_SMALL("Buffer subtitle frames, and make them accessible to the filterchain."), + .priv_size = sizeof(BufferSourceContext), + + .init = init_subtitle, + .uninit = uninit, + + .inputs = NULL, + FILTER_OUTPUTS(ssrc_sbuffer_outputs), + FILTER_QUERY_FUNC(query_formats), + .priv_class = &sbuffer_class, +}; diff --git a/libavfilter/buffersrc.h b/libavfilter/buffersrc.h index 08fbd18a47..929a2fa249 100644 --- a/libavfilter/buffersrc.h +++ b/libavfilter/buffersrc.h @@ -74,6 +74,7 @@ typedef struct AVBufferSrcParameters { /** * video: the pixel format, value corresponds to enum AVPixelFormat * audio: the sample format, value corresponds to enum AVSampleFormat + * subtitles: the subtitle format, value corresponds to enum AVSubtitleType */ int format; /** From patchwork Mon Dec 13 23:40:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 32463 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp6140622iog; Mon, 13 Dec 2021 15:42:40 -0800 (PST) X-Google-Smtp-Source: ABdhPJxWbNOkIK7MGSUjY11d8jwQUxt/XhZbXYVwu/2mr8I9zekgv8W9h0Q9dfz+8LOCFGICAzPi X-Received: by 2002:a05:6402:440b:: with SMTP id y11mr2779706eda.25.1639438960039; Mon, 13 Dec 2021 15:42:40 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id g3si26745367ejt.525.2021.12.13.15.42.39; Mon, 13 Dec 2021 15:42:40 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=r1raE6Zp; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 926BB68B0C6; Tue, 14 Dec 2021 01:40:49 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12olkn2104.outbound.protection.outlook.com [40.92.21.104]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9BA0368B03C for ; Tue, 14 Dec 2021 01:40:45 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=V9sW3fve1rck+9bACXaL3vFocEGs5ex8UQWBJKA9yswAPz4PRgKv4koXPU4wqDdc6lBitVDrRePr0Qi9mnPVIojWSG4/t8+RuPX5/rhb+9P3GHwayjVHhUVse187ptWpCZIl8ONIpNYCMtV4Z8mASV5EppdqE1seEa1Wo466UCidBVMXRJqRR6+NB0jhH2PXbKDwOdbk5A+A4lLHR62yklky0mSr2oWf79L7rqE5Sl7kFkuscygGi2Nkxa62iBvIrDYb5xXdRdL1pxH7zNGiYkbsLNF6fy+I5O01yedts9joWqZnUPoMuYwwdn/XRT0IlYZue2uekz0TzK/LJHma7g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=VDkhUdjRCU1JPqybF7reEyZPXTRv/wFFQLS0+qZd/7w=; b=ZFK0KzGtPvMZJSCuxiRype2bDrX9fCzbAEO5Kj66CjudPtzv6Z+p+nZhxp8BnnJZMpuy7CqRBi6W+LSV4H+CP9k17+J/4Zro9GXnFZqQH8G+TLUt6zfnl8qiCI9vrHZB3ogE6YkvS7jUEhzD7mCqlDyrFUKYgSFqHOtk/P+ydEkbBVICC8RgAL3r5jQ5ooTvg3TQE96AZMERTWtdjetRHKyMoxa+w6CpQdWUvOeaX6u2cY3SBagM1/BPUFgP3cRwGMoW+cD/9v46vpxkm37firtgxpOJ9ciNEQD4A79X8vTlhyFx0sNvEfoghGTAOY29ew/MryLsIJYn/Ic0jf2xAA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VDkhUdjRCU1JPqybF7reEyZPXTRv/wFFQLS0+qZd/7w=; b=r1raE6ZpzLyQVUA+ysgLKwavBzQf4rxKFBJ2mpmnhE6HL9uQEbFaUNi2Kz3O93ke6BlyzsyQ5UMqvnIGtCDYdQR+AbH1CTWo1/JwYC5MKWkMX0NNvMIt0YHwtaM/sH3zQgIXnHWdJAqX4RsOO8v4Sq7CaS4IQGBUw1yAj8yocE0wXmr36WGPyZONiJNu1FTWFKokjspIqLb+G9GXeamQWni5osSjjXsSn5P9kxCw9BfTylHACAZLzTWrw0fdi5wMX51B3PfTRzcQRwGy5QwwxNG1+GUl64CvlIsHw3imGl2/xsgUYkesrxMNtbvo3VRisW3budTu4xbilXN6jKIBJQ== Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::20) by DM8P223MB0397.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4778.17; Mon, 13 Dec 2021 23:40:43 +0000 Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775]) by DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775%7]) with mapi id 15.20.4778.018; Mon, 13 Dec 2021 23:40:43 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v24 12/21] avfilter/overlaygraphicsubs: Add overlaygraphicsubs and graphicsub2video filters Thread-Index: AQHX8HrY7teblWeBA06cxQSCvejswg== Date: Mon, 13 Dec 2021 23:40:43 +0000 Message-ID: References: <1ef42eb01b3627bbad539abd46206f320bb39697.1639438462.git.softworkz@hotmail.com> <4148d77a82d4b7e229fc81ce9bc1bc06904533f1.1639438462.git.softworkz@hotmail.com> <2ac798924ee984010d2e85f00bc992d9aa106171.1639438462.git.softworkz@hotmail.com> <300ea7188b331ca5e00dd7e04c03df8aba2a99c0.1639438462.git.softworkz@hotmail.com> <9474726b26f0d3e8f353d07caed2fbd33cdf473a.1639438462.git.softworkz@hotmail.com> <424522c96b5cdf6b03faa6bdceb1d25449463429.1639438462.git.softworkz@hotmail.com> <90caff245055005e6a4a890b3665da55aa853b31.1639438462.git.softworkz@hotmail.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [qi2/FtMd0kttXoQEke333NWfBsr01BZN] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 058192e8-4dc6-4079-88de-08d9be91faa0 x-ms-traffictypediagnostic: DM8P223MB0397:EE_ x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: RzYw2JS9b+qi6dbxmwxaQ3nQToRkD0NsqT+4X74Va4GV8JTND+DFILCZXEH7nD2FChG5VKdvZTOimW1weA/9VG9iQHLhbf0pzLKo3VGWMPRt3XN3aRnfNXRimlvvTKtZGkA1Wj1pNMx4lsyo3SG8Itt/VPUNwB3ALK6fXke299/oXbjzEqBV0l1BE8gmaVDtM2Tp1c9tUq6bHRRbsEuH8oPYDDuRrzbyHO2J+9Rb7RmsStGLWDQOEgSY1CKPcUVwmnFUqJMyN1WBi+/p+U/jMcyvTM3bzlvcK6IdJYpVwm1r0WR8IBxbMYJWt/sIHBniCJ+tokmtsrmWxgs6Be2Io7HXddXyBEKfORntj6VvYUDkPyzau/UQj2WRofM8J4yPt9Ky7trMLrnqz3ToZXoUen48mnPlRnukDOg5NKFjXKfFcmBR2g7LKsjrJR6mbdvuqcW5/ntl1PI112fATqIlnV5nt8QfnmWUGHDSVwT7b0+K7KCskZy0uokD535b97a+3FU0R0LK02CoaBeVkT/Yaikevv4WFtBYp/5D9i61sQKj543dtb8XT7tNM6PwQA5tlTWfkr0ThJV0yNOaBgpIEGUz5lfsydUHudb9F6IfbXO6Ibe9RHS8gFIejzmRMps2 x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?q?TFOw/6J/FhLd/9a/YsZNVNqyFb9u?= =?utf-8?q?xErGvVnieUJRADJM45HtvuQH3l7joSvAg7sGzSIyFqpsrmidUJC3jr5reM0dS3Cge?= =?utf-8?q?fXOAsH/gUC4oO/A9T99AJWVHsgxgILUVUh0oOTh44KiFQc95PBgEl96bW5mdmUiz0?= =?utf-8?q?UxS0mkeP2wk2+u9R5HMuveja5LTVT7JlJwoDQETlflEM5IdV9uH1/rf/JXoiympPs?= =?utf-8?q?H7IbSsfSJBJpKimDmV20thM/ifjEbyltMb00biCXLxKv/M3DYTsaCHzVS2NOopuDL?= =?utf-8?q?nH8+MYUvfhKQa8Zt0Y6cQNN0gnndpDHc3Jctmafc4QM18savipMmSCsCZso/u1o6U?= =?utf-8?q?a1K/AqwnQcQuagPOdeX4i4y8jyf5m27k9ypxaMp4qgnNdckeNAwITfcbjUdivteiN?= =?utf-8?q?JqImDOK4EnAW9H0n9uF8f3kdaP1wIBRfR+l+yD29EuEohCHozJ/wO2T9CYRVWVTb9?= =?utf-8?q?NW2oK3lKVHQgrX+VIbg8G/V/fi3z0u8BjH1NKtA2GCnBxfVayXug+I+3tN0Z+qSu4?= =?utf-8?q?VHIp1MZulo/IqFKVuRpSAJsPDxgoF+4RMRkgA9DSQGrZ65f+Dv6EOL5r/0KztmP9f?= =?utf-8?q?tU+Tg0Vm5xz3e/dFTttJujL5Q/jf3Rl9vVWI/Nv+rtFaP0xXvF3JHj86Jf9g6lAWS?= =?utf-8?q?YN8XmC6EM3cCRsQFXK8uafg7wLiaiaWYA+GXAd5Cfl7UF21gifaEMFtQRsmOqPYcK?= =?utf-8?q?wS+xMfpGdkFfTmaSsWXThIziEjvc2WWcMV5b8EE0zCEI4LfQ5KMAWn6N1o8KAWGJd?= =?utf-8?q?dfNPVY96HXjGEqpdhvWdkpC2PVsRmZL1EZeNxTjzl5Yh0TP75b+YiVo2fmEEXLWMH?= =?utf-8?q?/ifObaInJGnV19IwSEejiBhr7kC03WkA4ckp+odtYugydR6MQOwxX/ByVIP4Rmcvo?= =?utf-8?q?/0cEvTDBgAFw8E+NXwRrMabAGVZXwk+lq7LyKg4afB9g=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-1ff67.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 058192e8-4dc6-4079-88de-08d9be91faa0 X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Dec 2021 23:40:43.3109 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8P223MB0397 Subject: [FFmpeg-devel] [PATCH v24 12/21] avfilter/overlaygraphicsubs: Add overlaygraphicsubs and graphicsub2video filters X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: zREIcX+DWkyU - overlaygraphicsubs (VS -> V) Overlay graphic subtitles onto a video stream - graphicsub2video {S -> V) Converts graphic subtitles to video frames (with alpha) Gets auto-inserted for retaining compatibility with sub2video command lines Signed-off-by: softworkz --- doc/filters.texi | 118 +++++ libavfilter/Makefile | 2 + libavfilter/allfilters.c | 2 + libavfilter/vf_overlaygraphicsubs.c | 742 ++++++++++++++++++++++++++++ 4 files changed, 864 insertions(+) create mode 100644 libavfilter/vf_overlaygraphicsubs.c diff --git a/doc/filters.texi b/doc/filters.texi index cffec8168c..e7fd243ac6 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -25712,6 +25712,124 @@ tools. @c man end VIDEO SINKS +@chapter Subtitle Filters +@c man begin SUBTITLE FILTERS + +When you configure your FFmpeg build, you can disable any of the +existing filters using @code{--disable-filters}. + +Below is a description of the currently available subtitle filters. + +@section graphicsub2video + +Renders graphic subtitles as video frames. + +This filter replaces the previous "sub2video" hack which did the conversion implicitly and up-front as subtitle filtering wasn't possible at that time. +To retain compatibility with earlier sub2video command lines, this filter is being auto-inserted in those cases. + +For overlaying graphicsal subtitles it is recommended to use the 'overlay_graphicsubs' filter which is more efficient and takes less processing resources. + +This filter is still useful in cases where the overlay is done with hardware acceleration (e.g. overlay_qsv, overlay_vaapi, overlay_cuda) for preparing the overlay frames. + +Inputs: +@itemize +@item 0: Subtitles [BITMAP] +@end itemize + +Outputs: +@itemize +@item 0: Video [RGB32] +@end itemize + + +It accepts the following parameters: + +@table @option +@item size, s +Set the size of the output video frame. + +@end table + +@subsection Examples + +@itemize +@item +Overlay PGS subtitles +(not recommended - better use overlay_graphicsubs) +@example +ffmpeg -i "https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv" -filter_complex "[0:1]graphicsub2video[subs];[0:0][subs]overlay" output.mp4 +@end example + +@item +Overlay PGS subtitles implicitly +The graphicsub2video is inserted automatically for compatibility with legacy command lines. +@example +ffmpeg -i "https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv" -filter_complex "[0:0][0:1]overlay" output.mp4 +@end example +@end itemize + +@section overlaygraphicsubs + +Overlay graphic subtitles onto a video stream. + +This filter can blend graphical subtitles on a video stream directly, i.e. without creating full-size alpha images first. +The blending operation is limited to the area of the subtitle rectangles, which also means that no processing is done at times where no subtitles are to be displayed. + +Inputs: +@itemize +@item 0: Video [YUV420P, YUV422P, YUV444P, ARGB, RGBA, ABGR, BGRA, RGB24, BGR24] +@item 1: Subtitles [BITMAP] +@end itemize + +Outputs: +@itemize +@item 0: Video (same as input) +@end itemize + +It accepts the following parameters: + +@table @option +@item x +@item y +Set the expression for the x and y coordinates of the overlaid video +on the main video. Default value is "0" for both expressions. In case +the expression is invalid, it is set to a huge value (meaning that the +overlay will not be displayed within the output visible area). + +@item eof_action +See @ref{framesync}. + +@item eval +Set when the expressions for @option{x}, and @option{y} are evaluated. + +It accepts the following values: +@table @samp +@item init +only evaluate expressions once during the filter initialization or +when a command is processed + +@item frame +evaluate expressions for each incoming frame +@end table + +Default value is @samp{frame}. + +@item shortest +See @ref{framesync}. + +@end table + +@subsection Examples + +@itemize +@item +Overlay PGS subtitles +@example +ffmpeg -i "https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv" -filter_complex "[0:0][0:1]overlaygraphicsubs" output.mp4 +@end example +@end itemize +@c man end SUBTITLE FILTERS + @chapter Multimedia Filters @c man begin MULTIMEDIA FILTERS diff --git a/libavfilter/Makefile b/libavfilter/Makefile index a2f61d39f0..03ecf92ec2 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -296,6 +296,7 @@ OBJS-$(CONFIG_GBLUR_FILTER) += vf_gblur.o OBJS-$(CONFIG_GBLUR_VULKAN_FILTER) += vf_gblur_vulkan.o vulkan.o vulkan_filter.o OBJS-$(CONFIG_GEQ_FILTER) += vf_geq.o OBJS-$(CONFIG_GRADFUN_FILTER) += vf_gradfun.o +OBJS-$(CONFIG_GRAPHICSUB2VIDEO_FILTER) += vf_overlaygraphicsubs.o framesync.o OBJS-$(CONFIG_GRAPHMONITOR_FILTER) += f_graphmonitor.o OBJS-$(CONFIG_GRAYWORLD_FILTER) += vf_grayworld.o OBJS-$(CONFIG_GREYEDGE_FILTER) += vf_colorconstancy.o @@ -376,6 +377,7 @@ OBJS-$(CONFIG_OVERLAY_OPENCL_FILTER) += vf_overlay_opencl.o opencl.o \ opencl/overlay.o framesync.o OBJS-$(CONFIG_OVERLAY_QSV_FILTER) += vf_overlay_qsv.o framesync.o OBJS-$(CONFIG_OVERLAY_VULKAN_FILTER) += vf_overlay_vulkan.o vulkan.o vulkan_filter.o +OBJS-$(CONFIG_OVERLAYGRAPHICSUBS_FILTER) += vf_overlaygraphicsubs.o framesync.o OBJS-$(CONFIG_OWDENOISE_FILTER) += vf_owdenoise.o OBJS-$(CONFIG_PAD_FILTER) += vf_pad.o OBJS-$(CONFIG_PAD_OPENCL_FILTER) += vf_pad_opencl.o opencl.o opencl/pad.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index f98e1a594c..f8bf143fbe 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -358,6 +358,7 @@ extern const AVFilter ff_vf_oscilloscope; extern const AVFilter ff_vf_overlay; extern const AVFilter ff_vf_overlay_opencl; extern const AVFilter ff_vf_overlay_qsv; +extern const AVFilter ff_vf_overlaygraphicsubs; extern const AVFilter ff_vf_overlay_vulkan; extern const AVFilter ff_vf_overlay_cuda; extern const AVFilter ff_vf_owdenoise; @@ -544,6 +545,7 @@ extern const AVFilter ff_avf_showvolume; extern const AVFilter ff_avf_showwaves; extern const AVFilter ff_avf_showwavespic; extern const AVFilter ff_vaf_spectrumsynth; +extern const AVFilter ff_svf_graphicsub2video; /* multimedia sources */ extern const AVFilter ff_avsrc_amovie; diff --git a/libavfilter/vf_overlaygraphicsubs.c b/libavfilter/vf_overlaygraphicsubs.c new file mode 100644 index 0000000000..efbbbdcbc7 --- /dev/null +++ b/libavfilter/vf_overlaygraphicsubs.c @@ -0,0 +1,742 @@ +/* + * Copyright (c) 2021 softworkz (derived from vf_overlay) + * Copyright (c) 2010 Stefano Sabatini + * Copyright (c) 2010 Baptiste Coudurier + * Copyright (c) 2007 Bobby Bingham + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * overlay graphical subtitles on top of a video frame + */ + +#include "libavutil/eval.h" +#include "libavutil/imgutils.h" +#include "libavutil/opt.h" +#include "internal.h" +#include "drawutils.h" +#include "framesync.h" + +enum var_name { + VAR_MAIN_W, VAR_MW, + VAR_MAIN_H, VAR_MH, + VAR_OVERLAY_W, VAR_OW, + VAR_OVERLAY_H, VAR_OH, + VAR_HSUB, + VAR_VSUB, + VAR_X, + VAR_Y, + VAR_N, + VAR_POS, + VAR_T, + VAR_VARS_NB +}; + +typedef struct OverlaySubsContext { + const AVClass *class; + int x, y; ///< position of overlaid picture + int w, h; + AVFrame *outpicref; + + int main_is_packed_rgb; + uint8_t main_rgba_map[4]; + int main_has_alpha; + uint8_t overlay_rgba_map[4]; + int eval_mode; ///< EvalMode + + FFFrameSync fs; + + int main_pix_step[4]; ///< steps per pixel for each plane of the main output + int hsub, vsub; ///< chroma subsampling values + const AVPixFmtDescriptor *main_desc; ///< format descriptor for main input + + double var_values[VAR_VARS_NB]; + char *x_expr, *y_expr; + + AVExpr *x_pexpr, *y_pexpr; + + int pic_counter; +} OverlaySubsContext; + +static const char *const var_names[] = { + "main_w", "W", ///< width of the main video + "main_h", "H", ///< height of the main video + "overlay_w", "w", ///< width of the overlay video + "overlay_h", "h", ///< height of the overlay video + "hsub", + "vsub", + "x", + "y", + "n", ///< number of frame + "pos", ///< position in the file + "t", ///< timestamp expressed in seconds + NULL +}; + +#define MAIN 0 +#define OVERLAY 1 + +#define R 0 +#define G 1 +#define B 2 +#define A 3 + +#define Y 0 +#define U 1 +#define V 2 + +enum EvalMode { + EVAL_MODE_INIT, + EVAL_MODE_FRAME, + EVAL_MODE_NB +}; + +static av_cold void overlay_graphicsubs_uninit(AVFilterContext *ctx) +{ + OverlaySubsContext *s = ctx->priv; + + ff_framesync_uninit(&s->fs); + av_expr_free(s->x_pexpr); s->x_pexpr = NULL; + av_expr_free(s->y_pexpr); s->y_pexpr = NULL; +} + +static inline int normalize_xy(double d, int chroma_sub) +{ + if (isnan(d)) + return INT_MAX; + return (int)d & ~((1 << chroma_sub) - 1); +} + +static void eval_expr(AVFilterContext *ctx) +{ + OverlaySubsContext *s = ctx->priv; + + s->var_values[VAR_X] = av_expr_eval(s->x_pexpr, s->var_values, NULL); + s->var_values[VAR_Y] = av_expr_eval(s->y_pexpr, s->var_values, NULL); + /* It is necessary if x is expressed from y */ + s->var_values[VAR_X] = av_expr_eval(s->x_pexpr, s->var_values, NULL); + s->x = normalize_xy(s->var_values[VAR_X], s->hsub); + s->y = normalize_xy(s->var_values[VAR_Y], s->vsub); +} + +static int set_expr(AVExpr **pexpr, const char *expr, const char *option, void *log_ctx) +{ + int ret; + AVExpr *old = NULL; + + if (*pexpr) + old = *pexpr; + ret = av_expr_parse(pexpr, expr, var_names, + NULL, NULL, NULL, NULL, 0, log_ctx); + if (ret < 0) { + av_log(log_ctx, AV_LOG_ERROR, + "Error when evaluating the expression '%s' for %s\n", + expr, option); + *pexpr = old; + return ret; + } + + av_expr_free(old); + return 0; +} + +static int overlay_graphicsubs_query_formats(AVFilterContext *ctx) +{ + AVFilterFormats *formats; + AVFilterLink *inlink0 = ctx->inputs[0]; + AVFilterLink *inlink1 = ctx->inputs[1]; + AVFilterLink *outlink = ctx->outputs[0]; + int ret; + static const enum AVSubtitleType subtitle_fmts[] = { AV_SUBTITLE_FMT_BITMAP, AV_SUBTITLE_FMT_NONE }; + static const enum AVPixelFormat supported_pix_fmts[] = { + AV_PIX_FMT_YUV420P, AV_PIX_FMT_YUV422P, AV_PIX_FMT_YUV444P, + AV_PIX_FMT_ARGB, AV_PIX_FMT_RGBA, + AV_PIX_FMT_ABGR, AV_PIX_FMT_BGRA, + AV_PIX_FMT_RGB24, AV_PIX_FMT_BGR24, + AV_PIX_FMT_NONE + }; + + /* set input0 video formats */ + formats = ff_make_format_list(supported_pix_fmts); + if ((ret = ff_formats_ref(formats, &inlink0->outcfg.formats)) < 0) + return ret; + + /* set output0 video formats */ + if ((ret = ff_formats_ref(formats, &outlink->incfg.formats)) < 0) + return ret; + + /* set input1 subtitle formats */ + formats = ff_make_format_list(subtitle_fmts); + if ((ret = ff_formats_ref(formats, &inlink1->outcfg.formats)) < 0) + return ret; + + return 0; +} + +static int config_output(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + OverlaySubsContext *s = ctx->priv; + int ret; + + if ((ret = ff_framesync_init_dualinput(&s->fs, ctx)) < 0) + return ret; + + outlink->w = ctx->inputs[MAIN]->w; + outlink->h = ctx->inputs[MAIN]->h; + outlink->time_base = ctx->inputs[MAIN]->time_base; + + return ff_framesync_configure(&s->fs); +} + +// divide by 255 and round to nearest +// apply a fast variant: (X+127)/255 = ((X+127)*257+257)>>16 = ((X+128)*257)>>16 +#define FAST_DIV255(x) ((((x) + 128) * 257) >> 16) + +// calculate the non-pre-multiplied alpha, applying the general equation: +// alpha = alpha_overlay / ( (alpha_main + alpha_overlay) - (alpha_main * alpha_overlay) ) +// (((x) << 16) - ((x) << 9) + (x)) is a faster version of: 255 * 255 * x +// ((((x) + (y)) << 8) - ((x) + (y)) - (y) * (x)) is a faster version of: 255 * (x + y) +#define UNPREMULTIPLY_ALPHA(x, y) ((((x) << 16) - ((x) << 9) + (x)) / ((((x) + (y)) << 8) - ((x) + (y)) - (y) * (x))) + +/** + * Blend image in src to destination buffer dst at position (x, y). + */ +static av_always_inline void blend_packed_rgb(const AVFilterContext *ctx, + const AVFrame *dst, const AVSubtitleArea *src, + int x, int y, + int is_straight) +{ + OverlaySubsContext *s = ctx->priv; + int i, imax, j, jmax; + const int src_w = src->w; + const int src_h = src->h; + const int dst_w = dst->width; + const int dst_h = dst->height; + uint8_t alpha; ///< the amount of overlay to blend on to main + const int dr = s->main_rgba_map[R]; + const int dg = s->main_rgba_map[G]; + const int db = s->main_rgba_map[B]; + const int da = s->main_rgba_map[A]; + const int dstep = s->main_pix_step[0]; + const int sr = s->overlay_rgba_map[R]; + const int sg = s->overlay_rgba_map[G]; + const int sb = s->overlay_rgba_map[B]; + const int sa = s->overlay_rgba_map[A]; + int slice_start, slice_end; + uint8_t *S, *sp, *d, *dp; + + i = FFMAX(-y, 0); + imax = FFMIN3(-y + dst_h, FFMIN(src_h, dst_h), y + src_h); + + slice_start = i; + slice_end = i + imax; + + sp = src->buf[0]->data + slice_start * src->linesize[0]; + dp = dst->data[0] + (slice_start + y) * dst->linesize[0]; + + for (i = slice_start; i < slice_end; i++) { + j = FFMAX(-x, 0); + S = sp + j; + d = dp + ((x + j) * dstep); + + for (jmax = FFMIN(-x + dst_w, src_w); j < jmax; j++) { + uint32_t val = src->pal[*S]; + const uint8_t *sval = (uint8_t *)&val; + alpha = sval[sa]; + + // if the main channel has an alpha channel, alpha has to be calculated + // to create an un-premultiplied (straight) alpha value + if (s->main_has_alpha && alpha != 0 && alpha != 255) { + const uint8_t alpha_d = d[da]; + alpha = UNPREMULTIPLY_ALPHA(alpha, alpha_d); + } + + switch (alpha) { + case 0: + break; + case 255: + d[dr] = sval[sr]; + d[dg] = sval[sg]; + d[db] = sval[sb]; + break; + default: + // main_value = main_value * (1 - alpha) + overlay_value * alpha + // since alpha is in the range 0-255, the result must divided by 255 + d[dr] = is_straight ? FAST_DIV255(d[dr] * (255 - alpha) + sval[sr] * alpha) : + FFMIN(FAST_DIV255(d[dr] * (255 - alpha)) + sval[sr], 255); + d[dg] = is_straight ? FAST_DIV255(d[dg] * (255 - alpha) + sval[sg] * alpha) : + FFMIN(FAST_DIV255(d[dg] * (255 - alpha)) + sval[sg], 255); + d[db] = is_straight ? FAST_DIV255(d[db] * (255 - alpha) + sval[sb] * alpha) : + FFMIN(FAST_DIV255(d[db] * (255 - alpha)) + sval[sb], 255); + } + + if (s->main_has_alpha) { + switch (alpha) { + case 0: + break; + case 255: + d[da] = sval[sa]; + break; + default: + // apply alpha compositing: main_alpha += (1-main_alpha) * overlay_alpha + d[da] += FAST_DIV255((255 - d[da]) * S[sa]); + } + } + d += dstep; + S += 1; + } + dp += dst->linesize[0]; + sp += src->linesize[0]; + } +} + +static av_always_inline void blend_plane_8_8bits(const AVFilterContext *ctx, const AVFrame *dst, const AVSubtitleArea *area, + const uint32_t *yuv_pal, int src_w, int src_h, int dst_w, int dst_h, int plane, int hsub, int vsub, + int x, int y, int dst_plane, int dst_offset, int dst_step) +{ + const int src_wp = AV_CEIL_RSHIFT(src_w, hsub); + const int src_hp = AV_CEIL_RSHIFT(src_h, vsub); + const int dst_wp = AV_CEIL_RSHIFT(dst_w, hsub); + const int dst_hp = AV_CEIL_RSHIFT(dst_h, vsub); + const int yp = y >> vsub; + const int xp = x >> hsub; + uint8_t *s, *sp, *d, *dp, *dap; + int imax, i, j, jmax; + int slice_start, slice_end; + + i = FFMAX(-yp, 0); \ + imax = FFMIN3(-yp + dst_hp, FFMIN(src_hp, dst_hp), yp + src_hp); \ + + slice_start = i; + slice_end = i + imax; + + sp = area->buf[0]->data + (slice_start << vsub) * area->linesize[0]; + dp = dst->data[dst_plane] + (yp + slice_start) * dst->linesize[dst_plane] + dst_offset; + + dap = dst->data[3] + ((yp + slice_start) << vsub) * dst->linesize[3]; + + for (i = slice_start; i < slice_end; i++) { + j = FFMAX(-xp, 0); + d = dp + (xp + j) * dst_step; + s = sp + (j << hsub); + jmax = FFMIN(-xp + dst_wp, src_wp); + + for (; j < jmax; j++) { + uint32_t val = yuv_pal[*s]; + const uint8_t *sval = (uint8_t *)&val; + const int alpha = sval[3]; + const int max = 255, mid = 128; + const int d_int = *d; + const int sval_int = sval[plane]; + + switch (alpha) { + case 0: + break; + case 255: + *d = sval[plane]; + break; + default: + if (plane > 0) + *d = av_clip(FAST_DIV255((d_int - mid) * (max - alpha) + (sval_int - mid) * alpha) , -mid, mid) + mid; + else + *d = FAST_DIV255(d_int * (max - alpha) + sval_int * alpha); + break; + } + + d += dst_step; + s += 1 << hsub; + } + dp += dst->linesize[dst_plane]; + sp += (1 << vsub) * area->linesize[0]; + dap += (1 << vsub) * dst->linesize[3]; + } +} + +#define RGB2Y(r, g, b) (uint8_t)(((66 * (r) + 129 * (g) + 25 * (b) + 128) >> 8) + 16) +#define RGB2U(r, g, b) (uint8_t)(((-38 * (r) - 74 * (g) + 112 * (b) + 128) >> 8) + 128) +#define RGB2V(r, g, b) (uint8_t)(((112 * (r) - 94 * (g) - 18 * (b) + 128) >> 8) + 128) +/* Converts R8 G8 B8 color to YUV. */ +static av_always_inline void rgb_2_yuv(uint8_t r, uint8_t g, uint8_t b, uint8_t* y, uint8_t* u, uint8_t* v) +{ + *y = RGB2Y((int)r, (int)g, (int)b); + *u = RGB2U((int)r, (int)g, (int)b); + *v = RGB2V((int)r, (int)g, (int)b); +} + + +static av_always_inline void blend_yuv_8_8bits(AVFilterContext *ctx, AVFrame *dst, const AVSubtitleArea *area, int hsub, int vsub, int x, int y) +{ + OverlaySubsContext *s = ctx->priv; + const int src_w = area->w; + const int src_h = area->h; + const int dst_w = dst->width; + const int dst_h = dst->height; + const int sr = s->overlay_rgba_map[R]; + const int sg = s->overlay_rgba_map[G]; + const int sb = s->overlay_rgba_map[B]; + const int sa = s->overlay_rgba_map[A]; + uint32_t yuvpal[256]; + + for (int i = 0; i < 256; ++i) { + const uint8_t *rgba = (const uint8_t *)&area->pal[i]; + uint8_t *yuva = (uint8_t *)&yuvpal[i]; + rgb_2_yuv(rgba[sr], rgba[sg], rgba[sb], &yuva[Y], &yuva[U], &yuva[V]); + yuva[3] = rgba[sa]; + } + + blend_plane_8_8bits(ctx, dst, area, yuvpal, src_w, src_h, dst_w, dst_h, Y, 0, 0, x, y, s->main_desc->comp[Y].plane, s->main_desc->comp[Y].offset, s->main_desc->comp[Y].step); + blend_plane_8_8bits(ctx, dst, area, yuvpal, src_w, src_h, dst_w, dst_h, U, hsub, vsub, x, y, s->main_desc->comp[U].plane, s->main_desc->comp[U].offset, s->main_desc->comp[U].step); + blend_plane_8_8bits(ctx, dst, area, yuvpal, src_w, src_h, dst_w, dst_h, V, hsub, vsub, x, y, s->main_desc->comp[V].plane, s->main_desc->comp[V].offset, s->main_desc->comp[V].step); +} + +static int config_input_main(AVFilterLink *inlink) +{ + int ret; + AVFilterContext *ctx = inlink->dst; + OverlaySubsContext *s = inlink->dst->priv; + const AVPixFmtDescriptor *pix_desc = av_pix_fmt_desc_get(inlink->format); + + av_image_fill_max_pixsteps(s->main_pix_step, NULL, pix_desc); + ff_fill_rgba_map(s->overlay_rgba_map, AV_PIX_FMT_RGB32); // it's actually AV_PIX_FMT_PAL8); + + s->hsub = pix_desc->log2_chroma_w; + s->vsub = pix_desc->log2_chroma_h; + + s->main_desc = pix_desc; + + s->main_is_packed_rgb = ff_fill_rgba_map(s->main_rgba_map, inlink->format) >= 0; + s->main_has_alpha = !!(pix_desc->flags & AV_PIX_FMT_FLAG_ALPHA); + + /* Finish the configuration by evaluating the expressions + now when both inputs are configured. */ + s->var_values[VAR_MAIN_W ] = s->var_values[VAR_MW] = ctx->inputs[MAIN ]->w; + s->var_values[VAR_MAIN_H ] = s->var_values[VAR_MH] = ctx->inputs[MAIN ]->h; + s->var_values[VAR_OVERLAY_W] = s->var_values[VAR_OW] = ctx->inputs[OVERLAY]->w; + s->var_values[VAR_OVERLAY_H] = s->var_values[VAR_OH] = ctx->inputs[OVERLAY]->h; + s->var_values[VAR_HSUB] = 1<log2_chroma_w; + s->var_values[VAR_VSUB] = 1<log2_chroma_h; + s->var_values[VAR_X] = NAN; + s->var_values[VAR_Y] = NAN; + s->var_values[VAR_N] = 0; + s->var_values[VAR_T] = NAN; + s->var_values[VAR_POS] = NAN; + + if ((ret = set_expr(&s->x_pexpr, s->x_expr, "x", ctx)) < 0 || + (ret = set_expr(&s->y_pexpr, s->y_expr, "y", ctx)) < 0) + return ret; + + if (s->eval_mode == EVAL_MODE_INIT) { + eval_expr(ctx); + av_log(ctx, AV_LOG_VERBOSE, "x:%f xi:%d y:%f yi:%d\n", + s->var_values[VAR_X], s->x, + s->var_values[VAR_Y], s->y); + } + + av_log(ctx, AV_LOG_VERBOSE, + "main w:%d h:%d fmt:%s overlay w:%d h:%d fmt:%s\n", + ctx->inputs[MAIN]->w, ctx->inputs[MAIN]->h, + av_get_pix_fmt_name(ctx->inputs[MAIN]->format), + ctx->inputs[OVERLAY]->w, ctx->inputs[OVERLAY]->h, + av_get_pix_fmt_name(ctx->inputs[OVERLAY]->format)); + return 0; +} + +static int do_blend(FFFrameSync *fs) +{ + AVFilterContext *ctx = fs->parent; + AVFrame *mainpic, *second; + OverlaySubsContext *s = ctx->priv; + AVFilterLink *inlink = ctx->inputs[0]; + unsigned i; + int ret; + + ret = ff_framesync_dualinput_get_writable(fs, &mainpic, &second); + if (ret < 0) + return ret; + if (!second) + return ff_filter_frame(ctx->outputs[0], mainpic); + + if (s->eval_mode == EVAL_MODE_FRAME) { + int64_t pos = mainpic->pkt_pos; + + s->var_values[VAR_N] = (double)inlink->frame_count_out; + s->var_values[VAR_T] = mainpic->pts == AV_NOPTS_VALUE ? + NAN :(double)mainpic->pts * av_q2d(inlink->time_base); + s->var_values[VAR_POS] = pos == -1 ? NAN : (double)pos; + + s->var_values[VAR_OVERLAY_W] = s->var_values[VAR_OW] = second->width; + s->var_values[VAR_OVERLAY_H] = s->var_values[VAR_OH] = second->height; + s->var_values[VAR_MAIN_W ] = s->var_values[VAR_MW] = mainpic->width; + s->var_values[VAR_MAIN_H ] = s->var_values[VAR_MH] = mainpic->height; + + eval_expr(ctx); + av_log(ctx, AV_LOG_DEBUG, "n:%f t:%f pos:%f x:%f xi:%d y:%f yi:%d\n", + s->var_values[VAR_N], s->var_values[VAR_T], s->var_values[VAR_POS], + s->var_values[VAR_X], s->x, + s->var_values[VAR_Y], s->y); + } + + for (i = 0; i < second->num_subtitle_areas; i++) { + const AVSubtitleArea *sub_area = second->subtitle_areas[i]; + + if (sub_area->type != AV_SUBTITLE_FMT_BITMAP) { + av_log(NULL, AV_LOG_WARNING, "overlay_graphicsub: non-bitmap subtitle\n"); + return AVERROR_INVALIDDATA; + } + + switch (inlink->format) { + case AV_PIX_FMT_YUV420P: + blend_yuv_8_8bits(ctx, mainpic, sub_area, 1, 1, sub_area->x + s->x, sub_area->y + s->y); + break; + case AV_PIX_FMT_YUV422P: + blend_yuv_8_8bits(ctx, mainpic, sub_area, 1, 0, sub_area->x + s->x, sub_area->y + s->y); + break; + case AV_PIX_FMT_YUV444P: + blend_yuv_8_8bits(ctx, mainpic, sub_area, 0, 0, sub_area->x + s->x, sub_area->y + s->y); + break; + case AV_PIX_FMT_RGB24: + case AV_PIX_FMT_BGR24: + case AV_PIX_FMT_ARGB: + case AV_PIX_FMT_RGBA: + case AV_PIX_FMT_BGRA: + case AV_PIX_FMT_ABGR: + blend_packed_rgb(ctx, mainpic, sub_area, sub_area->x + s->x, sub_area->y + s->y, 1); + break; + default: + av_log(NULL, AV_LOG_ERROR, "Unsupported input pix fmt: %d\n", inlink->format); + return AVERROR(EINVAL); + } + } + + return ff_filter_frame(ctx->outputs[0], mainpic); +} + +static av_cold int overlay_graphicsubs_init(AVFilterContext *ctx) +{ + OverlaySubsContext *s = ctx->priv; + + s->fs.on_event = do_blend; + return 0; +} + +static int overlay_graphicsubs_activate(AVFilterContext *ctx) +{ + OverlaySubsContext *s = ctx->priv; + return ff_framesync_activate(&s->fs); +} + +static int graphicsub2video_query_formats(AVFilterContext *ctx) +{ + AVFilterFormats *formats; + AVFilterLink *inlink = ctx->inputs[0]; + AVFilterLink *outlink = ctx->outputs[0]; + static const enum AVSubtitleType subtitle_fmts[] = { AV_SUBTITLE_FMT_BITMAP, AV_SUBTITLE_FMT_NONE }; + static const enum AVPixelFormat pix_fmts[] = { AV_PIX_FMT_RGB32, AV_PIX_FMT_NONE }; + int ret; + + /* set input subtitle formats */ + formats = ff_make_format_list(subtitle_fmts); + if ((ret = ff_formats_ref(formats, &inlink->outcfg.formats)) < 0) + return ret; + + /* set output video formats */ + formats = ff_make_format_list(pix_fmts); + if ((ret = ff_formats_ref(formats, &outlink->incfg.formats)) < 0) + return ret; + + return 0; +} + +static int graphicsub2video_config_input(AVFilterLink *inlink) +{ + AVFilterContext *ctx = inlink->dst; + OverlaySubsContext *s = ctx->priv; + + if (s->w <= 0 || s->h <= 0) { + s->w = inlink->w; + s->h = inlink->h; + } + return 0; +} + +static int graphicsub2video_config_output(AVFilterLink *outlink) +{ + const AVFilterContext *ctx = outlink->src; + OverlaySubsContext *s = ctx->priv; + const AVPixFmtDescriptor *pix_desc = av_pix_fmt_desc_get(outlink->format); + + outlink->w = s->w; + outlink->h = s->h; + + if (outlink->w == 0 && outlink->h == 0) { + outlink->w = 1; + outlink->h = 1; + } + outlink->sample_aspect_ratio = (AVRational){1,1}; + outlink->time_base = ctx->inputs[0]->time_base; + + av_image_fill_max_pixsteps(s->main_pix_step, NULL, pix_desc); + ff_fill_rgba_map(s->overlay_rgba_map, AV_PIX_FMT_RGB32); + + s->hsub = pix_desc->log2_chroma_w; + s->vsub = pix_desc->log2_chroma_h; + + s->main_desc = pix_desc; + + s->main_is_packed_rgb = ff_fill_rgba_map(s->main_rgba_map, outlink->format) >= 0; + s->main_has_alpha = !!(pix_desc->flags & AV_PIX_FMT_FLAG_ALPHA); + + return 0; +} + +static int graphicsub2video_filter_frame(AVFilterLink *inlink, AVFrame *src_frame) +{ + AVFilterLink *outlink = inlink->dst->outputs[0]; + const AVFilterContext *ctx = outlink->src; + OverlaySubsContext *s = ctx->priv; + AVFrame *out; + const unsigned num_rects = src_frame->num_subtitle_areas; + unsigned int i; + + out = ff_get_video_buffer(outlink, outlink->w, outlink->h); + if (!out) { + av_frame_free(&src_frame); + return AVERROR(ENOMEM); + } + + // Maybe we should set these.. + ////out->color_primaries = AVCOL_PRI_BT709; + ////out->color_range = AVCOL_RANGE_MPEG; + ////out->color_trc = AVCOL_TRC_BT709; + + memset(out->data[0], 0, (size_t)out->linesize[0] * out->height); + + out->pts = out->pkt_dts = out->best_effort_timestamp = src_frame->pts; + out->coded_picture_number = out->display_picture_number = s->pic_counter++; + + for (i = 0; i < num_rects; i++) { + const AVSubtitleArea *sub_rect = src_frame->subtitle_areas[i]; + + if (sub_rect->type != AV_SUBTITLE_FMT_BITMAP) { + av_log(NULL, AV_LOG_WARNING, "sub2video: non-bitmap subtitle\n"); + av_frame_free(&src_frame); + return AVERROR_INVALIDDATA; + } + + blend_packed_rgb(inlink->dst, out, sub_rect, sub_rect->x, sub_rect->y, 1); + } + + av_log(inlink->dst, AV_LOG_DEBUG, "graphicsub2video output - size %dx%d pts: %"PRId64" areas: %d\n", out->width, out->height, out->pts, src_frame->num_subtitle_areas); + + av_frame_free(&src_frame); + return ff_filter_frame(outlink, out); +} + +#define OFFSET(x) offsetof(OverlaySubsContext, x) +#define FLAGS (AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_FILTERING_PARAM) + +static const AVOption overlaygraphicsubs_options[] = { + { "x", "set the x expression", OFFSET(x_expr), AV_OPT_TYPE_STRING, {.str = "0"}, 0, 0, .flags = FLAGS }, + { "y", "set the y expression", OFFSET(y_expr), AV_OPT_TYPE_STRING, {.str = "0"}, 0, 0, .flags = FLAGS }, + { "eof_action", "Action to take when encountering EOF from secondary input ", + OFFSET(fs.opt_eof_action), AV_OPT_TYPE_INT, { .i64 = EOF_ACTION_REPEAT }, + EOF_ACTION_REPEAT, EOF_ACTION_PASS, .flags = FLAGS, "eof_action" }, + { "repeat", "Repeat the previous frame.", 0, AV_OPT_TYPE_CONST, { .i64 = EOF_ACTION_REPEAT }, .flags = FLAGS, "eof_action" }, + { "endall", "End both streams.", 0, AV_OPT_TYPE_CONST, { .i64 = EOF_ACTION_ENDALL }, .flags = FLAGS, "eof_action" }, + { "pass", "Pass through the main input.", 0, AV_OPT_TYPE_CONST, { .i64 = EOF_ACTION_PASS }, .flags = FLAGS, "eof_action" }, + { "eval", "specify when to evaluate expressions", OFFSET(eval_mode), AV_OPT_TYPE_INT, {.i64 = EVAL_MODE_FRAME}, 0, EVAL_MODE_NB-1, FLAGS, "eval" }, + { "init", "eval expressions once during initialization", 0, AV_OPT_TYPE_CONST, {.i64=EVAL_MODE_INIT}, .flags = FLAGS, .unit = "eval" }, + { "frame", "eval expressions per-frame", 0, AV_OPT_TYPE_CONST, {.i64=EVAL_MODE_FRAME}, .flags = FLAGS, .unit = "eval" }, + { "shortest", "force termination when the shortest input terminates", OFFSET(fs.opt_shortest), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, .flags = FLAGS }, + { "repeatlast", "repeat overlay of the last overlay frame", OFFSET(fs.opt_repeatlast), AV_OPT_TYPE_BOOL, {.i64=1}, 0, 1, .flags = FLAGS }, + { NULL } +}; + +static const AVOption graphicsub2video_options[] = { + { "size", "set video size", OFFSET(w), AV_OPT_TYPE_IMAGE_SIZE, {.str = NULL}, 0, 0, .flags = FLAGS }, + { "s", "set video size", OFFSET(w), AV_OPT_TYPE_IMAGE_SIZE, {.str = NULL}, 0, 0, .flags = FLAGS }, + { NULL } +}; + +FRAMESYNC_DEFINE_CLASS(overlaygraphicsubs, OverlaySubsContext, fs); + +static const AVFilterPad overlaygraphicsubs_inputs[] = { + { + .name = "main", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_input_main, + .flags = AVFILTERPAD_FLAG_NEEDS_WRITABLE, + }, + { + .name = "overlay", + .type = AVMEDIA_TYPE_SUBTITLE, + }, +}; + +static const AVFilterPad overlaygraphicsubs_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_output, + }, +}; + +const AVFilter ff_vf_overlaygraphicsubs = { + .name = "overlaygraphicsubs", + .description = NULL_IF_CONFIG_SMALL("Overlay graphical subtitles on top of the input."), + .preinit = overlaygraphicsubs_framesync_preinit, + .init = overlay_graphicsubs_init, + .uninit = overlay_graphicsubs_uninit, + .priv_size = sizeof(OverlaySubsContext), + .priv_class = &overlaygraphicsubs_class, + .activate = overlay_graphicsubs_activate, + FILTER_INPUTS(overlaygraphicsubs_inputs), + FILTER_OUTPUTS(overlaygraphicsubs_outputs), + FILTER_QUERY_FUNC(overlay_graphicsubs_query_formats), +}; + +AVFILTER_DEFINE_CLASS(graphicsub2video); + +static const AVFilterPad graphicsub2video_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .filter_frame = graphicsub2video_filter_frame, + .config_props = graphicsub2video_config_input, + }, +}; + +static const AVFilterPad graphicsub2video_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = graphicsub2video_config_output, + }, +}; + +const AVFilter ff_svf_graphicsub2video = { + .name = "graphicsub2video", + .description = NULL_IF_CONFIG_SMALL("Convert graphical subtitles to video"), + .priv_size = sizeof(OverlaySubsContext), + .priv_class = &graphicsub2video_class, + FILTER_INPUTS(graphicsub2video_inputs), + FILTER_OUTPUTS(graphicsub2video_outputs), + FILTER_QUERY_FUNC(graphicsub2video_query_formats), +}; From patchwork Mon Dec 13 23:40:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 32464 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp6140781iog; Mon, 13 Dec 2021 15:42:51 -0800 (PST) X-Google-Smtp-Source: ABdhPJy0sZ8dq8zBbmBdpcSaAVHiiKwOC981B8Fbde0IiBHVqmZryw+vLd/6jZvWY9/AzR5OooX6 X-Received: by 2002:a17:907:7da0:: with SMTP id oz32mr1778241ejc.176.1639438970763; Mon, 13 Dec 2021 15:42:50 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id dm3si28796262ejc.316.2021.12.13.15.42.50; Mon, 13 Dec 2021 15:42:50 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=EVqEiysn; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 86A8C68B0D3; Tue, 14 Dec 2021 01:40:50 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12olkn2017.outbound.protection.outlook.com [40.92.21.17]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2D32768B0C7 for ; Tue, 14 Dec 2021 01:40:48 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LE9M+cMLjctao80mJ1XgoU1tWNUspk3O6TPywbtl1sLmgG9LHUbtvX5TGywNcr8fA/yl/PRhJ/qAUd433KHShroTsNKEtrSwZAHRSa0Q/oLhj1aBYQdEE8fNZB5gAMcFNZUp4CJbp5deT0WWrN1OtGfZmZ8BFUpoDwgq+YsDZhUEkIzyeGrCNbypFm/WLFDECXosH1IGGa7B89G02OI30lqA/TpfCDEJ2d2UgDaPKI/hw1wj90IRQQDj8Th3o81K9cKy0SDkp2SQwEMufkhQOJFUi0HgyM66JNlubmMOkqAewErTjhYrtlGTiKEv7VA+6oDf8kCivpWUWm9ZXcXxqg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=KIBrjMH+tDG6Qberv8KioM0w8W2GTD1/Bawxm0bwvS0=; b=T/GnLa6G52A0lLaXPTaADZUcDkOGVApoRQxS/H0z277L8DU0cFcg97NLiwfIbgt37z1SSvlLCcZHnFkyAwcW+JeJSJ6zeYKrxcdUtmHEFf4HIeJ8xWetupeglegXslz6FIBx7nh+AL4FoZNMNwHbUDXvM/etNT8mxdXORBN+YgT6YDdQq/4Hm+U5d1M3psYPse6/AoTiqzNqrWnz8znL+u9eaVWxYTZ5SlWKcx3bMatswbtkWqQ0DvVi7NN/72bOXhNpn8MC0Cz8AdVOrnSItDyog3O6SZ7OPlEWclkxrsgJu2HsrZl9spx32CsFWz6Ixr2Udo/ssL1k8Wm3OcJ2wQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KIBrjMH+tDG6Qberv8KioM0w8W2GTD1/Bawxm0bwvS0=; b=EVqEiysnJtJxOOfnkRsDb5GqygV+JMCrJM6egCuoQxgepd9CIemU2FK/VqTCmPZ2PtpltiUnv+g9XR/cAPXRHEWwlf38IRONcUKpQME0kz/L6rMJtxXkiMbKzRlRLKqKwLGqG6p6Dly52zJExhEdl0GJxmev+9OycXnVnkN8UjxDxV9vJjmsbYsCUZ+l7MRNmiQpmCwxDflOj3u3ExHMgQD2Iowjt4KJJpvcU5lpc9h9hkrrSaX+10kA3MelhSCwixeh5FuPm6uEmoO1AYx0roHic4G1hCEn7a3WiAuKWRJPd/Q7SjTgT3mIOxjCXUCY6CQ5SCabfeigiGYI3A9IGA== Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::20) by DM8P223MB0397.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4778.17; Mon, 13 Dec 2021 23:40:45 +0000 Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775]) by DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775%7]) with mapi id 15.20.4778.018; Mon, 13 Dec 2021 23:40:45 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v24 13/21] fftools/ffmpeg: Replace sub2video with subtitle frame filtering and use new frame-based subtitle encoding API Thread-Index: AQHX8HrZsNx/MTsUukCrhrL4/v0+Jw== Date: Mon, 13 Dec 2021 23:40:45 +0000 Message-ID: References: <1ef42eb01b3627bbad539abd46206f320bb39697.1639438462.git.softworkz@hotmail.com> <4148d77a82d4b7e229fc81ce9bc1bc06904533f1.1639438462.git.softworkz@hotmail.com> <2ac798924ee984010d2e85f00bc992d9aa106171.1639438462.git.softworkz@hotmail.com> <300ea7188b331ca5e00dd7e04c03df8aba2a99c0.1639438462.git.softworkz@hotmail.com> <9474726b26f0d3e8f353d07caed2fbd33cdf473a.1639438462.git.softworkz@hotmail.com> <424522c96b5cdf6b03faa6bdceb1d25449463429.1639438462.git.softworkz@hotmail.com> <90caff245055005e6a4a890b3665da55aa853b31.1639438462.git.softworkz@hotmail.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [JuQkfUIvh0vU47nuDBlQ2q/7c6CHMqWv] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: f88f2bd5-a554-4a12-06ce-08d9be91fbf8 x-ms-traffictypediagnostic: DM8P223MB0397:EE_ x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: sYdmkd8+8mmTaTkq4jnHFtPBzHXVRK4P9dY7aMrgKTzYKK/SKRKAHvjC/4DORS9C92sazDqRMjL76fb1+6DRAdxFeJ0gD3ucU9MIU3ChtOvWqseF38piOzGKvTY/2scOjcX1a3P4fe2kdc3WlyW255ioUKmhLZ2bLFRXF2qIHEbNPE4l72x451lbuAgDYgWwoNf6B/L4NrBrhyv0lTuuoxdB/9JhaUriv/C9TDcvs5mxFHp8GtgmhSfh6WqTebEp1h3QYTAbkys0rpXEtaSVOL8qdbttzVPmlKkze4NWSwqQiGqiScBNTUGJfmvbHcMmn7aSdAPCLko+sL+lI6JhO3U9wHBGeAjUKLrw/SMlik/KJWv6OCYYqjCkifFfi37ctiFV/p2GTIadfArqS7zbAi33mIdvTMt7FYS+9IclFqFUpdCb/teBlbtnyyy86O6oSv6TF12JMOWNqatVORr6XmOnN95PvxJvHapu/evIwsON19cdM3f0BKY55kM/t6SyU2sLeF+ficNEUQ+GhTq+BKrlc7LFUDFJvKZrdon8ApBXNEOtDV6BHbRO5SNASCogtChlXRBKrrql322dRnKYQw== x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?q?xGjWzYpM3QRrILrnzd78qldJbkyP?= =?utf-8?q?7kYn0kZUUI58Si6/8KMEzZCi6XB+bYAAGfXktJMIMo//WaaQmnhKzPienCOjRO5fQ?= =?utf-8?q?o94ot8ViwJk4Jg4cJH4aRP5j/03/lRbRF5xdEZEaWJ2olOCy0gCbeTxTZPMKAF9wm?= =?utf-8?q?whudt0L9dI3tjIUx1QFQZ/EVwrk4KagiyqWFKN9EbDuSiXJpJhQqM91r/vdI+dJ+G?= =?utf-8?q?I8OTyR7+EZFFzzmb80nzMcczvhHcTc3aOLTaRfYjXhlwtEgf6jF2xt6HVnVX3URrd?= =?utf-8?q?X/kctCzEAUSG9iilqmPgo0ypOq1/Kc0/zK7RW/oSjrg0hLLcVl0F6AfdyeCGEFnC1?= =?utf-8?q?EFnYBtUPavxQXZgvR5N5pe3K69UuG/qz7JJ8nR0WedANdypgQh8tDE8APfiDfwfzC?= =?utf-8?q?SbNNdPAFmfQH71v8uVFeAlEEBROaIqj5zmOjIoxD0rNFpGAMW2c8ByG8hdGQrP6W7?= =?utf-8?q?I8k9rgWTbzVpbVBZFgNgKbc9Zhf9KNHMT5j8vbQOo6HO3yldaE3e4mPsrAuECVvCi?= =?utf-8?q?jttRJSA7TJGUuqWcvcsk8FvuorRqlUBIfkUx9H/iK7aPEnqqZSisXjj+QcV/9gYIC?= =?utf-8?q?dyTnozIpgHnakqorLID6fkLphAi1aXGX/BVWW+ktLs6i3YnMCAUOwWSI4psfvry0z?= =?utf-8?q?zvaUdLwLs3u4EEvu05uvKL05IcC9Lj+JhfWaegOr8HJkAQU1k+ey1O7FC2qVj6gJe?= =?utf-8?q?oWP3QyXtbKeE6twBGodGsHLt0Z2O3z3QLOSzeK8EybU0cdx0ctbf/BzfLrQiC/fKn?= =?utf-8?q?VOncSaSmS2It0vrtFh7/QQP/gBl2nbOtbTzyDf/MZyNFOrdmvPzcS6T0aVqlf1ghj?= =?utf-8?q?7223PTqTKvTQpoLlw21BXxpDKGjA9fwogoEHd53lph3LMePlyCxcEYGGHbGKYcOiq?= =?utf-8?q?kZxvwGNIKFyno2vTj29M7dzjFRfFPIjCRaAckRBbr9YQ=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-1ff67.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: f88f2bd5-a554-4a12-06ce-08d9be91fbf8 X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Dec 2021 23:40:45.5063 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8P223MB0397 Subject: [FFmpeg-devel] [PATCH v24 13/21] fftools/ffmpeg: Replace sub2video with subtitle frame filtering and use new frame-based subtitle encoding API X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: EkQ2ANabIGjo This commit actually enables subtitle filtering in ffmpeg by sending and receiving subtitle frames to and from a filtergraph. The heartbeat functionality from the previous sub2video implementation is retained and applied to all subtitle frames (bitmap, text, ..). The other part of sub2video functionality is retained by auto-insertion of the new graphicsub2video filter. Justification for changed test refs: - sub2video The new results are identical excepting the last frame which is due to the implementation changes - sub2video_basic The previous results had some incorrect output because multiple frames had the same dts The non-empty content frames are visually identical, the different CRC is due to the different blending algorithm that is being used. - sub2video_time_limited The third frame in the previous ref was a repetition, which doesn't happen anymore with the new subtitle filtering. - sub-dvb Running ffprobe -show_frames on the source file shows that there are 7 subtitle frames with 0 rects in the source at the start and 2 at the end. This translates to the 14 and 4 additional entries in the new test results. - filter-overlay-dvdsub-2397 Overlay results have slightly different CRCs due to different blending implementation Signed-off-by: softworkz --- fftools/ffmpeg.c | 588 +++++++++++----------- fftools/ffmpeg.h | 15 +- fftools/ffmpeg_filter.c | 236 ++++++--- fftools/ffmpeg_hw.c | 2 +- fftools/ffmpeg_opt.c | 3 +- tests/ref/fate/filter-overlay-dvdsub-2397 | 181 ++++--- tests/ref/fate/sub-dvb | 162 +++--- tests/ref/fate/sub2video | 220 ++++---- tests/ref/fate/sub2video_basic | 135 ++--- tests/ref/fate/sub2video_time_limited | 4 +- 10 files changed, 800 insertions(+), 746 deletions(-) diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index bdeff9a12e..adbe7f667d 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -143,8 +143,6 @@ static int want_sdp = 1; static BenchmarkTimeStamps current_time; AVIOContext *progress_avio = NULL; -static uint8_t *subtitle_out; - InputStream **input_streams = NULL; int nb_input_streams = 0; InputFile **input_files = NULL; @@ -169,163 +167,6 @@ static int restore_tty; static void free_input_threads(void); #endif -/* sub2video hack: - Convert subtitles to video with alpha to insert them in filter graphs. - This is a temporary solution until libavfilter gets real subtitles support. - */ - -static int sub2video_get_blank_frame(InputStream *ist) -{ - int ret; - AVFrame *frame = ist->sub2video.frame; - - av_frame_unref(frame); - ist->sub2video.frame->width = ist->dec_ctx->width ? ist->dec_ctx->width : ist->sub2video.w; - ist->sub2video.frame->height = ist->dec_ctx->height ? ist->dec_ctx->height : ist->sub2video.h; - ist->sub2video.frame->format = AV_PIX_FMT_RGB32; - if ((ret = av_frame_get_buffer(frame, 0)) < 0) - return ret; - memset(frame->data[0], 0, frame->height * frame->linesize[0]); - return 0; -} - -static void sub2video_copy_rect(uint8_t *dst, int dst_linesize, int w, int h, - AVSubtitleRect *r) -{ - uint32_t *pal, *dst2; - uint8_t *src, *src2; - int x, y; - - if (r->type != SUBTITLE_BITMAP) { - av_log(NULL, AV_LOG_WARNING, "sub2video: non-bitmap subtitle\n"); - return; - } - if (r->x < 0 || r->x + r->w > w || r->y < 0 || r->y + r->h > h) { - av_log(NULL, AV_LOG_WARNING, "sub2video: rectangle (%d %d %d %d) overflowing %d %d\n", - r->x, r->y, r->w, r->h, w, h - ); - return; - } - - dst += r->y * dst_linesize + r->x * 4; - src = r->data[0]; - pal = (uint32_t *)r->data[1]; - for (y = 0; y < r->h; y++) { - dst2 = (uint32_t *)dst; - src2 = src; - for (x = 0; x < r->w; x++) - *(dst2++) = pal[*(src2++)]; - dst += dst_linesize; - src += r->linesize[0]; - } -} - -static void sub2video_push_ref(InputStream *ist, int64_t pts) -{ - AVFrame *frame = ist->sub2video.frame; - int i; - int ret; - - av_assert1(frame->data[0]); - ist->sub2video.last_pts = frame->pts = pts; - for (i = 0; i < ist->nb_filters; i++) { - ret = av_buffersrc_add_frame_flags(ist->filters[i]->filter, frame, - AV_BUFFERSRC_FLAG_KEEP_REF | - AV_BUFFERSRC_FLAG_PUSH); - if (ret != AVERROR_EOF && ret < 0) - av_log(NULL, AV_LOG_WARNING, "Error while add the frame to buffer source(%s).\n", - av_err2str(ret)); - } -} - -void sub2video_update(InputStream *ist, int64_t heartbeat_pts, AVSubtitle *sub) -{ - AVFrame *frame = ist->sub2video.frame; - int8_t *dst; - int dst_linesize; - int num_rects, i; - int64_t pts, end_pts; - - if (!frame) - return; - if (sub) { - pts = av_rescale_q(sub->pts + sub->start_display_time * 1000LL, - AV_TIME_BASE_Q, ist->st->time_base); - end_pts = av_rescale_q(sub->pts + sub->end_display_time * 1000LL, - AV_TIME_BASE_Q, ist->st->time_base); - num_rects = sub->num_rects; - } else { - /* If we are initializing the system, utilize current heartbeat - PTS as the start time, and show until the following subpicture - is received. Otherwise, utilize the previous subpicture's end time - as the fall-back value. */ - pts = ist->sub2video.initialize ? - heartbeat_pts : ist->sub2video.end_pts; - end_pts = INT64_MAX; - num_rects = 0; - } - if (sub2video_get_blank_frame(ist) < 0) { - av_log(ist->dec_ctx, AV_LOG_ERROR, - "Impossible to get a blank canvas.\n"); - return; - } - dst = frame->data [0]; - dst_linesize = frame->linesize[0]; - for (i = 0; i < num_rects; i++) - sub2video_copy_rect(dst, dst_linesize, frame->width, frame->height, sub->rects[i]); - sub2video_push_ref(ist, pts); - ist->sub2video.end_pts = end_pts; - ist->sub2video.initialize = 0; -} - -static void sub2video_heartbeat(InputStream *ist, int64_t pts) -{ - InputFile *infile = input_files[ist->file_index]; - int i, j, nb_reqs; - int64_t pts2; - - /* When a frame is read from a file, examine all sub2video streams in - the same file and send the sub2video frame again. Otherwise, decoded - video frames could be accumulating in the filter graph while a filter - (possibly overlay) is desperately waiting for a subtitle frame. */ - for (i = 0; i < infile->nb_streams; i++) { - InputStream *ist2 = input_streams[infile->ist_index + i]; - if (!ist2->sub2video.frame) - continue; - /* subtitles seem to be usually muxed ahead of other streams; - if not, subtracting a larger time here is necessary */ - pts2 = av_rescale_q(pts, ist->st->time_base, ist2->st->time_base) - 1; - /* do not send the heartbeat frame if the subtitle is already ahead */ - if (pts2 <= ist2->sub2video.last_pts) - continue; - if (pts2 >= ist2->sub2video.end_pts || ist2->sub2video.initialize) - /* if we have hit the end of the current displayed subpicture, - or if we need to initialize the system, update the - overlayed subpicture and its start/end times */ - sub2video_update(ist2, pts2 + 1, NULL); - for (j = 0, nb_reqs = 0; j < ist2->nb_filters; j++) - nb_reqs += av_buffersrc_get_nb_failed_requests(ist2->filters[j]->filter); - if (nb_reqs) - sub2video_push_ref(ist2, pts2); - } -} - -static void sub2video_flush(InputStream *ist) -{ - int i; - int ret; - - if (ist->sub2video.end_pts < INT64_MAX) - sub2video_update(ist, INT64_MAX, NULL); - for (i = 0; i < ist->nb_filters; i++) { - ret = av_buffersrc_add_frame(ist->filters[i]->filter, NULL); - if (ret != AVERROR_EOF && ret < 0) - av_log(NULL, AV_LOG_WARNING, "Flush the frame error.\n"); - } -} - -/* end of sub2video hack */ - static void term_exit_sigsafe(void) { #if HAVE_TERMIOS_H @@ -526,7 +367,6 @@ static void ffmpeg_cleanup(int ret) avfilter_graph_free(&fg->graph); for (j = 0; j < fg->nb_inputs; j++) { InputFilter *ifilter = fg->inputs[j]; - struct InputStream *ist = ifilter->ist; while (av_fifo_size(ifilter->frame_queue)) { AVFrame *frame; @@ -536,15 +376,6 @@ static void ffmpeg_cleanup(int ret) } av_fifo_freep(&ifilter->frame_queue); av_freep(&ifilter->displaymatrix); - if (ist->sub2video.sub_queue) { - while (av_fifo_size(ist->sub2video.sub_queue)) { - AVSubtitle sub; - av_fifo_generic_read(ist->sub2video.sub_queue, - &sub, sizeof(sub), NULL); - avsubtitle_free(&sub); - } - av_fifo_freep(&ist->sub2video.sub_queue); - } av_buffer_unref(&ifilter->hw_frames_ctx); av_freep(&ifilter->name); av_freep(&fg->inputs[j]); @@ -564,7 +395,7 @@ static void ffmpeg_cleanup(int ret) } av_freep(&filtergraphs); - av_freep(&subtitle_out); + ////av_freep(&subtitle_out); /* close files */ for (i = 0; i < nb_output_files; i++) { @@ -632,12 +463,14 @@ static void ffmpeg_cleanup(int ret) av_frame_free(&ist->decoded_frame); av_packet_free(&ist->pkt); av_dict_free(&ist->decoder_opts); - avsubtitle_free(&ist->prev_sub.subtitle); - av_frame_free(&ist->sub2video.frame); + av_frame_free(&ist->prev_sub.subtitle); av_freep(&ist->filters); av_freep(&ist->hwaccel_device); av_freep(&ist->dts_buffer); + av_frame_free(&ist->subtitle_heartbeat.recent_sub); + av_buffer_unref(&ist->subtitle_heartbeat.header); + avcodec_free_context(&ist->dec_ctx); av_freep(&input_streams[i]); @@ -1055,17 +888,61 @@ error: exit_program(1); } -static void do_subtitle_out(OutputFile *of, - OutputStream *ost, - AVSubtitle *sub) +static void encode_subtitle_frame(OutputFile *of, OutputStream *ost, AVFrame *frame, AVPacket *pkt, int64_t pts_offset) +{ + AVCodecContext *enc = ost->enc_ctx; + int ret; + + ost->frames_encoded++; + + ret = avcodec_send_frame(enc, frame); + if (ret < 0) + goto error; + + while (1) { + ret = avcodec_receive_packet(enc, pkt); + update_benchmark("encode_subtitles %d.%d", ost->file_index, ost->index); + if (ret == AVERROR(EAGAIN)) + break; + if (ret < 0) + goto error; + + if (debug_ts) { + av_log(NULL, AV_LOG_INFO, "encoder -> type:subtitles " + "pkt_pts:%s pkt_pts_time:%s pkt_dts:%s pkt_dts_time:%s\n", + av_ts2str(pkt->pts), av_ts2timestr(pkt->pts, &enc->time_base), + av_ts2str(pkt->dts), av_ts2timestr(pkt->dts, &enc->time_base)); + } + + pkt->pts = av_rescale_q(frame->subtitle_pts, AV_TIME_BASE_Q, ost->mux_timebase); + pkt->duration = av_rescale_q(frame->subtitle_end_time, (AVRational){ 1, 1000 }, ost->mux_timebase); + pkt->pts += pts_offset; + + pkt->dts = pkt->pts; + output_packet(of, pkt, ost, 0); + } + ost->sync_opts++; + ost->frame_number++; + + return; +error: + av_log(NULL, AV_LOG_FATAL, "Subtitle encoding failed - Error code: %d\n", ret); + exit_program(1); +} + +static void do_subtitle_out(OutputFile *of, OutputStream *ost, AVFrame *frame) { - int subtitle_out_max_size = 1024 * 1024; - int subtitle_out_size, nb, i; + int nb, i; AVCodecContext *enc; AVPacket *pkt = ost->pkt; int64_t pts; - if (sub->pts == AV_NOPTS_VALUE) { + if (!frame) + return; + + av_log(NULL, AV_LOG_TRACE, "do_subtitle_out: sub->pts: %"PRId64" frame->pts: %"PRId64"\n", frame->subtitle_pts, frame->pts); + + if (frame->subtitle_pts == AV_NOPTS_VALUE) { av_log(NULL, AV_LOG_ERROR, "Subtitle packets must have a pts\n"); if (exit_on_error) exit_program(1); @@ -1074,14 +951,6 @@ static void do_subtitle_out(OutputFile *of, enc = ost->enc_ctx; - if (!subtitle_out) { - subtitle_out = av_malloc(subtitle_out_max_size); - if (!subtitle_out) { - av_log(NULL, AV_LOG_FATAL, "Failed to allocate subtitle_out\n"); - exit_program(1); - } - } - /* Note: DVB subtitle need one packet to draw them and one other packet to clear them */ /* XXX: signal it in the codec context ? */ @@ -1091,50 +960,43 @@ static void do_subtitle_out(OutputFile *of, nb = 1; /* shift timestamp to honor -ss and make check_recording_time() work with -t */ - pts = sub->pts; + pts = frame->subtitle_pts; if (output_files[ost->file_index]->start_time != AV_NOPTS_VALUE) pts -= output_files[ost->file_index]->start_time; - for (i = 0; i < nb; i++) { - unsigned save_num_rects = sub->num_rects; - ost->sync_opts = av_rescale_q(pts, AV_TIME_BASE_Q, enc->time_base); - if (!check_recording_time(ost)) - return; + ost->sync_opts = av_rescale_q(pts, AV_TIME_BASE_Q, enc->time_base); + if (!check_recording_time(ost)) + return; - sub->pts = pts; - // start_display_time is required to be 0 - sub->pts += av_rescale_q(sub->start_display_time, (AVRational){ 1, 1000 }, AV_TIME_BASE_Q); - sub->end_display_time -= sub->start_display_time; - sub->start_display_time = 0; - if (i == 1) - sub->num_rects = 0; + frame->subtitle_pts = pts; + // subtitle_start_time is required to be 0 + frame->subtitle_pts += av_rescale_q(frame->subtitle_start_time, (AVRational){ 1, 1000 }, AV_TIME_BASE_Q); + frame->subtitle_end_time -= frame->subtitle_start_time; + frame->subtitle_start_time = 0; + + for (i = 0; i < nb; i++) { + int ret, got_packet = 0; + const unsigned save_num_rects = frame->num_subtitle_areas; + int64_t pts_offset = 0; ost->frames_encoded++; - subtitle_out_size = avcodec_encode_subtitle(enc, subtitle_out, - subtitle_out_max_size, sub); if (i == 1) - sub->num_rects = save_num_rects; - if (subtitle_out_size < 0) { - av_log(NULL, AV_LOG_FATAL, "Subtitle encoding failed\n"); - exit_program(1); - } + frame->num_subtitle_areas = 0; - av_packet_unref(pkt); - pkt->data = subtitle_out; - pkt->size = subtitle_out_size; - pkt->pts = av_rescale_q(sub->pts, AV_TIME_BASE_Q, ost->mux_timebase); - pkt->duration = av_rescale_q(sub->end_display_time, (AVRational){ 1, 1000 }, ost->mux_timebase); if (enc->codec_id == AV_CODEC_ID_DVB_SUBTITLE) { /* XXX: the pts correction is handled here. Maybe handling it in the codec would be better */ if (i == 0) - pkt->pts += av_rescale_q(sub->start_display_time, (AVRational){ 1, 1000 }, ost->mux_timebase); + pts_offset = av_rescale_q(frame->subtitle_start_time, (AVRational){ 1, 1000 }, ost->mux_timebase); else - pkt->pts += av_rescale_q(sub->end_display_time, (AVRational){ 1, 1000 }, ost->mux_timebase); + pts_offset = av_rescale_q(frame->subtitle_end_time, (AVRational){ 1, 1000 }, ost->mux_timebase); } - pkt->dts = pkt->pts; - output_packet(of, pkt, ost, 0); + + encode_subtitle_frame(of, ost, frame, pkt, pts_offset); + + if (i == 1) + frame->num_subtitle_areas = save_num_rects; } } @@ -1544,8 +1406,26 @@ static int reap_filters(int flush) } do_audio_out(of, ost, filtered_frame); break; + case AVMEDIA_TYPE_SUBTITLE: + + if (filtered_frame->format == AV_SUBTITLE_FMT_ASS && !enc->subtitle_header + && filtered_frame->subtitle_header) { + const char *subtitle_header = (char *)filtered_frame->subtitle_header->data; + enc->subtitle_header = (uint8_t *)av_strdup(subtitle_header); + if (!enc->subtitle_header) + return AVERROR(ENOMEM); + enc->subtitle_header_size = strlen(subtitle_header); + } + + if ((ost->enc_ctx->width == 0 || ost->enc_ctx->height == 0) + && filter->inputs[0]->w > 0 && filter->inputs[0]->h > 0 ) { + ost->enc_ctx->width = filter->inputs[0]->w; + ost->enc_ctx->height = filter->inputs[0]->h; + } + + do_subtitle_out(of, ost, filtered_frame); + break; default: - // TODO support subtitle filters av_assert0(0); } @@ -2138,7 +2018,8 @@ static int ifilter_has_all_input_formats(FilterGraph *fg) int i; for (i = 0; i < fg->nb_inputs; i++) { if (fg->inputs[i]->format < 0 && (fg->inputs[i]->type == AVMEDIA_TYPE_AUDIO || - fg->inputs[i]->type == AVMEDIA_TYPE_VIDEO)) + fg->inputs[i]->type == AVMEDIA_TYPE_VIDEO || + fg->inputs[i]->type == AVMEDIA_TYPE_SUBTITLE)) return 0; } return 1; @@ -2164,8 +2045,12 @@ static int ifilter_send_frame(InputFilter *ifilter, AVFrame *frame, int keep_ref ifilter->channel_layout != frame->channel_layout; break; case AVMEDIA_TYPE_VIDEO: + case AVMEDIA_TYPE_SUBTITLE: need_reinit |= ifilter->width != frame->width || ifilter->height != frame->height; + + if (need_reinit) + need_reinit = 1; break; } @@ -2243,7 +2128,7 @@ static int ifilter_send_eof(InputFilter *ifilter, int64_t pts) // the filtergraph was never configured if (ifilter->format < 0) ifilter_parameters_from_codecpar(ifilter, ifilter->ist->st->codecpar); - if (ifilter->format < 0 && (ifilter->type == AVMEDIA_TYPE_AUDIO || ifilter->type == AVMEDIA_TYPE_VIDEO)) { + if (ifilter->format < 0 && (ifilter->type == AVMEDIA_TYPE_AUDIO || ifilter->type == AVMEDIA_TYPE_VIDEO || ifilter->type == AVMEDIA_TYPE_SUBTITLE)) { av_log(NULL, AV_LOG_ERROR, "Cannot determine format of input stream %d:%d after EOF\n", ifilter->ist->file_index, ifilter->ist->st->index); return AVERROR_INVALIDDATA; } @@ -2281,7 +2166,7 @@ static int decode(AVCodecContext *avctx, AVFrame *frame, int *got_frame, AVPacke static int send_frame_to_filters(InputStream *ist, AVFrame *decoded_frame) { - int i, ret; + int i, ret = 0; av_assert1(ist->nb_filters > 0); /* ensure ret is initialized */ for (i = 0; i < ist->nb_filters; i++) { @@ -2482,81 +2367,223 @@ fail: return err < 0 ? err : ret; } -static int transcode_subtitles(InputStream *ist, AVPacket *pkt, int *got_output, +static void subtitle_resend_current(InputStream *ist, int64_t heartbeat_pts) +{ + AVFrame *frame; + int64_t pts, end_pts; + + if (ist->subtitle_heartbeat.recent_sub) { + frame = av_frame_clone(ist->subtitle_heartbeat.recent_sub); + if (!frame) { + av_log(ist->dec_ctx, AV_LOG_ERROR, "Unable to alloc frame (out of memory).\n"); + return; + } + + pts = heartbeat_pts; //av_rescale_q(frame->subtitle_pts + frame->subtitle_start_time * 1000LL, AV_TIME_BASE_Q, ist->st->time_base); + end_pts = av_rescale_q(frame->subtitle_pts + frame->subtitle_end_time * 1000LL, AV_TIME_BASE_Q, ist->st->time_base); + } else { + frame = av_frame_alloc(); + if (!frame) { + av_log(ist->dec_ctx, AV_LOG_ERROR, "Unable to alloc frame (out of memory).\n"); + return; + } + + frame->type = AVMEDIA_TYPE_SUBTITLE; + frame->format = avcodec_descriptor_get_subtitle_format(ist->dec_ctx->codec_descriptor); + + av_frame_get_buffer2(frame, 0); + + frame->width = ist->subtitle_heartbeat.w; + frame->height = ist->subtitle_heartbeat.h; + + pts = (ist->subtitle_heartbeat.end_pts < INT64_MAX && ist->subtitle_heartbeat.end_pts > 0) + ? ist->subtitle_heartbeat.end_pts : heartbeat_pts; + end_pts = INT64_MAX; + + frame->subtitle_pts = av_rescale_q(pts, ist->st->time_base, AV_TIME_BASE_Q); + frame->subtitle_end_time = 1000; + } + + av_log(NULL, AV_LOG_DEBUG, "subtitle_heartbeat: call subtitle_resend_current %"PRId64" \n", pts); + + frame->pts = pts; + ist->subtitle_heartbeat.last_pts = pts; + ist->subtitle_heartbeat.end_pts = end_pts; + + send_frame_to_filters(ist, frame); + + av_frame_free(&frame); +} + +static void subtitle_heartbeat(InputStream *ist, int64_t pts) +{ + int i; + int64_t pts2; + + if (ist->st->codecpar->codec_type != AVMEDIA_TYPE_VIDEO) + return; + + /* When a frame is read from a file, examine all subtitle streams in + the same file and send the subtitle frame again. Otherwise, decoded + video frames could be accumulating in the filter graph while a filter + (possibly overlay) is desperately waiting for a subtitle frame. */ + for (i = 0; i < nb_input_streams; i++) { + InputStream *ist2 = input_streams[i]; + if (!ist2->subtitle_heartbeat.is_active) + continue; + /* subtitles seem to be usually muxed ahead of other streams; + if not, subtracting a larger time here is necessary */ + pts2 = av_rescale_q(pts, ist->st->time_base, ist2->st->time_base) - 1; + /* do not send the heartbeat frame if the subtitle is already ahead */ + if (pts2 <= ist2->subtitle_heartbeat.last_pts) + continue; + if (pts2 >= ist2->subtitle_heartbeat.end_pts) { + /* if we have hit the end of the current displayed subpicture, + or if we need to initialize the system, update the + overlayed subpicture and its start/end times */ + if (ist2->subtitle_heartbeat.recent_sub) + av_frame_free(&ist2->subtitle_heartbeat.recent_sub); + + av_log(NULL, AV_LOG_DEBUG, "subtitle_heartbeat: clear + resend - pts: %"PRIi64"\n", pts2 + 1); + subtitle_resend_current(ist2, pts2 + 1); + continue; + } + if (!ist2->subtitle_heartbeat.check_buffer_requests) { + unsigned j, nb_reqs; + for (j = 0, nb_reqs = 0; j < ist2->nb_filters; j++) + nb_reqs += av_buffersrc_get_nb_failed_requests(ist2->filters[j]->filter); + if (nb_reqs) { + av_log(NULL, AV_LOG_DEBUG, "subtitle_heartbeat: resend - pts: %"PRIi64"\n", pts2); + subtitle_resend_current(ist2, pts2); + } + } + } +} + +static InputStream *get_input_stream(OutputStream *ost) +{ + if (ost->source_index >= 0) + return input_streams[ost->source_index]; + return NULL; +} + +static int decode_subtitles(InputStream *ist, AVPacket *pkt, int *got_output, int *decode_failed) { - AVSubtitle subtitle; - int free_sub = 1; - int i, ret = avcodec_decode_subtitle2(ist->dec_ctx, - &subtitle, got_output, pkt); + AVFrame *decoded_frame; + AVCodecContext *avctx = ist->dec_ctx; + int i = 0, ret = 0, err = 0; + int64_t pts, end_pts; + + if (!ist->decoded_frame && !(ist->decoded_frame = av_frame_alloc())) + return AVERROR(ENOMEM); + decoded_frame = ist->decoded_frame; + decoded_frame->type = AVMEDIA_TYPE_SUBTITLE; + decoded_frame->format = avcodec_descriptor_get_subtitle_format(avctx->codec_descriptor); + + if (!ist->subtitle_heartbeat.header && avctx->subtitle_header && avctx->subtitle_header_size > 0) { + ist->subtitle_heartbeat.header = av_buffer_allocz(avctx->subtitle_header_size + 1); + if (!ist->subtitle_heartbeat.header) + return AVERROR(ENOMEM); + memcpy(ist->subtitle_heartbeat.header->data, avctx->subtitle_header, avctx->subtitle_header_size); + } + + ret = decode(avctx, decoded_frame, got_output, pkt); - check_decode_result(NULL, got_output, ret); + if (ret != AVERROR_EOF) + check_decode_result(NULL, got_output, ret); if (ret < 0 || !*got_output) { *decode_failed = 1; - if (!pkt->size) - sub2video_flush(ist); + if (!pkt->size) { + // Flush + for (i = 0; i < ist->nb_filters; i++) { + ret = av_buffersrc_add_frame(ist->filters[i]->filter, NULL); + if (ret != AVERROR_EOF && ret < 0) + av_log(NULL, AV_LOG_WARNING, "Flush the frame error.\n"); + } + } return ret; } if (ist->fix_sub_duration) { int end = 1; - if (ist->prev_sub.got_output) { - end = av_rescale(subtitle.pts - ist->prev_sub.subtitle.pts, + if (ist->prev_sub.got_output && ist->prev_sub.subtitle) { + end = av_rescale(decoded_frame->subtitle_pts - ist->prev_sub.subtitle->subtitle_pts, 1000, AV_TIME_BASE); - if (end < ist->prev_sub.subtitle.end_display_time) { - av_log(ist->dec_ctx, AV_LOG_DEBUG, + if (end < ist->prev_sub.subtitle->subtitle_end_time) { + av_log(avctx, AV_LOG_DEBUG, "Subtitle duration reduced from %"PRId32" to %d%s\n", - ist->prev_sub.subtitle.end_display_time, end, + ist->prev_sub.subtitle->subtitle_end_time, end, end <= 0 ? ", dropping it" : ""); - ist->prev_sub.subtitle.end_display_time = end; + ist->prev_sub.subtitle->subtitle_end_time = end; } } FFSWAP(int, *got_output, ist->prev_sub.got_output); FFSWAP(int, ret, ist->prev_sub.ret); - FFSWAP(AVSubtitle, subtitle, ist->prev_sub.subtitle); + FFSWAP(AVFrame*, decoded_frame, ist->prev_sub.subtitle); if (end <= 0) - goto out; + return end; } - if (!*got_output) + if (!*got_output || !decoded_frame) return ret; - if (ist->sub2video.frame) { - sub2video_update(ist, INT64_MIN, &subtitle); - } else if (ist->nb_filters) { - if (!ist->sub2video.sub_queue) - ist->sub2video.sub_queue = av_fifo_alloc(8 * sizeof(AVSubtitle)); - if (!ist->sub2video.sub_queue) - exit_program(1); - if (!av_fifo_space(ist->sub2video.sub_queue)) { - ret = av_fifo_realloc2(ist->sub2video.sub_queue, 2 * av_fifo_size(ist->sub2video.sub_queue)); - if (ret < 0) - exit_program(1); + decoded_frame->type = AVMEDIA_TYPE_SUBTITLE; + decoded_frame->format = avcodec_descriptor_get_subtitle_format(avctx->codec_descriptor); + + ////if ((ret = av_frame_get_buffer2(decoded_frame, 0)) < 0) + //// return ret; + + if ((ret = av_buffer_replace(&decoded_frame->subtitle_header, ist->subtitle_heartbeat.header)) < 0) + return ret; + + pts = av_rescale_q(decoded_frame->subtitle_pts + decoded_frame->subtitle_start_time * 1000LL, + AV_TIME_BASE_Q, ist->st->time_base); + end_pts = av_rescale_q(decoded_frame->subtitle_pts + decoded_frame->subtitle_end_time * 1000LL, + AV_TIME_BASE_Q, ist->st->time_base); + + ist->subtitle_heartbeat.last_pts = decoded_frame->pts = pts; + ist->subtitle_heartbeat.end_pts = end_pts; + + if (ist->nb_filters > 0) { + AVFrame *filter_frame = av_frame_clone(decoded_frame); + if (!filter_frame) { + err = AVERROR(ENOMEM); + goto end; } - av_fifo_generic_write(ist->sub2video.sub_queue, &subtitle, sizeof(subtitle), NULL); - free_sub = 0; - } - if (!subtitle.num_rects) - goto out; + err = send_frame_to_filters(ist, filter_frame); + av_frame_free(&filter_frame); + } - ist->frames_decoded++; + if (err >= 0) { + for (i = 0; i < nb_output_streams; i++) { + OutputStream *ost = output_streams[i]; + InputStream *ist_src = get_input_stream(ost); - for (i = 0; i < nb_output_streams; i++) { - OutputStream *ost = output_streams[i]; + if (!ist_src || !check_output_constraints(ist, ost) + || ist_src != ist + || !ost->encoding_needed + || ost->enc->type != AVMEDIA_TYPE_SUBTITLE) + continue; - if (!check_output_constraints(ist, ost) || !ost->encoding_needed - || ost->enc->type != AVMEDIA_TYPE_SUBTITLE) - continue; + if (ost->filter && ost->filter->filter->nb_inputs > 0) + continue; - do_subtitle_out(output_files[ost->file_index], ost, &subtitle); + ////if (!ost->pkt && !((ost->pkt = av_packet_alloc()))) + //// exit_program(1); + do_subtitle_out(output_files[ost->file_index], ost, decoded_frame); + } } -out: - if (free_sub) - avsubtitle_free(&subtitle); - return ret; +end: + av_frame_free(&ist->subtitle_heartbeat.recent_sub); + ist->subtitle_heartbeat.recent_sub = av_frame_clone(decoded_frame); + + + av_frame_unref(decoded_frame); + return err < 0 ? err : ret; } static int send_filter_eof(InputStream *ist) @@ -2660,7 +2687,7 @@ static int process_input_packet(InputStream *ist, const AVPacket *pkt, int no_eo case AVMEDIA_TYPE_SUBTITLE: if (repeating) break; - ret = transcode_subtitles(ist, avpkt, &got_output, &decode_failed); + ret = decode_subtitles(ist, avpkt, &got_output, &decode_failed); if (!pkt && ret >= 0) ret = AVERROR_EOF; av_packet_unref(avpkt); @@ -2928,13 +2955,6 @@ FF_ENABLE_DEPRECATION_WARNINGS return 0; } -static InputStream *get_input_stream(OutputStream *ost) -{ - if (ost->source_index >= 0) - return input_streams[ost->source_index]; - return NULL; -} - static int compare_int64(const void *a, const void *b) { return FFDIFFSIGN(*(const int64_t *)a, *(const int64_t *)b); @@ -3413,7 +3433,7 @@ static int init_output_stream_encode(OutputStream *ost, AVFrame *frame) break; case AVMEDIA_TYPE_SUBTITLE: enc_ctx->time_base = AV_TIME_BASE_Q; - if (!enc_ctx->width) { + if (!enc_ctx->width && ost->source_index >= 0) { enc_ctx->width = input_streams[ost->source_index]->st->codecpar->width; enc_ctx->height = input_streams[ost->source_index]->st->codecpar->height; } @@ -3466,19 +3486,14 @@ static int init_output_stream(OutputStream *ost, AVFrame *frame, } if (ist && ist->dec->type == AVMEDIA_TYPE_SUBTITLE && ost->enc->type == AVMEDIA_TYPE_SUBTITLE) { - int input_props = 0, output_props = 0; - AVCodecDescriptor const *input_descriptor = - avcodec_descriptor_get(dec->codec_id); - AVCodecDescriptor const *output_descriptor = - avcodec_descriptor_get(ost->enc_ctx->codec_id); - if (input_descriptor) - input_props = input_descriptor->props & (AV_CODEC_PROP_TEXT_SUB | AV_CODEC_PROP_BITMAP_SUB); - if (output_descriptor) - output_props = output_descriptor->props & (AV_CODEC_PROP_TEXT_SUB | AV_CODEC_PROP_BITMAP_SUB); - if (input_props && output_props && input_props != output_props) { - snprintf(error, error_len, - "Subtitle encoding currently only possible from text to text " - "or bitmap to bitmap"); + AVCodecDescriptor const *input_descriptor = avcodec_descriptor_get(dec->codec_id); + AVCodecDescriptor const *output_descriptor = avcodec_descriptor_get(ost->enc_ctx->codec_id); + const enum AVSubtitleType in_subtitle_format = output_descriptor ? avcodec_descriptor_get_subtitle_format(input_descriptor) : AV_SUBTITLE_FMT_UNKNOWN; + const enum AVSubtitleType out_subtitle_format = output_descriptor ? avcodec_descriptor_get_subtitle_format(output_descriptor) : AV_SUBTITLE_FMT_UNKNOWN; + + if (ist->nb_filters == 0 && in_subtitle_format != AV_SUBTITLE_FMT_UNKNOWN && out_subtitle_format != AV_SUBTITLE_FMT_UNKNOWN + && in_subtitle_format != out_subtitle_format) { + snprintf(error, error_len, "Subtitle encoding is only possible from text to text or bitmap to bitmap"); return AVERROR_INVALIDDATA; } } @@ -4479,7 +4494,7 @@ static int process_input(int file_index) av_ts2timestr(input_files[ist->file_index]->ts_offset, &AV_TIME_BASE_Q)); } - sub2video_heartbeat(ist, pkt->pts); + subtitle_heartbeat(ist, pkt->pts); process_input_packet(ist, pkt, 0); @@ -4691,6 +4706,7 @@ static int transcode(void) /* at the end of stream, we must flush the decoder buffers */ for (i = 0; i < nb_input_streams; i++) { ist = input_streams[i]; + ist->subtitle_heartbeat.is_active = 0; if (!input_files[ist->file_index]->eof_reached) { process_input_packet(ist, NULL, 0); } diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index 9b200b806a..51da5ab151 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -349,17 +349,18 @@ typedef struct InputStream { struct { /* previous decoded subtitle and related variables */ int got_output; int ret; - AVSubtitle subtitle; + AVFrame *subtitle; } prev_sub; - struct sub2video { + struct subtitle_heartbeat { + int is_active; + int check_buffer_requests; int64_t last_pts; int64_t end_pts; - AVFifoBuffer *sub_queue; ///< queue of AVSubtitle* before filter init - AVFrame *frame; + AVFrame *recent_sub; int w, h; - unsigned int initialize; ///< marks if sub2video_update should force an initialization - } sub2video; + AVBufferRef *header; + } subtitle_heartbeat; /* decoded data from this stream goes into all those filters * currently video and audio only */ @@ -659,8 +660,6 @@ int filtergraph_is_simple(FilterGraph *fg); int init_simple_filtergraph(InputStream *ist, OutputStream *ost); int init_complex_filtergraph(FilterGraph *fg); -void sub2video_update(InputStream *ist, int64_t heartbeat_pts, AVSubtitle *sub); - int ifilter_parameters_from_frame(InputFilter *ifilter, const AVFrame *frame); int ffmpeg_parse_options(int argc, char **argv); diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c index 501a0acd61..d12fbdef37 100644 --- a/fftools/ffmpeg_filter.c +++ b/fftools/ffmpeg_filter.c @@ -22,6 +22,8 @@ #include "ffmpeg.h" +#include "libavutil/ass_split_internal.h" + #include "libavfilter/avfilter.h" #include "libavfilter/buffersink.h" #include "libavfilter/buffersrc.h" @@ -30,11 +32,9 @@ #include "libavutil/avstring.h" #include "libavutil/bprint.h" #include "libavutil/channel_layout.h" -#include "libavutil/display.h" #include "libavutil/opt.h" #include "libavutil/pixdesc.h" #include "libavutil/pixfmt.h" -#include "libavutil/imgutils.h" #include "libavutil/samplefmt.h" // FIXME: YUV420P etc. are actually supported with full color range, @@ -215,9 +215,8 @@ static void init_input_filter(FilterGraph *fg, AVFilterInOut *in) InputFilter *ifilter; int i; - // TODO: support other filter types - if (type != AVMEDIA_TYPE_VIDEO && type != AVMEDIA_TYPE_AUDIO) { - av_log(NULL, AV_LOG_FATAL, "Only video and audio filters supported " + if (type != AVMEDIA_TYPE_VIDEO && type != AVMEDIA_TYPE_AUDIO && type != AVMEDIA_TYPE_SUBTITLE) { + av_log(NULL, AV_LOG_FATAL, "Only video, audio and subtitle filters supported " "currently.\n"); exit_program(1); } @@ -238,8 +237,9 @@ static void init_input_filter(FilterGraph *fg, AVFilterInOut *in) for (i = 0; i < s->nb_streams; i++) { enum AVMediaType stream_type = s->streams[i]->codecpar->codec_type; if (stream_type != type && - !(stream_type == AVMEDIA_TYPE_SUBTITLE && - type == AVMEDIA_TYPE_VIDEO /* sub2video hack */)) + // in the followng case we auto-insert the graphicsub2video conversion filter + // for retaining compatibility with the previous sub2video hack + !(stream_type == AVMEDIA_TYPE_SUBTITLE && type == AVMEDIA_TYPE_VIDEO)) continue; if (check_stream_specifier(s, s->streams[i], *p == ':' ? p + 1 : p) == 1) { st = s->streams[i]; @@ -286,6 +286,17 @@ static void init_input_filter(FilterGraph *fg, AVFilterInOut *in) ifilter->type = ist->st->codecpar->codec_type; ifilter->name = describe_filter_link(fg, in, 1); + if (ist->st->codecpar->codec_type == AVMEDIA_TYPE_SUBTITLE && ist->dec_ctx) { + const AVCodecDescriptor *codec_descriptor = ist->dec_ctx->codec_descriptor; + if (!codec_descriptor) + codec_descriptor = avcodec_descriptor_get(ist->dec_ctx->codec_id); + + // For subtitles, we need to set the format here. Would we leave the format + // at -1, it would delay processing (ifilter_has_all_input_formats()) until + // the first subtile frame arrives, which could never happen in the worst case + fg->inputs[fg->nb_inputs - 1]->format = avcodec_descriptor_get_subtitle_format(codec_descriptor); + } + ifilter->frame_queue = av_fifo_alloc(8 * sizeof(AVFrame*)); if (!ifilter->frame_queue) exit_program(1); @@ -405,6 +416,39 @@ static int insert_filter(AVFilterContext **last_filter, int *pad_idx, return 0; } +static int configure_output_subtitle_filter(FilterGraph *fg, OutputFilter *ofilter, AVFilterInOut *out) +{ + OutputStream *ost = ofilter->ost; + AVFilterContext *last_filter = out->filter_ctx; + int pad_idx = out->pad_idx; + int ret; + char name[255]; + + snprintf(name, sizeof(name), "out_%d_%d", ost->file_index, ost->index); + ret = avfilter_graph_create_filter(&ofilter->filter, + avfilter_get_by_name("sbuffersink"), + name, NULL, NULL, fg->graph); + + if (ret < 0) { + av_log(NULL, AV_LOG_ERROR, "Unable to create filter sbuffersink\n"); + return ret; + } + + ////snprintf(name, sizeof(name), "trim_out_%d_%d", + //// ost->file_index, ost->index); + ////ret = insert_trim(of->start_time, of->recording_time, + //// &last_filter, &pad_idx, name); + ////if (ret < 0) + //// return ret; + + ////ost->st->codecpar->codec_tag = MKTAG('a', 's', 's', 's'); + + if ((ret = avfilter_link(last_filter, pad_idx, ofilter->filter, 0)) < 0) + return ret; + + return 0; +} + static int configure_output_video_filter(FilterGraph *fg, OutputFilter *ofilter, AVFilterInOut *out) { OutputStream *ost = ofilter->ost; @@ -585,7 +629,8 @@ static int configure_output_audio_filter(FilterGraph *fg, OutputFilter *ofilter, int i; for (i=0; ictx->nb_streams; i++) - if (of->ctx->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_VIDEO) + if (of->ctx->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_VIDEO || + of->ctx->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_SUBTITLE) break; if (ictx->nb_streams) { @@ -619,6 +664,7 @@ static int configure_output_filter(FilterGraph *fg, OutputFilter *ofilter, switch (avfilter_pad_get_type(out->filter_ctx->output_pads, out->pad_idx)) { case AVMEDIA_TYPE_VIDEO: return configure_output_video_filter(fg, ofilter, out); case AVMEDIA_TYPE_AUDIO: return configure_output_audio_filter(fg, ofilter, out); + case AVMEDIA_TYPE_SUBTITLE: return configure_output_subtitle_filter(fg, ofilter, out); default: av_assert0(0); return 0; } } @@ -638,51 +684,127 @@ void check_filter_outputs(void) } } -static int sub2video_prepare(InputStream *ist, InputFilter *ifilter) +static int configure_input_subtitle_filter(FilterGraph *fg, InputFilter *ifilter, + AVFilterInOut *in) { - AVFormatContext *avf = input_files[ist->file_index]->ctx; - int i, w, h; + AVFilterContext *last_filter; + const AVFilter *buffer_filt = avfilter_get_by_name("sbuffer"); + InputStream *ist = ifilter->ist; + AVBPrint args; + char name[255]; + int ret, pad_idx = 0; + int w, h; + AVBufferSrcParameters *par = av_buffersrc_parameters_alloc(); + enum AVMediaType media_type; + + if (!par) + return AVERROR(ENOMEM); + + par->format = AV_PIX_FMT_NONE; + + if (ist->dec_ctx->codec_type == AVMEDIA_TYPE_AUDIO) { + av_log(NULL, AV_LOG_ERROR, "Cannot connect subtitle filter to audio input\n"); + ret = AVERROR(EINVAL); + goto fail; + } + + if (ist->dec_ctx->codec_type == AVMEDIA_TYPE_VIDEO) { + av_log(NULL, AV_LOG_ERROR, "Cannot connect subtitle filter to video input\n"); + ret = AVERROR(EINVAL); + goto fail; + } + + if (!ist->subtitle_heartbeat.header && ist->dec_ctx->subtitle_header && ist->dec_ctx->subtitle_header_size > 0) { + ist->subtitle_heartbeat.header = av_buffer_allocz(ist->dec_ctx->subtitle_header_size + 1); + if (!ist->subtitle_heartbeat.header) + return AVERROR(ENOMEM); + memcpy(ist->subtitle_heartbeat.header->data, ist->dec_ctx->subtitle_header, ist->dec_ctx->subtitle_header_size); + } + + ist->subtitle_heartbeat.is_active = 1; - /* Compute the size of the canvas for the subtitles stream. - If the subtitles codecpar has set a size, use it. Otherwise use the - maximum dimensions of the video streams in the same file. */ w = ifilter->width; h = ifilter->height; + if (!(w && h)) { - for (i = 0; i < avf->nb_streams; i++) { - if (avf->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_VIDEO) { - w = FFMAX(w, avf->streams[i]->codecpar->width); - h = FFMAX(h, avf->streams[i]->codecpar->height); - } - } - if (!(w && h)) { - w = FFMAX(w, 720); - h = FFMAX(h, 576); - } - av_log(avf, AV_LOG_INFO, "sub2video: using %dx%d canvas\n", w, h); + w = ist->dec_ctx->width; + h = ist->dec_ctx->height; } - ist->sub2video.w = ifilter->width = w; - ist->sub2video.h = ifilter->height = h; - ifilter->width = ist->dec_ctx->width ? ist->dec_ctx->width : ist->sub2video.w; - ifilter->height = ist->dec_ctx->height ? ist->dec_ctx->height : ist->sub2video.h; + if (!(w && h) && ist->dec_ctx->subtitle_header) { + ASSSplitContext *ass_ctx = avpriv_ass_split((char *)ist->dec_ctx->subtitle_header); + ASS *ass = (ASS *)ass_ctx; + w = ass->script_info.play_res_x; + h = ass->script_info.play_res_y; + avpriv_ass_split_free(ass_ctx); + } - /* rectangles are AV_PIX_FMT_PAL8, but we have no guarantee that the - palettes for all rectangles are identical or compatible */ - ifilter->format = AV_PIX_FMT_RGB32; + ist->subtitle_heartbeat.w = w; + ist->subtitle_heartbeat.h = h; + av_log(ifilter, AV_LOG_INFO, "subtitle input filter: decoding size %dx%d\n", ist->subtitle_heartbeat.w, ist->subtitle_heartbeat.h); + + ifilter->width = w; + ifilter->height = h; + ist->dec_ctx->width = w; + ist->dec_ctx->height = h; + + ist->subtitle_heartbeat.last_pts = INT64_MIN; + + snprintf(name, sizeof(name), "graph %d subtitle input from stream %d:%d", fg->index, + ist->file_index, ist->st->index); - ist->sub2video.frame = av_frame_alloc(); - if (!ist->sub2video.frame) - return AVERROR(ENOMEM); - ist->sub2video.last_pts = INT64_MIN; - ist->sub2video.end_pts = INT64_MIN; - /* sub2video structure has been (re-)initialized. - Mark it as such so that the system will be - initialized with the first received heartbeat. */ - ist->sub2video.initialize = 1; + av_bprint_init(&args, 0, AV_BPRINT_SIZE_AUTOMATIC); + av_bprintf(&args, + "subtitle_type=%d:width=%d:height=%d:time_base=%d/%d:", + ifilter->format, ifilter->width, ifilter->height, + ist->st->time_base.num, ist->st->time_base.den); + if ((ret = avfilter_graph_create_filter(&ifilter->filter, buffer_filt, name, + args.str, NULL, fg->graph)) < 0) + goto fail; + + par->hw_frames_ctx = ifilter->hw_frames_ctx; + par->format = ifilter->format; + par->width = ifilter->width; + par->height = ifilter->height; + + ret = av_buffersrc_parameters_set(ifilter->filter, par); + if (ret < 0) + goto fail; + av_freep(&par); + last_filter = ifilter->filter; + + media_type = avfilter_pad_get_type(in->filter_ctx->input_pads, in->pad_idx); + if (media_type == AVMEDIA_TYPE_VIDEO) { + // If the subtitle frame size is unknown, try to find a video input + // and use its size for adding a subscale filter + for (int i = 0; i < fg->nb_inputs; i++) { + InputFilter *input = fg->inputs[i]; + if (input->type == AVMEDIA_TYPE_VIDEO && input->width && input->height) { + char subscale_params[64]; + snprintf(subscale_params, sizeof(subscale_params), "w=%d:h=%d", input->width, input->height); + ret = insert_filter(&last_filter, &pad_idx, "subscale", subscale_params); + if (ret < 0) + return ret; + break; + } + } + + // This is for sub2video compatibility + av_log(NULL, AV_LOG_INFO, "Auto-inserting graphicsub2video filter\n"); + ret = insert_filter(&last_filter, &pad_idx, "graphicsub2video", NULL); + if (ret < 0) + return ret; + } + + if ((ret = avfilter_link(last_filter, 0, in->filter_ctx, in->pad_idx)) < 0) + return ret; return 0; +fail: + av_freep(&par); + + return ret; } static int configure_input_video_filter(FilterGraph *fg, InputFilter *ifilter, @@ -701,8 +823,15 @@ static int configure_input_video_filter(FilterGraph *fg, InputFilter *ifilter, char name[255]; int ret, pad_idx = 0; int64_t tsoffset = 0; - AVBufferSrcParameters *par = av_buffersrc_parameters_alloc(); + AVBufferSrcParameters *par; + if (ist->dec_ctx->codec_type == AVMEDIA_TYPE_SUBTITLE) { + // Automatically insert conversion filter to retain compatibility + // with sub2video command lines + return configure_input_subtitle_filter(fg, ifilter, in); + } + + par = av_buffersrc_parameters_alloc(); if (!par) return AVERROR(ENOMEM); memset(par, 0, sizeof(*par)); @@ -717,12 +846,6 @@ static int configure_input_video_filter(FilterGraph *fg, InputFilter *ifilter, if (!fr.num) fr = av_guess_frame_rate(input_files[ist->file_index]->ctx, ist->st, NULL); - if (ist->dec_ctx->codec_type == AVMEDIA_TYPE_SUBTITLE) { - ret = sub2video_prepare(ist, ifilter); - if (ret < 0) - goto fail; - } - sar = ifilter->sample_aspect_ratio; if(!sar.den) sar = (AVRational){0,1}; @@ -734,7 +857,7 @@ static int configure_input_video_filter(FilterGraph *fg, InputFilter *ifilter, tb.num, tb.den, sar.num, sar.den); if (fr.num && fr.den) av_bprintf(&args, ":frame_rate=%d/%d", fr.num, fr.den); - snprintf(name, sizeof(name), "graph %d input from stream %d:%d", fg->index, + snprintf(name, sizeof(name), "graph %d video input from stream %d:%d", fg->index, ist->file_index, ist->st->index); @@ -940,6 +1063,7 @@ static int configure_input_filter(FilterGraph *fg, InputFilter *ifilter, switch (avfilter_pad_get_type(in->filter_ctx->input_pads, in->pad_idx)) { case AVMEDIA_TYPE_VIDEO: return configure_input_video_filter(fg, ifilter, in); case AVMEDIA_TYPE_AUDIO: return configure_input_audio_filter(fg, ifilter, in); + case AVMEDIA_TYPE_SUBTITLE: return configure_input_subtitle_filter(fg, ifilter, in); default: av_assert0(0); return 0; } } @@ -1133,19 +1257,6 @@ int configure_filtergraph(FilterGraph *fg) } } - /* process queued up subtitle packets */ - for (i = 0; i < fg->nb_inputs; i++) { - InputStream *ist = fg->inputs[i]->ist; - if (ist->sub2video.sub_queue && ist->sub2video.frame) { - while (av_fifo_size(ist->sub2video.sub_queue)) { - AVSubtitle tmp; - av_fifo_generic_read(ist->sub2video.sub_queue, &tmp, sizeof(tmp), NULL); - sub2video_update(ist, INT64_MIN, &tmp); - avsubtitle_free(&tmp); - } - } - } - return 0; fail: @@ -1168,6 +1279,7 @@ int ifilter_parameters_from_frame(InputFilter *ifilter, const AVFrame *frame) ifilter->sample_rate = frame->sample_rate; ifilter->channels = frame->channels; ifilter->channel_layout = frame->channel_layout; + ifilter->type = frame->type; av_freep(&ifilter->displaymatrix); sd = av_frame_get_side_data(frame, AV_FRAME_DATA_DISPLAYMATRIX); diff --git a/fftools/ffmpeg_hw.c b/fftools/ffmpeg_hw.c index 14e702bd92..be69d54aaf 100644 --- a/fftools/ffmpeg_hw.c +++ b/fftools/ffmpeg_hw.c @@ -449,7 +449,7 @@ int hw_device_setup_for_encode(OutputStream *ost) AVBufferRef *frames_ref = NULL; int i; - if (ost->filter) { + if (ost->filter && ost->filter->filter) { frames_ref = av_buffersink_get_hw_frames_ctx(ost->filter->filter); if (frames_ref && ((AVHWFramesContext*)frames_ref->data)->format == diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c index 9c820ab73f..476ef628e4 100644 --- a/fftools/ffmpeg_opt.c +++ b/fftools/ffmpeg_opt.c @@ -2211,8 +2211,9 @@ static void init_output_filter(OutputFilter *ofilter, OptionsContext *o, switch (ofilter->type) { case AVMEDIA_TYPE_VIDEO: ost = new_video_stream(o, oc, -1); break; case AVMEDIA_TYPE_AUDIO: ost = new_audio_stream(o, oc, -1); break; + case AVMEDIA_TYPE_SUBTITLE: ost = new_subtitle_stream(o, oc, -1); break; default: - av_log(NULL, AV_LOG_FATAL, "Only video and audio filters are supported " + av_log(NULL, AV_LOG_FATAL, "Only video, audio and subtitle filters are supported " "currently.\n"); exit_program(1); } diff --git a/tests/ref/fate/filter-overlay-dvdsub-2397 b/tests/ref/fate/filter-overlay-dvdsub-2397 index 483e5fa4e0..3bf198d6ad 100644 --- a/tests/ref/fate/filter-overlay-dvdsub-2397 +++ b/tests/ref/fate/filter-overlay-dvdsub-2397 @@ -490,368 +490,367 @@ 1, 3877, 3877, 10, 2013, 0x95a39f9c 1, 3887, 3887, 10, 2013, 0x4f7ea123 1, 3897, 3897, 10, 2013, 0x9efb9ba1 -0, 117, 117, 1, 518400, 0xbf8523da +0, 117, 117, 1, 518400, 0x61e0f688 1, 3907, 3907, 10, 2013, 0xf395b2cd 1, 3917, 3917, 10, 2013, 0x261a881e 1, 3927, 3927, 10, 2013, 0x7f2d9f72 1, 3937, 3937, 10, 2013, 0x0105b38d -0, 118, 118, 1, 518400, 0x41890ed6 +0, 118, 118, 1, 518400, 0xa47de755 1, 3952, 3952, 10, 2013, 0x0e5db67e 1, 3962, 3962, 10, 2013, 0xfc9baf97 -0, 119, 119, 1, 518400, 0x588534fc +0, 119, 119, 1, 518400, 0x52d52e73 1, 3972, 3972, 10, 2013, 0x8e02a1b1 1, 3982, 3982, 10, 2013, 0x6eecaac8 1, 3992, 3992, 10, 2013, 0xf5558f0c 1, 4002, 4002, 10, 2013, 0x512ba99b -0, 120, 120, 1, 518400, 0x2145ebc1 +0, 120, 120, 1, 518400, 0xc732e546 1, 4012, 4012, 10, 2013, 0x932b9932 1, 4022, 4022, 10, 2013, 0xc01ea987 -0, 121, 121, 1, 518400, 0x28bca595 +0, 121, 121, 1, 518400, 0xc36f9f14 1, 4038, 4038, 10, 2013, 0x10879cf7 1, 4048, 4048, 10, 2013, 0x90679338 1, 4058, 4058, 10, 2013, 0x077d8a9e 1, 4068, 4068, 10, 2013, 0x969fa57c -0, 122, 122, 1, 518400, 0x77dc951e +0, 122, 122, 1, 518400, 0x78428e8b 1, 4078, 4078, 10, 2013, 0xe049ab07 1, 4088, 4088, 10, 2013, 0xf535b3b3 1, 4098, 4098, 10, 2013, 0xfe76bd37 -0, 123, 123, 1, 518400, 0xe8924c17 +0, 123, 123, 1, 518400, 0xf0f8458d 1, 4108, 4108, 10, 2013, 0xde79ad8c 1, 4123, 4123, 10, 2013, 0xe89b9c47 1, 4133, 4133, 10, 2013, 0xc570b0f0 -0, 124, 124, 1, 518400, 0xadb4cccc +0, 124, 124, 1, 518400, 0x7083c653 1, 4143, 4143, 10, 2013, 0xee709cd9 1, 4153, 4153, 10, 2013, 0xcfe5afab 1, 4163, 4163, 10, 2013, 0x98ff8ce4 -0, 125, 125, 1, 518400, 0x1d7b56ac +0, 125, 125, 1, 518400, 0xa105502c 1, 4173, 4173, 10, 2013, 0x9d19b44c 1, 4183, 4183, 10, 2013, 0x4349917a 1, 4193, 4193, 10, 2013, 0xbf54a59a -0, 126, 126, 1, 518400, 0xad5739a4 +0, 126, 126, 1, 518400, 0xd411331a 1, 4208, 4208, 10, 2013, 0xc4a399e0 1, 4218, 4218, 10, 2013, 0x1bf58ff0 1, 4228, 4228, 10, 2013, 0x3518ac56 -0, 127, 127, 1, 518400, 0x2733d35a +0, 127, 127, 1, 518400, 0x83b0ccdb 1, 4238, 4238, 10, 2013, 0xcd38c1de 1, 4248, 4248, 10, 2013, 0xbe7d9c4d 1, 4258, 4258, 10, 2013, 0xe113a306 1, 4268, 4268, 10, 2013, 0x083197ea -0, 128, 128, 1, 518400, 0x78e76da2 +0, 128, 128, 1, 518400, 0xa9be671a 1, 4278, 4278, 10, 2013, 0x1929b1eb 1, 4294, 4294, 10, 2013, 0x5d6ea5af 1, 4304, 4304, 10, 2013, 0x05519d53 -0, 129, 129, 1, 518400, 0x6c076013 +0, 129, 129, 1, 518400, 0xaeb75983 1, 4314, 4314, 10, 2013, 0x5773b380 1, 4324, 4324, 10, 2013, 0xaa70a8f5 1, 4334, 4334, 10, 2013, 0x990db0ec -0, 130, 130, 1, 518400, 0x7854f2b1 +0, 130, 130, 1, 518400, 0x81f8ec13 1, 4344, 4344, 10, 2013, 0x91d3a623 1, 4354, 4354, 10, 2013, 0xc91f9824 1, 4364, 4364, 10, 2013, 0x1d058abf -0, 131, 131, 1, 518400, 0xd2ae1ecd +0, 131, 131, 1, 518400, 0x8aaa1839 1, 4379, 4379, 10, 2013, 0x8de1b8d5 1, 4389, 4389, 10, 2013, 0x7872b06b 1, 4399, 4399, 10, 2013, 0xa084c203 -0, 132, 132, 1, 518400, 0xf5eab38d +0, 132, 132, 1, 518400, 0xc98bacf5 1, 4409, 4409, 10, 2013, 0xff90ae8d 1, 4419, 4419, 10, 2013, 0x61dead8e 1, 4429, 4429, 10, 2013, 0xee76b284 -0, 133, 133, 1, 518400, 0x994d3e9c +0, 133, 133, 1, 518400, 0x31083804 1, 4439, 4439, 10, 2013, 0xe888af7f 1, 4449, 4449, 10, 2013, 0x5d57b115 1, 4464, 4464, 10, 2013, 0xcdbfb1d0 -0, 134, 134, 1, 518400, 0x95ab705a +0, 134, 134, 1, 518400, 0x540a69dc 1, 4474, 4474, 10, 2013, 0x2e28a952 1, 4484, 4484, 10, 2013, 0x4795a994 1, 4494, 4494, 10, 2013, 0x7e7ea304 1, 4504, 4504, 10, 2013, 0x9502c1e1 -0, 135, 135, 1, 518400, 0x3c83c5ce +0, 135, 135, 1, 518400, 0x80d3bf46 1, 4514, 4514, 10, 2013, 0xf7c78ab2 1, 4524, 4524, 10, 2013, 0x24049816 1, 4534, 4534, 10, 2013, 0x52089dcf -0, 136, 136, 1, 518400, 0xfa22c508 +0, 136, 136, 1, 518400, 0x2967be7f 1, 4550, 4550, 10, 2013, 0x2150a0b1 1, 4560, 4560, 10, 2013, 0x3c2e9b93 1, 4570, 4570, 10, 2013, 0x491f932b -0, 137, 137, 1, 518400, 0xddda1712 +0, 137, 137, 1, 518400, 0x5a3b1092 1, 4580, 4580, 10, 2013, 0x31359cf8 1, 4590, 4590, 10, 2013, 0x1b00ac3f 1, 4600, 4600, 10, 2013, 0x8d7ab3cb -0, 138, 138, 1, 518400, 0x985a3b93 +0, 138, 138, 1, 518400, 0x8741350b 1, 4610, 4610, 10, 2013, 0xb2c2a4de 1, 4620, 4620, 10, 2013, 0x80a4abf2 1, 4635, 4635, 10, 2013, 0x0701a4ee -0, 139, 139, 1, 518400, 0xea63c5e7 +0, 139, 139, 1, 518400, 0xd5a9bf60 1, 4645, 4645, 10, 2013, 0xdc1ba5bc 1, 4655, 4655, 10, 2013, 0x6083a8a4 1, 4665, 4665, 10, 2013, 0x6226ad45 -0, 140, 140, 1, 518400, 0xef64983d +0, 140, 140, 1, 518400, 0xc05f91ba 1, 4675, 4675, 10, 2013, 0x2732a205 1, 4685, 4685, 10, 2013, 0x0f62a0d3 1, 4695, 4695, 10, 2013, 0xc1799249 -0, 141, 141, 1, 518400, 0x747bb193 +0, 141, 141, 1, 518400, 0x3fdaab0b 1, 4705, 4705, 10, 2013, 0xbccfa9c8 1, 4720, 4720, 10, 2013, 0xded096e7 1, 4730, 4730, 10, 2013, 0x7f0daf43 -0, 142, 142, 1, 518400, 0xb8748862 +0, 142, 142, 1, 518400, 0xab7281d9 1, 4740, 4740, 10, 2013, 0xc47ea682 1, 4750, 4750, 10, 2013, 0x5a72b07a 1, 4760, 4760, 10, 2013, 0x386faa8c 1, 4770, 4770, 10, 2013, 0xf9919a91 -0, 143, 143, 1, 518400, 0xaab55a5f +0, 143, 143, 1, 518400, 0xc80053d6 1, 4780, 4780, 10, 2013, 0x4908897e 1, 4790, 4790, 10, 2013, 0x4882b594 -0, 144, 144, 1, 518400, 0x7b468add +0, 144, 144, 1, 518400, 0x6526845c 1, 4806, 4806, 10, 2013, 0x113e98d1 1, 4816, 4816, 10, 2013, 0x5098b30d 1, 4826, 4826, 10, 2013, 0x0ef7b857 1, 4836, 4836, 10, 2013, 0x216ea176 -0, 145, 145, 1, 518400, 0xf2078707 +0, 145, 145, 1, 518400, 0x1b788089 1, 4846, 4846, 10, 2013, 0xf906944a 1, 4856, 4856, 10, 2013, 0xee9b92fb 1, 4866, 4866, 10, 2013, 0xd6029209 -0, 146, 146, 1, 518400, 0x6a2d931e +0, 146, 146, 1, 518400, 0xfa8e8ca9 1, 4876, 4876, 10, 2013, 0x2256a12e 1, 4891, 4891, 10, 2013, 0x89de8e4a 1, 4901, 4901, 10, 2013, 0x0bf0a584 -0, 147, 147, 1, 518400, 0xbbe3c417 +0, 147, 147, 1, 518400, 0xb278bda1 1, 4911, 4911, 10, 2013, 0x6a5ebd58 1, 4921, 4921, 10, 2013, 0x3edd9aa4 1, 4931, 4931, 10, 2013, 0xbd66ac26 -0, 148, 148, 1, 518400, 0x6294e449 +0, 148, 148, 1, 518400, 0xb0c3ddca 1, 4941, 4941, 10, 2013, 0x313896ea 1, 4951, 4951, 10, 2013, 0x6b83a6a0 1, 4961, 4961, 10, 2013, 0x9aafb109 -0, 149, 149, 1, 518400, 0xa05721e7 +0, 149, 149, 1, 518400, 0x10351b53 1, 4976, 4976, 10, 2013, 0x5192a85a 1, 4986, 4986, 10, 2013, 0x1f919f79 1, 4996, 4996, 10, 2013, 0xc0799c40 -0, 150, 150, 1, 518400, 0x37749183 +0, 150, 150, 1, 518400, 0xc1408aee 1, 5006, 5006, 10, 2013, 0x2988bcd8 1, 5016, 5016, 10, 2013, 0x1482913a 1, 5026, 5026, 10, 2013, 0x74da9a94 1, 5036, 5036, 10, 2013, 0x763eb709 -0, 151, 151, 1, 518400, 0xf9d9dca0 +0, 151, 151, 1, 518400, 0xf016d615 1, 5046, 5046, 10, 2013, 0x1285b405 1, 5062, 5062, 10, 2013, 0xb6ab9dfc -0, 152, 152, 1, 518400, 0x5f8ccf08 +0, 152, 152, 1, 518400, 0xa768c892 1, 5072, 5072, 10, 2013, 0xe4c8bf19 1, 5082, 5082, 10, 2013, 0xabbbade8 1, 5092, 5092, 10, 2013, 0xf8b69d89 1, 5102, 5102, 10, 2013, 0xce04a866 -0, 153, 153, 1, 518400, 0x7303f77b +0, 153, 153, 1, 518400, 0x11c3f11e 1, 5112, 5112, 10, 2013, 0x07528abf 1, 5122, 5122, 10, 2013, 0x74fb98bf 1, 5132, 5132, 10, 2013, 0x579fb1c9 -0, 154, 154, 1, 518400, 0x22b0513f +0, 154, 154, 1, 518400, 0xcd9a4ac4 1, 5147, 5147, 10, 2013, 0x7ddea2ed 1, 5157, 5157, 10, 2013, 0x296caa2c 1, 5167, 5167, 10, 2013, 0x346d9c4f -0, 155, 155, 1, 518400, 0x330485d2 +0, 155, 155, 1, 518400, 0x4ade7f5e 1, 5177, 5177, 10, 2013, 0x3e1fba15 1, 5187, 5187, 10, 2013, 0x48a2908f 1, 5197, 5197, 10, 2013, 0xc1938d09 -0, 156, 156, 1, 518400, 0x7f83daea +0, 156, 156, 1, 518400, 0x655dd46b 1, 5207, 5207, 10, 2013, 0x0e96a060 1, 5217, 5217, 10, 2013, 0x7b6a9e06 1, 5232, 5232, 10, 2013, 0x5b779d28 -0, 157, 157, 1, 518400, 0xee19f2df +0, 157, 157, 1, 518400, 0x5ab5ec61 1, 5242, 5242, 10, 2013, 0xf600aca1 1, 5252, 5252, 10, 2013, 0x3a6c9e68 1, 5262, 5262, 10, 2013, 0x0c8dc1b0 -0, 158, 158, 1, 518400, 0xb71b1c77 +0, 158, 158, 1, 518400, 0x45dc15e6 1, 5272, 5272, 10, 2013, 0x26beb245 1, 5282, 5282, 10, 2013, 0x2bc09557 1, 5292, 5292, 10, 2013, 0x27fc8845 1, 5302, 5302, 10, 2013, 0x1025aa47 -0, 159, 159, 1, 518400, 0xbffc1856 +0, 159, 159, 1, 518400, 0x201911d3 1, 5318, 5318, 10, 2013, 0xc2e69baa 1, 5328, 5328, 10, 2013, 0xdb249b92 1, 5338, 5338, 10, 2013, 0x6ccda29e -0, 160, 160, 1, 518400, 0xabc125aa +0, 160, 160, 1, 518400, 0x0fbc1f46 1, 5348, 5348, 10, 2013, 0xeaf6a1cf 1, 5358, 5358, 10, 2013, 0x509ba397 1, 5368, 5368, 10, 2013, 0xfaf8a2df -0, 161, 161, 1, 518400, 0x5ee467f8 +0, 161, 161, 1, 518400, 0x7e316179 1, 5378, 5378, 10, 2013, 0x41388f28 1, 5388, 5388, 10, 2013, 0xfe5eab39 1, 5403, 5403, 10, 2013, 0xd5ffa066 -0, 162, 162, 1, 518400, 0x6c2cf168 +0, 162, 162, 1, 518400, 0x73bbeaed 1, 5413, 5413, 10, 2013, 0x6813a30a 1, 5423, 5423, 10, 2013, 0x9be89718 1, 5433, 5433, 10, 2013, 0xaec3a27b -0, 163, 163, 1, 518400, 0x63996b26 +0, 163, 163, 1, 518400, 0x3a7c648a 1, 5446, 5446, 10, 2013, 0x579a983e 1, 5456, 5456, 10, 2013, 0x98cea21f 1, 5466, 5466, 10, 2013, 0xca77a58a -0, 164, 164, 1, 518400, 0xb34d789a +0, 164, 164, 1, 518400, 0x9f707209 1, 5476, 5476, 10, 2013, 0xcbc3b1ee 1, 5486, 5486, 10, 2013, 0xf3bb8f07 1, 5496, 5496, 10, 2013, 0x6aeebd92 -0, 165, 165, 1, 518400, 0xf49c030f +0, 165, 165, 1, 518400, 0x9f25fc5c 1, 5506, 5506, 10, 2013, 0xe955a449 1, 5516, 5516, 10, 2013, 0x9436aa5b 1, 5531, 5531, 10, 2013, 0x4f0a8f9f -0, 166, 166, 1, 518400, 0x092dc41a +0, 166, 166, 1, 518400, 0x2ed8bd75 1, 5541, 5541, 10, 2013, 0x3551b22d 1, 5551, 5551, 10, 2013, 0x0959a3d4 1, 5561, 5561, 10, 2013, 0x2ed5a11b 1, 5571, 5571, 10, 2013, 0x8f52a5c3 -0, 167, 167, 1, 518400, 0x4134c577 +0, 167, 167, 1, 518400, 0xb493becb 1, 5581, 5581, 10, 2013, 0x6552978d 1, 5591, 5591, 10, 2013, 0x7dcca0c1 1, 5601, 5601, 10, 2013, 0xbcd4a3c9 -0, 168, 168, 1, 518400, 0x261de1ed +0, 168, 168, 1, 518400, 0x7df6db57 1, 5616, 5616, 10, 2013, 0xfe41a8d8 1, 5626, 5626, 10, 2013, 0xc85aae14 1, 5636, 5636, 10, 2013, 0x1185b346 -0, 169, 169, 1, 518400, 0xcbc8566a +0, 169, 169, 1, 518400, 0x1cb94fca 1, 5646, 5646, 10, 2013, 0xf7429a0d 1, 5656, 5656, 10, 2013, 0x48c2a160 1, 5666, 5666, 10, 2013, 0x9d85a85d -0, 170, 170, 1, 518400, 0x407a5c76 +0, 170, 170, 1, 518400, 0x70db55d8 1, 5676, 5676, 10, 2013, 0xbbe89fe9 1, 5686, 5686, 10, 2013, 0xea429fe2 1, 5702, 5702, 10, 2013, 0x221ca1d4 -0, 171, 171, 1, 518400, 0x1ed73bb2 +0, 171, 171, 1, 518400, 0xc1d9351b 1, 5712, 5712, 10, 2013, 0x394b925b 1, 5722, 5722, 10, 2013, 0x556dc26f 1, 5732, 5732, 10, 2013, 0xce21a5e1 -0, 172, 172, 1, 518400, 0x8467ddb5 +0, 172, 172, 1, 518400, 0xa4b0d717 1, 5742, 5742, 10, 2013, 0xbc87c0a8 1, 5752, 5752, 10, 2013, 0xbac4ac07 1, 5762, 5762, 10, 2013, 0xdeefa4aa 1, 5772, 5772, 10, 2013, 0x1f15b362 -0, 173, 173, 1, 518400, 0x0523dc73 +0, 173, 173, 1, 518400, 0x3730d5e9 1, 5787, 5787, 10, 2013, 0x6406b7b2 1, 5797, 5797, 10, 2013, 0x8030a03d -0, 174, 174, 1, 518400, 0x81f5e895 +0, 174, 174, 1, 518400, 0x9673e1ec 1, 5807, 5807, 10, 2013, 0x0373a5b1 1, 5817, 5817, 10, 2013, 0x34ef93da 1, 5827, 5827, 10, 2013, 0x94c198fe 1, 5837, 5837, 10, 2013, 0xfefcabad -0, 175, 175, 1, 518400, 0xfc74608d +0, 175, 175, 1, 518400, 0x877959d5 1, 5847, 5847, 10, 2013, 0x8755b3ec 1, 5857, 5857, 10, 2013, 0xe436a6fd 1, 5872, 5872, 10, 2013, 0x9cf5a11e -0, 176, 176, 1, 518400, 0xc4e0dae0 +0, 176, 176, 1, 518400, 0x04f3d421 1, 5882, 5882, 10, 2013, 0x03b8a98c 1, 5892, 5892, 10, 2013, 0x6216a138 1, 5902, 5902, 10, 2013, 0xd87b9f12 -0, 177, 177, 1, 518400, 0x98367f5b +0, 177, 177, 1, 518400, 0x4f3078bc 1, 5912, 5912, 10, 2013, 0x4ce99653 1, 5922, 5922, 10, 2013, 0x6c2ea9e2 1, 5932, 5932, 10, 2013, 0x918cae4c -0, 178, 178, 1, 518400, 0x0f1a869d +0, 178, 178, 1, 518400, 0x8a127ff8 1, 5942, 5942, 10, 2013, 0xd19fa5f2 1, 5958, 5958, 10, 2013, 0x0bdda7c6 1, 5968, 5968, 10, 2013, 0x0f9ab0ca -0, 179, 179, 1, 518400, 0x45b6ccf2 +0, 179, 179, 1, 518400, 0x5864c64f 1, 5978, 5978, 10, 2013, 0x410a92b1 1, 5988, 5988, 10, 2013, 0xcfbe9d1c 1, 5998, 5998, 10, 2013, 0x59ed9d15 -0, 180, 180, 1, 518400, 0x5f9ccb77 +0, 180, 180, 1, 518400, 0xdaccc4c0 1, 6008, 6008, 10, 2013, 0x4e129e27 1, 6018, 6018, 10, 2013, 0x7bb9ac0a 1, 6028, 6028, 10, 2013, 0x826ca82b -0, 181, 181, 1, 518400, 0x5f15ea31 +0, 181, 181, 1, 518400, 0xd999e376 1, 6043, 6043, 10, 2013, 0x9ad5a74b 1, 6053, 6053, 10, 2013, 0x6c5f969a 1, 6063, 6063, 10, 2013, 0x8479a0e5 -0, 182, 182, 1, 518400, 0x86369f27 +0, 182, 182, 1, 518400, 0x8af39876 1, 6073, 6073, 10, 2013, 0x165298ef 1, 6083, 6083, 10, 2013, 0xdcadb4a1 1, 6093, 6093, 10, 2013, 0xa90e987c 1, 6103, 6103, 10, 2013, 0x1ac5b510 -0, 183, 183, 1, 518400, 0x2e27f9fa +0, 183, 183, 1, 518400, 0x5e72f33d 1, 6113, 6113, 10, 2013, 0x66728d85 1, 6128, 6128, 10, 2013, 0xe4859fc5 1, 6138, 6138, 10, 2013, 0x9901786e -0, 184, 184, 1, 518400, 0xc029a44d +0, 184, 184, 1, 518400, 0x14af9d92 1, 6148, 6148, 10, 2013, 0x6aebb406 1, 6158, 6158, 10, 2013, 0x7d13a2cc 1, 6168, 6168, 10, 2013, 0x99b7a8cc -0, 185, 185, 1, 518400, 0xebee33b0 +0, 185, 185, 1, 518400, 0x50b82d10 1, 6178, 6178, 10, 2013, 0x80b8a624 1, 6188, 6188, 10, 2013, 0xbb6aa271 1, 6198, 6198, 10, 2013, 0x17af9e4a -0, 186, 186, 1, 518400, 0x19e5494f +0, 186, 186, 1, 518400, 0xc068429c 1, 6214, 6214, 10, 2013, 0xfaf0a8f1 1, 6224, 6224, 10, 2013, 0xd6849b93 1, 6234, 6234, 10, 2013, 0xe9829669 -0, 187, 187, 1, 518400, 0xf697bd7c +0, 187, 187, 1, 518400, 0x8934b6d1 1, 6244, 6244, 10, 2013, 0x7ec98944 1, 6254, 6254, 10, 2013, 0x2b2099a4 1, 6264, 6264, 10, 2013, 0x1033a82f -0, 188, 188, 1, 518400, 0x82569002 +0, 188, 188, 1, 518400, 0x11d08947 1, 6274, 6274, 10, 2013, 0x5ec88990 1, 6284, 6284, 10, 2013, 0xd2a19b3d 1, 6299, 6299, 10, 2013, 0xa377b268 -0, 189, 189, 1, 518400, 0xfcb6d707 +0, 189, 189, 1, 518400, 0x8a27d041 1, 6309, 6309, 10, 2013, 0xfa859901 1, 6319, 6319, 10, 2013, 0x1713955a 1, 6329, 6329, 10, 2013, 0x70aab0da 1, 6339, 6339, 10, 2013, 0xcdaea422 -0, 190, 190, 1, 518400, 0x82a9662b +0, 190, 190, 1, 518400, 0xab265f7d 1, 6349, 6349, 10, 2013, 0x65c3bf80 1, 6359, 6359, 10, 2013, 0x1d75a55f 1, 6369, 6369, 10, 2013, 0xa5bea4de -0, 191, 191, 1, 518400, 0x212e16ee +0, 191, 191, 1, 518400, 0xff491040 1, 6384, 6384, 10, 2013, 0x184db71c 1, 6394, 6394, 10, 2013, 0x99858ec8 1, 6404, 6404, 10, 2013, 0xb8f2aee5 -0, 192, 192, 1, 518400, 0x2ca34dca +0, 192, 192, 1, 518400, 0x822b4704 1, 6414, 6414, 10, 2013, 0x4435b2ef 1, 6424, 6424, 10, 2013, 0x8acfa6c7 1, 6434, 6434, 10, 2013, 0x42b4c01f -0, 193, 193, 1, 518400, 0xe9ebe0a5 +0, 193, 193, 1, 518400, 0x4523d9f4 1, 6444, 6444, 10, 2013, 0x6e308c13 1, 6454, 6454, 10, 2013, 0x8227a0f6 1, 6470, 6470, 10, 2013, 0x6f12a7a2 -0, 194, 194, 1, 518400, 0x4e6b6917 +0, 194, 194, 1, 518400, 0xfc3c626e 1, 6480, 6480, 10, 2013, 0x785392be 1, 6490, 6490, 10, 2013, 0x81849c2b 1, 6500, 6500, 10, 2013, 0x5cf2af65 -0, 195, 195, 1, 518400, 0x7dcf20ab +0, 195, 195, 1, 518400, 0x237319e5 1, 6510, 6510, 10, 2013, 0x0c6ca6b4 1, 6520, 6520, 10, 2013, 0x412fab9f 1, 6530, 6530, 10, 2013, 0x08e792b4 -0, 196, 196, 1, 518400, 0xf30fac97 +0, 196, 196, 1, 518400, 0x892ca5d8 1, 6540, 6540, 10, 2013, 0x407aace3 1, 6555, 6555, 10, 2013, 0xd26bac16 1, 6565, 6565, 10, 2013, 0xac8bb295 -0, 197, 197, 1, 518400, 0xcb9fc692 +0, 197, 197, 1, 518400, 0xc4c0bfc7 1, 6575, 6575, 10, 2013, 0xddd1949c 1, 6585, 6585, 10, 2013, 0x6b26b868 1, 6595, 6595, 10, 2013, 0x5eaba587 1, 6605, 6605, 10, 2013, 0xef0793b9 -0, 198, 198, 1, 518400, 0x5d05601e +0, 198, 198, 1, 518400, 0x57c85956 1, 6615, 6615, 10, 2013, 0xdef19bd6 1, 6625, 6625, 10, 2013, 0xca98a635 -0, 199, 199, 1, 518400, 0x456c1417 +0, 199, 199, 1, 518400, 0xd6300d46 1, 6640, 6640, 10, 2013, 0x06269a5a 1, 6650, 6650, 10, 2013, 0x32cb9952 1, 6660, 6660, 10, 2013, 0xf01fa95a 1, 6670, 6670, 10, 2013, 0xefab9e55 -0, 200, 200, 1, 518400, 0x9a0fd1ad +0, 200, 200, 1, 518400, 0xd3dacaec 1, 6680, 6680, 10, 2013, 0x55a3b63a 1, 6690, 6690, 10, 2013, 0xcd36a553 1, 6700, 6700, 10, 2013, 0x2ec19877 -0, 201, 201, 1, 518400, 0x55db9716 +0, 201, 201, 1, 518400, 0x65429052 1, 6710, 6710, 10, 2013, 0xc18b924c 1, 6726, 6726, 10, 2013, 0xf132b04c 1, 6736, 6736, 10, 2013, 0x7975a44d -0, 202, 202, 1, 518400, 0x1f0d40d6 +0, 202, 202, 1, 518400, 0xec803a15 1, 6746, 6746, 10, 2013, 0x2aaf94cb 1, 6756, 6756, 10, 2013, 0x58cfa60f 1, 6766, 6766, 10, 2013, 0x9757a658 -0, 203, 203, 1, 518400, 0x73695c82 +0, 203, 203, 1, 518400, 0x7a9a55c9 1, 6776, 6776, 10, 2013, 0x67ebc0d5 1, 6786, 6786, 10, 2013, 0x3c50a70e 1, 6796, 6796, 10, 2013, 0x9c5799c6 -0, 204, 204, 1, 518400, 0xb0f10812 +0, 204, 204, 1, 518400, 0xcac30160 1, 6811, 6811, 10, 2013, 0x018d85b2 1, 6821, 6821, 10, 2013, 0x5367a956 -0, 205, 205, 1, 518400, 0xdec18505 -0, 208, 208, 1, 518400, 0xb147b947 -0, 240, 240, 1, 518400, 0x9d2e3977 +0, 205, 205, 1, 518400, 0x7e187e4f +0, 208, 208, 1, 518400, 0x0be0b2a2 diff --git a/tests/ref/fate/sub-dvb b/tests/ref/fate/sub-dvb index cbd1801d64..8f33c75d70 100644 --- a/tests/ref/fate/sub-dvb +++ b/tests/ref/fate/sub-dvb @@ -1,75 +1,93 @@ #tb 0: 1/1000000 #media_type 0: subtitle #codec_id 0: dvb_subtitle -0, 15600000, 15600000, 159000, 1168, 0xd0f89d82 -0, 15759000, 15759000, 159000, 14, 0x064900eb -0, 15760000, 15760000, 239000, 1544, 0xe60f1751 -0, 15999000, 15999000, 239000, 14, 0x0729010b -0, 16000000, 16000000, 339000, 1658, 0xbe343093 -0, 16339000, 16339000, 339000, 14, 0x0809012b -0, 16340000, 16340000, 599000, 2343, 0xc68f07ef -0, 16939000, 16939000, 599000, 14, 0x08e9014b -0, 16940000, 16940000, 459000, 2568, 0x0ee657b1 -0, 17399000, 17399000, 459000, 14, 0x09c9016b -0, 17400000, 17400000, 359000, 3422, 0xba5b63ce -0, 17759000, 17759000, 359000, 14, 0x0aa9018b -0, 17760000, 17760000, 219000, 5078, 0x95b19902 -0, 17979000, 17979000, 219000, 14, 0x0b8901ab -0, 17980000, 17980000, 959000, 5808, 0xc9717b89 -0, 18939000, 18939000, 959000, 14, 0x0c6901cb -0, 18940000, 18940000, 219000, 6015, 0x0becbfac -0, 19159000, 19159000, 219000, 14, 0x064900eb -0, 19160000, 19160000, 259000, 6519, 0xfcd24d26 -0, 19419000, 19419000, 259000, 14, 0x0729010b -0, 19420000, 19420000, 99000, 7061, 0xf0320408 -0, 19519000, 19519000, 99000, 14, 0x0809012b -0, 19520000, 19520000, 219000, 4773, 0x66c93074 -0, 19739000, 19739000, 219000, 14, 0x08e9014b -0, 19740000, 19740000, 219000, 5546, 0x06052c81 -0, 19959000, 19959000, 219000, 14, 0x09c9016b -0, 19960000, 19960000, 239000, 5754, 0x904f7325 -0, 20199000, 20199000, 239000, 14, 0x0aa9018b -0, 20200000, 20200000, 139000, 6099, 0xe30cde07 -0, 20339000, 20339000, 139000, 14, 0x0b8901ab -0, 20340000, 20340000, 799000, 6839, 0x770fcb6c -0, 21139000, 21139000, 799000, 14, 0x0c6901cb -0, 21140000, 21140000, 239000, 4744, 0xa91e1b41 -0, 21379000, 21379000, 239000, 14, 0x064900eb -0, 21380000, 21380000, 339000, 5824, 0xcf6d782b -0, 21719000, 21719000, 339000, 14, 0x0729010b -0, 21720000, 21720000, 1439000, 6212, 0xabf8f7cf -0, 23159000, 23159000, 1439000, 14, 0x0809012b -0, 23160000, 23160000, 1319000, 7082, 0xd7ca10f2 -0, 24479000, 24479000, 1319000, 14, 0x08e9014b -0, 24480000, 24480000, 219000, 5345, 0x12b2cae0 -0, 24699000, 24699000, 219000, 14, 0x09c9016b -0, 24700000, 24700000, 219000, 5765, 0xc7d46192 -0, 24919000, 24919000, 219000, 14, 0x0aa9018b -0, 24920000, 24920000, 599000, 6557, 0xcb995d30 -0, 25519000, 25519000, 599000, 14, 0x0b8901ab -0, 25520000, 25520000, 219000, 7091, 0xe6ea0559 -0, 25739000, 25739000, 219000, 14, 0x0c6901cb -0, 25740000, 25740000, 239000, 7305, 0xb66c404e -0, 25979000, 25979000, 239000, 14, 0x064900eb -0, 25980000, 25980000, 359000, 7590, 0x0cc2a481 -0, 26339000, 26339000, 359000, 14, 0x0729010b -0, 26340000, 26340000, 219000, 4629, 0xe18cfea8 -0, 26559000, 26559000, 219000, 14, 0x0809012b -0, 26560000, 26560000, 719000, 4785, 0x82043fc0 -0, 27279000, 27279000, 719000, 14, 0x08e9014b -0, 27280000, 27280000, 459000, 6061, 0xbde7d245 -0, 27739000, 27739000, 459000, 14, 0x09c9016b -0, 27740000, 27740000, 239000, 6301, 0x92d01a51 -0, 27979000, 27979000, 239000, 14, 0x0aa9018b -0, 27980000, 27980000, 99000, 6736, 0xbd25a134 -0, 28079000, 28079000, 99000, 14, 0x0b8901ab -0, 28080000, 28080000, 219000, 7214, 0x7ef93c13 -0, 28299000, 28299000, 219000, 14, 0x0c6901cb -0, 28300000, 28300000, 239000, 7366, 0x5bed7fcd -0, 28539000, 28539000, 239000, 14, 0x064900eb -0, 28540000, 28540000, 599000, 4564, 0x7f4c014b -0, 29139000, 29139000, 599000, 14, 0x0729010b -0, 29140000, 29140000, 219000, 4637, 0x682626b7 -0, 29359000, 29359000, 219000, 14, 0x0809012b -0, 29360000, 29360000, 1679000, 5358, 0x29e30c48 -0, 31039000, 31039000, 1679000, 14, 0x08e9014b +0, 0, 0, 279000, 14, 0x05d900db +0, 279000, 279000, 279000, 14, 0x064900eb +0, 280000, 280000, 4999000, 14, 0x06b900fb +0, 5279000, 5279000, 4999000, 14, 0x0729010b +0, 5280000, 5280000, 5019000, 14, 0x0799011b +0, 10299000, 10299000, 5019000, 14, 0x0809012b +0, 10300000, 10300000, 3599000, 14, 0x0879013b +0, 13899000, 13899000, 3599000, 14, 0x08e9014b +0, 13900000, 13900000, 219000, 14, 0x0959015b +0, 14119000, 14119000, 219000, 14, 0x09c9016b +0, 14120000, 14120000, 1439000, 14, 0x0a39017b +0, 15559000, 15559000, 1439000, 14, 0x0aa9018b +0, 15560000, 15560000, 39000, 14, 0x0b19019b +0, 15599000, 15599000, 39000, 14, 0x0b8901ab +0, 15600000, 15600000, 159000, 1168, 0xd69da022 +0, 15759000, 15759000, 159000, 14, 0x0c6901cb +0, 15760000, 15760000, 239000, 1544, 0xc5f116f1 +0, 15999000, 15999000, 239000, 14, 0x064900eb +0, 16000000, 16000000, 339000, 1658, 0x73563033 +0, 16339000, 16339000, 339000, 14, 0x0729010b +0, 16340000, 16340000, 599000, 2343, 0x7ac2078f +0, 16939000, 16939000, 599000, 14, 0x0809012b +0, 16940000, 16940000, 459000, 2568, 0x6eaa5751 +0, 17399000, 17399000, 459000, 14, 0x08e9014b +0, 17400000, 17400000, 359000, 3422, 0xd9d0636e +0, 17759000, 17759000, 359000, 14, 0x09c9016b +0, 17760000, 17760000, 219000, 5078, 0x722c9862 +0, 17979000, 17979000, 219000, 14, 0x0aa9018b +0, 17980000, 17980000, 959000, 5808, 0x38dd7ae9 +0, 18939000, 18939000, 959000, 14, 0x0b8901ab +0, 18940000, 18940000, 219000, 6015, 0xd4d2c40c +0, 19159000, 19159000, 219000, 14, 0x0c6901cb +0, 19160000, 19160000, 259000, 6519, 0x08af4c86 +0, 19419000, 19419000, 259000, 14, 0x064900eb +0, 19420000, 19420000, 99000, 7061, 0xecf10368 +0, 19519000, 19519000, 99000, 14, 0x0729010b +0, 19520000, 19520000, 219000, 4773, 0xbee42fd4 +0, 19739000, 19739000, 219000, 14, 0x0809012b +0, 19740000, 19740000, 219000, 5546, 0xdb822be1 +0, 19959000, 19959000, 219000, 14, 0x08e9014b +0, 19960000, 19960000, 239000, 5754, 0xfdcc7285 +0, 20199000, 20199000, 239000, 14, 0x09c9016b +0, 20200000, 20200000, 139000, 6099, 0xa409dd67 +0, 20339000, 20339000, 139000, 14, 0x0aa9018b +0, 20340000, 20340000, 799000, 6839, 0xc5eecacc +0, 21139000, 21139000, 799000, 14, 0x0b8901ab +0, 21140000, 21140000, 239000, 4744, 0x4e451fa1 +0, 21379000, 21379000, 239000, 14, 0x0c6901cb +0, 21380000, 21380000, 339000, 5824, 0x5299778b +0, 21719000, 21719000, 339000, 14, 0x064900eb +0, 21720000, 21720000, 1439000, 6212, 0x6d15f72f +0, 23159000, 23159000, 1439000, 14, 0x0729010b +0, 23160000, 23160000, 1319000, 7082, 0xe5c91052 +0, 24479000, 24479000, 1319000, 14, 0x0809012b +0, 24480000, 24480000, 219000, 5345, 0x2e5eca40 +0, 24699000, 24699000, 219000, 14, 0x08e9014b +0, 24700000, 24700000, 219000, 5765, 0x118060f2 +0, 24919000, 24919000, 219000, 14, 0x09c9016b +0, 24920000, 24920000, 599000, 6557, 0x89275c90 +0, 25519000, 25519000, 599000, 14, 0x0aa9018b +0, 25520000, 25520000, 219000, 7091, 0x996904b9 +0, 25739000, 25739000, 219000, 14, 0x0b8901ab +0, 25740000, 25740000, 239000, 7305, 0xc23e44ae +0, 25979000, 25979000, 239000, 14, 0x0c6901cb +0, 25980000, 25980000, 359000, 7590, 0xc5a3a3e1 +0, 26339000, 26339000, 359000, 14, 0x064900eb +0, 26340000, 26340000, 219000, 4629, 0x7ad6fe08 +0, 26559000, 26559000, 219000, 14, 0x0729010b +0, 26560000, 26560000, 719000, 4785, 0xcd3f3f20 +0, 27279000, 27279000, 719000, 14, 0x0809012b +0, 27280000, 27280000, 459000, 6061, 0x8b04d1a5 +0, 27739000, 27739000, 459000, 14, 0x08e9014b +0, 27740000, 27740000, 239000, 6301, 0xe7de19b1 +0, 27979000, 27979000, 239000, 14, 0x09c9016b +0, 27980000, 27980000, 99000, 6736, 0x38b3a094 +0, 28079000, 28079000, 99000, 14, 0x0aa9018b +0, 28080000, 28080000, 219000, 7214, 0x0b783b73 +0, 28299000, 28299000, 219000, 14, 0x0b8901ab +0, 28300000, 28300000, 239000, 7366, 0x98bf842d +0, 28539000, 28539000, 239000, 14, 0x0c6901cb +0, 28540000, 28540000, 599000, 4564, 0x3d9600ab +0, 29139000, 29139000, 599000, 14, 0x064900eb +0, 29140000, 29140000, 219000, 4637, 0x01f02617 +0, 29359000, 29359000, 219000, 14, 0x0729010b +0, 29360000, 29360000, 1679000, 5358, 0x5b0f0ba8 +0, 31039000, 31039000, 1679000, 14, 0x0809012b +0, 31040000, 31040000, 359000, 14, 0x0879013b +0, 31399000, 31399000, 359000, 14, 0x08e9014b +0, 31400000, 31400000, 479000, 14, 0x0959015b +0, 31879000, 31879000, 479000, 14, 0x09c9016b diff --git a/tests/ref/fate/sub2video b/tests/ref/fate/sub2video index 80abe9c905..e8f7c86609 100644 --- a/tests/ref/fate/sub2video +++ b/tests/ref/fate/sub2video @@ -11,176 +11,134 @@ 0, 1, 1, 1, 518400, 0x4051c7f9 0, 2, 2, 1, 518400, 0xfb00e17e 1, 499000, 499000, 4960000, 1015, 0x19e092d2 -0, 3, 3, 1, 518400, 0x192abb74 -0, 4, 4, 1, 518400, 0x4669a88b -0, 5, 5, 1, 518400, 0xaababe00 -0, 6, 6, 1, 518400, 0x98a211a5 -0, 7, 7, 1, 518400, 0x440e7547 -0, 8, 8, 1, 518400, 0xca5bb496 -0, 9, 9, 1, 518400, 0xf86e0b0a -0, 10, 10, 1, 518400, 0xb80fa020 -0, 11, 11, 1, 518400, 0x41c2a54b -0, 12, 12, 1, 518400, 0x51baf353 -0, 13, 13, 1, 518400, 0x967ea7f3 -0, 14, 14, 1, 518400, 0x819e7f89 -0, 15, 15, 1, 518400, 0x192233e1 -0, 16, 16, 1, 518400, 0xc80a0eb3 -0, 17, 17, 1, 518400, 0x08260a23 -0, 18, 18, 1, 518400, 0xc92e2caf -0, 19, 19, 1, 518400, 0x3fe36eea -0, 20, 20, 1, 518400, 0x0891e8d5 -0, 21, 21, 1, 518400, 0x84655095 -0, 22, 22, 1, 518400, 0x9c7fa014 -0, 23, 23, 1, 518400, 0x9c43b656 -0, 24, 24, 1, 518400, 0x2cf46221 -0, 25, 25, 1, 518400, 0x7322e11c -0, 26, 26, 1, 518400, 0x45af1a84 -0, 27, 27, 1, 518400, 0x7b781071 -0, 28, 28, 1, 518400, 0x4f7c706c -0, 29, 29, 1, 518400, 0xb227603b -0, 30, 30, 1, 518400, 0x7b4b89c2 -0, 31, 31, 1, 518400, 0x456da21e -0, 32, 32, 1, 518400, 0xb691979f -0, 33, 33, 1, 518400, 0x0dfaa66d -0, 34, 34, 1, 518400, 0x191a6f23 -0, 35, 35, 1, 518400, 0xa03b2605 -0, 36, 36, 1, 518400, 0xb36aff87 -0, 37, 37, 1, 518400, 0xf5f0bc4a -0, 38, 38, 1, 518400, 0x863d701a -0, 39, 39, 1, 518400, 0xd11b4dce -0, 40, 40, 1, 518400, 0x969236bd -0, 41, 41, 1, 518400, 0xb60a485c -0, 42, 42, 1, 518400, 0xe9796621 -0, 43, 43, 1, 518400, 0x3e8fc04b -0, 44, 44, 1, 518400, 0xac9944e3 -0, 45, 45, 1, 518400, 0x01452b4d -0, 46, 46, 1, 518400, 0xb384f6d2 -0, 47, 47, 1, 518400, 0xde69683f -0, 48, 48, 1, 518400, 0x7df08fba -0, 49, 49, 1, 518400, 0xbab197ea +0, 3, 3, 1, 518400, 0x0f994baf +0, 4, 4, 1, 518400, 0x1be9369c +0, 5, 5, 1, 518400, 0x60de573c +0, 6, 6, 1, 518400, 0xc649b162 +0, 7, 7, 1, 518400, 0x05491b39 +0, 8, 8, 1, 518400, 0xdafc62cd +0, 9, 9, 1, 518400, 0xa582b72f +0, 10, 10, 1, 518400, 0xb208423a +0, 11, 11, 1, 518400, 0x686139e9 +0, 12, 12, 1, 518400, 0x9af78264 +0, 13, 13, 1, 518400, 0x7e6e203b +0, 14, 14, 1, 518400, 0x5193f953 +0, 15, 15, 1, 518400, 0xfe0eac88 +0, 16, 16, 1, 518400, 0xe20e7951 +0, 17, 17, 1, 518400, 0x8b226738 +0, 18, 18, 1, 518400, 0x7f776db2 +0, 19, 19, 1, 518400, 0xd49b8221 +0, 20, 20, 1, 518400, 0x264bbf4e +0, 21, 21, 1, 518400, 0xe95ff46e +0, 22, 22, 1, 518400, 0xbba12cbc +0, 23, 23, 1, 518400, 0xf3283572 +0, 24, 24, 1, 518400, 0xcacde1de +0, 25, 25, 1, 518400, 0x265d7266 +0, 26, 26, 1, 518400, 0xdab6bfdf +0, 27, 27, 1, 518400, 0xe7f7bb70 +0, 28, 28, 1, 518400, 0x503596b0 +0, 29, 29, 1, 518400, 0xef149508 +0, 30, 30, 1, 518400, 0x07ecc81b +0, 31, 31, 1, 518400, 0x5a73e9f6 +0, 32, 32, 1, 518400, 0xaff2e80f +0, 33, 33, 1, 518400, 0x68d20234 +0, 34, 34, 1, 518400, 0x20d8ce9b +0, 35, 35, 1, 518400, 0x8f7283f0 +0, 36, 36, 1, 518400, 0x279c5d9b +0, 37, 37, 1, 518400, 0xc19f1e78 +0, 38, 38, 1, 518400, 0x2349cee4 +0, 39, 39, 1, 518400, 0xcefda93f +0, 40, 40, 1, 518400, 0x8dd39341 +0, 41, 41, 1, 518400, 0x9b7ba700 +0, 42, 42, 1, 518400, 0x52bdc68a +0, 43, 43, 1, 518400, 0xe4de20e6 +0, 44, 44, 1, 518400, 0xa4b8a24d +0, 45, 45, 1, 518400, 0x5ad38366 +0, 46, 46, 1, 518400, 0xbfa14e65 +0, 47, 47, 1, 518400, 0x1a39c161 +0, 48, 48, 1, 518400, 0x18b1e440 +0, 49, 49, 1, 518400, 0xdf1fe495 1, 15355000, 15355000, 4733000, 2094, 0x3c171425 -0, 77, 77, 1, 518400, 0x902285d9 -0, 100, 100, 1, 518400, 0xbab197ea +0, 77, 77, 1, 518400, 0x34f1cb1b 1, 48797000, 48797000, 2560000, 2480, 0x7c0edf21 -0, 244, 244, 1, 518400, 0x7a11c812 -0, 257, 257, 1, 518400, 0xbab197ea +0, 244, 244, 1, 518400, 0x78d21417 +0, 257, 257, 1, 518400, 0x2a866db2 1, 51433000, 51433000, 2366000, 3059, 0xc95b8a05 -0, 258, 258, 1, 518400, 0x34cdddee -0, 269, 269, 1, 518400, 0xbab197ea 1, 53910000, 53910000, 2696000, 2095, 0x61bb15ed -0, 270, 270, 1, 518400, 0x4db4ce51 -0, 283, 283, 1, 518400, 0xbab197ea +0, 270, 270, 1, 518400, 0x75a34256 +0, 283, 283, 1, 518400, 0x6b37d703 1, 56663000, 56663000, 1262000, 1013, 0xc9ae89b7 -0, 284, 284, 1, 518400, 0xe6bc0ea9 -0, 290, 290, 1, 518400, 0xbab197ea +0, 290, 290, 1, 518400, 0x1bf2dacc 1, 58014000, 58014000, 1661000, 969, 0xe01878f0 -0, 291, 291, 1, 518400, 0xa8643af7 -0, 298, 298, 1, 518400, 0xbab197ea 1, 67724000, 67724000, 1365000, 844, 0xe7db4fc1 -0, 339, 339, 1, 518400, 0xb1885c67 -0, 345, 345, 1, 518400, 0xbab197ea +0, 339, 339, 1, 518400, 0x9242d323 1, 69175000, 69175000, 1558000, 802, 0xf48531ba -0, 346, 346, 1, 518400, 0x378e3fd0 -0, 354, 354, 1, 518400, 0xbab197ea +0, 346, 346, 1, 518400, 0x3b33c640 +0, 354, 354, 1, 518400, 0xcf341c1c 1, 70819000, 70819000, 1865000, 1709, 0xb4d5a1bd -0, 355, 355, 1, 518400, 0xa3782469 -0, 363, 363, 1, 518400, 0xbab197ea 1, 72762000, 72762000, 1968000, 2438, 0x99d7bc82 -0, 364, 364, 1, 518400, 0xba23a0d5 -0, 374, 374, 1, 518400, 0xbab197ea +0, 364, 364, 1, 518400, 0xe0141f2a +0, 374, 374, 1, 518400, 0x08602ed1 1, 74806000, 74806000, 1831000, 2116, 0x96514097 -0, 375, 375, 1, 518400, 0x129de2f8 -0, 383, 383, 1, 518400, 0xbab197ea 1, 76716000, 76716000, 1262000, 1822, 0xefccc72e -0, 384, 384, 1, 518400, 0x19772f0f -0, 390, 390, 1, 518400, 0xbab197ea +0, 384, 384, 1, 518400, 0x3fc415de +0, 390, 390, 1, 518400, 0xdd9ad4a0 1, 78051000, 78051000, 1524000, 987, 0x7b927a27 -0, 391, 391, 1, 518400, 0x56f54e73 -0, 398, 398, 1, 518400, 0xbab197ea +0, 398, 398, 1, 518400, 0x65897ecf 1, 79644000, 79644000, 2662000, 2956, 0x190778f7 -0, 399, 399, 1, 518400, 0x300b5247 1, 82380000, 82380000, 2764000, 3094, 0xc021b7d3 -0, 412, 412, 1, 518400, 0xbab197ea -0, 413, 413, 1, 518400, 0x6fd028fa -0, 426, 426, 1, 518400, 0xbab197ea +0, 412, 412, 1, 518400, 0x28eea6ae +0, 426, 426, 1, 518400, 0x7b93f4db 1, 85225000, 85225000, 2366000, 2585, 0x74d0048f -0, 427, 427, 1, 518400, 0x01f80e9d -0, 438, 438, 1, 518400, 0xbab197ea +0, 438, 438, 1, 518400, 0x2df4beb4 1, 87652000, 87652000, 1831000, 634, 0x8832fda1 -0, 439, 439, 1, 518400, 0xb48d90c0 -0, 447, 447, 1, 518400, 0xbab197ea 1, 91531000, 91531000, 2332000, 2080, 0x97a1146f -0, 458, 458, 1, 518400, 0xcb5a0173 -0, 469, 469, 1, 518400, 0xbab197ea +0, 458, 458, 1, 518400, 0xc314313d 1, 95510000, 95510000, 3299000, 2964, 0x8b8f6684 -0, 478, 478, 1, 518400, 0xb8a323e4 -0, 494, 494, 1, 518400, 0xbab197ea +0, 478, 478, 1, 518400, 0xb54e8bde +0, 494, 494, 1, 518400, 0xf3e21246 1, 98872000, 98872000, 2161000, 1875, 0x9002ef71 -0, 495, 495, 1, 518400, 0xc43518ba -0, 505, 505, 1, 518400, 0xbab197ea 1, 101124000, 101124000, 4096000, 3872, 0x20c6ed9c -0, 506, 506, 1, 518400, 0x04e38692 -0, 526, 526, 1, 518400, 0xbab197ea +0, 506, 506, 1, 518400, 0xc0b3b0a8 1, 105303000, 105303000, 2730000, 3094, 0xf203a663 -0, 527, 527, 1, 518400, 0x856b0ee5 -0, 540, 540, 1, 518400, 0xbab197ea +0, 527, 527, 1, 518400, 0xb7cdb42c 1, 108106000, 108106000, 2059000, 2404, 0x41a7b429 -0, 541, 541, 1, 518400, 0x3e5beee2 -0, 551, 551, 1, 518400, 0xbab197ea +0, 541, 541, 1, 518400, 0x55e80840 1, 141556000, 141556000, 1661000, 1088, 0xde20aa20 -0, 708, 708, 1, 518400, 0xb8bc1365 -0, 716, 716, 1, 518400, 0xbab197ea -0, 817, 817, 1, 518400, 0x83efa32d +0, 708, 708, 1, 518400, 0xd0b2dea5 +0, 817, 817, 1, 518400, 0xf95dab14 1, 163445000, 163445000, 1331000, 339, 0x8bd186ef -0, 824, 824, 1, 518400, 0xbab197ea -0, 840, 840, 1, 518400, 0x03ea0e90 +0, 840, 840, 1, 518400, 0xdad8e97c 1, 168049000, 168049000, 1900000, 1312, 0x0bf20e8d -0, 850, 850, 1, 518400, 0xbab197ea +0, 850, 850, 1, 518400, 0x6b97e924 1, 170035000, 170035000, 1524000, 1279, 0xb6c2dafe -0, 851, 851, 1, 518400, 0x8780239e -0, 858, 858, 1, 518400, 0xbab197ea -0, 861, 861, 1, 518400, 0x6eb72347 +0, 861, 861, 1, 518400, 0x983e2533 1, 172203000, 172203000, 1695000, 1826, 0x9a1ac769 -0, 869, 869, 1, 518400, 0xbab197ea 1, 173947000, 173947000, 1934000, 1474, 0xa9b03cdc -0, 870, 870, 1, 518400, 0x9c4a3a3d -0, 879, 879, 1, 518400, 0xbab197ea +0, 870, 870, 1, 518400, 0x855805b7 1, 175957000, 175957000, 1763000, 1019, 0x20409355 -0, 880, 880, 1, 518400, 0xc9ebfa89 -0, 889, 889, 1, 518400, 0xbab197ea -0, 946, 946, 1, 518400, 0xbaf801ef +0, 880, 880, 1, 518400, 0x66c5d637 +0, 946, 946, 1, 518400, 0x2e7f0368 1, 189295000, 189295000, 1968000, 1596, 0x408c726e -0, 956, 956, 1, 518400, 0xbab197ea 1, 191356000, 191356000, 1228000, 1517, 0xae8c5c2b -0, 957, 957, 1, 518400, 0x59f4e72f -0, 963, 963, 1, 518400, 0xbab197ea +0, 957, 957, 1, 518400, 0xa386ffc3 +0, 963, 963, 1, 518400, 0xb4f3023a 1, 192640000, 192640000, 1763000, 2506, 0xa458d6d4 -0, 964, 964, 1, 518400, 0x9d5b9d69 -0, 972, 972, 1, 518400, 0xbab197ea 1, 195193000, 195193000, 1092000, 1074, 0x397ba9a8 -0, 976, 976, 1, 518400, 0x923d1ce7 -0, 981, 981, 1, 518400, 0xbab197ea +0, 976, 976, 1, 518400, 0x4830e301 1, 196361000, 196361000, 1524000, 1715, 0x695ca41e -0, 982, 982, 1, 518400, 0x6e652cd2 -0, 989, 989, 1, 518400, 0xbab197ea +0, 982, 982, 1, 518400, 0x97962409 1, 197946000, 197946000, 1160000, 789, 0xc63a189e -0, 990, 990, 1, 518400, 0x25113966 -0, 996, 996, 1, 518400, 0xbab197ea +0, 990, 990, 1, 518400, 0xcaf5d357 +0, 996, 996, 1, 518400, 0xda492293 1, 199230000, 199230000, 1627000, 1846, 0xeea8c599 -0, 997, 997, 1, 518400, 0x2dc83609 -0, 1004, 1004, 1, 518400, 0xbab197ea 1, 200924000, 200924000, 1763000, 922, 0xd4a87222 -0, 1005, 1005, 1, 518400, 0x90483bc6 -0, 1013, 1013, 1, 518400, 0xbab197ea -0, 1053, 1053, 1, 518400, 0x3de86ab7 +0, 1005, 1005, 1, 518400, 0x64a9d75d +0, 1053, 1053, 1, 518400, 0x8cb7be11 1, 210600000, 210600000, 1831000, 665, 0x55580135 -0, 1062, 1062, 1, 518400, 0xbab197ea 1, 214771000, 214771000, 1558000, 1216, 0x50d1f6c5 -0, 1074, 1074, 1, 518400, 0x8c320e68 -0, 1082, 1082, 1, 518400, 0xbab197ea -0, 1128, 1128, 1, 518400, 0x81e977b2 +0, 1074, 1074, 1, 518400, 0x1f77f0e3 +0, 1128, 1128, 1, 518400, 0x5c96485d 1, 225640000, 225640000, 2127000, 2133, 0x670c11a5 -0, 1139, 1139, 1, 518400, 0xbab197ea +0, 1139, 1139, 1, 518400, 0x51f5f723 1, 227834000, 227834000, 1262000, 1264, 0xc1d9fc57 -0, 1140, 1140, 1, 518400, 0xb046dd30 -0, 1145, 1145, 1, 518400, 0xbab197ea diff --git a/tests/ref/fate/sub2video_basic b/tests/ref/fate/sub2video_basic index 5f72e292c9..41e02c057c 100644 --- a/tests/ref/fate/sub2video_basic +++ b/tests/ref/fate/sub2video_basic @@ -2,94 +2,47 @@ #media_type 0: video #codec_id 0: rawvideo #dimensions 0: 720x480 -#sar 0: 0/1 -0, 3312, 3312, 1, 1382400, 0x00000000 -0, 3312, 3312, 1, 1382400, 0x8c93c2ba -0, 3436, 3436, 1, 1382400, 0x00000000 -0, 3684, 3684, 1, 1382400, 0xb02e32ca -0, 3802, 3802, 1, 1382400, 0x00000000 -0, 4520, 4520, 1, 1382400, 0x83b71116 -0, 4584, 4584, 1, 1382400, 0x00000000 -0, 4586, 4586, 1, 1382400, 0x85547fd1 -0, 4645, 4645, 1, 1382400, 0x00000000 -0, 4648, 4648, 1, 1382400, 0x00000000 -0, 4648, 4648, 1, 1382400, 0xb6a8f181 -0, 4715, 4715, 1, 1382400, 0x00000000 -0, 4717, 4717, 1, 1382400, 0xb64d1a2c -0, 4748, 4748, 1, 1382400, 0x00000000 -0, 4750, 4750, 1, 1382400, 0x7b37ecf3 -0, 4792, 4792, 1, 1382400, 0x00000000 -0, 4993, 4993, 1, 1382400, 0xdc025bd1 -0, 5027, 5027, 1, 1382400, 0x00000000 -0, 5029, 5029, 1, 1382400, 0x688b294d -0, 5068, 5068, 1, 1382400, 0x00000000 -0, 5070, 5070, 1, 1382400, 0xa2b33d1b -0, 5117, 5117, 1, 1382400, 0x00000000 -0, 5119, 5119, 1, 1382400, 0xb3e525e3 -0, 5168, 5168, 1, 1382400, 0x00000000 -0, 5170, 5170, 1, 1382400, 0xaa8fbdd7 -0, 5216, 5216, 1, 1382400, 0x00000000 -0, 5218, 5218, 1, 1382400, 0x7b7f26dd -0, 5249, 5249, 1, 1382400, 0x00000000 -0, 5251, 5251, 1, 1382400, 0x15e2f836 -0, 5289, 5289, 1, 1382400, 0x00000000 -0, 5291, 5291, 1, 1382400, 0x0fee9b0c -0, 5358, 5358, 1, 1382400, 0x00000000 -0, 5360, 5360, 1, 1382400, 0x89d62791 -0, 5429, 5429, 1, 1382400, 0x00000000 -0, 5431, 5431, 1, 1382400, 0xa6a9fd74 -0, 5490, 5490, 1, 1382400, 0x00000000 -0, 5491, 5491, 1, 1382400, 0x7896178d -0, 5537, 5537, 1, 1382400, 0x00000000 -0, 5588, 5588, 1, 1382400, 0x01751a52 -0, 5647, 5647, 1, 1382400, 0x00000000 -0, 5688, 5688, 1, 1382400, 0xa3959c6f -0, 5770, 5770, 1, 1382400, 0x00000000 -0, 5772, 5772, 1, 1382400, 0x3d3ea47b -0, 5826, 5826, 1, 1382400, 0x00000000 -0, 5828, 5828, 1, 1382400, 0x593f8b24 -0, 5931, 5931, 1, 1382400, 0x00000000 -0, 5933, 5933, 1, 1382400, 0x171f05ba -0, 6001, 6001, 1, 1382400, 0x00000000 -0, 6003, 6003, 1, 1382400, 0xb014cdf1 -0, 6054, 6054, 1, 1382400, 0x00000000 -0, 6839, 6839, 1, 1382400, 0xd918e667 -0, 6880, 6880, 1, 1382400, 0x00000000 -0, 7386, 7386, 1, 1382400, 0xc9406331 -0, 7419, 7419, 1, 1382400, 0x00000000 -0, 7501, 7501, 1, 1382400, 0xaf08b10d -0, 7549, 7549, 1, 1382400, 0x00000000 -0, 7551, 7551, 1, 1382400, 0x00000000 -0, 7551, 7551, 1, 1382400, 0x853a9d93 -0, 7589, 7589, 1, 1382400, 0x00000000 -0, 7605, 7605, 1, 1382400, 0x7491a87d -0, 7647, 7647, 1, 1382400, 0x00000000 -0, 7649, 7649, 1, 1382400, 0xf7383c58 -0, 7697, 7697, 1, 1382400, 0x00000000 -0, 7699, 7699, 1, 1382400, 0xe66be411 -0, 7743, 7743, 1, 1382400, 0x00000000 -0, 8032, 8032, 1, 1382400, 0xd6850362 -0, 8082, 8082, 1, 1382400, 0x00000000 -0, 8084, 8084, 1, 1382400, 0x3e1ed109 -0, 8115, 8115, 1, 1382400, 0x00000000 -0, 8116, 8116, 1, 1382400, 0x39c1b7bd -0, 8160, 8160, 1, 1382400, 0x00000000 -0, 8180, 8180, 1, 1382400, 0x35b85f2e -0, 8207, 8207, 1, 1382400, 0x00000000 -0, 8209, 8209, 1, 1382400, 0x00000000 -0, 8209, 8209, 1, 1382400, 0x83f103e5 -0, 8247, 8247, 1, 1382400, 0x00000000 -0, 8249, 8249, 1, 1382400, 0xbc1ca9b3 -0, 8278, 8278, 1, 1382400, 0x00000000 -0, 8281, 8281, 1, 1382400, 0x94d4a51e -0, 8321, 8321, 1, 1382400, 0x00000000 -0, 8323, 8323, 1, 1382400, 0xf88cdfde -0, 8367, 8367, 1, 1382400, 0x00000000 -0, 8565, 8565, 1, 1382400, 0xdd51423b -0, 8611, 8611, 1, 1382400, 0x00000000 -0, 8669, 8669, 1, 1382400, 0x08259fa4 -0, 8708, 8708, 1, 1382400, 0x00000000 -0, 8941, 8941, 1, 1382400, 0x1663fa34 -0, 8994, 8994, 1, 1382400, 0x00000000 -0, 8996, 8996, 1, 1382400, 0xda2ceb55 -0, 9027, 9027, 1, 1382400, 0x00000000 +#sar 0: 1/1 +0, 3312, 3312, 1, 1382400, 0xc637b893 +0, 3684, 3684, 1, 1382400, 0x4c2960ca +0, 4520, 4520, 1, 1382400, 0x5fa18966 +0, 4586, 4586, 1, 1382400, 0x55f4b7b1 +0, 4648, 4648, 1, 1382400, 0xdfa4cf32 +0, 4717, 4717, 1, 1382400, 0x35023df8 +0, 4750, 4750, 1, 1382400, 0xed933219 +0, 4993, 4993, 1, 1382400, 0x1b26389a +0, 5029, 5029, 1, 1382400, 0xf0c7028b +0, 5070, 5070, 1, 1382400, 0x395f521d +0, 5119, 5119, 1, 1382400, 0x1ea87415 +0, 5170, 5170, 1, 1382400, 0xc6effdc1 +0, 5218, 5218, 1, 1382400, 0xba6846f8 +0, 5251, 5251, 1, 1382400, 0x033c5d5b +0, 5291, 5291, 1, 1382400, 0xef5abf66 +0, 5360, 5360, 1, 1382400, 0xec747954 +0, 5431, 5431, 1, 1382400, 0xfa34bcaf +0, 5491, 5491, 1, 1382400, 0x8b7a709b +0, 5588, 5588, 1, 1382400, 0xc333382f +0, 5688, 5688, 1, 1382400, 0xabe5dfcf +0, 5772, 5772, 1, 1382400, 0x56948101 +0, 5828, 5828, 1, 1382400, 0xb747834a +0, 5933, 5933, 1, 1382400, 0x3448baad +0, 6003, 6003, 1, 1382400, 0xaabe4f37 +0, 6839, 6839, 1, 1382400, 0x8a48cd6f +0, 7386, 7386, 1, 1382400, 0x49518c43 +0, 7501, 7501, 1, 1382400, 0x4a72fa21 +0, 7551, 7551, 1, 1382400, 0xa82f7de8 +0, 7605, 7605, 1, 1382400, 0xeba0b5f3 +0, 7649, 7649, 1, 1382400, 0xd6a91770 +0, 7699, 7699, 1, 1382400, 0x222f827c +0, 8032, 8032, 1, 1382400, 0x3270f4ff +0, 8084, 8084, 1, 1382400, 0x40813cb3 +0, 8116, 8116, 1, 1382400, 0x9d8fde41 +0, 8180, 8180, 1, 1382400, 0xc6d7a701 +0, 8209, 8209, 1, 1382400, 0x9d45f2dc +0, 8249, 8249, 1, 1382400, 0x8525ee40 +0, 8281, 8281, 1, 1382400, 0x5b26b98b +0, 8323, 8323, 1, 1382400, 0x51be311f +0, 8565, 8565, 1, 1382400, 0x00a4f2a3 +0, 8669, 8669, 1, 1382400, 0x40a445e8 +0, 8941, 8941, 1, 1382400, 0x43ef5128 +0, 8996, 8996, 1, 1382400, 0x3c3e3819 diff --git a/tests/ref/fate/sub2video_time_limited b/tests/ref/fate/sub2video_time_limited index 9fb6fb06f9..715af02fee 100644 --- a/tests/ref/fate/sub2video_time_limited +++ b/tests/ref/fate/sub2video_time_limited @@ -2,7 +2,5 @@ #media_type 0: video #codec_id 0: rawvideo #dimensions 0: 1920x1080 -#sar 0: 0/1 -0, 2, 2, 1, 8294400, 0x00000000 +#sar 0: 1/1 0, 2, 2, 1, 8294400, 0xa87c518f -0, 10, 10, 1, 8294400, 0xa87c518f From patchwork Mon Dec 13 23:40:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 32465 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp6140946iog; Mon, 13 Dec 2021 15:43:01 -0800 (PST) X-Google-Smtp-Source: ABdhPJz3dErbC63s3SmsPR0zSmszUnegLHrGhdlD7Nn6fN0nb84Js5rviQMsTL192qloe8BJNygW X-Received: by 2002:a17:906:5d0b:: with SMTP id g11mr1724008ejt.295.1639438981370; Mon, 13 Dec 2021 15:43:01 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id v14si16899946ejb.775.2021.12.13.15.43.00; Mon, 13 Dec 2021 15:43:01 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=WkmacAs9; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7270068B0DC; Tue, 14 Dec 2021 01:40:51 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12olkn2017.outbound.protection.outlook.com [40.92.21.17]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id DAAF368B0CC for ; Tue, 14 Dec 2021 01:40:49 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=g6qBhH9a6WidfGqPEu2AMxYPWzDrBD3TO01NNBR0p6swe7zKR6A0VY6SnUdGwylj1+XMnscRp2RNrDB2HIMk7gTY6cVYLxd1G57nS869rfZpDDXYFr5dLX+VlzSwYBbxfOGxXqlEbHZFaUTqMOjICnzIpi9gaSHB0pQ6VmwaZhPtlEU3c0iWA0Fbuq7PnO64mkxAou2CDwwEU8axYg7yuQ71puoNurO7KPepH4R079u6rUSPqwXVGHS7uQc1sxCv3/2tLkOzR1xuNKz7pK1MQ9uw/lLI66YpppSfWLWhyUGkIYLapyKhAD8+NkeHGRfve6eAzFQsZdRNn8cSOaHMfA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=PEr5v4/HVAs6sNUIpVFS6ON8WAePVTYCgWJ0N59LIZQ=; b=fEeJ9n/vni4rwgGreqCYF/i3BxHnbEvN9Io9sCZwX5t6ZQd9YHQWZy3JtIlXMHQx3NJBq79YQHiH6W/v4jBqcXKCvXKoeIV3tCoInDhxtHoDstNaEy/SPh2l7du7MwJiAuU8HTH8FfSv8Lbb3LtjkkGNa08P/Ekd4lKwPPyMSB6U4iK0hWL2lZVncApPjvsiiwdqxEPslGcUXCODk9WljJ6y5N1ljRV2EvWVms240oQHz0F439SmzEPdopA8FAUl99YMVf0gNL/OSwE7J9zOVDNQqXSMeLnGt++PzuwnvrqQDlGJvf3fKcKwMTnAVM6DEo23TssKQwWKgYbskqiGPw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PEr5v4/HVAs6sNUIpVFS6ON8WAePVTYCgWJ0N59LIZQ=; b=WkmacAs9+MIoSiFSDERIaW8nJ71u/PaaUIZuZz+nR6IuF4LmnjbG+zZ0cJ0+81I7Lj2vE/hPaNe7cSOOmA7o7QQqAnoBDDW5IDen5eBnWIGd+UBVyKMyIXuTLkEKsNLyWzdltDTe5k4wCGD3xhAAlEEeqP8htq5guvpSAUdO2A+JxXfSl49VC1+tVTELsGMCHLqndqKO81+0mMIQWbAY5KCGTdMYaSDz2/S1FjBjHIl2y9PsyJKaRJcZYCupvjDq4fdfXeqz6ihHKOZwFZTDoX9CX2nrWFHixFn/UEaOLDo7uKM7bhheFfzrOfKG4HW9wMhZzSCm0wa6HJqnd9Joiw== Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::20) by DM8P223MB0397.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4778.17; Mon, 13 Dec 2021 23:40:47 +0000 Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775]) by DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775%7]) with mapi id 15.20.4778.018; Mon, 13 Dec 2021 23:40:47 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v24 14/21] avfilter/avfilter: Fix hardcoded input index Thread-Index: AQHX8HrahOc7nZaVYkOfuwQA25ORvg== Date: Mon, 13 Dec 2021 23:40:47 +0000 Message-ID: References: <1ef42eb01b3627bbad539abd46206f320bb39697.1639438462.git.softworkz@hotmail.com> <4148d77a82d4b7e229fc81ce9bc1bc06904533f1.1639438462.git.softworkz@hotmail.com> <2ac798924ee984010d2e85f00bc992d9aa106171.1639438462.git.softworkz@hotmail.com> <300ea7188b331ca5e00dd7e04c03df8aba2a99c0.1639438462.git.softworkz@hotmail.com> <9474726b26f0d3e8f353d07caed2fbd33cdf473a.1639438462.git.softworkz@hotmail.com> <424522c96b5cdf6b03faa6bdceb1d25449463429.1639438462.git.softworkz@hotmail.com> <90caff245055005e6a4a890b3665da55aa853b31.1639438462.git.softworkz@hotmail.com> <0ebcf3ed2adb02d3ffb243a499b434c3bdd976a7.1639438462.git.softworkz@hotmail.com> In-Reply-To: <0ebcf3ed2adb02d3ffb243a499b434c3bdd976a7.1639438462.git.softworkz@hotmail.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [PBYevoWgBWKc1DptY0BG800W+BZf2D6R] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 3e69a9d9-8550-453a-d6bd-08d9be91fd11 x-ms-traffictypediagnostic: DM8P223MB0397:EE_ x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: RydOkS7sq4GStVaaMSkWsYGRCBppK6Mm6YQbmfS+btdG+WE0SIMsl3GIWVgH8owMQFbByfy+9ZfImLp5BxZT4lDQ9L+6bz2M/zuAHWFyoQ5JQh5uE5QPc0UQ4auEmYNoNlTRmFGBoZJEOo7B+GAh2FiEcJjA3hWUhvK+c7LZy3AdAd9k3gGbMLN0FrduXEUTsLJ98blJKVd7h0JK+sENVLnJRWiU/Qj3TrmpuG5Xw3TMnPcAXHpGx6+/rBVKlghMITkQGMBb3zp31VtA8DuQIf1uN9Eaieqh9vHnbFfC/SgKjd84X6HSOOKUTPWxHAh2EfTZXpoSbIFmnVkehwvzdZD/TI1Oc0aJ65Ncr3Tlnr3FcQmCsLNBBB5EEvsH0S4tfN1rQRWAu/HnT2JQNUCSyZONKfmXwhPrUsA7mwYrw6fpOWnufdYO2gn9URGamQYN90MhTs1uYmAMlQWU0aUUbhGK7TVdkRNqIB6sPiA6oXLDy6kznnobNx1/UNb/9Pp35E9D1JlERZI7ZJbEjHhawRlJgbNsn4qw3ZuHzo0uCpeNsRsK+C0povvFj9cy/q6kqgqDFD76W3id/JZZLI6xxg== x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?q?LwUzx4TW8d4HjuVjD03NxpDoIYfs?= =?utf-8?q?wn7fkndtwuTSqKgK14ckUB2U3AUGu/TtwKRdzXJU6UcqlK3FyyzjMegA6XdehbIOt?= =?utf-8?q?sLFLs4mG+y+ssyDSEs1iqHTKDM/kfYz+lNm0seSIZ6sXXmVhyyjDtXWCebmKXXC4C?= =?utf-8?q?FmVa75NNPe3P+eM6xzs6mXRMiEi18KzCkJfm8VFzPT3AYe29kU6e3ixqb2unpDdrI?= =?utf-8?q?L3hHW2Uh4YBlgYRvWBZEDtEOg4Qs0OmkItY3aQ4Ura+8jEmwkPBENJSkD+Aed1pj1?= =?utf-8?q?ZoIz6ciZ2uwNAmjxpYQelQk1O4yEmDhGXkxvpiAnkdDzjGw6zGQATv/mIRpPkjbK/?= =?utf-8?q?XOOlhtnx7FfXiw2SX4XG89wPy4OJqqdGM8h8lxokh0RQRp2kcLqbLtW70FZS+AO3n?= =?utf-8?q?SrKuqFHaTfXDII/qtxTq42VhNbsjyV2xD0ArtBrLOxPUoqXnhNPRwsN7a80rgpC/C?= =?utf-8?q?OCGNQ2tIZmwe+h5Ro5p//Acu2fm65e7K9BIZOlfjmR8gCfffzyJyLGDChxZ6RQcJP?= =?utf-8?q?d9eMfPhGIS+Dpi2rXM0jpFwuYRpjq/uOZkA+fuX6qCt7rUWg2/vee9Mn4ZV9o7upV?= =?utf-8?q?noXZPWTMmmgHWyVHzlBf2rTKFoD5TVQxF1g8Z8b3Ek1lb2H8ng5FwQsB5hLAHklo3?= =?utf-8?q?66rB7f7kczWe0bDQ9qudzTiR+GkqdlSbVTlrPKpZFAVIiFD5NIH80LH1uUIAN00YR?= =?utf-8?q?mqM0itFX65gXFZtShkCbtA6zu9m9qUfsgIGyaeVkqYeyQolex7O4pjeFzgDygl/9Q?= =?utf-8?q?pFaLtsxyKFf5f/zjG5xVRQQyCXbezgkNT06fhTUagNoHQtkd2YUhcqcORX+ZP1Hpv?= =?utf-8?q?7cli/3Rz0yKL058s+78muRq3enxDV5X8zm/3GfpFZ6K0xfby7L8ut4q1hq7oSwILC?= =?utf-8?q?0s4x40FntImLFiwhAAcX3wnnEBdqLM2CLSaR+2zpUoog=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-1ff67.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 3e69a9d9-8550-453a-d6bd-08d9be91fd11 X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Dec 2021 23:40:47.4617 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8P223MB0397 Subject: [FFmpeg-devel] [PATCH v24 14/21] avfilter/avfilter: Fix hardcoded input index X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: fg8L7cZtI762 This fix targets (rare) cases where multiple input pads have a .filter_frame function. ff_request_frame_to_filter needs to call ff_request_frame with the correct input pad instead of the hardcoded first one. Signed-off-by: softworkz --- libavfilter/avfilter.c | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-) diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c index 75d5e86539..aa9aa71f53 100644 --- a/libavfilter/avfilter.c +++ b/libavfilter/avfilter.c @@ -463,7 +463,7 @@ static int64_t guess_status_pts(AVFilterContext *ctx, int status, AVRational lin return AV_NOPTS_VALUE; } -static int ff_request_frame_to_filter(AVFilterLink *link) +static int ff_request_frame_to_filter(AVFilterLink *link, int input_index) { int ret = -1; @@ -472,8 +472,8 @@ static int ff_request_frame_to_filter(AVFilterLink *link) link->frame_blocked_in = 1; if (link->srcpad->request_frame) ret = link->srcpad->request_frame(link); - else if (link->src->inputs[0]) - ret = ff_request_frame(link->src->inputs[0]); + else if (link->src->inputs[input_index]) + ret = ff_request_frame(link->src->inputs[input_index]); if (ret < 0) { if (ret != AVERROR(EAGAIN) && ret != link->status_in) ff_avfilter_link_set_in_status(link, ret, guess_status_pts(link->src, ret, link->time_base)); @@ -1172,6 +1172,14 @@ static int forward_status_change(AVFilterContext *filter, AVFilterLink *in) { unsigned out = 0, progress = 0; int ret; + int input_index = 0; + + for (int i = 0; i < in->dst->nb_inputs; i++) { + if (&in->dst->input_pads[i] == in->dstpad) { + input_index = i; + break; + } + } av_assert0(!in->status_out); if (!filter->nb_outputs) { @@ -1181,7 +1189,7 @@ static int forward_status_change(AVFilterContext *filter, AVFilterLink *in) while (!in->status_out) { if (!filter->outputs[out]->status_in) { progress++; - ret = ff_request_frame_to_filter(filter->outputs[out]); + ret = ff_request_frame_to_filter(filter->outputs[out], input_index); if (ret < 0) return ret; } @@ -1218,7 +1226,7 @@ static int ff_filter_activate_default(AVFilterContext *filter) for (i = 0; i < filter->nb_outputs; i++) { if (filter->outputs[i]->frame_wanted_out && !filter->outputs[i]->frame_blocked_in) { - return ff_request_frame_to_filter(filter->outputs[i]); + return ff_request_frame_to_filter(filter->outputs[i], 0); } } return FFERROR_NOT_READY; From patchwork Mon Dec 13 23:40:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 32466 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp6141093iog; Mon, 13 Dec 2021 15:43:11 -0800 (PST) X-Google-Smtp-Source: ABdhPJwZF3QQFPzqZg+7t4Z0QT8nFICZL8wGyurxOAeKAs1Wy/GeWq9oq1TNrT1wA1saSiT6z5Pj X-Received: by 2002:a17:906:b04c:: with SMTP id bj12mr1656425ejb.716.1639438991135; Mon, 13 Dec 2021 15:43:11 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id do11si22962102ejc.174.2021.12.13.15.43.10; Mon, 13 Dec 2021 15:43:11 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=KQUCC5hx; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7251968B0E4; Tue, 14 Dec 2021 01:40:54 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12olkn2087.outbound.protection.outlook.com [40.92.21.87]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id ED2D468B0DF for ; Tue, 14 Dec 2021 01:40:51 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=K+PC4gP8F4PgvWqzasObrdbR/+My79HLEWXW7azSMvNp1Tym/HIhFrrhaN9lJcGttARl0LdvffOoExbrsauef7as5wLxBII8DWRaLZGsrHu+tntl45+PrK5jdTdjW388mNDRhIomIiBFpTjZVICWFBOic93zvq244izI4TjRj8NbT/QKJrFl5Qko5WXv052ns5t6F6x7Zo4Sxz+XXo+H8AKtHedjd410GiczVOAFEt2JdFZ1vBvwCPWsPotzeyiJiT/hrFUU0I9ZLdpfvX9+bJjCOm1tZZWH8dgkVRSPSgp0A9LFkdKgrFhiZJsJSgMmsKjMVL5DD0BiKHctQdm64Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=jFY9CycOlpiZsBtTrBfKDY7aFHXJoL2SHIZym1X2/X0=; b=Fxpw/ZWVpvFZ1xpeuiO+wVb11zj3rSZJKQY2dTDYQ3sHlT01udTtBVRW6zeuTOLE1DDj0iCmvgx0OeAoa96YvGo6h63XbJSYSvGPzR+38hoK8Z+iqa83FoTRWqWNzozc8jWC7Fa/pH2fE/JqI40q3Yti3/8L2sVcr+hnn/AyttFDbEetTjnEnlE3ApgvO+ZaoJjL2kZc1GYV5s8tM7s1ktLfc5w7W+Ezw+Aei7CPRe0N/vTXR9fCaFldnFc2qCqTqjwBQ6SWh0eZCeisdIScqUJinQFaUpNuxvCkGXxzn1aTy4CnBiy6/zKqPfJX0INqAh+ZGmfIxxdPTh87DpZxFQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jFY9CycOlpiZsBtTrBfKDY7aFHXJoL2SHIZym1X2/X0=; b=KQUCC5hxkuLznwh0UHaj+YAJD8BMf9qTSWYtqtSvFNZXWkQZchOF4Wwf47MSbSPQECSWEe6lCJmOTvV9xH+RJvl2qGCA7a1P+Gn9OBvlH4OeF+bJou69L1T+6dNP0jaBC3/yykICiogLhQqHMD9eIchiURrUrNMZQMtq1sDzjdOSZqonwzcXyvukUlSNve0xYgk7ARPTTLaMkTx98UTuI2jnGScbrRiu3JkETpGU3zwLfw0CA1tSDtbLvqPa90ik1SF9eg7GH6KtrIl+2/9Yt5pP94jZBMsoyc8h8KS+K1aQubn6dtVYHHXiuoUPmnvGmpv4XT0CplA6CddiORZWSA== Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::20) by DM8P223MB0397.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4778.17; Mon, 13 Dec 2021 23:40:49 +0000 Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775]) by DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775%7]) with mapi id 15.20.4778.018; Mon, 13 Dec 2021 23:40:49 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v24 15/21] avfilter/overlaytextsubs: Add overlaytextsubs and textsubs2video filters Thread-Index: AQHX8Hrb3l9cdJhuF0K+G/2Gf9xjlQ== Date: Mon, 13 Dec 2021 23:40:48 +0000 Message-ID: References: <1ef42eb01b3627bbad539abd46206f320bb39697.1639438462.git.softworkz@hotmail.com> <4148d77a82d4b7e229fc81ce9bc1bc06904533f1.1639438462.git.softworkz@hotmail.com> <2ac798924ee984010d2e85f00bc992d9aa106171.1639438462.git.softworkz@hotmail.com> <300ea7188b331ca5e00dd7e04c03df8aba2a99c0.1639438462.git.softworkz@hotmail.com> <9474726b26f0d3e8f353d07caed2fbd33cdf473a.1639438462.git.softworkz@hotmail.com> <424522c96b5cdf6b03faa6bdceb1d25449463429.1639438462.git.softworkz@hotmail.com> <90caff245055005e6a4a890b3665da55aa853b31.1639438462.git.softworkz@hotmail.com> <0ebcf3ed2adb02d3ffb243a499b434c3bdd976a7.1639438462.git.softworkz@hotmail.com> <4b5c1c1185075fd6e5d8ef70e07fdce6f896d699.1639438462.git.softworkz@hotmail.com> In-Reply-To: <4b5c1c1185075fd6e5d8ef70e07fdce6f896d699.1639438462.git.softworkz@hotmail.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [1NeX9onWGka6vPI0HbBiZ7WSuyqIPUMH] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 248660a5-aefd-4e1b-50c7-08d9be91fdfd x-ms-traffictypediagnostic: DM8P223MB0397:EE_ x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: y247qqEi7y4hQwiEV5KPU/b/LaaCWDXN+AiQOu5dSeCy8tMF9Xs0gtxWxQg5nQzql2Yz73kI7EF86d1E6wL2rkasrMVI9NNY7kMz67gcFhjtf+nOFH5xaTcEIHiRUl41w9orq/rwzWh5u465FFnKWrF26y4IJ+qNge8bffQpqTCgV/giAnR1NKcAwPRP7pftJlgoutwaeljx7uekKkAgJ/8eTcZrwHn/x/HcTkcRCqndF6wsI12jD/ijUQzrT9BQJbhcCNs5ZAnPCfl8f6T1izxpkdhZouGOze6NjA73oT5Tf7dmkBquJ+bPxj4BkAXxCTZBygsEtA9jxporqSa2TZ8WZQXWAv1MwWL9oCszZVLu8QGmIUbC1+yqhggi4edtTm75gFmFDyX5053HpOSlUxf/R1O6OKY+GKENpGOXF2juE6QLVmHQJvcHs27ukTYVv7R4M5PWjYyAuMAV5XF3OrzagREzk37BoXz94ktO9I3Wh1UjK7esvK8R0xxAR/G5YecmIZT44dmx5Jw6gfs6ySl7mUoLRRUnZDT20x3/Ak71W8i46lop++mqrW/2/dPlQJTQjsEZXvwZqC35dEMvME8GUs93tPTmZoY4T1D7Clm/BWVfMhq8nN3lppccgJa1Ly0THskq5FPW1USqwVuv//z2gj2CdaeGQhFdYu2ul/c= x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?q?mCOGTei24O4I7CZ0PKzAxTTTPofu?= =?utf-8?q?lTerDf3qQ8GGeP7aWmGD3AIs73tlheW5fnygZheqNL2qZwrWB6dWUzT2JmXBTwXJL?= =?utf-8?q?zZ4nYm3U+kj4iyyvLurHgxRyYEjhQbWhHuPgtlXWQh8CCdwdB/wSe1gMFlx1Xh90w?= =?utf-8?q?WZ0XTWyED9LieD+qnH/Tg+n24UvgQIWlEN6iVB7pRkEZjorMgXqRuFQWJaZURz+Aj?= =?utf-8?q?2Eb351iUuMviBpg5/Ei+7p0AcS43DUoDBPL9P2X15zwkd/Xa2rSbmziVmNk3UzxS1?= =?utf-8?q?H1GlBA3UT8UFsxrxo9BYYfjdL8Zy3vkF/XtQNYI3h6UtjH0FyPr2P14+8Wp891FpZ?= =?utf-8?q?4K1P5JnjHxjlmu9H9K+Y0R+7hjlbqOD6s/aAcuTPmTRf/lkJqYdrO/1X7vEfMn1tK?= =?utf-8?q?o9AtvnP+/HhmzJW3IUT/B2LFNWW4iXkfFgMGWfxWF/hE4yFu5HtTf3F7o0nsUrg62?= =?utf-8?q?P6Cfu10Yh3TcudcPd6JoWky8iXh8n16lTlfdW57mjQnKlm0pnX7bOwzELuFPAyH6W?= =?utf-8?q?HxV+eEN6wC0s+muwL4cO5FjxO/4cueneQ0f4uOOEgAEc1BdGy8utasvQesCFTb6Pb?= =?utf-8?q?md3xyat1/vV0kpWNteiQgHe52zV3rwM139R+srMXFazMFGVzEUMr0yfDY0smNdpf+?= =?utf-8?q?xyLD+8pg3v2wDreJFTApsjYpcIPJJeyhlxVTpy2wGWW7l8kUybaFzQhw7CVXbKYQj?= =?utf-8?q?ofHHv1vxaNpTeNh5iuTH8GdQv4meLhNRIGy2HAIjbl+T8Re0JZ2oVn+cuWpOhL1C/?= =?utf-8?q?7sUwGcTy4KuEW2Y7iZMZlAhPVzLSQ3ytMeRc/GC2WiipoD/XAdrWi2ZYfYhUoxgT5?= =?utf-8?q?lly0SF4FibhY/zp5dUHbVTAk4gD7agm9RBEBQg0KLsCbroRxpgJmKdHJdc7ECh/Yp?= =?utf-8?q?tuR21frk9X7wKrJAUYareXAw8+NoFInZYAja58SmnxUg=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-1ff67.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 248660a5-aefd-4e1b-50c7-08d9be91fdfd X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Dec 2021 23:40:48.9920 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8P223MB0397 Subject: [FFmpeg-devel] [PATCH v24 15/21] avfilter/overlaytextsubs: Add overlaytextsubs and textsubs2video filters X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: PM9YDwwXvp7c - overlaytextsubs {VS -> V) Overlay text subtitles onto a video stream. - textsubs2video {S -> V) Converts text subtitles to video frames Signed-off-by: softworkz --- configure | 2 + doc/filters.texi | 113 ++++++ libavfilter/Makefile | 2 + libavfilter/allfilters.c | 4 +- libavfilter/vf_overlaytextsubs.c | 643 +++++++++++++++++++++++++++++++ 5 files changed, 763 insertions(+), 1 deletion(-) create mode 100644 libavfilter/vf_overlaytextsubs.c diff --git a/configure b/configure index 0d0310cc9d..7f67a62b08 100755 --- a/configure +++ b/configure @@ -3666,6 +3666,7 @@ overlay_opencl_filter_deps="opencl" overlay_qsv_filter_deps="libmfx" overlay_qsv_filter_select="qsvvpp" overlay_vulkan_filter_deps="vulkan spirv_compiler" +overlaytextsubs_filter_deps="avcodec libass" owdenoise_filter_deps="gpl" pad_opencl_filter_deps="opencl" pan_filter_deps="swresample" @@ -3710,6 +3711,7 @@ superequalizer_filter_deps="avcodec" superequalizer_filter_select="rdft" surround_filter_deps="avcodec" surround_filter_select="rdft" +textsub2video_filter_deps="avcodec libass" tinterlace_filter_deps="gpl" tinterlace_merge_test_deps="tinterlace_filter" tinterlace_pad_test_deps="tinterlace_filter" diff --git a/doc/filters.texi b/doc/filters.texi index e7fd243ac6..5e00716083 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -25828,6 +25828,119 @@ Overlay PGS subtitles ffmpeg -i "https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv" -filter_complex "[0:0][0:1]overlaygraphicsubs" output.mp4 @end example @end itemize + +@section overlaytextsubs + +Overlay text subtitles onto a video stream. + +This filter supersedes the classic @ref{subtitles} filter opposed to which it does no longer require to open and access the source stream separately, which is often causing problems or doesn't even work for non-local or slow sources. + +Inputs: +@itemize +@item 0: Video [YUV420P, YUV422P, YUV444P, ARGB, RGBA, ABGR, BGRA, RGB24, BGR24] +@item 1: Subtitles [TEXT] +@end itemize + +Outputs: +@itemize +@item 0: Video (same as input) +@end itemize + +It accepts the following parameters: + +@table @option + +@item alpha +Process alpha channel, by default alpha channel is untouched. + +@item fonts_dir +Set a directory path containing fonts that can be used by the filter. +These fonts will be used in addition to whatever the font provider uses. + +@item default_font_path +Path to a font file to be used as the default font. + +@item font_size +Set the default font size. + +@item fontconfig_file +Path to ASS fontconfig configuration file. + +@item force_style +Override default style or script info parameters of the subtitles. It accepts a +string containing ASS style format @code{KEY=VALUE} couples separated by ",". + +@item margin +Set the rendering margin in pixels. + +@item render_latest_only +For rendering, alway use the latest event only, which is covering the given point in time +@end table + +@subsection Examples + +@itemize +@item +Overlay ASS subtitles with animations: +@example +ffmpeg -i "http://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.mkv" -filter_complex "[0:v]overlaytextsubs" -map 0 -y out.mkv +@end example +@end itemize + +@section textsub2video + +Converts text subtitles to video frames. + +For overlaying text subtitles onto video frames it is recommended to use the overlay_textsubs filter. +The textsub2video is useful for for creating transparent text-frames when overlay is done via hw acceleration + +Inputs: +@itemize +@item 0: Subtitles [TEXT] +@end itemize + +Outputs: +@itemize +@item 0: Video [RGB32] +@end itemize + +It accepts the following parameters: + +@table @option + +@item rate, r +Set the framerate for updating overlay frames. +Normally, overlay frames will only be updated each time when the subtitles to display are changing. +In cases where subtitles include advanced features (like animation), this parameter determines the frequency by which the overlay frames should be updated. + +@item size, s +Set the output frame size. +Allows to override the size of output video frames. + +@item fonts_dir +Set a directory path containing fonts that can be used by the filter. +These fonts will be used in addition to whatever the font provider uses. + +@item default_font_path +Path to a font file to be used as the default font. + +@item font_size +Set the default font size. + +@item fontconfig_file +Path to ASS fontconfig configuration file. + +@item force_style +Override default style or script info parameters of the subtitles. It accepts a +string containing ASS style format @code{KEY=VALUE} couples separated by ",". + +@item margin +Set the rendering margin in pixels. + +@item render_latest_only +For rendering, alway use the latest event only, which is covering the given point in time. +@end table + @c man end SUBTITLE FILTERS @chapter Multimedia Filters diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 03ecf92ec2..fe16fc147a 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -378,6 +378,7 @@ OBJS-$(CONFIG_OVERLAY_OPENCL_FILTER) += vf_overlay_opencl.o opencl.o \ OBJS-$(CONFIG_OVERLAY_QSV_FILTER) += vf_overlay_qsv.o framesync.o OBJS-$(CONFIG_OVERLAY_VULKAN_FILTER) += vf_overlay_vulkan.o vulkan.o vulkan_filter.o OBJS-$(CONFIG_OVERLAYGRAPHICSUBS_FILTER) += vf_overlaygraphicsubs.o framesync.o +OBJS-$(CONFIG_OVERLAYTEXTSUBS_FILTER) += vf_overlaytextsubs.o OBJS-$(CONFIG_OWDENOISE_FILTER) += vf_owdenoise.o OBJS-$(CONFIG_PAD_FILTER) += vf_pad.o OBJS-$(CONFIG_PAD_OPENCL_FILTER) += vf_pad_opencl.o opencl.o opencl/pad.o @@ -467,6 +468,7 @@ OBJS-$(CONFIG_SWAPRECT_FILTER) += vf_swaprect.o OBJS-$(CONFIG_SWAPUV_FILTER) += vf_swapuv.o OBJS-$(CONFIG_TBLEND_FILTER) += vf_blend.o framesync.o OBJS-$(CONFIG_TELECINE_FILTER) += vf_telecine.o +OBJS-$(CONFIG_TEXTSUB2VIDEO_FILTER) += vf_overlaytextsubs.o OBJS-$(CONFIG_THISTOGRAM_FILTER) += vf_histogram.o OBJS-$(CONFIG_THRESHOLD_FILTER) += vf_threshold.o framesync.o OBJS-$(CONFIG_THUMBNAIL_FILTER) += vf_thumbnail.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index f8bf143fbe..b3a4bf24aa 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -358,9 +358,10 @@ extern const AVFilter ff_vf_oscilloscope; extern const AVFilter ff_vf_overlay; extern const AVFilter ff_vf_overlay_opencl; extern const AVFilter ff_vf_overlay_qsv; -extern const AVFilter ff_vf_overlaygraphicsubs; extern const AVFilter ff_vf_overlay_vulkan; extern const AVFilter ff_vf_overlay_cuda; +extern const AVFilter ff_vf_overlaygraphicsubs; +extern const AVFilter ff_vf_overlaytextsubs; extern const AVFilter ff_vf_owdenoise; extern const AVFilter ff_vf_pad; extern const AVFilter ff_vf_pad_opencl; @@ -546,6 +547,7 @@ extern const AVFilter ff_avf_showwaves; extern const AVFilter ff_avf_showwavespic; extern const AVFilter ff_vaf_spectrumsynth; extern const AVFilter ff_svf_graphicsub2video; +extern const AVFilter ff_svf_textsub2video; /* multimedia sources */ extern const AVFilter ff_avsrc_amovie; diff --git a/libavfilter/vf_overlaytextsubs.c b/libavfilter/vf_overlaytextsubs.c new file mode 100644 index 0000000000..043423f50c --- /dev/null +++ b/libavfilter/vf_overlaytextsubs.c @@ -0,0 +1,643 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * overlay text subtitles on top of a video frame + */ + +#include +#include "libavutil/ass_internal.h" +#include "libavutil/thread.h" + +#include "drawutils.h" +#include "filters.h" + +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" + +typedef struct TextSubsContext { + const AVClass *class; + AVMutex mutex; + int is_mutex_initialized; + + ASS_Library *library; + ASS_Renderer *renderer; + ASS_Track *track; + + char *default_font_path; + char *fonts_dir; + char *fc_file; + double font_size; + char *force_style; + char *language; + int margin; + int render_latest_only; + + int alpha; + FFDrawContext draw; + + int got_header; + int out_w, out_h; + AVRational frame_rate; + AVFrame *last_frame; + int need_frame; + int eof; +} TextSubsContext; + +/* libass supports a log level ranging from 0 to 7 */ +static const int ass_libavfilter_log_level_map[] = { + AV_LOG_QUIET, /* 0 */ + AV_LOG_PANIC, /* 1 */ + AV_LOG_FATAL, /* 2 */ + AV_LOG_ERROR, /* 3 */ + AV_LOG_WARNING, /* 4 */ + AV_LOG_INFO, /* 5 */ + AV_LOG_VERBOSE, /* 6 */ + AV_LOG_DEBUG, /* 7 */ +}; + +static void ass_log(int ass_level, const char *fmt, va_list args, void *ctx) +{ + const int ass_level_clip = av_clip(ass_level, 0, FF_ARRAY_ELEMS(ass_libavfilter_log_level_map) - 1); + const int level = ass_libavfilter_log_level_map[ass_level_clip]; + + av_vlog(ctx, level, fmt, args); + av_log(ctx, level, "\n"); +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + TextSubsContext *s = ctx->priv; + + if (s->track) + ass_free_track(s->track); + if (s->renderer) + ass_renderer_done(s->renderer); + if (s->library) + ass_library_done(s->library); + + s->track = NULL; + s->renderer = NULL; + s->library = NULL; + + if (s->is_mutex_initialized) { + ff_mutex_destroy(&s->mutex); + s->is_mutex_initialized = 0; + } + + av_frame_free(&s->last_frame); +} + +static int overlay_textsubs_query_formats(AVFilterContext *ctx) +{ + AVFilterFormats *formats; + AVFilterLink *inlink0 = ctx->inputs[0]; + AVFilterLink *inlink1 = ctx->inputs[1]; + AVFilterLink *outlink = ctx->outputs[0]; + static const enum AVSubtitleType subtitle_fmts[] = { AV_SUBTITLE_FMT_ASS, AV_SUBTITLE_FMT_NONE }; + int ret; + + /* set input0 and output0 video formats */ + formats = ff_draw_supported_pixel_formats(0); + if ((ret = ff_formats_ref(formats, &inlink0->outcfg.formats)) < 0) + return ret; + if ((ret = ff_formats_ref(formats, &outlink->incfg.formats)) < 0) + return ret; + + /* set input1 subtitle formats */ + formats = ff_make_format_list(subtitle_fmts); + if ((ret = ff_formats_ref(formats, &inlink1->outcfg.formats)) < 0) + return ret; + + return 0; +} + +static int config_output(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + + outlink->w = ctx->inputs[0]->w; + outlink->h = ctx->inputs[0]->h; + outlink->time_base = ctx->inputs[0]->time_base; + + return 0; +} + +static int config_input_main(AVFilterLink *inlink) +{ + AVFilterContext *ctx = inlink->dst; + TextSubsContext *s = inlink->dst->priv; + int ret; + + ret = ff_draw_init(&s->draw, inlink->format, s->alpha ? FF_DRAW_PROCESS_ALPHA : 0); + if (ret < 0) { + av_log(ctx, AV_LOG_ERROR, "Could not initialize ff_draw.\n"); + return ret; + } + + ass_set_frame_size (s->renderer, inlink->w, inlink->h); + ass_set_pixel_aspect(s->renderer, av_q2d(inlink->sample_aspect_ratio)); + + av_log(ctx, AV_LOG_VERBOSE, "Subtitle screen: %dx%d\n\n\n\n", inlink->w, inlink->h); + + return 0; +} + +/* libass stores an RGBA color in the format RRGGBBTT, where TT is the transparency level */ +#define AR(c) ( (c)>>24) +#define AG(c) (((c)>>16)&0xFF) +#define AB(c) (((c)>>8) &0xFF) +#define AA(c) ((0xFF-(c)) &0xFF) + +static void overlay_ass_image(TextSubsContext *s, AVFrame *picref, + const ASS_Image *image) +{ + for (; image; image = image->next) { + uint8_t rgba_color[] = {AR(image->color), AG(image->color), AB(image->color), AA(image->color)}; + FFDrawColor color; + ff_draw_color(&s->draw, &color, rgba_color); + ff_blend_mask(&s->draw, &color, + picref->data, picref->linesize, + picref->width, picref->height, + image->bitmap, image->stride, image->w, image->h, + 3, 0, image->dst_x, image->dst_y); + } +} + +static void process_header(AVFilterContext *link, AVFrame *frame) +{ + TextSubsContext *s = link->priv; + ASS_Track *track = s->track; + ASS_Style *style; + int sid = 0; + + if (!track) + return; + + if (frame && frame->subtitle_header) { + char *subtitle_header = (char *)frame->subtitle_header->data; + ass_process_codec_private(s->track, subtitle_header, strlen(subtitle_header)); + } + else { + char* subtitle_header = avpriv_ass_get_subtitle_header_default(0); + if (!subtitle_header) + return; + + ass_process_codec_private(s->track, subtitle_header, strlen(subtitle_header)); + av_free(subtitle_header); + } + + if (s->language) + s->track->Language = av_strdup(s->language); + + if (!s->track->event_format) { + s->track->event_format = av_strdup("ReadOrder, Layer, Style, Name, MarginL, MarginR, MarginV, Effect, Text"); + } + + if (s->track->n_styles == 0) { + sid = ass_alloc_style(track); + style = &s->track->styles[sid]; + style->Name = av_strdup("Default"); + style->PrimaryColour = 0xffffff00; + style->SecondaryColour = 0x00ffff00; + style->OutlineColour = 0x00000000; + style->BackColour = 0x00000080; + style->Bold = 200; + style->ScaleX = 1.0; + style->ScaleY = 1.0; + style->Spacing = 0; + style->BorderStyle = 1; + style->Outline = 2; + style->Shadow = 3; + style->Alignment = 2; + } + else + style = &s->track->styles[sid]; + + style->FontSize = s->font_size; + style->MarginL = style->MarginR = style->MarginV = s->margin; + + track->default_style = sid; + + s->got_header = 1; +} + +static int filter_video_frame(AVFilterLink *inlink, AVFrame *frame) +{ + AVFilterContext *ctx = inlink->dst; + TextSubsContext *s = ctx->priv; + int detect_change = 0; + ASS_Image *image; + + int64_t time_ms = (int64_t)((double)frame->pts * av_q2d(inlink->time_base) * 1000); + + ff_mutex_lock(&s->mutex); + image = ass_render_frame(s->renderer, s->track, time_ms, &detect_change); + ff_mutex_unlock(&s->mutex); + + if (detect_change) + av_log(ctx, AV_LOG_DEBUG, "Change happened at time ms:%"PRId64"\n", time_ms); + + overlay_ass_image(s, frame, image); + + return ff_filter_frame(ctx->outputs[0], frame); +} + +static int filter_subtitle_frame(AVFilterLink *inlink, AVFrame *sub) +{ + AVFilterContext *ctx = inlink->dst; + TextSubsContext *s = ctx->priv; + const int64_t start_time = av_rescale_q(sub->subtitle_pts, AV_TIME_BASE_Q, av_make_q(1, 1000)); + const int64_t duration = sub->subtitle_end_time; + + // Postpone header processing until we receive a frame with content + if (!s->got_header && sub->num_subtitle_areas > 0) + process_header(ctx, sub); + + for (unsigned i = 0; i < sub->num_subtitle_areas; i++) { + char *ass_line = sub->subtitle_areas[i]->ass; + if (!ass_line) + break; + + ff_mutex_lock(&s->mutex); + ass_process_chunk(s->track, ass_line, strlen(ass_line), start_time, duration); + + if (s->render_latest_only && s->track->n_events > 1) { + const int64_t diff = s->track->events[s->track->n_events - 1].Start + - s->track->events[s->track->n_events - 2].Start; + if (s->track->events[s->track->n_events - 2].Duration > diff) + s->track->events[s->track->n_events - 2].Duration = diff; + } + + ff_mutex_unlock(&s->mutex); + } + + av_frame_free(&sub); + return 0; +} + +static av_cold int init(AVFilterContext *ctx) +{ + int ret; + TextSubsContext *s = ctx->priv; + + s->library = ass_library_init(); + + if (!s->library) { + av_log(ctx, AV_LOG_ERROR, "Could not initialize libass.\n"); + return AVERROR(EINVAL); + } + + ass_set_message_cb(s->library, ass_log, ctx); + + /* Initialize fonts */ + if (s->fonts_dir) + ass_set_fonts_dir(s->library, s->fonts_dir); + + ass_set_extract_fonts(s->library, 1); + + s->renderer = ass_renderer_init(s->library); + if (!s->renderer) { + av_log(ctx, AV_LOG_ERROR, "Could not initialize libass renderer.\n"); + return AVERROR(EINVAL); + } + + s->track = ass_new_track(s->library); + if (!s->track) { + av_log(ctx, AV_LOG_ERROR, "ass_new_track() failed!\n"); + return AVERROR(EINVAL); + } + + ass_set_fonts(s->renderer, s->default_font_path, NULL, 1, s->fc_file, 1); + + if (s->force_style) { + char **list = NULL; + char *temp = NULL; + char *ptr = av_strtok(s->force_style, ",", &temp); + int i = 0; + while (ptr) { + av_dynarray_add(&list, &i, ptr); + if (!list) { + return AVERROR(ENOMEM); + } + ptr = av_strtok(NULL, ",", &temp); + } + av_dynarray_add(&list, &i, NULL); + if (!list) { + return AVERROR(ENOMEM); + } + ass_set_style_overrides(s->library, list); + av_free(list); + } + + ret = ff_mutex_init(&s->mutex, NULL); + if (ret) { + av_log(ctx, AV_LOG_ERROR, "mutex initialiuzation failed! Error code: %d\n", ret); + return AVERROR(EINVAL); + } + + s->is_mutex_initialized = 1; + + return ret; +} + +static int textsub2video_query_formats(AVFilterContext *ctx) +{ + AVFilterFormats *formats; + AVFilterLink *inlink = ctx->inputs[0]; + AVFilterLink *outlink = ctx->outputs[0]; + static const enum AVSubtitleType subtitle_fmts[] = { AV_SUBTITLE_FMT_ASS, AV_SUBTITLE_FMT_NONE }; + int ret; + + /* set input0 subtitle format */ + formats = ff_make_format_list(subtitle_fmts); + if ((ret = ff_formats_ref(formats, &inlink->outcfg.formats)) < 0) + return ret; + + /* set output0 video format */ + formats = ff_draw_supported_pixel_formats(AV_PIX_FMT_FLAG_ALPHA); + if ((ret = ff_formats_ref(formats, &outlink->incfg.formats)) < 0) + return ret; + + return 0; +} + +static int textsub2video_config_input(AVFilterLink *inlink) +{ + AVFilterContext *ctx = inlink->dst; + TextSubsContext *s = ctx->priv; + + if (s->out_w <= 0 || s->out_h <= 0) { + s->out_w = inlink->w; + s->out_h = inlink->h; + } + + return 0; +} + +static int textsub2video_config_output(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + TextSubsContext *s = ctx->priv; + int ret; + + ret = ff_draw_init(&s->draw, outlink->format, FF_DRAW_PROCESS_ALPHA); + if (ret < 0) { + av_log(ctx, AV_LOG_ERROR, "Could not initialize ff_draw.\n"); + return ret; + } + + if (s->out_w <= 0 || s->out_h <= 0) { + av_log(ctx, AV_LOG_ERROR, "No output image size set.\n"); + return AVERROR(EINVAL); + } + + ass_set_frame_size (s->renderer, s->out_w, s->out_h); + + outlink->w = s->out_w; + outlink->h = s->out_h; + outlink->sample_aspect_ratio = (AVRational){1,1}; + outlink->frame_rate = s->frame_rate; + + return 0; +} + +static int textsub2video_request_frame(AVFilterLink *outlink) +{ + TextSubsContext *s = outlink->src->priv; + AVFilterLink *inlink = outlink->src->inputs[0]; + int64_t last_pts = outlink->current_pts; + int64_t next_pts, time_ms; + int i, detect_change = 0, status; + AVFrame *out; + ASS_Image *image; + + status = ff_outlink_get_status(inlink); + if (status == AVERROR_EOF) + return AVERROR_EOF; + + if (s->eof) + return AVERROR_EOF; + + if (inlink->current_pts == AV_NOPTS_VALUE) { // || outlink->current_pts > inlink->current_pts) { + int ret = ff_request_frame(inlink); + if (ret == AVERROR_EOF) { + s->eof = 1; + } + + if (ret != 0) + av_log(outlink->src, AV_LOG_DEBUG, "ff_request_frame returned: %d\n", ret); + + s->need_frame = 1; + return 0; + } + + if (last_pts == AV_NOPTS_VALUE) + next_pts = last_pts = inlink->current_pts * av_q2d(inlink->time_base) / av_q2d(outlink->time_base); + else + next_pts = last_pts + (int64_t)(1.0 / av_q2d(outlink->frame_rate) / av_q2d(outlink->time_base)); + + time_ms = (int64_t)((double)next_pts * av_q2d(outlink->time_base) * 1000); + + ff_mutex_lock(&s->mutex); + image = ass_render_frame(s->renderer, s->track, time_ms, &detect_change); + ff_mutex_unlock(&s->mutex); + + if (detect_change) + av_log(outlink->src, AV_LOG_VERBOSE, "Change happened at time ms:%"PRId64" pts:%"PRId64"\n", time_ms, next_pts); + else if (s->last_frame) { + out = av_frame_clone(s->last_frame); + if (!out) + return AVERROR(ENOMEM); + + out->pts = out->pkt_dts = out->best_effort_timestamp = next_pts; + return ff_filter_frame(outlink, out); + } + + out = ff_get_video_buffer(outlink, outlink->w, outlink->h); + if (!out) + return AVERROR(ENOMEM); + + for (i = 0; i < AV_NUM_DATA_POINTERS; i++) { + if (out->buf[i] && i != 1) + memset(out->buf[i]->data, 0, out->buf[i]->size); + } + + out->pts = out->pkt_dts = out->best_effort_timestamp = next_pts; + + if (image) + overlay_ass_image(s, out, image); + + av_frame_free(&s->last_frame); + + s->last_frame = av_frame_clone(out); + + return ff_filter_frame(outlink, out); +} + +static int textsub2video_filter_frame(AVFilterLink *inlink, AVFrame *sub) +{ + AVFilterContext *ctx = inlink->dst; + TextSubsContext *s = ctx->priv; + const int64_t start_time = av_rescale_q(sub->subtitle_pts, AV_TIME_BASE_Q, av_make_q(1, 1000)); + const int64_t duration = sub->subtitle_end_time; + + av_log(ctx, AV_LOG_VERBOSE, "textsub2video_filter_frame num_subtitle_rects: %d, start_time_ms: %"PRId64"\n", sub->num_subtitle_areas, start_time); + + if (!s->got_header && sub->num_subtitle_areas > 0) + process_header(ctx, sub); + + for (unsigned i = 0; i < sub->num_subtitle_areas; i++) { + char *ass_line = sub->subtitle_areas[i]->ass; + if (!ass_line) + break; + + ff_mutex_lock(&s->mutex); + ass_process_chunk(s->track, ass_line, strlen(ass_line), start_time, duration); + + if (s->render_latest_only && s->track->n_events > 1) { + const int64_t diff = s->track->events[s->track->n_events - 1].Start + - s->track->events[s->track->n_events - 2].Start; + if (s->track->events[s->track->n_events - 2].Duration > diff) + s->track->events[s->track->n_events - 2].Duration = diff; + } + + ff_mutex_unlock(&s->mutex); + } + + av_frame_free(&sub); + + if (s->need_frame) { + s->need_frame = 0; + return textsub2video_request_frame(ctx->outputs[0]); + } + + return 0; +} + +#define OFFSET(x) offsetof(TextSubsContext, x) +#define FLAGS (AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_FILTERING_PARAM) + +static const AVOption overlaytextsubs_options[] = { + {"alpha", "enable processing of alpha channel", OFFSET(alpha), AV_OPT_TYPE_BOOL, {.i64 = 0 }, 0, 1, .flags = FLAGS}, + {"font_size", "default font size", OFFSET(font_size), AV_OPT_TYPE_DOUBLE, {.dbl = 18.0}, 0.0, 100.0, .flags = FLAGS}, + {"force_style", "force subtitle style", OFFSET(force_style), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, .flags = FLAGS}, + {"margin", "default margin", OFFSET(margin), AV_OPT_TYPE_INT, {.i64 = 20 }, 0, INT_MAX, .flags = FLAGS}, + {"default_font_path", "path to default font", OFFSET(default_font_path), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, .flags = FLAGS}, + {"fonts_dir", "directory to scan for fonts", OFFSET(fonts_dir), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, .flags = FLAGS}, + {"fontsdir", "directory to scan for fonts", OFFSET(fonts_dir), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, .flags = FLAGS}, + {"fontconfig_file", "fontconfig file to load", OFFSET(fc_file), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, .flags = FLAGS}, + {"language", "default language", OFFSET(language), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, .flags = FLAGS}, + {"render_latest_only", "newest sub event for each time", OFFSET(render_latest_only), AV_OPT_TYPE_BOOL, {.i64 = 0 }, 0, 1, .flags = FLAGS}, + { .name = NULL } +}; + +static const AVOption textsub2video_options[] = { + {"rate", "set frame rate", OFFSET(frame_rate), AV_OPT_TYPE_VIDEO_RATE, {.str="8"}, 0, INT_MAX, .flags = FLAGS}, + {"r", "set frame rate", OFFSET(frame_rate), AV_OPT_TYPE_VIDEO_RATE, {.str="8"}, 0, INT_MAX, .flags = FLAGS}, + {"size", "set video size", OFFSET(out_w), AV_OPT_TYPE_IMAGE_SIZE, {.str = NULL}, 0, 0, .flags = FLAGS}, + {"s", "set video size", OFFSET(out_w), AV_OPT_TYPE_IMAGE_SIZE, {.str = NULL}, 0, 0, .flags = FLAGS}, + {"font_size", "default font size", OFFSET(font_size), AV_OPT_TYPE_DOUBLE, {.dbl = 18.0}, 0.0, 100.0, .flags = FLAGS}, + {"force_style", "force subtitle style", OFFSET(force_style), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, .flags = FLAGS}, + {"margin", "default margin", OFFSET(margin), AV_OPT_TYPE_INT, {.i64 = 20 }, 0, INT_MAX, .flags = FLAGS}, + {"default_font_path", "path to default font", OFFSET(default_font_path), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, .flags = FLAGS}, + {"fonts_dir", "directory to scan for fonts", OFFSET(fonts_dir), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, .flags = FLAGS}, + {"fontsdir", "directory to scan for fonts", OFFSET(fonts_dir), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, .flags = FLAGS}, + {"fontconfig_file", "fontconfig file to load", OFFSET(fc_file), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, .flags = FLAGS}, + {"language", "default language", OFFSET(language), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, .flags = FLAGS}, + {"render_latest_only", "newest sub event for each time", OFFSET(render_latest_only), AV_OPT_TYPE_BOOL, {.i64 = 0 }, 0, 1, .flags = FLAGS}, + { .name = NULL } +}; + +#if CONFIG_OVERLAYTEXTSUBS_FILTER + +AVFILTER_DEFINE_CLASS(overlaytextsubs); + +static const AVFilterPad overlaytextsubs_inputs[] = { + { + .name = "main", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_input_main, + .flags = AVFILTERPAD_FLAG_NEEDS_WRITABLE, + .filter_frame = filter_video_frame, + }, + { + .name = "overlay", + .type = AVMEDIA_TYPE_SUBTITLE, + .filter_frame = filter_subtitle_frame, + }, +}; + +static const AVFilterPad overlaytextsubs_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_output, + }, +}; + +const AVFilter ff_vf_overlaytextsubs = { + .name = "overlaytextsubs", + .description = NULL_IF_CONFIG_SMALL("Overlay textual subtitles on top of the input."), + .init = init, + .uninit = uninit, + .priv_size = sizeof(TextSubsContext), + .priv_class = &overlaytextsubs_class, + FILTER_INPUTS(overlaytextsubs_inputs), + FILTER_OUTPUTS(overlaytextsubs_outputs), + FILTER_QUERY_FUNC(overlay_textsubs_query_formats), +}; +#endif + +#if CONFIG_TEXTSUB2VIDEO_FILTER + +AVFILTER_DEFINE_CLASS(textsub2video); + +static const AVFilterPad textsub2video_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .config_props = textsub2video_config_input, + .filter_frame = textsub2video_filter_frame, + }, +}; + +static const AVFilterPad textsub2video_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = textsub2video_config_output, + .request_frame = textsub2video_request_frame, + }, +}; + +const AVFilter ff_svf_textsub2video = { + .name = "textsub2video", + .description = NULL_IF_CONFIG_SMALL("Convert textual subtitles to video frames"), + .init = init, + .uninit = uninit, + .priv_size = sizeof(TextSubsContext), + .priv_class = &textsub2video_class, + FILTER_INPUTS(textsub2video_inputs), + FILTER_OUTPUTS(textsub2video_outputs), + FILTER_QUERY_FUNC(textsub2video_query_formats), +}; +#endif From patchwork Mon Dec 13 23:40:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 32467 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp6141291iog; Mon, 13 Dec 2021 15:43:22 -0800 (PST) X-Google-Smtp-Source: ABdhPJxNQn8WImpC0LNVVBpIvc4rUCTfySeU/Gm9GyzoupE17QFtb8ybK9kakZccPqWT49Y1YDIn X-Received: by 2002:a05:6402:5cc:: with SMTP id n12mr2754190edx.246.1639439002631; Mon, 13 Dec 2021 15:43:22 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id b13si28408654ede.393.2021.12.13.15.43.22; Mon, 13 Dec 2021 15:43:22 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b="VWusKAm/"; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7529B68B0E7; Tue, 14 Dec 2021 01:40:55 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12olkn2087.outbound.protection.outlook.com [40.92.21.87]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 8A9D168B0D7 for ; Tue, 14 Dec 2021 01:40:52 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KFx63UTIbfitqEwdBta0S+06MLFxk9fSfb5kWjhsJ+AUVlKXtSV5fQUFSjB+mXDKU3x+cj3g2yMuYIBCLRMV9C35neiG6c7oUNxmkRti2aRjxdFxRVvoPCaHFZD1//JDxkfYe/aOS+aQNbJGpRur71HgapbQTQHzivPpE2Ck+XW6ULWvYtLJ2AZmgn0kJ+2MHr2GsBIAZkNZ9XS4VX7oaNWRgppiAf1cf6iEJlPOiTH4dPxaVtlLh6Hw1GLTBX9DK2WImm2SUHJJ8YfcQzh6BFZFLeOAzsOKIJFN/Kqithp3Nxmvq4qb2ZKpaPfOLid0vUgFDSBZvW0B0QJZ+mpptw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ntlEjMnOpwwB/8Ro9QyQKlyjm6VZYlu033LdyGk/WkY=; b=nejVQE8KL8MC4cVgEWsmsGuNr6h6dYydXZYON4mbrXpQf01QiDGrmAzGzo1M+2di2TJIrvlq/P8lHKdxPNB3LDZYPTysMlTPoTUxcZDUFhgzDs/IKQy0TBW2f1yS66cBriRvbr4TZQmITxFM9MDwWNqC5O8geiXPTdkcfjA02DcvWOlqvP+I0yLt7g7/EoWSBmOCJrQwM6erL+0s2BPcW3C/6QTvBR+Jy124AuxUBDoa8p1BS+uJBlZzcBlg+zz6J9yXJfpgBJqsTcDgyZks2HlppQLaBqOBBy97STZ+csNAPU25jBMQ/hb45lsB1ss7ASuE7TjZuWcByZV5eptQpA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ntlEjMnOpwwB/8Ro9QyQKlyjm6VZYlu033LdyGk/WkY=; b=VWusKAm/MKbsJnLlx7v5EaPDyIanUto4pef4mCinMe2hlO7V8eORmCEXCllCLrmt8Y/i8Y8eKcWtjbvcCPsUcxMmygy+zGNTGkDCy0OnpMToS/N1ek4TGKERSgeTL3XlNiC4VDW9IedS2wwLCxYlkKf9qFqw/71jz56m8d3bigJnMnckFSiRS0HYkj96zK0yJJtib+5u85G8tTzHSfQAtFxegFwTJnqvkT6IzLEXLDxptEGbSsNtg30BRz4rf4Gn4G0RNUYZJE602Mye1OaSnxAqz/pLjIoFf8xxUupy/lSOE0S6TADyPr1IENVGRLsmN76M/8iR70ug7fNBeq4rCw== Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::20) by DM8P223MB0397.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4778.17; Mon, 13 Dec 2021 23:40:51 +0000 Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775]) by DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775%7]) with mapi id 15.20.4778.018; Mon, 13 Dec 2021 23:40:51 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v24 16/21] avfilter/textmod: Add textmod, censor and show_speaker filters Thread-Index: AQHX8HrcIUrVnVM1OUW9B+zvjeHSXw== Date: Mon, 13 Dec 2021 23:40:51 +0000 Message-ID: References: <1ef42eb01b3627bbad539abd46206f320bb39697.1639438462.git.softworkz@hotmail.com> <4148d77a82d4b7e229fc81ce9bc1bc06904533f1.1639438462.git.softworkz@hotmail.com> <2ac798924ee984010d2e85f00bc992d9aa106171.1639438462.git.softworkz@hotmail.com> <300ea7188b331ca5e00dd7e04c03df8aba2a99c0.1639438462.git.softworkz@hotmail.com> <9474726b26f0d3e8f353d07caed2fbd33cdf473a.1639438462.git.softworkz@hotmail.com> <424522c96b5cdf6b03faa6bdceb1d25449463429.1639438462.git.softworkz@hotmail.com> <90caff245055005e6a4a890b3665da55aa853b31.1639438462.git.softworkz@hotmail.com> <0ebcf3ed2adb02d3ffb243a499b434c3bdd976a7.1639438462.git.softworkz@hotmail.com> <4b5c1c1185075fd6e5d8ef70e07fdce6f896d699.1639438462.git.softworkz@hotmail.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [Ef0NmRZl0o4ppgPxSwNzB+Vi1Ez9YoUI] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: a2766cc5-1ea8-47cc-3a63-08d9be91ff6a x-ms-traffictypediagnostic: DM8P223MB0397:EE_ x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: yzIuFUqDhF5DoQ5A1gfHO79hfbXVIUZST/igDJl9MPwoODLWmPsta9hJDP0CP7o4AhO7hNSolrRopnsMoBjagQuvCNhvMIecoMh+9lhOPNcb9rXJ0g9NXd8+dPaSdU0KHtPGw1rXfrElqBmdk1uWBJtLYBVk0+UDHyz2IdW1+HrC7fExhKJJK6w9BTH5YP+xTLXCX0Bu8DVJoBnPnp1NlX2kCJq69EmWBQt9luyz33HK6qat284DgSZujaMWz7Ds8UGuZQmr8R/7j2MqTlXOfvaV0NbpKERzG6MQPjXQ60bNpAYwrBG5WnjC+Cgp+lO8AlSKur2gbqxqhoWoHKEHDy5iuqhjRZU9gQe9pvYh8QC/mRNUSyFZ90Mp6hQxEFQ5xomS1ZIwz4Jr1f/GF36XGYupA+f0ZLwnRG8lLxWsT69nFhLL/QnKzdmKYiSeSh3xKieHHBR5noeEdeWK9+O8zgLc8Ojw6B4+nt/5gcAUQJFKmcmZ4bCPkPDMHa8zspULTg1KM4gM0QJ4itiUBkZZ+qFS059PQtY3oHsQMcbtrLQQgppc815rq2u4V7A6WgisXVWQc8jmKDNPdqhZs8tebZ1OqvFfXh1UHxcwpiqGMejfO4MPHwir4rAXjcl2sGBh/N6P3lt0mGdi+FWjRjQM+J5p0ql3BQRQ3pPckzc7zmE= x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?q?5skaPIOcTXh1+IW9jU6i+D4uB30s?= =?utf-8?q?WFlDj4W0argxb/9x5+kgdUjYt5xBu2va95iGF3tnvo2YCyLfPQgAIzO1Ai7gco6Ke?= =?utf-8?q?7nhrGrNASAXDQGXIYu2ppSjzlbW68kbe1BohT2UkFkgKAp7KA8NyrIUz2b4+U8D+Y?= =?utf-8?q?4YOXuB2JkaSs/YI+W6KnJImcijgmFYChW0JGNIC3kvM88EziDKOEjJSGtBPnpW7AE?= =?utf-8?q?MIWGxiB7vQV31lV3U1ySNhq2NZKoXV8nBntkqg7cPYVMeMFl1v8RrtZoFWHom6auV?= =?utf-8?q?eRLH72YcocyfDScJ0GamHNjJWjiK1EbAT9NBTtpH+EfYNitrTSKQ5OwIo04u4TPLc?= =?utf-8?q?awHWWCm465HxlbWzZHUz5YNKzYVqb5UOmPfCwpKKACGN2eOykDQiZxrsFr+h5u5bl?= =?utf-8?q?uK6RHq2cW86O2PLH5aQB1JsKhoVqDz857RRk3Kp2fFqrDoC5fqmbbCnl57rnv2klD?= =?utf-8?q?vQ2NtR0z0lTQTir8HxR+61GzWVmbCLCSyIVMXfhd1S3HxVJN/TIr04imm3hf2IisL?= =?utf-8?q?Qjz2mNElhf9h422XaHK7WlhyVWFUq0PVns3e4faN68Ttpl2+d9UvY1UZzKw7+AlyU?= =?utf-8?q?ygFC1MRSigCwJM71BOEmHdFVDrkJ/+zURHsN//xxns9Ub/iLAM2PID8lN+uGNt8jl?= =?utf-8?q?8R0ISV0dp7vQhteAJ6OUWHMn1qPztwqTOTO1lFIDFRW/X3O3nJ9Eu7QmzqOjVpnwi?= =?utf-8?q?lVFfY1RozLjl2S9cDNO70YEUucr4aFKftvnzb2ny4Fa/kqzvRImfyHW1Az55rwpUP?= =?utf-8?q?Dlm2VDvf/9EUIOGzoBvQ7v6nnvvYIDI7Z0xlQoZ3qWkPlnu2NQONh7ThAnLCDS+oQ?= =?utf-8?q?keIsTxHt+CLtA8Mn+lfucDcaSOMrS9YZxAfOo06ipXdPn2nAW96YSfjAQ+n+1eBvc?= =?utf-8?q?fw/YGdAA8dW+MfuhGFgQExd6PcQx7Ramo5FdSK9qP9Kg=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-1ff67.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: a2766cc5-1ea8-47cc-3a63-08d9be91ff6a X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Dec 2021 23:40:51.3924 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8P223MB0397 Subject: [FFmpeg-devel] [PATCH v24 16/21] avfilter/textmod: Add textmod, censor and show_speaker filters X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: gyUULGbg9ri1 s- textmod {S -> S) Modify subtitle text in a number of ways - censor {S -> S) Censor subtitles using a word list - show_speaker {S -> S) Prepend speaker names from ASS subtitles to the visible text lines Signed-off-by: softworkz --- doc/filters.texi | 206 ++++++++++++ libavfilter/Makefile | 5 + libavfilter/allfilters.c | 3 + libavfilter/sf_textmod.c | 697 +++++++++++++++++++++++++++++++++++++++ 4 files changed, 911 insertions(+) create mode 100644 libavfilter/sf_textmod.c diff --git a/doc/filters.texi b/doc/filters.texi index 5e00716083..6ef95c4926 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -25720,6 +25720,145 @@ existing filters using @code{--disable-filters}. Below is a description of the currently available subtitle filters. + +@section censor + +Censor selected words in text subtitles. + +Inputs: +@itemize +@item 0: Subtitles[TEXT] +@end itemize + +Outputs: +@itemize +@item 0: Subtitles[TEXT] +@end itemize + +It accepts the following parameters: + +@table @option +@item mode +The censoring mode to apply. + +Supported censoring modes are: + +@table @var +@item 0, keep_first_last +Replace all characters with the 'censor_char' except the first and the last character of a word. +For words with less than 4 characters, the last character will be replaced as well. +For words with less than 3 characters, the first character will be replaced as well. +@item 1, keep_first +Replace all characters with the 'censor_char' except the first character of a word. +For words with less than 3 characters, the first character will be replaced as well. +@item 2, all +Replace all characters with the 'censor_char'. +@end table + +@item words +A list of words to censor, separated by 'separator'. + +@item words_file +Specify a file from which to load the contents for the 'words' parameter. + +@item censor_char +Single character used as replacement for censoring. + +@item separator +Delimiter character for words. Used with replace_words and remove_words- Must be a single character. +The default is '.'. + +@end table + +@subsection Examples + +@itemize +@item +Censor a few given words with a pound character. +@example +ffmpeg -i "http://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.mkv" -filter_complex "[0:1]censor=words='diss,louder,hope,beam,word':censor_char='#'" -map 0 -y output.mkv +@end example +@end itemize + + +@section textmod + +Modify subtitle text in a number of ways. + +Inputs: +@itemize +@item 0: Subtitles[TEXT] +@end itemize + +Outputs: +@itemize +@item 0: Subtitles[TEXT] +@end itemize + +It accepts the following parameters: + +@table @option +@item mode +The kind of text modification to apply + +Supported operation modes are: + +@table @var +@item 0, leet +Convert subtitle text to 'leet speak'. It's primarily useful for testing as the modification will be visible with almost all text lines. +@item 1, to_upper +Change all text to upper case. Might improve readability. +@item 2, to_lower +Change all text to lower case. +@item 3, replace_chars +Replace one or more characters. Requires the find and replace parameters to be specified. +Both need to be equal in length. +The first char in find is replaced by the first char in replace, same for all subsequent chars. +@item 4, remove_chars +Remove certain characters. Requires the find parameter to be specified. +All chars in the find parameter string will be removed from all subtitle text. +@item 5, replace_words +Replace one or more words. Requires the find and replace parameters to be specified. Multiple words must be separated by the delimiter char specified vie the separator parameter (default: ','). +The number of words in the find and replace parameters needs to be equal. +The first word in find is replaced by the first word in replace, same for all subsequent words +@item 6, remove_words +Remove certain words. Requires the find parameter to be specified. Multiple words must be separated by the delimiter char specified vie the separator parameter (default: ','). +All words in the find parameter string will be removed from all subtitle text. +@end table + +@item find +Required for replace_chars, remove_chars, replace_words and remove_words. + +@item find_file +Specify a file from which to load the contents for the 'find' parameter. + +@item replace +Required for replace_chars and replace_words. + +@item replace_file +Specify a file from which to load the contents for the 'replace' parameter. + +@item separator +Delimiter character for words. Used with replace_words and remove_words- Must be a single character. +The default is '.'. + +@end table + +@subsection Examples + +@itemize +@item +Change all characters to upper case while keeping all styles and animations: +@example +ffmpeg -i "https://streams.videolan.org/ffmpeg/mkv_subtitles.mkv" -filter_complex "[0:s]textmod=mode=to_upper" -map 0 -y out.mkv +@end example +@item +Remove a set of symbol characters for am improved and smoother visual apperance: +@example +ffmpeg -i "https://streams.videolan.org/ffmpeg/mkv_subtitles.mkv" -filter_complex "[0:s]textmod=mode=remove_chars:find='$&#@*§'" -map 0 -y out.mkv +@end example +@end itemize + @section graphicsub2video Renders graphic subtitles as video frames. @@ -25887,6 +26026,73 @@ ffmpeg -i "http://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex. @end example @end itemize +@section showspeaker + +Prepend speaker names to subtitle lines (when available). + +Subtitles in ASS/SSA format are often including the names of the persons +or character for each subtitle line. The showspeaker filter adds those names +to the actual subtitle text to make it visible on playback. + +Inputs: +@itemize +@item 0: Subtitles[TEXT] +@end itemize + +Outputs: +@itemize +@item 0: Subtitles[TEXT] +@end itemize + +It accepts the following parameters: + +@table @option +@item format +The format for prepending speaker names. Default is 'square_brackets'. + +Supported operation modes are: + +@table @var +@item 0, square_brackets +Enclose the speaker name in square brackets, followed by space ('[speaker] text'). +@item 1, round_brackets +Enclose the speaker name in round brackets, followed by space ('(speaker) text'). +@item 2, colon +Separate the speaker name with a colon and space ('speaker: text'). +@item 3, plain +Separate the speaker name with a space only ('speaker text'). +@end table + +@item line_break +Set thís parameter to insert a line break between speaker name and text instead of the space character. + +@item style +Allows to set a specific style for the speaker name text. + +This can be either a named style that exists in the ass subtitle header script (e.g. 'Default') or an ass style override code. +Example: @{\\c&HDD0000&\\be1\\i1\\bord10@} +This sets the color to blue, enables edge blurring, italic font and a border of size 10. + +The behavior is as follows: + +- When the style parameter is not provided, the filter will find the first position in the event string that is actual text. + The speaker name will be inserted at this position. This allows to have the speaker name shown in the same style like the + regular text, in case the string would start with a sequence of style codes. +- When the style parameter is provided, everything will be prepended to the original text: + Style Code or Style name >> Speaker Name >> Style Reset Code >> Original Text + +@end table + +@subsection Examples + +@itemize +@item +Prepend speaker names with blue text, smooth edges and blend/overlay that onto the video. +@example +ffmpeg -i INPUT -filter_complex "showspeaker=format=colon:style='@{\\c&HDD0000&\\be1@}',[0:v]overlay_textsubs" +@end example +@end itemize + @section textsub2video Converts text subtitles to video frames. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index fe16fc147a..d47bdfa8f6 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -555,6 +555,11 @@ OBJS-$(CONFIG_YUVTESTSRC_FILTER) += vsrc_testsrc.o OBJS-$(CONFIG_NULLSINK_FILTER) += vsink_nullsink.o +# subtitle filters +OBJS-$(CONFIG_CENSOR_FILTER) += sf_textmod.o +OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o +OBJS-$(CONFIG_TEXTMOD_FILTER) += sf_textmod.o + # multimedia filters OBJS-$(CONFIG_ABITSCOPE_FILTER) += avf_abitscope.o OBJS-$(CONFIG_ADRAWGRAPH_FILTER) += f_drawgraph.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index b3a4bf24aa..cc8f43c925 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -546,6 +546,9 @@ extern const AVFilter ff_avf_showvolume; extern const AVFilter ff_avf_showwaves; extern const AVFilter ff_avf_showwavespic; extern const AVFilter ff_vaf_spectrumsynth; +extern const AVFilter ff_sf_censor; +extern const AVFilter ff_sf_showspeaker; +extern const AVFilter ff_sf_textmod; extern const AVFilter ff_svf_graphicsub2video; extern const AVFilter ff_svf_textsub2video; diff --git a/libavfilter/sf_textmod.c b/libavfilter/sf_textmod.c new file mode 100644 index 0000000000..d73f329b87 --- /dev/null +++ b/libavfilter/sf_textmod.c @@ -0,0 +1,697 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * text subtitle filter which allows to modify subtitle text in several ways + */ + +#include + +#include "libavutil/opt.h" +#include "internal.h" +#include "libavutil/ass_split_internal.h" +#include "libavutil/bprint.h" +#include "libavutil/file.h" + +static const char* leet_src = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"; +static const char* leet_dst = "abcd3f6#1jklmn0pq257uvwxyzAB(D3F6#1JKLMN0PQ257UVWXYZ"; + +enum TextModFilterType { + TM_TEXTMOD, + TM_CENSOR, + TM_SHOW_SPEAKER, +}; + +enum TextModOperation { + OP_LEET, + OP_TO_UPPER, + OP_TO_LOWER, + OP_REPLACE_CHARS, + OP_REMOVE_CHARS, + OP_REPLACE_WORDS, + OP_REMOVE_WORDS, + NB_OPS, +}; + +enum CensorMode { + CM_KEEP_FIRST_LAST, + CM_KEEP_FIRST, + CM_ALL, +}; + +enum ShowSpeakerMode { + SM_SQUARE_BRACKETS, + SM_ROUND_BRACKETS, + SM_COLON, + SM_PLAIN, +}; + +typedef struct TextModContext { + const AVClass *class; + enum AVSubtitleType format; + enum TextModFilterType filter_type; + enum TextModOperation operation; + enum CensorMode censor_mode; + enum ShowSpeakerMode speaker_mode; + char *find; + char *find_file; + char *style; + char *replace; + char *replace_file; + char *separator; + char *censor_char; + char **find_list; + int line_break; + int nb_find_list; + char **replace_list; + int nb_replace_list; +} TextModContext; + +static char **split_string(char *source, int *nb_elems, const char *delim) +{ + char **list = NULL; + char *temp = NULL; + char *ptr = av_strtok(source, delim, &temp); + + while (ptr) { + if (strlen(ptr)) { + av_dynarray_add(&list, nb_elems, ptr); + if (!list) + return NULL; + } + + ptr = av_strtok(NULL, delim, &temp); + } + + if (!list) + return NULL; + + for (int i = 0; i < *nb_elems; i++) { + list[i] = av_strdup(list[i]); + if (!list[i]) { + for (int n = 0; n < i; n++) + av_free(list[n]); + av_free(list); + return NULL; + } + } + + return list; +} + +static int load_text_from_file(AVFilterContext *ctx, const char *file_name, char **text, char separator) +{ + int err; + uint8_t *textbuf; + char *tmp; + size_t textbuf_size; + int offset = 0; + + if ((err = av_file_map(file_name, &textbuf, &textbuf_size, 0, ctx)) < 0) { + av_log(ctx, AV_LOG_ERROR, "The text file '%s' could not be read or is empty\n", file_name); + return err; + } + + if (textbuf_size > 1 && + (textbuf[0] == 0xFF && textbuf[1] == 0xFE + || textbuf[0] == 0xFE && textbuf[1] == 0xFF)) { + av_log(ctx, AV_LOG_ERROR, "UTF text files are not supported. File: %s\n", file_name); + return AVERROR(EINVAL); + } + + if (textbuf_size > 2 && textbuf[0] == 0xEF && textbuf[1] == 0xBB && textbuf[2] == 0xBF) + offset = 3; // UTF-8 + + if (textbuf_size > SIZE_MAX - 1 || !((tmp = av_strndup((char *)textbuf + offset, textbuf_size - offset)))) { + av_file_unmap(textbuf, textbuf_size); + return AVERROR(ENOMEM); + } + + av_file_unmap(textbuf, textbuf_size); + + for (size_t i = 0; i < strlen(tmp); i++) { + switch (tmp[i]) { + case '\n': + case '\r': + case '\f': + case '\v': + tmp[i] = separator; + } + } + + *text = tmp; + + return 0; +} + +static int load_files(AVFilterContext *ctx) +{ + TextModContext *s = ctx->priv; + int ret; + + if (!s->separator || strlen(s->separator) != 1) { + av_log(ctx, AV_LOG_ERROR, "A single character needs to be specified for the separator parameter.\n"); + return AVERROR(EINVAL); + } + + if (s->find_file && strlen(s->find_file)) { + ret = load_text_from_file(ctx, s->find_file, &s->find, s->separator[0]); + if (ret < 0 ) + return ret; + } + + if (s->replace_file && strlen(s->replace_file)) { + ret = load_text_from_file(ctx, s->replace_file, &s->replace, s->separator[0]); + if (ret < 0 ) + return ret; + } + + return 0; +} + +static int init_censor(AVFilterContext *ctx) +{ + TextModContext *s = ctx->priv; + int ret; + + s->filter_type = TM_CENSOR; + s->operation = OP_REPLACE_WORDS; + + ret = load_files(ctx); + if (ret < 0 ) + return ret; + + if (!s->find || !strlen(s->find)) { + av_log(ctx, AV_LOG_ERROR, "Either the 'words' or the 'words_file' parameter needs to be specified\n"); + return AVERROR(EINVAL); + } + + if (!s->censor_char || strlen(s->censor_char) != 1) { + av_log(ctx, AV_LOG_ERROR, "A single character needs to be specified for the censor_char parameter\n"); + return AVERROR(EINVAL); + } + + s->find_list = split_string(s->find, &s->nb_find_list, s->separator); + if (!s->find_list) + return AVERROR(ENOMEM); + + s->replace_list = av_calloc(s->nb_find_list, sizeof(char *)); + if (!s->replace_list) + return AVERROR(ENOMEM); + + for (int i = 0; i < s->nb_find_list; i++) { + size_t len, start = 0, end; + char *item = av_strdup(s->find_list[i]); + if (!item) + return AVERROR(ENOMEM); + + len = end = strlen(item); + + switch (s->censor_mode) { + case CM_KEEP_FIRST_LAST: + + if (len > 2) + start = 1; + if (len > 3) + end--; + + break; + case CM_KEEP_FIRST: + + if (len > 2) + start = 1; + + break; + } + + for (size_t n = start; n < end; n++) + item[n] = s->censor_char[0]; + + s->replace_list[i] = item; + } + + return 0; +} + +static int init_showspeaker(AVFilterContext *ctx) +{ + TextModContext *s = ctx->priv; + s->filter_type = TM_SHOW_SPEAKER; + + return 0; +} + +static int init(AVFilterContext *ctx) +{ + TextModContext *s = ctx->priv; + int ret; + + ret = load_files(ctx); + if (ret < 0 ) + return ret; + + switch (s->operation) { + case OP_REPLACE_CHARS: + case OP_REMOVE_CHARS: + case OP_REPLACE_WORDS: + case OP_REMOVE_WORDS: + if (!s->find || !strlen(s->find)) { + av_log(ctx, AV_LOG_ERROR, "Selected mode requires the 'find' parameter to be specified\n"); + return AVERROR(EINVAL); + } + break; + } + + switch (s->operation) { + case OP_REPLACE_CHARS: + case OP_REPLACE_WORDS: + if (!s->replace || !strlen(s->replace)) { + av_log(ctx, AV_LOG_ERROR, "Selected mode requires the 'replace' parameter to be specified\n"); + return AVERROR(EINVAL); + } + break; + } + + if (s->operation == OP_REPLACE_CHARS && strlen(s->find) != strlen(s->replace)) { + av_log(ctx, AV_LOG_ERROR, "Selected mode requires the 'find' and 'replace' parameters to have the same length\n"); + return AVERROR(EINVAL); + } + + if (s->operation == OP_REPLACE_WORDS || s->operation == OP_REMOVE_WORDS) { + if (!s->separator || strlen(s->separator) != 1) { + av_log(ctx, AV_LOG_ERROR, "Selected mode requires a single separator char to be specified\n"); + return AVERROR(EINVAL); + } + + s->find_list = split_string(s->find, &s->nb_find_list, s->separator); + if (!s->find_list) + return AVERROR(ENOMEM); + + if (s->operation == OP_REPLACE_WORDS) { + + s->replace_list = split_string(s->replace, &s->nb_replace_list, s->separator); + if (!s->replace_list) + return AVERROR(ENOMEM); + + if (s->nb_find_list != s->nb_replace_list) { + av_log(ctx, AV_LOG_ERROR, "The number of words in 'find' and 'replace' needs to be equal\n"); + return AVERROR(EINVAL); + } + } + } + + return 0; +} + +static void uninit(AVFilterContext *ctx) +{ + TextModContext *s = ctx->priv; + + for (int i = 0; i < s->nb_find_list; i++) + av_freep(&s->find_list[i]); + + s->nb_find_list = 0; + av_freep(&s->find_list); + + for (int i = 0; i < s->nb_replace_list; i++) + av_freep(&s->replace_list[i]); + + s->nb_replace_list = 0; + av_freep(&s->replace_list); +} + +static char *process_text(TextModContext *s, char *text) +{ + const char *char_src = s->find; + const char *char_dst = s->replace; + char *result = NULL; + int escape_level = 0, k = 0; + + switch (s->operation) { + case OP_LEET: + case OP_REPLACE_CHARS: + + if (s->operation == OP_LEET) { + char_src = leet_src; + char_dst = leet_dst; + } + + result = av_strdup(text); + if (!result) + return NULL; + + for (size_t n = 0; n < strlen(result); n++) { + if (result[n] == '{') + escape_level++; + + if (!escape_level) { + size_t len = strlen(char_src); + for (size_t t = 0; t < len; t++) { + if (result[n] == char_src[t]) { + result[n] = char_dst[t]; + break; + } + } + } + + if (result[n] == '}') + escape_level--; + } + + break; + case OP_TO_UPPER: + case OP_TO_LOWER: + + result = av_strdup(text); + if (!result) + return NULL; + + for (size_t n = 0; n < strlen(result); n++) { + if (result[n] == '{') + escape_level++; + if (!escape_level) + result[n] = s->operation == OP_TO_LOWER ? av_tolower(result[n]) : av_toupper(result[n]); + if (result[n] == '}') + escape_level--; + } + + break; + case OP_REMOVE_CHARS: + + result = av_strdup(text); + if (!result) + return NULL; + + for (size_t n = 0; n < strlen(result); n++) { + int skip_char = 0; + + if (result[n] == '{') + escape_level++; + + if (!escape_level) { + size_t len = strlen(char_src); + for (size_t t = 0; t < len; t++) { + if (result[n] == char_src[t]) { + skip_char = 1; + break; + } + } + } + + if (!skip_char) + result[k++] = result[n]; + + if (result[n] == '}') + escape_level--; + } + + result[k] = 0; + + break; + case OP_REPLACE_WORDS: + case OP_REMOVE_WORDS: + + result = av_strdup(text); + if (!result) + return NULL; + + for (int n = 0; n < s->nb_find_list; n++) { + char *tmp = result; + const char *replace = (s->operation == OP_REPLACE_WORDS) ? s->replace_list[n] : ""; + + result = av_strireplace(result, s->find_list[n], replace); + if (!result) + return NULL; + + av_free(tmp); + } + + break; + } + + return result; +} + +static char *process_dialog_show_speaker(TextModContext *s, char *ass_line) +{ + ASSDialog *dialog = avpriv_ass_split_dialog(NULL, ass_line); + int escape_level = 0; + unsigned pos = 0, len; + char *result, *text; + AVBPrint pbuf; + + if (!dialog) + return NULL; + + text = process_text(s, dialog->text); + if (!text) + return NULL; + + if (!dialog->name || !strlen(dialog->name) || !dialog->text || !strlen(dialog->text)) + return av_strdup(ass_line); + + // Find insertion point in case the line starts with style codes + len = (unsigned)strlen(dialog->text); + for (unsigned i = 0; i < len; i++) { + + if (dialog->text[i] == '{') + escape_level++; + + if (dialog->text[i] == '}') + escape_level--; + + if (escape_level == 0) { + pos = i; + break; + } + } + + if (s->style && strlen(s->style)) + // When a style is specified reset the insertion point + // (always add speaker plus style at the start in that case) + pos = 0; + + if (pos >= len - 1) + return av_strdup(ass_line); + + av_bprint_init(&pbuf, 1, AV_BPRINT_SIZE_UNLIMITED); + + if (pos > 0) { + av_bprint_append_data(&pbuf, dialog->text, pos); + } + + if (s->style && strlen(s->style)) { + if (s->style[0] == '{') + // Assume complete and valid style code, e.g. {\c&HFF0000&} + av_bprintf(&pbuf, "%s", s->style); + else + // Otherwise it must be a style name + av_bprintf(&pbuf, "{\\r%s}", s->style); + } + + switch (s->speaker_mode) { + case SM_SQUARE_BRACKETS: + av_bprintf(&pbuf, "[%s]", dialog->name); + break; + case SM_ROUND_BRACKETS: + av_bprintf(&pbuf, "(%s)", dialog->name); + break; + case SM_COLON: + av_bprintf(&pbuf, "%s:", dialog->name); + break; + case SM_PLAIN: + av_bprintf(&pbuf, "%s", dialog->name); + break; + } + + if (s->style && strlen(s->style)) { + // Reset line style + if (dialog->style && strlen(dialog->style) && !av_strcasecmp(dialog->style, "default")) + av_bprintf(&pbuf, "{\\r%s}", dialog->style); + else + av_bprintf(&pbuf, "{\\r}"); + } + + if (s->line_break) + av_bprintf(&pbuf, "\\N"); + else + av_bprintf(&pbuf, " "); + + av_bprint_append_data(&pbuf, dialog->text + pos, len - pos); + + av_bprint_finalize(&pbuf, &text); + + result = avpriv_ass_get_dialog(dialog->readorder, dialog->layer, dialog->style, dialog->name, text); + + av_free(text); + avpriv_ass_free_dialog(&dialog); + return result; +} + +static char *process_dialog(TextModContext *s, char *ass_line) +{ + ASSDialog *dialog; + char *result, *text; + + if (s->filter_type == TM_SHOW_SPEAKER) + return process_dialog_show_speaker(s, ass_line); + + dialog = avpriv_ass_split_dialog(NULL, ass_line); + if (!dialog) + return NULL; + + text = process_text(s, dialog->text); + if (!text) + return NULL; + + result = avpriv_ass_get_dialog(dialog->readorder, dialog->layer, dialog->style, dialog->name, text); + + av_free(text); + avpriv_ass_free_dialog(&dialog); + return result; +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *frame) +{ + TextModContext *s = inlink->dst->priv; + AVFilterLink *outlink = inlink->dst->outputs[0]; + int ret; + + outlink->format = inlink->format; + + ret = av_frame_make_writable(frame); + if (ret < 0) + return ret; + + for (unsigned i = 0; i < frame->num_subtitle_areas; i++) { + + AVSubtitleArea *area = frame->subtitle_areas[i]; + + if (area->ass) { + char *tmp = area->ass; + area->ass = process_dialog(s, area->ass); + av_free(tmp); + if (!area->ass) + return AVERROR(ENOMEM); + } + } + + return ff_filter_frame(outlink, frame); +} + +#define OFFSET(x) offsetof(TextModContext, x) +#define FLAGS (AV_OPT_FLAG_SUBTITLE_PARAM | AV_OPT_FLAG_FILTERING_PARAM) + +static const AVOption textmod_options[] = { + { "mode", "set operation mode", OFFSET(operation), AV_OPT_TYPE_INT, {.i64=OP_LEET}, OP_LEET, NB_OPS-1, FLAGS, "mode" }, + { "leet", "convert text to 'leet speak'", 0, AV_OPT_TYPE_CONST, {.i64=OP_LEET}, 0, 0, FLAGS, "mode" }, + { "to_upper", "change to upper case", 0, AV_OPT_TYPE_CONST, {.i64=OP_TO_UPPER}, 0, 0, FLAGS, "mode" }, + { "to_lower", "change to lower case", 0, AV_OPT_TYPE_CONST, {.i64=OP_TO_LOWER}, 0, 0, FLAGS, "mode" }, + { "replace_chars", "replace characters", 0, AV_OPT_TYPE_CONST, {.i64=OP_REPLACE_CHARS}, 0, 0, FLAGS, "mode" }, + { "remove_chars", "remove characters", 0, AV_OPT_TYPE_CONST, {.i64=OP_REMOVE_CHARS}, 0, 0, FLAGS, "mode" }, + { "replace_words", "replace words", 0, AV_OPT_TYPE_CONST, {.i64=OP_REPLACE_WORDS}, 0, 0, FLAGS, "mode" }, + { "remove_words", "remove words", 0, AV_OPT_TYPE_CONST, {.i64=OP_REMOVE_WORDS}, 0, 0, FLAGS, "mode" }, + { "find", "chars/words to find or remove", OFFSET(find), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, FLAGS, NULL }, + { "find_file", "load find param from file", OFFSET(find_file), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, FLAGS, NULL }, + { "replace", "chars/words to replace", OFFSET(replace), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, FLAGS, NULL }, + { "replace_file", "load replace param from file", OFFSET(replace_file), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, FLAGS, NULL }, + { "separator", "word separator", OFFSET(separator), AV_OPT_TYPE_STRING, {.str = ","}, 0, 0, FLAGS, NULL }, + { NULL }, +}; + + +static const AVOption censor_options[] = { + { "mode", "set censoring mode", OFFSET(censor_mode), AV_OPT_TYPE_INT, {.i64=CM_KEEP_FIRST_LAST}, 0, 2, FLAGS, "mode" }, + { "keep_first_last", "censor inner chars", 0, AV_OPT_TYPE_CONST, {.i64=CM_KEEP_FIRST_LAST}, 0, 0, FLAGS, "mode" }, + { "keep_first", "censor all but first char", 0, AV_OPT_TYPE_CONST, {.i64=CM_KEEP_FIRST}, 0, 0, FLAGS, "mode" }, + { "all", "censor all chars", 0, AV_OPT_TYPE_CONST, {.i64=CM_ALL}, 0, 0, FLAGS, "mode" }, + { "words", "list of words to censor", OFFSET(find), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, FLAGS, NULL }, + { "words_file", "path to word list file", OFFSET(find_file), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, FLAGS, NULL }, + { "separator", "word separator", OFFSET(separator), AV_OPT_TYPE_STRING, {.str = ","}, 0, 0, FLAGS, NULL }, + { "censor_char", "replacement character", OFFSET(censor_char), AV_OPT_TYPE_STRING, {.str = "*"}, 0, 0, FLAGS, NULL }, + { NULL }, +}; + +static const AVOption showspeaker_options[] = { + { "format", "speaker name formatting", OFFSET(speaker_mode), AV_OPT_TYPE_INT, {.i64=SM_SQUARE_BRACKETS}, 0, 2, FLAGS, "format" }, + { "square_brackets", "[speaker] text", 0, AV_OPT_TYPE_CONST, {.i64=SM_SQUARE_BRACKETS}, 0, 0, FLAGS, "format" }, + { "round_brackets", "(speaker) text", 0, AV_OPT_TYPE_CONST, {.i64=SM_ROUND_BRACKETS}, 0, 0, FLAGS, "format" }, + { "colon", "speaker: text", 0, AV_OPT_TYPE_CONST, {.i64=SM_COLON}, 0, 0, FLAGS, "format" }, + { "plain", "speaker text", 0, AV_OPT_TYPE_CONST, {.i64=SM_PLAIN}, 0, 0, FLAGS, "format" }, + { "line_break", "insert line break", OFFSET(line_break), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, FLAGS, NULL }, + { "style", "ass type name or style code", OFFSET(style), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, FLAGS, NULL }, + { NULL }, +}; + +AVFILTER_DEFINE_CLASS(textmod); +AVFILTER_DEFINE_CLASS(censor); +AVFILTER_DEFINE_CLASS(showspeaker); + +static const AVFilterPad inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .filter_frame = filter_frame, + }, +}; + +static const AVFilterPad outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + }, +}; + +const AVFilter ff_sf_textmod = { + .name = "textmod", + .description = NULL_IF_CONFIG_SMALL("Modify subtitle text in several ways"), + .init = init, + .uninit = uninit, + .priv_size = sizeof(TextModContext), + .priv_class = &textmod_class, + FILTER_INPUTS(inputs), + FILTER_OUTPUTS(outputs), + FILTER_SINGLE_SUBFMT(AV_SUBTITLE_FMT_ASS), +}; + +const AVFilter ff_sf_censor = { + .name = "censor", + .description = NULL_IF_CONFIG_SMALL("Censor words in subtitle text"), + .init = init_censor, + .uninit = uninit, + .priv_size = sizeof(TextModContext), + .priv_class = &censor_class, + FILTER_INPUTS(inputs), + FILTER_OUTPUTS(outputs), + FILTER_SINGLE_SUBFMT(AV_SUBTITLE_FMT_ASS), +}; + +const AVFilter ff_sf_showspeaker = { + .name = "showspeaker", + .description = NULL_IF_CONFIG_SMALL("Prepend speaker names to text subtitles (when available)"), + .init = init_showspeaker, + .uninit = uninit, + .priv_size = sizeof(TextModContext), + .priv_class = &showspeaker_class, + FILTER_INPUTS(inputs), + FILTER_OUTPUTS(outputs), + FILTER_SINGLE_SUBFMT(AV_SUBTITLE_FMT_ASS), +}; From patchwork Mon Dec 13 23:40:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 32468 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp6141488iog; Mon, 13 Dec 2021 15:43:34 -0800 (PST) X-Google-Smtp-Source: ABdhPJyUW9iXf1vL8XbgpoepAmVsyW8sk7eRaZwvd4nSN3Kik+8jqlrKhI0K6+YDGlY2/l+MV65y X-Received: by 2002:a50:bb2a:: with SMTP id y39mr2731029ede.348.1639439014043; Mon, 13 Dec 2021 15:43:34 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id g9si16331511ejk.428.2021.12.13.15.43.33; Mon, 13 Dec 2021 15:43:34 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=uCYGKMFo; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7660A68B0ED; Tue, 14 Dec 2021 01:40:59 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11olkn2058.outbound.protection.outlook.com [40.92.18.58]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 8F3B368B0D8 for ; Tue, 14 Dec 2021 01:40:57 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=EUdkaBJOdMZAawwRJAvzIVDfgVSQayz4HTPA43Dpmy+ooj80JUxH96dGxTOQXzxg7phK6blnv4WGbzidCLwCb7WQ/btHClrWrfnGh1yF0b7585HtuoHgavEU5j0xj4JUI6EtxngS9Zh18zbJZoo8DuCeV/1USC3SEI0SBb5e4a9OJQejFeRcoiify1Dl2bpYAcYAMR7BUvqkJH8Tt7Ho5jOZAKFjW2C1L5UolLuIQMxg3asYnMK9vrZBDoEMgvQiFz2qc2NtOVVzTOtuKt7642L4ntZR50SdXzKvRACDsvFkqMxtCxLqr1rtpR67tZhfZQY/i8m62wdY3Q2t+f0cRw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8Ceo1xIugAWYE8IwIkGlYhuBlWyRCi9Hh3Dth0WQ7sM=; b=KV+Yjx7XYmFIS37rhh0XQ30Js1ue6qv2HHk2s52iMwi+bZkiDtPVPv+dwaEJarB8plrYH7zScg9bXGdU67CNorYM0agVs4hx5ShK31J0tWBDdbFzhCJM2RSihje4YOSxO5avEkHEq2LgUYby06iy9XWg1LDPCw0HM+v/7DWj1FO25Ly+Tz/2HWj5oHFMPArEWO3FHHoU3iG8sIzIE0tH+LcW9hdDu89qA4/1C8BRyppBUYLRul68+TopDls3K0xnZkXm6I1ZTGA89mibn5oomCly+ZNeOjxkTgELNTmhUSQkc+A0dhZowuyEq3ezoPpf4hUhbby3507OVm3OgWNP2g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8Ceo1xIugAWYE8IwIkGlYhuBlWyRCi9Hh3Dth0WQ7sM=; b=uCYGKMFoiJUmHRSNdLHkjkwc2x8d3icktENY8xDycgn9Yjlu7x841F/uSz8L2WNcwD2bMyaAkoJtr2fAXYft10JygGRTtdwL2cuvInHt0GXN0wHPIGNMXmaUEIsU/oaPlbM7D188K90t/V0PORtHmUOodGkEVzeK1G2HbqvEhL83gkLu6pGjw3zOnlePOxra/NNx40Ifhz8v0ES9BVM7GKsHlTlQfbVMeXgivdtSky+LntehMVOxei1JkhpKv779gxpI/+HBjzxz9gSckR66hwTfY107lo0msAbPCy9RxT8oV6fEkcG75cVsNL4aYa++qrSHvWsdo7aL6M0ks2vgAA== Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::20) by DM8P223MB0397.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4778.17; Mon, 13 Dec 2021 23:40:54 +0000 Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775]) by DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775%7]) with mapi id 15.20.4778.018; Mon, 13 Dec 2021 23:40:54 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v24 17/21] avfilter/stripstyles: Add stripstyles filter Thread-Index: AQHX8HreVY0gr8CwNkSsCoXbQeojNg== Date: Mon, 13 Dec 2021 23:40:54 +0000 Message-ID: References: <1ef42eb01b3627bbad539abd46206f320bb39697.1639438462.git.softworkz@hotmail.com> <4148d77a82d4b7e229fc81ce9bc1bc06904533f1.1639438462.git.softworkz@hotmail.com> <2ac798924ee984010d2e85f00bc992d9aa106171.1639438462.git.softworkz@hotmail.com> <300ea7188b331ca5e00dd7e04c03df8aba2a99c0.1639438462.git.softworkz@hotmail.com> <9474726b26f0d3e8f353d07caed2fbd33cdf473a.1639438462.git.softworkz@hotmail.com> <424522c96b5cdf6b03faa6bdceb1d25449463429.1639438462.git.softworkz@hotmail.com> <90caff245055005e6a4a890b3665da55aa853b31.1639438462.git.softworkz@hotmail.com> <0ebcf3ed2adb02d3ffb243a499b434c3bdd976a7.1639438462.git.softworkz@hotmail.com> <4b5c1c1185075fd6e5d8ef70e07fdce6f896d699.1639438462.git.softworkz@hotmail.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [4JQ1hbcnzbPmg/cm3wG6aorSBPwx6HH8] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 86202cd9-f856-47c8-0c34-08d9be92016a x-ms-traffictypediagnostic: DM8P223MB0397:EE_ x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 1ScyDtPLbk+suXvYrRpBrkPAFyb+ZIKxkfkdDrSLCeZ/Plx2B26ATu+mL2u0JUEtjK8qxB8gLKDRcAg2DAzdsSS+tv5614h3kzVFEpc9qwDOZ04hgZn1jXuVwvmsjcVPrn1TBTXClOLvmxZyXIDLgXz2pBsGRGVrBWebPE+BipZCGSQsIrC+DfaA6TY21Bqiq5kvwdHHjT3/6E5hZzRifo0pEAlU+2RRuto5U8e7bRt8penFKAmCGu2AnpbBz3K1ksAJb5PUNCPGMXVImZeLmJotrXw2lhc8wFBiAy8znl5esn1ajl3Ms9agfyVFfprYYFzX97ceFSwHpPGjVDC43ZsZIXaHHom3X6WEOVNTbYFC/jyTfkwg80kXdyl1YO7gdtXRSjablsrMsElw4sMUIrrlR6dbUv9uJI1Nb0hQwrBlrIFpsNOso9WhFr1Ngt9X75MLECioHIuVdCeOWonuGwIY9eBZWH0Fpy9T9JoRdG77flftupwOcCNuL7iDgMSb0Nssss6vqDhcXZXJ/91JNbiIwYwgYt7qxl/UZYO1cI88n8JTgTo7uZwI/pGNg50GbIGIYiMh6WvEuoyc2LlLyVdh+9ZE7Z84lXbcR57R+OTPeLRmONbaGeGSNbbsiUQxLCw/0ZlJx6+6/QZ0pElZcCQPv2Daj1cgIEVxRHEFOY4= x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?q?yx1Q7A50AJnjK5KCGAa0f6p/j46v?= =?utf-8?q?+MhXe95zvTw7/YCW5jowZWjwI0R3Nf2KhUTQgLPz79iVnHHHAaWNw9cblYWNQnV2S?= =?utf-8?q?yp1bJy0fyJWQ0URR6f1BSG+ahERvGU8cz6zXY3ZqYks/czU3eW5RbaQDnQZAp5U6u?= =?utf-8?q?a0j6X6t8ObITxXaKiuL4aEWy9PoTWq49TTB7xxzJ1f/0dLOiabdWDnDQfRS/KFDk5?= =?utf-8?q?7aWUSAer/cfiZ4kW70ug75pyUWofM+k4/HTmir8+iCahjsVlfoP4KiJ6/dUtM3kW4?= =?utf-8?q?h3cfoZypfnfL8gh7nxG8UchOMy8yrK1f0zfQwCmHqJTkKUAYXojB5Baa5c0+4GOGH?= =?utf-8?q?s4qUDRZWdcDscY+rPFDJuAm+1TplpCURqVMBTQpshmviXhNssbKvzxFoyauWJX38z?= =?utf-8?q?Br5f+xhmfL0jDNrhuxloocnjaCa3kbfOr6IGonlPn/eevWq5DligwSluCque6qKul?= =?utf-8?q?/pjxS+gR7WW4Y2/rohgnWtIyzZ1UG1cRXXdeZN4QtLrl6Kd+WLRiV/j3lOJWPuxYc?= =?utf-8?q?F+ytRjFpoTVLO6zNPy8jwy08NV88W0QfmmIeFAwj1WUnRSuTt627SWQAAnLiVi+ab?= =?utf-8?q?xiVfQpq1s0Uh37ncsTV/iLeutezdZP6JNlEsOtLg/W8+u08rkmlWL7IUKYKZ6Kamz?= =?utf-8?q?dKDkbT4VqGBV/uiZdujZZQjv0jXoZB46lSoKfytYVVJJOs/yZrCyNZHOfcmefD2q7?= =?utf-8?q?u8Zz3J5CEgz25tkncFzAB1GztiU1FiT1xhGnOls1gyVa9ncAWPV3YgNvNCOIJYE4u?= =?utf-8?q?PiTrcJFK/PS93C8bX+5C/aFfFCk7NKjO+f4YglL0/Gx0u0qNPOT29Z09G5Q49V46J?= =?utf-8?q?ruy/ybftM669SH6XJnQxlJBUoW3eZW4GcKwpMOLBNV2UAXAio+ELVEb9yFBuYTMY0?= =?utf-8?q?4r30bmDa6sxoifNiKxSMID3MZv4mkNHK2DeBA5rPcS0Q=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-1ff67.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 86202cd9-f856-47c8-0c34-08d9be92016a X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Dec 2021 23:40:54.6899 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8P223MB0397 Subject: [FFmpeg-devel] [PATCH v24 17/21] avfilter/stripstyles: Add stripstyles filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: r7ZzqEhpQOrq - stripstyles {S -> S) Remove all inline styles from subtitle events Signed-off-by: softworkz --- doc/filters.texi | 37 +++++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/sf_stripstyles.c | 196 +++++++++++++++++++++++++++++++++++ 4 files changed, 235 insertions(+) create mode 100644 libavfilter/sf_stripstyles.c diff --git a/doc/filters.texi b/doc/filters.texi index 6ef95c4926..4806132cc1 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -25780,6 +25780,43 @@ ffmpeg -i "http://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex. @end example @end itemize +@section stripstyles + +Remove all inline styles from subtitle events. + +Inputs: +@itemize +@item 0: Subtitles[TEXT] +@end itemize + +Outputs: +@itemize +@item 0: Subtitles[TEXT] +@end itemize + +It accepts the following parameters: + +@table @option +@item remove_animated +Also remove text which is subject to animation (default: true) +Usually, animated text elements are used used in addition to static subtitle lines for creating effects, so in most cases it is safe to remove the animation content. +If subtitle text is missing, try setting this to false. + +@item select_layer +Process only ASS subtitle events from a specific layer. This allows to filter out certain effects where an ASS author duplicates the text onto multiple layers. + +@end table + +@subsection Examples + +@itemize +@item +Remove styles and animations from ASS subtitles and output events from ass layer 0 only. Then convert asn save as SRT stream: +@example +ffmpeg -i "https://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.mkv" -filter_complex "[0:1]stripstyles=select_layer=0" -map 0 -c:s srt output.mkv +@end example +@end itemize + @section textmod diff --git a/libavfilter/Makefile b/libavfilter/Makefile index d47bdfa8f6..d69fa61c91 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -559,6 +559,7 @@ OBJS-$(CONFIG_NULLSINK_FILTER) += vsink_nullsink.o OBJS-$(CONFIG_CENSOR_FILTER) += sf_textmod.o OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o OBJS-$(CONFIG_TEXTMOD_FILTER) += sf_textmod.o +OBJS-$(CONFIG_STRIPSTYLES_FILTER) += sf_stripstyles.o # multimedia filters OBJS-$(CONFIG_ABITSCOPE_FILTER) += avf_abitscope.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index cc8f43c925..32a0ebe71b 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -548,6 +548,7 @@ extern const AVFilter ff_avf_showwavespic; extern const AVFilter ff_vaf_spectrumsynth; extern const AVFilter ff_sf_censor; extern const AVFilter ff_sf_showspeaker; +extern const AVFilter ff_sf_stripstyles; extern const AVFilter ff_sf_textmod; extern const AVFilter ff_svf_graphicsub2video; extern const AVFilter ff_svf_textsub2video; diff --git a/libavfilter/sf_stripstyles.c b/libavfilter/sf_stripstyles.c new file mode 100644 index 0000000000..82cb9c7647 --- /dev/null +++ b/libavfilter/sf_stripstyles.c @@ -0,0 +1,196 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * text subtitle filter which removes inline-styles from subtitles + */ + +#include "libavutil/opt.h" +#include "internal.h" +#include "libavutil/ass_split_internal.h" +#include "libavutil/bprint.h" + +typedef struct StripStylesContext { + const AVClass *class; + enum AVSubtitleType format; + int remove_animated; + int select_layer; +} StripStylesContext; + +typedef struct DialogContext { + StripStylesContext* ss_ctx; + AVBPrint buffer; + int drawing_scale; + int is_animated; +} DialogContext; + +static void dialog_text_cb(void *priv, const char *text, int len) +{ + DialogContext *s = priv; + + av_log(s->ss_ctx, AV_LOG_DEBUG, "dialog_text_cb: %s\n", text); + + if (!s->drawing_scale && (!s->is_animated || !s->ss_ctx->remove_animated)) + av_bprint_append_data(&s->buffer, text, len); +} + +static void dialog_new_line_cb(void *priv, int forced) +{ + DialogContext *s = priv; + if (!s->drawing_scale && !s->is_animated) + av_bprint_append_data(&s->buffer, forced ? "\\N" : "\\n", 2); +} + +static void dialog_drawing_mode_cb(void *priv, int scale) +{ + DialogContext *s = priv; + s->drawing_scale = scale; +} + +static void dialog_animate_cb(void *priv, int t1, int t2, int accel, char *style) +{ + DialogContext *s = priv; + s->is_animated = 1; +} + +static void dialog_move_cb(void *priv, int x1, int y1, int x2, int y2, int t1, int t2) +{ + DialogContext *s = priv; + if (t1 >= 0 || t2 >= 0) + s->is_animated = 1; +} + +static const ASSCodesCallbacks dialog_callbacks = { + .text = dialog_text_cb, + .new_line = dialog_new_line_cb, + .drawing_mode = dialog_drawing_mode_cb, + .animate = dialog_animate_cb, + .move = dialog_move_cb, +}; + +static char *ass_get_line(int readorder, int layer, const char *style, + const char *speaker, const char *effect, const char *text) +{ + return av_asprintf("%d,%d,%s,%s,0,0,0,%s,%s", + readorder, layer, style ? style : "Default", + speaker ? speaker : "", effect, text); +} + +static char *process_dialog(StripStylesContext *s, const char *ass_line) +{ + DialogContext dlg_ctx = { .ss_ctx = s }; + ASSDialog *dialog = avpriv_ass_split_dialog(NULL, ass_line); + char *result = NULL; + + if (!dialog) + return NULL; + + if (s->select_layer >= 0 && dialog->layer != s->select_layer) + return NULL; + + dlg_ctx.ss_ctx = s; + + av_bprint_init(&dlg_ctx.buffer, 0, AV_BPRINT_SIZE_UNLIMITED); + + avpriv_ass_split_override_codes(&dialog_callbacks, &dlg_ctx, dialog->text); + + if (av_bprint_is_complete(&dlg_ctx.buffer) + && dlg_ctx.buffer.len > 0) + result = ass_get_line(dialog->readorder, dialog->layer, dialog->style, dialog->name, dialog->effect, dlg_ctx.buffer.str); + + av_bprint_finalize(&dlg_ctx.buffer, NULL); + avpriv_ass_free_dialog(&dialog); + + return result; +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *frame) +{ + StripStylesContext *s = inlink->dst->priv; + AVFilterLink *outlink = inlink->dst->outputs[0]; + int ret; + + outlink->format = inlink->format; + + ret = av_frame_make_writable(frame); + if (ret <0 ) { + av_frame_free(&frame); + return AVERROR(ENOMEM); + } + + for (unsigned i = 0; i < frame->num_subtitle_areas; i++) { + + AVSubtitleArea *area = frame->subtitle_areas[i]; + + if (area->ass) { + char *tmp = area->ass; + area->ass = process_dialog(s, area->ass); + + if (area->ass) { + av_log(inlink->dst, AV_LOG_INFO, "original: %d %s\n", i, tmp); + av_log(inlink->dst, AV_LOG_INFO, "stripped: %d %s\n", i, area->ass); + } + else + area->ass = NULL; + + av_free(tmp); + } + } + + return ff_filter_frame(outlink, frame); +} + +#define OFFSET(x) offsetof(StripStylesContext, x) +#define FLAGS (AV_OPT_FLAG_SUBTITLE_PARAM | AV_OPT_FLAG_FILTERING_PARAM) + +static const AVOption stripstyles_options[] = { + { "remove_animated", "remove animated text (default: yes)", OFFSET(remove_animated), AV_OPT_TYPE_BOOL, {.i64 = 1 }, 0, 1, FLAGS, 0 }, + { "select_layer", "process a specific ass layer only", OFFSET(remove_animated), AV_OPT_TYPE_INT, {.i64 = -1 }, -1, INT_MAX, FLAGS, 0 }, + { NULL }, +}; + +AVFILTER_DEFINE_CLASS(stripstyles); + +static const AVFilterPad inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .filter_frame = filter_frame, + }, +}; + +static const AVFilterPad outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + }, +}; + +const AVFilter ff_sf_stripstyles = { + .name = "stripstyles", + .description = NULL_IF_CONFIG_SMALL("Strip subtitle inline styles"), + .priv_size = sizeof(StripStylesContext), + .priv_class = &stripstyles_class, + FILTER_INPUTS(inputs), + FILTER_OUTPUTS(outputs), + FILTER_SINGLE_SUBFMT(AV_SUBTITLE_FMT_ASS), +}; + From patchwork Mon Dec 13 23:40:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 32469 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp6141680iog; Mon, 13 Dec 2021 15:43:45 -0800 (PST) X-Google-Smtp-Source: ABdhPJx4TB6U9wQoU5ruvg/4A9L61e39YYzfRP4LWmRFkrcXG96Rcu1I/rEAXXxmMsUB5RB/6pdH X-Received: by 2002:a17:906:3a4a:: with SMTP id a10mr1596504ejf.253.1639439024859; Mon, 13 Dec 2021 15:43:44 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id b20si16664982ejj.624.2021.12.13.15.43.44; Mon, 13 Dec 2021 15:43:44 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=Xi9jn+CA; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6CC9868B0F2; Tue, 14 Dec 2021 01:41:00 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11olkn2058.outbound.protection.outlook.com [40.92.18.58]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 42FD668B0D7 for ; Tue, 14 Dec 2021 01:40:58 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=lytMksscYuzIKbEJ35T2WusPswczfaFbaSnSvbIGzAwBqkfDl93TTmvnfHKs+0YlWdbmDt1Pp2L6Kwbl1c5ytE1Iunt3FWRh5ojYyxugIKL0spO87eXPi0aUOsFOD+foukGRHNd3dFUOhvV1oMzpjKtqg7a4SJ/GOwNlJf91AiIsrTbcGprFE+Ct/dHTL6rpcKHvkdydk1iqND05uuZO7lj/VR/uOpbVA9i1BY3pXz4aI+KzZGaNB8Dvr5VDINrEZTZskXlBZEH6S8WboAE4yScA24dRAwZrfrKikEW4bLtNN2SsHrfxDamST33qCHaIpHSik5YIlW+jyLQiPto9+g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=692U+h0A6T3oMOnhcIWerl+PPemvkotFkJ/n80t/QHw=; b=MN3R/9Wh/wC+MVZVOInlu/GTKItSYoez7SZckFmxNbuny48Lw5YDtNMV3J8fG3Q5HHFIDDT5xKNuLy/f63/Xj6xDfEbFr/ka9T/X1RrAlBpWQuXlb6M4dEQQGP2X98zdDj8PdZf1IE62N3zxIsLDU34KlyuJxSDKyIdwrRVd/hJ1B42sqV4yPkdoR4ODoB5qN9ZV617D1sDHZwXcn7XpFouJedoxjn4GamRao2MBzZkCFkd47UQ4vu0jkBixjxXdpCZExtYAATopSvad/zr21kcDn+3rbT7pLfqH6k8GbZ/iihc59bHgHPoljRPgv/t0wlYYPohij6zdz70ETXKigg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=692U+h0A6T3oMOnhcIWerl+PPemvkotFkJ/n80t/QHw=; b=Xi9jn+CANgSk1+us+03ViIHCf82kNhLCIRj7rNz9lrWQjZ5au4Ybhjbd9bsi3NhYHiYV8DKiGg2nTSI/l+S+xFDoDvBrTplV4avhna55ulvuF7zubDL+NAbb7oSM3ubxQPSMS3U6d5vkwvb8N8fzaE1ghV0KESL36gJdIby+1rKIXckbNTASs0+Cn0wOnodGTUW37MnZT5fTR6FFu1TI1DS2R1Ivala+kF/B6eKchDQ8ZfUMhE0vzROoMBtpZLbbWtsdcxrlcKIaxTkcNJf9SCObIXyZGRnvQH5Ablx37XKqpJywdBEQrIMVa6gA2YeyJZDk26m34LQwQQCq6ko/bA== Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::20) by DM8P223MB0397.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4778.17; Mon, 13 Dec 2021 23:40:56 +0000 Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775]) by DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775%7]) with mapi id 15.20.4778.018; Mon, 13 Dec 2021 23:40:56 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v24 18/21] avfilter/splitcc: Add splitcc filter for closed caption handling Thread-Index: AQHX8Hrg8AyHJcafRUSlPLBq99AOiw== Date: Mon, 13 Dec 2021 23:40:56 +0000 Message-ID: References: <1ef42eb01b3627bbad539abd46206f320bb39697.1639438462.git.softworkz@hotmail.com> <4148d77a82d4b7e229fc81ce9bc1bc06904533f1.1639438462.git.softworkz@hotmail.com> <2ac798924ee984010d2e85f00bc992d9aa106171.1639438462.git.softworkz@hotmail.com> <300ea7188b331ca5e00dd7e04c03df8aba2a99c0.1639438462.git.softworkz@hotmail.com> <9474726b26f0d3e8f353d07caed2fbd33cdf473a.1639438462.git.softworkz@hotmail.com> <424522c96b5cdf6b03faa6bdceb1d25449463429.1639438462.git.softworkz@hotmail.com> <90caff245055005e6a4a890b3665da55aa853b31.1639438462.git.softworkz@hotmail.com> <0ebcf3ed2adb02d3ffb243a499b434c3bdd976a7.1639438462.git.softworkz@hotmail.com> <4b5c1c1185075fd6e5d8ef70e07fdce6f896d699.1639438462.git.softworkz@hotmail.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [jVPCuWvgdN6TeSxNgSBaO/oPmARjAsjX] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 9ee9760e-1401-46fd-af47-08d9be920291 x-ms-traffictypediagnostic: DM8P223MB0397:EE_ x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: hWogvYO0mlo6HtnI4zJkDCoLkn3Nu8Pjz+wajc5M4eByiuiyL/Lp+GoMrhIP6rMnvRAM/0KYnrVLTXh/Gcb4e9z9KzrcxAhLpJheps0kuwZqNCMi64T/5F8Sz3Gc2L9kDYOx8R2RstWsaG6mhTZLhA+3qBONXpAOevYaTBOjmjBIot4lGoYZifFgFj4HoKw382McCIq+5EGZhQcCQgTU9XbTWpby11T4R9Ur5XG/2TS1QN6ScQ+HLKkNzpsXJ950yY3V/My9MFasfafUz6hB+EmuFrZ2IwdGiT56Hmdr7lC+WEu2At/VxSGm4L9XWKheT+1OlN0Z3F63HS9Rk5eJuKDt8iVsJrSiJ2UJxe+ST1EZwKiUg91AjKmbp3wJXBp494DBB3JXXh9rVw5SEgnFs49w5fYuZCE1wzv0lTdGY5vDEAS+YNuAuvrT6XRDWIyij/MhU9RfkNTPch1KJxBOBJJWM7LAmjGIwyJ4uGA5qLuaMOv2TqQsJhCBItdGlIHVtZNPVLZtmVSAk/pxUkiyCtqUA+S9hcbKpleyl07I46rP/smMgPnGKg4PRqgrE1V1RHLaoizHA2nSxOMXuMfDJbbh1fii5zyWoqvtuN9jvbAyPjTAuJ095aQswLXWdRdK x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?q?oLyioTYuIR9me+mNzuWMt4OaDcik?= =?utf-8?q?dQl3n3x8tRBkpk/iCdShl746+xnzu3cwPO7AGYi084meDXDcLdLds8wnIWHDehoAB?= =?utf-8?q?hspMX/EQptEGqKRVG3zWI04maUEeFpH+9csCAT+rcLkaGxuME5B9cny0vExA8pLTV?= =?utf-8?q?iLDDKS505HP9YWe5TqQvcpGpxo5yU1IxiAfEYO3T9Q6fAPtdUML9EJvg5zE2YIFmT?= =?utf-8?q?cSrJHGFjxbDeaRILywCNasFpp5IakkvnnxmraGanigB6Yyrju2X4uAAdqW27cgDhQ?= =?utf-8?q?9dvcQ6ef89FmdPbN7eeE+oqoQFLqwDtnKmeIAMZhtLs9+svMH3CF+4qqHzsEdskDC?= =?utf-8?q?5vOwJ0WUW5+uAFHSF6uzPjpStKn9FW7igxHQ9QjbhvRdjT7Xm99NvZfy5u9/nNN3K?= =?utf-8?q?cIJKL64WJrwCNbE1zmH7rkk78zOFKg+LbZ+z31m5NRVPqdGAlMOR7zWldGyiTvDpJ?= =?utf-8?q?g1qqBBRiL6KR4Dy3s21ZIIzFwfIkoC5zA/gCJZ52wv7ZWW0y6mP/hiJfGh98YsOVh?= =?utf-8?q?UI55hOnZE875tU+JoJClX39jpmPCJh8VkaTMH7BobsCNt5C3SZUPDu4GOIYX/PvEo?= =?utf-8?q?+N66BuhLFFODPs80aRzSVn9q5242ZVsXYiE1Bm0lkUy0rzAdKJfhAKQROtaikbtPF?= =?utf-8?q?RV1yM5O0MzM+ylk2jtvsh1/nOr7x249QzIFLnDpnENhuhvqgESaDlhsMIJtUxJlG4?= =?utf-8?q?8wbxkBhMiCuuMyw0jlzbOCk5SplQugI9MutFi5E1WY8lP5V60sD7AllUsT6FTv3nq?= =?utf-8?q?ZgyRSjBKjqFlQA7JfdF3SWV3e0fXoqRgdaN7IjmlKf3CDfIshN/mboFKtZPreTT2z?= =?utf-8?q?JogVcrUuFMc4gSxrn52Qd0laqy8mBJQYRqOQ7XN7AXaozeIvptvF+9c5It5mOhNlK?= =?utf-8?q?ps/RLTqZ4Q5teW8pfP5rXWq9a8//Wo98PmLuI/LPUU7g=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-1ff67.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 9ee9760e-1401-46fd-af47-08d9be920291 X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Dec 2021 23:40:56.6812 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8P223MB0397 Subject: [FFmpeg-devel] [PATCH v24 18/21] avfilter/splitcc: Add splitcc filter for closed caption handling X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: dwlBcIskhExs - splitcc {V -> VS) Extract closed-caption (A53) data from video frames as subtitle Frames ffmpeg -y -loglevel verbose -i "https://streams.videolan.org/streams/ts/CC/NewsStream-608-ac3.ts" -filter_complex "[0:v]splitcc[vid1],textmod=mode=remove_chars:find='@',[vid1]overlay_textsubs" output.mkv Signed-off-by: softworkz --- configure | 1 + doc/filters.texi | 63 +++++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/sf_splitcc.c | 378 +++++++++++++++++++++++++++++++++++++++ 5 files changed, 444 insertions(+) create mode 100644 libavfilter/sf_splitcc.c diff --git a/configure b/configure index 7f67a62b08..a6dad8fe9d 100755 --- a/configure +++ b/configure @@ -3704,6 +3704,7 @@ spp_filter_select="fft idctdsp fdctdsp me_cmp pixblockdsp" sr_filter_deps="avformat swscale" sr_filter_select="dnn" stereo3d_filter_deps="gpl" +splitcc_filter_deps="avcodec" subtitles_filter_deps="avformat avcodec libass" super2xsai_filter_deps="gpl" pixfmts_super2xsai_test_deps="super2xsai_filter" diff --git a/doc/filters.texi b/doc/filters.texi index 4806132cc1..53c7e91ef3 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -26130,6 +26130,69 @@ ffmpeg -i INPUT -filter_complex "showspeaker=format=colon:style='@{\\c&HDD0000&\ @end example @end itemize + +@section splitcc + +Split-out closed-caption/A53 subtitles from video frame side data. + +This filter provides an input and an output for video frames, which are just passed through without modification. +The second out provides subtitle frames which are extracted from video frame side data. + +Inputs: +@itemize +@item 0: Video [ALL] +@end itemize + +Outputs: +@itemize +@item 0: Video (same as input) +@item 1: Subtitles [TEXT] +@end itemize + +It accepts the following parameters: + +@table @option + +@item use_cc_styles +Emit closed caption style header. +This will make closed captions appear in white font with a black rectangle background. + +@item real_time +Emit subtitle events as they are decoded for real-time display. + +@item real_time_latency_msec +Minimum elapsed time between emitting real-time subtitle events. +Only applies to real_time mode. + +@item data_field +Select data field. Possible values: + +@table @samp +@item auto +Pick first one that appears. +@item first +@item second +@end table + +@end table + +@subsection Examples + +@itemize +@item +Extract closed captions as text subtitle stream and overlay it onto the video in cc style (black bar background): +@example +ffmpeg -i "https://streams.videolan.org/streams/ts/CC/NewsStream-608-ac3.ts" -filter_complex "[0:v:0]splitcc=use_cc_styles=1[vid1][sub1];[vid1][sub1]overlaytextsubs" output.mkv +@end example + +@item +A nicer variant, using realtime output from cc_dec and rendering it with the render_latest_only parameter from overlaytextsubs to avoid ghosting by timely overlap. +@example +ffmpeg -i "https://streams.videolan.org/streams/ts/CC/NewsStream-608-ac3.ts" -filter_complex "[0:v:0]splitcc=real_time=1:real_time_latency_msec=200[vid1][sub1];[vid1][sub1]overlaytextsubs=render_latest_only=1" output.mkv +@end example +@end itemize + + @section textsub2video Converts text subtitles to video frames. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index d69fa61c91..9477b56649 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -559,6 +559,7 @@ OBJS-$(CONFIG_NULLSINK_FILTER) += vsink_nullsink.o OBJS-$(CONFIG_CENSOR_FILTER) += sf_textmod.o OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o OBJS-$(CONFIG_TEXTMOD_FILTER) += sf_textmod.o +OBJS-$(CONFIG_SPLITCC_FILTER) += sf_splitcc.o OBJS-$(CONFIG_STRIPSTYLES_FILTER) += sf_stripstyles.o # multimedia filters diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 32a0ebe71b..65b6bea650 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -548,6 +548,7 @@ extern const AVFilter ff_avf_showwavespic; extern const AVFilter ff_vaf_spectrumsynth; extern const AVFilter ff_sf_censor; extern const AVFilter ff_sf_showspeaker; +extern const AVFilter ff_sf_splitcc; extern const AVFilter ff_sf_stripstyles; extern const AVFilter ff_sf_textmod; extern const AVFilter ff_svf_graphicsub2video; diff --git a/libavfilter/sf_splitcc.c b/libavfilter/sf_splitcc.c new file mode 100644 index 0000000000..3556d084d9 --- /dev/null +++ b/libavfilter/sf_splitcc.c @@ -0,0 +1,378 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * subtitle filter for splitting out closed-caption/A53 subtitles from video frame side data + */ + +#include "filters.h" +#include "libavutil/opt.h" +#include "subtitles.h" +#include "libavcodec/avcodec.h" + +static const AVRational ms_tb = {1, 1000}; + +typedef struct SplitCaptionsContext { + const AVClass *class; + enum AVSubtitleType format; + AVCodecContext *cc_dec; + int eof; + AVFrame *next_sub_frame; + AVFrame *empty_sub_frame; + int new_frame; + int64_t next_repetition_pts; + AVBufferRef *subtitle_header; + int use_cc_styles; + int real_time; + int real_time_latency_msec; + int data_field; + int scatter_realtime_output; +} SplitCaptionsContext; + +static int init(AVFilterContext *ctx) +{ + SplitCaptionsContext *s = ctx->priv; + AVDictionary *options = NULL; + + int ret; + const AVCodec *codec = avcodec_find_decoder(AV_CODEC_ID_EIA_608); + if (!codec) { + av_log(ctx, AV_LOG_ERROR, "failed to find EIA-608/708 decoder\n"); + return AVERROR_DECODER_NOT_FOUND; + } + + if (!((s->cc_dec = avcodec_alloc_context3(codec)))) { + av_log(ctx, AV_LOG_ERROR, "failed to allocate EIA-608/708 decoder\n"); + return AVERROR(ENOMEM); + } + + av_dict_set_int(&options, "real_time", s->real_time, 0); + av_dict_set_int(&options, "real_time_latency_msec", s->real_time_latency_msec, 0); + av_dict_set_int(&options, "data_field", s->data_field, 0); + + if ((ret = avcodec_open2(s->cc_dec, codec, &options)) < 0) { + av_log(ctx, AV_LOG_ERROR, "failed to open EIA-608/708 decoder: %i\n", ret); + return ret; + } + + if (s->use_cc_styles && s->cc_dec->subtitle_header && s->cc_dec->subtitle_header[0] != 0) { + char* subtitle_header = av_strdup((char *)s->cc_dec->subtitle_header); + if (!subtitle_header) + return AVERROR(ENOMEM); + s->subtitle_header = av_buffer_create((uint8_t *)subtitle_header, strlen(subtitle_header) + 1, NULL, NULL, 0); + if (!s->subtitle_header) { + av_free(subtitle_header); + return AVERROR(ENOMEM); + } + } + + return 0; +} + +static void uninit(AVFilterContext *ctx) +{ + SplitCaptionsContext *s = ctx->priv; + av_frame_free(&s->next_sub_frame); + av_frame_free(&s->empty_sub_frame); + av_buffer_unref(&s->subtitle_header); +} + +static int config_input(AVFilterLink *link) +{ + const SplitCaptionsContext *context = link->dst->priv; + + if (context->cc_dec) + context->cc_dec->pkt_timebase = link->time_base; + + return 0; +} + +static int query_formats(AVFilterContext *ctx) +{ + AVFilterFormats *formats; + AVFilterLink *inlink0 = ctx->inputs[0]; + AVFilterLink *outlink0 = ctx->outputs[0]; + AVFilterLink *outlink1 = ctx->outputs[1]; + static const enum AVSubtitleType subtitle_fmts[] = { AV_SUBTITLE_FMT_ASS, AV_SUBTITLE_FMT_NB }; + int ret; + + /* set input0 video formats */ + formats = ff_all_formats(AVMEDIA_TYPE_VIDEO); + if ((ret = ff_formats_ref(formats, &inlink0->outcfg.formats)) < 0) + return ret; + + /* set output0 video formats */ + if ((ret = ff_formats_ref(formats, &outlink0->incfg.formats)) < 0) + return ret; + + /* set output1 subtitle formats */ + formats = ff_make_format_list(subtitle_fmts); + if ((ret = ff_formats_ref(formats, &outlink1->incfg.formats)) < 0) + return ret; + + return 0; +} + +static int config_video_output(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + const AVFilterLink *inlink = outlink->src->inputs[0]; + + outlink->w = ctx->inputs[0]->w; + outlink->h = ctx->inputs[0]->h; + outlink->frame_rate = ctx->inputs[0]->frame_rate; + outlink->time_base = ctx->inputs[0]->time_base; + outlink->sample_aspect_ratio = ctx->inputs[0]->sample_aspect_ratio; + + if (inlink->hw_frames_ctx) + outlink->hw_frames_ctx = av_buffer_ref(inlink->hw_frames_ctx); + + return 0; +} + +static int config_sub_output(AVFilterLink *outlink) +{ + const AVFilterLink *inlink = outlink->src->inputs[0]; + + outlink->time_base = inlink->time_base; + outlink->format = AV_SUBTITLE_FMT_ASS; + outlink->frame_rate = (AVRational){5, 1}; + + return 0; +} + +static int request_sub_frame(AVFilterLink *outlink) +{ + SplitCaptionsContext *s = outlink->src->priv; + int status; + int64_t pts; + + if (!s->empty_sub_frame) { + s->empty_sub_frame = ff_get_subtitles_buffer(outlink, outlink->format); + if (!s->empty_sub_frame) + return AVERROR(ENOMEM); + } + + if (!s->eof && ff_inlink_acknowledge_status(outlink->src->inputs[0], &status, &pts)) { + if (status == AVERROR_EOF) + s->eof = 1; + } + + if (s->eof) + return AVERROR_EOF; + + if (s->next_sub_frame) { + + AVFrame *out = NULL; + s->next_sub_frame->pts++; + + if (s->new_frame) + out = av_frame_clone(s->next_sub_frame); + else if (s->empty_sub_frame) { + s->empty_sub_frame->pts = s->next_sub_frame->pts; + out = av_frame_clone(s->empty_sub_frame); + av_frame_copy_props(out, s->next_sub_frame); + } + + if (!out) + return AVERROR(ENOMEM); + + out->subtitle_pts = av_rescale_q(s->next_sub_frame->pts, outlink->time_base, AV_TIME_BASE_Q); + s->new_frame = 0; + + return ff_filter_frame(outlink, out); + } + + return 0; +} + +static int decode(AVCodecContext *avctx, AVFrame *frame, int *got_frame, AVPacket *pkt) +{ + int ret; + + *got_frame = 0; + + if (pkt) { + ret = avcodec_send_packet(avctx, pkt); + // In particular, we don't expect AVERROR(EAGAIN), because we read all + // decoded frames with avcodec_receive_frame() until done. + if (ret < 0 && ret != AVERROR_EOF) + return ret; + } + + ret = avcodec_receive_frame(avctx, frame); + if (ret < 0 && ret != AVERROR(EAGAIN)) + return ret; + if (ret >= 0) + *got_frame = 1; + + return 0; +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *frame) +{ + AVFrameSideData *sd; + SplitCaptionsContext *s = inlink->dst->priv; + AVFilterLink *outlink0 = inlink->dst->outputs[0]; + AVFilterLink *outlink1 = inlink->dst->outputs[1]; + AVPacket *pkt = NULL; + AVFrame *sub_out = NULL; + + int ret; + + outlink0->format = inlink->format; + + sd = av_frame_get_side_data(frame, AV_FRAME_DATA_A53_CC); + + if (sd) { + int got_output = 0; + + pkt = av_packet_alloc(); + pkt->buf = av_buffer_ref(sd->buf); + if (!pkt->buf) { + ret = AVERROR(ENOMEM); + goto fail; + } + + pkt->data = sd->data; + pkt->size = (int)sd->size; + pkt->pts = frame->pts; + + sub_out = ff_get_subtitles_buffer(outlink1, AV_SUBTITLE_FMT_ASS); + if (!sub_out) { + ret = AVERROR(ENOMEM); + goto fail; + } + + if ((ret = av_buffer_replace(&sub_out->subtitle_header, s->subtitle_header)) < 0) + goto fail; + + ret = decode(s->cc_dec, sub_out, &got_output, pkt); + + if (ret < 0) + goto fail; + + if (got_output) { + sub_out->pts = frame->pts; + av_frame_free(&s->next_sub_frame); + s->next_sub_frame = sub_out; + sub_out = NULL; + s->new_frame = 1; + s->next_sub_frame->pts = frame->pts; + + if ((ret = av_buffer_replace(&s->next_sub_frame->subtitle_header, s->subtitle_header)) < 0) + goto fail; + + if (s->real_time && s->scatter_realtime_output) { + if (s->next_repetition_pts) + s->next_sub_frame->pts = s->next_repetition_pts; + + s->next_sub_frame->subtitle_end_time = s->real_time_latency_msec; + s->next_repetition_pts = s->next_sub_frame->pts + av_rescale_q(s->real_time_latency_msec, ms_tb, inlink->time_base); + } + + ret = request_sub_frame(outlink1); + if (ret < 0) + goto fail; + } + } + + if (s->real_time && s->scatter_realtime_output && !s->new_frame && s->next_repetition_pts > 0 && frame->pts > s->next_repetition_pts) { + s->new_frame = 1; + s->next_sub_frame->pts = s->next_repetition_pts; + s->next_repetition_pts = s->next_sub_frame->pts + av_rescale_q(s->real_time_latency_msec, ms_tb, inlink->time_base); + } + + if (!s->next_sub_frame) { + s->next_sub_frame = ff_get_subtitles_buffer(outlink1, outlink1->format); + if (!s->next_sub_frame) { + ret = AVERROR(ENOMEM); + goto fail; + } + + s->next_sub_frame->subtitle_end_time = 100; + s->next_sub_frame->pts = frame->pts; + s->new_frame = 1; + + if ((ret = av_buffer_replace(&s->next_sub_frame->subtitle_header, s->subtitle_header)) < 0) + goto fail; + } + + ret = ff_filter_frame(outlink0, frame); + +fail: + av_packet_free(&pkt); + av_frame_free(&sub_out); + return ret; +} + +#define OFFSET(x) offsetof(SplitCaptionsContext, x) +#define FLAGS (AV_OPT_FLAG_SUBTITLE_PARAM | AV_OPT_FLAG_FILTERING_PARAM) + +static const AVOption split_cc_options[] = { + { "use_cc_styles", "Emit closed caption style header", OFFSET(use_cc_styles), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, FLAGS, NULL }, + { "real_time", "emit subtitle events as they are decoded for real-time display", OFFSET(real_time), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, FLAGS }, + { "real_time_latency_msec", "minimum elapsed time between emitting real-time subtitle events", OFFSET(real_time_latency_msec), AV_OPT_TYPE_INT, { .i64 = 200 }, 0, 500, FLAGS }, + { "scatter_realtime_output", "scatter output events to a duration of real_time_latency_msec", OFFSET(scatter_realtime_output), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, FLAGS }, + { "data_field", "select data field", OFFSET(data_field), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 1, FLAGS, "data_field" }, + { "auto", "pick first one that appears", 0, AV_OPT_TYPE_CONST, { .i64 =-1 }, 0, 0, FLAGS, "data_field" }, + { "first", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 0 }, 0, 0, FLAGS, "data_field" }, + { "second", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 1 }, 0, 0, FLAGS, "data_field" }, + { NULL }, +}; + +AVFILTER_DEFINE_CLASS(split_cc); + +static const AVFilterPad inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .filter_frame = filter_frame, + .config_props = config_input, + }, +}; + +static const AVFilterPad outputs[] = { + { + .name = "video_passthrough", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_video_output, + }, + { + .name = "subtitles", + .type = AVMEDIA_TYPE_SUBTITLE, + .request_frame = request_sub_frame, + .config_props = config_sub_output, + }, +}; + +const AVFilter ff_sf_splitcc = { + .name = "splitcc", + .description = NULL_IF_CONFIG_SMALL("Extract closed-caption (A53) data from video as subtitle stream."), + .init = init, + .uninit = uninit, + .priv_size = sizeof(SplitCaptionsContext), + .priv_class = &split_cc_class, + .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + FILTER_INPUTS(inputs), + FILTER_OUTPUTS(outputs), + FILTER_QUERY_FUNC(query_formats), +}; From patchwork Mon Dec 13 23:40:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 32470 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp6141858iog; Mon, 13 Dec 2021 15:43:56 -0800 (PST) X-Google-Smtp-Source: ABdhPJwYUxkbbjVO605x9c3Yd6yTxiEG9VUudG0N66aTJ8/+MjINjXjR8iu1xWmTH02pDCxXF5+z X-Received: by 2002:a17:906:7009:: with SMTP id n9mr1638218ejj.431.1639439036275; Mon, 13 Dec 2021 15:43:56 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id i3si18827241edu.486.2021.12.13.15.43.55; Mon, 13 Dec 2021 15:43:56 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=AuHJS29C; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7B87368B0EB; Tue, 14 Dec 2021 01:41:03 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11olkn2099.outbound.protection.outlook.com [40.92.18.99]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9170D68B0D8 for ; Tue, 14 Dec 2021 01:41:01 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Xow3EdTaeiMKNxERmfTIW569vt6aL7aEuk+7Tg4J+ve0QowgWkTO8t60OYeJ+fzJx4sv1/ip8iMeJ+Y12ZPRaZEaib/gEKyQpustH94xBR8TX45tJ9utZt5hZV7g8WkqbceChip1hx5N+6kLCiaaITb0pB89a0rKHIULjJDoa+jy7xsWZp2ME+mw9fxFpwhpbdR9ifzYYCsQYZhMD99ZeRdtqOECUE2m1QSFLKRa6oDHIgurIsrIf3xO/qCGQxJUpZ3SWb1z4Hcd9gwzABfZcURQI3lNIgdlCy5oOday4hlfRpyjnNvGPgovK+a1s19zSmMWtYAYAcQeyNpw/HM2Hg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=xZ0N/RGOSPidTlFlFsPAx+gWBKTZvFMzDcBDuRtfCcQ=; b=E1tRC5W73RkvLRkB5T/pQH2zJTHCtUHDr58MFfmJPW+DEeQMn9LQ3yhGr/eDhjjqizTtBhjSEDmB76HWq12AYfJUDg//y6JloVfss8aYdQ7ZWLu8cQjASfAjIUk+uaEI7akEjAqWO5oGmhSdfjbw4F7AFe2dtU+SqfIDU123FZjZWhd79G/f/l6zVk/W3kyolKTbBZSketmjHls3QPhUE7tMblEYO/7KSApgUNXMSnqSWYmPEeFdUAtCJGeX/6SvuzPEeZuuczTSho1FYnJuho59s43KpknlKaW9FL9+z4eFFe+sOtGFufHRbYQ1HDHcizwFBbhYmTuk/GatsSKk3w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xZ0N/RGOSPidTlFlFsPAx+gWBKTZvFMzDcBDuRtfCcQ=; b=AuHJS29Cs78t7twymsbxJPRj3l5hYhPlrDYaKTbnJmhLWVtsMUqEKBoG+YmcrPt0rsS4tnlfwZl6tpqN0aRrte/Ym5fmQr8ybA0sIenAeNYbGKr+zG2yTbD8qFoh/VZnAW5o/utY8G6t2fyQeUXj+bKzGcAd6vC7pA0vekYTdJpyiDHZ47N3Wz8WpWvXwyag3X1mntI/JihP1UWXtrnbjTlOxF6rXiADvqJ/iMnxYPtHeTR+1TMB0lMOm1UOfnRRTs+3DyJxUpKyXMQjUKKycx/9tpHvdAN416flUOXr5ISX4igfllOdSTpD/NwjamarJWHH2hh57tkktGYZWGjZ2w== Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::20) by DM8P223MB0397.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4778.17; Mon, 13 Dec 2021 23:40:59 +0000 Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775]) by DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775%7]) with mapi id 15.20.4778.018; Mon, 13 Dec 2021 23:40:59 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v24 19/21] avfilter/graphicsub2text: Add new graphicsub2text filter (OCR) Thread-Index: AQHX8HrhA1bQU64dsUWPAijhv41Jtg== Date: Mon, 13 Dec 2021 23:40:58 +0000 Message-ID: References: <1ef42eb01b3627bbad539abd46206f320bb39697.1639438462.git.softworkz@hotmail.com> <4148d77a82d4b7e229fc81ce9bc1bc06904533f1.1639438462.git.softworkz@hotmail.com> <2ac798924ee984010d2e85f00bc992d9aa106171.1639438462.git.softworkz@hotmail.com> <300ea7188b331ca5e00dd7e04c03df8aba2a99c0.1639438462.git.softworkz@hotmail.com> <9474726b26f0d3e8f353d07caed2fbd33cdf473a.1639438462.git.softworkz@hotmail.com> <424522c96b5cdf6b03faa6bdceb1d25449463429.1639438462.git.softworkz@hotmail.com> <90caff245055005e6a4a890b3665da55aa853b31.1639438462.git.softworkz@hotmail.com> <0ebcf3ed2adb02d3ffb243a499b434c3bdd976a7.1639438462.git.softworkz@hotmail.com> <4b5c1c1185075fd6e5d8ef70e07fdce6f896d699.1639438462.git.softworkz@hotmail.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [+Sdc5KGwKDvQ+GZERDF48xKazh/EHsfV] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 2b39e1fb-9038-44cb-6157-08d9be9203f1 x-ms-traffictypediagnostic: DM8P223MB0397:EE_ x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: OfD6T0aY8XRWaTf0DyQ8V8pqa08wWSRQTTU+4C2CBpZ3hDm+i0n07aDfbYCL/cGVgDalI0BKwF1YpJhWY2HJzwW35wYI/1CiFw+j7Q7K7cnXKsSEKvLi4FW+9lKVpzM3LtwBpnbTsU6fd6/u2iW1teA2c2QAgXyQkJvT4P2hZzQwYhoK4xHzqOXW5OuwgKHiY2i4nUHBv2j/QITB/jzd03GJyJF4ELFY3lAxDMEr3+dlMHvkkw1EI/wChhDN34SzW6avZJ44mRzhK43khSBufjz4td1ESiBKnT58pea6Zan4v58j+8f/i6c0W9MmSWtxxQ6ow36w0qvA5D7M+LYDmSFCALB2uJ8Any+nygdfjw7jlK7c/QXFr8CCPWxoWu9WdXokPCIgm3C0FsqhhzygpSKm6HeInNAa9vpYR+c580dKQT+jo6gAtZNM5qPIDyaXRZfG5tQWMQcL/JJn/CmEc+u/765m5xGUxPYiIFZW2VvFD3J5EP0domRG3VwVCzvH/KAWq1Miq8pqfhUR3ExTRKg8JnutKyvw1aUKC5XYbvLWTUI7s8HN+7z4CT6+r45MDAer7YDw0mk8+yCjedCdC42CUF48ST4JJQZOhk9qN2ks5mC0LcBcDJ+//UTziZVLnJTHKeNCprbCn/xWe4MEyXobdVRGjEwueRZ9jff1ww4= x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?q?0OTo/26bHvLj0gbI1H21WmVAnTNe?= =?utf-8?q?htyIxCCntarp9n1QHd2XdqhHrmt5mwSa6Jn+6hsPccdtY2Q5lr9Ghs2WYFbDP1X2e?= =?utf-8?q?jt1T2M1QLCoP+AUD4b+vW5VgacQaUyzJk+8Aq1W0ly/Z7iL5aoySVSTNczuYD2FXT?= =?utf-8?q?ZCS/zL8RQaPxpaedlQZbOKosKHFAMsheQsEEO4kw9AJgskv750Lsgs7ZqNIrLL2lI?= =?utf-8?q?8E7+Vf05oVxuTRvZaZQZUwVgBYf5tmpsnOdoYdTkA8LLs2f02a/6aKvnQQuWOuNMw?= =?utf-8?q?0n7S7LqSsUwCe4ekV1s/eTBh9U8L0KH8WmyXX20L2Ky+bodaIu64US4P+AmbJOTZ0?= =?utf-8?q?mbQXgr3urbJlcxXaI5czHA9lH+hxqZctc3EaWLMkfr3f4EBuajEpSXBKcbsfvszbL?= =?utf-8?q?/c9tpXoiZWTdD/pS7daqkM9+glvZHuXwLnT2mvSwhtBJbA4cxeN7U6DmrQq5+7CCK?= =?utf-8?q?jZOxTDIge6xlwNvTdfwwa0hBd92ihPvOr5248syjljFU24j30Q1ffaFhF1qFqQnj6?= =?utf-8?q?HUOSWuwMJl1AVlx0WOyRKrBmiDtUtcE3gnY4JMQ0uNHqx8JQ0Z5I31K2gqUZL+rb1?= =?utf-8?q?ckAvVG803j5o38JlHMS022CngAL2SGiU27wVdMsh8J7DaY5ce+9wQWxU6OOe3ukL6?= =?utf-8?q?dm1lLnk9VBksXsgHP0M5IaiurdaxceLX7o8Tp88bvWCWYwddqbQTOWLXVlp3C9R3V?= =?utf-8?q?V5/B0H2ctr5CU4GJUApCeMnZjv8TtYlwkPN35IMDrYeONOZY65A6V0CbjZ+aOt88B?= =?utf-8?q?zGwECWXXt3FrhhzXu3v7EZ6qkaCXIBRkXX3/VUt4hxAhse+eunDXF8Xps9ufTPVQ6?= =?utf-8?q?79ggUFvZMCa+nzLNXGNkCuWCEq/Yyh86pD2aw73yCHpNsK6B/HxffaouMJ9MMVQW0?= =?utf-8?q?bDxdoWVRf5tVPuc/i82MbXx2bOQiXDeu7Seikrl9sq4Q=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-1ff67.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 2b39e1fb-9038-44cb-6157-08d9be9203f1 X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Dec 2021 23:40:58.9124 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8P223MB0397 Subject: [FFmpeg-devel] [PATCH v24 19/21] avfilter/graphicsub2text: Add new graphicsub2text filter (OCR) X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: XcNOKQdwrvfL Signed-off-by: softworkz --- configure | 1 + doc/filters.texi | 55 +++++ libavfilter/Makefile | 2 + libavfilter/allfilters.c | 1 + libavfilter/sf_graphicsub2text.c | 354 +++++++++++++++++++++++++++++++ 5 files changed, 413 insertions(+) create mode 100644 libavfilter/sf_graphicsub2text.c diff --git a/configure b/configure index a6dad8fe9d..5b104be345 100755 --- a/configure +++ b/configure @@ -3640,6 +3640,7 @@ frei0r_filter_deps="frei0r" frei0r_src_filter_deps="frei0r" fspp_filter_deps="gpl" gblur_vulkan_filter_deps="vulkan spirv_compiler" +graphicsub2text_filter_deps="libtesseract" hflip_vulkan_filter_deps="vulkan spirv_compiler" histeq_filter_deps="gpl" hqdn3d_filter_deps="gpl" diff --git a/doc/filters.texi b/doc/filters.texi index 53c7e91ef3..d30021cc7e 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -25896,6 +25896,61 @@ ffmpeg -i "https://streams.videolan.org/ffmpeg/mkv_subtitles.mkv" -filter_comple @end example @end itemize +@section graphicsub2text + +Converts graphic subtitles to text subtitles by performing OCR. + +For this filter to be available, ffmpeg needs to be compiled with libtesseract (see https://github.com/tesseract-ocr/tesseract). +Language models need to be downloaded from https://github.com/tesseract-ocr/tessdata and put into as subfolder named 'tessdata' or into a folder specified via the environment variable 'TESSDATA_PREFIX'. +The path can also be specified via filter option (see below). + +Note: These models are including the data for both OCR modes. + +Inputs: +- 0: Subtitles [bitmap] + +Outputs: +- 0: Subtitles [text] + +It accepts the following parameters: + +@table @option +@item ocr_mode +The character recognition mode to use. + +Supported OCR modes are: + +@table @var +@item 0, tesseract +This is the classic libtesseract operation mode. It is fast but less accurate than LSTM. +@item 1, lstm +Newer OCR implementation based on ML models. Provides usually better results, requires more processing resources. +@item 2, both +Use a combination of both modes. +@end table + +@item tessdata_path +The path to a folder containing the language models to be used. + +@item language +The recognition language. It needs to match the first three characters of a language model file in the tessdata path. + +@end table + + +@subsection Examples + +@itemize +@item +Convert DVB graphic subtitles to ASS (text) subtitles + +Note: For this to work, you need to have the data file 'eng.traineddata' in a 'tessdata' subfolder (see above). +@example +ffmpeg ffmpeg -loglevel verbose -i "https://streams.videolan.org/streams/ts/video_subs_ttxt%2Bdvbsub.ts" -filter_complex "[0:13]graphicsub2text=ocr_mode=both" -c:s ass -y output.mkv +@end example +@end itemize + + @section graphicsub2video Renders graphic subtitles as video frames. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 9477b56649..3e98f63d28 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -297,6 +297,8 @@ OBJS-$(CONFIG_GBLUR_VULKAN_FILTER) += vf_gblur_vulkan.o vulkan.o vulka OBJS-$(CONFIG_GEQ_FILTER) += vf_geq.o OBJS-$(CONFIG_GRADFUN_FILTER) += vf_gradfun.o OBJS-$(CONFIG_GRAPHICSUB2VIDEO_FILTER) += vf_overlaygraphicsubs.o framesync.o +OBJS-$(CONFIG_GRAPHICSUB2TEXT_FILTER) += sf_graphicsub2text.o +OBJS-$(CONFIG_GRAPHICSUB2VIDEO_FILTER) += vf_overlaygraphicsubs.o framesync.o OBJS-$(CONFIG_GRAPHMONITOR_FILTER) += f_graphmonitor.o OBJS-$(CONFIG_GRAYWORLD_FILTER) += vf_grayworld.o OBJS-$(CONFIG_GREYEDGE_FILTER) += vf_colorconstancy.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 65b6bea650..dccad79728 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -547,6 +547,7 @@ extern const AVFilter ff_avf_showwaves; extern const AVFilter ff_avf_showwavespic; extern const AVFilter ff_vaf_spectrumsynth; extern const AVFilter ff_sf_censor; +extern const AVFilter ff_sf_graphicsub2text; extern const AVFilter ff_sf_showspeaker; extern const AVFilter ff_sf_splitcc; extern const AVFilter ff_sf_stripstyles; diff --git a/libavfilter/sf_graphicsub2text.c b/libavfilter/sf_graphicsub2text.c new file mode 100644 index 0000000000..ef10d60efd --- /dev/null +++ b/libavfilter/sf_graphicsub2text.c @@ -0,0 +1,354 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * subtitle filter to convert graphical subs to text subs via OCR + */ + +#include +#include + +#include "libavutil/opt.h" +#include "subtitles.h" + +typedef struct SubOcrContext { + const AVClass *class; + int w, h; + + TessBaseAPI *tapi; + TessOcrEngineMode ocr_mode; + char *tessdata_path; + char *language; + + int readorder_counter; + + AVFrame *pending_frame; +} SubOcrContext; + + +static int init(AVFilterContext *ctx) +{ + SubOcrContext *s = ctx->priv; + const char* tver = TessVersion(); + int ret; + + s->tapi = TessBaseAPICreate(); + + if (!s->tapi || !tver || !strlen(tver)) { + av_log(ctx, AV_LOG_ERROR, "Failed to access libtesseract\n"); + return AVERROR(ENOSYS); + } + + av_log(ctx, AV_LOG_VERBOSE, "Initializing libtesseract, version: %s\n", tver); + + ret = TessBaseAPIInit4(s->tapi, s->tessdata_path, s->language, s->ocr_mode, NULL, 0, NULL, NULL, 0, 1); + if (ret < 0 ) { + av_log(ctx, AV_LOG_ERROR, "Failed to initialize libtesseract. Error: %d\n", ret); + return AVERROR(ENOSYS); + } + + ret = TessBaseAPISetVariable(s->tapi, "tessedit_char_blacklist", "|"); + if (ret < 0 ) { + av_log(ctx, AV_LOG_ERROR, "Failed to set 'tessedit_char_blacklist'. Error: %d\n", ret); + return AVERROR(EINVAL); + } + + return 0; +} + +static void uninit(AVFilterContext *ctx) +{ + SubOcrContext *s = ctx->priv; + + if (s->tapi) { + TessBaseAPIEnd(s->tapi); + TessBaseAPIDelete(s->tapi); + } +} + +static int query_formats(AVFilterContext *ctx) +{ + AVFilterFormats *formats, *formats2; + AVFilterLink *inlink = ctx->inputs[0]; + AVFilterLink *outlink = ctx->outputs[0]; + static const enum AVSubtitleType in_fmts[] = { AV_SUBTITLE_FMT_BITMAP, AV_SUBTITLE_FMT_NONE }; + static const enum AVSubtitleType out_fmts[] = { AV_SUBTITLE_FMT_ASS, AV_SUBTITLE_FMT_NONE }; + int ret; + + /* set input format */ + formats = ff_make_format_list(in_fmts); + if ((ret = ff_formats_ref(formats, &inlink->outcfg.formats)) < 0) + return ret; + + /* set output format */ + formats2 = ff_make_format_list(out_fmts); + if ((ret = ff_formats_ref(formats2, &outlink->incfg.formats)) < 0) + return ret; + + return 0; +} + +static int config_input(AVFilterLink *inlink) +{ + AVFilterContext *ctx = inlink->dst; + SubOcrContext *s = ctx->priv; + + if (s->w <= 0 || s->h <= 0) { + s->w = inlink->w; + s->h = inlink->h; + } + return 0; +} + +static int config_output(AVFilterLink *outlink) +{ + const AVFilterContext *ctx = outlink->src; + SubOcrContext *s = ctx->priv; + + outlink->format = AV_SUBTITLE_FMT_ASS; + outlink->w = s->w; + outlink->h = s->h; + + return 0; +} + +static uint8_t* create_grayscale_image(AVFilterContext *ctx, AVSubtitleArea *area) +{ + uint8_t gray_pal[256]; + const size_t img_size = area->buf[0]->size; + const uint8_t* img = area->buf[0]->data; + uint8_t* gs_img = av_malloc(img_size); + + if (!gs_img) + return NULL; + + for (unsigned i = 0; i < 256; i++) { + const uint8_t *col = (uint8_t*)&area->pal[i]; + const int val = (int)col[3] * FFMAX3(col[0], col[1], col[2]); + gray_pal[i] = (uint8_t)(val >> 8); + } + + for (unsigned i = 0; i < img_size; i++) + gs_img[i] = 255 - gray_pal[img[i]]; + + return gs_img; +} + +static int convert_area(AVFilterContext *ctx, AVSubtitleArea *area) +{ + SubOcrContext *s = ctx->priv; + char *ocr_text = NULL; + int ret; + uint8_t *gs_img = create_grayscale_image(ctx, area); + + if (!gs_img) + return AVERROR(ENOMEM); + + area->type = AV_SUBTITLE_FMT_ASS; + TessBaseAPISetImage(s->tapi, gs_img, area->w, area->h, 1, area->linesize[0]); + TessBaseAPISetSourceResolution(s->tapi, 70); + + ret = TessBaseAPIRecognize(s->tapi, NULL); + if (ret == 0) + ocr_text = TessBaseAPIGetUTF8Text(s->tapi); + + if (!ocr_text) { + av_log(ctx, AV_LOG_WARNING, "OCR didn't return a text. ret=%d\n", ret); + area->ass = NULL; + } + else { + const size_t len = strlen(ocr_text); + + if (len > 0 && ocr_text[len - 1] == '\n') + ocr_text[len - 1] = 0; + + av_log(ctx, AV_LOG_VERBOSE, "OCR Result: %s\n", ocr_text); + + area->ass = av_strdup(ocr_text); + + TessDeleteText(ocr_text); + } + + av_freep(&gs_img); + av_buffer_unref(&area->buf[0]); + area->type = AV_SUBTITLE_FMT_ASS; + + return 0; +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *frame) +{ + AVFilterContext *ctx = inlink->dst; + SubOcrContext *s = ctx->priv; + AVFilterLink *outlink = inlink->dst->outputs[0]; + int ret, frame_sent = 0; + + if (s->pending_frame) { + const uint64_t pts_diff = frame->subtitle_pts - s->pending_frame->subtitle_pts; + + if (pts_diff == 0) { + // This is just a repetition of the previous frame, ignore it + av_frame_free(&frame); + return 0; + } + + s->pending_frame->subtitle_end_time = (uint32_t)(pts_diff / 1000); + + ret = ff_filter_frame(outlink, s->pending_frame); + s->pending_frame = NULL; + if (ret < 0) + return ret; + + frame_sent = 1; + + if (frame->num_subtitle_areas == 0) { + // No need to forward this empty frame + av_frame_free(&frame); + return 0; + } + } + + ret = av_frame_make_writable(frame); + + if (ret < 0) { + av_frame_free(&frame); + return ret; + } + + frame->format = AV_SUBTITLE_FMT_ASS; + + av_log(ctx, AV_LOG_DEBUG, "filter_frame sub_pts: %"PRIu64", start_time: %d, end_time: %d, num_areas: %d\n", + frame->subtitle_pts, frame->subtitle_start_time, frame->subtitle_end_time, frame->num_subtitle_areas); + + if (frame->num_subtitle_areas > 1 && + frame->subtitle_areas[0]->y > frame->subtitle_areas[frame->num_subtitle_areas - 1]->y) { + + for (unsigned i = 0; i < frame->num_subtitle_areas / 2; i++) + FFSWAP(AVSubtitleArea*, frame->subtitle_areas[i], frame->subtitle_areas[frame->num_subtitle_areas - i - 1]); + } + + for (unsigned i = 0; i < frame->num_subtitle_areas; i++) { + AVSubtitleArea *area = frame->subtitle_areas[i]; + + ret = convert_area(ctx, area); + if (ret < 0) + return ret; + + if (area->ass && area->ass[0] != '\0') { + char *tmp = area->ass; + + if (i == 0) + area->ass = avpriv_ass_get_dialog(s->readorder_counter++, 0, "Default", NULL, tmp); + else { + AVSubtitleArea* area0 = frame->subtitle_areas[0]; + char* tmp2 = area0->ass; + area0->ass = av_asprintf("%s\\N%s", area0->ass, tmp); + av_free(tmp2); + area->ass = NULL; + } + + av_free(tmp); + } + } + + if (frame->num_subtitle_areas > 1) { + for (unsigned i = 1; i < frame->num_subtitle_areas; i++) { + AVSubtitleArea* area = frame->subtitle_areas[i]; + + for (unsigned n = 0; n < FF_ARRAY_ELEMS(area->buf); n++) + av_buffer_unref(&area->buf[n]); + + av_freep(&area->text); + av_freep(&area->ass); + av_freep(&frame->subtitle_areas[i]); + } + + AVSubtitleArea* area0 = frame->subtitle_areas[0]; + av_freep(&frame->subtitle_areas); + frame->subtitle_areas = av_malloc_array(1, sizeof(AVSubtitleArea*)); + frame->subtitle_areas[0] = area0; + frame->num_subtitle_areas = 1; + } + + // When decoders can't determine the end time, they are setting it either to UINT32_NAX + // or 30s (dvbsub). + if (frame->num_subtitle_areas > 0 && frame->subtitle_end_time >= 30000) { + // Can't send it without end time, wait for the next frame to determine the end_display time + s->pending_frame = frame; + + if (frame_sent) + return 0; + + // To keep all going, send an empty frame instead + frame = ff_get_subtitles_buffer(outlink, AV_SUBTITLE_FMT_ASS); + if (!frame) + return AVERROR(ENOMEM); + + av_frame_copy_props(frame, s->pending_frame); + frame->subtitle_end_time = 1; + } + + return ff_filter_frame(outlink, frame); +} + +#define OFFSET(x) offsetof(SubOcrContext, x) +#define FLAGS (AV_OPT_FLAG_SUBTITLE_PARAM | AV_OPT_FLAG_FILTERING_PARAM) + +static const AVOption graphicsub2text_options[] = { + { "ocr_mode", "set ocr mode", OFFSET(ocr_mode), AV_OPT_TYPE_INT, {.i64=OEM_TESSERACT_ONLY}, OEM_TESSERACT_ONLY, 2, FLAGS, "ocr_mode" }, + { "tesseract", "classic tesseract ocr", 0, AV_OPT_TYPE_CONST, {.i64=OEM_TESSERACT_ONLY}, 0, 0, FLAGS, "ocr_mode" }, + { "lstm", "lstm (ML based)", 0, AV_OPT_TYPE_CONST, {.i64=OEM_LSTM_ONLY}, 0, 0, FLAGS, "ocr_mode" }, + { "both", "use both models combined", 0, AV_OPT_TYPE_CONST, {.i64=OEM_TESSERACT_LSTM_COMBINED}, 0, 0, FLAGS, "ocr_mode" }, + { "tessdata_path", "path to tesseract data", OFFSET(tessdata_path), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, FLAGS, NULL }, + { "language", "ocr language", OFFSET(language), AV_OPT_TYPE_STRING, {.str = "eng"}, 0, 0, FLAGS, NULL }, + { NULL }, +}; + +AVFILTER_DEFINE_CLASS(graphicsub2text); + +static const AVFilterPad inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .filter_frame = filter_frame, + .config_props = config_input, + }, +}; + +static const AVFilterPad outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .config_props = config_output, + }, +}; + +const AVFilter ff_sf_graphicsub2text = { + .name = "graphicsub2text", + .description = NULL_IF_CONFIG_SMALL("Convert graphical subtitles to text subtitles via OCR"), + .init = init, + .uninit = uninit, + .priv_size = sizeof(SubOcrContext), + .priv_class = &graphicsub2text_class, + FILTER_INPUTS(inputs), + FILTER_OUTPUTS(outputs), + FILTER_QUERY_FUNC(query_formats), +}; From patchwork Mon Dec 13 23:41:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 32471 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp6142047iog; Mon, 13 Dec 2021 15:44:08 -0800 (PST) X-Google-Smtp-Source: ABdhPJxafTEB4z+2mcRMd5bt6FuKalYDVnKAZyCF9g5YjxYh6XoxW7FZSrYt1h672iV3BI9cFDYG X-Received: by 2002:a17:906:9a06:: with SMTP id ai6mr1578463ejc.737.1639439048183; Mon, 13 Dec 2021 15:44:08 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id o2si20838896edc.544.2021.12.13.15.44.07; Mon, 13 Dec 2021 15:44:08 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=nk8TDauT; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 94EE268B111; Tue, 14 Dec 2021 01:41:06 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11olkn2056.outbound.protection.outlook.com [40.92.18.56]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 86AB968B0D8 for ; Tue, 14 Dec 2021 01:41:04 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=M6uy45CTbRuDs4Q3/O+8Myj6dO/1+Pf11JvEklgjqQD+5q4rRBDko0ETt38Wa97A7P8tLl9CFLo8E3Du5zp6Bk82VjMvKSEzQRcLKav3k6sxDdsQ6czBfxRMjQ1j8WQ4C4M5RRMf3z1YpgP7o/6Wa8pA+RdDCNlnD58p0QMUruPKsE+Z9eT57LHUwGikOLIEKZ+vx3yqkyAayknJsZ4DyKt1/A5HqmU4WgwxhD55OVNqDkEHEmfrTWYMsmMDZH9lwgICR27uan9OfVco9VMBdog8IYK86gIlDQUxpPvv/21bTaKlpIsrXzKukLbnZfHfj0lglOu3zsHVuyMCKjtxGg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=dcdbW/MvL92egKBY91KwoYx8ydYJ/nC/myP0SN5xhLo=; b=jzu2O++wWzRhMJ2bxkTEg0jNX8IKrj3l6g3jZUTgh+u75xe+GGecspiIoPtoRMSammBgfOZD2u3nUcEcRYkEcS6+k0BWI79MUA4MT1maMlrlgbVC4Iec94vDhnkF/Q9SvKHkTnrcaDgqtxJ41f3N2XtDP1Wt2YuoxHkPxtuweqdAaMtgyE7JFMIk7iCutjyWKPXbC505Ypi1kV4YH9wxDf/knopvIPQVQWRnXcxg+xDV6nLA+lBxDR9nmPTjNImXPYCV2bGTexZkFoZ9VJ2ji3Nl06d8BZ7Y8O4QymTf7wPT2L620W245jFdJqj7mPRxe59NhasVQLPvw55TBL56xQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=dcdbW/MvL92egKBY91KwoYx8ydYJ/nC/myP0SN5xhLo=; b=nk8TDauTrqe7d9vCo68iV4GAJ69ZnHVplwygrFvXBmFYJ9fgbJd8ebY9t5kb7Ve46tYz/bV/DBFPzwLjAEcOH9LRdLAHzGon7B4nh78B3btgGQwL6H30agv64a7y+T7/9kLHrRNXasT9v5HXnVWfY1TANfh8kRdbcgAfoCB8Jc6PSq81LiAqKema3N9arRNHLKyb8PpPb/DY35J/GpnxDEFj5GiQSsSLcgEzJquLmRinNEdKD7qbEuwgcXH3vm64KpH8oOz0+DgLrtpSRnEB60LJm1rV5pn5VQTELz+deLNSOWuDzecjSkDmIY/piZB/N7ip3b2X6fpb6LDbRI+UTA== Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::20) by DM8P223MB0397.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4778.17; Mon, 13 Dec 2021 23:41:01 +0000 Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775]) by DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775%7]) with mapi id 15.20.4778.018; Mon, 13 Dec 2021 23:41:01 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v24 20/21] avfilter/subscale: Add filter for scaling and/or re-arranging graphical subtitles Thread-Index: AQHX8HrjCfOVW5AkNkmFGFovYisx/g== Date: Mon, 13 Dec 2021 23:41:01 +0000 Message-ID: References: <1ef42eb01b3627bbad539abd46206f320bb39697.1639438462.git.softworkz@hotmail.com> <4148d77a82d4b7e229fc81ce9bc1bc06904533f1.1639438462.git.softworkz@hotmail.com> <2ac798924ee984010d2e85f00bc992d9aa106171.1639438462.git.softworkz@hotmail.com> <300ea7188b331ca5e00dd7e04c03df8aba2a99c0.1639438462.git.softworkz@hotmail.com> <9474726b26f0d3e8f353d07caed2fbd33cdf473a.1639438462.git.softworkz@hotmail.com> <424522c96b5cdf6b03faa6bdceb1d25449463429.1639438462.git.softworkz@hotmail.com> <90caff245055005e6a4a890b3665da55aa853b31.1639438462.git.softworkz@hotmail.com> <0ebcf3ed2adb02d3ffb243a499b434c3bdd976a7.1639438462.git.softworkz@hotmail.com> <4b5c1c1185075fd6e5d8ef70e07fdce6f896d699.1639438462.git.softworkz@hotmail.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [JRUACCFsMhcMVwVZ31Ekw2CBPecMpeKb] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: fac419a9-aa8d-4f43-aac2-08d9be92058f x-ms-traffictypediagnostic: DM8P223MB0397:EE_ x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: F6w2QHTQ0Z8c2iANVo8pwFAb5nxZRMbAIHDR1cpEKd13hVocJfUSey/5eY0jEEvG7NlHE9RehzAZWFPrW3Tc+P3DpLT5Vz7iqe02lMZd6wEKzDoju0C2SMspZEPoOHpf1Wr185DXACMdkAsDBhGv5JUcsm2PLHai/RRn9PO9s6OwFrqw/YlwuryzhfYq+YiHbZmR8bD3yI9HsJeN9HxHUZC4iMHrLwI5tcFHlw9THsUvG9V+bk59BzQnG9JTQxNK78a2RYfHd2eEmjLfaIhvjY+zxXZllPzDVlhprTmRq0NYbkpUn0Bhao77c8ANOtPSQQtpIfqpJbN1zqIzUVDL5+k2vtm+pPXXQ6cAmVjaM7rF6cbpdf7/l4nq1qmRwsiUm/8flqVmCwQG96vegcO0ZAANrfxwoaUgJxqD0Bt85K+5OQTL/fMHTuNGv26t1NnQFXGWv1EiIobvKWS333bgardB5hXZwDJRpczdFbWA5c7fMSsnLbPI8XeHr0KVWhTABHJjUUMOjNjUgAOleRYTtbADuVerml4vU6wt4HXsGx+HuU8d5h7JGvqK1d5x8WU7h8zU1dpdQhpteMaIKKrp1eyyzXDylHp+u0PcoZScMe6ajAnqxWtdN2POWRq9VWLZ x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?q?sYSh2lHCUQWHMwhDRvF/rlN7UnUC?= =?utf-8?q?gc+GrV7HVh72DFDllGmN91jKkRDd7EwqHIBO3R3YFISaXw4j9h2BERVPaQSssltws?= =?utf-8?q?wpLPI9h91sY6EcRvlx8qvDGv7avcUbkNmH6z6Wxa7TqScaf98PYR1/X4kzT3aFABC?= =?utf-8?q?J2gfY9Obo95Lp/DKYJcymWomhgEQwWoIRSdmJDyvHTVAii+VUDlG2FGQCHn9j2UuS?= =?utf-8?q?aYn6jiolXY5YUC325ERSgQFKmwhTJ9mxgA+wUURQo0YPHRQ/kPuEaIiC/R1Ad7yC1?= =?utf-8?q?SKapT4oCa4WyHbhhzzoFCbz41sJpASFs9trD8Fc2Pu/WQqmCJFnyO3sD4C6gN/vJw?= =?utf-8?q?VI6DtTa7VnR+FbUGtBULiuhoL6kV4t32ow2mRSSeFEqkOFQrTvIOnyDOgEKVBKMSX?= =?utf-8?q?NKCzHPVCFWd5Aexp4kUNSlhHIDKoybdBWtIehMMGgtB8ElisMYz29BM69zbLpuFSY?= =?utf-8?q?vh+TRyF1n6jjKK5rWruzV3gxEjbZKPlh7WB5cPGIAeSzbknPT/ujcax1eWQmNJLVG?= =?utf-8?q?iVfqdNL4BMfV7cfMi+pMFMWk8QdIb+TX4PYuU1vg4qsU4+h4YFhqBy8PbUjbytfKf?= =?utf-8?q?fkDaxvKM0Vc/qmM7iDy68/3Gtu56x167Ex+q0ujsFLgoVrJF9LQdy3R+71FlxEmzS?= =?utf-8?q?sFRrL7bywufwlbT+dbsVPRDeRvgf6HIF7w1RCTO7tSuRKhb7CZndpTzbSKrV1Jjzs?= =?utf-8?q?Z9nGKVux7tjlrbBNzAbdJjfNwV215wZ5FbFvwmH/WHHPwVoegDcPYagLgPtSmPujr?= =?utf-8?q?yCwlGKhkhbLd5DGc6tucNQ7vy/l+/otOaHDXenKW3CxHYO1t8MrOFXRo9S8wmcc4L?= =?utf-8?q?ltNBM06TpXTyctd3vydWWMPKAjueVumjv9BLRbSFXqhVnkjihc5j3z51UsRrT1YUu?= =?utf-8?q?6BmJcgWrFOZW7y6CJn3H0ZzYVq7bW0k1JrwvkTSaP7NA=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-1ff67.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: fac419a9-aa8d-4f43-aac2-08d9be92058f X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Dec 2021 23:41:01.6145 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8P223MB0397 Subject: [FFmpeg-devel] [PATCH v24 20/21] avfilter/subscale: Add filter for scaling and/or re-arranging graphical subtitles X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: zcJruNaQpxHQ Signed-off-by: softworkz --- configure | 1 + doc/filters.texi | 164 +++++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/sf_subscale.c | 883 ++++++++++++++++++++++++++++++++++++++ 5 files changed, 1050 insertions(+) create mode 100644 libavfilter/sf_subscale.c diff --git a/configure b/configure index 5b104be345..7e3ba34a9b 100755 --- a/configure +++ b/configure @@ -3706,6 +3706,7 @@ sr_filter_deps="avformat swscale" sr_filter_select="dnn" stereo3d_filter_deps="gpl" splitcc_filter_deps="avcodec" +subscale_filter_deps="swscale avcodec" subtitles_filter_deps="avformat avcodec libass" super2xsai_filter_deps="gpl" pixfmts_super2xsai_test_deps="super2xsai_filter" diff --git a/doc/filters.texi b/doc/filters.texi index d30021cc7e..8d33a238f6 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -26302,6 +26302,170 @@ Set the rendering margin in pixels. For rendering, alway use the latest event only, which is covering the given point in time. @end table +@section subscale + +Provides high-quality scaling and rearranging functionality for graphical subtitles. + +The subscale filter provides multiple approaches for manipulating +the size and position of graphical subtitle rectangles wich can +be combined or used separately. +Scaling is performed by converting the palettized subtitle bitmaps +to RGBA and re-quantization to palette colors afterwards via elbg algorithm. + +The two major operations are 'scale' and 're-arrange' with the +latter being separated as 'arrange_h' and 'arrange_v'. + + +Inputs: +- 0: Subtitles [bitmap] + +Outputs: +- 0: Subtitles [bitmap] + +It accepts the following parameters: + +@table @option + +@item w, width +Set the width of the output. +Width and height in case of graphical subtitles are just indicating +a virtual size for which the output (consisting of 0-n bitmap rectangles) +is intended to be displayed on. + +@item h, height +Set the height of the output. + +@item margin_h +Sets a horizontal margin to be preserverved when using any +of the arrange modes. + +@item margin_v +Sets a vertical margin to be preserverved when using any +of the arrange modes. + +@item force_original_aspect_ratio +Enable decreasing or increasing output video width or height if necessary to +keep the original aspect ratio. Possible values: + +@table @samp +@item disable +Scale the video as specified and disable this feature. + +@item decrease +The output video dimensions will automatically be decreased if needed. + +@item increase +The output video dimensions will automatically be increased if needed. + +@end table + + +@item scale_mode +Specifies how subtitle bitmaps should be scaled. +The scale factor is determined by the the factor between input +and output size. + +@table @samp +@item none +Do not apply any common scaling. + +@item uniform +Uniformly scale all subtitle bitmaps including their positions. + +@item uniform_no_reposition +Uniformly scale all subtitle bitmaps without changing positions. + +@end table + + +@item arrange_h +Specifies how subtitle bitmaps should be arranged horizontally. + +@item arrange_v +Specifies how subtitle bitmaps should be arranged vertically. + + +@table @samp +@item none +Do not rearrange subtitle bitmaps. + +@item margin_no_scale +Move subtitle bitmaps to be positioned inside the specified +margin (margin_h or margin_v) when possible and without scaling. + +@item margin_and_scale +Move subtitle bitmaps to be positioned inside the specified +margin (margin_h or margin_v) and scale in case it doesn't fit. + +@item snapalign_no_scale +Categorize subtitle bitmap positions as one of left/center/right +or top/bottom/middle based on original positioning and apply +these alignments for the target positioning. +No scaling will be applied. + +@item snapalign_and_scale +Categorize subtitle bitmap positions as one of left/center/right +or top/bottom/middle based on original positioning and apply +these alignments for the target positioning. +Bitmaps that do not fit inside the margins borders are +scaled to fit. +@end table + +@item eval +Set evaluation mode for the expressions (@option{width}, @option{height}). + +It accepts the following values: +@table @samp +@item init +Evaluate expressions only once during the filter initialization. + +@item frame +Evaluate expressions for each incoming frame. This is way slower than the +@samp{init} mode since it requires all the scalers to be re-computed, but it +allows advanced dynamic expressions. +@end table + +Default value is @samp{init}. + + +@item num_colors +Set the number of palette colors for output images. +Choose the maximum (256) when further processing is done (e.g. +overlaying on a video). +When subtitles will be encoded as bitmap subtitles (e.g. dvbsub), +a smaller number of palette colors (e.g. 4-16) might need to be used, depending +on the target format and codec. + +@item bitmap_width_align +@item bitmap_height_align +Make sure that subtitle bitmap sizes are a multiple of this +value. Default is 2. + +@end table + +@subsection Examples + +@itemize +@item +Uniformly scale down video and bitmap subtitles and encode +subtitles as dvbsub. +@example +ffmpeg -y -loglevel verbose -ss 32 -i "https://streams.videolan.org/samples/sub/largeres_vobsub.mkv" -filter_complex "[0:0]scale=480x270[vid1];[0:10]subscale=w=480:h=270:num_colors=16[sub1]" -map [vid1] -map [sub1] -c:s dvbsub -c:v libx265 output.ts +@end example +@item +Squeeze video vertically and arrange subtitle bitmaps +inside the video area without scaling, then overlay +subtitles onto the video. +@example +ffmpeg -y -loglevel verbose -ss 32 -i "https://streams.videolan.org/samples/sub/largeres_vobsub.mkv" -filter_complex "[0:0]scale=1920x400,setsar=1[vid1];[0:10]subscale=w=1920:h=400:scale_mode=none:margin_h=50:arrange_h=margin_no_scale:arrange_v=margin_no_scale:margin_v=50:num_colors=256[sub1];[vid1][sub1]overlay_graphicsubs" -c:v libx265 output.ts +@end example +@item +Scale both, a video and its embedded VOB subs, then encode them as separate streams into an MKV. +@example +ffmpeg -y -y -loglevel verbose -i "https://streams.videolan.org/samples/sub/largeres_vobsub.mkv" -filter_complex "[0:0]scale=720x576[vid1];[0:7]subscale=w=720:h=576:num_colors=16[sub1]" -map [vid1] -map [sub1] -c:s dvdsub out.mkv +@end example +@end itemize + @c man end SUBTITLE FILTERS @chapter Multimedia Filters diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 3e98f63d28..4795c72cd6 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -563,6 +563,7 @@ OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o OBJS-$(CONFIG_TEXTMOD_FILTER) += sf_textmod.o OBJS-$(CONFIG_SPLITCC_FILTER) += sf_splitcc.o OBJS-$(CONFIG_STRIPSTYLES_FILTER) += sf_stripstyles.o +OBJS-$(CONFIG_SUBSCALE_FILTER) += sf_subscale.o # multimedia filters OBJS-$(CONFIG_ABITSCOPE_FILTER) += avf_abitscope.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index dccad79728..8a0c31d6c2 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -551,6 +551,7 @@ extern const AVFilter ff_sf_graphicsub2text; extern const AVFilter ff_sf_showspeaker; extern const AVFilter ff_sf_splitcc; extern const AVFilter ff_sf_stripstyles; +extern const AVFilter ff_sf_subscale; extern const AVFilter ff_sf_textmod; extern const AVFilter ff_svf_graphicsub2video; extern const AVFilter ff_svf_textsub2video; diff --git a/libavfilter/sf_subscale.c b/libavfilter/sf_subscale.c new file mode 100644 index 0000000000..072312c913 --- /dev/null +++ b/libavfilter/sf_subscale.c @@ -0,0 +1,883 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * scale graphical subtitles filter + */ + +#include + +#include "drawutils.h" +#include "internal.h" +#include "scale_eval.h" +#include "libavutil/eval.h" +#include "libavutil/opt.h" +#include "libswscale/swscale.h" + +#include "libavcodec/elbg.h" + +static const char *const var_names[] = { + "in_w", "iw", + "in_h", "ih", + "out_w", "ow", + "out_h", "oh", + "a", + "sar", + "dar", + "margin_h", + "margin_v", + NULL +}; + +enum var_name { + VAR_IN_W, VAR_IW, + VAR_IN_H, VAR_IH, + VAR_OUT_W, VAR_OW, + VAR_OUT_H, VAR_OH, + VAR_A, + VAR_SAR, + VAR_DAR, + VARS_B_H, + VARS_B_V, + VARS_NB +}; + +enum EvalMode { + EVAL_MODE_INIT, + EVAL_MODE_FRAME, + EVAL_MODE_NB +}; + +enum SubScaleMode { + SSM_NONE, + SSM_UNIFORM, + SSM_UNIFORM_NO_REPOSITION, +}; + +enum SubArrangeMode { + SAM_NONE, + SAM_ENSUREMARGIN_NO_SCALE, + SAM_ENSUREMARGIN_AND_SCALE, + SAM_SNAPALIGNMENT_NO_SCALE, + SAM_SNAPALIGNMENT_AND_SCALE, +}; + +typedef struct SubScaleContext { + const AVClass *class; + struct SwsContext *sws; + AVDictionary *opts; + + int w, h; + + char *w_expr; ///< width expression string + char *h_expr; ///< height expression string + AVExpr *w_pexpr; + AVExpr *h_pexpr; + double var_values[VARS_NB]; + + int force_original_aspect_ratio; + int eval_mode; ///< expression evaluation mode + + int use_caching; + + // Scale Options + enum SubScaleMode scale_mode; + + // Arrange Options + enum SubArrangeMode arrange_mode_h; + enum SubArrangeMode arrange_mode_v; + int margin_h; + int margin_v; + char *margin_h_expr; + char *margin_v_expr; + AVExpr *margin_h_pexpr; + AVExpr *margin_v_pexpr; + + // Bitmap Options + int num_output_colors; + int bitmap_width_align; + int bitmap_height_align; + + // Color Quantization Fields + struct ELBGContext *ctx; + AVLFG lfg; + int *codeword; + int *codeword_closest_codebook_idxs; + int *codebook; + int r_idx, g_idx, b_idx, a_idx; + AVFrame *cache_frame; +} SubScaleContext; + + +static int config_output(AVFilterLink *outlink); + +static int check_exprs(AVFilterContext *ctx) +{ + const SubScaleContext *s = ctx->priv; + unsigned vars_w[VARS_NB] = { 0 }, vars_h[VARS_NB] = { 0 }; + + if (!s->w_pexpr && !s->h_pexpr) + return AVERROR(EINVAL); + + if (s->w_pexpr) + av_expr_count_vars(s->w_pexpr, vars_w, VARS_NB); + if (s->h_pexpr) + av_expr_count_vars(s->h_pexpr, vars_h, VARS_NB); + + if (vars_w[VAR_OUT_W] || vars_w[VAR_OW]) { + av_log(ctx, AV_LOG_ERROR, "Width expression cannot be self-referencing: '%s'.\n", s->w_expr); + return AVERROR(EINVAL); + } + + if (vars_h[VAR_OUT_H] || vars_h[VAR_OH]) { + av_log(ctx, AV_LOG_ERROR, "Height expression cannot be self-referencing: '%s'.\n", s->h_expr); + return AVERROR(EINVAL); + } + + if ((vars_w[VAR_OUT_H] || vars_w[VAR_OH]) && + (vars_h[VAR_OUT_W] || vars_h[VAR_OW])) { + av_log(ctx, AV_LOG_WARNING, "Circular references detected for width '%s' and height '%s' - possibly invalid.\n", s->w_expr, s->h_expr); + } + + if (s->margin_h_pexpr) + av_expr_count_vars(s->margin_h_pexpr, vars_w, VARS_NB); + if (s->margin_v_pexpr) + av_expr_count_vars(s->margin_v_pexpr, vars_h, VARS_NB); + + return 0; +} + +static int scale_parse_expr(AVFilterContext *ctx, char *str_expr, AVExpr **pexpr_ptr, const char *var, const char *args) +{ + SubScaleContext *s = ctx->priv; + int ret, is_inited = 0; + char *old_str_expr = NULL; + AVExpr *old_pexpr = NULL; + + if (str_expr) { + old_str_expr = av_strdup(str_expr); + if (!old_str_expr) + return AVERROR(ENOMEM); + av_opt_set(s, var, args, 0); + } + + if (*pexpr_ptr) { + old_pexpr = *pexpr_ptr; + *pexpr_ptr = NULL; + is_inited = 1; + } + + ret = av_expr_parse(pexpr_ptr, args, var_names, + NULL, NULL, NULL, NULL, 0, ctx); + if (ret < 0) { + av_log(ctx, AV_LOG_ERROR, "Cannot parse expression for %s: '%s'\n", var, args); + goto revert; + } + + ret = check_exprs(ctx); + if (ret < 0) + goto revert; + + if (is_inited && (ret = config_output(ctx->outputs[0])) < 0) + goto revert; + + av_expr_free(old_pexpr); + av_freep(&old_str_expr); + + return 0; + +revert: + av_expr_free(*pexpr_ptr); + *pexpr_ptr = NULL; + if (old_str_expr) { + av_opt_set(s, var, old_str_expr, 0); + av_free(old_str_expr); + } + if (old_pexpr) + *pexpr_ptr = old_pexpr; + + return ret; +} + +static av_cold int init_dict(AVFilterContext *ctx, AVDictionary **opts) +{ + SubScaleContext *s = ctx->priv; + uint8_t rgba_map[4]; + int ret; + + if (!s->w_expr) + av_opt_set(s, "w", "iw", 0); + if (!s->h_expr) + av_opt_set(s, "h", "ih", 0); + + ret = scale_parse_expr(ctx, NULL, &s->w_pexpr, "width", s->w_expr); + if (ret < 0) + return ret; + + ret = scale_parse_expr(ctx, NULL, &s->h_pexpr, "height", s->h_expr); + if (ret < 0) + return ret; + + av_log(ctx, AV_LOG_VERBOSE, "w:%s h:%s\n", + s->w_expr, s->h_expr); + + if (!s->margin_h_expr) + av_opt_set(s, "margin_h", "0", 0); + if (!s->margin_v_expr) + av_opt_set(s, "margin_v", "0", 0); + + ret = scale_parse_expr(ctx, NULL, &s->margin_h_pexpr, "margin_h", s->margin_h_expr); + if (ret < 0) + return ret; + + ret = scale_parse_expr(ctx, NULL, &s->margin_v_pexpr, "margin_v", s->margin_v_expr); + if (ret < 0) + return ret; + + s->opts = *opts; + *opts = NULL; + + ff_fill_rgba_map(&rgba_map[0], AV_PIX_FMT_RGB32); + + s->r_idx = rgba_map[0]; // R + s->g_idx = rgba_map[1]; // G + s->b_idx = rgba_map[2]; // B + s->a_idx = rgba_map[3]; // A + + av_lfg_init(&s->lfg, 123456789); + + + return 0; +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + SubScaleContext *s = ctx->priv; + + av_frame_free(&s->cache_frame); + + av_expr_free(s->w_pexpr); + av_expr_free(s->h_pexpr); + s->w_pexpr = s->h_pexpr = NULL; + + av_expr_free(s->margin_h_pexpr); + av_expr_free(s->margin_v_pexpr); + s->margin_h_pexpr = s->margin_v_pexpr = NULL; + + sws_freeContext(s->sws); + s->sws = NULL; + av_dict_free(&s->opts); + + avpriv_elbg_free(&s->ctx); + + av_freep(&s->codebook); + av_freep(&s->codeword); + av_freep(&s->codeword_closest_codebook_idxs); +} + +static int config_input(AVFilterLink *inlink) +{ + ////const AVFilterContext *ctx = inlink->dst; + ////SubScaleContext *s = ctx->priv; + + ////if (s->w <= 0 || s->h <= 0) { + //// s->w = inlink->w; + //// s->h = inlink->h; + ////} + return 0; +} + +static int scale_eval_dimensions(AVFilterContext *ctx) +{ + SubScaleContext *s = ctx->priv; + const AVFilterLink *inlink = ctx->inputs[0]; + char *expr; + int eval_w, eval_h, margin_h, margin_v; + int ret; + double res; + + s->var_values[VAR_IN_W] = s->var_values[VAR_IW] = inlink->w; + s->var_values[VAR_IN_H] = s->var_values[VAR_IH] = inlink->h; + s->var_values[VAR_OUT_W] = s->var_values[VAR_OW] = NAN; + s->var_values[VAR_OUT_H] = s->var_values[VAR_OH] = NAN; + s->var_values[VARS_B_H] = s->var_values[VARS_B_V] = 0; + s->var_values[VAR_A] = (double) inlink->w / inlink->h; + s->var_values[VAR_SAR] = inlink->sample_aspect_ratio.num ? + (double) inlink->sample_aspect_ratio.num / inlink->sample_aspect_ratio.den : 1; + s->var_values[VAR_DAR] = s->var_values[VAR_A] * s->var_values[VAR_SAR]; + + res = av_expr_eval(s->w_pexpr, s->var_values, NULL); + s->var_values[VAR_OUT_W] = s->var_values[VAR_OW] = (int) res == 0 ? inlink->w : (int) res; + + res = av_expr_eval(s->h_pexpr, s->var_values, NULL); + if (isnan(res)) { + expr = s->h_expr; + ret = AVERROR(EINVAL); + goto fail; + } + eval_h = s->var_values[VAR_OUT_H] = s->var_values[VAR_OH] = (int) res == 0 ? inlink->h : (int) res; + + res = av_expr_eval(s->w_pexpr, s->var_values, NULL); + if (isnan(res)) { + expr = s->w_expr; + ret = AVERROR(EINVAL); + goto fail; + } + eval_w = s->var_values[VAR_OUT_W] = s->var_values[VAR_OW] = (int) res == 0 ? inlink->w : (int) res; + + s->w = eval_w; + s->h = eval_h; + + res = av_expr_eval(s->margin_h_pexpr, s->var_values, NULL); + s->var_values[VARS_B_H] = (int)res; + + res = av_expr_eval(s->margin_v_pexpr, s->var_values, NULL); + if (isnan(res)) { + expr = s->margin_v_expr; + ret = AVERROR(EINVAL); + goto fail; + } + margin_v = s->var_values[VARS_B_V] = (int)res; + + res = av_expr_eval(s->margin_h_pexpr, s->var_values, NULL); + if (isnan(res)) { + expr = s->margin_h_expr; + ret = AVERROR(EINVAL); + goto fail; + } + margin_h = s->var_values[VARS_B_H] = (int)res; + + s->margin_h = margin_h; + s->margin_v = margin_v; + + return 0; + +fail: + av_log(ctx, AV_LOG_ERROR, + "Error when evaluating the expression '%s'.\n", expr); + return ret; +} + +static int config_output(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + AVFilterLink *inlink = outlink->src->inputs[0]; + SubScaleContext *s = ctx->priv; + int ret; + + outlink->format = AV_SUBTITLE_FMT_BITMAP; + outlink->time_base = ctx->inputs[0]->time_base; + + if ((ret = scale_eval_dimensions(ctx)) < 0) + goto fail; + + ff_scale_adjust_dimensions(inlink, &s->w, &s->h, + s->force_original_aspect_ratio, 2); + + if (s->w > INT_MAX || + s->h > INT_MAX || + (s->h * inlink->w) > INT_MAX || + (s->w * inlink->h) > INT_MAX) + av_log(ctx, AV_LOG_ERROR, "Rescaled value for width or height is too big.\n"); + + outlink->w = s->w; + outlink->h = s->h; + + if (s->sws) + sws_freeContext(s->sws); + + s->sws = sws_alloc_context(); + if (!s->sws) + return AVERROR(ENOMEM); + + av_opt_set_pixel_fmt(s->sws, "src_format", AV_PIX_FMT_PAL8, 0); + av_opt_set_int(s->sws, "dst_format", AV_PIX_FMT_RGB32, 0); + av_opt_set_int(s->sws, "threads", ff_filter_get_nb_threads(ctx), 0); + + if (s->opts) { + const AVDictionaryEntry *e = NULL; + while ((e = av_dict_get(s->opts, "", e, AV_DICT_IGNORE_SUFFIX))) { + if ((ret = av_opt_set(s->sws, e->key, e->value, 0)) < 0) + return ret; + } + } + + if ((ret = sws_init_context(s->sws, NULL, NULL)) < 0) + return ret; + + if (inlink->sample_aspect_ratio.num){ + outlink->sample_aspect_ratio = av_mul_q((AVRational){outlink->h * inlink->w, outlink->w * inlink->h}, inlink->sample_aspect_ratio); + } else + outlink->sample_aspect_ratio = inlink->sample_aspect_ratio; + + av_log(ctx, AV_LOG_VERBOSE, "Output size set to %dx%d.\n", outlink->w, outlink->h); + + return 0; +fail: + return ret; +} + +static int palettize_image(SubScaleContext *const s, const int w, const int h, const int src_linesize, uint8_t *src_data, + int dst_linesize, uint8_t *dst_data, uint32_t *dst_pal) +{ + int k, ret; + const int codeword_length = w * h; + + /* Re-Initialize */ + s->codeword = av_realloc_f(s->codeword, codeword_length, 4 * sizeof(*s->codeword)); + if (!s->codeword) + return AVERROR(ENOMEM); + + s->codeword_closest_codebook_idxs = + av_realloc_f(s->codeword_closest_codebook_idxs, codeword_length, + sizeof(*s->codeword_closest_codebook_idxs)); + if (!s->codeword_closest_codebook_idxs) + return AVERROR(ENOMEM); + + s->codebook = av_realloc_f(s->codebook, s->num_output_colors, 4 * sizeof(*s->codebook)); + if (!s->codebook) + return AVERROR(ENOMEM); + + /* build the codeword */ + k = 0; + for (int i = 0; i < h; i++) { + uint8_t *p = src_data; + for (int j = 0; j < w; j++) { + s->codeword[k++] = p[s->b_idx]; + s->codeword[k++] = p[s->g_idx]; + s->codeword[k++] = p[s->r_idx]; + s->codeword[k++] = p[s->a_idx]; + p += 4; + } + src_data += src_linesize; + } + + /* compute the codebook */ + ret = avpriv_elbg_do(&s->ctx, s->codeword, 4, + codeword_length, s->codebook, + s->num_output_colors, 1, + s->codeword_closest_codebook_idxs, &s->lfg, 0); + + if (ret < 0) + return ret; + + /* Write Palette */ + for (int i = 0; i < s->num_output_colors; i++) { + dst_pal[i] = s->codebook[i*4+3] << 24 | + (s->codebook[i*4+2] << 16) | + (s->codebook[i*4+1] << 8) | + s->codebook[i*4 ]; + } + + /* Write Image */ + k = 0; + for (int i = 0; i < h; i++) { + uint8_t *p = dst_data; + for (int j = 0; j < w; j++, p++) { + p[0] = s->codeword_closest_codebook_idxs[k++]; + } + + dst_data += dst_linesize; + } + + return ret; +} + +static int rescale_size(int64_t a, AVRational factor) +{ + const int64_t res = av_rescale_rnd(a, factor.num, factor.den, AV_ROUND_NEAR_INF); + if (res > INT32_MAX || res < 0) + return 0; + + return (int)res; +} + + +static int scale_area(AVFilterLink *link, AVSubtitleArea *area, const int target_width, const int target_height) +{ + const AVFilterContext *ctx = link->dst; + SubScaleContext *s = ctx->priv; + int ret; + + AVBufferRef *dst_buffer; + const uint8_t* data[2] = { area->buf[0]->data, (uint8_t *)&area->pal }; + const int dstW = FFALIGN(target_width, s->bitmap_width_align); + const int dstH = FFALIGN(target_height, s->bitmap_height_align); + const int tmp_linesize[2] = { FFALIGN(dstW * 4, 32), 0 }; + const int dst_linesize[2] = { dstW, 0 }; + uint8_t* tmp[2] = { 0, 0 }; + + AVBufferRef *tmp_buffer = av_buffer_allocz(tmp_linesize[0] * dstH); + if (!tmp_buffer) + return AVERROR(ENOMEM); + + if (!s->sws) + return 0; + + tmp[0] = tmp_buffer->data; + + s->sws = sws_getCachedContext(s->sws, area->w, area->h, AV_PIX_FMT_PAL8, + dstW, dstH, AV_PIX_FMT_RGB32, SWS_BICUBIC, NULL, NULL, NULL); + if (!s->sws) { + av_log(NULL, AV_LOG_FATAL, "Cannot initialize the conversion context. dstW=%d dstH=%d\n", dstW, dstH); + return AVERROR(EINVAL); + } + + // Rescale to ARGB + ret = sws_scale(s->sws, data, area->linesize, 0, area->h, tmp, tmp_linesize); + if (ret < 0) { + av_buffer_unref(&tmp_buffer); + return ret; + } + + // Alloc output buffer + dst_buffer = av_buffer_allocz(dst_linesize[0] * dstH); + if (!dst_buffer) { + av_buffer_unref(&tmp_buffer); + return AVERROR(ENOMEM); + } + + // Quantize to palettized image + ret = palettize_image(s, dstW, dstH, tmp_linesize[0], tmp[0], dst_linesize[0], dst_buffer->data, area->pal); + av_buffer_unref(&tmp_buffer); + + if (ret < 0) { + av_buffer_unref(&dst_buffer); + return ret; + } + + av_buffer_unref(&area->buf[0]); + ret = av_buffer_replace(&area->buf[0], dst_buffer); + if (ret < 0) { + av_buffer_unref(&dst_buffer); + return ret; + } + + area->w = dstW; + area->h = dstH; + area->linesize[0] = dst_linesize[0]; + area->nb_colors = s->num_output_colors; + + return ret; +} + +static int process_area(AVFilterLink *inlink, AVSubtitleArea *area, AVRational x_factor, AVRational y_factor) +{ + AVFilterContext *ctx = inlink->dst; + const SubScaleContext *s = ctx->priv; + int target_w, target_h, target_x, target_y; + const int border_l = s->margin_h; + const int border_r = s->w - s->margin_h; + const int border_t = s->margin_v; + const int border_b = s->h - s->margin_v; + + av_log(ctx, AV_LOG_DEBUG, "process_area - start: x/y: (%d:%d) size: %dx%d scale_mode: %d x-factor: %d:%d y-factor: %d:%d\n", + area->x, area->y, area->w, area->h, s->scale_mode, x_factor.num, x_factor.den, y_factor.num, y_factor.den); + + switch (s->scale_mode) { + case SSM_NONE: + target_w = area->w; + target_h = area->h; + target_x = area->x; + target_y = area->y; + break; + case SSM_UNIFORM: + target_w = rescale_size(area->w, x_factor); + target_h = rescale_size(area->h, y_factor); + target_x = rescale_size(area->x, x_factor); + target_y = rescale_size(area->y, y_factor); + break; + case SSM_UNIFORM_NO_REPOSITION: + target_w = rescale_size(area->w, x_factor); + target_h = rescale_size(area->h, y_factor); + target_x = area->x; + target_y = area->y; + break; + default: + av_log(ctx, AV_LOG_ERROR, "Invalid scale_mode: %d\n", s->scale_mode); + return AVERROR(EINVAL); + } + + av_log(ctx, AV_LOG_DEBUG, "process_area - scaled: x/y: (%d:%d) size: %dx%d.\n", target_x, target_y, target_w, target_h); + + + switch (s->arrange_mode_h) { + case SAM_ENSUREMARGIN_AND_SCALE: + case SAM_SNAPALIGNMENT_AND_SCALE: + { + // IF it doesn't fit - scale it + int max_width = s->w - 2 * s->margin_h; + + if (max_width < 2) + max_width = 2; + + if (target_w > max_width) { + target_h = (int)av_rescale(target_h, max_width, target_w); + target_w = max_width; + target_x = s->margin_h; + } + } + break; + } + + switch (s->arrange_mode_h) { + case SAM_NONE: + break; + case SAM_ENSUREMARGIN_NO_SCALE: + case SAM_ENSUREMARGIN_AND_SCALE: + // Left border + if (target_x < border_l) + target_x = border_l; + + // Right border + if (target_x + target_w > border_r) + target_x = border_r - target_w; + + break; + case SAM_SNAPALIGNMENT_NO_SCALE: + case SAM_SNAPALIGNMENT_AND_SCALE: + { + // Use original values to detect alignment + const int left_margin = area->x; + const int right_margin = inlink->w - area->x - area->w; + const AVRational diff_factor_r = { left_margin - right_margin, area->w }; + const float diff_factor = (float)av_q2d(diff_factor_r); + + if (diff_factor > 0.2f) { + // Right aligned + target_x = border_r - target_w; + } else if (diff_factor < -0.2f) { + // Left aligned + target_x = border_l; + } else { + // Centered + target_x = (inlink->w - area->w) / 2; + } + } + + break; + default: + av_log(ctx, AV_LOG_ERROR, "Invalid arrange_mode_h: %d\n", s->arrange_mode_h); + return AVERROR(EINVAL); + } + + av_log(ctx, AV_LOG_DEBUG, "process_area - arr_h: x/y: (%d:%d) size: %dx%d.\n", target_x, target_y, target_w, target_h); + + + switch (s->arrange_mode_v) { + case SAM_ENSUREMARGIN_AND_SCALE: + case SAM_SNAPALIGNMENT_AND_SCALE: + { + // IF it doesn't fit - scale it + int max_height = s->h - 2 * s->margin_v; + + if (max_height < 2) + max_height = 2; + + if (target_h > max_height) { + target_w = (int)av_rescale(target_w, max_height, target_h); + target_h = max_height; + target_y = s->margin_v; + } + } + break; + } + + switch (s->arrange_mode_v) { + case SAM_NONE: + break; + case SAM_ENSUREMARGIN_NO_SCALE: + case SAM_ENSUREMARGIN_AND_SCALE: + // Top border + if (target_y < border_t) + target_y = border_t; + + // Bottom border + if (target_y + target_h > border_b) + target_y = border_b - target_h; + + break; + case SAM_SNAPALIGNMENT_NO_SCALE: + case SAM_SNAPALIGNMENT_AND_SCALE: + { + // Use original values to detect alignment + const int top_margin = area->y; + const int bottom_margin = inlink->h - area->y - area->h; + const AVRational diff_factor_r = { top_margin - bottom_margin, area->h }; + const float diff_factor = (float)av_q2d(diff_factor_r); + + if (diff_factor > 0.2f) { + // Bottom aligned + target_y = border_b - target_h; + } else if (diff_factor < -0.2f) { + // Top aligned + target_y = border_t; + } else { + // Centered + target_y = (inlink->h - area->h) / 2; + } + } + + break; + default: + av_log(ctx, AV_LOG_ERROR, "Invalid arrange_mode_v: %d\n", s->arrange_mode_v); + return AVERROR(EINVAL); + } + + av_log(ctx, AV_LOG_VERBOSE, "process_area - arr_v: x/y: (%d:%d) size: %dx%d.\n", target_x, target_y, target_w, target_h); + + area->x = target_x; + area->y = target_y; + + if (area->w != target_w || area->h != target_h) + return scale_area(inlink, area, target_w, target_h); + + return 0; +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *frame) +{ + const AVFilterContext *ctx = inlink->dst; + SubScaleContext *s = ctx->priv; + AVFilterLink *outlink = ctx->outputs[0]; + int ret; + + // just forward empty frames + if (frame->num_subtitle_areas == 0) { + av_frame_free(&s->cache_frame); + return ff_filter_frame(outlink, frame); + } + + if (s->use_caching && s->cache_frame && s->cache_frame->subtitle_pts + && s->cache_frame->subtitle_pts == frame->subtitle_pts) { + AVFrame *out = av_frame_clone(s->cache_frame); + if (!out) + return AVERROR(ENOMEM); + + ret = av_frame_copy_props(out, frame); + if (ret < 0) + return ret; + + av_log(inlink->dst, AV_LOG_DEBUG, "subscale CACHED - size %dx%d pts: %"PRId64" areas: %d\n", frame->width, frame->height, frame->subtitle_pts, frame->num_subtitle_areas); + av_frame_free(&frame); + return ff_filter_frame(outlink, out); + } + + ret = av_frame_make_writable(frame); + if (ret >= 0) { + const AVRational x_factor = { .num = outlink->w, .den = inlink->w} ; + const AVRational y_factor = { .num = outlink->h, .den = inlink->h} ; + + for (unsigned i = 0; i < frame->num_subtitle_areas; ++i) { + AVSubtitleArea *area = frame->subtitle_areas[i]; + + ret = process_area(inlink, area, x_factor, y_factor); + if (ret < 0) + return ret; + } + + av_log(inlink->dst, AV_LOG_DEBUG, "subscale output - size %dx%d pts: %"PRId64" areas: %d\n", frame->width, frame->height, frame->subtitle_pts, frame->num_subtitle_areas); + + if (s->use_caching) { + av_frame_free(&s->cache_frame); + s->cache_frame = av_frame_clone(frame); + } + + return ff_filter_frame(outlink, frame); + } + + return ret; +} + +#define OFFSET(x) offsetof(SubScaleContext, x) +#define FLAGS (AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_FILTERING_PARAM) + +static const AVOption subscale_options[] = { + { "margin_h", "horizontal border", OFFSET(margin_h_expr), AV_OPT_TYPE_STRING, .flags = FLAGS }, + { "margin_v", "vertical border", OFFSET(margin_v_expr), AV_OPT_TYPE_STRING, .flags = FLAGS }, + { "w", "Output video width", OFFSET(w_expr), AV_OPT_TYPE_STRING, .flags = FLAGS }, + { "width", "Output video width", OFFSET(w_expr), AV_OPT_TYPE_STRING, .flags = FLAGS }, + { "h", "Output video height", OFFSET(h_expr), AV_OPT_TYPE_STRING, .flags = FLAGS }, + { "height","Output video height", OFFSET(h_expr), AV_OPT_TYPE_STRING, .flags = FLAGS }, + { "force_original_aspect_ratio", "decrease or increase w/h if necessary to keep the original AR", OFFSET(force_original_aspect_ratio), AV_OPT_TYPE_INT, { .i64 = 0}, 0, 2, FLAGS, "force_oar" }, + { "disable", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = 0 }, 0, 0, FLAGS, "force_oar" }, + { "decrease", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = 1 }, 0, 0, FLAGS, "force_oar" }, + { "increase", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = 2 }, 0, 0, FLAGS, "force_oar" }, + { "scale_mode", "specify how to scale subtitles", OFFSET(scale_mode), AV_OPT_TYPE_INT, {.i64 = SSM_UNIFORM}, 0, SSM_UNIFORM_NO_REPOSITION, FLAGS, "scale_mode" }, + { "none", "no common scaling", 0, AV_OPT_TYPE_CONST, {.i64=SSM_NONE}, .flags = FLAGS, .unit = "scale_mode" }, + { "uniform", "uniformly scale and reposition", 0, AV_OPT_TYPE_CONST, {.i64=SSM_UNIFORM}, .flags = FLAGS, .unit = "scale_mode" }, + { "uniform_no_reposition", "uniformly scale but keep positions", 0, AV_OPT_TYPE_CONST, {.i64=SSM_UNIFORM_NO_REPOSITION}, .flags = FLAGS, .unit = "scale_mode" }, + { "use_caching", "Cache output frames", OFFSET(use_caching), AV_OPT_TYPE_BOOL, { .i64 = 1}, 0, 1, .flags = FLAGS }, + + { "arrange_h", "specify how to arrange subtitles horizontally", OFFSET(arrange_mode_h), AV_OPT_TYPE_INT, {.i64 = SAM_NONE}, 0, SAM_SNAPALIGNMENT_AND_SCALE, FLAGS, "arrange" }, + { "arrange_v", "specify how to arrange subtitles vertically", OFFSET(arrange_mode_v), AV_OPT_TYPE_INT, {.i64 = SAM_NONE}, 0, SAM_SNAPALIGNMENT_AND_SCALE, FLAGS, "arrange" }, + { "none", "no repositioning", 0, AV_OPT_TYPE_CONST, {.i64=SAM_NONE}, .flags = FLAGS, .unit = "arrange" }, + { "margin_no_scale", "move subs inside border when possible", 0, AV_OPT_TYPE_CONST, {.i64=SAM_ENSUREMARGIN_NO_SCALE,}, .flags = FLAGS, .unit = "arrange" }, + { "margin_and_scale", "move subs inside border and scale as needed", 0, AV_OPT_TYPE_CONST, {.i64=SAM_ENSUREMARGIN_AND_SCALE,}, .flags = FLAGS, .unit = "arrange" }, + { "snapalign_no_scale", "snap subs to near/far/center when possible", 0, AV_OPT_TYPE_CONST, {.i64=SAM_SNAPALIGNMENT_NO_SCALE,}, .flags = FLAGS, .unit = "arrange" }, + { "snapalign_and_scale", "snap subs to near/far/center and scale as needed", 0, AV_OPT_TYPE_CONST, {.i64=SAM_SNAPALIGNMENT_AND_SCALE,}, .flags = FLAGS, .unit = "arrange" }, + + { "eval", "specify when to evaluate expressions", OFFSET(eval_mode), AV_OPT_TYPE_INT, {.i64 = EVAL_MODE_INIT}, 0, EVAL_MODE_NB-1, FLAGS, "eval" }, + { "init", "eval expressions once during initialization", 0, AV_OPT_TYPE_CONST, {.i64=EVAL_MODE_INIT}, .flags = FLAGS, .unit = "eval" }, + { "frame", "eval expressions during initialization and per-frame", 0, AV_OPT_TYPE_CONST, {.i64=EVAL_MODE_FRAME}, .flags = FLAGS, .unit = "eval" }, + { "num_colors", "number of palette colors in output", OFFSET(num_output_colors), AV_OPT_TYPE_INT, {.i64 = 256 }, 2, 256, .flags = FLAGS }, + { "bitmap_width_align", "Bitmap width alignment", OFFSET(bitmap_width_align), AV_OPT_TYPE_INT, {.i64 = 2 }, 1, 256, .flags = FLAGS }, + { "bitmap_height_align", "Bitmap height alignment", OFFSET(bitmap_height_align), AV_OPT_TYPE_INT, {.i64 = 2 }, 1, 256, .flags = FLAGS }, + { .name = NULL } +}; + +static const AVClass subscale_class = { + .class_name = "subscale", + .item_name = av_default_item_name, + .option = subscale_options, + .version = LIBAVUTIL_VERSION_INT, + .category = AV_CLASS_CATEGORY_FILTER, +}; + +static const AVFilterPad inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .filter_frame = filter_frame, + .config_props = config_input, + }, +}; + +static const AVFilterPad outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .config_props = config_output, + }, +}; + +const AVFilter ff_sf_subscale = { + .name = "subscale", + .description = NULL_IF_CONFIG_SMALL("Scale graphical subtitles."), + .init_dict = init_dict, + .uninit = uninit, + .priv_size = sizeof(SubScaleContext), + .priv_class = &subscale_class, + FILTER_INPUTS(inputs), + FILTER_OUTPUTS(outputs), + FILTER_SINGLE_SUBFMT(AV_SUBTITLE_FMT_BITMAP), +}; + From patchwork Mon Dec 13 23:41:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 32472 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp6142247iog; Mon, 13 Dec 2021 15:44:19 -0800 (PST) X-Google-Smtp-Source: ABdhPJzBLj/xqgOWt0XKtPB+aFpYHWz8A7vENpQWv5TH3KtSxyX3WW+PtYqO+ools8IAL9XgWTs2 X-Received: by 2002:a17:906:4aca:: with SMTP id u10mr1616608ejt.305.1639439059617; Mon, 13 Dec 2021 15:44:19 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id ds8si3069694ejc.812.2021.12.13.15.44.19; Mon, 13 Dec 2021 15:44:19 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=K0azXka6; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B4B5268B116; Tue, 14 Dec 2021 01:41:12 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11olkn2056.outbound.protection.outlook.com [40.92.18.56]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 562EA68B0F1 for ; Tue, 14 Dec 2021 01:41:05 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RS6ntIf9sYX18Odw6vI1RqTtH4aYGgQ8z/Dl5uWgNVgR7nWztWvPnMVWECQH5qJiAnNOuBoZmMpnYgU48wlUbgvCycT5p9G932hD6ieJCDmLO34P1oFgkxNwfJ6pGCEGCIMEekfmzk/mLRu0PcRU4+FmOe0vFRLclkqIZEAaSzIKv6MxHbj45lYnnQMUpUDrYxYOIb1Mdr30Dgv/AmT0YdZy4wHN0ueIJoUYYTuM1J4Xus9uKD2sr66a82G9VnnMcM1b4+6TQHgkPWX20ZT/sm7Vkqp1yQQjKpiWHk13lK2MnREuhW2Jl+lJOSZ6q5LrIVG4ezBO44Qq7MIAhCq8gA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=beKelU5ktEdlwmbGSfbo7fGgJRSVN3nM3vUjbZlQIgU=; b=JISIksTp6SmPOMC++OTEu/Z06iHuMA3wQvHRwQVe2yzkMmmyPsioEROH8e4rHhPv0CZm+vcg3URyC3u94rlMB71HDB+d1tWxUZCxEcqYVExIkfVpZjnwII3RRh50Sjw9ZyUujjei1lQ5Cnx+kaFPa+/8kWx6xH28i6r2/8NLq3hLjWIPQaQSZMg5/1nhW6xFvUHouf45GF6Oq8PqLB8J1zVAYxCDMb7Ahwyvfz+g38nVmJuKgEpJJaZoyoArG4yK17/n9oWvWim0kl//ndqO+eTlXAtoYEeTs0lQFUAxtTFG1b8mTEhG2SW4AOfUvzSTy6rbsviUm9NE/GPLatVzVA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=beKelU5ktEdlwmbGSfbo7fGgJRSVN3nM3vUjbZlQIgU=; b=K0azXka68e7DNQZedC6xzNKR+lTEuaD/8Q3XwvIy7bb8OPnsvZFbFdm73DBVUnaa/hI31sGXhyqwkxJwseNrGKKX5BYYTH49iYTZDvG/e0ya0DKPurLGgUvcnsb2YYzqCwGemoBYcAPjf9MBSqUNvCSH0X6cEoNrDc0oach4E8OtFt3CASH3E455R/08zkYPqy9uMyq30BYtGgzEflbLW5aOgkeZINtBPqC1CPzMH0pp7LnMKcNZ9MWRUec9SyWxyFrA8K+TH8tBp6EbojXxfTPgSLA4b72BdttMhcL95LG8lmYqzGFgS2HmNvaWRBu8SG8EbTnXQNi8D9/yXmlJDw== Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::20) by DM8P223MB0397.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4778.17; Mon, 13 Dec 2021 23:41:02 +0000 Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775]) by DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775%7]) with mapi id 15.20.4778.018; Mon, 13 Dec 2021 23:41:02 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v24 21/21] doc/APIchanges: update for subtitle filtering changes Thread-Index: AQHX8HrjeOuOqUw2fEmh87ZEfulXBA== Date: Mon, 13 Dec 2021 23:41:02 +0000 Message-ID: References: <1ef42eb01b3627bbad539abd46206f320bb39697.1639438462.git.softworkz@hotmail.com> <4148d77a82d4b7e229fc81ce9bc1bc06904533f1.1639438462.git.softworkz@hotmail.com> <2ac798924ee984010d2e85f00bc992d9aa106171.1639438462.git.softworkz@hotmail.com> <300ea7188b331ca5e00dd7e04c03df8aba2a99c0.1639438462.git.softworkz@hotmail.com> <9474726b26f0d3e8f353d07caed2fbd33cdf473a.1639438462.git.softworkz@hotmail.com> <424522c96b5cdf6b03faa6bdceb1d25449463429.1639438462.git.softworkz@hotmail.com> <90caff245055005e6a4a890b3665da55aa853b31.1639438462.git.softworkz@hotmail.com> <0ebcf3ed2adb02d3ffb243a499b434c3bdd976a7.1639438462.git.softworkz@hotmail.com> <4b5c1c1185075fd6e5d8ef70e07fdce6f896d699.1639438462.git.softworkz@hotmail.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [1KaZcxZoVDodS9NFXZThN5tkNXNhSERi] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: ea34f79a-2159-42a3-56b4-08d9be920647 x-ms-traffictypediagnostic: DM8P223MB0397:EE_ x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: PYpM0dEiRlyg6JXL524FbzTHg4gdhrahH8FowenZ9uCcG7XUGejGgCY8UNPIPDiFdMm6bz/TPgHFyeiES7PsT8HS0IbnV7GlRjaaKNurW1Btv1/sR1zTEWPNfdQiFUNkK8kCw77s0Njv0+1uvXM3E+U+BW8ypIyBXciGpYITXD+u+eGAAtqBKgu5S1Q8wo8F+Ekyt+zgY5DyzHuP0UYT8+C5ZB+qngJP3WocJmQPt6VPik88Bh9U3Z7/cTTuBEo/+NYi+VxVrqISflvG/kXavM+j8uOD1N+jywfhB8cdo8sN7X65T8U4ysM/ctAbcYj6edYEXIfBkrX+d9og1CpMkFW9dXR4IvRM9VvvOkA3Dyncjr5Px5ULnkdoWpJDHY+at3c91dwLaBOMyynZKB5ni/SJeeZDdu7LdjAdgNSJNlW49d3l8WkFNP9JSepGRtsug6XHDU3873iE2e3bFDfyOzam+tXEUF5euR7hgVcirsfDRk1enBMUnPNyqsiOoai91RRt5xuA8lgUCvzMqqxY3NTTwuSvyr23Uk215Wm0xxbRYIFUOrGSv8yhScElTGQX+XXup9DgxCXERJyNm0a3rw== x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?q?75ATQlRWEdAdVSsj2I0GzQPa8dPz?= =?utf-8?q?zTLsAnfMzWnT+8ZYML8IcvXL1i0a7TG7V39iLrDVDpdtJF3E/mMGa6Ivgx3vYfdd1?= =?utf-8?q?LY33BkABQJFCQ4WsnFsQZmPdEgpzv3UPspDq3+YuYSSFKSJfVVgY7sNWEZPYzUfF3?= =?utf-8?q?quqPwnyD9VgNBcxm/ly9ZCkBqVfOSYnws73pm4HCZgqxtPWY5s0VVd6RVEZmuwjvx?= =?utf-8?q?Fmfe3LvHqMEEX8f/j77tyiYQunrOiVpTGRa5l7DPspGR/1/eIrzy52KsfrU5nk7mP?= =?utf-8?q?dxVSBJsB/EZHlOzCVrUnK1+WVO2mmhxZyChZLzykP8pJ7PwiFBocKaf3/cx6HY81C?= =?utf-8?q?ceREDOOkYh3wy1WC7IQfwLCUFXBg/YuFDblTAKr0dndeQza8WDI12HCvis93/Iw7i?= =?utf-8?q?M0qq1rJrsXaibjD8QqaQ/GnH5sLbFd1m33pTte91FZpfvzJPrA4DRikofuxMuq+Z2?= =?utf-8?q?7vON+8fonWGW4Qa4Y5Hh8brFvsD8hlx6X1rGnNpVg+c6ljP33a6AFRMtr52ZFf/EC?= =?utf-8?q?3Ce7akyq47M+PNCJKQK4BLhPSEeTDb76N+KWrEuYCIUdbIWRGDdDQ9BArFLTEWnmQ?= =?utf-8?q?FOQCKJqo7ypsFDl8LoF45GeP9+Avvd08+sx5kROxndcOqXlCPFComextE56C3Kawc?= =?utf-8?q?1d9c34qL/bs9eWL5+T0DRgFAIdKIZnE4pdoyIV//BvLWOtVfT6FBpNWV8UPOpXBK+?= =?utf-8?q?4/XIzeWryWcr9ZDmq1esjCvXnSn4pmw5c/usPcN3n64nJYm4vmqytGTGymV9e2QuD?= =?utf-8?q?rwOGVOayFfG/BYM+iS3qI9QVvUIBATRRc17roAlLKcmPE7sRXEStr4RU9FCWqkQuW?= =?utf-8?q?9RopjTGuUjx7HOEMHfi1TsydqVKnet0V6zZ7+MYEMsjsGJvyI5ez9ee+5JGsh9fBH?= =?utf-8?q?o+U6AfD9PCw5p9sjh1bN3yKG+fpGo6P/kyNQIUJ+2WBA=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-1ff67.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: ea34f79a-2159-42a3-56b4-08d9be920647 X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Dec 2021 23:41:02.9317 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8P223MB0397 Subject: [FFmpeg-devel] [PATCH v24 21/21] doc/APIchanges: update for subtitle filtering changes X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: m3p6PAGQmRTn Signed-off-by: softworkz --- doc/APIchanges | 23 +++++++++++++++++++++++ libavcodec/version.h | 2 +- libavutil/version.h | 2 +- 3 files changed, 25 insertions(+), 2 deletions(-) diff --git a/doc/APIchanges b/doc/APIchanges index 17aa664ca3..1e12225b60 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -14,6 +14,29 @@ libavutil: 2021-04-27 API changes, most recent first: +2021-12-xx - xxxxxxxxxx - lavc 59.15.100 - avcodec.h + Deprecate avcodec_encode_subtitle(), use regular encode api now + +2021-12-xx - xxxxxxxxxx - lavc 59.15.100 - codec_desc.h + Add avcodec_descriptor_get_subtitle_format() + +2021-12-xx - xxxxxxxxxx - lavc 59.15.100 - avcodec.h + Deprecate avsubtitle_free() + Deprecate avcodec_decode_subtitle2(), use regular decode api now + +2021-12-xx - xxxxxxxxxx - lavu 57.12.100 - frame.h + Add AVMediaType field to AVFrame + Add Fields for carrying subtitle data to AVFrame + (subtitle_areas, subtitle_header, subtitle_pts, start/end time, etc.) + Add av_frame_get_buffer2() and deprecate av_frame_get_buffer() + +2021-12-xx - xxxxxxxxxx - lavu 57.12.100 - subfmt.h + Add struct AVSubtitleArea (replaces AVSubtitle) + Add av_get_subtitle_fmt_name() and av_get_subtitle_fmt() + +2021-12-xx - xxxxxxxxxx - lavu 57.12.100 - subfmt.h + Add enum AVSubtitleType (moved from lavc), add new values, deprecate existing + 2021-12-xx - xxxxxxxxxx - lavf 59.10.100 - avformat.h Add AVFormatContext io_close2 which returns an int diff --git a/libavcodec/version.h b/libavcodec/version.h index 8a0b94f5aa..db71fb1130 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -28,7 +28,7 @@ #include "libavutil/version.h" #define LIBAVCODEC_VERSION_MAJOR 59 -#define LIBAVCODEC_VERSION_MINOR 14 +#define LIBAVCODEC_VERSION_MINOR 15 #define LIBAVCODEC_VERSION_MICRO 100 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ diff --git a/libavutil/version.h b/libavutil/version.h index cc8caadf50..ed2786597d 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,7 +79,7 @@ */ #define LIBAVUTIL_VERSION_MAJOR 57 -#define LIBAVUTIL_VERSION_MINOR 11 +#define LIBAVUTIL_VERSION_MINOR 12 #define LIBAVUTIL_VERSION_MICRO 100 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \