From patchwork Tue Oct 25 09:13:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 38976 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2684216pzb; Tue, 25 Oct 2022 02:14:11 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4UPs8vrfE+pNIWVZMn9BgegfrabqC3UGSFPCu5/Hz6PbKrr1DW02DjHVRuv5QP+nJ/YD9h X-Received: by 2002:a17:907:5c2:b0:77e:def7:65d8 with SMTP id wg2-20020a17090705c200b0077edef765d8mr31991722ejb.487.1666689250904; Tue, 25 Oct 2022 02:14:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666689250; cv=none; d=google.com; s=arc-20160816; b=Vwwd3WtCEThRFu9UVI1MkRoq1bC/Om/moNGd8/zt4Qxtda2IzKYxkTMaMoO8LRGB33 sXmqxxshoppAQnkCTVp03RqInX6WFNH+rzfpuE2pzcgliO94iAuGI93oijwPdi4iX32F JxDS4fEEG+PXDVJOUepiRQGZcS+hLfEsuD8CfdQumAWlk8/UBh12W6nrKOmNic2WSqMQ t7hOvQjLMZpBu45Y/vCJitIYlGTtM2DW1tRWHPb9/+BqpGTthjvFnPKN/R3ZoEzYFot6 HnRIhGzDUAz4Nhnzl5mxOy5HoHZUbPevkArkzoWJgPtnAEZMnu4HWFmraH2SPkw46P4Y zkWA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:references :in-reply-to:message-id:from:dkim-signature:delivered-to; bh=QnB4Bwirozpx7+R8M+YuFNtMankLBW8dyovjvvF+WDc=; b=fOcz4Fr9BQTs99qKNv7FrfyK23Qw6OyGG3Krvu4RqKOBX27JfO9XwS7Z238aDNCuw9 S0Xy5M7yUKLEzMpelEq1JOoTzp0V5Qft2/VxZKo889/xFT+AirqWsJuhhaiH0ep4iP+7 NJj7oZnWlgqYF/Yf22MY14oO14cH9ccpQOawB5TM4AN2smChFdJUzPixH45xkvmEp3TS E+e8D9lmjLmKqW+uqvOtK0E4hXjoay4Ci0m1FpxJQsdi7P65s4l2uFCIONXdOd04mlan F47blChRtGHIGX3cx66zLmtXr65nSr3GgKq4bxQcbChKBLUc8Wj/1Et+lK0IHz7kcOHr 3ivQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=oBzMMZ8O; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id hp16-20020a1709073e1000b007306ac0faa0si2090954ejc.615.2022.10.25.02.14.10; Tue, 25 Oct 2022 02:14:10 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=oBzMMZ8O; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2A69068BB94; Tue, 25 Oct 2022 12:13:58 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pg1-f176.google.com (mail-pg1-f176.google.com [209.85.215.176]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 60B5A68B9D8 for ; Tue, 25 Oct 2022 12:13:51 +0300 (EEST) Received: by mail-pg1-f176.google.com with SMTP id b5so10916207pgb.6 for ; Tue, 25 Oct 2022 02:13:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=Q5ow4anFTCYy0VRyhIFwswBnIQF4Qfk6tm2/Y8mb+xg=; b=oBzMMZ8OMk4or4i3bXvPAbezzU4COVQz2XudtuO7ECgRyt9jAvZJg0Uz2GHs91nI0M 2CWgf5FNDiwFTwVQ32oUim96NysNqWRfvbcm3iXak1N+H8SEGl7LItivwTvhU5dxRuEx 4811rz7r+rVmwGn8+Gp3a0d03GylDnmBeGu3dSOLs+6Tc2C9kda/eHJ1mxgaZERgdmIY w5Q5Df2NaGQLvD0tLPZGiuJuoc7qEVN36Bgms8Id2LMkjP/JbztzEZ0TmRGwIve9diLt M6mDkqLT+3v4TDCLJ6ymuzLZtTXcDV/6TeOH0hBjRzifgq6pt7A1Mrc0vD8CB6/TGM1K I0Ow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=Q5ow4anFTCYy0VRyhIFwswBnIQF4Qfk6tm2/Y8mb+xg=; b=AhFIz6oGXlb2ATpMEIL8fE7amvyJwBnjV4c0zyzNlH9cbvmC6XPidnZToz0eIxCkA9 wx/O+PvTqJF8Ds7/9xblXW7VLGvOgIPh+TAbRMQ4rTZ4r6BFjpaXL8XkWia4cP1JGrKU sMnZJ61KjnezDZrm9OOJLan3/2egdLN7nWEQj6ZQB5hEnSR3kJpiCLMdh/nIjHZ6xhGr RYUvA/up/M6FFRLndvBBXXvCg8zmf3Ytc3Saz5BAMfKrjaVe9Bj35uFUgRsxSZTsdgtc 3JGFdcRLOBE+J0g8F6hv8T2BdNb2vFRwE5ABjQSNTvkzq4LymFHmg85jK9e6kW0BuhiI 5SYg== X-Gm-Message-State: ACrzQf3mgMOWMAZZprAK5yrC4GYQpLxEDHi0+A46nKwhGh7Txy6eMmS9 KjtKZcuMNDHdxlEoPkCChcabqZ6ixMQ= X-Received: by 2002:a05:6a00:134f:b0:563:7aa8:5cea with SMTP id k15-20020a056a00134f00b005637aa85ceamr37310000pfu.69.1666689229546; Tue, 25 Oct 2022 02:13:49 -0700 (PDT) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id e13-20020a17090301cd00b00177c488fea5sm894823plh.12.2022.10.25.02.13.48 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Oct 2022 02:13:48 -0700 (PDT) From: softworkz X-Google-Original-From: softworkz Message-Id: In-Reply-To: References: Date: Tue, 25 Oct 2022 09:13:22 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v9 01/25] avcodec, avutil: Move enum AVSubtitleType to avutil, add new and deprecate old values X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: softworkz Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: J83cN0TQL9S/ From: softworkz Signed-off-by: softworkz --- libavcodec/avcodec.h | 19 +------------ libavutil/Makefile | 1 + libavutil/subfmt.h | 68 ++++++++++++++++++++++++++++++++++++++++++++ libavutil/version.h | 5 ++-- 4 files changed, 73 insertions(+), 20 deletions(-) create mode 100644 libavutil/subfmt.h diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index 3edd8e2636..bf06b01e22 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -35,6 +35,7 @@ #include "libavutil/frame.h" #include "libavutil/log.h" #include "libavutil/pixfmt.h" +#include "libavutil/subfmt.h" #include "libavutil/rational.h" #include "codec.h" @@ -2257,24 +2258,6 @@ typedef struct AVHWAccel { * @} */ -enum AVSubtitleType { - SUBTITLE_NONE, - - SUBTITLE_BITMAP, ///< A bitmap, pict will be set - - /** - * Plain text, the text field must be set by the decoder and is - * authoritative. ass and pict fields may contain approximations. - */ - SUBTITLE_TEXT, - - /** - * Formatted text, the ass field must be set by the decoder and is - * authoritative. pict and text fields may contain approximations. - */ - SUBTITLE_ASS, -}; - #define AV_SUBTITLE_FLAG_FORCED 0x00000001 typedef struct AVSubtitleRect { diff --git a/libavutil/Makefile b/libavutil/Makefile index 3d9c07aea8..74380cf917 100644 --- a/libavutil/Makefile +++ b/libavutil/Makefile @@ -77,6 +77,7 @@ HEADERS = adler32.h \ sha512.h \ spherical.h \ stereo3d.h \ + subfmt.h \ threadmessage.h \ time.h \ timecode.h \ diff --git a/libavutil/subfmt.h b/libavutil/subfmt.h new file mode 100644 index 0000000000..791b45519f --- /dev/null +++ b/libavutil/subfmt.h @@ -0,0 +1,68 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVUTIL_SUBFMT_H +#define AVUTIL_SUBFMT_H + +#include "version.h" + +enum AVSubtitleType { + + /** + * Subtitle format unknown. + */ + AV_SUBTITLE_FMT_NONE = -1, + + /** + * Subtitle format unknown. + */ + AV_SUBTITLE_FMT_UNKNOWN = 0, +#if FF_API_OLD_SUBTITLES + SUBTITLE_NONE = 0, ///< Deprecated, use AV_SUBTITLE_FMT_NONE instead. +#endif + + /** + * Bitmap area in AVSubtitleRect.data, pixfmt AV_PIX_FMT_PAL8. + */ + AV_SUBTITLE_FMT_BITMAP = 1, +#if FF_API_OLD_SUBTITLES + SUBTITLE_BITMAP = 1, ///< Deprecated, use AV_SUBTITLE_FMT_BITMAP instead. +#endif + + /** + * Plain text in AVSubtitleRect.text. + */ + AV_SUBTITLE_FMT_TEXT = 2, +#if FF_API_OLD_SUBTITLES + SUBTITLE_TEXT = 2, ///< Deprecated, use AV_SUBTITLE_FMT_TEXT instead. +#endif + + /** + * Text Formatted as per ASS specification, contained AVSubtitleRect.ass. + */ + AV_SUBTITLE_FMT_ASS = 3, +#if FF_API_OLD_SUBTITLES + SUBTITLE_ASS = 3, ///< Deprecated, use AV_SUBTITLE_FMT_ASS instead. +#endif + + AV_SUBTITLE_FMT_NB, ///< number of subtitle formats, DO NOT USE THIS if you want to link with shared libav* because the number of formats might differ between versions. +}; + +#endif /* AVUTIL_SUBFMT_H */ diff --git a/libavutil/version.h b/libavutil/version.h index cb0c928bd0..009c628e35 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,8 +79,8 @@ */ #define LIBAVUTIL_VERSION_MAJOR 57 -#define LIBAVUTIL_VERSION_MINOR 39 -#define LIBAVUTIL_VERSION_MICRO 101 +#define LIBAVUTIL_VERSION_MINOR 40 +#define LIBAVUTIL_VERSION_MICRO 100 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \ LIBAVUTIL_VERSION_MINOR, \ @@ -115,6 +115,7 @@ #define FF_API_OLD_CHANNEL_LAYOUT (LIBAVUTIL_VERSION_MAJOR < 58) #define FF_API_AV_FOPEN_UTF8 (LIBAVUTIL_VERSION_MAJOR < 58) #define FF_API_PKT_DURATION (LIBAVUTIL_VERSION_MAJOR < 58) +#define FF_API_OLD_SUBTITLES (LIBAVUTIL_VERSION_MAJOR < 58) /** * @} From patchwork Tue Oct 25 09:13:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 38977 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2684287pzb; Tue, 25 Oct 2022 02:14:21 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5N4gs+3aCrnZJw9rx8NPGQjLRG/rMHt7fca0Dy7dVen/doqNFdxttWNJfkJvfQ4Mj9v9BK X-Received: by 2002:aa7:cad5:0:b0:454:88dc:2c22 with SMTP id l21-20020aa7cad5000000b0045488dc2c22mr34954748edt.352.1666689261275; Tue, 25 Oct 2022 02:14:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666689261; cv=none; d=google.com; s=arc-20160816; b=j4s08A85g6y8vD2UIbj3PT4fuxWVNhsrEm4kcNIHc2g2xYh37KuhHj8NgxRHyuZY6J fixaXwsaJKx7pLlWxCt7llMwo9v9+CKpyiZwWJugyfHKfY74yLuwfptksL2URK6LhBMd LL/RKK7iA01MNS4vpB2KztBJpaqiXNu8BWcKbOOelceNKchJPolGmr0qcc36AND187uH HPUq72K/ORV11Tssp366X0NYtHqmMiyiDLuSqxHx9k5rJqW2EJde02WHLryR2GQfovUN 8qXM6E8JYmt8F5dn8LrvTfVrDX0z57+AljaVjms/EvaeDn6Tr1qpczL4ZL1Bc8uvOT+E rinA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:references :in-reply-to:message-id:from:dkim-signature:delivered-to; bh=c6Uyd4cSn4NVgvuMgPag88OoysGWmpDrK+IPKLz5e+w=; b=RJqDkrwloghT9GXj54vx8vhppuFxr83/VyJIq9VdyW1zoK6qQUzT/LoJMPhnuNp49Y po0xbZinx9YL5zLdz3bS1+nE4OliPrG+rYDeuiYvIBDVW8lKtgA2Nph1FQL7Imvivrit 330ANi0ZvJgPZ9Ti35k5TsI2U802fAA+qU9wejEjJZ/ZBn1Hoky1m/WPUL05tmXM3817 MsMctFABELYL2RluuEAlhyyqd4MYBHB26DxqnJaIrZ4tkxDcR2qZePWK0zI6ovK038/k l85zTGxD0N3DS/k/EsKub5oXH4s0AvcMT1RuP9oB/8H91wvw+wi24nCEvMgX/8Cmu4a/ zs5Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=nPESxAsj; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id dn18-20020a17090794d200b0078e1b5b5bb8si2604113ejc.120.2022.10.25.02.14.20; Tue, 25 Oct 2022 02:14:21 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=nPESxAsj; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1414568BC1C; Tue, 25 Oct 2022 12:14:01 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f53.google.com (mail-pj1-f53.google.com [209.85.216.53]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9473A68B98B for ; Tue, 25 Oct 2022 12:13:52 +0300 (EEST) Received: by mail-pj1-f53.google.com with SMTP id r61-20020a17090a43c300b00212f4e9cccdso6429162pjg.5 for ; Tue, 25 Oct 2022 02:13:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=eLmYGiqGOGkueM7k/33Nkw1dJJZtJHWW+RqiX5hjzPc=; b=nPESxAsjLAj0L5p7JnbTV7X1f29tzNP49aYUImkOjrJ1d8CiSiT6o7W2sFLGdDTlIT 5/lHi76i3D7c+q9ZvqRYo5kAZbxHrrNj5OXF/5NJhKlwlY/Gsir8qMSbmdncyLDLH3W4 X5KHzCQhpCTSvhi8MuOVhqu+JobK/LNNa43k4vLM1sDyKPHyJC6Oq7AbV3rBJq2Y+eaN g7v9YkS3Qw+z4Aq47OS8geF+UrYXchTNacam7U8fqG7hmEHDQqTRYKor2oHWT3JOb/pM xvjawaD3QX49ScBxG7F4EeHwfmu2qpgoZr/0ToFEK3XNWQvsdvP0V2YD7Z9nmgIaIpPE aizQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=eLmYGiqGOGkueM7k/33Nkw1dJJZtJHWW+RqiX5hjzPc=; b=jc/B0HD8AzniAPI5d8yqqRH2LdTiXuBlQs+5Csu+kzy3TCBhKeUIXr+Lj5Gfg4SXSR mwQ9X1+L0nyyQV+DuuB6f4M4h7n8p1/3vCpy6rwPTp6OYE1q+9lSG2lZqRSUgccFz65+ o5cVCvirW8wP2qi2gNd6xbg4XXUJQMebegtd8NdEygORpprXhtLZOWAW53JhJB5ZwhoP cayi6U9CTFZG7gJH77OX0O8fwujXST5ZbVAz4i0n3MfvLk54Ga/eKhqprdKvq9/8tfXy b+REdrc/H/pJTDqwGFakuIK8fgK364zvi+1latfttMOTN9pSmmioHQ79UOrspB9SyzNq xtGg== X-Gm-Message-State: ACrzQf2fBJGls1oti4tCp0TRdEO65ZGLiZwO0k2dvu6/ewqwtCT+fOUK TRdKWl3/WdLTk4UUls3ide4bS6P/ZMA= X-Received: by 2002:a17:902:e845:b0:186:c8b9:24e1 with SMTP id t5-20020a170902e84500b00186c8b924e1mr679966plg.125.1666689230794; Tue, 25 Oct 2022 02:13:50 -0700 (PDT) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id c3-20020a17090a020300b00208c58d5a0esm4993660pjc.40.2022.10.25.02.13.49 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Oct 2022 02:13:50 -0700 (PDT) From: softworkz X-Google-Original-From: softworkz Message-Id: <95e9b5f39f921151f1da2aeba099088ddb1ed0ce.1666689226.git.ffmpegagent@gmail.com> In-Reply-To: References: Date: Tue, 25 Oct 2022 09:13:23 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v9 02/25] avutil/frame: Prepare AVFrame for subtitle handling X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: softworkz Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: z6I9d5GYtG3w From: softworkz Root commit for adding subtitle filtering capabilities. In detail: - Add type (AVMediaType) field to AVFrame Replaces previous way of distinction which was based on checking width and height to determine whether a frame is audio or video - Add subtitle fields to AVFrame - Add new struct AVSubtitleArea, similar to AVSubtitleRect, but different allocation logic. Cannot and must not be used interchangeably, hence the new struct Signed-off-by: softworkz --- libavutil/Makefile | 1 + libavutil/frame.c | 206 +++++++++++++++++++++++++++++++++++++++++---- libavutil/frame.h | 85 ++++++++++++++++++- libavutil/subfmt.c | 45 ++++++++++ libavutil/subfmt.h | 47 +++++++++++ 5 files changed, 363 insertions(+), 21 deletions(-) create mode 100644 libavutil/subfmt.c diff --git a/libavutil/Makefile b/libavutil/Makefile index 74380cf917..4ab4fcb01b 100644 --- a/libavutil/Makefile +++ b/libavutil/Makefile @@ -165,6 +165,7 @@ OBJS = adler32.o \ slicethread.o \ spherical.o \ stereo3d.o \ + subfmt.o \ threadmessage.o \ time.o \ timecode.o \ diff --git a/libavutil/frame.c b/libavutil/frame.c index de4ad1f94d..230673e4e1 100644 --- a/libavutil/frame.c +++ b/libavutil/frame.c @@ -26,6 +26,7 @@ #include "imgutils.h" #include "mem.h" #include "samplefmt.h" +#include "subfmt.h" #include "hwcontext.h" #if FF_API_OLD_CHANNEL_LAYOUT @@ -52,6 +53,9 @@ const char *av_get_colorspace_name(enum AVColorSpace val) return name[val]; } #endif + +static int frame_copy_subtitles(AVFrame *dst, const AVFrame *src, int copy_data); + static void get_frame_defaults(AVFrame *frame) { memset(frame, 0, sizeof(*frame)); @@ -77,7 +81,12 @@ FF_ENABLE_DEPRECATION_WARNINGS frame->colorspace = AVCOL_SPC_UNSPECIFIED; frame->color_range = AVCOL_RANGE_UNSPECIFIED; frame->chroma_location = AVCHROMA_LOC_UNSPECIFIED; - frame->flags = 0; + frame->num_subtitle_areas = 0; + frame->subtitle_areas = NULL; + frame->subtitle_header = NULL; + frame->repeat_sub = 0; + frame->subtitle_timing.start_pts = 0; + frame->subtitle_timing.duration = 0; } static void free_side_data(AVFrameSideData **ptr_sd) @@ -256,6 +265,23 @@ FF_ENABLE_DEPRECATION_WARNINGS } +static int get_subtitle_buffer(AVFrame *frame) +{ + // Buffers in AVFrame->buf[] are not used in case of subtitle frames. + // To accomodate with existing code, checking ->buf[0] to determine + // whether a frame is ref-counted or has data, we're adding a 1-byte + // buffer here, which marks the subtitle frame to contain data. + frame->buf[0] = av_buffer_alloc(1); + if (!frame->buf[0]) { + av_frame_unref(frame); + return AVERROR(ENOMEM); + } + + frame->extended_data = frame->data; + + return 0; +} + int av_frame_get_buffer(AVFrame *frame, int align) { if (frame->format < 0) @@ -263,23 +289,41 @@ int av_frame_get_buffer(AVFrame *frame, int align) FF_DISABLE_DEPRECATION_WARNINGS if (frame->width > 0 && frame->height > 0) - return get_video_buffer(frame, align); + frame->type = AVMEDIA_TYPE_VIDEO; else if (frame->nb_samples > 0 && (av_channel_layout_check(&frame->ch_layout) #if FF_API_OLD_CHANNEL_LAYOUT || frame->channel_layout || frame->channels > 0 #endif )) - return get_audio_buffer(frame, align); + frame->type = AVMEDIA_TYPE_AUDIO; FF_ENABLE_DEPRECATION_WARNINGS - return AVERROR(EINVAL); + return av_frame_get_buffer2(frame, align); +} + +int av_frame_get_buffer2(AVFrame *frame, int align) +{ + if (frame->format < 0) + return AVERROR(EINVAL); + + switch (frame->type) { + case AVMEDIA_TYPE_VIDEO: + return get_video_buffer(frame, align); + case AVMEDIA_TYPE_AUDIO: + return get_audio_buffer(frame, align); + case AVMEDIA_TYPE_SUBTITLE: + return get_subtitle_buffer(frame); + default: + return AVERROR(EINVAL); + } } static int frame_copy_props(AVFrame *dst, const AVFrame *src, int force_copy) { int ret, i; + dst->type = src->type; dst->key_frame = src->key_frame; dst->pict_type = src->pict_type; dst->sample_aspect_ratio = src->sample_aspect_ratio; @@ -316,6 +360,12 @@ FF_ENABLE_DEPRECATION_WARNINGS dst->colorspace = src->colorspace; dst->color_range = src->color_range; dst->chroma_location = src->chroma_location; + dst->repeat_sub = src->repeat_sub; + dst->subtitle_timing.start_pts = src->subtitle_timing.start_pts; + dst->subtitle_timing.duration = src->subtitle_timing.duration; + + if ((ret = av_buffer_replace(&dst->subtitle_header, src->subtitle_header)) < 0) + return ret; av_dict_copy(&dst->metadata, src->metadata, 0); @@ -363,6 +413,7 @@ FF_ENABLE_DEPRECATION_WARNINGS av_assert1(dst->ch_layout.nb_channels == 0 && dst->ch_layout.order == AV_CHANNEL_ORDER_UNSPEC); + dst->type = src->type; dst->format = src->format; dst->width = src->width; dst->height = src->height; @@ -395,7 +446,7 @@ FF_ENABLE_DEPRECATION_WARNINGS /* duplicate the frame data if it's not refcounted */ if (!src->buf[0]) { - ret = av_frame_get_buffer(dst, 0); + ret = av_frame_get_buffer2(dst, 0); if (ret < 0) goto fail; @@ -417,6 +468,10 @@ FF_ENABLE_DEPRECATION_WARNINGS } } + /* copy subtitle areas */ + if (src->type == AVMEDIA_TYPE_SUBTITLE) + frame_copy_subtitles(dst, src, 0); + if (src->extended_buf) { dst->extended_buf = av_calloc(src->nb_extended_buf, sizeof(*dst->extended_buf)); @@ -486,7 +541,7 @@ AVFrame *av_frame_clone(const AVFrame *src) void av_frame_unref(AVFrame *frame) { - int i; + unsigned i, n; if (!frame) return; @@ -505,6 +560,21 @@ void av_frame_unref(AVFrame *frame) av_buffer_unref(&frame->opaque_ref); av_buffer_unref(&frame->private_ref); + av_buffer_unref(&frame->subtitle_header); + + for (i = 0; i < frame->num_subtitle_areas; i++) { + AVSubtitleArea *area = frame->subtitle_areas[i]; + + for (n = 0; n < FF_ARRAY_ELEMS(area->buf); n++) + av_buffer_unref(&area->buf[n]); + + av_freep(&area->text); + av_freep(&area->ass); + av_free(area); + } + + av_freep(&frame->subtitle_areas); + if (frame->extended_data != frame->data) av_freep(&frame->extended_data); @@ -532,18 +602,28 @@ FF_ENABLE_DEPRECATION_WARNINGS int av_frame_is_writable(AVFrame *frame) { - int i, ret = 1; + AVSubtitleArea *area; + int ret = 1; /* assume non-refcounted frames are not writable */ if (!frame->buf[0]) return 0; - for (i = 0; i < FF_ARRAY_ELEMS(frame->buf); i++) + for (unsigned i = 0; i < FF_ARRAY_ELEMS(frame->buf); i++) if (frame->buf[i]) ret &= !!av_buffer_is_writable(frame->buf[i]); - for (i = 0; i < frame->nb_extended_buf; i++) + for (unsigned i = 0; i < frame->nb_extended_buf; i++) ret &= !!av_buffer_is_writable(frame->extended_buf[i]); + for (unsigned i = 0; i < frame->num_subtitle_areas; i++) { + area = frame->subtitle_areas[i]; + if (area) { + for (unsigned n = 0; n < FF_ARRAY_ELEMS(area->buf); n++) + if (area->buf[n]) + ret &= !!av_buffer_is_writable(area->buf[n]); + } + } + return ret; } @@ -556,6 +636,7 @@ int av_frame_make_writable(AVFrame *frame) return 0; memset(&tmp, 0, sizeof(tmp)); + tmp.type = frame->type; tmp.format = frame->format; tmp.width = frame->width; tmp.height = frame->height; @@ -575,7 +656,7 @@ FF_ENABLE_DEPRECATION_WARNINGS if (frame->hw_frames_ctx) ret = av_hwframe_get_buffer(frame->hw_frames_ctx, &tmp, 0); else - ret = av_frame_get_buffer(&tmp, 0); + ret = av_frame_get_buffer2(&tmp, 0); if (ret < 0) return ret; @@ -610,7 +691,12 @@ AVBufferRef *av_frame_get_plane_buffer(AVFrame *frame, int plane) uint8_t *data; int planes, i; - if (frame->nb_samples) { + switch(frame->type) { + case AVMEDIA_TYPE_VIDEO: + planes = 4; + break; + case AVMEDIA_TYPE_AUDIO: + { int channels = frame->ch_layout.nb_channels; #if FF_API_OLD_CHANNEL_LAYOUT @@ -624,8 +710,11 @@ FF_ENABLE_DEPRECATION_WARNINGS if (!channels) return NULL; planes = av_sample_fmt_is_planar(frame->format) ? channels : 1; - } else - planes = 4; + break; + } + default: + return NULL; + } if (plane < 0 || plane >= planes || !frame->extended_data[plane]) return NULL; @@ -768,22 +857,103 @@ FF_ENABLE_DEPRECATION_WARNINGS return 0; } +static int av_subtitle_area2area(AVSubtitleArea *dst, const AVSubtitleArea *src, int copy_data) +{ + dst->x = src->x; + dst->y = src->y; + dst->w = src->w; + dst->h = src->h; + dst->nb_colors = src->nb_colors; + dst->type = src->type; + dst->flags = src->flags; + + switch (dst->type) { + case AV_SUBTITLE_FMT_BITMAP: + + for (unsigned i = 0; i < AV_NUM_BUFFER_POINTERS; i++) { + if (src->h > 0 && src->w > 0 && src->buf[i]) { + dst->buf[0] = av_buffer_ref(src->buf[i]); + if (!dst->buf[i]) + return AVERROR(ENOMEM); + + if (copy_data) { + const int ret = av_buffer_make_writable(&dst->buf[i]); + if (ret < 0) + return ret; + } + + dst->linesize[i] = src->linesize[i]; + } + } + + memcpy(&dst->pal[0], &src->pal[0], sizeof(src->pal[0]) * 256); + + break; + case AV_SUBTITLE_FMT_TEXT: + + if (src->text) { + dst->text = av_strdup(src->text); + if (!dst->text) + return AVERROR(ENOMEM); + } + + break; + case AV_SUBTITLE_FMT_ASS: + + if (src->ass) { + dst->ass = av_strdup(src->ass); + if (!dst->ass) + return AVERROR(ENOMEM); + } + + break; + default: + + av_log(NULL, AV_LOG_ERROR, "Subtitle area has invalid format: %d", dst->type); + return AVERROR(ENOMEM); + } + + return 0; +} + +static int frame_copy_subtitles(AVFrame *dst, const AVFrame *src, int copy_data) +{ + dst->format = src->format; + + dst->num_subtitle_areas = src->num_subtitle_areas; + + if (src->num_subtitle_areas) { + dst->subtitle_areas = av_malloc_array(src->num_subtitle_areas, sizeof(AVSubtitleArea *)); + + for (unsigned i = 0; i < src->num_subtitle_areas; i++) { + dst->subtitle_areas[i] = av_mallocz(sizeof(AVSubtitleArea)); + av_subtitle_area2area(dst->subtitle_areas[i], src->subtitle_areas[i], copy_data); + } + } + + return 0; +} + int av_frame_copy(AVFrame *dst, const AVFrame *src) { if (dst->format != src->format || dst->format < 0) return AVERROR(EINVAL); -FF_DISABLE_DEPRECATION_WARNINGS - if (dst->width > 0 && dst->height > 0) + switch(dst->type) { + case AVMEDIA_TYPE_VIDEO: return frame_copy_video(dst, src); - else if (dst->nb_samples > 0 && + case AVMEDIA_TYPE_AUDIO: + if (dst->nb_samples > 0 && (av_channel_layout_check(&dst->ch_layout) #if FF_API_OLD_CHANNEL_LAYOUT || dst->channels > 0 #endif )) - return frame_copy_audio(dst, src); -FF_ENABLE_DEPRECATION_WARNINGS + return frame_copy_audio(dst, src); + break; + case AVMEDIA_TYPE_SUBTITLE: + return frame_copy_subtitles(dst, src, 1); + } return AVERROR(EINVAL); } diff --git a/libavutil/frame.h b/libavutil/frame.h index e60a82f6c0..9ce267bbc5 100644 --- a/libavutil/frame.h +++ b/libavutil/frame.h @@ -25,7 +25,6 @@ #ifndef AVUTIL_FRAME_H #define AVUTIL_FRAME_H -#include #include #include "avutil.h" @@ -35,6 +34,7 @@ #include "rational.h" #include "samplefmt.h" #include "pixfmt.h" +#include "subfmt.h" #include "version.h" @@ -293,7 +293,7 @@ typedef struct AVRegionOfInterest { } AVRegionOfInterest; /** - * This structure describes decoded (raw) audio or video data. + * This structure describes decoded (raw) audio, video or subtitle data. * * AVFrame must be allocated using av_frame_alloc(). Note that this only * allocates the AVFrame itself, the buffers for the data must be managed @@ -407,7 +407,7 @@ typedef struct AVFrame { /** * format of the frame, -1 if unknown or unset * Values correspond to enum AVPixelFormat for video frames, - * enum AVSampleFormat for audio) + * enum AVSampleFormat for audio, AVSubtitleType for subtitles) */ int format; @@ -712,6 +712,53 @@ typedef struct AVFrame { * Duration of the frame, in the same units as pts. 0 if unknown. */ int64_t duration; + + /** + * Media type of the frame (audio, video, subtitles..) + * + * See AVMEDIA_TYPE_xxx + */ + enum AVMediaType type; + + /** + * Number of items in the @ref subtitle_areas array. + */ + unsigned num_subtitle_areas; + + /** + * Array of subtitle areas, may be empty. + */ + AVSubtitleArea **subtitle_areas; + + /** + * Header containing style information for text subtitles. + */ + AVBufferRef *subtitle_header; + + /** + * Indicates that a subtitle frame is a repeated frame for arbitrating flow + * in a filter graph. + * The field subtitle_timing.start_pts always indicates the original presentation + * time, while the frame's pts field may be different. + */ + int repeat_sub; + + struct SubtitleTiming + { + /** + * The display start time, in AV_TIME_BASE. + * + * For subtitle frames, AVFrame.pts is populated from the packet pts value, + * which is not always the same as this value. + */ + int64_t start_pts; + + /** + * Display duration, in AV_TIME_BASE. + */ + int64_t duration; + + } subtitle_timing; } AVFrame; @@ -788,6 +835,8 @@ void av_frame_move_ref(AVFrame *dst, AVFrame *src); /** * Allocate new buffer(s) for audio or video data. * + * Note: For subtitle data, use av_frame_get_buffer2 + * * The following fields must be set on frame before calling this function: * - format (pixel format for video, sample format for audio) * - width and height for video @@ -807,9 +856,39 @@ void av_frame_move_ref(AVFrame *dst, AVFrame *src); * recommended to pass 0 here unless you know what you are doing. * * @return 0 on success, a negative AVERROR on error. + * + * @deprecated Use @ref av_frame_get_buffer2 instead and set @ref AVFrame.type + * before calling. */ +attribute_deprecated int av_frame_get_buffer(AVFrame *frame, int align); +/** + * Allocate new buffer(s) for audio, video or subtitle data. + * + * The following fields must be set on frame before calling this function: + * - format (pixel format for video, sample format for audio) + * - width and height for video + * - nb_samples and channel_layout for audio + * - type (AVMediaType) + * + * This function will fill AVFrame.data and AVFrame.buf arrays and, if + * necessary, allocate and fill AVFrame.extended_data and AVFrame.extended_buf. + * For planar formats, one buffer will be allocated for each plane. + * + * @warning: if frame already has been allocated, calling this function will + * leak memory. In addition, undefined behavior can occur in certain + * cases. + * + * @param frame frame in which to store the new buffers. + * @param align Required buffer size alignment. If equal to 0, alignment will be + * chosen automatically for the current CPU. It is highly + * recommended to pass 0 here unless you know what you are doing. + * + * @return 0 on success, a negative AVERROR on error. + */ +int av_frame_get_buffer2(AVFrame *frame, int align); + /** * Check if the frame data is writable. * diff --git a/libavutil/subfmt.c b/libavutil/subfmt.c new file mode 100644 index 0000000000..c72ebe2a43 --- /dev/null +++ b/libavutil/subfmt.c @@ -0,0 +1,45 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include +#include "common.h" +#include "subfmt.h" + +static const char sub_fmt_info[AV_SUBTITLE_FMT_NB][24] = { + [AV_SUBTITLE_FMT_UNKNOWN] = "Unknown subtitle format", + [AV_SUBTITLE_FMT_BITMAP] = "Graphical subtitles", + [AV_SUBTITLE_FMT_TEXT] = "Text subtitles (plain)", + [AV_SUBTITLE_FMT_ASS] = "Text subtitles (ass)", +}; + +const char *av_get_subtitle_fmt_name(enum AVSubtitleType sub_fmt) +{ + if (sub_fmt < 0 || sub_fmt >= AV_SUBTITLE_FMT_NB) + return NULL; + return sub_fmt_info[sub_fmt]; +} + +enum AVSubtitleType av_get_subtitle_fmt(const char *name) +{ + for (int i = 0; i < AV_SUBTITLE_FMT_NB; i++) + if (!strcmp(sub_fmt_info[i], name)) + return i; + return AV_SUBTITLE_FMT_NONE; +} diff --git a/libavutil/subfmt.h b/libavutil/subfmt.h index 791b45519f..3b8a0613b2 100644 --- a/libavutil/subfmt.h +++ b/libavutil/subfmt.h @@ -21,6 +21,9 @@ #ifndef AVUTIL_SUBFMT_H #define AVUTIL_SUBFMT_H +#include + +#include "buffer.h" #include "version.h" enum AVSubtitleType { @@ -65,4 +68,48 @@ enum AVSubtitleType { AV_SUBTITLE_FMT_NB, ///< number of subtitle formats, DO NOT USE THIS if you want to link with shared libav* because the number of formats might differ between versions. }; +typedef struct AVSubtitleArea { +#define AV_NUM_BUFFER_POINTERS 1 + + enum AVSubtitleType type; + int flags; + + int x; ///< top left corner of area. + int y; ///< top left corner of area. + int w; ///< width of area. + int h; ///< height of area. + int nb_colors; ///< number of colors in bitmap palette (@ref pal). + + /** + * Buffers and line sizes for the bitmap of this subtitle. + * + * @{ + */ + AVBufferRef *buf[AV_NUM_BUFFER_POINTERS]; + int linesize[AV_NUM_BUFFER_POINTERS]; + /** + * @} + */ + + uint32_t pal[256]; ///< RGBA palette for the bitmap. + + char *text; ///< 0-terminated plain UTF-8 text + char *ass; ///< 0-terminated ASS/SSA compatible event line. + +} AVSubtitleArea; + +/** + * Return the name of sub_fmt, or NULL if sub_fmt is not + * recognized. + */ +const char *av_get_subtitle_fmt_name(enum AVSubtitleType sub_fmt); + +/** + * Return a subtitle format corresponding to name, or AV_SUBTITLE_FMT_NONE + * on error. + * + * @param name Subtitle format name. + */ +enum AVSubtitleType av_get_subtitle_fmt(const char *name); + #endif /* AVUTIL_SUBFMT_H */ From patchwork Tue Oct 25 09:13:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 38978 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2684353pzb; Tue, 25 Oct 2022 02:14:32 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5Hgc5OcLo34HUxemDapTUG64GJeLvlSLFkE78+Wgz0YksrwKh0PrEHPTc1NwXt+3ZEs241 X-Received: by 2002:a17:907:763b:b0:7aa:1ae6:900b with SMTP id jy27-20020a170907763b00b007aa1ae6900bmr5883634ejc.416.1666689272362; Tue, 25 Oct 2022 02:14:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666689272; cv=none; d=google.com; s=arc-20160816; b=evjufe1hrU2ZYuE/jg2onFOhfDmGPRNjlmDpta97FAiWHnqj9e685rbi0NZ94/49+4 T4mx8BSvFAG2iwDMrQeJPtT/WCCDVmpqkuzdmKzZbBHPAWf/BTJ02L0tIL+/eUlnpOp8 DvAFw3E7EVA3ScalC9gR76J46cVfW9tJzH3DGAe4uLpJvYfsF/C9SgyQqTzHTX6D75N7 2OuXghsKXp5oG6mNUZfvImpuQ5ZAdrTeK6sqyRUrwVAoKQlm3HvXwL3gGJLeH1y4Z50P 2ynC1UCCHEA7ELJG7wztmNgp+qKPHhP3F57x0dE0BBk3ZAFjP27Qoo35o7qxcOhyB099 9kzA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:references :in-reply-to:message-id:from:dkim-signature:delivered-to; bh=MOSJipIte045KVLSeTczSu9DCQU0M7IbI/CWuknAPic=; b=D3IXVxpTF+FkssWJFo40LgcEr3FZuD/dpygFPsjSu25/SgsKUwt2AUFsxcqM4Oavoy IMcGWJglQl+k19ndUYQprq/82CDkrN450rpmwAsisvoAbIpl+QG7lKSm++Uoi6asvg9P LRxutJtHT/RQPjNhYAC9NahdT9g4oF2ZLz0YJQx0LEYafKYdlLfivVKf9/kiMRVxaWud Rody5gUivNc8DLd1S5k/pfk84JvyhPwT0ce15oS47Dk0IYjWiJ4DapFVNr13J/fOll1a eDJ4LfhHF6/+0+5yPSfqgPmVdcviTxRu3rXqlH5kqmz7zTuOw/XzlrPN7xOqWWwIGK7P naVw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=IJ72gqPp; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id d17-20020a170906305100b007317ad1f9a4si1922764ejd.310.2022.10.25.02.14.31; Tue, 25 Oct 2022 02:14:32 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=IJ72gqPp; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2F6F968BC88; Tue, 25 Oct 2022 12:14:03 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7E62968B9C0 for ; Tue, 25 Oct 2022 12:13:55 +0300 (EEST) Received: by mail-pl1-f179.google.com with SMTP id u6so10628479plq.12 for ; Tue, 25 Oct 2022 02:13:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=ZSKFZ8pHOGEGxjUj/zNVXaYGpi/CsnvD1MN5+m89hbM=; b=IJ72gqPpnCbNA05H0COogsI0RrcIaXp8v9TN8Dw7VZY12AK79adqmFlIXMBXy3IguO 13A7PI/4Sax0VIZiqq+RA8qYMKx5KgPgdo3SG0INpfT2Vx3P7yAJLjZ9cikaKLTJ/AYu M1zDjkYJG4SUKlgJcczae7x5ZXshN0uTEwiRyG1x+uFDBP93GSPp+tsO1+IvjQZ5M14O C9sAXi1dsPZRz9ZJdClPvTaZc3sT1ZbbppwK3CmTUfip+cV9pLCqSwmHchftgcr4DTrp pWtne95Rxcfr8IUzQHzZfvwUvtMXq1onqVOT+rXQ7l2kxmtMh5c9sTGRe89AUiKwZgof 1MqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=ZSKFZ8pHOGEGxjUj/zNVXaYGpi/CsnvD1MN5+m89hbM=; b=51TJlOVNETZbEs7b6VNUvP85pX/VzWXFSXEaceXRi30xNRQwXnV9BeDAzb9dXQ+tqt qnM38WLN1XtYFOzXQK/6J0bcqx7X/dPngJuAwjO87/r83IamJFQnvEVovqM2M+cb4Oyh LdDV5WHqR2CbN8TBdMsehR7UbBxbDVy0nRFexFV9UTQKyMU334swDJyi6rPcKt7/wUe6 KE7f+UHGPZjsZyGmggrHdkvZZ/puavXIQmOYpYSnkIB8vSL4sWeEaPBjolrSFTjzcRgw DWdprjv9JkODqi06D7e2b1v+qFckhwdT7Zndkj2ElDzypxuy8BbvFde8m4eQA9ooFrBz 1Ufw== X-Gm-Message-State: ACrzQf00Q/ME7R2mkTDfa2EgdHeGjHYm8VAFtstDCS3/w1uSnBVmOm13 9MFbIcs8+gfVbGjLdw9DkqdS2RSZJCc= X-Received: by 2002:a17:902:f54f:b0:186:a987:c733 with SMTP id h15-20020a170902f54f00b00186a987c733mr9492532plf.170.1666689233307; Tue, 25 Oct 2022 02:13:53 -0700 (PDT) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id u13-20020a17090ae00d00b0020a11217682sm1046055pjy.27.2022.10.25.02.13.52 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Oct 2022 02:13:52 -0700 (PDT) From: softworkz X-Google-Original-From: softworkz Message-Id: <3598ec3b9aad8e1ce49b104faa9b91fea799cbbd.1666689226.git.ffmpegagent@gmail.com> In-Reply-To: References: Date: Tue, 25 Oct 2022 09:13:24 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v9 03/25] avcodec/subtitles: Introduce new frame-based subtitle decoding API X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: softworkz Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: sjAXIKaacL6c From: softworkz - Modify avcodec_send_packet() to support subtitles via the regular frame based decoding API - Add decode_subtitle_shim() which takes subtitle frames, and serves as a compatibility shim to the legacy subtitle decoding API until all subtitle decoders are migrated to the frame-based API - Add additional methods for conversion between old and new API Signed-off-by: softworkz --- libavcodec/avcodec.c | 8 ++ libavcodec/avcodec.h | 10 ++- libavcodec/decode.c | 60 ++++++++++++-- libavcodec/internal.h | 18 +++++ libavcodec/utils.c | 184 ++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 271 insertions(+), 9 deletions(-) diff --git a/libavcodec/avcodec.c b/libavcodec/avcodec.c index a85d3c2309..3518dd2185 100644 --- a/libavcodec/avcodec.c +++ b/libavcodec/avcodec.c @@ -350,6 +350,14 @@ FF_ENABLE_DEPRECATION_WARNINGS goto free_and_end; } + // Set the subtitle type from the codec descriptor in case the decoder hasn't done itself + if (avctx->codec_type == AVMEDIA_TYPE_SUBTITLE && avctx->subtitle_type == AV_SUBTITLE_FMT_UNKNOWN) { + if(avctx->codec_descriptor->props & AV_CODEC_PROP_BITMAP_SUB) + avctx->subtitle_type = AV_SUBTITLE_FMT_BITMAP; + if(avctx->codec_descriptor->props & AV_CODEC_PROP_TEXT_SUB) + avctx->subtitle_type = AV_SUBTITLE_FMT_ASS; + } + #if FF_API_AVCTX_TIMEBASE if (avctx->framerate.num > 0 && avctx->framerate.den > 0) avctx->time_base = av_inv_q(av_mul_q(avctx->framerate, (AVRational){avctx->ticks_per_frame, 1})); diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index bf06b01e22..68b588861f 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -1700,7 +1700,7 @@ typedef struct AVCodecContext { /** * Header containing style information for text subtitles. - * For SUBTITLE_ASS subtitle type, it should contain the whole ASS + * For AV_SUBTITLE_FMT_ASS subtitle type, it should contain the whole ASS * [Script Info] and [V4+ Styles] section, plus the [Events] line and * the Format line following. It shouldn't include any Dialogue line. * - encoding: Set/allocated/freed by user (before avcodec_open2()) @@ -2058,6 +2058,8 @@ typedef struct AVCodecContext { * The decoder can then override during decoding as needed. */ AVChannelLayout ch_layout; + + enum AVSubtitleType subtitle_type; } AVCodecContext; /** @@ -2434,7 +2436,10 @@ int avcodec_close(AVCodecContext *avctx); * Free all allocated data in the given subtitle struct. * * @param sub AVSubtitle to free. + * + * @deprecated Use the regular frame based encode and decode APIs instead. */ +attribute_deprecated void avsubtitle_free(AVSubtitle *sub); /** @@ -2533,7 +2538,10 @@ enum AVChromaLocation avcodec_chroma_pos_to_enum(int xpos, int ypos); * must be freed with avsubtitle_free if *got_sub_ptr is set. * @param[in,out] got_sub_ptr Zero if no subtitle could be decompressed, otherwise, it is nonzero. * @param[in] avpkt The input AVPacket containing the input buffer. + * + * @deprecated Use the new decode API (avcodec_send_packet, avcodec_receive_frame) instead. */ +attribute_deprecated int avcodec_decode_subtitle2(AVCodecContext *avctx, AVSubtitle *sub, int *got_sub_ptr, AVPacket *avpkt); diff --git a/libavcodec/decode.c b/libavcodec/decode.c index 6be2d3d6ed..6a4702807a 100644 --- a/libavcodec/decode.c +++ b/libavcodec/decode.c @@ -634,6 +634,39 @@ FF_ENABLE_DEPRECATION_WARNINGS return ret; } +static int decode_subtitle2_priv(AVCodecContext *avctx, AVSubtitle *sub, + int *got_sub_ptr, AVPacket *avpkt); + +static int decode_subtitle_shim(AVCodecContext *avctx, AVFrame *frame, AVPacket *avpkt) +{ + int ret, got_sub_ptr = 0; + AVSubtitle subtitle = { 0 }; + + if (frame->buf[0]) + return AVERROR(EAGAIN); + + av_frame_unref(frame); + + ret = decode_subtitle2_priv(avctx, &subtitle, &got_sub_ptr, avpkt); + + if (ret >= 0 && got_sub_ptr) { + frame->type = AVMEDIA_TYPE_SUBTITLE; + frame->format = subtitle.format; + ret = av_frame_get_buffer2(frame, 0); + + if (ret >= 0) + ret = ff_frame_put_subtitle(frame, &subtitle); + + frame->width = avctx->width; + frame->height = avctx->height; + frame->pkt_dts = avpkt->dts; + } + + avsubtitle_free(&subtitle); + + return ret; +} + int attribute_align_arg avcodec_send_packet(AVCodecContext *avctx, const AVPacket *avpkt) { AVCodecInternal *avci = avctx->internal; @@ -648,6 +681,13 @@ int attribute_align_arg avcodec_send_packet(AVCodecContext *avctx, const AVPacke if (avpkt && !avpkt->size && avpkt->data) return AVERROR(EINVAL); + if (avctx->codec_type == AVMEDIA_TYPE_SUBTITLE) + // this does not exactly implement the avcodec_send_packet/avcodec_receive_frame API + // but we know that no subtitle decoder produces multiple AVSubtitles per packet through + // the legacy API, and this will be changed when migrating the subtitle decoders + // to the frame based decoding api + return decode_subtitle_shim(avctx, avci->buffer_frame, avpkt); + av_packet_unref(avci->buffer_pkt); if (avpkt && (avpkt->data || avpkt->side_data_elems)) { ret = av_packet_ref(avci->buffer_pkt, avpkt); @@ -707,7 +747,9 @@ int ff_decode_receive_frame(AVCodecContext *avctx, AVFrame *frame) if (avci->buffer_frame->buf[0]) { av_frame_move_ref(frame, avci->buffer_frame); - } else { + } else if (avctx->codec_type == AVMEDIA_TYPE_SUBTITLE) + return AVERROR(EAGAIN); + else { ret = decode_receive_frame_internal(avctx, frame); if (ret < 0) return ret; @@ -860,9 +902,8 @@ static int utf8_check(const uint8_t *str) return 1; } -int avcodec_decode_subtitle2(AVCodecContext *avctx, AVSubtitle *sub, - int *got_sub_ptr, - AVPacket *avpkt) +static int decode_subtitle2_priv(AVCodecContext *avctx, AVSubtitle *sub, + int *got_sub_ptr, AVPacket *avpkt) { int ret = 0; @@ -908,10 +949,7 @@ int avcodec_decode_subtitle2(AVCodecContext *avctx, AVSubtitle *sub, avctx->pkt_timebase, ms); } - if (avctx->codec_descriptor->props & AV_CODEC_PROP_BITMAP_SUB) - sub->format = 0; - else if (avctx->codec_descriptor->props & AV_CODEC_PROP_TEXT_SUB) - sub->format = 1; + sub->format = (uint16_t)avctx->subtitle_type; for (unsigned i = 0; i < sub->num_rects; i++) { if (avctx->sub_charenc_mode != FF_SUB_CHARENC_MODE_IGNORE && @@ -932,6 +970,12 @@ int avcodec_decode_subtitle2(AVCodecContext *avctx, AVSubtitle *sub, return ret; } +int avcodec_decode_subtitle2(AVCodecContext *avctx, AVSubtitle *sub, + int *got_sub_ptr, AVPacket *avpkt) +{ + return decode_subtitle2_priv(avctx, sub, got_sub_ptr, avpkt); +} + enum AVPixelFormat avcodec_default_get_format(struct AVCodecContext *avctx, const enum AVPixelFormat *fmt) { diff --git a/libavcodec/internal.h b/libavcodec/internal.h index 76a6ea6bc6..055c6735de 100644 --- a/libavcodec/internal.h +++ b/libavcodec/internal.h @@ -245,4 +245,22 @@ int64_t ff_guess_coded_bitrate(AVCodecContext *avctx); int ff_int_from_list_or_default(void *ctx, const char * val_name, int val, const int * array_valid_values, int default_value); +void ff_dvdsub_parse_palette(uint32_t *palette, const char *p); + +/** + * Copies subtitle data from AVSubtitle to AVFrame. + * + * @deprecated This is a compatibility method for interoperability with + * the legacy subtitle API. + */ +int ff_frame_put_subtitle(AVFrame* frame, const AVSubtitle* sub); + +/** + * Copies subtitle data from AVFrame to AVSubtitle. + * + * @deprecated This is a compatibility method for interoperability with + * the legacy subtitle API. + */ +int ff_frame_get_subtitle(AVSubtitle* sub, AVFrame* frame); + #endif /* AVCODEC_INTERNAL_H */ diff --git a/libavcodec/utils.c b/libavcodec/utils.c index 2b63a498b9..552cdff70f 100644 --- a/libavcodec/utils.c +++ b/libavcodec/utils.c @@ -823,6 +823,190 @@ FF_ENABLE_DEPRECATION_WARNINGS return FFMAX(0, duration); } +static int subtitle_area2rect(AVSubtitleRect *dst, const AVSubtitleArea *src) +{ + dst->x = src->x; + dst->y = src->y; + dst->w = src->w; + dst->h = src->h; + dst->nb_colors = src->nb_colors; + dst->type = src->type; + dst->flags = src->flags; + + switch (dst->type) { + case AV_SUBTITLE_FMT_BITMAP: + + if (src->h > 0 && src->w > 0 && src->buf[0]) { + uint32_t *pal; + AVBufferRef *buf = src->buf[0]; + dst->data[0] = av_mallocz(buf->size); + memcpy(dst->data[0], buf->data, buf->size); + dst->linesize[0] = src->linesize[0]; + + dst->data[1] = av_mallocz(256 * 4); + pal = (uint32_t *)dst->data[1]; + + for (unsigned i = 0; i < 256; i++) { + pal[i] = src->pal[i]; + } + } + + break; + case AV_SUBTITLE_FMT_TEXT: + + if (src->text) + dst->text = av_strdup(src->text); + else + dst->text = av_strdup(""); + + if (!dst->text) + return AVERROR(ENOMEM); + + break; + case AV_SUBTITLE_FMT_ASS: + + if (src->ass) + dst->ass = av_strdup(src->ass); + else + dst->ass = av_strdup(""); + + if (!dst->ass) + return AVERROR(ENOMEM); + + break; + default: + + av_log(NULL, AV_LOG_ERROR, "Subtitle rect has invalid format: %d", dst->type); + return AVERROR(EINVAL); + } + + return 0; +} + +static int subtitle_rect2area(AVSubtitleArea *dst, const AVSubtitleRect *src) +{ + dst->x = src->x; + dst->y = src->y; + dst->w = src->w; + dst->h = src->h; + dst->nb_colors = src->nb_colors; + dst->type = src->type; + dst->flags = src->flags; + + switch (dst->type) { + case AV_SUBTITLE_FMT_BITMAP: + + if (src->h > 0 && src->w > 0 && src->data[0]) { + AVBufferRef *buf = av_buffer_allocz(src->h * src->linesize[0]); + memcpy(buf->data, src->data[0], buf->size); + + dst->buf[0] = buf; + dst->linesize[0] = src->linesize[0]; + } + + if (src->data[1]) { + uint32_t *pal = (uint32_t *)src->data[1]; + + for (unsigned i = 0; i < 256; i++) { + dst->pal[i] = pal[i]; + } + } + + break; + case AV_SUBTITLE_FMT_TEXT: + + if (src->text) { + dst->text = av_strdup(src->text); + if (!dst->text) + return AVERROR(ENOMEM); + } + + break; + case AV_SUBTITLE_FMT_ASS: + + if (src->ass) { + dst->ass = av_strdup(src->ass); + if (!dst->ass) + return AVERROR(ENOMEM); + } + + break; + default: + + av_log(NULL, AV_LOG_ERROR, "Subtitle area has invalid format: %d", dst->type); + return AVERROR(EINVAL); + } + + return 0; +} + +/** + * Copies subtitle data from AVSubtitle (deprecated) to AVFrame + * + * @note This is a compatibility method for conversion to the legacy API + */ +int ff_frame_put_subtitle(AVFrame *frame, const AVSubtitle *sub) +{ + frame->format = sub->format; + frame->subtitle_timing.start_pts = sub->pts; + frame->subtitle_timing.start_pts += av_rescale_q(sub->start_display_time, (AVRational){ 1, 1000 }, AV_TIME_BASE_Q); + frame->subtitle_timing.duration = av_rescale_q(sub->end_display_time - sub->start_display_time, (AVRational){ 1, 1000 }, AV_TIME_BASE_Q); + + if (sub->num_rects) { + frame->subtitle_areas = av_malloc_array(sub->num_rects, sizeof(AVSubtitleArea*)); + if (!frame->subtitle_areas) + return AVERROR(ENOMEM); + + for (unsigned i = 0; i < sub->num_rects; i++) { + int ret; + frame->subtitle_areas[i] = av_mallocz(sizeof(AVSubtitleArea)); + if (!frame->subtitle_areas[i]) + return AVERROR(ENOMEM); + ret = subtitle_rect2area(frame->subtitle_areas[i], sub->rects[i]); + if (ret < 0) { + frame->num_subtitle_areas = i; + return ret; + } + } + } + + frame->num_subtitle_areas = sub->num_rects; + return 0; +} + +/** + * Copies subtitle data from AVFrame to AVSubtitle (deprecated) + * + * @note This is a compatibility method for conversion to the legacy API + */ +int ff_frame_get_subtitle(AVSubtitle *sub, AVFrame *frame) +{ + const int64_t duration_ms = av_rescale_q(frame->subtitle_timing.duration, AV_TIME_BASE_Q, (AVRational){ 1, 1000 }); + + sub->start_display_time = 0; + sub->end_display_time = (int32_t)duration_ms; + sub->pts = frame->subtitle_timing.start_pts; + + if (frame->num_subtitle_areas) { + sub->rects = av_malloc_array(frame->num_subtitle_areas, sizeof(AVSubtitleRect*)); + if (!sub->rects) + return AVERROR(ENOMEM); + + for (unsigned i = 0; i < frame->num_subtitle_areas; i++) { + int ret; + sub->rects[i] = av_mallocz(sizeof(AVSubtitleRect)); + ret = subtitle_area2rect(sub->rects[i], frame->subtitle_areas[i]); + if (ret < 0) { + sub->num_rects = i; + return ret; + } + } + } + + sub->num_rects = frame->num_subtitle_areas; + return 0; +} + int av_get_audio_frame_duration2(AVCodecParameters *par, int frame_bytes) { int channels = par->ch_layout.nb_channels; From patchwork Tue Oct 25 09:13:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 38979 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2684424pzb; Tue, 25 Oct 2022 02:14:42 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5manpqT0ZFIEu5aHbKo6LHY2B6qjBBULPHIeOtmBIUOV15rugSVG5q7VDMtfiwVbR5pjE6 X-Received: by 2002:a17:907:a4c:b0:77b:ba98:d3e with SMTP id be12-20020a1709070a4c00b0077bba980d3emr31719837ejc.13.1666689282340; Tue, 25 Oct 2022 02:14:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666689282; cv=none; d=google.com; s=arc-20160816; b=w2NxfnNzW/rIE9OVxtDWbiP1tkWMu6QMDyBq0xSzLPrVQMOk8HMU0r2G1gWO+KBGyR 772rsy8Yr8DD049topCshf+3vU2bpprIMWTWvcANAxfDLcFSkQOvwQwmjmSvnpbuogs+ 4wa5/GYGGm3/MoCCQfY44o2NpkRTxO+5FZVCiC3llloYK9CVTbzSTN5PiGNqMMzNS4zv eY5UIfaKmJNsPXP+KE1eXaZ8lNXOpabPqk7yL83FHHNVLx36jczhjp3fUpiea3j/di4k xFv71fVh3JWEVco+qAWjKh/n2gXUdTHFOtLpxnHgPRL6bgFXrL+kO5moCnsc8RQo2zsu OzMA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:references :in-reply-to:message-id:from:dkim-signature:delivered-to; bh=gvH7NSrkp3U6JzOFd+QvR3r1/z/D4ofhQBkHzc3QXoc=; b=rorXgnQR7ekENWZrY4+eu8/+qGsOydsQGEm46H7y/3YsAZ2laOitRBMSHQ7lQ6HMdE 34IvWTnNm2vmIuijp7d5ZwJLnT/+bEnhKkFoUfKVsE+HioLQV0YnWXmASHVjfitT8BoJ 0XKdp8cjL4IoWGUDDSHJLKffaDnd9G1ygQC+2xo/Dgz9wJxXM4uryqpyKXsv4g2MCgvM D8OJN3hpJW/PSibVc/GCTnyKTdoZfLk8wVUi7FIjJXEsGdckUBiw2bVY4V3B3t881sly 67GxQU/mzv4rdpPsFw14rEhr5POnol4Vh1Pn4vbvA4vWZrKQwfoFFe9AEPt+1rMAhHN2 9V4g== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=CbLaOF81; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id q19-20020a056402519300b004619253a760si2675960edd.312.2022.10.25.02.14.41; Tue, 25 Oct 2022 02:14:42 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=CbLaOF81; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 17CF468BCAF; Tue, 25 Oct 2022 12:14:04 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 25BA068BC2F for ; Tue, 25 Oct 2022 12:13:56 +0300 (EEST) Received: by mail-pj1-f52.google.com with SMTP id v13-20020a17090a6b0d00b0021332e5388fso2492290pjj.1 for ; Tue, 25 Oct 2022 02:13:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=Tn3/3HeVHLy3EIrrl6uty3JtRroSU9przyXWA2OhImU=; b=CbLaOF811NHd9JZet26WbXs8OsPOKp/8Zpj95NFNfTvLxQFpBpMG5YrX9jUEloszbz Tr70J6bxr0XTsak9Oi6yOU+vAF/hIGPFNOH1Nf6+XOsa4aBG8FmTUYtBYLbhI4D3K9JA keRB+7H4RZlLwiGZnZH2t5Gw1PRGC9q9LI4ZuxdQKv9eT/Vaea0yy1TAPdX2IY23FdhK qTIAWvIZksxjarC173K7ma0dOvUxw5aHNrUDLsVR08mRTx6hBbK4jKmjOfiLRjv0yeoJ LjfjwOpIzySGR7iQTO0dz/M1N0/tLyY9Z9r4nmKzj1nfYP0eDy40rlLEISsnWlKVFQL0 X9lQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=Tn3/3HeVHLy3EIrrl6uty3JtRroSU9przyXWA2OhImU=; b=o/uxWdExAfbUK70BynUOIoaOgjCaz9Yg0e5F6EuH7c4azVmh2uPDc9/cLjX1IT/69v 7rOqXURZh4SSrbZbRVEVT/AOrXnie54YpTjJqCjmirWscZAxTwcghRSOhTYbAbJyImR4 xJ3WxjhECeNN7R9ZfLpR8cRu3qUgtY+qA2iaeiPU38/c6DxbasWrZoXLemcHgn+n+Jqn V2utQZNeJWm8FqTVWR2FlEAavll/PICBWTsL94zC35UZZd5bJTenQUWOajF1+3q720pD +dRFyP9FCLyw4S0lxdNmN9/zreDbTacfXMgTAnuLnqDJzfhSbhDeSIVQpdSUiXbNnnBm CvTw== X-Gm-Message-State: ACrzQf0udkX+FtCi+eiMDjimYqCxk87KMwXIfp/S0jw+9gIBy+GIu7hF M+DAyU80SV0Dg7hYoF6EjDrgLn6Q0b4= X-Received: by 2002:a17:90b:1b4f:b0:213:353d:10e3 with SMTP id nv15-20020a17090b1b4f00b00213353d10e3mr5240929pjb.165.1666689234487; Tue, 25 Oct 2022 02:13:54 -0700 (PDT) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id t8-20020a1709027fc800b0018689e2c9dfsm875711plb.153.2022.10.25.02.13.53 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Oct 2022 02:13:53 -0700 (PDT) From: softworkz X-Google-Original-From: softworkz Message-Id: <76ed737a8e466cfec7fbcedfb26a2291745642e7.1666689226.git.ffmpegagent@gmail.com> In-Reply-To: References: Date: Tue, 25 Oct 2022 09:13:25 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v9 04/25] avcodec/libzvbi: set subtitle type X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: softworkz Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 769cZ0Jukmqn From: softworkz Signed-off-by: softworkz --- libavcodec/libzvbi-teletextdec.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/libavcodec/libzvbi-teletextdec.c b/libavcodec/libzvbi-teletextdec.c index 45e30eb01c..486aa1724e 100644 --- a/libavcodec/libzvbi-teletextdec.c +++ b/libavcodec/libzvbi-teletextdec.c @@ -751,10 +751,13 @@ static int teletext_init_decoder(AVCodecContext *avctx) switch (ctx->format_id) { case 0: + avctx->subtitle_type = AV_SUBTITLE_FMT_BITMAP; return 0; case 1: + avctx->subtitle_type = AV_SUBTITLE_FMT_ASS; return ff_ass_subtitle_header_default(avctx); case 2: + avctx->subtitle_type = AV_SUBTITLE_FMT_ASS; return my_ass_subtitle_header(avctx); } return AVERROR_BUG; From patchwork Tue Oct 25 09:13:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 38981 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2684472pzb; Tue, 25 Oct 2022 02:14:50 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6vAACZJwW9PGxo1VX/zaP17LKYGcJ29FcR5jBt1BMPs1wNOB6QLwl+s4Nr/2DMeFIzn8JX X-Received: by 2002:a05:6402:a46:b0:461:ed76:cb56 with SMTP id bt6-20020a0564020a4600b00461ed76cb56mr5934849edb.264.1666689290422; Tue, 25 Oct 2022 02:14:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666689290; cv=none; d=google.com; s=arc-20160816; b=saTYD6zG2e0WwHK1APwCSwd/BA+DS6QWB/UHOuKTLKzVHcWHLvFDYqvXXAoa2c+TpS e1pEEMnUs9f13tUOpiDJ87FVm/Wvyhquh9NtIMq1JC3RZvGTnwP7j3KPYfeeVvcfRq07 jRvTEDk4QpXgb+b2IntZCok7qgWu6Xpqn5O6lhtird6GvLClMnYc5EpJS42WzUwy89zd 5gYid8O0h+sbcWXH+nuYGn4L3fPMuwuDwTS2jHLzvl6F4Uj9JAe2k7KnXyeUJ2LWt9J7 HgngA9jkQaPqmU0m5Ki2UCLtubrVfU58+38pgEQxPKcYRQiLNLnM20IRFW3IbybXqouU Halw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:references :in-reply-to:message-id:from:dkim-signature:delivered-to; bh=77pS+/jtHJYjMI6CvOVvrbGk3iHeJem4CBImfLKA2gQ=; b=llJSN6pzX5lidOMH+m2oqLAcx5KBJlyuERzx+Eia3LLXlQ38wcVtlc3LY6LXpwgKKN G4FV9h3tAjgrjk7lIQeZk6Ya7TrjGjXOOGAugiEIxl87IaJV77W5cjFZAZHkd5a8vcu1 XZ5B9562/KAa1yqVcUHSh96+TPEpcREvFcQi6vuQwZUDowac3hXUJXR3OO6WLQPT1kN5 bnpqkbzEgCnXKI59Xc3Wr/Y1aWDYiBlc670CW7g5ElujKflqBN6DASowv7LA7hjI7zEN gzC2Ujyhde6zjvATP9LQBg0Iw7wbVIHBSFBPsaCik9H2O1zXJ8eN90XcEDWVI6tBzh3j kSNw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=gXKT20bh; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id b24-20020a17090636d800b0078200f886bbsi1988323ejc.361.2022.10.25.02.14.50; Tue, 25 Oct 2022 02:14:50 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=gXKT20bh; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id DCDFB68BCBB; Tue, 25 Oct 2022 12:14:04 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf1-f177.google.com (mail-pf1-f177.google.com [209.85.210.177]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6B90E68B85C for ; Tue, 25 Oct 2022 12:13:57 +0300 (EEST) Received: by mail-pf1-f177.google.com with SMTP id y1so11255842pfr.3 for ; Tue, 25 Oct 2022 02:13:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=OwAbOvynqRhMS61zCf3wNHPYflVi3DT6Sol+N98Ic8k=; b=gXKT20bhQyI9Rw/XPamQZPgmwLyFaoqxRaz5nArcXrHDdwEbD8UZYhpicszcXiW2a1 rYgqcGDyqZTwmIWSdmKDy9yq6ZLynhppPcInQpAMnTAB2H+0lwDoyWNLhlUvQcOZTs4I 4gIDKMbMQz2Ycl5yWaHUiF4uJeGI2cyvecfuvTyM0nhj0dtCXquLmL2mhftkDZUdlqJ7 nltTMitQnhBfyFi/6f7aH6F4iouiiwP5r0Gm+S+pcQy+0IPP/1UZ87VoKUoxl5s3a7eW EQNaCbAxl41t9tztPXdXquWtJyJ2nGkADWC7lCBqi0y0PCqWOI78ni7+Pls0zgw/M8kl 5b/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=OwAbOvynqRhMS61zCf3wNHPYflVi3DT6Sol+N98Ic8k=; b=INM60YNM4JYLmXJo+XfpAIBl/8N7IAfIBc5OLU6EeUjKyUHgc3nmQmVzJUxPFyDShu GyKzShw2ZKQd5L2ErJ8xqFnLnYWwWpJnE95rwmrZbOdTye7nwyuTLtKIwGWh5rZr2WND SYWgtAaaGEpfeGqvZiW7KEtlw8S0u+Yj+Z9aK0VzpWu/C2QaYv6kUDpH37wgEVjL1Ugi yr5VzqdyFSHoO+/Oyfa40ZzrD1dKVOvmCWcFkr2EPo7+kfe9PfZsp8dfQhkXYIWu5d6e fH4cMxDTghfFeH6bJNYVcjM99Vb4Ndzuul5+98djWg9d67PoI52eAYQKra8GlnxIwJHH 4Mtw== X-Gm-Message-State: ACrzQf3Y9hzkXpO5rz2U4JWmxsZUV82ULCxPoInuAIqpAUnDJekS0p70 Gj9ULiltAU04WoQ5q//2EocEX3dUXZw= X-Received: by 2002:a63:8bc8:0:b0:461:ff70:7546 with SMTP id j191-20020a638bc8000000b00461ff707546mr32400819pge.70.1666689235478; Tue, 25 Oct 2022 02:13:55 -0700 (PDT) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id w20-20020a170902ca1400b001714e7608fdsm864094pld.256.2022.10.25.02.13.54 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Oct 2022 02:13:54 -0700 (PDT) From: softworkz X-Google-Original-From: softworkz Message-Id: <20430719d3d5db8e23d96a6c416182e20da86264.1666689226.git.ffmpegagent@gmail.com> In-Reply-To: References: Date: Tue, 25 Oct 2022 09:13:26 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v9 05/25] avfilter/subtitles: Update vf_subtitles to use new decoding api X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: softworkz Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: VASkyWdu9yw1 From: softworkz Signed-off-by: softworkz --- libavfilter/vf_subtitles.c | 67 ++++++++++++++++++++++++++++++-------- 1 file changed, 54 insertions(+), 13 deletions(-) diff --git a/libavfilter/vf_subtitles.c b/libavfilter/vf_subtitles.c index 82e140e986..0ae156ad07 100644 --- a/libavfilter/vf_subtitles.c +++ b/libavfilter/vf_subtitles.c @@ -36,14 +36,12 @@ # include "libavformat/avformat.h" #endif #include "libavutil/avstring.h" -#include "libavutil/imgutils.h" #include "libavutil/opt.h" #include "libavutil/parseutils.h" #include "drawutils.h" #include "avfilter.h" #include "internal.h" #include "formats.h" -#include "video.h" typedef struct AssContext { const AVClass *class; @@ -304,8 +302,42 @@ static int attachment_is_font(AVStream * st) return 0; } +static int decode(AVCodecContext *avctx, AVFrame *frame, int *got_frame, AVPacket *pkt) +{ + int ret; + + *got_frame = 0; + + if (pkt) { + ret = avcodec_send_packet(avctx, pkt); + // In particular, we don't expect AVERROR(EAGAIN), because we read all + // decoded frames with avcodec_receive_frame() until done. + if (ret < 0 && ret != AVERROR_EOF) + return ret; + } + + ret = avcodec_receive_frame(avctx, frame); + if (ret < 0 && ret != AVERROR(EAGAIN)) + return ret; + if (ret >= 0) + *got_frame = 1; + + return 0; +} + AVFILTER_DEFINE_CLASS(subtitles); +static enum AVSubtitleType get_subtitle_format(const AVCodecDescriptor *codec_descriptor) +{ + if(codec_descriptor->props & AV_CODEC_PROP_BITMAP_SUB) + return AV_SUBTITLE_FMT_BITMAP; + + if(codec_descriptor->props & AV_CODEC_PROP_TEXT_SUB) + return AV_SUBTITLE_FMT_ASS; + + return AV_SUBTITLE_FMT_UNKNOWN; +} + static av_cold int init_subtitles(AVFilterContext *ctx) { int j, ret, sid; @@ -318,6 +350,7 @@ static av_cold int init_subtitles(AVFilterContext *ctx) AVStream *st; AVPacket pkt; AssContext *ass = ctx->priv; + enum AVSubtitleType subtitle_format; /* Init libass */ ret = init(ctx); @@ -398,13 +431,17 @@ static av_cold int init_subtitles(AVFilterContext *ctx) ret = AVERROR_DECODER_NOT_FOUND; goto end; } + dec_desc = avcodec_descriptor_get(st->codecpar->codec_id); - if (dec_desc && !(dec_desc->props & AV_CODEC_PROP_TEXT_SUB)) { + subtitle_format = get_subtitle_format(dec_desc); + + if (subtitle_format != AV_SUBTITLE_FMT_ASS) { av_log(ctx, AV_LOG_ERROR, - "Only text based subtitles are currently supported\n"); - ret = AVERROR_PATCHWELCOME; + "Only text based subtitles are supported by this filter\n"); + ret = AVERROR_INVALIDDATA; goto end; } + if (ass->charenc) av_dict_set(&codec_opts, "sub_charenc", ass->charenc, 0); @@ -460,27 +497,31 @@ static av_cold int init_subtitles(AVFilterContext *ctx) dec_ctx->subtitle_header_size); while (av_read_frame(fmt, &pkt) >= 0) { int i, got_subtitle; - AVSubtitle sub = {0}; + AVFrame *sub = av_frame_alloc(); + if (!sub) { + ret = AVERROR(ENOMEM); + goto end; + } if (pkt.stream_index == sid) { - ret = avcodec_decode_subtitle2(dec_ctx, &sub, &got_subtitle, &pkt); + ret = decode(dec_ctx, sub, &got_subtitle, &pkt); if (ret < 0) { av_log(ctx, AV_LOG_WARNING, "Error decoding: %s (ignored)\n", av_err2str(ret)); } else if (got_subtitle) { - const int64_t start_time = av_rescale_q(sub.pts, AV_TIME_BASE_Q, av_make_q(1, 1000)); - const int64_t duration = sub.end_display_time; - for (i = 0; i < sub.num_rects; i++) { - char *ass_line = sub.rects[i]->ass; + const int64_t start_time = av_rescale_q(sub->subtitle_timing.start_pts, AV_TIME_BASE_Q, av_make_q(1, 1000)); + const int64_t duration = av_rescale_q(sub->subtitle_timing.duration, AV_TIME_BASE_Q, av_make_q(1, 1000)); + for (i = 0; i < sub->num_subtitle_areas; i++) { + char *ass_line = sub->subtitle_areas[i]->ass; if (!ass_line) - break; + continue; ass_process_chunk(ass->track, ass_line, strlen(ass_line), start_time, duration); } } } av_packet_unref(&pkt); - avsubtitle_free(&sub); + av_frame_free(&sub); } end: From patchwork Tue Oct 25 09:13:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 38986 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2685274pzb; Tue, 25 Oct 2022 02:15:35 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4+gV6pBZ60frPXREkbtwxa6Rk1zw2WuV7jK8j/43FLgKnVgCL2FvfPgRAQCrQddnPHfY7t X-Received: by 2002:a17:907:78a:b0:782:2223:a7cd with SMTP id xd10-20020a170907078a00b007822223a7cdmr30795194ejb.532.1666689335197; Tue, 25 Oct 2022 02:15:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666689335; cv=none; d=google.com; s=arc-20160816; b=a8nvIGXplbWHhqwCoBRPgHSasAtyFnEKEwGjZn7f2pPjzRYAmJrP7tjcOwX5crmVDL 6YQaT9JF2YWnEp9lGfHvGx0uHETMQWJItmapsWlDfmiwek4xYueITjt5+xPkLb5BAsrc GV5yAEH3guZPHwLiNgSsPpc8m76Zt9e58SwVHR+70coWswWSTXUI2+WcJPykSgpiA7sw InG0eTJgV6XVBD5jONZqnPZMUhXlp+lGC09qUMGc36LbdzA01Wl3UUaJ0cXaAuLQ6itd Ac53ZUIXc4Ty4UzDpPaf1SXJGBmF2FPCpN7yOrnLKEQZm62PIMw6Hz6XWCl9o6n/En+m t4Wg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:references :in-reply-to:message-id:from:dkim-signature:delivered-to; bh=x0EE1p7SmVk5q433YjR3wxGyrx4m3QixeH2XGZ9NeuU=; b=NjNMX3K7WxEtTMLEJrxsXJNA9RT47mI6PQa7hvuUWCU08gxsJhgDzSwiAujgj+amY0 q6FrAxkzylVDMHq/MJ3vPUf1FoV2YLSV/uSrsVA+ogU4CZWnjMWp3i0i8NsVhKxo1t6Y SpQPd7ahIycOL8UHgQ+5fCX384piPVcVP2FQnXVsuOEqtv+e8+TbmJ8LLo71OGEdXebV /x7/98dikiMpqm9vgm3VPEVFvatVUG1PYa623AcF5Uu538PBy6aeatGXzI4A0tpIsgxN OdYPYs3fM4H/QQuFGzRFZF5WRTNNlqA3Y+8bal0a534rW+1X9D+X0zj1xrafVBvHiD9I NrvQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=VRDJlpco; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id qw5-20020a1709066a0500b00780636887fesi2393497ejc.797.2022.10.25.02.15.34; Tue, 25 Oct 2022 02:15:35 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=VRDJlpco; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D45A668BD1F; Tue, 25 Oct 2022 12:14:09 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pg1-f178.google.com (mail-pg1-f178.google.com [209.85.215.178]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4B55F68BCBB for ; Tue, 25 Oct 2022 12:13:59 +0300 (EEST) Received: by mail-pg1-f178.google.com with SMTP id 20so10934922pgc.5 for ; Tue, 25 Oct 2022 02:13:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=1TQQecol+ij9PerK5dlzYas6EbV0g9xq8KnOxV7b9vA=; b=VRDJlpcoPgDNHIYc6kgPY4ZXrU4G7/uoUB6LvJfO0Gdh6gIEiWx1UrR8IquveHJAsJ /sgQXnjtFkremboOZyj9MbevV6QY/cV7lDQ+1b/cjdjHEDJ7FjxQI1bfaMOHW3utaLV6 QKz165bbtqg1aC1Eq4L4l2rBo9fmdZlpWmQLGyQHUD5gN9N6iBC0Thtl41GQj8iyCGq8 rRUWOVY7CtpDShHOHrKHlQHUUdPxZrV7S9GrQxBTWribJpbPL6ErdHGuAGnyJe6x7V9O T/RCGLldXAnkEUIjr4K07w1xLvhEq0njUKj9Iuq7HpnuiyKa/y+trViE24cJmoeXTo51 0qsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=1TQQecol+ij9PerK5dlzYas6EbV0g9xq8KnOxV7b9vA=; b=fdPSEG+e/HeK4i9wbbonyWsDMFv77/oUijdCOCnu4R11tshl1/s2HYb5QsooI7AS0W MZFo+y/Vc70DMNDpv3vhO5fb4Xdqf/u4V50yZgQtTikFD0XxItAHfUHy8ptg0Rs45ugu xrWBr7IzxJPJdLRVl3pRj4c+dLJdHEmThguhstqgf29BGqOmp8VegYxdxHJfll1op6Zm Vhdxb5GXLAq9Ek42Hs2GLLhX8E8rDi8MqM++70zUdXsHZ5o4M2guh4doE66rnNIS6h+7 7MY/Rvz3N6zas1wOZ77lr1mfh71854rxfn8vEHbW2e0bm5lPOS60ML20bvnfgYke3k7T vlpw== X-Gm-Message-State: ACrzQf3M9xfxBzChhM8FYHnGYkSirCkNFwyYFKrotsmYHdbPTX10VMcP q8jE9k9i6RfW7eNEADSEJaK6DpbfVhY= X-Received: by 2002:a65:6cc4:0:b0:412:35fa:5bce with SMTP id g4-20020a656cc4000000b0041235fa5bcemr31193256pgw.466.1666689236781; Tue, 25 Oct 2022 02:13:56 -0700 (PDT) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id k88-20020a17090a4ce100b00212e60c7d9csm4690043pjh.41.2022.10.25.02.13.55 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Oct 2022 02:13:56 -0700 (PDT) From: softworkz X-Google-Original-From: softworkz Message-Id: In-Reply-To: References: Date: Tue, 25 Oct 2022 09:13:27 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v9 06/25] avcodec, avutil: Move ass helper functions to avutil as avpriv_ and extend ass dialog parsing X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: softworkz Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: SdIHJ6rW+7zl From: softworkz Also add - hard_space callback (for upcoming fix) - extensible callback (for future extension) - new API which allows tag filtering Signed-off-by: softworkz --- libavcodec/Makefile | 56 +++--- libavcodec/ass.h | 151 +++++---------- libavcodec/ass_split.h | 191 ------------------- libavcodec/assdec.c | 2 +- libavcodec/assenc.c | 2 +- libavcodec/ccaption_dec.c | 20 +- libavcodec/jacosubdec.c | 2 +- libavcodec/libaribb24.c | 2 +- libavcodec/libzvbi-teletextdec.c | 14 +- libavcodec/microdvddec.c | 7 +- libavcodec/movtextdec.c | 3 +- libavcodec/movtextenc.c | 20 +- libavcodec/mpl2dec.c | 2 +- libavcodec/realtextdec.c | 2 +- libavcodec/samidec.c | 2 +- libavcodec/srtdec.c | 2 +- libavcodec/srtenc.c | 16 +- libavcodec/subviewerdec.c | 2 +- libavcodec/textdec.c | 4 +- libavcodec/ttmlenc.c | 15 +- libavcodec/webvttdec.c | 2 +- libavcodec/webvttenc.c | 16 +- libavutil/Makefile | 2 + {libavcodec => libavutil}/ass.c | 115 ++++-------- libavutil/ass_internal.h | 135 ++++++++++++++ {libavcodec => libavutil}/ass_split.c | 179 +++++++++++++++--- libavutil/ass_split_internal.h | 254 ++++++++++++++++++++++++++ 27 files changed, 726 insertions(+), 492 deletions(-) delete mode 100644 libavcodec/ass_split.h rename {libavcodec => libavutil}/ass.c (59%) create mode 100644 libavutil/ass_internal.h rename {libavcodec => libavutil}/ass_split.c (71%) create mode 100644 libavutil/ass_split_internal.h diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 90c7f113a3..58544b40ba 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -224,10 +224,10 @@ OBJS-$(CONFIG_APNG_DECODER) += png.o pngdec.o pngdsp.o OBJS-$(CONFIG_APNG_ENCODER) += png.o pngenc.o OBJS-$(CONFIG_ARBC_DECODER) += arbc.o OBJS-$(CONFIG_ARGO_DECODER) += argo.o -OBJS-$(CONFIG_SSA_DECODER) += assdec.o ass.o -OBJS-$(CONFIG_SSA_ENCODER) += assenc.o ass.o -OBJS-$(CONFIG_ASS_DECODER) += assdec.o ass.o -OBJS-$(CONFIG_ASS_ENCODER) += assenc.o ass.o +OBJS-$(CONFIG_SSA_DECODER) += assdec.o +OBJS-$(CONFIG_SSA_ENCODER) += assenc.o +OBJS-$(CONFIG_ASS_DECODER) += assdec.o +OBJS-$(CONFIG_ASS_ENCODER) += assenc.o OBJS-$(CONFIG_ASV1_DECODER) += asvdec.o asv.o mpeg12data.o OBJS-$(CONFIG_ASV1_ENCODER) += asvenc.o asv.o mpeg12data.o OBJS-$(CONFIG_ASV2_DECODER) += asvdec.o asv.o mpeg12data.o @@ -269,7 +269,7 @@ OBJS-$(CONFIG_BRENDER_PIX_DECODER) += brenderpix.o OBJS-$(CONFIG_C93_DECODER) += c93.o OBJS-$(CONFIG_CAVS_DECODER) += cavs.o cavsdec.o cavsdsp.o \ cavsdata.o -OBJS-$(CONFIG_CCAPTION_DECODER) += ccaption_dec.o ass.o +OBJS-$(CONFIG_CCAPTION_DECODER) += ccaption_dec.o OBJS-$(CONFIG_CDGRAPHICS_DECODER) += cdgraphics.o OBJS-$(CONFIG_CDTOONS_DECODER) += cdtoons.o OBJS-$(CONFIG_CDXL_DECODER) += cdxl.o @@ -449,7 +449,7 @@ OBJS-$(CONFIG_INTERPLAY_ACM_DECODER) += interplayacm.o OBJS-$(CONFIG_INTERPLAY_DPCM_DECODER) += dpcm.o OBJS-$(CONFIG_INTERPLAY_VIDEO_DECODER) += interplayvideo.o OBJS-$(CONFIG_IPU_DECODER) += mpeg12dec.o mpeg12.o mpeg12data.o -OBJS-$(CONFIG_JACOSUB_DECODER) += jacosubdec.o ass.o +OBJS-$(CONFIG_JACOSUB_DECODER) += jacosubdec.o OBJS-$(CONFIG_JPEG2000_ENCODER) += j2kenc.o mqcenc.o mqc.o jpeg2000.o \ jpeg2000dwt.o OBJS-$(CONFIG_JPEG2000_DECODER) += jpeg2000dec.o jpeg2000.o jpeg2000dsp.o \ @@ -472,7 +472,7 @@ OBJS-$(CONFIG_MDEC_DECODER) += mdec.o mpeg12.o mpeg12data.o OBJS-$(CONFIG_MEDIA100_DECODER) += media100.o OBJS-$(CONFIG_METASOUND_DECODER) += metasound.o metasound_data.o \ twinvq.o -OBJS-$(CONFIG_MICRODVD_DECODER) += microdvddec.o ass.o +OBJS-$(CONFIG_MICRODVD_DECODER) += microdvddec.o OBJS-$(CONFIG_MIMIC_DECODER) += mimic.o OBJS-$(CONFIG_MISC4_DECODER) += misc4.o OBJS-$(CONFIG_MJPEG_DECODER) += mjpegdec.o mjpegdec_common.o @@ -488,8 +488,8 @@ OBJS-$(CONFIG_MLP_ENCODER) += mlpenc.o mlp.o OBJS-$(CONFIG_MMVIDEO_DECODER) += mmvideo.o OBJS-$(CONFIG_MOBICLIP_DECODER) += mobiclip.o OBJS-$(CONFIG_MOTIONPIXELS_DECODER) += motionpixels.o -OBJS-$(CONFIG_MOVTEXT_DECODER) += movtextdec.o ass.o -OBJS-$(CONFIG_MOVTEXT_ENCODER) += movtextenc.o ass_split.o +OBJS-$(CONFIG_MOVTEXT_DECODER) += movtextdec.o +OBJS-$(CONFIG_MOVTEXT_ENCODER) += movtextenc.o OBJS-$(CONFIG_MP1_DECODER) += mpegaudiodec_fixed.o OBJS-$(CONFIG_MP1FLOAT_DECODER) += mpegaudiodec_float.o OBJS-$(CONFIG_MP2_DECODER) += mpegaudiodec_fixed.o @@ -530,7 +530,7 @@ OBJS-$(CONFIG_MPEG4_MEDIACODEC_DECODER) += mediacodecdec.o OBJS-$(CONFIG_MPEG4_OMX_ENCODER) += omx.o OBJS-$(CONFIG_MPEG4_V4L2M2M_DECODER) += v4l2_m2m_dec.o OBJS-$(CONFIG_MPEG4_V4L2M2M_ENCODER) += v4l2_m2m_enc.o -OBJS-$(CONFIG_MPL2_DECODER) += mpl2dec.o ass.o +OBJS-$(CONFIG_MPL2_DECODER) += mpl2dec.o OBJS-$(CONFIG_MSA1_DECODER) += mss3.o OBJS-$(CONFIG_MSCC_DECODER) += mscc.o OBJS-$(CONFIG_MSNSIREN_DECODER) += siren.o @@ -580,7 +580,7 @@ OBJS-$(CONFIG_PHM_ENCODER) += pnmenc.o float2half.o OBJS-$(CONFIG_PHOTOCD_DECODER) += photocd.o OBJS-$(CONFIG_PICTOR_DECODER) += pictordec.o cga_data.o OBJS-$(CONFIG_PIXLET_DECODER) += pixlet.o -OBJS-$(CONFIG_PJS_DECODER) += textdec.o ass.o +OBJS-$(CONFIG_PJS_DECODER) += textdec.o OBJS-$(CONFIG_PNG_DECODER) += png.o pngdec.o pngdsp.o OBJS-$(CONFIG_PNG_ENCODER) += png.o pngenc.o OBJS-$(CONFIG_PPM_DECODER) += pnmdec.o pnm.o @@ -615,7 +615,7 @@ OBJS-$(CONFIG_RALF_DECODER) += ralf.o OBJS-$(CONFIG_RASC_DECODER) += rasc.o OBJS-$(CONFIG_RAWVIDEO_DECODER) += rawdec.o OBJS-$(CONFIG_RAWVIDEO_ENCODER) += rawenc.o -OBJS-$(CONFIG_REALTEXT_DECODER) += realtextdec.o ass.o +OBJS-$(CONFIG_REALTEXT_DECODER) += realtextdec.o OBJS-$(CONFIG_RL2_DECODER) += rl2.o OBJS-$(CONFIG_ROQ_DECODER) += roqvideodec.o roqvideo.o OBJS-$(CONFIG_ROQ_ENCODER) += roqvideoenc.o roqvideo.o elbg.o @@ -630,7 +630,7 @@ OBJS-$(CONFIG_RV20_DECODER) += rv10.o OBJS-$(CONFIG_RV20_ENCODER) += rv20enc.o OBJS-$(CONFIG_RV30_DECODER) += rv30.o rv34.o rv30dsp.o OBJS-$(CONFIG_RV40_DECODER) += rv40.o rv34.o rv40dsp.o -OBJS-$(CONFIG_SAMI_DECODER) += samidec.o ass.o htmlsubtitles.o +OBJS-$(CONFIG_SAMI_DECODER) += samidec.o htmlsubtitles.o OBJS-$(CONFIG_S302M_DECODER) += s302m.o OBJS-$(CONFIG_S302M_ENCODER) += s302menc.o OBJS-$(CONFIG_SANM_DECODER) += sanm.o @@ -666,13 +666,13 @@ OBJS-$(CONFIG_SPEEDHQ_ENCODER) += speedhq.o mpeg12data.o mpeg12enc.o spe OBJS-$(CONFIG_SPEEX_DECODER) += speexdec.o OBJS-$(CONFIG_SP5X_DECODER) += sp5xdec.o OBJS-$(CONFIG_SRGC_DECODER) += mscc.o -OBJS-$(CONFIG_SRT_DECODER) += srtdec.o ass.o htmlsubtitles.o -OBJS-$(CONFIG_SRT_ENCODER) += srtenc.o ass_split.o -OBJS-$(CONFIG_STL_DECODER) += textdec.o ass.o -OBJS-$(CONFIG_SUBRIP_DECODER) += srtdec.o ass.o htmlsubtitles.o -OBJS-$(CONFIG_SUBRIP_ENCODER) += srtenc.o ass_split.o -OBJS-$(CONFIG_SUBVIEWER1_DECODER) += textdec.o ass.o -OBJS-$(CONFIG_SUBVIEWER_DECODER) += subviewerdec.o ass.o +OBJS-$(CONFIG_SRT_DECODER) += srtdec.o htmlsubtitles.o +OBJS-$(CONFIG_SRT_ENCODER) += srtenc.o +OBJS-$(CONFIG_STL_DECODER) += textdec.o +OBJS-$(CONFIG_SUBRIP_DECODER) += srtdec.o htmlsubtitles.o +OBJS-$(CONFIG_SUBRIP_ENCODER) += srtenc.o +OBJS-$(CONFIG_SUBVIEWER1_DECODER) += textdec.o +OBJS-$(CONFIG_SUBVIEWER_DECODER) += subviewerdec.o OBJS-$(CONFIG_SUNRAST_DECODER) += sunrast.o OBJS-$(CONFIG_SUNRAST_ENCODER) += sunrastenc.o OBJS-$(CONFIG_LIBRSVG_DECODER) += librsvgdec.o @@ -682,8 +682,8 @@ OBJS-$(CONFIG_SVQ1_DECODER) += svq1dec.o svq1.o h263data.o OBJS-$(CONFIG_SVQ1_ENCODER) += svq1enc.o svq1.o h263data.o \ h263.o ituh263enc.o OBJS-$(CONFIG_SVQ3_DECODER) += svq3.o mpegutils.o h264data.o -OBJS-$(CONFIG_TEXT_DECODER) += textdec.o ass.o -OBJS-$(CONFIG_TEXT_ENCODER) += srtenc.o ass_split.o +OBJS-$(CONFIG_TEXT_DECODER) += textdec.o +OBJS-$(CONFIG_TEXT_ENCODER) += srtenc.o OBJS-$(CONFIG_TAK_DECODER) += takdec.o tak.o takdsp.o OBJS-$(CONFIG_TARGA_DECODER) += targa.o OBJS-$(CONFIG_TARGA_ENCODER) += targaenc.o rle.o @@ -703,7 +703,7 @@ OBJS-$(CONFIG_TSCC_DECODER) += tscc.o msrledec.o OBJS-$(CONFIG_TSCC2_DECODER) += tscc2.o OBJS-$(CONFIG_TTA_DECODER) += tta.o ttadata.o ttadsp.o OBJS-$(CONFIG_TTA_ENCODER) += ttaenc.o ttaencdsp.o ttadata.o -OBJS-$(CONFIG_TTML_ENCODER) += ttmlenc.o ass_split.o +OBJS-$(CONFIG_TTML_ENCODER) += ttmlenc.o OBJS-$(CONFIG_TWINVQ_DECODER) += twinvqdec.o twinvq.o metasound_data.o OBJS-$(CONFIG_TXD_DECODER) += txd.o OBJS-$(CONFIG_ULTI_DECODER) += ulti.o @@ -759,7 +759,7 @@ OBJS-$(CONFIG_VP9_MEDIACODEC_DECODER) += mediacodecdec.o OBJS-$(CONFIG_VP9_RKMPP_DECODER) += rkmppdec.o OBJS-$(CONFIG_VP9_VAAPI_ENCODER) += vaapi_encode_vp9.o OBJS-$(CONFIG_VP9_QSV_ENCODER) += qsvenc_vp9.o -OBJS-$(CONFIG_VPLAYER_DECODER) += textdec.o ass.o +OBJS-$(CONFIG_VPLAYER_DECODER) += textdec.o OBJS-$(CONFIG_VP9_V4L2M2M_DECODER) += v4l2_m2m_dec.o OBJS-$(CONFIG_VQA_DECODER) += vqavideo.o OBJS-$(CONFIG_VQC_DECODER) += vqcdec.o @@ -769,8 +769,8 @@ OBJS-$(CONFIG_WBMP_DECODER) += wbmpdec.o OBJS-$(CONFIG_WBMP_ENCODER) += wbmpenc.o OBJS-$(CONFIG_WCMV_DECODER) += wcmv.o OBJS-$(CONFIG_WEBP_DECODER) += webp.o -OBJS-$(CONFIG_WEBVTT_DECODER) += webvttdec.o ass.o -OBJS-$(CONFIG_WEBVTT_ENCODER) += webvttenc.o ass_split.o +OBJS-$(CONFIG_WEBVTT_DECODER) += webvttdec.o +OBJS-$(CONFIG_WEBVTT_ENCODER) += webvttenc.o OBJS-$(CONFIG_WMALOSSLESS_DECODER) += wmalosslessdec.o wma_common.o OBJS-$(CONFIG_WMAPRO_DECODER) += wmaprodec.o wma.o wma_common.o OBJS-$(CONFIG_WMAV1_DECODER) += wmadec.o wma.o wma_common.o aactab.o @@ -1057,7 +1057,7 @@ OBJS-$(CONFIG_PCM_ALAW_AT_ENCODER) += audiotoolboxenc.o OBJS-$(CONFIG_PCM_MULAW_AT_ENCODER) += audiotoolboxenc.o OBJS-$(CONFIG_LIBAOM_AV1_DECODER) += libaomdec.o libaom.o OBJS-$(CONFIG_LIBAOM_AV1_ENCODER) += libaomenc.o libaom.o -OBJS-$(CONFIG_LIBARIBB24_DECODER) += libaribb24.o ass.o +OBJS-$(CONFIG_LIBARIBB24_DECODER) += libaribb24.o OBJS-$(CONFIG_LIBCELT_DECODER) += libcelt_dec.o OBJS-$(CONFIG_LIBCODEC2_DECODER) += libcodec2.o OBJS-$(CONFIG_LIBCODEC2_ENCODER) += libcodec2.o @@ -1110,7 +1110,7 @@ OBJS-$(CONFIG_LIBX265_ENCODER) += libx265.o OBJS-$(CONFIG_LIBXAVS_ENCODER) += libxavs.o OBJS-$(CONFIG_LIBXAVS2_ENCODER) += libxavs2.o OBJS-$(CONFIG_LIBXVID_ENCODER) += libxvid.o -OBJS-$(CONFIG_LIBZVBI_TELETEXT_DECODER) += libzvbi-teletextdec.o ass.o +OBJS-$(CONFIG_LIBZVBI_TELETEXT_DECODER) += libzvbi-teletextdec.o # parsers OBJS-$(CONFIG_AAC_LATM_PARSER) += latm_parser.o diff --git a/libavcodec/ass.h b/libavcodec/ass.h index 4dffe923d9..8bc13d7ab8 100644 --- a/libavcodec/ass.h +++ b/libavcodec/ass.h @@ -23,124 +23,73 @@ #define AVCODEC_ASS_H #include "avcodec.h" -#include "libavutil/bprint.h" - -#define ASS_DEFAULT_PLAYRESX 384 -#define ASS_DEFAULT_PLAYRESY 288 - -/** - * @name Default values for ASS style - * @{ - */ -#define ASS_DEFAULT_FONT "Arial" -#define ASS_DEFAULT_FONT_SIZE 16 -#define ASS_DEFAULT_COLOR 0xffffff -#define ASS_DEFAULT_BACK_COLOR 0 -#define ASS_DEFAULT_BOLD 0 -#define ASS_DEFAULT_ITALIC 0 -#define ASS_DEFAULT_UNDERLINE 0 -#define ASS_DEFAULT_ALIGNMENT 2 -#define ASS_DEFAULT_BORDERSTYLE 1 -/** @} */ +#include "libavutil/ass_internal.h" typedef struct FFASSDecoderContext { int readorder; } FFASSDecoderContext; -/** - * Generate a suitable AVCodecContext.subtitle_header for SUBTITLE_ASS. - * Can specify all fields explicitly - * - * @param avctx pointer to the AVCodecContext - * @param play_res_x subtitle frame width - * @param play_res_y subtitle frame height - * @param font name of the default font face to use - * @param font_size default font size to use - * @param primary_color default text color to use (ABGR) - * @param secondary_color default secondary text color to use (ABGR) - * @param outline_color default outline color to use (ABGR) - * @param back_color default background color to use (ABGR) - * @param bold 1 for bold text, 0 for normal text - * @param italic 1 for italic text, 0 for normal text - * @param underline 1 for underline text, 0 for normal text - * @param border_style 1 for outline, 3 for opaque box - * @param alignment position of the text (left, center, top...), defined after - * the layout of the numpad (1-3 sub, 4-6 mid, 7-9 top) - * @return >= 0 on success otherwise an error code <0 - */ -int ff_ass_subtitle_header_full(AVCodecContext *avctx, +static inline int ff_ass_subtitle_header_full(AVCodecContext *avctx, int play_res_x, int play_res_y, const char *font, int font_size, int primary_color, int secondary_color, int outline_color, int back_color, int bold, int italic, int underline, - int border_style, int alignment); -/** - * Generate a suitable AVCodecContext.subtitle_header for SUBTITLE_ASS. - * - * @param avctx pointer to the AVCodecContext - * @param font name of the default font face to use - * @param font_size default font size to use - * @param color default text color to use (ABGR) - * @param back_color default background color to use (ABGR) - * @param bold 1 for bold text, 0 for normal text - * @param italic 1 for italic text, 0 for normal text - * @param underline 1 for underline text, 0 for normal text - * @param alignment position of the text (left, center, top...), defined after - * the layout of the numpad (1-3 sub, 4-6 mid, 7-9 top) - * @return >= 0 on success otherwise an error code <0 - */ -int ff_ass_subtitle_header(AVCodecContext *avctx, - const char *font, int font_size, - int color, int back_color, - int bold, int italic, int underline, - int border_style, int alignment); + int border_style, int alignment) +{ + avctx->subtitle_header = (uint8_t *)avpriv_ass_get_subtitle_header_full( + play_res_x, play_res_y, font, font_size, + primary_color, secondary_color, outline_color, + back_color, bold,italic,underline,border_style,alignment, + !(avctx->flags & AV_CODEC_FLAG_BITEXACT)); -/** - * Generate a suitable AVCodecContext.subtitle_header for SUBTITLE_ASS - * with default style. - * - * @param avctx pointer to the AVCodecContext - * @return >= 0 on success otherwise an error code <0 - */ -int ff_ass_subtitle_header_default(AVCodecContext *avctx); + if (!avctx->subtitle_header) + return AVERROR(ENOMEM); + avctx->subtitle_header_size = strlen((char *)avctx->subtitle_header); + return 0; +} -/** - * Craft an ASS dialog string. - */ -char *ff_ass_get_dialog(int readorder, int layer, const char *style, - const char *speaker, const char *text); +static inline int ff_ass_subtitle_header_default(AVCodecContext *avctx) +{ + avctx->subtitle_header = (uint8_t *)avpriv_ass_get_subtitle_header_default(!(avctx->flags & AV_CODEC_FLAG_BITEXACT)); -/** - * Add an ASS dialog to a subtitle. - */ -int ff_ass_add_rect(AVSubtitle *sub, const char *dialog, - int readorder, int layer, const char *style, - const char *speaker); + if (!avctx->subtitle_header) + return AVERROR(ENOMEM); + avctx->subtitle_header_size = strlen((char *)avctx->subtitle_header); + return 0; +} + +static inline void ff_ass_decoder_flush(AVCodecContext *avctx) +{ + FFASSDecoderContext *s = avctx->priv_data; + if (!(avctx->flags2 & AV_CODEC_FLAG2_RO_FLUSH_NOOP)) + s->readorder = 0; +} /** * Add an ASS dialog to a subtitle. */ -int ff_ass_add_rect2(AVSubtitle *sub, const char *dialog, - int readorder, int layer, const char *style, - const char *speaker, unsigned *nb_rect_allocated); +static inline int avpriv_ass_add_rect(AVSubtitle *sub, const char *dialog, + int readorder, int layer, const char *style, + const char *speaker) +{ + char *ass_str; + AVSubtitleRect **rects; -/** - * Helper to flush a text subtitles decoder making use of the - * FFASSDecoderContext. - */ -void ff_ass_decoder_flush(AVCodecContext *avctx); + rects = av_realloc_array(sub->rects, sub->num_rects+1, sizeof(*sub->rects)); + if (!rects) + return AVERROR(ENOMEM); + sub->rects = rects; + rects[sub->num_rects] = av_mallocz(sizeof(*rects[0])); + if (!rects[sub->num_rects]) + return AVERROR(ENOMEM); + rects[sub->num_rects]->type = SUBTITLE_ASS; + ass_str = avpriv_ass_get_dialog(readorder, layer, style, speaker, dialog); + if (!ass_str) + return AVERROR(ENOMEM); + rects[sub->num_rects]->ass = ass_str; + sub->num_rects++; + return 0; +} -/** - * Escape a text subtitle using ASS syntax into an AVBPrint buffer. - * Newline characters will be escaped to \N. - * - * @param buf pointer to an initialized AVBPrint buffer - * @param p source text - * @param size size of the source text - * @param linebreaks additional newline chars, which will be escaped to \N - * @param keep_ass_markup braces and backslash will not be escaped if set - */ -void ff_ass_bprint_text_event(AVBPrint *buf, const char *p, int size, - const char *linebreaks, int keep_ass_markup); #endif /* AVCODEC_ASS_H */ diff --git a/libavcodec/ass_split.h b/libavcodec/ass_split.h deleted file mode 100644 index a45fb9b8a1..0000000000 --- a/libavcodec/ass_split.h +++ /dev/null @@ -1,191 +0,0 @@ -/* - * SSA/ASS spliting functions - * Copyright (c) 2010 Aurelien Jacobs - * - * This file is part of FFmpeg. - * - * FFmpeg is free software; you can redistribute it and/or - * modify it under the terms of the GNU Lesser General Public - * License as published by the Free Software Foundation; either - * version 2.1 of the License, or (at your option) any later version. - * - * FFmpeg is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public - * License along with FFmpeg; if not, write to the Free Software - * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA - */ - -#ifndef AVCODEC_ASS_SPLIT_H -#define AVCODEC_ASS_SPLIT_H - -/** - * fields extracted from the [Script Info] section - */ -typedef struct { - char *script_type; /**< SSA script format version (eg. v4.00) */ - char *collisions; /**< how subtitles are moved to prevent collisions */ - int play_res_x; /**< video width that ASS coords are referring to */ - int play_res_y; /**< video height that ASS coords are referring to */ - float timer; /**< time multiplier to apply to SSA clock (in %) */ -} ASSScriptInfo; - -/** - * fields extracted from the [V4(+) Styles] section - */ -typedef struct { - char *name; /**< name of the tyle (case sensitive) */ - char *font_name; /**< font face (case sensitive) */ - int font_size; /**< font height */ - int primary_color; /**< color that a subtitle will normally appear in */ - int secondary_color; - int outline_color; /**< color for outline in ASS, called tertiary in SSA */ - int back_color; /**< color of the subtitle outline or shadow */ - int bold; /**< whether text is bold (1) or not (0) */ - int italic; /**< whether text is italic (1) or not (0) */ - int underline; /**< whether text is underlined (1) or not (0) */ - int strikeout; - float scalex; - float scaley; - float spacing; - float angle; - int border_style; - float outline; - float shadow; - int alignment; /**< position of the text (left, center, top...), - defined after the layout of the numpad - (1-3 sub, 4-6 mid, 7-9 top) */ - int margin_l; - int margin_r; - int margin_v; - int alpha_level; - int encoding; -} ASSStyle; - -/** - * fields extracted from the [Events] section - */ -typedef struct { - int readorder; - int layer; /**< higher numbered layers are drawn over lower numbered */ - int start; /**< start time of the dialog in centiseconds */ - int end; /**< end time of the dialog in centiseconds */ - char *style; /**< name of the ASSStyle to use with this dialog */ - char *name; - int margin_l; - int margin_r; - int margin_v; - char *effect; - char *text; /**< actual text which will be displayed as a subtitle, - can include style override control codes (see - ff_ass_split_override_codes()) */ -} ASSDialog; - -/** - * structure containing the whole split ASS data - */ -typedef struct { - ASSScriptInfo script_info; /**< general information about the SSA script*/ - ASSStyle *styles; /**< array of split out styles */ - int styles_count; /**< number of ASSStyle in the styles array */ - ASSDialog *dialogs; /**< array of split out dialogs */ - int dialogs_count; /**< number of ASSDialog in the dialogs array*/ -} ASS; - -/** - * This struct can be casted to ASS to access to the split data. - */ -typedef struct ASSSplitContext ASSSplitContext; - -/** - * Split a full ASS file or a ASS header from a string buffer and store - * the split structure in a newly allocated context. - * - * @param buf String containing the ASS formatted data. - * @return Newly allocated struct containing split data. - */ -ASSSplitContext *ff_ass_split(const char *buf); - -/** - * Free a dialogue obtained from ff_ass_split_dialog(). - */ -void ff_ass_free_dialog(ASSDialog **dialogp); - -/** - * Split one ASS Dialogue line from a string buffer. - * - * @param ctx Context previously initialized by ff_ass_split(). - * @param buf String containing the ASS "Dialogue" line. - * @return Pointer to the split ASSDialog. Must be freed with ff_ass_free_dialog() - */ -ASSDialog *ff_ass_split_dialog(ASSSplitContext *ctx, const char *buf); - -/** - * Free all the memory allocated for an ASSSplitContext. - * - * @param ctx Context previously initialized by ff_ass_split(). - */ -void ff_ass_split_free(ASSSplitContext *ctx); - - -/** - * Set of callback functions corresponding to each override codes that can - * be encountered in a "Dialogue" Text field. - */ -typedef struct { - /** - * @defgroup ass_styles ASS styles - * @{ - */ - void (*text)(void *priv, const char *text, int len); - void (*new_line)(void *priv, int forced); - void (*style)(void *priv, char style, int close); - void (*color)(void *priv, unsigned int /* color */, unsigned int color_id); - void (*alpha)(void *priv, int alpha, int alpha_id); - void (*font_name)(void *priv, const char *name); - void (*font_size)(void *priv, int size); - void (*alignment)(void *priv, int alignment); - void (*cancel_overrides)(void *priv, const char *style); - /** @} */ - - /** - * @defgroup ass_functions ASS functions - * @{ - */ - void (*move)(void *priv, int x1, int y1, int x2, int y2, int t1, int t2); - void (*origin)(void *priv, int x, int y); - /** @} */ - - /** - * @defgroup ass_end end of Dialogue Event - * @{ - */ - void (*end)(void *priv); - /** @} */ -} ASSCodesCallbacks; - -/** - * Split override codes out of a ASS "Dialogue" Text field. - * - * @param callbacks Set of callback functions called for each override code - * encountered. - * @param priv Opaque pointer passed to the callback functions. - * @param buf The ASS "Dialogue" Text field to split. - * @return >= 0 on success otherwise an error code <0 - */ -int ff_ass_split_override_codes(const ASSCodesCallbacks *callbacks, void *priv, - const char *buf); - -/** - * Find an ASSStyle structure by its name. - * - * @param ctx Context previously initialized by ff_ass_split(). - * @param style name of the style to search for. - * @return the ASSStyle corresponding to style, or NULL if style can't be found - */ -ASSStyle *ff_ass_style_get(ASSSplitContext *ctx, const char *style); - -#endif /* AVCODEC_ASS_SPLIT_H */ diff --git a/libavcodec/assdec.c b/libavcodec/assdec.c index 89d7b51894..2571118ee9 100644 --- a/libavcodec/assdec.c +++ b/libavcodec/assdec.c @@ -22,9 +22,9 @@ #include #include "avcodec.h" -#include "ass.h" #include "codec_internal.h" #include "config_components.h" +#include "libavutil/ass_internal.h" #include "libavutil/internal.h" #include "libavutil/mem.h" diff --git a/libavcodec/assenc.c b/libavcodec/assenc.c index db6fd25dd7..9bbcdd9054 100644 --- a/libavcodec/assenc.c +++ b/libavcodec/assenc.c @@ -24,8 +24,8 @@ #include #include "avcodec.h" -#include "ass.h" #include "codec_internal.h" +#include "libavutil/ass_internal.h" #include "libavutil/avstring.h" #include "libavutil/internal.h" #include "libavutil/mem.h" diff --git a/libavcodec/ccaption_dec.c b/libavcodec/ccaption_dec.c index 61eda9ff56..6636ec34ac 100644 --- a/libavcodec/ccaption_dec.c +++ b/libavcodec/ccaption_dec.c @@ -272,15 +272,12 @@ static av_cold int init_decoder(AVCodecContext *avctx) ctx->bg_color = CCCOL_BLACK; ctx->rollup = 2; ctx->cursor_row = 10; - ret = ff_ass_subtitle_header(avctx, "Monospace", + ret = ff_ass_subtitle_header_full(avctx, ASS_DEFAULT_PLAYRESX, ASS_DEFAULT_PLAYRESY, "Monospace", ASS_DEFAULT_FONT_SIZE, - ASS_DEFAULT_COLOR, - ASS_DEFAULT_BACK_COLOR, - ASS_DEFAULT_BOLD, - ASS_DEFAULT_ITALIC, - ASS_DEFAULT_UNDERLINE, - 3, - ASS_DEFAULT_ALIGNMENT); + ASS_DEFAULT_COLOR, ASS_DEFAULT_COLOR, + ASS_DEFAULT_BACK_COLOR, ASS_DEFAULT_BACK_COLOR, + ASS_DEFAULT_BOLD, ASS_DEFAULT_ITALIC, ASS_DEFAULT_UNDERLINE, + 3, ASS_DEFAULT_ALIGNMENT); if (ret < 0) { return ret; } @@ -850,7 +847,6 @@ static int decode(AVCodecContext *avctx, AVSubtitle *sub, int len = avpkt->size; int ret = 0; int i; - unsigned nb_rect_allocated = 0; for (i = 0; i < len; i += 3) { uint8_t hi, cc_type = bptr[i] & 1; @@ -887,7 +883,7 @@ static int decode(AVCodecContext *avctx, AVSubtitle *sub, AV_TIME_BASE_Q, ms_tb); else sub->end_display_time = -1; - ret = ff_ass_add_rect2(sub, ctx->buffer[bidx].str, ctx->readorder++, 0, NULL, NULL, &nb_rect_allocated); + ret = avpriv_ass_add_rect(sub, ctx->buffer[bidx].str, ctx->readorder++, 0, NULL, NULL); if (ret < 0) return ret; ctx->last_real_time = sub->pts; @@ -897,7 +893,7 @@ static int decode(AVCodecContext *avctx, AVSubtitle *sub, if (!bptr && !ctx->real_time && ctx->buffer[!ctx->buffer_index].str[0]) { bidx = !ctx->buffer_index; - ret = ff_ass_add_rect2(sub, ctx->buffer[bidx].str, ctx->readorder++, 0, NULL, NULL, &nb_rect_allocated); + ret = avpriv_ass_add_rect(sub, ctx->buffer[bidx].str, ctx->readorder++, 0, NULL, NULL); if (ret < 0) return ret; sub->pts = ctx->buffer_time[1]; @@ -915,7 +911,7 @@ static int decode(AVCodecContext *avctx, AVSubtitle *sub, capture_screen(ctx); ctx->buffer_changed = 0; - ret = ff_ass_add_rect2(sub, ctx->buffer[bidx].str, ctx->readorder++, 0, NULL, NULL, &nb_rect_allocated); + ret = avpriv_ass_add_rect(sub, ctx->buffer[bidx].str, ctx->readorder++, 0, NULL, NULL); if (ret < 0) return ret; sub->end_display_time = -1; diff --git a/libavcodec/jacosubdec.c b/libavcodec/jacosubdec.c index 08349a9ec8..9108050291 100644 --- a/libavcodec/jacosubdec.c +++ b/libavcodec/jacosubdec.c @@ -182,7 +182,7 @@ static int jacosub_decode_frame(AVCodecContext *avctx, AVSubtitle *sub, av_bprint_init(&buffer, JSS_MAX_LINESIZE, JSS_MAX_LINESIZE); jacosub_to_ass(avctx, &buffer, ptr); - ret = ff_ass_add_rect(sub, buffer.str, s->readorder++, 0, NULL, NULL); + ret = avpriv_ass_add_rect(sub, buffer.str, s->readorder++, 0, NULL, NULL); av_bprint_finalize(&buffer, NULL); if (ret < 0) return ret; diff --git a/libavcodec/libaribb24.c b/libavcodec/libaribb24.c index f40517e22e..e69ffb5653 100644 --- a/libavcodec/libaribb24.c +++ b/libavcodec/libaribb24.c @@ -274,7 +274,7 @@ next_region: av_log(avctx, AV_LOG_DEBUG, "Styled ASS line: %s\n", buf.str); - ret = ff_ass_add_rect(sub, buf.str, b24->read_order++, + ret = avpriv_ass_add_rect(sub, buf.str, b24->read_order++, 0, NULL, NULL); } diff --git a/libavcodec/libzvbi-teletextdec.c b/libavcodec/libzvbi-teletextdec.c index 486aa1724e..1afc13ce21 100644 --- a/libavcodec/libzvbi-teletextdec.c +++ b/libavcodec/libzvbi-teletextdec.c @@ -153,12 +153,12 @@ static char *create_ass_text(TeletextContext *ctx, const char *text) AVBPrint buf; av_bprint_init(&buf, 0, AV_BPRINT_SIZE_UNLIMITED); - ff_ass_bprint_text_event(&buf, text, strlen(text), "", 0); + avpriv_ass_bprint_text_event(&buf, text, strlen(text), "", 0); if (!av_bprint_is_complete(&buf)) { av_bprint_finalize(&buf, NULL); return NULL; } - dialog = ff_ass_get_dialog(ctx->readorder++, 0, NULL, NULL, buf.str); + dialog = avpriv_ass_get_dialog(ctx->readorder++, 0, NULL, NULL, buf.str); av_bprint_finalize(&buf, NULL); return dialog; } @@ -225,7 +225,7 @@ static int gen_sub_text(TeletextContext *ctx, AVSubtitleRect *sub_rect, vbi_page } av_log(ctx, AV_LOG_DEBUG, "subtext:%s:txetbus\n", sub_rect->ass); } else { - sub_rect->type = SUBTITLE_NONE; + sub_rect->type = AV_SUBTITLE_FMT_NONE; } av_bprint_finalize(&buf, NULL); return 0; @@ -395,7 +395,7 @@ static int gen_sub_ass(TeletextContext *ctx, AVSubtitleRect *sub_rect, vbi_page if (buf.len) { sub_rect->type = SUBTITLE_ASS; - sub_rect->ass = ff_ass_get_dialog(ctx->readorder++, 0, is_subtitle_page ? "Subtitle" : "Teletext", NULL, buf.str); + sub_rect->ass = avpriv_ass_get_dialog(ctx->readorder++, 0, is_subtitle_page ? "Subtitle" : "Teletext", NULL, buf.str); if (!sub_rect->ass) { av_bprint_finalize(&buf, NULL); @@ -403,7 +403,7 @@ static int gen_sub_ass(TeletextContext *ctx, AVSubtitleRect *sub_rect, vbi_page } av_log(ctx, AV_LOG_DEBUG, "subtext:%s:txetbus\n", sub_rect->ass); } else { - sub_rect->type = SUBTITLE_NONE; + sub_rect->type = AV_SUBTITLE_FMT_NONE; } av_bprint_finalize(&buf, NULL); return 0; @@ -463,7 +463,7 @@ static int gen_sub_bitmap(TeletextContext *ctx, AVSubtitleRect *sub_rect, vbi_pa if (vc >= vcend) { av_log(ctx, AV_LOG_DEBUG, "dropping empty page %3x\n", page->pgno); - sub_rect->type = SUBTITLE_NONE; + sub_rect->type = AV_SUBTITLE_FMT_NONE; return 0; } @@ -696,7 +696,7 @@ static int teletext_decode_frame(AVCodecContext *avctx, AVSubtitle *sub, sub->num_rects = 0; sub->pts = ctx->pages->pts; - if (ctx->pages->sub_rect->type != SUBTITLE_NONE) { + if (ctx->pages->sub_rect->type != AV_SUBTITLE_FMT_NONE) { sub->rects = av_malloc(sizeof(*sub->rects)); if (sub->rects) { sub->num_rects = 1; diff --git a/libavcodec/microdvddec.c b/libavcodec/microdvddec.c index 786a3845fd..d254b1cd93 100644 --- a/libavcodec/microdvddec.c +++ b/libavcodec/microdvddec.c @@ -309,7 +309,7 @@ static int microdvd_decode_frame(AVCodecContext *avctx, AVSubtitle *sub, } } if (new_line.len) { - int ret = ff_ass_add_rect(sub, new_line.str, s->readorder++, 0, NULL, NULL); + int ret = avpriv_ass_add_rect(sub, new_line.str, s->readorder++, 0, NULL, NULL); av_bprint_finalize(&new_line, NULL); if (ret < 0) return ret; @@ -362,8 +362,9 @@ static int microdvd_init(AVCodecContext *avctx) } } } - return ff_ass_subtitle_header(avctx, font_buf.str, font_size, color, - ASS_DEFAULT_BACK_COLOR, bold, italic, + return ff_ass_subtitle_header_full(avctx, ASS_DEFAULT_PLAYRESX, ASS_DEFAULT_PLAYRESY, + font_buf.str, font_size, color, color, + ASS_DEFAULT_BACK_COLOR, ASS_DEFAULT_BACK_COLOR, bold, italic, underline, ASS_DEFAULT_BORDERSTYLE, alignment); } diff --git a/libavcodec/movtextdec.c b/libavcodec/movtextdec.c index f799252bf2..3c9e84c5e9 100644 --- a/libavcodec/movtextdec.c +++ b/libavcodec/movtextdec.c @@ -22,7 +22,6 @@ #include "avcodec.h" #include "ass.h" #include "libavutil/opt.h" -#include "libavutil/avstring.h" #include "libavutil/common.h" #include "libavutil/bprint.h" #include "libavutil/intreadwrite.h" @@ -553,7 +552,7 @@ static int mov_text_decode_frame(AVCodecContext *avctx, AVSubtitle *sub, } else text_to_ass(&buf, ptr, end, avctx); - ret = ff_ass_add_rect(sub, buf.str, m->readorder++, 0, NULL, NULL); + ret = avpriv_ass_add_rect(sub, buf.str, m->readorder++, 0, NULL, NULL); av_bprint_finalize(&buf, NULL); if (ret < 0) return ret; diff --git a/libavcodec/movtextenc.c b/libavcodec/movtextenc.c index 7aa74d7c9d..6f87e30904 100644 --- a/libavcodec/movtextenc.c +++ b/libavcodec/movtextenc.c @@ -26,8 +26,8 @@ #include "libavutil/intreadwrite.h" #include "libavutil/mem.h" #include "libavutil/common.h" -#include "ass_split.h" -#include "ass.h" +#include "libavutil/ass_split_internal.h" +#include "libavutil/ass_internal.h" #include "bytestream.h" #include "codec_internal.h" @@ -167,7 +167,7 @@ static int mov_text_encode_close(AVCodecContext *avctx) { MovTextContext *s = avctx->priv_data; - ff_ass_split_free(s->ass_ctx); + avpriv_ass_split_free(s->ass_ctx); av_freep(&s->style_attributes); av_freep(&s->fonts); av_bprint_finalize(&s->buffer, NULL); @@ -222,7 +222,7 @@ static int encode_sample_description(AVCodecContext *avctx) else s->font_scale_factor = 1; - style = ff_ass_style_get(s->ass_ctx, "Default"); + style = avpriv_ass_style_get(s->ass_ctx, "Default"); if (!style && ass->styles_count) { style = &ass->styles[0]; } @@ -329,7 +329,7 @@ static av_cold int mov_text_encode_init(AVCodecContext *avctx) av_bprint_init(&s->buffer, 0, AV_BPRINT_SIZE_UNLIMITED); - s->ass_ctx = ff_ass_split(avctx->subtitle_header); + s->ass_ctx = avpriv_ass_split(avctx->subtitle_header); if (!s->ass_ctx) return AVERROR_INVALIDDATA; ret = encode_sample_description(avctx); @@ -566,7 +566,7 @@ static void mov_text_ass_style_set(MovTextContext *s, ASSStyle *style) static void mov_text_dialog(MovTextContext *s, ASSDialog *dialog) { - ASSStyle *style = ff_ass_style_get(s->ass_ctx, dialog->style); + ASSStyle *style = avpriv_ass_style_get(s->ass_ctx, dialog->style); s->ass_dialog_style = style; mov_text_ass_style_set(s, style); @@ -580,7 +580,7 @@ static void mov_text_cancel_overrides_cb(void *priv, const char *style_name) if (!style_name || !*style_name) style = s->ass_dialog_style; else - style= ff_ass_style_get(s->ass_ctx, style_name); + style= avpriv_ass_style_get(s->ass_ctx, style_name); mov_text_ass_style_set(s, style); } @@ -652,12 +652,12 @@ static int mov_text_encode_frame(AVCodecContext *avctx, unsigned char *buf, return AVERROR(EINVAL); } - dialog = ff_ass_split_dialog(s->ass_ctx, ass); + dialog = avpriv_ass_split_dialog(s->ass_ctx, ass); if (!dialog) return AVERROR(ENOMEM); mov_text_dialog(s, dialog); - ff_ass_split_override_codes(&mov_text_callbacks, s, dialog->text); - ff_ass_free_dialog(&dialog); + avpriv_ass_split_override_codes(&mov_text_callbacks, s, dialog->text); + avpriv_ass_free_dialog(&dialog); } if (s->buffer.len > UINT16_MAX) diff --git a/libavcodec/mpl2dec.c b/libavcodec/mpl2dec.c index 8639acf0b0..896323156c 100644 --- a/libavcodec/mpl2dec.c +++ b/libavcodec/mpl2dec.c @@ -73,7 +73,7 @@ static int mpl2_decode_frame(AVCodecContext *avctx, AVSubtitle *sub, av_bprint_init(&buf, 0, AV_BPRINT_SIZE_UNLIMITED); if (ptr && avpkt->size > 0 && *ptr && !mpl2_event_to_ass(&buf, ptr)) - ret = ff_ass_add_rect(sub, buf.str, s->readorder++, 0, NULL, NULL); + ret = avpriv_ass_add_rect(sub, buf.str, s->readorder++, 0, NULL, NULL); av_bprint_finalize(&buf, NULL); if (ret < 0) return ret; diff --git a/libavcodec/realtextdec.c b/libavcodec/realtextdec.c index b09ff78ea4..4b17f31662 100644 --- a/libavcodec/realtextdec.c +++ b/libavcodec/realtextdec.c @@ -66,7 +66,7 @@ static int realtext_decode_frame(AVCodecContext *avctx, AVSubtitle *sub, av_bprint_init(&buf, 0, 4096); if (ptr && avpkt->size > 0 && !rt_event_to_ass(&buf, ptr)) - ret = ff_ass_add_rect(sub, buf.str, s->readorder++, 0, NULL, NULL); + ret = avpriv_ass_add_rect(sub, buf.str, s->readorder++, 0, NULL, NULL); av_bprint_finalize(&buf, NULL); if (ret < 0) return ret; diff --git a/libavcodec/samidec.c b/libavcodec/samidec.c index e84c4bc6b8..07ef1cb0d0 100644 --- a/libavcodec/samidec.c +++ b/libavcodec/samidec.c @@ -143,7 +143,7 @@ static int sami_decode_frame(AVCodecContext *avctx, AVSubtitle *sub, if (ret < 0) return ret; // TODO: pass escaped sami->encoded_source.str as source - ret = ff_ass_add_rect(sub, sami->full.str, sami->readorder++, 0, NULL, NULL); + ret = avpriv_ass_add_rect(sub, sami->full.str, sami->readorder++, 0, NULL, NULL); if (ret < 0) return ret; } diff --git a/libavcodec/srtdec.c b/libavcodec/srtdec.c index 8697f5b149..9eb30fe7d2 100644 --- a/libavcodec/srtdec.c +++ b/libavcodec/srtdec.c @@ -79,7 +79,7 @@ static int srt_decode_frame(AVCodecContext *avctx, AVSubtitle *sub, ret = srt_to_ass(avctx, &buffer, avpkt->data, x1, y1, x2, y2); if (ret >= 0) - ret = ff_ass_add_rect(sub, buffer.str, s->readorder++, 0, NULL, NULL); + ret = avpriv_ass_add_rect(sub, buffer.str, s->readorder++, 0, NULL, NULL); av_bprint_finalize(&buffer, NULL); if (ret < 0) return ret; diff --git a/libavcodec/srtenc.c b/libavcodec/srtenc.c index 7e36a2200c..245e6a37c5 100644 --- a/libavcodec/srtenc.c +++ b/libavcodec/srtenc.c @@ -25,9 +25,9 @@ #include "avcodec.h" #include "libavutil/avstring.h" #include "libavutil/bprint.h" -#include "ass_split.h" -#include "ass.h" #include "codec_internal.h" +#include "libavutil/ass_split_internal.h" +#include "libavutil/ass_internal.h" #define SRT_STACK_SIZE 64 @@ -96,7 +96,7 @@ static void srt_stack_push_pop(SRTContext *s, const char c, int close) static void srt_style_apply(SRTContext *s, const char *style) { - ASSStyle *st = ff_ass_style_get(s->ass_ctx, style); + ASSStyle *st = avpriv_ass_style_get(s->ass_ctx, style); if (st) { int c = st->primary_color & 0xFFFFFF; if (st->font_name && strcmp(st->font_name, ASS_DEFAULT_FONT) || @@ -137,7 +137,7 @@ static av_cold int srt_encode_init(AVCodecContext *avctx) { SRTContext *s = avctx->priv_data; s->avctx = avctx; - s->ass_ctx = ff_ass_split(avctx->subtitle_header); + s->ass_ctx = avpriv_ass_split(avctx->subtitle_header); av_bprint_init(&s->buffer, 0, AV_BPRINT_SIZE_UNLIMITED); return s->ass_ctx ? 0 : AVERROR_INVALIDDATA; } @@ -247,14 +247,14 @@ static int encode_frame(AVCodecContext *avctx, return AVERROR(EINVAL); } - dialog = ff_ass_split_dialog(s->ass_ctx, ass); + dialog = avpriv_ass_split_dialog(s->ass_ctx, ass); if (!dialog) return AVERROR(ENOMEM); s->alignment_applied = 0; if (avctx->codec_id == AV_CODEC_ID_SUBRIP) srt_style_apply(s, dialog->style); - ff_ass_split_override_codes(cb, s, dialog->text); - ff_ass_free_dialog(&dialog); + avpriv_ass_split_override_codes(cb, s, dialog->text); + avpriv_ass_free_dialog(&dialog); } if (!av_bprint_is_complete(&s->buffer)) @@ -286,7 +286,7 @@ static int text_encode_frame(AVCodecContext *avctx, static int srt_encode_close(AVCodecContext *avctx) { SRTContext *s = avctx->priv_data; - ff_ass_split_free(s->ass_ctx); + avpriv_ass_split_free(s->ass_ctx); av_bprint_finalize(&s->buffer, NULL); return 0; } diff --git a/libavcodec/subviewerdec.c b/libavcodec/subviewerdec.c index 209f69505f..f860d05c00 100644 --- a/libavcodec/subviewerdec.c +++ b/libavcodec/subviewerdec.c @@ -57,7 +57,7 @@ static int subviewer_decode_frame(AVCodecContext *avctx, AVSubtitle *sub, av_bprint_init(&buf, 0, AV_BPRINT_SIZE_UNLIMITED); if (ptr && avpkt->size > 0 && !subviewer_event_to_ass(&buf, ptr)) - ret = ff_ass_add_rect(sub, buf.str, s->readorder++, 0, NULL, NULL); + ret = avpriv_ass_add_rect(sub, buf.str, s->readorder++, 0, NULL, NULL); av_bprint_finalize(&buf, NULL); if (ret < 0) return ret; diff --git a/libavcodec/textdec.c b/libavcodec/textdec.c index b9aebff002..604175ddc2 100644 --- a/libavcodec/textdec.c +++ b/libavcodec/textdec.c @@ -55,8 +55,8 @@ static int text_decode_frame(AVCodecContext *avctx, AVSubtitle *sub, av_bprint_init(&buf, 0, AV_BPRINT_SIZE_UNLIMITED); if (ptr && avpkt->size > 0 && *ptr) { - ff_ass_bprint_text_event(&buf, ptr, avpkt->size, text->linebreaks, text->keep_ass_markup); - ret = ff_ass_add_rect(sub, buf.str, text->readorder++, 0, NULL, NULL); + avpriv_ass_bprint_text_event(&buf, ptr, avpkt->size, text->linebreaks, text->keep_ass_markup); + ret = avpriv_ass_add_rect(sub, buf.str, text->readorder++, 0, NULL, NULL); } av_bprint_finalize(&buf, NULL); if (ret < 0) diff --git a/libavcodec/ttmlenc.c b/libavcodec/ttmlenc.c index fb05c38968..c3f873147c 100644 --- a/libavcodec/ttmlenc.c +++ b/libavcodec/ttmlenc.c @@ -32,8 +32,7 @@ #include "libavutil/avstring.h" #include "libavutil/bprint.h" #include "libavutil/internal.h" -#include "ass_split.h" -#include "ass.h" +#include "libavutil/ass_split_internal.h" #include "ttmlenc.h" typedef struct { @@ -95,7 +94,7 @@ static int ttml_encode_frame(AVCodecContext *avctx, uint8_t *buf, return AVERROR(EINVAL); } - dialog = ff_ass_split_dialog(s->ass_ctx, ass); + dialog = avpriv_ass_split_dialog(s->ass_ctx, ass); if (!dialog) return AVERROR(ENOMEM); @@ -107,7 +106,7 @@ static int ttml_encode_frame(AVCodecContext *avctx, uint8_t *buf, av_bprintf(&s->buffer, "\">"); } - ret = ff_ass_split_override_codes(&ttml_callbacks, s, dialog->text); + ret = avpriv_ass_split_override_codes(&ttml_callbacks, s, dialog->text); if (ret < 0) { int log_level = (ret != AVERROR_INVALIDDATA || avctx->err_recognition & AV_EF_EXPLODE) ? @@ -118,7 +117,7 @@ static int ttml_encode_frame(AVCodecContext *avctx, uint8_t *buf, av_err2str(ret)); if (log_level == AV_LOG_ERROR) { - ff_ass_free_dialog(&dialog); + avpriv_ass_free_dialog(&dialog); return ret; } } @@ -126,7 +125,7 @@ static int ttml_encode_frame(AVCodecContext *avctx, uint8_t *buf, if (dialog->style) av_bprintf(&s->buffer, ""); - ff_ass_free_dialog(&dialog); + avpriv_ass_free_dialog(&dialog); } if (!av_bprint_is_complete(&s->buffer)) @@ -148,7 +147,7 @@ static av_cold int ttml_encode_close(AVCodecContext *avctx) { TTMLContext *s = avctx->priv_data; - ff_ass_split_free(s->ass_ctx); + avpriv_ass_split_free(s->ass_ctx); av_bprint_finalize(&s->buffer, NULL); @@ -372,7 +371,7 @@ static av_cold int ttml_encode_init(AVCodecContext *avctx) av_bprint_init(&s->buffer, 0, AV_BPRINT_SIZE_UNLIMITED); - if (!(s->ass_ctx = ff_ass_split(avctx->subtitle_header))) { + if (!(s->ass_ctx = avpriv_ass_split(avctx->subtitle_header))) { return AVERROR_INVALIDDATA; } diff --git a/libavcodec/webvttdec.c b/libavcodec/webvttdec.c index 690f00dc47..b255484464 100644 --- a/libavcodec/webvttdec.c +++ b/libavcodec/webvttdec.c @@ -90,7 +90,7 @@ static int webvtt_decode_frame(AVCodecContext *avctx, AVSubtitle *sub, av_bprint_init(&buf, 0, AV_BPRINT_SIZE_UNLIMITED); if (ptr && avpkt->size > 0 && !webvtt_event_to_ass(&buf, ptr)) - ret = ff_ass_add_rect(sub, buf.str, s->readorder++, 0, NULL, NULL); + ret = avpriv_ass_add_rect(sub, buf.str, s->readorder++, 0, NULL, NULL); av_bprint_finalize(&buf, NULL); if (ret < 0) return ret; diff --git a/libavcodec/webvttenc.c b/libavcodec/webvttenc.c index 4369aacb74..2df31f8fba 100644 --- a/libavcodec/webvttenc.c +++ b/libavcodec/webvttenc.c @@ -24,9 +24,9 @@ #include "avcodec.h" #include "libavutil/avstring.h" #include "libavutil/bprint.h" -#include "ass_split.h" -#include "ass.h" #include "codec_internal.h" +#include "libavutil/ass_split_internal.h" +#include "libavutil/ass_internal.h" #define WEBVTT_STACK_SIZE 64 typedef struct { @@ -93,7 +93,7 @@ static void webvtt_stack_push_pop(WebVTTContext *s, const char c, int close) static void webvtt_style_apply(WebVTTContext *s, const char *style) { - ASSStyle *st = ff_ass_style_get(s->ass_ctx, style); + ASSStyle *st = avpriv_ass_style_get(s->ass_ctx, style); if (st) { if (st->bold != ASS_DEFAULT_BOLD) { webvtt_print(s, ""); @@ -172,12 +172,12 @@ static int webvtt_encode_frame(AVCodecContext *avctx, return AVERROR(EINVAL); } - dialog = ff_ass_split_dialog(s->ass_ctx, ass); + dialog = avpriv_ass_split_dialog(s->ass_ctx, ass); if (!dialog) return AVERROR(ENOMEM); webvtt_style_apply(s, dialog->style); - ff_ass_split_override_codes(&webvtt_callbacks, s, dialog->text); - ff_ass_free_dialog(&dialog); + avpriv_ass_split_override_codes(&webvtt_callbacks, s, dialog->text); + avpriv_ass_free_dialog(&dialog); } if (!av_bprint_is_complete(&s->buffer)) @@ -197,7 +197,7 @@ static int webvtt_encode_frame(AVCodecContext *avctx, static int webvtt_encode_close(AVCodecContext *avctx) { WebVTTContext *s = avctx->priv_data; - ff_ass_split_free(s->ass_ctx); + avpriv_ass_split_free(s->ass_ctx); av_bprint_finalize(&s->buffer, NULL); return 0; } @@ -206,7 +206,7 @@ static av_cold int webvtt_encode_init(AVCodecContext *avctx) { WebVTTContext *s = avctx->priv_data; s->avctx = avctx; - s->ass_ctx = ff_ass_split(avctx->subtitle_header); + s->ass_ctx = avpriv_ass_split(avctx->subtitle_header); av_bprint_init(&s->buffer, 0, AV_BPRINT_SIZE_UNLIMITED); return s->ass_ctx ? 0 : AVERROR_INVALIDDATA; } diff --git a/libavutil/Makefile b/libavutil/Makefile index 4ab4fcb01b..0eba374477 100644 --- a/libavutil/Makefile +++ b/libavutil/Makefile @@ -103,6 +103,8 @@ BUILT_HEADERS = avconfig.h \ OBJS = adler32.o \ aes.o \ aes_ctr.o \ + ass.o \ + ass_split.o \ audio_fifo.o \ avstring.o \ avsscanf.o \ diff --git a/libavcodec/ass.c b/libavutil/ass.c similarity index 59% rename from libavcodec/ass.c rename to libavutil/ass.c index 728cfb1ab5..1a46f8d589 100644 --- a/libavcodec/ass.c +++ b/libavutil/ass.c @@ -19,22 +19,23 @@ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ -#include "avcodec.h" -#include "ass.h" +#include "ass_internal.h" + +#include "subfmt.h" #include "libavutil/avstring.h" #include "libavutil/bprint.h" #include "libavutil/common.h" #include "version.h" -int ff_ass_subtitle_header_full(AVCodecContext *avctx, - int play_res_x, int play_res_y, - const char *font, int font_size, - int primary_color, int secondary_color, - int outline_color, int back_color, - int bold, int italic, int underline, - int border_style, int alignment) +char* avpriv_ass_get_subtitle_header_full(int play_res_x, int play_res_y, + const char *font, int font_size, + int primary_color, int secondary_color, + int outline_color, int back_color, + int bold, int italic, int underline, + int border_style, int alignment, + int print_av_version) { - avctx->subtitle_header = av_asprintf( + char* header = av_asprintf( "[Script Info]\r\n" "; Script generated by FFmpeg/Lavc%s\r\n" "ScriptType: v4.00+\r\n" @@ -69,34 +70,31 @@ int ff_ass_subtitle_header_full(AVCodecContext *avctx, "\r\n" "[Events]\r\n" "Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text\r\n", - !(avctx->flags & AV_CODEC_FLAG_BITEXACT) ? AV_STRINGIFY(LIBAVCODEC_VERSION) : "", + print_av_version ? AV_STRINGIFY(LIBAVCODEC_VERSION) : "", play_res_x, play_res_y, font, font_size, primary_color, secondary_color, outline_color, back_color, -bold, -italic, -underline, border_style, alignment); - if (!avctx->subtitle_header) - return AVERROR(ENOMEM); - avctx->subtitle_header_size = strlen(avctx->subtitle_header); - return 0; + return header; } -int ff_ass_subtitle_header(AVCodecContext *avctx, - const char *font, int font_size, +char* avpriv_ass_get_subtitle_header(const char *font, int font_size, int color, int back_color, int bold, int italic, int underline, - int border_style, int alignment) + int border_style, int alignment, + int print_av_version) { - return ff_ass_subtitle_header_full(avctx, - ASS_DEFAULT_PLAYRESX, ASS_DEFAULT_PLAYRESY, - font, font_size, color, color, - back_color, back_color, - bold, italic, underline, - border_style, alignment); + return avpriv_ass_get_subtitle_header_full(ASS_DEFAULT_PLAYRESX, ASS_DEFAULT_PLAYRESY, + font, font_size, color, color, + back_color, back_color, + bold, italic, underline, + border_style, alignment, + print_av_version); } -int ff_ass_subtitle_header_default(AVCodecContext *avctx) +char* avpriv_ass_get_subtitle_header_default(int print_av_version) { - return ff_ass_subtitle_header(avctx, ASS_DEFAULT_FONT, + return avpriv_ass_get_subtitle_header(ASS_DEFAULT_FONT, ASS_DEFAULT_FONT_SIZE, ASS_DEFAULT_COLOR, ASS_DEFAULT_BACK_COLOR, @@ -104,10 +102,11 @@ int ff_ass_subtitle_header_default(AVCodecContext *avctx) ASS_DEFAULT_ITALIC, ASS_DEFAULT_UNDERLINE, ASS_DEFAULT_BORDERSTYLE, - ASS_DEFAULT_ALIGNMENT); + ASS_DEFAULT_ALIGNMENT, + print_av_version); } -char *ff_ass_get_dialog(int readorder, int layer, const char *style, +char *avpriv_ass_get_dialog(int readorder, int layer, const char *style, const char *speaker, const char *text) { return av_asprintf("%d,%d,%s,%s,0,0,0,,%s", @@ -115,61 +114,17 @@ char *ff_ass_get_dialog(int readorder, int layer, const char *style, speaker ? speaker : "", text); } -int ff_ass_add_rect2(AVSubtitle *sub, const char *dialog, - int readorder, int layer, const char *style, - const char *speaker, unsigned *nb_rect_allocated) -{ - AVSubtitleRect **rects = sub->rects, *rect; - char *ass_str; - uint64_t new_nb = 0; - - if (sub->num_rects >= UINT_MAX) - return AVERROR(ENOMEM); - - if (nb_rect_allocated && *nb_rect_allocated <= sub->num_rects) { - if (sub->num_rects < UINT_MAX / 17 * 16) { - new_nb = sub->num_rects + sub->num_rects/16 + 1; - } else - new_nb = UINT_MAX; - } else if (!nb_rect_allocated) - new_nb = sub->num_rects + 1; - - if (new_nb) { - rects = av_realloc_array(rects, new_nb, sizeof(*sub->rects)); - if (!rects) - return AVERROR(ENOMEM); - if (nb_rect_allocated) - *nb_rect_allocated = new_nb; - sub->rects = rects; - } - - rect = av_mallocz(sizeof(*rect)); - if (!rect) - return AVERROR(ENOMEM); - rects[sub->num_rects++] = rect; - rect->type = SUBTITLE_ASS; - ass_str = ff_ass_get_dialog(readorder, layer, style, speaker, dialog); - if (!ass_str) - return AVERROR(ENOMEM); - rect->ass = ass_str; - return 0; -} - -int ff_ass_add_rect(AVSubtitle *sub, const char *dialog, - int readorder, int layer, const char *style, - const char *speaker) +char *avpriv_ass_get_dialog_ex(int readorder, int layer, const char *style, + const char *speaker, int margin_l, int margin_r, + int margin_v, const char *effect, const char *text) { - return ff_ass_add_rect2(sub, dialog, readorder, layer, style, speaker, NULL); -} - -void ff_ass_decoder_flush(AVCodecContext *avctx) -{ - FFASSDecoderContext *s = avctx->priv_data; - if (!(avctx->flags2 & AV_CODEC_FLAG2_RO_FLUSH_NOOP)) - s->readorder = 0; + return av_asprintf("%d,%d,%s,%s,%d,%d,%d,%s,%s", + readorder, layer, style ? style : "Default", + speaker ? speaker : "", margin_l, margin_r, + margin_v, effect ? effect : "", text); } -void ff_ass_bprint_text_event(AVBPrint *buf, const char *p, int size, +void avpriv_ass_bprint_text_event(AVBPrint *buf, const char *p, int size, const char *linebreaks, int keep_ass_markup) { const char *p_end = p + size; diff --git a/libavutil/ass_internal.h b/libavutil/ass_internal.h new file mode 100644 index 0000000000..fc4e6232fb --- /dev/null +++ b/libavutil/ass_internal.h @@ -0,0 +1,135 @@ +/* + * SSA/ASS common functions + * Copyright (c) 2010 Aurelien Jacobs + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVUTIL_ASS_INTERNAL_H +#define AVUTIL_ASS_INTERNAL_H + +#include "subfmt.h" +#include "libavutil/bprint.h" + +#define ASS_DEFAULT_PLAYRESX 384 +#define ASS_DEFAULT_PLAYRESY 288 + +/** + * @name Default values for ASS style + * @{ + */ +#define ASS_DEFAULT_FONT "Arial" +#define ASS_DEFAULT_FONT_SIZE 16 +#define ASS_DEFAULT_COLOR 0xffffff +#define ASS_DEFAULT_BACK_COLOR 0 +#define ASS_DEFAULT_BOLD 0 +#define ASS_DEFAULT_ITALIC 0 +#define ASS_DEFAULT_UNDERLINE 0 +#define ASS_DEFAULT_ALIGNMENT 2 +#define ASS_DEFAULT_BORDERSTYLE 1 +/** @} */ + +/** + * Generate a suitable AVCodecContext.subtitle_header for SUBTITLE_ASS. + * Can specify all fields explicitly + * + * @param play_res_x subtitle frame width + * @param play_res_y subtitle frame height + * @param font name of the default font face to use + * @param font_size default font size to use + * @param primary_color default text color to use (ABGR) + * @param secondary_color default secondary text color to use (ABGR) + * @param outline_color default outline color to use (ABGR) + * @param back_color default background color to use (ABGR) + * @param bold 1 for bold text, 0 for normal text + * @param italic 1 for italic text, 0 for normal text + * @param underline 1 for underline text, 0 for normal text + * @param border_style 1 for outline, 3 for opaque box + * @param alignment position of the text (left, center, top...), defined after + * the layout of the numpad (1-3 sub, 4-6 mid, 7-9 top) + * @param print_av_version include library version in header + * @return a string containing the subtitle header that needs + * to be released via av_free() + */ +char* avpriv_ass_get_subtitle_header_full(int play_res_x, int play_res_y, + const char *font, int font_size, + int primary_color, int secondary_color, + int outline_color, int back_color, + int bold, int italic, int underline, + int border_style, int alignment, + int print_av_version); + +/** + * Generate a suitable AVCodecContext.subtitle_header for SUBTITLE_ASS. + * + * @param font name of the default font face to use + * @param font_size default font size to use + * @param color default text color to use (ABGR) + * @param back_color default background color to use (ABGR) + * @param bold 1 for bold text, 0 for normal text + * @param italic 1 for italic text, 0 for normal text + * @param underline 1 for underline text, 0 for normal text + * @param border_style 1 for outline, 3 for opaque box + * @param alignment position of the text (left, center, top...), defined after + * the layout of the numpad (1-3 sub, 4-6 mid, 7-9 top) + * @param print_av_version include library version in header + * @return a string containing the subtitle header that needs + * to be released via av_free() + */ +char* avpriv_ass_get_subtitle_header(const char *font, int font_size, + int color, int back_color, + int bold, int italic, int underline, + int border_style, int alignment, + int print_av_version); + +/** + * Generate a suitable AVCodecContext.subtitle_header for SUBTITLE_ASS + * with default style. + * + * @param print_av_version include library version in header + * @return a string containing the subtitle header that needs + * to be released via av_free() + */ +char* avpriv_ass_get_subtitle_header_default(int print_av_version); + +/** + * Craft an ASS dialog string. + */ +char *avpriv_ass_get_dialog(int readorder, int layer, const char *style, + const char *speaker, const char *text); + +/** + * Craft an ASS dialog string. + */ +char *avpriv_ass_get_dialog_ex(int readorder, int layer, const char *style, + const char *speaker, int margin_l, int margin_r, + int margin_v, const char *effect, const char *text); + +/** + * Escape a text subtitle using ASS syntax into an AVBPrint buffer. + * Newline characters will be escaped to \N. + * + * @param buf pointer to an initialized AVBPrint buffer + * @param p source text + * @param size size of the source text + * @param linebreaks additional newline chars, which will be escaped to \N + * @param keep_ass_markup braces and backslash will not be escaped if set + */ +void avpriv_ass_bprint_text_event(AVBPrint *buf, const char *p, int size, + const char *linebreaks, int keep_ass_markup); + +#endif /* AVUTIL_ASS_INTERNAL_H */ diff --git a/libavcodec/ass_split.c b/libavutil/ass_split.c similarity index 71% rename from libavcodec/ass_split.c rename to libavutil/ass_split.c index 73ef6196c5..3e5e24e2fa 100644 --- a/libavcodec/ass_split.c +++ b/libavutil/ass_split.c @@ -28,7 +28,7 @@ #include "libavutil/error.h" #include "libavutil/macros.h" #include "libavutil/mem.h" -#include "ass_split.h" +#include "ass_split_internal.h" typedef enum { ASS_STR, @@ -379,7 +379,7 @@ static int ass_split(ASSSplitContext *ctx, const char *buf) return buf ? 0 : AVERROR_INVALIDDATA; } -ASSSplitContext *ff_ass_split(const char *buf) +ASSSplitContext *avpriv_ass_split(const char *buf) { ASSSplitContext *ctx = av_mallocz(sizeof(*ctx)); if (!ctx) @@ -388,7 +388,7 @@ ASSSplitContext *ff_ass_split(const char *buf) buf += 3; ctx->current_section = -1; if (ass_split(ctx, buf) < 0) { - ff_ass_split_free(ctx); + avpriv_ass_split_free(ctx); return NULL; } return ctx; @@ -418,7 +418,7 @@ static void free_section(ASSSplitContext *ctx, const ASSSection *section) av_freep((uint8_t *)&ctx->ass + section->offset); } -void ff_ass_free_dialog(ASSDialog **dialogp) +void avpriv_ass_free_dialog(ASSDialog **dialogp) { ASSDialog *dialog = *dialogp; if (!dialog) @@ -430,7 +430,7 @@ void ff_ass_free_dialog(ASSDialog **dialogp) av_freep(dialogp); } -ASSDialog *ff_ass_split_dialog(ASSSplitContext *ctx, const char *buf) +ASSDialog *avpriv_ass_split_dialog(ASSSplitContext *ctx, const char *buf) { int i; static const ASSFields fields[] = { @@ -457,7 +457,7 @@ ASSDialog *ff_ass_split_dialog(ASSSplitContext *ctx, const char *buf) buf = skip_space(buf); len = last ? strlen(buf) : strcspn(buf, ","); if (len >= INT_MAX) { - ff_ass_free_dialog(&dialog); + avpriv_ass_free_dialog(&dialog); return NULL; } convert_func[type](ptr, buf, len); @@ -467,7 +467,7 @@ ASSDialog *ff_ass_split_dialog(ASSSplitContext *ctx, const char *buf) return dialog; } -void ff_ass_split_free(ASSSplitContext *ctx) +void avpriv_ass_split_free(ASSSplitContext *ctx) { if (ctx) { int i; @@ -479,87 +479,209 @@ void ff_ass_split_free(ASSSplitContext *ctx) } } +static int ass_remove_empty_braces(AVBPrint* buffer) +{ + char* tmp; + int ret = 0, n = 0; -int ff_ass_split_override_codes(const ASSCodesCallbacks *callbacks, void *priv, - const char *buf) + if (buffer == NULL || buffer->len == 0 || !av_bprint_is_complete(buffer)) + return 0; + + ret = av_bprint_finalize(buffer, &tmp); + if (ret) + return ret; + + for (unsigned i = 0; i < buffer->len; i++) { + if (tmp[i] == '{' && tmp[i+1] == '}') + i++; + else + tmp[n++] = tmp[i]; + } + + tmp[n++] = '\0'; + + av_bprint_init(buffer, n, n); + av_bprint_append_data(buffer, tmp, n - 1); + av_free(tmp); + + return ret; +} + +static void ass_write_filtered_line(AVBPrint* buffer, const char *buf, int len, enum ASSSplitComponents keep_flags, enum ASSSplitComponents split_component) +{ + if (buffer == NULL || buf == NULL || len == 0) + return; + + if (split_component != ASS_SPLIT_ANY && !(keep_flags & split_component)) + return; + + + av_bprint_append_data(buffer, buf, len - 1); +} + +int avpriv_ass_filter_override_codes(const ASSCodesCallbacks *callbacks, void *priv, const char *buf, AVBPrint* outbuffer, enum ASSSplitComponents keep_flags) { const char *text = NULL; char new_line[2]; - int text_len = 0; + int text_len = 0, ret = 0; while (buf && *buf) { - if (text && callbacks->text && - (sscanf(buf, "\\%1[nN]", new_line) == 1 || - !strncmp(buf, "{\\", 2))) { - callbacks->text(priv, text, text_len); + + if (text && (sscanf(buf, "\\%1[nN]", new_line) == 1 || !strncmp(buf, "{\\", 2))) { + ass_write_filtered_line(outbuffer, text, text_len + 1, keep_flags, ASS_SPLIT_TEXT | ASS_SPLIT_TEXT2); + + if (callbacks->text) + callbacks->text(priv, text, text_len); text = NULL; } + if (sscanf(buf, "\\%1[nN]", new_line) == 1) { if (callbacks->new_line) callbacks->new_line(priv, new_line[0] == 'N'); + ass_write_filtered_line(outbuffer, buf, 3, keep_flags, ASS_SPLIT_ANY); buf += 2; } else if (!strncmp(buf, "{\\", 2)) { + ass_write_filtered_line(outbuffer, buf, 2, keep_flags, ASS_SPLIT_ANY); buf++; while (*buf == '\\') { - char style[2], c[2], sep[2], c_num[2] = "0", tmp[128] = {0}; + char style[4], c[2], axis[3], sep[3], c_num[2] = "0", tmp[128] = {0}; unsigned int color = 0xFFFFFFFF; - int len, size = -1, an = -1, alpha = -1; - int x1, y1, x2, y2, t1 = -1, t2 = -1; + int len, size = -1, an = -1, alpha = -1, scale = 0; + float f1 = 1; + int x1, y1, x2, y2, x3, t1 = -1, t2 = -1, t3 = -1, t4 = -1, accel = 1; if (sscanf(buf, "\\%1[bisu]%1[01\\}]%n", style, c, &len) > 1) { int close = c[0] == '0' ? 1 : c[0] == '1' ? 0 : -1; len += close != -1; + switch (c[0]) { + case 'b': + ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_FONT_BOLD); + break; + case 'u': + ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_FONT_UNDERLINE); + break; + case 'i': + ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_FONT_ITALIC); + break; + case 'a': + ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_FONT_STRIKEOUT); + break; + } if (callbacks->style) callbacks->style(priv, style[0], close); } else if (sscanf(buf, "\\c%1[\\}]%n", sep, &len) > 0 || sscanf(buf, "\\c&H%X&%1[\\}]%n", &color, sep, &len) > 1 || sscanf(buf, "\\%1[1234]c%1[\\}]%n", c_num, sep, &len) > 1 || sscanf(buf, "\\%1[1234]c&H%X&%1[\\}]%n", c_num, &color, sep, &len) > 2) { + ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_COLOR); if (callbacks->color) callbacks->color(priv, color, c_num[0] - '0'); } else if (sscanf(buf, "\\alpha%1[\\}]%n", sep, &len) > 0 || sscanf(buf, "\\alpha&H%2X&%1[\\}]%n", &alpha, sep, &len) > 1 || sscanf(buf, "\\%1[1234]a%1[\\}]%n", c_num, sep, &len) > 1 || sscanf(buf, "\\%1[1234]a&H%2X&%1[\\}]%n", c_num, &alpha, sep, &len) > 2) { + ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_ALPHA); if (callbacks->alpha) callbacks->alpha(priv, alpha, c_num[0] - '0'); } else if (sscanf(buf, "\\fn%1[\\}]%n", sep, &len) > 0 || sscanf(buf, "\\fn%127[^\\}]%1[\\}]%n", tmp, sep, &len) > 1) { + ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_FONT_NAME); if (callbacks->font_name) callbacks->font_name(priv, tmp[0] ? tmp : NULL); } else if (sscanf(buf, "\\fs%1[\\}]%n", sep, &len) > 0 || sscanf(buf, "\\fs%u%1[\\}]%n", &size, sep, &len) > 1) { + ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_FONT_SIZE); if (callbacks->font_size) callbacks->font_size(priv, size); + } else if (sscanf(buf, "\\fscx%1[\\}]%n", sep, &len) > 0 || + sscanf(buf, "\\fscx%f%1[\\}]%n", &f1, sep, &len) > 1) { + ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_FONT_SCALE); + } else if (sscanf(buf, "\\fscy%1[\\}]%n", sep, &len) > 0 || + sscanf(buf, "\\fscy%f%1[\\}]%n", &f1, sep, &len) > 1) { + ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_FONT_SCALE); + } else if (sscanf(buf, "\\fsp%1[\\}]%n", sep, &len) > 0 || + sscanf(buf, "\\fsp%u%1[\\}]%n", &size, sep, &len) > 1) { + ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_FONT_SPACING); + } else if (sscanf(buf, "\\fe%1[\\}]%n", sep, &len) > 0 || + sscanf(buf, "\\fe%u%1[\\}]%n", &size, sep, &len) > 1) { + ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_FONT_CHARSET); + } else if (sscanf(buf, "\\bord%1[\\}]%n", sep, &len) > 0 || + sscanf(buf, "\\bord%u%1[\\}]%n", &size, sep, &len) > 1) { + ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_TEXT_BORDER); + } else if (sscanf(buf, "\\shad%1[\\}]%n", sep, &len) > 0 || + sscanf(buf, "\\shad%u%1[\\}]%n", &size, sep, &len) > 1) { + ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_TEXT_SHADOW); + } else if (sscanf(buf, "\\fr%1[\\}]%n", sep, &len) > 0 || + sscanf(buf, "\\fr%u%1[\\}]%n", &x1, sep, &len) > 1 || + sscanf(buf, "\\fr%1[xyz]%1[\\}]%n", axis, sep, &len) > 1 || + sscanf(buf, "\\fr%1[xyz]%u%1[\\}]%n", axis, &size, sep, &len) > 2) { + ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_TEXT_ROTATE); + } else if (sscanf(buf, "\\blur%1[\\}]%n", sep, &len) > 0 || + sscanf(buf, "\\blur%u%1[\\}]%n", &size, sep, &len) > 1) { + ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_TEXT_BLUR); + } else if (sscanf(buf, "\\be%1[\\}]%n", sep, &len) > 0 || + sscanf(buf, "\\be%u%1[\\}]%n", &size, sep, &len) > 1) { + ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_TEXT_BLUR); + } else if (sscanf(buf, "\\q%1[\\}]%n", sep, &len) > 0 || + sscanf(buf, "\\q%u%1[\\}]%n", &size, sep, &len) > 1) { + ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_TEXT_WRAP); } else if (sscanf(buf, "\\a%1[\\}]%n", sep, &len) > 0 || sscanf(buf, "\\a%2u%1[\\}]%n", &an, sep, &len) > 1 || sscanf(buf, "\\an%1[\\}]%n", sep, &len) > 0 || sscanf(buf, "\\an%1u%1[\\}]%n", &an, sep, &len) > 1) { if (an != -1 && buf[2] != 'n') an = (an&3) + (an&4 ? 6 : an&8 ? 3 : 0); + ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_TEXT_ALIGNMENT); if (callbacks->alignment) callbacks->alignment(priv, an); } else if (sscanf(buf, "\\r%1[\\}]%n", sep, &len) > 0 || sscanf(buf, "\\r%127[^\\}]%1[\\}]%n", tmp, sep, &len) > 1) { + ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_CANCELLING); if (callbacks->cancel_overrides) callbacks->cancel_overrides(priv, tmp); } else if (sscanf(buf, "\\move(%d,%d,%d,%d)%1[\\}]%n", &x1, &y1, &x2, &y2, sep, &len) > 4 || sscanf(buf, "\\move(%d,%d,%d,%d,%d,%d)%1[\\}]%n", &x1, &y1, &x2, &y2, &t1, &t2, sep, &len) > 6) { + ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_MOVE); if (callbacks->move) callbacks->move(priv, x1, y1, x2, y2, t1, t2); } else if (sscanf(buf, "\\pos(%d,%d)%1[\\}]%n", &x1, &y1, sep, &len) > 2) { + ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_POS); if (callbacks->move) callbacks->move(priv, x1, y1, x1, y1, -1, -1); } else if (sscanf(buf, "\\org(%d,%d)%1[\\}]%n", &x1, &y1, sep, &len) > 2) { + ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_ORIGIN); if (callbacks->origin) callbacks->origin(priv, x1, y1); + } else if (sscanf(buf, "\\t(%1[\\}]%n", sep, &len) > 0 || + sscanf(buf, "\\t(%d,%d,%1[\\}]%n", &t1, &t2, sep, &len) > 2 || + sscanf(buf, "\\t(%d,%d,%d,%1[\\}]%n", &t1, &t2, &accel, sep, &len) > 3) { + + len = strcspn(buf, ")") + 2; + + ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_ANIMATE); + if (callbacks->animate) + callbacks->animate(priv, t1, t2, accel, tmp); + } else if (sscanf(buf, "\\fade(%d,%d,%d,%d,%d,%d,%d)%1[\\}]%n", &x1, &x2, &x3, &t1, &t2, &t3, &t4, sep, &len) > 7) { + ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_FADE); + } else if (sscanf(buf, "\\fad(%d,%d)%1[\\}]%n", &t1, &t2, sep, &len) > 2) { + ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_FADE); + } else if (sscanf(buf, "\\clip(%d,%d,%d,%d)%1[\\}]%n", &x1, &y1, &x2, &y2, sep, &len) > 4) { + ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_CLIP); + } else if (sscanf(buf, "\\p%1[\\}]%n", sep, &len) > 0 || + sscanf(buf, "\\p%u%1[\\}]%n", &scale, sep, &len) > 1) { + ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_DRAW); + if (callbacks->drawing_mode) + callbacks->drawing_mode(priv, scale); } else { - len = strcspn(buf+1, "\\}") + 2; /* skip unknown code */ - } + len = strcspn(buf+1, "\\}") + 2; /* unknown code */ + ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_UNKNOWN); + } buf += len - 1; } if (*buf++ != '}') return AVERROR_INVALIDDATA; - } else { + + ass_write_filtered_line(outbuffer, "}", 2, keep_flags, ASS_SPLIT_ANY); + } else { if (!text) { text = buf; text_len = 1; @@ -568,14 +690,27 @@ int ff_ass_split_override_codes(const ASSCodesCallbacks *callbacks, void *priv, buf++; } } + if (text) + ass_write_filtered_line(outbuffer, text, text_len + 1, keep_flags, ASS_SPLIT_TEXT | ASS_SPLIT_TEXT2); if (text && callbacks->text) callbacks->text(priv, text, text_len); if (callbacks->end) callbacks->end(priv); - return 0; + + if (outbuffer) + ret = ass_remove_empty_braces(outbuffer); + + return ret; +} + + +int avpriv_ass_split_override_codes(const ASSCodesCallbacks *callbacks, void *priv, + const char *buf) +{ + return avpriv_ass_filter_override_codes(callbacks, priv, buf, NULL, 0); } -ASSStyle *ff_ass_style_get(ASSSplitContext *ctx, const char *style) +ASSStyle *avpriv_ass_style_get(ASSSplitContext *ctx, const char *style) { ASS *ass = &ctx->ass; int i; diff --git a/libavutil/ass_split_internal.h b/libavutil/ass_split_internal.h new file mode 100644 index 0000000000..5d82aa8886 --- /dev/null +++ b/libavutil/ass_split_internal.h @@ -0,0 +1,254 @@ +/* + * SSA/ASS spliting functions + * Copyright (c) 2010 Aurelien Jacobs + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVUTIL_ASS_SPLIT_INTERNAL_H +#define AVUTIL_ASS_SPLIT_INTERNAL_H + +#include "bprint.h" + +enum ASSSplitComponents +{ + ASS_SPLIT_ANY = 0, + ASS_SPLIT_TEXT = (1 << 0), + ASS_SPLIT_TEXT2 = (1 << 1), // Same semantics as ASS_SPLIT_TEXT. To work around help output default display. + ASS_SPLIT_COLOR = (1 << 2), + ASS_SPLIT_ALPHA = (1 << 3), + ASS_SPLIT_FONT_NAME = (1 << 4), + ASS_SPLIT_FONT_SIZE = (1 << 5), + ASS_SPLIT_FONT_SCALE = (1 << 6), + ASS_SPLIT_FONT_SPACING = (1 << 7), + ASS_SPLIT_FONT_CHARSET = (1 << 8), + ASS_SPLIT_FONT_BOLD = (1 << 9), + ASS_SPLIT_FONT_ITALIC = (1 << 10), + ASS_SPLIT_FONT_UNDERLINE = (1 << 11), + ASS_SPLIT_FONT_STRIKEOUT = (1 << 12), + ASS_SPLIT_TEXT_BORDER = (1 << 13), + ASS_SPLIT_TEXT_SHADOW = (1 << 14), + ASS_SPLIT_TEXT_ROTATE = (1 << 15), + ASS_SPLIT_TEXT_BLUR = (1 << 16), + ASS_SPLIT_TEXT_WRAP = (1 << 17), + ASS_SPLIT_TEXT_ALIGNMENT = (1 << 18), + ASS_SPLIT_CANCELLING = (1 << 19), + ASS_SPLIT_MOVE = (1 << 20), + ASS_SPLIT_POS = (1 << 21), + ASS_SPLIT_ORIGIN = (1 << 22), + ASS_SPLIT_DRAW = (1 << 23), + ASS_SPLIT_ANIMATE = (1 << 24), + ASS_SPLIT_FADE = (1 << 25), + ASS_SPLIT_CLIP = (1 << 26), + ASS_SPLIT_UNKNOWN = (1 << 27), + + ASS_SPLIT_BASIC = ASS_SPLIT_TEXT2 | ASS_SPLIT_COLOR | ASS_SPLIT_ALPHA | ASS_SPLIT_FONT_NAME | ASS_SPLIT_FONT_SIZE | ASS_SPLIT_FONT_SCALE | ASS_SPLIT_FONT_SPACING | ASS_SPLIT_FONT_CHARSET | ASS_SPLIT_FONT_BOLD | ASS_SPLIT_FONT_ITALIC | ASS_SPLIT_FONT_UNDERLINE | ASS_SPLIT_FONT_STRIKEOUT | ASS_SPLIT_TEXT_BORDER | ASS_SPLIT_TEXT_SHADOW | ASS_SPLIT_TEXT_WRAP | ASS_SPLIT_TEXT_ALIGNMENT | ASS_SPLIT_POS | ASS_SPLIT_CANCELLING, + ASS_SPLIT_ALL_KNOWN = ASS_SPLIT_TEXT2 | ASS_SPLIT_COLOR | ASS_SPLIT_ALPHA | ASS_SPLIT_FONT_NAME | ASS_SPLIT_FONT_SIZE | ASS_SPLIT_FONT_SCALE | ASS_SPLIT_FONT_SPACING | ASS_SPLIT_FONT_CHARSET | ASS_SPLIT_FONT_BOLD | ASS_SPLIT_FONT_ITALIC | ASS_SPLIT_FONT_UNDERLINE | ASS_SPLIT_FONT_STRIKEOUT | ASS_SPLIT_TEXT_BORDER | ASS_SPLIT_TEXT_SHADOW | ASS_SPLIT_TEXT_ROTATE | ASS_SPLIT_TEXT_BLUR | ASS_SPLIT_TEXT_WRAP | ASS_SPLIT_TEXT_ALIGNMENT | ASS_SPLIT_CANCELLING | ASS_SPLIT_POS | ASS_SPLIT_MOVE | ASS_SPLIT_ORIGIN | ASS_SPLIT_DRAW | ASS_SPLIT_ANIMATE | ASS_SPLIT_FADE | ASS_SPLIT_CLIP, +}; + + /** + * fields extracted from the [Script Info] section + */ + typedef struct { + char *script_type; /**< SSA script format version (eg. v4.00) */ + char *collisions; /**< how subtitles are moved to prevent collisions */ + int play_res_x; /**< video width that ASS coords are referring to */ + int play_res_y; /**< video height that ASS coords are referring to */ + float timer; /**< time multiplier to apply to SSA clock (in %) */ +} ASSScriptInfo; + +/** + * fields extracted from the [V4(+) Styles] section + */ +typedef struct { + char *name; /**< name of the tyle (case sensitive) */ + char *font_name; /**< font face (case sensitive) */ + int font_size; /**< font height */ + int primary_color; /**< color that a subtitle will normally appear in */ + int secondary_color; + int outline_color; /**< color for outline in ASS, called tertiary in SSA */ + int back_color; /**< color of the subtitle outline or shadow */ + int bold; /**< whether text is bold (1) or not (0) */ + int italic; /**< whether text is italic (1) or not (0) */ + int underline; /**< whether text is underlined (1) or not (0) */ + int strikeout; + float scalex; + float scaley; + float spacing; + float angle; + int border_style; + float outline; + float shadow; + int alignment; /**< position of the text (left, center, top...), + defined after the layout of the numpad + (1-3 sub, 4-6 mid, 7-9 top) */ + int margin_l; + int margin_r; + int margin_v; + int alpha_level; + int encoding; +} ASSStyle; + +/** + * fields extracted from the [Events] section + */ +typedef struct { + int readorder; + int layer; /**< higher numbered layers are drawn over lower numbered */ + int start; /**< start time of the dialog in centiseconds */ + int end; /**< end time of the dialog in centiseconds */ + char *style; /**< name of the ASSStyle to use with this dialog */ + char *name; + int margin_l; + int margin_r; + int margin_v; + char *effect; + char *text; /**< actual text which will be displayed as a subtitle, + can include style override control codes (see + avpriv_ass_split_override_codes()) */ +} ASSDialog; + +/** + * structure containing the whole split ASS data + */ +typedef struct { + ASSScriptInfo script_info; /**< general information about the SSA script*/ + ASSStyle *styles; /**< array of split out styles */ + int styles_count; /**< number of ASSStyle in the styles array */ + ASSDialog *dialogs; /**< array of split out dialogs */ + int dialogs_count; /**< number of ASSDialog in the dialogs array*/ +} ASS; + +/** + * This struct can be casted to ASS to access to the split data. + */ +typedef struct ASSSplitContext ASSSplitContext; + +/** + * Split a full ASS file or a ASS header from a string buffer and store + * the split structure in a newly allocated context. + * + * @param buf String containing the ASS formatted data. + * @return Newly allocated struct containing split data. + */ +ASSSplitContext *avpriv_ass_split(const char *buf); + +/** + * Free a dialogue obtained from avpriv_ass_split_dialog(). + */ +void avpriv_ass_free_dialog(ASSDialog **dialogp); + +/** + * Split one ASS Dialogue line from a string buffer. + * + * @param ctx Context previously initialized by ff_ass_split(). + * @param buf String containing the ASS "Dialogue" line. + * @return Pointer to the split ASSDialog. Must be freed with + * ff_ass_free_dialog() + */ +ASSDialog *avpriv_ass_split_dialog(ASSSplitContext *ctx, const char *buf); + +/** + * Free all the memory allocated for an ASSSplitContext. + * + * @param ctx Context previously initialized by ff_ass_split(). + */ +void avpriv_ass_split_free(ASSSplitContext *ctx); + +/** + * Set of callback functions corresponding to each override codes that can + * be encountered in a "Dialogue" Text field. + */ +typedef struct { + /** + * @defgroup ass_styles ASS styles + * @{ + */ + void (*text)(void *priv, const char *text, int len); + void (*hard_space)(void *priv); + void (*new_line)(void *priv, int forced); + void (*style)(void *priv, char style, int close); + void (*color)(void *priv, unsigned int /* color */, unsigned int color_id); + void (*alpha)(void *priv, int alpha, int alpha_id); + void (*font_name)(void *priv, const char *name); + void (*font_size)(void *priv, int size); + void (*alignment)(void *priv, int alignment); + void (*cancel_overrides)(void *priv, const char *style); + /** @} */ + + /** + * @defgroup ass_functions ASS functions + * @{ + */ + void (*move)(void *priv, int x1, int y1, int x2, int y2, int t1, int t2); + void (*animate)(void *priv, int t1, int t2, int accel, char *style); + void (*origin)(void *priv, int x, int y); + void (*drawing_mode)(void *priv, int scale); + /** @} */ + + /** + * @defgroup ass_ext ASS extensible parsing callback + * @{ + */ + void (*ext)(void *priv, int ext_id, const char *text, int p1, int p2); + /** @} */ + + /** + * @defgroup ass_end end of Dialogue Event + * @{ + */ + void (*end)(void *priv); + /** @} */ +} ASSCodesCallbacks; + +/** + * Split override codes out of a ASS "Dialogue" Text field. + * + * @param callbacks Set of callback functions called for each override code + * encountered. + * @param priv Opaque pointer passed to the callback functions. + * @param buf The ASS "Dialogue" Text field to split. + * @param outbuffer The output buffer. + * @param keep_flags Flags for filtering ass codes. + * @return >= 0 on success otherwise an error code <0 + */ +int avpriv_ass_filter_override_codes(const ASSCodesCallbacks *callbacks, + void *priv, const char *buf, + AVBPrint *outbuffer, enum ASSSplitComponents keep_flags); + +/** + * Split override codes out of a ASS "Dialogue" Text field. + * + * @param callbacks Set of callback functions called for each override code + * encountered. + * @param priv Opaque pointer passed to the callback functions. + * @param buf The ASS "Dialogue" Text field to split. + * @return >= 0 on success otherwise an error code <0 + */ +int avpriv_ass_split_override_codes(const ASSCodesCallbacks *callbacks, + void *priv, const char *buf); + +/** + * Find an ASSStyle structure by its name. + * + * @param ctx Context previously initialized by ff_ass_split(). + * @param style name of the style to search for. + * @return the ASSStyle corresponding to style, or NULL if style can't be found + */ +ASSStyle *avpriv_ass_style_get(ASSSplitContext *ctx, const char *style); + +#endif /* AVUTIL_ASS_SPLIT_INTERNAL_H */ From patchwork Tue Oct 25 09:13:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 38984 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2685090pzb; Tue, 25 Oct 2022 02:15:17 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6ua84MKVtQSc7RrT8JumB3B0GfQVg+MmKMXyevqFmYh1zttbu9/oieB2Zqi1C047F5CYGN X-Received: by 2002:a05:6402:114a:b0:454:85e4:2295 with SMTP id g10-20020a056402114a00b0045485e42295mr35691164edw.348.1666689316836; Tue, 25 Oct 2022 02:15:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666689316; cv=none; d=google.com; s=arc-20160816; b=SDFUMZQjomSI872gxbWm/oLxL9xFOw59+iOSgk4KDybzAG2Rx0yZQNVOBlEDXGQ19d qEErb42H/RPoEC70PwYwu2CTf//K0CfIq7jEnmoSWVCYsG77R2F1gCphIaNuk8W8YBw6 hahKJoo8c0AWmSpVwtpV1577xN/p9HbQZBE35s7O/cncABiR7R0UOSSZJNHJqVhYZcoQ WydDrUqvxvORQfsr38wrj5HdomzUBnBGntpmCebj5Qg4Qttxcg7JolKeqAVA2raJNYzg DFpB9KWNq0thZhgp8Pa6iITSaftKtNPwtSTajEHM7FWXXYV/J1ONz3pYDPWEFnguYaJQ cBJQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:references :in-reply-to:message-id:from:dkim-signature:delivered-to; bh=9dcD3VyQ4HcTRMQGjLR/62td+tPobtWv5cfWDpZ9kso=; b=AQnXcuiws4wdJqOjg2jGLGkPwio7n2OqjU2p70WITp01VISiIjcltYB+sh5c8tn4E4 s3eW/ZKTZk1ueX2S1Ovb9x5P2uqErbQwJzqUvOZQ/5leT110a4gbPaQVu6LByyenRc7M wmzP09tAKzEgapIVtJjSsaMN7ZF5W+0AVpEfKDJ/TnCaGr0tewsVo18hCHHD5yNarLr4 ufvzm8exCAZllOHrT2LpwWtfx8UOo7q9YT8dokTcowiMyjmjETIr/WpevZ0aZ/gQRakM 8qt4AlNG4V5rYviKbbhNlN/DjHcL5Z2ALOhdAcf2haxdKX7kkkof61lnbWeZQvl2UjRz y5DA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=I+bymQSj; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id r18-20020a1709060d5200b0078232bfe3a0si1870157ejh.331.2022.10.25.02.15.16; Tue, 25 Oct 2022 02:15:16 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=I+bymQSj; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0F9E768BC89; Tue, 25 Oct 2022 12:14:08 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f42.google.com (mail-pj1-f42.google.com [209.85.216.42]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5C60768BC71 for ; Tue, 25 Oct 2022 12:13:58 +0300 (EEST) Received: by mail-pj1-f42.google.com with SMTP id t10-20020a17090a4e4a00b0020af4bcae10so11313026pjl.3 for ; Tue, 25 Oct 2022 02:13:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=mN9hz1wJlmZKp4M4qFPr7BCuPqMfQlsCsWbToZjK9v8=; b=I+bymQSjFXhOgKLA25B4rBpZ22nQL+aUgsL5zGHo9g0vTimHu0GhONVwPG/FMowJAJ 23BfjSOMmO0jCHPEMQYrB2oJyl+Z46qwmvkx5e8lXIauoZZLT/plaQcLKjAG4akLeMHK jkBjzNR3KbUBtqTSRFoDOIkLMDio0fcwFJGvJCKU+RfcnAkySBpFbKi6oD8tWo5x9LX7 PaDHFHlcYUm00mmmHkaBinCTVYfB0rqzU89Quz87kFUwWQgQPbtzOMNZdoqLsOc0tPdF FawogDxwwDauWbwj8cebrBJrfQDSMbjmPg1XS8fAJIxEcgnYXdloy33vG+KroiF4RyfV 5DJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=mN9hz1wJlmZKp4M4qFPr7BCuPqMfQlsCsWbToZjK9v8=; b=Nsmb+GTy6TagG2XJRGQurfJG+Z38sWFEcmFn7mjaqvrTZbd3ZF72rbV+7hJyxfwnD8 j22SnEEzg5kqwCxLZslhXwMcBKJDT5FYb8WOUu1iOfNE9/Zx1+5o37quFFDfLjRZSCf8 gtpXbX8mZDI3r/rPkemYT3ZCjShBNgJ/eLmU8u2RZRaXZH5bgcE/V4T5IBQVZ3yzss7h qFolyskb8HY4NS4PANN1RXVJYVLFxsbHY49WuhYSha7kJ6UtGe8QjColGvOKeQaLyAxw 5eDW+CWkp/UwUVmFu7GaBevsuP0m9w0jz0PRPRBpl51BysiScwW6CZ/ASQwdAg4HirzY czkQ== X-Gm-Message-State: ACrzQf36fbwn5AR2Qj4rs1ZA15uzxmcTwgQ+OPCVtzTOo7ErCR/Se9ee Jp9alBWEANW65qhYrnOv95POLOMjLnM= X-Received: by 2002:a17:90a:1508:b0:212:e56b:2b18 with SMTP id l8-20020a17090a150800b00212e56b2b18mr18187963pja.35.1666689237838; Tue, 25 Oct 2022 02:13:57 -0700 (PDT) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id t2-20020a170902e84200b00176dc67df44sm885133plg.132.2022.10.25.02.13.57 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Oct 2022 02:13:57 -0700 (PDT) From: softworkz X-Google-Original-From: softworkz Message-Id: <4acdb3ace6d380e16a47606bd8f828ac60981309.1666689226.git.ffmpegagent@gmail.com> In-Reply-To: References: Date: Tue, 25 Oct 2022 09:13:28 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v9 07/25] avcodec/subtitles: Replace deprecated enum values X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: softworkz Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: tBqkt0BLKqDC From: softworkz Signed-off-by: softworkz --- libavcodec/ass.h | 2 +- libavcodec/assdec.c | 2 +- libavcodec/dvbsubdec.c | 2 +- libavcodec/dvdsubdec.c | 2 +- libavcodec/dvdsubenc.c | 2 +- libavcodec/pgssubdec.c | 2 +- libavcodec/xsubdec.c | 2 +- 7 files changed, 7 insertions(+), 7 deletions(-) diff --git a/libavcodec/ass.h b/libavcodec/ass.h index 8bc13d7ab8..43c5ad651a 100644 --- a/libavcodec/ass.h +++ b/libavcodec/ass.h @@ -83,7 +83,7 @@ static inline int avpriv_ass_add_rect(AVSubtitle *sub, const char *dialog, rects[sub->num_rects] = av_mallocz(sizeof(*rects[0])); if (!rects[sub->num_rects]) return AVERROR(ENOMEM); - rects[sub->num_rects]->type = SUBTITLE_ASS; + rects[sub->num_rects]->type = AV_SUBTITLE_FMT_ASS; ass_str = avpriv_ass_get_dialog(readorder, layer, style, speaker, dialog); if (!ass_str) return AVERROR(ENOMEM); diff --git a/libavcodec/assdec.c b/libavcodec/assdec.c index 2571118ee9..a274d7e92e 100644 --- a/libavcodec/assdec.c +++ b/libavcodec/assdec.c @@ -53,7 +53,7 @@ static int ass_decode_frame(AVCodecContext *avctx, AVSubtitle *sub, if (!sub->rects[0]) return AVERROR(ENOMEM); sub->num_rects = 1; - sub->rects[0]->type = SUBTITLE_ASS; + sub->rects[0]->type = AV_SUBTITLE_FMT_ASS; sub->rects[0]->ass = av_strdup(avpkt->data); if (!sub->rects[0]->ass) return AVERROR(ENOMEM); diff --git a/libavcodec/dvbsubdec.c b/libavcodec/dvbsubdec.c index bcc607d1d7..c864ba3fb6 100644 --- a/libavcodec/dvbsubdec.c +++ b/libavcodec/dvbsubdec.c @@ -796,7 +796,7 @@ static int save_subtitle_set(AVCodecContext *avctx, AVSubtitle *sub, int *got_ou rect->w = region->width; rect->h = region->height; rect->nb_colors = (1 << region->depth); - rect->type = SUBTITLE_BITMAP; + rect->type = AV_SUBTITLE_FMT_BITMAP; rect->linesize[0] = region->width; clut = get_clut(ctx, region->clut); diff --git a/libavcodec/dvdsubdec.c b/libavcodec/dvdsubdec.c index a5da0d7b08..bf97b916c9 100644 --- a/libavcodec/dvdsubdec.c +++ b/libavcodec/dvdsubdec.c @@ -405,7 +405,7 @@ static int decode_dvd_subtitles(DVDSubContext *ctx, AVSubtitle *sub_header, sub_header->rects[0]->y = y1; sub_header->rects[0]->w = w; sub_header->rects[0]->h = h; - sub_header->rects[0]->type = SUBTITLE_BITMAP; + sub_header->rects[0]->type = AV_SUBTITLE_FMT_BITMAP; sub_header->rects[0]->linesize[0] = w; sub_header->rects[0]->flags = is_menu ? AV_SUBTITLE_FLAG_FORCED : 0; } diff --git a/libavcodec/dvdsubenc.c b/libavcodec/dvdsubenc.c index 0874aaa02d..624e21c000 100644 --- a/libavcodec/dvdsubenc.c +++ b/libavcodec/dvdsubenc.c @@ -269,7 +269,7 @@ static int encode_dvd_subtitles(AVCodecContext *avctx, if (rects == 0 || !h->rects) return AVERROR(EINVAL); for (i = 0; i < rects; i++) - if (h->rects[i]->type != SUBTITLE_BITMAP) { + if (h->rects[i]->type != AV_SUBTITLE_FMT_BITMAP) { av_log(avctx, AV_LOG_ERROR, "Bitmap subtitle required\n"); return AVERROR(EINVAL); } diff --git a/libavcodec/pgssubdec.c b/libavcodec/pgssubdec.c index 5f76f12615..65ba3ad261 100644 --- a/libavcodec/pgssubdec.c +++ b/libavcodec/pgssubdec.c @@ -534,7 +534,7 @@ static int display_end_segment(AVCodecContext *avctx, AVSubtitle *sub, if (!rect) return AVERROR(ENOMEM); sub->rects[sub->num_rects++] = rect; - rect->type = SUBTITLE_BITMAP; + rect->type = AV_SUBTITLE_FMT_BITMAP; /* Process bitmap */ object = find_object(ctx->presentation.objects[i].id, &ctx->objects); diff --git a/libavcodec/xsubdec.c b/libavcodec/xsubdec.c index f86b7c58e7..1b167e1d0a 100644 --- a/libavcodec/xsubdec.c +++ b/libavcodec/xsubdec.c @@ -107,7 +107,7 @@ static int decode_frame(AVCodecContext *avctx, AVSubtitle *sub, sub->num_rects = 1; rect->x = x; rect->y = y; rect->w = w; rect->h = h; - rect->type = SUBTITLE_BITMAP; + rect->type = AV_SUBTITLE_FMT_BITMAP; rect->linesize[0] = w; rect->data[0] = av_malloc(w * h); rect->nb_colors = 4; From patchwork Tue Oct 25 09:13:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 38992 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2685967pzb; Tue, 25 Oct 2022 02:16:48 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5twBZA3U4YVHIpegTFF6L3VSGtogHilXRI9Gr9NdPB0+nQgYE06okQuEV4/wPs+j0mFNjF X-Received: by 2002:a17:906:cc52:b0:78d:d477:5b7f with SMTP id mm18-20020a170906cc5200b0078dd4775b7fmr31221819ejb.558.1666689407865; Tue, 25 Oct 2022 02:16:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666689407; cv=none; d=google.com; s=arc-20160816; b=sg971FjinkBNGWYLPl1JDr7OHEqBucUbMdqiBS3QUfNceGOUaCFKtZ+sBpVnpv7M7H mGmxxBuaJc1hz1z04O/uy8PSAPJy9gKedbNLyOLBB1hMM2cgNnsDTE9PPm4KKiWasCSz qHyApBWaD581/YdYqbox8uLhFvIZvnbQbmTKhZEfBz1gHgujNU9AzLprTwDmG1SSh2a7 vCH8HpfVQXgTRZTDE//h2tdk+IRR1VNdaOBDm5nN6boA9DPWzvEBwu1FgUHRgJfKnT24 SlYRvJl12cYjD/SsTqmvuG+z6j3uMubt6AqYPQcty7OeSUFH++6U55vcZZ8TFhPy8+3u d+ag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:references :in-reply-to:message-id:from:dkim-signature:delivered-to; bh=TkARt0Uk7Zhu5Icc8kFdLvOO67eu11F0dZdMsfEM7aQ=; b=RHGhU+8SgEVqjwtRdh+rBSWgmAM65PGZVRrqV82z0tQKrmIh99cvSoxT0H8ugt7Mqv 23mRYfAQxVtJaWDCXvlpy0lyoWJH66ekx/PG3dc3aZn2uXLET1YQOCCdfbcb9t37Lv/d JAgVobAtK5p3+4gz6ePd8jnz12SGtSypwsmZ3zNUln23DSXzsL9K90uP1Ptvnc94b4WJ 3paSHwZblJdP3ew7z6nkBHUzp/5EgXusU7es5SnAFtfwDyi/BtwyJp1J9EAmQcdKFqcC vfoiMNc62mMHoAUvLc1mYcCS/L/jEbrf55g2Ur3/9/WodadWY9s/Si2yecQkQ0mkWC+F IqvQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=q763ZLAi; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id m3-20020a1709066d0300b007877f3132d9si1992001ejr.438.2022.10.25.02.16.47; Tue, 25 Oct 2022 02:16:47 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=q763ZLAi; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9230868BD6B; Tue, 25 Oct 2022 12:14:16 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-oi1-f176.google.com (mail-oi1-f176.google.com [209.85.167.176]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 8406D68BD39 for ; Tue, 25 Oct 2022 12:14:11 +0300 (EEST) Received: by mail-oi1-f176.google.com with SMTP id i7so130078oif.4 for ; Tue, 25 Oct 2022 02:14:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=MqxJM5GYqKq8zpp0Uzt4LwrVXckDhdLJC9lJBEy/4tM=; b=q763ZLAi+6BT02/NDwD8moz7KnsDByWwKL032KWmdBDBJ1/aKf2lEYMlZD7h7+c+AL Wx7unq6/7jU4nVUoL7NietATK370MQVT+uKUJssgWOfJ0DepTbZGmTR3Bx54A9GeRema Y1eEYEkEv++9MkTSKBSzQGc/1ZD5GccelGktWoI95tf0mr76UG+AWhZoUbLgKYhl1qzI BCwotcK5Byb+85fhsz3nB2R2RfItWBSTKVLAiWKvFjie+CrQg7ox3OkQtWahxSxtzqV1 VQcROJ2l+i5owQY7PAXecBqeRdfjmeVTS3ODsf+MgyMjT6/tfytfdDBdFtOqNMcqObZ3 nhOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=MqxJM5GYqKq8zpp0Uzt4LwrVXckDhdLJC9lJBEy/4tM=; b=zlPEWw+LL9XG0EgNPbnHvoc9KvUFWOHzM5Ua1VuP8zcfPX1fSsm9Yn4Veo9vkWoQW/ cbXmQ3uCqIsTy+ehlZTbm4Y+snKqhZKrzYhGg+aox+bdSVUUUdIEr2/ZH383BN9snshr hbUlWmGGitLbFOEtqY8sIdAbPnm29Dq3qhhfxbA7tnGuSb8XEr/24uteyRemZrgweeBl LHDaow8M5wuHzx0ZmybHH+apNoXAIT7fx47s3G8Oecxa6yvXRHtM8ivdMTohwQFu3V8P 6aaBVrFLl3//2IfWwmGpX4rm1sfb2NO3w/qGy4kbkwryE/7qBunvCVIV8zCTJ6YF+V4s k4lQ== X-Gm-Message-State: ACrzQf078xAS1ZPwGIWUA+kslh0To2I3bkmajiZOfXMS2WCgjnjUETT0 8gKZpIY/Tttbvsj+7eREn7LNWKep+cM= X-Received: by 2002:a17:90a:8504:b0:212:c1f4:ddf1 with SMTP id l4-20020a17090a850400b00212c1f4ddf1mr28914526pjn.224.1666689239012; Tue, 25 Oct 2022 02:13:59 -0700 (PDT) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id p25-20020a635b19000000b0044a4025cea1sm882088pgb.90.2022.10.25.02.13.58 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Oct 2022 02:13:58 -0700 (PDT) From: softworkz X-Google-Original-From: softworkz Message-Id: In-Reply-To: References: Date: Tue, 25 Oct 2022 09:13:29 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v9 08/25] fftools/play, probe: Adjust for subtitle changes X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: softworkz Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: nFuYD/y4E37o From: softworkz Signed-off-by: softworkz --- fftools/ffplay.c | 102 +++++++++++++++++++++------------------------- fftools/ffprobe.c | 47 +++++++++++++-------- 2 files changed, 77 insertions(+), 72 deletions(-) diff --git a/fftools/ffplay.c b/fftools/ffplay.c index bcc00afe31..4ec186a0b3 100644 --- a/fftools/ffplay.c +++ b/fftools/ffplay.c @@ -153,7 +153,6 @@ typedef struct Clock { /* Common struct for handling all types of decoded data and allocated render buffers. */ typedef struct Frame { AVFrame *frame; - AVSubtitle sub; int serial; double pts; /* presentation timestamp for the frame */ double duration; /* estimated duration of the frame */ @@ -574,7 +573,7 @@ static int decoder_init(Decoder *d, AVCodecContext *avctx, PacketQueue *queue, S return 0; } -static int decoder_decode_frame(Decoder *d, AVFrame *frame, AVSubtitle *sub) { +static int decoder_decode_frame(Decoder *d, AVFrame *frame) { int ret = AVERROR(EAGAIN); for (;;) { @@ -608,6 +607,9 @@ static int decoder_decode_frame(Decoder *d, AVFrame *frame, AVSubtitle *sub) { } } break; + case AVMEDIA_TYPE_SUBTITLE: + ret = avcodec_receive_frame(d->avctx, frame); + break; } if (ret == AVERROR_EOF) { d->finished = d->pkt_serial; @@ -640,25 +642,11 @@ static int decoder_decode_frame(Decoder *d, AVFrame *frame, AVSubtitle *sub) { av_packet_unref(d->pkt); } while (1); - if (d->avctx->codec_type == AVMEDIA_TYPE_SUBTITLE) { - int got_frame = 0; - ret = avcodec_decode_subtitle2(d->avctx, sub, &got_frame, d->pkt); - if (ret < 0) { - ret = AVERROR(EAGAIN); - } else { - if (got_frame && !d->pkt->data) { - d->packet_pending = 1; - } - ret = got_frame ? 0 : (d->pkt->data ? AVERROR(EAGAIN) : AVERROR_EOF); - } - av_packet_unref(d->pkt); + if (avcodec_send_packet(d->avctx, d->pkt) == AVERROR(EAGAIN)) { + av_log(d->avctx, AV_LOG_ERROR, "Receive_frame and send_packet both returned EAGAIN, which is an API violation.\n"); + d->packet_pending = 1; } else { - if (avcodec_send_packet(d->avctx, d->pkt) == AVERROR(EAGAIN)) { - av_log(d->avctx, AV_LOG_ERROR, "Receive_frame and send_packet both returned EAGAIN, which is an API violation.\n"); - d->packet_pending = 1; - } else { - av_packet_unref(d->pkt); - } + av_packet_unref(d->pkt); } } } @@ -671,7 +659,6 @@ static void decoder_destroy(Decoder *d) { static void frame_queue_unref_item(Frame *vp) { av_frame_unref(vp->frame); - avsubtitle_free(&vp->sub); } static int frame_queue_init(FrameQueue *f, PacketQueue *pktq, int max_size, int keep_last) @@ -969,7 +956,7 @@ static void video_image_display(VideoState *is) if (frame_queue_nb_remaining(&is->subpq) > 0) { sp = frame_queue_peek(&is->subpq); - if (vp->pts >= sp->pts + ((float) sp->sub.start_display_time / 1000)) { + if (vp->pts >= sp->pts) { if (!sp->uploaded) { uint8_t* pixels[4]; int pitch[4]; @@ -981,25 +968,27 @@ static void video_image_display(VideoState *is) if (realloc_texture(&is->sub_texture, SDL_PIXELFORMAT_ARGB8888, sp->width, sp->height, SDL_BLENDMODE_BLEND, 1) < 0) return; - for (i = 0; i < sp->sub.num_rects; i++) { - AVSubtitleRect *sub_rect = sp->sub.rects[i]; + for (i = 0; i < sp->frame->num_subtitle_areas; i++) { + AVSubtitleArea *area = sp->frame->subtitle_areas[i]; + SDL_Rect sdl_rect = { .x = area->x, .y = area->y, .w = area->w, .h = area->h }; - sub_rect->x = av_clip(sub_rect->x, 0, sp->width ); - sub_rect->y = av_clip(sub_rect->y, 0, sp->height); - sub_rect->w = av_clip(sub_rect->w, 0, sp->width - sub_rect->x); - sub_rect->h = av_clip(sub_rect->h, 0, sp->height - sub_rect->y); + area->x = av_clip(area->x, 0, sp->width ); + area->y = av_clip(area->y, 0, sp->height); + area->w = av_clip(area->w, 0, sp->width - area->x); + area->h = av_clip(area->h, 0, sp->height - area->y); is->sub_convert_ctx = sws_getCachedContext(is->sub_convert_ctx, - sub_rect->w, sub_rect->h, AV_PIX_FMT_PAL8, - sub_rect->w, sub_rect->h, AV_PIX_FMT_BGRA, + area->w, area->h, AV_PIX_FMT_PAL8, + area->w, area->h, AV_PIX_FMT_BGRA, 0, NULL, NULL, NULL); if (!is->sub_convert_ctx) { av_log(NULL, AV_LOG_FATAL, "Cannot initialize the conversion context\n"); return; } - if (!SDL_LockTexture(is->sub_texture, (SDL_Rect *)sub_rect, (void **)pixels, pitch)) { - sws_scale(is->sub_convert_ctx, (const uint8_t * const *)sub_rect->data, sub_rect->linesize, - 0, sub_rect->h, pixels, pitch); + if (!SDL_LockTexture(is->sub_texture, &sdl_rect, (void **)pixels, pitch)) { + const uint8_t* data[2] = { area->buf[0]->data, (uint8_t *)&area->pal }; + sws_scale(is->sub_convert_ctx, data, area->linesize, + 0, area->h, pixels, pitch); SDL_UnlockTexture(is->sub_texture); } } @@ -1028,16 +1017,18 @@ static void video_image_display(VideoState *is) #if USE_ONEPASS_SUBTITLE_RENDER SDL_RenderCopy(renderer, is->sub_texture, NULL, &rect); #else - int i; + unsigned i; double xratio = (double)rect.w / (double)sp->width; double yratio = (double)rect.h / (double)sp->height; - for (i = 0; i < sp->sub.num_rects; i++) { - SDL_Rect *sub_rect = (SDL_Rect*)sp->sub.rects[i]; - SDL_Rect target = {.x = rect.x + sub_rect->x * xratio, - .y = rect.y + sub_rect->y * yratio, - .w = sub_rect->w * xratio, - .h = sub_rect->h * yratio}; - SDL_RenderCopy(renderer, is->sub_texture, sub_rect, &target); + for (i = 0; i < sp->frame->num_subtitle_areas; i++) { + AVSubtitleArea *area = sp->frame->subtitle_areas[i]; + SDL_Rect sub_rect = { .x = area->x, .y = area->y, + .w = area->w, .h = area->h}; + SDL_Rect target = {.x = rect.x + sub_rect.x * xratio, + .y = rect.y + sub_rect.y * yratio, + .w = sub_rect.w * xratio, + .h = sub_rect.h * yratio}; + SDL_RenderCopy(renderer, is->sub_texture, &sub_rect, &target); } #endif } @@ -1641,19 +1632,20 @@ retry: sp2 = NULL; if (sp->serial != is->subtitleq.serial - || (is->vidclk.pts > (sp->pts + ((float) sp->sub.end_display_time / 1000))) - || (sp2 && is->vidclk.pts > (sp2->pts + ((float) sp2->sub.start_display_time / 1000)))) + || (is->vidclk.pts > (sp->pts + ((float) sp->frame->subtitle_timing.duration / AV_TIME_BASE))) + || (sp2 && is->vidclk.pts > (sp2->pts))) { if (sp->uploaded) { int i; - for (i = 0; i < sp->sub.num_rects; i++) { - AVSubtitleRect *sub_rect = sp->sub.rects[i]; + for (i = 0; i < sp->frame->num_subtitle_areas; i++) { + AVSubtitleArea *area = sp->frame->subtitle_areas[i]; + SDL_Rect sdl_rect = { .x = area->x, .y = area->y, .w = area->w, .h = area->h }; uint8_t *pixels; int pitch, j; - if (!SDL_LockTexture(is->sub_texture, (SDL_Rect *)sub_rect, (void **)&pixels, &pitch)) { - for (j = 0; j < sub_rect->h; j++, pixels += pitch) - memset(pixels, 0, sub_rect->w << 2); + if (!SDL_LockTexture(is->sub_texture, &sdl_rect, (void **)&pixels, &pitch)) { + for (j = 0; j < area->h; j++, pixels += pitch) + memset(pixels, 0, area->w << 2); SDL_UnlockTexture(is->sub_texture); } } @@ -1764,7 +1756,7 @@ static int get_video_frame(VideoState *is, AVFrame *frame) { int got_picture; - if ((got_picture = decoder_decode_frame(&is->viddec, frame, NULL)) < 0) + if ((got_picture = decoder_decode_frame(&is->viddec, frame)) < 0) return -1; if (got_picture) { @@ -2040,7 +2032,7 @@ static int audio_thread(void *arg) return AVERROR(ENOMEM); do { - if ((got_frame = decoder_decode_frame(&is->auddec, frame, NULL)) < 0) + if ((got_frame = decoder_decode_frame(&is->auddec, frame)) < 0) goto the_end; if (got_frame) { @@ -2237,14 +2229,14 @@ static int subtitle_thread(void *arg) if (!(sp = frame_queue_peek_writable(&is->subpq))) return 0; - if ((got_subtitle = decoder_decode_frame(&is->subdec, NULL, &sp->sub)) < 0) + if ((got_subtitle = decoder_decode_frame(&is->subdec, sp->frame)) < 0) break; pts = 0; - if (got_subtitle && sp->sub.format == 0) { - if (sp->sub.pts != AV_NOPTS_VALUE) - pts = sp->sub.pts / (double)AV_TIME_BASE; + if (got_subtitle && sp->frame->format == AV_SUBTITLE_FMT_BITMAP) { + if (sp->frame->subtitle_timing.start_pts != AV_NOPTS_VALUE) + pts = (double)sp->frame->subtitle_timing.start_pts / (double)AV_TIME_BASE; sp->pts = pts; sp->serial = is->subdec.pkt_serial; sp->width = is->subdec.avctx->width; @@ -2254,7 +2246,7 @@ static int subtitle_thread(void *arg) /* now we can update the picture count */ frame_queue_push(&is->subpq); } else if (got_subtitle) { - avsubtitle_free(&sp->sub); + av_frame_free(&sp->frame); } } return 0; diff --git a/fftools/ffprobe.c b/fftools/ffprobe.c index 9b7e82fd8c..460b979400 100644 --- a/fftools/ffprobe.c +++ b/fftools/ffprobe.c @@ -2528,22 +2528,42 @@ static void show_packet(WriterContext *w, InputFile *ifile, AVPacket *pkt, int p fflush(stdout); } -static void show_subtitle(WriterContext *w, AVSubtitle *sub, AVStream *stream, +static void show_subtitle(WriterContext *w, AVFrame *sub, AVStream *stream, AVFormatContext *fmt_ctx) { AVBPrint pbuf; + const char *s; av_bprint_init(&pbuf, 1, AV_BPRINT_SIZE_UNLIMITED); writer_print_section_header(w, SECTION_ID_SUBTITLE); print_str ("media_type", "subtitle"); - print_ts ("pts", sub->pts); - print_time("pts_time", sub->pts, &AV_TIME_BASE_Q); - print_int ("format", sub->format); - print_int ("start_display_time", sub->start_display_time); - print_int ("end_display_time", sub->end_display_time); - print_int ("num_rects", sub->num_rects); + print_ts ("pts", sub->subtitle_timing.start_pts); + print_time("pts_time", sub->subtitle_timing.start_pts, &AV_TIME_BASE_Q); + print_time("duration", sub->subtitle_timing.duration, &AV_TIME_BASE_Q); + + // Remain compatible with previous outputs + switch (sub->format) { + case AV_SUBTITLE_FMT_BITMAP: + print_int ("format", 0); + break; + case AV_SUBTITLE_FMT_TEXT: + print_int ("format", 1); + break; + case AV_SUBTITLE_FMT_ASS: + print_int ("format", 1); + break; + default: + print_int ("format", -1); + break; + } + + s = av_get_subtitle_fmt_name(sub->format); + if (s) print_str ("format_str", s); + else print_str_opt("format_str", "unknown"); + + print_int ("num_subtitle_rects", sub->num_subtitle_areas); writer_print_section_footer(w); @@ -2722,7 +2742,6 @@ static av_always_inline int process_frame(WriterContext *w, AVFormatContext *fmt_ctx = ifile->fmt_ctx; AVCodecContext *dec_ctx = ifile->streams[pkt->stream_index].dec_ctx; AVCodecParameters *par = ifile->streams[pkt->stream_index].st->codecpar; - AVSubtitle sub; int ret = 0, got_frame = 0; clear_log(1); @@ -2730,6 +2749,7 @@ static av_always_inline int process_frame(WriterContext *w, switch (par->codec_type) { case AVMEDIA_TYPE_VIDEO: case AVMEDIA_TYPE_AUDIO: + case AVMEDIA_TYPE_SUBTITLE: if (*packet_new) { ret = avcodec_send_packet(dec_ctx, pkt); if (ret == AVERROR(EAGAIN)) { @@ -2748,12 +2768,6 @@ static av_always_inline int process_frame(WriterContext *w, } } break; - - case AVMEDIA_TYPE_SUBTITLE: - if (*packet_new) - ret = avcodec_decode_subtitle2(dec_ctx, &sub, &got_frame, pkt); - *packet_new = 0; - break; default: *packet_new = 0; } @@ -2768,12 +2782,11 @@ static av_always_inline int process_frame(WriterContext *w, nb_streams_frames[pkt->stream_index]++; if (do_show_frames) if (is_sub) - show_subtitle(w, &sub, ifile->streams[pkt->stream_index].st, fmt_ctx); + show_subtitle(w, frame, ifile->streams[pkt->stream_index].st, fmt_ctx); else show_frame(w, frame, ifile->streams[pkt->stream_index].st, fmt_ctx); - if (is_sub) - avsubtitle_free(&sub); } + return got_frame || *packet_new; } From patchwork Tue Oct 25 09:13:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 38982 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2684518pzb; Tue, 25 Oct 2022 02:14:59 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6cCmKJtbGiBzSo7POyY5Gjl9uaTYNAmUCCRdeOqPq6fhDyngL3Lm/eycwaUxNdW2F1OVuP X-Received: by 2002:a05:6402:5207:b0:462:2426:48fc with SMTP id s7-20020a056402520700b00462242648fcmr1402813edd.93.1666689298919; Tue, 25 Oct 2022 02:14:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666689298; cv=none; d=google.com; s=arc-20160816; b=H8Q2rcPaElu0pOl1Ig/6hCuFy8HN9gIjpCVOhdSNgyGEzkHxMUzk2tjX2k1zRAVah/ BxAj0F/ZkDHcF0YJz3ceIsFry/jjEnlVhTZwcgYRi6xxFd+J8Ojj7baxxQn4KoNIHwhb CJK0tqwHvZxQ//zyR55y366WhzMHOebYILrPjBWaqa75Ievac8hBIJ9/BGLG1TecNyVf GX0P1NyN2glGfjItDoAuNjw/wEAr/FAWGerVc+WIt/RIhO32/2FgUHJ4p/Fw2xENvBTm QblC0X+Krpw6kZk/lPmiLXXTO4l300F0pUSLcb6VJuZNmNEEHAMQ+XxDSdspXtJ9bk0x tXRg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:references :in-reply-to:message-id:from:dkim-signature:delivered-to; bh=OJlsCKMQC4eGamzGa2DyUruCavumRYBWd7KYR+WjXK8=; b=Zl+8B/d0OE0B4zur7CSzhtuCtMwTEzVHz7udNA1hzEMgDab9IyVCwez6PLgdamNVEV 2XrZ60SA/Gtp5eCNRrSgF0Wa/BoEVQ9Fhj9gnXbJfx6sVxjYglO5c/4lsO27O/PMDTLy l7Eaz27quZod+88CaCAhAF4lgoFKsvn3wkSvATP6B3/WGgbc6pOwC0/dWCpiaoVvkQ37 JMCTsGSkLesDojZWJTIN1Mslt1j0vIeoj2uOmW86SSITdbgKIqv1Jzmui3LLDwN8mRhC iLBfvBl2fZvG/nwUxIk8WRGwMoWDmg/ZyBDfqpWnU9h1qMrLE0UTeljVjLJixeLS7TQB 0IPQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=VIKWdi5O; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id gb24-20020a170907961800b007aa3373e802si2214669ejc.660.2022.10.25.02.14.58; Tue, 25 Oct 2022 02:14:58 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=VIKWdi5O; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id DF38968BC71; Tue, 25 Oct 2022 12:14:05 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9134C68BC3E for ; Tue, 25 Oct 2022 12:14:01 +0300 (EEST) Received: by mail-pl1-f180.google.com with SMTP id c2so2447227plz.11 for ; Tue, 25 Oct 2022 02:14:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=3RMhou2wJmR42zACoesomNs+7Fxvt4YtKlP7MzDsX0E=; b=VIKWdi5OL4MRo6VLS0oxYJJ3wAycm+tDFD72ui02eGd5gzWF4NO49s7F8SdGYEISdE ckC/UYXJYwm25OUyHB+DhnMF0LREHtE7mwI/gWr7H9LJQyGhwQwYYX12FpYG6vpa09xC Gv5jNGdFkMyDOTZk8hfM+1qQ8DGjkxjpbww7vRjAS4IawcR0BKhdtwxkxQForThAMGza maf2nZp3MqqvK76P9/pvk+YiYWy8bVq2qAZVmKLt7Hx7RH4+63I6ehGyQI+21X5LgGA5 y41P9O6/hg56xzgqdyj29jCKIzeErG3Q/Ilauuqf06d8qCM3a9AeUa9akteHKfSdHP2m UKGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=3RMhou2wJmR42zACoesomNs+7Fxvt4YtKlP7MzDsX0E=; b=Nzg7+OHeC2iPH2I4RTv2uvvxObrXkO56oUPkY6HCJIwVMeAler2HIuWV/j/dPLJ4bf x51+JZDhdnKXLbVs4BrctT2dUbXbITEkrCEBQoJ/Z/SnYpypkkaXZmFn6rASHfgmMafu YC8jGVzUCXGhRRItnZE1Q50FUvejx21EffpRZSx0GJZcaZtV1j4pgUPrzoD3HKy0RDAw Zqolsg9Ugl3F7jI3jPipmZiZLlZKzXVR0WRCbX2Nr/JwgfOtEgU/UVzS+NWWRfIIctco KVwMQVDoXmwYtgCPFpL+sjgXAVBeQWpzEdVt8D/UiKx1gI54zT4pDJ019gvFuhAPqnCZ Kd7Q== X-Gm-Message-State: ACrzQf17NfQKpZMiKY3LjlIWVn8Nf7z006Bx+wTY9qqrdYrcID87r32n t9NObO1tkCXUAIdRndqGT6/QVrKQVmo= X-Received: by 2002:a17:902:7b8f:b0:186:a1c4:e648 with SMTP id w15-20020a1709027b8f00b00186a1c4e648mr10521976pll.37.1666689239963; Tue, 25 Oct 2022 02:13:59 -0700 (PDT) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id f1-20020a170902684100b00172fad607b3sm871077pln.207.2022.10.25.02.13.59 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Oct 2022 02:13:59 -0700 (PDT) From: softworkz X-Google-Original-From: softworkz Message-Id: <2ea0e7130348dae1e9982bd9e8737d79a1a708b0.1666689226.git.ffmpegagent@gmail.com> In-Reply-To: References: Date: Tue, 25 Oct 2022 09:13:30 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v9 09/25] avfilter/subtitles: Add subtitles.c for subtitle frame allocation X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: softworkz Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: YvCLxQzPFcGH From: softworkz Analog to avfilter/video.c and avfilter/audio.c Signed-off-by: softworkz --- libavfilter/Makefile | 1 + libavfilter/avfilter.c | 4 +++ libavfilter/internal.h | 1 + libavfilter/subtitles.c | 63 +++++++++++++++++++++++++++++++++++++++++ libavfilter/subtitles.h | 44 ++++++++++++++++++++++++++++ 5 files changed, 113 insertions(+) create mode 100644 libavfilter/subtitles.c create mode 100644 libavfilter/subtitles.h diff --git a/libavfilter/Makefile b/libavfilter/Makefile index ff2a06c262..6db90028e0 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -21,6 +21,7 @@ OBJS = allfilters.o \ framequeue.o \ graphdump.o \ graphparser.o \ + subtitles.o \ version.o \ video.o \ diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c index cc5505e65b..e49d76c14a 100644 --- a/libavfilter/avfilter.c +++ b/libavfilter/avfilter.c @@ -43,6 +43,7 @@ #include "formats.h" #include "framepool.h" #include "internal.h" +#include "subtitles.h" static void tlog_ref(void *ctx, AVFrame *ref, int end) { @@ -1462,6 +1463,9 @@ int ff_inlink_make_frame_writable(AVFilterLink *link, AVFrame **rframe) case AVMEDIA_TYPE_AUDIO: out = ff_get_audio_buffer(link, frame->nb_samples); break; + case AVMEDIA_TYPE_SUBTITLE: + out = ff_get_subtitles_buffer(link, link->format); + break; default: return AVERROR(EINVAL); } diff --git a/libavfilter/internal.h b/libavfilter/internal.h index aaf2c6c584..1a0752e4ee 100644 --- a/libavfilter/internal.h +++ b/libavfilter/internal.h @@ -89,6 +89,7 @@ struct AVFilterPad { union { AVFrame *(*video)(AVFilterLink *link, int w, int h); AVFrame *(*audio)(AVFilterLink *link, int nb_samples); + AVFrame *(*subtitle)(AVFilterLink *link, int format); } get_buffer; /** diff --git a/libavfilter/subtitles.c b/libavfilter/subtitles.c new file mode 100644 index 0000000000..951bfd612c --- /dev/null +++ b/libavfilter/subtitles.c @@ -0,0 +1,63 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/common.h" + +#include "subtitles.h" +#include "avfilter.h" +#include "internal.h" + + +AVFrame *ff_null_get_subtitles_buffer(AVFilterLink *link, int format) +{ + return ff_get_subtitles_buffer(link->dst->outputs[0], format); +} + +AVFrame *ff_default_get_subtitles_buffer(AVFilterLink *link, int format) +{ + AVFrame *frame; + + frame = av_frame_alloc(); + if (!frame) + return NULL; + + frame->format = format; + frame->type = AVMEDIA_TYPE_SUBTITLE; + + if (av_frame_get_buffer2(frame, 0) < 0) { + av_frame_free(&frame); + return NULL; + } + + return frame; +} + +AVFrame *ff_get_subtitles_buffer(AVFilterLink *link, int format) +{ + AVFrame *ret = NULL; + + if (link->dstpad->get_buffer.subtitle) + ret = link->dstpad->get_buffer.subtitle(link, format); + + if (!ret) + ret = ff_default_get_subtitles_buffer(link, format); + + return ret; +} diff --git a/libavfilter/subtitles.h b/libavfilter/subtitles.h new file mode 100644 index 0000000000..4a9115126e --- /dev/null +++ b/libavfilter/subtitles.h @@ -0,0 +1,44 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVFILTER_SUBTITLES_H +#define AVFILTER_SUBTITLES_H + +#include "avfilter.h" +#include "internal.h" + +/** default handler for get_subtitles_buffer() for subtitle inputs */ +AVFrame *ff_default_get_subtitles_buffer(AVFilterLink *link, int format); + +/** get_subtitles_buffer() handler for filters which simply pass subtitles along */ +AVFrame *ff_null_get_subtitles_buffer(AVFilterLink *link, int format); + +/** + * Request a subtitles frame with a specific set of permissions. + * + * @param link the output link to the filter from which the buffer will + * be requested + * @param format The subtitles format. + * @return A reference to the frame. This must be unreferenced with + * av_frame_free when you are finished with it. +*/ +AVFrame *ff_get_subtitles_buffer(AVFilterLink *link, int format); + +#endif /* AVFILTER_SUBTITLES_H */ From patchwork Tue Oct 25 09:13:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 38983 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2684963pzb; Tue, 25 Oct 2022 02:15:08 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4+hmCAA69LzdlHEuPmqzKvXVA6E12ibfwJ9rEjOLd4a5NFMo/0dtuebz6Vx9Gw+NV7L29k X-Received: by 2002:a17:907:97c7:b0:7a5:ad82:f31c with SMTP id js7-20020a17090797c700b007a5ad82f31cmr10201724ejc.497.1666689308049; Tue, 25 Oct 2022 02:15:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666689308; cv=none; d=google.com; s=arc-20160816; b=GWJQcnTUtj+oW3RrPv12jeuSingWLTr7T5c7Zv1NwqevEzlDi1epxNdOQxM3d4QAMt PXnVxTw67czBDw+IyfFNYWz7bQmRgGoJV5yEdhPbFYhJGl0vbZqtEDshk+stWcqp00oK Rj9DXEiDz0Vkf56A8aTKxc5DyT7cCqh6Q1bkzdbY18sHUu910WNGI2YyZZM1dE10itZW HAicH2Q9XM20Y7LPoTMC2XbRAVAjA4GaRC9HsLE512exDsS4/KsFLND97ldoQMAJzLdu 2atuoW3PCmCrckrslMVXEXI+qpCIZxMAuR0M+ADjYhywiiXfuLvdgesYfPv7Um+Mjisb cWGg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:references :in-reply-to:message-id:from:dkim-signature:delivered-to; bh=a2WaM0fzhmLu0TWHQp5BL9bIMHwq2Vrne/9e4xDNxsw=; b=Tr5DN4wm/H1BdcK16HTbGy/n/uG5BBMIv0VPag7LbVwXbCz+xHOtwBjRV5wy/acKYk lzX7o6TAZiZE1FXpFdKpxfZBslcieU10L0jTDeHcZoNG6RAHlRVt4vS0+tW3SQqhw9IH oYGKn0oLT5bOW9/w+Fz2XsUJkMPtRuOrvpUSSXld4mFBTbzO5p2kZvoJRqvMbAn1xw7D M+tzWBNBnl9NeM1hDnxoxIAMyPU4ZBKNpIiYy7WrAVowU+E/NelAmx4Hy8F6use+IZiJ E/KGSAbmGpnAXdaY2ZKNelz879AkRQvi7m7qvbsQPmiPD74g8lY9kKj7YtXcxuHLX66L xK/Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=g3kQSNvB; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id d17-20020a170906305100b007317ad1f9a4si1923902ejd.310.2022.10.25.02.15.07; Tue, 25 Oct 2022 02:15:08 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=g3kQSNvB; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 06B9E68BD18; Tue, 25 Oct 2022 12:14:07 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 20C6868BC71 for ; Tue, 25 Oct 2022 12:14:02 +0300 (EEST) Received: by mail-pl1-f176.google.com with SMTP id c2so2447263plz.11 for ; Tue, 25 Oct 2022 02:14:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=BaTjIvOE/dpwuL9yKzdaz3SevcpM3pY5YGZC9AhD9KQ=; b=g3kQSNvBwasdarjJ/+S+jyghvxFM1dapccieL9PyQEpC0yBxXmudTWSkKFpo1paBFA sW9eZOatJ4YD269Ez+pEwfkANzMv+Cwx8b0cx72JRlFnZiAafIWWEqS727VjaSF7s5/r xEZzzlL41XvXYlVEfUhhK2lrJE/5ec5YQ46wD1O5YaXlbCf3QDsLKhKtO6Nz0YN8awV6 tx/HQTV0Rwnm5Y+KyJaVSIRBhlj9WF4LSuXq3F6bSrWrmHAvWx4N5lkkI1veueLAmw3t wgt5owkigOX+REE7lMBqMMUvi9ffY0lcdYddRgqcwAb8UYQ/EXcckWwp+rG+TI/Nn+57 hb6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=BaTjIvOE/dpwuL9yKzdaz3SevcpM3pY5YGZC9AhD9KQ=; b=uryrvdNgLiHXn3spoUXX8Be/XQAKYqN07yFiiURsr0UFtlWVKicW0kzmT87mpc4EW/ 4ZMX0MF9KpZWA2L39Jc6JIloTlnekNOeyZThkzNPkztW7ebkFd7IZLuPNcCYbdHnaNkN ERd91LZ5rO7558PhKMLVW2C1S2WkEssjxJl3s9tR1k+bCAu6euXl7QX04OTpXlFNqWDM pgzGGOtaUPEgXLGj4YkGZLRH0FRVICTNmmlpKdtvIGCeyx6gqfEkJcGkzZSv/2n2fdzO S58JaOcG8Ah44qH5CZOHj1MwQsmzQEh7YEo9DA6jTPmaEPc72xowtkdmkj4d9FJxT7Uu ZgkA== X-Gm-Message-State: ACrzQf3kWqnDGB6i3LasSywTcrmENCM0NRIxiZGUDy4e2QQkul36Vmw6 x4yVQ6xveibO2JOX801pE1Jzn5ZIj2o= X-Received: by 2002:a17:90b:1e43:b0:213:1efe:9815 with SMTP id pi3-20020a17090b1e4300b002131efe9815mr8661481pjb.164.1666689241175; Tue, 25 Oct 2022 02:14:01 -0700 (PDT) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id b9-20020a1709027e0900b0017f92d7fe2csm854236plm.288.2022.10.25.02.14.00 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Oct 2022 02:14:00 -0700 (PDT) From: softworkz X-Google-Original-From: softworkz Message-Id: In-Reply-To: References: Date: Tue, 25 Oct 2022 09:13:31 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v9 10/25] avfilter/avfilter: Handle subtitle frames X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: softworkz Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Mf8XrM8UJLgg From: softworkz Signed-off-by: softworkz --- libavfilter/avfilter.c | 20 +++++++++++++++++--- libavfilter/avfilter.h | 11 +++++++++++ libavfilter/avfiltergraph.c | 5 +++++ libavfilter/formats.c | 16 ++++++++++++++++ libavfilter/formats.h | 3 +++ libavfilter/internal.h | 18 +++++++++++++++--- 6 files changed, 67 insertions(+), 6 deletions(-) diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c index e49d76c14a..b40a2fc7cd 100644 --- a/libavfilter/avfilter.c +++ b/libavfilter/avfilter.c @@ -54,7 +54,8 @@ static void tlog_ref(void *ctx, AVFrame *ref, int end) ref->linesize[0], ref->linesize[1], ref->linesize[2], ref->linesize[3], ref->pts, ref->pkt_pos); - if (ref->width) { + switch(ref->type) { + case AVMEDIA_TYPE_VIDEO: ff_tlog(ctx, " a:%d/%d s:%dx%d i:%c iskey:%d type:%c", ref->sample_aspect_ratio.num, ref->sample_aspect_ratio.den, ref->width, ref->height, @@ -62,8 +63,8 @@ static void tlog_ref(void *ctx, AVFrame *ref, int end) ref->top_field_first ? 'T' : 'B', /* Top / Bottom */ ref->key_frame, av_get_picture_type_char(ref->pict_type)); - } - if (ref->nb_samples) { + break; + case AVMEDIA_TYPE_AUDIO: AVBPrint bprint; av_bprint_init(&bprint, 1, AV_BPRINT_SIZE_UNLIMITED); @@ -73,6 +74,7 @@ static void tlog_ref(void *ctx, AVFrame *ref, int end) ref->nb_samples, ref->sample_rate); av_bprint_finalize(&bprint, NULL); + break; } ff_tlog(ctx, "]%s", end ? "\n" : ""); @@ -356,6 +358,14 @@ int avfilter_config_links(AVFilterContext *filter) if (!link->time_base.num && !link->time_base.den) link->time_base = (AVRational) {1, link->sample_rate}; + + break; + + case AVMEDIA_TYPE_SUBTITLE: + if (!link->time_base.num && !link->time_base.den) + link->time_base = inlink ? inlink->time_base : AV_TIME_BASE_Q; + + break; } if (link->src->nb_inputs && link->src->inputs[0]->hw_frames_ctx && @@ -1023,6 +1033,10 @@ int ff_filter_frame(AVFilterLink *link, AVFrame *frame) av_assert1(frame->width == link->w); av_assert1(frame->height == link->h); } + } else if (link->type == AVMEDIA_TYPE_SUBTITLE) { + if (frame->format != link->format) { + av_log(link->dst, AV_LOG_WARNING, "Subtitle format change from %d to %d\n", link->format, frame->format); + } } else { if (frame->format != link->format) { av_log(link->dst, AV_LOG_ERROR, "Format change is not supported\n"); diff --git a/libavfilter/avfilter.h b/libavfilter/avfilter.h index 6d68ebece4..82f6b21520 100644 --- a/libavfilter/avfilter.h +++ b/libavfilter/avfilter.h @@ -45,6 +45,7 @@ #include "libavutil/log.h" #include "libavutil/samplefmt.h" #include "libavutil/pixfmt.h" +#include "libavutil/subfmt.h" #include "libavutil/rational.h" #include "libavfilter/version_major.h" @@ -356,6 +357,12 @@ typedef struct AVFilter { * and outputs use the same sample rate and channel count/layout. */ const enum AVSampleFormat *samples_list; + /** + * Analogous to pixels, but delimited by AV_SUBTITLE_FMT_NONE + * and restricted to filters that only have AVMEDIA_TYPE_SUBTITLE + * inputs and outputs. + */ + const enum AVSubtitleType *subs_list; /** * Equivalent to { pix_fmt, AV_PIX_FMT_NONE } as pixels_list. */ @@ -364,6 +371,10 @@ typedef struct AVFilter { * Equivalent to { sample_fmt, AV_SAMPLE_FMT_NONE } as samples_list. */ enum AVSampleFormat sample_fmt; + /** + * Equivalent to { sub_fmt, AV_SUBTITLE_FMT_NONE } as subs_list. + */ + enum AVSubtitleType sub_fmt; } formats; int priv_size; ///< size of private data to allocate for the filter diff --git a/libavfilter/avfiltergraph.c b/libavfilter/avfiltergraph.c index 53f468494d..f7547467f0 100644 --- a/libavfilter/avfiltergraph.c +++ b/libavfilter/avfiltergraph.c @@ -309,6 +309,8 @@ static int filter_link_check_formats(void *log, AVFilterLink *link, AVFilterForm return ret; break; + case AVMEDIA_TYPE_SUBTITLE: + return 0; default: av_assert0(!"reached"); } @@ -439,6 +441,9 @@ static int query_formats(AVFilterGraph *graph, void *log_ctx) if (!link) continue; + if (link->type == AVMEDIA_TYPE_SUBTITLE) + continue; + neg = ff_filter_get_negotiation(link); av_assert0(neg); for (neg_step = 1; neg_step < neg->nb_mergers; neg_step++) { diff --git a/libavfilter/formats.c b/libavfilter/formats.c index e8c2888c0c..12585ed428 100644 --- a/libavfilter/formats.c +++ b/libavfilter/formats.c @@ -19,6 +19,7 @@ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ +#include "libavutil/subfmt.h" #include "libavutil/avassert.h" #include "libavutil/channel_layout.h" #include "libavutil/common.h" @@ -491,6 +492,13 @@ AVFilterFormats *ff_all_formats(enum AVMediaType type) return NULL; fmt++; } + } else if (type == AVMEDIA_TYPE_SUBTITLE) { + if (ff_add_format(&ret, AV_SUBTITLE_FMT_BITMAP) < 0) + return NULL; + if (ff_add_format(&ret, AV_SUBTITLE_FMT_ASS) < 0) + return NULL; + if (ff_add_format(&ret, AV_SUBTITLE_FMT_TEXT) < 0) + return NULL; } return ret; @@ -774,6 +782,10 @@ int ff_default_query_formats(AVFilterContext *ctx) type = AVMEDIA_TYPE_AUDIO; formats = ff_make_format_list(f->formats.samples_list); break; + case FF_FILTER_FORMATS_SUBFMTS_LIST: + type = AVMEDIA_TYPE_SUBTITLE; + formats = ff_make_format_list(f->formats.subs_list); + break; case FF_FILTER_FORMATS_SINGLE_PIXFMT: type = AVMEDIA_TYPE_VIDEO; formats = ff_make_formats_list_singleton(f->formats.pix_fmt); @@ -782,6 +794,10 @@ int ff_default_query_formats(AVFilterContext *ctx) type = AVMEDIA_TYPE_AUDIO; formats = ff_make_formats_list_singleton(f->formats.sample_fmt); break; + case FF_FILTER_FORMATS_SINGLE_SUBFMT: + type = AVMEDIA_TYPE_SUBTITLE; + formats = ff_make_formats_list_singleton(f->formats.sub_fmt); + break; default: av_assert2(!"Unreachable"); /* Intended fallthrough */ diff --git a/libavfilter/formats.h b/libavfilter/formats.h index 22224dce2d..6cf952a059 100644 --- a/libavfilter/formats.h +++ b/libavfilter/formats.h @@ -183,6 +183,9 @@ av_warn_unused_result int ff_add_channel_layout(AVFilterChannelLayouts **l, const AVChannelLayout *channel_layout); +av_warn_unused_result +int ff_add_subtitle_type(AVFilterFormats **avff, int64_t fmt); + /** * Add *ref as a new reference to f. */ diff --git a/libavfilter/internal.h b/libavfilter/internal.h index 1a0752e4ee..dc56960eef 100644 --- a/libavfilter/internal.h +++ b/libavfilter/internal.h @@ -148,9 +148,11 @@ static av_always_inline int ff_filter_execute(AVFilterContext *ctx, avfilter_act enum FilterFormatsState { /** - * The default value meaning that this filter supports all formats - * and (for audio) sample rates and channel layouts/counts as long - * as these properties agree for all inputs and outputs. + * The default value meaning that this filter supports + * - For video: all formats + * - For audio: all sample rates and channel layouts/counts + * - For subtitles: all subtitle formats + * as long as these properties agree for all inputs and outputs. * This state is only allowed in case all inputs and outputs actually * have the same type. * The union is unused in this state. @@ -161,8 +163,10 @@ enum FilterFormatsState { FF_FILTER_FORMATS_QUERY_FUNC, ///< formats.query active. FF_FILTER_FORMATS_PIXFMT_LIST, ///< formats.pixels_list active. FF_FILTER_FORMATS_SAMPLEFMTS_LIST, ///< formats.samples_list active. + FF_FILTER_FORMATS_SUBFMTS_LIST, ///< formats.subs_list active. FF_FILTER_FORMATS_SINGLE_PIXFMT, ///< formats.pix_fmt active FF_FILTER_FORMATS_SINGLE_SAMPLEFMT, ///< formats.sample_fmt active. + FF_FILTER_FORMATS_SINGLE_SUBFMT, ///< formats.sub_fmt active. }; #define FILTER_QUERY_FUNC(func) \ @@ -174,16 +178,24 @@ enum FilterFormatsState { #define FILTER_SAMPLEFMTS_ARRAY(array) \ .formats.samples_list = array, \ .formats_state = FF_FILTER_FORMATS_SAMPLEFMTS_LIST +#define FILTER_SUBFMTS_ARRAY(array) \ + .formats.subs_list = array, \ + .formats_state = FF_FILTER_FORMATS_SUBFMTS_LIST #define FILTER_PIXFMTS(...) \ FILTER_PIXFMTS_ARRAY(((const enum AVPixelFormat []) { __VA_ARGS__, AV_PIX_FMT_NONE })) #define FILTER_SAMPLEFMTS(...) \ FILTER_SAMPLEFMTS_ARRAY(((const enum AVSampleFormat[]) { __VA_ARGS__, AV_SAMPLE_FMT_NONE })) +#define FILTER_SUBFMTS(...) \ + FILTER_SUBFMTS_ARRAY(((const enum AVSubtitleType[]) { __VA_ARGS__, AV_SUBTITLE_FMT_NONE })) #define FILTER_SINGLE_PIXFMT(pix_fmt_) \ .formats.pix_fmt = pix_fmt_, \ .formats_state = FF_FILTER_FORMATS_SINGLE_PIXFMT #define FILTER_SINGLE_SAMPLEFMT(sample_fmt_) \ .formats.sample_fmt = sample_fmt_, \ .formats_state = FF_FILTER_FORMATS_SINGLE_SAMPLEFMT +#define FILTER_SINGLE_SUBFMT(sub_fmt_) \ + .formats.sub_fmt = sub_fmt_, \ + .formats_state = FF_FILTER_FORMATS_SINGLE_SUBFMT #define FILTER_INOUTPADS(inout, array) \ .inout = array, \ From patchwork Tue Oct 25 09:13:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 38985 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2685191pzb; Tue, 25 Oct 2022 02:15:25 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7bJzgL5HcEgZKpNfHijJe2VhK+sfubo9NNMgyNlUZ5bjEA/GWRL0W269BfWZZIX0g7/yMO X-Received: by 2002:a17:907:9688:b0:7a5:74eb:d12b with SMTP id hd8-20020a170907968800b007a574ebd12bmr10054257ejc.60.1666689325413; Tue, 25 Oct 2022 02:15:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666689325; cv=none; d=google.com; s=arc-20160816; b=ecJtL1rs6A3RWt/oTo46j6M4t1gHw4N9GrMmEqp2h0wFdxXRL3Oc4y1lxZXiOLHHDh EkeQiLYAtjeHbmDdn0rAHbWqs49zqd9OSM6mgv7Hp8BKgl+gfVD8DhRDZlaZYzChz6wo bBb5dJuSxZDUWb3PRTMKc7Qc2xgWF873CThv6bBPCDn4E1JE4HMJ8cRgB1y1nCGT1WPy 8ivl/vLX+DmqQd9aAQwELecOgjsmxExmiqQ55PEnc1Sb7WSFdpVNB5WQZ7pMsx30YI7a S57Sc5ArRg8j6rbkiuRR8lcs/QaVmo9KTyBioIdgeRgYnWF1RmBsOBLZ8mwMi6ADjfGq lGMQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:references :in-reply-to:message-id:from:dkim-signature:delivered-to; bh=7jXbtK8pNlI4iUuNJ4K9ovkspvfEF3MSbxh4YG/3Byo=; b=xplfKXZyMrH+XqPTsNJ+Q3tkN7xWWlZ6c6V8Pv0yo+CcUf8kT2sRZMvnEC4JlS5Tfm oSE18O0QqxbNMeJWUBzVoqAVSJpE1i12XEUDqAxWZNCADmAfALoC/wsD8/tvEx2CG7kc tVyrQES4Mk8XQxyqocnuATQk93MWZQGKPaOEplFFJM4nQYVrTdyNvWMMIMivrCvgZQwY Wwgb78JqsvYyr+ycrHuI6/0KMUGSNDaKpXCZ854BhPPNJ6haNDnQRCaMY3bdWG9UhfOk 8YyOlxkDi9A/aHmdc7Yl1oF4Fy+VL+jYaXiIniAuQWWRj9Jj3r33ZMInJL+cH4RIjKLx MWGg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=cq3CVStu; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id ji20-20020a170907981400b00783698dd8b4si2437267ejc.722.2022.10.25.02.15.24; Tue, 25 Oct 2022 02:15:25 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=cq3CVStu; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0E36B68BA9D; Tue, 25 Oct 2022 12:14:09 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pg1-f174.google.com (mail-pg1-f174.google.com [209.85.215.174]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 36E9B68B85C for ; Tue, 25 Oct 2022 12:14:03 +0300 (EEST) Received: by mail-pg1-f174.google.com with SMTP id 128so10946308pga.1 for ; Tue, 25 Oct 2022 02:14:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=YfKPiEDxU7JWTn0qHJlasal2N9e2SU9ex1lyvaxrkuY=; b=cq3CVStunSJWfZj+4RJbJ5bkB1AQwsCBDvNQPqjuYKRw2qiTph03/2nekgoCyzTJol np/aGbFXi8W7Tte1yRY8op/pbN9Ysx7Htihtyn9VvhjCSp6Gmqz0pLVnjgXX0k8SfJBf WEScSkfj6nX+31CZPvByB9/FhFn8q4AKt06UHrJMFhKX6j6QiYdl906FagzCJLLQURCc qOEWfxvCP1q2vMmEVuCoKQBxHkJqV9eN2jAoRto759nLxMdUiFYHwUt92PGeSsGVrxAP 88glKcR8oduc7IFhBuIYCIG6mc8FeRgXRrcp2cOUxx3UvFqbTzKJMvhhc2O1x69e+cg4 eIXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=YfKPiEDxU7JWTn0qHJlasal2N9e2SU9ex1lyvaxrkuY=; b=FDkU2BGeJXFF/f9zHsC0rq5Kv0rbXpBHuKSRwqcZ0zuxXAqjmKSuUEp4XHHvQeEfoP Y9TaQYgUr+bc/Cy1tAxhgBjVi2POePz/lBfRAPNz0JndRDSFmEe6fz4a831h4z/GWA15 tvK5OxMYQbLpmu2CMiCV5zyiNFpkXmbFaqJgmGxdzwHSwhPC6g6vID6K5K3mbE/DoGTV lk7mmfgWvfYZ7QPkR2Z2BDokfRprH4GlOiK2osH915JTh/rGiCO4j4I9RxrQRXZ/MWh+ w0QX/FmX1MSoQ6CZQO2zgfxk40naprg/PcDxUPsxZHKmmL+mB3xQcu3TVci50K3X/pYP JMwg== X-Gm-Message-State: ACrzQf2rqzuBaIonRThzhk8hbXRFwAxsXy9W+vAnpV+UXn06EwcmOaWu RmWs1cxll04iYGlixbF6pssHUGD4qvQ= X-Received: by 2002:a63:2b41:0:b0:46e:9364:eb07 with SMTP id r62-20020a632b41000000b0046e9364eb07mr23822317pgr.46.1666689242328; Tue, 25 Oct 2022 02:14:02 -0700 (PDT) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id a13-20020a17090ae20d00b0020a821e97fbsm1054038pjz.13.2022.10.25.02.14.01 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Oct 2022 02:14:01 -0700 (PDT) From: softworkz X-Google-Original-From: softworkz Message-Id: <0b31d698845594ba7e660f5e5cd288fff62ca03b.1666689226.git.ffmpegagent@gmail.com> In-Reply-To: References: Date: Tue, 25 Oct 2022 09:13:32 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v9 11/25] avfilter/avfilter: Fix hardcoded input index X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: softworkz Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: DnIkqbz5dWKx From: softworkz This fix targets (rare) cases where multiple input pads have a .filter_frame function. ff_request_frame_to_filter needs to call ff_request_frame with the correct input pad instead of the hardcoded first one. Signed-off-by: softworkz --- libavfilter/avfilter.c | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-) diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c index b40a2fc7cd..6cd83ae34a 100644 --- a/libavfilter/avfilter.c +++ b/libavfilter/avfilter.c @@ -461,7 +461,7 @@ static int64_t guess_status_pts(AVFilterContext *ctx, int status, AVRational lin return AV_NOPTS_VALUE; } -static int ff_request_frame_to_filter(AVFilterLink *link) +static int ff_request_frame_to_filter(AVFilterLink *link, int input_index) { int ret = -1; @@ -470,8 +470,8 @@ static int ff_request_frame_to_filter(AVFilterLink *link) link->frame_blocked_in = 1; if (link->srcpad->request_frame) ret = link->srcpad->request_frame(link); - else if (link->src->inputs[0]) - ret = ff_request_frame(link->src->inputs[0]); + else if (link->src->inputs[input_index]) + ret = ff_request_frame(link->src->inputs[input_index]); if (ret < 0) { if (ret != AVERROR(EAGAIN) && ret != link->status_in) ff_avfilter_link_set_in_status(link, ret, guess_status_pts(link->src, ret, link->time_base)); @@ -1171,6 +1171,14 @@ static int forward_status_change(AVFilterContext *filter, AVFilterLink *in) { unsigned out = 0, progress = 0; int ret; + int input_index = 0; + + for (int i = 0; i < in->dst->nb_inputs; i++) { + if (&in->dst->input_pads[i] == in->dstpad) { + input_index = i; + break; + } + } av_assert0(!in->status_out); if (!filter->nb_outputs) { @@ -1180,7 +1188,7 @@ static int forward_status_change(AVFilterContext *filter, AVFilterLink *in) while (!in->status_out) { if (!filter->outputs[out]->status_in) { progress++; - ret = ff_request_frame_to_filter(filter->outputs[out]); + ret = ff_request_frame_to_filter(filter->outputs[out], input_index); if (ret < 0) return ret; } @@ -1217,7 +1225,7 @@ static int ff_filter_activate_default(AVFilterContext *filter) for (i = 0; i < filter->nb_outputs; i++) { if (filter->outputs[i]->frame_wanted_out && !filter->outputs[i]->frame_blocked_in) { - return ff_request_frame_to_filter(filter->outputs[i]); + return ff_request_frame_to_filter(filter->outputs[i], 0); } } return FFERROR_NOT_READY; From patchwork Tue Oct 25 09:13:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 38975 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2685364pzb; Tue, 25 Oct 2022 02:15:46 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7lNFu6UOqf0k5kwxeb58Nb+gx53pqAWNcGprt0KU8Gf8iyJQwqHpDiCY+WVdUZWP4Z1ufn X-Received: by 2002:a17:906:6791:b0:78d:4051:969f with SMTP id q17-20020a170906679100b0078d4051969fmr31901134ejp.171.1666689346161; Tue, 25 Oct 2022 02:15:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666689346; cv=none; d=google.com; s=arc-20160816; b=C/L1+EJgeFUGy3nw2xw0DaymGL7aoBNsLV5BRo5jvrpeYqyUZs9Y6oXD6ArMjCyHM8 PtAMAho200T0kHBzNX4wbzhgyNG+X6L5oAzKJ/AzRShTnX4/0k+ldNeT5DEB7jOyETLh hufd+1wYhUHmnhfbi4Xn1boW3/iHW8gNviZAxVWHVUrnMsNRGAthaGwceZZDE52Rjnq0 W3RP5Mn0LBzj+TrEvZ0NI+vXMtTObDJ72t8l4Kd9iehDSIY8ckqapiYgLWXoskdkodT1 o3ZWWiRYkd+S2mG2VaCwr43JxhbJelBW/m2xY2lY1nWK60K57jRRheW24Nd9Wu6frkyc eEaQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:references :in-reply-to:message-id:from:dkim-signature:delivered-to; bh=5JxqJaygJDPS4tlv1E08jC10QBA8M+yL0IH/Y6a5MnY=; b=B+sql9KB+lFaoq4MnMk6FDbd2JINltUrwXqmVFyDqMZ3vOtd7dMgeZc3jJxe5jOkwq Eij13bC8ywP+lBbYDv0spwWQ7VvPm8OOW2UUht20OmpEFKLCZhUz2TDF8bLZ0sHUGDt2 tg8AP63+X2lZQlUjB7AGctc9ZRh0cFCH2EJ6efcMbyL7id2zZHKZ2IvyZnS0LzM8/yYh ZSMi/9D15YagM3+9nh7fpKqxQPSA5+OqYJrhoXJwXWvy2TSpkz1jN0QAK7W1sS8+301Z nzMOAHskXLOXYkHygzzUYAb0OtxLlyz8+X3VfQ639Pu1/LzUMomQyLKM73mUsXHsOQzt LfaQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=DWn8W2vJ; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id b9-20020a170906708900b0078d8614e809si1941591ejk.504.2022.10.25.02.15.45; Tue, 25 Oct 2022 02:15:46 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=DWn8W2vJ; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B9BB468BD38; Tue, 25 Oct 2022 12:14:10 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3340468BCCE for ; Tue, 25 Oct 2022 12:14:05 +0300 (EEST) Received: by mail-pj1-f52.google.com with SMTP id q9-20020a17090a178900b00212fe7c6bbeso5344041pja.4 for ; Tue, 25 Oct 2022 02:14:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=8mBq9SZ4u+dJ7tGA2IMtMgEb3DDSKovIkqH4+sl0N8M=; b=DWn8W2vJjsboKH5dlnmbvIOxiA7yPhGDgmy9qHeNBAnp2x6g0ZNqgpT0PED/C2oEyC t5BfQKJf0SeuEwRskS39d3khaRMhS6UANjJE2DlEyDc8hFx7rf5QKo01svRheag9fjSi j94U1O+MDDn+mCKnI//H8qgdGPeWDdthZUQmMZqJ1Qz1b3YiyzB54oAWTdZtDDjaSjNm /cTln9DqqXgS6GuCUFPKMXgUqFbATO4dY7zUROVi9qkX2ndVkI1pl8cF1Dp81yCgg+nP znRPP8SP2eEYfMiPJtVxEFjH7lXV3H6a5rvE2e2UBsXP7eHW8jK3HnHEHSI6Ywqq2HOq xQcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=8mBq9SZ4u+dJ7tGA2IMtMgEb3DDSKovIkqH4+sl0N8M=; b=zTOMS6Vprop2PJNPC+zwXkkUsoQI99vj6716ugdp/zX17arorcQdVzBzpD3ehtsN3x qSUz4heF9LcqHsqUHZ9wG0O+59HyM0Ej8yx67Xx+gZxAjx0MheJWHDW8NTdsDaicELDY DjXAHdoAWLxoAsUujMqihvbySjEPe++BS3c+5Q7yUMWjT6Un1C4yFiLaAvE+N7CMpnw1 CPvjTdfUYomEgXskg2wmdlb9J3CfHC+U083g5FY6gttnFo0fQ5h5xVEa2WpSPuzp0xXp cMAG+1SbP96/0NKJu1l+GN6iMNgodNxBMPHJjT4SJe2ZMABC55ZfRcydDcTq6QqjvbtQ 0iog== X-Gm-Message-State: ACrzQf2N57RqWJ9CfH7r7nq1CRm/Y0cjiQbyYP15N3WgZs2iNrNgGVc3 A2WqSmTDyyxQnzZfXlnvYwOqiJcgqdA= X-Received: by 2002:a17:90b:2306:b0:212:533:3188 with SMTP id mt6-20020a17090b230600b0021205333188mr31320261pjb.210.1666689243511; Tue, 25 Oct 2022 02:14:03 -0700 (PDT) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id pj14-20020a17090b4f4e00b00212cf2fe8c3sm1284090pjb.1.2022.10.25.02.14.02 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Oct 2022 02:14:02 -0700 (PDT) From: softworkz X-Google-Original-From: softworkz Message-Id: <066b754236ca1cad56b9fabff408271a347036e6.1666689226.git.ffmpegagent@gmail.com> In-Reply-To: References: Date: Tue, 25 Oct 2022 09:13:33 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v9 12/25] avfilter/sbuffer: Add sbuffersrc and sbuffersink filters X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: softworkz Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: kAjTCViabRPN From: softworkz Signed-off-by: softworkz --- configure | 2 +- libavfilter/allfilters.c | 2 ++ libavfilter/buffersink.c | 54 ++++++++++++++++++++++++++++++ libavfilter/buffersink.h | 7 ++++ libavfilter/buffersrc.c | 72 ++++++++++++++++++++++++++++++++++++++++ libavfilter/buffersrc.h | 1 + 6 files changed, 137 insertions(+), 1 deletion(-) diff --git a/configure b/configure index 431fa5bf7a..b3f242469b 100755 --- a/configure +++ b/configure @@ -7954,7 +7954,7 @@ print_enabled_components(){ fi done if [ "$name" = "filter_list" ]; then - for c in asrc_abuffer vsrc_buffer asink_abuffer vsink_buffer; do + for c in asrc_abuffer vsrc_buffer ssrc_sbuffer asink_abuffer vsink_buffer ssink_sbuffer; do printf " &ff_%s,\n" $c >> $TMPH done fi diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 119de40b25..173d85b982 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -577,8 +577,10 @@ extern const AVFilter ff_avsrc_movie; * being the same while having different 'types'). */ extern const AVFilter ff_asrc_abuffer; extern const AVFilter ff_vsrc_buffer; +extern const AVFilter ff_ssrc_sbuffer; extern const AVFilter ff_asink_abuffer; extern const AVFilter ff_vsink_buffer; +extern const AVFilter ff_ssink_sbuffer; extern const AVFilter ff_af_afifo; extern const AVFilter ff_vf_fifo; diff --git a/libavfilter/buffersink.c b/libavfilter/buffersink.c index e269cf72d1..204e9bad6c 100644 --- a/libavfilter/buffersink.c +++ b/libavfilter/buffersink.c @@ -30,6 +30,8 @@ #include "libavutil/internal.h" #include "libavutil/opt.h" +#include "libavcodec/avcodec.h" + #define FF_INTERNAL_FIELDS 1 #include "framequeue.h" @@ -61,6 +63,10 @@ typedef struct BufferSinkContext { int *sample_rates; ///< list of accepted sample rates int sample_rates_size; + /* only used for subtitles */ + enum AVSubtitleType *subtitle_types; ///< list of accepted subtitle types, must be terminated with -1 + int subtitle_types_size; + AVFrame *peeked_frame; } BufferSinkContext; @@ -372,6 +378,28 @@ static int asink_query_formats(AVFilterContext *ctx) return 0; } +static int ssink_query_formats(AVFilterContext *ctx) +{ + BufferSinkContext *buf = ctx->priv; + AVFilterFormats *formats = NULL; + unsigned i; + int ret; + + CHECK_LIST_SIZE(subtitle_types) + if (buf->subtitle_types_size) { + for (i = 0; i < NB_ITEMS(buf->subtitle_types); i++) + if ((ret = ff_add_format(&formats, buf->subtitle_types[i])) < 0) + return ret; + if ((ret = ff_set_common_formats(ctx, formats)) < 0) + return ret; + } else { + if ((ret = ff_default_query_formats(ctx)) < 0) + return ret; + } + + return 0; +} + #define OFFSET(x) offsetof(BufferSinkContext, x) #define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM static const AVOption buffersink_options[] = { @@ -395,9 +423,16 @@ static const AVOption abuffersink_options[] = { { NULL }, }; #undef FLAGS +#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_SUBTITLE_PARAM +static const AVOption sbuffersink_options[] = { + { "subtitle_types", "set the supported subtitle formats", OFFSET(subtitle_types), AV_OPT_TYPE_BINARY, .flags = FLAGS }, + { NULL }, +}; +#undef FLAGS AVFILTER_DEFINE_CLASS(buffersink); AVFILTER_DEFINE_CLASS(abuffersink); +AVFILTER_DEFINE_CLASS(sbuffersink); static const AVFilterPad avfilter_vsink_buffer_inputs[] = { { @@ -436,3 +471,22 @@ const AVFilter ff_asink_abuffer = { .outputs = NULL, FILTER_QUERY_FUNC(asink_query_formats), }; + +static const AVFilterPad avfilter_ssink_sbuffer_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + }, +}; + +const AVFilter ff_ssink_sbuffer = { + .name = "sbuffersink", + .description = NULL_IF_CONFIG_SMALL("Buffer subtitle frames, and make them available to the end of the filter graph."), + .priv_class = &sbuffersink_class, + .priv_size = sizeof(BufferSinkContext), + .init = common_init, + .activate = activate, + FILTER_INPUTS(avfilter_ssink_sbuffer_inputs), + .outputs = NULL, + FILTER_QUERY_FUNC(ssink_query_formats), +}; diff --git a/libavfilter/buffersink.h b/libavfilter/buffersink.h index 01e7c747d8..42c164e670 100644 --- a/libavfilter/buffersink.h +++ b/libavfilter/buffersink.h @@ -128,6 +128,13 @@ typedef struct AVABufferSinkParams { */ attribute_deprecated AVABufferSinkParams *av_abuffersink_params_alloc(void); + +/** + * Deprecated and unused struct to use for initializing an sbuffersink context. + */ +typedef struct AVSBufferSinkParams { + const int *subtitle_type; +} AVSBufferSinkParams; #endif /** diff --git a/libavfilter/buffersrc.c b/libavfilter/buffersrc.c index ae8bba19b0..21465fe5e3 100644 --- a/libavfilter/buffersrc.c +++ b/libavfilter/buffersrc.c @@ -39,6 +39,7 @@ #include "formats.h" #include "internal.h" #include "video.h" +#include "libavcodec/avcodec.h" typedef struct BufferSourceContext { const AVClass *class; @@ -63,6 +64,9 @@ typedef struct BufferSourceContext { char *channel_layout_str; AVChannelLayout ch_layout; + /* subtitle only */ + enum AVSubtitleType subtitle_type; + int eof; } BufferSourceContext; @@ -143,6 +147,13 @@ FF_ENABLE_DEPRECATION_WARNINGS return ret; } break; + case AVMEDIA_TYPE_SUBTITLE: + s->subtitle_type = param->format; + if (param->width > 0) + s->w = param->width; + if (param->height > 0) + s->h = param->height; + break; default: return AVERROR_BUG; } @@ -224,6 +235,8 @@ FF_ENABLE_DEPRECATION_WARNINGS CHECK_AUDIO_PARAM_CHANGE(ctx, s, frame->sample_rate, frame->ch_layout, frame->format, frame->pts); break; + case AVMEDIA_TYPE_SUBTITLE: + break; default: return AVERROR(EINVAL); } @@ -303,6 +316,7 @@ unsigned av_buffersrc_get_nb_failed_requests(AVFilterContext *buffer_src) #define OFFSET(x) offsetof(BufferSourceContext, x) #define A AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_AUDIO_PARAM #define V AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM +#define S AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_SUBTITLE_PARAM static const AVOption buffer_options[] = { { "width", NULL, OFFSET(w), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, V }, @@ -332,6 +346,16 @@ static const AVOption abuffer_options[] = { AVFILTER_DEFINE_CLASS(abuffer); +static const AVOption sbuffer_options[] = { + { "time_base", NULL, OFFSET(time_base), AV_OPT_TYPE_RATIONAL, { .dbl = 0 }, 0, INT_MAX, S }, + { "subtitle_type", NULL, OFFSET(subtitle_type), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, S }, + { "width", NULL, OFFSET(w), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, V }, + { "height", NULL, OFFSET(h), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, V }, + { NULL }, +}; + +AVFILTER_DEFINE_CLASS(sbuffer); + static av_cold int init_audio(AVFilterContext *ctx) { BufferSourceContext *s = ctx->priv; @@ -400,6 +424,21 @@ FF_ENABLE_DEPRECATION_WARNINGS return ret; } +static av_cold int init_subtitle(AVFilterContext *ctx) +{ + BufferSourceContext *c = ctx->priv; + + if (c->subtitle_type == AV_SUBTITLE_FMT_BITMAP) + av_log(ctx, AV_LOG_VERBOSE, "graphical subtitles - w:%d h:%d tb:%d/%d\n", + c->w, c->h, c->time_base.num, c->time_base.den); + else + av_log(ctx, AV_LOG_VERBOSE, "text subtitles - w:%d h:%d tb:%d/%d\n", + c->w, c->h, c->time_base.num, c->time_base.den); + + return 0; +} + + static av_cold void uninit(AVFilterContext *ctx) { BufferSourceContext *s = ctx->priv; @@ -433,6 +472,11 @@ static int query_formats(AVFilterContext *ctx) if ((ret = ff_set_common_channel_layouts(ctx, channel_layouts)) < 0) return ret; break; + case AVMEDIA_TYPE_SUBTITLE: + if ((ret = ff_add_format (&formats, c->subtitle_type)) < 0 || + (ret = ff_set_common_formats (ctx , formats )) < 0) + return ret; + break; default: return AVERROR(EINVAL); } @@ -463,6 +507,11 @@ static int config_props(AVFilterLink *link) return ret; } break; + case AVMEDIA_TYPE_SUBTITLE: + link->format = c->subtitle_type; + link->w = c->w; + link->h = c->h; + break; default: return AVERROR(EINVAL); } @@ -527,3 +576,26 @@ const AVFilter ff_asrc_abuffer = { FILTER_QUERY_FUNC(query_formats), .priv_class = &abuffer_class, }; + +static const AVFilterPad avfilter_ssrc_sbuffer_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .request_frame = request_frame, + .config_props = config_props, + }, +}; + +const AVFilter ff_ssrc_sbuffer = { + .name = "sbuffer", + .description = NULL_IF_CONFIG_SMALL("Buffer subtitle frames, and make them accessible to the filterchain."), + .priv_size = sizeof(BufferSourceContext), + + .init = init_subtitle, + .uninit = uninit, + + .inputs = NULL, + FILTER_OUTPUTS(avfilter_ssrc_sbuffer_outputs), + FILTER_QUERY_FUNC(query_formats), + .priv_class = &sbuffer_class, +}; diff --git a/libavfilter/buffersrc.h b/libavfilter/buffersrc.h index 3b248b37cd..45a657bbb6 100644 --- a/libavfilter/buffersrc.h +++ b/libavfilter/buffersrc.h @@ -74,6 +74,7 @@ typedef struct AVBufferSrcParameters { /** * video: the pixel format, value corresponds to enum AVPixelFormat * audio: the sample format, value corresponds to enum AVSampleFormat + * subtitles: the subtitle format, value corresponds to enum AVSubtitleType */ int format; /** From patchwork Tue Oct 25 09:13:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 38987 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2685447pzb; Tue, 25 Oct 2022 02:15:56 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4d1AUHWazKB43kXtQ7v4FAIMpSCKpzIjuHShfI4fL9uUvmFwzDnj+lUKNKpuf52Gom6j0g X-Received: by 2002:a17:907:a057:b0:7ac:8e6a:8cd7 with SMTP id gz23-20020a170907a05700b007ac8e6a8cd7mr1350469ejc.678.1666689355822; Tue, 25 Oct 2022 02:15:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666689355; cv=none; d=google.com; s=arc-20160816; b=w6f8hPqOSw77pcdqtcFd0IIw7xu2B0N/RLWt7f13vM9K6NDuK7t+Bfxed5SamjRqp8 YPivK6jc/aqF9TySXDWvxBhKzLL/iKkRPhUdYD9WMbKgIiznankm7II3BQ6H/W40jC85 a/rRV/P80+zr38bO0w3I2aiKmps/qy97OpyVPSlk4ClSNcRcKbomhoJv2IGpGGWzdd0r OAro+DhKdp23XHUyNreaT3b1xqPG650Zuh76FJf4Md95O3lZ1K3HUi9tZkSqfmhLgjv4 lPPEUYONaqQawa0WaB86NY3HZr+2Opglacm2jZsFgIpZh0s//g4uBC7g+PlmzTQBAhK5 i6GA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:references :in-reply-to:message-id:from:dkim-signature:delivered-to; bh=amjg1hIj49VKwIKrMBL6JEdpg6fqNoPHnV98WTIzcbQ=; b=Iwp/ScvwXk1rD7Z/tSrbJXFsEzuRdfLLPN/W57QJI22G1smqfq/05QhsI1k/K9ALz8 Go8HGy5W19PLj2PfSytkoxqot9kUs8vWtoe69sc6Ql7zcr7uywv+2od/GFlEcgm7k6E2 yz1e+QeTVPO//XyO8m+V+tmgdEvW74dVMNfQZzAZmazE9z5bX7O3v+cM15XV50StgK8n 028EwlBKkOg3EI5EMXZJzuJqSnkPt7f4DZR/jY8aggo10t3UU4FDSGKwt74MZh7Aa2A/ QX1qg3LA6xQHKraapyJSnPkWDb6TlUVFpQfglRecaSAUWyhjn8feMSk3qgGMguZJW1Vv HNaQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=Qh8y8TyT; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id y14-20020a17090668ce00b0078cff711da6si1700395ejr.976.2022.10.25.02.15.55; Tue, 25 Oct 2022 02:15:55 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=Qh8y8TyT; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id BFB9368BD39; Tue, 25 Oct 2022 12:14:11 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf1-f178.google.com (mail-pf1-f178.google.com [209.85.210.178]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D9D6168BD18 for ; Tue, 25 Oct 2022 12:14:06 +0300 (EEST) Received: by mail-pf1-f178.google.com with SMTP id m6so11295004pfb.0 for ; Tue, 25 Oct 2022 02:14:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=h8cR74C0eQwVPq9feV5zEvVdERj3tfg7ZKy5rVn6hHE=; b=Qh8y8TyTI/XWCTpQa8l6MKHcS+kgMNjDauSWEPPf8j/zJy7wmmUi4MBIaKZyUvSDpw mn54mEqYR0G49qA+7szUz+5zPb8ov8bBTUWPy4zHftb9AAIU9alfAla6yb9r08+26+t2 beW+5KjZ525zti4OhJ9ulygbmUJtnRa9H7HdJ/kvwtrfm8dco+CT0YJFTz/OuzXGYUhR C4OyyWh/z15gWBtN3xO5NR662WmhCaKUxx6zAdeWannf00lcu8A1BkjF+hYqu3JRs0f8 iNDT9PAppqTwfkmf/8qAerTGYvXPS3X0You4xn5vbGQ0eAidHIgdJnJHTqaBmr+WJZzC 7zUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=h8cR74C0eQwVPq9feV5zEvVdERj3tfg7ZKy5rVn6hHE=; b=ncT8UNyAwGQHPM0/Tr+DX7lErEk8eglG19BuxTPtahSoAQqEPgkmAnEZIfb4UtwxEa I9glX3vsE31BR+xrQQf96reV1C5K22HdhVlW8IVrcQzQH8GKW/c3nVzHOk35j5TyqDpd WGDvjlUGUxx/ItpJRunvCbHLeq6F6iLGlUCiDAhG+duotllVo+RhC1UH0cYu3xWMttMC RhimjE0eQvku67jtIaQOXA7IRCAIzUSZhKLmkogdNUvywFwxwiMmxGv9I0QfrLnx7WUi LzheMI5QPaLzGA+BohYbdotHIipqjm3lOw4wajCFd9na8VGsaPI4f+Eue5JqfNKo1KED zwnw== X-Gm-Message-State: ACrzQf11f+nZwemBRuLqMvvjprgl3hyJ8C1UaCB+kHy1xh9iPT+kiiIZ a2Z/1dtaQou4o5BFwiw1ifow+uJoIrw= X-Received: by 2002:a05:6a00:248b:b0:56b:830d:bbdf with SMTP id c11-20020a056a00248b00b0056b830dbbdfmr16222448pfv.47.1666689244574; Tue, 25 Oct 2022 02:14:04 -0700 (PDT) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id v10-20020a170902b7ca00b00174be817124sm869951plz.221.2022.10.25.02.14.03 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Oct 2022 02:14:04 -0700 (PDT) From: softworkz X-Google-Original-From: softworkz Message-Id: <2dce3f8101492dfa4fccab78e384895e80d99476.1666689226.git.ffmpegagent@gmail.com> In-Reply-To: References: Date: Tue, 25 Oct 2022 09:13:34 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v9 13/25] avfilter/overlaygraphicsubs: Add overlaygraphicsubs and graphicsub2video filters X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: softworkz Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: KJj8Y29DzbPa From: softworkz - overlaygraphicsubs (VS -> V) Overlay graphic subtitles onto a video stream - graphicsub2video {S -> V) Converts graphic subtitles to video frames (with alpha) Gets auto-inserted for retaining compatibility with sub2video command lines Signed-off-by: softworkz --- doc/filters.texi | 118 +++++ libavfilter/Makefile | 2 + libavfilter/allfilters.c | 2 + libavfilter/vf_overlaygraphicsubs.c | 765 ++++++++++++++++++++++++++++ 4 files changed, 887 insertions(+) create mode 100644 libavfilter/vf_overlaygraphicsubs.c diff --git a/doc/filters.texi b/doc/filters.texi index bcd19cf931..70a371ee0e 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -27364,6 +27364,124 @@ tools. @c man end VIDEO SINKS +@chapter Subtitle Filters +@c man begin SUBTITLE FILTERS + +When you configure your FFmpeg build, you can disable any of the +existing filters using @code{--disable-filters}. + +Below is a description of the currently available subtitle filters. + +@section graphicsub2video + +Renders graphic subtitles as video frames. + +This filter replaces the previous "sub2video" hack which did the conversion implicitly and up-front as subtitle filtering wasn't possible at that time. +To retain compatibility with earlier sub2video command lines, this filter is being auto-inserted in those cases. + +For overlaying graphicsal subtitles it is recommended to use the 'overlaygraphicsubs' filter which is more efficient and takes less processing resources. + +This filter is still useful in cases where the overlay is done with hardware acceleration (e.g. overlay_qsv, overlay_vaapi, overlay_cuda) for preparing the overlay frames. + +Inputs: +@itemize +@item 0: Subtitles [BITMAP] +@end itemize + +Outputs: +@itemize +@item 0: Video [RGB32] +@end itemize + + +It accepts the following parameters: + +@table @option +@item size, s +Set the size of the output video frame. + +@end table + +@subsection Examples + +@itemize +@item +Overlay PGS subtitles +(not recommended - better use overlaygraphicsubs) +@example +ffmpeg -i "https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv" -filter_complex "[0:1]graphicsub2video[subs];[0:0][subs]overlay" output.mp4 +@end example + +@item +Overlay PGS subtitles implicitly +The graphicsub2video is inserted automatically for compatibility with legacy command lines. +@example +ffmpeg -i "https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv" -filter_complex "[0:0][0:1]overlay" output.mp4 +@end example +@end itemize + +@section overlaygraphicsubs + +Overlay graphic subtitles onto a video stream. + +This filter can blend graphical subtitles on a video stream directly, i.e. without creating full-size alpha images first. +The blending operation is limited to the area of the subtitle rectangles, which also means that no processing is done at times where no subtitles are to be displayed. + +Inputs: +@itemize +@item 0: Video [YUV420P, YUV422P, YUV444P, ARGB, RGBA, ABGR, BGRA, RGB24, BGR24] +@item 1: Subtitles [BITMAP] +@end itemize + +Outputs: +@itemize +@item 0: Video (same as input) +@end itemize + +It accepts the following parameters: + +@table @option +@item x +@item y +Set the expression for the x and y coordinates of the overlaid video +on the main video. Default value is "0" for both expressions. In case +the expression is invalid, it is set to a huge value (meaning that the +overlay will not be displayed within the output visible area). + +@item eof_action +See @ref{framesync}. + +@item eval +Set when the expressions for @option{x}, and @option{y} are evaluated. + +It accepts the following values: +@table @samp +@item init +only evaluate expressions once during the filter initialization or +when a command is processed + +@item frame +evaluate expressions for each incoming frame +@end table + +Default value is @samp{frame}. + +@item shortest +See @ref{framesync}. + +@end table + +@subsection Examples + +@itemize +@item +Overlay PGS subtitles +@example +ffmpeg -i "https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv" -filter_complex "[0:0][0:1]overlaygraphicsubs" output.mp4 +@end example +@end itemize +@c man end SUBTITLE FILTERS + @chapter Multimedia Filters @c man begin MULTIMEDIA FILTERS diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 6db90028e0..0dae913b16 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -314,6 +314,7 @@ OBJS-$(CONFIG_GBLUR_FILTER) += vf_gblur.o OBJS-$(CONFIG_GBLUR_VULKAN_FILTER) += vf_gblur_vulkan.o vulkan.o vulkan_filter.o OBJS-$(CONFIG_GEQ_FILTER) += vf_geq.o OBJS-$(CONFIG_GRADFUN_FILTER) += vf_gradfun.o +OBJS-$(CONFIG_GRAPHICSUB2VIDEO_FILTER) += vf_overlaygraphicsubs.o framesync.o OBJS-$(CONFIG_GRAPHMONITOR_FILTER) += f_graphmonitor.o OBJS-$(CONFIG_GRAYWORLD_FILTER) += vf_grayworld.o OBJS-$(CONFIG_GREYEDGE_FILTER) += vf_colorconstancy.o @@ -398,6 +399,7 @@ OBJS-$(CONFIG_OVERLAY_OPENCL_FILTER) += vf_overlay_opencl.o opencl.o \ OBJS-$(CONFIG_OVERLAY_QSV_FILTER) += vf_overlay_qsv.o framesync.o OBJS-$(CONFIG_OVERLAY_VAAPI_FILTER) += vf_overlay_vaapi.o framesync.o vaapi_vpp.o OBJS-$(CONFIG_OVERLAY_VULKAN_FILTER) += vf_overlay_vulkan.o vulkan.o vulkan_filter.o +OBJS-$(CONFIG_OVERLAYGRAPHICSUBS_FILTER) += vf_overlaygraphicsubs.o framesync.o OBJS-$(CONFIG_OWDENOISE_FILTER) += vf_owdenoise.o OBJS-$(CONFIG_PAD_FILTER) += vf_pad.o OBJS-$(CONFIG_PAD_OPENCL_FILTER) += vf_pad_opencl.o opencl.o opencl/pad.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 173d85b982..71896ce904 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -374,6 +374,7 @@ extern const AVFilter ff_vf_overlay_qsv; extern const AVFilter ff_vf_overlay_vaapi; extern const AVFilter ff_vf_overlay_vulkan; extern const AVFilter ff_vf_overlay_cuda; +extern const AVFilter ff_vf_overlaygraphicsubs; extern const AVFilter ff_vf_owdenoise; extern const AVFilter ff_vf_pad; extern const AVFilter ff_vf_pad_opencl; @@ -564,6 +565,7 @@ extern const AVFilter ff_avf_showspectrumpic; extern const AVFilter ff_avf_showvolume; extern const AVFilter ff_avf_showwaves; extern const AVFilter ff_avf_showwavespic; +extern const AVFilter ff_svf_graphicsub2video; extern const AVFilter ff_vaf_spectrumsynth; /* multimedia sources */ diff --git a/libavfilter/vf_overlaygraphicsubs.c b/libavfilter/vf_overlaygraphicsubs.c new file mode 100644 index 0000000000..7b26d8ef37 --- /dev/null +++ b/libavfilter/vf_overlaygraphicsubs.c @@ -0,0 +1,765 @@ +/* + * Copyright (c) 2021 softworkz (derived from vf_overlay) + * Copyright (c) 2010 Stefano Sabatini + * Copyright (c) 2010 Baptiste Coudurier + * Copyright (c) 2007 Bobby Bingham + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * overlay graphical subtitles on top of a video frame + */ + +#include "libavutil/eval.h" +#include "libavutil/imgutils.h" +#include "libavutil/opt.h" +#include "internal.h" +#include "drawutils.h" +#include "framesync.h" + +enum var_name { + VAR_MAIN_W, VAR_MW, + VAR_MAIN_H, VAR_MH, + VAR_OVERLAY_W, VAR_OW, + VAR_OVERLAY_H, VAR_OH, + VAR_HSUB, + VAR_VSUB, + VAR_X, + VAR_Y, + VAR_N, + VAR_POS, + VAR_T, + VAR_VARS_NB +}; + +typedef struct OverlaySubsContext { + const AVClass *class; + int x, y; ///< position of overlaid picture + int w, h; + AVFrame *outpicref; + + int main_is_packed_rgb; + uint8_t main_rgba_map[4]; + int main_has_alpha; + uint8_t overlay_rgba_map[4]; + int eval_mode; ///< EvalMode + int use_caching; + AVFrame *cache_frame; + + FFFrameSync fs; + + int main_pix_step[4]; ///< steps per pixel for each plane of the main output + int hsub, vsub; ///< chroma subsampling values + const AVPixFmtDescriptor *main_desc; ///< format descriptor for main input + + double var_values[VAR_VARS_NB]; + char *x_expr, *y_expr; + + AVExpr *x_pexpr, *y_pexpr; + + int pic_counter; +} OverlaySubsContext; + +static const char *const var_names[] = { + "main_w", "W", ///< width of the main video + "main_h", "H", ///< height of the main video + "overlay_w", "w", ///< width of the overlay video + "overlay_h", "h", ///< height of the overlay video + "hsub", + "vsub", + "x", + "y", + "n", ///< number of frame + "pos", ///< position in the file + "t", ///< timestamp expressed in seconds + NULL +}; + +#define MAIN 0 +#define OVERLAY 1 + +#define R 0 +#define G 1 +#define B 2 +#define A 3 + +#define Y 0 +#define U 1 +#define V 2 + +enum EvalMode { + EVAL_MODE_INIT, + EVAL_MODE_FRAME, + EVAL_MODE_NB +}; + +static av_cold void overlay_graphicsubs_uninit(AVFilterContext *ctx) +{ + OverlaySubsContext *s = ctx->priv; + + av_frame_free(&s->cache_frame); + ff_framesync_uninit(&s->fs); + av_expr_free(s->x_pexpr); s->x_pexpr = NULL; + av_expr_free(s->y_pexpr); s->y_pexpr = NULL; +} + +static inline int normalize_xy(double d, int chroma_sub) +{ + if (isnan(d)) + return INT_MAX; + return (int)d & ~((1 << chroma_sub) - 1); +} + +static void eval_expr(AVFilterContext *ctx) +{ + OverlaySubsContext *s = ctx->priv; + + s->var_values[VAR_X] = av_expr_eval(s->x_pexpr, s->var_values, NULL); + s->var_values[VAR_Y] = av_expr_eval(s->y_pexpr, s->var_values, NULL); + /* It is necessary if x is expressed from y */ + s->var_values[VAR_X] = av_expr_eval(s->x_pexpr, s->var_values, NULL); + s->x = normalize_xy(s->var_values[VAR_X], s->hsub); + s->y = normalize_xy(s->var_values[VAR_Y], s->vsub); +} + +static int set_expr(AVExpr **pexpr, const char *expr, const char *option, void *log_ctx) +{ + int ret; + AVExpr *old = NULL; + + if (*pexpr) + old = *pexpr; + ret = av_expr_parse(pexpr, expr, var_names, + NULL, NULL, NULL, NULL, 0, log_ctx); + if (ret < 0) { + av_log(log_ctx, AV_LOG_ERROR, + "Error when evaluating the expression '%s' for %s\n", + expr, option); + *pexpr = old; + return ret; + } + + av_expr_free(old); + return 0; +} + +static int overlay_graphicsubs_query_formats(AVFilterContext *ctx) +{ + AVFilterFormats *formats; + AVFilterLink *inlink0 = ctx->inputs[0]; + AVFilterLink *inlink1 = ctx->inputs[1]; + AVFilterLink *outlink = ctx->outputs[0]; + int ret; + static const enum AVSubtitleType subtitle_fmts[] = { AV_SUBTITLE_FMT_BITMAP, AV_SUBTITLE_FMT_NONE }; + static const enum AVPixelFormat supported_pix_fmts[] = { + AV_PIX_FMT_YUV420P, AV_PIX_FMT_YUV422P, AV_PIX_FMT_YUV444P, + AV_PIX_FMT_ARGB, AV_PIX_FMT_RGBA, + AV_PIX_FMT_ABGR, AV_PIX_FMT_BGRA, + AV_PIX_FMT_RGB24, AV_PIX_FMT_BGR24, + AV_PIX_FMT_NONE + }; + + /* set input0 video formats */ + formats = ff_make_format_list(supported_pix_fmts); + if ((ret = ff_formats_ref(formats, &inlink0->outcfg.formats)) < 0) + return ret; + + /* set output0 video formats */ + if ((ret = ff_formats_ref(formats, &outlink->incfg.formats)) < 0) + return ret; + + /* set input1 subtitle formats */ + formats = ff_make_format_list(subtitle_fmts); + if ((ret = ff_formats_ref(formats, &inlink1->outcfg.formats)) < 0) + return ret; + + return 0; +} + +static int config_output(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + OverlaySubsContext *s = ctx->priv; + int ret; + + if ((ret = ff_framesync_init_dualinput(&s->fs, ctx)) < 0) + return ret; + + outlink->w = ctx->inputs[MAIN]->w; + outlink->h = ctx->inputs[MAIN]->h; + outlink->time_base = ctx->inputs[MAIN]->time_base; + outlink->frame_rate = ctx->inputs[MAIN]->frame_rate; + + return ff_framesync_configure(&s->fs); +} + +// divide by 255 and round to nearest +// apply a fast variant: (X+127)/255 = ((X+127)*257+257)>>16 = ((X+128)*257)>>16 +#define FAST_DIV255(x) ((((x) + 128) * 257) >> 16) + +// calculate the non-pre-multiplied alpha, applying the general equation: +// alpha = alpha_overlay / ( (alpha_main + alpha_overlay) - (alpha_main * alpha_overlay) ) +// (((x) << 16) - ((x) << 9) + (x)) is a faster version of: 255 * 255 * x +// ((((x) + (y)) << 8) - ((x) + (y)) - (y) * (x)) is a faster version of: 255 * (x + y) +#define UNPREMULTIPLY_ALPHA(x, y) ((((x) << 16) - ((x) << 9) + (x)) / ((((x) + (y)) << 8) - ((x) + (y)) - (y) * (x))) + +/** + * Blend image in src to destination buffer dst at position (x, y). + */ +static av_always_inline void blend_packed_rgb(const AVFilterContext *ctx, + const AVFrame *dst, const AVSubtitleArea *src, + int x, int y, + int is_straight) +{ + OverlaySubsContext *s = ctx->priv; + int i, imax, j, jmax; + const int src_w = src->w; + const int src_h = src->h; + const int dst_w = dst->width; + const int dst_h = dst->height; + uint8_t alpha; ///< the amount of overlay to blend on to main + const int dr = s->main_rgba_map[R]; + const int dg = s->main_rgba_map[G]; + const int db = s->main_rgba_map[B]; + const int da = s->main_rgba_map[A]; + const int dstep = s->main_pix_step[0]; + const int sr = s->overlay_rgba_map[R]; + const int sg = s->overlay_rgba_map[G]; + const int sb = s->overlay_rgba_map[B]; + const int sa = s->overlay_rgba_map[A]; + int slice_start, slice_end; + uint8_t *S, *sp, *d, *dp; + + i = FFMAX(-y, 0); + imax = FFMIN3(-y + dst_h, FFMIN(src_h, dst_h), y + src_h); + + slice_start = i; + slice_end = i + imax; + + sp = src->buf[0]->data + slice_start * src->linesize[0]; + dp = dst->data[0] + (slice_start + y) * dst->linesize[0]; + + for (i = slice_start; i < slice_end; i++) { + j = FFMAX(-x, 0); + S = sp + j; + d = dp + ((x + j) * dstep); + + for (jmax = FFMIN(-x + dst_w, src_w); j < jmax; j++) { + uint32_t val = src->pal[*S]; + const uint8_t *sval = (uint8_t *)&val; + alpha = sval[sa]; + + // if the main channel has an alpha channel, alpha has to be calculated + // to create an un-premultiplied (straight) alpha value + if (s->main_has_alpha && alpha != 0 && alpha != 255) { + const uint8_t alpha_d = d[da]; + alpha = UNPREMULTIPLY_ALPHA(alpha, alpha_d); + } + + switch (alpha) { + case 0: + break; + case 255: + d[dr] = sval[sr]; + d[dg] = sval[sg]; + d[db] = sval[sb]; + break; + default: + // main_value = main_value * (1 - alpha) + overlay_value * alpha + // since alpha is in the range 0-255, the result must divided by 255 + d[dr] = is_straight ? FAST_DIV255(d[dr] * (255 - alpha) + sval[sr] * alpha) : + FFMIN(FAST_DIV255(d[dr] * (255 - alpha)) + sval[sr], 255); + d[dg] = is_straight ? FAST_DIV255(d[dg] * (255 - alpha) + sval[sg] * alpha) : + FFMIN(FAST_DIV255(d[dg] * (255 - alpha)) + sval[sg], 255); + d[db] = is_straight ? FAST_DIV255(d[db] * (255 - alpha) + sval[sb] * alpha) : + FFMIN(FAST_DIV255(d[db] * (255 - alpha)) + sval[sb], 255); + } + + if (s->main_has_alpha) { + switch (alpha) { + case 0: + break; + case 255: + d[da] = sval[sa]; + break; + default: + // apply alpha compositing: main_alpha += (1-main_alpha) * overlay_alpha + d[da] += FAST_DIV255((255 - d[da]) * S[sa]); + } + } + d += dstep; + S += 1; + } + dp += dst->linesize[0]; + sp += src->linesize[0]; + } +} + +static av_always_inline void blend_plane_8_8bits(const AVFilterContext *ctx, const AVFrame *dst, const AVSubtitleArea *area, + const uint32_t *yuv_pal, int src_w, int src_h, int dst_w, int dst_h, int plane, int hsub, int vsub, + int x, int y, int dst_plane, int dst_offset, int dst_step) +{ + const int src_wp = AV_CEIL_RSHIFT(src_w, hsub); + const int src_hp = AV_CEIL_RSHIFT(src_h, vsub); + const int dst_wp = AV_CEIL_RSHIFT(dst_w, hsub); + const int dst_hp = AV_CEIL_RSHIFT(dst_h, vsub); + const int yp = y >> vsub; + const int xp = x >> hsub; + uint8_t *s, *sp, *d, *dp, *dap; + int imax, i, j, jmax; + int slice_start, slice_end; + + i = FFMAX(-yp, 0); \ + imax = FFMIN3(-yp + dst_hp, FFMIN(src_hp, dst_hp), yp + src_hp); \ + + slice_start = i; + slice_end = i + imax; + + sp = area->buf[0]->data + (slice_start << vsub) * area->linesize[0]; + dp = dst->data[dst_plane] + (yp + slice_start) * dst->linesize[dst_plane] + dst_offset; + + dap = dst->data[3] + ((yp + slice_start) << vsub) * dst->linesize[3]; + + for (i = slice_start; i < slice_end; i++) { + j = FFMAX(-xp, 0); + d = dp + (xp + j) * dst_step; + s = sp + (j << hsub); + jmax = FFMIN(-xp + dst_wp, src_wp); + + for (; j < jmax; j++) { + uint32_t val = yuv_pal[*s]; + const uint8_t *sval = (uint8_t *)&val; + const int alpha = sval[3]; + const int max = 255, mid = 128; + const int d_int = *d; + const int sval_int = sval[plane]; + + switch (alpha) { + case 0: + break; + case 255: + *d = sval[plane]; + break; + default: + if (plane > 0) + *d = av_clip(FAST_DIV255((d_int - mid) * (max - alpha) + (sval_int - mid) * alpha) , -mid, mid) + mid; + else + *d = FAST_DIV255(d_int * (max - alpha) + sval_int * alpha); + break; + } + + d += dst_step; + s += 1 << hsub; + } + dp += dst->linesize[dst_plane]; + sp += (1 << vsub) * area->linesize[0]; + dap += (1 << vsub) * dst->linesize[3]; + } +} + +#define RGB2Y(r, g, b) (uint8_t)(((66 * (r) + 129 * (g) + 25 * (b) + 128) >> 8) + 16) +#define RGB2U(r, g, b) (uint8_t)(((-38 * (r) - 74 * (g) + 112 * (b) + 128) >> 8) + 128) +#define RGB2V(r, g, b) (uint8_t)(((112 * (r) - 94 * (g) - 18 * (b) + 128) >> 8) + 128) +/* Converts R8 G8 B8 color to YUV. */ +static av_always_inline void rgb_2_yuv(uint8_t r, uint8_t g, uint8_t b, uint8_t* y, uint8_t* u, uint8_t* v) +{ + *y = RGB2Y((int)r, (int)g, (int)b); + *u = RGB2U((int)r, (int)g, (int)b); + *v = RGB2V((int)r, (int)g, (int)b); +} + + +static av_always_inline void blend_yuv_8_8bits(AVFilterContext *ctx, AVFrame *dst, const AVSubtitleArea *area, int hsub, int vsub, int x, int y) +{ + OverlaySubsContext *s = ctx->priv; + const int src_w = area->w; + const int src_h = area->h; + const int dst_w = dst->width; + const int dst_h = dst->height; + const int sr = s->overlay_rgba_map[R]; + const int sg = s->overlay_rgba_map[G]; + const int sb = s->overlay_rgba_map[B]; + const int sa = s->overlay_rgba_map[A]; + uint32_t yuvpal[256]; + + for (int i = 0; i < 256; ++i) { + const uint8_t *rgba = (const uint8_t *)&area->pal[i]; + uint8_t *yuva = (uint8_t *)&yuvpal[i]; + rgb_2_yuv(rgba[sr], rgba[sg], rgba[sb], &yuva[Y], &yuva[U], &yuva[V]); + yuva[3] = rgba[sa]; + } + + blend_plane_8_8bits(ctx, dst, area, yuvpal, src_w, src_h, dst_w, dst_h, Y, 0, 0, x, y, s->main_desc->comp[Y].plane, s->main_desc->comp[Y].offset, s->main_desc->comp[Y].step); + blend_plane_8_8bits(ctx, dst, area, yuvpal, src_w, src_h, dst_w, dst_h, U, hsub, vsub, x, y, s->main_desc->comp[U].plane, s->main_desc->comp[U].offset, s->main_desc->comp[U].step); + blend_plane_8_8bits(ctx, dst, area, yuvpal, src_w, src_h, dst_w, dst_h, V, hsub, vsub, x, y, s->main_desc->comp[V].plane, s->main_desc->comp[V].offset, s->main_desc->comp[V].step); +} + +static int config_input_main(AVFilterLink *inlink) +{ + int ret; + AVFilterContext *ctx = inlink->dst; + OverlaySubsContext *s = inlink->dst->priv; + const AVPixFmtDescriptor *pix_desc = av_pix_fmt_desc_get(inlink->format); + + av_image_fill_max_pixsteps(s->main_pix_step, NULL, pix_desc); + ff_fill_rgba_map(s->overlay_rgba_map, AV_PIX_FMT_RGB32); // it's actually AV_PIX_FMT_PAL8); + + s->hsub = pix_desc->log2_chroma_w; + s->vsub = pix_desc->log2_chroma_h; + + s->main_desc = pix_desc; + + s->main_is_packed_rgb = ff_fill_rgba_map(s->main_rgba_map, inlink->format) >= 0; + s->main_has_alpha = !!(pix_desc->flags & AV_PIX_FMT_FLAG_ALPHA); + + /* Finish the configuration by evaluating the expressions + now when both inputs are configured. */ + s->var_values[VAR_MAIN_W ] = s->var_values[VAR_MW] = ctx->inputs[MAIN ]->w; + s->var_values[VAR_MAIN_H ] = s->var_values[VAR_MH] = ctx->inputs[MAIN ]->h; + s->var_values[VAR_OVERLAY_W] = s->var_values[VAR_OW] = ctx->inputs[OVERLAY]->w; + s->var_values[VAR_OVERLAY_H] = s->var_values[VAR_OH] = ctx->inputs[OVERLAY]->h; + s->var_values[VAR_HSUB] = 1<log2_chroma_w; + s->var_values[VAR_VSUB] = 1<log2_chroma_h; + s->var_values[VAR_X] = NAN; + s->var_values[VAR_Y] = NAN; + s->var_values[VAR_N] = 0; + s->var_values[VAR_T] = NAN; + s->var_values[VAR_POS] = NAN; + + if ((ret = set_expr(&s->x_pexpr, s->x_expr, "x", ctx)) < 0 || + (ret = set_expr(&s->y_pexpr, s->y_expr, "y", ctx)) < 0) + return ret; + + if (s->eval_mode == EVAL_MODE_INIT) { + eval_expr(ctx); + av_log(ctx, AV_LOG_VERBOSE, "x:%f xi:%d y:%f yi:%d\n", + s->var_values[VAR_X], s->x, + s->var_values[VAR_Y], s->y); + } + + av_log(ctx, AV_LOG_VERBOSE, + "main w:%d h:%d fmt:%s overlay w:%d h:%d fmt:%s\n", + ctx->inputs[MAIN]->w, ctx->inputs[MAIN]->h, + av_get_pix_fmt_name(ctx->inputs[MAIN]->format), + ctx->inputs[OVERLAY]->w, ctx->inputs[OVERLAY]->h, + av_get_pix_fmt_name(ctx->inputs[OVERLAY]->format)); + return 0; +} + +static int do_blend(FFFrameSync *fs) +{ + AVFilterContext *ctx = fs->parent; + AVFrame *mainpic, *second; + OverlaySubsContext *s = ctx->priv; + AVFilterLink *inlink = ctx->inputs[0]; + unsigned i; + int ret; + + ret = ff_framesync_dualinput_get_writable(fs, &mainpic, &second); + if (ret < 0) + return ret; + if (!second) + return ff_filter_frame(ctx->outputs[0], mainpic); + + if (s->eval_mode == EVAL_MODE_FRAME) { + int64_t pos = mainpic->pkt_pos; + + s->var_values[VAR_N] = (double)inlink->frame_count_out; + s->var_values[VAR_T] = mainpic->pts == AV_NOPTS_VALUE ? + NAN :(double)mainpic->pts * av_q2d(inlink->time_base); + s->var_values[VAR_POS] = pos == -1 ? NAN : (double)pos; + + s->var_values[VAR_OVERLAY_W] = s->var_values[VAR_OW] = second->width; + s->var_values[VAR_OVERLAY_H] = s->var_values[VAR_OH] = second->height; + s->var_values[VAR_MAIN_W ] = s->var_values[VAR_MW] = mainpic->width; + s->var_values[VAR_MAIN_H ] = s->var_values[VAR_MH] = mainpic->height; + + eval_expr(ctx); + av_log(ctx, AV_LOG_DEBUG, "n:%f t:%f pos:%f x:%f xi:%d y:%f yi:%d\n", + s->var_values[VAR_N], s->var_values[VAR_T], s->var_values[VAR_POS], + s->var_values[VAR_X], s->x, + s->var_values[VAR_Y], s->y); + } + + for (i = 0; i < second->num_subtitle_areas; i++) { + const AVSubtitleArea *sub_area = second->subtitle_areas[i]; + + if (sub_area->type != AV_SUBTITLE_FMT_BITMAP) { + av_log(NULL, AV_LOG_WARNING, "overlay_graphicsubs: non-bitmap subtitle\n"); + return AVERROR_INVALIDDATA; + } + + switch (inlink->format) { + case AV_PIX_FMT_YUV420P: + blend_yuv_8_8bits(ctx, mainpic, sub_area, 1, 1, sub_area->x + s->x, sub_area->y + s->y); + break; + case AV_PIX_FMT_YUV422P: + blend_yuv_8_8bits(ctx, mainpic, sub_area, 1, 0, sub_area->x + s->x, sub_area->y + s->y); + break; + case AV_PIX_FMT_YUV444P: + blend_yuv_8_8bits(ctx, mainpic, sub_area, 0, 0, sub_area->x + s->x, sub_area->y + s->y); + break; + case AV_PIX_FMT_RGB24: + case AV_PIX_FMT_BGR24: + case AV_PIX_FMT_ARGB: + case AV_PIX_FMT_RGBA: + case AV_PIX_FMT_BGRA: + case AV_PIX_FMT_ABGR: + blend_packed_rgb(ctx, mainpic, sub_area, sub_area->x + s->x, sub_area->y + s->y, 1); + break; + default: + av_log(NULL, AV_LOG_ERROR, "Unsupported input pix fmt: %d\n", inlink->format); + return AVERROR(EINVAL); + } + } + + return ff_filter_frame(ctx->outputs[0], mainpic); +} + +static av_cold int overlay_graphicsubs_init(AVFilterContext *ctx) +{ + OverlaySubsContext *s = ctx->priv; + + s->fs.on_event = do_blend; + return 0; +} + +static int overlay_graphicsubs_activate(AVFilterContext *ctx) +{ + OverlaySubsContext *s = ctx->priv; + return ff_framesync_activate(&s->fs); +} + +static int graphicsub2video_query_formats(AVFilterContext *ctx) +{ + AVFilterFormats *formats; + AVFilterLink *inlink = ctx->inputs[0]; + AVFilterLink *outlink = ctx->outputs[0]; + static const enum AVSubtitleType subtitle_fmts[] = { AV_SUBTITLE_FMT_BITMAP, AV_SUBTITLE_FMT_NONE }; + static const enum AVPixelFormat pix_fmts[] = { AV_PIX_FMT_RGB32, AV_PIX_FMT_NONE }; + int ret; + + /* set input subtitle formats */ + formats = ff_make_format_list(subtitle_fmts); + if ((ret = ff_formats_ref(formats, &inlink->outcfg.formats)) < 0) + return ret; + + /* set output video formats */ + formats = ff_make_format_list(pix_fmts); + if ((ret = ff_formats_ref(formats, &outlink->incfg.formats)) < 0) + return ret; + + return 0; +} + +static int graphicsub2video_config_input(AVFilterLink *inlink) +{ + AVFilterContext *ctx = inlink->dst; + OverlaySubsContext *s = ctx->priv; + + if (s->w <= 0 || s->h <= 0) { + s->w = inlink->w; + s->h = inlink->h; + } + return 0; +} + +static int graphicsub2video_config_output(AVFilterLink *outlink) +{ + const AVFilterContext *ctx = outlink->src; + OverlaySubsContext *s = ctx->priv; + const AVPixFmtDescriptor *pix_desc = av_pix_fmt_desc_get(outlink->format); + + outlink->w = s->w; + outlink->h = s->h; + + if (outlink->w == 0 && outlink->h == 0) { + outlink->w = 1; + outlink->h = 1; + } + outlink->sample_aspect_ratio = (AVRational){1,1}; + outlink->time_base = ctx->inputs[0]->time_base; + outlink->frame_rate = ctx->inputs[0]->frame_rate; + + av_image_fill_max_pixsteps(s->main_pix_step, NULL, pix_desc); + ff_fill_rgba_map(s->overlay_rgba_map, AV_PIX_FMT_RGB32); + + s->hsub = pix_desc->log2_chroma_w; + s->vsub = pix_desc->log2_chroma_h; + + s->main_desc = pix_desc; + + s->main_is_packed_rgb = ff_fill_rgba_map(s->main_rgba_map, outlink->format) >= 0; + s->main_has_alpha = !!(pix_desc->flags & AV_PIX_FMT_FLAG_ALPHA); + + return 0; +} + +static int graphicsub2video_filter_frame(AVFilterLink *inlink, AVFrame *frame) +{ + AVFilterLink *outlink = inlink->dst->outputs[0]; + const AVFilterContext *ctx = outlink->src; + OverlaySubsContext *s = ctx->priv; + AVFrame *out; + const unsigned num_rects = frame->num_subtitle_areas; + unsigned int i; + int ret; + + if (s->use_caching && s->cache_frame && frame->repeat_sub + && s->cache_frame->subtitle_timing.start_pts == frame->subtitle_timing.start_pts) { + out = av_frame_clone(s->cache_frame); + if (!out) + return AVERROR(ENOMEM); + + ret = av_frame_copy_props(out, frame); + if (ret < 0) + return ret; + + av_log(inlink->dst, AV_LOG_DEBUG, "graphicsub2video CACHED - size %dx%d pts: %"PRId64" areas: %d\n", frame->width, frame->height, frame->subtitle_timing.start_pts, frame->num_subtitle_areas); + av_frame_free(&frame); + return ff_filter_frame(outlink, out); + } + + + out = ff_get_video_buffer(outlink, outlink->w, outlink->h); + if (!out) { + av_frame_free(&frame); + return AVERROR(ENOMEM); + } + + memset(out->data[0], 0, (size_t)out->linesize[0] * out->height); + + out->pts = out->pkt_dts = out->best_effort_timestamp = frame->pts; + out->coded_picture_number = out->display_picture_number = s->pic_counter++; + + for (i = 0; i < num_rects; i++) { + const AVSubtitleArea *sub_rect = frame->subtitle_areas[i]; + + if (sub_rect->type != AV_SUBTITLE_FMT_BITMAP) { + av_log(NULL, AV_LOG_WARNING, "graphicsub2video: non-bitmap subtitle\n"); + av_frame_free(&frame); + return AVERROR_INVALIDDATA; + } + + blend_packed_rgb(inlink->dst, out, sub_rect, sub_rect->x, sub_rect->y, 1); + } + + av_log(inlink->dst, AV_LOG_DEBUG, "graphicsub2video output - size %dx%d pts: %"PRId64" areas: %d\n", out->width, out->height, out->pts, frame->num_subtitle_areas); + + if (s->use_caching) { + av_frame_free(&s->cache_frame); + s->cache_frame = av_frame_clone(out); + } + + av_frame_free(&frame); + return ff_filter_frame(outlink, out); +} + +#define OFFSET(x) offsetof(OverlaySubsContext, x) +#define FLAGS (AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_FILTERING_PARAM) + +static const AVOption overlaygraphicsubs_options[] = { + { "x", "set the x expression", OFFSET(x_expr), AV_OPT_TYPE_STRING, {.str = "0"}, 0, 0, .flags = FLAGS }, + { "y", "set the y expression", OFFSET(y_expr), AV_OPT_TYPE_STRING, {.str = "0"}, 0, 0, .flags = FLAGS }, + { "eof_action", "Action to take when encountering EOF from secondary input ", + OFFSET(fs.opt_eof_action), AV_OPT_TYPE_INT, { .i64 = EOF_ACTION_REPEAT }, + EOF_ACTION_REPEAT, EOF_ACTION_PASS, .flags = FLAGS, "eof_action" }, + { "repeat", "Repeat the previous frame.", 0, AV_OPT_TYPE_CONST, { .i64 = EOF_ACTION_REPEAT }, .flags = FLAGS, "eof_action" }, + { "endall", "End both streams.", 0, AV_OPT_TYPE_CONST, { .i64 = EOF_ACTION_ENDALL }, .flags = FLAGS, "eof_action" }, + { "pass", "Pass through the main input.", 0, AV_OPT_TYPE_CONST, { .i64 = EOF_ACTION_PASS }, .flags = FLAGS, "eof_action" }, + { "eval", "specify when to evaluate expressions", OFFSET(eval_mode), AV_OPT_TYPE_INT, {.i64 = EVAL_MODE_FRAME}, 0, EVAL_MODE_NB-1, FLAGS, "eval" }, + { "init", "eval expressions once during initialization", 0, AV_OPT_TYPE_CONST, {.i64=EVAL_MODE_INIT}, .flags = FLAGS, .unit = "eval" }, + { "frame", "eval expressions per-frame", 0, AV_OPT_TYPE_CONST, {.i64=EVAL_MODE_FRAME}, .flags = FLAGS, .unit = "eval" }, + { "shortest", "force termination when the shortest input terminates", OFFSET(fs.opt_shortest), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, .flags = FLAGS }, + { "repeatlast", "repeat overlay of the last overlay frame", OFFSET(fs.opt_repeatlast), AV_OPT_TYPE_BOOL, {.i64=1}, 0, 1, .flags = FLAGS }, + { NULL } +}; + +static const AVOption graphicsub2video_options[] = { + { "size", "set output frame size", OFFSET(w), AV_OPT_TYPE_IMAGE_SIZE, {.str = NULL}, 0, 0, .flags = FLAGS }, + { "s", "set output frame size", OFFSET(w), AV_OPT_TYPE_IMAGE_SIZE, {.str = NULL}, 0, 0, .flags = FLAGS }, + { "use_caching", "Cache output frames", OFFSET(use_caching), AV_OPT_TYPE_BOOL, { .i64 = 1}, 0, 1, .flags = FLAGS }, + { NULL } +}; + +FRAMESYNC_DEFINE_CLASS(overlaygraphicsubs, OverlaySubsContext, fs); + +static const AVFilterPad overlaygraphicsubs_inputs[] = { + { + .name = "main", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_input_main, + .flags = AVFILTERPAD_FLAG_NEEDS_WRITABLE, + }, + { + .name = "overlay", + .type = AVMEDIA_TYPE_SUBTITLE, + }, +}; + +static const AVFilterPad overlaygraphicsubs_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_output, + }, +}; + +const AVFilter ff_vf_overlaygraphicsubs = { + .name = "overlaygraphicsubs", + .description = NULL_IF_CONFIG_SMALL("Overlay graphical subtitles on top of the input."), + .preinit = overlaygraphicsubs_framesync_preinit, + .init = overlay_graphicsubs_init, + .uninit = overlay_graphicsubs_uninit, + .priv_size = sizeof(OverlaySubsContext), + .priv_class = &overlaygraphicsubs_class, + .activate = overlay_graphicsubs_activate, + FILTER_INPUTS(overlaygraphicsubs_inputs), + FILTER_OUTPUTS(overlaygraphicsubs_outputs), + FILTER_QUERY_FUNC(overlay_graphicsubs_query_formats), +}; + +AVFILTER_DEFINE_CLASS(graphicsub2video); + +static const AVFilterPad graphicsub2video_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .filter_frame = graphicsub2video_filter_frame, + .config_props = graphicsub2video_config_input, + }, +}; + +static const AVFilterPad graphicsub2video_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = graphicsub2video_config_output, + }, +}; + +const AVFilter ff_svf_graphicsub2video = { + .name = "graphicsub2video", + .description = NULL_IF_CONFIG_SMALL("Convert graphical subtitles to video"), + .priv_size = sizeof(OverlaySubsContext), + .priv_class = &graphicsub2video_class, + FILTER_INPUTS(graphicsub2video_inputs), + FILTER_OUTPUTS(graphicsub2video_outputs), + FILTER_QUERY_FUNC(graphicsub2video_query_formats), +}; From patchwork Tue Oct 25 09:13:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 38988 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2685518pzb; Tue, 25 Oct 2022 02:16:05 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6LGpLmlZQIOHywcvCCZDjznNFDtPHtwsughSzW71YHUVVCNkkdlldePDaY6sZUm9U3TGy3 X-Received: by 2002:a05:6402:31f4:b0:461:604d:2607 with SMTP id dy20-20020a05640231f400b00461604d2607mr18460115edb.330.1666689365773; Tue, 25 Oct 2022 02:16:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666689365; cv=none; d=google.com; s=arc-20160816; b=KF33I3lw1TLhvf7DRxG3wl5G5CphPZqiUivihLqwYJyRwxFtLPQzR+QcO7ZIJo7C6s h5SZ7YioSFN19WkVegQC0E0Mahg1Z3gaeqsB355LRDIPzEHD1IGOsyqUbAZoriZZMR5B bWnuVGuYh/F45NPy/9uxqHTJ0wmyqar4nhCJLUQsBUZ7X3IOOq/DG2ZWk1e1lPJghWMJ zRnWyMWLc4QU6poADZ3rUYlMSTOMo6Q8oz1fFtqYxVeee2hoih8l8rDKNvwqWnGN8FTO I86rigduVqLxa8BCW9tJDeDQ1JFQW4PfiYT5/n3R0RaasEtTrR0qcnNwdHdB6DgY/tzl 7/MA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:references :in-reply-to:message-id:from:dkim-signature:delivered-to; bh=ULrv3lNPO4nHEHn2pc5UVFXVN0pbIT2NpLhPn60qS9A=; b=F9mdxbydqLOHLM2oeDw+REX1d58xoNT+RGELeK8kSy8mxxbk0oPsY0o/U+/Lw0qr6w uMcyllXCeNfNEDyTv4yOTX25Udmu4nSR//RynuGHF6y9WvE4ogBTrTex/ZepkKEHUmbs aYi0VGTWtFwFucMszrwgUFsZQ43xVaFkdlGqgYqTMFJRh+bh4TGM7dTqEofo5dG9noLw +rGYTIFfyIwzqPNA0Cp/zezROTbsKpbqZWeYgByK9J5N6mQNK6IGmw3NnlI2APUFZmiv ubtg7FUU4j71SH/janvZQTor0We0G/rIOOb96rwJSwam1YizgicZcC/8VPwY8BtnSjjR djWA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b="NGMOYv/Z"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id i5-20020a1709064fc500b0078d93245e34si2383462ejw.793.2022.10.25.02.16.05; Tue, 25 Oct 2022 02:16:05 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b="NGMOYv/Z"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B138868BD1A; Tue, 25 Oct 2022 12:14:12 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f54.google.com (mail-pj1-f54.google.com [209.85.216.54]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 090C468BD1F for ; Tue, 25 Oct 2022 12:14:07 +0300 (EEST) Received: by mail-pj1-f54.google.com with SMTP id b11so4068969pjp.2 for ; Tue, 25 Oct 2022 02:14:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=ueIQY4ZDM0F4sjyxhlDt0af8Eq33oB45rt68sem90I0=; b=NGMOYv/Z/t185VikFW7Nt3b79/z/hyxTdV4QJNaja5FhsKAtr+smyjzda8z7DCuFjd 11SmpwB+knZpHxzLLENwRRFUOYl8YjqErZisc4/vsHV6/Z+5vAcFiyFekKmaAZB6+2dy FVcB1ZsSP8RRNAc/xCtQaxaiG2FpzCrb+QspV1XhsXgUF9Txf/adyhdoClrGjjf3HHUS 3FktKYUPvltTnJcUGu2TuLzEVFBHojuWw7gdaHQTtdSd13j+bTIHHHgwVChc6YeRFCfS PItKWQ+qmlJH792+3LGEmGhS+xuim+QFGObRJBp+ox7jGoj3Of40G4BLAzv84zkZS6Jh S9kQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=ueIQY4ZDM0F4sjyxhlDt0af8Eq33oB45rt68sem90I0=; b=iVwwM3TYUvDtFEVP30ai8/0G6udVMd/uo+7ALygwZui3VpUfeASe73nGi6q1axjWOj ZLVjXJ1zOy0gN4FGItMouRxadd6mMHLaneidHpzIViUmbCT48jqMNmuXXjzxK1IRIJlE VWFEpKT1YntoRgT2pyYWoeQa3XrH2OsRHGNTmmmPC/me3yt1PHpdRVZLBMS8w6GsT0iy nrez5hpa61kK5lpg6fcxN7JEFd4hKRK+MWElyPOFnSRLevmArxso/rm0iijjcynS4Z0l JGrtLX3F/NNFNrl8bkqIXuhtkeRthDrkbVZ5xFQ5sevkrtzu0J06rhZOxm7nr+Mtkybt fEfg== X-Gm-Message-State: ACrzQf19gc2qL/7YA3Wvj3+IUT6f364l4Rm4WHKZCoKUlAABf3d7URdr ljvDRjboDuQIwedboIuGtls1t2Bpql0= X-Received: by 2002:a17:90b:314c:b0:20d:83c1:5297 with SMTP id ip12-20020a17090b314c00b0020d83c15297mr43150479pjb.18.1666689245867; Tue, 25 Oct 2022 02:14:05 -0700 (PDT) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id m126-20020a632684000000b004588814ca73sm913460pgm.21.2022.10.25.02.14.05 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Oct 2022 02:14:05 -0700 (PDT) From: softworkz X-Google-Original-From: softworkz Message-Id: <14d291dbaee386d5c00b9ec7daa692a7c105ba94.1666689226.git.ffmpegagent@gmail.com> In-Reply-To: References: Date: Tue, 25 Oct 2022 09:13:35 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v9 14/25] avfilter/overlaytextsubs: Add overlaytextsubs and textsubs2video filters X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: softworkz Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: ico8/DmQzbPH From: softworkz - overlaytextsubs {VS -> V) Overlay text subtitles onto a video stream. - textsubs2video {S -> V) Converts text subtitles to video frames Signed-off-by: softworkz --- configure | 2 + doc/filters.texi | 113 +++++ libavfilter/Makefile | 2 + libavfilter/allfilters.c | 2 + libavfilter/vf_overlaytextsubs.c | 680 +++++++++++++++++++++++++++++++ 5 files changed, 799 insertions(+) create mode 100644 libavfilter/vf_overlaytextsubs.c diff --git a/configure b/configure index b3f242469b..61a7bd1438 100755 --- a/configure +++ b/configure @@ -3716,6 +3716,7 @@ overlay_qsv_filter_deps="libmfx" overlay_qsv_filter_select="qsvvpp" overlay_vaapi_filter_deps="vaapi VAProcPipelineCaps_blend_flags" overlay_vulkan_filter_deps="vulkan spirv_compiler" +overlaytextsubs_filter_deps="avcodec libass" owdenoise_filter_deps="gpl" pad_opencl_filter_deps="opencl" pan_filter_deps="swresample" @@ -3755,6 +3756,7 @@ stereo3d_filter_deps="gpl" subtitles_filter_deps="avformat avcodec libass" super2xsai_filter_deps="gpl" pixfmts_super2xsai_test_deps="super2xsai_filter" +textsub2video_filter_deps="avcodec libass" tinterlace_filter_deps="gpl" tinterlace_merge_test_deps="tinterlace_filter" tinterlace_pad_test_deps="tinterlace_filter" diff --git a/doc/filters.texi b/doc/filters.texi index 70a371ee0e..fa6fd57cdc 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -27480,6 +27480,119 @@ Overlay PGS subtitles ffmpeg -i "https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv" -filter_complex "[0:0][0:1]overlaygraphicsubs" output.mp4 @end example @end itemize + +@section overlaytextsubs + +Overlay text subtitles onto a video stream. + +This filter supersedes the classic @ref{subtitles} filter opposed to which it does no longer require to open and access the source stream separately, which is often causing problems or doesn't even work for non-local or slow sources. + +Inputs: +@itemize +@item 0: Video [YUV420P, YUV422P, YUV444P, ARGB, RGBA, ABGR, BGRA, RGB24, BGR24] +@item 1: Subtitles [TEXT] +@end itemize + +Outputs: +@itemize +@item 0: Video (same as input) +@end itemize + +It accepts the following parameters: + +@table @option + +@item alpha +Process alpha channel, by default alpha channel is untouched. + +@item fonts_dir +Set a directory path containing fonts that can be used by the filter. +These fonts will be used in addition to whatever the font provider uses. + +@item default_font_path +Path to a font file to be used as the default font. + +@item font_size +Set the default font size. + +@item fontconfig_file +Path to ASS fontconfig configuration file. + +@item force_style +Override default style or script info parameters of the subtitles. It accepts a +string containing ASS style format @code{KEY=VALUE} couples separated by ",". + +@item margin +Set the rendering margin in pixels. + +@item render_latest_only +For rendering, alway use the latest event only, which is covering the given point in time +@end table + +@subsection Examples + +@itemize +@item +Overlay ASS subtitles with animations: +@example +ffmpeg -i "http://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.mkv" -filter_complex "[0:v]overlaytextsubs" -map 0 -y out.mkv +@end example +@end itemize + +@section textsub2video + +Converts text subtitles to video frames. + +For overlaying text subtitles onto video frames it is recommended to use the overlaytextsubs filter. +The textsub2video is useful for for creating transparent text-frames when overlay is done via hw acceleration + +Inputs: +@itemize +@item 0: Subtitles [TEXT] +@end itemize + +Outputs: +@itemize +@item 0: Video [RGB32] +@end itemize + +It accepts the following parameters: + +@table @option + +@item rate, r +Set the framerate for updating overlay frames. +Normally, overlay frames will only be updated each time when the subtitles to display are changing. +In cases where subtitles include advanced features (like animation), this parameter determines the frequency by which the overlay frames should be updated. + +@item size, s +Set the output frame size. +Allows to override the size of output video frames. + +@item fonts_dir +Set a directory path containing fonts that can be used by the filter. +These fonts will be used in addition to whatever the font provider uses. + +@item default_font_path +Path to a font file to be used as the default font. + +@item font_size +Set the default font size. + +@item fontconfig_file +Path to ASS fontconfig configuration file. + +@item force_style +Override default style or script info parameters of the subtitles. It accepts a +string containing ASS style format @code{KEY=VALUE} couples separated by ",". + +@item margin +Set the rendering margin in pixels. + +@item render_latest_only +For rendering, alway use the latest event only, which is covering the given point in time. +@end table + @c man end SUBTITLE FILTERS @chapter Multimedia Filters diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 0dae913b16..232d3cc0f8 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -400,6 +400,7 @@ OBJS-$(CONFIG_OVERLAY_QSV_FILTER) += vf_overlay_qsv.o framesync.o OBJS-$(CONFIG_OVERLAY_VAAPI_FILTER) += vf_overlay_vaapi.o framesync.o vaapi_vpp.o OBJS-$(CONFIG_OVERLAY_VULKAN_FILTER) += vf_overlay_vulkan.o vulkan.o vulkan_filter.o OBJS-$(CONFIG_OVERLAYGRAPHICSUBS_FILTER) += vf_overlaygraphicsubs.o framesync.o +OBJS-$(CONFIG_OVERLAYTEXTSUBS_FILTER) += vf_overlaytextsubs.o OBJS-$(CONFIG_OWDENOISE_FILTER) += vf_owdenoise.o OBJS-$(CONFIG_PAD_FILTER) += vf_pad.o OBJS-$(CONFIG_PAD_OPENCL_FILTER) += vf_pad_opencl.o opencl.o opencl/pad.o @@ -493,6 +494,7 @@ OBJS-$(CONFIG_SWAPRECT_FILTER) += vf_swaprect.o OBJS-$(CONFIG_SWAPUV_FILTER) += vf_swapuv.o OBJS-$(CONFIG_TBLEND_FILTER) += vf_blend.o framesync.o OBJS-$(CONFIG_TELECINE_FILTER) += vf_telecine.o +OBJS-$(CONFIG_TEXTSUB2VIDEO_FILTER) += vf_overlaytextsubs.o OBJS-$(CONFIG_THISTOGRAM_FILTER) += vf_histogram.o OBJS-$(CONFIG_THRESHOLD_FILTER) += vf_threshold.o framesync.o OBJS-$(CONFIG_THUMBNAIL_FILTER) += vf_thumbnail.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 71896ce904..a04af0869a 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -375,6 +375,7 @@ extern const AVFilter ff_vf_overlay_vaapi; extern const AVFilter ff_vf_overlay_vulkan; extern const AVFilter ff_vf_overlay_cuda; extern const AVFilter ff_vf_overlaygraphicsubs; +extern const AVFilter ff_vf_overlaytextsubs; extern const AVFilter ff_vf_owdenoise; extern const AVFilter ff_vf_pad; extern const AVFilter ff_vf_pad_opencl; @@ -566,6 +567,7 @@ extern const AVFilter ff_avf_showvolume; extern const AVFilter ff_avf_showwaves; extern const AVFilter ff_avf_showwavespic; extern const AVFilter ff_svf_graphicsub2video; +extern const AVFilter ff_svf_textsub2video; extern const AVFilter ff_vaf_spectrumsynth; /* multimedia sources */ diff --git a/libavfilter/vf_overlaytextsubs.c b/libavfilter/vf_overlaytextsubs.c new file mode 100644 index 0000000000..1e43289bec --- /dev/null +++ b/libavfilter/vf_overlaytextsubs.c @@ -0,0 +1,680 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * overlay text subtitles on top of a video frame + */ + +#include "config_components.h" + +#include +#include "libavutil/ass_internal.h" +#include "libavutil/thread.h" + +#include "drawutils.h" +#include "filters.h" + +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" + +typedef struct TextSubsContext { + const AVClass *class; + AVMutex mutex; + int is_mutex_initialized; + + ASS_Library *library; + ASS_Renderer *renderer; + ASS_Track *track; + + char *default_font_path; + char *fonts_dir; + char *fc_file; + double font_size; + char *force_style; + char *language; + int margin; + int render_latest_only; + + int alpha; + FFDrawContext draw; + + int got_header; + int out_w, out_h; + AVRational frame_rate; + AVFrame *last_frame; + int need_frame; + int eof; +} TextSubsContext; + +/* libass supports a log level ranging from 0 to 7 */ +static const int ass_libavfilter_log_level_map[] = { + AV_LOG_QUIET, /* 0 */ + AV_LOG_PANIC, /* 1 */ + AV_LOG_FATAL, /* 2 */ + AV_LOG_ERROR, /* 3 */ + AV_LOG_WARNING, /* 4 */ + AV_LOG_INFO, /* 5 */ + AV_LOG_VERBOSE, /* 6 */ + AV_LOG_DEBUG, /* 7 */ +}; + +static void ass_log(int ass_level, const char *fmt, va_list args, void *ctx) +{ + const int ass_level_clip = av_clip(ass_level, 0, FF_ARRAY_ELEMS(ass_libavfilter_log_level_map) - 1); + const int level = ass_libavfilter_log_level_map[ass_level_clip]; + + av_vlog(ctx, level, fmt, args); + av_log(ctx, level, "\n"); +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + TextSubsContext *s = ctx->priv; + + if (s->track) + ass_free_track(s->track); + if (s->renderer) + ass_renderer_done(s->renderer); + if (s->library) + ass_library_done(s->library); + + s->track = NULL; + s->renderer = NULL; + s->library = NULL; + + if (s->is_mutex_initialized) { + ff_mutex_destroy(&s->mutex); + s->is_mutex_initialized = 0; + } + + av_frame_free(&s->last_frame); +} + +static int overlay_textsubs_query_formats(AVFilterContext *ctx) +{ + AVFilterFormats *formats; + AVFilterLink *inlink0 = ctx->inputs[0]; + AVFilterLink *inlink1 = ctx->inputs[1]; + AVFilterLink *outlink = ctx->outputs[0]; + static const enum AVSubtitleType subtitle_fmts[] = { AV_SUBTITLE_FMT_ASS, AV_SUBTITLE_FMT_NONE }; + int ret; + + /* set input0 and output0 video formats */ + formats = ff_draw_supported_pixel_formats(0); + if ((ret = ff_formats_ref(formats, &inlink0->outcfg.formats)) < 0) + return ret; + if ((ret = ff_formats_ref(formats, &outlink->incfg.formats)) < 0) + return ret; + + /* set input1 subtitle formats */ + formats = ff_make_format_list(subtitle_fmts); + if ((ret = ff_formats_ref(formats, &inlink1->outcfg.formats)) < 0) + return ret; + + return 0; +} + +static int config_output(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + + outlink->w = ctx->inputs[0]->w; + outlink->h = ctx->inputs[0]->h; + outlink->time_base = ctx->inputs[0]->time_base; + outlink->frame_rate = ctx->inputs[0]->frame_rate; + + return 0; +} + +static int config_input_main(AVFilterLink *inlink) +{ + AVFilterContext *ctx = inlink->dst; + TextSubsContext *s = inlink->dst->priv; + int ret; + + ret = ff_draw_init(&s->draw, inlink->format, s->alpha ? FF_DRAW_PROCESS_ALPHA : 0); + if (ret < 0) { + av_log(ctx, AV_LOG_ERROR, "Could not initialize ff_draw.\n"); + return ret; + } + + ass_set_frame_size (s->renderer, inlink->w, inlink->h); + ass_set_pixel_aspect(s->renderer, av_q2d(inlink->sample_aspect_ratio)); + + av_log(ctx, AV_LOG_VERBOSE, "Subtitle screen: %dx%d\n\n\n\n", inlink->w, inlink->h); + + return 0; +} + +/* libass stores an RGBA color in the format RRGGBBTT, where TT is the transparency level */ +#define AR(c) ( (c)>>24) +#define AG(c) (((c)>>16)&0xFF) +#define AB(c) (((c)>>8) &0xFF) +#define AA(c) ((0xFF-(c)) &0xFF) + +static void overlay_ass_image(TextSubsContext *s, AVFrame *picref, + const ASS_Image *image) +{ + for (; image; image = image->next) { + uint8_t rgba_color[] = {AR(image->color), AG(image->color), AB(image->color), AA(image->color)}; + FFDrawColor color; + ff_draw_color(&s->draw, &color, rgba_color); + ff_blend_mask(&s->draw, &color, + picref->data, picref->linesize, + picref->width, picref->height, + image->bitmap, image->stride, image->w, image->h, + 3, 0, image->dst_x, image->dst_y); + } +} + +static void process_header(AVFilterContext *link, AVFrame *frame) +{ + TextSubsContext *s = link->priv; + ASS_Track *track = s->track; + ASS_Style *style; + int sid = 0; + + if (!track) + return; + + if (frame && frame->subtitle_header) { + char *subtitle_header = (char *)frame->subtitle_header->data; + ass_process_codec_private(s->track, subtitle_header, strlen(subtitle_header)); + } + else { + char* subtitle_header = avpriv_ass_get_subtitle_header_default(0); + if (!subtitle_header) + return; + + ass_process_codec_private(s->track, subtitle_header, strlen(subtitle_header)); + av_free(subtitle_header); + } + + if (s->language) + s->track->Language = av_strdup(s->language); + + if (!s->track->event_format) { + s->track->event_format = av_strdup("ReadOrder, Layer, Style, Name, MarginL, MarginR, MarginV, Effect, Text"); + } + + if (s->track->n_styles == 0) { + sid = ass_alloc_style(track); + style = &s->track->styles[sid]; + style->Name = av_strdup("Default"); + style->PrimaryColour = 0xffffff00; + style->SecondaryColour = 0x00ffff00; + style->OutlineColour = 0x00000000; + style->BackColour = 0x00000080; + style->Bold = 200; + style->ScaleX = 1.0; + style->ScaleY = 1.0; + style->Spacing = 0; + style->BorderStyle = 1; + style->Outline = 2; + style->Shadow = 3; + style->Alignment = 2; + } + else + style = &s->track->styles[sid]; + + style->FontSize = s->font_size; + style->MarginL = style->MarginR = style->MarginV = s->margin; + + track->default_style = sid; + + s->got_header = 1; +} + +static int filter_video_frame(AVFilterLink *inlink, AVFrame *frame) +{ + AVFilterContext *ctx = inlink->dst; + TextSubsContext *s = ctx->priv; + int detect_change = 0; + ASS_Image *image; + + + int64_t time_ms = (int64_t)((double)frame->pts * av_q2d(inlink->time_base) * 1000); + int64_t time_ms1 = (int64_t)((double)ctx->inputs[1]->current_pts * av_q2d(ctx->inputs[1]->time_base) * 1000); + + if (time_ms1 < time_ms + 1000) + ff_request_frame(ctx->inputs[1]); + + av_log(ctx, AV_LOG_DEBUG, "filter_video_frame - video: %"PRId64"ms sub: %"PRId64"ms rel %d\n", time_ms, time_ms1, (time_ms1 < time_ms)); + + ff_mutex_lock(&s->mutex); + image = ass_render_frame(s->renderer, s->track, time_ms, &detect_change); + ff_mutex_unlock(&s->mutex); + + if (detect_change) + av_log(ctx, AV_LOG_DEBUG, "Change happened at time ms:%"PRId64"\n", time_ms); + + overlay_ass_image(s, frame, image); + + return ff_filter_frame(ctx->outputs[0], frame); +} + +static int filter_subtitle_frame(AVFilterLink *inlink, AVFrame *frame) +{ + AVFilterContext *ctx = inlink->dst; + TextSubsContext *s = ctx->priv; + const int64_t start_time = av_rescale_q(frame->subtitle_timing.start_pts, AV_TIME_BASE_Q, av_make_q(1, 1000)); + const int64_t duration = av_rescale_q(frame->subtitle_timing.duration, AV_TIME_BASE_Q, av_make_q(1, 1000)); + const int64_t frame_time = (int64_t)((double)frame->pts * av_q2d(inlink->time_base) * 1000); + + // Postpone header processing until we receive a frame with content + if (!s->got_header && frame->num_subtitle_areas > 0) + process_header(ctx, frame); + + av_log(ctx, AV_LOG_DEBUG, "filter_subtitle_frame dur: %"PRId64"ms frame: %"PRId64"ms sub: %"PRId64"ms repeat_sub %d\n", duration, frame_time, start_time, frame->repeat_sub); + + if (frame->repeat_sub) + goto exit; + + ff_mutex_lock(&s->mutex); + + if (s->render_latest_only && s->track->n_events > 0) { + const int64_t previous_start_time = s->track->events[s->track->n_events - 1].Start; + const int64_t diff = start_time - previous_start_time; + for (int i = s->track->n_events - 1; i >= 0; i--) { + if (previous_start_time != s->track->events[i].Start) + break; + + if (s->track->events[i].Duration > diff) + s->track->events[i].Duration = diff; + } + } + + for (unsigned i = 0; i < frame->num_subtitle_areas; i++) { + char *ass_line = frame->subtitle_areas[i]->ass; + if (!ass_line) + continue; + + ass_process_chunk(s->track, ass_line, strlen(ass_line), start_time, duration); + } + + ff_mutex_unlock(&s->mutex); + +exit: + av_frame_free(&frame); + return 0; +} + +static av_cold int init(AVFilterContext *ctx) +{ + int ret; + TextSubsContext *s = ctx->priv; + + s->library = ass_library_init(); + + if (!s->library) { + av_log(ctx, AV_LOG_ERROR, "Could not initialize libass.\n"); + return AVERROR(EINVAL); + } + + ass_set_message_cb(s->library, ass_log, ctx); + + /* Initialize fonts */ + if (s->fonts_dir) + ass_set_fonts_dir(s->library, s->fonts_dir); + + ass_set_extract_fonts(s->library, 1); + + s->renderer = ass_renderer_init(s->library); + if (!s->renderer) { + av_log(ctx, AV_LOG_ERROR, "Could not initialize libass renderer.\n"); + return AVERROR(EINVAL); + } + + s->track = ass_new_track(s->library); + if (!s->track) { + av_log(ctx, AV_LOG_ERROR, "ass_new_track() failed!\n"); + return AVERROR(EINVAL); + } + + ass_set_check_readorder(s->track, 0); + + ass_set_fonts(s->renderer, s->default_font_path, NULL, 1, s->fc_file, 1); + + if (s->force_style) { + char **list = NULL; + char *temp = NULL; + char *ptr = av_strtok(s->force_style, ",", &temp); + int i = 0; + while (ptr) { + av_dynarray_add(&list, &i, ptr); + if (!list) { + return AVERROR(ENOMEM); + } + ptr = av_strtok(NULL, ",", &temp); + } + av_dynarray_add(&list, &i, NULL); + if (!list) { + return AVERROR(ENOMEM); + } + ass_set_style_overrides(s->library, list); + av_free(list); + } + + ret = ff_mutex_init(&s->mutex, NULL); + if (ret) { + av_log(ctx, AV_LOG_ERROR, "mutex initialiuzation failed! Error code: %d\n", ret); + return AVERROR(EINVAL); + } + + s->is_mutex_initialized = 1; + + return ret; +} + +static int textsub2video_query_formats(AVFilterContext *ctx) +{ + AVFilterFormats *formats; + AVFilterLink *inlink = ctx->inputs[0]; + AVFilterLink *outlink = ctx->outputs[0]; + static const enum AVSubtitleType subtitle_fmts[] = { AV_SUBTITLE_FMT_ASS, AV_SUBTITLE_FMT_NONE }; + int ret; + + /* set input0 subtitle format */ + formats = ff_make_format_list(subtitle_fmts); + if ((ret = ff_formats_ref(formats, &inlink->outcfg.formats)) < 0) + return ret; + + /* set output0 video format */ + formats = ff_draw_supported_pixel_formats(AV_PIX_FMT_FLAG_ALPHA); + if ((ret = ff_formats_ref(formats, &outlink->incfg.formats)) < 0) + return ret; + + return 0; +} + +static int textsub2video_config_input(AVFilterLink *inlink) +{ + AVFilterContext *ctx = inlink->dst; + TextSubsContext *s = ctx->priv; + + if (s->out_w <= 0 || s->out_h <= 0) { + s->out_w = inlink->w; + s->out_h = inlink->h; + } + + return 0; +} + +static int textsub2video_config_output(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + TextSubsContext *s = ctx->priv; + int ret; + + ret = ff_draw_init(&s->draw, outlink->format, FF_DRAW_PROCESS_ALPHA); + if (ret < 0) { + av_log(ctx, AV_LOG_ERROR, "Could not initialize ff_draw.\n"); + return ret; + } + + if (s->out_w <= 0 || s->out_h <= 0) { + av_log(ctx, AV_LOG_ERROR, "No output image size set.\n"); + return AVERROR(EINVAL); + } + + ass_set_frame_size (s->renderer, s->out_w, s->out_h); + + outlink->w = s->out_w; + outlink->h = s->out_h; + outlink->sample_aspect_ratio = (AVRational){1,1}; + outlink->frame_rate = s->frame_rate; + + return 0; +} + +static int textsub2video_request_frame(AVFilterLink *outlink) +{ + TextSubsContext *s = outlink->src->priv; + AVFilterLink *inlink = outlink->src->inputs[0]; + int64_t last_pts = outlink->current_pts; + int64_t next_pts, time_ms; + int i, detect_change = 0, status; + AVFrame *out; + ASS_Image *image; + + status = ff_outlink_get_status(inlink); + if (status == AVERROR_EOF) + return AVERROR_EOF; + + if (s->eof) + return AVERROR_EOF; + + if (inlink->current_pts == AV_NOPTS_VALUE) { // || outlink->current_pts > inlink->current_pts) { + int ret = ff_request_frame(inlink); + if (ret == AVERROR_EOF) { + s->eof = 1; + } + + if (ret != 0) + av_log(outlink->src, AV_LOG_DEBUG, "ff_request_frame returned: %d\n", ret); + + s->need_frame = 1; + return 0; + } + + if (last_pts == AV_NOPTS_VALUE) + next_pts = last_pts = inlink->current_pts * av_q2d(inlink->time_base) / av_q2d(outlink->time_base); + else + next_pts = last_pts + (int64_t)(1.0 / av_q2d(outlink->frame_rate) / av_q2d(outlink->time_base)); + + time_ms = (int64_t)((double)next_pts * av_q2d(outlink->time_base) * 1000); + + ff_mutex_lock(&s->mutex); + image = ass_render_frame(s->renderer, s->track, time_ms, &detect_change); + ff_mutex_unlock(&s->mutex); + + if (detect_change) + av_log(outlink->src, AV_LOG_VERBOSE, "Change happened at time ms:%"PRId64" pts:%"PRId64"\n", time_ms, next_pts); + else if (s->last_frame) { + out = av_frame_clone(s->last_frame); + if (!out) + return AVERROR(ENOMEM); + + out->pts = out->pkt_dts = out->best_effort_timestamp = next_pts; + return ff_filter_frame(outlink, out); + } + + out = ff_get_video_buffer(outlink, outlink->w, outlink->h); + if (!out) + return AVERROR(ENOMEM); + + for (i = 0; i < AV_NUM_DATA_POINTERS; i++) { + if (out->buf[i] && i != 1) + memset(out->buf[i]->data, 0, out->buf[i]->size); + } + + out->pts = out->pkt_dts = out->best_effort_timestamp = next_pts; + + if (image) + overlay_ass_image(s, out, image); + + av_frame_free(&s->last_frame); + + s->last_frame = av_frame_clone(out); + + return ff_filter_frame(outlink, out); +} + +static int textsub2video_filter_frame(AVFilterLink *inlink, AVFrame *frame) +{ + AVFilterContext *ctx = inlink->dst; + TextSubsContext *s = ctx->priv; + const int64_t start_time = av_rescale_q(frame->subtitle_timing.start_pts, AV_TIME_BASE_Q, av_make_q(1, 1000)); + const int64_t duration = av_rescale_q(frame->subtitle_timing.duration, AV_TIME_BASE_Q, av_make_q(1, 1000)); + + av_log(ctx, AV_LOG_VERBOSE, "textsub2video_filter_frame num_subtitle_rects: %d, start_time_ms: %"PRId64"\n", frame->num_subtitle_areas, start_time); + + if (!s->got_header && frame->num_subtitle_areas > 0) + process_header(ctx, frame); + + if (frame->repeat_sub) + goto exit; + + ff_mutex_lock(&s->mutex); + + if (s->render_latest_only && s->track->n_events > 0) { + const int64_t previous_start_time = s->track->events[s->track->n_events - 1].Start; + const int64_t diff = start_time - previous_start_time; + for (int i = s->track->n_events - 1; i >= 0; i--) { + if (previous_start_time != s->track->events[i].Start) + break; + + if (s->track->events[i].Duration > diff) + s->track->events[i].Duration = diff; + + } + } + + for (unsigned i = 0; i < frame->num_subtitle_areas; i++) { + char *ass_line = frame->subtitle_areas[i]->ass; + if (!ass_line) + continue; + + ass_process_chunk(s->track, ass_line, strlen(ass_line), start_time, duration); + } + + + ff_mutex_unlock(&s->mutex); + +exit: + av_frame_free(&frame); + + if (s->need_frame) { + s->need_frame = 0; + return textsub2video_request_frame(ctx->outputs[0]); + } + + return 0; +} + +#define OFFSET(x) offsetof(TextSubsContext, x) +#define FLAGS (AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_FILTERING_PARAM) + +static const AVOption overlaytextsubs_options[] = { + {"alpha", "enable processing of alpha channel", OFFSET(alpha), AV_OPT_TYPE_BOOL, {.i64 = 0 }, 0, 1, .flags = FLAGS}, + {"font_size", "default font size", OFFSET(font_size), AV_OPT_TYPE_DOUBLE, {.dbl = 18.0}, 0.0, 100.0, .flags = FLAGS}, + {"force_style", "force subtitle style", OFFSET(force_style), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, .flags = FLAGS}, + {"margin", "default margin", OFFSET(margin), AV_OPT_TYPE_INT, {.i64 = 20 }, 0, INT_MAX, .flags = FLAGS}, + {"default_font_path", "path to default font", OFFSET(default_font_path), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, .flags = FLAGS}, + {"fonts_dir", "directory to scan for fonts", OFFSET(fonts_dir), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, .flags = FLAGS}, + {"fontsdir", "directory to scan for fonts", OFFSET(fonts_dir), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, .flags = FLAGS}, + {"fontconfig_file", "fontconfig file to load", OFFSET(fc_file), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, .flags = FLAGS}, + {"language", "default language", OFFSET(language), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, .flags = FLAGS}, + {"render_latest_only", "newest sub event for each time", OFFSET(render_latest_only), AV_OPT_TYPE_BOOL, {.i64 = 0 }, 0, 1, .flags = FLAGS}, + { .name = NULL } +}; + +static const AVOption textsub2video_options[] = { + {"rate", "set frame rate", OFFSET(frame_rate), AV_OPT_TYPE_VIDEO_RATE, {.str="8"}, 0, INT_MAX, .flags = FLAGS}, + {"r", "set frame rate", OFFSET(frame_rate), AV_OPT_TYPE_VIDEO_RATE, {.str="8"}, 0, INT_MAX, .flags = FLAGS}, + {"size", "set video size", OFFSET(out_w), AV_OPT_TYPE_IMAGE_SIZE, {.str = NULL}, 0, 0, .flags = FLAGS}, + {"s", "set video size", OFFSET(out_w), AV_OPT_TYPE_IMAGE_SIZE, {.str = NULL}, 0, 0, .flags = FLAGS}, + {"font_size", "default font size", OFFSET(font_size), AV_OPT_TYPE_DOUBLE, {.dbl = 18.0}, 0.0, 100.0, .flags = FLAGS}, + {"force_style", "force subtitle style", OFFSET(force_style), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, .flags = FLAGS}, + {"margin", "default margin", OFFSET(margin), AV_OPT_TYPE_INT, {.i64 = 20 }, 0, INT_MAX, .flags = FLAGS}, + {"default_font_path", "path to default font", OFFSET(default_font_path), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, .flags = FLAGS}, + {"fonts_dir", "directory to scan for fonts", OFFSET(fonts_dir), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, .flags = FLAGS}, + {"fontsdir", "directory to scan for fonts", OFFSET(fonts_dir), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, .flags = FLAGS}, + {"fontconfig_file", "fontconfig file to load", OFFSET(fc_file), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, .flags = FLAGS}, + {"language", "default language", OFFSET(language), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, .flags = FLAGS}, + {"render_latest_only", "newest sub event for each time", OFFSET(render_latest_only), AV_OPT_TYPE_BOOL, {.i64 = 0 }, 0, 1, .flags = FLAGS}, + { .name = NULL } +}; + +#if CONFIG_OVERLAYTEXTSUBS_FILTER + +AVFILTER_DEFINE_CLASS(overlaytextsubs); + +static const AVFilterPad overlaytextsubs_inputs[] = { + { + .name = "main", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_input_main, + .flags = AVFILTERPAD_FLAG_NEEDS_WRITABLE, + .filter_frame = filter_video_frame, + }, + { + .name = "overlay", + .type = AVMEDIA_TYPE_SUBTITLE, + .filter_frame = filter_subtitle_frame, + }, +}; + +static const AVFilterPad overlaytextsubs_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_output, + }, +}; + +const AVFilter ff_vf_overlaytextsubs = { + .name = "overlaytextsubs", + .description = NULL_IF_CONFIG_SMALL("Overlay textual subtitles on top of the input."), + .init = init, + .uninit = uninit, + .priv_size = sizeof(TextSubsContext), + .priv_class = &overlaytextsubs_class, + FILTER_INPUTS(overlaytextsubs_inputs), + FILTER_OUTPUTS(overlaytextsubs_outputs), + FILTER_QUERY_FUNC(overlay_textsubs_query_formats), +}; +#endif + +#if CONFIG_TEXTSUB2VIDEO_FILTER + +AVFILTER_DEFINE_CLASS(textsub2video); + +static const AVFilterPad textsub2video_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .config_props = textsub2video_config_input, + .filter_frame = textsub2video_filter_frame, + }, +}; + +static const AVFilterPad textsub2video_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = textsub2video_config_output, + .request_frame = textsub2video_request_frame, + }, +}; + +const AVFilter ff_svf_textsub2video = { + .name = "textsub2video", + .description = NULL_IF_CONFIG_SMALL("Convert textual subtitles to video frames"), + .init = init, + .uninit = uninit, + .priv_size = sizeof(TextSubsContext), + .priv_class = &textsub2video_class, + FILTER_INPUTS(textsub2video_inputs), + FILTER_OUTPUTS(textsub2video_outputs), + FILTER_QUERY_FUNC(textsub2video_query_formats), +}; +#endif From patchwork Tue Oct 25 09:13:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 38989 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2685607pzb; Tue, 25 Oct 2022 02:16:16 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4b9bkx9IF/uEosN8kmusYX90ggJJX6DlHzpWPpZrKtMSBvU4fqZBOFWD/yRbgPnWTNsWI4 X-Received: by 2002:aa7:df04:0:b0:45b:cb0d:856a with SMTP id c4-20020aa7df04000000b0045bcb0d856amr35687660edy.180.1666689376350; Tue, 25 Oct 2022 02:16:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666689376; cv=none; d=google.com; s=arc-20160816; b=f/vrsDWgPXlE79wfwUnGfFR/qvZJIXoEE+e9SvWyOXCT5V9PlavV3gOaIszh69J5tH raETPFMoaJXcxOCtFmEkj0o/zwb83yVJY+HLrxuuQMCaO0CbmEfivzh1lzTtuOPaIT/l 1W5ouBhc+NocZcP5Te9D+cfm/Aay20yrsPmDEZIdDRzcqP1jDIKFCfcXOVgQQ/FQDSnX ueW1ktYoKNtrsMwlqrmcNejERnz8XF6Sr6PRhlW8+uOxgXHsBVFVu0uG1ifH3GjFXYeR 0oNWzTI61xZUzqBtnxpbC4+2QczwxBlYiw1OmdmhOu2dDlogxzW+AReCX/wWE7PSjmHQ s2uw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:references :in-reply-to:message-id:from:dkim-signature:delivered-to; bh=+/z51XtCpecTZCajtZADeRWxwOCuC1+x6Q5nlI0wcj4=; b=B+5eB102rU6K2S1uFT5rOb4VU917g++6CnPYyIgFuIN4lkTZpi5TQPRZwgezYKRBWc bJzu5hfXWnvLp9dz+iOI0STvtFJsu4AEmaQcAsSpEFSOLI6795rYIfo9Vr869ariFgfx RdQSrQ6Tr5Zj3qlFTdNpM9itjoakJreU5qmMrJBgR4GONqI3nbFZ16N6pYVPgz/fxjGE wY0w5hUf76j+bIAmEJ/jjde8tJRZo9eogHdyC2tXDO0a5srAaw0vdJuP7YQYpw04yvl+ k7ITFpWGLqpPxAbYM+SleyR0J0NZ5fz8R7u9ggxu0gnE10QcHSxetTQZ/CfRrv0C8q94 HLZA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=TNpFIsDY; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id hp5-20020a1709073e0500b00799074c80c7si2581987ejc.672.2022.10.25.02.16.15; Tue, 25 Oct 2022 02:16:16 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=TNpFIsDY; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8AB5C68BD51; Tue, 25 Oct 2022 12:14:13 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f50.google.com (mail-pj1-f50.google.com [209.85.216.50]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1D58668BCBF for ; Tue, 25 Oct 2022 12:14:08 +0300 (EEST) Received: by mail-pj1-f50.google.com with SMTP id ez6so10205004pjb.1 for ; Tue, 25 Oct 2022 02:14:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=Xu643Do27z44rGh1wB3KblUYi4v435td9SJ7hMC5b9k=; b=TNpFIsDY4ZWLAxJXIGLAk9DNdt6guPDZHDmyqwkUVZxfChfCqTvNauU7mmIa3U2Dpb 5o3o+rZIbnpjvAb2d8XmOJQboopGQGEz+1UNIgmeNDA1gUJ3bgZfDBSF3M88irBY1OPs Ye4hk9QfPjEAf4HJ+llhrqv7h+oeajl/DnubsE91vAe8elSA57XMFwF2qYY96B7C6eAW 3bxRJ3n4thmfmapkbIhCKsI9JIT34sDQt1LJtmRHJQZVlbxodyW1sd1OcXys4zwxZpFX H/x8MRxpjWyv/UR2N5w4ykHV2f1W2EiqMTZXtip6nGpgQQx18UsOqzmDmJgRQ/C3b82E Y4cw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=Xu643Do27z44rGh1wB3KblUYi4v435td9SJ7hMC5b9k=; b=oY363ognx6/FidMuFAD4t7OnUadFzmgXoQHE5p7SuS4f2ZCW3M7H31cWfLWF3UmcHb GyFRGVwdmqw8/V6d+1mTbs5/Hk7FG+MGh1HuvgQbIoekQLqK+RQW92+JSsl1RdAj54S/ oBDSLYEguwloOZbo96bKTpl5U4cnFq5vfNEBbnTaBqlmbMU95TQAzi+QlFf7EdMVR5N+ kmzyllbU4SxFmeCICC2G1NrgDDrfim9HLJ56/RYGUFpemQfSiZoDs5JvYU6gD0JtAvEn /G/tKZY8mfCFkgFjoAQZmrPZufUnK8BChkrblfM/BvFR7CoUmBq9nxttgL+7zkxQn3e2 kJ7w== X-Gm-Message-State: ACrzQf1TZSJz68ANgo/gECbb+B5LVzWJAEJs8cY2DqzogWGGq4AmkKiR +R2uIFrdfXM9yXyprrUMoXzXALJGkMg= X-Received: by 2002:a17:902:b907:b0:178:9d11:c978 with SMTP id bf7-20020a170902b90700b001789d11c978mr37775089plb.90.1666689247022; Tue, 25 Oct 2022 02:14:07 -0700 (PDT) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id cp9-20020a170902e78900b001865e69b4d7sm862900plb.264.2022.10.25.02.14.06 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Oct 2022 02:14:06 -0700 (PDT) From: softworkz X-Google-Original-From: softworkz Message-Id: <15a8984fb7de42cecf341d5e184ee47ffc1f9681.1666689226.git.ffmpegagent@gmail.com> In-Reply-To: References: Date: Tue, 25 Oct 2022 09:13:36 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v9 15/25] avfilter/textmod: Add textmod, censor and show_speaker filters X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: softworkz Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 7kU+FnqSnoxm From: softworkz - textmod {S -> S) Modify subtitle text in a number of ways - censor {S -> S) Censor subtitles using a word list - show_speaker {S -> S) Prepend speaker names from ASS subtitles to the visible text lines Signed-off-by: softworkz --- doc/filters.texi | 206 ++++++++++++ libavfilter/Makefile | 5 + libavfilter/allfilters.c | 5 + libavfilter/sf_textmod.c | 710 +++++++++++++++++++++++++++++++++++++++ 4 files changed, 926 insertions(+) create mode 100644 libavfilter/sf_textmod.c diff --git a/doc/filters.texi b/doc/filters.texi index fa6fd57cdc..1f7989e92f 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -27372,6 +27372,145 @@ existing filters using @code{--disable-filters}. Below is a description of the currently available subtitle filters. + +@section censor + +Censor selected words in text subtitles. + +Inputs: +@itemize +@item 0: Subtitles[TEXT] +@end itemize + +Outputs: +@itemize +@item 0: Subtitles[TEXT] +@end itemize + +It accepts the following parameters: + +@table @option +@item mode +The censoring mode to apply. + +Supported censoring modes are: + +@table @var +@item 0, keep_first_last +Replace all characters with the 'censor_char' except the first and the last character of a word. +For words with less than 4 characters, the last character will be replaced as well. +For words with less than 3 characters, the first character will be replaced as well. +@item 1, keep_first +Replace all characters with the 'censor_char' except the first character of a word. +For words with less than 3 characters, the first character will be replaced as well. +@item 2, all +Replace all characters with the 'censor_char'. +@end table + +@item words +A list of words to censor, separated by 'separator'. + +@item words_file +Specify a file from which to load the contents for the 'words' parameter. + +@item censor_char +Single character used as replacement for censoring. + +@item separator +Delimiter character for words. Used with replace_words and remove_words- Must be a single character. +The default is '.'. + +@end table + +@subsection Examples + +@itemize +@item +Censor a few given words with a pound character. +@example +ffmpeg -i "http://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.mkv" -filter_complex "[0:1]censor=words='diss,louder,hope,beam,word':censor_char='#'" -map 0 -y output.mkv +@end example +@end itemize + + +@section textmod + +Modify subtitle text in a number of ways. + +Inputs: +@itemize +@item 0: Subtitles[TEXT] +@end itemize + +Outputs: +@itemize +@item 0: Subtitles[TEXT] +@end itemize + +It accepts the following parameters: + +@table @option +@item mode +The kind of text modification to apply + +Supported operation modes are: + +@table @var +@item 0, leet +Convert subtitle text to 'leet speak'. It's primarily useful for testing as the modification will be visible with almost all text lines. +@item 1, to_upper +Change all text to upper case. Might improve readability. +@item 2, to_lower +Change all text to lower case. +@item 3, replace_chars +Replace one or more characters. Requires the find and replace parameters to be specified. +Both need to be equal in length. +The first char in find is replaced by the first char in replace, same for all subsequent chars. +@item 4, remove_chars +Remove certain characters. Requires the find parameter to be specified. +All chars in the find parameter string will be removed from all subtitle text. +@item 5, replace_words +Replace one or more words. Requires the find and replace parameters to be specified. Multiple words must be separated by the delimiter char specified vie the separator parameter (default: ','). +The number of words in the find and replace parameters needs to be equal. +The first word in find is replaced by the first word in replace, same for all subsequent words +@item 6, remove_words +Remove certain words. Requires the find parameter to be specified. Multiple words must be separated by the delimiter char specified vie the separator parameter (default: ','). +All words in the find parameter string will be removed from all subtitle text. +@end table + +@item find +Required for replace_chars, remove_chars, replace_words and remove_words. + +@item find_file +Specify a file from which to load the contents for the 'find' parameter. + +@item replace +Required for replace_chars and replace_words. + +@item replace_file +Specify a file from which to load the contents for the 'replace' parameter. + +@item separator +Delimiter character for words. Used with replace_words and remove_words- Must be a single character. +The default is '.'. + +@end table + +@subsection Examples + +@itemize +@item +Change all characters to upper case while keeping all styles and animations: +@example +ffmpeg -i "https://streams.videolan.org/ffmpeg/mkv_subtitles.mkv" -filter_complex "[0:s]textmod=mode=to_upper" -map 0 -y out.mkv +@end example +@item +Remove a set of symbol characters for am improved and smoother visual apperance: +@example +ffmpeg -i "https://streams.videolan.org/ffmpeg/mkv_subtitles.mkv" -filter_complex "[0:s]textmod=mode=remove_chars:find='$&#@*§'" -map 0 -y out.mkv +@end example +@end itemize + @section graphicsub2video Renders graphic subtitles as video frames. @@ -27539,6 +27678,73 @@ ffmpeg -i "http://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex. @end example @end itemize +@section showspeaker + +Prepend speaker names to subtitle lines (when available). + +Subtitles in ASS/SSA format are often including the names of the persons +or character for each subtitle line. The showspeaker filter adds those names +to the actual subtitle text to make it visible on playback. + +Inputs: +@itemize +@item 0: Subtitles[TEXT] +@end itemize + +Outputs: +@itemize +@item 0: Subtitles[TEXT] +@end itemize + +It accepts the following parameters: + +@table @option +@item format +The format for prepending speaker names. Default is 'square_brackets'. + +Supported operation modes are: + +@table @var +@item 0, square_brackets +Enclose the speaker name in square brackets, followed by space ('[speaker] text'). +@item 1, round_brackets +Enclose the speaker name in round brackets, followed by space ('(speaker) text'). +@item 2, colon +Separate the speaker name with a colon and space ('speaker: text'). +@item 3, plain +Separate the speaker name with a space only ('speaker text'). +@end table + +@item line_break +Set thís parameter to insert a line break between speaker name and text instead of the space character. + +@item style +Allows to set a specific style for the speaker name text. + +This can be either a named style that exists in the ass subtitle header script (e.g. 'Default') or an ass style override code. +Example: @{\\c&HDD0000&\\be1\\i1\\bord10@} +This sets the color to blue, enables edge blurring, italic font and a border of size 10. + +The behavior is as follows: + +- When the style parameter is not provided, the filter will find the first position in the event string that is actual text. + The speaker name will be inserted at this position. This allows to have the speaker name shown in the same style like the + regular text, in case the string would start with a sequence of style codes. +- When the style parameter is provided, everything will be prepended to the original text: + Style Code or Style name >> Speaker Name >> Style Reset Code >> Original Text + +@end table + +@subsection Examples + +@itemize +@item +Prepend speaker names with blue text, smooth edges and blend/overlay that onto the video. +@example +ffmpeg -i INPUT -filter_complex "showspeaker=format=colon:style='@{\\c&HDD0000&\\be1@}',[0:v]overlaytextsubs" +@end example +@end itemize + @section textsub2video Converts text subtitles to video frames. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 232d3cc0f8..5d82b3e8c4 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -587,6 +587,11 @@ OBJS-$(CONFIG_YUVTESTSRC_FILTER) += vsrc_testsrc.o OBJS-$(CONFIG_NULLSINK_FILTER) += vsink_nullsink.o +# subtitle filters +OBJS-$(CONFIG_CENSOR_FILTER) += sf_textmod.o +OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o +OBJS-$(CONFIG_TEXTMOD_FILTER) += sf_textmod.o + # multimedia filters OBJS-$(CONFIG_A3DSCOPE_FILTER) += avf_a3dscope.o OBJS-$(CONFIG_ABITSCOPE_FILTER) += avf_abitscope.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index a04af0869a..d7f24ab7db 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -575,6 +575,11 @@ extern const AVFilter ff_avsrc_avsynctest; extern const AVFilter ff_avsrc_amovie; extern const AVFilter ff_avsrc_movie; +/* subtitle filters */ +extern const AVFilter ff_sf_censor; +extern const AVFilter ff_sf_showspeaker; +extern const AVFilter ff_sf_textmod; + /* those filters are part of public or internal API, * they are formatted to not be found by the grep * as they are manually added again (due to their 'names' diff --git a/libavfilter/sf_textmod.c b/libavfilter/sf_textmod.c new file mode 100644 index 0000000000..9651e07c65 --- /dev/null +++ b/libavfilter/sf_textmod.c @@ -0,0 +1,710 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * text subtitle filter which allows to modify subtitle text in several ways + */ + +#include + +#include "libavutil/opt.h" +#include "internal.h" +#include "libavutil/ass_split_internal.h" +#include "libavutil/bprint.h" +#include "libavutil/file.h" + +static const char* leet_src = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"; +static const char* leet_dst = "abcd3f6#1jklmn0pq257uvwxyzAB(D3F6#1JKLMN0PQ257UVWXYZ"; + +enum TextModFilterType { + TM_TEXTMOD, + TM_CENSOR, + TM_SHOW_SPEAKER, +}; + +enum TextModOperation { + OP_LEET, + OP_TO_UPPER, + OP_TO_LOWER, + OP_REPLACE_CHARS, + OP_REMOVE_CHARS, + OP_REPLACE_WORDS, + OP_REMOVE_WORDS, + NB_OPS, +}; + +enum CensorMode { + CM_KEEP_FIRST_LAST, + CM_KEEP_FIRST, + CM_ALL, +}; + +enum ShowSpeakerMode { + SM_SQUARE_BRACKETS, + SM_ROUND_BRACKETS, + SM_COLON, + SM_PLAIN, +}; + +typedef struct TextModContext { + const AVClass *class; + enum AVSubtitleType format; + enum TextModFilterType filter_type; + enum TextModOperation operation; + enum CensorMode censor_mode; + enum ShowSpeakerMode speaker_mode; + char *find; + char *find_file; + char *style; + char *replace; + char *replace_file; + char *separator; + char *censor_char; + char **find_list; + int line_break; + int nb_find_list; + char **replace_list; + int nb_replace_list; +} TextModContext; + +static char **split_string(char *source, int *nb_elems, const char *delim) +{ + char **list = NULL; + char *temp = NULL; + char *ptr = av_strtok(source, delim, &temp); + + while (ptr) { + if (strlen(ptr)) { + av_dynarray_add(&list, nb_elems, ptr); + if (!list) + return NULL; + } + + ptr = av_strtok(NULL, delim, &temp); + } + + if (!list) + return NULL; + + for (int i = 0; i < *nb_elems; i++) { + list[i] = av_strdup(list[i]); + if (!list[i]) { + for (int n = 0; n < i; n++) + av_free(list[n]); + av_free(list); + return NULL; + } + } + + return list; +} + +static int load_text_from_file(AVFilterContext *ctx, const char *file_name, char **text, char separator) +{ + int err; + uint8_t *textbuf; + char *tmp; + size_t textbuf_size; + int offset = 0; + + if ((err = av_file_map(file_name, &textbuf, &textbuf_size, 0, ctx)) < 0) { + av_log(ctx, AV_LOG_ERROR, "The text file '%s' could not be read or is empty\n", file_name); + return err; + } + + if (textbuf_size > 1 && + (textbuf[0] == 0xFF && textbuf[1] == 0xFE + || textbuf[0] == 0xFE && textbuf[1] == 0xFF)) { + av_log(ctx, AV_LOG_ERROR, "UTF text files are not supported. File: %s\n", file_name); + return AVERROR(EINVAL); + } + + if (textbuf_size > 2 && textbuf[0] == 0xEF && textbuf[1] == 0xBB && textbuf[2] == 0xBF) + offset = 3; // UTF-8 + + if (textbuf_size > SIZE_MAX - 1 || !((tmp = av_strndup((char *)textbuf + offset, textbuf_size - offset)))) { + av_file_unmap(textbuf, textbuf_size); + return AVERROR(ENOMEM); + } + + av_file_unmap(textbuf, textbuf_size); + + for (size_t i = 0; i < strlen(tmp); i++) { + switch (tmp[i]) { + case '\n': + case '\r': + case '\f': + case '\v': + tmp[i] = separator; + } + } + + *text = tmp; + + return 0; +} + +static int load_files(AVFilterContext *ctx) +{ + TextModContext *s = ctx->priv; + int ret; + + if (!s->separator || strlen(s->separator) != 1) { + av_log(ctx, AV_LOG_ERROR, "A single character needs to be specified for the separator parameter.\n"); + return AVERROR(EINVAL); + } + + if (s->find_file && strlen(s->find_file)) { + ret = load_text_from_file(ctx, s->find_file, &s->find, s->separator[0]); + if (ret < 0 ) + return ret; + } + + if (s->replace_file && strlen(s->replace_file)) { + ret = load_text_from_file(ctx, s->replace_file, &s->replace, s->separator[0]); + if (ret < 0 ) + return ret; + } + + return 0; +} + +static int init_censor(AVFilterContext *ctx) +{ + TextModContext *s = ctx->priv; + int ret; + + s->filter_type = TM_CENSOR; + s->operation = OP_REPLACE_WORDS; + + ret = load_files(ctx); + if (ret < 0 ) + return ret; + + if (!s->find || !strlen(s->find)) { + av_log(ctx, AV_LOG_ERROR, "Either the 'words' or the 'words_file' parameter needs to be specified\n"); + return AVERROR(EINVAL); + } + + if (!s->censor_char || strlen(s->censor_char) != 1) { + av_log(ctx, AV_LOG_ERROR, "A single character needs to be specified for the censor_char parameter\n"); + return AVERROR(EINVAL); + } + + s->find_list = split_string(s->find, &s->nb_find_list, s->separator); + if (!s->find_list) + return AVERROR(ENOMEM); + + s->replace_list = av_calloc(s->nb_find_list, sizeof(char *)); + if (!s->replace_list) + return AVERROR(ENOMEM); + + for (int i = 0; i < s->nb_find_list; i++) { + size_t len, start = 0, end; + char *item = av_strdup(s->find_list[i]); + if (!item) + return AVERROR(ENOMEM); + + len = end = strlen(item); + + switch (s->censor_mode) { + case CM_KEEP_FIRST_LAST: + + if (len > 2) + start = 1; + if (len > 3) + end--; + + break; + case CM_KEEP_FIRST: + + if (len > 2) + start = 1; + + break; + } + + for (size_t n = start; n < end; n++) + item[n] = s->censor_char[0]; + + s->replace_list[i] = item; + } + + return 0; +} + +static int init_showspeaker(AVFilterContext *ctx) +{ + TextModContext *s = ctx->priv; + s->filter_type = TM_SHOW_SPEAKER; + + return 0; +} + +static int init(AVFilterContext *ctx) +{ + TextModContext *s = ctx->priv; + int ret; + + ret = load_files(ctx); + if (ret < 0 ) + return ret; + + switch (s->operation) { + case OP_REPLACE_CHARS: + case OP_REMOVE_CHARS: + case OP_REPLACE_WORDS: + case OP_REMOVE_WORDS: + if (!s->find || !strlen(s->find)) { + av_log(ctx, AV_LOG_ERROR, "Selected mode requires the 'find' parameter to be specified\n"); + return AVERROR(EINVAL); + } + break; + } + + switch (s->operation) { + case OP_REPLACE_CHARS: + case OP_REPLACE_WORDS: + if (!s->replace || !strlen(s->replace)) { + av_log(ctx, AV_LOG_ERROR, "Selected mode requires the 'replace' parameter to be specified\n"); + return AVERROR(EINVAL); + } + break; + } + + if (s->operation == OP_REPLACE_CHARS && strlen(s->find) != strlen(s->replace)) { + av_log(ctx, AV_LOG_ERROR, "Selected mode requires the 'find' and 'replace' parameters to have the same length\n"); + return AVERROR(EINVAL); + } + + if (s->operation == OP_REPLACE_WORDS || s->operation == OP_REMOVE_WORDS) { + if (!s->separator || strlen(s->separator) != 1) { + av_log(ctx, AV_LOG_ERROR, "Selected mode requires a single separator char to be specified\n"); + return AVERROR(EINVAL); + } + + s->find_list = split_string(s->find, &s->nb_find_list, s->separator); + if (!s->find_list) + return AVERROR(ENOMEM); + + if (s->operation == OP_REPLACE_WORDS) { + + s->replace_list = split_string(s->replace, &s->nb_replace_list, s->separator); + if (!s->replace_list) + return AVERROR(ENOMEM); + + if (s->nb_find_list != s->nb_replace_list) { + av_log(ctx, AV_LOG_ERROR, "The number of words in 'find' and 'replace' needs to be equal\n"); + return AVERROR(EINVAL); + } + } + } + + return 0; +} + +static void uninit(AVFilterContext *ctx) +{ + TextModContext *s = ctx->priv; + + for (int i = 0; i < s->nb_find_list; i++) + av_freep(&s->find_list[i]); + + s->nb_find_list = 0; + av_freep(&s->find_list); + + for (int i = 0; i < s->nb_replace_list; i++) + av_freep(&s->replace_list[i]); + + s->nb_replace_list = 0; + av_freep(&s->replace_list); +} + +static char *process_text(TextModContext *s, char *text) +{ + const char *char_src = s->find; + const char *char_dst = s->replace; + char *result = NULL; + int escape_level = 0, k = 0; + + switch (s->operation) { + case OP_LEET: + case OP_REPLACE_CHARS: + + if (s->operation == OP_LEET) { + char_src = leet_src; + char_dst = leet_dst; + } + + result = av_strdup(text); + if (!result) + return NULL; + + for (size_t n = 0; n < strlen(result); n++) { + if (result[n] == '{') + escape_level++; + + if (!escape_level) { + size_t len = strlen(char_src); + for (size_t t = 0; t < len; t++) { + if (result[n] == char_src[t]) { + result[n] = char_dst[t]; + break; + } + } + } + + if (result[n] == '}') + escape_level--; + } + + break; + case OP_TO_UPPER: + case OP_TO_LOWER: + + result = av_strdup(text); + if (!result) + return NULL; + + for (size_t n = 0; n < strlen(result); n++) { + if (result[n] == '{') + escape_level++; + if (!escape_level) + result[n] = s->operation == OP_TO_LOWER ? av_tolower(result[n]) : av_toupper(result[n]); + if (result[n] == '}') + escape_level--; + } + + break; + case OP_REMOVE_CHARS: + + result = av_strdup(text); + if (!result) + return NULL; + + for (size_t n = 0; n < strlen(result); n++) { + int skip_char = 0; + + if (result[n] == '{') + escape_level++; + + if (!escape_level) { + size_t len = strlen(char_src); + for (size_t t = 0; t < len; t++) { + if (result[n] == char_src[t]) { + skip_char = 1; + break; + } + } + } + + if (!skip_char) + result[k++] = result[n]; + + if (result[n] == '}') + escape_level--; + } + + result[k] = 0; + + break; + case OP_REPLACE_WORDS: + case OP_REMOVE_WORDS: + + result = av_strdup(text); + if (!result) + return NULL; + + for (int n = 0; n < s->nb_find_list; n++) { + char *tmp = result; + const char *replace = (s->operation == OP_REPLACE_WORDS) ? s->replace_list[n] : ""; + + result = av_strireplace(result, s->find_list[n], replace); + if (!result) + return NULL; + + av_free(tmp); + } + + break; + } + + return result; +} + +static char *process_dialog_show_speaker(TextModContext *s, char *ass_line) +{ + ASSDialog *dialog = avpriv_ass_split_dialog(NULL, ass_line); + int escape_level = 0; + unsigned pos = 0, len; + char *result, *text; + AVBPrint pbuf; + + if (!dialog) + return NULL; + + text = process_text(s, dialog->text); + if (!text) + return NULL; + + if (!dialog->name || !strlen(dialog->name) || !dialog->text || !strlen(dialog->text)) + return av_strdup(ass_line); + + // Find insertion point in case the line starts with style codes + len = (unsigned)strlen(dialog->text); + for (unsigned i = 0; i < len; i++) { + + if (dialog->text[i] == '{') + escape_level++; + + if (dialog->text[i] == '}') + escape_level--; + + if (escape_level == 0) { + pos = i; + break; + } + } + + if (s->style && strlen(s->style)) + // When a style is specified reset the insertion point + // (always add speaker plus style at the start in that case) + pos = 0; + + if (pos >= len - 1) + return av_strdup(ass_line); + + av_bprint_init(&pbuf, 1, AV_BPRINT_SIZE_UNLIMITED); + + if (pos > 0) { + av_bprint_append_data(&pbuf, dialog->text, pos); + } + + if (s->style && strlen(s->style)) { + if (s->style[0] == '{') + // Assume complete and valid style code, e.g. {\c&HFF0000&} + av_bprintf(&pbuf, "%s", s->style); + else + // Otherwise it must be a style name + av_bprintf(&pbuf, "{\\r%s}", s->style); + } + + switch (s->speaker_mode) { + case SM_SQUARE_BRACKETS: + av_bprintf(&pbuf, "[%s]", dialog->name); + break; + case SM_ROUND_BRACKETS: + av_bprintf(&pbuf, "(%s)", dialog->name); + break; + case SM_COLON: + av_bprintf(&pbuf, "%s:", dialog->name); + break; + case SM_PLAIN: + av_bprintf(&pbuf, "%s", dialog->name); + break; + } + + if (s->style && strlen(s->style)) { + // Reset line style + if (dialog->style && strlen(dialog->style) && !av_strcasecmp(dialog->style, "default")) + av_bprintf(&pbuf, "{\\r%s}", dialog->style); + else + av_bprintf(&pbuf, "{\\r}"); + } + + if (s->line_break) + av_bprintf(&pbuf, "\\N"); + else + av_bprintf(&pbuf, " "); + + av_bprint_append_data(&pbuf, dialog->text + pos, len - pos); + + av_bprint_finalize(&pbuf, &text); + + result = avpriv_ass_get_dialog_ex(dialog->readorder, dialog->layer, dialog->style, dialog->name, dialog->margin_l, dialog->margin_r, dialog->margin_v, dialog->effect, text); + + av_free(text); + avpriv_ass_free_dialog(&dialog); + return result; +} + +static char *process_dialog(TextModContext *s, char *ass_line) +{ + ASSDialog *dialog; + char *result, *text; + + if (s->filter_type == TM_SHOW_SPEAKER) + return process_dialog_show_speaker(s, ass_line); + + dialog = avpriv_ass_split_dialog(NULL, ass_line); + if (!dialog) + return NULL; + + text = process_text(s, dialog->text); + if (!text) + return NULL; + + result = avpriv_ass_get_dialog_ex(dialog->readorder, dialog->layer, dialog->style, dialog->name, dialog->margin_l, dialog->margin_r, dialog->margin_v, dialog->effect, text); + + av_free(text); + avpriv_ass_free_dialog(&dialog); + return result; +} + +static int config_output(AVFilterLink *outlink) +{ + AVFilterLink *inlink = outlink->src->inputs[0]; + + outlink->w = inlink->w; + outlink->h = inlink->h; + outlink->time_base = inlink->time_base; + outlink->frame_rate = inlink->frame_rate; + + return 0; +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *frame) +{ + TextModContext *s = inlink->dst->priv; + AVFilterLink *outlink = inlink->dst->outputs[0]; + int ret; + + outlink->format = inlink->format; + + ret = av_frame_make_writable(frame); + if (ret < 0) + return ret; + + for (unsigned i = 0; i < frame->num_subtitle_areas; i++) { + + AVSubtitleArea *area = frame->subtitle_areas[i]; + + if (area->ass) { + char *tmp = area->ass; + area->ass = process_dialog(s, area->ass); + av_free(tmp); + if (!area->ass) + return AVERROR(ENOMEM); + } + } + + return ff_filter_frame(outlink, frame); +} + +#define OFFSET(x) offsetof(TextModContext, x) +#define FLAGS (AV_OPT_FLAG_SUBTITLE_PARAM | AV_OPT_FLAG_FILTERING_PARAM) + +static const AVOption textmod_options[] = { + { "mode", "set operation mode", OFFSET(operation), AV_OPT_TYPE_INT, {.i64=OP_LEET}, OP_LEET, NB_OPS-1, FLAGS, "mode" }, + { "leet", "convert text to 'leet speak'", 0, AV_OPT_TYPE_CONST, {.i64=OP_LEET}, 0, 0, FLAGS, "mode" }, + { "to_upper", "change to upper case", 0, AV_OPT_TYPE_CONST, {.i64=OP_TO_UPPER}, 0, 0, FLAGS, "mode" }, + { "to_lower", "change to lower case", 0, AV_OPT_TYPE_CONST, {.i64=OP_TO_LOWER}, 0, 0, FLAGS, "mode" }, + { "replace_chars", "replace characters", 0, AV_OPT_TYPE_CONST, {.i64=OP_REPLACE_CHARS}, 0, 0, FLAGS, "mode" }, + { "remove_chars", "remove characters", 0, AV_OPT_TYPE_CONST, {.i64=OP_REMOVE_CHARS}, 0, 0, FLAGS, "mode" }, + { "replace_words", "replace words", 0, AV_OPT_TYPE_CONST, {.i64=OP_REPLACE_WORDS}, 0, 0, FLAGS, "mode" }, + { "remove_words", "remove words", 0, AV_OPT_TYPE_CONST, {.i64=OP_REMOVE_WORDS}, 0, 0, FLAGS, "mode" }, + { "find", "chars/words to find or remove", OFFSET(find), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, FLAGS, NULL }, + { "find_file", "load find param from file", OFFSET(find_file), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, FLAGS, NULL }, + { "replace", "chars/words to replace", OFFSET(replace), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, FLAGS, NULL }, + { "replace_file", "load replace param from file", OFFSET(replace_file), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, FLAGS, NULL }, + { "separator", "word separator", OFFSET(separator), AV_OPT_TYPE_STRING, {.str = ","}, 0, 0, FLAGS, NULL }, + { NULL }, +}; + + +static const AVOption censor_options[] = { + { "mode", "set censoring mode", OFFSET(censor_mode), AV_OPT_TYPE_INT, {.i64=CM_KEEP_FIRST_LAST}, 0, 2, FLAGS, "mode" }, + { "keep_first_last", "censor inner chars", 0, AV_OPT_TYPE_CONST, {.i64=CM_KEEP_FIRST_LAST}, 0, 0, FLAGS, "mode" }, + { "keep_first", "censor all but first char", 0, AV_OPT_TYPE_CONST, {.i64=CM_KEEP_FIRST}, 0, 0, FLAGS, "mode" }, + { "all", "censor all chars", 0, AV_OPT_TYPE_CONST, {.i64=CM_ALL}, 0, 0, FLAGS, "mode" }, + { "words", "list of words to censor", OFFSET(find), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, FLAGS, NULL }, + { "words_file", "path to word list file", OFFSET(find_file), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, FLAGS, NULL }, + { "separator", "word separator", OFFSET(separator), AV_OPT_TYPE_STRING, {.str = ","}, 0, 0, FLAGS, NULL }, + { "censor_char", "replacement character", OFFSET(censor_char), AV_OPT_TYPE_STRING, {.str = "*"}, 0, 0, FLAGS, NULL }, + { NULL }, +}; + +static const AVOption showspeaker_options[] = { + { "format", "speaker name formatting", OFFSET(speaker_mode), AV_OPT_TYPE_INT, {.i64=SM_SQUARE_BRACKETS}, 0, 2, FLAGS, "format" }, + { "square_brackets", "[speaker] text", 0, AV_OPT_TYPE_CONST, {.i64=SM_SQUARE_BRACKETS}, 0, 0, FLAGS, "format" }, + { "round_brackets", "(speaker) text", 0, AV_OPT_TYPE_CONST, {.i64=SM_ROUND_BRACKETS}, 0, 0, FLAGS, "format" }, + { "colon", "speaker: text", 0, AV_OPT_TYPE_CONST, {.i64=SM_COLON}, 0, 0, FLAGS, "format" }, + { "plain", "speaker text", 0, AV_OPT_TYPE_CONST, {.i64=SM_PLAIN}, 0, 0, FLAGS, "format" }, + { "line_break", "insert line break", OFFSET(line_break), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, FLAGS, NULL }, + { "style", "ass type name or style code", OFFSET(style), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, FLAGS, NULL }, + { NULL }, +}; + +AVFILTER_DEFINE_CLASS(textmod); +AVFILTER_DEFINE_CLASS(censor); +AVFILTER_DEFINE_CLASS(showspeaker); + +static const AVFilterPad inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .filter_frame = filter_frame, + }, +}; + +static const AVFilterPad outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .config_props = config_output, + }, +}; + +const AVFilter ff_sf_textmod = { + .name = "textmod", + .description = NULL_IF_CONFIG_SMALL("Modify subtitle text in several ways"), + .init = init, + .uninit = uninit, + .priv_size = sizeof(TextModContext), + .priv_class = &textmod_class, + FILTER_INPUTS(inputs), + FILTER_OUTPUTS(outputs), + FILTER_SINGLE_SUBFMT(AV_SUBTITLE_FMT_ASS), +}; + +const AVFilter ff_sf_censor = { + .name = "censor", + .description = NULL_IF_CONFIG_SMALL("Censor words in subtitle text"), + .init = init_censor, + .uninit = uninit, + .priv_size = sizeof(TextModContext), + .priv_class = &censor_class, + FILTER_INPUTS(inputs), + FILTER_OUTPUTS(outputs), + FILTER_SINGLE_SUBFMT(AV_SUBTITLE_FMT_ASS), +}; + +const AVFilter ff_sf_showspeaker = { + .name = "showspeaker", + .description = NULL_IF_CONFIG_SMALL("Prepend speaker names to text subtitles (when available)"), + .init = init_showspeaker, + .uninit = uninit, + .priv_size = sizeof(TextModContext), + .priv_class = &showspeaker_class, + FILTER_INPUTS(inputs), + FILTER_OUTPUTS(outputs), + FILTER_SINGLE_SUBFMT(AV_SUBTITLE_FMT_ASS), +}; From patchwork Tue Oct 25 09:13:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 38991 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2685864pzb; Tue, 25 Oct 2022 02:16:37 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4cOWZswo0+RNU7h6RQZusTJ5KAKuF6bL7xkeTT1mui7OLNBVgb1OkE3hqhdqNK4e383bus X-Received: by 2002:a05:6402:254d:b0:45d:3ed3:8aa9 with SMTP id l13-20020a056402254d00b0045d3ed38aa9mr33992886edb.157.1666689397572; Tue, 25 Oct 2022 02:16:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666689397; cv=none; d=google.com; s=arc-20160816; b=cJFnC2TgXYKf/4bQmKcVfxiebd/lrRIy+WvNGosCFmyV3VjTQS/iARi+g2Lw4GWxvM AP3yVPD1x+sisNRccayxu1RjWPSfLoAGl4Ayr0DSJKP2kTYERDoG10eMiQUIvqtgfods ni+VqwAGMc1OfXp1iqhLMVQY9BZynGsysQ6r3MVwuilLCddJzK5bHV6QvXqeZGy5aoyd Btdee07pIh79b8goR75qhrJZ5tSpeeD3WitZ7q/CnVD9KILktVcz2MucQjp4+HnLe/d+ Z6eS/k3Z9xDR4+A8OJbFd/JSwMAdnvzJn5+830WSotx853US9uFUQj0qr4t1ir0piYwZ /p7A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:references :in-reply-to:message-id:from:dkim-signature:delivered-to; bh=w4x4r+g99JcajEg+W6IdJahTSTMgbreeTwMtEjGW2MY=; b=rUi1nqn3+qEJsEUF8IzZcVmCIpSAd+yAHRrZjaTli6R5BTQfpsNlkTsFUKCUVK93Rs 6KirSuNE9yLOF3/KrmelWoIQpaSTU6mowwdRS9zzDLbh3yAIJqo6MfUWF6PqQ/Htat5I U/nQ2EtDQ4UJat4veGHWtpwm+yGPSudKNDX258pLQQY7k1og5n6dBx2oSzfDw3MiVfzl gv9NjQucseJr7EQlESYXmwkXfCnBglJaoSBiSiJvOMI+EgIP4YcIgUaKZ2adRqekhLq2 vweHcv/LhwMIyLr0b5rpCnsBtcFPXqVW2LalvyGlN4YrZL2iqd3MzoAb37tVOrvKzbYD yDnw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=PijXDei7; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id x12-20020aa7d6cc000000b0045cf2d0357asi2062595edr.34.2022.10.25.02.16.37; Tue, 25 Oct 2022 02:16:37 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=PijXDei7; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A8F9968BD5E; Tue, 25 Oct 2022 12:14:15 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id AE84268BC55 for ; Tue, 25 Oct 2022 12:14:10 +0300 (EEST) Received: by mail-pl1-f174.google.com with SMTP id jo13so7050155plb.13 for ; Tue, 25 Oct 2022 02:14:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=Xmsfl0TjrIaclLavJpdxatTTsDvPaML9a8dExQ5N9/s=; b=PijXDei7pF7GqlPh/RdfMzLLrMaW4klYb/viPpwhSJ5Bc+qFljXWFFYR8reOs7i/VF 5bDXshJARZzUodMmaGn/0io5dNPCUqmjcOG++r6YBbZA3whf1XcrV+2WVCDOOzjhO/WT vOJgrhT8CegpzMGGoqlvEpB2RpE2o+0djIjFBG/k9uTbi3wcbDrEnVGYg59nUPuKZXs7 18z6MEcFGUOHMo70rsyBPAx2aYmCAQR/SYVwzlO0ZorHGOQUk4t9HbaOm2mLogEBjE7H 18PcH/Ss9uAv+Ih3PEa9rXqP2hCFUCy4B3Pw9h8Xlu6qp/Vvf/ig/wKRGuOppQFpTOdA nbQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=Xmsfl0TjrIaclLavJpdxatTTsDvPaML9a8dExQ5N9/s=; b=w2CCCvVFXvh/OazVWdW2IkZl4XL4ZU+tZQO+GljgucvnMYNYnGSJOy5TAPpW2Kod8V /FyDYZSKPw+l06gYelTytb4wt/iUd+8lvl12nKTTaFqEFjC62vQqyNh1IGOCWAa6UAZM a1cRNQem1eYexQZp/vqAtolzJBAkgJlyUdCptsbkssWve7xgoIZ4w/f5lIAlqeOkD2jR JNKb2Z32rFlv+SlkPxypt3djkR1BGhWg7PASeB53xY+VG0frdbcTxnDg1Pvv+2J1+WZJ VwRGJF2/SJGYFaHhzm5Mn46BzpDz7TR+CEZM/0k2w0iau0A51uxBYPdyQMNMKyOCW7R+ a4/w== X-Gm-Message-State: ACrzQf0lRkXR+Pk+HYruILxhEZuRIiFvhuDuWadO6GIzQ8t+1bmdWAXd w/NZm1/gwCGbsOCnzGOg3h0l+HC1aQI= X-Received: by 2002:a17:90b:254b:b0:200:a860:5bf9 with SMTP id nw11-20020a17090b254b00b00200a8605bf9mr77539663pjb.176.1666689248683; Tue, 25 Oct 2022 02:14:08 -0700 (PDT) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id s66-20020a625e45000000b0056bbd286cf4sm991666pfb.167.2022.10.25.02.14.07 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Oct 2022 02:14:07 -0700 (PDT) From: softworkz X-Google-Original-From: softworkz Message-Id: <5433d2751a5870c30ce3e34dfb1f72ab75ac47a0.1666689226.git.ffmpegagent@gmail.com> In-Reply-To: References: Date: Tue, 25 Oct 2022 09:13:37 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v9 16/25] avfilter/stripstyles: Add stripstyles filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: softworkz Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: qux8J8omV7A4 From: softworkz - stripstyles {S -> S) Remove all inline styles from subtitle events Signed-off-by: softworkz --- doc/filters.texi | 37 ++++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/sf_stripstyles.c | 237 +++++++++++++++++++++++++++++++++++ 4 files changed, 276 insertions(+) create mode 100644 libavfilter/sf_stripstyles.c diff --git a/doc/filters.texi b/doc/filters.texi index 1f7989e92f..383e020c8d 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -27432,6 +27432,43 @@ ffmpeg -i "http://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex. @end example @end itemize +@section stripstyles + +Remove all inline styles from subtitle events. + +Inputs: +@itemize +@item 0: Subtitles[TEXT] +@end itemize + +Outputs: +@itemize +@item 0: Subtitles[TEXT] +@end itemize + +It accepts the following parameters: + +@table @option +@item remove_animated +Also remove text which is subject to animation (default: true) +Usually, animated text elements are used used in addition to static subtitle lines for creating effects, so in most cases it is safe to remove the animation content. +If subtitle text is missing, try setting this to false. + +@item select_layer +Process only ASS subtitle events from a specific layer. This allows to filter out certain effects where an ASS author duplicates the text onto multiple layers. + +@end table + +@subsection Examples + +@itemize +@item +Remove styles and animations from ASS subtitles and output events from ass layer 0 only. Then convert asn save as SRT stream: +@example +ffmpeg -i "https://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.mkv" -filter_complex "[0:1]stripstyles=select_layer=0" -map 0 -c:s srt output.mkv +@end example +@end itemize + @section textmod diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 5d82b3e8c4..01b7440cce 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -590,6 +590,7 @@ OBJS-$(CONFIG_NULLSINK_FILTER) += vsink_nullsink.o # subtitle filters OBJS-$(CONFIG_CENSOR_FILTER) += sf_textmod.o OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o +OBJS-$(CONFIG_STRIPSTYLES_FILTER) += sf_stripstyles.o OBJS-$(CONFIG_TEXTMOD_FILTER) += sf_textmod.o # multimedia filters diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index d7f24ab7db..33bfae7358 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -578,6 +578,7 @@ extern const AVFilter ff_avsrc_movie; /* subtitle filters */ extern const AVFilter ff_sf_censor; extern const AVFilter ff_sf_showspeaker; +extern const AVFilter ff_sf_stripstyles; extern const AVFilter ff_sf_textmod; /* those filters are part of public or internal API, diff --git a/libavfilter/sf_stripstyles.c b/libavfilter/sf_stripstyles.c new file mode 100644 index 0000000000..78dc6f3ef4 --- /dev/null +++ b/libavfilter/sf_stripstyles.c @@ -0,0 +1,237 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * text subtitle filter which removes inline-styles from subtitles + */ + +#include "libavutil/opt.h" +#include "internal.h" +#include "libavutil/ass_internal.h" +#include "libavutil/ass_split_internal.h" +#include "libavutil/bprint.h" + +typedef struct StripStylesContext { + const AVClass *class; + enum AVSubtitleType format; + int remove_animated; + enum ASSSplitComponents keep_flags; + int select_layer; +} StripStylesContext; + +typedef struct DialogContext { + StripStylesContext* ss_ctx; + AVBPrint buffer; + int drawing_scale; + int is_animated; + int plain_text_length; +} DialogContext; + +static void dialog_text_cb(void *priv, const char *text, int len) +{ + DialogContext *s = priv; + + av_log(s->ss_ctx, AV_LOG_DEBUG, "dialog_text_cb: %s\n", text); + + if (!s->drawing_scale && (!s->is_animated || !s->ss_ctx->remove_animated)) + s->plain_text_length += len; + ////av_bprint_append_data(&s->buffer, text, len); +} + +static void dialog_new_line_cb(void *priv, int forced) +{ + DialogContext *s = priv; + if (!s->drawing_scale && !s->is_animated) + s->plain_text_length += 2; + ////av_bprint_append_data(&s->buffer, forced ? "\\N" : "\\n", 2); +} + +static void dialog_drawing_mode_cb(void *priv, int scale) +{ + DialogContext *s = priv; + s->drawing_scale = scale; +} + +static void dialog_animate_cb(void *priv, int t1, int t2, int accel, char *style) +{ + DialogContext *s = priv; + s->is_animated = 1; +} + +static void dialog_move_cb(void *priv, int x1, int y1, int x2, int y2, int t1, int t2) +{ + DialogContext *s = priv; + if (t1 >= 0 || t2 >= 0) + s->is_animated = 1; +} + +static const ASSCodesCallbacks dialog_callbacks = { + .text = dialog_text_cb, + .new_line = dialog_new_line_cb, + .drawing_mode = dialog_drawing_mode_cb, + .animate = dialog_animate_cb, + .move = dialog_move_cb, +}; + +static char *process_dialog(StripStylesContext *s, const char *ass_line) +{ + DialogContext dlg_ctx = { .ss_ctx = s }; + ASSDialog *dialog = avpriv_ass_split_dialog(NULL, ass_line); + char *result = NULL; + + if (!dialog) + return NULL; + + if (s->select_layer >= 0 && dialog->layer != s->select_layer) { + avpriv_ass_free_dialog(&dialog); + return NULL; + } + + dlg_ctx.ss_ctx = s; + + av_bprint_init(&dlg_ctx.buffer, 512, AV_BPRINT_SIZE_UNLIMITED); + + avpriv_ass_filter_override_codes(&dialog_callbacks, &dlg_ctx, dialog->text, &dlg_ctx.buffer, s->keep_flags); + + if (av_bprint_is_complete(&dlg_ctx.buffer) && dlg_ctx.buffer.len > 0 && dlg_ctx.plain_text_length > 0) + result = avpriv_ass_get_dialog_ex(dialog->readorder, dialog->layer, dialog->style, dialog->name, dialog->margin_l, dialog->margin_r, dialog->margin_v, dialog->effect, dlg_ctx.buffer.str); + + av_bprint_finalize(&dlg_ctx.buffer, NULL); + avpriv_ass_free_dialog(&dialog); + + return result; +} + +static int config_output(AVFilterLink *outlink) +{ + AVFilterLink *inlink = outlink->src->inputs[0]; + + outlink->w = inlink->w; + outlink->h = inlink->h; + outlink->time_base = inlink->time_base; + outlink->frame_rate = inlink->frame_rate; + + return 0; +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *frame) +{ + StripStylesContext *s = inlink->dst->priv; + AVFilterLink *outlink = inlink->dst->outputs[0]; + int ret; + + outlink->format = inlink->format; + + ret = av_frame_make_writable(frame); + if (ret <0 ) { + av_frame_free(&frame); + return AVERROR(ENOMEM); + } + + for (unsigned i = 0; i < frame->num_subtitle_areas; i++) { + + AVSubtitleArea *area = frame->subtitle_areas[i]; + + if (area->ass) { + char *tmp = area->ass; + area->ass = process_dialog(s, area->ass); + + if (area->ass) { + av_log(inlink->dst, AV_LOG_DEBUG, "original: %d %s\n", i, tmp); + av_log(inlink->dst, AV_LOG_DEBUG, "stripped: %d %s\n", i, area->ass); + } + else + area->ass = NULL; + + av_free(tmp); + } + } + + return ff_filter_frame(outlink, frame); +} + +#define OFFSET(x) offsetof(StripStylesContext, x) +#define FLAGS (AV_OPT_FLAG_SUBTITLE_PARAM | AV_OPT_FLAG_FILTERING_PARAM) + +static const AVOption stripstyles_options[] = { + { "keep_flags", "flags to control which override codes to keep", OFFSET(keep_flags), AV_OPT_TYPE_FLAGS, { .i64 = ASS_SPLIT_TEXT }, .flags = FLAGS, .unit = "keepflags" }, + { "basic", "keep static style tags only", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_BASIC }, .flags=FLAGS, .unit = "keepflags" }, + { "all_known", "keep all known tags", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_ALL_KNOWN }, .flags=FLAGS, .unit = "keepflags" }, + { "text", "keep text content", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_TEXT }, .flags=FLAGS, .unit = "keepflags" }, + { "color", "keep color tags (\\c, \\c)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_COLOR }, .flags=FLAGS, .unit = "keepflags" }, + { "alpha", "keep color alpha tags (\\alpha, \\a)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_ALPHA }, .flags=FLAGS, .unit = "keepflags" }, + { "font_name", "keep font name tags (\\fn)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_FONT_NAME }, .flags=FLAGS, .unit = "keepflags" }, + { "font_size", "keep font size tags (\\fs)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_FONT_SIZE }, .flags=FLAGS, .unit = "keepflags" }, + { "font_scale", "keep font scale tags (\\fscx, \\fscy)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_FONT_SCALE }, .flags=FLAGS, .unit = "keepflags" }, + { "font_spacing", "keep font spacing tags (\\fsp)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_FONT_SPACING }, .flags=FLAGS, .unit = "keepflags" }, + { "font_charset", "keep font charset tags (\\fe)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_FONT_CHARSET }, .flags=FLAGS, .unit = "keepflags" }, + { "font_bold", "keep font bold tags (\\b)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_FONT_BOLD }, .flags=FLAGS, .unit = "keepflags" }, + { "font_italic", "keep font italic tags (\\i)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_FONT_ITALIC }, .flags=FLAGS, .unit = "keepflags" }, + { "font_underline", "keep font underline tags (\\u)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_FONT_UNDERLINE }, .flags=FLAGS, .unit = "keepflags" }, + { "font_strikeout", "keep font strikeout tags (\\s)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_FONT_STRIKEOUT }, .flags=FLAGS, .unit = "keepflags" }, + { "text_border", "keep text border tags (\\bord)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_TEXT_BORDER }, .flags=FLAGS, .unit = "keepflags" }, + { "text_shadow", "keep text shadow tags (\\shad)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_TEXT_SHADOW }, .flags=FLAGS, .unit = "keepflags" }, + { "text_rotate", "keep text rotate tags (\\fr)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_TEXT_ROTATE }, .flags=FLAGS, .unit = "keepflags" }, + { "text_blur", "keep text blur tags (\\blur, \\be)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_TEXT_BLUR }, .flags=FLAGS, .unit = "keepflags" }, + { "text_wrap", "keep text wrap tags (\\q)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_TEXT_WRAP }, .flags=FLAGS, .unit = "keepflags" }, + { "text_align", "keep text align tags (\\a, \\an)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_TEXT_ALIGNMENT }, .flags=FLAGS, .unit = "keepflags" }, + { "reset_override", "keep override reset tags (\\r)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_CANCELLING }, .flags=FLAGS, .unit = "keepflags" }, + { "move", "keep move tags (\\move)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_MOVE }, .flags=FLAGS, .unit = "keepflags" }, + { "pos", "keep position tags (\\pos)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_POS }, .flags=FLAGS, .unit = "keepflags" }, + { "origin", "keep origin tags (\\org)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_ORIGIN }, .flags=FLAGS, .unit = "keepflags" }, + { "draw", "keep drawing tags (\\p)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_DRAW }, .flags=FLAGS, .unit = "keepflags" }, + { "animate", "keep animation tags (\\t)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_ANIMATE }, .flags=FLAGS, .unit = "keepflags" }, + { "fade", "keep fade tags (\\fad, \\fade)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_FADE }, .flags=FLAGS, .unit = "keepflags" }, + { "clip", "keep clip tags (\\clip)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_CLIP }, .flags=FLAGS, .unit = "keepflags" }, + { "unknown", "keep unknown tags", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_UNKNOWN }, .flags=FLAGS, .unit = "keepflags" }, + { "remove_animated", "remove animated text (default: yes)", OFFSET(remove_animated), AV_OPT_TYPE_BOOL, {.i64 = 1 }, 0, 1, FLAGS, 0 }, + { "select_layer", "process a specific ass layer only", OFFSET(select_layer), AV_OPT_TYPE_INT, {.i64 = -1 }, -1, INT_MAX, FLAGS, 0 }, + { NULL }, +}; + +AVFILTER_DEFINE_CLASS(stripstyles); + +static const AVFilterPad inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .filter_frame = filter_frame, + }, +}; + +static const AVFilterPad outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .config_props = config_output, + }, +}; + +const AVFilter ff_sf_stripstyles = { + .name = "stripstyles", + .description = NULL_IF_CONFIG_SMALL("Strip subtitle inline styles"), + .priv_size = sizeof(StripStylesContext), + .priv_class = &stripstyles_class, + FILTER_INPUTS(inputs), + FILTER_OUTPUTS(outputs), + FILTER_SINGLE_SUBFMT(AV_SUBTITLE_FMT_ASS), +}; + From patchwork Tue Oct 25 09:13:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 38990 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2685750pzb; Tue, 25 Oct 2022 02:16:27 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7rPobVJsDrR27MU7DTcDuW59mNKfmSGBWnmpDcDctrgjwzsO6lcNUdu4YTQUuM3d8fNnNI X-Received: by 2002:a17:907:25c1:b0:77b:9966:ccf1 with SMTP id ae1-20020a17090725c100b0077b9966ccf1mr31408436ejc.763.1666689387393; Tue, 25 Oct 2022 02:16:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666689387; cv=none; d=google.com; s=arc-20160816; b=Gj5BXeUKT0fh18mGCcJneiEwJg+zbbf98Va/8fF4RbIhVIWH5v/UD/sVWp873f3ejF YScMaCRnN41xsm1XpNnq25EwqRUPfGPB8y5J8hK5D07ji1hvf+ZGcQZzSsN1JgOSR8B8 OO1ZpMmArKTBvyow3rWnh+fwgxZqSgaanByVVuPAoiwmsIlFYgeyvPMdrfZgQvLRhCWK 3KM1N1N45qYvl2AXjsrnyxFI4CYLMuVhBHdasnyzF9w4WUu0+JJrxDVVUPox5bKCK7d2 rwqOCW5hgtbzQrgOkHjewHEed/P+Om/W+FsWTauhxmbtmR7wG+kBskqxmwJpRnH7ZCFK WzsQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:references :in-reply-to:message-id:from:dkim-signature:delivered-to; bh=Da8HyyRnKxWaCMaYNFehae1NycuayR4Rrk0QznphbME=; b=kNIesH4JqZmYHHbUKRax5MW+pCHreQKazE1AE1Qh+88/K6LnUo84PJzYuhCtb78m1c n43edfjzUqZGJdP8Rm5/iR7br9aoxz3JjLGa1ic4hqRxLOoH5ZKi0Hv5WM3p5sy4r98q W7XTf1Tgom/fLJwRzHAgLRv6dljlyeZXRcZvjlmOyRmciyUBVnHfv9UtD21FMv4VUMc7 BWqawh9KrQJeoS8EdvgdGuTN2LzlvMENG9rdZ6kjd8FZRljN0sn6AsRc61CAWfBfKSoC dHmKct+FHowfVcb2oxViVbQS3Zbylm2rHtXOQZZ1g1EpxsA1e0xQtCp3bxwfTeWFZNZ6 F9/A== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=RZdrE7kp; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a29-20020a50c31d000000b0046207099f6bsi358379edb.613.2022.10.25.02.16.26; Tue, 25 Oct 2022 02:16:27 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=RZdrE7kp; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 88DCD68BD5B; Tue, 25 Oct 2022 12:14:14 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f53.google.com (mail-pj1-f53.google.com [209.85.216.53]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5654568BD33 for ; Tue, 25 Oct 2022 12:14:10 +0300 (EEST) Received: by mail-pj1-f53.google.com with SMTP id r61-20020a17090a43c300b00212f4e9cccdso6429720pjg.5 for ; Tue, 25 Oct 2022 02:14:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=p+TddTXRWPDsAqNr/EOPnvH6oXCa+h2C5VFHRHtxgB4=; b=RZdrE7kp+vXG9ZKNvgivaF9t1qNfpotSuUIK4h8EyRBdADaWccrE4RR+InKMywTyzy 6B/tBLQrcvhBl0J2GSSb90zCXxbM1LGAJ7NH23mYTSwfC3N4dwNwzT+BUhmji9SCp0Kk aCEwRiOOsFvxCBEOxGtDLYsn3ChsHk70Adb9ufIo8SyOMHH85VXm+ROmImJdyUbpM4+a +wcDWrBVpq6QT/GymQpyaek/K+nfkUoRLsWH/K1oxaIIm1PaluNnDSCYYOWNe8vhVHbP TUdEYBlpZPDjYZtgF/rwPCN6gtAESBKwUTdBsx9V0NWhGfR3LHdpE2spwG9QKRrLWC2Y C37w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=p+TddTXRWPDsAqNr/EOPnvH6oXCa+h2C5VFHRHtxgB4=; b=flWKwyssN+vMVnLTwO2uBK1sLXR4zUUaLrCID0nOGKArPMG9gzlKi8MJ7iG9EmHZgg nVHcmUzJSwy0bW8YHJkdd1iIIfY8HumrwGYlwSsiqNEuTAq2dqRs0mhwhARO9b3F45Eg zx+UPnGC/cRt3H5AWYLs1LHS/vmnxbZDDtVt0KY0F5ObEhHVDspnmCduR8Ktl2zM4yL8 HrHDYRRIHHCvu8SZMZk3RKj6itzX4TfaZRyFNEDHOnAQcjpyF4BSh18ywGifkSIVV5DP O2SkpnxKND7sbVuTmy4RTaV2d/n+1+ZUZDtpQB3/JnrBn/CEusJTGnFvmTC55wkdYP9E sUjg== X-Gm-Message-State: ACrzQf3Rpnrz0RajCgH5lihsjkNG653bhx3sOmlrVwEpgS3sKl52sZcs e4x8uybcjh36DueW9Gfo8oTbij//crw= X-Received: by 2002:a17:902:e74f:b0:186:a981:2334 with SMTP id p15-20020a170902e74f00b00186a9812334mr9238148plf.88.1666689249637; Tue, 25 Oct 2022 02:14:09 -0700 (PDT) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id f8-20020a17090a120800b00203ab277966sm5193820pja.7.2022.10.25.02.14.08 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Oct 2022 02:14:09 -0700 (PDT) From: softworkz X-Google-Original-From: softworkz Message-Id: <39bb78e449a1df67ed5b3b0fafa86af7e115cb36.1666689226.git.ffmpegagent@gmail.com> In-Reply-To: References: Date: Tue, 25 Oct 2022 09:13:38 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v9 17/25] avfilter/splitcc: Add splitcc filter for closed caption handling X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: softworkz Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: pc1Kavr8WNZS From: softworkz - splitcc {V -> VS) Extract closed-caption (A53) data from video frames as subtitle Frames ffmpeg -y -loglevel verbose -i "https://streams.videolan.org/streams /ts/CC/NewsStream-608-ac3.ts" -filter_complex "[0:v]splitcc[vid1], textmod=mode=remove_chars:find='@',[vid1]overlay_textsubs" output.mkv Signed-off-by: softworkz --- configure | 1 + doc/filters.texi | 63 +++++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/sf_splitcc.c | 395 +++++++++++++++++++++++++++++++++++++++ 5 files changed, 461 insertions(+) create mode 100644 libavfilter/sf_splitcc.c diff --git a/configure b/configure index 61a7bd1438..a992a4795f 100755 --- a/configure +++ b/configure @@ -3753,6 +3753,7 @@ spp_filter_select="fft idctdsp fdctdsp me_cmp pixblockdsp" sr_filter_deps="avformat swscale" sr_filter_select="dnn" stereo3d_filter_deps="gpl" +splitcc_filter_deps="avcodec" subtitles_filter_deps="avformat avcodec libass" super2xsai_filter_deps="gpl" pixfmts_super2xsai_test_deps="super2xsai_filter" diff --git a/doc/filters.texi b/doc/filters.texi index 383e020c8d..d1954b280f 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -27782,6 +27782,69 @@ ffmpeg -i INPUT -filter_complex "showspeaker=format=colon:style='@{\\c&HDD0000&\ @end example @end itemize + +@section splitcc + +Split-out closed-caption/A53 subtitles from video frame side data. + +This filter provides an input and an output for video frames, which are just passed through without modification. +The second out provides subtitle frames which are extracted from video frame side data. + +Inputs: +@itemize +@item 0: Video [ALL] +@end itemize + +Outputs: +@itemize +@item 0: Video (same as input) +@item 1: Subtitles [TEXT] +@end itemize + +It accepts the following parameters: + +@table @option + +@item use_cc_styles +Emit closed caption style header. +This will make closed captions appear in white font with a black rectangle background. + +@item real_time +Emit subtitle events as they are decoded for real-time display. + +@item real_time_latency_msec +Minimum elapsed time between emitting real-time subtitle events. +Only applies to real_time mode. + +@item data_field +Select data field. Possible values: + +@table @samp +@item auto +Pick first one that appears. +@item first +@item second +@end table + +@end table + +@subsection Examples + +@itemize +@item +Extract closed captions as text subtitle stream and overlay it onto the video in cc style (black bar background): +@example +ffmpeg -i "https://streams.videolan.org/streams/ts/CC/NewsStream-608-ac3.ts" -filter_complex "[0:v:0]splitcc=use_cc_styles=1[vid1][sub1];[vid1][sub1]overlaytextsubs" output.mkv +@end example + +@item +A nicer variant, using realtime output from cc_dec and rendering it with the render_latest_only parameter from overlaytextsubs to avoid ghosting by timely overlap. +@example +ffmpeg -i "https://streams.videolan.org/streams/ts/CC/NewsStream-608-ac3.ts" -filter_complex "[0:v:0]splitcc=real_time=1:real_time_latency_msec=200[vid1][sub1];[vid1][sub1]overlaytextsubs=render_latest_only=1" output.mkv +@end example +@end itemize + + @section textsub2video Converts text subtitles to video frames. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 01b7440cce..66c2210a46 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -590,6 +590,7 @@ OBJS-$(CONFIG_NULLSINK_FILTER) += vsink_nullsink.o # subtitle filters OBJS-$(CONFIG_CENSOR_FILTER) += sf_textmod.o OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o +OBJS-$(CONFIG_SPLITCC_FILTER) += sf_splitcc.o OBJS-$(CONFIG_STRIPSTYLES_FILTER) += sf_stripstyles.o OBJS-$(CONFIG_TEXTMOD_FILTER) += sf_textmod.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 33bfae7358..529ccbf760 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -578,6 +578,7 @@ extern const AVFilter ff_avsrc_movie; /* subtitle filters */ extern const AVFilter ff_sf_censor; extern const AVFilter ff_sf_showspeaker; +extern const AVFilter ff_sf_splitcc; extern const AVFilter ff_sf_stripstyles; extern const AVFilter ff_sf_textmod; diff --git a/libavfilter/sf_splitcc.c b/libavfilter/sf_splitcc.c new file mode 100644 index 0000000000..14235e822c --- /dev/null +++ b/libavfilter/sf_splitcc.c @@ -0,0 +1,395 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * subtitle filter for splitting out closed-caption/A53 subtitles from video frame side data + */ + +#include "filters.h" +#include "libavutil/opt.h" +#include "subtitles.h" +#include "libavcodec/avcodec.h" + +static const AVRational ms_tb = {1, 1000}; + +typedef struct SplitCaptionsContext { + const AVClass *class; + enum AVSubtitleType format; + AVCodecContext *cc_dec; + int eof; + AVFrame *next_sub_frame; + AVFrame *empty_sub_frame; + int new_frame; + int64_t next_repetition_pts; + int had_keyframe; + AVBufferRef *subtitle_header; + int use_cc_styles; + int real_time; + int real_time_latency_msec; + int data_field; + int scatter_realtime_output; +} SplitCaptionsContext; + +static int64_t ms_to_avtb(int64_t ms) +{ + return av_rescale_q(ms, (AVRational){ 1, 1000 }, AV_TIME_BASE_Q); +} + +static int init(AVFilterContext *ctx) +{ + SplitCaptionsContext *s = ctx->priv; + AVDictionary *options = NULL; + + int ret; + const AVCodec *codec = avcodec_find_decoder(AV_CODEC_ID_EIA_608); + if (!codec) { + av_log(ctx, AV_LOG_ERROR, "failed to find EIA-608/708 decoder\n"); + return AVERROR_DECODER_NOT_FOUND; + } + + if (!((s->cc_dec = avcodec_alloc_context3(codec)))) { + av_log(ctx, AV_LOG_ERROR, "failed to allocate EIA-608/708 decoder\n"); + return AVERROR(ENOMEM); + } + + av_dict_set_int(&options, "real_time", s->real_time, 0); + av_dict_set_int(&options, "real_time_latency_msec", s->real_time_latency_msec, 0); + av_dict_set_int(&options, "data_field", s->data_field, 0); + + if ((ret = avcodec_open2(s->cc_dec, codec, &options)) < 0) { + av_log(ctx, AV_LOG_ERROR, "failed to open EIA-608/708 decoder: %i\n", ret); + return ret; + } + + if (s->use_cc_styles && s->cc_dec->subtitle_header && s->cc_dec->subtitle_header[0] != 0) { + char* subtitle_header = av_strdup((char *)s->cc_dec->subtitle_header); + if (!subtitle_header) + return AVERROR(ENOMEM); + s->subtitle_header = av_buffer_create((uint8_t *)subtitle_header, strlen(subtitle_header) + 1, NULL, NULL, 0); + if (!s->subtitle_header) { + av_free(subtitle_header); + return AVERROR(ENOMEM); + } + } + + return 0; +} + +static void uninit(AVFilterContext *ctx) +{ + SplitCaptionsContext *s = ctx->priv; + av_frame_free(&s->next_sub_frame); + av_frame_free(&s->empty_sub_frame); + av_buffer_unref(&s->subtitle_header); +} + +static int config_input(AVFilterLink *link) +{ + const SplitCaptionsContext *context = link->dst->priv; + + if (context->cc_dec) + context->cc_dec->pkt_timebase = link->time_base; + + return 0; +} + +static int query_formats(AVFilterContext *ctx) +{ + AVFilterFormats *formats; + AVFilterLink *inlink0 = ctx->inputs[0]; + AVFilterLink *outlink0 = ctx->outputs[0]; + AVFilterLink *outlink1 = ctx->outputs[1]; + static const enum AVSubtitleType subtitle_fmts[] = { AV_SUBTITLE_FMT_ASS, AV_SUBTITLE_FMT_NONE }; + int ret; + + /* set input0 video formats */ + formats = ff_all_formats(AVMEDIA_TYPE_VIDEO); + if ((ret = ff_formats_ref(formats, &inlink0->outcfg.formats)) < 0) + return ret; + + /* set output0 video formats */ + if ((ret = ff_formats_ref(formats, &outlink0->incfg.formats)) < 0) + return ret; + + /* set output1 subtitle formats */ + formats = ff_make_format_list(subtitle_fmts); + if ((ret = ff_formats_ref(formats, &outlink1->incfg.formats)) < 0) + return ret; + + return 0; +} + +static int config_video_output(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + const AVFilterLink *inlink = outlink->src->inputs[0]; + + outlink->w = ctx->inputs[0]->w; + outlink->h = ctx->inputs[0]->h; + outlink->frame_rate = ctx->inputs[0]->frame_rate; + outlink->time_base = ctx->inputs[0]->time_base; + outlink->sample_aspect_ratio = ctx->inputs[0]->sample_aspect_ratio; + + if (inlink->hw_frames_ctx) + outlink->hw_frames_ctx = av_buffer_ref(inlink->hw_frames_ctx); + + return 0; +} + +static int config_sub_output(AVFilterLink *outlink) +{ + SplitCaptionsContext *s = outlink->src->priv; + const AVFilterLink *inlink = outlink->src->inputs[0]; + + outlink->time_base = inlink->time_base; + outlink->format = AV_SUBTITLE_FMT_ASS; + outlink->frame_rate = (AVRational){1000, s->real_time_latency_msec}; + + return 0; +} + +static int request_sub_frame(AVFilterLink *outlink) +{ + SplitCaptionsContext *s = outlink->src->priv; + int status; + int64_t pts; + + if (!s->empty_sub_frame) { + s->empty_sub_frame = ff_get_subtitles_buffer(outlink, outlink->format); + if (!s->empty_sub_frame) + return AVERROR(ENOMEM); + } + + if (!s->eof && ff_inlink_acknowledge_status(outlink->src->inputs[0], &status, &pts)) { + if (status == AVERROR_EOF) + s->eof = 1; + } + + if (s->eof) + return AVERROR_EOF; + + if (s->next_sub_frame) { + + AVFrame *out = NULL; + s->next_sub_frame->pts++; + + if (s->new_frame) + out = av_frame_clone(s->next_sub_frame); + else if (s->empty_sub_frame) { + s->empty_sub_frame->pts = s->next_sub_frame->pts; + out = av_frame_clone(s->empty_sub_frame); + av_frame_copy_props(out, s->next_sub_frame); + out->repeat_sub = 1; + } + + if (!out) + return AVERROR(ENOMEM); + + out->subtitle_timing.start_pts = av_rescale_q(s->next_sub_frame->pts, outlink->time_base, AV_TIME_BASE_Q); + s->new_frame = 0; + + return ff_filter_frame(outlink, out); + } + + return 0; +} + +static int decode(AVCodecContext *avctx, AVFrame *frame, int *got_frame, AVPacket *pkt) +{ + int ret; + + *got_frame = 0; + + if (pkt) { + ret = avcodec_send_packet(avctx, pkt); + // In particular, we don't expect AVERROR(EAGAIN), because we read all + // decoded frames with avcodec_receive_frame() until done. + if (ret < 0 && ret != AVERROR_EOF) + return ret; + } + + ret = avcodec_receive_frame(avctx, frame); + if (ret < 0 && ret != AVERROR(EAGAIN)) + return ret; + if (ret >= 0) + *got_frame = 1; + + return 0; +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *frame) +{ + AVFrameSideData *sd; + SplitCaptionsContext *s = inlink->dst->priv; + AVFilterLink *outlink0 = inlink->dst->outputs[0]; + AVFilterLink *outlink1 = inlink->dst->outputs[1]; + AVPacket *pkt = NULL; + AVFrame *sub_out = NULL; + + int ret; + + outlink0->format = inlink->format; + + sd = av_frame_get_side_data(frame, AV_FRAME_DATA_A53_CC); + + if (sd && (s->had_keyframe || frame->key_frame)) { + int got_output = 0; + + s->had_keyframe = 1; + pkt = av_packet_alloc(); + pkt->buf = av_buffer_ref(sd->buf); + if (!pkt->buf) { + ret = AVERROR(ENOMEM); + goto fail; + } + + pkt->data = pkt->buf->data; + pkt->size = pkt->buf->size; + pkt->pts = av_rescale_q(frame->pts, inlink->time_base, AV_TIME_BASE_Q); + + sub_out = ff_get_subtitles_buffer(outlink1, AV_SUBTITLE_FMT_ASS); + if (!sub_out) { + ret = AVERROR(ENOMEM); + goto fail; + } + + if ((ret = av_buffer_replace(&sub_out->subtitle_header, s->subtitle_header)) < 0) + goto fail; + + ret = decode(s->cc_dec, sub_out, &got_output, pkt); + + ////if (got_output) { + //// av_log(inlink->dst, AV_LOG_INFO, "CC Packet PTS: %"PRId64" got_output: %d out_frame_pts: %"PRId64" out_sub_pts: %"PRId64"\n", pkt->pts, got_output, sub_out->pts, sub_out->subtitle_pts); + ////} + + av_packet_free(&pkt); + + if (ret < 0) { + av_log(inlink->dst, AV_LOG_ERROR, "Decode error: %d \n", ret); + goto fail; + } + + if (got_output) { + sub_out->pts = frame->pts; + av_frame_free(&s->next_sub_frame); + s->next_sub_frame = sub_out; + sub_out = NULL; + s->new_frame = 1; + s->next_sub_frame->pts = frame->pts; + + if ((ret = av_buffer_replace(&s->next_sub_frame->subtitle_header, s->subtitle_header)) < 0) + goto fail; + + if (s->real_time && s->scatter_realtime_output) { + if (s->next_repetition_pts) + s->next_sub_frame->pts = s->next_repetition_pts; + + s->next_sub_frame->subtitle_timing.duration = ms_to_avtb(s->real_time_latency_msec); + s->next_repetition_pts = s->next_sub_frame->pts + av_rescale_q(s->real_time_latency_msec, ms_tb, inlink->time_base); + } + + ret = request_sub_frame(outlink1); + if (ret < 0) + goto fail; + } + } + + if (s->real_time && s->scatter_realtime_output && !s->new_frame && s->next_repetition_pts > 0 && frame->pts > s->next_repetition_pts) { + s->new_frame = 1; + s->next_sub_frame->pts = s->next_repetition_pts; + s->next_repetition_pts = s->next_sub_frame->pts + av_rescale_q(s->real_time_latency_msec, ms_tb, inlink->time_base); + } + + if (!s->next_sub_frame) { + s->next_sub_frame = ff_get_subtitles_buffer(outlink1, outlink1->format); + if (!s->next_sub_frame) { + ret = AVERROR(ENOMEM); + goto fail; + } + + s->next_sub_frame->subtitle_timing.duration = ms_to_avtb(s->real_time_latency_msec); + s->next_sub_frame->pts = frame->pts; + s->new_frame = 1; + + if ((ret = av_buffer_replace(&s->next_sub_frame->subtitle_header, s->subtitle_header)) < 0) + goto fail; + } + + ret = ff_filter_frame(outlink0, frame); + +fail: + av_packet_free(&pkt); + av_frame_free(&sub_out); + return ret; +} + +#define OFFSET(x) offsetof(SplitCaptionsContext, x) +#define FLAGS (AV_OPT_FLAG_SUBTITLE_PARAM | AV_OPT_FLAG_FILTERING_PARAM) + +static const AVOption split_cc_options[] = { + { "use_cc_styles", "Emit closed caption style header", OFFSET(use_cc_styles), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, FLAGS, NULL }, + { "real_time", "emit subtitle events as they are decoded for real-time display", OFFSET(real_time), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, FLAGS }, + { "real_time_latency_msec", "minimum elapsed time between emitting real-time subtitle events", OFFSET(real_time_latency_msec), AV_OPT_TYPE_INT, { .i64 = 200 }, 0, 500, FLAGS }, + { "scatter_realtime_output", "scatter output events to a duration of real_time_latency_msec", OFFSET(scatter_realtime_output), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, FLAGS }, + { "data_field", "select data field", OFFSET(data_field), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 1, FLAGS, "data_field" }, + { "auto", "pick first one that appears", 0, AV_OPT_TYPE_CONST, { .i64 =-1 }, 0, 0, FLAGS, "data_field" }, + { "first", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 0 }, 0, 0, FLAGS, "data_field" }, + { "second", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 1 }, 0, 0, FLAGS, "data_field" }, + { NULL }, +}; + +AVFILTER_DEFINE_CLASS(split_cc); + +static const AVFilterPad inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .filter_frame = filter_frame, + .config_props = config_input, + }, +}; + +static const AVFilterPad outputs[] = { + { + .name = "video_passthrough", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_video_output, + }, + { + .name = "subtitles", + .type = AVMEDIA_TYPE_SUBTITLE, + .request_frame = request_sub_frame, + .config_props = config_sub_output, + }, +}; + +const AVFilter ff_sf_splitcc = { + .name = "splitcc", + .description = NULL_IF_CONFIG_SMALL("Extract closed-caption (A53) data from video as subtitle stream."), + .init = init, + .uninit = uninit, + .priv_size = sizeof(SplitCaptionsContext), + .priv_class = &split_cc_class, + .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + FILTER_INPUTS(inputs), + FILTER_OUTPUTS(outputs), + FILTER_QUERY_FUNC(query_formats), +}; From patchwork Tue Oct 25 09:13:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 38993 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2686070pzb; Tue, 25 Oct 2022 02:16:58 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7RT/240HvX3KMcmDePjrHLxnaIWg7WCOmLqpGdzcQYpsZo+a/l4nnEE/8JTLNvKJrHEfbD X-Received: by 2002:a05:6402:4441:b0:454:8a74:5459 with SMTP id o1-20020a056402444100b004548a745459mr34011075edb.155.1666689418347; Tue, 25 Oct 2022 02:16:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666689418; cv=none; d=google.com; s=arc-20160816; b=m0o69LG/ys+PON5QrkR/udOWTw7A8GfUHOmQZHQ7uNh6GC1SVXEw5mqZMjoSQTBUq9 RNTnBH0PS3vzzMev6cHHUFOFUCMyhjoQuakT+Hh8RFNjI7yj12tD+3JtK7KpGScqppml JORxk9LqPf4BN0NZ8kotr44FkW/0jPAIIvnOXl1priYb3i/cdY8Vrf0RRkFN28yLnBqW bZ4akZJYckYQwVPPxz2bh51dhvuw84xG5Ls7N27nInTZtKvBPi1E7et+lG+f6df3Jkuo E2k91Cz8CTwUQ/JdQCDGCJEfe7AeD0P71tiZTyQ4kSr0uY0vL/3T0Ng5dGCQsnBI3jpK cXAA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:references :in-reply-to:message-id:from:dkim-signature:delivered-to; bh=7bGxatD/IUjbB4slifU583O7hl/QRlbGp7AopRFKpIQ=; b=q9WBR4n1ycMRcnrVSnpRE8npBN8J22cNsu1HMIfuMxEGAfRkDbgzixf4zl3gbGPWJT eM29UTf3sFfG2UrN6YCta8q0esNhWR7ta51M2fEIuoTmSuvRv/at5Qz4Ud9JwSzes3RV 1tsHQ21rFO8LPg6SCOgEt7RLVcdFnzExRVxRvyowp2M5IgIRSDE2fm/NjS82UKrMO0L7 nyaIZ9hq2QIuOs7GBtCrFMXEHofWqFfvjqOgk/bKhRqmJj4A3XO1v3lXrcQ/zVNQSQSj 5BbZJKHCAzBbB58G2CAJcoGKwi0YXY3Ha7jWENpBmpuX/1sMwMh5XxBgPSIA1Rbtut8a fjnA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=OQxf1XHu; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a13-20020a1709066d4d00b007804b5a2c48si2036564ejt.521.2022.10.25.02.16.57; Tue, 25 Oct 2022 02:16:58 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=OQxf1XHu; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7CF8868BCA5; Tue, 25 Oct 2022 12:14:17 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9C7B668BC41 for ; Tue, 25 Oct 2022 12:14:12 +0300 (EEST) Received: by mail-pl1-f180.google.com with SMTP id p3so9706962pld.10 for ; Tue, 25 Oct 2022 02:14:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=kLsySdvHFgYk7hckt3+8m95DepGGIZrSvfYNib7I6go=; b=OQxf1XHuYeSdqRii/XRIgFIfYlheO3NsQFVC+l7mQfTXVUO4FKc6IAZJjbBnYXrBr7 jzdWjlDTR0wLqbC2S9rxcjIQTb0SE1yjuSoLEkgWJgx4qom1jHl1Yy/XZEE43AOVKPRE 3zIZkM0045FTRYO+oDnBZDDT3PBx3cJaWJW1vkbMu1W8Bn6hP5FuLPce36m/fwoToiNU 1wTY6M1e45gTOKmhFyN/RC7Xf+fheg6n6eqpcRJ8lfaK248cDd9clcM9q52x5UA+BpUA gzbB+vhL4EI3EdIyuk5duRJ/XInEHjvw8ZFaj4ELYK12H3ObClc4wsdsESKG9I1rj1gq 4orQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=kLsySdvHFgYk7hckt3+8m95DepGGIZrSvfYNib7I6go=; b=KabkbTMK+ZJzXTve5FJLaLZFC4oa2rL1Zkk6X/Hisi5dZHa/ASTJwBlK9S7t6nCxkG QU6yzZdCg+jSCjE9IL6hcLHIOBEMF5PUxxoPW8//dhSKnICrd+JA8IIY5T6/R+kIavgh bct3x+H3ZTZMSMH8OCdQy9DqWDbBpPy29f0tgti06hBG2DpsYdZ8gU1UqCG/2i78ZAxk XH6Q+crZDnAVPvtuhRzMhn0eYtxFitXK6GN4kBzM9v9L0jbn6zXJolf4r2Gf0oS7ysEJ 5lDCXxmKonjYuy1D3/l0J0D7JwptoREwiMZzaqCmd2AFAwUzjOgvCnszm6JAAVz8df0r kmCg== X-Gm-Message-State: ACrzQf0D2SADR7BCSYSr48ig/TLPQuhRQGkM1JRyph4oPoB+lcM4kxc3 JjZNaLf4VsuoP6W7r3WX3NSi82uVBZ4= X-Received: by 2002:a17:902:a584:b0:186:6040:87f9 with SMTP id az4-20020a170902a58400b00186604087f9mr28401375plb.36.1666689250537; Tue, 25 Oct 2022 02:14:10 -0700 (PDT) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id n2-20020aa79842000000b0056c360af4e3sm144513pfq.9.2022.10.25.02.14.09 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Oct 2022 02:14:10 -0700 (PDT) From: softworkz X-Google-Original-From: softworkz Message-Id: <80f4c93633fe628c7984f069c69602283b55571a.1666689226.git.ffmpegagent@gmail.com> In-Reply-To: References: Date: Tue, 25 Oct 2022 09:13:39 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v9 18/25] avfilter/graphicsub2text: Add new graphicsub2text filter (OCR) X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: softworkz Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 0SGfYt7IHfe1 From: softworkz Signed-off-by: softworkz --- configure | 1 + doc/filters.texi | 55 ++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/sf_graphicsub2text.c | 1137 ++++++++++++++++++++++++++++++ 5 files changed, 1195 insertions(+) create mode 100644 libavfilter/sf_graphicsub2text.c diff --git a/configure b/configure index a992a4795f..815e8e067f 100755 --- a/configure +++ b/configure @@ -3687,6 +3687,7 @@ frei0r_filter_deps="frei0r" frei0r_src_filter_deps="frei0r" fspp_filter_deps="gpl" gblur_vulkan_filter_deps="vulkan spirv_compiler" +graphicsub2text_filter_deps="libtesseract" hflip_vulkan_filter_deps="vulkan spirv_compiler" histeq_filter_deps="gpl" hqdn3d_filter_deps="gpl" diff --git a/doc/filters.texi b/doc/filters.texi index d1954b280f..6b05864706 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -27548,6 +27548,61 @@ ffmpeg -i "https://streams.videolan.org/ffmpeg/mkv_subtitles.mkv" -filter_comple @end example @end itemize +@section graphicsub2text + +Converts graphic subtitles to text subtitles by performing OCR. + +For this filter to be available, ffmpeg needs to be compiled with libtesseract (see https://github.com/tesseract-ocr/tesseract). +Language models need to be downloaded from https://github.com/tesseract-ocr/tessdata and put into as subfolder named 'tessdata' or into a folder specified via the environment variable 'TESSDATA_PREFIX'. +The path can also be specified via filter option (see below). + +Note: These models are including the data for both OCR modes. + +Inputs: +- 0: Subtitles [bitmap] + +Outputs: +- 0: Subtitles [text] + +It accepts the following parameters: + +@table @option +@item ocr_mode +The character recognition mode to use. + +Supported OCR modes are: + +@table @var +@item 0, tesseract +This is the classic libtesseract operation mode. It is fast but less accurate than LSTM. +@item 1, lstm +Newer OCR implementation based on ML models. Provides usually better results, requires more processing resources. +@item 2, both +Use a combination of both modes. +@end table + +@item tessdata_path +The path to a folder containing the language models to be used. + +@item language +The recognition language. It needs to match the first three characters of a language model file in the tessdata path. + +@end table + + +@subsection Examples + +@itemize +@item +Convert DVB graphic subtitles to ASS (text) subtitles + +Note: For this to work, you need to have the data file 'eng.traineddata' in a 'tessdata' subfolder (see above). +@example +ffmpeg -loglevel verbose -i "https://streams.videolan.org/streams/ts/video_subs_ttxt%2Bdvbsub.ts" -filter_complex "[0:13]graphicsub2text=delay_when_no_duration=1" -c:s ass -y output.mkv +@end example +@end itemize + + @section graphicsub2video Renders graphic subtitles as video frames. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 66c2210a46..03dd8ed5ef 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -314,6 +314,7 @@ OBJS-$(CONFIG_GBLUR_FILTER) += vf_gblur.o OBJS-$(CONFIG_GBLUR_VULKAN_FILTER) += vf_gblur_vulkan.o vulkan.o vulkan_filter.o OBJS-$(CONFIG_GEQ_FILTER) += vf_geq.o OBJS-$(CONFIG_GRADFUN_FILTER) += vf_gradfun.o +OBJS-$(CONFIG_GRAPHICSUB2TEXT_FILTER) += sf_graphicsub2text.o OBJS-$(CONFIG_GRAPHICSUB2VIDEO_FILTER) += vf_overlaygraphicsubs.o framesync.o OBJS-$(CONFIG_GRAPHMONITOR_FILTER) += f_graphmonitor.o OBJS-$(CONFIG_GRAYWORLD_FILTER) += vf_grayworld.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 529ccbf760..a1137e6c25 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -577,6 +577,7 @@ extern const AVFilter ff_avsrc_movie; /* subtitle filters */ extern const AVFilter ff_sf_censor; +extern const AVFilter ff_sf_graphicsub2text; extern const AVFilter ff_sf_showspeaker; extern const AVFilter ff_sf_splitcc; extern const AVFilter ff_sf_stripstyles; diff --git a/libavfilter/sf_graphicsub2text.c b/libavfilter/sf_graphicsub2text.c new file mode 100644 index 0000000000..47c7030939 --- /dev/null +++ b/libavfilter/sf_graphicsub2text.c @@ -0,0 +1,1137 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * subtitle filter to convert graphical subs to text subs via OCR + */ + +#include +#include + +#include "drawutils.h" +#include "libavutil/opt.h" +#include "subtitles.h" + +#include "libavcodec/elbg.h" + +enum { + RFLAGS_NONE = 0, + RFLAGS_HALIGN = 1 << 0, + RFLAGS_VALIGN = 1 << 1, + RFLAGS_FBOLD = 1 << 2, + RFLAGS_FITALIC = 1 << 3, + RFLAGS_FUNDERLINE = 1 << 4, + RFLAGS_FONT = 1 << 5, + RFLAGS_FONTSIZE = 1 << 6, + RFLAGS_COLOR = 1 << 7, + RFLAGS_OUTLINECOLOR = 1 << 8, + RFLAGS_ALL = RFLAGS_HALIGN | RFLAGS_VALIGN | RFLAGS_FBOLD | RFLAGS_FITALIC | RFLAGS_FUNDERLINE | + RFLAGS_FONT | RFLAGS_FONTSIZE | RFLAGS_COLOR | RFLAGS_OUTLINECOLOR, +}; + +typedef struct SubOcrContext { + const AVClass *class; + int w, h; + + TessBaseAPI *tapi; + TessOcrEngineMode ocr_mode; + char *tessdata_path; + char *language; + int preprocess_images; + int dump_bitmaps; + int delay_when_no_duration; + int recognize; + double font_size_factor; + + int readorder_counter; + + AVFrame *pending_frame; + AVBufferRef *subtitle_header; + AVBPrint buffer; + + // Color Quantization Fields + struct ELBGContext *ctx; + AVLFG lfg; + int *codeword; + int *codeword_closest_codebook_idxs; + int *codebook; + int r_idx, g_idx, b_idx, a_idx; + int64_t last_subtitle_pts; +} SubOcrContext; + +typedef struct OcrImageProps { + int background_color_index; + int fill_color_index; + +} OcrImageProps; + +static int64_t ms_to_avtb(int64_t ms) +{ + return av_rescale_q(ms, (AVRational){ 1, 1000 }, AV_TIME_BASE_Q); +} + +static int create_ass_header(AVFilterContext* ctx) +{ + SubOcrContext* s = ctx->priv; + + if (!(s->w && s->h)) { + av_log(ctx, AV_LOG_WARNING, "create_ass_header: no width and height specified!\n"); + s->w = ASS_DEFAULT_PLAYRESX; + s->h = ASS_DEFAULT_PLAYRESY; + } + + char* subtitle_header_text = avpriv_ass_get_subtitle_header_full(s->w, s->h, ASS_DEFAULT_FONT, ASS_DEFAULT_FONT_SIZE, + ASS_DEFAULT_COLOR, ASS_DEFAULT_COLOR, ASS_DEFAULT_BACK_COLOR, ASS_DEFAULT_BACK_COLOR, ASS_DEFAULT_BOLD, + ASS_DEFAULT_ITALIC, ASS_DEFAULT_UNDERLINE, ASS_DEFAULT_BORDERSTYLE, ASS_DEFAULT_ALIGNMENT, 0); + + if (!subtitle_header_text) + return AVERROR(ENOMEM); + + s->subtitle_header = av_buffer_create((uint8_t*)subtitle_header_text, strlen(subtitle_header_text) + 1, NULL, NULL, 0); + + if (!s->subtitle_header) + return AVERROR(ENOMEM); + + return 0; +} + +static int init(AVFilterContext *ctx) +{ + SubOcrContext *s = ctx->priv; + const char* tver = TessVersion(); + uint8_t rgba_map[4]; + int ret; + + s->tapi = TessBaseAPICreate(); + + if (!s->tapi || !tver || !strlen(tver)) { + av_log(ctx, AV_LOG_ERROR, "Failed to access libtesseract\n"); + return AVERROR(ENOSYS); + } + + av_log(ctx, AV_LOG_VERBOSE, "Initializing libtesseract, version: %s\n", tver); + + ret = TessBaseAPIInit4(s->tapi, s->tessdata_path, s->language, s->ocr_mode, NULL, 0, NULL, NULL, 0, 1); + if (ret < 0 ) { + av_log(ctx, AV_LOG_ERROR, "Failed to initialize libtesseract. Error: %d\n", ret); + return AVERROR(ENOSYS); + } + + ret = TessBaseAPISetVariable(s->tapi, "tessedit_char_blacklist", "|"); + if (ret < 0 ) { + av_log(ctx, AV_LOG_ERROR, "Failed to set 'tessedit_char_blacklist'. Error: %d\n", ret); + return AVERROR(EINVAL); + } + + av_bprint_init(&s->buffer, 0, AV_BPRINT_SIZE_UNLIMITED); + + ff_fill_rgba_map(&rgba_map[0], AV_PIX_FMT_RGB32); + + s->r_idx = rgba_map[0]; // R + s->g_idx = rgba_map[1]; // G + s->b_idx = rgba_map[2]; // B + s->a_idx = rgba_map[3]; // A + + av_lfg_init(&s->lfg, 123456789); + + return 0; +} + +static void uninit(AVFilterContext *ctx) +{ + SubOcrContext *s = ctx->priv; + + av_buffer_unref(&s->subtitle_header); + av_bprint_finalize(&s->buffer, NULL); + + if (s->tapi) { + TessBaseAPIEnd(s->tapi); + TessBaseAPIDelete(s->tapi); + } + + avpriv_elbg_free(&s->ctx); +} + +static int query_formats(AVFilterContext *ctx) +{ + AVFilterFormats *formats, *formats2; + AVFilterLink *inlink = ctx->inputs[0]; + AVFilterLink *outlink = ctx->outputs[0]; + static const enum AVSubtitleType in_fmts[] = { AV_SUBTITLE_FMT_BITMAP, AV_SUBTITLE_FMT_NONE }; + static const enum AVSubtitleType out_fmts[] = { AV_SUBTITLE_FMT_ASS, AV_SUBTITLE_FMT_NONE }; + int ret; + + /* set input format */ + formats = ff_make_format_list(in_fmts); + if ((ret = ff_formats_ref(formats, &inlink->outcfg.formats)) < 0) + return ret; + + /* set output format */ + formats2 = ff_make_format_list(out_fmts); + if ((ret = ff_formats_ref(formats2, &outlink->incfg.formats)) < 0) + return ret; + + return 0; +} + +static int config_input(AVFilterLink *inlink) +{ + AVFilterContext *ctx = inlink->dst; + SubOcrContext *s = ctx->priv; + + if (s->w <= 0 || s->h <= 0) { + s->w = inlink->w; + s->h = inlink->h; + } + + return create_ass_header(ctx); +} + +static int config_output(AVFilterLink *outlink) +{ + AVFilterLink *inlink = outlink->src->inputs[0]; + const AVFilterContext *ctx = outlink->src; + SubOcrContext *s = ctx->priv; + + outlink->format = AV_SUBTITLE_FMT_ASS; + outlink->w = s->w; + outlink->h = s->h; + outlink->time_base = inlink->time_base; + outlink->frame_rate = inlink->frame_rate; + + return 0; +} + +static void free_subtitle_area(AVSubtitleArea *area) +{ + for (unsigned n = 0; n < FF_ARRAY_ELEMS(area->buf); n++) + av_buffer_unref(&area->buf[n]); + + av_freep(&area->text); + av_freep(&area->ass); + av_free(area); + +} + +static AVSubtitleArea *copy_subtitle_area(const AVSubtitleArea *src) +{ + AVSubtitleArea *dst = av_mallocz(sizeof(AVSubtitleArea)); + + if (!dst) + return NULL; + + dst->x = src->x; + dst->y = src->y; + dst->w = src->w; + dst->h = src->h; + dst->nb_colors = src->nb_colors; + dst->type = src->type; + dst->flags = src->flags; + + for (unsigned i = 0; i < AV_NUM_BUFFER_POINTERS; i++) { + if (src->h > 0 && src->w > 0 && src->buf[i]) { + dst->buf[0] = av_buffer_ref(src->buf[i]); + if (!dst->buf[i]) + return NULL; + + const int ret = av_buffer_make_writable(&dst->buf[i]); + if (ret < 0) + return NULL; + + dst->linesize[i] = src->linesize[i]; + } + } + + memcpy(&dst->pal[0], &src->pal[0], sizeof(src->pal[0]) * 256); + + + return dst; +} + +static int quantize_image_colors(SubOcrContext *const s, AVSubtitleArea *subtitle_area) +{ + const int num_quantized_colors = 3; + int k, ret; + const int codeword_length = subtitle_area->w * subtitle_area->h; + uint8_t *src_data = subtitle_area->buf[0]->data; + + if (subtitle_area->nb_colors <= num_quantized_colors) { + av_log(s, AV_LOG_DEBUG, "No need to quantize colors. Color count: %d\n", subtitle_area->nb_colors); + return 0; + } + + // Convert palette to grayscale + for (int i = 0; i < subtitle_area->nb_colors; i++) { + uint8_t *color = (uint8_t *)&subtitle_area->pal[i]; + const uint8_t average = (uint8_t)(((int)color[s->r_idx] + color[s->g_idx] + color[s->b_idx]) / 3); + color[s->b_idx] = average; + color[s->g_idx] = average; + color[s->r_idx] = average; + } + + /* Re-Initialize */ + s->codeword = av_realloc_f(s->codeword, codeword_length, 4 * sizeof(*s->codeword)); + if (!s->codeword) + return AVERROR(ENOMEM); + + s->codeword_closest_codebook_idxs = av_realloc_f(s->codeword_closest_codebook_idxs, + codeword_length, sizeof(*s->codeword_closest_codebook_idxs)); + if (!s->codeword_closest_codebook_idxs) + return AVERROR(ENOMEM); + + s->codebook = av_realloc_f(s->codebook, num_quantized_colors, 4 * sizeof(*s->codebook)); + if (!s->codebook) + return AVERROR(ENOMEM); + + /* build the codeword */ + k = 0; + for (int i = 0; i < subtitle_area->h; i++) { + uint8_t *p = src_data; + for (int j = 0; j < subtitle_area->w; j++) { + const uint8_t *color = (uint8_t *)&subtitle_area->pal[*p]; + s->codeword[k++] = color[s->b_idx]; + s->codeword[k++] = color[s->g_idx]; + s->codeword[k++] = color[s->r_idx]; + s->codeword[k++] = color[s->a_idx]; + p++; + } + src_data += subtitle_area->linesize[0]; + } + + /* compute the codebook */ + ret = avpriv_elbg_do(&s->ctx, s->codeword, 4, codeword_length, s->codebook, + num_quantized_colors, 1, s->codeword_closest_codebook_idxs, &s->lfg, 0); + if (ret < 0) + return ret; + + /* Write Palette */ + for (int i = 0; i < num_quantized_colors; i++) { + subtitle_area->pal[i] = s->codebook[i*4+3] << 24 | + (s->codebook[i*4+2] << 16) | + (s->codebook[i*4+1] << 8) | + (s->codebook[i*4 ] << 0); + } + + + av_log(s, AV_LOG_DEBUG, "Quantized colors from %d to %d\n", subtitle_area->nb_colors, num_quantized_colors); + + subtitle_area->nb_colors = num_quantized_colors; + src_data = subtitle_area->buf[0]->data; + + /* Write Image */ + k = 0; + for (int i = 0; i < subtitle_area->h; i++) { + uint8_t *p = src_data; + for (int j = 0; j < subtitle_area->w; j++, p++) { + p[0] = (uint8_t)s->codeword_closest_codebook_idxs[k++]; + } + + src_data += subtitle_area->linesize[0]; + } + + return ret; +} + +#define MEASURE_LINE_COUNT 6 + +static uint8_t get_background_color_index(SubOcrContext *const s, const AVSubtitleArea *subtitle_area) +{ + const int linesize = subtitle_area->linesize[0]; + int index_counts[256] = {0}; + const unsigned int line_offsets[MEASURE_LINE_COUNT] = { + 0, + linesize, + 2 * linesize, + (subtitle_area->h - 3) * linesize, + (subtitle_area->h - 2) * linesize, + (subtitle_area->h - 1) * linesize + }; + + const uint8_t *src_data = subtitle_area->buf[0]->data; + const uint8_t tl = src_data[0]; + const uint8_t tr = src_data[subtitle_area->w - 1]; + const uint8_t bl = src_data[(subtitle_area->h - 1) * linesize + 0]; + const uint8_t br = src_data[(subtitle_area->h - 1) * linesize + subtitle_area->w - 1]; + uint8_t max_index = 0; + int max_count; + + // When all corner pixels are equal, assume that as background color + if (tl == tr == bl == br || subtitle_area->h < 6) + return tl; + + for (unsigned int i = 0; i < MEASURE_LINE_COUNT; i++) { + uint8_t *p = subtitle_area->buf[0]->data + line_offsets[i]; + for (int k = 0; k < subtitle_area->w; k++) + index_counts[p[k]]++; + } + + max_count = index_counts[0]; + + for (uint8_t i = 1; i < subtitle_area->nb_colors; i++) { + if (index_counts[i] > max_count) { + max_count = index_counts[i]; + max_index = i; + } + } + + return max_index; +} + +static uint8_t get_text_color_index(SubOcrContext *const s, const AVSubtitleArea *subtitle_area, const uint8_t bg_color_index, uint8_t *outline_color_index) +{ + const int linesize = subtitle_area->linesize[0]; + int index_counts[256] = {0}; + uint8_t last_index = bg_color_index; + int max_count, min_req_count; + uint8_t max_index = 0; + + for (int i = 3; i < subtitle_area->h - 3; i += 5) { + const uint8_t *p = subtitle_area->buf[0]->data + ((ptrdiff_t)linesize * i); + for (int k = 0; k < subtitle_area->w; k++) { + const uint8_t cur_index = p[k]; + + // When color hasn't changed, continue + if (cur_index == last_index) + continue; + + if (cur_index != bg_color_index) + index_counts[cur_index]++; + + last_index = cur_index; + } + } + + max_count = index_counts[0]; + + for (uint8_t i = 1; i < subtitle_area->nb_colors; i++) { + if (index_counts[i] > max_count) { + max_count = index_counts[i]; + max_index = i; + } + } + + min_req_count = max_count / 3; + + for (uint8_t i = 1; i < subtitle_area->nb_colors; i++) { + if (index_counts[i] < min_req_count) + index_counts[i] = 0; + } + + *outline_color_index = max_index; + + index_counts[max_index] = 0; + max_count = 0; + + for (uint8_t i = 0; i < subtitle_area->nb_colors; i++) { + if (index_counts[i] > max_count) { + max_count = index_counts[i]; + max_index = i; + } + } + + if (*outline_color_index == max_index) + *outline_color_index = 255; + + return max_index; +} + +static void make_image_binary(SubOcrContext *const s, AVSubtitleArea *subtitle_area, const uint8_t text_color_index) +{ + for (int i = 0; i < subtitle_area->nb_colors; i++) { + + if (i != text_color_index) + subtitle_area->pal[i] = 0xffffffff; + else + subtitle_area->pal[i] = 0xff000000; + } +} + +static int get_crop_region(SubOcrContext *const s, const AVSubtitleArea *subtitle_area, uint8_t text_color_index, int *x, int *y, int *w, int *h) +{ + const int linesize = subtitle_area->linesize[0]; + int max_y = 0, max_x = 0; + int min_y = subtitle_area->h - 1, min_x = subtitle_area->w - 1; + + for (int i = 0; i < subtitle_area->h; i += 3) { + const uint8_t *p = subtitle_area->buf[0]->data + ((ptrdiff_t)linesize * i); + for (int k = 0; k < subtitle_area->w; k += 2) { + if (p[k] == text_color_index) { + min_y = FFMIN(min_y, i); + min_x = FFMIN(min_x, k); + max_y = FFMAX(max_y, i); + max_x = FFMAX(max_x, k); + } + } + } + + if (max_y <= min_y || max_x <= min_x) { + av_log(s, AV_LOG_WARNING, "Unable to detect crop region\n"); + *x = 0; + *y = 0; + *w = subtitle_area->w; + *h = subtitle_area->h; + } else { + *x = FFMAX(min_x - 10, 0); + *y = FFMAX(min_y - 10, 0); + *w = FFMIN(max_x + 10 - *x, (subtitle_area->w - *x)); + *h = FFMIN(max_y + 10 - *y, (subtitle_area->h - *y)); + } + + return 0; +} + +static int crop_area_bitmap(SubOcrContext *const s, AVSubtitleArea *subtitle_area, int x, int y, int w, int h) +{ + const int linesize = subtitle_area->linesize[0]; + AVBufferRef *dst = av_buffer_allocz(h * w); + uint8_t *d; + + if (!dst) + return AVERROR(ENOMEM); + + d = dst->data; + + for (int i = y; i < y + h; i++) { + const uint8_t *p = subtitle_area->buf[0]->data + ((ptrdiff_t)linesize * i); + for (int k = x; k < x + w; k++) { + *d = p[k]; + d++; + } + } + + subtitle_area->w = w; + subtitle_area->h = h; + subtitle_area->x += x; + subtitle_area->y += y; + subtitle_area->linesize[0] = w; + av_buffer_replace(&subtitle_area->buf[0], dst); + + av_buffer_unref(&dst); + return 0; +} + +#define R 0 +#define G 1 +#define B 2 +#define A 3 + +static int print_code(AVBPrint *buf, int in_code, const char *fmt, ...) +{ + va_list vl; + + if (!in_code) + av_bprint_chars(buf, '{', 1); + + va_start(vl, fmt); + av_vbprintf(buf, fmt, vl); + va_end(vl); + + return 1; +} + +static int end_code(AVBPrint *buf, int in_code) +{ + if (in_code) + av_bprint_chars(buf, '}', 1); + return 0; +} + +static uint8_t* create_grayscale_image(AVFilterContext *ctx, AVSubtitleArea *area, int invert) +{ + uint8_t gray_pal[256]; + const size_t img_size = area->buf[0]->size; + const uint8_t* img = area->buf[0]->data; + uint8_t* gs_img = av_malloc(img_size); + + if (!gs_img) + return NULL; + + for (unsigned i = 0; i < 256; i++) { + const uint8_t *col = (uint8_t*)&area->pal[i]; + const int val = (int)col[3] * FFMAX3(col[0], col[1], col[2]); + gray_pal[i] = (uint8_t)(val >> 8); + } + + if (invert) + for (unsigned i = 0; i < img_size; i++) + gs_img[i] = 255 - gray_pal[img[i]]; + else + for (unsigned i = 0; i < img_size; i++) + gs_img[i] = gray_pal[img[i]]; + + return gs_img; +} + +static uint8_t* create_bitmap_image(AVFilterContext *ctx, AVSubtitleArea *area, const uint8_t text_color_index) +{ + const size_t img_size = area->buf[0]->size; + const uint8_t* img = area->buf[0]->data; + uint8_t* gs_img = av_malloc(img_size); + + if (!gs_img) + return NULL; + + for (unsigned i = 0; i < img_size; i++) { + if (img[i] == text_color_index) + gs_img[i] = 0; + else + gs_img[i] = 255; + } + + return gs_img; +} + +static void png_save(AVFilterContext *ctx, const char *filename, AVSubtitleArea *area) +{ + int x, y; + int v; + FILE *f; + char fname[40]; + const uint8_t *data = area->buf[0]->data; + + snprintf(fname, sizeof(fname), "%s.ppm", filename); + + f = fopen(fname, "wb"); + if (!f) { + perror(fname); + return; + } + fprintf(f, "P6\n" + "%d %d\n" + "%d\n", + area->w, area->h, 255); + for(y = 0; y < area->h; y++) { + for(x = 0; x < area->w; x++) { + const uint8_t index = data[y * area->linesize[0] + x]; + v = (int)area->pal[index]; + putc(v >> 16 & 0xff, f); + putc(v >> 8 & 0xff, f); + putc(v >> 0 & 0xff, f); + } + } + + fclose(f); +} + +static int get_max_index(int score[256]) +{ + int max_val = 0, max_index = 0; + + for (int i = 0; i < 256; i++) { + if (score[i] > max_val) { + max_val = score[i]; + max_index = i; + } + } + + return max_index; +} + +static int get_word_colors(AVFilterContext *ctx, TessResultIterator* ri, const AVSubtitleArea* area, const AVSubtitleArea* original_area, + uint8_t bg_color_index, uint8_t text_color_index, uint8_t outline_color_index, + uint32_t* bg_color, uint32_t* text_color, uint32_t* outline_color) +{ + int left = 0, top = 0, right = 0, bottom = 0, ret; + int bg_score[256] = {0}, text_score[256] = {0}, outline_score[256] = {0}; + int max_index; + + ret = TessPageIteratorBoundingBox((TessPageIterator*)ri, RIL_WORD, &left, &top, &right, &bottom); + if (ret < 0) { + av_log(ctx, AV_LOG_WARNING, "get_word_colors: IteratorBoundingBox failed: %d\n", ret); + return ret; + } + + if (left >= area->w || right >= area->w || top >= area->h || bottom >= area->h) { + av_log(ctx, AV_LOG_WARNING, "get_word_colors: word bounding box (l: %d, t: %d r: %d, b: %d) out of image bounds (%dx%d)\n", left,top, right, bottom, area->w, area->h); + return AVERROR(EINVAL); + } + + for (int y = top; y < bottom; y += 3) { + uint8_t *p = area->buf[0]->data + (y * area->linesize[0]) + left; + uint8_t *porig = original_area->buf[0]->data + (y * original_area->linesize[0]) + left; + uint8_t current_index = 255; + + for (int x = left; x < right; x++, p++, porig++) { + + if (*p == current_index) { + if (*p == bg_color_index) + bg_score[*porig]++; + if (*p == text_color_index) + text_score[*porig]++; + if (*p == outline_color_index) + outline_score[*porig]++; + } + + current_index = *p; + } + } + + max_index = get_max_index(bg_score); + if (bg_score[max_index] > 0) + *bg_color = original_area->pal[max_index]; + + max_index = get_max_index(text_score); + if (text_score[max_index] > 0) + *text_color = original_area->pal[max_index]; + + max_index = get_max_index(outline_score); + if (outline_score[max_index] > 0) + *outline_color = original_area->pal[max_index]; + + return 0; +} + +static int convert_area(AVFilterContext *ctx, AVSubtitleArea *area, const AVFrame *frame, const unsigned area_index, int *margin_v) +{ + SubOcrContext *s = ctx->priv; + char *ocr_text = NULL; + int ret = 0; + uint8_t *gs_img; + uint8_t bg_color_index; + uint8_t text_color_index = 255; + uint8_t outline_color_index = 255; + char filename[32]; + AVSubtitleArea *original_area = copy_subtitle_area(area); + + if (!original_area) + return AVERROR(ENOMEM); + + if (area->w < 6 || area->h < 6) { + area->ass = NULL; + goto exit; + } + + if (s->dump_bitmaps) { + snprintf(filename, sizeof(filename), "graphicsub2text_%"PRId64"_%d_original", frame->subtitle_timing.start_pts, area_index); + png_save(ctx, filename, area); + } + + if (s->preprocess_images) { + ret = quantize_image_colors(s, area); + if (ret < 0) + goto exit; + if (s->dump_bitmaps && original_area->nb_colors != area->nb_colors) { + snprintf(filename, sizeof(filename), "graphicsub2text_%"PRId64"_%d_quantized", frame->subtitle_timing.start_pts, area_index); + png_save(ctx, filename, area); + } + } + + bg_color_index = get_background_color_index(s, area); + + if (s->preprocess_images) { + int x, y, w, h; + + for (int i = 0; i < area->nb_colors; ++i) { + av_log(s, AV_LOG_DEBUG, "Color #%d: %0.8X\n", i, area->pal[i]); + } + + text_color_index = get_text_color_index(s, area, bg_color_index, &outline_color_index); + + get_crop_region(s, area, text_color_index, &x, &y, &w, &h); + + if ((ret = crop_area_bitmap(s, area, x, y, w, h) < 0)) + goto exit; + + if ((ret = crop_area_bitmap(s, original_area, x, y, w, h) < 0)) + goto exit; + + make_image_binary(s, area, text_color_index); + + if (s->dump_bitmaps) { + snprintf(filename, sizeof(filename), "graphicsub2text_%"PRId64"_%d_preprocessed", frame->subtitle_timing.start_pts, area_index); + png_save(ctx, filename, area); + } + + gs_img = create_bitmap_image(ctx, area, text_color_index); + } else + gs_img = create_grayscale_image(ctx, area, 1); + + if (!gs_img) { + ret = AVERROR(ENOMEM); + goto exit; + } + + area->type = AV_SUBTITLE_FMT_ASS; + TessBaseAPISetImage(s->tapi, gs_img, area->w, area->h, 1, area->linesize[0]); + + TessBaseAPISetSourceResolution(s->tapi, 72); + + ret = TessBaseAPIRecognize(s->tapi, NULL); + if (ret == 0) + ocr_text = TessBaseAPIGetUTF8Text(s->tapi); + + if (!ocr_text || !strlen(ocr_text)) { + av_log(ctx, AV_LOG_WARNING, "OCR didn't return a text. ret=%d\n", ret); + area->ass = NULL; + + goto exit; + } + + const size_t len = strlen(ocr_text); + if (len > 0 && ocr_text[len - 1] == '\n') + ocr_text[len - 1] = 0; + + av_log(ctx, AV_LOG_VERBOSE, "OCR Result: %s\n", ocr_text); + + area->ass = av_strdup(ocr_text); + TessDeleteText(ocr_text); + + // End of simple recognition + + if (s->recognize != RFLAGS_NONE) { + TessResultIterator* ri = 0; + const TessPageIteratorLevel level = RIL_WORD; + int cur_is_bold = 0, cur_is_italic = 0, cur_is_underlined = 0, cur_pointsize = 0; + uint32_t cur_text_color = 0, cur_bg_color = 0, cur_outline_color = 0; + + char *cur_font_name = NULL; + int valign = 0; // 0: bottom, 4: top, 8 middle + int halign = 2; // 1: left, 2: center, 3: right + int in_code = 0; + double font_factor = (0.000666 * (s->h - 480) + 1) * s->font_size_factor; + + av_freep(&area->ass); + av_bprint_clear(&s->buffer); + + ri = TessBaseAPIGetIterator(s->tapi); + + // Horizontal Alignment + if (s->w && s->recognize & RFLAGS_HALIGN) { + int left_margin = area->x; + int right_margin = s->w - area->x - area->w; + double relative_diff = ((double)left_margin - right_margin) / s->w; + + if (FFABS(relative_diff) < 0.1) + halign = 2; // center + else if (relative_diff > 0) + halign = 3; // right + else + halign = 1; // left + } + + // Vertical Alignment + if (s->h && frame->height && s->recognize & RFLAGS_VALIGN) { + int left = 0, top = 0, right = 0, bottom = 0; + + TessPageIteratorBoundingBox((TessPageIterator*)ri, RIL_TEXTLINE, &left, &top, &right, &bottom); + av_log(s, AV_LOG_DEBUG, "RIL_TEXTLINE - TOP: %d BOTTOM: %d HEIGHT: %d\n", top, bottom, bottom - top); + + TessPageIteratorBoundingBox((TessPageIterator*)ri, RIL_BLOCK, &left, &top, &right, &bottom); + + const int vertical_pos = area->y + area->h / 2; + if (vertical_pos < s->h / 3) { + *margin_v = area->y + top; + valign = 4; + } + else if (vertical_pos < s->h / 3 * 2) { + *margin_v = 0; + valign = 8; + } else { + *margin_v = frame->height - area->y - area->h; + valign = 0; + } + } + + if (*margin_v < 0) + *margin_v = 0; + + // Set alignment when not default (2) + if ((valign | halign) != 2) + in_code = print_code(&s->buffer, in_code, "\\a%d", valign | halign); + + do { + int is_bold, is_italic, is_underlined, is_monospace, is_serif, is_smallcaps, pointsize, font_id; + char* word; + const char *font_name = TessResultIteratorWordFontAttributes(ri, &is_bold, &is_italic, &is_underlined, &is_monospace, &is_serif, &is_smallcaps, &pointsize, &font_id); + uint32_t text_color = 0, bg_color = 0, outline_color = 0; + + if (cur_is_underlined && !is_underlined && s->recognize & RFLAGS_FUNDERLINE) + in_code = print_code(&s->buffer, in_code, "\\u0"); + + if (cur_is_bold && !is_bold && s->recognize & RFLAGS_FBOLD) + in_code = print_code(&s->buffer, in_code, "\\b0"); + + if (cur_is_italic && !is_italic && s->recognize & RFLAGS_FITALIC) + in_code = print_code(&s->buffer, in_code, "\\i0"); + + + if (TessPageIteratorIsAtBeginningOf((TessPageIterator*)ri, RIL_TEXTLINE) && !TessPageIteratorIsAtBeginningOf((TessPageIterator*)ri, RIL_BLOCK)) { + in_code = end_code(&s->buffer, in_code); + av_bprintf(&s->buffer, "\\N"); + } + + if (get_word_colors(ctx, ri, area, original_area, bg_color_index, text_color_index, outline_color_index, &bg_color, &text_color, &outline_color) == 0) { + + if (text_color > 0 && cur_text_color != text_color && s->recognize & RFLAGS_COLOR) { + const uint8_t* tval = (uint8_t*)&text_color; + const int color = (int)tval[R] << 16 | (int)tval[G] << 8 | tval[B]; + + in_code = print_code(&s->buffer, in_code, "\\1c&H%0.6X&", color); + if (tval[A] != 255) + in_code = print_code(&s->buffer, in_code, "\\1a&H%0.2X&", 255 - tval[A]); + } + + if (outline_color > 0 && cur_outline_color != outline_color && s->recognize & RFLAGS_OUTLINECOLOR) { + const uint8_t* tval = (uint8_t*)&outline_color; + const int color = (int)tval[R] << 16 | (int)tval[G] << 8 | tval[B]; + + in_code = print_code(&s->buffer, in_code, "\\3c&H%0.6X&\\bord2", color); + in_code = print_code(&s->buffer, in_code, "\\3a&H%0.2X&", FFMIN(255 - tval[A], 30)); + } + + cur_text_color = text_color; + cur_outline_color = outline_color; + } + + if (font_name && strlen(font_name) && s->recognize & RFLAGS_FONT) { + if (!cur_font_name || !strlen(cur_font_name) || strcmp(cur_font_name, font_name) != 0) { + char *sanitized_font_name = av_strireplace(font_name, "_", " "); + if (!sanitized_font_name) { + ret = AVERROR(ENOMEM); + goto exit; + } + + in_code = print_code(&s->buffer, in_code, "\\fn%s", sanitized_font_name); + av_freep(&sanitized_font_name); + + if (cur_font_name) + av_freep(&cur_font_name); + cur_font_name = av_strdup(font_name); + if (!cur_font_name) { + ret = AVERROR(ENOMEM); + goto exit; + } + } + } + + if (pointsize > 0 && pointsize != cur_pointsize && s->recognize & RFLAGS_FONTSIZE) { + float change_factor = (float)(FFABS(pointsize - cur_pointsize)) / FFMAX(pointsize, cur_pointsize); + + // Avoid small changes due to recognition variance + if (change_factor > 0.12f) { + av_log(s, AV_LOG_DEBUG, "pointsize - pointsize: %d\n", pointsize); + in_code = print_code(&s->buffer, in_code, "\\fs%d", (int)(pointsize * font_factor)); + cur_pointsize = pointsize; + } + } + + if (is_italic && !cur_is_italic && s->recognize & RFLAGS_FITALIC) + in_code = print_code(&s->buffer, in_code, "\\i1"); + + if (is_bold && !cur_is_bold && s->recognize & RFLAGS_FBOLD) + in_code = print_code(&s->buffer, in_code, "\\b1"); + + if (is_underlined && !cur_is_underlined && s->recognize & RFLAGS_FUNDERLINE) + in_code = print_code(&s->buffer, in_code, "\\u1"); + + in_code = end_code(&s->buffer, in_code); + + cur_is_underlined = is_underlined; + cur_is_bold = is_bold; + cur_is_italic = is_italic; + + if (!TessPageIteratorIsAtBeginningOf((TessPageIterator*)ri, RIL_TEXTLINE)) + av_bprint_chars(&s->buffer, ' ', 1); + + word = TessResultIteratorGetUTF8Text(ri, level); + av_bprint_append_data(&s->buffer, word, strlen(word)); + TessDeleteText(word); + + } while (TessResultIteratorNext(ri, level)); + + if (!av_bprint_is_complete(&s->buffer)) + ret = AVERROR(ENOMEM); + else { + av_log(ctx, AV_LOG_VERBOSE, "ASS Result: %s\n", s->buffer.str); + area->ass = av_strdup(s->buffer.str); + } + + TessResultIteratorDelete(ri); + av_freep(&cur_font_name); + } + +exit: + free_subtitle_area(original_area); + av_freep(&gs_img); + av_buffer_unref(&area->buf[0]); + area->type = AV_SUBTITLE_FMT_ASS; + + return ret; +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *frame) +{ + AVFilterContext *ctx = inlink->dst; + SubOcrContext *s = ctx->priv; + AVFilterLink *outlink = inlink->dst->outputs[0]; + int ret, frame_sent = 0; + + if (s->pending_frame && !frame->repeat_sub) { + const int64_t pts_diff = frame->subtitle_timing.start_pts - s->pending_frame->subtitle_timing.start_pts; + + if (pts_diff == 0) { + // This is just a repetition of the previous frame, ignore it + av_frame_free(&frame); + return 0; + } + + s->pending_frame->subtitle_timing.duration = pts_diff; + + if ((ret = av_buffer_replace(&s->pending_frame->subtitle_header, s->subtitle_header)) < 0) + return ret; + + ret = ff_filter_frame(outlink, s->pending_frame); + s->pending_frame = NULL; + if (ret < 0) + return ret; + + frame_sent = 1; + s->last_subtitle_pts = frame->subtitle_timing.start_pts; + } + + if (frame->repeat_sub) { + // Ignore repeated frame + av_frame_free(&frame); + return 0; + } + + s->last_subtitle_pts = frame->subtitle_timing.start_pts; + + ret = av_frame_make_writable(frame); + + if (ret < 0) { + av_frame_free(&frame); + return ret; + } + + frame->format = AV_SUBTITLE_FMT_ASS; + + av_log(ctx, AV_LOG_VERBOSE, "filter_frame sub_pts: %"PRIu64", duration: %"PRIu64", num_areas: %d\n", + frame->subtitle_timing.start_pts, frame->subtitle_timing.duration, frame->num_subtitle_areas); + + if (frame->num_subtitle_areas > 1 && + frame->subtitle_areas[0]->y > frame->subtitle_areas[frame->num_subtitle_areas - 1]->y) { + + for (unsigned i = 0; i < frame->num_subtitle_areas / 2; i++) + FFSWAP(AVSubtitleArea*, frame->subtitle_areas[i], frame->subtitle_areas[frame->num_subtitle_areas - i - 1]); + } + + for (int i = 0; i < frame->num_subtitle_areas; i++) { + AVSubtitleArea *area = frame->subtitle_areas[i]; + int margin_v = 0; + + ret = convert_area(ctx, area, frame, i, &margin_v); + if (ret < 0) + return ret; + + if (area->ass && area->ass[0] != '\0') { + + const int layer = s->recognize ? i : 0; + char *tmp = area->ass; + area->ass = avpriv_ass_get_dialog_ex(s->readorder_counter++, layer, "Default", NULL, 0, 0, margin_v, NULL, tmp); + av_free(tmp); + } + } + + // When decoders can't determine the end time, they are setting it either to UINT32_NAX + // or 30s (dvbsub). + if (s->delay_when_no_duration && frame->subtitle_timing.duration >= ms_to_avtb(29000)) { + // Can't send it without end time, wait for the next frame to determine the end_display time + s->pending_frame = frame; + + if (frame_sent) + return 0; + + // To keep all going, send an empty frame instead + frame = ff_get_subtitles_buffer(outlink, AV_SUBTITLE_FMT_ASS); + if (!frame) + return AVERROR(ENOMEM); + + av_frame_copy_props(frame, s->pending_frame); + frame->subtitle_timing.start_pts = 0; + frame->subtitle_timing.duration = 1; + frame->repeat_sub = 1; + } + + if ((ret = av_buffer_replace(&frame->subtitle_header, s->subtitle_header)) < 0) + return ret; + + return ff_filter_frame(outlink, frame); +} + +#define OFFSET(x) offsetof(SubOcrContext, x) +#define FLAGS (AV_OPT_FLAG_SUBTITLE_PARAM | AV_OPT_FLAG_FILTERING_PARAM) + +static const AVOption graphicsub2text_options[] = { + { "delay_when_no_duration", "delay output when duration is unknown", OFFSET(delay_when_no_duration), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, FLAGS, NULL }, + { "dump_bitmaps", "save processed bitmaps as .ppm", OFFSET(dump_bitmaps), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, FLAGS, NULL }, + { "font_size_factor", "font size adjustment factor", OFFSET(font_size_factor), AV_OPT_TYPE_DOUBLE, { .dbl = 1.0 }, 0.2, 5, FLAGS, NULL }, + { "language", "ocr language", OFFSET(language), AV_OPT_TYPE_STRING, { .str = "eng" }, 0, 0, FLAGS, NULL }, + { "ocr_mode", "set ocr mode", OFFSET(ocr_mode), AV_OPT_TYPE_INT, { .i64=OEM_TESSERACT_ONLY }, OEM_TESSERACT_ONLY, 2, FLAGS, "ocr_mode" }, + { "tesseract", "classic tesseract ocr", 0, AV_OPT_TYPE_CONST, { .i64=OEM_TESSERACT_ONLY }, 0, 0, FLAGS, "ocr_mode" }, + { "lstm", "lstm (ML based)", 0, AV_OPT_TYPE_CONST, { .i64=OEM_LSTM_ONLY}, 0, 0, FLAGS, "ocr_mode" }, + { "both", "use both models combined", 0, AV_OPT_TYPE_CONST, { .i64=OEM_TESSERACT_LSTM_COMBINED }, 0, 0, FLAGS, "ocr_mode" }, + { "preprocess_images", "reduce colors, remove outlines", OFFSET(preprocess_images), AV_OPT_TYPE_BOOL, { .i64 = 1 }, 0, 1, FLAGS, NULL }, + { "recognize", "detect fonts, styles and colors", OFFSET(recognize), AV_OPT_TYPE_FLAGS, { .i64 = RFLAGS_ALL}, 0, INT_MAX, FLAGS, "reco_flags" }, + { "none", "no format detection", 0, AV_OPT_TYPE_CONST, { .i64 = RFLAGS_NONE }, 0, 0, FLAGS, "reco_flags" }, + { "halign", "horizontal alignment", 0, AV_OPT_TYPE_CONST, { .i64 = RFLAGS_HALIGN }, 0, 0, FLAGS, "reco_flags" }, + { "valign", "vertical alignment", 0, AV_OPT_TYPE_CONST, { .i64 = RFLAGS_VALIGN }, 0, 0, FLAGS, "reco_flags" }, + { "bold", "font bold", 0, AV_OPT_TYPE_CONST, { .i64 = RFLAGS_FBOLD }, 0, 0, FLAGS, "reco_flags" }, + { "italic", "font italic", 0, AV_OPT_TYPE_CONST, { .i64 = RFLAGS_FITALIC }, 0, 0, FLAGS, "reco_flags" }, + { "underline", "font underline", 0, AV_OPT_TYPE_CONST, { .i64 = RFLAGS_FUNDERLINE }, 0, 0, FLAGS, "reco_flags" }, + { "font", "font name", 0, AV_OPT_TYPE_CONST, { .i64 = RFLAGS_FONT }, 0, 0, FLAGS, "reco_flags" }, + { "fontsize", "font size", 0, AV_OPT_TYPE_CONST, { .i64 = RFLAGS_FONTSIZE }, 0, 0, FLAGS, "reco_flags" }, + { "color", "font color", 0, AV_OPT_TYPE_CONST, { .i64 = RFLAGS_COLOR }, 0, 0, FLAGS, "reco_flags" }, + { "outlinecolor", "outline color", 0, AV_OPT_TYPE_CONST, { .i64 = RFLAGS_OUTLINECOLOR }, 0, 0, FLAGS, "reco_flags" }, + { "tessdata_path", "path to tesseract data", OFFSET(tessdata_path), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, FLAGS, NULL }, + { NULL }, +}; + +AVFILTER_DEFINE_CLASS(graphicsub2text); + +static const AVFilterPad inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .filter_frame = filter_frame, + .config_props = config_input, + }, +}; + +static const AVFilterPad outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .config_props = config_output, + }, +}; + +const AVFilter ff_sf_graphicsub2text = { + .name = "graphicsub2text", + .description = NULL_IF_CONFIG_SMALL("Convert graphical subtitles to text subtitles via OCR"), + .init = init, + .uninit = uninit, + .priv_size = sizeof(SubOcrContext), + .priv_class = &graphicsub2text_class, + FILTER_INPUTS(inputs), + FILTER_OUTPUTS(outputs), + FILTER_QUERY_FUNC(query_formats), +}; From patchwork Tue Oct 25 09:13:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 38999 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2686579pzb; Tue, 25 Oct 2022 02:18:12 -0700 (PDT) X-Google-Smtp-Source: AMsMyM53xitskFfGqDuHHdvrVAOSmxZ4bw66gCFJxNDk++qGJbBimmVLAoS6VYdXShEClSEaOZ+7 X-Received: by 2002:a05:6402:3641:b0:45c:4231:ddcc with SMTP id em1-20020a056402364100b0045c4231ddccmr34304101edb.224.1666689491794; Tue, 25 Oct 2022 02:18:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666689491; cv=none; d=google.com; s=arc-20160816; b=Np5zydezZoUsuZog1E9vTgDJr5HVjKHEzxXLuYyObZlWmPeMWIbL4CD/i4t9P26SN6 yxKOABhvKJRRgk0CPJCZvVIS6tHqSnfZM6dQ38cncJ9F07gNcEUcCY+EHzpVZ3Rcx9KZ OWyqa5ewPA98NZcWfNd/wf7SIPlMtLhpOsg5vKpbgYT7k6c6U5L45ycQ4FdMVXIUBcFn iAS+E5aiF0VE5rOJJxBUHy8j9J2anNI213v60tAWPSLUfbs4DycB+yB4VgEhL+Vil9Cy 6XjRF+uYivMHlJfSVkKdxNnXeNPz08cdvJr4k9r+WwHwAcYJnPsTz1cz8g+9rM9MdJeU MACw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:references :in-reply-to:message-id:from:dkim-signature:delivered-to; bh=ZhVKzxLWgdiXxi2cqh33Ivr3Bwt99NuVcFTQlGeaSAs=; b=EQZz8SBhzoe8r6u4iE2ZzQDUhkmRvSPwpmmDdhriNsH1d/gvJH3VPR0tkC/Z5cpKAQ OTYqQ38oks7C8sQTvkRmmrbFQD9DPDYeaM4XRiZgEfJoQoHiO1lUMkDEmdnl/E10cYKQ Yshi8mc/9Di2n5d59F+3icj6R2cynyd6Mq33m3N2bd1krfSlaSzTrWJJwbx05lBbxi/I WEZDgUqWkkEJW664Harl20xFcGmyPtqJ3DtyPvKgbdlfperFYCXCwC7C8yAQJ5GLeyNR pfggEf7xV+laMkIjga9pK6LeWgPlQRoOi/391D9BAFOwfY3JJl/6S7geXYaVEfGBVkKy SanA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=eyOYL0UL; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id pj8-20020a170906d78800b00780e844b2eesi1763280ejb.482.2022.10.25.02.18.11; Tue, 25 Oct 2022 02:18:11 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=eyOYL0UL; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 133C168BDA7; Tue, 25 Oct 2022 12:14:25 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pg1-f171.google.com (mail-pg1-f171.google.com [209.85.215.171]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2AE0E68BD98 for ; Tue, 25 Oct 2022 12:14:23 +0300 (EEST) Received: by mail-pg1-f171.google.com with SMTP id r18so10896963pgr.12 for ; Tue, 25 Oct 2022 02:14:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=HwgiEcQL4qccmXwv0p4+B0Y+wbpOuWdaE4avDKp8CUk=; b=eyOYL0UL5njgob9Dnma1Bk+jWMnDnkXJVWJ6yw0pGGnhKjFeBO2HrOlshqbemo4Q69 10spIwao1xORTJk2NQp1QcIme55zUkBMihs6xyODSyL3UUBI+BO7aREGWo08ZhrGvR5+ XaDzoOrdAAssSFP0CrOoAn28kPY0PSu4EpewESfuhfizBBUkFt4egokMk0myOfgxZbbj 1lZ704hTpbK3YA+yVVTaw6JMy6uMik/JfQsCxDiTDZyD9oEXkM7/U51/+QP+9F6tanzB UJA+J9LrplDFeWrpwyung5vjR4YFWAS8T4Sgv4cBhAd9r8lWVknWirBq1P1bx6Ax4Xz6 yiPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=HwgiEcQL4qccmXwv0p4+B0Y+wbpOuWdaE4avDKp8CUk=; b=0VGv7i4IyYQdYYB3WMU25VszvBNSMUkuUiiE0VrF0cTYz2EgKh/xdWAp7x7FRfCANX b8Dq4w1OyT7pOGMtInBP5kf9KDRXkQzeWdowfflQnmISehPDS8/WXFnCUfk/tqZI/JZa C6XlYcSsS3cTm464p1oL84UkFBlHrlLBenmNcDsbXfnmJdwdndo+VlYRqRTj7D/626FK gSPOrK6aIBH89pan8FGurv4AKW3zo1RXKj50T7ft0Dl1QgoBAD83JNPXuHIf3DI5b5V5 0lDVZhFpmOpzP5oHrV4qfVhm1xdbTpTvPZA5Nl7KWmhWNzWA0TmdHfYQD85iV9HgNmv8 0skQ== X-Gm-Message-State: ACrzQf0I2/aLumkzsQWBWjAWftlLmYIh4mZlMh/ybwnF/aNDqBiUiVkR GenR0RE0FT0aawUILCt/LkFhP4GSNHU= X-Received: by 2002:a05:6a00:a21:b0:562:99d6:c30a with SMTP id p33-20020a056a000a2100b0056299d6c30amr36429165pfh.35.1666689251551; Tue, 25 Oct 2022 02:14:11 -0700 (PDT) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id n13-20020a170903404d00b00182305a787fsm886323pla.62.2022.10.25.02.14.10 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Oct 2022 02:14:11 -0700 (PDT) From: softworkz X-Google-Original-From: softworkz Message-Id: <4e6888ca877a314c0a2b24152988205a304fb1a7.1666689226.git.ffmpegagent@gmail.com> In-Reply-To: References: Date: Tue, 25 Oct 2022 09:13:40 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v9 19/25] avfilter/subscale: Add filter for scaling and/or re-arranging graphical subtitles X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: softworkz Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: mO5yLND49k3P From: softworkz Signed-off-by: softworkz --- configure | 1 + doc/filters.texi | 164 +++++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/sf_subscale.c | 884 ++++++++++++++++++++++++++++++++++++++ 5 files changed, 1051 insertions(+) create mode 100644 libavfilter/sf_subscale.c diff --git a/configure b/configure index 815e8e067f..8968851571 100755 --- a/configure +++ b/configure @@ -3755,6 +3755,7 @@ sr_filter_deps="avformat swscale" sr_filter_select="dnn" stereo3d_filter_deps="gpl" splitcc_filter_deps="avcodec" +subscale_filter_deps="swscale avcodec" subtitles_filter_deps="avformat avcodec libass" super2xsai_filter_deps="gpl" pixfmts_super2xsai_test_deps="super2xsai_filter" diff --git a/doc/filters.texi b/doc/filters.texi index 6b05864706..5b058e8f3f 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -27954,6 +27954,170 @@ Set the rendering margin in pixels. For rendering, alway use the latest event only, which is covering the given point in time. @end table +@section subscale + +Provides high-quality scaling and rearranging functionality for graphical subtitles. + +The subscale filter provides multiple approaches for manipulating +the size and position of graphical subtitle rectangles wich can +be combined or used separately. +Scaling is performed by converting the palettized subtitle bitmaps +to RGBA and re-quantization to palette colors afterwards via elbg algorithm. + +The two major operations are 'scale' and 're-arrange' with the +latter being separated as 'arrange_h' and 'arrange_v'. + + +Inputs: +- 0: Subtitles [bitmap] + +Outputs: +- 0: Subtitles [bitmap] + +It accepts the following parameters: + +@table @option + +@item w, width +Set the width of the output. +Width and height in case of graphical subtitles are just indicating +a virtual size for which the output (consisting of 0-n bitmap rectangles) +is intended to be displayed on. + +@item h, height +Set the height of the output. + +@item margin_h +Sets a horizontal margin to be preserverved when using any +of the arrange modes. + +@item margin_v +Sets a vertical margin to be preserverved when using any +of the arrange modes. + +@item force_original_aspect_ratio +Enable decreasing or increasing output video width or height if necessary to +keep the original aspect ratio. Possible values: + +@table @samp +@item disable +Scale the video as specified and disable this feature. + +@item decrease +The output video dimensions will automatically be decreased if needed. + +@item increase +The output video dimensions will automatically be increased if needed. + +@end table + + +@item scale_mode +Specifies how subtitle bitmaps should be scaled. +The scale factor is determined by the the factor between input +and output size. + +@table @samp +@item none +Do not apply any common scaling. + +@item uniform +Uniformly scale all subtitle bitmaps including their positions. + +@item uniform_no_reposition +Uniformly scale all subtitle bitmaps without changing positions. + +@end table + + +@item arrange_h +Specifies how subtitle bitmaps should be arranged horizontally. + +@item arrange_v +Specifies how subtitle bitmaps should be arranged vertically. + + +@table @samp +@item none +Do not rearrange subtitle bitmaps. + +@item margin_no_scale +Move subtitle bitmaps to be positioned inside the specified +margin (margin_h or margin_v) when possible and without scaling. + +@item margin_and_scale +Move subtitle bitmaps to be positioned inside the specified +margin (margin_h or margin_v) and scale in case it doesn't fit. + +@item snapalign_no_scale +Categorize subtitle bitmap positions as one of left/center/right +or top/bottom/middle based on original positioning and apply +these alignments for the target positioning. +No scaling will be applied. + +@item snapalign_and_scale +Categorize subtitle bitmap positions as one of left/center/right +or top/bottom/middle based on original positioning and apply +these alignments for the target positioning. +Bitmaps that do not fit inside the margins borders are +scaled to fit. +@end table + +@item eval +Set evaluation mode for the expressions (@option{width}, @option{height}). + +It accepts the following values: +@table @samp +@item init +Evaluate expressions only once during the filter initialization. + +@item frame +Evaluate expressions for each incoming frame. This is way slower than the +@samp{init} mode since it requires all the scalers to be re-computed, but it +allows advanced dynamic expressions. +@end table + +Default value is @samp{init}. + + +@item num_colors +Set the number of palette colors for output images. +Choose the maximum (256) when further processing is done (e.g. +overlaying on a video). +When subtitles will be encoded as bitmap subtitles (e.g. dvbsub), +a smaller number of palette colors (e.g. 4-16) might need to be used, depending +on the target format and codec. + +@item bitmap_width_align +@item bitmap_height_align +Make sure that subtitle bitmap sizes are a multiple of this +value. Default is 2. + +@end table + +@subsection Examples + +@itemize +@item +Uniformly scale down video and bitmap subtitles and encode +subtitles as dvbsub. +@example +ffmpeg -y -loglevel verbose -ss 32 -i "https://streams.videolan.org/samples/sub/largeres_vobsub.mkv" -filter_complex "[0:0]scale=480x270[vid1];[0:10]subscale=w=480:h=270:num_colors=16[sub1]" -map [vid1] -map [sub1] -c:s dvbsub -c:v libx265 output.ts +@end example +@item +Squeeze video vertically and arrange subtitle bitmaps +inside the video area without scaling, then overlay +subtitles onto the video. +@example +ffmpeg -y -loglevel verbose -ss 32 -i "https://streams.videolan.org/samples/sub/largeres_vobsub.mkv" -filter_complex "[0:0]scale=1920x400,setsar=1[vid1];[0:10]subscale=w=1920:h=400:scale_mode=none:margin_h=50:arrange_h=margin_no_scale:arrange_v=margin_no_scale:margin_v=50:num_colors=256[sub1];[vid1][sub1]overlaygraphicsubs" -c:v libx265 output.ts +@end example +@item +Scale both, a video and its embedded VOB subs, then encode them as separate streams into an MKV. +@example +ffmpeg -y -y -loglevel verbose -i "https://streams.videolan.org/samples/sub/largeres_vobsub.mkv" -filter_complex "[0:0]scale=720x576[vid1];[0:7]subscale=w=720:h=576:num_colors=16[sub1]" -map [vid1] -map [sub1] -c:s dvdsub out.mkv +@end example +@end itemize + @c man end SUBTITLE FILTERS @chapter Multimedia Filters diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 03dd8ed5ef..9dbac707d1 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -593,6 +593,7 @@ OBJS-$(CONFIG_CENSOR_FILTER) += sf_textmod.o OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o OBJS-$(CONFIG_SPLITCC_FILTER) += sf_splitcc.o OBJS-$(CONFIG_STRIPSTYLES_FILTER) += sf_stripstyles.o +OBJS-$(CONFIG_SUBSCALE_FILTER) += sf_subscale.o OBJS-$(CONFIG_TEXTMOD_FILTER) += sf_textmod.o # multimedia filters diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index a1137e6c25..9c64f3f585 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -581,6 +581,7 @@ extern const AVFilter ff_sf_graphicsub2text; extern const AVFilter ff_sf_showspeaker; extern const AVFilter ff_sf_splitcc; extern const AVFilter ff_sf_stripstyles; +extern const AVFilter ff_sf_subscale; extern const AVFilter ff_sf_textmod; /* those filters are part of public or internal API, diff --git a/libavfilter/sf_subscale.c b/libavfilter/sf_subscale.c new file mode 100644 index 0000000000..04f0f16c27 --- /dev/null +++ b/libavfilter/sf_subscale.c @@ -0,0 +1,884 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * scale graphical subtitles filter + */ + +#include + +#include "drawutils.h" +#include "internal.h" +#include "scale_eval.h" +#include "libavutil/eval.h" +#include "libavutil/opt.h" +#include "libswscale/swscale.h" + +#include "libavcodec/elbg.h" + +static const char *const var_names[] = { + "in_w", "iw", + "in_h", "ih", + "out_w", "ow", + "out_h", "oh", + "a", + "sar", + "dar", + "margin_h", + "margin_v", + NULL +}; + +enum var_name { + VAR_IN_W, VAR_IW, + VAR_IN_H, VAR_IH, + VAR_OUT_W, VAR_OW, + VAR_OUT_H, VAR_OH, + VAR_A, + VAR_SAR, + VAR_DAR, + VARS_B_H, + VARS_B_V, + VARS_NB +}; + +enum EvalMode { + EVAL_MODE_INIT, + EVAL_MODE_FRAME, + EVAL_MODE_NB +}; + +enum SubScaleMode { + SSM_NONE, + SSM_UNIFORM, + SSM_UNIFORM_NO_REPOSITION, +}; + +enum SubArrangeMode { + SAM_NONE, + SAM_ENSUREMARGIN_NO_SCALE, + SAM_ENSUREMARGIN_AND_SCALE, + SAM_SNAPALIGNMENT_NO_SCALE, + SAM_SNAPALIGNMENT_AND_SCALE, +}; + +typedef struct SubScaleContext { + const AVClass *class; + struct SwsContext *sws; + AVDictionary *opts; + + int w, h; + + char *w_expr; ///< width expression string + char *h_expr; ///< height expression string + AVExpr *w_pexpr; + AVExpr *h_pexpr; + double var_values[VARS_NB]; + + int force_original_aspect_ratio; + int eval_mode; ///< expression evaluation mode + + int use_caching; + + // Scale Options + enum SubScaleMode scale_mode; + + // Arrange Options + enum SubArrangeMode arrange_mode_h; + enum SubArrangeMode arrange_mode_v; + int margin_h; + int margin_v; + char *margin_h_expr; + char *margin_v_expr; + AVExpr *margin_h_pexpr; + AVExpr *margin_v_pexpr; + + // Bitmap Options + int num_output_colors; + int bitmap_width_align; + int bitmap_height_align; + + // Color Quantization Fields + struct ELBGContext *ctx; + AVLFG lfg; + int *codeword; + int *codeword_closest_codebook_idxs; + int *codebook; + int r_idx, g_idx, b_idx, a_idx; + AVFrame *cache_frame; +} SubScaleContext; + + +static int config_output(AVFilterLink *outlink); + +static int check_exprs(AVFilterContext *ctx) +{ + const SubScaleContext *s = ctx->priv; + unsigned vars_w[VARS_NB] = { 0 }, vars_h[VARS_NB] = { 0 }; + + if (!s->w_pexpr && !s->h_pexpr) + return AVERROR(EINVAL); + + if (s->w_pexpr) + av_expr_count_vars(s->w_pexpr, vars_w, VARS_NB); + if (s->h_pexpr) + av_expr_count_vars(s->h_pexpr, vars_h, VARS_NB); + + if (vars_w[VAR_OUT_W] || vars_w[VAR_OW]) { + av_log(ctx, AV_LOG_ERROR, "Width expression cannot be self-referencing: '%s'.\n", s->w_expr); + return AVERROR(EINVAL); + } + + if (vars_h[VAR_OUT_H] || vars_h[VAR_OH]) { + av_log(ctx, AV_LOG_ERROR, "Height expression cannot be self-referencing: '%s'.\n", s->h_expr); + return AVERROR(EINVAL); + } + + if ((vars_w[VAR_OUT_H] || vars_w[VAR_OH]) && + (vars_h[VAR_OUT_W] || vars_h[VAR_OW])) { + av_log(ctx, AV_LOG_WARNING, "Circular references detected for width '%s' and height '%s' - possibly invalid.\n", s->w_expr, s->h_expr); + } + + if (s->margin_h_pexpr) + av_expr_count_vars(s->margin_h_pexpr, vars_w, VARS_NB); + if (s->margin_v_pexpr) + av_expr_count_vars(s->margin_v_pexpr, vars_h, VARS_NB); + + return 0; +} + +static int scale_parse_expr(AVFilterContext *ctx, char *str_expr, AVExpr **pexpr_ptr, const char *var, const char *args) +{ + SubScaleContext *s = ctx->priv; + int ret, is_inited = 0; + char *old_str_expr = NULL; + AVExpr *old_pexpr = NULL; + + if (str_expr) { + old_str_expr = av_strdup(str_expr); + if (!old_str_expr) + return AVERROR(ENOMEM); + av_opt_set(s, var, args, 0); + } + + if (*pexpr_ptr) { + old_pexpr = *pexpr_ptr; + *pexpr_ptr = NULL; + is_inited = 1; + } + + ret = av_expr_parse(pexpr_ptr, args, var_names, + NULL, NULL, NULL, NULL, 0, ctx); + if (ret < 0) { + av_log(ctx, AV_LOG_ERROR, "Cannot parse expression for %s: '%s'\n", var, args); + goto revert; + } + + ret = check_exprs(ctx); + if (ret < 0) + goto revert; + + if (is_inited && (ret = config_output(ctx->outputs[0])) < 0) + goto revert; + + av_expr_free(old_pexpr); + av_freep(&old_str_expr); + + return 0; + +revert: + av_expr_free(*pexpr_ptr); + *pexpr_ptr = NULL; + if (old_str_expr) { + av_opt_set(s, var, old_str_expr, 0); + av_free(old_str_expr); + } + if (old_pexpr) + *pexpr_ptr = old_pexpr; + + return ret; +} + +static av_cold int init_dict(AVFilterContext *ctx, AVDictionary **opts) +{ + SubScaleContext *s = ctx->priv; + uint8_t rgba_map[4]; + int ret; + + if (!s->w_expr) + av_opt_set(s, "w", "iw", 0); + if (!s->h_expr) + av_opt_set(s, "h", "ih", 0); + + ret = scale_parse_expr(ctx, NULL, &s->w_pexpr, "width", s->w_expr); + if (ret < 0) + return ret; + + ret = scale_parse_expr(ctx, NULL, &s->h_pexpr, "height", s->h_expr); + if (ret < 0) + return ret; + + av_log(ctx, AV_LOG_VERBOSE, "w:%s h:%s\n", + s->w_expr, s->h_expr); + + if (!s->margin_h_expr) + av_opt_set(s, "margin_h", "0", 0); + if (!s->margin_v_expr) + av_opt_set(s, "margin_v", "0", 0); + + ret = scale_parse_expr(ctx, NULL, &s->margin_h_pexpr, "margin_h", s->margin_h_expr); + if (ret < 0) + return ret; + + ret = scale_parse_expr(ctx, NULL, &s->margin_v_pexpr, "margin_v", s->margin_v_expr); + if (ret < 0) + return ret; + + s->opts = *opts; + *opts = NULL; + + ff_fill_rgba_map(&rgba_map[0], AV_PIX_FMT_RGB32); + + s->r_idx = rgba_map[0]; // R + s->g_idx = rgba_map[1]; // G + s->b_idx = rgba_map[2]; // B + s->a_idx = rgba_map[3]; // A + + av_lfg_init(&s->lfg, 123456789); + + + return 0; +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + SubScaleContext *s = ctx->priv; + + av_frame_free(&s->cache_frame); + + av_expr_free(s->w_pexpr); + av_expr_free(s->h_pexpr); + s->w_pexpr = s->h_pexpr = NULL; + + av_expr_free(s->margin_h_pexpr); + av_expr_free(s->margin_v_pexpr); + s->margin_h_pexpr = s->margin_v_pexpr = NULL; + + sws_freeContext(s->sws); + s->sws = NULL; + av_dict_free(&s->opts); + + avpriv_elbg_free(&s->ctx); + + av_freep(&s->codebook); + av_freep(&s->codeword); + av_freep(&s->codeword_closest_codebook_idxs); +} + +static int config_input(AVFilterLink *inlink) +{ + ////const AVFilterContext *ctx = inlink->dst; + ////SubScaleContext *s = ctx->priv; + + ////if (s->w <= 0 || s->h <= 0) { + //// s->w = inlink->w; + //// s->h = inlink->h; + ////} + return 0; +} + +static int scale_eval_dimensions(AVFilterContext *ctx) +{ + SubScaleContext *s = ctx->priv; + const AVFilterLink *inlink = ctx->inputs[0]; + char *expr; + int eval_w, eval_h, margin_h, margin_v; + int ret; + double res; + + s->var_values[VAR_IN_W] = s->var_values[VAR_IW] = inlink->w; + s->var_values[VAR_IN_H] = s->var_values[VAR_IH] = inlink->h; + s->var_values[VAR_OUT_W] = s->var_values[VAR_OW] = NAN; + s->var_values[VAR_OUT_H] = s->var_values[VAR_OH] = NAN; + s->var_values[VARS_B_H] = s->var_values[VARS_B_V] = 0; + s->var_values[VAR_A] = (double) inlink->w / inlink->h; + s->var_values[VAR_SAR] = inlink->sample_aspect_ratio.num ? + (double) inlink->sample_aspect_ratio.num / inlink->sample_aspect_ratio.den : 1; + s->var_values[VAR_DAR] = s->var_values[VAR_A] * s->var_values[VAR_SAR]; + + res = av_expr_eval(s->w_pexpr, s->var_values, NULL); + s->var_values[VAR_OUT_W] = s->var_values[VAR_OW] = (int) res == 0 ? inlink->w : (int) res; + + res = av_expr_eval(s->h_pexpr, s->var_values, NULL); + if (isnan(res)) { + expr = s->h_expr; + ret = AVERROR(EINVAL); + goto fail; + } + eval_h = s->var_values[VAR_OUT_H] = s->var_values[VAR_OH] = (int) res == 0 ? inlink->h : (int) res; + + res = av_expr_eval(s->w_pexpr, s->var_values, NULL); + if (isnan(res)) { + expr = s->w_expr; + ret = AVERROR(EINVAL); + goto fail; + } + eval_w = s->var_values[VAR_OUT_W] = s->var_values[VAR_OW] = (int) res == 0 ? inlink->w : (int) res; + + s->w = eval_w; + s->h = eval_h; + + res = av_expr_eval(s->margin_h_pexpr, s->var_values, NULL); + s->var_values[VARS_B_H] = (int)res; + + res = av_expr_eval(s->margin_v_pexpr, s->var_values, NULL); + if (isnan(res)) { + expr = s->margin_v_expr; + ret = AVERROR(EINVAL); + goto fail; + } + margin_v = s->var_values[VARS_B_V] = (int)res; + + res = av_expr_eval(s->margin_h_pexpr, s->var_values, NULL); + if (isnan(res)) { + expr = s->margin_h_expr; + ret = AVERROR(EINVAL); + goto fail; + } + margin_h = s->var_values[VARS_B_H] = (int)res; + + s->margin_h = margin_h; + s->margin_v = margin_v; + + return 0; + +fail: + av_log(ctx, AV_LOG_ERROR, + "Error when evaluating the expression '%s'.\n", expr); + return ret; +} + +static int config_output(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + AVFilterLink *inlink = outlink->src->inputs[0]; + SubScaleContext *s = ctx->priv; + int ret; + + outlink->format = AV_SUBTITLE_FMT_BITMAP; + outlink->time_base = ctx->inputs[0]->time_base; + outlink->frame_rate = ctx->inputs[0]->frame_rate; + + if ((ret = scale_eval_dimensions(ctx)) < 0) + goto fail; + + ff_scale_adjust_dimensions(inlink, &s->w, &s->h, + s->force_original_aspect_ratio, 2); + + if (s->w > INT_MAX || + s->h > INT_MAX || + (s->h * inlink->w) > INT_MAX || + (s->w * inlink->h) > INT_MAX) + av_log(ctx, AV_LOG_ERROR, "Rescaled value for width or height is too big.\n"); + + outlink->w = s->w; + outlink->h = s->h; + + if (s->sws) + sws_freeContext(s->sws); + + s->sws = sws_alloc_context(); + if (!s->sws) + return AVERROR(ENOMEM); + + av_opt_set_pixel_fmt(s->sws, "src_format", AV_PIX_FMT_PAL8, 0); + av_opt_set_int(s->sws, "dst_format", AV_PIX_FMT_RGB32, 0); + av_opt_set_int(s->sws, "threads", ff_filter_get_nb_threads(ctx), 0); + + if (s->opts) { + const AVDictionaryEntry *e = NULL; + while ((e = av_dict_get(s->opts, "", e, AV_DICT_IGNORE_SUFFIX))) { + if ((ret = av_opt_set(s->sws, e->key, e->value, 0)) < 0) + return ret; + } + } + + if ((ret = sws_init_context(s->sws, NULL, NULL)) < 0) + return ret; + + if (inlink->sample_aspect_ratio.num){ + outlink->sample_aspect_ratio = av_mul_q((AVRational){outlink->h * inlink->w, outlink->w * inlink->h}, inlink->sample_aspect_ratio); + } else + outlink->sample_aspect_ratio = inlink->sample_aspect_ratio; + + av_log(ctx, AV_LOG_VERBOSE, "Output size set to %dx%d.\n", outlink->w, outlink->h); + + return 0; +fail: + return ret; +} + +static int palettize_image(SubScaleContext *const s, const int w, const int h, const int src_linesize, uint8_t *src_data, + int dst_linesize, uint8_t *dst_data, uint32_t *dst_pal) +{ + int k, ret; + const int codeword_length = w * h; + + /* Re-Initialize */ + s->codeword = av_realloc_f(s->codeword, codeword_length, 4 * sizeof(*s->codeword)); + if (!s->codeword) + return AVERROR(ENOMEM); + + s->codeword_closest_codebook_idxs = + av_realloc_f(s->codeword_closest_codebook_idxs, codeword_length, + sizeof(*s->codeword_closest_codebook_idxs)); + if (!s->codeword_closest_codebook_idxs) + return AVERROR(ENOMEM); + + s->codebook = av_realloc_f(s->codebook, s->num_output_colors, 4 * sizeof(*s->codebook)); + if (!s->codebook) + return AVERROR(ENOMEM); + + /* build the codeword */ + k = 0; + for (int i = 0; i < h; i++) { + uint8_t *p = src_data; + for (int j = 0; j < w; j++) { + s->codeword[k++] = p[s->b_idx]; + s->codeword[k++] = p[s->g_idx]; + s->codeword[k++] = p[s->r_idx]; + s->codeword[k++] = p[s->a_idx]; + p += 4; + } + src_data += src_linesize; + } + + /* compute the codebook */ + ret = avpriv_elbg_do(&s->ctx, s->codeword, 4, + codeword_length, s->codebook, + s->num_output_colors, 1, + s->codeword_closest_codebook_idxs, &s->lfg, 0); + + if (ret < 0) + return ret; + + /* Write Palette */ + for (int i = 0; i < s->num_output_colors; i++) { + dst_pal[i] = s->codebook[i*4+3] << 24 | + (s->codebook[i*4+2] << 16) | + (s->codebook[i*4+1] << 8) | + s->codebook[i*4 ]; + } + + /* Write Image */ + k = 0; + for (int i = 0; i < h; i++) { + uint8_t *p = dst_data; + for (int j = 0; j < w; j++, p++) { + p[0] = s->codeword_closest_codebook_idxs[k++]; + } + + dst_data += dst_linesize; + } + + return ret; +} + +static int rescale_size(int64_t a, AVRational factor) +{ + const int64_t res = av_rescale_rnd(a, factor.num, factor.den, AV_ROUND_NEAR_INF); + if (res > INT32_MAX || res < 0) + return 0; + + return (int)res; +} + + +static int scale_area(AVFilterLink *link, AVSubtitleArea *area, const int target_width, const int target_height) +{ + const AVFilterContext *ctx = link->dst; + SubScaleContext *s = ctx->priv; + int ret; + + AVBufferRef *dst_buffer; + const uint8_t* data[2] = { area->buf[0]->data, (uint8_t *)&area->pal }; + const int dstW = FFALIGN(target_width, s->bitmap_width_align); + const int dstH = FFALIGN(target_height, s->bitmap_height_align); + const int tmp_linesize[2] = { FFALIGN(dstW * 4, 32), 0 }; + const int dst_linesize[2] = { dstW, 0 }; + uint8_t* tmp[2] = { 0, 0 }; + + AVBufferRef *tmp_buffer = av_buffer_allocz(tmp_linesize[0] * dstH); + if (!tmp_buffer) + return AVERROR(ENOMEM); + + if (!s->sws) + return 0; + + tmp[0] = tmp_buffer->data; + + s->sws = sws_getCachedContext(s->sws, area->w, area->h, AV_PIX_FMT_PAL8, + dstW, dstH, AV_PIX_FMT_RGB32, SWS_BICUBIC, NULL, NULL, NULL); + if (!s->sws) { + av_log(NULL, AV_LOG_FATAL, "Cannot initialize the conversion context. dstW=%d dstH=%d\n", dstW, dstH); + return AVERROR(EINVAL); + } + + // Rescale to ARGB + ret = sws_scale(s->sws, data, area->linesize, 0, area->h, tmp, tmp_linesize); + if (ret < 0) { + av_buffer_unref(&tmp_buffer); + return ret; + } + + // Alloc output buffer + dst_buffer = av_buffer_allocz(dst_linesize[0] * dstH); + if (!dst_buffer) { + av_buffer_unref(&tmp_buffer); + return AVERROR(ENOMEM); + } + + // Quantize to palettized image + ret = palettize_image(s, dstW, dstH, tmp_linesize[0], tmp[0], dst_linesize[0], dst_buffer->data, area->pal); + av_buffer_unref(&tmp_buffer); + + if (ret < 0) { + av_buffer_unref(&dst_buffer); + return ret; + } + + av_buffer_unref(&area->buf[0]); + ret = av_buffer_replace(&area->buf[0], dst_buffer); + if (ret < 0) { + av_buffer_unref(&dst_buffer); + return ret; + } + + area->w = dstW; + area->h = dstH; + area->linesize[0] = dst_linesize[0]; + area->nb_colors = s->num_output_colors; + + return ret; +} + +static int process_area(AVFilterLink *inlink, AVSubtitleArea *area, AVRational x_factor, AVRational y_factor) +{ + AVFilterContext *ctx = inlink->dst; + const SubScaleContext *s = ctx->priv; + int target_w, target_h, target_x, target_y; + const int border_l = s->margin_h; + const int border_r = s->w - s->margin_h; + const int border_t = s->margin_v; + const int border_b = s->h - s->margin_v; + + av_log(ctx, AV_LOG_DEBUG, "process_area - start: x/y: (%d:%d) size: %dx%d scale_mode: %d x-factor: %d:%d y-factor: %d:%d\n", + area->x, area->y, area->w, area->h, s->scale_mode, x_factor.num, x_factor.den, y_factor.num, y_factor.den); + + switch (s->scale_mode) { + case SSM_NONE: + target_w = area->w; + target_h = area->h; + target_x = area->x; + target_y = area->y; + break; + case SSM_UNIFORM: + target_w = rescale_size(area->w, x_factor); + target_h = rescale_size(area->h, y_factor); + target_x = rescale_size(area->x, x_factor); + target_y = rescale_size(area->y, y_factor); + break; + case SSM_UNIFORM_NO_REPOSITION: + target_w = rescale_size(area->w, x_factor); + target_h = rescale_size(area->h, y_factor); + target_x = area->x; + target_y = area->y; + break; + default: + av_log(ctx, AV_LOG_ERROR, "Invalid scale_mode: %d\n", s->scale_mode); + return AVERROR(EINVAL); + } + + av_log(ctx, AV_LOG_DEBUG, "process_area - scaled: x/y: (%d:%d) size: %dx%d.\n", target_x, target_y, target_w, target_h); + + + switch (s->arrange_mode_h) { + case SAM_ENSUREMARGIN_AND_SCALE: + case SAM_SNAPALIGNMENT_AND_SCALE: + { + // IF it doesn't fit - scale it + int max_width = s->w - 2 * s->margin_h; + + if (max_width < 2) + max_width = 2; + + if (target_w > max_width) { + target_h = (int)av_rescale(target_h, max_width, target_w); + target_w = max_width; + target_x = s->margin_h; + } + } + break; + } + + switch (s->arrange_mode_h) { + case SAM_NONE: + break; + case SAM_ENSUREMARGIN_NO_SCALE: + case SAM_ENSUREMARGIN_AND_SCALE: + // Left border + if (target_x < border_l) + target_x = border_l; + + // Right border + if (target_x + target_w > border_r) + target_x = border_r - target_w; + + break; + case SAM_SNAPALIGNMENT_NO_SCALE: + case SAM_SNAPALIGNMENT_AND_SCALE: + { + // Use original values to detect alignment + const int left_margin = area->x; + const int right_margin = inlink->w - area->x - area->w; + const AVRational diff_factor_r = { left_margin - right_margin, area->w }; + const float diff_factor = (float)av_q2d(diff_factor_r); + + if (diff_factor > 0.2f) { + // Right aligned + target_x = border_r - target_w; + } else if (diff_factor < -0.2f) { + // Left aligned + target_x = border_l; + } else { + // Centered + target_x = (inlink->w - area->w) / 2; + } + } + + break; + default: + av_log(ctx, AV_LOG_ERROR, "Invalid arrange_mode_h: %d\n", s->arrange_mode_h); + return AVERROR(EINVAL); + } + + av_log(ctx, AV_LOG_DEBUG, "process_area - arr_h: x/y: (%d:%d) size: %dx%d.\n", target_x, target_y, target_w, target_h); + + + switch (s->arrange_mode_v) { + case SAM_ENSUREMARGIN_AND_SCALE: + case SAM_SNAPALIGNMENT_AND_SCALE: + { + // IF it doesn't fit - scale it + int max_height = s->h - 2 * s->margin_v; + + if (max_height < 2) + max_height = 2; + + if (target_h > max_height) { + target_w = (int)av_rescale(target_w, max_height, target_h); + target_h = max_height; + target_y = s->margin_v; + } + } + break; + } + + switch (s->arrange_mode_v) { + case SAM_NONE: + break; + case SAM_ENSUREMARGIN_NO_SCALE: + case SAM_ENSUREMARGIN_AND_SCALE: + // Top border + if (target_y < border_t) + target_y = border_t; + + // Bottom border + if (target_y + target_h > border_b) + target_y = border_b - target_h; + + break; + case SAM_SNAPALIGNMENT_NO_SCALE: + case SAM_SNAPALIGNMENT_AND_SCALE: + { + // Use original values to detect alignment + const int top_margin = area->y; + const int bottom_margin = inlink->h - area->y - area->h; + const AVRational diff_factor_r = { top_margin - bottom_margin, area->h }; + const float diff_factor = (float)av_q2d(diff_factor_r); + + if (diff_factor > 0.2f) { + // Bottom aligned + target_y = border_b - target_h; + } else if (diff_factor < -0.2f) { + // Top aligned + target_y = border_t; + } else { + // Centered + target_y = (inlink->h - area->h) / 2; + } + } + + break; + default: + av_log(ctx, AV_LOG_ERROR, "Invalid arrange_mode_v: %d\n", s->arrange_mode_v); + return AVERROR(EINVAL); + } + + av_log(ctx, AV_LOG_VERBOSE, "process_area - arr_v: x/y: (%d:%d) size: %dx%d.\n", target_x, target_y, target_w, target_h); + + area->x = target_x; + area->y = target_y; + + if (area->w != target_w || area->h != target_h) + return scale_area(inlink, area, target_w, target_h); + + return 0; +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *frame) +{ + const AVFilterContext *ctx = inlink->dst; + SubScaleContext *s = ctx->priv; + AVFilterLink *outlink = ctx->outputs[0]; + int ret; + + // just forward empty frames + if (frame->num_subtitle_areas == 0) { + av_frame_free(&s->cache_frame); + return ff_filter_frame(outlink, frame); + } + + if (s->use_caching && s->cache_frame && frame->repeat_sub + && s->cache_frame->subtitle_timing.start_pts == frame->subtitle_timing.start_pts) { + AVFrame *out = av_frame_clone(s->cache_frame); + if (!out) + return AVERROR(ENOMEM); + + ret = av_frame_copy_props(out, frame); + if (ret < 0) + return ret; + + av_log(inlink->dst, AV_LOG_DEBUG, "subscale CACHED - size %dx%d pts: %"PRId64" areas: %d\n", frame->width, frame->height, frame->subtitle_timing.start_pts, frame->num_subtitle_areas); + av_frame_free(&frame); + return ff_filter_frame(outlink, out); + } + + ret = av_frame_make_writable(frame); + if (ret >= 0) { + const AVRational x_factor = { .num = outlink->w, .den = inlink->w} ; + const AVRational y_factor = { .num = outlink->h, .den = inlink->h} ; + + for (unsigned i = 0; i < frame->num_subtitle_areas; ++i) { + AVSubtitleArea *area = frame->subtitle_areas[i]; + + ret = process_area(inlink, area, x_factor, y_factor); + if (ret < 0) + return ret; + } + + av_log(inlink->dst, AV_LOG_DEBUG, "subscale output - size %dx%d pts: %"PRId64" areas: %d\n", frame->width, frame->height, frame->subtitle_timing.start_pts, frame->num_subtitle_areas); + + if (s->use_caching) { + av_frame_free(&s->cache_frame); + s->cache_frame = av_frame_clone(frame); + } + + return ff_filter_frame(outlink, frame); + } + + return ret; +} + +#define OFFSET(x) offsetof(SubScaleContext, x) +#define FLAGS (AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_FILTERING_PARAM) + +static const AVOption subscale_options[] = { + { "margin_h", "horizontal border", OFFSET(margin_h_expr), AV_OPT_TYPE_STRING, .flags = FLAGS }, + { "margin_v", "vertical border", OFFSET(margin_v_expr), AV_OPT_TYPE_STRING, .flags = FLAGS }, + { "w", "Output video width", OFFSET(w_expr), AV_OPT_TYPE_STRING, .flags = FLAGS }, + { "width", "Output video width", OFFSET(w_expr), AV_OPT_TYPE_STRING, .flags = FLAGS }, + { "h", "Output video height", OFFSET(h_expr), AV_OPT_TYPE_STRING, .flags = FLAGS }, + { "height","Output video height", OFFSET(h_expr), AV_OPT_TYPE_STRING, .flags = FLAGS }, + { "force_original_aspect_ratio", "decrease or increase w/h if necessary to keep the original AR", OFFSET(force_original_aspect_ratio), AV_OPT_TYPE_INT, { .i64 = 0}, 0, 2, FLAGS, "force_oar" }, + { "disable", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = 0 }, 0, 0, FLAGS, "force_oar" }, + { "decrease", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = 1 }, 0, 0, FLAGS, "force_oar" }, + { "increase", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = 2 }, 0, 0, FLAGS, "force_oar" }, + { "scale_mode", "specify how to scale subtitles", OFFSET(scale_mode), AV_OPT_TYPE_INT, {.i64 = SSM_UNIFORM}, 0, SSM_UNIFORM_NO_REPOSITION, FLAGS, "scale_mode" }, + { "none", "no common scaling", 0, AV_OPT_TYPE_CONST, {.i64=SSM_NONE}, .flags = FLAGS, .unit = "scale_mode" }, + { "uniform", "uniformly scale and reposition", 0, AV_OPT_TYPE_CONST, {.i64=SSM_UNIFORM}, .flags = FLAGS, .unit = "scale_mode" }, + { "uniform_no_reposition", "uniformly scale but keep positions", 0, AV_OPT_TYPE_CONST, {.i64=SSM_UNIFORM_NO_REPOSITION}, .flags = FLAGS, .unit = "scale_mode" }, + { "use_caching", "Cache output frames", OFFSET(use_caching), AV_OPT_TYPE_BOOL, { .i64 = 1}, 0, 1, .flags = FLAGS }, + + { "arrange_h", "specify how to arrange subtitles horizontally", OFFSET(arrange_mode_h), AV_OPT_TYPE_INT, {.i64 = SAM_NONE}, 0, SAM_SNAPALIGNMENT_AND_SCALE, FLAGS, "arrange" }, + { "arrange_v", "specify how to arrange subtitles vertically", OFFSET(arrange_mode_v), AV_OPT_TYPE_INT, {.i64 = SAM_NONE}, 0, SAM_SNAPALIGNMENT_AND_SCALE, FLAGS, "arrange" }, + { "none", "no repositioning", 0, AV_OPT_TYPE_CONST, {.i64=SAM_NONE}, .flags = FLAGS, .unit = "arrange" }, + { "margin_no_scale", "move subs inside border when possible", 0, AV_OPT_TYPE_CONST, {.i64=SAM_ENSUREMARGIN_NO_SCALE,}, .flags = FLAGS, .unit = "arrange" }, + { "margin_and_scale", "move subs inside border and scale as needed", 0, AV_OPT_TYPE_CONST, {.i64=SAM_ENSUREMARGIN_AND_SCALE,}, .flags = FLAGS, .unit = "arrange" }, + { "snapalign_no_scale", "snap subs to near/far/center when possible", 0, AV_OPT_TYPE_CONST, {.i64=SAM_SNAPALIGNMENT_NO_SCALE,}, .flags = FLAGS, .unit = "arrange" }, + { "snapalign_and_scale", "snap subs to near/far/center and scale as needed", 0, AV_OPT_TYPE_CONST, {.i64=SAM_SNAPALIGNMENT_AND_SCALE,}, .flags = FLAGS, .unit = "arrange" }, + + { "eval", "specify when to evaluate expressions", OFFSET(eval_mode), AV_OPT_TYPE_INT, {.i64 = EVAL_MODE_INIT}, 0, EVAL_MODE_NB-1, FLAGS, "eval" }, + { "init", "eval expressions once during initialization", 0, AV_OPT_TYPE_CONST, {.i64=EVAL_MODE_INIT}, .flags = FLAGS, .unit = "eval" }, + { "frame", "eval expressions during initialization and per-frame", 0, AV_OPT_TYPE_CONST, {.i64=EVAL_MODE_FRAME}, .flags = FLAGS, .unit = "eval" }, + { "num_colors", "number of palette colors in output", OFFSET(num_output_colors), AV_OPT_TYPE_INT, {.i64 = 256 }, 2, 256, .flags = FLAGS }, + { "bitmap_width_align", "Bitmap width alignment", OFFSET(bitmap_width_align), AV_OPT_TYPE_INT, {.i64 = 2 }, 1, 256, .flags = FLAGS }, + { "bitmap_height_align", "Bitmap height alignment", OFFSET(bitmap_height_align), AV_OPT_TYPE_INT, {.i64 = 2 }, 1, 256, .flags = FLAGS }, + { .name = NULL } +}; + +static const AVClass subscale_class = { + .class_name = "subscale", + .item_name = av_default_item_name, + .option = subscale_options, + .version = LIBAVUTIL_VERSION_INT, + .category = AV_CLASS_CATEGORY_FILTER, +}; + +static const AVFilterPad inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .filter_frame = filter_frame, + .config_props = config_input, + }, +}; + +static const AVFilterPad outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .config_props = config_output, + }, +}; + +const AVFilter ff_sf_subscale = { + .name = "subscale", + .description = NULL_IF_CONFIG_SMALL("Scale graphical subtitles."), + .init_dict = init_dict, + .uninit = uninit, + .priv_size = sizeof(SubScaleContext), + .priv_class = &subscale_class, + FILTER_INPUTS(inputs), + FILTER_OUTPUTS(outputs), + FILTER_SINGLE_SUBFMT(AV_SUBTITLE_FMT_BITMAP), +}; + From patchwork Tue Oct 25 09:13:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 38994 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2686166pzb; Tue, 25 Oct 2022 02:17:08 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7Yq99ItE9O942uUdamKHGBU0xuXaTuM/Dpn/kbTi8vV59hCsPAUX8E8gm1IkwMvfXhNcji X-Received: by 2002:aa7:c491:0:b0:461:7f0c:c573 with SMTP id m17-20020aa7c491000000b004617f0cc573mr14821950edq.8.1666689428041; Tue, 25 Oct 2022 02:17:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666689428; cv=none; d=google.com; s=arc-20160816; b=JyWjzGZerxI0Z13Jhn+3ploZYQ6eNVw+izgRJKjhHEGs2nAdvUH8Qs4K+o9vR0dSor 9lzSZoUljAgpMg1zlp8TsFqib05qcLkxzEJbuxl9UhBlayAqFoeaTOB7miA34td7lv1Z Tgg14kqRfdbIib/oROp4YX60Q6Qy+nvKZzPm89a/BbE3VdkQ2P6a2aShc6FLmQTGKyL/ JkPuKO2GjSeEVu15aIz6WiqVrvFCkJZOv1MlhZHW1p9MDMg/tUuEuvmnTnVlOgY0pqJu 7qLtzjgDChVJI7DeL3IFhWQ7S37y6HHqV6KVKRAMS69IEHky2cuQgX4ofJN7ZbXrR9bm LOZA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:references :in-reply-to:message-id:from:dkim-signature:delivered-to; bh=FqPXkR1G/Oc9m+hP1c8qQtnATzHD/4g3hAag2RyrLZ4=; b=c/1fzs9cFnsd+E4h4nCH4zsIh1se6M/kpRHatXx9FZENTm0cV8Hn3arh6UmyVBgFqd sD9gxZH5OC2QgNBsUWZwLlFkXtsR552fZ+WqMUDwp36hrCMesLZizMhPJHPzVGT2TO+D 6d5I0K2pUknEG/3f4KLasPg+8IVERiahuaq3NnDoFbcpRLBGZUTbr29XouISZ8jYZzzv rOuTSDeBtaePWc1fG+wq7RriXWE4ncoptZArAGsCrEQLKIk71CRV9aF0qiYY31WP0o6a GRwJ6EpSYa2FHwY8eRv6S5gxBWEIAU3ksGngePUFGTEaId7EFiGyDTjem6fJx7dBJijn K+Tg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=cp6TXXMs; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id c20-20020aa7c754000000b0045878af0adbsi2025861eds.393.2022.10.25.02.17.07; Tue, 25 Oct 2022 02:17:08 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=cp6TXXMs; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 83CB868BD7D; Tue, 25 Oct 2022 12:14:18 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf1-f172.google.com (mail-pf1-f172.google.com [209.85.210.172]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id F246168BD5D for ; Tue, 25 Oct 2022 12:14:14 +0300 (EEST) Received: by mail-pf1-f172.google.com with SMTP id y13so6694070pfp.7 for ; Tue, 25 Oct 2022 02:14:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=U3C6zAPJ90isDnbq7hWumI9NFA0Mg+WsozavtQilfoM=; b=cp6TXXMsJ7igVElATRiuQILxye0hjC93dSMilL8UcDNwNjpaXIlSEtuoruy0NAO0wk vOah4iAICTjGp+L0KMExRHr3U8A+f0KcJf+bbH8+v8OUDW9OoaRJmVlB+24o9f/H8qE9 CMNLlShRk8ayJFHHf0ZY23VpNFhQ3mYNQa81SWp+TvCr3u7uq/3qG8TaZjn250lXveNq zXHY5x1dd0UfAkppUWB3BEc1659MlqYW6moJ5dP/JS3aDoLf0vQvQ8hI/ayQ8LY5cT6m bLwUcaNUY+mqvhSUj1908v52nE7cSQheGFx93lPk9lcG2Vgc1/UAdgOpuqC82WQ3JocJ 5zAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=U3C6zAPJ90isDnbq7hWumI9NFA0Mg+WsozavtQilfoM=; b=GOKmQ48iIxzwtwnC2ZutvNOEhEEZ+1lLOgqQitKr/BZd22Ew+4aPHDzVIfryWOaCbe i3M1Ix1F+22Si1Pk6AuE01A+TIoiqigYoxl0dZeSPN9Nv9jz+ZlcZQEF8J7qLze3oPaB TIKal86f6Wubyl/PQ3eIUIfHuV9JxUc/mwDtQq2VahuhtvMVn9DFqr4tKtyK1n3jeTpo p9cLCpyLtaQBBeg2dnxRp4KtKMZMHlqJW/jglClwZMScekloOf2SDbP5OwbySBIbb8vf efQ40O7NgCR1Nt+NDBys4ykh3XT+7zuIRUhf+dQAoUPi+dKrpdkTmtA/sa3PFCow744P V8Nw== X-Gm-Message-State: ACrzQf0NZ1ywuXHwfuohcVXvN2A+m4TVYW/KdqGEKgoywEpifJdLtb4r /jBA7OPmqrXvCm+7WQ4zoP/UI7UZV6Y= X-Received: by 2002:a63:ed01:0:b0:445:4345:4a21 with SMTP id d1-20020a63ed01000000b0044543454a21mr32164328pgi.404.1666689253007; Tue, 25 Oct 2022 02:14:13 -0700 (PDT) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id n66-20020a622745000000b0053b723a74f7sm1037638pfn.90.2022.10.25.02.14.11 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Oct 2022 02:14:12 -0700 (PDT) From: softworkz X-Google-Original-From: softworkz Message-Id: <5874f5c452381e99b40903313c2be26f063977bb.1666689226.git.ffmpegagent@gmail.com> In-Reply-To: References: Date: Tue, 25 Oct 2022 09:13:41 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v9 20/25] avfilter/subfeed: add subtitle feed filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: softworkz Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: HQfhGbVn31EN From: softworkz Signed-off-by: softworkz --- libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/sf_subfeed.c | 412 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 414 insertions(+) create mode 100644 libavfilter/sf_subfeed.c diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 9dbac707d1..4a8badeddf 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -593,6 +593,7 @@ OBJS-$(CONFIG_CENSOR_FILTER) += sf_textmod.o OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o OBJS-$(CONFIG_SPLITCC_FILTER) += sf_splitcc.o OBJS-$(CONFIG_STRIPSTYLES_FILTER) += sf_stripstyles.o +OBJS-$(CONFIG_SUBFEED_FILTER) += sf_subfeed.o OBJS-$(CONFIG_SUBSCALE_FILTER) += sf_subscale.o OBJS-$(CONFIG_TEXTMOD_FILTER) += sf_textmod.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 9c64f3f585..fe5f815731 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -581,6 +581,7 @@ extern const AVFilter ff_sf_graphicsub2text; extern const AVFilter ff_sf_showspeaker; extern const AVFilter ff_sf_splitcc; extern const AVFilter ff_sf_stripstyles; +extern const AVFilter ff_sf_subfeed; extern const AVFilter ff_sf_subscale; extern const AVFilter ff_sf_textmod; diff --git a/libavfilter/sf_subfeed.c b/libavfilter/sf_subfeed.c new file mode 100644 index 0000000000..3227861ac0 --- /dev/null +++ b/libavfilter/sf_subfeed.c @@ -0,0 +1,412 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * subtitle filter for feeding subtitle frames into a filtergraph in a contiguous way + * + * + * also supports + * - duration fixup + * delaying a subtitle event with unknown duration and infer duration from the + * start time of the subsequent subtitle + * - scattering + * splitting a subtitle event with unknown duration into multiple ones with + * a short and fixed duration + * + */ + +#include "filters.h" +#include "libavutil/opt.h" +#include "subtitles.h" +#include "libavutil/avassert.h" + +enum SubFeedMode { + FM_REPEAT, + FM_SCATTER, + FM_FORWARD, +}; + +typedef struct SubFeedContext { + const AVClass *class; + enum AVSubtitleType format; + enum SubFeedMode mode; + + AVRational frame_rate; + int fix_durations; + int fix_overlap; + + int current_frame_isnew; + int eof; + int got_first_input; + int need_frame; + int64_t next_pts_offset; + int64_t recent_subtitle_pts; + + int64_t counter; + + /** + * Queue of frames waiting to be filtered. + */ + FFFrameQueue fifo; + +} SubFeedContext; + +static int64_t ms_to_avtb(int64_t ms) +{ + return av_rescale_q(ms, (AVRational){ 1, 1000 }, AV_TIME_BASE_Q); +} + +static int64_t avtb_to_ms(int64_t avtb) +{ + return av_rescale_q(avtb, AV_TIME_BASE_Q, (AVRational){ 1, 1000 }); +} + +static int init(AVFilterContext *ctx) +{ + SubFeedContext *s = ctx->priv; + + ff_framequeue_init(&s->fifo, NULL); + + return 0; +} + +static void uninit(AVFilterContext *ctx) +{ + SubFeedContext *s = ctx->priv; + ff_framequeue_free(&s->fifo); +} + +static int config_input(AVFilterLink *link) +{ + ////const subfeedContext *context = link->dst->priv; + + return 0; +} + +static int query_formats(AVFilterContext *ctx) +{ + AVFilterFormats *formats; + AVFilterLink *inlink0 = ctx->inputs[0]; + AVFilterLink *outlink0 = ctx->outputs[0]; + static const enum AVSubtitleType subtitle_fmts[] = { AV_SUBTITLE_FMT_BITMAP, AV_SUBTITLE_FMT_ASS, AV_SUBTITLE_FMT_NB }; + int ret; + + formats = ff_make_format_list(subtitle_fmts); + + if ((ret = ff_formats_ref(formats, &inlink0->outcfg.formats)) < 0) + return ret; + + if ((ret = ff_formats_ref(formats, &outlink0->incfg.formats)) < 0) + return ret; + + return 0; +} + +static int config_output(AVFilterLink *outlink) +{ + SubFeedContext *s = outlink->src->priv; + const AVFilterLink *inlink = outlink->src->inputs[0]; + + outlink->time_base = AV_TIME_BASE_Q; + outlink->format = inlink->format; + outlink->w = inlink->w; + outlink->h = inlink->h; + + if (s->mode == FM_FORWARD) + outlink->frame_rate = (AVRational) { 1, 0 }; + else + outlink->frame_rate = s->frame_rate; + + return 0; +} + +static int request_frame(AVFilterLink *outlink) +{ + SubFeedContext *s = outlink->src->priv; + AVFilterLink *inlink = outlink->src->inputs[0]; + int64_t last_pts = outlink->current_pts; + int64_t next_pts; + int64_t interval = ms_to_avtb((int64_t)(av_q2d(av_inv_q(outlink->frame_rate)) * 1000)); + AVFrame *out; + int status; + + if (s->mode == FM_FORWARD) + return ff_request_frame(inlink); + + s->counter++; + if (interval == 0) + interval = ms_to_avtb(200); + + status = ff_outlink_get_status(inlink); + if (status == AVERROR_EOF) + s->eof = 1; + + if (s->eof) + return AVERROR_EOF; + + if (!s->got_first_input && inlink->current_pts != AV_NOPTS_VALUE) { + + s->got_first_input = 1; + next_pts = av_rescale_q(inlink->current_pts, inlink->time_base, AV_TIME_BASE_Q); + if (next_pts < last_pts) + next_pts = last_pts + interval; + + } else if (last_pts == AV_NOPTS_VALUE) + next_pts = av_rescale_q(inlink->current_pts, inlink->time_base, AV_TIME_BASE_Q); + else + next_pts = last_pts + interval; + + if (next_pts == AV_NOPTS_VALUE) + next_pts = 0; + + if (s->next_pts_offset) { + av_log(outlink->src, AV_LOG_VERBOSE, "Subtracting next_pts_offset: %"PRId64" \n", s->next_pts_offset); + next_pts -= s->next_pts_offset; + s->next_pts_offset = 0; + } + +retry: + if (ff_framequeue_queued_frames(&s->fifo) && !s->current_frame_isnew) { + + const AVFrame *current_frame = ff_framequeue_peek(&s->fifo, 0); + const int64_t sub_end_time = current_frame->subtitle_timing.start_pts + current_frame->subtitle_timing.duration; + + if (ff_framequeue_queued_frames(&s->fifo) > 1) { + const AVFrame *next_frame = ff_framequeue_peek(&s->fifo, 1); + if (next_pts + interval > next_frame->subtitle_timing.start_pts) { + AVFrame *remove_frame = ff_framequeue_take(&s->fifo); + av_frame_free(&remove_frame); + s->current_frame_isnew = 1; + goto retry; + } + } + + if (next_pts > sub_end_time) { + AVFrame *remove_frame = ff_framequeue_take(&s->fifo); + av_frame_free(&remove_frame); + s->current_frame_isnew = 1; + goto retry; + } + } + + if (ff_framequeue_queued_frames(&s->fifo)) { + AVFrame *current_frame = ff_framequeue_peek(&s->fifo, 0); + + if (current_frame && current_frame->subtitle_timing.start_pts <= next_pts + interval) { + if (!s->current_frame_isnew) + current_frame->repeat_sub++; + + out = av_frame_clone(current_frame); + + if (!out) + return AVERROR(ENOMEM); + + if (!s->current_frame_isnew) { + out->pts = next_pts; + } else { + out->pts = out->subtitle_timing.start_pts; + + if (out->pts < next_pts) + out->pts = next_pts; + + s->next_pts_offset = (out->pts - next_pts) % interval; + } + + if (s->mode == FM_SCATTER) { + const int64_t sub_end_time = current_frame->subtitle_timing.start_pts + current_frame->subtitle_timing.duration; + + if (s->current_frame_isnew == 1 && current_frame->subtitle_timing.start_pts < out->pts) { + const int64_t diff = out->pts - current_frame->subtitle_timing.start_pts; + current_frame->subtitle_timing.duration -= diff; + } + + out->repeat_sub = 0; + out->subtitle_timing.start_pts = out->pts; + out->subtitle_timing.duration = interval; + av_assert1(out->pts >= next_pts); + av_assert1(out->pts < next_pts + interval); + av_assert1(out->pts < sub_end_time); + + if (out->pts > next_pts) + out->subtitle_timing.duration -= out->pts - next_pts; + + if (sub_end_time < next_pts + interval) { + const int64_t diff = next_pts + interval - sub_end_time; + av_assert1(diff <= out->subtitle_timing.duration); + out->subtitle_timing.duration -= diff; + } + } + + s->current_frame_isnew = 0; + s->recent_subtitle_pts = out->subtitle_timing.start_pts; + + av_log(outlink->src, AV_LOG_DEBUG, "Output1 frame pts: %"PRId64" subtitle_pts: %"PRId64" repeat_frame: %d\n", + out->pts, out->subtitle_timing.start_pts, out->repeat_sub); + + return ff_filter_frame(outlink, out); + } + } + + if (ff_framequeue_queued_frames(&s->fifo) == 0) { + status = ff_request_frame(inlink); + if (status == AVERROR_EOF) { + s->eof = 1; + return status; + } + + if (s->counter > 1 && s->counter % 2) + return 0; + } + + out = ff_get_subtitles_buffer(outlink, outlink->format); + out->pts = next_pts; + out->repeat_sub = 1; + out->subtitle_timing.start_pts = s->recent_subtitle_pts; + + av_log(outlink->src, AV_LOG_DEBUG, "Output2 frame pts: %"PRId64" subtitle_pts: %"PRId64" repeat_frame: %d\n", + out->pts, out->subtitle_timing.start_pts, out->repeat_sub); + + return ff_filter_frame(outlink, out); +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *frame) +{ + AVFilterContext *ctx = inlink->dst; + SubFeedContext *s = inlink->dst->priv; + AVFilterLink *outlink = inlink->dst->outputs[0]; + const int64_t index = (int64_t)ff_framequeue_queued_frames(&s->fifo) - 1; + size_t nb_queued_frames; + int ret = 0; + + av_log(ctx, AV_LOG_VERBOSE, "frame.pts: %"PRId64" (AVTB: %"PRId64") - subtitle_timing.start_pts: %"PRId64" subtitle_timing.duration: %"PRId64" - format: %d\n", + frame->pts, av_rescale_q(frame->pts, inlink->time_base, AV_TIME_BASE_Q), frame->subtitle_timing.start_pts, frame->subtitle_timing.duration, frame->format); + + frame->pts = av_rescale_q(frame->pts, inlink->time_base, AV_TIME_BASE_Q); + + if (index < 0) { + s->current_frame_isnew = 1; + } else if (s->fix_durations || s->fix_overlap) { + AVFrame *previous_frame = ff_framequeue_peek(&s->fifo, index); + const int64_t pts_diff = frame->subtitle_timing.start_pts - previous_frame->subtitle_timing.start_pts; + nb_queued_frames = ff_framequeue_queued_frames(&s->fifo); + + if (s->fix_durations && pts_diff > 0 && previous_frame->subtitle_timing.duration > ms_to_avtb(29000)) { + av_log(ctx, AV_LOG_VERBOSE, "Previous frame (index #%"PRId64") has a duration of %"PRId64" ms, setting to %"PRId64" ms\n", + index, avtb_to_ms(previous_frame->subtitle_timing.duration), avtb_to_ms(frame->subtitle_timing.start_pts - previous_frame->subtitle_timing.start_pts)); + previous_frame->subtitle_timing.duration = frame->subtitle_timing.start_pts - previous_frame->subtitle_timing.start_pts; + } + + if (s->fix_overlap && pts_diff > 0 && previous_frame->subtitle_timing.duration > pts_diff) { + av_log(ctx, AV_LOG_VERBOSE, "Detected overlap from previous frame (index #%"PRId64") which had a duration of %"PRId64" ms, setting to the pts_diff which is %"PRId64" ms\n", + index, avtb_to_ms(previous_frame->subtitle_timing.duration), avtb_to_ms(pts_diff)); + previous_frame->subtitle_timing.duration = pts_diff; + } + + if (pts_diff <= 0) { + av_log(ctx, AV_LOG_WARNING, "The pts_diff to the previous frame (index #%"PRId64") is <= 0: %"PRId64" ms. The previous frame duration is %"PRId64" ms.\n", + index, avtb_to_ms(pts_diff), avtb_to_ms(previous_frame->subtitle_timing.duration)); + + if (s->fix_overlap) { + av_log(ctx, AV_LOG_VERBOSE, "Removing previous frame\n"); + previous_frame = ff_framequeue_take(&s->fifo); + while (nb_queued_frames > 1) { + ff_framequeue_add(&s->fifo, previous_frame); + previous_frame = ff_framequeue_take(&s->fifo); + nb_queued_frames--; + } + } + } + } + + ff_framequeue_add(&s->fifo, frame); + + nb_queued_frames = ff_framequeue_queued_frames(&s->fifo); + + if (nb_queued_frames > 3) + av_log(ctx, AV_LOG_WARNING, "frame queue count: %zu\n", nb_queued_frames); + + if (s->mode == FM_FORWARD && nb_queued_frames) { + + AVFrame *first_frame = ff_framequeue_peek(&s->fifo, 0); + + if (s->fix_overlap && nb_queued_frames < 2) { + av_log(ctx, AV_LOG_VERBOSE, "Return no frame since we have less than 2\n"); + return 0; + } + + if (s->fix_durations && first_frame->subtitle_timing.duration > ms_to_avtb(29000)) { + av_log(ctx, AV_LOG_VERBOSE, "Return no frame because first frame duration is %"PRId64" ms\n", avtb_to_ms(first_frame->subtitle_timing.duration)); + return 0; + } + + first_frame = ff_framequeue_take(&s->fifo); + return ff_filter_frame(outlink, first_frame); + } + + return ret; +} + +#define OFFSET(x) offsetof(SubFeedContext, x) +#define FLAGS (AV_OPT_FLAG_SUBTITLE_PARAM | AV_OPT_FLAG_FILTERING_PARAM) + +static const AVOption subfeed_options[] = { + { "fix_durations", "delay output and determine duration from next frame", OFFSET(fix_durations), AV_OPT_TYPE_BOOL, { .i64=1 }, 0, 1, FLAGS, NULL }, + { "fix_overlap", "delay output and adjust durations to prevent overlap", OFFSET(fix_overlap), AV_OPT_TYPE_BOOL, { .i64=0 }, 0, 1, FLAGS, NULL }, + { "mode", "set feed mode", OFFSET(mode), AV_OPT_TYPE_INT, { .i64=FM_REPEAT }, FM_REPEAT, FM_FORWARD, FLAGS, "mode" }, + { "repeat", "repeat recent while valid, send empty otherwise", 0, AV_OPT_TYPE_CONST, { .i64=FM_REPEAT }, 0, 0, FLAGS, "mode" }, + { "scatter", "subdivide subtitles into 1/framerate segments", 0, AV_OPT_TYPE_CONST, { .i64=FM_SCATTER }, 0, 0, FLAGS, "mode" }, + { "forward", "forward only (clears output framerate)", 0, AV_OPT_TYPE_CONST, { .i64=FM_FORWARD }, 0, 0, FLAGS, "mode" }, + { "rate", "output frame rate", OFFSET(frame_rate), AV_OPT_TYPE_VIDEO_RATE, { .str = "5"}, 0, INT_MAX, FLAGS, NULL },\ + { "r", "output frame rate", OFFSET(frame_rate), AV_OPT_TYPE_VIDEO_RATE, { .str = "5"}, 0, INT_MAX, FLAGS, NULL },\ + { NULL }, +}; + +AVFILTER_DEFINE_CLASS(subfeed); + +static const AVFilterPad inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .filter_frame = filter_frame, + .config_props = config_input, + }, +}; + +static const AVFilterPad outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .request_frame = request_frame, + .config_props = config_output, + }, +}; + +const AVFilter ff_sf_subfeed = { + .name = "subfeed", + .description = NULL_IF_CONFIG_SMALL("Control subtitle frame timing and flow in a filtergraph"), + .init = init, + .uninit = uninit, + .priv_size = sizeof(SubFeedContext), + .priv_class = &subfeed_class, + FILTER_INPUTS(inputs), + FILTER_OUTPUTS(outputs), + FILTER_QUERY_FUNC(query_formats), +}; From patchwork Tue Oct 25 09:13:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 38995 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2686244pzb; Tue, 25 Oct 2022 02:17:18 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5ZPhJOvGYkMJX21O0bxuyFebXLY3pX+4mDMzuRdi8H7ju4SKKeLbwy8PGUuEEjz46BXzpE X-Received: by 2002:a05:6402:688:b0:461:a1c1:b66d with SMTP id f8-20020a056402068800b00461a1c1b66dmr11930859edy.191.1666689438546; Tue, 25 Oct 2022 02:17:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666689438; cv=none; d=google.com; s=arc-20160816; b=NC67Xds0kpbjsSDm6OiOGWaL4T2wczQcFERu0bvB7Q+va9SOd13OXlyGjaKw8MPTTB 2JSghrfpyaNW86KqHg8ebRhPhCGzTHmGrgpkMAa8qSv3kwhTQjLYUuA0aYF7gWBlmZFN bPhmdkLmN9Zr6isLpKVnVlLKYd8WaCsYE1+/Rkio6IX9NdRpM5NLBtksZVRNh2d15JKW +cc2ihOyQr3GMKD6zri2AAoEsWGSz/sqrjcMFSqnylKsythW0vlVKhYD5weaORFihBKs rxcgBFMQdAK/MA6KZH7jcWX3bMppkYFGLzwnWI8YBPQguzdViMEbr2uZw9V8vY43YmgJ 5N9Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:references :in-reply-to:message-id:from:dkim-signature:delivered-to; bh=ssjItSE0PxUwbg9NkrJh+AnSOlVxX0AGuh8b6j25mmA=; b=PD/pkvcTgxgmevr0j91suHGStV295m7mEAI9wRLXlnNiH/kfhR0ZwtbMBjyU07HBtF QbDHf9dblHessN/WGalxBto/aN7LTjgayThhPajzGgUwUba0Z2eoK6fgpvqAKIU1ltjH 18I2kfE4/8AI5MmnMB7PQ+hKp4glSeYzNwn7hSxZ0l9447jBvHYniHJ8NDAbYDhCU08U dwjKrYiTzC4MFPvY7ImK/jbxdYI4gTdJki6Rbc4KsyHGsJ0ys5PiqP/F4G96xTHCwR4j uzmPIIAJO0g4U7gSP7dvGuZGsrgR+f6rxGDUuJaZ2MtRo29LaSC25dzH7/4H8sZns4l0 KBUg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=PlUWuSuR; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id z19-20020a05640240d300b0046230e4fd03si237137edb.615.2022.10.25.02.17.18; Tue, 25 Oct 2022 02:17:18 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=PlUWuSuR; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8487168BD2B; Tue, 25 Oct 2022 12:14:19 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2B45A68BD65 for ; Tue, 25 Oct 2022 12:14:15 +0300 (EEST) Received: by mail-pf1-f174.google.com with SMTP id d10so11245709pfh.6 for ; Tue, 25 Oct 2022 02:14:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=N/TiiB+XrT/QLeXdAl438Ev+pfX0nepCQiv5Bxk04bU=; b=PlUWuSuRRsTt2Ce/jS6eUDzw+QSNtQyrudF0KBzBLvXPeb6tqfq7YSS0bSyidRGT6R TiLTNXgWojo3oPUNH3LZwNMj+WV001CvSFvS4RK37t1NOVF+psOK9tjUuL5Ue71yY/Bw foGJVDCbQHLxCxwIjsxyJ2vomc7LyccSNKEpjGYUoP57L1a4VVXjeAMgjgJSMihr0l2G HTeZS68hyGEYBFOI5b4TjIMl9e30pkHgKy5NO/ZR/KGXjZ1zQLKSoW8ZrfcYNEOgIUZt tPyGSuQtyEsiOxU6G1B6qJRmgQ+J/fwOis3nG0d9DfC6ZOdY/ieNHgxqtPUsFC/ULIzT tdSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=N/TiiB+XrT/QLeXdAl438Ev+pfX0nepCQiv5Bxk04bU=; b=lczzYx7WqJlWWdqjNVi67HX+szb9VCT6K6mT+2PQ7IS/y8WAy94zeNZ/jdz/7PbRnf 07G4oFfxZotGW4qmhXQrolkWiZrk1U504JQCqiI/dEfS07JmRUYTPIqQAC+Pul2LrW4d Ly5bF5hAkunIkM2nyQEcV4CJjyzfblIfZ7r14UGb3OAWt9gTTHFCFoMKdMPAA5BVmfVR GrYNoRs9EluzaC7f0JNGHVsE8/L1X8dBD2u9Di7B5N44Me9Eg2WEKxzxpsvlube6qISV uvvBuDIpc3g81yWG0GIZSjB84k6lD7LdOZRNRRdglfkCh8jedGrnfJSqMqp2Xkog6Emg P0pw== X-Gm-Message-State: ACrzQf0xaE8QITDrR4Iq6T4iX76bRlTYDx8PD40KJOJ1+uto5GN86mHi ZHAKAMij87v/UzV7J8MktYJyGPeEKc4= X-Received: by 2002:a63:dc4e:0:b0:46e:acf4:7589 with SMTP id f14-20020a63dc4e000000b0046eacf47589mr21274363pgj.170.1666689254038; Tue, 25 Oct 2022 02:14:14 -0700 (PDT) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id z24-20020a17090acb1800b00212d4cbcbfdsm1033327pjt.22.2022.10.25.02.14.13 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Oct 2022 02:14:13 -0700 (PDT) From: softworkz X-Google-Original-From: softworkz Message-Id: <938d0ef0cccd2ed7c84812abf0bf9a4ecd6cf081.1666689226.git.ffmpegagent@gmail.com> In-Reply-To: References: Date: Tue, 25 Oct 2022 09:13:42 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v9 21/25] avfilter/text2graphicsub: Added text2graphicsub subtitle filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: softworkz Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: KelRTri9bvTJ From: softworkz Added a text2graphicsub subtitle filter which converts text-based subtitle tracks to bitmap-based subtitle tracks. The filter uses libass to render the subtitles. It takes as parameters an output height and width, as well as a number of colors in the output palette as well as sources of fonts. All its arguments are optional. Reviewed-by: softworkz Signed-off-by: softworkz Signed-off-by: tcoza --- configure | 1 + doc/filters.texi | 51 +++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/sf_text2graphicsub.c | 634 +++++++++++++++++++++++++++++++ 5 files changed, 688 insertions(+) create mode 100644 libavfilter/sf_text2graphicsub.c diff --git a/configure b/configure index 8968851571..3703aa216b 100755 --- a/configure +++ b/configure @@ -3759,6 +3759,7 @@ subscale_filter_deps="swscale avcodec" subtitles_filter_deps="avformat avcodec libass" super2xsai_filter_deps="gpl" pixfmts_super2xsai_test_deps="super2xsai_filter" +text2graphicsub_filter_deps="avformat avcodec libass" textsub2video_filter_deps="avcodec libass" tinterlace_filter_deps="gpl" tinterlace_merge_test_deps="tinterlace_filter" diff --git a/doc/filters.texi b/doc/filters.texi index 5b058e8f3f..ebd398386b 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -28118,6 +28118,57 @@ ffmpeg -y -y -loglevel verbose -i "https://streams.videolan.org/samples/sub/larg @end example @end itemize +@section text2graphicsub + +Converts a text-based subtitle track to a bitmap-based subtitle track. + +The text2graphicsub filter uses libass to render all the subtitle +frames in a text-based subtitle track (such as srt or ass) to allow +its encoding with a bitmap-based subtitle encoder (such as dvd_subtitle). + +Inputs: +- 0: Subtitles [text] + +Outputs: +- 0: Subtitles [bitmap] + +It accepts the following parameters: + +@table @option + +@item s, size +Set the size of the output. Default from input track. + +@item n, num_colors +Set the number of palette colors for output images, +Range [2,256]. Default 16. + +@item f, filename +Set the media container from which to extract fonts required +for rendering the subtitles, usually the same as the input +track's media container. Can be omitted. + +@item fd, fontsdir +Set the directory from which to load fonts required for +rendering the subtitles. Can be used with 'fonts'. Can be omitted. + +@item ss, stripstyles +Remove certain styles from an ass track which do not render well. +Stripped styles include blurs, fades, and other animations. +Default yes. + +@end table + +@subsection Examples + +@itemize +@item +Convert a video's text-based subtitle track to a dvd_subtitle subtitle track. +@example +ffmpeg -i video.mkv -map 0:v -map 0:a -filter_complex [0:s]text2graphicsub=f=video.mkv[dvd_sub] -map [dvd_sub] -c copy -c:s dvd_subtitle output.mkv +@end example +@end itemize + @c man end SUBTITLE FILTERS @chapter Multimedia Filters diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 4a8badeddf..c6dfa517e8 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -595,6 +595,7 @@ OBJS-$(CONFIG_SPLITCC_FILTER) += sf_splitcc.o OBJS-$(CONFIG_STRIPSTYLES_FILTER) += sf_stripstyles.o OBJS-$(CONFIG_SUBFEED_FILTER) += sf_subfeed.o OBJS-$(CONFIG_SUBSCALE_FILTER) += sf_subscale.o +OBJS-$(CONFIG_TEXT2GRAPHICSUB_FILTER) += sf_text2graphicsub.o OBJS-$(CONFIG_TEXTMOD_FILTER) += sf_textmod.o # multimedia filters diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index fe5f815731..8779141aad 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -583,6 +583,7 @@ extern const AVFilter ff_sf_splitcc; extern const AVFilter ff_sf_stripstyles; extern const AVFilter ff_sf_subfeed; extern const AVFilter ff_sf_subscale; +extern const AVFilter ff_sf_text2graphicsub; extern const AVFilter ff_sf_textmod; /* those filters are part of public or internal API, diff --git a/libavfilter/sf_text2graphicsub.c b/libavfilter/sf_text2graphicsub.c new file mode 100644 index 0000000000..1fd7a76c53 --- /dev/null +++ b/libavfilter/sf_text2graphicsub.c @@ -0,0 +1,634 @@ +/* + * Copyright (c) 2021 tcoza + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * convert text subtitles to bitmap subtitles filter + */ + +#include +#include "libavutil/avstring.h" + +#include "internal.h" +#include "avfilter.h" +#include "drawutils.h" +#include "libavutil/opt.h" +#include "libavutil/buffer.h" +#include "libavutil/internal.h" +#include "libavformat/avformat.h" +#include "libavcodec/elbg.h" +#include "libavutil/ass_split_internal.h" +#include "libavutil/ass_internal.h" +#include "libavutil/lfg.h" + +typedef struct PalettizeContext { + int *codeword, *codebook; + int *codeword_closest_codebook_idxs; + int r_idx, g_idx, b_idx, a_idx; + struct ELBGContext *elbg; + AVLFG lfg; +} PalettizeContext; + +typedef struct Text2GraphicSubContext { + const AVClass *class; + ASS_Library *library; + ASS_Renderer *renderer; + ASS_Track *track; + PalettizeContext *palettize_context; + FFDrawContext draw_context; + struct { int width; int height; } size; + int num_colors, stripstyles; + char *filename, *fontsdir, *force_style; + int got_header; +} Text2GraphicSubContext; + +typedef struct DialogContext { + int is_animated; + int has_text; +} DialogContext; + +static void dialog_text_cb(void *priv, const char *text, int len) +{ + DialogContext *s = priv; + if (!s->is_animated ) + s->has_text = 1; +} + +static void dialog_drawing_mode_cb(void *priv, int scale) +{ + DialogContext *s = priv; + s->is_animated = 1; +} + +static void dialog_animate_cb(void *priv, int t1, int t2, int accel, char *style) +{ + DialogContext *s = priv; + s->is_animated = 1; +} + +static void dialog_move_cb(void *priv, int x1, int y1, int x2, int y2, int t1, int t2) +{ + DialogContext *s = priv; + if (t1 >= 0 || t2 >= 0) + s->is_animated = 1; +} + +static const ASSCodesCallbacks dialog_callbacks = { + .text = dialog_text_cb, + .drawing_mode = dialog_drawing_mode_cb, + .animate = dialog_animate_cb, + .move = dialog_move_cb, +}; + +static char *process_dialog(const char *ass_line) +{ + DialogContext dlg_ctx = { 0 }; + ASSDialog *dialog = avpriv_ass_split_dialog(NULL, ass_line); + AVBPrint buffer; + char *result = NULL; + + if (!dialog) + return NULL; + + av_bprint_init(&buffer, 512, AV_BPRINT_SIZE_UNLIMITED); + + avpriv_ass_filter_override_codes(&dialog_callbacks, &dlg_ctx, dialog->text, &buffer, ASS_SPLIT_BASIC); + + if (av_bprint_is_complete(&buffer) && buffer.len > 0 && dlg_ctx.has_text > 0) + result = avpriv_ass_get_dialog_ex(dialog->readorder, dialog->layer, dialog->style, dialog->name, dialog->margin_l, dialog->margin_r, dialog->margin_v, dialog->effect, buffer.str); + + av_bprint_finalize(&buffer, NULL); + avpriv_ass_free_dialog(&dialog); + + return result; +} + +static void palettize_image(PalettizeContext *const s, + const int w, const int h, + uint8_t *src_data, const int src_linesize, + uint8_t *dst_data, const int dst_linesize, + uint32_t *dst_pal, const int num_colors) +{ + const int codeword_length = w * h; + + /* Re-Initialize */ + s->codeword = av_realloc_f(s->codeword, codeword_length, 4 * sizeof(*s->codeword)); + s->codeword_closest_codebook_idxs = av_realloc_f( + s->codeword_closest_codebook_idxs, codeword_length, + sizeof(*s->codeword_closest_codebook_idxs)); + s->codebook = av_realloc_f(s->codebook, num_colors, 4 * sizeof(*s->codebook)); + + /* build the codeword */ + for (int k = 0, i = 0; i < h; i++) + for (int j = 0; j < w; j++) { + s->codeword[k++] = src_data[i * src_linesize + j * 4 + s->b_idx]; + s->codeword[k++] = src_data[i * src_linesize + j * 4 + s->g_idx]; + s->codeword[k++] = src_data[i * src_linesize + j * 4 + s->r_idx]; + s->codeword[k++] = src_data[i * src_linesize + j * 4 + s->a_idx]; + } + + /* compute the codebook */ + avpriv_elbg_do(&s->elbg, s->codeword, 4, + codeword_length, s->codebook, num_colors, 1, + s->codeword_closest_codebook_idxs, &s->lfg, 0); + + /* Write Palette */ + for (int i = 0; i < num_colors; i++) + dst_pal[i] = (s->codebook[i*4+3] << 24) | + (s->codebook[i*4+2] << 16) | + (s->codebook[i*4+1] << 8) | + (s->codebook[i*4+0] << 0); + + /* Write Image */ + for (int k = 0, i = 0; i < h; i++) + for (int j = 0; j < w; j++) + dst_data[i * dst_linesize + j] = + s->codeword_closest_codebook_idxs[k++]; +} + +static void init_palettizecontext(PalettizeContext **palettizecontext) +{ + uint8_t rgba_map[4]; + PalettizeContext *context = (PalettizeContext *)av_malloc(sizeof(PalettizeContext)); + context->codebook = NULL; + context->codeword = NULL; + context->codeword_closest_codebook_idxs = NULL; + context->elbg = NULL; + av_lfg_init(&context->lfg, 0xACBADF); + ff_fill_rgba_map(&rgba_map[0], AV_PIX_FMT_RGB32); + context->r_idx = rgba_map[0]; // R + context->g_idx = rgba_map[1]; // G + context->b_idx = rgba_map[2]; // B + context->a_idx = rgba_map[3]; // A + *palettizecontext = context; +} + +static void free_palettizecontext(PalettizeContext **palettizecontext) +{ + PalettizeContext *context = *palettizecontext; + if (context) { + av_freep(&context->codebook); + av_freep(&context->codeword); + av_freep(&context->codeword_closest_codebook_idxs); + avpriv_elbg_free(&context->elbg); + av_free(context); + *palettizecontext = NULL; + } +} + +/* libass supports a log level ranging from 0 to 7 */ +static const int ass_libavfilter_log_level_map[] = { + [0] = AV_LOG_FATAL, /* MSGL_FATAL */ + [1] = AV_LOG_ERROR, /* MSGL_ERR */ + [2] = AV_LOG_WARNING, /* MSGL_WARN */ + [3] = AV_LOG_WARNING, /* */ + [4] = AV_LOG_INFO, /* MSGL_INFO */ + [5] = AV_LOG_INFO, /* */ + [6] = AV_LOG_VERBOSE, /* MSGL_V */ + [7] = AV_LOG_DEBUG, /* MSGL_DBG2 */ +}; + +static void ass_log(int ass_level, const char *fmt, va_list args, void *ctx) +{ + const int ass_level_clip = av_clip(ass_level, 0, + FF_ARRAY_ELEMS(ass_libavfilter_log_level_map) - 1); + const int level = ass_libavfilter_log_level_map[ass_level_clip]; + av_vlog(ctx, level, fmt, args); + av_log(ctx, level, "\n"); +} + +static const char * const font_mimetypes[] = { + "font/ttf", + "font/otf", + "font/sfnt", + "font/woff", + "font/woff2", + "application/font-sfnt", + "application/font-woff", + "application/x-truetype-font", + "application/vnd.ms-opentype", + "application/x-font-ttf", + NULL +}; + +static int stream_is_font(AVStream* st) +{ + const AVDictionaryEntry *tag = NULL; + + if (st->codecpar->codec_type != AVMEDIA_TYPE_ATTACHMENT) + return 0; + + tag = av_dict_get(st->metadata, "mimetype", NULL, AV_DICT_MATCH_CASE); + if (!tag) return 0; + for (int i = 0; font_mimetypes[i]; i++) + if (av_strcasecmp(font_mimetypes[i], tag->value) == 0) + return 1; + return 0; +} + +#define AR(c) (((c)>>24)&0xFF) +#define AG(c) (((c)>>16)&0xFF) +#define AB(c) (((c)>> 8)&0xFF) +#define AA(c) ((0xFF-(c))&0xFF) + +static void set_area_bounds(ASS_Image *image, AVSubtitleArea *area) +{ + int x_min = INT_MAX, y_min = INT_MAX; + int x_max = 0, y_max = 0; + int stride_max = 0; + + for (ASS_Image *img = image; img != NULL; img = img->next) + { + x_min = FFMIN(x_min, img->dst_x); + y_min = FFMIN(y_min, img->dst_y); + x_max = FFMAX(x_max, img->dst_x + img->w); + y_max = FFMAX(y_max, img->dst_y + img->h); + stride_max = FFMAX(stride_max, img->dst_x + img->stride); + } + + area->x = x_min; + area->y = y_min; + area->w = FFALIGN(x_max - x_min, 2); + area->h = FFALIGN(y_max - y_min, 2); + area->linesize[0] = area->w; //stride_max - x_min; +} + +static void ass_image_to_area_palletization(Text2GraphicSubContext *context, ASS_Image *image, AVSubtitleArea *area) +{ + size_t image_rgba_size; + uint8_t *image_rgba; + + set_area_bounds(image, area); + av_log(context, AV_LOG_VERBOSE, "set_area_bounds %d,%d %dx%d\n", area->x, area->y, area->w, area->h); + + // Create rgba image + image_rgba_size = (area->linesize[0] * area->h) * 4 * sizeof(uint8_t); + image_rgba = (uint8_t *)av_mallocz(image_rgba_size); + for (; image != NULL; image = image->next) + { + uint8_t rgba_color[] = {AR(image->color), AG(image->color), AB(image->color), AA(image->color)}; + int linesize = area->linesize[0] * 4; + FFDrawColor color; + ff_draw_color(&context->draw_context, &color, rgba_color); + ff_blend_mask(&context->draw_context, &color, + &image_rgba, &linesize, area->w, area->h, + image->bitmap, image->stride, image->w, image->h, + 3, 0, image->dst_x - area->x, image->dst_y - area->y); + } + + area->nb_colors = context->num_colors; + area->buf[0] = av_buffer_alloc(image_rgba_size / 4); + + palettize_image(context->palettize_context, + area->w, area->h, + image_rgba, area->linesize[0] * 4, + area->buf[0]->data, area->linesize[0], + &area->pal[0], context->num_colors); + + // Clean up + av_free(image_rgba); +} + +static void process_header(const AVFilterContext *link, const AVFrame *frame) +{ + Text2GraphicSubContext *s = link->priv; + ASS_Track *track = s->track; + ASS_Style *style; + int sid = 0; + + if (!track) + return; + + if (frame && frame->subtitle_header) { + char *subtitle_header = (char *)frame->subtitle_header->data; + ass_process_codec_private(s->track, subtitle_header, strlen(subtitle_header)); + } + else { + char* subtitle_header = avpriv_ass_get_subtitle_header_default(0); + if (!subtitle_header) + return; + + ass_process_codec_private(s->track, subtitle_header, strlen(subtitle_header)); + av_free(subtitle_header); + } + + if (!s->track->event_format) { + s->track->event_format = av_strdup("ReadOrder, Layer, Style, Name, MarginL, MarginR, MarginV, Effect, Text"); + } + + if (s->track->n_styles == 0) { + sid = ass_alloc_style(track); + style = &s->track->styles[sid]; + style->Name = av_strdup("Default"); + style->PrimaryColour = 0xffffff00; + style->SecondaryColour = 0x00ffff00; + style->OutlineColour = 0x00000000; + style->BackColour = 0x00000080; + style->Bold = 200; + style->ScaleX = 1.0; + style->ScaleY = 1.0; + style->Spacing = 0; + style->BorderStyle = 1; + style->Outline = 2; + style->Shadow = 3; + style->Alignment = 2; + track->default_style = sid; + } + + s->got_header = 1; +} + +// Main interface functions + +static av_cold int init(AVFilterContext *ctx) +{ + Text2GraphicSubContext *context = ctx->priv; + + context->library = ass_library_init(); + ass_set_message_cb(context->library, ass_log, context); + ass_set_fonts_dir(context->library, context->fontsdir); + ass_set_extract_fonts(context->library, 1); + + if (context->filename) { + AVFormatContext *video = NULL; + int ret = avformat_open_input(&video, context->filename, NULL, NULL); + if (ret < 0) { + av_log(ctx, AV_LOG_ERROR, "Unable to open %s\n", context->filename); + return ret; + } + for (int i = 0; i < video->nb_streams; i++) { + const AVDictionaryEntry *tag; + AVStream *st = video->streams[i]; + if (!stream_is_font(st)) continue; + tag = av_dict_get(st->metadata, "filename", NULL, AV_DICT_MATCH_CASE); + if (!tag) continue; + av_log(NULL, AV_LOG_DEBUG, "Loading attached font: %s\n", tag->value); + ass_add_font(context->library, tag->value, + (char *)st->codecpar->extradata, + st->codecpar->extradata_size); + } + avformat_close_input(&video); + } + + context->renderer = ass_renderer_init(context->library); + ass_set_pixel_aspect(context->renderer, 1); + ass_set_shaper(context->renderer, 0); + ass_set_fonts(context->renderer, NULL, NULL, 1, NULL, 1); + + context->track = ass_new_track(context->library); + if (!context->track) { + av_log(ctx, AV_LOG_ERROR, "ass_new_track() failed!\n"); + return AVERROR(EINVAL); + } + + ass_set_check_readorder(context->track, 0); + + if (context->force_style) { + char **list = NULL; + char *temp = NULL; + char *ptr = av_strtok(context->force_style, ",", &temp); + int i = 0; + while (ptr) { + av_dynarray_add(&list, &i, ptr); + if (!list) { + return AVERROR(ENOMEM); + } + ptr = av_strtok(NULL, ",", &temp); + } + av_dynarray_add(&list, &i, NULL); + if (!list) { + return AVERROR(ENOMEM); + } + ass_set_style_overrides(context->library, list); + av_free(list); + } + + init_palettizecontext(&context->palettize_context); + + ff_draw_init(&context->draw_context, AV_PIX_FMT_BGRA, FF_DRAW_PROCESS_ALPHA); + + return 0; +} + +static int config_input(AVFilterLink *inlink) +{ + const AVFilterContext *ctx = inlink->dst; + Text2GraphicSubContext *context = ctx->priv; + if (!context->size.width) context->size.width = inlink->w; + if (!context->size.height) context->size.height = inlink->h; + if (!context->size.height || !context->size.width) { + av_log(NULL, AV_LOG_ERROR, "A positive height and width are required to render subtitles\n"); + return AVERROR_EXIT; + } + ass_set_frame_size(context->renderer, context->size.width, context->size.height); + ass_set_storage_size(context->renderer, inlink->w, inlink->h); + + return 0; +} + +static int config_output(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + Text2GraphicSubContext *context = ctx->priv; + + outlink->time_base = AV_TIME_BASE_Q; + outlink->format = AV_SUBTITLE_FMT_BITMAP; + outlink->w = context->size.width; + outlink->h = context->size.height; + + return 0; +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *frame) +{ + const AVFilterContext *ctx = inlink->dst; + Text2GraphicSubContext *context = ctx->priv; + AVFilterLink *outlink = ctx->outputs[0]; + AVSubtitleArea *area; + int ret; + unsigned processed_area_cnt = 0; + const int64_t start_time = av_rescale_q(frame->subtitle_timing.start_pts, AV_TIME_BASE_Q, av_make_q(1, 1000)); + const int64_t duration = av_rescale_q(frame->subtitle_timing.duration, AV_TIME_BASE_Q, av_make_q(1, 1000)); + ASS_Image *image; + + // Postpone header processing until we receive a frame with content + if (!context->got_header && frame->num_subtitle_areas > 0) + process_header(ctx, frame); + + if (frame->repeat_sub || frame->num_subtitle_areas == 0) { + av_frame_free(&frame); + return 0; + } + + ret = av_frame_make_writable(frame); + if (ret < 0) + return ret; + + if (context->stripstyles) { + for (unsigned r = 0; r < frame->num_subtitle_areas; r++) { + + area = frame->subtitle_areas[r]; + + if (area->ass) { + char *tmp = area->ass; + area->ass = process_dialog(area->ass); + + if (area->ass) { + av_log(inlink->dst, AV_LOG_DEBUG, "original: %d %s\n", r, tmp); + av_log(inlink->dst, AV_LOG_DEBUG, "stripped: %d %s\n", r, area->ass); + } + + av_free(tmp); + } + } + } + + for (unsigned r = 0; r < frame->num_subtitle_areas; r++) + { + area = frame->subtitle_areas[r]; + if (area->type != AV_SUBTITLE_FMT_ASS || area->ass == NULL) + continue; + + ass_process_chunk(context->track, area->ass, strlen(area->ass), start_time, duration); + processed_area_cnt++; + + } + + if (processed_area_cnt == 0) { + av_frame_free(&frame); + return 0; + } + + for (unsigned r = 1; r < frame->num_subtitle_areas; r++) + av_free(frame->subtitle_areas[r]); + + frame->num_subtitle_areas = 1; + area = frame->subtitle_areas[0]; + + image = ass_render_frame(context->renderer, context->track, start_time + duration / 2, NULL); + if (image == NULL) { + av_log(NULL, AV_LOG_WARNING, "failed to render ass: %s\n", area->ass); + return 0; + } + + // TODO: Split into multiple bitmaps + + ass_image_to_area_palletization(context, image, area); + area->type = AV_SUBTITLE_FMT_BITMAP; + + av_log(NULL, AV_LOG_DEBUG, "successfully rendered ass: %s\n", area->ass); + + frame->width = context->size.width; + frame->height = context->size.height; + frame->format = AV_SUBTITLE_FMT_BITMAP; + return ff_filter_frame(outlink, frame); +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + Text2GraphicSubContext *context = ctx->priv; + free_palettizecontext(&context->palettize_context); + if (context->track) ass_free_track(context->track); + if (context->renderer) ass_renderer_done(context->renderer); + if (context->library) ass_library_done(context->library); + context->track = NULL; + context->renderer = NULL; + context->library = NULL; +} + +// Copied from sf_graphicsub2text, with formats swapped +static int query_formats(AVFilterContext *ctx) +{ + AVFilterFormats *formats, *formats2; + AVFilterLink *inlink = ctx->inputs[0]; + AVFilterLink *outlink = ctx->outputs[0]; + static const enum AVSubtitleType in_fmts[] = { AV_SUBTITLE_FMT_ASS, AV_SUBTITLE_FMT_NONE }; + static const enum AVSubtitleType out_fmts[] = { AV_SUBTITLE_FMT_BITMAP, AV_SUBTITLE_FMT_NONE }; + int ret; + + /* set input format */ + formats = ff_make_format_list(in_fmts); + if ((ret = ff_formats_ref(formats, &inlink->outcfg.formats)) < 0) + return ret; + + /* set output format */ + formats2 = ff_make_format_list(out_fmts); + if ((ret = ff_formats_ref(formats2, &outlink->incfg.formats)) < 0) + return ret; + + return 0; +} + +// Filter structures + +#define OFFSET(x) offsetof(Text2GraphicSubContext, x) +#define FLAGS (AV_OPT_FLAG_SUBTITLE_PARAM|AV_OPT_FLAG_FILTERING_PARAM) + +static const AVOption text2graphicsub_options[] = { + { "s", "output size", OFFSET(size), AV_OPT_TYPE_IMAGE_SIZE, .default_val.str = NULL, .flags = FLAGS }, + { "size", "output size", OFFSET(size), AV_OPT_TYPE_IMAGE_SIZE, .default_val.str = NULL, .flags = FLAGS }, + { "n", "number of output colors", OFFSET(num_colors), AV_OPT_TYPE_INT, .min = 2, .max = 256, .default_val.i64 = 16, .flags = FLAGS }, + { "num_colors", "number of output colors", OFFSET(num_colors), AV_OPT_TYPE_INT, .min = 2, .max = 256, .default_val.i64 = 16, .flags = FLAGS }, + { "ss", "strip animations and blur styles", OFFSET(stripstyles), AV_OPT_TYPE_BOOL, .min = 0, .max = 1, .default_val.i64 = 1, .flags = FLAGS }, + { "stripstyles", "strip animations and blur styles", OFFSET(stripstyles), AV_OPT_TYPE_BOOL, .min = 0, .max = 1, .default_val.i64 = 1, .flags = FLAGS }, + { "force_style", "enforce subtitle styles", OFFSET(force_style), AV_OPT_TYPE_STRING, .default_val.str = NULL, .flags = FLAGS }, + { "f", "media file from which to load fonts", OFFSET(filename), AV_OPT_TYPE_STRING, .default_val.str = NULL, .flags = FLAGS }, + { "filename", "media file from which to load fonts", OFFSET(filename), AV_OPT_TYPE_STRING, .default_val.str = NULL, .flags = FLAGS }, + { "fd", "fonts directory", OFFSET(fontsdir), AV_OPT_TYPE_STRING, .default_val.str = NULL, .flags = FLAGS }, + { "fontsdir", "fonts directory", OFFSET(fontsdir), AV_OPT_TYPE_STRING, .default_val.str = NULL, .flags = FLAGS }, + { .name = NULL } +}; + +static const AVClass text2graphicsub_class = { + .class_name = "text2graphicsub", + .item_name = av_default_item_name, + .option = text2graphicsub_options, + .version = LIBAVUTIL_VERSION_INT, + .category = AV_CLASS_CATEGORY_FILTER, +}; + +static const AVFilterPad inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .filter_frame = filter_frame, + .config_props = config_input, + }, +}; + +static const AVFilterPad outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .config_props = config_output, + }, +}; + +const AVFilter ff_sf_text2graphicsub = { + .name = "text2graphicsub", + .description = NULL_IF_CONFIG_SMALL("Convert text subtitles to bitmap subtitles."), + .init = init, + .uninit = uninit, + .priv_size = sizeof(Text2GraphicSubContext), + .priv_class = &text2graphicsub_class, + FILTER_INPUTS(inputs), + FILTER_OUTPUTS(outputs), + FILTER_QUERY_FUNC(query_formats) +}; From patchwork Tue Oct 25 09:13:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 38996 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2686301pzb; Tue, 25 Oct 2022 02:17:28 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5yB9fuXoAW6AQAjSt5fFByY+7d49W9kZL2sb3tPADLSpL0usXu2e8xAYL3bMwxEwgQK5rF X-Received: by 2002:a17:907:7611:b0:78d:9d69:adf9 with SMTP id jx17-20020a170907761100b0078d9d69adf9mr32015182ejc.283.1666689448222; Tue, 25 Oct 2022 02:17:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666689448; cv=none; d=google.com; s=arc-20160816; b=ztA3kRe0Tr7qfCZ7M5CQ6Z0jL327igLMzFJxVh6N9LUHNThddOSJeuzXyrzRa0v2Kn Jd+GVWX3rubdCxy+MZmPtcX5RM+Ux4sdMqkdwa6Jpzg7Yo1b8P996va397S4fmv6eGOn 3nv/LWd7G4ka5RCbzQLZoIdKJeEJLYeQ/rMma9nMHBoGw90ucvTE4fRjLfqTo9SOmivF kB4ieAKcGg5MSxfSXBoXFPbTt45Gb82rdfMMOcSkVD7sPbVzVBf9mVjE5xU6cV0eOyQ6 VM2036ztSW2er7xBcQcysSAJ+2MjeFAsD/X5LjyhA+G09eJogsd3fqdiuWzCFNCAsB2A ul/A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:references :in-reply-to:message-id:from:dkim-signature:delivered-to; bh=OEfP0yq2YA1K4FU+2C6akvijFXQmWtlcjuJnOlvQq8o=; b=ufK0qfMlsOWPpqeJAZI8i9GqjkAwC9lyQ2Jt8DsLaYg+WXbof/Wu/xWLi5bSikRcat aWaUrKeZSEcQSSTkX9/CHd2G1ochoi+CTnm/df6Sn2JTgCTJeTj6wNyyZdD7/BlgfdbI LviUJ2oL4TH9h+NrCgXYWq6rdWh2lowAVTIRbdqiZZCtP4nK6MuogaMbBYxOfY8Uro+t KMCAwiWfkIvPEMey6mcXblWzfPyOAdcLr+xSidNpdGkK5cCnmle3IUAJr/mQSCmViPm9 4YG0VyFoGTSrNljvIsCaYNixzy9KteqDHkT3pbxb6fP6Tozh7QSM+YCXHQ2UtY2iySxB rWfw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=KwmLpqW2; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id y12-20020a056402270c00b00461a5271515si2375598edd.618.2022.10.25.02.17.27; Tue, 25 Oct 2022 02:17:28 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=KwmLpqW2; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 71B2A68BD88; Tue, 25 Oct 2022 12:14:20 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pg1-f180.google.com (mail-pg1-f180.google.com [209.85.215.180]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2B09468BD54 for ; Tue, 25 Oct 2022 12:14:16 +0300 (EEST) Received: by mail-pg1-f180.google.com with SMTP id 78so10891970pgb.13 for ; Tue, 25 Oct 2022 02:14:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=wCVNypdZUr4sr0/9iw4olFWEYmefmkewWHt/Bx6bvjY=; b=KwmLpqW2cC8yVlx6/8kKVK63TpN305KPjUtRq3PMK44Wrhf8VFaMkNR/tf7owMVTAo RQDs/cXL4U4O2LiwewAnosUHX5S0pA8gme/Izdk218kiTeV1DIoRTOsusQDQb0bAH179 Gp39HssDd1f0ZLBnGRkHKVi2jmA8+nVBqlTQw4PdIHgJ+LX0aeuoW2kNzcUa6pkw/AUP DiGFsjOuw7c4jCUl1yYp/UZawAoL13qJ9zyeV0D+VkwNc2HfWnORUuCuRnNfCBVnZS4n 1Bxj6adOIEYGdDDr/aAncU8rk9fH8n3U2vyK+aBGoZ7oLzwg7UQKBzueUlSw8umnOB7A qnjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=wCVNypdZUr4sr0/9iw4olFWEYmefmkewWHt/Bx6bvjY=; b=q5nPjcLgEuQIifsh7JugEYnizbvnUyr/WrPy7hjd6dZ38FluPM2G2/JstaaRk/r7Pb xahxiAKIgpMFlPzLIi8AWABn6cK0OTA76K6rGl8tOHNVo/r8wazLAsxzZlqJfr+3Pj7v LwnInWWP1ftShY5WKRxPL/cbXj8049sRv/vW8/BkIsXP+5jYEypZSDptQhnWXUGRHnjv VVRRrLrgd0qWwzwwnEfH0TYV87xZtxsVROENK0KiYa0YTFhllERLx1zUTZkVQ9EYhxAr 9ao+pBZceAf8wO1dIMECVql1NN7ubeXrM7ZDUBTMvbcOlL7wNU4OKDqFehFHvy/CoSrA h/IQ== X-Gm-Message-State: ACrzQf33iwMHXW++a9mKtE/lIX2QGz8Ny5lu7EzAPmOIbGVRRWC12+RZ ArbR89oOsVOccaAhdFClGH7iCtRgQTg= X-Received: by 2002:aa7:8c15:0:b0:56b:ead2:3950 with SMTP id c21-20020aa78c15000000b0056bead23950mr7601950pfd.77.1666689255155; Tue, 25 Oct 2022 02:14:15 -0700 (PDT) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id w184-20020a6282c1000000b0056bb21cd7basm1011293pfd.51.2022.10.25.02.14.14 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Oct 2022 02:14:14 -0700 (PDT) From: softworkz X-Google-Original-From: softworkz Message-Id: <99e5c1195224849c756451093270048a5c7037c6.1666689226.git.ffmpegagent@gmail.com> In-Reply-To: References: Date: Tue, 25 Oct 2022 09:13:43 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v9 22/25] avfilter/snull, strim: Add snull and strim filters X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: softworkz Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: QdUxgIBOUac3 From: softworkz Signed-off-by: softworkz --- configure | 2 +- libavfilter/Makefile | 2 ++ libavfilter/allfilters.c | 2 ++ libavfilter/sf_snull.c | 61 ++++++++++++++++++++++++++++++++++++++++ libavfilter/trim.c | 60 ++++++++++++++++++++++++++++++++++++++- 5 files changed, 125 insertions(+), 2 deletions(-) create mode 100644 libavfilter/sf_snull.c diff --git a/configure b/configure index 3703aa216b..3d830be10f 100755 --- a/configure +++ b/configure @@ -3842,7 +3842,7 @@ avutil_extralibs="d3d11va_extralibs nanosleep_extralibs pthreads_extralibs vaapi # programs ffmpeg_deps="avcodec avfilter avformat threads" ffmpeg_select="aformat_filter anull_filter atrim_filter format_filter - hflip_filter null_filter + snull_filter strim_filter hflip_filter null_filter transpose_filter trim_filter vflip_filter" ffmpeg_suggest="ole32 psapi shell32" ffplay_deps="avcodec avformat swscale swresample sdl2" diff --git a/libavfilter/Makefile b/libavfilter/Makefile index c6dfa517e8..4e5d33a43f 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -591,7 +591,9 @@ OBJS-$(CONFIG_NULLSINK_FILTER) += vsink_nullsink.o # subtitle filters OBJS-$(CONFIG_CENSOR_FILTER) += sf_textmod.o OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o +OBJS-$(CONFIG_SNULL_FILTER) += sf_snull.o OBJS-$(CONFIG_SPLITCC_FILTER) += sf_splitcc.o +OBJS-$(CONFIG_STRIM_FILTER) += trim.o OBJS-$(CONFIG_STRIPSTYLES_FILTER) += sf_stripstyles.o OBJS-$(CONFIG_SUBFEED_FILTER) += sf_subfeed.o OBJS-$(CONFIG_SUBSCALE_FILTER) += sf_subscale.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 8779141aad..9c84347659 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -579,7 +579,9 @@ extern const AVFilter ff_avsrc_movie; extern const AVFilter ff_sf_censor; extern const AVFilter ff_sf_graphicsub2text; extern const AVFilter ff_sf_showspeaker; +extern const AVFilter ff_sf_snull; extern const AVFilter ff_sf_splitcc; +extern const AVFilter ff_sf_strim; extern const AVFilter ff_sf_stripstyles; extern const AVFilter ff_sf_subfeed; extern const AVFilter ff_sf_subscale; diff --git a/libavfilter/sf_snull.c b/libavfilter/sf_snull.c new file mode 100644 index 0000000000..815436cfff --- /dev/null +++ b/libavfilter/sf_snull.c @@ -0,0 +1,61 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * null subtitle filter + */ + +#include "avfilter.h" +#include "internal.h" +#include "libavutil/internal.h" + +static int sconfig_output(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + AVFilterLink *inlink = ctx->inputs[0]; + + outlink->format = inlink->format; + + return 0; +} + +static const AVFilterPad avfilter_sf_snull_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + }, +}; + +static const AVFilterPad avfilter_sf_snull_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .config_props = sconfig_output, + }, +}; + +const AVFilter ff_sf_snull = { + .name = "snull", + .description = NULL_IF_CONFIG_SMALL("Pass the source unchanged to the output."), + .flags = AVFILTER_FLAG_METADATA_ONLY, + FILTER_INPUTS(avfilter_sf_snull_inputs), + FILTER_OUTPUTS(avfilter_sf_snull_outputs), +}; diff --git a/libavfilter/trim.c b/libavfilter/trim.c index ee6e821cd2..a25366497b 100644 --- a/libavfilter/trim.c +++ b/libavfilter/trim.c @@ -83,7 +83,7 @@ static int config_input(AVFilterLink *inlink) { AVFilterContext *ctx = inlink->dst; TrimContext *s = ctx->priv; - AVRational tb = (inlink->type == AVMEDIA_TYPE_VIDEO) ? + AVRational tb = (inlink->type != AVMEDIA_TYPE_AUDIO) ? inlink->time_base : (AVRational){ 1, inlink->sample_rate }; if (s->start_time != INT64_MAX) { @@ -369,3 +369,61 @@ const AVFilter ff_af_atrim = { FILTER_OUTPUTS(atrim_outputs), }; #endif // CONFIG_ATRIM_FILTER + +#if CONFIG_STRIM_FILTER + +static int sconfig_output(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + AVFilterLink *inlink = ctx->inputs[0]; + + outlink->format = inlink->format; + outlink->w = inlink->w; + outlink->h = inlink->h; + + return 0; +} + + +#define FLAGS (AV_OPT_FLAG_SUBTITLE_PARAM | AV_OPT_FLAG_FILTERING_PARAM) +static const AVOption strim_options[] = { + COMMON_OPTS + { "start_frame", "Number of the first frame that should be passed " + "to the output", OFFSET(start_frame), AV_OPT_TYPE_INT64, { .i64 = -1 }, -1, INT64_MAX, FLAGS }, + { "end_frame", "Number of the first frame that should be dropped " + "again", OFFSET(end_frame), AV_OPT_TYPE_INT64, { .i64 = INT64_MAX }, 0, INT64_MAX, FLAGS }, + { NULL } +}; +#undef FLAGS + +AVFILTER_DEFINE_CLASS(strim); + +static const AVFilterPad strim_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .filter_frame = trim_filter_frame, + .config_props = config_input, + }, +}; + +static const AVFilterPad strim_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .config_props = sconfig_output, + }, +}; + +const AVFilter ff_sf_strim = { + .name = "strim", + .description = NULL_IF_CONFIG_SMALL("Pick one continuous section from the input, drop the rest."), + .init = init, + .priv_size = sizeof(TrimContext), + .priv_class = &strim_class, + .flags = AVFILTER_FLAG_METADATA_ONLY, + FILTER_INPUTS(strim_inputs), + FILTER_OUTPUTS(strim_outputs), +}; +#endif // CONFIG_STRIM_FILTER + From patchwork Tue Oct 25 09:13:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 38997 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2686376pzb; Tue, 25 Oct 2022 02:17:39 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5WsUXtz9LwGmAaxOGohdW9GOVlrUmJrmtqLuVoPBA72pTRSk8+JUUCn1bDwXoaxBMmx8Jz X-Received: by 2002:a17:907:78a:b0:782:2223:a7cd with SMTP id xd10-20020a170907078a00b007822223a7cdmr30802629ejb.532.1666689459138; Tue, 25 Oct 2022 02:17:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666689459; cv=none; d=google.com; s=arc-20160816; b=zfdJHqcVPAYNEjtt2Ox9wcWnxaw5hC2wKuZ7QIH6wGXyfgaoTmXAWvPvFVd4bAy0uq cDCJIhss72VlQ4VFSdKBjHHu28UvdEsZ+4X4qKbKRRUNTH7A78ai33ObHDvk02XZHnK0 5LA5clEiLRcUwOlq+vOH8lVRfZOp2EIypt5PfDszGBMSQ8d0vfqZovcbO6dyDrJKbsFv sq8yv3wRTNH72v659fjM4MkXLbdymkqdSMa+ipOpKfMJxSuus3OKNjvPBBDvWOF4XcO6 OuPl/34Bt51QMq6WMgEn+8YSq1ZTQTlCxAZs6//ynCPislSR5iAWbRiSfN2f+j/V0BJU zTaA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:references :in-reply-to:message-id:from:dkim-signature:delivered-to; bh=PCA74ovJVmYyn3fx6MqkUwDEiIP0BNFN39eWBgXQCR8=; b=WBRyZWhdSJQnkW7gq4ZPbc8b4/62yRLZ/3N3WrCm292aaCJGEwYsElvF5z0+85XdU+ b0aV53oa/yR2WmUDAEOFjWgnU/4nwU7Xr8iozNwhzWobxBoR57FUc8sXZvr+1uMnoJOV vNwmn4cY5YVI2IlHCSOpU75ARRYkAdgW3dJryVze/lllfAz2OjqLV8g73VUupdkN+s3s nKjnlzVVjnQfrl2r3yfr23BK5ldh6VHk1APJNGav1eYWJLAXO0u2FGpX9E1diOc2esSs cw9LjQCSfpQYBLrrgUdzGxuar4Jc9XtGTrSeTaKTQamm4oJH7FEY4Z/Uf2wBeaMVvBeN ngZA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b="TXowak/L"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id xg11-20020a170907320b00b007919f213511si2334451ejb.951.2022.10.25.02.17.38; Tue, 25 Oct 2022 02:17:39 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b="TXowak/L"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 496EE68BD92; Tue, 25 Oct 2022 12:14:21 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 87FF768BD7F for ; Tue, 25 Oct 2022 12:14:18 +0300 (EEST) Received: by mail-pl1-f178.google.com with SMTP id j12so10656231plj.5 for ; Tue, 25 Oct 2022 02:14:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=uIOSt5XiXDnbTuZdIDvq4beRj9/o/Wb7BS+lLeI0pcA=; b=TXowak/LoXkf/GjVB26NiwTzP5z3r7jEWkXAeeeulIQRwIKmm3LecMIcbO3b3zScyj rifZ+OCvEVtRpIkliegVDLKQvgVaOezgJLlepZlrmX55E06PeNp8fJxsgCVf3cN0Td8g NeItY8RZ7G2WxbcxDPtZplIxq25VmUcPHXMrYLPdKTq9CAmDEmtRSR4S9/78nmA6YtT5 yNsTrOZs+iW/TkPpyHYCnhjScjorOtANVas6NZOGKgqQtgYf3UM9c90bmcSrwjgrki2m 2IvysZSo947GINr8Jd79s9lq5JVFUdARHFZAXfkXeLTDx0Bl0vHBYLXoIihRlPvWkkFV mkTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=uIOSt5XiXDnbTuZdIDvq4beRj9/o/Wb7BS+lLeI0pcA=; b=Vx0a/Bj1dQ4qY6Zhqb/fspLpVYmcfXSUmQzDwjDEhaKxOWBQFHqVET1URLtPUZo5Vs ZDAbfN0LbyEVG8zYDU/RUlrq9koUTLMdN5Bu1RkBpAAfo9ehTHyk9UTw7z3Rg7iROzYD +7ySRWG3wF/WW2es38a563nMr7zXPdr3NhvsoYYYABAL+gWndWMxuqY45HDZODL96D1p p0+FeOv9n/291J01wES14KPjs0vjWlzCS9A1EiJOqAH3zax4dxQsDCijV/DdjzLGeCvz kSEFK+YWfoK2LHSq0TzLHkUCNU0is+7oQ9fCPd4/zys7hclf7hGFQr/wJ1tManvUcNZh fOGQ== X-Gm-Message-State: ACrzQf2wkBDCyU4peUWh5TGyzhVWjAZVeSHQBn2O80OoDgL3MLPBLt36 d49mbalSQpXXRtOXFuQ/RybRHUbqu4M= X-Received: by 2002:a17:902:f605:b0:186:6feb:7eb4 with SMTP id n5-20020a170902f60500b001866feb7eb4mr24166452plg.134.1666689256146; Tue, 25 Oct 2022 02:14:16 -0700 (PDT) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id y2-20020a17090322c200b00180033438a0sm884574plg.106.2022.10.25.02.14.15 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Oct 2022 02:14:15 -0700 (PDT) From: softworkz X-Google-Original-From: softworkz Message-Id: <300c14faaa3e95ec6f2785a0099d815264ecc9a7.1666689226.git.ffmpegagent@gmail.com> In-Reply-To: References: Date: Tue, 25 Oct 2022 09:13:44 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v9 23/25] avcodec/subtitles: Migrate subtitle encoders to frame-based API X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: softworkz Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: NH52aV9NOr0h From: softworkz and provide a compatibility shim for the legacy api Signed-off-by: softworkz --- libavcodec/assenc.c | 189 ++++++++++++++++++++++++++++++------ libavcodec/avcodec.h | 5 +- libavcodec/codec_internal.h | 12 --- libavcodec/dvbsubenc.c | 96 ++++++++++-------- libavcodec/dvdsubenc.c | 103 ++++++++++++-------- libavcodec/encode.c | 61 +++++++++++- libavcodec/movtextenc.c | 114 ++++++++++++++++------ libavcodec/srtenc.c | 108 ++++++++++++++------- libavcodec/tests/avcodec.c | 5 +- libavcodec/ttmlenc.c | 101 ++++++++++++++----- libavcodec/utils.c | 1 - libavcodec/webvttenc.c | 86 +++++++++++----- libavcodec/xsubenc.c | 88 ++++++++++------- 13 files changed, 689 insertions(+), 280 deletions(-) diff --git a/libavcodec/assenc.c b/libavcodec/assenc.c index 9bbcdd9054..45fd5a217f 100644 --- a/libavcodec/assenc.c +++ b/libavcodec/assenc.c @@ -25,67 +25,192 @@ #include "avcodec.h" #include "codec_internal.h" +#include "encode.h" #include "libavutil/ass_internal.h" #include "libavutil/avstring.h" #include "libavutil/internal.h" #include "libavutil/mem.h" +typedef struct { + AVCodecContext *avctx; + AVFrame* current_frame; + int have_frame; + int current_area; +} AssEncContext; + +static void check_write_header(AVCodecContext* avctx, const AVFrame* frame) +{ + if (avctx->extradata_size) + return; + + if (frame->subtitle_header && frame->subtitle_header->size > 0) { + const char* subtitle_header = (char*)frame->subtitle_header->data; + avctx->extradata_size = strlen(subtitle_header); + avctx->extradata = av_mallocz(frame->subtitle_header->size + 1); + memcpy(avctx->extradata, subtitle_header, avctx->extradata_size); + avctx->extradata[avctx->extradata_size] = 0; + } + + if (!avctx->extradata_size) { + const char* subtitle_header = avpriv_ass_get_subtitle_header_default(0); + if (!subtitle_header) + return; + + avctx->extradata_size = strlen(subtitle_header); + avctx->extradata = av_mallocz(avctx->extradata_size + 1); + memcpy(avctx->extradata, subtitle_header, avctx->extradata_size); + avctx->extradata[avctx->extradata_size] = 0; + av_freep(&subtitle_header); + } +} + static av_cold int ass_encode_init(AVCodecContext *avctx) { - avctx->extradata = av_malloc(avctx->subtitle_header_size + 1); - if (!avctx->extradata) - return AVERROR(ENOMEM); - memcpy(avctx->extradata, avctx->subtitle_header, avctx->subtitle_header_size); - avctx->extradata_size = avctx->subtitle_header_size; - avctx->extradata[avctx->extradata_size] = 0; + AssEncContext *s = avctx->priv_data; + + if (avctx->subtitle_header_size) { + avctx->extradata = av_malloc(avctx->subtitle_header_size + 1); + if (!avctx->extradata) + return AVERROR(ENOMEM); + memcpy(avctx->extradata, avctx->subtitle_header, avctx->subtitle_header_size); + avctx->extradata_size = avctx->subtitle_header_size; + avctx->extradata[avctx->extradata_size] = 0; + } + + s->current_frame = av_frame_alloc(); + return 0; +} + +static av_cold int ass_encode_close(AVCodecContext *avctx) +{ + AssEncContext *s = avctx->priv_data; + av_frame_free(&s->current_frame); return 0; } -static int ass_encode_frame(AVCodecContext *avctx, - unsigned char *buf, int bufsize, - const AVSubtitle *sub) +////static int ass_encode_frame(AVCodecContext* avctx, AVPacket* avpkt, +//// const AVFrame* frame, int* got_packet) +////{ +//// int ret; +//// size_t req_len = 0, total_len = 0; +//// +//// check_write_header(avctx, frame); +//// +//// for (unsigned i = 0; i < frame->num_subtitle_areas; i++) { +//// const char *ass = frame->subtitle_areas[i]->ass; +//// +//// if (frame->subtitle_areas[i]->type != AV_SUBTITLE_FMT_ASS) { +//// av_log(avctx, AV_LOG_ERROR, "Only AV_SUBTITLE_FMT_ASS type supported.\n"); +//// return AVERROR(EINVAL); +//// } +//// +//// if (ass) +//// req_len += strlen(ass); +//// } +//// +//// ret = ff_get_encode_buffer(avctx, avpkt, req_len + 1, 0); +//// if (ret < 0) { +//// av_log(avctx, AV_LOG_ERROR, "Error getting output packet.\n"); +//// return ret; +//// } +//// +//// for (unsigned i = 0; i < frame->num_subtitle_areas; i++) { +//// const char *ass = frame->subtitle_areas[i]->ass; +//// +//// if (ass) { +//// size_t len = av_strlcpy((char *)avpkt->data + total_len, ass, avpkt->size - total_len); +//// total_len += len; +//// } +//// } +//// +//// avpkt->size = total_len; +//// *got_packet = total_len > 0; +//// +//// return 0; +////} + +static int ass_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) { - int i, len, total_len = 0; + AssEncContext *s = avctx->priv_data; + int ret; + + if (!s->have_frame) { + s->current_area = 0; + ret = ff_encode_get_frame(avctx, s->current_frame); + + if (ret < 0) { + av_frame_unref(s->current_frame); + return ret; + } + + s->have_frame = 1; + } - for (i=0; inum_rects; i++) { - const char *ass = sub->rects[i]->ass; + check_write_header(avctx, s->current_frame); - if (sub->rects[i]->type != SUBTITLE_ASS) { - av_log(avctx, AV_LOG_ERROR, "Only SUBTITLE_ASS type supported.\n"); + if (s->current_frame->repeat_sub) { + av_frame_unref(s->current_frame); + s->have_frame = 0; + return AVERROR(EAGAIN); + } + + if (s->current_area < s->current_frame->num_subtitle_areas) { + const AVSubtitleArea *area = s->current_frame->subtitle_areas[s->current_area]; + const char *ass = area->ass; + + if (area->type != AV_SUBTITLE_FMT_ASS) { + av_log(avctx, AV_LOG_ERROR, "Only AV_SUBTITLE_FMT_ASS type supported.\n"); return AVERROR(EINVAL); } - len = av_strlcpy(buf+total_len, ass, bufsize-total_len); + if (ass) { + size_t len = strlen(ass); + + ret = ff_get_encode_buffer(avctx, avpkt, len + 1, 0); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Error getting output packet.\n"); + return ret; + } + + len = av_strlcpy((char *)avpkt->data, ass, avpkt->size); - if (len > bufsize-total_len-1) { - av_log(avctx, AV_LOG_ERROR, "Buffer too small for ASS event.\n"); - return AVERROR_BUFFER_TOO_SMALL; + avpkt->size = len; } - total_len += len; + s->current_area++; } - return total_len; + if (s->current_area < s->current_frame->num_subtitle_areas) + return 0; + + av_frame_unref(s->current_frame); + s->have_frame = 0; + + return 0; } #if CONFIG_SSA_ENCODER const FFCodec ff_ssa_encoder = { - .p.name = "ssa", - CODEC_LONG_NAME("ASS (Advanced SubStation Alpha) subtitle"), - .p.type = AVMEDIA_TYPE_SUBTITLE, - .p.id = AV_CODEC_ID_ASS, - .init = ass_encode_init, - FF_CODEC_ENCODE_SUB_CB(ass_encode_frame), + .p.name = "ssa", + .p.long_name = NULL_IF_CONFIG_SMALL("ASS (Advanced SubStation Alpha) subtitle"), + .p.type = AVMEDIA_TYPE_SUBTITLE, + .p.id = AV_CODEC_ID_ASS, + .priv_data_size = sizeof(AssEncContext), + .init = ass_encode_init, + .close = ass_encode_close, + FF_CODEC_RECEIVE_PACKET_CB(ass_receive_packet), }; #endif #if CONFIG_ASS_ENCODER const FFCodec ff_ass_encoder = { - .p.name = "ass", - CODEC_LONG_NAME("ASS (Advanced SubStation Alpha) subtitle"), - .p.type = AVMEDIA_TYPE_SUBTITLE, - .p.id = AV_CODEC_ID_ASS, - .init = ass_encode_init, - FF_CODEC_ENCODE_SUB_CB(ass_encode_frame), + .p.name = "ass", + .p.long_name = NULL_IF_CONFIG_SMALL("ASS (Advanced SubStation Alpha) subtitle"), + .p.type = AVMEDIA_TYPE_SUBTITLE, + .priv_data_size = sizeof(AssEncContext), + .p.id = AV_CODEC_ID_ASS, + .init = ass_encode_init, + .close = ass_encode_close, + FF_CODEC_RECEIVE_PACKET_CB(ass_receive_packet), }; #endif diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index 68b588861f..c1be88fd11 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -3024,10 +3024,13 @@ void av_parser_close(AVCodecParserContext *s); * @{ */ + /** + * @deprecated Use @ref avcodec_encode_subtitle2() instead. + */ +attribute_deprecated int avcodec_encode_subtitle(AVCodecContext *avctx, uint8_t *buf, int buf_size, const AVSubtitle *sub); - /** * @} */ diff --git a/libavcodec/codec_internal.h b/libavcodec/codec_internal.h index e3b77e6dea..1aa7ed151e 100644 --- a/libavcodec/codec_internal.h +++ b/libavcodec/codec_internal.h @@ -108,9 +108,6 @@ enum FFCodecType { /* The codec is an encoder using the encode callback; * audio and video codecs only. */ FF_CODEC_CB_TYPE_ENCODE, - /* The codec is an encoder using the encode_sub callback; - * subtitle codecs only. */ - FF_CODEC_CB_TYPE_ENCODE_SUB, /* The codec is an encoder using the receive_packet callback; * audio and video codecs only. */ FF_CODEC_CB_TYPE_RECEIVE_PACKET, @@ -213,12 +210,6 @@ typedef struct FFCodec { */ int (*encode)(struct AVCodecContext *avctx, struct AVPacket *avpkt, const struct AVFrame *frame, int *got_packet_ptr); - /** - * Encode subtitles to a raw buffer. - * cb is in this state if cb_type is FF_CODEC_CB_TYPE_ENCODE_SUB. - */ - int (*encode_sub)(struct AVCodecContext *avctx, uint8_t *buf, - int buf_size, const struct AVSubtitle *sub); /** * Encode API with decoupled frame/packet dataflow. * cb is in this state if cb_type is FF_CODEC_CB_TYPE_RECEIVE_PACKET. @@ -307,9 +298,6 @@ typedef struct FFCodec { #define FF_CODEC_ENCODE_CB(func) \ .cb_type = FF_CODEC_CB_TYPE_ENCODE, \ .cb.encode = (func) -#define FF_CODEC_ENCODE_SUB_CB(func) \ - .cb_type = FF_CODEC_CB_TYPE_ENCODE_SUB, \ - .cb.encode_sub = (func) #define FF_CODEC_RECEIVE_PACKET_CB(func) \ .cb_type = FF_CODEC_CB_TYPE_RECEIVE_PACKET, \ .cb.receive_packet = (func) diff --git a/libavcodec/dvbsubenc.c b/libavcodec/dvbsubenc.c index 822e3a5309..da6543443c 100644 --- a/libavcodec/dvbsubenc.c +++ b/libavcodec/dvbsubenc.c @@ -21,6 +21,7 @@ #include "avcodec.h" #include "bytestream.h" #include "codec_internal.h" +#include "encode.h" #include "libavutil/colorspace.h" typedef struct DVBSubtitleContext { @@ -269,21 +270,29 @@ static int dvb_encode_rle8(uint8_t **pq, int buf_size, return len; } -static int dvbsub_encode(AVCodecContext *avctx, uint8_t *outbuf, int buf_size, - const AVSubtitle *h) +static int dvbsub_encode(AVCodecContext* avctx, AVPacket* avpkt, + const AVFrame* frame, int* got_packet) { DVBSubtitleContext *s = avctx->priv_data; uint8_t *q, *pseg_len; int page_id, region_id, clut_id, object_id, i, bpp_index, page_state; - - - q = outbuf; + size_t buf_size; + int ret; page_id = 1; - if (h->num_rects && !h->rects) + if (frame->num_subtitle_areas && !frame->subtitle_areas) return AVERROR(EINVAL); + ret = ff_get_encode_buffer(avctx, avpkt, 1024 * 1024, 0); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Error getting output packet.\n"); + return ret; + } + + buf_size = avpkt->size; + q = avpkt->data; + if (avctx->width > 0 && avctx->height > 0) { if (buf_size < 11) return AVERROR_BUFFER_TOO_SMALL; @@ -302,7 +311,7 @@ static int dvbsub_encode(AVCodecContext *avctx, uint8_t *outbuf, int buf_size, /* page composition segment */ - if (buf_size < 8 + h->num_rects * 6) + if (buf_size < 8 + frame->num_subtitle_areas * 6) return AVERROR_BUFFER_TOO_SMALL; *q++ = 0x0f; /* sync_byte */ *q++ = 0x10; /* segment_type */ @@ -314,30 +323,30 @@ static int dvbsub_encode(AVCodecContext *avctx, uint8_t *outbuf, int buf_size, /* page_version = 0 + page_state */ *q++ = (s->object_version << 4) | (page_state << 2) | 3; - for (region_id = 0; region_id < h->num_rects; region_id++) { + for (region_id = 0; region_id < frame->num_subtitle_areas; region_id++) { *q++ = region_id; *q++ = 0xff; /* reserved */ - bytestream_put_be16(&q, h->rects[region_id]->x); /* left pos */ - bytestream_put_be16(&q, h->rects[region_id]->y); /* top pos */ + bytestream_put_be16(&q, frame->subtitle_areas[region_id]->x); /* left pos */ + bytestream_put_be16(&q, frame->subtitle_areas[region_id]->y); /* top pos */ } bytestream_put_be16(&pseg_len, q - pseg_len - 2); - buf_size -= 8 + h->num_rects * 6; + buf_size -= 8 + frame->num_subtitle_areas * 6; - if (h->num_rects) { - for (clut_id = 0; clut_id < h->num_rects; clut_id++) { - if (buf_size < 6 + h->rects[clut_id]->nb_colors * 6) + if (frame->num_subtitle_areas) { + for (clut_id = 0; clut_id < frame->num_subtitle_areas; clut_id++) { + if (buf_size < 6 + frame->subtitle_areas[clut_id]->nb_colors * 6) return AVERROR_BUFFER_TOO_SMALL; /* CLUT segment */ - if (h->rects[clut_id]->nb_colors <= 4) { + if (frame->subtitle_areas[clut_id]->nb_colors <= 4) { /* 2 bpp, some decoders do not support it correctly */ bpp_index = 0; - } else if (h->rects[clut_id]->nb_colors <= 16) { + } else if (frame->subtitle_areas[clut_id]->nb_colors <= 16) { /* 4 bpp, standard encoding */ bpp_index = 1; - } else if (h->rects[clut_id]->nb_colors <= 256) { + } else if (frame->subtitle_areas[clut_id]->nb_colors <= 256) { /* 8 bpp, standard encoding */ bpp_index = 2; } else { @@ -354,12 +363,12 @@ static int dvbsub_encode(AVCodecContext *avctx, uint8_t *outbuf, int buf_size, *q++ = clut_id; *q++ = (0 << 4) | 0xf; /* version = 0 */ - for(i = 0; i < h->rects[clut_id]->nb_colors; i++) { + for(i = 0; i < frame->subtitle_areas[clut_id]->nb_colors; i++) { *q++ = i; /* clut_entry_id */ *q++ = (1 << (7 - bpp_index)) | (0xf << 1) | 1; /* 2 bits/pixel full range */ { int a, r, g, b; - uint32_t x= ((uint32_t*)h->rects[clut_id]->data[1])[i]; + uint32_t x= ((uint32_t*)frame->subtitle_areas[clut_id]->pal)[i]; a = (x >> 24) & 0xff; r = (x >> 16) & 0xff; g = (x >> 8) & 0xff; @@ -373,22 +382,22 @@ static int dvbsub_encode(AVCodecContext *avctx, uint8_t *outbuf, int buf_size, } bytestream_put_be16(&pseg_len, q - pseg_len - 2); - buf_size -= 6 + h->rects[clut_id]->nb_colors * 6; + buf_size -= 6 + frame->subtitle_areas[clut_id]->nb_colors * 6; } - if (buf_size < h->num_rects * 22) + if (buf_size < frame->num_subtitle_areas * 22) return AVERROR_BUFFER_TOO_SMALL; - for (region_id = 0; region_id < h->num_rects; region_id++) { + for (region_id = 0; region_id < frame->num_subtitle_areas; region_id++) { /* region composition segment */ - if (h->rects[region_id]->nb_colors <= 4) { + if (frame->subtitle_areas[region_id]->nb_colors <= 4) { /* 2 bpp, some decoders do not support it correctly */ bpp_index = 0; - } else if (h->rects[region_id]->nb_colors <= 16) { + } else if (frame->subtitle_areas[region_id]->nb_colors <= 16) { /* 4 bpp, standard encoding */ bpp_index = 1; - } else if (h->rects[region_id]->nb_colors <= 256) { + } else if (frame->subtitle_areas[region_id]->nb_colors <= 256) { /* 8 bpp, standard encoding */ bpp_index = 2; } else { @@ -402,8 +411,8 @@ static int dvbsub_encode(AVCodecContext *avctx, uint8_t *outbuf, int buf_size, q += 2; /* segment length */ *q++ = region_id; *q++ = (s->object_version << 4) | (0 << 3) | 0x07; /* version , no fill */ - bytestream_put_be16(&q, h->rects[region_id]->w); /* region width */ - bytestream_put_be16(&q, h->rects[region_id]->h); /* region height */ + bytestream_put_be16(&q, frame->subtitle_areas[region_id]->w); /* region width */ + bytestream_put_be16(&q, frame->subtitle_areas[region_id]->h); /* region height */ *q++ = ((1 + bpp_index) << 5) | ((1 + bpp_index) << 2) | 0x03; *q++ = region_id; /* clut_id == region_id */ *q++ = 0; /* 8 bit fill colors */ @@ -417,9 +426,9 @@ static int dvbsub_encode(AVCodecContext *avctx, uint8_t *outbuf, int buf_size, bytestream_put_be16(&pseg_len, q - pseg_len - 2); } - buf_size -= h->num_rects * 22; + buf_size -= frame->num_subtitle_areas * 22; - for (object_id = 0; object_id < h->num_rects; object_id++) { + for (object_id = 0; object_id < frame->num_subtitle_areas; object_id++) { int (*dvb_encode_rle)(uint8_t **pq, int buf_size, const uint8_t *bitmap, int linesize, int w, int h); @@ -428,13 +437,13 @@ static int dvbsub_encode(AVCodecContext *avctx, uint8_t *outbuf, int buf_size, return AVERROR_BUFFER_TOO_SMALL; /* bpp_index maths */ - if (h->rects[object_id]->nb_colors <= 4) { + if (frame->subtitle_areas[object_id]->nb_colors <= 4) { /* 2 bpp, some decoders do not support it correctly */ dvb_encode_rle = dvb_encode_rle2; - } else if (h->rects[object_id]->nb_colors <= 16) { + } else if (frame->subtitle_areas[object_id]->nb_colors <= 16) { /* 4 bpp, standard encoding */ dvb_encode_rle = dvb_encode_rle4; - } else if (h->rects[object_id]->nb_colors <= 256) { + } else if (frame->subtitle_areas[object_id]->nb_colors <= 256) { /* 8 bpp, standard encoding */ dvb_encode_rle = dvb_encode_rle8; } else { @@ -464,19 +473,19 @@ static int dvbsub_encode(AVCodecContext *avctx, uint8_t *outbuf, int buf_size, top_ptr = q; ret = dvb_encode_rle(&q, buf_size, - h->rects[object_id]->data[0], - h->rects[object_id]->w * 2, - h->rects[object_id]->w, - h->rects[object_id]->h >> 1); + frame->subtitle_areas[object_id]->buf[0]->data, + frame->subtitle_areas[object_id]->w * 2, + frame->subtitle_areas[object_id]->w, + frame->subtitle_areas[object_id]->h >> 1); if (ret < 0) return ret; buf_size -= ret; bottom_ptr = q; ret = dvb_encode_rle(&q, buf_size, - h->rects[object_id]->data[0] + h->rects[object_id]->w, - h->rects[object_id]->w * 2, - h->rects[object_id]->w, - h->rects[object_id]->h >> 1); + frame->subtitle_areas[object_id]->buf[0]->data + frame->subtitle_areas[object_id]->w, + frame->subtitle_areas[object_id]->w * 2, + frame->subtitle_areas[object_id]->w, + frame->subtitle_areas[object_id]->h >> 1); if (ret < 0) return ret; buf_size -= ret; @@ -503,7 +512,10 @@ static int dvbsub_encode(AVCodecContext *avctx, uint8_t *outbuf, int buf_size, buf_size -= 6; s->object_version = (s->object_version + 1) & 0xf; - return q - outbuf; + avpkt->size = q - avpkt->data; + *got_packet = 1; + + return 0; } const FFCodec ff_dvbsub_encoder = { @@ -512,5 +524,5 @@ const FFCodec ff_dvbsub_encoder = { .p.type = AVMEDIA_TYPE_SUBTITLE, .p.id = AV_CODEC_ID_DVB_SUBTITLE, .priv_data_size = sizeof(DVBSubtitleContext), - FF_CODEC_ENCODE_SUB_CB(dvbsub_encode), + FF_CODEC_ENCODE_CB(dvbsub_encode), }; diff --git a/libavcodec/dvdsubenc.c b/libavcodec/dvdsubenc.c index 624e21c000..408ac938c9 100644 --- a/libavcodec/dvdsubenc.c +++ b/libavcodec/dvdsubenc.c @@ -22,6 +22,8 @@ #include "bytestream.h" #include "codec_internal.h" #include "dvdsub.h" +#include "encode.h" +#include "internal.h" #include "libavutil/avassert.h" #include "libavutil/bprint.h" #include "libavutil/imgutils.h" @@ -115,15 +117,14 @@ static int color_distance(uint32_t a, uint32_t b) * Count colors used in a rectangle, quantizing alpha and grouping by * nearest global palette entry. */ -static void count_colors(AVCodecContext *avctx, unsigned hits[33], - const AVSubtitleRect *r) +static void count_colors(const AVCodecContext *avctx, unsigned hits[33], + const AVSubtitleArea *r) { DVDSubtitleContext *dvdc = avctx->priv_data; unsigned count[256] = { 0 }; - uint32_t *palette = (uint32_t *)r->data[1]; uint32_t color; int x, y, i, j, match, d, best_d, av_uninit(best_j); - uint8_t *p = r->data[0]; + uint8_t *p = r->buf[0]->data; for (y = 0; y < r->h; y++) { for (x = 0; x < r->w; x++) @@ -133,7 +134,7 @@ static void count_colors(AVCodecContext *avctx, unsigned hits[33], for (i = 0; i < 256; i++) { if (!count[i]) /* avoid useless search */ continue; - color = palette[i]; + color = r->pal[i]; /* 0: transparent, 1-16: semi-transparent, 17-33 opaque */ match = color < 0x33000000 ? 0 : color < 0xCC000000 ? 1 : 17; if (match) { @@ -233,13 +234,13 @@ static void build_color_map(AVCodecContext *avctx, int cmap[], } } -static void copy_rectangle(AVSubtitleRect *dst, AVSubtitleRect *src, int cmap[]) +static void copy_rectangle(AVSubtitleArea*dst, AVSubtitleArea *src, int cmap[]) { int x, y; uint8_t *p, *q; - p = src->data[0]; - q = dst->data[0] + (src->x - dst->x) + + p = src->buf[0]->data; + q = dst->buf[0]->data + (src->x - dst->x) + (src->y - dst->y) * dst->linesize[0]; for (y = 0; y < src->h; y++) { for (x = 0; x < src->w; x++) @@ -249,51 +250,57 @@ static void copy_rectangle(AVSubtitleRect *dst, AVSubtitleRect *src, int cmap[]) } } -static int encode_dvd_subtitles(AVCodecContext *avctx, - uint8_t *outbuf, int outbuf_size, - const AVSubtitle *h) +static int encode_dvd_subtitles(AVCodecContext* avctx, AVPacket* avpkt, + const AVFrame* frame, int* got_packet) { DVDSubtitleContext *dvdc = avctx->priv_data; uint8_t *q, *qq; int offset1, offset2; - int i, rects = h->num_rects, ret; + int ret = 0; + unsigned i, rects = frame->num_subtitle_areas; unsigned global_palette_hits[33] = { 0 }; int cmap[256]; int out_palette[4]; int out_alpha[4]; - AVSubtitleRect vrect; - uint8_t *vrect_data = NULL; + AVSubtitleArea vrect; + uint8_t* vrect_data = NULL, *outbuf; int x2, y2; int forced = 0; + int outbuf_size; + int64_t duration_ms; - if (rects == 0 || !h->rects) + if (rects == 0) + return 0; + + if (!frame->subtitle_areas) return AVERROR(EINVAL); + for (i = 0; i < rects; i++) - if (h->rects[i]->type != AV_SUBTITLE_FMT_BITMAP) { + if (frame->subtitle_areas[i]->type != AV_SUBTITLE_FMT_BITMAP) { av_log(avctx, AV_LOG_ERROR, "Bitmap subtitle required\n"); return AVERROR(EINVAL); } /* Mark this subtitle forced if any of the rectangles is forced. */ for (i = 0; i < rects; i++) - if ((h->rects[i]->flags & AV_SUBTITLE_FLAG_FORCED) != 0) { + if ((frame->subtitle_areas[i]->flags & AV_SUBTITLE_FLAG_FORCED) != 0) { forced = 1; break; } - vrect = *h->rects[0]; + vrect = *frame->subtitle_areas[0]; if (rects > 1) { /* DVD subtitles can have only one rectangle: build a virtual rectangle containing all actual rectangles. The data of the rectangles will be copied later, when the palette is decided, because the rectangles may have different palettes. */ - int xmin = h->rects[0]->x, xmax = xmin + h->rects[0]->w; - int ymin = h->rects[0]->y, ymax = ymin + h->rects[0]->h; + int xmin = frame->subtitle_areas[0]->x, xmax = xmin + frame->subtitle_areas[0]->w; + int ymin = frame->subtitle_areas[0]->y, ymax = ymin + frame->subtitle_areas[0]->h; for (i = 1; i < rects; i++) { - xmin = FFMIN(xmin, h->rects[i]->x); - ymin = FFMIN(ymin, h->rects[i]->y); - xmax = FFMAX(xmax, h->rects[i]->x + h->rects[i]->w); - ymax = FFMAX(ymax, h->rects[i]->y + h->rects[i]->h); + xmin = FFMIN(xmin, frame->subtitle_areas[i]->x); + ymin = FFMIN(ymin, frame->subtitle_areas[i]->y); + xmax = FFMAX(xmax, frame->subtitle_areas[i]->x + frame->subtitle_areas[i]->w); + ymax = FFMAX(ymax, frame->subtitle_areas[i]->y + frame->subtitle_areas[i]->h); } vrect.x = xmin; vrect.y = ymin; @@ -305,27 +312,29 @@ static int encode_dvd_subtitles(AVCodecContext *avctx, /* Count pixels outside the virtual rectangle as transparent */ global_palette_hits[0] = vrect.w * vrect.h; for (i = 0; i < rects; i++) - global_palette_hits[0] -= h->rects[i]->w * h->rects[i]->h; + global_palette_hits[0] -= frame->subtitle_areas[i]->w * frame->subtitle_areas[i]->h; } for (i = 0; i < rects; i++) - count_colors(avctx, global_palette_hits, h->rects[i]); + count_colors(avctx, global_palette_hits, frame->subtitle_areas[i]); select_palette(avctx, out_palette, out_alpha, global_palette_hits); if (rects > 1) { - if (!(vrect_data = av_calloc(vrect.w, vrect.h))) + + vrect.buf[0] = av_buffer_allocz((size_t)vrect.w * vrect.h); + if (!vrect.buf[0]) return AVERROR(ENOMEM); - vrect.data [0] = vrect_data; + vrect.linesize[0] = vrect.w; for (i = 0; i < rects; i++) { - build_color_map(avctx, cmap, (uint32_t *)h->rects[i]->data[1], + build_color_map(avctx, cmap, frame->subtitle_areas[i]->pal, out_palette, out_alpha); - copy_rectangle(&vrect, h->rects[i], cmap); + copy_rectangle(&vrect, frame->subtitle_areas[i], cmap); } for (i = 0; i < 4; i++) cmap[i] = i; } else { - build_color_map(avctx, cmap, (uint32_t *)h->rects[0]->data[1], + build_color_map(avctx, cmap, frame->subtitle_areas[0]->pal, out_palette, out_alpha); } @@ -336,6 +345,16 @@ static int encode_dvd_subtitles(AVCodecContext *avctx, out_palette[i], out_alpha[i] >> 4); av_log(avctx, AV_LOG_DEBUG, "\n"); + + ret = ff_get_encode_buffer(avctx, avpkt, 4 + vrect.w * vrect.h / 2 + 17 + 21, 0); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Error getting output packet.\n"); + return ret; + } + + outbuf_size = avpkt->size; + outbuf = avpkt->data; + // encode data block q = outbuf + 4; offset1 = q - outbuf; @@ -345,10 +364,10 @@ static int encode_dvd_subtitles(AVCodecContext *avctx, ret = AVERROR_BUFFER_TOO_SMALL; goto fail; } - dvd_encode_rle(&q, vrect.data[0], vrect.w * 2, + dvd_encode_rle(&q, vrect.buf[0]->data, vrect.w * 2, vrect.w, (vrect.h + 1) >> 1, cmap); offset2 = q - outbuf; - dvd_encode_rle(&q, vrect.data[0] + vrect.w, vrect.w * 2, + dvd_encode_rle(&q, vrect.buf[0]->data + vrect.w, vrect.w * 2, vrect.w, vrect.h >> 1, cmap); if (dvdc->even_rows_fix && (vrect.h & 1)) { @@ -363,7 +382,7 @@ static int encode_dvd_subtitles(AVCodecContext *avctx, bytestream_put_be16(&qq, q - outbuf); // send start display command - bytestream_put_be16(&q, (h->start_display_time*90) >> 10); + bytestream_put_be16(&q, 0); bytestream_put_be16(&q, (q - outbuf) /*- 2 */ + 8 + 12 + 2); *q++ = 0x03; // palette - 4 nibbles *q++ = (out_palette[3] << 4) | out_palette[2]; @@ -401,7 +420,8 @@ static int encode_dvd_subtitles(AVCodecContext *avctx, *q++ = 0xff; // terminating command // send stop display command last - bytestream_put_be16(&q, (h->end_display_time*90) >> 10); + duration_ms = av_rescale_q(frame->subtitle_timing.duration, AV_TIME_BASE_Q, (AVRational){ 1, 1000 }); + bytestream_put_be16(&q, (duration_ms*90) >> 10); bytestream_put_be16(&q, (q - outbuf) - 2 /*+ 4*/); *q++ = 0x02; // set end *q++ = 0xff; // terminating command @@ -410,7 +430,9 @@ static int encode_dvd_subtitles(AVCodecContext *avctx, bytestream_put_be16(&qq, q - outbuf); av_log(NULL, AV_LOG_DEBUG, "subtitle_packet size=%"PTRDIFF_SPECIFIER"\n", q - outbuf); - ret = q - outbuf; + avpkt->size = q - outbuf; + ret = 0; + *got_packet = 1; fail: av_free(vrect_data); @@ -474,14 +496,13 @@ static int dvdsub_init(AVCodecContext *avctx) return 0; } -static int dvdsub_encode(AVCodecContext *avctx, - unsigned char *buf, int buf_size, - const AVSubtitle *sub) +static int dvdsub_encode(struct AVCodecContext* avctx, struct AVPacket* avpkt, + const struct AVFrame* frame, int* got_packet) { //DVDSubtitleContext *s = avctx->priv_data; int ret; - ret = encode_dvd_subtitles(avctx, buf, buf_size, sub); + ret = encode_dvd_subtitles(avctx, avpkt, frame, got_packet); return ret; } @@ -506,7 +527,7 @@ const FFCodec ff_dvdsub_encoder = { .p.type = AVMEDIA_TYPE_SUBTITLE, .p.id = AV_CODEC_ID_DVD_SUBTITLE, .init = dvdsub_init, - FF_CODEC_ENCODE_SUB_CB(dvdsub_encode), + FF_CODEC_ENCODE_CB(dvdsub_encode), .p.priv_class = &dvdsubenc_class, .priv_data_size = sizeof(DVDSubtitleContext), }; diff --git a/libavcodec/encode.c b/libavcodec/encode.c index fbe2c97cd6..27785c0242 100644 --- a/libavcodec/encode.c +++ b/libavcodec/encode.c @@ -161,17 +161,70 @@ fail: return ret; } -int avcodec_encode_subtitle(AVCodecContext *avctx, uint8_t *buf, int buf_size, - const AVSubtitle *sub) +int avcodec_encode_subtitle(AVCodecContext *avctx, uint8_t *buf, int buf_size, const AVSubtitle *sub) { - int ret; + int ret = 0; + AVFrame *frame = NULL; + AVPacket* avpkt = NULL; + if (sub->start_display_time) { av_log(avctx, AV_LOG_ERROR, "start_display_time must be 0.\n"); return -1; } - ret = ffcodec(avctx->codec)->cb.encode_sub(avctx, buf, buf_size, sub); + memset(buf, 0, buf_size); + // Create a temporary frame for calling the regular api: + frame = av_frame_alloc(); + if (!frame) { + ret = AVERROR(ENOMEM); + goto exit; + } + + frame->format = sub->format; + frame->type = AVMEDIA_TYPE_SUBTITLE; + ret = av_frame_get_buffer2(frame, 0); + if (ret < 0) + goto exit; + + // Create a temporary packet + avpkt = av_packet_alloc(); + if (!avpkt) { + ret = AVERROR(ENOMEM); + goto exit; + } + + // Copy legacy subtitle data to temp frame + ret = ff_frame_put_subtitle(frame, sub); + if (ret < 0) + goto exit; + + ret = avcodec_send_frame(avctx, frame); + if (ret < 0) + goto exit; + + ret = avcodec_receive_packet(avctx, avpkt); + + if (ret < 0 && ret != AVERROR(EAGAIN)) + goto exit; + + //ret = avctx->codec->encode2(avctx, avpkt, frame, &got_packet); + avctx->frame_number++; + + if (avpkt->size) { + if (avpkt->size > buf_size) { + ret = AVERROR_BUFFER_TOO_SMALL; + goto exit; + } + + memcpy(buf, avpkt->data, avpkt->size); + ret = avpkt->size; + } + +exit: + + av_packet_free(&avpkt); + av_frame_free(&frame); return ret; } diff --git a/libavcodec/movtextenc.c b/libavcodec/movtextenc.c index 6f87e30904..2d5f0347a5 100644 --- a/libavcodec/movtextenc.c +++ b/libavcodec/movtextenc.c @@ -30,6 +30,7 @@ #include "libavutil/ass_internal.h" #include "bytestream.h" #include "codec_internal.h" +#include "encode.h" #define STYLE_FLAG_BOLD (1<<0) #define STYLE_FLAG_ITALIC (1<<1) @@ -73,6 +74,7 @@ typedef struct { AVCodecContext *avctx; ASSSplitContext *ass_ctx; + int is_default_ass_context; ASSStyle *ass_dialog_style; StyleBox *style_attributes; unsigned count; @@ -329,7 +331,7 @@ static av_cold int mov_text_encode_init(AVCodecContext *avctx) av_bprint_init(&s->buffer, 0, AV_BPRINT_SIZE_UNLIMITED); - s->ass_ctx = avpriv_ass_split(avctx->subtitle_header); + s->ass_ctx = avpriv_ass_split((char*)avctx->subtitle_header); if (!s->ass_ctx) return AVERROR_INVALIDDATA; ret = encode_sample_description(avctx); @@ -633,56 +635,112 @@ static const ASSCodesCallbacks mov_text_callbacks = { .end = mov_text_end_cb, }; -static int mov_text_encode_frame(AVCodecContext *avctx, unsigned char *buf, - int bufsize, const AVSubtitle *sub) +static void ensure_ass_context(AVCodecContext* avctx, const AVFrame* frame) +{ + MovTextContext* s = avctx->priv_data; + int ret; + + if (s->ass_ctx && !s->is_default_ass_context) + // We already have a (non-default context) + return; + + if (!frame->num_subtitle_areas) + // Don't need ass context for processing empty subtitle frames + return; + + // The frame has content, so we need to set up a context + if (frame->subtitle_header && frame->subtitle_header->size > 0) { + const char* subtitle_header = (char*)frame->subtitle_header->data; + avpriv_ass_split_free(s->ass_ctx); + s->ass_ctx = avpriv_ass_split(subtitle_header); + s->is_default_ass_context = 0; + } + else if (!s->ass_ctx) { + char* subtitle_header = avpriv_ass_get_subtitle_header_default(0); + if (!subtitle_header) + return; + + s->ass_ctx = avpriv_ass_split(subtitle_header); + s->is_default_ass_context = 1; + av_free(subtitle_header); + } + + if (s->ass_ctx && !avctx->extradata_size) { + ret = encode_sample_description(avctx); + if (ret < 0) + av_log(avctx, AV_LOG_ERROR, "Error during encode_sample_description().\n"); + } +} + +static int mov_text_encode_frame(AVCodecContext *avctx, AVPacket *avpkt, + const AVFrame *frame, int *got_packet) { MovTextContext *s = avctx->priv_data; ASSDialog *dialog; - int i, length; + int i, ret = 0; + size_t j; + uint8_t* buf; + + ensure_ass_context(avctx, frame); s->text_pos = 0; s->count = 0; s->box_flags = 0; av_bprint_clear(&s->buffer); - for (i = 0; i < sub->num_rects; i++) { - const char *ass = sub->rects[i]->ass; + for (i = 0; i < frame->num_subtitle_areas; i++) { + const char *ass = frame->subtitle_areas[i]->ass; - if (sub->rects[i]->type != SUBTITLE_ASS) { - av_log(avctx, AV_LOG_ERROR, "Only SUBTITLE_ASS type supported.\n"); + if (frame->subtitle_areas[i]->type != AV_SUBTITLE_FMT_ASS) { + av_log(avctx, AV_LOG_ERROR, "Only AV_SUBTITLE_FMT_ASS type supported.\n"); return AVERROR(EINVAL); } - dialog = avpriv_ass_split_dialog(s->ass_ctx, ass); - if (!dialog) - return AVERROR(ENOMEM); - mov_text_dialog(s, dialog); - avpriv_ass_split_override_codes(&mov_text_callbacks, s, dialog->text); - avpriv_ass_free_dialog(&dialog); + if (ass) { + + if (i > 0) + mov_text_new_line_cb(s, 0); + + dialog = avpriv_ass_split_dialog(s->ass_ctx, ass); + if (!dialog) + return AVERROR(ENOMEM); + mov_text_dialog(s, dialog); + avpriv_ass_split_override_codes(&mov_text_callbacks, s, dialog->text); + avpriv_ass_free_dialog(&dialog); + + } } - if (s->buffer.len > UINT16_MAX) - return AVERROR(ERANGE); - AV_WB16(buf, s->buffer.len); - buf += 2; + if (!av_bprint_is_complete(&s->buffer)) { + return AVERROR(ENOMEM); + } - for (size_t j = 0; j < box_count; j++) + for (j = 0; j < box_count; j++) { box_types[j].encode(s); + } - if (!av_bprint_is_complete(&s->buffer)) - return AVERROR(ENOMEM); + ret = ff_get_encode_buffer(avctx, avpkt, s->buffer.len + 3, 0); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Error getting output packet.\n"); + return ret; + } - if (!s->buffer.len) - return 0; + buf = avpkt->data; - if (s->buffer.len > bufsize - 3) { + AV_WB16(buf, s->buffer.len); + buf += 2; + + if (s->buffer.len > avpkt->size - 3) { av_log(avctx, AV_LOG_ERROR, "Buffer too small for ASS event.\n"); - return AVERROR_BUFFER_TOO_SMALL; + ret = AVERROR_BUFFER_TOO_SMALL; + goto exit; } memcpy(buf, s->buffer.str, s->buffer.len); - length = s->buffer.len + 2; + avpkt->size = s->buffer.len + 2; + *got_packet = 1; - return length; +exit: + return ret; } #define OFFSET(x) offsetof(MovTextContext, x) @@ -707,7 +765,7 @@ const FFCodec ff_movtext_encoder = { .priv_data_size = sizeof(MovTextContext), .p.priv_class = &mov_text_encoder_class, .init = mov_text_encode_init, - FF_CODEC_ENCODE_SUB_CB(mov_text_encode_frame), + FF_CODEC_ENCODE_CB(mov_text_encode_frame), .close = mov_text_encode_close, .caps_internal = FF_CODEC_CAP_INIT_CLEANUP, }; diff --git a/libavcodec/srtenc.c b/libavcodec/srtenc.c index 245e6a37c5..1fa1a2da26 100644 --- a/libavcodec/srtenc.c +++ b/libavcodec/srtenc.c @@ -23,6 +23,7 @@ #include #include "avcodec.h" +#include "encode.h" #include "libavutil/avstring.h" #include "libavutil/bprint.h" #include "codec_internal.h" @@ -35,6 +36,7 @@ typedef struct { AVCodecContext *avctx; ASSSplitContext *ass_ctx; + int is_default_ass_context; AVBPrint buffer; char stack[SRT_STACK_SIZE]; int stack_ptr; @@ -132,14 +134,13 @@ static void srt_style_apply(SRTContext *s, const char *style) } } - static av_cold int srt_encode_init(AVCodecContext *avctx) { SRTContext *s = avctx->priv_data; s->avctx = avctx; - s->ass_ctx = avpriv_ass_split(avctx->subtitle_header); + s->ass_ctx = avpriv_ass_split((char *)avctx->subtitle_header); av_bprint_init(&s->buffer, 0, AV_BPRINT_SIZE_UNLIMITED); - return s->ass_ctx ? 0 : AVERROR_INVALIDDATA; + return 0; } static void srt_text_cb(void *priv, const char *text, int len) @@ -229,58 +230,95 @@ static const ASSCodesCallbacks text_callbacks = { .new_line = srt_new_line_cb, }; -static int encode_frame(AVCodecContext *avctx, - unsigned char *buf, int bufsize, const AVSubtitle *sub, - const ASSCodesCallbacks *cb) +static void ensure_ass_context(SRTContext* s, const AVFrame* frame) +{ + if (s->ass_ctx && !s->is_default_ass_context) + // We already have a (non-default context) + return; + + if (!frame->num_subtitle_areas) + // Don't need ass context for processing empty subtitle frames + return; + + // The frame has content, so we need to set up a context + if (frame->subtitle_header && frame->subtitle_header->size > 0) { + const char* subtitle_header = (char*)frame->subtitle_header->data; + avpriv_ass_split_free(s->ass_ctx); + s->ass_ctx = avpriv_ass_split(subtitle_header); + s->is_default_ass_context = 0; + } + else if (!s->ass_ctx) { + char* subtitle_header = avpriv_ass_get_subtitle_header_default(0); + if (!subtitle_header) + return; + + s->ass_ctx = avpriv_ass_split(subtitle_header); + s->is_default_ass_context = 1; + av_free(subtitle_header); + } +} + +static int encode_frame(AVCodecContext* avctx, AVPacket* avpkt, + const AVFrame* frame, int* got_packet, const ASSCodesCallbacks* cb) { SRTContext *s = avctx->priv_data; ASSDialog *dialog; - int i; + int i, ret; + + ensure_ass_context(s, frame); av_bprint_clear(&s->buffer); - for (i=0; inum_rects; i++) { - const char *ass = sub->rects[i]->ass; + for (i=0; i< frame->num_subtitle_areas; i++) { + const char *ass = frame->subtitle_areas[i]->ass; - if (sub->rects[i]->type != SUBTITLE_ASS) { - av_log(avctx, AV_LOG_ERROR, "Only SUBTITLE_ASS type supported.\n"); + if (frame->subtitle_areas[i]->type != AV_SUBTITLE_FMT_ASS) { + av_log(avctx, AV_LOG_ERROR, "Only AV_SUBTITLE_FMT_ASS type supported.\n"); return AVERROR(EINVAL); } - dialog = avpriv_ass_split_dialog(s->ass_ctx, ass); - if (!dialog) - return AVERROR(ENOMEM); - s->alignment_applied = 0; - if (avctx->codec_id == AV_CODEC_ID_SUBRIP) - srt_style_apply(s, dialog->style); - avpriv_ass_split_override_codes(cb, s, dialog->text); - avpriv_ass_free_dialog(&dialog); + if (ass) { + + if (i > 0) + srt_new_line_cb(s, 0); + + dialog = avpriv_ass_split_dialog(s->ass_ctx, ass); + if (!dialog) + return AVERROR(ENOMEM); + s->alignment_applied = 0; + if (avctx->codec_id == AV_CODEC_ID_SUBRIP) + srt_style_apply(s, dialog->style); + avpriv_ass_split_override_codes(cb, s, dialog->text); + avpriv_ass_free_dialog(&dialog); + } } if (!av_bprint_is_complete(&s->buffer)) return AVERROR(ENOMEM); - if (!s->buffer.len) - return 0; - if (s->buffer.len > bufsize) { - av_log(avctx, AV_LOG_ERROR, "Buffer too small for ASS event.\n"); - return AVERROR_BUFFER_TOO_SMALL; + ret = ff_get_encode_buffer(avctx, avpkt, s->buffer.len + 1, 0); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Error getting output packet.\n"); + return ret; } - memcpy(buf, s->buffer.str, s->buffer.len); - return s->buffer.len; + memcpy(avpkt->data, s->buffer.str, s->buffer.len); + avpkt->size = s->buffer.len; + *got_packet = 1; + + return 0; } -static int srt_encode_frame(AVCodecContext *avctx, - unsigned char *buf, int bufsize, const AVSubtitle *sub) +static int srt_encode_frame(AVCodecContext* avctx, AVPacket* avpkt, + const AVFrame* frame, int* got_packet) { - return encode_frame(avctx, buf, bufsize, sub, &srt_callbacks); + return encode_frame(avctx, avpkt, frame, got_packet, &srt_callbacks); } -static int text_encode_frame(AVCodecContext *avctx, - unsigned char *buf, int bufsize, const AVSubtitle *sub) +static int text_encode_frame(AVCodecContext* avctx, AVPacket* avpkt, + const AVFrame* frame, int* got_packet) { - return encode_frame(avctx, buf, bufsize, sub, &text_callbacks); + return encode_frame(avctx, avpkt, frame, got_packet, &text_callbacks); } static int srt_encode_close(AVCodecContext *avctx) @@ -300,7 +338,7 @@ const FFCodec ff_srt_encoder = { .p.id = AV_CODEC_ID_SUBRIP, .priv_data_size = sizeof(SRTContext), .init = srt_encode_init, - FF_CODEC_ENCODE_SUB_CB(srt_encode_frame), + FF_CODEC_ENCODE_CB(srt_encode_frame), .close = srt_encode_close, }; #endif @@ -313,7 +351,7 @@ const FFCodec ff_subrip_encoder = { .p.id = AV_CODEC_ID_SUBRIP, .priv_data_size = sizeof(SRTContext), .init = srt_encode_init, - FF_CODEC_ENCODE_SUB_CB(srt_encode_frame), + FF_CODEC_ENCODE_CB(srt_encode_frame), .close = srt_encode_close, }; #endif @@ -326,7 +364,7 @@ const FFCodec ff_text_encoder = { .p.id = AV_CODEC_ID_TEXT, .priv_data_size = sizeof(SRTContext), .init = srt_encode_init, - FF_CODEC_ENCODE_SUB_CB(text_encode_frame), + FF_CODEC_ENCODE_CB(text_encode_frame), .close = srt_encode_close, }; #endif diff --git a/libavcodec/tests/avcodec.c b/libavcodec/tests/avcodec.c index 3288a85f64..ed8b9212bb 100644 --- a/libavcodec/tests/avcodec.c +++ b/libavcodec/tests/avcodec.c @@ -109,7 +109,6 @@ int main(void){ is_decoder = 1; break; case FF_CODEC_CB_TYPE_ENCODE: - case FF_CODEC_CB_TYPE_ENCODE_SUB: case FF_CODEC_CB_TYPE_RECEIVE_PACKET: is_encoder = 1; break; @@ -125,15 +124,13 @@ int main(void){ #define CHECK(TYPE, type) (codec2->cb_type == FF_CODEC_CB_TYPE_ ## TYPE && !codec2->cb.type) if (CHECK(DECODE, decode) || CHECK(DECODE_SUB, decode_sub) || CHECK(RECEIVE_PACKET, receive_packet) || - CHECK(ENCODE, encode) || CHECK(ENCODE_SUB, encode_sub) || + CHECK(ENCODE, encode) || CHECK(RECEIVE_FRAME, receive_frame)) { ERR_EXT("Codec %s does not implement its %s callback.\n", is_decoder ? "decoding" : "encoding"); } #undef CHECK if (is_encoder) { - if ((codec->type == AVMEDIA_TYPE_SUBTITLE) != (codec2->cb_type == FF_CODEC_CB_TYPE_ENCODE_SUB)) - ERR("Encoder %s is both subtitle encoder and not subtitle encoder."); if (codec2->update_thread_context || codec2->update_thread_context_for_user || codec2->bsfs) ERR("Encoder %s has decoder-only thread functions or bsf.\n"); if (codec->type == AVMEDIA_TYPE_AUDIO) { diff --git a/libavcodec/ttmlenc.c b/libavcodec/ttmlenc.c index c3f873147c..128b50c53a 100644 --- a/libavcodec/ttmlenc.c +++ b/libavcodec/ttmlenc.c @@ -33,11 +33,17 @@ #include "libavutil/bprint.h" #include "libavutil/internal.h" #include "libavutil/ass_split_internal.h" +#include "libavutil/ass_internal.h" #include "ttmlenc.h" +#include "encode.h" + + typedef struct { AVCodecContext *avctx; ASSSplitContext *ass_ctx; + int is_default_ass_context; + int extradata_written; AVBPrint buffer; } TTMLContext; @@ -76,28 +82,75 @@ static const ASSCodesCallbacks ttml_callbacks = { .new_line = ttml_new_line_cb, }; -static int ttml_encode_frame(AVCodecContext *avctx, uint8_t *buf, - int bufsize, const AVSubtitle *sub) +static int ttml_write_header_content(AVCodecContext* avctx); + +static void ensure_ass_context(AVCodecContext* avctx, const AVFrame* frame) +{ + TTMLContext* s = avctx->priv_data; + int ret; + + if (s->ass_ctx && !s->is_default_ass_context) + // We already have a (non-default context) + return; + + if (!frame->num_subtitle_areas) + // Don't need ass context for processing empty subtitle frames + return; + + // The frame has content, so we need to set up a context + if (frame->subtitle_header && frame->subtitle_header->size > 0) { + const char* subtitle_header = (char*)frame->subtitle_header->data; + avpriv_ass_split_free(s->ass_ctx); + s->ass_ctx = avpriv_ass_split(subtitle_header); + s->is_default_ass_context = 0; + } + else if (!s->ass_ctx) { + char* subtitle_header = avpriv_ass_get_subtitle_header_default(0); + if (!subtitle_header) + return; + + s->ass_ctx = avpriv_ass_split(subtitle_header); + s->is_default_ass_context = 1; + av_free(subtitle_header); + } + + if (s->ass_ctx && !s->extradata_written) { + s->extradata_written = 1; + if ((ret = ttml_write_header_content(avctx)) < 0) { + av_log(avctx, AV_LOG_ERROR, "Error writing header content.\n"); + } + } +} + +static int ttml_encode_frame(AVCodecContext* avctx, AVPacket* avpkt, + const AVFrame* frame, int* got_packet) { TTMLContext *s = avctx->priv_data; ASSDialog *dialog; - int i; + int i, ret; + + ensure_ass_context(avctx, frame); av_bprint_clear(&s->buffer); - for (i=0; inum_rects; i++) { - const char *ass = sub->rects[i]->ass; - int ret; + for (i=0; i< frame->num_subtitle_areas; i++) { + const char *ass = frame->subtitle_areas[i]->ass; - if (sub->rects[i]->type != SUBTITLE_ASS) { - av_log(avctx, AV_LOG_ERROR, "Only SUBTITLE_ASS type supported.\n"); + if (frame->subtitle_areas[i]->type != AV_SUBTITLE_FMT_ASS) { + av_log(avctx, AV_LOG_ERROR, "Only AV_SUBTITLE_FMT_ASS type supported.\n"); return AVERROR(EINVAL); } + if (!ass) + continue; + dialog = avpriv_ass_split_dialog(s->ass_ctx, ass); if (!dialog) return AVERROR(ENOMEM); + if (i > 0) + ttml_new_line_cb(s, 0); + if (dialog->style) { av_bprintf(&s->buffer, "buffer, dialog->style, NULL, @@ -130,17 +183,19 @@ static int ttml_encode_frame(AVCodecContext *avctx, uint8_t *buf, if (!av_bprint_is_complete(&s->buffer)) return AVERROR(ENOMEM); - if (!s->buffer.len) - return 0; - - // force null-termination, so in case our destination buffer is - // too small, the return value is larger than bufsize minus null. - if (av_strlcpy(buf, s->buffer.str, bufsize) > bufsize - 1) { - av_log(avctx, AV_LOG_ERROR, "Buffer too small for TTML event.\n"); - return AVERROR_BUFFER_TOO_SMALL; + + ret = ff_get_encode_buffer(avctx, avpkt, s->buffer.len + 3, 0); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Error getting output packet.\n"); + return ret; } - return s->buffer.len; + av_strlcpy(avpkt->data, s->buffer.str, avpkt->size); + + avpkt->size = s->buffer.len; + *got_packet = 1; + + return 0; } static av_cold int ttml_encode_close(AVCodecContext *avctx) @@ -370,13 +425,13 @@ static av_cold int ttml_encode_init(AVCodecContext *avctx) s->avctx = avctx; av_bprint_init(&s->buffer, 0, AV_BPRINT_SIZE_UNLIMITED); + s->ass_ctx = avpriv_ass_split((char*)avctx->subtitle_header); - if (!(s->ass_ctx = avpriv_ass_split(avctx->subtitle_header))) { - return AVERROR_INVALIDDATA; - } + if (s->ass_ctx) { + if (ret = ttml_write_header_content(avctx) < 0) + return ret; - if ((ret = ttml_write_header_content(avctx)) < 0) { - return ret; + s->extradata_written = 1; } return 0; @@ -389,7 +444,7 @@ const FFCodec ff_ttml_encoder = { .p.id = AV_CODEC_ID_TTML, .priv_data_size = sizeof(TTMLContext), .init = ttml_encode_init, - FF_CODEC_ENCODE_SUB_CB(ttml_encode_frame), + FF_CODEC_ENCODE_CB(ttml_encode_frame), .close = ttml_encode_close, .caps_internal = FF_CODEC_CAP_INIT_CLEANUP, }; diff --git a/libavcodec/utils.c b/libavcodec/utils.c index 552cdff70f..3c88952f2b 100644 --- a/libavcodec/utils.c +++ b/libavcodec/utils.c @@ -76,7 +76,6 @@ int av_codec_is_encoder(const AVCodec *avcodec) { const FFCodec *const codec = ffcodec(avcodec); return codec && (codec->cb_type == FF_CODEC_CB_TYPE_ENCODE || - codec->cb_type == FF_CODEC_CB_TYPE_ENCODE_SUB || codec->cb_type == FF_CODEC_CB_TYPE_RECEIVE_PACKET); } diff --git a/libavcodec/webvttenc.c b/libavcodec/webvttenc.c index 2df31f8fba..a21a922ef0 100644 --- a/libavcodec/webvttenc.c +++ b/libavcodec/webvttenc.c @@ -22,6 +22,7 @@ #include #include "avcodec.h" +#include "encode.h" #include "libavutil/avstring.h" #include "libavutil/bprint.h" #include "codec_internal.h" @@ -32,6 +33,7 @@ typedef struct { AVCodecContext *avctx; ASSSplitContext *ass_ctx; + int is_default_ass_context; AVBPrint buffer; unsigned timestamp_end; int count; @@ -155,43 +157,81 @@ static const ASSCodesCallbacks webvtt_callbacks = { .end = webvtt_end_cb, }; -static int webvtt_encode_frame(AVCodecContext *avctx, - unsigned char *buf, int bufsize, const AVSubtitle *sub) +static void ensure_ass_context(WebVTTContext* s, const AVFrame* frame) +{ + if (s->ass_ctx && !s->is_default_ass_context) + // We already have a (non-default context) + return; + + if (!frame->num_subtitle_areas) + // Don't need ass context for processing empty subtitle frames + return; + + // The frame has content, so we need to set up a context + if (frame->subtitle_header && frame->subtitle_header->size > 0) { + const char* subtitle_header = (char*)frame->subtitle_header->data; + avpriv_ass_split_free(s->ass_ctx); + s->ass_ctx = avpriv_ass_split(subtitle_header); + s->is_default_ass_context = 0; + } + else if (!s->ass_ctx) { + char* subtitle_header = avpriv_ass_get_subtitle_header_default(0); + if (!subtitle_header) + return; + + s->ass_ctx = avpriv_ass_split(subtitle_header); + s->is_default_ass_context = 1; + av_free(subtitle_header); + } +} + +static int webvtt_encode_frame(AVCodecContext* avctx, AVPacket* avpkt, + const AVFrame* frame, int* got_packet) { WebVTTContext *s = avctx->priv_data; ASSDialog *dialog; - int i; + int ret, i; + + ensure_ass_context(s, frame); av_bprint_clear(&s->buffer); - for (i=0; inum_rects; i++) { - const char *ass = sub->rects[i]->ass; + for (i=0; i< frame->num_subtitle_areas; i++) { + const char *ass = frame->subtitle_areas[i]->ass; - if (sub->rects[i]->type != SUBTITLE_ASS) { - av_log(avctx, AV_LOG_ERROR, "Only SUBTITLE_ASS type supported.\n"); + if (frame->subtitle_areas[i]->type != AV_SUBTITLE_FMT_ASS) { + av_log(avctx, AV_LOG_ERROR, "Only AV_SUBTITLE_FMT_ASS type supported.\n"); return AVERROR(EINVAL); } - dialog = avpriv_ass_split_dialog(s->ass_ctx, ass); - if (!dialog) - return AVERROR(ENOMEM); - webvtt_style_apply(s, dialog->style); - avpriv_ass_split_override_codes(&webvtt_callbacks, s, dialog->text); - avpriv_ass_free_dialog(&dialog); + if (ass) { + + if (i > 0) + webvtt_new_line_cb(s, 0); + + dialog = avpriv_ass_split_dialog(s->ass_ctx, ass); + if (!dialog) + return AVERROR(ENOMEM); + webvtt_style_apply(s, dialog->style); + avpriv_ass_split_override_codes(&webvtt_callbacks, s, dialog->text); + avpriv_ass_free_dialog(&dialog); + } } if (!av_bprint_is_complete(&s->buffer)) return AVERROR(ENOMEM); - if (!s->buffer.len) - return 0; - if (s->buffer.len > bufsize) { - av_log(avctx, AV_LOG_ERROR, "Buffer too small for ASS event.\n"); - return AVERROR_BUFFER_TOO_SMALL; + ret = ff_get_encode_buffer(avctx, avpkt, s->buffer.len + 1, 0); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Error getting output packet.\n"); + return ret; } - memcpy(buf, s->buffer.str, s->buffer.len); - return s->buffer.len; + memcpy(avpkt->data, s->buffer.str, s->buffer.len); + avpkt->size = s->buffer.len; + *got_packet = s->buffer.len > 0; + + return 0; } static int webvtt_encode_close(AVCodecContext *avctx) @@ -206,9 +246,9 @@ static av_cold int webvtt_encode_init(AVCodecContext *avctx) { WebVTTContext *s = avctx->priv_data; s->avctx = avctx; - s->ass_ctx = avpriv_ass_split(avctx->subtitle_header); + s->ass_ctx = avpriv_ass_split((char*)avctx->subtitle_header); av_bprint_init(&s->buffer, 0, AV_BPRINT_SIZE_UNLIMITED); - return s->ass_ctx ? 0 : AVERROR_INVALIDDATA; + return 0; } const FFCodec ff_webvtt_encoder = { @@ -218,6 +258,6 @@ const FFCodec ff_webvtt_encoder = { .p.id = AV_CODEC_ID_WEBVTT, .priv_data_size = sizeof(WebVTTContext), .init = webvtt_encode_init, - FF_CODEC_ENCODE_SUB_CB(webvtt_encode_frame), + FF_CODEC_ENCODE_CB(webvtt_encode_frame), .close = webvtt_encode_close, }; diff --git a/libavcodec/xsubenc.c b/libavcodec/xsubenc.c index 6f417c5958..10147bae8e 100644 --- a/libavcodec/xsubenc.c +++ b/libavcodec/xsubenc.c @@ -23,6 +23,7 @@ #include "avcodec.h" #include "bytestream.h" #include "codec_internal.h" +#include "encode.h" #include "put_bits.h" /** @@ -111,39 +112,56 @@ static int make_tc(uint64_t ms, int *tc) return ms > 99; } -static int xsub_encode(AVCodecContext *avctx, unsigned char *buf, - int bufsize, const AVSubtitle *h) +static int xsub_encode(AVCodecContext* avctx, AVPacket* avpkt, + const AVFrame* frame, int* got_packet) { - uint64_t startTime = h->pts / 1000; // FIXME: need better solution... - uint64_t endTime = startTime + h->end_display_time - h->start_display_time; + const int64_t duration_ms = (int64_t)((double)frame->subtitle_timing.duration * av_q2d(AV_TIME_BASE_Q) * 1000); + const uint64_t startTime = (int64_t)((double)frame->subtitle_timing.start_pts * av_q2d(AV_TIME_BASE_Q) * 1000); + const uint64_t endTime = startTime + duration_ms; int start_tc[4], end_tc[4]; - uint8_t *hdr = buf + 27; // Point behind the timestamp + uint8_t *hdr; uint8_t *rlelenptr; uint16_t width, height; - int i; + int i, ret; PutBitContext pb; + uint8_t* buf; + int64_t req_size; - if (bufsize < 27 + 7*2 + 4*3) { - av_log(avctx, AV_LOG_ERROR, "Buffer too small for XSUB header.\n"); - return AVERROR_BUFFER_TOO_SMALL; + if (!frame->num_subtitle_areas) { + // Don't encode empty sub events + return 0; } + // Estimate size (timestamp 27, header 7*2 + 4*3, padding 10) + req_size = 27 + 7*2 + 4*3 + 10; + req_size += frame->subtitle_areas[0]->linesize[0] * frame->subtitle_areas[0]->h; + req_size += 256; // Palette + + ret = ff_get_encode_buffer(avctx, avpkt, req_size, 0); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Error getting output packet.\n"); + return ret; + } + + buf = avpkt->data; + hdr = avpkt->data + 27; // Point behind the timestamp + // TODO: support multiple rects - if (h->num_rects != 1) - av_log(avctx, AV_LOG_WARNING, "Only single rects supported (%d in subtitle.)\n", h->num_rects); + if (frame->num_subtitle_areas != 1) + av_log(avctx, AV_LOG_WARNING, "Only single rects supported (%d in subtitle.)\n", frame->num_subtitle_areas); // TODO: render text-based subtitles into bitmaps - if (!h->rects[0]->data[0] || !h->rects[0]->data[1]) { + if (!frame->subtitle_areas[0]->buf[0]->data || !frame->subtitle_areas[0]->pal) { av_log(avctx, AV_LOG_WARNING, "No subtitle bitmap available.\n"); return AVERROR(EINVAL); } // TODO: color reduction, similar to dvdsub encoder - if (h->rects[0]->nb_colors > 4) - av_log(avctx, AV_LOG_WARNING, "No more than 4 subtitle colors supported (%d found.)\n", h->rects[0]->nb_colors); + if (frame->subtitle_areas[0]->nb_colors > 4) + av_log(avctx, AV_LOG_WARNING, "No more than 4 subtitle colors supported (%d found.)\n", frame->subtitle_areas[0]->nb_colors); // TODO: Palette swapping if color zero is not transparent - if (((uint32_t *)h->rects[0]->data[1])[0] & 0xff000000) + if (((uint32_t *)frame->subtitle_areas[0]->pal)[0] & 0xff000000) av_log(avctx, AV_LOG_WARNING, "Color index 0 is not transparent. Transparency will be messed up.\n"); if (make_tc(startTime, start_tc) || make_tc(endTime, end_tc)) { @@ -151,7 +169,7 @@ static int xsub_encode(AVCodecContext *avctx, unsigned char *buf, return AVERROR(EINVAL); } - snprintf(buf, 28, + snprintf((char *)avpkt->data, 28, "[%02d:%02d:%02d.%03d-%02d:%02d:%02d.%03d]", start_tc[3], start_tc[2], start_tc[1], start_tc[0], end_tc[3], end_tc[2], end_tc[1], end_tc[0]); @@ -160,45 +178,47 @@ static int xsub_encode(AVCodecContext *avctx, unsigned char *buf, // 2 pixels required on either side of subtitle. // Possibly due to limitations of hardware renderers. // TODO: check if the bitmap is already padded - width = FFALIGN(h->rects[0]->w, 2) + PADDING * 2; - height = FFALIGN(h->rects[0]->h, 2); + width = FFALIGN(frame->subtitle_areas[0]->w, 2) + PADDING * 2; + height = FFALIGN(frame->subtitle_areas[0]->h, 2); bytestream_put_le16(&hdr, width); bytestream_put_le16(&hdr, height); - bytestream_put_le16(&hdr, h->rects[0]->x); - bytestream_put_le16(&hdr, h->rects[0]->y); - bytestream_put_le16(&hdr, h->rects[0]->x + width -1); - bytestream_put_le16(&hdr, h->rects[0]->y + height -1); + bytestream_put_le16(&hdr, frame->subtitle_areas[0]->x); + bytestream_put_le16(&hdr, frame->subtitle_areas[0]->y); + bytestream_put_le16(&hdr, frame->subtitle_areas[0]->x + width -1); + bytestream_put_le16(&hdr, frame->subtitle_areas[0]->y + height -1); rlelenptr = hdr; // Will store length of first field here later. hdr+=2; // Palette for (i=0; i<4; i++) - bytestream_put_be24(&hdr, ((uint32_t *)h->rects[0]->data[1])[i]); + bytestream_put_be24(&hdr, ((uint32_t *)frame->subtitle_areas[0]->pal)[i]); // Bitmap // RLE buffer. Reserve 2 bytes for possible padding after the last row. - init_put_bits(&pb, hdr, bufsize - (hdr - buf) - 2); - if (xsub_encode_rle(&pb, h->rects[0]->data[0], - h->rects[0]->linesize[0] * 2, - h->rects[0]->w, (h->rects[0]->h + 1) >> 1)) + init_put_bits(&pb, hdr, avpkt->size - (hdr - buf) - 2); + if (xsub_encode_rle(&pb, frame->subtitle_areas[0]->buf[0]->data, + frame->subtitle_areas[0]->linesize[0] * 2, + frame->subtitle_areas[0]->w, (frame->subtitle_areas[0]->h + 1) >> 1)) return AVERROR_BUFFER_TOO_SMALL; bytestream_put_le16(&rlelenptr, put_bytes_count(&pb, 0)); // Length of first field - if (xsub_encode_rle(&pb, h->rects[0]->data[0] + h->rects[0]->linesize[0], - h->rects[0]->linesize[0] * 2, - h->rects[0]->w, h->rects[0]->h >> 1)) + if (xsub_encode_rle(&pb, frame->subtitle_areas[0]->buf[0]->data + frame->subtitle_areas[0]->linesize[0], + frame->subtitle_areas[0]->linesize[0] * 2, + frame->subtitle_areas[0]->w, frame->subtitle_areas[0]->h >> 1)) return AVERROR_BUFFER_TOO_SMALL; // Enforce total height to be a multiple of 2 - if (h->rects[0]->h & 1) { - put_xsub_rle(&pb, h->rects[0]->w, PADDING_COLOR); + if (frame->subtitle_areas[0]->h & 1) { + put_xsub_rle(&pb, frame->subtitle_areas[0]->w, PADDING_COLOR); } flush_put_bits(&pb); - return hdr - buf + put_bytes_output(&pb); + avpkt->size = hdr - buf + put_bytes_output(&pb); + *got_packet = 1; + return 0; } static av_cold int xsub_encoder_init(AVCodecContext *avctx) @@ -217,5 +237,5 @@ const FFCodec ff_xsub_encoder = { .p.type = AVMEDIA_TYPE_SUBTITLE, .p.id = AV_CODEC_ID_XSUB, .init = xsub_encoder_init, - FF_CODEC_ENCODE_SUB_CB(xsub_encode), + FF_CODEC_ENCODE_CB(xsub_encode), }; From patchwork Tue Oct 25 09:13:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 38998 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2686501pzb; Tue, 25 Oct 2022 02:17:59 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5UObFnEPDWp+SbwDxP6xkY7Z/qyFtVu4cKls+7n9SHsZwsblOMzZsyaMmNPKUKcFK02Uxg X-Received: by 2002:a17:907:320c:b0:77b:6f08:9870 with SMTP id xg12-20020a170907320c00b0077b6f089870mr31456344ejb.249.1666689479209; Tue, 25 Oct 2022 02:17:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666689479; cv=none; d=google.com; s=arc-20160816; b=XNamwn21g28DAVPXsJks9mjqeaECvT9ilGPW/7rHmsOz16DpVQ1ocK/0Mw5FxeqH+m 2AWdyhoo85nTmF53A+B6TputALlnN1zM5OLjIFO7ANoK94KWblovnbToDIr5VmCuiTI6 JqvGNyEBb+Qk34Z3T+HSHdOgn/Qaz2eUZEl8PfcaPLjZEQM1F8BduEhGNH0HCMp7kAbj jnIREYIuVtVoI0QlU4CZ9gskV9cuQJmNgFYP9bC8tR1u5v4Pkuxj4IOgDcOKpMN0hTTt 51JWMwgK6KON2e7lrt4V3bB8fNnZsFzT3ewZ1LksBrp5RXew8pBYaYuKf5waDk3eOq+N q+Iw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:references :in-reply-to:message-id:from:dkim-signature:delivered-to; bh=1zriloKhR5mMs0Af/JkRYRoHzH1lCJnvWdDp285UNz8=; b=Z6XH5ZGVagUg+FUCYLwUtLinOxGrWOw1L+sVETeXTN8/IMEsyIaGqLN5edDXny9F6f TBI3cV7JybS2aT/L42CM62pnjaV9zUmRlouquc/+yLE36/yNiCpFWLdZb7x3DJ9s45AY AkY0B1mXoxdVrMdeIk5vCKVTBKf2uUidjHtwVEN1XPQBLARnFfvIIg44mMs5RiDy9sCK aR4i0EoRWTWEjQmsTWyMD/N4CC1qlihXgYJPuXTNilag1NQzejwvKBEzQC8O9QP6adBz PchsWN1Vvm0dLzlf1CFFH3y31jQGizyGUEnF3xkvlgxuUICcSTbDczOx9CDSn62OnBB5 PsgQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=iEAhJZSC; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id w1-20020a056402268100b0045166555fddsi2885888edd.481.2022.10.25.02.17.58; Tue, 25 Oct 2022 02:17:59 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=iEAhJZSC; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2917768BDA2; Tue, 25 Oct 2022 12:14:23 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com [209.85.210.171]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B27DD68BD8B for ; Tue, 25 Oct 2022 12:14:20 +0300 (EEST) Received: by mail-pf1-f171.google.com with SMTP id g16so5803944pfr.12 for ; Tue, 25 Oct 2022 02:14:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=AlnJjD3nXhlyfaGGzH4xX1J9/Ue8/lBSx1eajDIZVSY=; b=iEAhJZSCOFZsS2b1+xb8MHWJvJ0irXOrT1k+R3/ZjEDvNuqaWTgOy0dQaonbwJ3joj BCZk0U3iua7DOroJ1MOpe5uJ8G5e7ropykWgsf+YFcNqj0okOCgOHMnRDm0LHYFWF5fz FFjHPQAwNdazWzYth1UVUxsNd6v41BnwbBRGTh4jG5f3X94tdX6zHkd58TGvRNKEky9x A7HsMLgMeRj98WBZ4yn5geSDok0YoEz7VEASrRIVVSPnvUWSrFPW/9jFWDsy4rS+mF2P KPRu8XI1uqSoBag2mg9Nbp9Uduqd5R2yTRomPdTry3elMRqCAZ+mP/yC6xCQLfWwuBf7 wsow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=AlnJjD3nXhlyfaGGzH4xX1J9/Ue8/lBSx1eajDIZVSY=; b=JDo4VxRBNAKVDaZ8O0JV18qJqMwsWVNVHUzooMOuQ8aI+3a7uyeE2D0g87nfCzHxJF 0sUG+JONXgmdjOYqnYaFwlgNZR/cggN2OXNVwbIEyFnI4VRvqcYtdWKiSw3DjKg41/wE 1/zgszTgYpyayYBxXv0/CMDCb6D0WplJmMhJc3YmHDW9G6y5e7lYXyWWKn8G6pO6+5CY kdaaXsgrydUS1NUolq52lNP6t0Gf8gY+cZc0K1XE7bNEYGU3DhpnXjjFF0Q+w3jhZvaY vpcQ7j8E4lrduykcKqpoVbKSfqMc1tkEUo5hcAqJ8bubZLbgS27totkGlD6zAAQUpkOc Y4TA== X-Gm-Message-State: ACrzQf0IMbzQIgVsIt41FSocVQnSnRfh5fIIpQqjtAzD6QXTP5cgPkl5 DU/Q/3EG2Hk63P6dq6vXtqHbfiW/ALQ= X-Received: by 2002:a65:4c46:0:b0:460:f598:d038 with SMTP id l6-20020a654c46000000b00460f598d038mr32813972pgr.99.1666689257722; Tue, 25 Oct 2022 02:14:17 -0700 (PDT) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id b5-20020a170902e94500b001868da70685sm873055pll.235.2022.10.25.02.14.16 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Oct 2022 02:14:16 -0700 (PDT) From: softworkz X-Google-Original-From: softworkz Message-Id: <6046dde1251f3f867eae283401f75124571fdad8.1666689226.git.ffmpegagent@gmail.com> In-Reply-To: References: Date: Tue, 25 Oct 2022 09:13:45 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v9 24/25] fftools/ffmpeg: Introduce subtitle filtering and new frame-based subtitle encoding X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: softworkz Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: rcUHmfki2bcs From: softworkz This commit actually enables subtitle filtering in ffmpeg by sending and receiving subtitle frames to and from a filtergraph. The heartbeat functionality from the previous sub2video implementation is removed and now provided by the 'subfeed' filter. The other part of sub2video functionality is retained by auto-insertion of the new graphicsub2video filter. Justification for changed test refs: - sub2video The previous results were incorrect. The command line for this test specifies -r 5 (5 frames per second), which is now fulfilled by the additional frames in the reference output. Example: The first subtitle time is 499000, the second is 15355000, which means 0.5s and 15.35s with a difference of 14.85s. 15s * 5fps = 75 frames and that's now the exact number of video frames between these two subtitle events. - sub2video_basic The previous results had some incorrect output because multiple frames had the same dts The non-empty content frames are visually identical, the different CRC is due to the different blending algorithm that is being used. - sub2video_time_limited Subtitle frames are emitted to the filter graphs at a 5 fps rate by default. The time limit for this test is 15s * 5fps = 75 frames which matches the count in the new ref. - sub-dvb Running ffprobe -show_frames on the source file shows that there are 7 subtitle frames with 0 rects in the source at the start and 2 at the end. This translates to the 14 and 4 additional entries in the new test results. - filter-overlay-dvdsub-2397 Overlay results have slightly different CRCs due to different blending implementation - sub-scc The first entry is no longer in the output because it is before the actual start time and the strim filter removes such entries now (like for video and audio) Signed-off-by: softworkz --- fftools/ffmpeg.c | 604 +++++----- fftools/ffmpeg.h | 17 +- fftools/ffmpeg_filter.c | 268 ++++- fftools/ffmpeg_hw.c | 2 +- fftools/ffmpeg_mux_init.c | 28 +- fftools/ffmpeg_opt.c | 1 + tests/ref/fate/filter-overlay-dvdsub-2397 | 181 ++- tests/ref/fate/sub-dvb | 162 +-- tests/ref/fate/sub-scc | 1 - tests/ref/fate/sub2video | 1092 +++++++++++++++++- tests/ref/fate/sub2video_basic | 1238 +++++++++++++++++++-- tests/ref/fate/sub2video_time_limited | 78 +- 12 files changed, 3024 insertions(+), 648 deletions(-) diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index e57486fd4a..e130bc3e8d 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -159,163 +159,6 @@ static struct termios oldtty; static int restore_tty; #endif -/* sub2video hack: - Convert subtitles to video with alpha to insert them in filter graphs. - This is a temporary solution until libavfilter gets real subtitles support. - */ - -static int sub2video_get_blank_frame(InputStream *ist) -{ - int ret; - AVFrame *frame = ist->sub2video.frame; - - av_frame_unref(frame); - ist->sub2video.frame->width = ist->dec_ctx->width ? ist->dec_ctx->width : ist->sub2video.w; - ist->sub2video.frame->height = ist->dec_ctx->height ? ist->dec_ctx->height : ist->sub2video.h; - ist->sub2video.frame->format = AV_PIX_FMT_RGB32; - if ((ret = av_frame_get_buffer(frame, 0)) < 0) - return ret; - memset(frame->data[0], 0, frame->height * frame->linesize[0]); - return 0; -} - -static void sub2video_copy_rect(uint8_t *dst, int dst_linesize, int w, int h, - AVSubtitleRect *r) -{ - uint32_t *pal, *dst2; - uint8_t *src, *src2; - int x, y; - - if (r->type != SUBTITLE_BITMAP) { - av_log(NULL, AV_LOG_WARNING, "sub2video: non-bitmap subtitle\n"); - return; - } - if (r->x < 0 || r->x + r->w > w || r->y < 0 || r->y + r->h > h) { - av_log(NULL, AV_LOG_WARNING, "sub2video: rectangle (%d %d %d %d) overflowing %d %d\n", - r->x, r->y, r->w, r->h, w, h - ); - return; - } - - dst += r->y * dst_linesize + r->x * 4; - src = r->data[0]; - pal = (uint32_t *)r->data[1]; - for (y = 0; y < r->h; y++) { - dst2 = (uint32_t *)dst; - src2 = src; - for (x = 0; x < r->w; x++) - *(dst2++) = pal[*(src2++)]; - dst += dst_linesize; - src += r->linesize[0]; - } -} - -static void sub2video_push_ref(InputStream *ist, int64_t pts) -{ - AVFrame *frame = ist->sub2video.frame; - int i; - int ret; - - av_assert1(frame->data[0]); - ist->sub2video.last_pts = frame->pts = pts; - for (i = 0; i < ist->nb_filters; i++) { - ret = av_buffersrc_add_frame_flags(ist->filters[i]->filter, frame, - AV_BUFFERSRC_FLAG_KEEP_REF | - AV_BUFFERSRC_FLAG_PUSH); - if (ret != AVERROR_EOF && ret < 0) - av_log(NULL, AV_LOG_WARNING, "Error while add the frame to buffer source(%s).\n", - av_err2str(ret)); - } -} - -void sub2video_update(InputStream *ist, int64_t heartbeat_pts, AVSubtitle *sub) -{ - AVFrame *frame = ist->sub2video.frame; - int8_t *dst; - int dst_linesize; - int num_rects, i; - int64_t pts, end_pts; - - if (!frame) - return; - if (sub) { - pts = av_rescale_q(sub->pts + sub->start_display_time * 1000LL, - AV_TIME_BASE_Q, ist->st->time_base); - end_pts = av_rescale_q(sub->pts + sub->end_display_time * 1000LL, - AV_TIME_BASE_Q, ist->st->time_base); - num_rects = sub->num_rects; - } else { - /* If we are initializing the system, utilize current heartbeat - PTS as the start time, and show until the following subpicture - is received. Otherwise, utilize the previous subpicture's end time - as the fall-back value. */ - pts = ist->sub2video.initialize ? - heartbeat_pts : ist->sub2video.end_pts; - end_pts = INT64_MAX; - num_rects = 0; - } - if (sub2video_get_blank_frame(ist) < 0) { - av_log(NULL, AV_LOG_ERROR, - "Impossible to get a blank canvas.\n"); - return; - } - dst = frame->data [0]; - dst_linesize = frame->linesize[0]; - for (i = 0; i < num_rects; i++) - sub2video_copy_rect(dst, dst_linesize, frame->width, frame->height, sub->rects[i]); - sub2video_push_ref(ist, pts); - ist->sub2video.end_pts = end_pts; - ist->sub2video.initialize = 0; -} - -static void sub2video_heartbeat(InputStream *ist, int64_t pts) -{ - InputFile *infile = input_files[ist->file_index]; - int i, j, nb_reqs; - int64_t pts2; - - /* When a frame is read from a file, examine all sub2video streams in - the same file and send the sub2video frame again. Otherwise, decoded - video frames could be accumulating in the filter graph while a filter - (possibly overlay) is desperately waiting for a subtitle frame. */ - for (i = 0; i < infile->nb_streams; i++) { - InputStream *ist2 = input_streams[infile->ist_index + i]; - if (!ist2->sub2video.frame) - continue; - /* subtitles seem to be usually muxed ahead of other streams; - if not, subtracting a larger time here is necessary */ - pts2 = av_rescale_q(pts, ist->st->time_base, ist2->st->time_base) - 1; - /* do not send the heartbeat frame if the subtitle is already ahead */ - if (pts2 <= ist2->sub2video.last_pts) - continue; - if (pts2 >= ist2->sub2video.end_pts || ist2->sub2video.initialize) - /* if we have hit the end of the current displayed subpicture, - or if we need to initialize the system, update the - overlayed subpicture and its start/end times */ - sub2video_update(ist2, pts2 + 1, NULL); - for (j = 0, nb_reqs = 0; j < ist2->nb_filters; j++) - nb_reqs += av_buffersrc_get_nb_failed_requests(ist2->filters[j]->filter); - if (nb_reqs) - sub2video_push_ref(ist2, pts2); - } -} - -static void sub2video_flush(InputStream *ist) -{ - int i; - int ret; - - if (ist->sub2video.end_pts < INT64_MAX) - sub2video_update(ist, INT64_MAX, NULL); - for (i = 0; i < ist->nb_filters; i++) { - ret = av_buffersrc_add_frame(ist->filters[i]->filter, NULL); - if (ret != AVERROR_EOF && ret < 0) - av_log(NULL, AV_LOG_WARNING, "Flush the frame error.\n"); - } -} - -/* end of sub2video hack */ - static void term_exit_sigsafe(void) { #if HAVE_TERMIOS_H @@ -516,7 +359,6 @@ static void ffmpeg_cleanup(int ret) avfilter_graph_free(&fg->graph); for (j = 0; j < fg->nb_inputs; j++) { InputFilter *ifilter = fg->inputs[j]; - struct InputStream *ist = ifilter->ist; if (ifilter->frame_queue) { AVFrame *frame; @@ -525,12 +367,6 @@ static void ffmpeg_cleanup(int ret) av_fifo_freep2(&ifilter->frame_queue); } av_freep(&ifilter->displaymatrix); - if (ist->sub2video.sub_queue) { - AVSubtitle sub; - while (av_fifo_read(ist->sub2video.sub_queue, &sub, 1) >= 0) - avsubtitle_free(&sub); - av_fifo_freep2(&ist->sub2video.sub_queue); - } av_buffer_unref(&ifilter->hw_frames_ctx); av_freep(&ifilter->name); av_freep(&fg->inputs[j]); @@ -563,15 +399,19 @@ static void ffmpeg_cleanup(int ret) for (i = 0; i < nb_input_streams; i++) { InputStream *ist = input_streams[i]; + if (ist->prev_sub.subtitle == ist->decoded_frame) + // Prevent double-free + ist->prev_sub.subtitle = NULL; + av_frame_free(&ist->decoded_frame); av_packet_free(&ist->pkt); av_dict_free(&ist->decoder_opts); - avsubtitle_free(&ist->prev_sub.subtitle); - av_frame_free(&ist->sub2video.frame); + av_frame_free(&ist->prev_sub.subtitle); av_freep(&ist->filters); av_freep(&ist->hwaccel_device); av_freep(&ist->dts_buffer); + av_buffer_unref(&ist->subtitle_header); avcodec_free_context(&ist->dec_ctx); avcodec_parameters_free(&ist->par); @@ -974,23 +814,80 @@ static void do_audio_out(OutputFile *of, OutputStream *ost, exit_program(1); } -static void do_subtitle_out(OutputFile *of, - OutputStream *ost, - AVSubtitle *sub) +static void encode_subtitle_frame(OutputFile *of, OutputStream *ost, AVFrame *frame, AVPacket *pkt, int64_t pts_offset) +{ + AVCodecContext *enc = ost->enc_ctx; + int ret; + + ost->frames_encoded++; + + ret = avcodec_send_frame(enc, frame); + if (ret < 0) + goto error; + + while (1) { + ret = avcodec_receive_packet(enc, pkt); + update_benchmark("encode_subtitles %d.%d", ost->file_index, ost->index); + if (ret == AVERROR(EAGAIN)) + break; + if (ret < 0) + goto error; + + if (debug_ts) { + av_log(NULL, AV_LOG_INFO, "encoder -> type:subtitles " + "pkt_pts:%s pkt_pts_time:%s pkt_dts:%s pkt_dts_time:%s\n", + av_ts2str(pkt->pts), av_ts2timestr(pkt->pts, &enc->time_base), + av_ts2str(pkt->dts), av_ts2timestr(pkt->dts, &enc->time_base)); + } + + pkt->pts = av_rescale_q(frame->subtitle_timing.start_pts, AV_TIME_BASE_Q, ost->mux_timebase); + pkt->duration = av_rescale_q(frame->subtitle_timing.duration, AV_TIME_BASE_Q, ost->mux_timebase); + pkt->pts += pts_offset; + + pkt->dts = pkt->pts; + of_output_packet(of, pkt, ost, 0); + } + ost->next_pts++; + + return; +error: + av_log(NULL, AV_LOG_FATAL, "Subtitle encoding failed - Error code: %d\n", ret); + exit_program(1); +} + +static void do_subtitle_out(OutputFile *of, OutputStream *ost, AVFrame *frame) { - int subtitle_out_max_size = 1024 * 1024; - int subtitle_out_size, nb, i, ret; + int nb, i; AVCodecContext *enc; AVPacket *pkt = ost->pkt; int64_t pts; - if (sub->pts == AV_NOPTS_VALUE) { + if (!frame) + return; + + av_log(NULL, AV_LOG_DEBUG, "do_subtitle_out: sub->pts: %"PRId64" frame->pts: %"PRId64"\n", frame->subtitle_timing.start_pts, frame->pts); + + if (frame->subtitle_timing.start_pts == AV_NOPTS_VALUE) { av_log(NULL, AV_LOG_ERROR, "Subtitle packets must have a pts\n"); if (exit_on_error) exit_program(1); return; } + if (frame->repeat_sub) { + av_log(NULL, AV_LOG_DEBUG, "Ignoring repeated subtitle frame\n"); + return; + } + + ////if (ost->last_subtitle_pts && ost->last_subtitle_pts == frame->subtitle_timing.start_pts) { + //// av_log(NULL, AV_LOG_DEBUG, "Ignoring subtitle frame with duplicate subtitle_pts\n"); + //// return; + ////} + + ost->last_subtitle_pts = frame->subtitle_timing.start_pts; + + init_output_stream_wrapper(ost, frame, 1); + enc = ost->enc_ctx; /* Note: DVB subtitle need one packet to draw them and one other @@ -1002,50 +899,38 @@ static void do_subtitle_out(OutputFile *of, nb = 1; /* shift timestamp to honor -ss and make check_recording_time() work with -t */ - pts = sub->pts; + pts = frame->subtitle_timing.start_pts; if (output_files[ost->file_index]->start_time != AV_NOPTS_VALUE) pts -= output_files[ost->file_index]->start_time; - for (i = 0; i < nb; i++) { - unsigned save_num_rects = sub->num_rects; - if (!check_recording_time(ost, pts, AV_TIME_BASE_Q)) - return; + ost->next_pts = av_rescale_q(pts, AV_TIME_BASE_Q, enc->time_base); + if (!check_recording_time(ost, ost->next_pts, ost->enc_ctx->time_base)) + return; - ret = av_new_packet(pkt, subtitle_out_max_size); - if (ret < 0) - report_and_exit(AVERROR(ENOMEM)); + frame->subtitle_timing.start_pts = pts; - sub->pts = pts; - // start_display_time is required to be 0 - sub->pts += av_rescale_q(sub->start_display_time, (AVRational){ 1, 1000 }, AV_TIME_BASE_Q); - sub->end_display_time -= sub->start_display_time; - sub->start_display_time = 0; - if (i == 1) - sub->num_rects = 0; + for (i = 0; i < nb; i++) { + const unsigned save_num_rects = frame->num_subtitle_areas; + int64_t pts_offset = 0; ost->frames_encoded++; - subtitle_out_size = avcodec_encode_subtitle(enc, pkt->data, pkt->size, sub); if (i == 1) - sub->num_rects = save_num_rects; - if (subtitle_out_size < 0) { - av_log(NULL, AV_LOG_FATAL, "Subtitle encoding failed\n"); - exit_program(1); - } + frame->num_subtitle_areas = 0; - av_shrink_packet(pkt, subtitle_out_size); - pkt->pts = av_rescale_q(sub->pts, AV_TIME_BASE_Q, ost->mux_timebase); - pkt->duration = av_rescale_q(sub->end_display_time, (AVRational){ 1, 1000 }, ost->mux_timebase); if (enc->codec_id == AV_CODEC_ID_DVB_SUBTITLE) { /* XXX: the pts correction is handled here. Maybe handling it in the codec would be better */ if (i == 0) - pkt->pts += av_rescale_q(sub->start_display_time, (AVRational){ 1, 1000 }, ost->mux_timebase); + pts_offset = 0; else - pkt->pts += av_rescale_q(sub->end_display_time, (AVRational){ 1, 1000 }, ost->mux_timebase); + pts_offset = av_rescale_q(frame->subtitle_timing.duration, AV_TIME_BASE_Q, ost->mux_timebase); } - pkt->dts = pkt->pts; - of_output_packet(of, pkt, ost, 0); + + encode_subtitle_frame(of, ost, frame, pkt, pts_offset); + + if (i == 1) + frame->num_subtitle_areas = save_num_rects; } } @@ -1351,8 +1236,26 @@ static int reap_filters(int flush) } do_audio_out(of, ost, filtered_frame); break; + case AVMEDIA_TYPE_SUBTITLE: + + if (filtered_frame->format == AV_SUBTITLE_FMT_ASS && !enc->subtitle_header + && filtered_frame->subtitle_header) { + const char *subtitle_header = (char *)filtered_frame->subtitle_header->data; + enc->subtitle_header = (uint8_t *)av_strdup(subtitle_header); + if (!enc->subtitle_header) + return AVERROR(ENOMEM); + enc->subtitle_header_size = strlen(subtitle_header); + } + + if ((ost->enc_ctx->width == 0 || ost->enc_ctx->height == 0) + && filter->inputs[0]->w > 0 && filter->inputs[0]->h > 0 ) { + ost->enc_ctx->width = filter->inputs[0]->w; + ost->enc_ctx->height = filter->inputs[0]->h; + } + + do_subtitle_out(of, ost, filtered_frame); + break; default: - // TODO support subtitle filters av_assert0(0); } @@ -1875,7 +1778,8 @@ static int ifilter_has_all_input_formats(FilterGraph *fg) int i; for (i = 0; i < fg->nb_inputs; i++) { if (fg->inputs[i]->format < 0 && (fg->inputs[i]->type == AVMEDIA_TYPE_AUDIO || - fg->inputs[i]->type == AVMEDIA_TYPE_VIDEO)) + fg->inputs[i]->type == AVMEDIA_TYPE_VIDEO || + fg->inputs[i]->type == AVMEDIA_TYPE_SUBTITLE)) return 0; } return 1; @@ -1902,6 +1806,18 @@ static int ifilter_send_frame(InputFilter *ifilter, AVFrame *frame, int keep_ref case AVMEDIA_TYPE_VIDEO: need_reinit |= ifilter->width != frame->width || ifilter->height != frame->height; + + if (need_reinit) + need_reinit = 1; + break; + case AVMEDIA_TYPE_SUBTITLE: + need_reinit |= ifilter->width != frame->width || + ifilter->height != frame->height; + + need_reinit &= (ifilter->width == 0 || ifilter->height == 0); + + if (need_reinit) + need_reinit = 1; break; } @@ -1973,12 +1889,9 @@ static int ifilter_send_eof(InputFilter *ifilter, int64_t pts) return ret; } else { // the filtergraph was never configured - if (ifilter->format < 0) { - ret = ifilter_parameters_from_codecpar(ifilter, ifilter->ist->par); - if (ret < 0) - return ret; - } - if (ifilter->format < 0 && (ifilter->type == AVMEDIA_TYPE_AUDIO || ifilter->type == AVMEDIA_TYPE_VIDEO)) { + if (ifilter->format < 0) + ifilter_parameters_from_codecpar(ifilter, ifilter->ist->st->codecpar); + if (ifilter->format < 0 && (ifilter->type == AVMEDIA_TYPE_AUDIO || ifilter->type == AVMEDIA_TYPE_VIDEO || ifilter->type == AVMEDIA_TYPE_SUBTITLE)) { av_log(NULL, AV_LOG_ERROR, "Cannot determine format of input stream %d:%d after EOF\n", ifilter->ist->file_index, ifilter->ist->st->index); return AVERROR_INVALIDDATA; } @@ -2016,7 +1929,7 @@ static int decode(AVCodecContext *avctx, AVFrame *frame, int *got_frame, AVPacke static int send_frame_to_filters(InputStream *ist, AVFrame *decoded_frame) { - int i, ret; + int i, ret = 0; av_assert1(ist->nb_filters > 0); /* ensure ret is initialized */ for (i = 0; i < ist->nb_filters; i++) { @@ -2216,77 +2129,198 @@ fail: return err < 0 ? err : ret; } -static int transcode_subtitles(InputStream *ist, AVPacket *pkt, int *got_output, +static void subtitle_send_kickoff(InputStream *ist, int64_t heartbeat_pts) +{ + AVFrame *frame; + int64_t pts; + + frame = av_frame_alloc(); + if (!frame) { + av_log(ist->dec_ctx, AV_LOG_ERROR, "Unable to alloc frame (out of memory).\n"); + return; + } + + frame->type = AVMEDIA_TYPE_SUBTITLE; + av_frame_get_buffer2(frame, 0); + + frame->format = (uint16_t)ist->dec_ctx->subtitle_type; + + pts = FFMAX(heartbeat_pts, ist->subtitle_kickoff.last_pts) + av_rescale_q(10, (AVRational){ 1, 1000 }, ist->st->time_base); + + frame->width = ist->subtitle_kickoff.w; + frame->height = ist->subtitle_kickoff.h; + frame->subtitle_timing.start_pts = av_rescale_q(pts, ist->st->time_base, AV_TIME_BASE_Q); + frame->subtitle_timing.duration = av_rescale_q(10, (AVRational){ 1, 1000 }, AV_TIME_BASE_Q); + frame->pts = pts; + + av_log(NULL, AV_LOG_WARNING, "subtitle_kickoff: call subtitle_resend_current %"PRId64" frame->format: %d\n", pts, frame->format); + + ist->subtitle_kickoff.last_pts = pts; + + send_frame_to_filters(ist, frame); + + av_frame_free(&frame); +} + +// Opposed to the earlier "subtitle hearbeat", this is primarily aimed at +// sending an initial subtitle frame to the filters for propagating the initial +// timing values and to avoid that too much time is spent on a single "HW" decoder +// which could overflow its buffer pool otherwise +static void subtitle_kickoff(InputStream *ist, int64_t pts) +{ + int i; + int64_t pts2; + int64_t diff; + + if (ist->st->codecpar->codec_type != AVMEDIA_TYPE_VIDEO) + return; + + for (i = 0; i < nb_input_streams; i++) { + InputStream *ist2 = input_streams[i]; + unsigned j, nb_reqs; + + if (!ist2->subtitle_kickoff.is_active) + continue; + /* It's primarily the initial send that matters and which propagates + * the right start times to subtitle filters depending on input from decoding */ + diff = av_rescale_q(5000, (AVRational){ 1, 1000 }, ist2->st->time_base); + pts2 = av_rescale_q(pts, ist->st->time_base, ist2->st->time_base); + + /* do not send a kickoff frame if the subtitle is already ahead */ + if (pts2 - diff <= ist2->subtitle_kickoff.last_pts) + continue; + + for (j = 0, nb_reqs = 0; j < ist2->nb_filters; j++) { + if (ist2->filters[j]->filter == NULL) { + subtitle_send_kickoff(ist2, pts2); + return; + } + nb_reqs += av_buffersrc_get_nb_failed_requests(ist2->filters[j]->filter); + } + if (nb_reqs) { + av_log(NULL, AV_LOG_WARNING, "subtitle_kickoff: resend - pts: %"PRIi64"\n", pts2); + subtitle_send_kickoff(ist2, pts2); + } + } +} + +static InputStream *get_input_stream(OutputStream *ost) +{ + if (ost->source_index >= 0) + return input_streams[ost->source_index]; + return NULL; +} + +static int decode_subtitles(InputStream *ist, AVPacket *pkt, int *got_output, int *decode_failed) { - AVSubtitle subtitle; - int free_sub = 1; - int ret = avcodec_decode_subtitle2(ist->dec_ctx, - &subtitle, got_output, pkt); + AVFrame *decoded_frame; + AVCodecContext *avctx = ist->dec_ctx; + int i = 0, ret = 0, err = 0; + int64_t pts; + + if (!ist->decoded_frame && !((ist->decoded_frame = av_frame_alloc()))) + return AVERROR(ENOMEM); + decoded_frame = ist->decoded_frame; + decoded_frame->type = AVMEDIA_TYPE_SUBTITLE; + decoded_frame->format = (uint16_t)avctx->subtitle_type; + + if (!ist->subtitle_header && avctx->subtitle_header && avctx->subtitle_header_size > 0) { + ist->subtitle_header = av_buffer_allocz(avctx->subtitle_header_size + 1); + if (!ist->subtitle_header) + return AVERROR(ENOMEM); + memcpy(ist->subtitle_header->data, avctx->subtitle_header, avctx->subtitle_header_size); + } - check_decode_result(NULL, got_output, ret); + ret = decode(avctx, decoded_frame, got_output, pkt); + + if (ret != AVERROR_EOF) + check_decode_result(NULL, got_output, ret); if (ret < 0 || !*got_output) { *decode_failed = 1; - if (!pkt->size) - sub2video_flush(ist); + if (!pkt->size) { + // Flush + for (i = 0; i < ist->nb_filters; i++) { + if (ist->filters[i]->filter != NULL) { + ret = av_buffersrc_add_frame(ist->filters[i]->filter, NULL); + if (ret != AVERROR_EOF && ret < 0) + av_log(NULL, AV_LOG_WARNING, "Flush the frame error.\n"); + } + } + } return ret; } if (ist->fix_sub_duration) { - int end = 1; - if (ist->prev_sub.got_output) { - end = av_rescale(subtitle.pts - ist->prev_sub.subtitle.pts, - 1000, AV_TIME_BASE); - if (end < ist->prev_sub.subtitle.end_display_time) { - av_log(NULL, AV_LOG_DEBUG, - "Subtitle duration reduced from %"PRId32" to %d%s\n", - ist->prev_sub.subtitle.end_display_time, end, - end <= 0 ? ", dropping it" : ""); - ist->prev_sub.subtitle.end_display_time = end; + int64_t end = 1; + if (ist->prev_sub.got_output && ist->prev_sub.subtitle) { + + const int64_t duration = ist->prev_sub.subtitle->subtitle_timing.duration; + end = decoded_frame->subtitle_timing.start_pts - ist->prev_sub.subtitle->subtitle_timing.start_pts; + + if (end < duration) { + av_log(avctx, AV_LOG_DEBUG, "Subtitle duration reduced from %"PRId64" to %"PRId64"%s\n", + duration, end, end <= 0 ? ", dropping it" : ""); + ist->prev_sub.subtitle->subtitle_timing.duration = end; } } - FFSWAP(int, *got_output, ist->prev_sub.got_output); - FFSWAP(int, ret, ist->prev_sub.ret); - FFSWAP(AVSubtitle, subtitle, ist->prev_sub.subtitle); + FFSWAP(int, *got_output, ist->prev_sub.got_output); + FFSWAP(int, ret, ist->prev_sub.ret); + FFSWAP(AVFrame*, ist->decoded_frame, ist->prev_sub.subtitle); + decoded_frame = ist->decoded_frame; if (end <= 0) - goto out; + return (int)end; } - if (!*got_output) + if (!*got_output || !decoded_frame) return ret; - if (ist->sub2video.frame) { - sub2video_update(ist, INT64_MIN, &subtitle); - } else if (ist->nb_filters) { - if (!ist->sub2video.sub_queue) - ist->sub2video.sub_queue = av_fifo_alloc2(8, sizeof(AVSubtitle), AV_FIFO_FLAG_AUTO_GROW); - if (!ist->sub2video.sub_queue) - report_and_exit(AVERROR(ENOMEM)); + decoded_frame->type = AVMEDIA_TYPE_SUBTITLE; - ret = av_fifo_write(ist->sub2video.sub_queue, &subtitle, 1); - if (ret < 0) - exit_program(1); - free_sub = 0; + if (decoded_frame->format == AV_SUBTITLE_FMT_UNKNOWN) + decoded_frame->format = (uint16_t)avctx->subtitle_type; + + if ((ret = av_buffer_replace(&decoded_frame->subtitle_header, ist->subtitle_header)) < 0) + return ret; + + pts = av_rescale_q(decoded_frame->subtitle_timing.start_pts, AV_TIME_BASE_Q, ist->st->time_base); + if (ist->subtitle_kickoff.last_pts > 0 && pts <= ist->subtitle_kickoff.last_pts) + pts = ist->subtitle_kickoff.last_pts + 1; + + ist->subtitle_kickoff.last_pts = decoded_frame->pts = pts; + + if (ist->nb_filters > 0) { + AVFrame *filter_frame = av_frame_clone(decoded_frame); + if (!filter_frame) { + err = AVERROR(ENOMEM); + goto end; + } + + err = send_frame_to_filters(ist, filter_frame); + av_frame_free(&filter_frame); } - if (!subtitle.num_rects) - goto out; + if (err >= 0) { + for (OutputStream *ost = ost_iter(NULL); ost; ost = ost_iter(ost)) { + InputStream *ist_src = get_input_stream(ost); - ist->frames_decoded++; + if (!ist_src || !check_output_constraints(ist, ost) + || ist_src != ist + || !ost->enc_ctx + || ost->enc_ctx->codec_type != AVMEDIA_TYPE_SUBTITLE) + continue; - for (OutputStream *ost = ost_iter(NULL); ost; ost = ost_iter(ost)) { - if (!check_output_constraints(ist, ost) || !ost->enc_ctx - || ost->enc_ctx->codec_type != AVMEDIA_TYPE_SUBTITLE) - continue; + if (ost->filter && ost->filter->filter->nb_inputs > 0) + continue; - do_subtitle_out(output_files[ost->file_index], ost, &subtitle); + do_subtitle_out(output_files[ost->file_index], ost, decoded_frame); + } } -out: - if (free_sub) - avsubtitle_free(&subtitle); - return ret; +end: + av_frame_unref(decoded_frame); + return err < 0 ? err : ret; } static int send_filter_eof(InputStream *ist) @@ -2393,7 +2427,7 @@ static int process_input_packet(InputStream *ist, const AVPacket *pkt, int no_eo case AVMEDIA_TYPE_SUBTITLE: if (repeating) break; - ret = transcode_subtitles(ist, avpkt, &got_output, &decode_failed); + ret = decode_subtitles(ist, avpkt, &got_output, &decode_failed); if (!pkt && ret >= 0) ret = AVERROR_EOF; av_packet_unref(avpkt); @@ -2611,13 +2645,6 @@ static int init_input_stream(int ist_index, char *error, int error_len) return 0; } -static InputStream *get_input_stream(OutputStream *ost) -{ - if (ost->source_index >= 0) - return input_streams[ost->source_index]; - return NULL; -} - static int compare_int64(const void *a, const void *b) { return FFDIFFSIGN(*(const int64_t *)a, *(const int64_t *)b); @@ -3006,9 +3033,9 @@ static int init_output_stream_encode(OutputStream *ost, AVFrame *frame) break; case AVMEDIA_TYPE_SUBTITLE: enc_ctx->time_base = AV_TIME_BASE_Q; - if (!enc_ctx->width) { - enc_ctx->width = input_streams[ost->source_index]->par->width; - enc_ctx->height = input_streams[ost->source_index]->par->height; + if (!enc_ctx->width && ost->source_index >= 0) { + enc_ctx->width = input_streams[ost->source_index]->st->codecpar->width; + enc_ctx->height = input_streams[ost->source_index]->st->codecpar->height; } if (dec_ctx && dec_ctx->subtitle_header) { /* ASS code assumes this buffer is null terminated so add extra byte. */ @@ -3057,6 +3084,17 @@ static int init_output_stream_encode(OutputStream *ost, AVFrame *frame) return 0; } +static enum AVSubtitleType get_subtitle_format(const AVCodecDescriptor *codec_descriptor) +{ + if(codec_descriptor->props & AV_CODEC_PROP_BITMAP_SUB) + return AV_SUBTITLE_FMT_BITMAP; + + if(codec_descriptor->props & AV_CODEC_PROP_TEXT_SUB) + return AV_SUBTITLE_FMT_ASS; + + return AV_SUBTITLE_FMT_UNKNOWN; +} + static int init_output_stream(OutputStream *ost, AVFrame *frame, char *error, int error_len) { @@ -3064,13 +3102,23 @@ static int init_output_stream(OutputStream *ost, AVFrame *frame, if (ost->enc_ctx) { const AVCodec *codec = ost->enc_ctx->codec; + AVCodecContext *dec = NULL; InputStream *ist; ret = init_output_stream_encode(ost, frame); if (ret < 0) return ret; - ist = get_input_stream(ost); + if ((ist = get_input_stream(ost))) + dec = ist->dec_ctx; + if (dec && dec->subtitle_header) { + /* ASS code assumes this buffer is null terminated so add extra byte. */ + ost->enc_ctx->subtitle_header = av_mallocz(dec->subtitle_header_size + 1); + if (!ost->enc_ctx->subtitle_header) + return AVERROR(ENOMEM); + memcpy(ost->enc_ctx->subtitle_header, dec->subtitle_header, dec->subtitle_header_size); + ost->enc_ctx->subtitle_header_size = dec->subtitle_header_size; + } if (!av_dict_get(ost->encoder_opts, "threads", NULL, 0)) av_dict_set(&ost->encoder_opts, "threads", "auto", 0); @@ -3082,6 +3130,18 @@ static int init_output_stream(OutputStream *ost, AVFrame *frame, return ret; } + if (ist && ist->dec->type == AVMEDIA_TYPE_SUBTITLE && ost->enc_ctx->codec_type == AVMEDIA_TYPE_SUBTITLE) { + AVCodecDescriptor const *output_descriptor = avcodec_descriptor_get(ost->enc_ctx->codec_id); + const enum AVSubtitleType in_subtitle_format = (uint16_t)dec->subtitle_type; + const enum AVSubtitleType out_subtitle_format = output_descriptor ? get_subtitle_format(output_descriptor) : AV_SUBTITLE_FMT_UNKNOWN; + + if (ist->nb_filters == 0 && in_subtitle_format != AV_SUBTITLE_FMT_UNKNOWN && out_subtitle_format != AV_SUBTITLE_FMT_UNKNOWN + && in_subtitle_format != out_subtitle_format) { + snprintf(error, error_len, "Subtitle encoding is only possible from text to text or bitmap to bitmap"); + return AVERROR_INVALIDDATA; + } + } + if ((ret = avcodec_open2(ost->enc_ctx, codec, &ost->encoder_opts)) < 0) { if (ret == AVERROR_EXPERIMENTAL) abort_codec_experimental(codec, 1); @@ -3230,10 +3290,11 @@ static int transcode_init(void) * to be configured with the correct audio frame size, which is only * known after the encoder is initialized. */ - for (ost = ost_iter(NULL); ost; ost = ost_iter(ost)) { + for (OutputStream *ost = ost_iter(NULL); ost; ost = ost_iter(ost)) { if (ost->enc_ctx && - (ost->st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO || - ost->st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO)) + (ost->enc_ctx->codec_type == AVMEDIA_TYPE_VIDEO || + ost->enc_ctx->codec_type == AVMEDIA_TYPE_AUDIO || + ost->enc_ctx->codec_type == AVMEDIA_TYPE_SUBTITLE)) continue; ret = init_output_stream_wrapper(ost, NULL, 0); @@ -3742,7 +3803,7 @@ static int process_input(int file_index) av_ts2timestr(input_files[ist->file_index]->ts_offset, &AV_TIME_BASE_Q)); } - sub2video_heartbeat(ist, pkt->pts); + subtitle_kickoff(ist, pkt->pts); process_input_packet(ist, pkt, 0); @@ -3943,6 +4004,7 @@ static int transcode(void) /* at the end of stream, we must flush the decoder buffers */ for (i = 0; i < nb_input_streams; i++) { ist = input_streams[i]; + ist->subtitle_kickoff.is_active = 0; if (!input_files[ist->file_index]->eof_reached) { process_input_packet(ist, NULL, 0); } diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index 5966cac60e..06858de046 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -386,17 +386,16 @@ typedef struct InputStream { struct { /* previous decoded subtitle and related variables */ int got_output; int ret; - AVSubtitle subtitle; + AVFrame *subtitle; } prev_sub; - struct sub2video { + struct subtitle_kickoff { + int is_active; int64_t last_pts; - int64_t end_pts; - AVFifo *sub_queue; ///< queue of AVSubtitle* before filter init - AVFrame *frame; int w, h; - unsigned int initialize; ///< marks if sub2video_update should force an initialization - } sub2video; + } subtitle_kickoff; + + AVBufferRef *subtitle_header; /* decoded data from this stream goes into all those filters * currently video and audio only */ @@ -513,6 +512,8 @@ typedef struct OutputStream { // everything before it should be discarded int64_t ts_copy_start; + /* subtitle_pts values of the last subtitle frame having arrived for encoding */ + int64_t last_subtitle_pts; // the timebase of the packets sent to the muxer AVRational mux_timebase; AVRational enc_timebase; @@ -713,8 +714,6 @@ int filtergraph_is_simple(FilterGraph *fg); int init_simple_filtergraph(InputStream *ist, OutputStream *ost); int init_complex_filtergraph(FilterGraph *fg); -void sub2video_update(InputStream *ist, int64_t heartbeat_pts, AVSubtitle *sub); - int ifilter_parameters_from_frame(InputFilter *ifilter, const AVFrame *frame); int ffmpeg_parse_options(int argc, char **argv); diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c index 718826a485..57a6c8ee16 100644 --- a/fftools/ffmpeg_filter.c +++ b/fftools/ffmpeg_filter.c @@ -22,6 +22,8 @@ #include "ffmpeg.h" +#include "libavutil/ass_split_internal.h" + #include "libavfilter/avfilter.h" #include "libavfilter/buffersink.h" #include "libavfilter/buffersrc.h" @@ -30,11 +32,9 @@ #include "libavutil/avstring.h" #include "libavutil/bprint.h" #include "libavutil/channel_layout.h" -#include "libavutil/display.h" #include "libavutil/opt.h" #include "libavutil/pixdesc.h" #include "libavutil/pixfmt.h" -#include "libavutil/imgutils.h" #include "libavutil/samplefmt.h" // FIXME: YUV420P etc. are actually supported with full color range, @@ -235,9 +235,8 @@ static void init_input_filter(FilterGraph *fg, AVFilterInOut *in) InputFilter *ifilter; int i; - // TODO: support other filter types - if (type != AVMEDIA_TYPE_VIDEO && type != AVMEDIA_TYPE_AUDIO) { - av_log(NULL, AV_LOG_FATAL, "Only video and audio filters supported " + if (type != AVMEDIA_TYPE_VIDEO && type != AVMEDIA_TYPE_AUDIO && type != AVMEDIA_TYPE_SUBTITLE) { + av_log(NULL, AV_LOG_FATAL, "Only video, audio and subtitle filters supported " "currently.\n"); exit_program(1); } @@ -258,8 +257,9 @@ static void init_input_filter(FilterGraph *fg, AVFilterInOut *in) for (i = 0; i < s->nb_streams; i++) { enum AVMediaType stream_type = s->streams[i]->codecpar->codec_type; if (stream_type != type && - !(stream_type == AVMEDIA_TYPE_SUBTITLE && - type == AVMEDIA_TYPE_VIDEO /* sub2video hack */)) + // in the followng case we auto-insert the graphicsub2video conversion filter + // for retaining compatibility with the previous sub2video hack + !(stream_type == AVMEDIA_TYPE_SUBTITLE && type == AVMEDIA_TYPE_VIDEO)) continue; if (check_stream_specifier(s, s->streams[i], *p == ':' ? p + 1 : p) == 1) { st = s->streams[i]; @@ -364,6 +364,21 @@ static int insert_trim(int64_t start_time, int64_t duration, const char *name = (type == AVMEDIA_TYPE_VIDEO) ? "trim" : "atrim"; int ret = 0; + switch (type) { + case AVMEDIA_TYPE_VIDEO: + name = "trim"; + break; + case AVMEDIA_TYPE_AUDIO: + name = "atrim"; + break; + case AVMEDIA_TYPE_SUBTITLE: + name = "strim"; + break; + default: + av_log(NULL, AV_LOG_ERROR, "insert_trim: Invalid media type: %d\n", type); + return AVERROR_INVALIDDATA; + } + if (duration == INT64_MAX && start_time == AV_NOPTS_VALUE) return 0; @@ -426,6 +441,40 @@ static int insert_filter(AVFilterContext **last_filter, int *pad_idx, return 0; } +static int configure_output_subtitle_filter(FilterGraph *fg, OutputFilter *ofilter, AVFilterInOut *out) +{ + OutputStream *ost = ofilter->ost; + OutputFile *of = output_files[ost->file_index]; + AVFilterContext *last_filter = out->filter_ctx; + int pad_idx = out->pad_idx; + int ret; + char name[255]; + + snprintf(name, sizeof(name), "out_%d_%d", ost->file_index, ost->index); + ret = avfilter_graph_create_filter(&ofilter->filter, + avfilter_get_by_name("sbuffersink"), + name, NULL, NULL, fg->graph); + + if (ret < 0) { + av_log(NULL, AV_LOG_ERROR, "Unable to create filter sbuffersink\n"); + return ret; + } + + snprintf(name, sizeof(name), "trim_out_%d_%d", + ost->file_index, ost->index); + ret = insert_trim(of->start_time, of->recording_time, + &last_filter, &pad_idx, name); + if (ret < 0) + return ret; + + ////ost->st->codecpar->codec_tag = MKTAG('a', 's', 's', 's'); + + if ((ret = avfilter_link(last_filter, pad_idx, ofilter->filter, 0)) < 0) + return ret; + + return 0; +} + static int configure_output_video_filter(FilterGraph *fg, OutputFilter *ofilter, AVFilterInOut *out) { OutputStream *ost = ofilter->ost; @@ -609,7 +658,8 @@ static int configure_output_audio_filter(FilterGraph *fg, OutputFilter *ofilter, int i; for (i = 0; i < of->nb_streams; i++) - if (of->streams[i]->st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO) + if (of->streams[i]->st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO || + of->streams[i]->st->codecpar->codec_type == AVMEDIA_TYPE_SUBTITLE) break; if (i < of->nb_streams) { @@ -643,6 +693,7 @@ static int configure_output_filter(FilterGraph *fg, OutputFilter *ofilter, switch (avfilter_pad_get_type(out->filter_ctx->output_pads, out->pad_idx)) { case AVMEDIA_TYPE_VIDEO: return configure_output_video_filter(fg, ofilter, out); case AVMEDIA_TYPE_AUDIO: return configure_output_audio_filter(fg, ofilter, out); + case AVMEDIA_TYPE_SUBTITLE: return configure_output_subtitle_filter(fg, ofilter, out); default: av_assert0(0); return 0; } } @@ -662,51 +713,154 @@ void check_filter_outputs(void) } } -static int sub2video_prepare(InputStream *ist, InputFilter *ifilter) +static int configure_input_subtitle_filter(FilterGraph *fg, InputFilter *ifilter, + AVFilterInOut *in) { - AVFormatContext *avf = input_files[ist->file_index]->ctx; - int i, w, h; + AVFilterContext *last_filter; + const AVFilter *buffer_filt = avfilter_get_by_name("sbuffer"); + InputStream *ist = ifilter->ist; + InputFile *f = input_files[ist->file_index]; + AVBPrint args; + char name[255]; + int ret, pad_idx = 0; + int w, h; + int64_t tsoffset = 0; + AVBufferSrcParameters *par = av_buffersrc_parameters_alloc(); + enum AVMediaType media_type; + + if (!par) + return AVERROR(ENOMEM); + + par->format = AV_PIX_FMT_NONE; + + if (ist->dec_ctx->codec_type == AVMEDIA_TYPE_AUDIO) { + av_log(NULL, AV_LOG_ERROR, "Cannot connect subtitle filter to audio input\n"); + ret = AVERROR(EINVAL); + goto fail; + } + + if (ist->dec_ctx->codec_type == AVMEDIA_TYPE_VIDEO) { + av_log(NULL, AV_LOG_ERROR, "Cannot connect subtitle filter to video input\n"); + ret = AVERROR(EINVAL); + goto fail; + } + + if (ist->st->codecpar->codec_type == AVMEDIA_TYPE_SUBTITLE && ist->dec_ctx) { + // For subtitles, we need to set the format here. Would we leave the format + // at -1, it would delay processing (ifilter_has_all_input_formats()) until + // the first subtile frame arrives, which could never happen in the worst case + ifilter->format = (uint16_t)ist->dec_ctx->subtitle_type; + } + + ist->subtitle_kickoff.is_active = 1; - /* Compute the size of the canvas for the subtitles stream. - If the subtitles codecpar has set a size, use it. Otherwise use the - maximum dimensions of the video streams in the same file. */ w = ifilter->width; h = ifilter->height; + if (!(w && h)) { - for (i = 0; i < avf->nb_streams; i++) { - if (avf->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_VIDEO) { - w = FFMAX(w, avf->streams[i]->codecpar->width); - h = FFMAX(h, avf->streams[i]->codecpar->height); + w = ist->dec_ctx->width; + h = ist->dec_ctx->height; + } + + if (!(w && h) && ist->dec_ctx->subtitle_header) { + ASSSplitContext *ass_ctx = avpriv_ass_split((char *)ist->dec_ctx->subtitle_header); + ASS *ass = (ASS *)ass_ctx; + w = ass->script_info.play_res_x; + h = ass->script_info.play_res_y; + avpriv_ass_split_free(ass_ctx); + } + + ist->subtitle_kickoff.w = w; + ist->subtitle_kickoff.h = h; + av_log(ifilter, AV_LOG_INFO, "subtitle input filter: decoding size %dx%d\n", ist->subtitle_kickoff.w, ist->subtitle_kickoff.h); + + ifilter->width = w; + ifilter->height = h; + ist->dec_ctx->width = w; + ist->dec_ctx->height = h; + + ist->subtitle_kickoff.last_pts = INT64_MIN; + + snprintf(name, sizeof(name), "graph %d subtitle input from stream %d:%d", fg->index, + ist->file_index, ist->st->index); + + + av_bprint_init(&args, 0, AV_BPRINT_SIZE_AUTOMATIC); + av_bprintf(&args, + "subtitle_type=%d:width=%d:height=%d:time_base=%d/%d:", + ifilter->format, ifilter->width, ifilter->height, + ist->st->time_base.num, ist->st->time_base.den); + if ((ret = avfilter_graph_create_filter(&ifilter->filter, buffer_filt, name, + args.str, NULL, fg->graph)) < 0) + goto fail; + + par->hw_frames_ctx = ifilter->hw_frames_ctx; + par->format = ifilter->format; + par->width = ifilter->width; + par->height = ifilter->height; + + ret = av_buffersrc_parameters_set(ifilter->filter, par); + if (ret < 0) + goto fail; + av_freep(&par); + last_filter = ifilter->filter; + + // This is for sub2video compatibility + media_type = avfilter_pad_get_type(in->filter_ctx->input_pads, in->pad_idx); + if (media_type == AVMEDIA_TYPE_VIDEO) { + int subscale_w = w, subscale_h = h; + + av_log(NULL, AV_LOG_INFO, "Auto-inserting subfeed filter\n"); + ret = insert_filter(&last_filter, &pad_idx, "subfeed", NULL); + if (ret < 0) + return ret; + + if (!(subscale_w && subscale_h)) { + // If the subtitle frame size is unknown, try to find a video input + // and use its size for adding a subscale filter + for (int i = 0; i < fg->nb_inputs; i++) { + InputFilter *input = fg->inputs[i]; + if (input->type == AVMEDIA_TYPE_VIDEO && input->width && input->height) { + subscale_w = input->width; + subscale_h = input->height; + break; + } } } - if (!(w && h)) { - w = FFMAX(w, 720); - h = FFMAX(h, 576); - } - av_log(avf, AV_LOG_INFO, "sub2video: using %dx%d canvas\n", w, h); - } - ist->sub2video.w = ifilter->width = w; - ist->sub2video.h = ifilter->height = h; - ifilter->width = ist->dec_ctx->width ? ist->dec_ctx->width : ist->sub2video.w; - ifilter->height = ist->dec_ctx->height ? ist->dec_ctx->height : ist->sub2video.h; + if (subscale_w && subscale_h) { + char subscale_params[64]; + snprintf(subscale_params, sizeof(subscale_params), "w=%d:h=%d", subscale_w, subscale_h); + ret = insert_filter(&last_filter, &pad_idx, "subscale", subscale_params); + if (ret < 0) + return ret; + } - /* rectangles are AV_PIX_FMT_PAL8, but we have no guarantee that the - palettes for all rectangles are identical or compatible */ - ifilter->format = AV_PIX_FMT_RGB32; + av_log(NULL, AV_LOG_INFO, "Auto-inserting graphicsub2video filter\n"); + ret = insert_filter(&last_filter, &pad_idx, "graphicsub2video", NULL); + if (ret < 0) + return ret; + } - ist->sub2video.frame = av_frame_alloc(); - if (!ist->sub2video.frame) - return AVERROR(ENOMEM); - ist->sub2video.last_pts = INT64_MIN; - ist->sub2video.end_pts = INT64_MIN; + if (copy_ts) { + tsoffset = f->start_time == AV_NOPTS_VALUE ? 0 : f->start_time; + if (!start_at_zero && f->ctx->start_time != AV_NOPTS_VALUE) + tsoffset += f->ctx->start_time; + } + ret = insert_trim(((f->start_time == AV_NOPTS_VALUE) || !f->accurate_seek) ? + AV_NOPTS_VALUE : tsoffset, f->recording_time, + &last_filter, &pad_idx, name); + if (ret < 0) + return ret; - /* sub2video structure has been (re-)initialized. - Mark it as such so that the system will be - initialized with the first received heartbeat. */ - ist->sub2video.initialize = 1; + if ((ret = avfilter_link(last_filter, 0, in->filter_ctx, in->pad_idx)) < 0) + return ret; return 0; +fail: + av_freep(&par); + + return ret; } static int configure_input_video_filter(FilterGraph *fg, InputFilter *ifilter, @@ -725,8 +879,15 @@ static int configure_input_video_filter(FilterGraph *fg, InputFilter *ifilter, char name[255]; int ret, pad_idx = 0; int64_t tsoffset = 0; - AVBufferSrcParameters *par = av_buffersrc_parameters_alloc(); + AVBufferSrcParameters *par; + + if (ist->dec_ctx->codec_type == AVMEDIA_TYPE_SUBTITLE) { + // Automatically insert conversion filter to retain compatibility + // with sub2video command lines + return configure_input_subtitle_filter(fg, ifilter, in); + } + par = av_buffersrc_parameters_alloc(); if (!par) return AVERROR(ENOMEM); memset(par, 0, sizeof(*par)); @@ -741,12 +902,6 @@ static int configure_input_video_filter(FilterGraph *fg, InputFilter *ifilter, if (!fr.num) fr = ist->framerate_guessed; - if (ist->dec_ctx->codec_type == AVMEDIA_TYPE_SUBTITLE) { - ret = sub2video_prepare(ist, ifilter); - if (ret < 0) - goto fail; - } - sar = ifilter->sample_aspect_ratio; if(!sar.den) sar = (AVRational){0,1}; @@ -758,7 +913,7 @@ static int configure_input_video_filter(FilterGraph *fg, InputFilter *ifilter, tb.num, tb.den, sar.num, sar.den); if (fr.num && fr.den) av_bprintf(&args, ":frame_rate=%d/%d", fr.num, fr.den); - snprintf(name, sizeof(name), "graph %d input from stream %d:%d", fg->index, + snprintf(name, sizeof(name), "graph %d video input from stream %d:%d", fg->index, ist->file_index, ist->st->index); @@ -923,6 +1078,7 @@ static int configure_input_filter(FilterGraph *fg, InputFilter *ifilter, switch (avfilter_pad_get_type(in->filter_ctx->input_pads, in->pad_idx)) { case AVMEDIA_TYPE_VIDEO: return configure_input_video_filter(fg, ifilter, in); case AVMEDIA_TYPE_AUDIO: return configure_input_audio_filter(fg, ifilter, in); + case AVMEDIA_TYPE_SUBTITLE: return configure_input_subtitle_filter(fg, ifilter, in); default: av_assert0(0); return 0; } } @@ -941,7 +1097,8 @@ static int filter_is_buffersrc(const AVFilterContext *f) { return f->nb_inputs == 0 && (!strcmp(f->filter->name, "buffer") || - !strcmp(f->filter->name, "abuffer")); + !strcmp(f->filter->name, "abuffer") || + !strcmp(f->filter->name, "sbuffer")); } static int graph_is_meta(AVFilterGraph *graph) @@ -1110,18 +1267,6 @@ int configure_filtergraph(FilterGraph *fg) } } - /* process queued up subtitle packets */ - for (i = 0; i < fg->nb_inputs; i++) { - InputStream *ist = fg->inputs[i]->ist; - if (ist->sub2video.sub_queue && ist->sub2video.frame) { - AVSubtitle tmp; - while (av_fifo_read(ist->sub2video.sub_queue, &tmp, 1) >= 0) { - sub2video_update(ist, INT64_MIN, &tmp); - avsubtitle_free(&tmp); - } - } - } - return 0; fail: @@ -1141,6 +1286,7 @@ int ifilter_parameters_from_frame(InputFilter *ifilter, const AVFrame *frame) ifilter->width = frame->width; ifilter->height = frame->height; ifilter->sample_aspect_ratio = frame->sample_aspect_ratio; + ifilter->type = frame->type; ifilter->sample_rate = frame->sample_rate; ret = av_channel_layout_copy(&ifilter->ch_layout, &frame->ch_layout); diff --git a/fftools/ffmpeg_hw.c b/fftools/ffmpeg_hw.c index 88fa782470..a9e241b540 100644 --- a/fftools/ffmpeg_hw.c +++ b/fftools/ffmpeg_hw.c @@ -449,7 +449,7 @@ int hw_device_setup_for_encode(OutputStream *ost) AVBufferRef *frames_ref = NULL; int i; - if (ost->filter) { + if (ost->filter && ost->filter->filter) { frames_ref = av_buffersink_get_hw_frames_ctx(ost->filter->filter); if (frames_ref && ((AVHWFramesContext*)frames_ref->data)->format == diff --git a/fftools/ffmpeg_mux_init.c b/fftools/ffmpeg_mux_init.c index c77260c24b..b79f540b21 100644 --- a/fftools/ffmpeg_mux_init.c +++ b/fftools/ffmpeg_mux_init.c @@ -381,11 +381,15 @@ static char *get_ost_filters(OptionsContext *o, AVFormatContext *oc, if (ost->filters_script) return file_read(ost->filters_script); - else if (ost->filters) + if (ost->filters) return av_strdup(ost->filters); - return av_strdup(st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO ? - "null" : "anull"); + switch (st->codecpar->codec_type) { + case AVMEDIA_TYPE_VIDEO: return av_strdup("null"); + case AVMEDIA_TYPE_AUDIO: return av_strdup("anull"); + case AVMEDIA_TYPE_SUBTITLE: return av_strdup("snull"); + default: av_assert0(0); return NULL; + } } static void check_streamcopy_filters(OptionsContext *o, AVFormatContext *oc, @@ -797,15 +801,23 @@ static OutputStream *new_subtitle_stream(Muxer *mux, OptionsContext *o, int sour ost = new_output_stream(mux, o, AVMEDIA_TYPE_SUBTITLE, source_index); st = ost->st; + MATCH_PER_STREAM_OPT(filter_scripts, str, ost->filters_script, mux->fc, st) + MATCH_PER_STREAM_OPT(filters, str, ost->filters, mux->fc, st) + if (ost->enc_ctx) { AVCodecContext *subtitle_enc = ost->enc_ctx; char *frame_size = NULL; + ost->enc_ctx->codec_type = AVMEDIA_TYPE_SUBTITLE; MATCH_PER_STREAM_OPT(frame_sizes, str, frame_size, mux->fc, st); if (frame_size && av_parse_video_size(&subtitle_enc->width, &subtitle_enc->height, frame_size) < 0) { av_log(NULL, AV_LOG_FATAL, "Invalid frame size: %s.\n", frame_size); exit_program(1); } + + ost->avfilter = get_ost_filters(o, mux->fc, ost); + if (!ost->avfilter) + exit_program(1); } return ost; @@ -819,8 +831,9 @@ static void init_output_filter(OutputFilter *ofilter, OptionsContext *o, switch (ofilter->type) { case AVMEDIA_TYPE_VIDEO: ost = new_video_stream(mux, o, -1); break; case AVMEDIA_TYPE_AUDIO: ost = new_audio_stream(mux, o, -1); break; + case AVMEDIA_TYPE_SUBTITLE: ost = new_subtitle_stream(mux, o, -1); break; default: - av_log(NULL, AV_LOG_FATAL, "Only video and audio filters are supported " + av_log(NULL, AV_LOG_FATAL, "Only video, audio and subtitle filters are supported " "currently.\n"); exit_program(1); } @@ -1765,7 +1778,8 @@ int of_open(OptionsContext *o, const char *filename) ist->processing_needed = 1; if (ost->st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO || - ost->st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO) { + ost->st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO || + ost->st->codecpar->codec_type == AVMEDIA_TYPE_SUBTITLE) { err = init_simple_filtergraph(ist, ost); if (err < 0) { av_log(NULL, AV_LOG_ERROR, @@ -1811,6 +1825,10 @@ int of_open(OptionsContext *o, const char *filename) } else if (c->ch_layouts) { f->ch_layouts = c->ch_layouts; } + break; + case AVMEDIA_TYPE_SUBTITLE: + f->format = ost->enc_ctx->subtitle_type; + break; } } diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c index 9245e02813..2402a591f6 100644 --- a/fftools/ffmpeg_opt.c +++ b/fftools/ffmpeg_opt.c @@ -915,6 +915,7 @@ static void add_input_streams(OptionsContext *o, AVFormatContext *ic) av_log(NULL, AV_LOG_FATAL, "Invalid canvas size: %s.\n", canvas_size); exit_program(1); } + ist->subtitle_kickoff.is_active = 1; break; } case AVMEDIA_TYPE_ATTACHMENT: diff --git a/tests/ref/fate/filter-overlay-dvdsub-2397 b/tests/ref/fate/filter-overlay-dvdsub-2397 index 7df4f50776..0c0f35c634 100644 --- a/tests/ref/fate/filter-overlay-dvdsub-2397 +++ b/tests/ref/fate/filter-overlay-dvdsub-2397 @@ -489,368 +489,367 @@ 1, 3877, 3877, 10, 2013, 0x95a39f9c 1, 3887, 3887, 10, 2013, 0x4f7ea123 1, 3897, 3897, 10, 2013, 0x9efb9ba1 -0, 117, 117, 1, 518400, 0xbf8523da +0, 117, 117, 1, 518400, 0xc44e1d4c 1, 3907, 3907, 10, 2013, 0xf395b2cd 1, 3917, 3917, 10, 2013, 0x261a881e 1, 3927, 3927, 10, 2013, 0x7f2d9f72 1, 3937, 3937, 10, 2013, 0x0105b38d -0, 118, 118, 1, 518400, 0x41890ed6 +0, 118, 118, 1, 518400, 0xad1a084c 1, 3952, 3952, 10, 2013, 0x0e5db67e 1, 3962, 3962, 10, 2013, 0xfc9baf97 -0, 119, 119, 1, 518400, 0x588534fc +0, 119, 119, 1, 518400, 0x52d52e73 1, 3972, 3972, 10, 2013, 0x8e02a1b1 1, 3982, 3982, 10, 2013, 0x6eecaac8 1, 3992, 3992, 10, 2013, 0xf5558f0c 1, 4002, 4002, 10, 2013, 0x512ba99b -0, 120, 120, 1, 518400, 0x2145ebc1 +0, 120, 120, 1, 518400, 0xc732e546 1, 4012, 4012, 10, 2013, 0x932b9932 1, 4022, 4022, 10, 2013, 0xc01ea987 -0, 121, 121, 1, 518400, 0x28bca595 +0, 121, 121, 1, 518400, 0xc36f9f14 1, 4038, 4038, 10, 2013, 0x10879cf7 1, 4048, 4048, 10, 2013, 0x90679338 1, 4058, 4058, 10, 2013, 0x077d8a9e 1, 4068, 4068, 10, 2013, 0x969fa57c -0, 122, 122, 1, 518400, 0x77dc951e +0, 122, 122, 1, 518400, 0x78428e8b 1, 4078, 4078, 10, 2013, 0xe049ab07 1, 4088, 4088, 10, 2013, 0xf535b3b3 1, 4098, 4098, 10, 2013, 0xfe76bd37 -0, 123, 123, 1, 518400, 0xe8924c17 +0, 123, 123, 1, 518400, 0xf0f8458d 1, 4108, 4108, 10, 2013, 0xde79ad8c 1, 4123, 4123, 10, 2013, 0xe89b9c47 1, 4133, 4133, 10, 2013, 0xc570b0f0 -0, 124, 124, 1, 518400, 0xadb4cccc +0, 124, 124, 1, 518400, 0x7083c653 1, 4143, 4143, 10, 2013, 0xee709cd9 1, 4153, 4153, 10, 2013, 0xcfe5afab 1, 4163, 4163, 10, 2013, 0x98ff8ce4 -0, 125, 125, 1, 518400, 0x1d7b56ac +0, 125, 125, 1, 518400, 0xa105502c 1, 4173, 4173, 10, 2013, 0x9d19b44c 1, 4183, 4183, 10, 2013, 0x4349917a 1, 4193, 4193, 10, 2013, 0xbf54a59a -0, 126, 126, 1, 518400, 0xad5739a4 +0, 126, 126, 1, 518400, 0xd411331a 1, 4208, 4208, 10, 2013, 0xc4a399e0 1, 4218, 4218, 10, 2013, 0x1bf58ff0 1, 4228, 4228, 10, 2013, 0x3518ac56 -0, 127, 127, 1, 518400, 0x2733d35a +0, 127, 127, 1, 518400, 0x83b0ccdb 1, 4238, 4238, 10, 2013, 0xcd38c1de 1, 4248, 4248, 10, 2013, 0xbe7d9c4d 1, 4258, 4258, 10, 2013, 0xe113a306 1, 4268, 4268, 10, 2013, 0x083197ea -0, 128, 128, 1, 518400, 0x78e76da2 +0, 128, 128, 1, 518400, 0xa9be671a 1, 4278, 4278, 10, 2013, 0x1929b1eb 1, 4294, 4294, 10, 2013, 0x5d6ea5af 1, 4304, 4304, 10, 2013, 0x05519d53 -0, 129, 129, 1, 518400, 0x6c076013 +0, 129, 129, 1, 518400, 0xaeb75983 1, 4314, 4314, 10, 2013, 0x5773b380 1, 4324, 4324, 10, 2013, 0xaa70a8f5 1, 4334, 4334, 10, 2013, 0x990db0ec -0, 130, 130, 1, 518400, 0x7854f2b1 +0, 130, 130, 1, 518400, 0x81f8ec13 1, 4344, 4344, 10, 2013, 0x91d3a623 1, 4354, 4354, 10, 2013, 0xc91f9824 1, 4364, 4364, 10, 2013, 0x1d058abf -0, 131, 131, 1, 518400, 0xd2ae1ecd +0, 131, 131, 1, 518400, 0x8aaa1839 1, 4379, 4379, 10, 2013, 0x8de1b8d5 1, 4389, 4389, 10, 2013, 0x7872b06b 1, 4399, 4399, 10, 2013, 0xa084c203 -0, 132, 132, 1, 518400, 0xf5eab38d +0, 132, 132, 1, 518400, 0xc98bacf5 1, 4409, 4409, 10, 2013, 0xff90ae8d 1, 4419, 4419, 10, 2013, 0x61dead8e 1, 4429, 4429, 10, 2013, 0xee76b284 -0, 133, 133, 1, 518400, 0x994d3e9c +0, 133, 133, 1, 518400, 0x31083804 1, 4439, 4439, 10, 2013, 0xe888af7f 1, 4449, 4449, 10, 2013, 0x5d57b115 1, 4464, 4464, 10, 2013, 0xcdbfb1d0 -0, 134, 134, 1, 518400, 0x95ab705a +0, 134, 134, 1, 518400, 0x540a69dc 1, 4474, 4474, 10, 2013, 0x2e28a952 1, 4484, 4484, 10, 2013, 0x4795a994 1, 4494, 4494, 10, 2013, 0x7e7ea304 1, 4504, 4504, 10, 2013, 0x9502c1e1 -0, 135, 135, 1, 518400, 0x3c83c5ce +0, 135, 135, 1, 518400, 0x80d3bf46 1, 4514, 4514, 10, 2013, 0xf7c78ab2 1, 4524, 4524, 10, 2013, 0x24049816 1, 4534, 4534, 10, 2013, 0x52089dcf -0, 136, 136, 1, 518400, 0xfa22c508 +0, 136, 136, 1, 518400, 0x2967be7f 1, 4550, 4550, 10, 2013, 0x2150a0b1 1, 4560, 4560, 10, 2013, 0x3c2e9b93 1, 4570, 4570, 10, 2013, 0x491f932b -0, 137, 137, 1, 518400, 0xddda1712 +0, 137, 137, 1, 518400, 0x5a3b1092 1, 4580, 4580, 10, 2013, 0x31359cf8 1, 4590, 4590, 10, 2013, 0x1b00ac3f 1, 4600, 4600, 10, 2013, 0x8d7ab3cb -0, 138, 138, 1, 518400, 0x985a3b93 +0, 138, 138, 1, 518400, 0x8741350b 1, 4610, 4610, 10, 2013, 0xb2c2a4de 1, 4620, 4620, 10, 2013, 0x80a4abf2 1, 4635, 4635, 10, 2013, 0x0701a4ee -0, 139, 139, 1, 518400, 0xea63c5e7 +0, 139, 139, 1, 518400, 0xd5a9bf60 1, 4645, 4645, 10, 2013, 0xdc1ba5bc 1, 4655, 4655, 10, 2013, 0x6083a8a4 1, 4665, 4665, 10, 2013, 0x6226ad45 -0, 140, 140, 1, 518400, 0xef64983d +0, 140, 140, 1, 518400, 0xc05f91ba 1, 4675, 4675, 10, 2013, 0x2732a205 1, 4685, 4685, 10, 2013, 0x0f62a0d3 1, 4695, 4695, 10, 2013, 0xc1799249 -0, 141, 141, 1, 518400, 0x747bb193 +0, 141, 141, 1, 518400, 0x3fdaab0b 1, 4705, 4705, 10, 2013, 0xbccfa9c8 1, 4720, 4720, 10, 2013, 0xded096e7 1, 4730, 4730, 10, 2013, 0x7f0daf43 -0, 142, 142, 1, 518400, 0xb8748862 +0, 142, 142, 1, 518400, 0xab7281d9 1, 4740, 4740, 10, 2013, 0xc47ea682 1, 4750, 4750, 10, 2013, 0x5a72b07a 1, 4760, 4760, 10, 2013, 0x386faa8c 1, 4770, 4770, 10, 2013, 0xf9919a91 -0, 143, 143, 1, 518400, 0xaab55a5f +0, 143, 143, 1, 518400, 0xc80053d6 1, 4780, 4780, 10, 2013, 0x4908897e 1, 4790, 4790, 10, 2013, 0x4882b594 -0, 144, 144, 1, 518400, 0x7b468add +0, 144, 144, 1, 518400, 0x6526845c 1, 4806, 4806, 10, 2013, 0x113e98d1 1, 4816, 4816, 10, 2013, 0x5098b30d 1, 4826, 4826, 10, 2013, 0x0ef7b857 1, 4836, 4836, 10, 2013, 0x216ea176 -0, 145, 145, 1, 518400, 0xf2078707 +0, 145, 145, 1, 518400, 0x1b788089 1, 4846, 4846, 10, 2013, 0xf906944a 1, 4856, 4856, 10, 2013, 0xee9b92fb 1, 4866, 4866, 10, 2013, 0xd6029209 -0, 146, 146, 1, 518400, 0x6a2d931e +0, 146, 146, 1, 518400, 0xfa8e8ca9 1, 4876, 4876, 10, 2013, 0x2256a12e 1, 4891, 4891, 10, 2013, 0x89de8e4a 1, 4901, 4901, 10, 2013, 0x0bf0a584 -0, 147, 147, 1, 518400, 0xbbe3c417 +0, 147, 147, 1, 518400, 0xb278bda1 1, 4911, 4911, 10, 2013, 0x6a5ebd58 1, 4921, 4921, 10, 2013, 0x3edd9aa4 1, 4931, 4931, 10, 2013, 0xbd66ac26 -0, 148, 148, 1, 518400, 0x6294e449 +0, 148, 148, 1, 518400, 0xb0c3ddca 1, 4941, 4941, 10, 2013, 0x313896ea 1, 4951, 4951, 10, 2013, 0x6b83a6a0 1, 4961, 4961, 10, 2013, 0x9aafb109 -0, 149, 149, 1, 518400, 0xa05721e7 +0, 149, 149, 1, 518400, 0x10351b53 1, 4976, 4976, 10, 2013, 0x5192a85a 1, 4986, 4986, 10, 2013, 0x1f919f79 1, 4996, 4996, 10, 2013, 0xc0799c40 -0, 150, 150, 1, 518400, 0x37749183 +0, 150, 150, 1, 518400, 0xc1408aee 1, 5006, 5006, 10, 2013, 0x2988bcd8 1, 5016, 5016, 10, 2013, 0x1482913a 1, 5026, 5026, 10, 2013, 0x74da9a94 1, 5036, 5036, 10, 2013, 0x763eb709 -0, 151, 151, 1, 518400, 0xf9d9dca0 +0, 151, 151, 1, 518400, 0xf016d615 1, 5046, 5046, 10, 2013, 0x1285b405 1, 5062, 5062, 10, 2013, 0xb6ab9dfc -0, 152, 152, 1, 518400, 0x5f8ccf08 +0, 152, 152, 1, 518400, 0xa768c892 1, 5072, 5072, 10, 2013, 0xe4c8bf19 1, 5082, 5082, 10, 2013, 0xabbbade8 1, 5092, 5092, 10, 2013, 0xf8b69d89 1, 5102, 5102, 10, 2013, 0xce04a866 -0, 153, 153, 1, 518400, 0x7303f77b +0, 153, 153, 1, 518400, 0x11c3f11e 1, 5112, 5112, 10, 2013, 0x07528abf 1, 5122, 5122, 10, 2013, 0x74fb98bf 1, 5132, 5132, 10, 2013, 0x579fb1c9 -0, 154, 154, 1, 518400, 0x22b0513f +0, 154, 154, 1, 518400, 0xcd9a4ac4 1, 5147, 5147, 10, 2013, 0x7ddea2ed 1, 5157, 5157, 10, 2013, 0x296caa2c 1, 5167, 5167, 10, 2013, 0x346d9c4f -0, 155, 155, 1, 518400, 0x330485d2 +0, 155, 155, 1, 518400, 0x4ade7f5e 1, 5177, 5177, 10, 2013, 0x3e1fba15 1, 5187, 5187, 10, 2013, 0x48a2908f 1, 5197, 5197, 10, 2013, 0xc1938d09 -0, 156, 156, 1, 518400, 0x7f83daea +0, 156, 156, 1, 518400, 0x655dd46b 1, 5207, 5207, 10, 2013, 0x0e96a060 1, 5217, 5217, 10, 2013, 0x7b6a9e06 1, 5232, 5232, 10, 2013, 0x5b779d28 -0, 157, 157, 1, 518400, 0xee19f2df +0, 157, 157, 1, 518400, 0x5ab5ec61 1, 5242, 5242, 10, 2013, 0xf600aca1 1, 5252, 5252, 10, 2013, 0x3a6c9e68 1, 5262, 5262, 10, 2013, 0x0c8dc1b0 -0, 158, 158, 1, 518400, 0xb71b1c77 +0, 158, 158, 1, 518400, 0x45dc15e6 1, 5272, 5272, 10, 2013, 0x26beb245 1, 5282, 5282, 10, 2013, 0x2bc09557 1, 5292, 5292, 10, 2013, 0x27fc8845 1, 5302, 5302, 10, 2013, 0x1025aa47 -0, 159, 159, 1, 518400, 0xbffc1856 +0, 159, 159, 1, 518400, 0x201911d3 1, 5318, 5318, 10, 2013, 0xc2e69baa 1, 5328, 5328, 10, 2013, 0xdb249b92 1, 5338, 5338, 10, 2013, 0x6ccda29e -0, 160, 160, 1, 518400, 0xabc125aa +0, 160, 160, 1, 518400, 0x0fbc1f46 1, 5348, 5348, 10, 2013, 0xeaf6a1cf 1, 5358, 5358, 10, 2013, 0x509ba397 1, 5368, 5368, 10, 2013, 0xfaf8a2df -0, 161, 161, 1, 518400, 0x5ee467f8 +0, 161, 161, 1, 518400, 0x7e316179 1, 5378, 5378, 10, 2013, 0x41388f28 1, 5388, 5388, 10, 2013, 0xfe5eab39 1, 5403, 5403, 10, 2013, 0xd5ffa066 -0, 162, 162, 1, 518400, 0x6c2cf168 +0, 162, 162, 1, 518400, 0x73bbeaed 1, 5413, 5413, 10, 2013, 0x6813a30a 1, 5423, 5423, 10, 2013, 0x9be89718 1, 5433, 5433, 10, 2013, 0xaec3a27b -0, 163, 163, 1, 518400, 0x63996b26 +0, 163, 163, 1, 518400, 0x3a7c648a 1, 5446, 5446, 10, 2013, 0x579a983e 1, 5456, 5456, 10, 2013, 0x98cea21f 1, 5466, 5466, 10, 2013, 0xca77a58a -0, 164, 164, 1, 518400, 0xb34d789a +0, 164, 164, 1, 518400, 0x9f707209 1, 5476, 5476, 10, 2013, 0xcbc3b1ee 1, 5486, 5486, 10, 2013, 0xf3bb8f07 1, 5496, 5496, 10, 2013, 0x6aeebd92 -0, 165, 165, 1, 518400, 0xf49c030f +0, 165, 165, 1, 518400, 0x9f25fc5c 1, 5506, 5506, 10, 2013, 0xe955a449 1, 5516, 5516, 10, 2013, 0x9436aa5b 1, 5531, 5531, 10, 2013, 0x4f0a8f9f -0, 166, 166, 1, 518400, 0x092dc41a +0, 166, 166, 1, 518400, 0x2ed8bd75 1, 5541, 5541, 10, 2013, 0x3551b22d 1, 5551, 5551, 10, 2013, 0x0959a3d4 1, 5561, 5561, 10, 2013, 0x2ed5a11b 1, 5571, 5571, 10, 2013, 0x8f52a5c3 -0, 167, 167, 1, 518400, 0x4134c577 +0, 167, 167, 1, 518400, 0xb493becb 1, 5581, 5581, 10, 2013, 0x6552978d 1, 5591, 5591, 10, 2013, 0x7dcca0c1 1, 5601, 5601, 10, 2013, 0xbcd4a3c9 -0, 168, 168, 1, 518400, 0x261de1ed +0, 168, 168, 1, 518400, 0x7df6db57 1, 5616, 5616, 10, 2013, 0xfe41a8d8 1, 5626, 5626, 10, 2013, 0xc85aae14 1, 5636, 5636, 10, 2013, 0x1185b346 -0, 169, 169, 1, 518400, 0xcbc8566a +0, 169, 169, 1, 518400, 0x1cb94fca 1, 5646, 5646, 10, 2013, 0xf7429a0d 1, 5656, 5656, 10, 2013, 0x48c2a160 1, 5666, 5666, 10, 2013, 0x9d85a85d -0, 170, 170, 1, 518400, 0x407a5c76 +0, 170, 170, 1, 518400, 0x70db55d8 1, 5676, 5676, 10, 2013, 0xbbe89fe9 1, 5686, 5686, 10, 2013, 0xea429fe2 1, 5702, 5702, 10, 2013, 0x221ca1d4 -0, 171, 171, 1, 518400, 0x1ed73bb2 +0, 171, 171, 1, 518400, 0xc1d9351b 1, 5712, 5712, 10, 2013, 0x394b925b 1, 5722, 5722, 10, 2013, 0x556dc26f 1, 5732, 5732, 10, 2013, 0xce21a5e1 -0, 172, 172, 1, 518400, 0x8467ddb5 +0, 172, 172, 1, 518400, 0xa4b0d717 1, 5742, 5742, 10, 2013, 0xbc87c0a8 1, 5752, 5752, 10, 2013, 0xbac4ac07 1, 5762, 5762, 10, 2013, 0xdeefa4aa 1, 5772, 5772, 10, 2013, 0x1f15b362 -0, 173, 173, 1, 518400, 0x0523dc73 +0, 173, 173, 1, 518400, 0x3730d5e9 1, 5787, 5787, 10, 2013, 0x6406b7b2 1, 5797, 5797, 10, 2013, 0x8030a03d -0, 174, 174, 1, 518400, 0x81f5e895 +0, 174, 174, 1, 518400, 0x9673e1ec 1, 5807, 5807, 10, 2013, 0x0373a5b1 1, 5817, 5817, 10, 2013, 0x34ef93da 1, 5827, 5827, 10, 2013, 0x94c198fe 1, 5837, 5837, 10, 2013, 0xfefcabad -0, 175, 175, 1, 518400, 0xfc74608d +0, 175, 175, 1, 518400, 0x877959d5 1, 5847, 5847, 10, 2013, 0x8755b3ec 1, 5857, 5857, 10, 2013, 0xe436a6fd 1, 5872, 5872, 10, 2013, 0x9cf5a11e -0, 176, 176, 1, 518400, 0xc4e0dae0 +0, 176, 176, 1, 518400, 0x04f3d421 1, 5882, 5882, 10, 2013, 0x03b8a98c 1, 5892, 5892, 10, 2013, 0x6216a138 1, 5902, 5902, 10, 2013, 0xd87b9f12 -0, 177, 177, 1, 518400, 0x98367f5b +0, 177, 177, 1, 518400, 0x4f3078bc 1, 5912, 5912, 10, 2013, 0x4ce99653 1, 5922, 5922, 10, 2013, 0x6c2ea9e2 1, 5932, 5932, 10, 2013, 0x918cae4c -0, 178, 178, 1, 518400, 0x0f1a869d +0, 178, 178, 1, 518400, 0x8a127ff8 1, 5942, 5942, 10, 2013, 0xd19fa5f2 1, 5958, 5958, 10, 2013, 0x0bdda7c6 1, 5968, 5968, 10, 2013, 0x0f9ab0ca -0, 179, 179, 1, 518400, 0x45b6ccf2 +0, 179, 179, 1, 518400, 0x5864c64f 1, 5978, 5978, 10, 2013, 0x410a92b1 1, 5988, 5988, 10, 2013, 0xcfbe9d1c 1, 5998, 5998, 10, 2013, 0x59ed9d15 -0, 180, 180, 1, 518400, 0x5f9ccb77 +0, 180, 180, 1, 518400, 0xdaccc4c0 1, 6008, 6008, 10, 2013, 0x4e129e27 1, 6018, 6018, 10, 2013, 0x7bb9ac0a 1, 6028, 6028, 10, 2013, 0x826ca82b -0, 181, 181, 1, 518400, 0x5f15ea31 +0, 181, 181, 1, 518400, 0xd999e376 1, 6043, 6043, 10, 2013, 0x9ad5a74b 1, 6053, 6053, 10, 2013, 0x6c5f969a 1, 6063, 6063, 10, 2013, 0x8479a0e5 -0, 182, 182, 1, 518400, 0x86369f27 +0, 182, 182, 1, 518400, 0x8af39876 1, 6073, 6073, 10, 2013, 0x165298ef 1, 6083, 6083, 10, 2013, 0xdcadb4a1 1, 6093, 6093, 10, 2013, 0xa90e987c 1, 6103, 6103, 10, 2013, 0x1ac5b510 -0, 183, 183, 1, 518400, 0x2e27f9fa +0, 183, 183, 1, 518400, 0x5e72f33d 1, 6113, 6113, 10, 2013, 0x66728d85 1, 6128, 6128, 10, 2013, 0xe4859fc5 1, 6138, 6138, 10, 2013, 0x9901786e -0, 184, 184, 1, 518400, 0xc029a44d +0, 184, 184, 1, 518400, 0x14af9d92 1, 6148, 6148, 10, 2013, 0x6aebb406 1, 6158, 6158, 10, 2013, 0x7d13a2cc 1, 6168, 6168, 10, 2013, 0x99b7a8cc -0, 185, 185, 1, 518400, 0xebee33b0 +0, 185, 185, 1, 518400, 0x50b82d10 1, 6178, 6178, 10, 2013, 0x80b8a624 1, 6188, 6188, 10, 2013, 0xbb6aa271 1, 6198, 6198, 10, 2013, 0x17af9e4a -0, 186, 186, 1, 518400, 0x19e5494f +0, 186, 186, 1, 518400, 0xc068429c 1, 6214, 6214, 10, 2013, 0xfaf0a8f1 1, 6224, 6224, 10, 2013, 0xd6849b93 1, 6234, 6234, 10, 2013, 0xe9829669 -0, 187, 187, 1, 518400, 0xf697bd7c +0, 187, 187, 1, 518400, 0x8934b6d1 1, 6244, 6244, 10, 2013, 0x7ec98944 1, 6254, 6254, 10, 2013, 0x2b2099a4 1, 6264, 6264, 10, 2013, 0x1033a82f -0, 188, 188, 1, 518400, 0x82569002 +0, 188, 188, 1, 518400, 0x11d08947 1, 6274, 6274, 10, 2013, 0x5ec88990 1, 6284, 6284, 10, 2013, 0xd2a19b3d 1, 6299, 6299, 10, 2013, 0xa377b268 -0, 189, 189, 1, 518400, 0xfcb6d707 +0, 189, 189, 1, 518400, 0x8a27d041 1, 6309, 6309, 10, 2013, 0xfa859901 1, 6319, 6319, 10, 2013, 0x1713955a 1, 6329, 6329, 10, 2013, 0x70aab0da 1, 6339, 6339, 10, 2013, 0xcdaea422 -0, 190, 190, 1, 518400, 0x82a9662b +0, 190, 190, 1, 518400, 0xab265f7d 1, 6349, 6349, 10, 2013, 0x65c3bf80 1, 6359, 6359, 10, 2013, 0x1d75a55f 1, 6369, 6369, 10, 2013, 0xa5bea4de -0, 191, 191, 1, 518400, 0x212e16ee +0, 191, 191, 1, 518400, 0xff491040 1, 6384, 6384, 10, 2013, 0x184db71c 1, 6394, 6394, 10, 2013, 0x99858ec8 1, 6404, 6404, 10, 2013, 0xb8f2aee5 -0, 192, 192, 1, 518400, 0x2ca34dca +0, 192, 192, 1, 518400, 0x822b4704 1, 6414, 6414, 10, 2013, 0x4435b2ef 1, 6424, 6424, 10, 2013, 0x8acfa6c7 1, 6434, 6434, 10, 2013, 0x42b4c01f -0, 193, 193, 1, 518400, 0xe9ebe0a5 +0, 193, 193, 1, 518400, 0x4523d9f4 1, 6444, 6444, 10, 2013, 0x6e308c13 1, 6454, 6454, 10, 2013, 0x8227a0f6 1, 6470, 6470, 10, 2013, 0x6f12a7a2 -0, 194, 194, 1, 518400, 0x4e6b6917 +0, 194, 194, 1, 518400, 0xfc3c626e 1, 6480, 6480, 10, 2013, 0x785392be 1, 6490, 6490, 10, 2013, 0x81849c2b 1, 6500, 6500, 10, 2013, 0x5cf2af65 -0, 195, 195, 1, 518400, 0x7dcf20ab +0, 195, 195, 1, 518400, 0x237319e5 1, 6510, 6510, 10, 2013, 0x0c6ca6b4 1, 6520, 6520, 10, 2013, 0x412fab9f 1, 6530, 6530, 10, 2013, 0x08e792b4 -0, 196, 196, 1, 518400, 0xf30fac97 +0, 196, 196, 1, 518400, 0x892ca5d8 1, 6540, 6540, 10, 2013, 0x407aace3 1, 6555, 6555, 10, 2013, 0xd26bac16 1, 6565, 6565, 10, 2013, 0xac8bb295 -0, 197, 197, 1, 518400, 0xcb9fc692 +0, 197, 197, 1, 518400, 0xc4c0bfc7 1, 6575, 6575, 10, 2013, 0xddd1949c 1, 6585, 6585, 10, 2013, 0x6b26b868 1, 6595, 6595, 10, 2013, 0x5eaba587 1, 6605, 6605, 10, 2013, 0xef0793b9 -0, 198, 198, 1, 518400, 0x5d05601e +0, 198, 198, 1, 518400, 0x57c85956 1, 6615, 6615, 10, 2013, 0xdef19bd6 1, 6625, 6625, 10, 2013, 0xca98a635 -0, 199, 199, 1, 518400, 0x456c1417 +0, 199, 199, 1, 518400, 0xd6300d46 1, 6640, 6640, 10, 2013, 0x06269a5a 1, 6650, 6650, 10, 2013, 0x32cb9952 1, 6660, 6660, 10, 2013, 0xf01fa95a 1, 6670, 6670, 10, 2013, 0xefab9e55 -0, 200, 200, 1, 518400, 0x9a0fd1ad +0, 200, 200, 1, 518400, 0xd3dacaec 1, 6680, 6680, 10, 2013, 0x55a3b63a 1, 6690, 6690, 10, 2013, 0xcd36a553 1, 6700, 6700, 10, 2013, 0x2ec19877 -0, 201, 201, 1, 518400, 0x55db9716 +0, 201, 201, 1, 518400, 0x65429052 1, 6710, 6710, 10, 2013, 0xc18b924c 1, 6726, 6726, 10, 2013, 0xf132b04c 1, 6736, 6736, 10, 2013, 0x7975a44d -0, 202, 202, 1, 518400, 0x1f0d40d6 +0, 202, 202, 1, 518400, 0xec803a15 1, 6746, 6746, 10, 2013, 0x2aaf94cb 1, 6756, 6756, 10, 2013, 0x58cfa60f 1, 6766, 6766, 10, 2013, 0x9757a658 -0, 203, 203, 1, 518400, 0x73695c82 +0, 203, 203, 1, 518400, 0x7a9a55c9 1, 6776, 6776, 10, 2013, 0x67ebc0d5 1, 6786, 6786, 10, 2013, 0x3c50a70e 1, 6796, 6796, 10, 2013, 0x9c5799c6 -0, 204, 204, 1, 518400, 0xb0f10812 +0, 204, 204, 1, 518400, 0xcac30160 1, 6811, 6811, 10, 2013, 0x018d85b2 1, 6821, 6821, 10, 2013, 0x5367a956 -0, 205, 205, 1, 518400, 0xdec18505 -0, 208, 208, 1, 518400, 0xb147b947 -0, 240, 240, 1, 518400, 0x9d2e3977 +0, 205, 205, 1, 518400, 0x7e187e4f +0, 208, 208, 1, 518400, 0x0be0b2a2 diff --git a/tests/ref/fate/sub-dvb b/tests/ref/fate/sub-dvb index cbd1801d64..8f33c75d70 100644 --- a/tests/ref/fate/sub-dvb +++ b/tests/ref/fate/sub-dvb @@ -1,75 +1,93 @@ #tb 0: 1/1000000 #media_type 0: subtitle #codec_id 0: dvb_subtitle -0, 15600000, 15600000, 159000, 1168, 0xd0f89d82 -0, 15759000, 15759000, 159000, 14, 0x064900eb -0, 15760000, 15760000, 239000, 1544, 0xe60f1751 -0, 15999000, 15999000, 239000, 14, 0x0729010b -0, 16000000, 16000000, 339000, 1658, 0xbe343093 -0, 16339000, 16339000, 339000, 14, 0x0809012b -0, 16340000, 16340000, 599000, 2343, 0xc68f07ef -0, 16939000, 16939000, 599000, 14, 0x08e9014b -0, 16940000, 16940000, 459000, 2568, 0x0ee657b1 -0, 17399000, 17399000, 459000, 14, 0x09c9016b -0, 17400000, 17400000, 359000, 3422, 0xba5b63ce -0, 17759000, 17759000, 359000, 14, 0x0aa9018b -0, 17760000, 17760000, 219000, 5078, 0x95b19902 -0, 17979000, 17979000, 219000, 14, 0x0b8901ab -0, 17980000, 17980000, 959000, 5808, 0xc9717b89 -0, 18939000, 18939000, 959000, 14, 0x0c6901cb -0, 18940000, 18940000, 219000, 6015, 0x0becbfac -0, 19159000, 19159000, 219000, 14, 0x064900eb -0, 19160000, 19160000, 259000, 6519, 0xfcd24d26 -0, 19419000, 19419000, 259000, 14, 0x0729010b -0, 19420000, 19420000, 99000, 7061, 0xf0320408 -0, 19519000, 19519000, 99000, 14, 0x0809012b -0, 19520000, 19520000, 219000, 4773, 0x66c93074 -0, 19739000, 19739000, 219000, 14, 0x08e9014b -0, 19740000, 19740000, 219000, 5546, 0x06052c81 -0, 19959000, 19959000, 219000, 14, 0x09c9016b -0, 19960000, 19960000, 239000, 5754, 0x904f7325 -0, 20199000, 20199000, 239000, 14, 0x0aa9018b -0, 20200000, 20200000, 139000, 6099, 0xe30cde07 -0, 20339000, 20339000, 139000, 14, 0x0b8901ab -0, 20340000, 20340000, 799000, 6839, 0x770fcb6c -0, 21139000, 21139000, 799000, 14, 0x0c6901cb -0, 21140000, 21140000, 239000, 4744, 0xa91e1b41 -0, 21379000, 21379000, 239000, 14, 0x064900eb -0, 21380000, 21380000, 339000, 5824, 0xcf6d782b -0, 21719000, 21719000, 339000, 14, 0x0729010b -0, 21720000, 21720000, 1439000, 6212, 0xabf8f7cf -0, 23159000, 23159000, 1439000, 14, 0x0809012b -0, 23160000, 23160000, 1319000, 7082, 0xd7ca10f2 -0, 24479000, 24479000, 1319000, 14, 0x08e9014b -0, 24480000, 24480000, 219000, 5345, 0x12b2cae0 -0, 24699000, 24699000, 219000, 14, 0x09c9016b -0, 24700000, 24700000, 219000, 5765, 0xc7d46192 -0, 24919000, 24919000, 219000, 14, 0x0aa9018b -0, 24920000, 24920000, 599000, 6557, 0xcb995d30 -0, 25519000, 25519000, 599000, 14, 0x0b8901ab -0, 25520000, 25520000, 219000, 7091, 0xe6ea0559 -0, 25739000, 25739000, 219000, 14, 0x0c6901cb -0, 25740000, 25740000, 239000, 7305, 0xb66c404e -0, 25979000, 25979000, 239000, 14, 0x064900eb -0, 25980000, 25980000, 359000, 7590, 0x0cc2a481 -0, 26339000, 26339000, 359000, 14, 0x0729010b -0, 26340000, 26340000, 219000, 4629, 0xe18cfea8 -0, 26559000, 26559000, 219000, 14, 0x0809012b -0, 26560000, 26560000, 719000, 4785, 0x82043fc0 -0, 27279000, 27279000, 719000, 14, 0x08e9014b -0, 27280000, 27280000, 459000, 6061, 0xbde7d245 -0, 27739000, 27739000, 459000, 14, 0x09c9016b -0, 27740000, 27740000, 239000, 6301, 0x92d01a51 -0, 27979000, 27979000, 239000, 14, 0x0aa9018b -0, 27980000, 27980000, 99000, 6736, 0xbd25a134 -0, 28079000, 28079000, 99000, 14, 0x0b8901ab -0, 28080000, 28080000, 219000, 7214, 0x7ef93c13 -0, 28299000, 28299000, 219000, 14, 0x0c6901cb -0, 28300000, 28300000, 239000, 7366, 0x5bed7fcd -0, 28539000, 28539000, 239000, 14, 0x064900eb -0, 28540000, 28540000, 599000, 4564, 0x7f4c014b -0, 29139000, 29139000, 599000, 14, 0x0729010b -0, 29140000, 29140000, 219000, 4637, 0x682626b7 -0, 29359000, 29359000, 219000, 14, 0x0809012b -0, 29360000, 29360000, 1679000, 5358, 0x29e30c48 -0, 31039000, 31039000, 1679000, 14, 0x08e9014b +0, 0, 0, 279000, 14, 0x05d900db +0, 279000, 279000, 279000, 14, 0x064900eb +0, 280000, 280000, 4999000, 14, 0x06b900fb +0, 5279000, 5279000, 4999000, 14, 0x0729010b +0, 5280000, 5280000, 5019000, 14, 0x0799011b +0, 10299000, 10299000, 5019000, 14, 0x0809012b +0, 10300000, 10300000, 3599000, 14, 0x0879013b +0, 13899000, 13899000, 3599000, 14, 0x08e9014b +0, 13900000, 13900000, 219000, 14, 0x0959015b +0, 14119000, 14119000, 219000, 14, 0x09c9016b +0, 14120000, 14120000, 1439000, 14, 0x0a39017b +0, 15559000, 15559000, 1439000, 14, 0x0aa9018b +0, 15560000, 15560000, 39000, 14, 0x0b19019b +0, 15599000, 15599000, 39000, 14, 0x0b8901ab +0, 15600000, 15600000, 159000, 1168, 0xd69da022 +0, 15759000, 15759000, 159000, 14, 0x0c6901cb +0, 15760000, 15760000, 239000, 1544, 0xc5f116f1 +0, 15999000, 15999000, 239000, 14, 0x064900eb +0, 16000000, 16000000, 339000, 1658, 0x73563033 +0, 16339000, 16339000, 339000, 14, 0x0729010b +0, 16340000, 16340000, 599000, 2343, 0x7ac2078f +0, 16939000, 16939000, 599000, 14, 0x0809012b +0, 16940000, 16940000, 459000, 2568, 0x6eaa5751 +0, 17399000, 17399000, 459000, 14, 0x08e9014b +0, 17400000, 17400000, 359000, 3422, 0xd9d0636e +0, 17759000, 17759000, 359000, 14, 0x09c9016b +0, 17760000, 17760000, 219000, 5078, 0x722c9862 +0, 17979000, 17979000, 219000, 14, 0x0aa9018b +0, 17980000, 17980000, 959000, 5808, 0x38dd7ae9 +0, 18939000, 18939000, 959000, 14, 0x0b8901ab +0, 18940000, 18940000, 219000, 6015, 0xd4d2c40c +0, 19159000, 19159000, 219000, 14, 0x0c6901cb +0, 19160000, 19160000, 259000, 6519, 0x08af4c86 +0, 19419000, 19419000, 259000, 14, 0x064900eb +0, 19420000, 19420000, 99000, 7061, 0xecf10368 +0, 19519000, 19519000, 99000, 14, 0x0729010b +0, 19520000, 19520000, 219000, 4773, 0xbee42fd4 +0, 19739000, 19739000, 219000, 14, 0x0809012b +0, 19740000, 19740000, 219000, 5546, 0xdb822be1 +0, 19959000, 19959000, 219000, 14, 0x08e9014b +0, 19960000, 19960000, 239000, 5754, 0xfdcc7285 +0, 20199000, 20199000, 239000, 14, 0x09c9016b +0, 20200000, 20200000, 139000, 6099, 0xa409dd67 +0, 20339000, 20339000, 139000, 14, 0x0aa9018b +0, 20340000, 20340000, 799000, 6839, 0xc5eecacc +0, 21139000, 21139000, 799000, 14, 0x0b8901ab +0, 21140000, 21140000, 239000, 4744, 0x4e451fa1 +0, 21379000, 21379000, 239000, 14, 0x0c6901cb +0, 21380000, 21380000, 339000, 5824, 0x5299778b +0, 21719000, 21719000, 339000, 14, 0x064900eb +0, 21720000, 21720000, 1439000, 6212, 0x6d15f72f +0, 23159000, 23159000, 1439000, 14, 0x0729010b +0, 23160000, 23160000, 1319000, 7082, 0xe5c91052 +0, 24479000, 24479000, 1319000, 14, 0x0809012b +0, 24480000, 24480000, 219000, 5345, 0x2e5eca40 +0, 24699000, 24699000, 219000, 14, 0x08e9014b +0, 24700000, 24700000, 219000, 5765, 0x118060f2 +0, 24919000, 24919000, 219000, 14, 0x09c9016b +0, 24920000, 24920000, 599000, 6557, 0x89275c90 +0, 25519000, 25519000, 599000, 14, 0x0aa9018b +0, 25520000, 25520000, 219000, 7091, 0x996904b9 +0, 25739000, 25739000, 219000, 14, 0x0b8901ab +0, 25740000, 25740000, 239000, 7305, 0xc23e44ae +0, 25979000, 25979000, 239000, 14, 0x0c6901cb +0, 25980000, 25980000, 359000, 7590, 0xc5a3a3e1 +0, 26339000, 26339000, 359000, 14, 0x064900eb +0, 26340000, 26340000, 219000, 4629, 0x7ad6fe08 +0, 26559000, 26559000, 219000, 14, 0x0729010b +0, 26560000, 26560000, 719000, 4785, 0xcd3f3f20 +0, 27279000, 27279000, 719000, 14, 0x0809012b +0, 27280000, 27280000, 459000, 6061, 0x8b04d1a5 +0, 27739000, 27739000, 459000, 14, 0x08e9014b +0, 27740000, 27740000, 239000, 6301, 0xe7de19b1 +0, 27979000, 27979000, 239000, 14, 0x09c9016b +0, 27980000, 27980000, 99000, 6736, 0x38b3a094 +0, 28079000, 28079000, 99000, 14, 0x0aa9018b +0, 28080000, 28080000, 219000, 7214, 0x0b783b73 +0, 28299000, 28299000, 219000, 14, 0x0b8901ab +0, 28300000, 28300000, 239000, 7366, 0x98bf842d +0, 28539000, 28539000, 239000, 14, 0x0c6901cb +0, 28540000, 28540000, 599000, 4564, 0x3d9600ab +0, 29139000, 29139000, 599000, 14, 0x064900eb +0, 29140000, 29140000, 219000, 4637, 0x01f02617 +0, 29359000, 29359000, 219000, 14, 0x0729010b +0, 29360000, 29360000, 1679000, 5358, 0x5b0f0ba8 +0, 31039000, 31039000, 1679000, 14, 0x0809012b +0, 31040000, 31040000, 359000, 14, 0x0879013b +0, 31399000, 31399000, 359000, 14, 0x08e9014b +0, 31400000, 31400000, 479000, 14, 0x0959015b +0, 31879000, 31879000, 479000, 14, 0x09c9016b diff --git a/tests/ref/fate/sub-scc b/tests/ref/fate/sub-scc index 62cbf6fa4a..b02e6e95b5 100644 --- a/tests/ref/fate/sub-scc +++ b/tests/ref/fate/sub-scc @@ -11,7 +11,6 @@ Style: Default,Monospace,16,&Hffffff,&Hffffff,&H0,&H0,0,0,0,0,100,100,0,0,3,1,0, [Events] Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text -Dialogue: 0,0:00:-2.-47,0:00:00.70,Default,,0,0,0,,{\an7}{\pos(76,228)}WE HAVE FOUND A WITCH !\N{\an7}{\pos(76,243)}MAY WE BURN HER ? Dialogue: 0,0:00:00.69,0:00:03.29,Default,,0,0,0,,{\an7}{\pos(115,228)}[ Crowd ]\N{\an7}{\pos(115,243)}BURN HER ! BURN HER ! Dialogue: 0,0:00:03.30,0:00:07.07,Default,,0,0,0,,{\an7}{\pos(38,197)}HOW DO YOU KNOW\N{\an7}{\pos(38,213)}SHE IS A WITCH ?\N{\an7}{\pos(153,243)}SHE LOOKS LIKE ONE ! Dialogue: 0,0:00:07.07,0:00:09.27,Default,,0,0,0,,{\an7}{\pos(192,228)}[ Shouting\N{\an7}{\pos(192,243)}\h\hAffirmations ] diff --git a/tests/ref/fate/sub2video b/tests/ref/fate/sub2video index 80abe9c905..7e89ee0ccd 100644 --- a/tests/ref/fate/sub2video +++ b/tests/ref/fate/sub2video @@ -58,129 +58,1137 @@ 0, 47, 47, 1, 518400, 0xde69683f 0, 48, 48, 1, 518400, 0x7df08fba 0, 49, 49, 1, 518400, 0xbab197ea +0, 50, 50, 1, 518400, 0xbab197ea +0, 51, 51, 1, 518400, 0xbab197ea +0, 52, 52, 1, 518400, 0xbab197ea +0, 53, 53, 1, 518400, 0xbab197ea +0, 54, 54, 1, 518400, 0xbab197ea +0, 55, 55, 1, 518400, 0xbab197ea +0, 56, 56, 1, 518400, 0xbab197ea +0, 57, 57, 1, 518400, 0xbab197ea +0, 58, 58, 1, 518400, 0xbab197ea +0, 59, 59, 1, 518400, 0xbab197ea +0, 60, 60, 1, 518400, 0xbab197ea +0, 61, 61, 1, 518400, 0xbab197ea +0, 62, 62, 1, 518400, 0xbab197ea +0, 63, 63, 1, 518400, 0xbab197ea +0, 64, 64, 1, 518400, 0xbab197ea +0, 65, 65, 1, 518400, 0xbab197ea +0, 66, 66, 1, 518400, 0xbab197ea +0, 67, 67, 1, 518400, 0xbab197ea +0, 68, 68, 1, 518400, 0xbab197ea +0, 69, 69, 1, 518400, 0xbab197ea +0, 70, 70, 1, 518400, 0xbab197ea +0, 71, 71, 1, 518400, 0xbab197ea +0, 72, 72, 1, 518400, 0xbab197ea +0, 73, 73, 1, 518400, 0xbab197ea +0, 74, 74, 1, 518400, 0xbab197ea +0, 75, 75, 1, 518400, 0xbab197ea +0, 76, 76, 1, 518400, 0xbab197ea 1, 15355000, 15355000, 4733000, 2094, 0x3c171425 0, 77, 77, 1, 518400, 0x902285d9 -0, 100, 100, 1, 518400, 0xbab197ea +0, 78, 78, 1, 518400, 0x902285d9 +0, 79, 79, 1, 518400, 0x902285d9 +0, 80, 80, 1, 518400, 0x902285d9 +0, 81, 81, 1, 518400, 0x902285d9 +0, 82, 82, 1, 518400, 0x902285d9 +0, 83, 83, 1, 518400, 0x902285d9 +0, 84, 84, 1, 518400, 0x902285d9 +0, 85, 85, 1, 518400, 0x902285d9 +0, 86, 86, 1, 518400, 0x902285d9 +0, 87, 87, 1, 518400, 0x902285d9 +0, 88, 88, 1, 518400, 0x902285d9 +0, 89, 89, 1, 518400, 0x902285d9 +0, 90, 90, 1, 518400, 0x902285d9 +0, 91, 91, 1, 518400, 0x902285d9 +0, 92, 92, 1, 518400, 0x902285d9 +0, 93, 93, 1, 518400, 0x902285d9 +0, 94, 94, 1, 518400, 0x902285d9 +0, 95, 95, 1, 518400, 0x902285d9 +0, 96, 96, 1, 518400, 0x902285d9 +0, 97, 97, 1, 518400, 0x902285d9 +0, 98, 98, 1, 518400, 0x902285d9 +0, 99, 99, 1, 518400, 0x902285d9 +0, 100, 100, 1, 518400, 0x902285d9 +0, 101, 101, 1, 518400, 0xbab197ea +0, 102, 102, 1, 518400, 0xbab197ea +0, 103, 103, 1, 518400, 0xbab197ea +0, 104, 104, 1, 518400, 0xbab197ea +0, 105, 105, 1, 518400, 0xbab197ea +0, 106, 106, 1, 518400, 0xbab197ea +0, 107, 107, 1, 518400, 0xbab197ea +0, 108, 108, 1, 518400, 0xbab197ea +0, 109, 109, 1, 518400, 0xbab197ea +0, 110, 110, 1, 518400, 0xbab197ea +0, 111, 111, 1, 518400, 0xbab197ea +0, 112, 112, 1, 518400, 0xbab197ea +0, 113, 113, 1, 518400, 0xbab197ea +0, 114, 114, 1, 518400, 0xbab197ea +0, 115, 115, 1, 518400, 0xbab197ea +0, 116, 116, 1, 518400, 0xbab197ea +0, 117, 117, 1, 518400, 0xbab197ea +0, 118, 118, 1, 518400, 0xbab197ea +0, 119, 119, 1, 518400, 0xbab197ea +0, 120, 120, 1, 518400, 0xbab197ea +0, 121, 121, 1, 518400, 0xbab197ea +0, 122, 122, 1, 518400, 0xbab197ea +0, 123, 123, 1, 518400, 0xbab197ea +0, 124, 124, 1, 518400, 0xbab197ea +0, 125, 125, 1, 518400, 0xbab197ea +0, 126, 126, 1, 518400, 0xbab197ea +0, 127, 127, 1, 518400, 0xbab197ea +0, 128, 128, 1, 518400, 0xbab197ea +0, 129, 129, 1, 518400, 0xbab197ea +0, 130, 130, 1, 518400, 0xbab197ea +0, 131, 131, 1, 518400, 0xbab197ea +0, 132, 132, 1, 518400, 0xbab197ea +0, 133, 133, 1, 518400, 0xbab197ea +0, 134, 134, 1, 518400, 0xbab197ea +0, 135, 135, 1, 518400, 0xbab197ea +0, 136, 136, 1, 518400, 0xbab197ea +0, 137, 137, 1, 518400, 0xbab197ea +0, 138, 138, 1, 518400, 0xbab197ea +0, 139, 139, 1, 518400, 0xbab197ea +0, 140, 140, 1, 518400, 0xbab197ea +0, 141, 141, 1, 518400, 0xbab197ea +0, 142, 142, 1, 518400, 0xbab197ea +0, 143, 143, 1, 518400, 0xbab197ea +0, 144, 144, 1, 518400, 0xbab197ea +0, 145, 145, 1, 518400, 0xbab197ea +0, 146, 146, 1, 518400, 0xbab197ea +0, 147, 147, 1, 518400, 0xbab197ea +0, 148, 148, 1, 518400, 0xbab197ea +0, 149, 149, 1, 518400, 0xbab197ea +0, 150, 150, 1, 518400, 0xbab197ea +0, 151, 151, 1, 518400, 0xbab197ea +0, 152, 152, 1, 518400, 0xbab197ea +0, 153, 153, 1, 518400, 0xbab197ea +0, 154, 154, 1, 518400, 0xbab197ea +0, 155, 155, 1, 518400, 0xbab197ea +0, 156, 156, 1, 518400, 0xbab197ea +0, 157, 157, 1, 518400, 0xbab197ea +0, 158, 158, 1, 518400, 0xbab197ea +0, 159, 159, 1, 518400, 0xbab197ea +0, 160, 160, 1, 518400, 0xbab197ea +0, 161, 161, 1, 518400, 0xbab197ea +0, 162, 162, 1, 518400, 0xbab197ea +0, 163, 163, 1, 518400, 0xbab197ea +0, 164, 164, 1, 518400, 0xbab197ea +0, 165, 165, 1, 518400, 0xbab197ea +0, 166, 166, 1, 518400, 0xbab197ea +0, 167, 167, 1, 518400, 0xbab197ea +0, 168, 168, 1, 518400, 0xbab197ea +0, 169, 169, 1, 518400, 0xbab197ea +0, 170, 170, 1, 518400, 0xbab197ea +0, 171, 171, 1, 518400, 0xbab197ea +0, 172, 172, 1, 518400, 0xbab197ea +0, 173, 173, 1, 518400, 0xbab197ea +0, 174, 174, 1, 518400, 0xbab197ea +0, 175, 175, 1, 518400, 0xbab197ea +0, 176, 176, 1, 518400, 0xbab197ea +0, 177, 177, 1, 518400, 0xbab197ea +0, 178, 178, 1, 518400, 0xbab197ea +0, 179, 179, 1, 518400, 0xbab197ea +0, 180, 180, 1, 518400, 0xbab197ea +0, 181, 181, 1, 518400, 0xbab197ea +0, 182, 182, 1, 518400, 0xbab197ea +0, 183, 183, 1, 518400, 0xbab197ea +0, 184, 184, 1, 518400, 0xbab197ea +0, 185, 185, 1, 518400, 0xbab197ea +0, 186, 186, 1, 518400, 0xbab197ea +0, 187, 187, 1, 518400, 0xbab197ea +0, 188, 188, 1, 518400, 0xbab197ea +0, 189, 189, 1, 518400, 0xbab197ea +0, 190, 190, 1, 518400, 0xbab197ea +0, 191, 191, 1, 518400, 0xbab197ea +0, 192, 192, 1, 518400, 0xbab197ea +0, 193, 193, 1, 518400, 0xbab197ea +0, 194, 194, 1, 518400, 0xbab197ea +0, 195, 195, 1, 518400, 0xbab197ea +0, 196, 196, 1, 518400, 0xbab197ea +0, 197, 197, 1, 518400, 0xbab197ea +0, 198, 198, 1, 518400, 0xbab197ea +0, 199, 199, 1, 518400, 0xbab197ea +0, 200, 200, 1, 518400, 0xbab197ea +0, 201, 201, 1, 518400, 0xbab197ea +0, 202, 202, 1, 518400, 0xbab197ea +0, 203, 203, 1, 518400, 0xbab197ea +0, 204, 204, 1, 518400, 0xbab197ea +0, 205, 205, 1, 518400, 0xbab197ea +0, 206, 206, 1, 518400, 0xbab197ea +0, 207, 207, 1, 518400, 0xbab197ea +0, 208, 208, 1, 518400, 0xbab197ea +0, 209, 209, 1, 518400, 0xbab197ea +0, 210, 210, 1, 518400, 0xbab197ea +0, 211, 211, 1, 518400, 0xbab197ea +0, 212, 212, 1, 518400, 0xbab197ea +0, 213, 213, 1, 518400, 0xbab197ea +0, 214, 214, 1, 518400, 0xbab197ea +0, 215, 215, 1, 518400, 0xbab197ea +0, 216, 216, 1, 518400, 0xbab197ea +0, 217, 217, 1, 518400, 0xbab197ea +0, 218, 218, 1, 518400, 0xbab197ea +0, 219, 219, 1, 518400, 0xbab197ea +0, 220, 220, 1, 518400, 0xbab197ea +0, 221, 221, 1, 518400, 0xbab197ea +0, 222, 222, 1, 518400, 0xbab197ea +0, 223, 223, 1, 518400, 0xbab197ea +0, 224, 224, 1, 518400, 0xbab197ea +0, 225, 225, 1, 518400, 0xbab197ea +0, 226, 226, 1, 518400, 0xbab197ea +0, 227, 227, 1, 518400, 0xbab197ea +0, 228, 228, 1, 518400, 0xbab197ea +0, 229, 229, 1, 518400, 0xbab197ea +0, 230, 230, 1, 518400, 0xbab197ea +0, 231, 231, 1, 518400, 0xbab197ea +0, 232, 232, 1, 518400, 0xbab197ea +0, 233, 233, 1, 518400, 0xbab197ea +0, 234, 234, 1, 518400, 0xbab197ea +0, 235, 235, 1, 518400, 0xbab197ea +0, 236, 236, 1, 518400, 0xbab197ea +0, 237, 237, 1, 518400, 0xbab197ea +0, 238, 238, 1, 518400, 0xbab197ea +0, 239, 239, 1, 518400, 0xbab197ea +0, 240, 240, 1, 518400, 0xbab197ea +0, 241, 241, 1, 518400, 0xbab197ea +0, 242, 242, 1, 518400, 0xbab197ea +0, 243, 243, 1, 518400, 0xbab197ea 1, 48797000, 48797000, 2560000, 2480, 0x7c0edf21 0, 244, 244, 1, 518400, 0x7a11c812 -0, 257, 257, 1, 518400, 0xbab197ea +0, 245, 245, 1, 518400, 0x7a11c812 +0, 246, 246, 1, 518400, 0x7a11c812 +0, 247, 247, 1, 518400, 0x7a11c812 +0, 248, 248, 1, 518400, 0x7a11c812 +0, 249, 249, 1, 518400, 0x7a11c812 +0, 250, 250, 1, 518400, 0x7a11c812 +0, 251, 251, 1, 518400, 0x7a11c812 +0, 252, 252, 1, 518400, 0x7a11c812 +0, 253, 253, 1, 518400, 0x7a11c812 +0, 254, 254, 1, 518400, 0x7a11c812 +0, 255, 255, 1, 518400, 0x7a11c812 +0, 256, 256, 1, 518400, 0x7a11c812 +0, 257, 257, 1, 518400, 0x34cdddee 1, 51433000, 51433000, 2366000, 3059, 0xc95b8a05 0, 258, 258, 1, 518400, 0x34cdddee -0, 269, 269, 1, 518400, 0xbab197ea +0, 259, 259, 1, 518400, 0x34cdddee +0, 260, 260, 1, 518400, 0x34cdddee +0, 261, 261, 1, 518400, 0x34cdddee +0, 262, 262, 1, 518400, 0x34cdddee +0, 263, 263, 1, 518400, 0x34cdddee +0, 264, 264, 1, 518400, 0x34cdddee +0, 265, 265, 1, 518400, 0x34cdddee +0, 266, 266, 1, 518400, 0x34cdddee +0, 267, 267, 1, 518400, 0x34cdddee +0, 268, 268, 1, 518400, 0x34cdddee +0, 269, 269, 1, 518400, 0x34cdddee 1, 53910000, 53910000, 2696000, 2095, 0x61bb15ed 0, 270, 270, 1, 518400, 0x4db4ce51 -0, 283, 283, 1, 518400, 0xbab197ea +0, 271, 271, 1, 518400, 0x4db4ce51 +0, 272, 272, 1, 518400, 0x4db4ce51 +0, 273, 273, 1, 518400, 0x4db4ce51 +0, 274, 274, 1, 518400, 0x4db4ce51 +0, 275, 275, 1, 518400, 0x4db4ce51 +0, 276, 276, 1, 518400, 0x4db4ce51 +0, 277, 277, 1, 518400, 0x4db4ce51 +0, 278, 278, 1, 518400, 0x4db4ce51 +0, 279, 279, 1, 518400, 0x4db4ce51 +0, 280, 280, 1, 518400, 0x4db4ce51 +0, 281, 281, 1, 518400, 0x4db4ce51 +0, 282, 282, 1, 518400, 0x4db4ce51 +0, 283, 283, 1, 518400, 0xe6bc0ea9 1, 56663000, 56663000, 1262000, 1013, 0xc9ae89b7 0, 284, 284, 1, 518400, 0xe6bc0ea9 -0, 290, 290, 1, 518400, 0xbab197ea +0, 285, 285, 1, 518400, 0xe6bc0ea9 +0, 286, 286, 1, 518400, 0xe6bc0ea9 +0, 287, 287, 1, 518400, 0xe6bc0ea9 +0, 288, 288, 1, 518400, 0xe6bc0ea9 +0, 289, 289, 1, 518400, 0xe6bc0ea9 +0, 290, 290, 1, 518400, 0xa8643af7 1, 58014000, 58014000, 1661000, 969, 0xe01878f0 0, 291, 291, 1, 518400, 0xa8643af7 -0, 298, 298, 1, 518400, 0xbab197ea +0, 292, 292, 1, 518400, 0xa8643af7 +0, 293, 293, 1, 518400, 0xa8643af7 +0, 294, 294, 1, 518400, 0xa8643af7 +0, 295, 295, 1, 518400, 0xa8643af7 +0, 296, 296, 1, 518400, 0xa8643af7 +0, 297, 297, 1, 518400, 0xa8643af7 +0, 298, 298, 1, 518400, 0xa8643af7 +0, 299, 299, 1, 518400, 0xbab197ea +0, 300, 300, 1, 518400, 0xbab197ea +0, 301, 301, 1, 518400, 0xbab197ea +0, 302, 302, 1, 518400, 0xbab197ea +0, 303, 303, 1, 518400, 0xbab197ea +0, 304, 304, 1, 518400, 0xbab197ea +0, 305, 305, 1, 518400, 0xbab197ea +0, 306, 306, 1, 518400, 0xbab197ea +0, 307, 307, 1, 518400, 0xbab197ea +0, 308, 308, 1, 518400, 0xbab197ea +0, 309, 309, 1, 518400, 0xbab197ea +0, 310, 310, 1, 518400, 0xbab197ea +0, 311, 311, 1, 518400, 0xbab197ea +0, 312, 312, 1, 518400, 0xbab197ea +0, 313, 313, 1, 518400, 0xbab197ea +0, 314, 314, 1, 518400, 0xbab197ea +0, 315, 315, 1, 518400, 0xbab197ea +0, 316, 316, 1, 518400, 0xbab197ea +0, 317, 317, 1, 518400, 0xbab197ea +0, 318, 318, 1, 518400, 0xbab197ea +0, 319, 319, 1, 518400, 0xbab197ea +0, 320, 320, 1, 518400, 0xbab197ea +0, 321, 321, 1, 518400, 0xbab197ea +0, 322, 322, 1, 518400, 0xbab197ea +0, 323, 323, 1, 518400, 0xbab197ea +0, 324, 324, 1, 518400, 0xbab197ea +0, 325, 325, 1, 518400, 0xbab197ea +0, 326, 326, 1, 518400, 0xbab197ea +0, 327, 327, 1, 518400, 0xbab197ea +0, 328, 328, 1, 518400, 0xbab197ea +0, 329, 329, 1, 518400, 0xbab197ea +0, 330, 330, 1, 518400, 0xbab197ea +0, 331, 331, 1, 518400, 0xbab197ea +0, 332, 332, 1, 518400, 0xbab197ea +0, 333, 333, 1, 518400, 0xbab197ea +0, 334, 334, 1, 518400, 0xbab197ea +0, 335, 335, 1, 518400, 0xbab197ea +0, 336, 336, 1, 518400, 0xbab197ea +0, 337, 337, 1, 518400, 0xbab197ea +0, 338, 338, 1, 518400, 0xbab197ea 1, 67724000, 67724000, 1365000, 844, 0xe7db4fc1 0, 339, 339, 1, 518400, 0xb1885c67 -0, 345, 345, 1, 518400, 0xbab197ea +0, 340, 340, 1, 518400, 0xb1885c67 +0, 341, 341, 1, 518400, 0xb1885c67 +0, 342, 342, 1, 518400, 0xb1885c67 +0, 343, 343, 1, 518400, 0xb1885c67 +0, 344, 344, 1, 518400, 0xb1885c67 +0, 345, 345, 1, 518400, 0xb1885c67 1, 69175000, 69175000, 1558000, 802, 0xf48531ba 0, 346, 346, 1, 518400, 0x378e3fd0 -0, 354, 354, 1, 518400, 0xbab197ea +0, 347, 347, 1, 518400, 0x378e3fd0 +0, 348, 348, 1, 518400, 0x378e3fd0 +0, 349, 349, 1, 518400, 0x378e3fd0 +0, 350, 350, 1, 518400, 0x378e3fd0 +0, 351, 351, 1, 518400, 0x378e3fd0 +0, 352, 352, 1, 518400, 0x378e3fd0 +0, 353, 353, 1, 518400, 0x378e3fd0 +0, 354, 354, 1, 518400, 0xa3782469 1, 70819000, 70819000, 1865000, 1709, 0xb4d5a1bd 0, 355, 355, 1, 518400, 0xa3782469 -0, 363, 363, 1, 518400, 0xbab197ea +0, 356, 356, 1, 518400, 0xa3782469 +0, 357, 357, 1, 518400, 0xa3782469 +0, 358, 358, 1, 518400, 0xa3782469 +0, 359, 359, 1, 518400, 0xa3782469 +0, 360, 360, 1, 518400, 0xa3782469 +0, 361, 361, 1, 518400, 0xa3782469 +0, 362, 362, 1, 518400, 0xa3782469 +0, 363, 363, 1, 518400, 0xa3782469 1, 72762000, 72762000, 1968000, 2438, 0x99d7bc82 0, 364, 364, 1, 518400, 0xba23a0d5 -0, 374, 374, 1, 518400, 0xbab197ea +0, 365, 365, 1, 518400, 0xba23a0d5 +0, 366, 366, 1, 518400, 0xba23a0d5 +0, 367, 367, 1, 518400, 0xba23a0d5 +0, 368, 368, 1, 518400, 0xba23a0d5 +0, 369, 369, 1, 518400, 0xba23a0d5 +0, 370, 370, 1, 518400, 0xba23a0d5 +0, 371, 371, 1, 518400, 0xba23a0d5 +0, 372, 372, 1, 518400, 0xba23a0d5 +0, 373, 373, 1, 518400, 0xba23a0d5 +0, 374, 374, 1, 518400, 0x129de2f8 1, 74806000, 74806000, 1831000, 2116, 0x96514097 0, 375, 375, 1, 518400, 0x129de2f8 -0, 383, 383, 1, 518400, 0xbab197ea +0, 376, 376, 1, 518400, 0x129de2f8 +0, 377, 377, 1, 518400, 0x129de2f8 +0, 378, 378, 1, 518400, 0x129de2f8 +0, 379, 379, 1, 518400, 0x129de2f8 +0, 380, 380, 1, 518400, 0x129de2f8 +0, 381, 381, 1, 518400, 0x129de2f8 +0, 382, 382, 1, 518400, 0x129de2f8 +0, 383, 383, 1, 518400, 0x129de2f8 1, 76716000, 76716000, 1262000, 1822, 0xefccc72e 0, 384, 384, 1, 518400, 0x19772f0f -0, 390, 390, 1, 518400, 0xbab197ea +0, 385, 385, 1, 518400, 0x19772f0f +0, 386, 386, 1, 518400, 0x19772f0f +0, 387, 387, 1, 518400, 0x19772f0f +0, 388, 388, 1, 518400, 0x19772f0f +0, 389, 389, 1, 518400, 0x19772f0f +0, 390, 390, 1, 518400, 0x56f54e73 1, 78051000, 78051000, 1524000, 987, 0x7b927a27 0, 391, 391, 1, 518400, 0x56f54e73 -0, 398, 398, 1, 518400, 0xbab197ea +0, 392, 392, 1, 518400, 0x56f54e73 +0, 393, 393, 1, 518400, 0x56f54e73 +0, 394, 394, 1, 518400, 0x56f54e73 +0, 395, 395, 1, 518400, 0x56f54e73 +0, 396, 396, 1, 518400, 0x56f54e73 +0, 397, 397, 1, 518400, 0x56f54e73 +0, 398, 398, 1, 518400, 0x300b5247 1, 79644000, 79644000, 2662000, 2956, 0x190778f7 0, 399, 399, 1, 518400, 0x300b5247 +0, 400, 400, 1, 518400, 0x300b5247 +0, 401, 401, 1, 518400, 0x300b5247 +0, 402, 402, 1, 518400, 0x300b5247 +0, 403, 403, 1, 518400, 0x300b5247 +0, 404, 404, 1, 518400, 0x300b5247 +0, 405, 405, 1, 518400, 0x300b5247 +0, 406, 406, 1, 518400, 0x300b5247 +0, 407, 407, 1, 518400, 0x300b5247 +0, 408, 408, 1, 518400, 0x300b5247 +0, 409, 409, 1, 518400, 0x300b5247 +0, 410, 410, 1, 518400, 0x300b5247 +0, 411, 411, 1, 518400, 0x300b5247 1, 82380000, 82380000, 2764000, 3094, 0xc021b7d3 -0, 412, 412, 1, 518400, 0xbab197ea +0, 412, 412, 1, 518400, 0x6fd028fa 0, 413, 413, 1, 518400, 0x6fd028fa -0, 426, 426, 1, 518400, 0xbab197ea +0, 414, 414, 1, 518400, 0x6fd028fa +0, 415, 415, 1, 518400, 0x6fd028fa +0, 416, 416, 1, 518400, 0x6fd028fa +0, 417, 417, 1, 518400, 0x6fd028fa +0, 418, 418, 1, 518400, 0x6fd028fa +0, 419, 419, 1, 518400, 0x6fd028fa +0, 420, 420, 1, 518400, 0x6fd028fa +0, 421, 421, 1, 518400, 0x6fd028fa +0, 422, 422, 1, 518400, 0x6fd028fa +0, 423, 423, 1, 518400, 0x6fd028fa +0, 424, 424, 1, 518400, 0x6fd028fa +0, 425, 425, 1, 518400, 0x6fd028fa +0, 426, 426, 1, 518400, 0x01f80e9d 1, 85225000, 85225000, 2366000, 2585, 0x74d0048f 0, 427, 427, 1, 518400, 0x01f80e9d -0, 438, 438, 1, 518400, 0xbab197ea +0, 428, 428, 1, 518400, 0x01f80e9d +0, 429, 429, 1, 518400, 0x01f80e9d +0, 430, 430, 1, 518400, 0x01f80e9d +0, 431, 431, 1, 518400, 0x01f80e9d +0, 432, 432, 1, 518400, 0x01f80e9d +0, 433, 433, 1, 518400, 0x01f80e9d +0, 434, 434, 1, 518400, 0x01f80e9d +0, 435, 435, 1, 518400, 0x01f80e9d +0, 436, 436, 1, 518400, 0x01f80e9d +0, 437, 437, 1, 518400, 0x01f80e9d +0, 438, 438, 1, 518400, 0xb48d90c0 1, 87652000, 87652000, 1831000, 634, 0x8832fda1 0, 439, 439, 1, 518400, 0xb48d90c0 -0, 447, 447, 1, 518400, 0xbab197ea +0, 440, 440, 1, 518400, 0xb48d90c0 +0, 441, 441, 1, 518400, 0xb48d90c0 +0, 442, 442, 1, 518400, 0xb48d90c0 +0, 443, 443, 1, 518400, 0xb48d90c0 +0, 444, 444, 1, 518400, 0xb48d90c0 +0, 445, 445, 1, 518400, 0xb48d90c0 +0, 446, 446, 1, 518400, 0xb48d90c0 +0, 447, 447, 1, 518400, 0xb48d90c0 +0, 448, 448, 1, 518400, 0xbab197ea +0, 449, 449, 1, 518400, 0xbab197ea +0, 450, 450, 1, 518400, 0xbab197ea +0, 451, 451, 1, 518400, 0xbab197ea +0, 452, 452, 1, 518400, 0xbab197ea +0, 453, 453, 1, 518400, 0xbab197ea +0, 454, 454, 1, 518400, 0xbab197ea +0, 455, 455, 1, 518400, 0xbab197ea +0, 456, 456, 1, 518400, 0xbab197ea +0, 457, 457, 1, 518400, 0xbab197ea 1, 91531000, 91531000, 2332000, 2080, 0x97a1146f 0, 458, 458, 1, 518400, 0xcb5a0173 -0, 469, 469, 1, 518400, 0xbab197ea +0, 459, 459, 1, 518400, 0xcb5a0173 +0, 460, 460, 1, 518400, 0xcb5a0173 +0, 461, 461, 1, 518400, 0xcb5a0173 +0, 462, 462, 1, 518400, 0xcb5a0173 +0, 463, 463, 1, 518400, 0xcb5a0173 +0, 464, 464, 1, 518400, 0xcb5a0173 +0, 465, 465, 1, 518400, 0xcb5a0173 +0, 466, 466, 1, 518400, 0xcb5a0173 +0, 467, 467, 1, 518400, 0xcb5a0173 +0, 468, 468, 1, 518400, 0xcb5a0173 +0, 469, 469, 1, 518400, 0xcb5a0173 +0, 470, 470, 1, 518400, 0xbab197ea +0, 471, 471, 1, 518400, 0xbab197ea +0, 472, 472, 1, 518400, 0xbab197ea +0, 473, 473, 1, 518400, 0xbab197ea +0, 474, 474, 1, 518400, 0xbab197ea +0, 475, 475, 1, 518400, 0xbab197ea +0, 476, 476, 1, 518400, 0xbab197ea +0, 477, 477, 1, 518400, 0xbab197ea 1, 95510000, 95510000, 3299000, 2964, 0x8b8f6684 0, 478, 478, 1, 518400, 0xb8a323e4 -0, 494, 494, 1, 518400, 0xbab197ea +0, 479, 479, 1, 518400, 0xb8a323e4 +0, 480, 480, 1, 518400, 0xb8a323e4 +0, 481, 481, 1, 518400, 0xb8a323e4 +0, 482, 482, 1, 518400, 0xb8a323e4 +0, 483, 483, 1, 518400, 0xb8a323e4 +0, 484, 484, 1, 518400, 0xb8a323e4 +0, 485, 485, 1, 518400, 0xb8a323e4 +0, 486, 486, 1, 518400, 0xb8a323e4 +0, 487, 487, 1, 518400, 0xb8a323e4 +0, 488, 488, 1, 518400, 0xb8a323e4 +0, 489, 489, 1, 518400, 0xb8a323e4 +0, 490, 490, 1, 518400, 0xb8a323e4 +0, 491, 491, 1, 518400, 0xb8a323e4 +0, 492, 492, 1, 518400, 0xb8a323e4 +0, 493, 493, 1, 518400, 0xb8a323e4 +0, 494, 494, 1, 518400, 0xc43518ba 1, 98872000, 98872000, 2161000, 1875, 0x9002ef71 0, 495, 495, 1, 518400, 0xc43518ba -0, 505, 505, 1, 518400, 0xbab197ea +0, 496, 496, 1, 518400, 0xc43518ba +0, 497, 497, 1, 518400, 0xc43518ba +0, 498, 498, 1, 518400, 0xc43518ba +0, 499, 499, 1, 518400, 0xc43518ba +0, 500, 500, 1, 518400, 0xc43518ba +0, 501, 501, 1, 518400, 0xc43518ba +0, 502, 502, 1, 518400, 0xc43518ba +0, 503, 503, 1, 518400, 0xc43518ba +0, 504, 504, 1, 518400, 0xc43518ba +0, 505, 505, 1, 518400, 0xc43518ba 1, 101124000, 101124000, 4096000, 3872, 0x20c6ed9c 0, 506, 506, 1, 518400, 0x04e38692 -0, 526, 526, 1, 518400, 0xbab197ea +0, 507, 507, 1, 518400, 0x04e38692 +0, 508, 508, 1, 518400, 0x04e38692 +0, 509, 509, 1, 518400, 0x04e38692 +0, 510, 510, 1, 518400, 0x04e38692 +0, 511, 511, 1, 518400, 0x04e38692 +0, 512, 512, 1, 518400, 0x04e38692 +0, 513, 513, 1, 518400, 0x04e38692 +0, 514, 514, 1, 518400, 0x04e38692 +0, 515, 515, 1, 518400, 0x04e38692 +0, 516, 516, 1, 518400, 0x04e38692 +0, 517, 517, 1, 518400, 0x04e38692 +0, 518, 518, 1, 518400, 0x04e38692 +0, 519, 519, 1, 518400, 0x04e38692 +0, 520, 520, 1, 518400, 0x04e38692 +0, 521, 521, 1, 518400, 0x04e38692 +0, 522, 522, 1, 518400, 0x04e38692 +0, 523, 523, 1, 518400, 0x04e38692 +0, 524, 524, 1, 518400, 0x04e38692 +0, 525, 525, 1, 518400, 0x04e38692 +0, 526, 526, 1, 518400, 0x04e38692 1, 105303000, 105303000, 2730000, 3094, 0xf203a663 0, 527, 527, 1, 518400, 0x856b0ee5 -0, 540, 540, 1, 518400, 0xbab197ea +0, 528, 528, 1, 518400, 0x856b0ee5 +0, 529, 529, 1, 518400, 0x856b0ee5 +0, 530, 530, 1, 518400, 0x856b0ee5 +0, 531, 531, 1, 518400, 0x856b0ee5 +0, 532, 532, 1, 518400, 0x856b0ee5 +0, 533, 533, 1, 518400, 0x856b0ee5 +0, 534, 534, 1, 518400, 0x856b0ee5 +0, 535, 535, 1, 518400, 0x856b0ee5 +0, 536, 536, 1, 518400, 0x856b0ee5 +0, 537, 537, 1, 518400, 0x856b0ee5 +0, 538, 538, 1, 518400, 0x856b0ee5 +0, 539, 539, 1, 518400, 0x856b0ee5 +0, 540, 540, 1, 518400, 0x856b0ee5 1, 108106000, 108106000, 2059000, 2404, 0x41a7b429 0, 541, 541, 1, 518400, 0x3e5beee2 -0, 551, 551, 1, 518400, 0xbab197ea +0, 542, 542, 1, 518400, 0x3e5beee2 +0, 543, 543, 1, 518400, 0x3e5beee2 +0, 544, 544, 1, 518400, 0x3e5beee2 +0, 545, 545, 1, 518400, 0x3e5beee2 +0, 546, 546, 1, 518400, 0x3e5beee2 +0, 547, 547, 1, 518400, 0x3e5beee2 +0, 548, 548, 1, 518400, 0x3e5beee2 +0, 549, 549, 1, 518400, 0x3e5beee2 +0, 550, 550, 1, 518400, 0x3e5beee2 +0, 551, 551, 1, 518400, 0x3e5beee2 +0, 552, 552, 1, 518400, 0xbab197ea +0, 553, 553, 1, 518400, 0xbab197ea +0, 554, 554, 1, 518400, 0xbab197ea +0, 555, 555, 1, 518400, 0xbab197ea +0, 556, 556, 1, 518400, 0xbab197ea +0, 557, 557, 1, 518400, 0xbab197ea +0, 558, 558, 1, 518400, 0xbab197ea +0, 559, 559, 1, 518400, 0xbab197ea +0, 560, 560, 1, 518400, 0xbab197ea +0, 561, 561, 1, 518400, 0xbab197ea +0, 562, 562, 1, 518400, 0xbab197ea +0, 563, 563, 1, 518400, 0xbab197ea +0, 564, 564, 1, 518400, 0xbab197ea +0, 565, 565, 1, 518400, 0xbab197ea +0, 566, 566, 1, 518400, 0xbab197ea +0, 567, 567, 1, 518400, 0xbab197ea +0, 568, 568, 1, 518400, 0xbab197ea +0, 569, 569, 1, 518400, 0xbab197ea +0, 570, 570, 1, 518400, 0xbab197ea +0, 571, 571, 1, 518400, 0xbab197ea +0, 572, 572, 1, 518400, 0xbab197ea +0, 573, 573, 1, 518400, 0xbab197ea +0, 574, 574, 1, 518400, 0xbab197ea +0, 575, 575, 1, 518400, 0xbab197ea +0, 576, 576, 1, 518400, 0xbab197ea +0, 577, 577, 1, 518400, 0xbab197ea +0, 578, 578, 1, 518400, 0xbab197ea +0, 579, 579, 1, 518400, 0xbab197ea +0, 580, 580, 1, 518400, 0xbab197ea +0, 581, 581, 1, 518400, 0xbab197ea +0, 582, 582, 1, 518400, 0xbab197ea +0, 583, 583, 1, 518400, 0xbab197ea +0, 584, 584, 1, 518400, 0xbab197ea +0, 585, 585, 1, 518400, 0xbab197ea +0, 586, 586, 1, 518400, 0xbab197ea +0, 587, 587, 1, 518400, 0xbab197ea +0, 588, 588, 1, 518400, 0xbab197ea +0, 589, 589, 1, 518400, 0xbab197ea +0, 590, 590, 1, 518400, 0xbab197ea +0, 591, 591, 1, 518400, 0xbab197ea +0, 592, 592, 1, 518400, 0xbab197ea +0, 593, 593, 1, 518400, 0xbab197ea +0, 594, 594, 1, 518400, 0xbab197ea +0, 595, 595, 1, 518400, 0xbab197ea +0, 596, 596, 1, 518400, 0xbab197ea +0, 597, 597, 1, 518400, 0xbab197ea +0, 598, 598, 1, 518400, 0xbab197ea +0, 599, 599, 1, 518400, 0xbab197ea +0, 600, 600, 1, 518400, 0xbab197ea +0, 601, 601, 1, 518400, 0xbab197ea +0, 602, 602, 1, 518400, 0xbab197ea +0, 603, 603, 1, 518400, 0xbab197ea +0, 604, 604, 1, 518400, 0xbab197ea +0, 605, 605, 1, 518400, 0xbab197ea +0, 606, 606, 1, 518400, 0xbab197ea +0, 607, 607, 1, 518400, 0xbab197ea +0, 608, 608, 1, 518400, 0xbab197ea +0, 609, 609, 1, 518400, 0xbab197ea +0, 610, 610, 1, 518400, 0xbab197ea +0, 611, 611, 1, 518400, 0xbab197ea +0, 612, 612, 1, 518400, 0xbab197ea +0, 613, 613, 1, 518400, 0xbab197ea +0, 614, 614, 1, 518400, 0xbab197ea +0, 615, 615, 1, 518400, 0xbab197ea +0, 616, 616, 1, 518400, 0xbab197ea +0, 617, 617, 1, 518400, 0xbab197ea +0, 618, 618, 1, 518400, 0xbab197ea +0, 619, 619, 1, 518400, 0xbab197ea +0, 620, 620, 1, 518400, 0xbab197ea +0, 621, 621, 1, 518400, 0xbab197ea +0, 622, 622, 1, 518400, 0xbab197ea +0, 623, 623, 1, 518400, 0xbab197ea +0, 624, 624, 1, 518400, 0xbab197ea +0, 625, 625, 1, 518400, 0xbab197ea +0, 626, 626, 1, 518400, 0xbab197ea +0, 627, 627, 1, 518400, 0xbab197ea +0, 628, 628, 1, 518400, 0xbab197ea +0, 629, 629, 1, 518400, 0xbab197ea +0, 630, 630, 1, 518400, 0xbab197ea +0, 631, 631, 1, 518400, 0xbab197ea +0, 632, 632, 1, 518400, 0xbab197ea +0, 633, 633, 1, 518400, 0xbab197ea +0, 634, 634, 1, 518400, 0xbab197ea +0, 635, 635, 1, 518400, 0xbab197ea +0, 636, 636, 1, 518400, 0xbab197ea +0, 637, 637, 1, 518400, 0xbab197ea +0, 638, 638, 1, 518400, 0xbab197ea +0, 639, 639, 1, 518400, 0xbab197ea +0, 640, 640, 1, 518400, 0xbab197ea +0, 641, 641, 1, 518400, 0xbab197ea +0, 642, 642, 1, 518400, 0xbab197ea +0, 643, 643, 1, 518400, 0xbab197ea +0, 644, 644, 1, 518400, 0xbab197ea +0, 645, 645, 1, 518400, 0xbab197ea +0, 646, 646, 1, 518400, 0xbab197ea +0, 647, 647, 1, 518400, 0xbab197ea +0, 648, 648, 1, 518400, 0xbab197ea +0, 649, 649, 1, 518400, 0xbab197ea +0, 650, 650, 1, 518400, 0xbab197ea +0, 651, 651, 1, 518400, 0xbab197ea +0, 652, 652, 1, 518400, 0xbab197ea +0, 653, 653, 1, 518400, 0xbab197ea +0, 654, 654, 1, 518400, 0xbab197ea +0, 655, 655, 1, 518400, 0xbab197ea +0, 656, 656, 1, 518400, 0xbab197ea +0, 657, 657, 1, 518400, 0xbab197ea +0, 658, 658, 1, 518400, 0xbab197ea +0, 659, 659, 1, 518400, 0xbab197ea +0, 660, 660, 1, 518400, 0xbab197ea +0, 661, 661, 1, 518400, 0xbab197ea +0, 662, 662, 1, 518400, 0xbab197ea +0, 663, 663, 1, 518400, 0xbab197ea +0, 664, 664, 1, 518400, 0xbab197ea +0, 665, 665, 1, 518400, 0xbab197ea +0, 666, 666, 1, 518400, 0xbab197ea +0, 667, 667, 1, 518400, 0xbab197ea +0, 668, 668, 1, 518400, 0xbab197ea +0, 669, 669, 1, 518400, 0xbab197ea +0, 670, 670, 1, 518400, 0xbab197ea +0, 671, 671, 1, 518400, 0xbab197ea +0, 672, 672, 1, 518400, 0xbab197ea +0, 673, 673, 1, 518400, 0xbab197ea +0, 674, 674, 1, 518400, 0xbab197ea +0, 675, 675, 1, 518400, 0xbab197ea +0, 676, 676, 1, 518400, 0xbab197ea +0, 677, 677, 1, 518400, 0xbab197ea +0, 678, 678, 1, 518400, 0xbab197ea +0, 679, 679, 1, 518400, 0xbab197ea +0, 680, 680, 1, 518400, 0xbab197ea +0, 681, 681, 1, 518400, 0xbab197ea +0, 682, 682, 1, 518400, 0xbab197ea +0, 683, 683, 1, 518400, 0xbab197ea +0, 684, 684, 1, 518400, 0xbab197ea +0, 685, 685, 1, 518400, 0xbab197ea +0, 686, 686, 1, 518400, 0xbab197ea +0, 687, 687, 1, 518400, 0xbab197ea +0, 688, 688, 1, 518400, 0xbab197ea +0, 689, 689, 1, 518400, 0xbab197ea +0, 690, 690, 1, 518400, 0xbab197ea +0, 691, 691, 1, 518400, 0xbab197ea +0, 692, 692, 1, 518400, 0xbab197ea +0, 693, 693, 1, 518400, 0xbab197ea +0, 694, 694, 1, 518400, 0xbab197ea +0, 695, 695, 1, 518400, 0xbab197ea +0, 696, 696, 1, 518400, 0xbab197ea +0, 697, 697, 1, 518400, 0xbab197ea +0, 698, 698, 1, 518400, 0xbab197ea +0, 699, 699, 1, 518400, 0xbab197ea +0, 700, 700, 1, 518400, 0xbab197ea +0, 701, 701, 1, 518400, 0xbab197ea +0, 702, 702, 1, 518400, 0xbab197ea +0, 703, 703, 1, 518400, 0xbab197ea +0, 704, 704, 1, 518400, 0xbab197ea +0, 705, 705, 1, 518400, 0xbab197ea +0, 706, 706, 1, 518400, 0xbab197ea +0, 707, 707, 1, 518400, 0xbab197ea 1, 141556000, 141556000, 1661000, 1088, 0xde20aa20 0, 708, 708, 1, 518400, 0xb8bc1365 -0, 716, 716, 1, 518400, 0xbab197ea +0, 709, 709, 1, 518400, 0xb8bc1365 +0, 710, 710, 1, 518400, 0xb8bc1365 +0, 711, 711, 1, 518400, 0xb8bc1365 +0, 712, 712, 1, 518400, 0xb8bc1365 +0, 713, 713, 1, 518400, 0xb8bc1365 +0, 714, 714, 1, 518400, 0xb8bc1365 +0, 715, 715, 1, 518400, 0xb8bc1365 +0, 716, 716, 1, 518400, 0xb8bc1365 +0, 717, 717, 1, 518400, 0xbab197ea +0, 718, 718, 1, 518400, 0xbab197ea +0, 719, 719, 1, 518400, 0xbab197ea +0, 720, 720, 1, 518400, 0xbab197ea +0, 721, 721, 1, 518400, 0xbab197ea +0, 722, 722, 1, 518400, 0xbab197ea +0, 723, 723, 1, 518400, 0xbab197ea +0, 724, 724, 1, 518400, 0xbab197ea +0, 725, 725, 1, 518400, 0xbab197ea +0, 726, 726, 1, 518400, 0xbab197ea +0, 727, 727, 1, 518400, 0xbab197ea +0, 728, 728, 1, 518400, 0xbab197ea +0, 729, 729, 1, 518400, 0xbab197ea +0, 730, 730, 1, 518400, 0xbab197ea +0, 731, 731, 1, 518400, 0xbab197ea +0, 732, 732, 1, 518400, 0xbab197ea +0, 733, 733, 1, 518400, 0xbab197ea +0, 734, 734, 1, 518400, 0xbab197ea +0, 735, 735, 1, 518400, 0xbab197ea +0, 736, 736, 1, 518400, 0xbab197ea +0, 737, 737, 1, 518400, 0xbab197ea +0, 738, 738, 1, 518400, 0xbab197ea +0, 739, 739, 1, 518400, 0xbab197ea +0, 740, 740, 1, 518400, 0xbab197ea +0, 741, 741, 1, 518400, 0xbab197ea +0, 742, 742, 1, 518400, 0xbab197ea +0, 743, 743, 1, 518400, 0xbab197ea +0, 744, 744, 1, 518400, 0xbab197ea +0, 745, 745, 1, 518400, 0xbab197ea +0, 746, 746, 1, 518400, 0xbab197ea +0, 747, 747, 1, 518400, 0xbab197ea +0, 748, 748, 1, 518400, 0xbab197ea +0, 749, 749, 1, 518400, 0xbab197ea +0, 750, 750, 1, 518400, 0xbab197ea +0, 751, 751, 1, 518400, 0xbab197ea +0, 752, 752, 1, 518400, 0xbab197ea +0, 753, 753, 1, 518400, 0xbab197ea +0, 754, 754, 1, 518400, 0xbab197ea +0, 755, 755, 1, 518400, 0xbab197ea +0, 756, 756, 1, 518400, 0xbab197ea +0, 757, 757, 1, 518400, 0xbab197ea +0, 758, 758, 1, 518400, 0xbab197ea +0, 759, 759, 1, 518400, 0xbab197ea +0, 760, 760, 1, 518400, 0xbab197ea +0, 761, 761, 1, 518400, 0xbab197ea +0, 762, 762, 1, 518400, 0xbab197ea +0, 763, 763, 1, 518400, 0xbab197ea +0, 764, 764, 1, 518400, 0xbab197ea +0, 765, 765, 1, 518400, 0xbab197ea +0, 766, 766, 1, 518400, 0xbab197ea +0, 767, 767, 1, 518400, 0xbab197ea +0, 768, 768, 1, 518400, 0xbab197ea +0, 769, 769, 1, 518400, 0xbab197ea +0, 770, 770, 1, 518400, 0xbab197ea +0, 771, 771, 1, 518400, 0xbab197ea +0, 772, 772, 1, 518400, 0xbab197ea +0, 773, 773, 1, 518400, 0xbab197ea +0, 774, 774, 1, 518400, 0xbab197ea +0, 775, 775, 1, 518400, 0xbab197ea +0, 776, 776, 1, 518400, 0xbab197ea +0, 777, 777, 1, 518400, 0xbab197ea +0, 778, 778, 1, 518400, 0xbab197ea +0, 779, 779, 1, 518400, 0xbab197ea +0, 780, 780, 1, 518400, 0xbab197ea +0, 781, 781, 1, 518400, 0xbab197ea +0, 782, 782, 1, 518400, 0xbab197ea +0, 783, 783, 1, 518400, 0xbab197ea +0, 784, 784, 1, 518400, 0xbab197ea +0, 785, 785, 1, 518400, 0xbab197ea +0, 786, 786, 1, 518400, 0xbab197ea +0, 787, 787, 1, 518400, 0xbab197ea +0, 788, 788, 1, 518400, 0xbab197ea +0, 789, 789, 1, 518400, 0xbab197ea +0, 790, 790, 1, 518400, 0xbab197ea +0, 791, 791, 1, 518400, 0xbab197ea +0, 792, 792, 1, 518400, 0xbab197ea +0, 793, 793, 1, 518400, 0xbab197ea +0, 794, 794, 1, 518400, 0xbab197ea +0, 795, 795, 1, 518400, 0xbab197ea +0, 796, 796, 1, 518400, 0xbab197ea +0, 797, 797, 1, 518400, 0xbab197ea +0, 798, 798, 1, 518400, 0xbab197ea +0, 799, 799, 1, 518400, 0xbab197ea +0, 800, 800, 1, 518400, 0xbab197ea +0, 801, 801, 1, 518400, 0xbab197ea +0, 802, 802, 1, 518400, 0xbab197ea +0, 803, 803, 1, 518400, 0xbab197ea +0, 804, 804, 1, 518400, 0xbab197ea +0, 805, 805, 1, 518400, 0xbab197ea +0, 806, 806, 1, 518400, 0xbab197ea +0, 807, 807, 1, 518400, 0xbab197ea +0, 808, 808, 1, 518400, 0xbab197ea +0, 809, 809, 1, 518400, 0xbab197ea +0, 810, 810, 1, 518400, 0xbab197ea +0, 811, 811, 1, 518400, 0xbab197ea +0, 812, 812, 1, 518400, 0xbab197ea +0, 813, 813, 1, 518400, 0xbab197ea +0, 814, 814, 1, 518400, 0xbab197ea +0, 815, 815, 1, 518400, 0xbab197ea +0, 816, 816, 1, 518400, 0xbab197ea 0, 817, 817, 1, 518400, 0x83efa32d 1, 163445000, 163445000, 1331000, 339, 0x8bd186ef -0, 824, 824, 1, 518400, 0xbab197ea +0, 818, 818, 1, 518400, 0x83efa32d +0, 819, 819, 1, 518400, 0x83efa32d +0, 820, 820, 1, 518400, 0x83efa32d +0, 821, 821, 1, 518400, 0x83efa32d +0, 822, 822, 1, 518400, 0x83efa32d +0, 823, 823, 1, 518400, 0x83efa32d +0, 824, 824, 1, 518400, 0x83efa32d +0, 825, 825, 1, 518400, 0xbab197ea +0, 826, 826, 1, 518400, 0xbab197ea +0, 827, 827, 1, 518400, 0xbab197ea +0, 828, 828, 1, 518400, 0xbab197ea +0, 829, 829, 1, 518400, 0xbab197ea +0, 830, 830, 1, 518400, 0xbab197ea +0, 831, 831, 1, 518400, 0xbab197ea +0, 832, 832, 1, 518400, 0xbab197ea +0, 833, 833, 1, 518400, 0xbab197ea +0, 834, 834, 1, 518400, 0xbab197ea +0, 835, 835, 1, 518400, 0xbab197ea +0, 836, 836, 1, 518400, 0xbab197ea +0, 837, 837, 1, 518400, 0xbab197ea +0, 838, 838, 1, 518400, 0xbab197ea +0, 839, 839, 1, 518400, 0xbab197ea 0, 840, 840, 1, 518400, 0x03ea0e90 1, 168049000, 168049000, 1900000, 1312, 0x0bf20e8d -0, 850, 850, 1, 518400, 0xbab197ea +0, 841, 841, 1, 518400, 0x03ea0e90 +0, 842, 842, 1, 518400, 0x03ea0e90 +0, 843, 843, 1, 518400, 0x03ea0e90 +0, 844, 844, 1, 518400, 0x03ea0e90 +0, 845, 845, 1, 518400, 0x03ea0e90 +0, 846, 846, 1, 518400, 0x03ea0e90 +0, 847, 847, 1, 518400, 0x03ea0e90 +0, 848, 848, 1, 518400, 0x03ea0e90 +0, 849, 849, 1, 518400, 0x03ea0e90 +0, 850, 850, 1, 518400, 0x8780239e 1, 170035000, 170035000, 1524000, 1279, 0xb6c2dafe 0, 851, 851, 1, 518400, 0x8780239e -0, 858, 858, 1, 518400, 0xbab197ea +0, 852, 852, 1, 518400, 0x8780239e +0, 853, 853, 1, 518400, 0x8780239e +0, 854, 854, 1, 518400, 0x8780239e +0, 855, 855, 1, 518400, 0x8780239e +0, 856, 856, 1, 518400, 0x8780239e +0, 857, 857, 1, 518400, 0x8780239e +0, 858, 858, 1, 518400, 0x8780239e +0, 859, 859, 1, 518400, 0xbab197ea +0, 860, 860, 1, 518400, 0xbab197ea 0, 861, 861, 1, 518400, 0x6eb72347 1, 172203000, 172203000, 1695000, 1826, 0x9a1ac769 -0, 869, 869, 1, 518400, 0xbab197ea +0, 862, 862, 1, 518400, 0x6eb72347 +0, 863, 863, 1, 518400, 0x6eb72347 +0, 864, 864, 1, 518400, 0x6eb72347 +0, 865, 865, 1, 518400, 0x6eb72347 +0, 866, 866, 1, 518400, 0x6eb72347 +0, 867, 867, 1, 518400, 0x6eb72347 +0, 868, 868, 1, 518400, 0x6eb72347 +0, 869, 869, 1, 518400, 0x6eb72347 1, 173947000, 173947000, 1934000, 1474, 0xa9b03cdc 0, 870, 870, 1, 518400, 0x9c4a3a3d -0, 879, 879, 1, 518400, 0xbab197ea +0, 871, 871, 1, 518400, 0x9c4a3a3d +0, 872, 872, 1, 518400, 0x9c4a3a3d +0, 873, 873, 1, 518400, 0x9c4a3a3d +0, 874, 874, 1, 518400, 0x9c4a3a3d +0, 875, 875, 1, 518400, 0x9c4a3a3d +0, 876, 876, 1, 518400, 0x9c4a3a3d +0, 877, 877, 1, 518400, 0x9c4a3a3d +0, 878, 878, 1, 518400, 0x9c4a3a3d +0, 879, 879, 1, 518400, 0x9c4a3a3d 1, 175957000, 175957000, 1763000, 1019, 0x20409355 0, 880, 880, 1, 518400, 0xc9ebfa89 -0, 889, 889, 1, 518400, 0xbab197ea +0, 881, 881, 1, 518400, 0xc9ebfa89 +0, 882, 882, 1, 518400, 0xc9ebfa89 +0, 883, 883, 1, 518400, 0xc9ebfa89 +0, 884, 884, 1, 518400, 0xc9ebfa89 +0, 885, 885, 1, 518400, 0xc9ebfa89 +0, 886, 886, 1, 518400, 0xc9ebfa89 +0, 887, 887, 1, 518400, 0xc9ebfa89 +0, 888, 888, 1, 518400, 0xc9ebfa89 +0, 889, 889, 1, 518400, 0xc9ebfa89 +0, 890, 890, 1, 518400, 0xbab197ea +0, 891, 891, 1, 518400, 0xbab197ea +0, 892, 892, 1, 518400, 0xbab197ea +0, 893, 893, 1, 518400, 0xbab197ea +0, 894, 894, 1, 518400, 0xbab197ea +0, 895, 895, 1, 518400, 0xbab197ea +0, 896, 896, 1, 518400, 0xbab197ea +0, 897, 897, 1, 518400, 0xbab197ea +0, 898, 898, 1, 518400, 0xbab197ea +0, 899, 899, 1, 518400, 0xbab197ea +0, 900, 900, 1, 518400, 0xbab197ea +0, 901, 901, 1, 518400, 0xbab197ea +0, 902, 902, 1, 518400, 0xbab197ea +0, 903, 903, 1, 518400, 0xbab197ea +0, 904, 904, 1, 518400, 0xbab197ea +0, 905, 905, 1, 518400, 0xbab197ea +0, 906, 906, 1, 518400, 0xbab197ea +0, 907, 907, 1, 518400, 0xbab197ea +0, 908, 908, 1, 518400, 0xbab197ea +0, 909, 909, 1, 518400, 0xbab197ea +0, 910, 910, 1, 518400, 0xbab197ea +0, 911, 911, 1, 518400, 0xbab197ea +0, 912, 912, 1, 518400, 0xbab197ea +0, 913, 913, 1, 518400, 0xbab197ea +0, 914, 914, 1, 518400, 0xbab197ea +0, 915, 915, 1, 518400, 0xbab197ea +0, 916, 916, 1, 518400, 0xbab197ea +0, 917, 917, 1, 518400, 0xbab197ea +0, 918, 918, 1, 518400, 0xbab197ea +0, 919, 919, 1, 518400, 0xbab197ea +0, 920, 920, 1, 518400, 0xbab197ea +0, 921, 921, 1, 518400, 0xbab197ea +0, 922, 922, 1, 518400, 0xbab197ea +0, 923, 923, 1, 518400, 0xbab197ea +0, 924, 924, 1, 518400, 0xbab197ea +0, 925, 925, 1, 518400, 0xbab197ea +0, 926, 926, 1, 518400, 0xbab197ea +0, 927, 927, 1, 518400, 0xbab197ea +0, 928, 928, 1, 518400, 0xbab197ea +0, 929, 929, 1, 518400, 0xbab197ea +0, 930, 930, 1, 518400, 0xbab197ea +0, 931, 931, 1, 518400, 0xbab197ea +0, 932, 932, 1, 518400, 0xbab197ea +0, 933, 933, 1, 518400, 0xbab197ea +0, 934, 934, 1, 518400, 0xbab197ea +0, 935, 935, 1, 518400, 0xbab197ea +0, 936, 936, 1, 518400, 0xbab197ea +0, 937, 937, 1, 518400, 0xbab197ea +0, 938, 938, 1, 518400, 0xbab197ea +0, 939, 939, 1, 518400, 0xbab197ea +0, 940, 940, 1, 518400, 0xbab197ea +0, 941, 941, 1, 518400, 0xbab197ea +0, 942, 942, 1, 518400, 0xbab197ea +0, 943, 943, 1, 518400, 0xbab197ea +0, 944, 944, 1, 518400, 0xbab197ea +0, 945, 945, 1, 518400, 0xbab197ea 0, 946, 946, 1, 518400, 0xbaf801ef 1, 189295000, 189295000, 1968000, 1596, 0x408c726e -0, 956, 956, 1, 518400, 0xbab197ea +0, 947, 947, 1, 518400, 0xbaf801ef +0, 948, 948, 1, 518400, 0xbaf801ef +0, 949, 949, 1, 518400, 0xbaf801ef +0, 950, 950, 1, 518400, 0xbaf801ef +0, 951, 951, 1, 518400, 0xbaf801ef +0, 952, 952, 1, 518400, 0xbaf801ef +0, 953, 953, 1, 518400, 0xbaf801ef +0, 954, 954, 1, 518400, 0xbaf801ef +0, 955, 955, 1, 518400, 0xbaf801ef +0, 956, 956, 1, 518400, 0xbaf801ef 1, 191356000, 191356000, 1228000, 1517, 0xae8c5c2b 0, 957, 957, 1, 518400, 0x59f4e72f -0, 963, 963, 1, 518400, 0xbab197ea +0, 958, 958, 1, 518400, 0x59f4e72f +0, 959, 959, 1, 518400, 0x59f4e72f +0, 960, 960, 1, 518400, 0x59f4e72f +0, 961, 961, 1, 518400, 0x59f4e72f +0, 962, 962, 1, 518400, 0x59f4e72f +0, 963, 963, 1, 518400, 0x9d5b9d69 1, 192640000, 192640000, 1763000, 2506, 0xa458d6d4 0, 964, 964, 1, 518400, 0x9d5b9d69 -0, 972, 972, 1, 518400, 0xbab197ea +0, 965, 965, 1, 518400, 0x9d5b9d69 +0, 966, 966, 1, 518400, 0x9d5b9d69 +0, 967, 967, 1, 518400, 0x9d5b9d69 +0, 968, 968, 1, 518400, 0x9d5b9d69 +0, 969, 969, 1, 518400, 0x9d5b9d69 +0, 970, 970, 1, 518400, 0x9d5b9d69 +0, 971, 971, 1, 518400, 0x9d5b9d69 +0, 972, 972, 1, 518400, 0x9d5b9d69 +0, 973, 973, 1, 518400, 0xbab197ea +0, 974, 974, 1, 518400, 0xbab197ea +0, 975, 975, 1, 518400, 0xbab197ea 1, 195193000, 195193000, 1092000, 1074, 0x397ba9a8 0, 976, 976, 1, 518400, 0x923d1ce7 -0, 981, 981, 1, 518400, 0xbab197ea +0, 977, 977, 1, 518400, 0x923d1ce7 +0, 978, 978, 1, 518400, 0x923d1ce7 +0, 979, 979, 1, 518400, 0x923d1ce7 +0, 980, 980, 1, 518400, 0x923d1ce7 +0, 981, 981, 1, 518400, 0x923d1ce7 1, 196361000, 196361000, 1524000, 1715, 0x695ca41e 0, 982, 982, 1, 518400, 0x6e652cd2 -0, 989, 989, 1, 518400, 0xbab197ea +0, 983, 983, 1, 518400, 0x6e652cd2 +0, 984, 984, 1, 518400, 0x6e652cd2 +0, 985, 985, 1, 518400, 0x6e652cd2 +0, 986, 986, 1, 518400, 0x6e652cd2 +0, 987, 987, 1, 518400, 0x6e652cd2 +0, 988, 988, 1, 518400, 0x6e652cd2 +0, 989, 989, 1, 518400, 0x6e652cd2 1, 197946000, 197946000, 1160000, 789, 0xc63a189e 0, 990, 990, 1, 518400, 0x25113966 -0, 996, 996, 1, 518400, 0xbab197ea +0, 991, 991, 1, 518400, 0x25113966 +0, 992, 992, 1, 518400, 0x25113966 +0, 993, 993, 1, 518400, 0x25113966 +0, 994, 994, 1, 518400, 0x25113966 +0, 995, 995, 1, 518400, 0x25113966 +0, 996, 996, 1, 518400, 0x2dc83609 1, 199230000, 199230000, 1627000, 1846, 0xeea8c599 0, 997, 997, 1, 518400, 0x2dc83609 -0, 1004, 1004, 1, 518400, 0xbab197ea +0, 998, 998, 1, 518400, 0x2dc83609 +0, 999, 999, 1, 518400, 0x2dc83609 +0, 1000, 1000, 1, 518400, 0x2dc83609 +0, 1001, 1001, 1, 518400, 0x2dc83609 +0, 1002, 1002, 1, 518400, 0x2dc83609 +0, 1003, 1003, 1, 518400, 0x2dc83609 +0, 1004, 1004, 1, 518400, 0x2dc83609 1, 200924000, 200924000, 1763000, 922, 0xd4a87222 0, 1005, 1005, 1, 518400, 0x90483bc6 -0, 1013, 1013, 1, 518400, 0xbab197ea +0, 1006, 1006, 1, 518400, 0x90483bc6 +0, 1007, 1007, 1, 518400, 0x90483bc6 +0, 1008, 1008, 1, 518400, 0x90483bc6 +0, 1009, 1009, 1, 518400, 0x90483bc6 +0, 1010, 1010, 1, 518400, 0x90483bc6 +0, 1011, 1011, 1, 518400, 0x90483bc6 +0, 1012, 1012, 1, 518400, 0x90483bc6 +0, 1013, 1013, 1, 518400, 0x90483bc6 +0, 1014, 1014, 1, 518400, 0xbab197ea +0, 1015, 1015, 1, 518400, 0xbab197ea +0, 1016, 1016, 1, 518400, 0xbab197ea +0, 1017, 1017, 1, 518400, 0xbab197ea +0, 1018, 1018, 1, 518400, 0xbab197ea +0, 1019, 1019, 1, 518400, 0xbab197ea +0, 1020, 1020, 1, 518400, 0xbab197ea +0, 1021, 1021, 1, 518400, 0xbab197ea +0, 1022, 1022, 1, 518400, 0xbab197ea +0, 1023, 1023, 1, 518400, 0xbab197ea +0, 1024, 1024, 1, 518400, 0xbab197ea +0, 1025, 1025, 1, 518400, 0xbab197ea +0, 1026, 1026, 1, 518400, 0xbab197ea +0, 1027, 1027, 1, 518400, 0xbab197ea +0, 1028, 1028, 1, 518400, 0xbab197ea +0, 1029, 1029, 1, 518400, 0xbab197ea +0, 1030, 1030, 1, 518400, 0xbab197ea +0, 1031, 1031, 1, 518400, 0xbab197ea +0, 1032, 1032, 1, 518400, 0xbab197ea +0, 1033, 1033, 1, 518400, 0xbab197ea +0, 1034, 1034, 1, 518400, 0xbab197ea +0, 1035, 1035, 1, 518400, 0xbab197ea +0, 1036, 1036, 1, 518400, 0xbab197ea +0, 1037, 1037, 1, 518400, 0xbab197ea +0, 1038, 1038, 1, 518400, 0xbab197ea +0, 1039, 1039, 1, 518400, 0xbab197ea +0, 1040, 1040, 1, 518400, 0xbab197ea +0, 1041, 1041, 1, 518400, 0xbab197ea +0, 1042, 1042, 1, 518400, 0xbab197ea +0, 1043, 1043, 1, 518400, 0xbab197ea +0, 1044, 1044, 1, 518400, 0xbab197ea +0, 1045, 1045, 1, 518400, 0xbab197ea +0, 1046, 1046, 1, 518400, 0xbab197ea +0, 1047, 1047, 1, 518400, 0xbab197ea +0, 1048, 1048, 1, 518400, 0xbab197ea +0, 1049, 1049, 1, 518400, 0xbab197ea +0, 1050, 1050, 1, 518400, 0xbab197ea +0, 1051, 1051, 1, 518400, 0xbab197ea +0, 1052, 1052, 1, 518400, 0xbab197ea 0, 1053, 1053, 1, 518400, 0x3de86ab7 1, 210600000, 210600000, 1831000, 665, 0x55580135 -0, 1062, 1062, 1, 518400, 0xbab197ea +0, 1054, 1054, 1, 518400, 0x3de86ab7 +0, 1055, 1055, 1, 518400, 0x3de86ab7 +0, 1056, 1056, 1, 518400, 0x3de86ab7 +0, 1057, 1057, 1, 518400, 0x3de86ab7 +0, 1058, 1058, 1, 518400, 0x3de86ab7 +0, 1059, 1059, 1, 518400, 0x3de86ab7 +0, 1060, 1060, 1, 518400, 0x3de86ab7 +0, 1061, 1061, 1, 518400, 0x3de86ab7 +0, 1062, 1062, 1, 518400, 0x3de86ab7 +0, 1063, 1063, 1, 518400, 0xbab197ea +0, 1064, 1064, 1, 518400, 0xbab197ea +0, 1065, 1065, 1, 518400, 0xbab197ea +0, 1066, 1066, 1, 518400, 0xbab197ea +0, 1067, 1067, 1, 518400, 0xbab197ea +0, 1068, 1068, 1, 518400, 0xbab197ea +0, 1069, 1069, 1, 518400, 0xbab197ea +0, 1070, 1070, 1, 518400, 0xbab197ea +0, 1071, 1071, 1, 518400, 0xbab197ea +0, 1072, 1072, 1, 518400, 0xbab197ea +0, 1073, 1073, 1, 518400, 0xbab197ea 1, 214771000, 214771000, 1558000, 1216, 0x50d1f6c5 0, 1074, 1074, 1, 518400, 0x8c320e68 -0, 1082, 1082, 1, 518400, 0xbab197ea +0, 1075, 1075, 1, 518400, 0x8c320e68 +0, 1076, 1076, 1, 518400, 0x8c320e68 +0, 1077, 1077, 1, 518400, 0x8c320e68 +0, 1078, 1078, 1, 518400, 0x8c320e68 +0, 1079, 1079, 1, 518400, 0x8c320e68 +0, 1080, 1080, 1, 518400, 0x8c320e68 +0, 1081, 1081, 1, 518400, 0x8c320e68 +0, 1082, 1082, 1, 518400, 0x8c320e68 +0, 1083, 1083, 1, 518400, 0xbab197ea +0, 1084, 1084, 1, 518400, 0xbab197ea +0, 1085, 1085, 1, 518400, 0xbab197ea +0, 1086, 1086, 1, 518400, 0xbab197ea +0, 1087, 1087, 1, 518400, 0xbab197ea +0, 1088, 1088, 1, 518400, 0xbab197ea +0, 1089, 1089, 1, 518400, 0xbab197ea +0, 1090, 1090, 1, 518400, 0xbab197ea +0, 1091, 1091, 1, 518400, 0xbab197ea +0, 1092, 1092, 1, 518400, 0xbab197ea +0, 1093, 1093, 1, 518400, 0xbab197ea +0, 1094, 1094, 1, 518400, 0xbab197ea +0, 1095, 1095, 1, 518400, 0xbab197ea +0, 1096, 1096, 1, 518400, 0xbab197ea +0, 1097, 1097, 1, 518400, 0xbab197ea +0, 1098, 1098, 1, 518400, 0xbab197ea +0, 1099, 1099, 1, 518400, 0xbab197ea +0, 1100, 1100, 1, 518400, 0xbab197ea +0, 1101, 1101, 1, 518400, 0xbab197ea +0, 1102, 1102, 1, 518400, 0xbab197ea +0, 1103, 1103, 1, 518400, 0xbab197ea +0, 1104, 1104, 1, 518400, 0xbab197ea +0, 1105, 1105, 1, 518400, 0xbab197ea +0, 1106, 1106, 1, 518400, 0xbab197ea +0, 1107, 1107, 1, 518400, 0xbab197ea +0, 1108, 1108, 1, 518400, 0xbab197ea +0, 1109, 1109, 1, 518400, 0xbab197ea +0, 1110, 1110, 1, 518400, 0xbab197ea +0, 1111, 1111, 1, 518400, 0xbab197ea +0, 1112, 1112, 1, 518400, 0xbab197ea +0, 1113, 1113, 1, 518400, 0xbab197ea +0, 1114, 1114, 1, 518400, 0xbab197ea +0, 1115, 1115, 1, 518400, 0xbab197ea +0, 1116, 1116, 1, 518400, 0xbab197ea +0, 1117, 1117, 1, 518400, 0xbab197ea +0, 1118, 1118, 1, 518400, 0xbab197ea +0, 1119, 1119, 1, 518400, 0xbab197ea +0, 1120, 1120, 1, 518400, 0xbab197ea +0, 1121, 1121, 1, 518400, 0xbab197ea +0, 1122, 1122, 1, 518400, 0xbab197ea +0, 1123, 1123, 1, 518400, 0xbab197ea +0, 1124, 1124, 1, 518400, 0xbab197ea +0, 1125, 1125, 1, 518400, 0xbab197ea +0, 1126, 1126, 1, 518400, 0xbab197ea +0, 1127, 1127, 1, 518400, 0xbab197ea 0, 1128, 1128, 1, 518400, 0x81e977b2 1, 225640000, 225640000, 2127000, 2133, 0x670c11a5 -0, 1139, 1139, 1, 518400, 0xbab197ea +0, 1129, 1129, 1, 518400, 0x81e977b2 +0, 1130, 1130, 1, 518400, 0x81e977b2 +0, 1131, 1131, 1, 518400, 0x81e977b2 +0, 1132, 1132, 1, 518400, 0x81e977b2 +0, 1133, 1133, 1, 518400, 0x81e977b2 +0, 1134, 1134, 1, 518400, 0x81e977b2 +0, 1135, 1135, 1, 518400, 0x81e977b2 +0, 1136, 1136, 1, 518400, 0x81e977b2 +0, 1137, 1137, 1, 518400, 0x81e977b2 +0, 1138, 1138, 1, 518400, 0x81e977b2 +0, 1139, 1139, 1, 518400, 0xb046dd30 1, 227834000, 227834000, 1262000, 1264, 0xc1d9fc57 0, 1140, 1140, 1, 518400, 0xb046dd30 -0, 1145, 1145, 1, 518400, 0xbab197ea +0, 1141, 1141, 1, 518400, 0xb046dd30 diff --git a/tests/ref/fate/sub2video_basic b/tests/ref/fate/sub2video_basic index 5f72e292c9..9bd4e8b114 100644 --- a/tests/ref/fate/sub2video_basic +++ b/tests/ref/fate/sub2video_basic @@ -1,95 +1,1149 @@ -#tb 0: 1/25 +#tb 0: 1/5 #media_type 0: video #codec_id 0: rawvideo #dimensions 0: 720x480 -#sar 0: 0/1 -0, 3312, 3312, 1, 1382400, 0x00000000 -0, 3312, 3312, 1, 1382400, 0x8c93c2ba -0, 3436, 3436, 1, 1382400, 0x00000000 -0, 3684, 3684, 1, 1382400, 0xb02e32ca -0, 3802, 3802, 1, 1382400, 0x00000000 -0, 4520, 4520, 1, 1382400, 0x83b71116 -0, 4584, 4584, 1, 1382400, 0x00000000 -0, 4586, 4586, 1, 1382400, 0x85547fd1 -0, 4645, 4645, 1, 1382400, 0x00000000 -0, 4648, 4648, 1, 1382400, 0x00000000 -0, 4648, 4648, 1, 1382400, 0xb6a8f181 -0, 4715, 4715, 1, 1382400, 0x00000000 -0, 4717, 4717, 1, 1382400, 0xb64d1a2c -0, 4748, 4748, 1, 1382400, 0x00000000 -0, 4750, 4750, 1, 1382400, 0x7b37ecf3 -0, 4792, 4792, 1, 1382400, 0x00000000 -0, 4993, 4993, 1, 1382400, 0xdc025bd1 -0, 5027, 5027, 1, 1382400, 0x00000000 -0, 5029, 5029, 1, 1382400, 0x688b294d -0, 5068, 5068, 1, 1382400, 0x00000000 -0, 5070, 5070, 1, 1382400, 0xa2b33d1b -0, 5117, 5117, 1, 1382400, 0x00000000 -0, 5119, 5119, 1, 1382400, 0xb3e525e3 -0, 5168, 5168, 1, 1382400, 0x00000000 -0, 5170, 5170, 1, 1382400, 0xaa8fbdd7 -0, 5216, 5216, 1, 1382400, 0x00000000 -0, 5218, 5218, 1, 1382400, 0x7b7f26dd -0, 5249, 5249, 1, 1382400, 0x00000000 -0, 5251, 5251, 1, 1382400, 0x15e2f836 -0, 5289, 5289, 1, 1382400, 0x00000000 -0, 5291, 5291, 1, 1382400, 0x0fee9b0c -0, 5358, 5358, 1, 1382400, 0x00000000 -0, 5360, 5360, 1, 1382400, 0x89d62791 -0, 5429, 5429, 1, 1382400, 0x00000000 -0, 5431, 5431, 1, 1382400, 0xa6a9fd74 -0, 5490, 5490, 1, 1382400, 0x00000000 -0, 5491, 5491, 1, 1382400, 0x7896178d -0, 5537, 5537, 1, 1382400, 0x00000000 -0, 5588, 5588, 1, 1382400, 0x01751a52 -0, 5647, 5647, 1, 1382400, 0x00000000 -0, 5688, 5688, 1, 1382400, 0xa3959c6f -0, 5770, 5770, 1, 1382400, 0x00000000 -0, 5772, 5772, 1, 1382400, 0x3d3ea47b -0, 5826, 5826, 1, 1382400, 0x00000000 -0, 5828, 5828, 1, 1382400, 0x593f8b24 -0, 5931, 5931, 1, 1382400, 0x00000000 -0, 5933, 5933, 1, 1382400, 0x171f05ba -0, 6001, 6001, 1, 1382400, 0x00000000 -0, 6003, 6003, 1, 1382400, 0xb014cdf1 -0, 6054, 6054, 1, 1382400, 0x00000000 -0, 6839, 6839, 1, 1382400, 0xd918e667 -0, 6880, 6880, 1, 1382400, 0x00000000 -0, 7386, 7386, 1, 1382400, 0xc9406331 -0, 7419, 7419, 1, 1382400, 0x00000000 -0, 7501, 7501, 1, 1382400, 0xaf08b10d -0, 7549, 7549, 1, 1382400, 0x00000000 -0, 7551, 7551, 1, 1382400, 0x00000000 -0, 7551, 7551, 1, 1382400, 0x853a9d93 -0, 7589, 7589, 1, 1382400, 0x00000000 -0, 7605, 7605, 1, 1382400, 0x7491a87d -0, 7647, 7647, 1, 1382400, 0x00000000 -0, 7649, 7649, 1, 1382400, 0xf7383c58 -0, 7697, 7697, 1, 1382400, 0x00000000 -0, 7699, 7699, 1, 1382400, 0xe66be411 -0, 7743, 7743, 1, 1382400, 0x00000000 -0, 8032, 8032, 1, 1382400, 0xd6850362 -0, 8082, 8082, 1, 1382400, 0x00000000 -0, 8084, 8084, 1, 1382400, 0x3e1ed109 -0, 8115, 8115, 1, 1382400, 0x00000000 -0, 8116, 8116, 1, 1382400, 0x39c1b7bd -0, 8160, 8160, 1, 1382400, 0x00000000 -0, 8180, 8180, 1, 1382400, 0x35b85f2e -0, 8207, 8207, 1, 1382400, 0x00000000 -0, 8209, 8209, 1, 1382400, 0x00000000 -0, 8209, 8209, 1, 1382400, 0x83f103e5 -0, 8247, 8247, 1, 1382400, 0x00000000 -0, 8249, 8249, 1, 1382400, 0xbc1ca9b3 -0, 8278, 8278, 1, 1382400, 0x00000000 -0, 8281, 8281, 1, 1382400, 0x94d4a51e -0, 8321, 8321, 1, 1382400, 0x00000000 -0, 8323, 8323, 1, 1382400, 0xf88cdfde -0, 8367, 8367, 1, 1382400, 0x00000000 -0, 8565, 8565, 1, 1382400, 0xdd51423b -0, 8611, 8611, 1, 1382400, 0x00000000 -0, 8669, 8669, 1, 1382400, 0x08259fa4 -0, 8708, 8708, 1, 1382400, 0x00000000 -0, 8941, 8941, 1, 1382400, 0x1663fa34 -0, 8994, 8994, 1, 1382400, 0x00000000 -0, 8996, 8996, 1, 1382400, 0xda2ceb55 -0, 9027, 9027, 1, 1382400, 0x00000000 +#sar 0: 1/1 +0, 662, 662, 1, 1382400, 0xc637b893 +0, 663, 663, 1, 1382400, 0xc637b893 +0, 664, 664, 1, 1382400, 0xc637b893 +0, 665, 665, 1, 1382400, 0xc637b893 +0, 666, 666, 1, 1382400, 0xc637b893 +0, 667, 667, 1, 1382400, 0xc637b893 +0, 668, 668, 1, 1382400, 0xc637b893 +0, 669, 669, 1, 1382400, 0xc637b893 +0, 670, 670, 1, 1382400, 0xc637b893 +0, 671, 671, 1, 1382400, 0xc637b893 +0, 672, 672, 1, 1382400, 0xc637b893 +0, 673, 673, 1, 1382400, 0xc637b893 +0, 674, 674, 1, 1382400, 0xc637b893 +0, 675, 675, 1, 1382400, 0xc637b893 +0, 676, 676, 1, 1382400, 0xc637b893 +0, 677, 677, 1, 1382400, 0xc637b893 +0, 678, 678, 1, 1382400, 0xc637b893 +0, 679, 679, 1, 1382400, 0xc637b893 +0, 680, 680, 1, 1382400, 0xc637b893 +0, 681, 681, 1, 1382400, 0xc637b893 +0, 682, 682, 1, 1382400, 0xc637b893 +0, 683, 683, 1, 1382400, 0xc637b893 +0, 684, 684, 1, 1382400, 0xc637b893 +0, 685, 685, 1, 1382400, 0xc637b893 +0, 686, 686, 1, 1382400, 0xc637b893 +0, 687, 687, 1, 1382400, 0x00000000 +0, 688, 688, 1, 1382400, 0x00000000 +0, 689, 689, 1, 1382400, 0x00000000 +0, 690, 690, 1, 1382400, 0x00000000 +0, 691, 691, 1, 1382400, 0x00000000 +0, 692, 692, 1, 1382400, 0x00000000 +0, 693, 693, 1, 1382400, 0x00000000 +0, 694, 694, 1, 1382400, 0x00000000 +0, 695, 695, 1, 1382400, 0x00000000 +0, 696, 696, 1, 1382400, 0x00000000 +0, 697, 697, 1, 1382400, 0x00000000 +0, 698, 698, 1, 1382400, 0x00000000 +0, 699, 699, 1, 1382400, 0x00000000 +0, 700, 700, 1, 1382400, 0x00000000 +0, 701, 701, 1, 1382400, 0x00000000 +0, 702, 702, 1, 1382400, 0x00000000 +0, 703, 703, 1, 1382400, 0x00000000 +0, 704, 704, 1, 1382400, 0x00000000 +0, 705, 705, 1, 1382400, 0x00000000 +0, 706, 706, 1, 1382400, 0x00000000 +0, 707, 707, 1, 1382400, 0x00000000 +0, 708, 708, 1, 1382400, 0x00000000 +0, 709, 709, 1, 1382400, 0x00000000 +0, 710, 710, 1, 1382400, 0x00000000 +0, 711, 711, 1, 1382400, 0x00000000 +0, 712, 712, 1, 1382400, 0x00000000 +0, 713, 713, 1, 1382400, 0x00000000 +0, 714, 714, 1, 1382400, 0x00000000 +0, 715, 715, 1, 1382400, 0x00000000 +0, 716, 716, 1, 1382400, 0x00000000 +0, 717, 717, 1, 1382400, 0x00000000 +0, 718, 718, 1, 1382400, 0x00000000 +0, 719, 719, 1, 1382400, 0x00000000 +0, 720, 720, 1, 1382400, 0x00000000 +0, 721, 721, 1, 1382400, 0x00000000 +0, 722, 722, 1, 1382400, 0x00000000 +0, 723, 723, 1, 1382400, 0x00000000 +0, 724, 724, 1, 1382400, 0x00000000 +0, 725, 725, 1, 1382400, 0x00000000 +0, 726, 726, 1, 1382400, 0x00000000 +0, 727, 727, 1, 1382400, 0x00000000 +0, 728, 728, 1, 1382400, 0x00000000 +0, 729, 729, 1, 1382400, 0x00000000 +0, 730, 730, 1, 1382400, 0x00000000 +0, 731, 731, 1, 1382400, 0x00000000 +0, 732, 732, 1, 1382400, 0x00000000 +0, 733, 733, 1, 1382400, 0x00000000 +0, 734, 734, 1, 1382400, 0x00000000 +0, 735, 735, 1, 1382400, 0x00000000 +0, 737, 737, 1, 1382400, 0x4c2960ca +0, 737, 737, 1, 1382400, 0x4c2960ca +0, 738, 738, 1, 1382400, 0x4c2960ca +0, 739, 739, 1, 1382400, 0x4c2960ca +0, 740, 740, 1, 1382400, 0x4c2960ca +0, 741, 741, 1, 1382400, 0x4c2960ca +0, 742, 742, 1, 1382400, 0x4c2960ca +0, 743, 743, 1, 1382400, 0x4c2960ca +0, 744, 744, 1, 1382400, 0x4c2960ca +0, 745, 745, 1, 1382400, 0x4c2960ca +0, 746, 746, 1, 1382400, 0x4c2960ca +0, 747, 747, 1, 1382400, 0x4c2960ca +0, 748, 748, 1, 1382400, 0x4c2960ca +0, 749, 749, 1, 1382400, 0x4c2960ca +0, 750, 750, 1, 1382400, 0x4c2960ca +0, 751, 751, 1, 1382400, 0x4c2960ca +0, 752, 752, 1, 1382400, 0x4c2960ca +0, 753, 753, 1, 1382400, 0x4c2960ca +0, 754, 754, 1, 1382400, 0x4c2960ca +0, 755, 755, 1, 1382400, 0x4c2960ca +0, 756, 756, 1, 1382400, 0x4c2960ca +0, 757, 757, 1, 1382400, 0x4c2960ca +0, 758, 758, 1, 1382400, 0x4c2960ca +0, 759, 759, 1, 1382400, 0x4c2960ca +0, 760, 760, 1, 1382400, 0x00000000 +0, 761, 761, 1, 1382400, 0x00000000 +0, 762, 762, 1, 1382400, 0x00000000 +0, 763, 763, 1, 1382400, 0x00000000 +0, 764, 764, 1, 1382400, 0x00000000 +0, 765, 765, 1, 1382400, 0x00000000 +0, 766, 766, 1, 1382400, 0x00000000 +0, 767, 767, 1, 1382400, 0x00000000 +0, 768, 768, 1, 1382400, 0x00000000 +0, 769, 769, 1, 1382400, 0x00000000 +0, 770, 770, 1, 1382400, 0x00000000 +0, 771, 771, 1, 1382400, 0x00000000 +0, 772, 772, 1, 1382400, 0x00000000 +0, 773, 773, 1, 1382400, 0x00000000 +0, 774, 774, 1, 1382400, 0x00000000 +0, 775, 775, 1, 1382400, 0x00000000 +0, 776, 776, 1, 1382400, 0x00000000 +0, 777, 777, 1, 1382400, 0x00000000 +0, 778, 778, 1, 1382400, 0x00000000 +0, 779, 779, 1, 1382400, 0x00000000 +0, 780, 780, 1, 1382400, 0x00000000 +0, 781, 781, 1, 1382400, 0x00000000 +0, 782, 782, 1, 1382400, 0x00000000 +0, 783, 783, 1, 1382400, 0x00000000 +0, 784, 784, 1, 1382400, 0x00000000 +0, 785, 785, 1, 1382400, 0x00000000 +0, 786, 786, 1, 1382400, 0x00000000 +0, 787, 787, 1, 1382400, 0x00000000 +0, 788, 788, 1, 1382400, 0x00000000 +0, 789, 789, 1, 1382400, 0x00000000 +0, 790, 790, 1, 1382400, 0x00000000 +0, 791, 791, 1, 1382400, 0x00000000 +0, 792, 792, 1, 1382400, 0x00000000 +0, 793, 793, 1, 1382400, 0x00000000 +0, 794, 794, 1, 1382400, 0x00000000 +0, 795, 795, 1, 1382400, 0x00000000 +0, 796, 796, 1, 1382400, 0x00000000 +0, 797, 797, 1, 1382400, 0x00000000 +0, 798, 798, 1, 1382400, 0x00000000 +0, 799, 799, 1, 1382400, 0x00000000 +0, 800, 800, 1, 1382400, 0x00000000 +0, 801, 801, 1, 1382400, 0x00000000 +0, 802, 802, 1, 1382400, 0x00000000 +0, 803, 803, 1, 1382400, 0x00000000 +0, 804, 804, 1, 1382400, 0x00000000 +0, 805, 805, 1, 1382400, 0x00000000 +0, 806, 806, 1, 1382400, 0x00000000 +0, 807, 807, 1, 1382400, 0x00000000 +0, 808, 808, 1, 1382400, 0x00000000 +0, 809, 809, 1, 1382400, 0x00000000 +0, 810, 810, 1, 1382400, 0x00000000 +0, 811, 811, 1, 1382400, 0x00000000 +0, 812, 812, 1, 1382400, 0x00000000 +0, 813, 813, 1, 1382400, 0x00000000 +0, 814, 814, 1, 1382400, 0x00000000 +0, 815, 815, 1, 1382400, 0x00000000 +0, 816, 816, 1, 1382400, 0x00000000 +0, 817, 817, 1, 1382400, 0x00000000 +0, 818, 818, 1, 1382400, 0x00000000 +0, 819, 819, 1, 1382400, 0x00000000 +0, 820, 820, 1, 1382400, 0x00000000 +0, 821, 821, 1, 1382400, 0x00000000 +0, 822, 822, 1, 1382400, 0x00000000 +0, 823, 823, 1, 1382400, 0x00000000 +0, 824, 824, 1, 1382400, 0x00000000 +0, 825, 825, 1, 1382400, 0x00000000 +0, 826, 826, 1, 1382400, 0x00000000 +0, 827, 827, 1, 1382400, 0x00000000 +0, 828, 828, 1, 1382400, 0x00000000 +0, 829, 829, 1, 1382400, 0x00000000 +0, 830, 830, 1, 1382400, 0x00000000 +0, 831, 831, 1, 1382400, 0x00000000 +0, 832, 832, 1, 1382400, 0x00000000 +0, 833, 833, 1, 1382400, 0x00000000 +0, 834, 834, 1, 1382400, 0x00000000 +0, 835, 835, 1, 1382400, 0x00000000 +0, 836, 836, 1, 1382400, 0x00000000 +0, 837, 837, 1, 1382400, 0x00000000 +0, 838, 838, 1, 1382400, 0x00000000 +0, 839, 839, 1, 1382400, 0x00000000 +0, 840, 840, 1, 1382400, 0x00000000 +0, 841, 841, 1, 1382400, 0x00000000 +0, 842, 842, 1, 1382400, 0x00000000 +0, 843, 843, 1, 1382400, 0x00000000 +0, 844, 844, 1, 1382400, 0x00000000 +0, 845, 845, 1, 1382400, 0x00000000 +0, 846, 846, 1, 1382400, 0x00000000 +0, 847, 847, 1, 1382400, 0x00000000 +0, 848, 848, 1, 1382400, 0x00000000 +0, 849, 849, 1, 1382400, 0x00000000 +0, 850, 850, 1, 1382400, 0x00000000 +0, 851, 851, 1, 1382400, 0x00000000 +0, 852, 852, 1, 1382400, 0x00000000 +0, 853, 853, 1, 1382400, 0x00000000 +0, 854, 854, 1, 1382400, 0x00000000 +0, 855, 855, 1, 1382400, 0x00000000 +0, 856, 856, 1, 1382400, 0x00000000 +0, 857, 857, 1, 1382400, 0x00000000 +0, 858, 858, 1, 1382400, 0x00000000 +0, 859, 859, 1, 1382400, 0x00000000 +0, 860, 860, 1, 1382400, 0x00000000 +0, 861, 861, 1, 1382400, 0x00000000 +0, 862, 862, 1, 1382400, 0x00000000 +0, 863, 863, 1, 1382400, 0x00000000 +0, 864, 864, 1, 1382400, 0x00000000 +0, 865, 865, 1, 1382400, 0x00000000 +0, 866, 866, 1, 1382400, 0x00000000 +0, 867, 867, 1, 1382400, 0x00000000 +0, 868, 868, 1, 1382400, 0x00000000 +0, 869, 869, 1, 1382400, 0x00000000 +0, 870, 870, 1, 1382400, 0x00000000 +0, 871, 871, 1, 1382400, 0x00000000 +0, 872, 872, 1, 1382400, 0x00000000 +0, 873, 873, 1, 1382400, 0x00000000 +0, 874, 874, 1, 1382400, 0x00000000 +0, 875, 875, 1, 1382400, 0x00000000 +0, 876, 876, 1, 1382400, 0x00000000 +0, 877, 877, 1, 1382400, 0x00000000 +0, 878, 878, 1, 1382400, 0x00000000 +0, 879, 879, 1, 1382400, 0x00000000 +0, 880, 880, 1, 1382400, 0x00000000 +0, 881, 881, 1, 1382400, 0x00000000 +0, 882, 882, 1, 1382400, 0x00000000 +0, 883, 883, 1, 1382400, 0x00000000 +0, 884, 884, 1, 1382400, 0x00000000 +0, 885, 885, 1, 1382400, 0x00000000 +0, 886, 886, 1, 1382400, 0x00000000 +0, 887, 887, 1, 1382400, 0x00000000 +0, 888, 888, 1, 1382400, 0x00000000 +0, 889, 889, 1, 1382400, 0x00000000 +0, 890, 890, 1, 1382400, 0x00000000 +0, 891, 891, 1, 1382400, 0x00000000 +0, 892, 892, 1, 1382400, 0x00000000 +0, 893, 893, 1, 1382400, 0x00000000 +0, 894, 894, 1, 1382400, 0x00000000 +0, 895, 895, 1, 1382400, 0x00000000 +0, 896, 896, 1, 1382400, 0x00000000 +0, 897, 897, 1, 1382400, 0x00000000 +0, 898, 898, 1, 1382400, 0x00000000 +0, 899, 899, 1, 1382400, 0x00000000 +0, 900, 900, 1, 1382400, 0x00000000 +0, 901, 901, 1, 1382400, 0x00000000 +0, 902, 902, 1, 1382400, 0x00000000 +0, 904, 904, 1, 1382400, 0x5fa18966 +0, 904, 904, 1, 1382400, 0x5fa18966 +0, 905, 905, 1, 1382400, 0x5fa18966 +0, 906, 906, 1, 1382400, 0x5fa18966 +0, 907, 907, 1, 1382400, 0x5fa18966 +0, 908, 908, 1, 1382400, 0x5fa18966 +0, 909, 909, 1, 1382400, 0x5fa18966 +0, 910, 910, 1, 1382400, 0x5fa18966 +0, 911, 911, 1, 1382400, 0x5fa18966 +0, 912, 912, 1, 1382400, 0x5fa18966 +0, 913, 913, 1, 1382400, 0x5fa18966 +0, 914, 914, 1, 1382400, 0x5fa18966 +0, 915, 915, 1, 1382400, 0x5fa18966 +0, 916, 916, 1, 1382400, 0x5fa18966 +0, 917, 917, 1, 1382400, 0x55f4b7b1 +0, 918, 918, 1, 1382400, 0x55f4b7b1 +0, 919, 919, 1, 1382400, 0x55f4b7b1 +0, 920, 920, 1, 1382400, 0x55f4b7b1 +0, 921, 921, 1, 1382400, 0x55f4b7b1 +0, 922, 922, 1, 1382400, 0x55f4b7b1 +0, 923, 923, 1, 1382400, 0x55f4b7b1 +0, 924, 924, 1, 1382400, 0x55f4b7b1 +0, 925, 925, 1, 1382400, 0x55f4b7b1 +0, 926, 926, 1, 1382400, 0x55f4b7b1 +0, 927, 927, 1, 1382400, 0x55f4b7b1 +0, 928, 928, 1, 1382400, 0x55f4b7b1 +0, 929, 929, 1, 1382400, 0x00000000 +0, 930, 930, 1, 1382400, 0xdfa4cf32 +0, 931, 931, 1, 1382400, 0xdfa4cf32 +0, 932, 932, 1, 1382400, 0xdfa4cf32 +0, 933, 933, 1, 1382400, 0xdfa4cf32 +0, 934, 934, 1, 1382400, 0xdfa4cf32 +0, 935, 935, 1, 1382400, 0xdfa4cf32 +0, 936, 936, 1, 1382400, 0xdfa4cf32 +0, 937, 937, 1, 1382400, 0xdfa4cf32 +0, 938, 938, 1, 1382400, 0xdfa4cf32 +0, 939, 939, 1, 1382400, 0xdfa4cf32 +0, 940, 940, 1, 1382400, 0xdfa4cf32 +0, 941, 941, 1, 1382400, 0xdfa4cf32 +0, 942, 942, 1, 1382400, 0xdfa4cf32 +0, 943, 943, 1, 1382400, 0x35023df8 +0, 944, 944, 1, 1382400, 0x35023df8 +0, 945, 945, 1, 1382400, 0x35023df8 +0, 946, 946, 1, 1382400, 0x35023df8 +0, 947, 947, 1, 1382400, 0x35023df8 +0, 948, 948, 1, 1382400, 0x35023df8 +0, 949, 949, 1, 1382400, 0x35023df8 +0, 950, 950, 1, 1382400, 0xed933219 +0, 951, 951, 1, 1382400, 0xed933219 +0, 952, 952, 1, 1382400, 0xed933219 +0, 953, 953, 1, 1382400, 0xed933219 +0, 954, 954, 1, 1382400, 0xed933219 +0, 955, 955, 1, 1382400, 0xed933219 +0, 956, 956, 1, 1382400, 0xed933219 +0, 957, 957, 1, 1382400, 0xed933219 +0, 958, 958, 1, 1382400, 0x00000000 +0, 959, 959, 1, 1382400, 0x00000000 +0, 960, 960, 1, 1382400, 0x00000000 +0, 961, 961, 1, 1382400, 0x00000000 +0, 962, 962, 1, 1382400, 0x00000000 +0, 963, 963, 1, 1382400, 0x00000000 +0, 964, 964, 1, 1382400, 0x00000000 +0, 965, 965, 1, 1382400, 0x00000000 +0, 966, 966, 1, 1382400, 0x00000000 +0, 967, 967, 1, 1382400, 0x00000000 +0, 968, 968, 1, 1382400, 0x00000000 +0, 969, 969, 1, 1382400, 0x00000000 +0, 970, 970, 1, 1382400, 0x00000000 +0, 971, 971, 1, 1382400, 0x00000000 +0, 972, 972, 1, 1382400, 0x00000000 +0, 973, 973, 1, 1382400, 0x00000000 +0, 974, 974, 1, 1382400, 0x00000000 +0, 975, 975, 1, 1382400, 0x00000000 +0, 976, 976, 1, 1382400, 0x00000000 +0, 977, 977, 1, 1382400, 0x00000000 +0, 978, 978, 1, 1382400, 0x00000000 +0, 979, 979, 1, 1382400, 0x00000000 +0, 980, 980, 1, 1382400, 0x00000000 +0, 981, 981, 1, 1382400, 0x00000000 +0, 982, 982, 1, 1382400, 0x00000000 +0, 983, 983, 1, 1382400, 0x00000000 +0, 984, 984, 1, 1382400, 0x00000000 +0, 985, 985, 1, 1382400, 0x00000000 +0, 986, 986, 1, 1382400, 0x00000000 +0, 987, 987, 1, 1382400, 0x00000000 +0, 988, 988, 1, 1382400, 0x00000000 +0, 989, 989, 1, 1382400, 0x00000000 +0, 990, 990, 1, 1382400, 0x00000000 +0, 991, 991, 1, 1382400, 0x00000000 +0, 992, 992, 1, 1382400, 0x00000000 +0, 993, 993, 1, 1382400, 0x00000000 +0, 994, 994, 1, 1382400, 0x00000000 +0, 995, 995, 1, 1382400, 0x00000000 +0, 996, 996, 1, 1382400, 0x00000000 +0, 997, 997, 1, 1382400, 0x00000000 +0, 999, 999, 1, 1382400, 0x1b26389a +0, 999, 999, 1, 1382400, 0x1b26389a +0, 1000, 1000, 1, 1382400, 0x1b26389a +0, 1001, 1001, 1, 1382400, 0x1b26389a +0, 1002, 1002, 1, 1382400, 0x1b26389a +0, 1003, 1003, 1, 1382400, 0x1b26389a +0, 1004, 1004, 1, 1382400, 0x1b26389a +0, 1006, 1006, 1, 1382400, 0xf0c7028b +0, 1006, 1006, 1, 1382400, 0xf0c7028b +0, 1007, 1007, 1, 1382400, 0xf0c7028b +0, 1008, 1008, 1, 1382400, 0xf0c7028b +0, 1009, 1009, 1, 1382400, 0xf0c7028b +0, 1010, 1010, 1, 1382400, 0xf0c7028b +0, 1011, 1011, 1, 1382400, 0xf0c7028b +0, 1012, 1012, 1, 1382400, 0xf0c7028b +0, 1013, 1013, 1, 1382400, 0xf0c7028b +0, 1014, 1014, 1, 1382400, 0x395f521d +0, 1015, 1015, 1, 1382400, 0x395f521d +0, 1016, 1016, 1, 1382400, 0x395f521d +0, 1017, 1017, 1, 1382400, 0x395f521d +0, 1018, 1018, 1, 1382400, 0x395f521d +0, 1019, 1019, 1, 1382400, 0x395f521d +0, 1020, 1020, 1, 1382400, 0x395f521d +0, 1021, 1021, 1, 1382400, 0x395f521d +0, 1022, 1022, 1, 1382400, 0x395f521d +0, 1024, 1024, 1, 1382400, 0x1ea87415 +0, 1024, 1024, 1, 1382400, 0x1ea87415 +0, 1025, 1025, 1, 1382400, 0x1ea87415 +0, 1026, 1026, 1, 1382400, 0x1ea87415 +0, 1027, 1027, 1, 1382400, 0x1ea87415 +0, 1028, 1028, 1, 1382400, 0x1ea87415 +0, 1029, 1029, 1, 1382400, 0x1ea87415 +0, 1030, 1030, 1, 1382400, 0x1ea87415 +0, 1031, 1031, 1, 1382400, 0x1ea87415 +0, 1032, 1032, 1, 1382400, 0x1ea87415 +0, 1033, 1033, 1, 1382400, 0x1ea87415 +0, 1034, 1034, 1, 1382400, 0xc6effdc1 +0, 1035, 1035, 1, 1382400, 0xc6effdc1 +0, 1036, 1036, 1, 1382400, 0xc6effdc1 +0, 1037, 1037, 1, 1382400, 0xc6effdc1 +0, 1038, 1038, 1, 1382400, 0xc6effdc1 +0, 1039, 1039, 1, 1382400, 0xc6effdc1 +0, 1040, 1040, 1, 1382400, 0xc6effdc1 +0, 1041, 1041, 1, 1382400, 0xc6effdc1 +0, 1042, 1042, 1, 1382400, 0xc6effdc1 +0, 1044, 1044, 1, 1382400, 0xba6846f8 +0, 1044, 1044, 1, 1382400, 0xba6846f8 +0, 1045, 1045, 1, 1382400, 0xba6846f8 +0, 1046, 1046, 1, 1382400, 0xba6846f8 +0, 1047, 1047, 1, 1382400, 0xba6846f8 +0, 1048, 1048, 1, 1382400, 0xba6846f8 +0, 1049, 1049, 1, 1382400, 0xba6846f8 +0, 1050, 1050, 1, 1382400, 0x033c5d5b +0, 1051, 1051, 1, 1382400, 0x033c5d5b +0, 1052, 1052, 1, 1382400, 0x033c5d5b +0, 1053, 1053, 1, 1382400, 0x033c5d5b +0, 1054, 1054, 1, 1382400, 0x033c5d5b +0, 1055, 1055, 1, 1382400, 0x033c5d5b +0, 1056, 1056, 1, 1382400, 0x033c5d5b +0, 1057, 1057, 1, 1382400, 0x033c5d5b +0, 1058, 1058, 1, 1382400, 0x00000000 +0, 1059, 1059, 1, 1382400, 0xef5abf66 +0, 1060, 1060, 1, 1382400, 0xef5abf66 +0, 1061, 1061, 1, 1382400, 0xef5abf66 +0, 1062, 1062, 1, 1382400, 0xef5abf66 +0, 1063, 1063, 1, 1382400, 0xef5abf66 +0, 1064, 1064, 1, 1382400, 0xef5abf66 +0, 1065, 1065, 1, 1382400, 0xef5abf66 +0, 1066, 1066, 1, 1382400, 0xef5abf66 +0, 1067, 1067, 1, 1382400, 0xef5abf66 +0, 1068, 1068, 1, 1382400, 0xef5abf66 +0, 1069, 1069, 1, 1382400, 0xef5abf66 +0, 1070, 1070, 1, 1382400, 0xef5abf66 +0, 1071, 1071, 1, 1382400, 0xef5abf66 +0, 1072, 1072, 1, 1382400, 0x00000000 +0, 1073, 1073, 1, 1382400, 0xec747954 +0, 1074, 1074, 1, 1382400, 0xec747954 +0, 1075, 1075, 1, 1382400, 0xec747954 +0, 1076, 1076, 1, 1382400, 0xec747954 +0, 1077, 1077, 1, 1382400, 0xec747954 +0, 1078, 1078, 1, 1382400, 0xec747954 +0, 1079, 1079, 1, 1382400, 0xec747954 +0, 1080, 1080, 1, 1382400, 0xec747954 +0, 1081, 1081, 1, 1382400, 0xec747954 +0, 1082, 1082, 1, 1382400, 0xec747954 +0, 1083, 1083, 1, 1382400, 0xec747954 +0, 1084, 1084, 1, 1382400, 0xec747954 +0, 1085, 1085, 1, 1382400, 0xec747954 +0, 1086, 1086, 1, 1382400, 0x00000000 +0, 1087, 1087, 1, 1382400, 0xfa34bcaf +0, 1088, 1088, 1, 1382400, 0xfa34bcaf +0, 1089, 1089, 1, 1382400, 0xfa34bcaf +0, 1090, 1090, 1, 1382400, 0xfa34bcaf +0, 1091, 1091, 1, 1382400, 0xfa34bcaf +0, 1092, 1092, 1, 1382400, 0xfa34bcaf +0, 1093, 1093, 1, 1382400, 0xfa34bcaf +0, 1094, 1094, 1, 1382400, 0xfa34bcaf +0, 1095, 1095, 1, 1382400, 0xfa34bcaf +0, 1096, 1096, 1, 1382400, 0xfa34bcaf +0, 1097, 1097, 1, 1382400, 0xfa34bcaf +0, 1098, 1098, 1, 1382400, 0x00000000 +0, 1099, 1099, 1, 1382400, 0x8b7a709b +0, 1100, 1100, 1, 1382400, 0x8b7a709b +0, 1101, 1101, 1, 1382400, 0x8b7a709b +0, 1102, 1102, 1, 1382400, 0x8b7a709b +0, 1103, 1103, 1, 1382400, 0x8b7a709b +0, 1104, 1104, 1, 1382400, 0x8b7a709b +0, 1105, 1105, 1, 1382400, 0x8b7a709b +0, 1106, 1106, 1, 1382400, 0x8b7a709b +0, 1107, 1107, 1, 1382400, 0x00000000 +0, 1108, 1108, 1, 1382400, 0x00000000 +0, 1109, 1109, 1, 1382400, 0x00000000 +0, 1110, 1110, 1, 1382400, 0x00000000 +0, 1111, 1111, 1, 1382400, 0x00000000 +0, 1112, 1112, 1, 1382400, 0x00000000 +0, 1113, 1113, 1, 1382400, 0x00000000 +0, 1114, 1114, 1, 1382400, 0x00000000 +0, 1115, 1115, 1, 1382400, 0x00000000 +0, 1116, 1116, 1, 1382400, 0x00000000 +0, 1118, 1118, 1, 1382400, 0xc333382f +0, 1118, 1118, 1, 1382400, 0xc333382f +0, 1119, 1119, 1, 1382400, 0xc333382f +0, 1120, 1120, 1, 1382400, 0xc333382f +0, 1121, 1121, 1, 1382400, 0xc333382f +0, 1122, 1122, 1, 1382400, 0xc333382f +0, 1123, 1123, 1, 1382400, 0xc333382f +0, 1124, 1124, 1, 1382400, 0xc333382f +0, 1125, 1125, 1, 1382400, 0xc333382f +0, 1126, 1126, 1, 1382400, 0xc333382f +0, 1127, 1127, 1, 1382400, 0xc333382f +0, 1128, 1128, 1, 1382400, 0xc333382f +0, 1129, 1129, 1, 1382400, 0x00000000 +0, 1130, 1130, 1, 1382400, 0x00000000 +0, 1131, 1131, 1, 1382400, 0x00000000 +0, 1132, 1132, 1, 1382400, 0x00000000 +0, 1133, 1133, 1, 1382400, 0x00000000 +0, 1134, 1134, 1, 1382400, 0x00000000 +0, 1135, 1135, 1, 1382400, 0x00000000 +0, 1136, 1136, 1, 1382400, 0x00000000 +0, 1138, 1138, 1, 1382400, 0xabe5dfcf +0, 1138, 1138, 1, 1382400, 0xabe5dfcf +0, 1139, 1139, 1, 1382400, 0xabe5dfcf +0, 1140, 1140, 1, 1382400, 0xabe5dfcf +0, 1141, 1141, 1, 1382400, 0xabe5dfcf +0, 1142, 1142, 1, 1382400, 0xabe5dfcf +0, 1143, 1143, 1, 1382400, 0xabe5dfcf +0, 1144, 1144, 1, 1382400, 0xabe5dfcf +0, 1145, 1145, 1, 1382400, 0xabe5dfcf +0, 1146, 1146, 1, 1382400, 0xabe5dfcf +0, 1147, 1147, 1, 1382400, 0xabe5dfcf +0, 1148, 1148, 1, 1382400, 0xabe5dfcf +0, 1149, 1149, 1, 1382400, 0xabe5dfcf +0, 1150, 1150, 1, 1382400, 0xabe5dfcf +0, 1151, 1151, 1, 1382400, 0xabe5dfcf +0, 1152, 1152, 1, 1382400, 0xabe5dfcf +0, 1153, 1153, 1, 1382400, 0xabe5dfcf +0, 1154, 1154, 1, 1382400, 0x56948101 +0, 1155, 1155, 1, 1382400, 0x56948101 +0, 1156, 1156, 1, 1382400, 0x56948101 +0, 1157, 1157, 1, 1382400, 0x56948101 +0, 1158, 1158, 1, 1382400, 0x56948101 +0, 1159, 1159, 1, 1382400, 0x56948101 +0, 1160, 1160, 1, 1382400, 0x56948101 +0, 1161, 1161, 1, 1382400, 0x56948101 +0, 1162, 1162, 1, 1382400, 0x56948101 +0, 1163, 1163, 1, 1382400, 0x56948101 +0, 1164, 1164, 1, 1382400, 0x56948101 +0, 1166, 1166, 1, 1382400, 0xb747834a +0, 1166, 1166, 1, 1382400, 0xb747834a +0, 1167, 1167, 1, 1382400, 0xb747834a +0, 1168, 1168, 1, 1382400, 0xb747834a +0, 1169, 1169, 1, 1382400, 0xb747834a +0, 1170, 1170, 1, 1382400, 0xb747834a +0, 1171, 1171, 1, 1382400, 0xb747834a +0, 1172, 1172, 1, 1382400, 0xb747834a +0, 1173, 1173, 1, 1382400, 0xb747834a +0, 1174, 1174, 1, 1382400, 0xb747834a +0, 1175, 1175, 1, 1382400, 0xb747834a +0, 1176, 1176, 1, 1382400, 0xb747834a +0, 1177, 1177, 1, 1382400, 0xb747834a +0, 1178, 1178, 1, 1382400, 0xb747834a +0, 1179, 1179, 1, 1382400, 0xb747834a +0, 1180, 1180, 1, 1382400, 0xb747834a +0, 1181, 1181, 1, 1382400, 0xb747834a +0, 1182, 1182, 1, 1382400, 0xb747834a +0, 1183, 1183, 1, 1382400, 0xb747834a +0, 1184, 1184, 1, 1382400, 0xb747834a +0, 1185, 1185, 1, 1382400, 0xb747834a +0, 1187, 1187, 1, 1382400, 0x3448baad +0, 1187, 1187, 1, 1382400, 0x3448baad +0, 1188, 1188, 1, 1382400, 0x3448baad +0, 1189, 1189, 1, 1382400, 0x3448baad +0, 1190, 1190, 1, 1382400, 0x3448baad +0, 1191, 1191, 1, 1382400, 0x3448baad +0, 1192, 1192, 1, 1382400, 0x3448baad +0, 1193, 1193, 1, 1382400, 0x3448baad +0, 1194, 1194, 1, 1382400, 0x3448baad +0, 1195, 1195, 1, 1382400, 0x3448baad +0, 1196, 1196, 1, 1382400, 0x3448baad +0, 1197, 1197, 1, 1382400, 0x3448baad +0, 1198, 1198, 1, 1382400, 0x3448baad +0, 1199, 1199, 1, 1382400, 0x3448baad +0, 1200, 1200, 1, 1382400, 0x00000000 +0, 1201, 1201, 1, 1382400, 0xaabe4f37 +0, 1202, 1202, 1, 1382400, 0xaabe4f37 +0, 1203, 1203, 1, 1382400, 0xaabe4f37 +0, 1204, 1204, 1, 1382400, 0xaabe4f37 +0, 1205, 1205, 1, 1382400, 0xaabe4f37 +0, 1206, 1206, 1, 1382400, 0xaabe4f37 +0, 1207, 1207, 1, 1382400, 0xaabe4f37 +0, 1208, 1208, 1, 1382400, 0xaabe4f37 +0, 1209, 1209, 1, 1382400, 0xaabe4f37 +0, 1210, 1210, 1, 1382400, 0xaabe4f37 +0, 1211, 1211, 1, 1382400, 0x00000000 +0, 1212, 1212, 1, 1382400, 0x00000000 +0, 1213, 1213, 1, 1382400, 0x00000000 +0, 1214, 1214, 1, 1382400, 0x00000000 +0, 1215, 1215, 1, 1382400, 0x00000000 +0, 1216, 1216, 1, 1382400, 0x00000000 +0, 1217, 1217, 1, 1382400, 0x00000000 +0, 1218, 1218, 1, 1382400, 0x00000000 +0, 1219, 1219, 1, 1382400, 0x00000000 +0, 1220, 1220, 1, 1382400, 0x00000000 +0, 1221, 1221, 1, 1382400, 0x00000000 +0, 1222, 1222, 1, 1382400, 0x00000000 +0, 1223, 1223, 1, 1382400, 0x00000000 +0, 1224, 1224, 1, 1382400, 0x00000000 +0, 1225, 1225, 1, 1382400, 0x00000000 +0, 1226, 1226, 1, 1382400, 0x00000000 +0, 1227, 1227, 1, 1382400, 0x00000000 +0, 1228, 1228, 1, 1382400, 0x00000000 +0, 1229, 1229, 1, 1382400, 0x00000000 +0, 1230, 1230, 1, 1382400, 0x00000000 +0, 1231, 1231, 1, 1382400, 0x00000000 +0, 1232, 1232, 1, 1382400, 0x00000000 +0, 1233, 1233, 1, 1382400, 0x00000000 +0, 1234, 1234, 1, 1382400, 0x00000000 +0, 1235, 1235, 1, 1382400, 0x00000000 +0, 1236, 1236, 1, 1382400, 0x00000000 +0, 1237, 1237, 1, 1382400, 0x00000000 +0, 1238, 1238, 1, 1382400, 0x00000000 +0, 1239, 1239, 1, 1382400, 0x00000000 +0, 1240, 1240, 1, 1382400, 0x00000000 +0, 1241, 1241, 1, 1382400, 0x00000000 +0, 1242, 1242, 1, 1382400, 0x00000000 +0, 1243, 1243, 1, 1382400, 0x00000000 +0, 1244, 1244, 1, 1382400, 0x00000000 +0, 1245, 1245, 1, 1382400, 0x00000000 +0, 1246, 1246, 1, 1382400, 0x00000000 +0, 1247, 1247, 1, 1382400, 0x00000000 +0, 1248, 1248, 1, 1382400, 0x00000000 +0, 1249, 1249, 1, 1382400, 0x00000000 +0, 1250, 1250, 1, 1382400, 0x00000000 +0, 1251, 1251, 1, 1382400, 0x00000000 +0, 1252, 1252, 1, 1382400, 0x00000000 +0, 1253, 1253, 1, 1382400, 0x00000000 +0, 1254, 1254, 1, 1382400, 0x00000000 +0, 1255, 1255, 1, 1382400, 0x00000000 +0, 1256, 1256, 1, 1382400, 0x00000000 +0, 1257, 1257, 1, 1382400, 0x00000000 +0, 1258, 1258, 1, 1382400, 0x00000000 +0, 1259, 1259, 1, 1382400, 0x00000000 +0, 1260, 1260, 1, 1382400, 0x00000000 +0, 1261, 1261, 1, 1382400, 0x00000000 +0, 1262, 1262, 1, 1382400, 0x00000000 +0, 1263, 1263, 1, 1382400, 0x00000000 +0, 1264, 1264, 1, 1382400, 0x00000000 +0, 1265, 1265, 1, 1382400, 0x00000000 +0, 1266, 1266, 1, 1382400, 0x00000000 +0, 1267, 1267, 1, 1382400, 0x00000000 +0, 1268, 1268, 1, 1382400, 0x00000000 +0, 1269, 1269, 1, 1382400, 0x00000000 +0, 1270, 1270, 1, 1382400, 0x00000000 +0, 1271, 1271, 1, 1382400, 0x00000000 +0, 1272, 1272, 1, 1382400, 0x00000000 +0, 1273, 1273, 1, 1382400, 0x00000000 +0, 1274, 1274, 1, 1382400, 0x00000000 +0, 1275, 1275, 1, 1382400, 0x00000000 +0, 1276, 1276, 1, 1382400, 0x00000000 +0, 1277, 1277, 1, 1382400, 0x00000000 +0, 1278, 1278, 1, 1382400, 0x00000000 +0, 1279, 1279, 1, 1382400, 0x00000000 +0, 1280, 1280, 1, 1382400, 0x00000000 +0, 1281, 1281, 1, 1382400, 0x00000000 +0, 1282, 1282, 1, 1382400, 0x00000000 +0, 1283, 1283, 1, 1382400, 0x00000000 +0, 1284, 1284, 1, 1382400, 0x00000000 +0, 1285, 1285, 1, 1382400, 0x00000000 +0, 1286, 1286, 1, 1382400, 0x00000000 +0, 1287, 1287, 1, 1382400, 0x00000000 +0, 1288, 1288, 1, 1382400, 0x00000000 +0, 1289, 1289, 1, 1382400, 0x00000000 +0, 1290, 1290, 1, 1382400, 0x00000000 +0, 1291, 1291, 1, 1382400, 0x00000000 +0, 1292, 1292, 1, 1382400, 0x00000000 +0, 1293, 1293, 1, 1382400, 0x00000000 +0, 1294, 1294, 1, 1382400, 0x00000000 +0, 1295, 1295, 1, 1382400, 0x00000000 +0, 1296, 1296, 1, 1382400, 0x00000000 +0, 1297, 1297, 1, 1382400, 0x00000000 +0, 1298, 1298, 1, 1382400, 0x00000000 +0, 1299, 1299, 1, 1382400, 0x00000000 +0, 1300, 1300, 1, 1382400, 0x00000000 +0, 1301, 1301, 1, 1382400, 0x00000000 +0, 1302, 1302, 1, 1382400, 0x00000000 +0, 1303, 1303, 1, 1382400, 0x00000000 +0, 1304, 1304, 1, 1382400, 0x00000000 +0, 1305, 1305, 1, 1382400, 0x00000000 +0, 1306, 1306, 1, 1382400, 0x00000000 +0, 1307, 1307, 1, 1382400, 0x00000000 +0, 1308, 1308, 1, 1382400, 0x00000000 +0, 1309, 1309, 1, 1382400, 0x00000000 +0, 1310, 1310, 1, 1382400, 0x00000000 +0, 1311, 1311, 1, 1382400, 0x00000000 +0, 1312, 1312, 1, 1382400, 0x00000000 +0, 1313, 1313, 1, 1382400, 0x00000000 +0, 1314, 1314, 1, 1382400, 0x00000000 +0, 1315, 1315, 1, 1382400, 0x00000000 +0, 1316, 1316, 1, 1382400, 0x00000000 +0, 1317, 1317, 1, 1382400, 0x00000000 +0, 1318, 1318, 1, 1382400, 0x00000000 +0, 1319, 1319, 1, 1382400, 0x00000000 +0, 1320, 1320, 1, 1382400, 0x00000000 +0, 1321, 1321, 1, 1382400, 0x00000000 +0, 1322, 1322, 1, 1382400, 0x00000000 +0, 1323, 1323, 1, 1382400, 0x00000000 +0, 1324, 1324, 1, 1382400, 0x00000000 +0, 1325, 1325, 1, 1382400, 0x00000000 +0, 1326, 1326, 1, 1382400, 0x00000000 +0, 1327, 1327, 1, 1382400, 0x00000000 +0, 1328, 1328, 1, 1382400, 0x00000000 +0, 1329, 1329, 1, 1382400, 0x00000000 +0, 1330, 1330, 1, 1382400, 0x00000000 +0, 1331, 1331, 1, 1382400, 0x00000000 +0, 1332, 1332, 1, 1382400, 0x00000000 +0, 1333, 1333, 1, 1382400, 0x00000000 +0, 1334, 1334, 1, 1382400, 0x00000000 +0, 1335, 1335, 1, 1382400, 0x00000000 +0, 1336, 1336, 1, 1382400, 0x00000000 +0, 1337, 1337, 1, 1382400, 0x00000000 +0, 1338, 1338, 1, 1382400, 0x00000000 +0, 1339, 1339, 1, 1382400, 0x00000000 +0, 1340, 1340, 1, 1382400, 0x00000000 +0, 1341, 1341, 1, 1382400, 0x00000000 +0, 1342, 1342, 1, 1382400, 0x00000000 +0, 1343, 1343, 1, 1382400, 0x00000000 +0, 1344, 1344, 1, 1382400, 0x00000000 +0, 1345, 1345, 1, 1382400, 0x00000000 +0, 1346, 1346, 1, 1382400, 0x00000000 +0, 1347, 1347, 1, 1382400, 0x00000000 +0, 1348, 1348, 1, 1382400, 0x00000000 +0, 1349, 1349, 1, 1382400, 0x00000000 +0, 1350, 1350, 1, 1382400, 0x00000000 +0, 1351, 1351, 1, 1382400, 0x00000000 +0, 1352, 1352, 1, 1382400, 0x00000000 +0, 1353, 1353, 1, 1382400, 0x00000000 +0, 1354, 1354, 1, 1382400, 0x00000000 +0, 1355, 1355, 1, 1382400, 0x00000000 +0, 1356, 1356, 1, 1382400, 0x00000000 +0, 1357, 1357, 1, 1382400, 0x00000000 +0, 1358, 1358, 1, 1382400, 0x00000000 +0, 1359, 1359, 1, 1382400, 0x00000000 +0, 1360, 1360, 1, 1382400, 0x00000000 +0, 1361, 1361, 1, 1382400, 0x00000000 +0, 1362, 1362, 1, 1382400, 0x00000000 +0, 1363, 1363, 1, 1382400, 0x00000000 +0, 1364, 1364, 1, 1382400, 0x00000000 +0, 1365, 1365, 1, 1382400, 0x00000000 +0, 1366, 1366, 1, 1382400, 0x00000000 +0, 1368, 1368, 1, 1382400, 0x8a48cd6f +0, 1368, 1368, 1, 1382400, 0x8a48cd6f +0, 1369, 1369, 1, 1382400, 0x8a48cd6f +0, 1370, 1370, 1, 1382400, 0x8a48cd6f +0, 1371, 1371, 1, 1382400, 0x8a48cd6f +0, 1372, 1372, 1, 1382400, 0x8a48cd6f +0, 1373, 1373, 1, 1382400, 0x8a48cd6f +0, 1374, 1374, 1, 1382400, 0x8a48cd6f +0, 1375, 1375, 1, 1382400, 0x8a48cd6f +0, 1376, 1376, 1, 1382400, 0x00000000 +0, 1377, 1377, 1, 1382400, 0x00000000 +0, 1378, 1378, 1, 1382400, 0x00000000 +0, 1379, 1379, 1, 1382400, 0x00000000 +0, 1380, 1380, 1, 1382400, 0x00000000 +0, 1381, 1381, 1, 1382400, 0x00000000 +0, 1382, 1382, 1, 1382400, 0x00000000 +0, 1383, 1383, 1, 1382400, 0x00000000 +0, 1384, 1384, 1, 1382400, 0x00000000 +0, 1385, 1385, 1, 1382400, 0x00000000 +0, 1386, 1386, 1, 1382400, 0x00000000 +0, 1387, 1387, 1, 1382400, 0x00000000 +0, 1388, 1388, 1, 1382400, 0x00000000 +0, 1389, 1389, 1, 1382400, 0x00000000 +0, 1390, 1390, 1, 1382400, 0x00000000 +0, 1391, 1391, 1, 1382400, 0x00000000 +0, 1392, 1392, 1, 1382400, 0x00000000 +0, 1393, 1393, 1, 1382400, 0x00000000 +0, 1394, 1394, 1, 1382400, 0x00000000 +0, 1395, 1395, 1, 1382400, 0x00000000 +0, 1396, 1396, 1, 1382400, 0x00000000 +0, 1397, 1397, 1, 1382400, 0x00000000 +0, 1398, 1398, 1, 1382400, 0x00000000 +0, 1399, 1399, 1, 1382400, 0x00000000 +0, 1400, 1400, 1, 1382400, 0x00000000 +0, 1401, 1401, 1, 1382400, 0x00000000 +0, 1402, 1402, 1, 1382400, 0x00000000 +0, 1403, 1403, 1, 1382400, 0x00000000 +0, 1404, 1404, 1, 1382400, 0x00000000 +0, 1405, 1405, 1, 1382400, 0x00000000 +0, 1406, 1406, 1, 1382400, 0x00000000 +0, 1407, 1407, 1, 1382400, 0x00000000 +0, 1408, 1408, 1, 1382400, 0x00000000 +0, 1409, 1409, 1, 1382400, 0x00000000 +0, 1410, 1410, 1, 1382400, 0x00000000 +0, 1411, 1411, 1, 1382400, 0x00000000 +0, 1412, 1412, 1, 1382400, 0x00000000 +0, 1413, 1413, 1, 1382400, 0x00000000 +0, 1414, 1414, 1, 1382400, 0x00000000 +0, 1415, 1415, 1, 1382400, 0x00000000 +0, 1416, 1416, 1, 1382400, 0x00000000 +0, 1417, 1417, 1, 1382400, 0x00000000 +0, 1418, 1418, 1, 1382400, 0x00000000 +0, 1419, 1419, 1, 1382400, 0x00000000 +0, 1420, 1420, 1, 1382400, 0x00000000 +0, 1421, 1421, 1, 1382400, 0x00000000 +0, 1422, 1422, 1, 1382400, 0x00000000 +0, 1423, 1423, 1, 1382400, 0x00000000 +0, 1424, 1424, 1, 1382400, 0x00000000 +0, 1425, 1425, 1, 1382400, 0x00000000 +0, 1426, 1426, 1, 1382400, 0x00000000 +0, 1427, 1427, 1, 1382400, 0x00000000 +0, 1428, 1428, 1, 1382400, 0x00000000 +0, 1429, 1429, 1, 1382400, 0x00000000 +0, 1430, 1430, 1, 1382400, 0x00000000 +0, 1431, 1431, 1, 1382400, 0x00000000 +0, 1432, 1432, 1, 1382400, 0x00000000 +0, 1433, 1433, 1, 1382400, 0x00000000 +0, 1434, 1434, 1, 1382400, 0x00000000 +0, 1435, 1435, 1, 1382400, 0x00000000 +0, 1436, 1436, 1, 1382400, 0x00000000 +0, 1437, 1437, 1, 1382400, 0x00000000 +0, 1438, 1438, 1, 1382400, 0x00000000 +0, 1439, 1439, 1, 1382400, 0x00000000 +0, 1440, 1440, 1, 1382400, 0x00000000 +0, 1441, 1441, 1, 1382400, 0x00000000 +0, 1442, 1442, 1, 1382400, 0x00000000 +0, 1443, 1443, 1, 1382400, 0x00000000 +0, 1444, 1444, 1, 1382400, 0x00000000 +0, 1445, 1445, 1, 1382400, 0x00000000 +0, 1446, 1446, 1, 1382400, 0x00000000 +0, 1447, 1447, 1, 1382400, 0x00000000 +0, 1448, 1448, 1, 1382400, 0x00000000 +0, 1449, 1449, 1, 1382400, 0x00000000 +0, 1450, 1450, 1, 1382400, 0x00000000 +0, 1451, 1451, 1, 1382400, 0x00000000 +0, 1452, 1452, 1, 1382400, 0x00000000 +0, 1453, 1453, 1, 1382400, 0x00000000 +0, 1454, 1454, 1, 1382400, 0x00000000 +0, 1455, 1455, 1, 1382400, 0x00000000 +0, 1456, 1456, 1, 1382400, 0x00000000 +0, 1457, 1457, 1, 1382400, 0x00000000 +0, 1458, 1458, 1, 1382400, 0x00000000 +0, 1459, 1459, 1, 1382400, 0x00000000 +0, 1460, 1460, 1, 1382400, 0x00000000 +0, 1461, 1461, 1, 1382400, 0x00000000 +0, 1462, 1462, 1, 1382400, 0x00000000 +0, 1463, 1463, 1, 1382400, 0x00000000 +0, 1464, 1464, 1, 1382400, 0x00000000 +0, 1465, 1465, 1, 1382400, 0x00000000 +0, 1466, 1466, 1, 1382400, 0x00000000 +0, 1467, 1467, 1, 1382400, 0x00000000 +0, 1468, 1468, 1, 1382400, 0x00000000 +0, 1469, 1469, 1, 1382400, 0x00000000 +0, 1470, 1470, 1, 1382400, 0x00000000 +0, 1471, 1471, 1, 1382400, 0x00000000 +0, 1472, 1472, 1, 1382400, 0x00000000 +0, 1473, 1473, 1, 1382400, 0x00000000 +0, 1474, 1474, 1, 1382400, 0x00000000 +0, 1475, 1475, 1, 1382400, 0x00000000 +0, 1477, 1477, 1, 1382400, 0x49518c43 +0, 1477, 1477, 1, 1382400, 0x49518c43 +0, 1478, 1478, 1, 1382400, 0x49518c43 +0, 1479, 1479, 1, 1382400, 0x49518c43 +0, 1480, 1480, 1, 1382400, 0x49518c43 +0, 1481, 1481, 1, 1382400, 0x49518c43 +0, 1482, 1482, 1, 1382400, 0x49518c43 +0, 1483, 1483, 1, 1382400, 0x49518c43 +0, 1484, 1484, 1, 1382400, 0x00000000 +0, 1485, 1485, 1, 1382400, 0x00000000 +0, 1486, 1486, 1, 1382400, 0x00000000 +0, 1487, 1487, 1, 1382400, 0x00000000 +0, 1488, 1488, 1, 1382400, 0x00000000 +0, 1489, 1489, 1, 1382400, 0x00000000 +0, 1490, 1490, 1, 1382400, 0x00000000 +0, 1491, 1491, 1, 1382400, 0x00000000 +0, 1492, 1492, 1, 1382400, 0x00000000 +0, 1493, 1493, 1, 1382400, 0x00000000 +0, 1494, 1494, 1, 1382400, 0x00000000 +0, 1495, 1495, 1, 1382400, 0x00000000 +0, 1496, 1496, 1, 1382400, 0x00000000 +0, 1497, 1497, 1, 1382400, 0x00000000 +0, 1498, 1498, 1, 1382400, 0x00000000 +0, 1500, 1500, 1, 1382400, 0x4a72fa21 +0, 1500, 1500, 1, 1382400, 0x4a72fa21 +0, 1501, 1501, 1, 1382400, 0x4a72fa21 +0, 1502, 1502, 1, 1382400, 0x4a72fa21 +0, 1503, 1503, 1, 1382400, 0x4a72fa21 +0, 1504, 1504, 1, 1382400, 0x4a72fa21 +0, 1505, 1505, 1, 1382400, 0x4a72fa21 +0, 1506, 1506, 1, 1382400, 0x4a72fa21 +0, 1507, 1507, 1, 1382400, 0x4a72fa21 +0, 1508, 1508, 1, 1382400, 0x4a72fa21 +0, 1509, 1509, 1, 1382400, 0x4a72fa21 +0, 1510, 1510, 1, 1382400, 0x00000000 +0, 1511, 1511, 1, 1382400, 0xa82f7de8 +0, 1512, 1512, 1, 1382400, 0xa82f7de8 +0, 1513, 1513, 1, 1382400, 0xa82f7de8 +0, 1514, 1514, 1, 1382400, 0xa82f7de8 +0, 1515, 1515, 1, 1382400, 0xa82f7de8 +0, 1516, 1516, 1, 1382400, 0xa82f7de8 +0, 1517, 1517, 1, 1382400, 0xa82f7de8 +0, 1518, 1518, 1, 1382400, 0x00000000 +0, 1519, 1519, 1, 1382400, 0x00000000 +0, 1521, 1521, 1, 1382400, 0xeba0b5f3 +0, 1521, 1521, 1, 1382400, 0xeba0b5f3 +0, 1522, 1522, 1, 1382400, 0xeba0b5f3 +0, 1523, 1523, 1, 1382400, 0xeba0b5f3 +0, 1524, 1524, 1, 1382400, 0xeba0b5f3 +0, 1525, 1525, 1, 1382400, 0xeba0b5f3 +0, 1526, 1526, 1, 1382400, 0xeba0b5f3 +0, 1527, 1527, 1, 1382400, 0xeba0b5f3 +0, 1528, 1528, 1, 1382400, 0xeba0b5f3 +0, 1530, 1530, 1, 1382400, 0xd6a91770 +0, 1530, 1530, 1, 1382400, 0xd6a91770 +0, 1531, 1531, 1, 1382400, 0xd6a91770 +0, 1532, 1532, 1, 1382400, 0xd6a91770 +0, 1533, 1533, 1, 1382400, 0xd6a91770 +0, 1534, 1534, 1, 1382400, 0xd6a91770 +0, 1535, 1535, 1, 1382400, 0xd6a91770 +0, 1536, 1536, 1, 1382400, 0xd6a91770 +0, 1537, 1537, 1, 1382400, 0xd6a91770 +0, 1538, 1538, 1, 1382400, 0xd6a91770 +0, 1539, 1539, 1, 1382400, 0x00000000 +0, 1540, 1540, 1, 1382400, 0x222f827c +0, 1541, 1541, 1, 1382400, 0x222f827c +0, 1542, 1542, 1, 1382400, 0x222f827c +0, 1543, 1543, 1, 1382400, 0x222f827c +0, 1544, 1544, 1, 1382400, 0x222f827c +0, 1545, 1545, 1, 1382400, 0x222f827c +0, 1546, 1546, 1, 1382400, 0x222f827c +0, 1547, 1547, 1, 1382400, 0x222f827c +0, 1548, 1548, 1, 1382400, 0x222f827c +0, 1549, 1549, 1, 1382400, 0x00000000 +0, 1550, 1550, 1, 1382400, 0x00000000 +0, 1551, 1551, 1, 1382400, 0x00000000 +0, 1552, 1552, 1, 1382400, 0x00000000 +0, 1553, 1553, 1, 1382400, 0x00000000 +0, 1554, 1554, 1, 1382400, 0x00000000 +0, 1555, 1555, 1, 1382400, 0x00000000 +0, 1556, 1556, 1, 1382400, 0x00000000 +0, 1557, 1557, 1, 1382400, 0x00000000 +0, 1558, 1558, 1, 1382400, 0x00000000 +0, 1559, 1559, 1, 1382400, 0x00000000 +0, 1560, 1560, 1, 1382400, 0x00000000 +0, 1561, 1561, 1, 1382400, 0x00000000 +0, 1562, 1562, 1, 1382400, 0x00000000 +0, 1563, 1563, 1, 1382400, 0x00000000 +0, 1564, 1564, 1, 1382400, 0x00000000 +0, 1565, 1565, 1, 1382400, 0x00000000 +0, 1566, 1566, 1, 1382400, 0x00000000 +0, 1567, 1567, 1, 1382400, 0x00000000 +0, 1568, 1568, 1, 1382400, 0x00000000 +0, 1569, 1569, 1, 1382400, 0x00000000 +0, 1570, 1570, 1, 1382400, 0x00000000 +0, 1571, 1571, 1, 1382400, 0x00000000 +0, 1572, 1572, 1, 1382400, 0x00000000 +0, 1573, 1573, 1, 1382400, 0x00000000 +0, 1574, 1574, 1, 1382400, 0x00000000 +0, 1575, 1575, 1, 1382400, 0x00000000 +0, 1576, 1576, 1, 1382400, 0x00000000 +0, 1577, 1577, 1, 1382400, 0x00000000 +0, 1578, 1578, 1, 1382400, 0x00000000 +0, 1579, 1579, 1, 1382400, 0x00000000 +0, 1580, 1580, 1, 1382400, 0x00000000 +0, 1581, 1581, 1, 1382400, 0x00000000 +0, 1582, 1582, 1, 1382400, 0x00000000 +0, 1583, 1583, 1, 1382400, 0x00000000 +0, 1584, 1584, 1, 1382400, 0x00000000 +0, 1585, 1585, 1, 1382400, 0x00000000 +0, 1586, 1586, 1, 1382400, 0x00000000 +0, 1587, 1587, 1, 1382400, 0x00000000 +0, 1588, 1588, 1, 1382400, 0x00000000 +0, 1589, 1589, 1, 1382400, 0x00000000 +0, 1590, 1590, 1, 1382400, 0x00000000 +0, 1591, 1591, 1, 1382400, 0x00000000 +0, 1592, 1592, 1, 1382400, 0x00000000 +0, 1593, 1593, 1, 1382400, 0x00000000 +0, 1594, 1594, 1, 1382400, 0x00000000 +0, 1595, 1595, 1, 1382400, 0x00000000 +0, 1596, 1596, 1, 1382400, 0x00000000 +0, 1597, 1597, 1, 1382400, 0x00000000 +0, 1598, 1598, 1, 1382400, 0x00000000 +0, 1599, 1599, 1, 1382400, 0x00000000 +0, 1600, 1600, 1, 1382400, 0x00000000 +0, 1601, 1601, 1, 1382400, 0x00000000 +0, 1602, 1602, 1, 1382400, 0x00000000 +0, 1603, 1603, 1, 1382400, 0x00000000 +0, 1604, 1604, 1, 1382400, 0x00000000 +0, 1606, 1606, 1, 1382400, 0x3270f4ff +0, 1606, 1606, 1, 1382400, 0x3270f4ff +0, 1607, 1607, 1, 1382400, 0x3270f4ff +0, 1608, 1608, 1, 1382400, 0x3270f4ff +0, 1609, 1609, 1, 1382400, 0x3270f4ff +0, 1610, 1610, 1, 1382400, 0x3270f4ff +0, 1611, 1611, 1, 1382400, 0x3270f4ff +0, 1612, 1612, 1, 1382400, 0x3270f4ff +0, 1613, 1613, 1, 1382400, 0x3270f4ff +0, 1614, 1614, 1, 1382400, 0x3270f4ff +0, 1615, 1615, 1, 1382400, 0x3270f4ff +0, 1617, 1617, 1, 1382400, 0x40813cb3 +0, 1617, 1617, 1, 1382400, 0x40813cb3 +0, 1618, 1618, 1, 1382400, 0x40813cb3 +0, 1619, 1619, 1, 1382400, 0x40813cb3 +0, 1620, 1620, 1, 1382400, 0x40813cb3 +0, 1621, 1621, 1, 1382400, 0x40813cb3 +0, 1622, 1622, 1, 1382400, 0x40813cb3 +0, 1623, 1623, 1, 1382400, 0x9d8fde41 +0, 1624, 1624, 1, 1382400, 0x9d8fde41 +0, 1625, 1625, 1, 1382400, 0x9d8fde41 +0, 1626, 1626, 1, 1382400, 0x9d8fde41 +0, 1627, 1627, 1, 1382400, 0x9d8fde41 +0, 1628, 1628, 1, 1382400, 0x9d8fde41 +0, 1629, 1629, 1, 1382400, 0x9d8fde41 +0, 1630, 1630, 1, 1382400, 0x9d8fde41 +0, 1631, 1631, 1, 1382400, 0x9d8fde41 +0, 1632, 1632, 1, 1382400, 0x00000000 +0, 1633, 1633, 1, 1382400, 0x00000000 +0, 1634, 1634, 1, 1382400, 0x00000000 +0, 1636, 1636, 1, 1382400, 0xc6d7a701 +0, 1636, 1636, 1, 1382400, 0xc6d7a701 +0, 1637, 1637, 1, 1382400, 0xc6d7a701 +0, 1638, 1638, 1, 1382400, 0xc6d7a701 +0, 1639, 1639, 1, 1382400, 0xc6d7a701 +0, 1640, 1640, 1, 1382400, 0xc6d7a701 +0, 1642, 1642, 1, 1382400, 0x9d45f2dc +0, 1642, 1642, 1, 1382400, 0x9d45f2dc +0, 1643, 1643, 1, 1382400, 0x9d45f2dc +0, 1644, 1644, 1, 1382400, 0x9d45f2dc +0, 1645, 1645, 1, 1382400, 0x9d45f2dc +0, 1646, 1646, 1, 1382400, 0x9d45f2dc +0, 1647, 1647, 1, 1382400, 0x9d45f2dc +0, 1648, 1648, 1, 1382400, 0x9d45f2dc +0, 1649, 1649, 1, 1382400, 0x00000000 +0, 1650, 1650, 1, 1382400, 0x8525ee40 +0, 1651, 1651, 1, 1382400, 0x8525ee40 +0, 1652, 1652, 1, 1382400, 0x8525ee40 +0, 1653, 1653, 1, 1382400, 0x8525ee40 +0, 1654, 1654, 1, 1382400, 0x8525ee40 +0, 1655, 1655, 1, 1382400, 0x8525ee40 +0, 1656, 1656, 1, 1382400, 0x5b26b98b +0, 1657, 1657, 1, 1382400, 0x5b26b98b +0, 1658, 1658, 1, 1382400, 0x5b26b98b +0, 1659, 1659, 1, 1382400, 0x5b26b98b +0, 1660, 1660, 1, 1382400, 0x5b26b98b +0, 1661, 1661, 1, 1382400, 0x5b26b98b +0, 1662, 1662, 1, 1382400, 0x5b26b98b +0, 1663, 1663, 1, 1382400, 0x5b26b98b +0, 1664, 1664, 1, 1382400, 0x00000000 +0, 1665, 1665, 1, 1382400, 0x51be311f +0, 1666, 1666, 1, 1382400, 0x51be311f +0, 1667, 1667, 1, 1382400, 0x51be311f +0, 1668, 1668, 1, 1382400, 0x51be311f +0, 1669, 1669, 1, 1382400, 0x51be311f +0, 1670, 1670, 1, 1382400, 0x51be311f +0, 1671, 1671, 1, 1382400, 0x51be311f +0, 1672, 1672, 1, 1382400, 0x51be311f +0, 1673, 1673, 1, 1382400, 0x00000000 +0, 1674, 1674, 1, 1382400, 0x00000000 +0, 1675, 1675, 1, 1382400, 0x00000000 +0, 1676, 1676, 1, 1382400, 0x00000000 +0, 1677, 1677, 1, 1382400, 0x00000000 +0, 1678, 1678, 1, 1382400, 0x00000000 +0, 1679, 1679, 1, 1382400, 0x00000000 +0, 1680, 1680, 1, 1382400, 0x00000000 +0, 1681, 1681, 1, 1382400, 0x00000000 +0, 1682, 1682, 1, 1382400, 0x00000000 +0, 1683, 1683, 1, 1382400, 0x00000000 +0, 1684, 1684, 1, 1382400, 0x00000000 +0, 1685, 1685, 1, 1382400, 0x00000000 +0, 1686, 1686, 1, 1382400, 0x00000000 +0, 1687, 1687, 1, 1382400, 0x00000000 +0, 1688, 1688, 1, 1382400, 0x00000000 +0, 1689, 1689, 1, 1382400, 0x00000000 +0, 1690, 1690, 1, 1382400, 0x00000000 +0, 1691, 1691, 1, 1382400, 0x00000000 +0, 1692, 1692, 1, 1382400, 0x00000000 +0, 1693, 1693, 1, 1382400, 0x00000000 +0, 1694, 1694, 1, 1382400, 0x00000000 +0, 1695, 1695, 1, 1382400, 0x00000000 +0, 1696, 1696, 1, 1382400, 0x00000000 +0, 1697, 1697, 1, 1382400, 0x00000000 +0, 1698, 1698, 1, 1382400, 0x00000000 +0, 1699, 1699, 1, 1382400, 0x00000000 +0, 1700, 1700, 1, 1382400, 0x00000000 +0, 1701, 1701, 1, 1382400, 0x00000000 +0, 1702, 1702, 1, 1382400, 0x00000000 +0, 1703, 1703, 1, 1382400, 0x00000000 +0, 1704, 1704, 1, 1382400, 0x00000000 +0, 1705, 1705, 1, 1382400, 0x00000000 +0, 1706, 1706, 1, 1382400, 0x00000000 +0, 1707, 1707, 1, 1382400, 0x00000000 +0, 1708, 1708, 1, 1382400, 0x00000000 +0, 1709, 1709, 1, 1382400, 0x00000000 +0, 1710, 1710, 1, 1382400, 0x00000000 +0, 1711, 1711, 1, 1382400, 0x00000000 +0, 1713, 1713, 1, 1382400, 0x00a4f2a3 +0, 1713, 1713, 1, 1382400, 0x00a4f2a3 +0, 1714, 1714, 1, 1382400, 0x00a4f2a3 +0, 1715, 1715, 1, 1382400, 0x00a4f2a3 +0, 1716, 1716, 1, 1382400, 0x00a4f2a3 +0, 1717, 1717, 1, 1382400, 0x00a4f2a3 +0, 1718, 1718, 1, 1382400, 0x00a4f2a3 +0, 1719, 1719, 1, 1382400, 0x00a4f2a3 +0, 1720, 1720, 1, 1382400, 0x00a4f2a3 +0, 1721, 1721, 1, 1382400, 0x00a4f2a3 +0, 1722, 1722, 1, 1382400, 0x00000000 +0, 1723, 1723, 1, 1382400, 0x00000000 +0, 1724, 1724, 1, 1382400, 0x00000000 +0, 1725, 1725, 1, 1382400, 0x00000000 +0, 1726, 1726, 1, 1382400, 0x00000000 +0, 1727, 1727, 1, 1382400, 0x00000000 +0, 1728, 1728, 1, 1382400, 0x00000000 +0, 1729, 1729, 1, 1382400, 0x00000000 +0, 1730, 1730, 1, 1382400, 0x00000000 +0, 1731, 1731, 1, 1382400, 0x00000000 +0, 1732, 1732, 1, 1382400, 0x00000000 +0, 1734, 1734, 1, 1382400, 0x40a445e8 +0, 1734, 1734, 1, 1382400, 0x40a445e8 +0, 1735, 1735, 1, 1382400, 0x40a445e8 +0, 1736, 1736, 1, 1382400, 0x40a445e8 +0, 1737, 1737, 1, 1382400, 0x40a445e8 +0, 1738, 1738, 1, 1382400, 0x40a445e8 +0, 1739, 1739, 1, 1382400, 0x40a445e8 +0, 1740, 1740, 1, 1382400, 0x40a445e8 +0, 1741, 1741, 1, 1382400, 0x40a445e8 +0, 1742, 1742, 1, 1382400, 0x00000000 +0, 1743, 1743, 1, 1382400, 0x00000000 +0, 1744, 1744, 1, 1382400, 0x00000000 +0, 1745, 1745, 1, 1382400, 0x00000000 +0, 1746, 1746, 1, 1382400, 0x00000000 +0, 1747, 1747, 1, 1382400, 0x00000000 +0, 1748, 1748, 1, 1382400, 0x00000000 +0, 1749, 1749, 1, 1382400, 0x00000000 +0, 1750, 1750, 1, 1382400, 0x00000000 +0, 1751, 1751, 1, 1382400, 0x00000000 +0, 1752, 1752, 1, 1382400, 0x00000000 +0, 1753, 1753, 1, 1382400, 0x00000000 +0, 1754, 1754, 1, 1382400, 0x00000000 +0, 1755, 1755, 1, 1382400, 0x00000000 +0, 1756, 1756, 1, 1382400, 0x00000000 +0, 1757, 1757, 1, 1382400, 0x00000000 +0, 1758, 1758, 1, 1382400, 0x00000000 +0, 1759, 1759, 1, 1382400, 0x00000000 +0, 1760, 1760, 1, 1382400, 0x00000000 +0, 1761, 1761, 1, 1382400, 0x00000000 +0, 1762, 1762, 1, 1382400, 0x00000000 +0, 1763, 1763, 1, 1382400, 0x00000000 +0, 1764, 1764, 1, 1382400, 0x00000000 +0, 1765, 1765, 1, 1382400, 0x00000000 +0, 1766, 1766, 1, 1382400, 0x00000000 +0, 1767, 1767, 1, 1382400, 0x00000000 +0, 1768, 1768, 1, 1382400, 0x00000000 +0, 1769, 1769, 1, 1382400, 0x00000000 +0, 1770, 1770, 1, 1382400, 0x00000000 +0, 1771, 1771, 1, 1382400, 0x00000000 +0, 1772, 1772, 1, 1382400, 0x00000000 +0, 1773, 1773, 1, 1382400, 0x00000000 +0, 1774, 1774, 1, 1382400, 0x00000000 +0, 1775, 1775, 1, 1382400, 0x00000000 +0, 1776, 1776, 1, 1382400, 0x00000000 +0, 1777, 1777, 1, 1382400, 0x00000000 +0, 1778, 1778, 1, 1382400, 0x00000000 +0, 1779, 1779, 1, 1382400, 0x00000000 +0, 1780, 1780, 1, 1382400, 0x00000000 +0, 1781, 1781, 1, 1382400, 0x00000000 +0, 1782, 1782, 1, 1382400, 0x00000000 +0, 1783, 1783, 1, 1382400, 0x00000000 +0, 1784, 1784, 1, 1382400, 0x00000000 +0, 1785, 1785, 1, 1382400, 0x00000000 +0, 1786, 1786, 1, 1382400, 0x00000000 +0, 1788, 1788, 1, 1382400, 0x43ef5128 +0, 1788, 1788, 1, 1382400, 0x43ef5128 +0, 1789, 1789, 1, 1382400, 0x43ef5128 +0, 1790, 1790, 1, 1382400, 0x43ef5128 +0, 1791, 1791, 1, 1382400, 0x43ef5128 +0, 1792, 1792, 1, 1382400, 0x43ef5128 +0, 1793, 1793, 1, 1382400, 0x43ef5128 +0, 1794, 1794, 1, 1382400, 0x43ef5128 +0, 1795, 1795, 1, 1382400, 0x43ef5128 +0, 1796, 1796, 1, 1382400, 0x43ef5128 +0, 1797, 1797, 1, 1382400, 0x43ef5128 +0, 1798, 1798, 1, 1382400, 0x43ef5128 +0, 1799, 1799, 1, 1382400, 0x3c3e3819 +0, 1800, 1800, 1, 1382400, 0x3c3e3819 +0, 1801, 1801, 1, 1382400, 0x3c3e3819 +0, 1802, 1802, 1, 1382400, 0x3c3e3819 +0, 1803, 1803, 1, 1382400, 0x3c3e3819 +0, 1804, 1804, 1, 1382400, 0x3c3e3819 +0, 1805, 1805, 1, 1382400, 0x00000000 diff --git a/tests/ref/fate/sub2video_time_limited b/tests/ref/fate/sub2video_time_limited index 9fb6fb06f9..2fbd91a7eb 100644 --- a/tests/ref/fate/sub2video_time_limited +++ b/tests/ref/fate/sub2video_time_limited @@ -1,8 +1,80 @@ -#tb 0: 1/25 +#tb 0: 1/5 #media_type 0: video #codec_id 0: rawvideo #dimensions 0: 1920x1080 -#sar 0: 0/1 -0, 2, 2, 1, 8294400, 0x00000000 +#sar 0: 1/1 +0, 0, 0, 1, 8294400, 0xa87c518f +0, 1, 1, 1, 8294400, 0xa87c518f 0, 2, 2, 1, 8294400, 0xa87c518f +0, 3, 3, 1, 8294400, 0xa87c518f +0, 4, 4, 1, 8294400, 0xa87c518f +0, 5, 5, 1, 8294400, 0xa87c518f +0, 6, 6, 1, 8294400, 0xa87c518f +0, 7, 7, 1, 8294400, 0xa87c518f +0, 8, 8, 1, 8294400, 0xa87c518f +0, 9, 9, 1, 8294400, 0xa87c518f 0, 10, 10, 1, 8294400, 0xa87c518f +0, 11, 11, 1, 8294400, 0xa87c518f +0, 12, 12, 1, 8294400, 0xa87c518f +0, 13, 13, 1, 8294400, 0xa87c518f +0, 14, 14, 1, 8294400, 0xa87c518f +0, 15, 15, 1, 8294400, 0xa87c518f +0, 16, 16, 1, 8294400, 0xa87c518f +0, 17, 17, 1, 8294400, 0xa87c518f +0, 18, 18, 1, 8294400, 0xa87c518f +0, 19, 19, 1, 8294400, 0xa87c518f +0, 20, 20, 1, 8294400, 0xa87c518f +0, 21, 21, 1, 8294400, 0xa87c518f +0, 22, 22, 1, 8294400, 0xa87c518f +0, 23, 23, 1, 8294400, 0xa87c518f +0, 24, 24, 1, 8294400, 0xa87c518f +0, 25, 25, 1, 8294400, 0xa87c518f +0, 26, 26, 1, 8294400, 0xa87c518f +0, 27, 27, 1, 8294400, 0xa87c518f +0, 28, 28, 1, 8294400, 0xa87c518f +0, 29, 29, 1, 8294400, 0xa87c518f +0, 30, 30, 1, 8294400, 0xa87c518f +0, 31, 31, 1, 8294400, 0xa87c518f +0, 32, 32, 1, 8294400, 0xa87c518f +0, 33, 33, 1, 8294400, 0xa87c518f +0, 34, 34, 1, 8294400, 0xa87c518f +0, 35, 35, 1, 8294400, 0xa87c518f +0, 36, 36, 1, 8294400, 0xa87c518f +0, 37, 37, 1, 8294400, 0xa87c518f +0, 38, 38, 1, 8294400, 0xa87c518f +0, 39, 39, 1, 8294400, 0xa87c518f +0, 40, 40, 1, 8294400, 0xa87c518f +0, 41, 41, 1, 8294400, 0xa87c518f +0, 42, 42, 1, 8294400, 0xa87c518f +0, 43, 43, 1, 8294400, 0xa87c518f +0, 44, 44, 1, 8294400, 0xa87c518f +0, 45, 45, 1, 8294400, 0xa87c518f +0, 46, 46, 1, 8294400, 0xa87c518f +0, 47, 47, 1, 8294400, 0xa87c518f +0, 48, 48, 1, 8294400, 0xa87c518f +0, 49, 49, 1, 8294400, 0xa87c518f +0, 50, 50, 1, 8294400, 0xa87c518f +0, 51, 51, 1, 8294400, 0xa87c518f +0, 52, 52, 1, 8294400, 0xa87c518f +0, 53, 53, 1, 8294400, 0xa87c518f +0, 54, 54, 1, 8294400, 0xa87c518f +0, 55, 55, 1, 8294400, 0xa87c518f +0, 56, 56, 1, 8294400, 0xa87c518f +0, 57, 57, 1, 8294400, 0xa87c518f +0, 58, 58, 1, 8294400, 0xa87c518f +0, 59, 59, 1, 8294400, 0xa87c518f +0, 60, 60, 1, 8294400, 0xa87c518f +0, 61, 61, 1, 8294400, 0xa87c518f +0, 62, 62, 1, 8294400, 0xa87c518f +0, 63, 63, 1, 8294400, 0xa87c518f +0, 64, 64, 1, 8294400, 0xa87c518f +0, 65, 65, 1, 8294400, 0xa87c518f +0, 66, 66, 1, 8294400, 0xa87c518f +0, 67, 67, 1, 8294400, 0xa87c518f +0, 68, 68, 1, 8294400, 0xa87c518f +0, 69, 69, 1, 8294400, 0xa87c518f +0, 70, 70, 1, 8294400, 0xa87c518f +0, 71, 71, 1, 8294400, 0xa87c518f +0, 72, 72, 1, 8294400, 0xa87c518f +0, 73, 73, 1, 8294400, 0xa87c518f +0, 74, 74, 1, 8294400, 0xa87c518f From patchwork Tue Oct 25 09:13:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 38980 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2686430pzb; Tue, 25 Oct 2022 02:17:49 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4s6qkGcrUsBC+RHfLdm142K0cP9mLXIyHKYLf6H8seX8Q8HQnvGU5nIs3S8skXADx24I3W X-Received: by 2002:a17:907:3ea9:b0:78d:fdf0:88fe with SMTP id hs41-20020a1709073ea900b0078dfdf088femr32742539ejc.667.1666689468903; Tue, 25 Oct 2022 02:17:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666689468; cv=none; d=google.com; s=arc-20160816; b=pGH6lyFkvOWeWxuahc5fJ7R91iCml704I0275RSoXXUafFgdKFStuj6Eu2SC2+XyZi KkwkLvawSy2I1gdToLNKstWwKPTM4xn+3lbNcDcsEUMxEU87DeZZPp90R/WhaOf4lpIX jr9wsYzzg1sRPtWdHGoh1wl1HA+gMUsHg/lwyBaHd+BkphkAEvDPyYJEDjfExw9mB3H8 hOXh7EmFR6j+v/a18CfgdZTZnheAWegbFj4iHTbtxL3u7TzlFTVL6c55CSMH2L13NZmB lR10b0/MU5pKUgIXlAAnKXPUCXtYnpdnbmtg7nGpVhruCyjD5BTTs4HHsYAycYyD8B3z KtFg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:references :in-reply-to:message-id:from:dkim-signature:delivered-to; bh=qQAb1yx8ZGCgmBkmTUZlZL/f8QYRB0RqnQsPHzFsQyM=; b=THXC+saABgABfOoarpP6IyQw8p7r7dpteow/3dSgTeZ9rJGgc4qpH/68qTPHNcWjdg lCJO01HdCE9Ok9j3f0VMKZ33ipEmVDWnsx5EJaGGAJHSYaRdi7c/nqf3h/CaX4rdCSjP eFxrV49vwyWFIOkJx+jLiTxV3Jkbi+PkHQdBVm0QOe/4hbg8xC1bg/ot1EkZ1Ek3tIV1 0YV6503TOAmRlTq+nFxgAfMBv3BxzALWU2CfkufS0RjknzRnEAeIHTBXNm42NeZbo+oI 11u/ghDxmYExSILxN8OQ+tukkZMdTI6FFkqx1JCTGlD384WdYrro/zMZJaddAh0Jcx8l 8vxA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=dmF6VOmH; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id gt12-20020a1709072d8c00b0078d1faeb619si2883517ejc.777.2022.10.25.02.17.48; Tue, 25 Oct 2022 02:17:48 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=dmF6VOmH; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 47EBB68BD9F; Tue, 25 Oct 2022 12:14:22 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com [209.85.210.171]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B208C68BD84 for ; Tue, 25 Oct 2022 12:14:20 +0300 (EEST) Received: by mail-pf1-f171.google.com with SMTP id g62so7227587pfb.10 for ; Tue, 25 Oct 2022 02:14:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=d0XqOcvlQCaW2AQnlWuouX8iEt5sctwyaH4eLUsrptw=; b=dmF6VOmH6XKaucGaiLX+MTMSRM9JDQ6NE9CuNDd0NhGUGwXKFbbYsozapr7etDbb1r yENcSxklG4KY9HdV3MK+mxWvZIGYrk2oe8V1XojHD5HlvTI5t6LAjQtmr3SWvYy2yuP9 OdR9wJpt8sLnU6aLI7I9vLhMkflN4lLDYfYfRQjtuZ0dqwnMgZMwC84G0e3xaqzra0t0 EDi8b7C3QQhSaApNx5/sX2Vtj9RxCBKfEHfRkToQDapXk6s5e3688htdLzSM8YLLcpLb UGLpKjobRqFJMxnUbbNpGXA6WwsRYXkbFExc1Mic7c5rHe/UNAJfMO2kn06cRE1NDGXQ 0tSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=d0XqOcvlQCaW2AQnlWuouX8iEt5sctwyaH4eLUsrptw=; b=IkJcGCKMOVtyvOEMXlMo3eRkd/jOpONu5hH4KSi/GOzdk7X8ypQZ5Uus/MIRLZ+QRX FcOJ42Wqj1X6YTChiEsAzZpbZWaZ9GmlvRNw3AeoT8MiYYHrC8pxro3UkQPJFGw/jm33 F6lgnd9J9pE3DQL3cg+GykwuVgKxK7dl7HeTT0Z/AFT8rpiz8z4IxJgZVK9cQ+KBcJCw blcGfHrEDxJr0ArXyn+0DjKWV5e1N321WfBeXOdBN4D0f+S4EavsVroVfo1SYDfhjNIK YZ+WN/ZE1rRjVl0w6EgbgR3y49TJTfRB+c0yIZWdU7X1GHG9R3jduhOUYbWYMKmBbahK eRyw== X-Gm-Message-State: ACrzQf0AXPj2yPhaiLw8vsbtG8r229QkE13rgJtPIdVD1nN79gf8b4se QYHtDUV3sAUV1BtANADTbXKeTft80kk= X-Received: by 2002:a05:6a02:47:b0:46e:ea48:9543 with SMTP id az7-20020a056a02004700b0046eea489543mr12345033pgb.6.1666689258746; Tue, 25 Oct 2022 02:14:18 -0700 (PDT) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id oc12-20020a17090b1c0c00b00212735c8898sm5118299pjb.30.2022.10.25.02.14.18 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Oct 2022 02:14:18 -0700 (PDT) From: softworkz X-Google-Original-From: softworkz Message-Id: In-Reply-To: References: Date: Tue, 25 Oct 2022 09:13:46 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v9 25/25] avcodec/dvbsubdec: Fix conditions for fallback to default resolution X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: softworkz Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Rg2FnurnyPD7 From: softworkz The previous code expected a segment of type CLUT definition to exist in order to accept a set of segments to be complete. This was an incorrect assumption as the presence of a CLUT segment is not mandatory. (version 1.6.1 of the spec is probably a bit more clear about this than earlier versions: https://www.etsi.org/deliver/etsi_en/ 300700_300799/300743/01.06.01_20/en_300743v010601a.pdf) The flawed condition prevented proper fallback to using the default resolution for the decoding context. Signed-off-by: softworkz --- libavcodec/dvbsubdec.c | 51 +++++++++++++++++++++++++----------------- 1 file changed, 30 insertions(+), 21 deletions(-) diff --git a/libavcodec/dvbsubdec.c b/libavcodec/dvbsubdec.c index c864ba3fb6..4df350574d 100644 --- a/libavcodec/dvbsubdec.c +++ b/libavcodec/dvbsubdec.c @@ -34,7 +34,7 @@ #define DVBSUB_CLUT_SEGMENT 0x12 #define DVBSUB_OBJECT_SEGMENT 0x13 #define DVBSUB_DISPLAYDEFINITION_SEGMENT 0x14 -#define DVBSUB_DISPLAY_SEGMENT 0x80 +#define DVBSUB_END_DISPLAY_SEGMENT 0x80 #define cm (ff_crop_tab + MAX_NEG_CROP) @@ -1450,8 +1450,12 @@ static int dvbsub_decode(AVCodecContext *avctx, AVSubtitle *sub, int segment_length; int i; int ret = 0; - int got_segment = 0; - int got_dds = 0; + //int got_segment = 0; + int got_page = 0; + int got_region = 0; + int got_object = 0; + int got_end_display = 0; + int got_displaydef = 0; ff_dlog(avctx, "DVB sub packet:\n"); @@ -1496,34 +1500,28 @@ static int dvbsub_decode(AVCodecContext *avctx, AVSubtitle *sub, switch (segment_type) { case DVBSUB_PAGE_SEGMENT: ret = dvbsub_parse_page_segment(avctx, p, segment_length, sub, got_sub_ptr); - got_segment |= 1; + got_page = 1; break; case DVBSUB_REGION_SEGMENT: ret = dvbsub_parse_region_segment(avctx, p, segment_length); - got_segment |= 2; + got_region = 1; break; case DVBSUB_CLUT_SEGMENT: ret = dvbsub_parse_clut_segment(avctx, p, segment_length); if (ret < 0) goto end; - got_segment |= 4; break; case DVBSUB_OBJECT_SEGMENT: ret = dvbsub_parse_object_segment(avctx, p, segment_length); - got_segment |= 8; + got_object = 1; break; case DVBSUB_DISPLAYDEFINITION_SEGMENT: ret = dvbsub_parse_display_definition_segment(avctx, p, segment_length); - got_dds = 1; + got_displaydef = 1; break; - case DVBSUB_DISPLAY_SEGMENT: + case DVBSUB_END_DISPLAY_SEGMENT: ret = dvbsub_display_end_segment(avctx, p, segment_length, sub, got_sub_ptr); - if (got_segment == 15 && !got_dds && !avctx->width && !avctx->height) { - // Default from ETSI EN 300 743 V1.3.1 (7.2.1) - avctx->width = 720; - avctx->height = 576; - } - got_segment |= 16; + got_end_display = 1; break; default: ff_dlog(avctx, "Subtitling segment type 0x%x, page id %d, length %d\n", @@ -1536,13 +1534,24 @@ static int dvbsub_decode(AVCodecContext *avctx, AVSubtitle *sub, p += segment_length; } - // Some streams do not send a display segment but if we have all the other - // segments then we need no further data. - if (got_segment == 15) { - av_log(avctx, AV_LOG_DEBUG, "Missing display_end_segment, emulating\n"); - dvbsub_display_end_segment(avctx, p, 0, sub, got_sub_ptr); - } + // Even though not mandated by the spec, we're imposing a minimum requirement + // for a useful packet to have at least one page, region and object segment. + if (got_page && got_region && got_object && got_end_display) { + + if (!got_displaydef && !avctx->width && !avctx->height) { + // Default from ETSI EN 300 743 V1.3.1 (7.2.1) + avctx->width = 720; + avctx->height = 576; + } + + // Some streams do not send an end-of-display segment but if we have all the other + // segments then we need no further data. + if (!got_end_display) { + av_log(avctx, AV_LOG_DEBUG, "Missing display_end_segment, emulating\n"); + dvbsub_display_end_segment(avctx, p, 0, sub, got_sub_ptr); + } + } end: if (ret < 0) { return ret;