From patchwork Sun Sep 12 03:21:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 30175 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp2862807iov; Sat, 11 Sep 2021 20:21:55 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwYZSSMVcRLGLB6JtOsmt9pWJObR7fQjPeOF9iUWf9xWHbDlOTm2H2ZQAZnLyTc7HDbkp1u X-Received: by 2002:a05:6402:205:: with SMTP id t5mr6050381edv.81.1631416915472; Sat, 11 Sep 2021 20:21:55 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id z29si925903ejl.555.2021.09.11.20.21.55; Sat, 11 Sep 2021 20:21:55 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=moZwSmMz; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0335368A8F1; Sun, 12 Sep 2021 06:21:44 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10olkn2062.outbound.protection.outlook.com [40.92.41.62]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A0B1268A899 for ; Sun, 12 Sep 2021 06:21:37 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=THPlxbuLKLkXtdtQnPsI4mlx/rY9xuTDlqXi22EBtYZWlgacNKtEzz+PC1QBy9JRsjUn9vp3+Xqlq7tNFwr9aYJF7H+2k4jl0V2sbcJSO8J+DXRHwLZmDBe6g0tBuWIgVTbvhLb9jb6MeWu8DOCyp7BPpLr6iRa9FFjBn3n+HVn5Ap1YWyolQmB5v2Wursau3ZncBL4vvft5r7SR7Nc3fbtc4S4Jckyfg4GEosy446PIqOEOtAD+sTF4crLjbrHPGqwnyzao9TQv2e2ZbR2NBqWvbNpU/hZNfBpjZtpkkpLlh+xDmApxPtsVeZV/sxVyuBN4b8KRh69SX9VVKobi/Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=OsuaGWm8RukLdvOdtGCl2lh/WC2K1i1y+PDsKd2u/Uk=; b=D9I/cm+EgRR6qhWanlaZqa0aZVAJQCbYWgpREbeWvecoZ1XX5mM8VucmVJHz5sNdRi3M44MakQGIhw3OSMNG4s7NW0GMON11Jf///AKXQoGqvuz+8zr4oR3jhcK4ArwMr+bYmjt20KOzbhDU/l44fURbYYxqumdMc2lM6XTTyjGkoKCOitha00t0PTmzYUFGciqVKOLm1rqUCF9EdScJRutFqc6malCC6KHNTvM52Qxu8NiCPpPXpEf3E/s/ieGCbVHjcWsGKkg0FuZVpttxoiHJ1tWjU5HrtsbdjxD3jPKpvGDGtWY6QWqmz9JAN21jFlJWqmb+DV6K5q5gRZvymQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OsuaGWm8RukLdvOdtGCl2lh/WC2K1i1y+PDsKd2u/Uk=; b=moZwSmMzTXxmS2cIDwPnDyj9Zv5my9CQpuZXZXZu7bkLEEP4nJSwj2RZO0dGGj2lpHWorTPmVeVEL/sMLXPO5zyjHtvBlsefiGVihcXhX2wcYKJZ/qFk+f/uf3gxXl6zFARnSYfjVkS26vNxaPIO3YVAmRQds3FHtdXcWU7sON+yBph+n4WfTwGz/oR6tZJIvcTl5/lqLL/bRHAP4EzpTFqBnqZhx6NJDfJuv0onN5LNBR6pmAwGC163l7pcug4I9MJ7zmayk2X8hN8XAy8S8zJ6W4FnT9GSKbrJ3Ougqx4fRMCVf0Q6YPgxyOKf8P51ZdRFHk4v9SC0+a3px6AuqA== Received: from MN2PR04MB5981.namprd04.prod.outlook.com (2603:10b6:208:da::10) by MN2PR04MB5728.namprd04.prod.outlook.com (2603:10b6:208:3d::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Sun, 12 Sep 2021 03:21:31 +0000 Received: from MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb]) by MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb%5]) with mapi id 15.20.4500.018; Sun, 12 Sep 2021 03:21:31 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v5 01/12] avutil/frame: Subtitle Filtering - Add AVMediaType property to AVFrame Thread-Index: AQHXp4VHNRKJaoAhykKCVkdSxIqNmg== Date: Sun, 12 Sep 2021 03:21:31 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [SZdXHzxrV7dKwlUNPNMyvoi9QXevPCVq9L9cHk2NTFk=] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 2b524c38-70c5-4878-2964-08d9759c6a82 x-ms-traffictypediagnostic: MN2PR04MB5728: x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 1zeXejXXxpqYCClSXmrLZB6A0FAtSGMD7DAEaAY5ajRSB1eiwmXyrnyw7J2fTHO46LTfvP956g9MCpQ7ObPwfwn86Ogk0xc7oL/IB/X2VguhUV/JtlbOp64P5yZEuE9ZD9AXVK3g/9xzpTKMJScb8bj/FfqPnl/nl+qSVQvKgJ7LqKB3ihroSJBN25WztFxW9Ueh5ZnV/fFYuCsmoJCFp6fu2e6iJPUj0Bx3PQBCqGCNCADOzsqWnMGUXjMuEn/HVwQL2rmYZDz5EH7OKuCTEfI7csAAPZjmJVJxPKsKKV4fYZWgp0ubH/9Fv7N5C7Ou0MREPCA6sgDU4DY8dRmNIVEq9cb8LxbKeDerDC77BAyebt6hug25F+YmLUFENDYY9pKMVndlr7NtJk7KAAmnlz+QfQNJUAMYSO14LJj/86M6PZecxlhcVkEb7mV+RQNC x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: on83DoMDMK9Kuy9lPTB7x/wuoY7k0cb6eQq5Fs1CJfskRQSLgK0jL61OmN5UW/sX7tpqUZilpOUEgiXCRPqv98dmNsBvDnssqzGhxnxTCUWxPF8Cp5F5wgSN26MpHAPYUTspnnQnxfYuRigZA3ejhQ== x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-3174-20-msonline-outlook-529c7.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR04MB5981.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 2b524c38-70c5-4878-2964-08d9759c6a82 X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Sep 2021 03:21:31.1039 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR04MB5728 Subject: [FFmpeg-devel] [PATCH v5 01/12] avutil/frame: Subtitle Filtering - Add AVMediaType property to AVFrame X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Wsu+s8ORDVDw This is the root commit for adding subtitle filtering capabilities. Adding the media type property to AVFrame replaces the previous way of distinction which was based on checking width and height to determine whether a frame is audio or video. Signed-off-by: softworkz --- libavutil/frame.c | 85 ++++++++++++++++++++++++++++++++++++--------- libavutil/frame.h | 39 +++++++++++++++++++-- libavutil/version.h | 2 +- 3 files changed, 106 insertions(+), 20 deletions(-) diff --git a/libavutil/frame.c b/libavutil/frame.c index b0ceaf7145..ef2867d318 100644 --- a/libavutil/frame.c +++ b/libavutil/frame.c @@ -103,6 +103,7 @@ AVFrame *av_frame_alloc(void) if (!frame) return NULL; + frame->type = AVMEDIA_TYPE_UNKNOWN; frame->extended_data = NULL; get_frame_defaults(frame); @@ -244,22 +245,37 @@ static int get_audio_buffer(AVFrame *frame, int align) } int av_frame_get_buffer(AVFrame *frame, int align) +{ + if (frame->width > 0 && frame->height > 0) + frame->type = AVMEDIA_TYPE_VIDEO; + else if (frame->nb_samples > 0 && (frame->channel_layout || frame->channels > 0)) + frame->type = AVMEDIA_TYPE_AUDIO; + + return av_frame_get_buffer2(frame, align); +} + +int av_frame_get_buffer2(AVFrame *frame, int align) { if (frame->format < 0) return AVERROR(EINVAL); - if (frame->width > 0 && frame->height > 0) + switch(frame->type) { + case AVMEDIA_TYPE_VIDEO: return get_video_buffer(frame, align); - else if (frame->nb_samples > 0 && (frame->channel_layout || frame->channels > 0)) + case AVMEDIA_TYPE_AUDIO: return get_audio_buffer(frame, align); - - return AVERROR(EINVAL); + case AVMEDIA_TYPE_SUBTITLE: + return 0; + default: + return AVERROR(EINVAL); + } } static int frame_copy_props(AVFrame *dst, const AVFrame *src, int force_copy) { int ret, i; + dst->type = src->type; dst->key_frame = src->key_frame; dst->pict_type = src->pict_type; dst->sample_aspect_ratio = src->sample_aspect_ratio; @@ -331,6 +347,7 @@ int av_frame_ref(AVFrame *dst, const AVFrame *src) av_assert1(dst->width == 0 && dst->height == 0); av_assert1(dst->channels == 0); + dst->type = src->type; dst->format = src->format; dst->width = src->width; dst->height = src->height; @@ -344,7 +361,7 @@ int av_frame_ref(AVFrame *dst, const AVFrame *src) /* duplicate the frame data if it's not refcounted */ if (!src->buf[0]) { - ret = av_frame_get_buffer(dst, 0); + ret = av_frame_get_buffer2(dst, 0); if (ret < 0) goto fail; @@ -499,6 +516,7 @@ int av_frame_make_writable(AVFrame *frame) return 0; memset(&tmp, 0, sizeof(tmp)); + tmp.type = frame->type; tmp.format = frame->format; tmp.width = frame->width; tmp.height = frame->height; @@ -509,7 +527,7 @@ int av_frame_make_writable(AVFrame *frame) if (frame->hw_frames_ctx) ret = av_hwframe_get_buffer(frame->hw_frames_ctx, &tmp, 0); else - ret = av_frame_get_buffer(&tmp, 0); + ret = av_frame_get_buffer2(&tmp, 0); if (ret < 0) return ret; @@ -544,14 +562,22 @@ AVBufferRef *av_frame_get_plane_buffer(AVFrame *frame, int plane) uint8_t *data; int planes, i; - if (frame->nb_samples) { - int channels = frame->channels; - if (!channels) - return NULL; - CHECK_CHANNELS_CONSISTENCY(frame); - planes = av_sample_fmt_is_planar(frame->format) ? channels : 1; - } else + switch(frame->type) { + case AVMEDIA_TYPE_VIDEO: planes = 4; + break; + case AVMEDIA_TYPE_AUDIO: + { + int channels = frame->channels; + if (!channels) + return NULL; + CHECK_CHANNELS_CONSISTENCY(frame); + planes = av_sample_fmt_is_planar(frame->format) ? channels : 1; + break; + } + default: + return NULL; + } if (plane < 0 || plane >= planes || !frame->extended_data[plane]) return NULL; @@ -675,17 +701,42 @@ static int frame_copy_audio(AVFrame *dst, const AVFrame *src) return 0; } +static int frame_copy_subtitles(AVFrame *dst, const AVFrame *src) +{ + unsigned i; + + dst->format = src->format; + + dst->num_subtitle_rects = src->num_subtitle_rects; + + if (src->num_subtitle_rects) { + dst->subtitle_rects = av_malloc_array(src->num_subtitle_rects, sizeof(AVSubtitleRect *)); + + for (i = 0; i < src->num_subtitle_rects; i++) { + AVSubtitleRect *rect = src->subtitle_rects[i]; + rect->nb_refs++; + dst->subtitle_rects[i] = rect; + } + } + + return 0; +} + int av_frame_copy(AVFrame *dst, const AVFrame *src) { if (dst->format != src->format || dst->format < 0) return AVERROR(EINVAL); - if (dst->width > 0 && dst->height > 0) + switch(dst->type) { + case AVMEDIA_TYPE_VIDEO: return frame_copy_video(dst, src); - else if (dst->nb_samples > 0 && dst->channels > 0) + case AVMEDIA_TYPE_AUDIO: return frame_copy_audio(dst, src); - - return AVERROR(EINVAL); + case AVMEDIA_TYPE_SUBTITLE: + return frame_copy_subtitles(dst, src); + default: + return AVERROR(EINVAL); + } } void av_frame_remove_side_data(AVFrame *frame, enum AVFrameSideDataType type) diff --git a/libavutil/frame.h b/libavutil/frame.h index ff2540a20f..005eed9d18 100644 --- a/libavutil/frame.h +++ b/libavutil/frame.h @@ -271,7 +271,7 @@ typedef struct AVRegionOfInterest { } AVRegionOfInterest; /** - * This structure describes decoded (raw) audio or video data. + * This structure describes decoded (raw) audio, video or subtitle data. * * AVFrame must be allocated using av_frame_alloc(). Note that this only * allocates the AVFrame itself, the buffers for the data must be managed @@ -302,6 +302,13 @@ typedef struct AVRegionOfInterest { */ typedef struct AVFrame { #define AV_NUM_DATA_POINTERS 8 + /** + * Media type of the frame (audio, video, subtitles..) + * + * See AVMEDIA_TYPE_xxx + */ + enum AVMediaType type; + /** * pointer to the picture/channel planes. * This might be different from the first allocated byte @@ -371,7 +378,7 @@ typedef struct AVFrame { /** * format of the frame, -1 if unknown or unset * Values correspond to enum AVPixelFormat for video frames, - * enum AVSampleFormat for audio) + * enum AVSampleFormat for audio, AVSubtitleType for subtitles) */ int format; @@ -721,6 +728,8 @@ void av_frame_move_ref(AVFrame *dst, AVFrame *src); /** * Allocate new buffer(s) for audio or video data. * + * Note: For subtitle data, use av_frame_get_buffer2 + * * The following fields must be set on frame before calling this function: * - format (pixel format for video, sample format for audio) * - width and height for video @@ -743,6 +752,32 @@ void av_frame_move_ref(AVFrame *dst, AVFrame *src); */ int av_frame_get_buffer(AVFrame *frame, int align); +/** + * Allocate new buffer(s) for audio, video or subtitle data. + * + * The following fields must be set on frame before calling this function: + * - format (pixel format for video, sample format for audio) + * - width and height for video + * - nb_samples and channel_layout for audio + * - type (AVMediaType) + * + * This function will fill AVFrame.data and AVFrame.buf arrays and, if + * necessary, allocate and fill AVFrame.extended_data and AVFrame.extended_buf. + * For planar formats, one buffer will be allocated for each plane. + * + * @warning: if frame already has been allocated, calling this function will + * leak memory. In addition, undefined behavior can occur in certain + * cases. + * + * @param frame frame in which to store the new buffers. + * @param align Required buffer size alignment. If equal to 0, alignment will be + * chosen automatically for the current CPU. It is highly + * recommended to pass 0 here unless you know what you are doing. + * + * @return 0 on success, a negative AVERROR on error. + */ +int av_frame_get_buffer2(AVFrame *frame, int align); + /** * Check if the frame data is writable. * diff --git a/libavutil/version.h b/libavutil/version.h index f220e192c8..2b5ac540a4 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,7 +79,7 @@ */ #define LIBAVUTIL_VERSION_MAJOR 57 -#define LIBAVUTIL_VERSION_MINOR 5 +#define LIBAVUTIL_VERSION_MINOR 6 #define LIBAVUTIL_VERSION_MICRO 100 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \ From patchwork Sun Sep 12 03:21:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 30180 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp2862893iov; Sat, 11 Sep 2021 20:22:08 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwI3k6a+RIMcb2Vvt9OIf0oCN7QpydlzdAalWk8C0AQZpcMs7QFKjR0QyCIk4RaMNkMLBN7 X-Received: by 2002:a05:6402:14c3:: with SMTP id f3mr6102012edx.312.1631416928171; Sat, 11 Sep 2021 20:22:08 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id nd33si3747401ejc.139.2021.09.11.20.22.07; Sat, 11 Sep 2021 20:22:08 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=QXVb11g9; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4A8F268A92E; Sun, 12 Sep 2021 06:21:50 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10olkn2062.outbound.protection.outlook.com [40.92.41.62]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2777868A901 for ; Sun, 12 Sep 2021 06:21:43 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SZ2JTuK4CsQcNw2pp0/RrhBhhU/wbo+z1U8TC99CcRmiFnBUoL47ybqaADxyWKpQe947NhnnOr1I81svbrEU2+RkUGL5bJtoYr0jGaa1xFmRLwCcVkdwiU0nvkdqdNr2CvgV6dcD7V/QM2b2XYHg4udhdDyWXr4jFE0BHKhrVACn9OBdrlg8nDbGL7Dw8JiFg4gE+h/X0OOxAOTAx8kZvw0ol58V2ca8UDK8GHG+L2Wx7+/zl5b2jr+U0k3WX7oJm0DIhpQowvzLI2gi0zTeUJUZBJIFbmSVj32tC0F0Lf/j035p5N5kpnh1nBMeSFAEOyXiY2rNY3dcYiHcByk1iw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=30HhpAwTBICc00bf6R5JOaDGhVKCHBCOJAjPBRh+Swo=; b=LR8LgYKz2BwRBribPMWZKjIDPyX0JBANltSFn63AKHa33RWt5uRdO/Ie9GOaSXLeyURo2XFbhO1Af3/GoikXRrYKK68ljHHPBspz8EhYS4Vxg/SmVWV0PP0mYbgBm5aLSJroo/xl417FmolGgWiMOMm8AJ9nsqRnBgxm4wWpJ6RkyH4bEz405lIyHP1s2xex9KGxizCs/m0ZAZsE2qALPbzCf4OdzpNhZdBeOCyOTRGQNR+EBcl1WkTEUBkDSUShgjC/TLsQwrQZxGRtNtYZ7tV+dp5ORD2scbqpWcCt7e7EMyqcg3FInjYVfhrLRPq7L+skwZkFN4Lbp7vYS2P+Tw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=30HhpAwTBICc00bf6R5JOaDGhVKCHBCOJAjPBRh+Swo=; b=QXVb11g9eaXpxWzPDTfBnvrOBHmTea9YlHyDVbW30KBbUGwVnMuIGj8R6B0Nx0tWGFLD5G3chISj07CjYymkl2zLM7bfmdUiht6ojze9AeH2nU3xix+EEFTUFeDvAnfs1qiNPpTo6Pzkrt4ldl39xOxsNBFN1J4LNisU7V0EjVYDjRHabnldAycAqw3xGmLGKsNNpGxfRcYXOvjMDUCXPVP43pm3tAQr/fNnWkRaix+E/F62hNBHzJ7h5+V95Fjeeqog1+9/tF2VQKcmuXWLqNF3QMpbui+e82Ou7HGRwwP4aEFOzBCv3mxKICDNXtJPBvcFiUbWsCK+VHICv8pKug== Received: from MN2PR04MB5981.namprd04.prod.outlook.com (2603:10b6:208:da::10) by MN2PR04MB5728.namprd04.prod.outlook.com (2603:10b6:208:3d::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Sun, 12 Sep 2021 03:21:37 +0000 Received: from MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb]) by MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb%5]) with mapi id 15.20.4500.018; Sun, 12 Sep 2021 03:21:37 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v5 02/12] global: Merge AVSubtitle into AVFrame Thread-Index: AQHXp4VLULMa9R3Fu0+u6pWXhWgwtQ== Date: Sun, 12 Sep 2021 03:21:37 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [VkPo2LeIzaiGenofx/C+BgeyNLdmlDr7AM/QbghRfGk=] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 118a5889-de4f-49a1-54ed-08d9759c6e37 x-ms-traffictypediagnostic: MN2PR04MB5728: x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: IWarwvCtfYCKIitvOy8YQYVGPxAGfBw9xF3wohrQzEEQFMkmjmaJoE6eE85lGLSIFKFn58lOH7wS4YfZo3Q4jJHjIyUSpo/sELsrL7cslUY1YMaCU2bi9pTfPlbTmugDgq/9OHg8/oFSiorUe/Mh5h/27YJ5qcuYQmZgYSkp7riLy/Ng4qAA15jnFsXmlfGo66DfKqA+Jvaxeje6aJf30+fOn6gqnPibya2ScDz1BX7f5njpcRjpYkyOdKrgCw5gX1KGA5T2ZyAsfkb0oHCHrc2rCykPEor84XK2AUMkhKTnwBL8Fc2Fuw9f+VZ7Sab/7UImj3W0PGWT9nEvo5Tg8QILw+/MXE4spKDZdS8MpHj+MZEiNOkCqp2bQ/1wEfDUhQ/bZVW84yyI7VuNbSV7LWScBvKBuxYSQqUDY/qLr1aIf/Fsz1l7iKUmenrm6/xF x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: EeqgE+Ba1+sT032ll9IX3aFYj9lEwQ2w3fnzIp7Za6d04ed3fT1ePk66sshvszsp2d6xjk1GA4L0bsCPgF+TbmneBMRERLFed0sICWBTemMJsKhGjdfYvZ4mcq9qXR5L3ErxG9SM1csuEv5L5g3hgg== x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-3174-20-msonline-outlook-529c7.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR04MB5981.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 118a5889-de4f-49a1-54ed-08d9759c6e37 X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Sep 2021 03:21:37.2574 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR04MB5728 Subject: [FFmpeg-devel] [PATCH v5 02/12] global: Merge AVSubtitle into AVFrame X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: K4Zc23B9pDEL Signed-off-by: softworkz --- libavcodec/ass.c | 16 ++-- libavcodec/ass.h | 8 +- libavcodec/assdec.c | 18 ++--- libavcodec/assenc.c | 10 +-- libavcodec/avcodec.c | 19 ----- libavcodec/avcodec.h | 81 ++++---------------- libavcodec/ccaption_dec.c | 28 +++---- libavcodec/codec.h | 4 +- libavcodec/codec_desc.h | 4 +- libavcodec/decode.c | 23 +++--- libavcodec/dvbsubdec.c | 48 ++++++------ libavcodec/dvbsubenc.c | 72 +++++++++--------- libavcodec/dvdsubdec.c | 122 +++++++++++++++---------------- libavcodec/dvdsubenc.c | 40 +++++----- libavcodec/encode.c | 6 +- libavcodec/jacosubdec.c | 4 +- libavcodec/libzvbi-teletextdec.c | 16 ++-- libavcodec/microdvddec.c | 4 +- libavcodec/movtextdec.c | 4 +- libavcodec/movtextenc.c | 10 +-- libavcodec/mpl2dec.c | 4 +- libavcodec/pgssubdec.c | 68 ++++++++--------- libavcodec/realtextdec.c | 4 +- libavcodec/samidec.c | 4 +- libavcodec/srtdec.c | 4 +- libavcodec/srtenc.c | 14 ++-- libavcodec/subviewerdec.c | 4 +- libavcodec/textdec.c | 4 +- libavcodec/ttmlenc.c | 10 +-- libavcodec/utils.c | 11 +++ libavcodec/webvttdec.c | 4 +- libavcodec/webvttenc.c | 10 +-- libavcodec/xsubdec.c | 58 +++++++-------- libavcodec/xsubenc.c | 48 ++++++------ libavfilter/vf_subtitles.c | 28 +++---- libavformat/utils.c | 5 +- libavutil/Makefile | 2 + libavutil/frame.c | 41 ++++++++++- libavutil/frame.h | 12 +++ libavutil/subfmt.c | 52 +++++++++++++ libavutil/subfmt.h | 94 ++++++++++++++++++++++++ 41 files changed, 576 insertions(+), 442 deletions(-) create mode 100644 libavutil/subfmt.c create mode 100644 libavutil/subfmt.h diff --git a/libavcodec/ass.c b/libavcodec/ass.c index 907e2d7b88..415ef12911 100644 --- a/libavcodec/ass.c +++ b/libavcodec/ass.c @@ -114,26 +114,26 @@ char *ff_ass_get_dialog(int readorder, int layer, const char *style, speaker ? speaker : "", text); } -int ff_ass_add_rect(AVSubtitle *sub, const char *dialog, +int ff_ass_add_rect(AVFrame *sub, const char *dialog, int readorder, int layer, const char *style, const char *speaker) { char *ass_str; AVSubtitleRect **rects; - rects = av_realloc_array(sub->rects, sub->num_rects+1, sizeof(*sub->rects)); + rects = av_realloc_array(sub->subtitle_rects, sub->num_subtitle_rects+1, sizeof(*sub->subtitle_rects)); if (!rects) return AVERROR(ENOMEM); - sub->rects = rects; - rects[sub->num_rects] = av_mallocz(sizeof(*rects[0])); - if (!rects[sub->num_rects]) + sub->subtitle_rects = rects; + rects[sub->num_subtitle_rects] = av_mallocz(sizeof(*rects[0])); + if (!rects[sub->num_subtitle_rects]) return AVERROR(ENOMEM); - rects[sub->num_rects]->type = SUBTITLE_ASS; + rects[sub->num_subtitle_rects]->type = AV_SUBTITLE_FMT_ASS; ass_str = ff_ass_get_dialog(readorder, layer, style, speaker, dialog); if (!ass_str) return AVERROR(ENOMEM); - rects[sub->num_rects]->ass = ass_str; - sub->num_rects++; + rects[sub->num_subtitle_rects]->ass = ass_str; + sub->num_subtitle_rects++; return 0; } diff --git a/libavcodec/ass.h b/libavcodec/ass.h index 2c260e4e78..de31f35c8a 100644 --- a/libavcodec/ass.h +++ b/libavcodec/ass.h @@ -48,7 +48,7 @@ typedef struct FFASSDecoderContext { } FFASSDecoderContext; /** - * Generate a suitable AVCodecContext.subtitle_header for SUBTITLE_ASS. + * Generate a suitable AVCodecContext.subtitle_header for AV_SUBTITLE_FMT_ASS. * Can specify all fields explicitly * * @param avctx pointer to the AVCodecContext @@ -76,7 +76,7 @@ int ff_ass_subtitle_header_full(AVCodecContext *avctx, int bold, int italic, int underline, int border_style, int alignment); /** - * Generate a suitable AVCodecContext.subtitle_header for SUBTITLE_ASS. + * Generate a suitable AVCodecContext.subtitle_header for AV_SUBTITLE_FMT_ASS. * * @param avctx pointer to the AVCodecContext * @param font name of the default font face to use @@ -97,7 +97,7 @@ int ff_ass_subtitle_header(AVCodecContext *avctx, int border_style, int alignment); /** - * Generate a suitable AVCodecContext.subtitle_header for SUBTITLE_ASS + * Generate a suitable AVCodecContext.subtitle_header for AV_SUBTITLE_FMT_ASS * with default style. * * @param avctx pointer to the AVCodecContext @@ -114,7 +114,7 @@ char *ff_ass_get_dialog(int readorder, int layer, const char *style, /** * Add an ASS dialog to a subtitle. */ -int ff_ass_add_rect(AVSubtitle *sub, const char *dialog, +int ff_ass_add_rect(AVFrame *sub, const char *dialog, int readorder, int layer, const char *style, const char *speaker); diff --git a/libavcodec/assdec.c b/libavcodec/assdec.c index 319279490c..0df08ba756 100644 --- a/libavcodec/assdec.c +++ b/libavcodec/assdec.c @@ -42,21 +42,21 @@ static av_cold int ass_decode_init(AVCodecContext *avctx) static int ass_decode_frame(AVCodecContext *avctx, void *data, int *got_sub_ptr, AVPacket *avpkt) { - AVSubtitle *sub = data; + AVFrame *sub = data; if (avpkt->size <= 0) return avpkt->size; - sub->rects = av_malloc(sizeof(*sub->rects)); - if (!sub->rects) + sub->subtitle_rects = av_malloc(sizeof(*sub->subtitle_rects)); + if (!sub->subtitle_rects) return AVERROR(ENOMEM); - sub->rects[0] = av_mallocz(sizeof(*sub->rects[0])); - if (!sub->rects[0]) + sub->subtitle_rects[0] = av_mallocz(sizeof(*sub->subtitle_rects[0])); + if (!sub->subtitle_rects[0]) return AVERROR(ENOMEM); - sub->num_rects = 1; - sub->rects[0]->type = SUBTITLE_ASS; - sub->rects[0]->ass = av_strdup(avpkt->data); - if (!sub->rects[0]->ass) + sub->num_subtitle_rects = 1; + sub->subtitle_rects[0]->type = AV_SUBTITLE_FMT_ASS; + sub->subtitle_rects[0]->ass = av_strdup(avpkt->data); + if (!sub->subtitle_rects[0]->ass) return AVERROR(ENOMEM); *got_sub_ptr = 1; return avpkt->size; diff --git a/libavcodec/assenc.c b/libavcodec/assenc.c index a6d107ded2..84745a4487 100644 --- a/libavcodec/assenc.c +++ b/libavcodec/assenc.c @@ -41,15 +41,15 @@ static av_cold int ass_encode_init(AVCodecContext *avctx) static int ass_encode_frame(AVCodecContext *avctx, unsigned char *buf, int bufsize, - const AVSubtitle *sub) + const AVFrame *sub) { int i, len, total_len = 0; - for (i=0; inum_rects; i++) { - const char *ass = sub->rects[i]->ass; + for (i=0; inum_subtitle_rects; i++) { + const char *ass = sub->subtitle_rects[i]->ass; - if (sub->rects[i]->type != SUBTITLE_ASS) { - av_log(avctx, AV_LOG_ERROR, "Only SUBTITLE_ASS type supported.\n"); + if (sub->subtitle_rects[i]->type != AV_SUBTITLE_FMT_ASS) { + av_log(avctx, AV_LOG_ERROR, "Only AV_SUBTITLE_FMT_ASS type supported.\n"); return AVERROR(EINVAL); } diff --git a/libavcodec/avcodec.c b/libavcodec/avcodec.c index 2dd7dd84e0..963f52c4bd 100644 --- a/libavcodec/avcodec.c +++ b/libavcodec/avcodec.c @@ -426,25 +426,6 @@ void avcodec_flush_buffers(AVCodecContext *avctx) av_bsf_flush(avci->bsf); } -void avsubtitle_free(AVSubtitle *sub) -{ - int i; - - for (i = 0; i < sub->num_rects; i++) { - av_freep(&sub->rects[i]->data[0]); - av_freep(&sub->rects[i]->data[1]); - av_freep(&sub->rects[i]->data[2]); - av_freep(&sub->rects[i]->data[3]); - av_freep(&sub->rects[i]->text); - av_freep(&sub->rects[i]->ass); - av_freep(&sub->rects[i]); - } - - av_freep(&sub->rects); - - memset(sub, 0, sizeof(*sub)); -} - av_cold int avcodec_close(AVCodecContext *avctx) { int i; diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index ffd58c333f..a7a2df8cf4 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -35,6 +35,7 @@ #include "libavutil/frame.h" #include "libavutil/log.h" #include "libavutil/pixfmt.h" +#include "libavutil/subfmt.h" #include "libavutil/rational.h" #include "codec.h" @@ -1670,7 +1671,7 @@ typedef struct AVCodecContext { /** * Header containing style information for text subtitles. - * For SUBTITLE_ASS subtitle type, it should contain the whole ASS + * For AV_SUBTITLE_FMT_ASS subtitle type, it should contain the whole ASS * [Script Info] and [V4+ Styles] section, plus the [Events] line and * the Format line following. It shouldn't include any Dialogue line. * - encoding: Set/allocated/freed by user (before avcodec_open2()) @@ -2233,63 +2234,8 @@ typedef struct AVHWAccel { * @} */ -enum AVSubtitleType { - SUBTITLE_NONE, - - SUBTITLE_BITMAP, ///< A bitmap, pict will be set - - /** - * Plain text, the text field must be set by the decoder and is - * authoritative. ass and pict fields may contain approximations. - */ - SUBTITLE_TEXT, - - /** - * Formatted text, the ass field must be set by the decoder and is - * authoritative. pict and text fields may contain approximations. - */ - SUBTITLE_ASS, -}; - #define AV_SUBTITLE_FLAG_FORCED 0x00000001 -typedef struct AVSubtitleRect { - int x; ///< top left corner of pict, undefined when pict is not set - int y; ///< top left corner of pict, undefined when pict is not set - int w; ///< width of pict, undefined when pict is not set - int h; ///< height of pict, undefined when pict is not set - int nb_colors; ///< number of colors in pict, undefined when pict is not set - - /** - * data+linesize for the bitmap of this subtitle. - * Can be set for text/ass as well once they are rendered. - */ - uint8_t *data[4]; - int linesize[4]; - - enum AVSubtitleType type; - - char *text; ///< 0 terminated plain UTF-8 text - - /** - * 0 terminated ASS/SSA compatible event line. - * The presentation of this is unaffected by the other values in this - * struct. - */ - char *ass; - - int flags; -} AVSubtitleRect; - -typedef struct AVSubtitle { - uint16_t format; /* 0 = graphics */ - uint32_t start_display_time; /* relative to packet pts, in ms */ - uint32_t end_display_time; /* relative to packet pts, in ms */ - unsigned num_rects; - AVSubtitleRect **rects; - int64_t pts; ///< Same as packet pts, in AV_TIME_BASE -} AVSubtitle; - /** * Return the LIBAVCODEC_VERSION_INT constant. */ @@ -2425,13 +2371,6 @@ int avcodec_open2(AVCodecContext *avctx, const AVCodec *codec, AVDictionary **op */ int avcodec_close(AVCodecContext *avctx); -/** - * Free all allocated data in the given subtitle struct. - * - * @param sub AVSubtitle to free. - */ -void avsubtitle_free(AVSubtitle *sub); - /** * @} */ @@ -2518,12 +2457,12 @@ enum AVChromaLocation avcodec_chroma_pos_to_enum(int xpos, int ypos); * before packets may be fed to the decoder. * * @param avctx the codec context - * @param[out] sub The preallocated AVSubtitle in which the decoded subtitle will be stored, - * must be freed with avsubtitle_free if *got_sub_ptr is set. + * @param[out] sub The preallocated AVFrame in which the decoded subtitle will be stored, + * must be freed with av_frame_free if *got_sub_ptr is set. * @param[in,out] got_sub_ptr Zero if no subtitle could be decompressed, otherwise, it is nonzero. * @param[in] avpkt The input AVPacket containing the input buffer. */ -int avcodec_decode_subtitle2(AVCodecContext *avctx, AVSubtitle *sub, +int avcodec_decode_subtitle2(AVCodecContext *avctx, AVFrame *sub, int *got_sub_ptr, AVPacket *avpkt); @@ -3004,7 +2943,7 @@ void av_parser_close(AVCodecParserContext *s); */ int avcodec_encode_subtitle(AVCodecContext *avctx, uint8_t *buf, int buf_size, - const AVSubtitle *sub); + const AVFrame *sub); /** @@ -3120,6 +3059,14 @@ void avcodec_flush_buffers(AVCodecContext *avctx); */ int av_get_audio_frame_duration(AVCodecContext *avctx, int frame_bytes); +/** + * Return subtitle format from a codec descriptor + * + * @param codec_descriptor codec descriptor + * @return the subtitle type (e.g. bitmap, text) + */ +enum AVSubtitleType av_get_subtitle_format_from_codecdesc(const AVCodecDescriptor *codec_descriptor); + /* memory */ /** diff --git a/libavcodec/ccaption_dec.c b/libavcodec/ccaption_dec.c index 27c61527f6..33cc88f705 100644 --- a/libavcodec/ccaption_dec.c +++ b/libavcodec/ccaption_dec.c @@ -841,8 +841,8 @@ static int process_cc608(CCaptionSubContext *ctx, uint8_t hi, uint8_t lo) static int decode(AVCodecContext *avctx, void *data, int *got_sub, AVPacket *avpkt) { CCaptionSubContext *ctx = avctx->priv_data; - AVSubtitle *sub = data; - int64_t in_time = sub->pts; + AVFrame *sub = data; + int64_t in_time = sub->subtitle_pts; int64_t start_time; int64_t end_time; int bidx = ctx->buffer_index; @@ -879,17 +879,17 @@ static int decode(AVCodecContext *avctx, void *data, int *got_sub, AVPacket *avp if (ctx->buffer[bidx].str[0] || ctx->real_time) { ff_dlog(ctx, "cdp writing data (%s)\n", ctx->buffer[bidx].str); start_time = ctx->buffer_time[0]; - sub->pts = start_time; + sub->subtitle_pts = start_time; end_time = ctx->buffer_time[1]; if (!ctx->real_time) - sub->end_display_time = av_rescale_q(end_time - start_time, + sub->subtitle_end_time = av_rescale_q(end_time - start_time, AV_TIME_BASE_Q, ms_tb); else - sub->end_display_time = -1; + sub->subtitle_end_time = -1; ret = ff_ass_add_rect(sub, ctx->buffer[bidx].str, ctx->readorder++, 0, NULL, NULL); if (ret < 0) return ret; - ctx->last_real_time = sub->pts; + ctx->last_real_time = sub->subtitle_pts; ctx->screen_touched = 0; } } @@ -899,16 +899,16 @@ static int decode(AVCodecContext *avctx, void *data, int *got_sub, AVPacket *avp ret = ff_ass_add_rect(sub, ctx->buffer[bidx].str, ctx->readorder++, 0, NULL, NULL); if (ret < 0) return ret; - sub->pts = ctx->buffer_time[1]; - sub->end_display_time = av_rescale_q(ctx->buffer_time[1] - ctx->buffer_time[0], + sub->subtitle_pts = ctx->buffer_time[1]; + sub->subtitle_end_time = av_rescale_q(ctx->buffer_time[1] - ctx->buffer_time[0], AV_TIME_BASE_Q, ms_tb); - if (sub->end_display_time == 0) - sub->end_display_time = ctx->buffer[bidx].len * 20; + if (sub->subtitle_end_time == 0) + sub->subtitle_end_time = ctx->buffer[bidx].len * 20; } if (ctx->real_time && ctx->screen_touched && - sub->pts >= ctx->last_real_time + av_rescale_q(ctx->real_time_latency_msec, ms_tb, AV_TIME_BASE_Q)) { - ctx->last_real_time = sub->pts; + sub->subtitle_pts >= ctx->last_real_time + av_rescale_q(ctx->real_time_latency_msec, ms_tb, AV_TIME_BASE_Q)) { + ctx->last_real_time = sub->subtitle_pts; ctx->screen_touched = 0; capture_screen(ctx); @@ -917,10 +917,10 @@ static int decode(AVCodecContext *avctx, void *data, int *got_sub, AVPacket *avp ret = ff_ass_add_rect(sub, ctx->buffer[bidx].str, ctx->readorder++, 0, NULL, NULL); if (ret < 0) return ret; - sub->end_display_time = -1; + sub->subtitle_end_time = -1; } - *got_sub = sub->num_rects > 0; + *got_sub = sub->num_subtitle_rects > 0; return ret; } diff --git a/libavcodec/codec.h b/libavcodec/codec.h index 8f12705066..785a6bf27e 100644 --- a/libavcodec/codec.h +++ b/libavcodec/codec.h @@ -188,7 +188,7 @@ typedef struct AVProfile { typedef struct AVCodecDefault AVCodecDefault; struct AVCodecContext; -struct AVSubtitle; +struct AVFrame; struct AVPacket; /** @@ -283,7 +283,7 @@ typedef struct AVCodec { int (*init)(struct AVCodecContext *); int (*encode_sub)(struct AVCodecContext *, uint8_t *buf, int buf_size, - const struct AVSubtitle *sub); + const struct AVFrame *sub); /** * Encode data to an AVPacket. * diff --git a/libavcodec/codec_desc.h b/libavcodec/codec_desc.h index 126b52df47..387863a041 100644 --- a/libavcodec/codec_desc.h +++ b/libavcodec/codec_desc.h @@ -92,12 +92,12 @@ typedef struct AVCodecDescriptor { #define AV_CODEC_PROP_REORDER (1 << 3) /** * Subtitle codec is bitmap based - * Decoded AVSubtitle data can be read from the AVSubtitleRect->pict field. + * Decoded AVFrame data can be read from the AVSubtitleRect->pict field. */ #define AV_CODEC_PROP_BITMAP_SUB (1 << 16) /** * Subtitle codec is text based. - * Decoded AVSubtitle data can be read from the AVSubtitleRect->ass field. + * Decoded AVFrame data can be read from the AVSubtitleRect->ass field. */ #define AV_CODEC_PROP_TEXT_SUB (1 << 17) diff --git a/libavcodec/decode.c b/libavcodec/decode.c index 643f9d6a30..37dd69070e 100644 --- a/libavcodec/decode.c +++ b/libavcodec/decode.c @@ -712,10 +712,10 @@ int attribute_align_arg avcodec_receive_frame(AVCodecContext *avctx, AVFrame *fr return 0; } -static void get_subtitle_defaults(AVSubtitle *sub) +static void get_subtitle_defaults(AVFrame *sub) { memset(sub, 0, sizeof(*sub)); - sub->pts = AV_NOPTS_VALUE; + sub->subtitle_pts = AV_NOPTS_VALUE; } #define UTF8_MAX_BYTES 4 /* 5 and 6 bytes sequences should not be used */ @@ -799,7 +799,7 @@ static int utf8_check(const uint8_t *str) return 1; } -int avcodec_decode_subtitle2(AVCodecContext *avctx, AVSubtitle *sub, +int avcodec_decode_subtitle2(AVCodecContext *avctx, AVFrame *sub, int *got_sub_ptr, AVPacket *avpkt) { @@ -828,31 +828,28 @@ int avcodec_decode_subtitle2(AVCodecContext *avctx, AVSubtitle *sub, return ret; if (avctx->pkt_timebase.num && avpkt->pts != AV_NOPTS_VALUE) - sub->pts = av_rescale_q(avpkt->pts, + sub->subtitle_pts = av_rescale_q(avpkt->pts, avctx->pkt_timebase, AV_TIME_BASE_Q); ret = avctx->codec->decode(avctx, sub, got_sub_ptr, pkt); av_assert1((ret >= 0) >= !!*got_sub_ptr && !!*got_sub_ptr >= !!sub->num_rects); - if (sub->num_rects && !sub->end_display_time && avpkt->duration && + if (sub->num_subtitle_rects && !sub->subtitle_end_time && avpkt->duration && avctx->pkt_timebase.num) { AVRational ms = { 1, 1000 }; - sub->end_display_time = av_rescale_q(avpkt->duration, + sub->subtitle_end_time = av_rescale_q(avpkt->duration, avctx->pkt_timebase, ms); } - if (avctx->codec_descriptor->props & AV_CODEC_PROP_BITMAP_SUB) - sub->format = 0; - else if (avctx->codec_descriptor->props & AV_CODEC_PROP_TEXT_SUB) - sub->format = 1; + sub->format = av_get_subtitle_format_from_codecdesc(avctx->codec_descriptor); - for (unsigned i = 0; i < sub->num_rects; i++) { + for (unsigned i = 0; i < sub->num_subtitle_rects; i++) { if (avctx->sub_charenc_mode != FF_SUB_CHARENC_MODE_IGNORE && - sub->rects[i]->ass && !utf8_check(sub->rects[i]->ass)) { + sub->subtitle_rects[i]->ass && !utf8_check(sub->subtitle_rects[i]->ass)) { av_log(avctx, AV_LOG_ERROR, "Invalid UTF-8 in decoded subtitles text; " "maybe missing -sub_charenc option\n"); - avsubtitle_free(sub); + av_frame_free(&sub); ret = AVERROR_INVALIDDATA; break; } diff --git a/libavcodec/dvbsubdec.c b/libavcodec/dvbsubdec.c index e45c14e878..aeb88df06f 100644 --- a/libavcodec/dvbsubdec.c +++ b/libavcodec/dvbsubdec.c @@ -725,7 +725,7 @@ static void compute_default_clut(DVBSubContext *ctx, uint8_t *clut, AVSubtitleRe } -static int save_subtitle_set(AVCodecContext *avctx, AVSubtitle *sub, int *got_output) +static int save_subtitle_set(AVCodecContext *avctx, AVFrame *sub, int *got_output) { DVBSubContext *ctx = avctx->priv_data; DVBSubRegionDisplay *display; @@ -745,34 +745,34 @@ static int save_subtitle_set(AVCodecContext *avctx, AVSubtitle *sub, int *got_ou } /* Not touching AVSubtitles again*/ - if (sub->num_rects) { + if (sub->num_subtitle_rects) { avpriv_request_sample(ctx, "Different Version of Segment asked Twice"); return AVERROR_PATCHWELCOME; } for (display = ctx->display_list; display; display = display->next) { region = get_region(ctx, display->region_id); if (region && region->dirty) - sub->num_rects++; + sub->num_subtitle_rects++; } if (ctx->compute_edt == 0) { - sub->end_display_time = ctx->time_out * 1000; + sub->subtitle_end_time = ctx->time_out * 1000; *got_output = 1; } else if (ctx->prev_start != AV_NOPTS_VALUE) { - sub->end_display_time = av_rescale_q((sub->pts - ctx->prev_start ), AV_TIME_BASE_Q, (AVRational){ 1, 1000 }) - 1; + sub->subtitle_end_time = av_rescale_q((sub->subtitle_pts - ctx->prev_start ), AV_TIME_BASE_Q, (AVRational){ 1, 1000 }) - 1; *got_output = 1; } - if (sub->num_rects > 0) { + if (sub->num_subtitle_rects > 0) { - sub->rects = av_mallocz_array(sizeof(*sub->rects), sub->num_rects); - if (!sub->rects) { + sub->subtitle_rects = av_mallocz_array(sizeof(*sub->subtitle_rects), sub->num_subtitle_rects); + if (!sub->subtitle_rects) { ret = AVERROR(ENOMEM); goto fail; } - for (i = 0; i < sub->num_rects; i++) { - sub->rects[i] = av_mallocz(sizeof(*sub->rects[i])); - if (!sub->rects[i]) { + for (i = 0; i < sub->num_subtitle_rects; i++) { + sub->subtitle_rects[i] = av_mallocz(sizeof(*sub->subtitle_rects[i])); + if (!sub->subtitle_rects[i]) { ret = AVERROR(ENOMEM); goto fail; } @@ -789,13 +789,13 @@ static int save_subtitle_set(AVCodecContext *avctx, AVSubtitle *sub, int *got_ou if (!region->dirty) continue; - rect = sub->rects[i]; + rect = sub->subtitle_rects[i]; rect->x = display->x_pos + offset_x; rect->y = display->y_pos + offset_y; rect->w = region->width; rect->h = region->height; rect->nb_colors = (1 << region->depth); - rect->type = SUBTITLE_BITMAP; + rect->type = AV_SUBTITLE_FMT_BITMAP; rect->linesize[0] = region->width; clut = get_clut(ctx, region->clut); @@ -846,18 +846,18 @@ static int save_subtitle_set(AVCodecContext *avctx, AVSubtitle *sub, int *got_ou return 0; fail: - if (sub->rects) { - for (i=0; i < sub->num_rects; i++) { - rect = sub->rects[i]; + if (sub->subtitle_rects) { + for (i=0; i < sub->num_subtitle_rects; i++) { + rect = sub->subtitle_rects[i]; if (rect) { av_freep(&rect->data[0]); av_freep(&rect->data[1]); } - av_freep(&sub->rects[i]); + av_freep(&sub->subtitle_rects[i]); } - av_freep(&sub->rects); + av_freep(&sub->subtitle_rects); } - sub->num_rects = 0; + sub->num_subtitle_rects = 0; return ret; } @@ -1291,7 +1291,7 @@ static int dvbsub_parse_region_segment(AVCodecContext *avctx, } static int dvbsub_parse_page_segment(AVCodecContext *avctx, - const uint8_t *buf, int buf_size, AVSubtitle *sub, int *got_output) + const uint8_t *buf, int buf_size, AVFrame *sub, int *got_output) { DVBSubContext *ctx = avctx->priv_data; DVBSubRegionDisplay *display; @@ -1598,7 +1598,7 @@ static int dvbsub_parse_display_definition_segment(AVCodecContext *avctx, } static int dvbsub_display_end_segment(AVCodecContext *avctx, const uint8_t *buf, - int buf_size, AVSubtitle *sub,int *got_output) + int buf_size, AVFrame *sub,int *got_output) { DVBSubContext *ctx = avctx->priv_data; @@ -1617,7 +1617,7 @@ static int dvbsub_decode(AVCodecContext *avctx, const uint8_t *buf = avpkt->data; int buf_size = avpkt->size; DVBSubContext *ctx = avctx->priv_data; - AVSubtitle *sub = data; + AVFrame *sub = data; const uint8_t *p, *p_end; int segment_type; int page_id; @@ -1720,11 +1720,11 @@ static int dvbsub_decode(AVCodecContext *avctx, end: if (ret < 0) { *got_sub_ptr = 0; - avsubtitle_free(sub); + av_frame_free(&sub); return ret; } else { if (ctx->compute_edt == 1) - FFSWAP(int64_t, ctx->prev_start, sub->pts); + FFSWAP(int64_t, ctx->prev_start, sub->subtitle_pts); } return p - buf; diff --git a/libavcodec/dvbsubenc.c b/libavcodec/dvbsubenc.c index 322fc27cb4..480092867e 100644 --- a/libavcodec/dvbsubenc.c +++ b/libavcodec/dvbsubenc.c @@ -269,7 +269,7 @@ static int dvb_encode_rle8(uint8_t **pq, int buf_size, } static int dvbsub_encode(AVCodecContext *avctx, uint8_t *outbuf, int buf_size, - const AVSubtitle *h) + const AVFrame *h) { DVBSubtitleContext *s = avctx->priv_data; uint8_t *q, *pseg_len; @@ -280,7 +280,7 @@ static int dvbsub_encode(AVCodecContext *avctx, uint8_t *outbuf, int buf_size, page_id = 1; - if (h->num_rects && !h->rects) + if (h->num_subtitle_rects && !h->subtitle_rects) return AVERROR(EINVAL); if (avctx->width > 0 && avctx->height > 0) { @@ -301,7 +301,7 @@ static int dvbsub_encode(AVCodecContext *avctx, uint8_t *outbuf, int buf_size, /* page composition segment */ - if (buf_size < 8 + h->num_rects * 6) + if (buf_size < 8 + h->num_subtitle_rects * 6) return AVERROR_BUFFER_TOO_SMALL; *q++ = 0x0f; /* sync_byte */ *q++ = 0x10; /* segment_type */ @@ -313,30 +313,30 @@ static int dvbsub_encode(AVCodecContext *avctx, uint8_t *outbuf, int buf_size, /* page_version = 0 + page_state */ *q++ = (s->object_version << 4) | (page_state << 2) | 3; - for (region_id = 0; region_id < h->num_rects; region_id++) { + for (region_id = 0; region_id < h->num_subtitle_rects; region_id++) { *q++ = region_id; *q++ = 0xff; /* reserved */ - bytestream_put_be16(&q, h->rects[region_id]->x); /* left pos */ - bytestream_put_be16(&q, h->rects[region_id]->y); /* top pos */ + bytestream_put_be16(&q, h->subtitle_rects[region_id]->x); /* left pos */ + bytestream_put_be16(&q, h->subtitle_rects[region_id]->y); /* top pos */ } bytestream_put_be16(&pseg_len, q - pseg_len - 2); - buf_size -= 8 + h->num_rects * 6; + buf_size -= 8 + h->num_subtitle_rects * 6; - if (h->num_rects) { - for (clut_id = 0; clut_id < h->num_rects; clut_id++) { - if (buf_size < 6 + h->rects[clut_id]->nb_colors * 6) + if (h->num_subtitle_rects) { + for (clut_id = 0; clut_id < h->num_subtitle_rects; clut_id++) { + if (buf_size < 6 + h->subtitle_rects[clut_id]->nb_colors * 6) return AVERROR_BUFFER_TOO_SMALL; /* CLUT segment */ - if (h->rects[clut_id]->nb_colors <= 4) { + if (h->subtitle_rects[clut_id]->nb_colors <= 4) { /* 2 bpp, some decoders do not support it correctly */ bpp_index = 0; - } else if (h->rects[clut_id]->nb_colors <= 16) { + } else if (h->subtitle_rects[clut_id]->nb_colors <= 16) { /* 4 bpp, standard encoding */ bpp_index = 1; - } else if (h->rects[clut_id]->nb_colors <= 256) { + } else if (h->subtitle_rects[clut_id]->nb_colors <= 256) { /* 8 bpp, standard encoding */ bpp_index = 2; } else { @@ -353,12 +353,12 @@ static int dvbsub_encode(AVCodecContext *avctx, uint8_t *outbuf, int buf_size, *q++ = clut_id; *q++ = (0 << 4) | 0xf; /* version = 0 */ - for(i = 0; i < h->rects[clut_id]->nb_colors; i++) { + for(i = 0; i < h->subtitle_rects[clut_id]->nb_colors; i++) { *q++ = i; /* clut_entry_id */ *q++ = (1 << (7 - bpp_index)) | (0xf << 1) | 1; /* 2 bits/pixel full range */ { int a, r, g, b; - uint32_t x= ((uint32_t*)h->rects[clut_id]->data[1])[i]; + uint32_t x= ((uint32_t*)h->subtitle_rects[clut_id]->data[1])[i]; a = (x >> 24) & 0xff; r = (x >> 16) & 0xff; g = (x >> 8) & 0xff; @@ -372,22 +372,22 @@ static int dvbsub_encode(AVCodecContext *avctx, uint8_t *outbuf, int buf_size, } bytestream_put_be16(&pseg_len, q - pseg_len - 2); - buf_size -= 6 + h->rects[clut_id]->nb_colors * 6; + buf_size -= 6 + h->subtitle_rects[clut_id]->nb_colors * 6; } - if (buf_size < h->num_rects * 22) + if (buf_size < h->num_subtitle_rects * 22) return AVERROR_BUFFER_TOO_SMALL; - for (region_id = 0; region_id < h->num_rects; region_id++) { + for (region_id = 0; region_id < h->num_subtitle_rects; region_id++) { /* region composition segment */ - if (h->rects[region_id]->nb_colors <= 4) { + if (h->subtitle_rects[region_id]->nb_colors <= 4) { /* 2 bpp, some decoders do not support it correctly */ bpp_index = 0; - } else if (h->rects[region_id]->nb_colors <= 16) { + } else if (h->subtitle_rects[region_id]->nb_colors <= 16) { /* 4 bpp, standard encoding */ bpp_index = 1; - } else if (h->rects[region_id]->nb_colors <= 256) { + } else if (h->subtitle_rects[region_id]->nb_colors <= 256) { /* 8 bpp, standard encoding */ bpp_index = 2; } else { @@ -401,8 +401,8 @@ static int dvbsub_encode(AVCodecContext *avctx, uint8_t *outbuf, int buf_size, q += 2; /* segment length */ *q++ = region_id; *q++ = (s->object_version << 4) | (0 << 3) | 0x07; /* version , no fill */ - bytestream_put_be16(&q, h->rects[region_id]->w); /* region width */ - bytestream_put_be16(&q, h->rects[region_id]->h); /* region height */ + bytestream_put_be16(&q, h->subtitle_rects[region_id]->w); /* region width */ + bytestream_put_be16(&q, h->subtitle_rects[region_id]->h); /* region height */ *q++ = ((1 + bpp_index) << 5) | ((1 + bpp_index) << 2) | 0x03; *q++ = region_id; /* clut_id == region_id */ *q++ = 0; /* 8 bit fill colors */ @@ -416,9 +416,9 @@ static int dvbsub_encode(AVCodecContext *avctx, uint8_t *outbuf, int buf_size, bytestream_put_be16(&pseg_len, q - pseg_len - 2); } - buf_size -= h->num_rects * 22; + buf_size -= h->num_subtitle_rects * 22; - for (object_id = 0; object_id < h->num_rects; object_id++) { + for (object_id = 0; object_id < h->num_subtitle_rects; object_id++) { int (*dvb_encode_rle)(uint8_t **pq, int buf_size, const uint8_t *bitmap, int linesize, int w, int h); @@ -427,13 +427,13 @@ static int dvbsub_encode(AVCodecContext *avctx, uint8_t *outbuf, int buf_size, return AVERROR_BUFFER_TOO_SMALL; /* bpp_index maths */ - if (h->rects[object_id]->nb_colors <= 4) { + if (h->subtitle_rects[object_id]->nb_colors <= 4) { /* 2 bpp, some decoders do not support it correctly */ dvb_encode_rle = dvb_encode_rle2; - } else if (h->rects[object_id]->nb_colors <= 16) { + } else if (h->subtitle_rects[object_id]->nb_colors <= 16) { /* 4 bpp, standard encoding */ dvb_encode_rle = dvb_encode_rle4; - } else if (h->rects[object_id]->nb_colors <= 256) { + } else if (h->subtitle_rects[object_id]->nb_colors <= 256) { /* 8 bpp, standard encoding */ dvb_encode_rle = dvb_encode_rle8; } else { @@ -463,19 +463,19 @@ static int dvbsub_encode(AVCodecContext *avctx, uint8_t *outbuf, int buf_size, top_ptr = q; ret = dvb_encode_rle(&q, buf_size, - h->rects[object_id]->data[0], - h->rects[object_id]->w * 2, - h->rects[object_id]->w, - h->rects[object_id]->h >> 1); + h->subtitle_rects[object_id]->data[0], + h->subtitle_rects[object_id]->w * 2, + h->subtitle_rects[object_id]->w, + h->subtitle_rects[object_id]->h >> 1); if (ret < 0) return ret; buf_size -= ret; bottom_ptr = q; ret = dvb_encode_rle(&q, buf_size, - h->rects[object_id]->data[0] + h->rects[object_id]->w, - h->rects[object_id]->w * 2, - h->rects[object_id]->w, - h->rects[object_id]->h >> 1); + h->subtitle_rects[object_id]->data[0] + h->subtitle_rects[object_id]->w, + h->subtitle_rects[object_id]->w * 2, + h->subtitle_rects[object_id]->w, + h->subtitle_rects[object_id]->h >> 1); if (ret < 0) return ret; buf_size -= ret; diff --git a/libavcodec/dvdsubdec.c b/libavcodec/dvdsubdec.c index 52259f0730..6c7b315026 100644 --- a/libavcodec/dvdsubdec.c +++ b/libavcodec/dvdsubdec.c @@ -200,24 +200,24 @@ static void guess_palette(DVDSubContext* ctx, } } -static void reset_rects(AVSubtitle *sub_header) +static void reset_rects(AVFrame *sub_header) { int i; - if (sub_header->rects) { - for (i = 0; i < sub_header->num_rects; i++) { - av_freep(&sub_header->rects[i]->data[0]); - av_freep(&sub_header->rects[i]->data[1]); - av_freep(&sub_header->rects[i]); + if (sub_header->subtitle_rects) { + for (i = 0; i < sub_header->num_subtitle_rects; i++) { + av_freep(&sub_header->subtitle_rects[i]->data[0]); + av_freep(&sub_header->subtitle_rects[i]->data[1]); + av_freep(&sub_header->subtitle_rects[i]); } - av_freep(&sub_header->rects); - sub_header->num_rects = 0; + av_freep(&sub_header->subtitle_rects); + sub_header->num_subtitle_rects = 0; } } #define READ_OFFSET(a) (big_offsets ? AV_RB32(a) : AV_RB16(a)) -static int decode_dvd_subtitles(DVDSubContext *ctx, AVSubtitle *sub_header, +static int decode_dvd_subtitles(DVDSubContext *ctx, AVFrame *sub_header, const uint8_t *buf, int buf_size) { int cmd_pos, pos, cmd, x1, y1, x2, y2, next_cmd_pos; @@ -273,11 +273,11 @@ static int decode_dvd_subtitles(DVDSubContext *ctx, AVSubtitle *sub_header, break; case 0x01: /* set start date */ - sub_header->start_display_time = (date << 10) / 90; + sub_header->subtitle_start_time = (date << 10) / 90; break; case 0x02: /* set end date */ - sub_header->end_display_time = (date << 10) / 90; + sub_header->subtitle_end_time = (date << 10) / 90; break; case 0x03: /* set colormap */ @@ -371,14 +371,14 @@ static int decode_dvd_subtitles(DVDSubContext *ctx, AVSubtitle *sub_header, if (w > 0 && h > 1) { reset_rects(sub_header); memset(ctx->used_color, 0, sizeof(ctx->used_color)); - sub_header->rects = av_mallocz(sizeof(*sub_header->rects)); - if (!sub_header->rects) + sub_header->subtitle_rects = av_mallocz(sizeof(*sub_header->subtitle_rects)); + if (!sub_header->subtitle_rects) goto fail; - sub_header->rects[0] = av_mallocz(sizeof(AVSubtitleRect)); - if (!sub_header->rects[0]) + sub_header->subtitle_rects[0] = av_mallocz(sizeof(AVSubtitleRect)); + if (!sub_header->subtitle_rects[0]) goto fail; - sub_header->num_rects = 1; - bitmap = sub_header->rects[0]->data[0] = av_malloc(w * h); + sub_header->num_subtitle_rects = 1; + bitmap = sub_header->subtitle_rects[0]->data[0] = av_malloc(w * h); if (!bitmap) goto fail; if (decode_rle(bitmap, w * 2, w, (h + 1) / 2, ctx->used_color, @@ -387,28 +387,28 @@ static int decode_dvd_subtitles(DVDSubContext *ctx, AVSubtitle *sub_header, if (decode_rle(bitmap + w, w * 2, w, h / 2, ctx->used_color, buf, offset2, buf_size, is_8bit) < 0) goto fail; - sub_header->rects[0]->data[1] = av_mallocz(AVPALETTE_SIZE); - if (!sub_header->rects[0]->data[1]) + sub_header->subtitle_rects[0]->data[1] = av_mallocz(AVPALETTE_SIZE); + if (!sub_header->subtitle_rects[0]->data[1]) goto fail; if (is_8bit) { if (!yuv_palette) goto fail; - sub_header->rects[0]->nb_colors = 256; + sub_header->subtitle_rects[0]->nb_colors = 256; yuv_a_to_rgba(yuv_palette, alpha, - (uint32_t *)sub_header->rects[0]->data[1], + (uint32_t *)sub_header->subtitle_rects[0]->data[1], 256); } else { - sub_header->rects[0]->nb_colors = 4; - guess_palette(ctx, (uint32_t*)sub_header->rects[0]->data[1], + sub_header->subtitle_rects[0]->nb_colors = 4; + guess_palette(ctx, (uint32_t*)sub_header->subtitle_rects[0]->data[1], 0xffff00); } - sub_header->rects[0]->x = x1; - sub_header->rects[0]->y = y1; - sub_header->rects[0]->w = w; - sub_header->rects[0]->h = h; - sub_header->rects[0]->type = SUBTITLE_BITMAP; - sub_header->rects[0]->linesize[0] = w; - sub_header->rects[0]->flags = is_menu ? AV_SUBTITLE_FLAG_FORCED : 0; + sub_header->subtitle_rects[0]->x = x1; + sub_header->subtitle_rects[0]->y = y1; + sub_header->subtitle_rects[0]->w = w; + sub_header->subtitle_rects[0]->h = h; + sub_header->subtitle_rects[0]->type = AV_SUBTITLE_FMT_BITMAP; + sub_header->subtitle_rects[0]->linesize[0] = w; + sub_header->subtitle_rects[0]->flags = is_menu ? AV_SUBTITLE_FLAG_FORCED : 0; } } if (next_cmd_pos < cmd_pos) { @@ -419,7 +419,7 @@ static int decode_dvd_subtitles(DVDSubContext *ctx, AVSubtitle *sub_header, break; cmd_pos = next_cmd_pos; } - if (sub_header->num_rects > 0) + if (sub_header->num_subtitle_rects > 0) return is_menu; fail: reset_rects(sub_header); @@ -439,18 +439,18 @@ static int is_transp(const uint8_t *buf, int pitch, int n, } /* return 0 if empty rectangle, 1 if non empty */ -static int find_smallest_bounding_rectangle(DVDSubContext *ctx, AVSubtitle *s) +static int find_smallest_bounding_rectangle(DVDSubContext *ctx, AVFrame *s) { uint8_t transp_color[256] = { 0 }; int y1, y2, x1, x2, y, w, h, i; uint8_t *bitmap; int transparent = 1; - if (s->num_rects == 0 || !s->rects || s->rects[0]->w <= 0 || s->rects[0]->h <= 0) + if (s->num_subtitle_rects == 0 || !s->subtitle_rects || s->subtitle_rects[0]->w <= 0 || s->subtitle_rects[0]->h <= 0) return 0; - for(i = 0; i < s->rects[0]->nb_colors; i++) { - if ((((uint32_t *)s->rects[0]->data[1])[i] >> 24) == 0) { + for(i = 0; i < s->subtitle_rects[0]->nb_colors; i++) { + if ((((uint32_t *)s->subtitle_rects[0]->data[1])[i] >> 24) == 0) { transp_color[i] = 1; } else if (ctx->used_color[i]) transparent = 0; @@ -458,25 +458,25 @@ static int find_smallest_bounding_rectangle(DVDSubContext *ctx, AVSubtitle *s) if (transparent) return 0; y1 = 0; - while (y1 < s->rects[0]->h && is_transp(s->rects[0]->data[0] + y1 * s->rects[0]->linesize[0], - 1, s->rects[0]->w, transp_color)) + while (y1 < s->subtitle_rects[0]->h && is_transp(s->subtitle_rects[0]->data[0] + y1 * s->subtitle_rects[0]->linesize[0], + 1, s->subtitle_rects[0]->w, transp_color)) y1++; - if (y1 == s->rects[0]->h) { - av_freep(&s->rects[0]->data[0]); - s->rects[0]->w = s->rects[0]->h = 0; + if (y1 == s->subtitle_rects[0]->h) { + av_freep(&s->subtitle_rects[0]->data[0]); + s->subtitle_rects[0]->w = s->subtitle_rects[0]->h = 0; return 0; } - y2 = s->rects[0]->h - 1; - while (y2 > 0 && is_transp(s->rects[0]->data[0] + y2 * s->rects[0]->linesize[0], 1, - s->rects[0]->w, transp_color)) + y2 = s->subtitle_rects[0]->h - 1; + while (y2 > 0 && is_transp(s->subtitle_rects[0]->data[0] + y2 * s->subtitle_rects[0]->linesize[0], 1, + s->subtitle_rects[0]->w, transp_color)) y2--; x1 = 0; - while (x1 < (s->rects[0]->w - 1) && is_transp(s->rects[0]->data[0] + x1, s->rects[0]->linesize[0], - s->rects[0]->h, transp_color)) + while (x1 < (s->subtitle_rects[0]->w - 1) && is_transp(s->subtitle_rects[0]->data[0] + x1, s->subtitle_rects[0]->linesize[0], + s->subtitle_rects[0]->h, transp_color)) x1++; - x2 = s->rects[0]->w - 1; - while (x2 > 0 && is_transp(s->rects[0]->data[0] + x2, s->rects[0]->linesize[0], s->rects[0]->h, + x2 = s->subtitle_rects[0]->w - 1; + while (x2 > 0 && is_transp(s->subtitle_rects[0]->data[0] + x2, s->subtitle_rects[0]->linesize[0], s->subtitle_rects[0]->h, transp_color)) x2--; w = x2 - x1 + 1; @@ -485,15 +485,15 @@ static int find_smallest_bounding_rectangle(DVDSubContext *ctx, AVSubtitle *s) if (!bitmap) return 1; for(y = 0; y < h; y++) { - memcpy(bitmap + w * y, s->rects[0]->data[0] + x1 + (y1 + y) * s->rects[0]->linesize[0], w); + memcpy(bitmap + w * y, s->subtitle_rects[0]->data[0] + x1 + (y1 + y) * s->subtitle_rects[0]->linesize[0], w); } - av_freep(&s->rects[0]->data[0]); - s->rects[0]->data[0] = bitmap; - s->rects[0]->linesize[0] = w; - s->rects[0]->w = w; - s->rects[0]->h = h; - s->rects[0]->x += x1; - s->rects[0]->y += y1; + av_freep(&s->subtitle_rects[0]->data[0]); + s->subtitle_rects[0]->data[0] = bitmap; + s->subtitle_rects[0]->linesize[0] = w; + s->subtitle_rects[0]->w = w; + s->subtitle_rects[0]->h = h; + s->subtitle_rects[0]->x += x1; + s->subtitle_rects[0]->y += y1; return 1; } @@ -554,7 +554,7 @@ static int dvdsub_decode(AVCodecContext *avctx, DVDSubContext *ctx = avctx->priv_data; const uint8_t *buf = avpkt->data; int buf_size = avpkt->size; - AVSubtitle *sub = data; + AVFrame *sub = data; int appended = 0; int is_menu; @@ -586,7 +586,7 @@ static int dvdsub_decode(AVCodecContext *avctx, if (!is_menu && find_smallest_bounding_rectangle(ctx, sub) == 0) goto no_subtitle; - if (ctx->forced_subs_only && !(sub->rects[0]->flags & AV_SUBTITLE_FLAG_FORCED)) + if (ctx->forced_subs_only && !(sub->subtitle_rects[0]->flags & AV_SUBTITLE_FLAG_FORCED)) goto no_subtitle; #if defined(DEBUG) @@ -595,10 +595,10 @@ static int dvdsub_decode(AVCodecContext *avctx, snprintf(ppm_name, sizeof(ppm_name), "/tmp/%05d.ppm", ctx->sub_id++); ff_dlog(NULL, "start=%d ms end =%d ms\n", - sub->start_display_time, - sub->end_display_time); - ppm_save(ppm_name, sub->rects[0]->data[0], - sub->rects[0]->w, sub->rects[0]->h, (uint32_t*) sub->rects[0]->data[1]); + sub->subtitle_start_time, + sub->subtitle_end_time); + ppm_save(ppm_name, sub->subtitle_rects[0]->data[0], + sub->subtitle_rects[0]->w, sub->subtitle_rects[0]->h, (uint32_t*) sub->subtitle_rects[0]->data[1]); } #endif diff --git a/libavcodec/dvdsubenc.c b/libavcodec/dvdsubenc.c index ff4fbed39d..53df1251ed 100644 --- a/libavcodec/dvdsubenc.c +++ b/libavcodec/dvdsubenc.c @@ -250,12 +250,12 @@ static void copy_rectangle(AVSubtitleRect *dst, AVSubtitleRect *src, int cmap[]) static int encode_dvd_subtitles(AVCodecContext *avctx, uint8_t *outbuf, int outbuf_size, - const AVSubtitle *h) + const AVFrame *h) { DVDSubtitleContext *dvdc = avctx->priv_data; uint8_t *q, *qq; int offset1, offset2; - int i, rects = h->num_rects, ret; + int i, rects = h->num_subtitle_rects, ret; unsigned global_palette_hits[33] = { 0 }; int cmap[256]; int out_palette[4]; @@ -265,34 +265,34 @@ static int encode_dvd_subtitles(AVCodecContext *avctx, int x2, y2; int forced = 0; - if (rects == 0 || !h->rects) + if (rects == 0 || !h->subtitle_rects) return AVERROR(EINVAL); for (i = 0; i < rects; i++) - if (h->rects[i]->type != SUBTITLE_BITMAP) { + if (h->subtitle_rects[i]->type != AV_SUBTITLE_FMT_BITMAP) { av_log(avctx, AV_LOG_ERROR, "Bitmap subtitle required\n"); return AVERROR(EINVAL); } /* Mark this subtitle forced if any of the rectangles is forced. */ for (i = 0; i < rects; i++) - if ((h->rects[i]->flags & AV_SUBTITLE_FLAG_FORCED) != 0) { + if ((h->subtitle_rects[i]->flags & AV_SUBTITLE_FLAG_FORCED) != 0) { forced = 1; break; } - vrect = *h->rects[0]; + vrect = *h->subtitle_rects[0]; if (rects > 1) { /* DVD subtitles can have only one rectangle: build a virtual rectangle containing all actual rectangles. The data of the rectangles will be copied later, when the palette is decided, because the rectangles may have different palettes. */ - int xmin = h->rects[0]->x, xmax = xmin + h->rects[0]->w; - int ymin = h->rects[0]->y, ymax = ymin + h->rects[0]->h; + int xmin = h->subtitle_rects[0]->x, xmax = xmin + h->subtitle_rects[0]->w; + int ymin = h->subtitle_rects[0]->y, ymax = ymin + h->subtitle_rects[0]->h; for (i = 1; i < rects; i++) { - xmin = FFMIN(xmin, h->rects[i]->x); - ymin = FFMIN(ymin, h->rects[i]->y); - xmax = FFMAX(xmax, h->rects[i]->x + h->rects[i]->w); - ymax = FFMAX(ymax, h->rects[i]->y + h->rects[i]->h); + xmin = FFMIN(xmin, h->subtitle_rects[i]->x); + ymin = FFMIN(ymin, h->subtitle_rects[i]->y); + xmax = FFMAX(xmax, h->subtitle_rects[i]->x + h->subtitle_rects[i]->w); + ymax = FFMAX(ymax, h->subtitle_rects[i]->y + h->subtitle_rects[i]->h); } vrect.x = xmin; vrect.y = ymin; @@ -304,11 +304,11 @@ static int encode_dvd_subtitles(AVCodecContext *avctx, /* Count pixels outside the virtual rectangle as transparent */ global_palette_hits[0] = vrect.w * vrect.h; for (i = 0; i < rects; i++) - global_palette_hits[0] -= h->rects[i]->w * h->rects[i]->h; + global_palette_hits[0] -= h->subtitle_rects[i]->w * h->subtitle_rects[i]->h; } for (i = 0; i < rects; i++) - count_colors(avctx, global_palette_hits, h->rects[i]); + count_colors(avctx, global_palette_hits, h->subtitle_rects[i]); select_palette(avctx, out_palette, out_alpha, global_palette_hits); if (rects > 1) { @@ -317,14 +317,14 @@ static int encode_dvd_subtitles(AVCodecContext *avctx, vrect.data [0] = vrect_data; vrect.linesize[0] = vrect.w; for (i = 0; i < rects; i++) { - build_color_map(avctx, cmap, (uint32_t *)h->rects[i]->data[1], + build_color_map(avctx, cmap, (uint32_t *)h->subtitle_rects[i]->data[1], out_palette, out_alpha); - copy_rectangle(&vrect, h->rects[i], cmap); + copy_rectangle(&vrect, h->subtitle_rects[i], cmap); } for (i = 0; i < 4; i++) cmap[i] = i; } else { - build_color_map(avctx, cmap, (uint32_t *)h->rects[0]->data[1], + build_color_map(avctx, cmap, (uint32_t *)h->subtitle_rects[0]->data[1], out_palette, out_alpha); } @@ -362,7 +362,7 @@ static int encode_dvd_subtitles(AVCodecContext *avctx, bytestream_put_be16(&qq, q - outbuf); // send start display command - bytestream_put_be16(&q, (h->start_display_time*90) >> 10); + bytestream_put_be16(&q, (h->subtitle_start_time*90) >> 10); bytestream_put_be16(&q, (q - outbuf) /*- 2 */ + 8 + 12 + 2); *q++ = 0x03; // palette - 4 nibbles *q++ = (out_palette[3] << 4) | out_palette[2]; @@ -394,7 +394,7 @@ static int encode_dvd_subtitles(AVCodecContext *avctx, *q++ = 0xff; // terminating command // send stop display command last - bytestream_put_be16(&q, (h->end_display_time*90) >> 10); + bytestream_put_be16(&q, (h->subtitle_end_time*90) >> 10); bytestream_put_be16(&q, (q - outbuf) - 2 /*+ 4*/); *q++ = 0x02; // set end *q++ = 0xff; // terminating command @@ -469,7 +469,7 @@ static int dvdsub_init(AVCodecContext *avctx) static int dvdsub_encode(AVCodecContext *avctx, unsigned char *buf, int buf_size, - const AVSubtitle *sub) + const AVFrame *sub) { //DVDSubtitleContext *s = avctx->priv_data; int ret; diff --git a/libavcodec/encode.c b/libavcodec/encode.c index 98dfbfdff3..289f6e88f1 100644 --- a/libavcodec/encode.c +++ b/libavcodec/encode.c @@ -141,11 +141,11 @@ fail: } int avcodec_encode_subtitle(AVCodecContext *avctx, uint8_t *buf, int buf_size, - const AVSubtitle *sub) + const AVFrame *sub) { int ret; - if (sub->start_display_time) { - av_log(avctx, AV_LOG_ERROR, "start_display_time must be 0.\n"); + if (sub->subtitle_start_time) { + av_log(avctx, AV_LOG_ERROR, "subtitle_start_time must be 0.\n"); return -1; } diff --git a/libavcodec/jacosubdec.c b/libavcodec/jacosubdec.c index 698895a86b..4c87d0f06d 100644 --- a/libavcodec/jacosubdec.c +++ b/libavcodec/jacosubdec.c @@ -166,7 +166,7 @@ static int jacosub_decode_frame(AVCodecContext *avctx, void *data, int *got_sub_ptr, AVPacket *avpkt) { int ret; - AVSubtitle *sub = data; + AVFrame *sub = data; const char *ptr = avpkt->data; FFASSDecoderContext *s = avctx->priv_data; @@ -190,7 +190,7 @@ static int jacosub_decode_frame(AVCodecContext *avctx, } end: - *got_sub_ptr = sub->num_rects > 0; + *got_sub_ptr = sub->num_subtitle_rects > 0; return avpkt->size; } diff --git a/libavcodec/libzvbi-teletextdec.c b/libavcodec/libzvbi-teletextdec.c index 1073d6a0bd..b597042c52 100644 --- a/libavcodec/libzvbi-teletextdec.c +++ b/libavcodec/libzvbi-teletextdec.c @@ -215,7 +215,7 @@ static int gen_sub_text(TeletextContext *ctx, AVSubtitleRect *sub_rect, vbi_page } if (buf.len) { - sub_rect->type = SUBTITLE_ASS; + sub_rect->type = AV_SUBTITLE_FMT_ASS; sub_rect->ass = create_ass_text(ctx, buf.str); if (!sub_rect->ass) { @@ -224,7 +224,7 @@ static int gen_sub_text(TeletextContext *ctx, AVSubtitleRect *sub_rect, vbi_page } av_log(ctx, AV_LOG_DEBUG, "subtext:%s:txetbus\n", sub_rect->ass); } else { - sub_rect->type = SUBTITLE_NONE; + sub_rect->type = AV_SUBTITLE_FMT_NONE; } av_bprint_finalize(&buf, NULL); return 0; @@ -393,7 +393,7 @@ static int gen_sub_ass(TeletextContext *ctx, AVSubtitleRect *sub_rect, vbi_page } if (buf.len) { - sub_rect->type = SUBTITLE_ASS; + sub_rect->type = AV_SUBTITLE_FMT_ASS; sub_rect->ass = ff_ass_get_dialog(ctx->readorder++, 0, is_subtitle_page ? "Subtitle" : "Teletext", NULL, buf.str); if (!sub_rect->ass) { @@ -402,7 +402,7 @@ static int gen_sub_ass(TeletextContext *ctx, AVSubtitleRect *sub_rect, vbi_page } av_log(ctx, AV_LOG_DEBUG, "subtext:%s:txetbus\n", sub_rect->ass); } else { - sub_rect->type = SUBTITLE_NONE; + sub_rect->type = AV_SUBTITLE_FMT_NONE; } av_bprint_finalize(&buf, NULL); return 0; @@ -462,7 +462,7 @@ static int gen_sub_bitmap(TeletextContext *ctx, AVSubtitleRect *sub_rect, vbi_pa if (vc >= vcend) { av_log(ctx, AV_LOG_DEBUG, "dropping empty page %3x\n", page->pgno); - sub_rect->type = SUBTITLE_NONE; + sub_rect->type = AV_SUBTITLE_FMT_NONE; return 0; } @@ -500,7 +500,7 @@ static int gen_sub_bitmap(TeletextContext *ctx, AVSubtitleRect *sub_rect, vbi_pa } ((uint32_t *)sub_rect->data[1])[VBI_TRANSPARENT_BLACK] = RGBA(0, 0, 0, 0); ((uint32_t *)sub_rect->data[1])[VBI_TRANSPARENT_BLACK + VBI_NB_COLORS] = RGBA(0, 0, 0, 0); - sub_rect->type = SUBTITLE_BITMAP; + sub_rect->type = AV_SUBTITLE_FMT_BITMAP; return 0; } @@ -639,7 +639,7 @@ static int slice_to_vbi_lines(TeletextContext *ctx, uint8_t* buf, int size) static int teletext_decode_frame(AVCodecContext *avctx, void *data, int *got_sub_ptr, AVPacket *pkt) { TeletextContext *ctx = avctx->priv_data; - AVSubtitle *sub = data; + AVFrame *sub = data; int ret = 0; if (!ctx->vbi) { @@ -695,7 +695,7 @@ static int teletext_decode_frame(AVCodecContext *avctx, void *data, int *got_sub sub->num_rects = 0; sub->pts = ctx->pages->pts; - if (ctx->pages->sub_rect->type != SUBTITLE_NONE) { + if (ctx->pages->sub_rect->type != AV_SUBTITLE_FMT_NONE) { sub->rects = av_malloc(sizeof(*sub->rects)); if (sub->rects) { sub->num_rects = 1; diff --git a/libavcodec/microdvddec.c b/libavcodec/microdvddec.c index c45fe043bf..8618c94bf2 100644 --- a/libavcodec/microdvddec.c +++ b/libavcodec/microdvddec.c @@ -277,7 +277,7 @@ static void microdvd_close_no_persistent_tags(AVBPrint *new_line, static int microdvd_decode_frame(AVCodecContext *avctx, void *data, int *got_sub_ptr, AVPacket *avpkt) { - AVSubtitle *sub = data; + AVFrame *sub = data; AVBPrint new_line; char *line = avpkt->data; char *end = avpkt->data + avpkt->size; @@ -316,7 +316,7 @@ static int microdvd_decode_frame(AVCodecContext *avctx, return ret; } - *got_sub_ptr = sub->num_rects > 0; + *got_sub_ptr = sub->num_subtitle_rects > 0; return avpkt->size; } diff --git a/libavcodec/movtextdec.c b/libavcodec/movtextdec.c index 4e14ae5900..55fc1db69d 100644 --- a/libavcodec/movtextdec.c +++ b/libavcodec/movtextdec.c @@ -461,7 +461,7 @@ static int mov_text_init(AVCodecContext *avctx) { static int mov_text_decode_frame(AVCodecContext *avctx, void *data, int *got_sub_ptr, AVPacket *avpkt) { - AVSubtitle *sub = data; + AVFrame *sub = data; MovTextContext *m = avctx->priv_data; int ret; AVBPrint buf; @@ -549,7 +549,7 @@ static int mov_text_decode_frame(AVCodecContext *avctx, av_bprint_finalize(&buf, NULL); if (ret < 0) return ret; - *got_sub_ptr = sub->num_rects > 0; + *got_sub_ptr = sub->num_subtitle_rects > 0; return avpkt->size; } diff --git a/libavcodec/movtextenc.c b/libavcodec/movtextenc.c index 2ae5a9bf0b..315fb9732a 100644 --- a/libavcodec/movtextenc.c +++ b/libavcodec/movtextenc.c @@ -635,7 +635,7 @@ static const ASSCodesCallbacks mov_text_callbacks = { }; static int mov_text_encode_frame(AVCodecContext *avctx, unsigned char *buf, - int bufsize, const AVSubtitle *sub) + int bufsize, const AVFrame *sub) { MovTextContext *s = avctx->priv_data; ASSDialog *dialog; @@ -646,11 +646,11 @@ static int mov_text_encode_frame(AVCodecContext *avctx, unsigned char *buf, s->text_pos = 0; s->count = 0; s->box_flags = 0; - for (i = 0; i < sub->num_rects; i++) { - const char *ass = sub->rects[i]->ass; + for (i = 0; i < sub->num_subtitle_rects; i++) { + const char *ass = sub->subtitle_rects[i]->ass; - if (sub->rects[i]->type != SUBTITLE_ASS) { - av_log(avctx, AV_LOG_ERROR, "Only SUBTITLE_ASS type supported.\n"); + if (sub->subtitle_rects[i]->type != AV_SUBTITLE_FMT_ASS) { + av_log(avctx, AV_LOG_ERROR, "Only AV_SUBTITLE_FMT_ASS type supported.\n"); return AVERROR(EINVAL); } diff --git a/libavcodec/mpl2dec.c b/libavcodec/mpl2dec.c index 61e47050ec..76efc0ce4f 100644 --- a/libavcodec/mpl2dec.c +++ b/libavcodec/mpl2dec.c @@ -68,7 +68,7 @@ static int mpl2_decode_frame(AVCodecContext *avctx, void *data, { int ret = 0; AVBPrint buf; - AVSubtitle *sub = data; + AVFrame *sub = data; const char *ptr = avpkt->data; FFASSDecoderContext *s = avctx->priv_data; @@ -78,7 +78,7 @@ static int mpl2_decode_frame(AVCodecContext *avctx, void *data, av_bprint_finalize(&buf, NULL); if (ret < 0) return ret; - *got_sub_ptr = sub->num_rects > 0; + *got_sub_ptr = sub->num_subtitle_rects > 0; return avpkt->size; } diff --git a/libavcodec/pgssubdec.c b/libavcodec/pgssubdec.c index 55eda4c2a9..1e4557ed42 100644 --- a/libavcodec/pgssubdec.c +++ b/libavcodec/pgssubdec.c @@ -498,28 +498,28 @@ static int parse_presentation_segment(AVCodecContext *avctx, static int display_end_segment(AVCodecContext *avctx, void *data, const uint8_t *buf, int buf_size) { - AVSubtitle *sub = data; + AVFrame *sub = data; PGSSubContext *ctx = avctx->priv_data; int64_t pts; PGSSubPalette *palette; int i, ret; - pts = ctx->presentation.pts != AV_NOPTS_VALUE ? ctx->presentation.pts : sub->pts; + pts = ctx->presentation.pts != AV_NOPTS_VALUE ? ctx->presentation.pts : sub->subtitle_pts; memset(sub, 0, sizeof(*sub)); - sub->pts = pts; + sub->subtitle_pts = pts; ctx->presentation.pts = AV_NOPTS_VALUE; - sub->start_display_time = 0; + sub->subtitle_start_time = 0; // There is no explicit end time for PGS subtitles. The end time // is defined by the start of the next sub which may contain no // objects (i.e. clears the previous sub) - sub->end_display_time = UINT32_MAX; - sub->format = 0; + sub->subtitle_end_time = UINT32_MAX; + sub->format = AV_SUBTITLE_FMT_BITMAP; // Blank if last object_count was 0. if (!ctx->presentation.object_count) return 1; - sub->rects = av_mallocz_array(ctx->presentation.object_count, sizeof(*sub->rects)); - if (!sub->rects) { + sub->subtitle_rects = av_mallocz_array(ctx->presentation.object_count, sizeof(*sub->subtitle_rects)); + if (!sub->subtitle_rects) { return AVERROR(ENOMEM); } palette = find_palette(ctx->presentation.palette_id, &ctx->palettes); @@ -527,19 +527,19 @@ static int display_end_segment(AVCodecContext *avctx, void *data, // Missing palette. Should only happen with damaged streams. av_log(avctx, AV_LOG_ERROR, "Invalid palette id %d\n", ctx->presentation.palette_id); - avsubtitle_free(sub); + av_frame_free(&sub); return AVERROR_INVALIDDATA; } for (i = 0; i < ctx->presentation.object_count; i++) { PGSSubObject *object; - sub->rects[i] = av_mallocz(sizeof(*sub->rects[0])); - if (!sub->rects[i]) { - avsubtitle_free(sub); + sub->subtitle_rects[i] = av_mallocz(sizeof(*sub->subtitle_rects[0])); + if (!sub->subtitle_rects[i]) { + av_frame_free(&sub); return AVERROR(ENOMEM); } - sub->num_rects++; - sub->rects[i]->type = SUBTITLE_BITMAP; + sub->num_subtitle_rects++; + sub->subtitle_rects[i]->type = AV_SUBTITLE_FMT_BITMAP; /* Process bitmap */ object = find_object(ctx->presentation.objects[i].id, &ctx->objects); @@ -548,54 +548,54 @@ static int display_end_segment(AVCodecContext *avctx, void *data, av_log(avctx, AV_LOG_ERROR, "Invalid object id %d\n", ctx->presentation.objects[i].id); if (avctx->err_recognition & AV_EF_EXPLODE) { - avsubtitle_free(sub); + av_frame_free(&sub); return AVERROR_INVALIDDATA; } // Leaves rect empty with 0 width and height. continue; } if (ctx->presentation.objects[i].composition_flag & 0x40) - sub->rects[i]->flags |= AV_SUBTITLE_FLAG_FORCED; + sub->subtitle_rects[i]->flags |= AV_SUBTITLE_FLAG_FORCED; - sub->rects[i]->x = ctx->presentation.objects[i].x; - sub->rects[i]->y = ctx->presentation.objects[i].y; + sub->subtitle_rects[i]->x = ctx->presentation.objects[i].x; + sub->subtitle_rects[i]->y = ctx->presentation.objects[i].y; if (object->rle) { - sub->rects[i]->w = object->w; - sub->rects[i]->h = object->h; + sub->subtitle_rects[i]->w = object->w; + sub->subtitle_rects[i]->h = object->h; - sub->rects[i]->linesize[0] = object->w; + sub->subtitle_rects[i]->linesize[0] = object->w; if (object->rle_remaining_len) { av_log(avctx, AV_LOG_ERROR, "RLE data length %u is %u bytes shorter than expected\n", object->rle_data_len, object->rle_remaining_len); if (avctx->err_recognition & AV_EF_EXPLODE) { - avsubtitle_free(sub); + av_frame_free(&sub); return AVERROR_INVALIDDATA; } } - ret = decode_rle(avctx, sub->rects[i], object->rle, object->rle_data_len); + ret = decode_rle(avctx, sub->subtitle_rects[i], object->rle, object->rle_data_len); if (ret < 0) { if ((avctx->err_recognition & AV_EF_EXPLODE) || ret == AVERROR(ENOMEM)) { - avsubtitle_free(sub); + av_frame_free(&sub); return ret; } - sub->rects[i]->w = 0; - sub->rects[i]->h = 0; + sub->subtitle_rects[i]->w = 0; + sub->subtitle_rects[i]->h = 0; continue; } } /* Allocate memory for colors */ - sub->rects[i]->nb_colors = 256; - sub->rects[i]->data[1] = av_mallocz(AVPALETTE_SIZE); - if (!sub->rects[i]->data[1]) { - avsubtitle_free(sub); + sub->subtitle_rects[i]->nb_colors = 256; + sub->subtitle_rects[i]->data[1] = av_mallocz(AVPALETTE_SIZE); + if (!sub->subtitle_rects[i]->data[1]) { + av_frame_free(&sub); return AVERROR(ENOMEM); } if (!ctx->forced_subs_only || ctx->presentation.objects[i].composition_flag & 0x40) - memcpy(sub->rects[i]->data[1], palette->clut, sub->rects[i]->nb_colors * sizeof(uint32_t)); + memcpy(sub->subtitle_rects[i]->data[1], palette->clut, sub->subtitle_rects[i]->nb_colors * sizeof(uint32_t)); } return 1; } @@ -605,7 +605,7 @@ static int decode(AVCodecContext *avctx, void *data, int *got_sub_ptr, { const uint8_t *buf = avpkt->data; int buf_size = avpkt->size; - + AVFrame *sub = data; const uint8_t *buf_end; uint8_t segment_type; int segment_length; @@ -649,7 +649,7 @@ static int decode(AVCodecContext *avctx, void *data, int *got_sub_ptr, ret = parse_object_segment(avctx, buf, segment_length); break; case PRESENTATION_SEGMENT: - ret = parse_presentation_segment(avctx, buf, segment_length, ((AVSubtitle*)(data))->pts); + ret = parse_presentation_segment(avctx, buf, segment_length, sub->subtitle_pts); break; case WINDOW_SEGMENT: /* @@ -678,7 +678,7 @@ static int decode(AVCodecContext *avctx, void *data, int *got_sub_ptr, break; } if (ret < 0 && (avctx->err_recognition & AV_EF_EXPLODE)) { - avsubtitle_free(data); + av_frame_free(&sub); *got_sub_ptr = 0; return ret; } diff --git a/libavcodec/realtextdec.c b/libavcodec/realtextdec.c index 11b586d493..8ce24e0bcc 100644 --- a/libavcodec/realtextdec.c +++ b/libavcodec/realtextdec.c @@ -60,7 +60,7 @@ static int realtext_decode_frame(AVCodecContext *avctx, void *data, int *got_sub_ptr, AVPacket *avpkt) { int ret = 0; - AVSubtitle *sub = data; + AVFrame *sub = data; const char *ptr = avpkt->data; FFASSDecoderContext *s = avctx->priv_data; AVBPrint buf; @@ -71,7 +71,7 @@ static int realtext_decode_frame(AVCodecContext *avctx, av_bprint_finalize(&buf, NULL); if (ret < 0) return ret; - *got_sub_ptr = sub->num_rects > 0; + *got_sub_ptr = sub->num_subtitle_rects > 0; return avpkt->size; } diff --git a/libavcodec/samidec.c b/libavcodec/samidec.c index 32d07447b4..2c25edefea 100644 --- a/libavcodec/samidec.c +++ b/libavcodec/samidec.c @@ -135,7 +135,7 @@ end: static int sami_decode_frame(AVCodecContext *avctx, void *data, int *got_sub_ptr, AVPacket *avpkt) { - AVSubtitle *sub = data; + AVFrame *sub = data; const char *ptr = avpkt->data; SAMIContext *sami = avctx->priv_data; @@ -148,7 +148,7 @@ static int sami_decode_frame(AVCodecContext *avctx, if (ret < 0) return ret; } - *got_sub_ptr = sub->num_rects > 0; + *got_sub_ptr = sub->num_subtitle_rects > 0; return avpkt->size; } diff --git a/libavcodec/srtdec.c b/libavcodec/srtdec.c index 4f16226b83..557ffab45e 100644 --- a/libavcodec/srtdec.c +++ b/libavcodec/srtdec.c @@ -56,7 +56,7 @@ static int srt_to_ass(AVCodecContext *avctx, AVBPrint *dst, static int srt_decode_frame(AVCodecContext *avctx, void *data, int *got_sub_ptr, AVPacket *avpkt) { - AVSubtitle *sub = data; + AVFrame *sub = data; AVBPrint buffer; int x1 = -1, y1 = -1, x2 = -1, y2 = -1; int ret; @@ -83,7 +83,7 @@ static int srt_decode_frame(AVCodecContext *avctx, if (ret < 0) return ret; - *got_sub_ptr = sub->num_rects > 0; + *got_sub_ptr = sub->num_subtitle_rects > 0; return avpkt->size; } diff --git a/libavcodec/srtenc.c b/libavcodec/srtenc.c index 2e3ac55770..f4a9de1e70 100644 --- a/libavcodec/srtenc.c +++ b/libavcodec/srtenc.c @@ -228,7 +228,7 @@ static const ASSCodesCallbacks text_callbacks = { }; static int encode_frame(AVCodecContext *avctx, - unsigned char *buf, int bufsize, const AVSubtitle *sub, + unsigned char *buf, int bufsize, const AVFrame *sub, const ASSCodesCallbacks *cb) { SRTContext *s = avctx->priv_data; @@ -237,11 +237,11 @@ static int encode_frame(AVCodecContext *avctx, av_bprint_clear(&s->buffer); - for (i=0; inum_rects; i++) { - const char *ass = sub->rects[i]->ass; + for (i=0; inum_subtitle_rects; i++) { + const char *ass = sub->subtitle_rects[i]->ass; - if (sub->rects[i]->type != SUBTITLE_ASS) { - av_log(avctx, AV_LOG_ERROR, "Only SUBTITLE_ASS type supported.\n"); + if (sub->subtitle_rects[i]->type != AV_SUBTITLE_FMT_ASS) { + av_log(avctx, AV_LOG_ERROR, "Only AV_SUBTITLE_FMT_ASS type supported.\n"); return AVERROR(EINVAL); } @@ -270,13 +270,13 @@ static int encode_frame(AVCodecContext *avctx, } static int srt_encode_frame(AVCodecContext *avctx, - unsigned char *buf, int bufsize, const AVSubtitle *sub) + unsigned char *buf, int bufsize, const AVFrame *sub) { return encode_frame(avctx, buf, bufsize, sub, &srt_callbacks); } static int text_encode_frame(AVCodecContext *avctx, - unsigned char *buf, int bufsize, const AVSubtitle *sub) + unsigned char *buf, int bufsize, const AVFrame *sub) { return encode_frame(avctx, buf, bufsize, sub, &text_callbacks); } diff --git a/libavcodec/subviewerdec.c b/libavcodec/subviewerdec.c index 5c650d0cde..e4eff51d33 100644 --- a/libavcodec/subviewerdec.c +++ b/libavcodec/subviewerdec.c @@ -51,7 +51,7 @@ static int subviewer_decode_frame(AVCodecContext *avctx, void *data, int *got_sub_ptr, AVPacket *avpkt) { int ret = 0; - AVSubtitle *sub = data; + AVFrame *sub = data; const char *ptr = avpkt->data; FFASSDecoderContext *s = avctx->priv_data; AVBPrint buf; @@ -62,7 +62,7 @@ static int subviewer_decode_frame(AVCodecContext *avctx, av_bprint_finalize(&buf, NULL); if (ret < 0) return ret; - *got_sub_ptr = sub->num_rects > 0; + *got_sub_ptr = sub->num_subtitle_rects > 0; return avpkt->size; } diff --git a/libavcodec/textdec.c b/libavcodec/textdec.c index 308553660a..274b53f74d 100644 --- a/libavcodec/textdec.c +++ b/libavcodec/textdec.c @@ -48,7 +48,7 @@ static int text_decode_frame(AVCodecContext *avctx, void *data, { int ret = 0; AVBPrint buf; - AVSubtitle *sub = data; + AVFrame *sub = data; const char *ptr = avpkt->data; TextContext *text = avctx->priv_data; @@ -60,7 +60,7 @@ static int text_decode_frame(AVCodecContext *avctx, void *data, av_bprint_finalize(&buf, NULL); if (ret < 0) return ret; - *got_sub_ptr = sub->num_rects > 0; + *got_sub_ptr = sub->num_subtitle_rects > 0; return avpkt->size; } diff --git a/libavcodec/ttmlenc.c b/libavcodec/ttmlenc.c index ad2eddfdd5..3a439db98e 100644 --- a/libavcodec/ttmlenc.c +++ b/libavcodec/ttmlenc.c @@ -78,7 +78,7 @@ static const ASSCodesCallbacks ttml_callbacks = { }; static int ttml_encode_frame(AVCodecContext *avctx, uint8_t *buf, - int bufsize, const AVSubtitle *sub) + int bufsize, const AVFrame *sub) { TTMLContext *s = avctx->priv_data; ASSDialog *dialog; @@ -86,12 +86,12 @@ static int ttml_encode_frame(AVCodecContext *avctx, uint8_t *buf, av_bprint_clear(&s->buffer); - for (i=0; inum_rects; i++) { - const char *ass = sub->rects[i]->ass; + for (i=0; inum_subtitle_rects; i++) { + const char *ass = sub->subtitle_rects[i]->ass; int ret; - if (sub->rects[i]->type != SUBTITLE_ASS) { - av_log(avctx, AV_LOG_ERROR, "Only SUBTITLE_ASS type supported.\n"); + if (sub->subtitle_rects[i]->type != AV_SUBTITLE_FMT_ASS) { + av_log(avctx, AV_LOG_ERROR, "Only AV_SUBTITLE_FMT_ASS type supported.\n"); return AVERROR(EINVAL); } diff --git a/libavcodec/utils.c b/libavcodec/utils.c index cfc07cbcb8..767dfabd1e 100644 --- a/libavcodec/utils.c +++ b/libavcodec/utils.c @@ -818,6 +818,17 @@ int av_get_audio_frame_duration(AVCodecContext *avctx, int frame_bytes) return FFMAX(0, duration); } +enum AVSubtitleType av_get_subtitle_format_from_codecdesc(const AVCodecDescriptor *codec_descriptor) +{ + if(codec_descriptor->props & AV_CODEC_PROP_BITMAP_SUB) + return AV_SUBTITLE_FMT_BITMAP; + + if(codec_descriptor->props & AV_CODEC_PROP_TEXT_SUB) + return AV_SUBTITLE_FMT_ASS; + + return AV_SUBTITLE_FMT_UNKNOWN; +} + int av_get_audio_frame_duration2(AVCodecParameters *par, int frame_bytes) { int duration = get_audio_frame_duration(par->codec_id, par->sample_rate, diff --git a/libavcodec/webvttdec.c b/libavcodec/webvttdec.c index 0093f328fa..23b0c5245c 100644 --- a/libavcodec/webvttdec.c +++ b/libavcodec/webvttdec.c @@ -84,7 +84,7 @@ static int webvtt_decode_frame(AVCodecContext *avctx, void *data, int *got_sub_ptr, AVPacket *avpkt) { int ret = 0; - AVSubtitle *sub = data; + AVFrame *sub = data; const char *ptr = avpkt->data; FFASSDecoderContext *s = avctx->priv_data; AVBPrint buf; @@ -95,7 +95,7 @@ static int webvtt_decode_frame(AVCodecContext *avctx, av_bprint_finalize(&buf, NULL); if (ret < 0) return ret; - *got_sub_ptr = sub->num_rects > 0; + *got_sub_ptr = sub->num_subtitle_rects > 0; return avpkt->size; } diff --git a/libavcodec/webvttenc.c b/libavcodec/webvttenc.c index 89b49e42bf..287a246156 100644 --- a/libavcodec/webvttenc.c +++ b/libavcodec/webvttenc.c @@ -156,7 +156,7 @@ static const ASSCodesCallbacks webvtt_callbacks = { }; static int webvtt_encode_frame(AVCodecContext *avctx, - unsigned char *buf, int bufsize, const AVSubtitle *sub) + unsigned char *buf, int bufsize, const AVFrame *sub) { WebVTTContext *s = avctx->priv_data; ASSDialog *dialog; @@ -164,11 +164,11 @@ static int webvtt_encode_frame(AVCodecContext *avctx, av_bprint_clear(&s->buffer); - for (i=0; inum_rects; i++) { - const char *ass = sub->rects[i]->ass; + for (i=0; inum_subtitle_rects; i++) { + const char *ass = sub->subtitle_rects[i]->ass; - if (sub->rects[i]->type != SUBTITLE_ASS) { - av_log(avctx, AV_LOG_ERROR, "Only SUBTITLE_ASS type supported.\n"); + if (sub->subtitle_rects[i]->type != AV_SUBTITLE_FMT_ASS) { + av_log(avctx, AV_LOG_ERROR, "Only AV_SUBTITLE_FMT_ASS type supported.\n"); return AVERROR(EINVAL); } diff --git a/libavcodec/xsubdec.c b/libavcodec/xsubdec.c index 979399bae6..f45edcc0ca 100644 --- a/libavcodec/xsubdec.c +++ b/libavcodec/xsubdec.c @@ -51,7 +51,7 @@ static int decode_frame(AVCodecContext *avctx, void *data, int *got_sub_ptr, AVPacket *avpkt) { const uint8_t *buf = avpkt->data; int buf_size = avpkt->size; - AVSubtitle *sub = data; + AVFrame *sub = data; const uint8_t *buf_end = buf + buf_size; uint8_t *bitmap; int w, h, x, y, i, ret; @@ -72,8 +72,8 @@ static int decode_frame(AVCodecContext *avctx, void *data, int *got_sub_ptr, } if (avpkt->pts != AV_NOPTS_VALUE) packet_time = av_rescale_q(avpkt->pts, AV_TIME_BASE_Q, (AVRational){1, 1000}); - sub->start_display_time = parse_timecode(buf + 1, packet_time); - sub->end_display_time = parse_timecode(buf + 14, packet_time); + sub->subtitle_start_time = parse_timecode(buf + 1, packet_time); + sub->subtitle_end_time = parse_timecode(buf + 14, packet_time); buf += 27; // read header @@ -96,51 +96,51 @@ static int decode_frame(AVCodecContext *avctx, void *data, int *got_sub_ptr, return AVERROR_INVALIDDATA; // allocate sub and set values - sub->rects = av_mallocz(sizeof(*sub->rects)); - if (!sub->rects) + sub->subtitle_rects = av_mallocz(sizeof(*sub->subtitle_rects)); + if (!sub->subtitle_rects) return AVERROR(ENOMEM); - sub->rects[0] = av_mallocz(sizeof(*sub->rects[0])); - if (!sub->rects[0]) { - av_freep(&sub->rects); + sub->subtitle_rects[0] = av_mallocz(sizeof(*sub->subtitle_rects[0])); + if (!sub->subtitle_rects[0]) { + av_freep(&sub->subtitle_rects); return AVERROR(ENOMEM); } - sub->rects[0]->x = x; sub->rects[0]->y = y; - sub->rects[0]->w = w; sub->rects[0]->h = h; - sub->rects[0]->type = SUBTITLE_BITMAP; - sub->rects[0]->linesize[0] = w; - sub->rects[0]->data[0] = av_malloc(w * h); - sub->rects[0]->nb_colors = 4; - sub->rects[0]->data[1] = av_mallocz(AVPALETTE_SIZE); - if (!sub->rects[0]->data[0] || !sub->rects[0]->data[1]) { - av_freep(&sub->rects[0]->data[1]); - av_freep(&sub->rects[0]->data[0]); - av_freep(&sub->rects[0]); - av_freep(&sub->rects); + sub->subtitle_rects[0]->x = x; sub->subtitle_rects[0]->y = y; + sub->subtitle_rects[0]->w = w; sub->subtitle_rects[0]->h = h; + sub->subtitle_rects[0]->type = AV_SUBTITLE_FMT_BITMAP; + sub->subtitle_rects[0]->linesize[0] = w; + sub->subtitle_rects[0]->data[0] = av_malloc(w * h); + sub->subtitle_rects[0]->nb_colors = 4; + sub->subtitle_rects[0]->data[1] = av_mallocz(AVPALETTE_SIZE); + if (!sub->subtitle_rects[0]->data[0] || !sub->subtitle_rects[0]->data[1]) { + av_freep(&sub->subtitle_rects[0]->data[1]); + av_freep(&sub->subtitle_rects[0]->data[0]); + av_freep(&sub->subtitle_rects[0]); + av_freep(&sub->subtitle_rects); return AVERROR(ENOMEM); } - sub->num_rects = 1; + sub->num_subtitle_rects = 1; // read palette - for (i = 0; i < sub->rects[0]->nb_colors; i++) - ((uint32_t*)sub->rects[0]->data[1])[i] = bytestream_get_be24(&buf); + for (i = 0; i < sub->subtitle_rects[0]->nb_colors; i++) + ((uint32_t*)sub->subtitle_rects[0]->data[1])[i] = bytestream_get_be24(&buf); if (!has_alpha) { // make all except background (first entry) non-transparent - for (i = 1; i < sub->rects[0]->nb_colors; i++) - ((uint32_t *)sub->rects[0]->data[1])[i] |= 0xff000000; + for (i = 1; i < sub->subtitle_rects[0]->nb_colors; i++) + ((uint32_t *)sub->subtitle_rects[0]->data[1])[i] |= 0xff000000; } else { - for (i = 0; i < sub->rects[0]->nb_colors; i++) - ((uint32_t *)sub->rects[0]->data[1])[i] |= (unsigned)*buf++ << 24; + for (i = 0; i < sub->subtitle_rects[0]->nb_colors; i++) + ((uint32_t *)sub->subtitle_rects[0]->data[1])[i] |= (unsigned)*buf++ << 24; } // process RLE-compressed data if ((ret = init_get_bits8(&gb, buf, buf_end - buf)) < 0) return ret; - bitmap = sub->rects[0]->data[0]; + bitmap = sub->subtitle_rects[0]->data[0]; for (y = 0; y < h; y++) { // interlaced: do odd lines - if (y == (h + 1) / 2) bitmap = sub->rects[0]->data[0] + w; + if (y == (h + 1) / 2) bitmap = sub->subtitle_rects[0]->data[0] + w; for (x = 0; x < w; ) { int log2 = ff_log2_tab[show_bits(&gb, 8)]; int run = get_bits(&gb, 14 - 4 * (log2 >> 1)); diff --git a/libavcodec/xsubenc.c b/libavcodec/xsubenc.c index 03d0dc2d86..6508af9a73 100644 --- a/libavcodec/xsubenc.c +++ b/libavcodec/xsubenc.c @@ -112,10 +112,10 @@ static int make_tc(uint64_t ms, int *tc) } static int xsub_encode(AVCodecContext *avctx, unsigned char *buf, - int bufsize, const AVSubtitle *h) + int bufsize, const AVFrame *h) { - uint64_t startTime = h->pts / 1000; // FIXME: need better solution... - uint64_t endTime = startTime + h->end_display_time - h->start_display_time; + uint64_t startTime = h->subtitle_pts / 1000; // FIXME: need better solution... + uint64_t endTime = startTime + h->subtitle_end_time - h->subtitle_start_time; int start_tc[4], end_tc[4]; uint8_t *hdr = buf + 27; // Point behind the timestamp uint8_t *rlelenptr; @@ -129,21 +129,21 @@ static int xsub_encode(AVCodecContext *avctx, unsigned char *buf, } // TODO: support multiple rects - if (h->num_rects != 1) - av_log(avctx, AV_LOG_WARNING, "Only single rects supported (%d in subtitle.)\n", h->num_rects); + if (h->num_subtitle_rects != 1) + av_log(avctx, AV_LOG_WARNING, "Only single rects supported (%d in subtitle.)\n", h->num_subtitle_rects); // TODO: render text-based subtitles into bitmaps - if (!h->rects[0]->data[0] || !h->rects[0]->data[1]) { + if (!h->subtitle_rects[0]->data[0] || !h->subtitle_rects[0]->data[1]) { av_log(avctx, AV_LOG_WARNING, "No subtitle bitmap available.\n"); return AVERROR(EINVAL); } // TODO: color reduction, similar to dvdsub encoder - if (h->rects[0]->nb_colors > 4) - av_log(avctx, AV_LOG_WARNING, "No more than 4 subtitle colors supported (%d found.)\n", h->rects[0]->nb_colors); + if (h->subtitle_rects[0]->nb_colors > 4) + av_log(avctx, AV_LOG_WARNING, "No more than 4 subtitle colors supported (%d found.)\n", h->subtitle_rects[0]->nb_colors); // TODO: Palette swapping if color zero is not transparent - if (((uint32_t *)h->rects[0]->data[1])[0] & 0xff000000) + if (((uint32_t *)h->subtitle_rects[0]->data[1])[0] & 0xff000000) av_log(avctx, AV_LOG_WARNING, "Color index 0 is not transparent. Transparency will be messed up.\n"); if (make_tc(startTime, start_tc) || make_tc(endTime, end_tc)) { @@ -160,40 +160,40 @@ static int xsub_encode(AVCodecContext *avctx, unsigned char *buf, // 2 pixels required on either side of subtitle. // Possibly due to limitations of hardware renderers. // TODO: check if the bitmap is already padded - width = FFALIGN(h->rects[0]->w, 2) + PADDING * 2; - height = FFALIGN(h->rects[0]->h, 2); + width = FFALIGN(h->subtitle_rects[0]->w, 2) + PADDING * 2; + height = FFALIGN(h->subtitle_rects[0]->h, 2); bytestream_put_le16(&hdr, width); bytestream_put_le16(&hdr, height); - bytestream_put_le16(&hdr, h->rects[0]->x); - bytestream_put_le16(&hdr, h->rects[0]->y); - bytestream_put_le16(&hdr, h->rects[0]->x + width -1); - bytestream_put_le16(&hdr, h->rects[0]->y + height -1); + bytestream_put_le16(&hdr, h->subtitle_rects[0]->x); + bytestream_put_le16(&hdr, h->subtitle_rects[0]->y); + bytestream_put_le16(&hdr, h->subtitle_rects[0]->x + width -1); + bytestream_put_le16(&hdr, h->subtitle_rects[0]->y + height -1); rlelenptr = hdr; // Will store length of first field here later. hdr+=2; // Palette for (i=0; i<4; i++) - bytestream_put_be24(&hdr, ((uint32_t *)h->rects[0]->data[1])[i]); + bytestream_put_be24(&hdr, ((uint32_t *)h->subtitle_rects[0]->data[1])[i]); // Bitmap // RLE buffer. Reserve 2 bytes for possible padding after the last row. init_put_bits(&pb, hdr, bufsize - (hdr - buf) - 2); - if (xsub_encode_rle(&pb, h->rects[0]->data[0], - h->rects[0]->linesize[0] * 2, - h->rects[0]->w, (h->rects[0]->h + 1) >> 1)) + if (xsub_encode_rle(&pb, h->subtitle_rects[0]->data[0], + h->subtitle_rects[0]->linesize[0] * 2, + h->subtitle_rects[0]->w, (h->subtitle_rects[0]->h + 1) >> 1)) return AVERROR_BUFFER_TOO_SMALL; bytestream_put_le16(&rlelenptr, put_bytes_count(&pb, 0)); // Length of first field - if (xsub_encode_rle(&pb, h->rects[0]->data[0] + h->rects[0]->linesize[0], - h->rects[0]->linesize[0] * 2, - h->rects[0]->w, h->rects[0]->h >> 1)) + if (xsub_encode_rle(&pb, h->subtitle_rects[0]->data[0] + h->subtitle_rects[0]->linesize[0], + h->subtitle_rects[0]->linesize[0] * 2, + h->subtitle_rects[0]->w, h->subtitle_rects[0]->h >> 1)) return AVERROR_BUFFER_TOO_SMALL; // Enforce total height to be a multiple of 2 - if (h->rects[0]->h & 1) { - put_xsub_rle(&pb, h->rects[0]->w, PADDING_COLOR); + if (h->subtitle_rects[0]->h & 1) { + put_xsub_rle(&pb, h->subtitle_rects[0]->w, PADDING_COLOR); } flush_put_bits(&pb); diff --git a/libavfilter/vf_subtitles.c b/libavfilter/vf_subtitles.c index d0bafcd3cf..c536854c45 100644 --- a/libavfilter/vf_subtitles.c +++ b/libavfilter/vf_subtitles.c @@ -35,14 +35,12 @@ # include "libavformat/avformat.h" #endif #include "libavutil/avstring.h" -#include "libavutil/imgutils.h" #include "libavutil/opt.h" #include "libavutil/parseutils.h" #include "drawutils.h" #include "avfilter.h" #include "internal.h" #include "formats.h" -#include "video.h" typedef struct AssContext { const AVClass *class; @@ -306,6 +304,7 @@ static av_cold int init_subtitles(AVFilterContext *ctx) AVStream *st; AVPacket pkt; AssContext *ass = ctx->priv; + enum AVSubtitleType subtitle_format; /* Init libass */ ret = init(ctx); @@ -386,16 +385,19 @@ static av_cold int init_subtitles(AVFilterContext *ctx) ret = AVERROR_DECODER_NOT_FOUND; goto end; } + dec_desc = avcodec_descriptor_get(st->codecpar->codec_id); - if (dec_desc && !(dec_desc->props & AV_CODEC_PROP_TEXT_SUB)) { + subtitle_format = av_get_subtitle_format_from_codecdesc(dec_desc); + + if (subtitle_format != AV_SUBTITLE_FMT_ASS) { av_log(ctx, AV_LOG_ERROR, - "Only text based subtitles are currently supported\n"); - ret = AVERROR_PATCHWELCOME; + "Only text based subtitles are supported by this filter\n"); + ret = AVERROR_INVALIDDATA; goto end; } + if (ass->charenc) av_dict_set(&codec_opts, "sub_charenc", ass->charenc, 0); - av_dict_set(&codec_opts, "sub_text_format", "ass", 0); dec_ctx = avcodec_alloc_context3(dec); if (!dec_ctx) { @@ -449,18 +451,18 @@ static av_cold int init_subtitles(AVFilterContext *ctx) dec_ctx->subtitle_header_size); while (av_read_frame(fmt, &pkt) >= 0) { int i, got_subtitle; - AVSubtitle sub = {0}; + AVFrame *sub = av_frame_alloc(); if (pkt.stream_index == sid) { - ret = avcodec_decode_subtitle2(dec_ctx, &sub, &got_subtitle, &pkt); + ret = avcodec_decode_subtitle2(dec_ctx, sub, &got_subtitle, &pkt); if (ret < 0) { av_log(ctx, AV_LOG_WARNING, "Error decoding: %s (ignored)\n", av_err2str(ret)); } else if (got_subtitle) { - const int64_t start_time = av_rescale_q(sub.pts, AV_TIME_BASE_Q, av_make_q(1, 1000)); - const int64_t duration = sub.end_display_time; - for (i = 0; i < sub.num_rects; i++) { - char *ass_line = sub.rects[i]->ass; + const int64_t start_time = av_rescale_q(sub->subtitle_pts, AV_TIME_BASE_Q, av_make_q(1, 1000)); + const int64_t duration = sub->subtitle_end_time; + for (i = 0; i < sub->num_subtitle_rects; i++) { + char *ass_line = sub->subtitle_rects[i]->ass; if (!ass_line) break; ass_process_chunk(ass->track, ass_line, strlen(ass_line), @@ -469,7 +471,7 @@ static av_cold int init_subtitles(AVFilterContext *ctx) } } av_packet_unref(&pkt); - avsubtitle_free(&sub); + av_frame_free(&sub); } end: diff --git a/libavformat/utils.c b/libavformat/utils.c index b56190d2da..f83dadd477 100644 --- a/libavformat/utils.c +++ b/libavformat/utils.c @@ -2949,7 +2949,6 @@ static int try_decode_frame(AVFormatContext *s, AVStream *st, const AVCodec *codec; int got_picture = 1, ret = 0; AVFrame *frame = av_frame_alloc(); - AVSubtitle subtitle; AVPacket pkt = *avpkt; int do_skip_frame = 0; enum AVDiscard skip_frame; @@ -3020,10 +3019,8 @@ static int try_decode_frame(AVFormatContext *s, AVStream *st, if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) ret = 0; } else if (avctx->codec_type == AVMEDIA_TYPE_SUBTITLE) { - ret = avcodec_decode_subtitle2(avctx, &subtitle, + ret = avcodec_decode_subtitle2(avctx, frame, &got_picture, &pkt); - if (got_picture) - avsubtitle_free(&subtitle); if (ret >= 0) pkt.size = 0; } diff --git a/libavutil/Makefile b/libavutil/Makefile index 410ac636f7..04e1101bf3 100644 --- a/libavutil/Makefile +++ b/libavutil/Makefile @@ -74,6 +74,7 @@ HEADERS = adler32.h \ sha512.h \ spherical.h \ stereo3d.h \ + subfmt.h \ threadmessage.h \ time.h \ timecode.h \ @@ -159,6 +160,7 @@ OBJS = adler32.o \ slicethread.o \ spherical.o \ stereo3d.o \ + subfmt.o \ threadmessage.o \ time.o \ timecode.o \ diff --git a/libavutil/frame.c b/libavutil/frame.c index ef2867d318..1855de9b3d 100644 --- a/libavutil/frame.c +++ b/libavutil/frame.c @@ -72,7 +72,12 @@ static void get_frame_defaults(AVFrame *frame) frame->colorspace = AVCOL_SPC_UNSPECIFIED; frame->color_range = AVCOL_RANGE_UNSPECIFIED; frame->chroma_location = AVCHROMA_LOC_UNSPECIFIED; - frame->flags = 0; + frame->subtitle_start_time = 0; + frame->subtitle_end_time = 0; + frame->num_subtitle_rects = 0; + frame->subtitle_rects = NULL; + frame->subtitle_pts = 0; + frame->subtitle_header = NULL; } static void free_side_data(AVFrameSideData **ptr_sd) @@ -306,6 +311,9 @@ static int frame_copy_props(AVFrame *dst, const AVFrame *src, int force_copy) dst->colorspace = src->colorspace; dst->color_range = src->color_range; dst->chroma_location = src->chroma_location; + dst->subtitle_start_time = src->subtitle_start_time; + dst->subtitle_end_time = src->subtitle_end_time; + dst->subtitle_pts = src->subtitle_pts; av_dict_copy(&dst->metadata, src->metadata, 0); @@ -359,6 +367,19 @@ int av_frame_ref(AVFrame *dst, const AVFrame *src) if (ret < 0) goto fail; + /* duplicate subtitle rects */ + dst->num_subtitle_rects = src->num_subtitle_rects; + + if (src->num_subtitle_rects) { + dst->subtitle_rects = av_malloc_array(src->num_subtitle_rects, sizeof(AVSubtitleRect *)); + + for (i = 0; i < src->num_subtitle_rects; i++) { + AVSubtitleRect *rect = src->subtitle_rects[i]; + rect->nb_refs++; + dst->subtitle_rects[i] = rect; + } + } + /* duplicate the frame data if it's not refcounted */ if (!src->buf[0]) { ret = av_frame_get_buffer2(dst, 0); @@ -472,6 +493,24 @@ void av_frame_unref(AVFrame *frame) av_buffer_unref(&frame->opaque_ref); av_buffer_unref(&frame->private_ref); + for (i = 0; i < frame->num_subtitle_rects; i++) { + AVSubtitleRect *rect = frame->subtitle_rects[i]; + + if (rect && rect->nb_refs > 0) + rect->nb_refs--; + else { + av_freep(&rect->data[0]); + av_freep(&rect->data[1]); + av_freep(&rect->data[2]); + av_freep(&rect->data[3]); + av_freep(&rect->text); + av_freep(&rect->ass); + av_freep(&rect); + } + } + + av_freep(&frame->subtitle_rects); + get_frame_defaults(frame); } diff --git a/libavutil/frame.h b/libavutil/frame.h index 005eed9d18..4e2cfe19b3 100644 --- a/libavutil/frame.h +++ b/libavutil/frame.h @@ -34,6 +34,7 @@ #include "rational.h" #include "samplefmt.h" #include "pixfmt.h" +#include "subfmt.h" #include "version.h" @@ -469,6 +470,17 @@ typedef struct AVFrame { */ uint64_t channel_layout; + uint32_t subtitle_start_time; /* display start time, relative to packet pts, in ms */ + uint32_t subtitle_end_time; /* display end time, relative to packet pts, in ms */ + unsigned num_subtitle_rects; + AVSubtitleRect **subtitle_rects; + int64_t subtitle_pts; ///< Same as packet pts, in AV_TIME_BASE + + /** + * Header containing style information for text subtitles. + */ + char *subtitle_header; + /** * AVBuffer references backing the data for this frame. If all elements of * this array are NULL, then this frame is not reference counted. This array diff --git a/libavutil/subfmt.c b/libavutil/subfmt.c new file mode 100644 index 0000000000..9a19d01aa3 --- /dev/null +++ b/libavutil/subfmt.c @@ -0,0 +1,52 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include +#include "common.h" +#include "subfmt.h" + +typedef struct SubtitleFmtInfo { + enum AVSubtitleType fmt; + char* name; +} SubtitleFmtInfo; + +static SubtitleFmtInfo sub_fmt_info[AV_SUBTITLE_FMT_NB] = { + {.fmt = AV_SUBTITLE_FMT_UNKNOWN, .name = "Unknown subtitle format", }, + {.fmt = AV_SUBTITLE_FMT_BITMAP, .name = "Graphical subtitles", }, + {.fmt = AV_SUBTITLE_FMT_TEXT, .name = "Text subtitles (plain)", }, + {.fmt = AV_SUBTITLE_FMT_ASS, .name = "Text subtitles (ass)", }, +}; + +const char *av_get_subtitle_fmt_name(enum AVSubtitleType sub_fmt) +{ + if (sub_fmt < 0 || sub_fmt >= AV_SUBTITLE_FMT_NB) + return NULL; + return sub_fmt_info[sub_fmt].name; +} + +enum AVSubtitleType av_get_subtitle_fmt(const char *name) +{ + int i; + + for (i = 0; i < AV_SUBTITLE_FMT_NB; i++) + if (!strcmp(sub_fmt_info[i].name, name)) + return i; + return AV_SUBTITLE_FMT_NONE; +} diff --git a/libavutil/subfmt.h b/libavutil/subfmt.h new file mode 100644 index 0000000000..31dabc957c --- /dev/null +++ b/libavutil/subfmt.h @@ -0,0 +1,94 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVUTIL_SUBFMT_H +#define AVUTIL_SUBFMT_H + +#include + +enum AVSubtitleType { + + /** + * Subtitle format unknown. + */ + AV_SUBTITLE_FMT_NONE = -1, + + /** + * Subtitle format unknown. + */ + AV_SUBTITLE_FMT_UNKNOWN = 0, + + /** + * Bitmap area in AVSubtitleRect.data, pixfmt AV_PIX_FMT_PAL8. + */ + AV_SUBTITLE_FMT_BITMAP = 1, + + /** + * Plain text in AVSubtitleRect.text. + */ + AV_SUBTITLE_FMT_TEXT = 2, + + /** + * Text Formatted as per ASS specification, contained AVSubtitleRect.ass. + */ + AV_SUBTITLE_FMT_ASS = 3, + + AV_SUBTITLE_FMT_NB, +}; + +typedef struct AVSubtitleRect { + unsigned nb_refs; + int x; ///< top left corner of pict, undefined when pict is not set + int y; ///< top left corner of pict, undefined when pict is not set + int w; ///< width of pict, undefined when pict is not set + int h; ///< height of pict, undefined when pict is not set + int nb_colors; ///< number of colors in pict, undefined when pict is not set + + /** + * data+linesize for the bitmap of this subtitle. + */ + uint8_t *data[4]; + int linesize[4]; + + enum AVSubtitleType type; + + char *text; ///< 0 terminated plain UTF-8 text + + /** + * 0-terminated ASS/SSA compatible event line. + */ + char *ass; + + int flags; +} AVSubtitleRect; + +/** + * Return the name of sub_fmt, or NULL if sub_fmt is not + * recognized. + */ +const char *av_get_subtitle_fmt_name(enum AVSubtitleType sub_fmt); + +/** + * Return a subtitle format corresponding to name, or AV_SUBTITLE_FMT_NONE + * on error. + */ +enum AVSubtitleType av_get_subtitle_fmt(const char *name); + +#endif /* AVUTIL_SUBFMT_H */ From patchwork Sun Sep 12 03:21:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 30170 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp2862967iov; Sat, 11 Sep 2021 20:22:20 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw3cqaIEBmuRM6O1zWlHsNKNXaQxb0GpkbhVRDL/fWtP7xCZIPPp6I3S/rtwOcXi/L3oL1q X-Received: by 2002:a05:6402:2810:: with SMTP id h16mr6051176ede.133.1631416940125; Sat, 11 Sep 2021 20:22:20 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id i23si3386669edv.93.2021.09.11.20.22.19; Sat, 11 Sep 2021 20:22:20 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=jI4Im4gj; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6AD7068A990; Sun, 12 Sep 2021 06:21:55 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10olkn2062.outbound.protection.outlook.com [40.92.41.62]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C61C168A927 for ; Sun, 12 Sep 2021 06:21:48 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cT6K3N2HVfs0Zy2ZwOvATBu065kGw18ypHcg8OZyTqA5qr+lrJR9uMUo9xui8d5Bhlglhaxzh+AT58hABMlzu5GdM/4JtAL/EQhVjveJcs7+PYmi3sYalIr4cgwsgSTy6OIQ5eM4eN0Izo1Tmip/G4kcfaSDCYJFgsdK+DWzr6IpvHWdSJ9MZvOItX6XFRZsAqfgTloVFyUN7zjddTcO3fTp7EW7kXCKwQCEcvC4gkabSsViGhvVkckPSmYoIbQanli+8+CBo4KZwpQwM5nuHBLTFuGYz9YheSkollwuR1RtW6p9HMLymi21vWnARPO9AvkMbr7a3EQ8FImeThTL6Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=CTnr2xAACyWYuhVgVKxLbRv+CTzIHqRhTayCAOK/PiM=; b=Bb58Fr/Hs2oWWIIyE1K3iyFtyQ7R6p1RrYQt0y4xkBJbJDBJIFd3cqu1J+AJtlrdjgUbv5TE4FbZCoqSo18wcFdWgqzIOMn+YlVb+otrP7oyira5oM3TS9o/e810/sqk5LnULFS9Ebwvm1gUXbRVR/BwVbPede8nmMbvlZoGtgGi9rYkMpqF9fnMdeAxS8vk9zSRY3aMDKcw7Fu0mneceLdse0VXppVd8ZRNFgHrMl/jZvhGj85nqivcILX9+kGrIg7imiVDljrTMjfeZ4zJiVYy+8fQGl7F1f7owneFfTp0q9rPrb2x3SqW5dpCdCFhqTEw2uejbabN3p55zkVOxw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CTnr2xAACyWYuhVgVKxLbRv+CTzIHqRhTayCAOK/PiM=; b=jI4Im4gj5yHifH+UvgtoN1KFReBl/dmVaKyTHdniRU+3HgMlCG9sU4jUPMhPVpr1jrVVuW/whTxhUmMXVQ6dOZUP4aaG01UT/ZcbrBRLKeO1gISjsBZ2XNkDHXnfbQAAdtGCxIpjgsVWXnqXqbOCywbWKYH0yj3mp3zHQfA82apkGFXb/H4IxwWH1EqVGOWcmSRKrVd4tizq0D/PweNB1E7I46bUWkcq/aj/RvgepZ2YEzq7myBAkNwL2f87Xq2WIfrWz0gphVMz5r+TDzh3AMthywcQToKSmPR6vcPtWmXtqYob7V5w1lv7Guk80Jqp9WPprZD2LJsb/B+XQLJ6pg== Received: from MN2PR04MB5981.namprd04.prod.outlook.com (2603:10b6:208:da::10) by MN2PR04MB5728.namprd04.prod.outlook.com (2603:10b6:208:3d::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Sun, 12 Sep 2021 03:21:40 +0000 Received: from MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb]) by MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb%5]) with mapi id 15.20.4500.018; Sun, 12 Sep 2021 03:21:40 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v5 03/12] fftools/play, probe: Adjust for subtitle format type change Thread-Index: AQHXp4VNWN7hl5mljk2QvECa/KqKaA== Date: Sun, 12 Sep 2021 03:21:40 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [ZtFrtEOtlGU1EXSA0IxCskSYgBh7+a67939tK/XsTj8=] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 4f8127b6-831d-4843-dbe3-08d9759c6fec x-ms-traffictypediagnostic: MN2PR04MB5728: x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: JjxTq/PmnitAYpAfK/aQMZy7X57DHgeGdZ0oe3yJTNNk2MeRsS1hSsBEHeFOjDtY1lpt3HpFzc3xAURSK9n7LJd7tA2BhRT8AEqC23mctnX+PtpNHUqXquoszjKiXAuuKqPflll0GpaCrr0xEr4ATQosogMQ/KcNrqVvJ2+meyglYC1Svz93/DjVgkF3HeEnMomIPlz+rI3+1WFKMHHDvn+et4Oxyovc5CjMz2O5ONwSl5Cxk0Hz5qMAqdQCirb1rmjI0p65YUkj/nF6EGy2CIoFIJMrDiu8X9nolGb6gpilvjM/M3hEYs2p9y4PC6hSYvrPVKuT4maqGTkCNzBete6OyBSJiwovY3kynV9/kZcSGOJhHTXwjTlc9m9XtSIDog3aSSNAC+6HygLv1tzZPD+E1KFwska5I2LoFrN9eOWFq0OqKxFbPz4/Ed+q/tyW x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: o1+itOcmrfDJ6ARVw5W8fwpIjlDl8LyBvTAu96xUVwznu2lIVV8ueobWJYwej4aULYHlZzkDNV7sazuKMLJlfhdgSuH027tPRX40wOo0srhssLcIjBHeEzNQQNUHaRRGqvPej+AUPCAcUEHEOQXjGQ== x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-3174-20-msonline-outlook-529c7.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR04MB5981.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 4f8127b6-831d-4843-dbe3-08d9759c6fec X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Sep 2021 03:21:40.2207 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR04MB5728 Subject: [FFmpeg-devel] [PATCH v5 03/12] fftools/play, probe: Adjust for subtitle format type change X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: mewjOoy6NphN Signed-off-by: softworkz --- fftools/ffplay.c | 30 ++++++++++++++--------------- fftools/ffprobe.c | 49 +++++++++++++++++++++++++++++++++++------------ 2 files changed, 51 insertions(+), 28 deletions(-) diff --git a/fftools/ffplay.c b/fftools/ffplay.c index 46758b9f55..b1cb44e804 100644 --- a/fftools/ffplay.c +++ b/fftools/ffplay.c @@ -152,7 +152,6 @@ typedef struct Clock { /* Common struct for handling all types of decoded data and allocated render buffers. */ typedef struct Frame { AVFrame *frame; - AVSubtitle sub; int serial; double pts; /* presentation timestamp for the frame */ double duration; /* estimated duration of the frame */ @@ -586,7 +585,7 @@ static int decoder_init(Decoder *d, AVCodecContext *avctx, PacketQueue *queue, S return 0; } -static int decoder_decode_frame(Decoder *d, AVFrame *frame, AVSubtitle *sub) { +static int decoder_decode_frame(Decoder *d, AVFrame *frame, AVFrame *sub) { int ret = AVERROR(EAGAIN); for (;;) { @@ -683,7 +682,6 @@ static void decoder_destroy(Decoder *d) { static void frame_queue_unref_item(Frame *vp) { av_frame_unref(vp->frame); - avsubtitle_free(&vp->sub); } static int frame_queue_init(FrameQueue *f, PacketQueue *pktq, int max_size, int keep_last) @@ -981,7 +979,7 @@ static void video_image_display(VideoState *is) if (frame_queue_nb_remaining(&is->subpq) > 0) { sp = frame_queue_peek(&is->subpq); - if (vp->pts >= sp->pts + ((float) sp->sub.start_display_time / 1000)) { + if (vp->pts >= sp->pts + ((float) sp->frame->subtitle_start_time / 1000)) { if (!sp->uploaded) { uint8_t* pixels[4]; int pitch[4]; @@ -993,8 +991,8 @@ static void video_image_display(VideoState *is) if (realloc_texture(&is->sub_texture, SDL_PIXELFORMAT_ARGB8888, sp->width, sp->height, SDL_BLENDMODE_BLEND, 1) < 0) return; - for (i = 0; i < sp->sub.num_rects; i++) { - AVSubtitleRect *sub_rect = sp->sub.rects[i]; + for (i = 0; i < sp->frame->num_subtitle_rects; i++) { + AVSubtitleRect *sub_rect = sp->frame->subtitle_rects[i]; sub_rect->x = av_clip(sub_rect->x, 0, sp->width ); sub_rect->y = av_clip(sub_rect->y, 0, sp->height); @@ -1041,7 +1039,7 @@ static void video_image_display(VideoState *is) int i; double xratio = (double)rect.w / (double)sp->width; double yratio = (double)rect.h / (double)sp->height; - for (i = 0; i < sp->sub.num_rects; i++) { + for (i = 0; i < sp->sub.num_subtitle_rects; i++) { SDL_Rect *sub_rect = (SDL_Rect*)sp->sub.rects[i]; SDL_Rect target = {.x = rect.x + sub_rect->x * xratio, .y = rect.y + sub_rect->y * yratio, @@ -1651,13 +1649,13 @@ retry: sp2 = NULL; if (sp->serial != is->subtitleq.serial - || (is->vidclk.pts > (sp->pts + ((float) sp->sub.end_display_time / 1000))) - || (sp2 && is->vidclk.pts > (sp2->pts + ((float) sp2->sub.start_display_time / 1000)))) + || (is->vidclk.pts > (sp->pts + ((float) sp->frame->subtitle_end_time / 1000))) + || (sp2 && is->vidclk.pts > (sp2->pts + ((float) sp2->frame->subtitle_start_time / 1000)))) { if (sp->uploaded) { int i; - for (i = 0; i < sp->sub.num_rects; i++) { - AVSubtitleRect *sub_rect = sp->sub.rects[i]; + for (i = 0; i < sp->frame->num_subtitle_rects; i++) { + AVSubtitleRect *sub_rect = sp->frame->subtitle_rects[i]; uint8_t *pixels; int pitch, j; @@ -2245,14 +2243,14 @@ static int subtitle_thread(void *arg) if (!(sp = frame_queue_peek_writable(&is->subpq))) return 0; - if ((got_subtitle = decoder_decode_frame(&is->subdec, NULL, &sp->sub)) < 0) + if ((got_subtitle = decoder_decode_frame(&is->subdec, NULL, sp->frame)) < 0) break; pts = 0; - if (got_subtitle && sp->sub.format == 0) { - if (sp->sub.pts != AV_NOPTS_VALUE) - pts = sp->sub.pts / (double)AV_TIME_BASE; + if (got_subtitle && sp->frame->format == AV_SUBTITLE_FMT_BITMAP) { + if (sp->frame->subtitle_pts != AV_NOPTS_VALUE) + pts = sp->frame->subtitle_pts / (double)AV_TIME_BASE; sp->pts = pts; sp->serial = is->subdec.pkt_serial; sp->width = is->subdec.avctx->width; @@ -2262,7 +2260,7 @@ static int subtitle_thread(void *arg) /* now we can update the picture count */ frame_queue_push(&is->subpq); } else if (got_subtitle) { - avsubtitle_free(&sp->sub); + av_frame_free(&sp->frame); } } return 0; diff --git a/fftools/ffprobe.c b/fftools/ffprobe.c index acfec09656..5e899a09db 100644 --- a/fftools/ffprobe.c +++ b/fftools/ffprobe.c @@ -2208,22 +2208,43 @@ static void show_packet(WriterContext *w, InputFile *ifile, AVPacket *pkt, int p fflush(stdout); } -static void show_subtitle(WriterContext *w, AVSubtitle *sub, AVStream *stream, +static void show_subtitle(WriterContext *w, AVFrame *sub, AVStream *stream, AVFormatContext *fmt_ctx) { AVBPrint pbuf; + const char *s; av_bprint_init(&pbuf, 1, AV_BPRINT_SIZE_UNLIMITED); writer_print_section_header(w, SECTION_ID_SUBTITLE); print_str ("media_type", "subtitle"); - print_ts ("pts", sub->pts); - print_time("pts_time", sub->pts, &AV_TIME_BASE_Q); - print_int ("format", sub->format); - print_int ("start_display_time", sub->start_display_time); - print_int ("end_display_time", sub->end_display_time); - print_int ("num_rects", sub->num_rects); + print_ts ("pts", sub->subtitle_pts); + print_time("pts_time", sub->subtitle_pts, &AV_TIME_BASE_Q); + + // Remain compatible with previous outputs? + switch (sub->format) { + case AV_SUBTITLE_FMT_BITMAP: + print_int ("format", 0); + break; + case AV_SUBTITLE_FMT_TEXT: + print_int ("format", 1); + break; + case AV_SUBTITLE_FMT_ASS: + print_int ("format", 1); + break; + default: + print_int ("format", -1); + break; + } + + s = av_get_subtitle_fmt_name(sub->format); + if (s) print_str ("format_str", s); + else print_str_opt("format_str", "unknown"); + + print_int ("start_display_time", sub->subtitle_start_time); + print_int ("end_display_time", sub->subtitle_end_time); + print_int ("num_subtitle_rects", sub->num_subtitle_rects); writer_print_section_footer(w); @@ -2388,7 +2409,7 @@ static av_always_inline int process_frame(WriterContext *w, AVFormatContext *fmt_ctx = ifile->fmt_ctx; AVCodecContext *dec_ctx = ifile->streams[pkt->stream_index].dec_ctx; AVCodecParameters *par = ifile->streams[pkt->stream_index].st->codecpar; - AVSubtitle sub; + AVFrame *sub = NULL; int ret = 0, got_frame = 0; clear_log(1); @@ -2416,8 +2437,12 @@ static av_always_inline int process_frame(WriterContext *w, break; case AVMEDIA_TYPE_SUBTITLE: - if (*packet_new) - ret = avcodec_decode_subtitle2(dec_ctx, &sub, &got_frame, pkt); + if (*packet_new) { + sub = av_frame_alloc(); + if (!sub) + return AVERROR(ENOMEM); + ret = avcodec_decode_subtitle2(dec_ctx, sub, &got_frame, pkt); + } *packet_new = 0; break; default: @@ -2434,11 +2459,11 @@ static av_always_inline int process_frame(WriterContext *w, nb_streams_frames[pkt->stream_index]++; if (do_show_frames) if (is_sub) - show_subtitle(w, &sub, ifile->streams[pkt->stream_index].st, fmt_ctx); + show_subtitle(w, sub, ifile->streams[pkt->stream_index].st, fmt_ctx); else show_frame(w, frame, ifile->streams[pkt->stream_index].st, fmt_ctx); if (is_sub) - avsubtitle_free(&sub); + av_frame_free(&sub); } return got_frame || *packet_new; } From patchwork Sun Sep 12 03:21:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 30172 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp2863048iov; Sat, 11 Sep 2021 20:22:32 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzMgbV6CT2H52DHDlou1vpJoAkGKhc0KWLN/TPVjnVs3PedUjX8JMg+zpJpeWu6FCWW7NUS X-Received: by 2002:a17:906:6547:: with SMTP id u7mr5430804ejn.544.1631416952360; Sat, 11 Sep 2021 20:22:32 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id v12si3210074ejv.679.2021.09.11.20.22.31; Sat, 11 Sep 2021 20:22:32 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=TUpL+z9z; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8862768A9C4; Sun, 12 Sep 2021 06:21:56 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10olkn2062.outbound.protection.outlook.com [40.92.41.62]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 595EA68A477 for ; Sun, 12 Sep 2021 06:21:54 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=EAWxqdzhTMuz2T953bblenUA8jYLkKfjo3bmtkYaJ6EMmThdrXrrCtw3clTofCUUoRck4ACjA896fKwZczSd7ywyGmNwwzH37Gju0nh1LBP4lO4Fi+hhm9PWqb/JmoTfFHOlBZdOF0raEzUtbGsyF+cl4DaLxkz9yKJl6L0tT5XY4QjQkOS2oG6Ab9ADgmdMcwxOwJOK5PFOxSkalL3nFqsSggmzyW4l4Z/1PwVHbDkLd+WByyROc70/3iqv9jaQ+mQNMYBRxc+haskoxWaXG5Qbopxaqxtxjvt1Epr9MxGOUQdUn2usInH9aLtXrHJ5G6i2zdE5OBZ47G8jAAsPaQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=9tdPekbq9BohPeGZ8c3XoikwISayjlKaSRFXmIdHzL0=; b=Pg6h7n/bWGHCzzuQreM3Kg/+DTG589LfxNrkzm0GvDG18ZVvqZUB+x+H+ffM55W1pF1pGV6VE3IqKtNFhVsxiU+hXgbUu93WJp3/auiDqQ6DqhkdooUuUr7UNMHk6qqlkl/8b8iCOU/FpzoJwQ9zihG62SHw63qzWyvjPWomiU8dObUXtLEhUkOvb4IQji77g0uIVor9qzUm0hQZx6wRwJJLslvHBKT1Vey8E1MhMF0VV7njNf7oROmAj5yRlD0+yUAgQz1MUATfgN64pMhbKr2t8wLD+3b47AxhcTLDnZHZfgESjNlTTsZ19Lnbh/EH/SrxRo0LRhMrWHMaBskL2Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9tdPekbq9BohPeGZ8c3XoikwISayjlKaSRFXmIdHzL0=; b=TUpL+z9zul0qjdwrRMZ1R0xTwN+lnvhcqFcMum9qH0IMDC9ajNXYtUm8EdHC30X/OrAmtegoHtLoKfNd/lFUOCoD9RM1o/8ksRiE7bgiJYywfvP8++w2vzteZyuztG8t3sxsKCQybfB1GAcl/mUo8BSDt0dveL1hagWTk8cxiDuFHk0CB8yCiw5Aily8rsLN8TNx3Ewg9ZmzuVCQoVt8Y9w3jSTrM/ecr1bEdwqlXnU2fsS0d1cyc9bi8+TUSRUKyG2xxxOL/X/zk8Y1V1hse0/0W47LS2fP2OrWqNJf+Cp1EAhu43eGQZGcruuqK2uVlX0Bhn404jiPXerXjhJ9LQ== Received: from MN2PR04MB5981.namprd04.prod.outlook.com (2603:10b6:208:da::10) by MN2PR04MB5728.namprd04.prod.outlook.com (2603:10b6:208:3d::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Sun, 12 Sep 2021 03:21:44 +0000 Received: from MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb]) by MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb%5]) with mapi id 15.20.4500.018; Sun, 12 Sep 2021 03:21:44 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v5 04/12] fftools/ffmpeg: Replace sub2video with subtitle frame filtering Thread-Index: AQHXp4VPrmQ40Tn3nkGpH9MXEwW/JQ== Date: Sun, 12 Sep 2021 03:21:43 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [TKTpjR6Vp2JBYba4uMp5sZmbVwWnO/3lr8zeZLKvTME=] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 8748a00f-290c-46b8-2997-08d9759c722d x-ms-traffictypediagnostic: MN2PR04MB5728: x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 5YrV03lItbaRfp+DJt28WV+TCUkQzhOsEYjLdo2XJB7viEUMmJRiUtYvX6BfobgV0bPgT03LT8gW8R/ECPi4+sTJ0m97yk9+A0tUxfHuryD4vX0PtC+IyhQEKoVZtGi6nSFKrsb+SaD92Qm10HdMI7mJnyzdIjRKMK6/2nt1qzd/fYx1gFadGXOwzEeAhXBkM3BgKsf5CUF6eaO5dmR6ZIQXOseJiE1uLPSoLXXH5ht/UU0uOVr44b3K25Y5VHId7EbDmJv82Zwmyh9GBCDrtS8IpG9CG4YUo34NAJOb9ZGl9wDFPFtmeSvYmqCQFebEoo6WI6h2y1JCzJu7YW7LCKjN2oIybarRYiT31Mq6UZac3qy3LR4yMyqXduf4uvu1sNzRFHltRsRHRSVrhzYWN17OIo6adLCfuDpVAIxDygZZ7cKxFbr2H/rbiLuZHym5 x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: m0hOoMj4M5b7eCj4q3uYLGf8BoJZkbUtMv+YegYskOEHRZmc72yKRoazf2leOKSxhOGvoDT103SJQimFDIynWes4wFisw2ntlwc+dK/k3nkflT4Kr8haW8QMQcj5jf4m5v8UwQ7x0WM1+EDsOQSj4Q== x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-3174-20-msonline-outlook-529c7.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR04MB5981.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 8748a00f-290c-46b8-2997-08d9759c722d X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Sep 2021 03:21:43.8707 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR04MB5728 Subject: [FFmpeg-devel] [PATCH v5 04/12] fftools/ffmpeg: Replace sub2video with subtitle frame filtering X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: vmtmn87y5Njn Signed-off-by: softworkz --- fftools/ffmpeg.c | 435 +++++++++------------- fftools/ffmpeg.h | 14 +- fftools/ffmpeg_filter.c | 198 +++++++--- fftools/ffmpeg_hw.c | 2 +- fftools/ffmpeg_opt.c | 3 +- tests/ref/fate/filter-overlay-dvdsub-2397 | 181 +++++---- tests/ref/fate/sub-dvb | 162 ++++---- tests/ref/fate/sub2video | 178 ++++----- tests/ref/fate/sub2video_basic | 93 +++-- tests/ref/fate/sub2video_time_limited | 6 +- 10 files changed, 623 insertions(+), 649 deletions(-) diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index a9bb9d964d..c828388b34 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -169,163 +169,6 @@ static int restore_tty; static void free_input_threads(void); #endif -/* sub2video hack: - Convert subtitles to video with alpha to insert them in filter graphs. - This is a temporary solution until libavfilter gets real subtitles support. - */ - -static int sub2video_get_blank_frame(InputStream *ist) -{ - int ret; - AVFrame *frame = ist->sub2video.frame; - - av_frame_unref(frame); - ist->sub2video.frame->width = ist->dec_ctx->width ? ist->dec_ctx->width : ist->sub2video.w; - ist->sub2video.frame->height = ist->dec_ctx->height ? ist->dec_ctx->height : ist->sub2video.h; - ist->sub2video.frame->format = AV_PIX_FMT_RGB32; - if ((ret = av_frame_get_buffer(frame, 0)) < 0) - return ret; - memset(frame->data[0], 0, frame->height * frame->linesize[0]); - return 0; -} - -static void sub2video_copy_rect(uint8_t *dst, int dst_linesize, int w, int h, - AVSubtitleRect *r) -{ - uint32_t *pal, *dst2; - uint8_t *src, *src2; - int x, y; - - if (r->type != SUBTITLE_BITMAP) { - av_log(NULL, AV_LOG_WARNING, "sub2video: non-bitmap subtitle\n"); - return; - } - if (r->x < 0 || r->x + r->w > w || r->y < 0 || r->y + r->h > h) { - av_log(NULL, AV_LOG_WARNING, "sub2video: rectangle (%d %d %d %d) overflowing %d %d\n", - r->x, r->y, r->w, r->h, w, h - ); - return; - } - - dst += r->y * dst_linesize + r->x * 4; - src = r->data[0]; - pal = (uint32_t *)r->data[1]; - for (y = 0; y < r->h; y++) { - dst2 = (uint32_t *)dst; - src2 = src; - for (x = 0; x < r->w; x++) - *(dst2++) = pal[*(src2++)]; - dst += dst_linesize; - src += r->linesize[0]; - } -} - -static void sub2video_push_ref(InputStream *ist, int64_t pts) -{ - AVFrame *frame = ist->sub2video.frame; - int i; - int ret; - - av_assert1(frame->data[0]); - ist->sub2video.last_pts = frame->pts = pts; - for (i = 0; i < ist->nb_filters; i++) { - ret = av_buffersrc_add_frame_flags(ist->filters[i]->filter, frame, - AV_BUFFERSRC_FLAG_KEEP_REF | - AV_BUFFERSRC_FLAG_PUSH); - if (ret != AVERROR_EOF && ret < 0) - av_log(NULL, AV_LOG_WARNING, "Error while add the frame to buffer source(%s).\n", - av_err2str(ret)); - } -} - -void sub2video_update(InputStream *ist, int64_t heartbeat_pts, AVSubtitle *sub) -{ - AVFrame *frame = ist->sub2video.frame; - int8_t *dst; - int dst_linesize; - int num_rects, i; - int64_t pts, end_pts; - - if (!frame) - return; - if (sub) { - pts = av_rescale_q(sub->pts + sub->start_display_time * 1000LL, - AV_TIME_BASE_Q, ist->st->time_base); - end_pts = av_rescale_q(sub->pts + sub->end_display_time * 1000LL, - AV_TIME_BASE_Q, ist->st->time_base); - num_rects = sub->num_rects; - } else { - /* If we are initializing the system, utilize current heartbeat - PTS as the start time, and show until the following subpicture - is received. Otherwise, utilize the previous subpicture's end time - as the fall-back value. */ - pts = ist->sub2video.initialize ? - heartbeat_pts : ist->sub2video.end_pts; - end_pts = INT64_MAX; - num_rects = 0; - } - if (sub2video_get_blank_frame(ist) < 0) { - av_log(ist->dec_ctx, AV_LOG_ERROR, - "Impossible to get a blank canvas.\n"); - return; - } - dst = frame->data [0]; - dst_linesize = frame->linesize[0]; - for (i = 0; i < num_rects; i++) - sub2video_copy_rect(dst, dst_linesize, frame->width, frame->height, sub->rects[i]); - sub2video_push_ref(ist, pts); - ist->sub2video.end_pts = end_pts; - ist->sub2video.initialize = 0; -} - -static void sub2video_heartbeat(InputStream *ist, int64_t pts) -{ - InputFile *infile = input_files[ist->file_index]; - int i, j, nb_reqs; - int64_t pts2; - - /* When a frame is read from a file, examine all sub2video streams in - the same file and send the sub2video frame again. Otherwise, decoded - video frames could be accumulating in the filter graph while a filter - (possibly overlay) is desperately waiting for a subtitle frame. */ - for (i = 0; i < infile->nb_streams; i++) { - InputStream *ist2 = input_streams[infile->ist_index + i]; - if (!ist2->sub2video.frame) - continue; - /* subtitles seem to be usually muxed ahead of other streams; - if not, subtracting a larger time here is necessary */ - pts2 = av_rescale_q(pts, ist->st->time_base, ist2->st->time_base) - 1; - /* do not send the heartbeat frame if the subtitle is already ahead */ - if (pts2 <= ist2->sub2video.last_pts) - continue; - if (pts2 >= ist2->sub2video.end_pts || ist2->sub2video.initialize) - /* if we have hit the end of the current displayed subpicture, - or if we need to initialize the system, update the - overlayed subpicture and its start/end times */ - sub2video_update(ist2, pts2 + 1, NULL); - for (j = 0, nb_reqs = 0; j < ist2->nb_filters; j++) - nb_reqs += av_buffersrc_get_nb_failed_requests(ist2->filters[j]->filter); - if (nb_reqs) - sub2video_push_ref(ist2, pts2); - } -} - -static void sub2video_flush(InputStream *ist) -{ - int i; - int ret; - - if (ist->sub2video.end_pts < INT64_MAX) - sub2video_update(ist, INT64_MAX, NULL); - for (i = 0; i < ist->nb_filters; i++) { - ret = av_buffersrc_add_frame(ist->filters[i]->filter, NULL); - if (ret != AVERROR_EOF && ret < 0) - av_log(NULL, AV_LOG_WARNING, "Flush the frame error.\n"); - } -} - -/* end of sub2video hack */ - static void term_exit_sigsafe(void) { #if HAVE_TERMIOS_H @@ -526,7 +369,6 @@ static void ffmpeg_cleanup(int ret) avfilter_graph_free(&fg->graph); for (j = 0; j < fg->nb_inputs; j++) { InputFilter *ifilter = fg->inputs[j]; - struct InputStream *ist = ifilter->ist; while (av_fifo_size(ifilter->frame_queue)) { AVFrame *frame; @@ -535,15 +377,6 @@ static void ffmpeg_cleanup(int ret) av_frame_free(&frame); } av_fifo_freep(&ifilter->frame_queue); - if (ist->sub2video.sub_queue) { - while (av_fifo_size(ist->sub2video.sub_queue)) { - AVSubtitle sub; - av_fifo_generic_read(ist->sub2video.sub_queue, - &sub, sizeof(sub), NULL); - avsubtitle_free(&sub); - } - av_fifo_freep(&ist->sub2video.sub_queue); - } av_buffer_unref(&ifilter->hw_frames_ctx); av_freep(&ifilter->name); av_freep(&fg->inputs[j]); @@ -635,12 +468,14 @@ static void ffmpeg_cleanup(int ret) av_frame_free(&ist->filter_frame); av_packet_free(&ist->pkt); av_dict_free(&ist->decoder_opts); - avsubtitle_free(&ist->prev_sub.subtitle); - av_frame_free(&ist->sub2video.frame); + av_frame_free(&ist->prev_sub.subtitle); av_freep(&ist->filters); av_freep(&ist->hwaccel_device); av_freep(&ist->dts_buffer); + av_frame_free(&ist->subtitle_heartbeat.recent_sub); + av_freep(&ist->subtitle_heartbeat.header); + avcodec_free_context(&ist->dec_ctx); av_freep(&input_streams[i]); @@ -1062,7 +897,7 @@ error: static void do_subtitle_out(OutputFile *of, OutputStream *ost, - AVSubtitle *sub) + AVFrame *frame) { int subtitle_out_max_size = 1024 * 1024; int subtitle_out_size, nb, i; @@ -1070,7 +905,12 @@ static void do_subtitle_out(OutputFile *of, AVPacket *pkt = ost->pkt; int64_t pts; - if (sub->pts == AV_NOPTS_VALUE) { + if (!frame) + return; + + av_log(NULL, AV_LOG_TRACE, "do_subtitle_out: sub->pts: %"PRId64" frame->pts: %"PRId64"\n", frame->subtitle_pts, frame->pts); + + if (frame->subtitle_pts == AV_NOPTS_VALUE) { av_log(NULL, AV_LOG_ERROR, "Subtitle packets must have a pts\n"); if (exit_on_error) exit_program(1); @@ -1096,30 +936,29 @@ static void do_subtitle_out(OutputFile *of, nb = 1; /* shift timestamp to honor -ss and make check_recording_time() work with -t */ - pts = sub->pts; + pts = frame->subtitle_pts; if (output_files[ost->file_index]->start_time != AV_NOPTS_VALUE) pts -= output_files[ost->file_index]->start_time; for (i = 0; i < nb; i++) { - unsigned save_num_rects = sub->num_rects; + unsigned save_num_rects = frame->num_subtitle_rects; ost->sync_opts = av_rescale_q(pts, AV_TIME_BASE_Q, enc->time_base); if (!check_recording_time(ost)) return; - sub->pts = pts; - // start_display_time is required to be 0 - sub->pts += av_rescale_q(sub->start_display_time, (AVRational){ 1, 1000 }, AV_TIME_BASE_Q); - sub->end_display_time -= sub->start_display_time; - sub->start_display_time = 0; + frame->subtitle_pts = pts; + // subtitle_start_time is required to be 0 + frame->subtitle_pts += av_rescale_q(frame->subtitle_start_time, (AVRational){ 1, 1000 }, AV_TIME_BASE_Q); + frame->subtitle_end_time -= frame->subtitle_start_time; + frame->subtitle_start_time = 0; if (i == 1) - sub->num_rects = 0; + frame->num_subtitle_rects = 0; ost->frames_encoded++; - subtitle_out_size = avcodec_encode_subtitle(enc, subtitle_out, - subtitle_out_max_size, sub); + subtitle_out_size = avcodec_encode_subtitle(enc, subtitle_out, subtitle_out_max_size, frame); if (i == 1) - sub->num_rects = save_num_rects; + frame->num_subtitle_rects = save_num_rects; if (subtitle_out_size < 0) { av_log(NULL, AV_LOG_FATAL, "Subtitle encoding failed\n"); exit_program(1); @@ -1128,15 +967,15 @@ static void do_subtitle_out(OutputFile *of, av_packet_unref(pkt); pkt->data = subtitle_out; pkt->size = subtitle_out_size; - pkt->pts = av_rescale_q(sub->pts, AV_TIME_BASE_Q, ost->mux_timebase); - pkt->duration = av_rescale_q(sub->end_display_time, (AVRational){ 1, 1000 }, ost->mux_timebase); + pkt->pts = av_rescale_q(frame->subtitle_pts, AV_TIME_BASE_Q, ost->mux_timebase); + pkt->duration = av_rescale_q(frame->subtitle_end_time, (AVRational){ 1, 1000 }, ost->mux_timebase); if (enc->codec_id == AV_CODEC_ID_DVB_SUBTITLE) { /* XXX: the pts correction is handled here. Maybe handling it in the codec would be better */ if (i == 0) - pkt->pts += av_rescale_q(sub->start_display_time, (AVRational){ 1, 1000 }, ost->mux_timebase); + pkt->pts += av_rescale_q(frame->subtitle_start_time, (AVRational){ 1, 1000 }, ost->mux_timebase); else - pkt->pts += av_rescale_q(sub->end_display_time, (AVRational){ 1, 1000 }, ost->mux_timebase); + pkt->pts += av_rescale_q(frame->subtitle_end_time, (AVRational){ 1, 1000 }, ost->mux_timebase); } pkt->dts = pkt->pts; output_packet(of, pkt, ost, 0); @@ -1577,8 +1416,11 @@ static int reap_filters(int flush) } do_audio_out(of, ost, filtered_frame); break; + case AVMEDIA_TYPE_SUBTITLE: + + do_subtitle_out(of, ost, filtered_frame); + break; default: - // TODO support subtitle filters av_assert0(0); } @@ -2174,7 +2016,8 @@ static int ifilter_has_all_input_formats(FilterGraph *fg) int i; for (i = 0; i < fg->nb_inputs; i++) { if (fg->inputs[i]->format < 0 && (fg->inputs[i]->type == AVMEDIA_TYPE_AUDIO || - fg->inputs[i]->type == AVMEDIA_TYPE_VIDEO)) + fg->inputs[i]->type == AVMEDIA_TYPE_VIDEO || + fg->inputs[i]->type == AVMEDIA_TYPE_SUBTITLE)) return 0; } return 1; @@ -2271,7 +2114,7 @@ static int ifilter_send_eof(InputFilter *ifilter, int64_t pts) // the filtergraph was never configured if (ifilter->format < 0) ifilter_parameters_from_codecpar(ifilter, ifilter->ist->st->codecpar); - if (ifilter->format < 0 && (ifilter->type == AVMEDIA_TYPE_AUDIO || ifilter->type == AVMEDIA_TYPE_VIDEO)) { + if (ifilter->format < 0 && (ifilter->type == AVMEDIA_TYPE_AUDIO || ifilter->type == AVMEDIA_TYPE_VIDEO || ifilter->type == AVMEDIA_TYPE_SUBTITLE)) { av_log(NULL, AV_LOG_ERROR, "Cannot determine format of input stream %d:%d after EOF\n", ifilter->ist->file_index, ifilter->ist->st->index); return AVERROR_INVALIDDATA; } @@ -2526,83 +2369,177 @@ fail: return err < 0 ? err : ret; } -static int transcode_subtitles(InputStream *ist, AVPacket *pkt, int *got_output, +static void subtitle_resend_current(InputStream *ist, int64_t heartbeat_pts) +{ + AVFrame *frame; + int64_t pts, end_pts; + + if (ist->subtitle_heartbeat.recent_sub) { + frame = av_frame_clone(ist->subtitle_heartbeat.recent_sub); + + pts = av_rescale_q(frame->subtitle_pts + frame->subtitle_start_time * 1000LL, AV_TIME_BASE_Q, ist->st->time_base); + end_pts = av_rescale_q(frame->subtitle_pts + frame->subtitle_end_time * 1000LL, AV_TIME_BASE_Q, ist->st->time_base); + } + else { + frame = av_frame_alloc(); + if (!frame) { + av_log(ist->dec_ctx, AV_LOG_ERROR, "Unable to alloc frame (out of memory).\n"); + return; + } + + frame->type = AVMEDIA_TYPE_SUBTITLE; + frame->format = av_get_subtitle_format_from_codecdesc(ist->dec_ctx->codec_descriptor); + + frame->width = ist->subtitle_heartbeat.w; + frame->height = ist->subtitle_heartbeat.h; + + + pts = ist->subtitle_heartbeat.end_pts <= 0 ? heartbeat_pts : ist->subtitle_heartbeat.end_pts; + end_pts = INT64_MAX; + + frame->subtitle_pts = av_rescale_q(pts, ist->st->time_base, AV_TIME_BASE_Q); + frame->subtitle_end_time = 1000; + } + + frame->pts = pts; + ist->subtitle_heartbeat.last_pts = pts; + ist->subtitle_heartbeat.end_pts = end_pts; + + send_frame_to_filters(ist, frame); +} + +static void subtitle_heartbeat(InputStream *ist, int64_t pts) +{ + const InputFile *infile = input_files[ist->file_index]; + int i; + int64_t pts2; + + /* When a frame is read from a file, examine all subtitle streams in + the same file and send the subtitle frame again. Otherwise, decoded + video frames could be accumulating in the filter graph while a filter + (possibly overlay) is desperately waiting for a subtitle frame. */ + for (i = 0; i < infile->nb_streams; i++) { + InputStream *ist2 = input_streams[infile->ist_index + i]; + if (!ist2->subtitle_heartbeat.is_active) + continue; + /* subtitles seem to be usually muxed ahead of other streams; + if not, subtracting a larger time here is necessary */ + pts2 = av_rescale_q(pts, ist->st->time_base, ist2->st->time_base) - 1; + /* do not send the heartbeat frame if the subtitle is already ahead */ + if (pts2 <= ist2->subtitle_heartbeat.last_pts) + continue; + if (pts2 >= ist2->subtitle_heartbeat.end_pts) { + /* if we have hit the end of the current displayed subpicture, + or if we need to initialize the system, update the + overlayed subpicture and its start/end times */ + if (ist2->subtitle_heartbeat.recent_sub) + av_frame_free(&ist2->subtitle_heartbeat.recent_sub); + subtitle_resend_current(ist2, pts2 + 1); + } + //for (j = 0, nb_reqs = 0; j < ist2->nb_filters; j++) + // nb_reqs += av_buffersrc_get_nb_failed_requests(ist2->filters[j]->filter); + //if (nb_reqs) + // subtitle_heartbeat_resend_current(ist2, pts2); + } +} + +static int decode_subtitles(InputStream *ist, AVPacket *pkt, int *got_output, int *decode_failed) { - AVSubtitle subtitle; - int free_sub = 1; - int i, ret = avcodec_decode_subtitle2(ist->dec_ctx, - &subtitle, got_output, pkt); + AVFrame *decoded_frame; + AVCodecContext *avctx = ist->dec_ctx; + int i = 0, ret = 0, err = 0; + int64_t pts, end_pts; + + if (!ist->decoded_frame && !(ist->decoded_frame = av_frame_alloc())) + return AVERROR(ENOMEM); + if (!ist->filter_frame && !(ist->filter_frame = av_frame_alloc())) + return AVERROR(ENOMEM); + + decoded_frame = ist->decoded_frame; + decoded_frame->type = AVMEDIA_TYPE_SUBTITLE; + + if (!ist->subtitle_heartbeat.header && avctx->subtitle_header) + ist->subtitle_heartbeat.header = av_strdup((char *)avctx->subtitle_header); - check_decode_result(NULL, got_output, ret); + ret = avcodec_decode_subtitle2(avctx, decoded_frame, got_output, pkt); + decoded_frame->subtitle_header = ist->subtitle_heartbeat.header; + + if (ret != AVERROR_EOF) + check_decode_result(NULL, got_output, ret); if (ret < 0 || !*got_output) { *decode_failed = 1; - if (!pkt->size) - sub2video_flush(ist); + if (!pkt->size) { + // Flush + for (i = 0; i < ist->nb_filters; i++) { + ret = av_buffersrc_add_frame(ist->filters[i]->filter, NULL); + if (ret != AVERROR_EOF && ret < 0) + av_log(NULL, AV_LOG_WARNING, "Flush the frame error.\n"); + } + } return ret; } if (ist->fix_sub_duration) { int end = 1; - if (ist->prev_sub.got_output) { - end = av_rescale(subtitle.pts - ist->prev_sub.subtitle.pts, + if (ist->prev_sub.got_output && ist->prev_sub.subtitle) { + end = av_rescale(decoded_frame->subtitle_pts - ist->prev_sub.subtitle->subtitle_pts, 1000, AV_TIME_BASE); - if (end < ist->prev_sub.subtitle.end_display_time) { - av_log(ist->dec_ctx, AV_LOG_DEBUG, + if (end < ist->prev_sub.subtitle->subtitle_end_time) { + av_log(avctx, AV_LOG_DEBUG, "Subtitle duration reduced from %"PRId32" to %d%s\n", - ist->prev_sub.subtitle.end_display_time, end, + ist->prev_sub.subtitle->subtitle_end_time, end, end <= 0 ? ", dropping it" : ""); - ist->prev_sub.subtitle.end_display_time = end; + ist->prev_sub.subtitle->subtitle_end_time = end; } } FFSWAP(int, *got_output, ist->prev_sub.got_output); FFSWAP(int, ret, ist->prev_sub.ret); - FFSWAP(AVSubtitle, subtitle, ist->prev_sub.subtitle); + FFSWAP(AVFrame*, decoded_frame, ist->prev_sub.subtitle); if (end <= 0) - goto out; + return end; } if (!*got_output) return ret; - if (ist->sub2video.frame) { - sub2video_update(ist, INT64_MIN, &subtitle); - } else if (ist->nb_filters) { - if (!ist->sub2video.sub_queue) - ist->sub2video.sub_queue = av_fifo_alloc(8 * sizeof(AVSubtitle)); - if (!ist->sub2video.sub_queue) - exit_program(1); - if (!av_fifo_space(ist->sub2video.sub_queue)) { - ret = av_fifo_realloc2(ist->sub2video.sub_queue, 2 * av_fifo_size(ist->sub2video.sub_queue)); - if (ret < 0) - exit_program(1); - } - av_fifo_generic_write(ist->sub2video.sub_queue, &subtitle, sizeof(subtitle), NULL); - free_sub = 0; - } + decoded_frame->format = av_get_subtitle_format_from_codecdesc(avctx->codec_descriptor); + decoded_frame->type = AVMEDIA_TYPE_SUBTITLE; - if (!subtitle.num_rects) - goto out; + if ((ret = av_frame_get_buffer2(decoded_frame, 0)) < 0) + return ret; - ist->frames_decoded++; + pts = av_rescale_q(decoded_frame->subtitle_pts + decoded_frame->subtitle_start_time * 1000LL, + AV_TIME_BASE_Q, ist->st->time_base); + end_pts = av_rescale_q(decoded_frame->subtitle_pts + decoded_frame->subtitle_end_time * 1000LL, + AV_TIME_BASE_Q, ist->st->time_base); - for (i = 0; i < nb_output_streams; i++) { - OutputStream *ost = output_streams[i]; + ist->subtitle_heartbeat.last_pts = decoded_frame->pts = pts; + ist->subtitle_heartbeat.end_pts = end_pts; - if (!ost->pkt && !(ost->pkt = av_packet_alloc())) - exit_program(1); - if (!check_output_constraints(ist, ost) || !ost->encoding_needed - || ost->enc->type != AVMEDIA_TYPE_SUBTITLE) - continue; + if (ist->nb_filters == 0) { + for (i = 0; i < nb_output_streams; i++) { + OutputStream *ost = output_streams[i]; - do_subtitle_out(output_files[ost->file_index], ost, &subtitle); + if (!ost->pkt && !(ost->pkt = av_packet_alloc())) + exit_program(1); + if (!check_output_constraints(ist, ost) || !ost->encoding_needed + || ost->enc->type != AVMEDIA_TYPE_SUBTITLE) + continue; + + do_subtitle_out(output_files[ost->file_index], ost, decoded_frame); + } } + else + err = send_frame_to_filters(ist, decoded_frame); -out: - if (free_sub) - avsubtitle_free(&subtitle); - return ret; + av_frame_free(&ist->subtitle_heartbeat.recent_sub); + ist->subtitle_heartbeat.recent_sub = av_frame_clone(decoded_frame); + + av_frame_unref(ist->filter_frame); + av_frame_unref(decoded_frame); + return err < 0 ? err : ret; } static int send_filter_eof(InputStream *ist) @@ -2710,7 +2647,7 @@ static int process_input_packet(InputStream *ist, const AVPacket *pkt, int no_eo case AVMEDIA_TYPE_SUBTITLE: if (repeating) break; - ret = transcode_subtitles(ist, avpkt, &got_output, &decode_failed); + ret = decode_subtitles(ist, avpkt, &got_output, &decode_failed); if (!pkt && ret >= 0) ret = AVERROR_EOF; av_packet_unref(avpkt); @@ -2983,8 +2920,6 @@ FF_ENABLE_DEPRECATION_WARNINGS av_log(NULL, AV_LOG_WARNING, "Warning using DVB subtitles for filtering and output at the same time is not fully supported, also see -compute_edt [0|1]\n"); } - av_dict_set(&ist->decoder_opts, "sub_text_format", "ass", AV_DICT_DONT_OVERWRITE); - /* Useful for subtitles retiming by lavf (FIXME), skipping samples in * audio, and video decoders such as cuvid or mediacodec */ ist->dec_ctx->pkt_timebase = ist->st->time_base; @@ -3582,19 +3517,14 @@ static int init_output_stream(OutputStream *ost, AVFrame *frame, } if (ist && ist->dec->type == AVMEDIA_TYPE_SUBTITLE && ost->enc->type == AVMEDIA_TYPE_SUBTITLE) { - int input_props = 0, output_props = 0; - AVCodecDescriptor const *input_descriptor = - avcodec_descriptor_get(dec->codec_id); - AVCodecDescriptor const *output_descriptor = - avcodec_descriptor_get(ost->enc_ctx->codec_id); - if (input_descriptor) - input_props = input_descriptor->props & (AV_CODEC_PROP_TEXT_SUB | AV_CODEC_PROP_BITMAP_SUB); - if (output_descriptor) - output_props = output_descriptor->props & (AV_CODEC_PROP_TEXT_SUB | AV_CODEC_PROP_BITMAP_SUB); - if (input_props && output_props && input_props != output_props) { - snprintf(error, error_len, - "Subtitle encoding currently only possible from text to text " - "or bitmap to bitmap"); + AVCodecDescriptor const *input_descriptor = avcodec_descriptor_get(dec->codec_id); + AVCodecDescriptor const *output_descriptor = avcodec_descriptor_get(ost->enc_ctx->codec_id); + const enum AVSubtitleType in_subtitle_format = output_descriptor ? av_get_subtitle_format_from_codecdesc(input_descriptor) : AV_SUBTITLE_FMT_UNKNOWN; + const enum AVSubtitleType out_subtitle_format = output_descriptor ? av_get_subtitle_format_from_codecdesc(output_descriptor) : AV_SUBTITLE_FMT_UNKNOWN; + + if (in_subtitle_format != AV_SUBTITLE_FMT_UNKNOWN && out_subtitle_format != AV_SUBTITLE_FMT_UNKNOWN + && in_subtitle_format != out_subtitle_format) { + snprintf(error, error_len, "Subtitle encoding is only possible from text to text or bitmap to bitmap"); return AVERROR_INVALIDDATA; } } @@ -4627,7 +4557,7 @@ static int process_input(int file_index) av_ts2timestr(input_files[ist->file_index]->ts_offset, &AV_TIME_BASE_Q)); } - sub2video_heartbeat(ist, pkt->pts); + subtitle_heartbeat(ist, pkt->pts); process_input_packet(ist, pkt, 0); @@ -4839,6 +4769,7 @@ static int transcode(void) /* at the end of stream, we must flush the decoder buffers */ for (i = 0; i < nb_input_streams; i++) { ist = input_streams[i]; + ist->subtitle_heartbeat.is_active = 0; if (!input_files[ist->file_index]->eof_reached) { process_input_packet(ist, NULL, 0); } diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index 1194bb0cae..5cc2031653 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -348,17 +348,17 @@ typedef struct InputStream { struct { /* previous decoded subtitle and related variables */ int got_output; int ret; - AVSubtitle subtitle; + AVFrame *subtitle; } prev_sub; - struct sub2video { + struct subtitle_heartbeat { + int is_active; int64_t last_pts; int64_t end_pts; - AVFifoBuffer *sub_queue; ///< queue of AVSubtitle* before filter init - AVFrame *frame; + AVFrame *recent_sub; int w, h; - unsigned int initialize; ///< marks if sub2video_update should force an initialization - } sub2video; + char *header; + } subtitle_heartbeat; /* decoded data from this stream goes into all those filters * currently video and audio only */ @@ -660,8 +660,6 @@ int filtergraph_is_simple(FilterGraph *fg); int init_simple_filtergraph(InputStream *ist, OutputStream *ost); int init_complex_filtergraph(FilterGraph *fg); -void sub2video_update(InputStream *ist, int64_t heartbeat_pts, AVSubtitle *sub); - int ifilter_parameters_from_frame(InputFilter *ifilter, const AVFrame *frame); int ffmpeg_parse_options(int argc, char **argv); diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c index 6b7b6ca1b3..983b49cb8b 100644 --- a/fftools/ffmpeg_filter.c +++ b/fftools/ffmpeg_filter.c @@ -22,6 +22,9 @@ #include "ffmpeg.h" +#include +#include + #include "libavfilter/avfilter.h" #include "libavfilter/buffersink.h" #include "libavfilter/buffersrc.h" @@ -221,9 +224,8 @@ static void init_input_filter(FilterGraph *fg, AVFilterInOut *in) enum AVMediaType type = avfilter_pad_get_type(in->filter_ctx->input_pads, in->pad_idx); int i; - // TODO: support other filter types - if (type != AVMEDIA_TYPE_VIDEO && type != AVMEDIA_TYPE_AUDIO) { - av_log(NULL, AV_LOG_FATAL, "Only video and audio filters supported " + if (type != AVMEDIA_TYPE_VIDEO && type != AVMEDIA_TYPE_AUDIO && type != AVMEDIA_TYPE_SUBTITLE) { + av_log(NULL, AV_LOG_FATAL, "Only video, audio and subtitle filters supported " "currently.\n"); exit_program(1); } @@ -244,8 +246,9 @@ static void init_input_filter(FilterGraph *fg, AVFilterInOut *in) for (i = 0; i < s->nb_streams; i++) { enum AVMediaType stream_type = s->streams[i]->codecpar->codec_type; if (stream_type != type && - !(stream_type == AVMEDIA_TYPE_SUBTITLE && - type == AVMEDIA_TYPE_VIDEO /* sub2video hack */)) + // in the followng case we auto-insert the graphicsub2video conversion filter + // for retaining compatibility with the previous sub2video hack + !(stream_type == AVMEDIA_TYPE_SUBTITLE && type == AVMEDIA_TYPE_VIDEO)) continue; if (check_stream_specifier(s, s->streams[i], *p == ':' ? p + 1 : p) == 1) { st = s->streams[i]; @@ -416,6 +419,37 @@ static int insert_filter(AVFilterContext **last_filter, int *pad_idx, return 0; } +static int configure_output_subtitle_filter(FilterGraph *fg, OutputFilter *ofilter, AVFilterInOut *out) +{ + OutputStream *ost = ofilter->ost; + AVFilterContext *last_filter = out->filter_ctx; + int pad_idx = out->pad_idx; + int ret; + char name[255]; + + snprintf(name, sizeof(name), "out_%d_%d", ost->file_index, ost->index); + ret = avfilter_graph_create_filter(&ofilter->filter, + avfilter_get_by_name("sbuffersink"), + name, NULL, NULL, fg->graph); + + if (ret < 0) + return ret; + + ////snprintf(name, sizeof(name), "trim_out_%d_%d", + //// ost->file_index, ost->index); + ////ret = insert_trim(of->start_time, of->recording_time, + //// &last_filter, &pad_idx, name); + ////if (ret < 0) + //// return ret; + + ////ost->st->codecpar->codec_tag = MKTAG('a', 's', 's', 's'); + + if ((ret = avfilter_link(last_filter, pad_idx, ofilter->filter, 0)) < 0) + return ret; + + return 0; +} + static int configure_output_video_filter(FilterGraph *fg, OutputFilter *ofilter, AVFilterInOut *out) { char *pix_fmts; @@ -594,7 +628,8 @@ static int configure_output_audio_filter(FilterGraph *fg, OutputFilter *ofilter, int i; for (i=0; ictx->nb_streams; i++) - if (of->ctx->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_VIDEO) + if (of->ctx->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_VIDEO || + of->ctx->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_SUBTITLE) break; if (ictx->nb_streams) { @@ -628,6 +663,7 @@ static int configure_output_filter(FilterGraph *fg, OutputFilter *ofilter, switch (avfilter_pad_get_type(out->filter_ctx->output_pads, out->pad_idx)) { case AVMEDIA_TYPE_VIDEO: return configure_output_video_filter(fg, ofilter, out); case AVMEDIA_TYPE_AUDIO: return configure_output_audio_filter(fg, ofilter, out); + case AVMEDIA_TYPE_SUBTITLE: return configure_output_subtitle_filter(fg, ofilter, out); default: av_assert0(0); return 0; } } @@ -647,51 +683,103 @@ void check_filter_outputs(void) } } -static int sub2video_prepare(InputStream *ist, InputFilter *ifilter) +static int configure_input_subtitle_filter(FilterGraph *fg, InputFilter *ifilter, + AVFilterInOut *in) { - AVFormatContext *avf = input_files[ist->file_index]->ctx; - int i, w, h; + AVFilterContext *last_filter; + const AVFilter *buffer_filt = avfilter_get_by_name("sbuffer"); + InputStream *ist = ifilter->ist; + AVBPrint args; + char name[255]; + int ret, pad_idx = 0; + int w, h; + AVBufferSrcParameters *par = av_buffersrc_parameters_alloc(); + + if (!par) + return AVERROR(ENOMEM); + memset(par, 0, sizeof(*par)); + par->format = AV_PIX_FMT_NONE; + + if (ist->dec_ctx->codec_type == AVMEDIA_TYPE_AUDIO) { + av_log(NULL, AV_LOG_ERROR, "Cannot connect subtitle filter to audio input\n"); + ret = AVERROR(EINVAL); + goto fail; + } + + if (ist->dec_ctx->codec_type == AVMEDIA_TYPE_VIDEO) { + av_log(NULL, AV_LOG_ERROR, "Cannot connect subtitle filter to video input\n"); + ret = AVERROR(EINVAL); + goto fail; + } + + if (!ist->subtitle_heartbeat.header && ist->dec_ctx->subtitle_header) + ist->subtitle_heartbeat.header = av_strdup((char *)ist->dec_ctx->subtitle_header); + + ist->subtitle_heartbeat.is_active = 1; - /* Compute the size of the canvas for the subtitles stream. - If the subtitles codecpar has set a size, use it. Otherwise use the - maximum dimensions of the video streams in the same file. */ w = ifilter->width; h = ifilter->height; + if (!(w && h)) { - for (i = 0; i < avf->nb_streams; i++) { - if (avf->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_VIDEO) { - w = FFMAX(w, avf->streams[i]->codecpar->width); - h = FFMAX(h, avf->streams[i]->codecpar->height); - } - } - if (!(w && h)) { - w = FFMAX(w, 720); - h = FFMAX(h, 576); - } - av_log(avf, AV_LOG_INFO, "sub2video: using %dx%d canvas\n", w, h); + w = ist->dec_ctx->width; + h = ist->dec_ctx->height; } - ist->sub2video.w = ifilter->width = w; - ist->sub2video.h = ifilter->height = h; - ifilter->width = ist->dec_ctx->width ? ist->dec_ctx->width : ist->sub2video.w; - ifilter->height = ist->dec_ctx->height ? ist->dec_ctx->height : ist->sub2video.h; + if (!(w && h) && ist->dec_ctx->subtitle_header) { + ASSSplitContext *ass_ctx = ff_ass_split((char *)ist->dec_ctx->subtitle_header); + ASS *ass = (ASS *)ass_ctx; + w = ass->script_info.play_res_x; + h = ass->script_info.play_res_y; + ff_ass_split_free(ass_ctx); + } - /* rectangles are AV_PIX_FMT_PAL8, but we have no guarantee that the - palettes for all rectangles are identical or compatible */ - ifilter->format = AV_PIX_FMT_RGB32; + ist->subtitle_heartbeat.w = w; + ist->subtitle_heartbeat.h = h; + av_log(ifilter, AV_LOG_INFO, "subtitle input filter: decoding size %dx%d\n", ist->subtitle_heartbeat.w, ist->subtitle_heartbeat.h); - ist->sub2video.frame = av_frame_alloc(); - if (!ist->sub2video.frame) - return AVERROR(ENOMEM); - ist->sub2video.last_pts = INT64_MIN; - ist->sub2video.end_pts = INT64_MIN; + ifilter->width = w; + ifilter->height = h; + + ist->subtitle_heartbeat.last_pts = INT64_MIN; + + snprintf(name, sizeof(name), "graph %d subtitle input from stream %d:%d", fg->index, + ist->file_index, ist->st->index); + + + av_bprint_init(&args, 0, AV_BPRINT_SIZE_AUTOMATIC); + av_bprintf(&args, + "subtitle_type=%d:time_base=%d/%d:", + ifilter->format, + ist->st->time_base.num, ist->st->time_base.den); + if ((ret = avfilter_graph_create_filter(&ifilter->filter, buffer_filt, name, + args.str, NULL, fg->graph)) < 0) + goto fail; - /* sub2video structure has been (re-)initialized. - Mark it as such so that the system will be - initialized with the first received heartbeat. */ - ist->sub2video.initialize = 1; + par->hw_frames_ctx = ifilter->hw_frames_ctx; + par->format = ifilter->format; + par->width = ifilter->width; + par->height = ifilter->height; + + ret = av_buffersrc_parameters_set(ifilter->filter, par); + if (ret < 0) + goto fail; + av_freep(&par); + last_filter = ifilter->filter; + + if (in->filter_ctx->input_pads[in->pad_idx].type == AVMEDIA_TYPE_VIDEO) { + ret = insert_filter(&last_filter, &pad_idx, "graphicsub2video", NULL); + if (ret < 0) + return ret; + } + + if ((ret = avfilter_link(last_filter, 0, in->filter_ctx, in->pad_idx)) < 0) + return ret; return 0; +fail: + av_freep(&par); + + return ret; } static int configure_input_video_filter(FilterGraph *fg, InputFilter *ifilter, @@ -709,8 +797,15 @@ static int configure_input_video_filter(FilterGraph *fg, InputFilter *ifilter, char name[255]; int ret, pad_idx = 0; int64_t tsoffset = 0; - AVBufferSrcParameters *par = av_buffersrc_parameters_alloc(); + AVBufferSrcParameters *par; + if (ist->dec_ctx->codec_type == AVMEDIA_TYPE_SUBTITLE) { + // Automatically insert conversion filter to retain compatibility + // with sub2video command lines + return configure_input_subtitle_filter(fg, ifilter, in); + } + + par = av_buffersrc_parameters_alloc(); if (!par) return AVERROR(ENOMEM); memset(par, 0, sizeof(*par)); @@ -725,12 +820,6 @@ static int configure_input_video_filter(FilterGraph *fg, InputFilter *ifilter, if (!fr.num) fr = av_guess_frame_rate(input_files[ist->file_index]->ctx, ist->st, NULL); - if (ist->dec_ctx->codec_type == AVMEDIA_TYPE_SUBTITLE) { - ret = sub2video_prepare(ist, ifilter); - if (ret < 0) - goto fail; - } - sar = ifilter->sample_aspect_ratio; if(!sar.den) sar = (AVRational){0,1}; @@ -742,7 +831,7 @@ static int configure_input_video_filter(FilterGraph *fg, InputFilter *ifilter, tb.num, tb.den, sar.num, sar.den); if (fr.num && fr.den) av_bprintf(&args, ":frame_rate=%d/%d", fr.num, fr.den); - snprintf(name, sizeof(name), "graph %d input from stream %d:%d", fg->index, + snprintf(name, sizeof(name), "graph %d video input from stream %d:%d", fg->index, ist->file_index, ist->st->index); @@ -938,6 +1027,7 @@ static int configure_input_filter(FilterGraph *fg, InputFilter *ifilter, switch (avfilter_pad_get_type(in->filter_ctx->input_pads, in->pad_idx)) { case AVMEDIA_TYPE_VIDEO: return configure_input_video_filter(fg, ifilter, in); case AVMEDIA_TYPE_AUDIO: return configure_input_audio_filter(fg, ifilter, in); + case AVMEDIA_TYPE_SUBTITLE: return configure_input_subtitle_filter(fg, ifilter, in); default: av_assert0(0); return 0; } } @@ -1105,19 +1195,6 @@ int configure_filtergraph(FilterGraph *fg) } } - /* process queued up subtitle packets */ - for (i = 0; i < fg->nb_inputs; i++) { - InputStream *ist = fg->inputs[i]->ist; - if (ist->sub2video.sub_queue && ist->sub2video.frame) { - while (av_fifo_size(ist->sub2video.sub_queue)) { - AVSubtitle tmp; - av_fifo_generic_read(ist->sub2video.sub_queue, &tmp, sizeof(tmp), NULL); - sub2video_update(ist, INT64_MIN, &tmp); - avsubtitle_free(&tmp); - } - } - } - return 0; fail: @@ -1138,6 +1215,7 @@ int ifilter_parameters_from_frame(InputFilter *ifilter, const AVFrame *frame) ifilter->sample_rate = frame->sample_rate; ifilter->channels = frame->channels; ifilter->channel_layout = frame->channel_layout; + ifilter->type = frame->type; if (frame->hw_frames_ctx) { ifilter->hw_frames_ctx = av_buffer_ref(frame->hw_frames_ctx); diff --git a/fftools/ffmpeg_hw.c b/fftools/ffmpeg_hw.c index 41aaf776d7..f8b49fd5ac 100644 --- a/fftools/ffmpeg_hw.c +++ b/fftools/ffmpeg_hw.c @@ -449,7 +449,7 @@ int hw_device_setup_for_encode(OutputStream *ost) AVBufferRef *frames_ref = NULL; int i; - if (ost->filter) { + if (ost->filter && ost->filter->filter) { frames_ref = av_buffersink_get_hw_frames_ctx(ost->filter->filter); if (frames_ref && ((AVHWFramesContext*)frames_ref->data)->format == diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c index 6ca28cf974..21b5228bb2 100644 --- a/fftools/ffmpeg_opt.c +++ b/fftools/ffmpeg_opt.c @@ -2149,8 +2149,9 @@ static void init_output_filter(OutputFilter *ofilter, OptionsContext *o, switch (ofilter->type) { case AVMEDIA_TYPE_VIDEO: ost = new_video_stream(o, oc, -1); break; case AVMEDIA_TYPE_AUDIO: ost = new_audio_stream(o, oc, -1); break; + case AVMEDIA_TYPE_SUBTITLE: ost = new_subtitle_stream(o, oc, -1); break; default: - av_log(NULL, AV_LOG_FATAL, "Only video and audio filters are supported " + av_log(NULL, AV_LOG_FATAL, "Only video, audio and subtitle filters are supported " "currently.\n"); exit_program(1); } diff --git a/tests/ref/fate/filter-overlay-dvdsub-2397 b/tests/ref/fate/filter-overlay-dvdsub-2397 index 483e5fa4e0..1f505ba2b2 100644 --- a/tests/ref/fate/filter-overlay-dvdsub-2397 +++ b/tests/ref/fate/filter-overlay-dvdsub-2397 @@ -490,368 +490,367 @@ 1, 3877, 3877, 10, 2013, 0x95a39f9c 1, 3887, 3887, 10, 2013, 0x4f7ea123 1, 3897, 3897, 10, 2013, 0x9efb9ba1 -0, 117, 117, 1, 518400, 0xbf8523da +0, 117, 117, 1, 518400, 0x61e0f688 1, 3907, 3907, 10, 2013, 0xf395b2cd 1, 3917, 3917, 10, 2013, 0x261a881e 1, 3927, 3927, 10, 2013, 0x7f2d9f72 1, 3937, 3937, 10, 2013, 0x0105b38d -0, 118, 118, 1, 518400, 0x41890ed6 +0, 118, 118, 1, 518400, 0xa47de755 1, 3952, 3952, 10, 2013, 0x0e5db67e 1, 3962, 3962, 10, 2013, 0xfc9baf97 -0, 119, 119, 1, 518400, 0x588534fc +0, 119, 119, 1, 518400, 0xaade171d 1, 3972, 3972, 10, 2013, 0x8e02a1b1 1, 3982, 3982, 10, 2013, 0x6eecaac8 1, 3992, 3992, 10, 2013, 0xf5558f0c 1, 4002, 4002, 10, 2013, 0x512ba99b -0, 120, 120, 1, 518400, 0x2145ebc1 +0, 120, 120, 1, 518400, 0xbc06d530 1, 4012, 4012, 10, 2013, 0x932b9932 1, 4022, 4022, 10, 2013, 0xc01ea987 -0, 121, 121, 1, 518400, 0x28bca595 +0, 121, 121, 1, 518400, 0xd4709c99 1, 4038, 4038, 10, 2013, 0x10879cf7 1, 4048, 4048, 10, 2013, 0x90679338 1, 4058, 4058, 10, 2013, 0x077d8a9e 1, 4068, 4068, 10, 2013, 0x969fa57c -0, 122, 122, 1, 518400, 0x77dc951e +0, 122, 122, 1, 518400, 0x12989321 1, 4078, 4078, 10, 2013, 0xe049ab07 1, 4088, 4088, 10, 2013, 0xf535b3b3 1, 4098, 4098, 10, 2013, 0xfe76bd37 -0, 123, 123, 1, 518400, 0xe8924c17 +0, 123, 123, 1, 518400, 0x3c875430 1, 4108, 4108, 10, 2013, 0xde79ad8c 1, 4123, 4123, 10, 2013, 0xe89b9c47 1, 4133, 4133, 10, 2013, 0xc570b0f0 -0, 124, 124, 1, 518400, 0xadb4cccc +0, 124, 124, 1, 518400, 0x8b18e05f 1, 4143, 4143, 10, 2013, 0xee709cd9 1, 4153, 4153, 10, 2013, 0xcfe5afab 1, 4163, 4163, 10, 2013, 0x98ff8ce4 -0, 125, 125, 1, 518400, 0x1d7b56ac +0, 125, 125, 1, 518400, 0xba537174 1, 4173, 4173, 10, 2013, 0x9d19b44c 1, 4183, 4183, 10, 2013, 0x4349917a 1, 4193, 4193, 10, 2013, 0xbf54a59a -0, 126, 126, 1, 518400, 0xad5739a4 +0, 126, 126, 1, 518400, 0x0f625b1a 1, 4208, 4208, 10, 2013, 0xc4a399e0 1, 4218, 4218, 10, 2013, 0x1bf58ff0 1, 4228, 4228, 10, 2013, 0x3518ac56 -0, 127, 127, 1, 518400, 0x2733d35a +0, 127, 127, 1, 518400, 0x16b103e4 1, 4238, 4238, 10, 2013, 0xcd38c1de 1, 4248, 4248, 10, 2013, 0xbe7d9c4d 1, 4258, 4258, 10, 2013, 0xe113a306 1, 4268, 4268, 10, 2013, 0x083197ea -0, 128, 128, 1, 518400, 0x78e76da2 +0, 128, 128, 1, 518400, 0xf1d8a234 1, 4278, 4278, 10, 2013, 0x1929b1eb 1, 4294, 4294, 10, 2013, 0x5d6ea5af 1, 4304, 4304, 10, 2013, 0x05519d53 -0, 129, 129, 1, 518400, 0x6c076013 +0, 129, 129, 1, 518400, 0x93109d3f 1, 4314, 4314, 10, 2013, 0x5773b380 1, 4324, 4324, 10, 2013, 0xaa70a8f5 1, 4334, 4334, 10, 2013, 0x990db0ec -0, 130, 130, 1, 518400, 0x7854f2b1 +0, 130, 130, 1, 518400, 0x902b37c7 1, 4344, 4344, 10, 2013, 0x91d3a623 1, 4354, 4354, 10, 2013, 0xc91f9824 1, 4364, 4364, 10, 2013, 0x1d058abf -0, 131, 131, 1, 518400, 0xd2ae1ecd +0, 131, 131, 1, 518400, 0xc41862c2 1, 4379, 4379, 10, 2013, 0x8de1b8d5 1, 4389, 4389, 10, 2013, 0x7872b06b 1, 4399, 4399, 10, 2013, 0xa084c203 -0, 132, 132, 1, 518400, 0xf5eab38d +0, 132, 132, 1, 518400, 0xf733feea 1, 4409, 4409, 10, 2013, 0xff90ae8d 1, 4419, 4419, 10, 2013, 0x61dead8e 1, 4429, 4429, 10, 2013, 0xee76b284 -0, 133, 133, 1, 518400, 0x994d3e9c +0, 133, 133, 1, 518400, 0x813b9340 1, 4439, 4439, 10, 2013, 0xe888af7f 1, 4449, 4449, 10, 2013, 0x5d57b115 1, 4464, 4464, 10, 2013, 0xcdbfb1d0 -0, 134, 134, 1, 518400, 0x95ab705a +0, 134, 134, 1, 518400, 0xc55fc66f 1, 4474, 4474, 10, 2013, 0x2e28a952 1, 4484, 4484, 10, 2013, 0x4795a994 1, 4494, 4494, 10, 2013, 0x7e7ea304 1, 4504, 4504, 10, 2013, 0x9502c1e1 -0, 135, 135, 1, 518400, 0x3c83c5ce +0, 135, 135, 1, 518400, 0xbd401cc5 1, 4514, 4514, 10, 2013, 0xf7c78ab2 1, 4524, 4524, 10, 2013, 0x24049816 1, 4534, 4534, 10, 2013, 0x52089dcf -0, 136, 136, 1, 518400, 0xfa22c508 +0, 136, 136, 1, 518400, 0x9d202a03 1, 4550, 4550, 10, 2013, 0x2150a0b1 1, 4560, 4560, 10, 2013, 0x3c2e9b93 1, 4570, 4570, 10, 2013, 0x491f932b -0, 137, 137, 1, 518400, 0xddda1712 +0, 137, 137, 1, 518400, 0x822f8062 1, 4580, 4580, 10, 2013, 0x31359cf8 1, 4590, 4590, 10, 2013, 0x1b00ac3f 1, 4600, 4600, 10, 2013, 0x8d7ab3cb -0, 138, 138, 1, 518400, 0x985a3b93 +0, 138, 138, 1, 518400, 0xe011aad7 1, 4610, 4610, 10, 2013, 0xb2c2a4de 1, 4620, 4620, 10, 2013, 0x80a4abf2 1, 4635, 4635, 10, 2013, 0x0701a4ee -0, 139, 139, 1, 518400, 0xea63c5e7 +0, 139, 139, 1, 518400, 0xb0934609 1, 4645, 4645, 10, 2013, 0xdc1ba5bc 1, 4655, 4655, 10, 2013, 0x6083a8a4 1, 4665, 4665, 10, 2013, 0x6226ad45 -0, 140, 140, 1, 518400, 0xef64983d +0, 140, 140, 1, 518400, 0xb22a25e0 1, 4675, 4675, 10, 2013, 0x2732a205 1, 4685, 4685, 10, 2013, 0x0f62a0d3 1, 4695, 4695, 10, 2013, 0xc1799249 -0, 141, 141, 1, 518400, 0x747bb193 +0, 141, 141, 1, 518400, 0x4a715018 1, 4705, 4705, 10, 2013, 0xbccfa9c8 1, 4720, 4720, 10, 2013, 0xded096e7 1, 4730, 4730, 10, 2013, 0x7f0daf43 -0, 142, 142, 1, 518400, 0xb8748862 +0, 142, 142, 1, 518400, 0x63f24360 1, 4740, 4740, 10, 2013, 0xc47ea682 1, 4750, 4750, 10, 2013, 0x5a72b07a 1, 4760, 4760, 10, 2013, 0x386faa8c 1, 4770, 4770, 10, 2013, 0xf9919a91 -0, 143, 143, 1, 518400, 0xaab55a5f +0, 143, 143, 1, 518400, 0x93002157 1, 4780, 4780, 10, 2013, 0x4908897e 1, 4790, 4790, 10, 2013, 0x4882b594 -0, 144, 144, 1, 518400, 0x7b468add +0, 144, 144, 1, 518400, 0xab6b616b 1, 4806, 4806, 10, 2013, 0x113e98d1 1, 4816, 4816, 10, 2013, 0x5098b30d 1, 4826, 4826, 10, 2013, 0x0ef7b857 1, 4836, 4836, 10, 2013, 0x216ea176 -0, 145, 145, 1, 518400, 0xf2078707 +0, 145, 145, 1, 518400, 0x331375bf 1, 4846, 4846, 10, 2013, 0xf906944a 1, 4856, 4856, 10, 2013, 0xee9b92fb 1, 4866, 4866, 10, 2013, 0xd6029209 -0, 146, 146, 1, 518400, 0x6a2d931e +0, 146, 146, 1, 518400, 0x4d8992ca 1, 4876, 4876, 10, 2013, 0x2256a12e 1, 4891, 4891, 10, 2013, 0x89de8e4a 1, 4901, 4901, 10, 2013, 0x0bf0a584 -0, 147, 147, 1, 518400, 0xbbe3c417 +0, 147, 147, 1, 518400, 0x60ebc989 1, 4911, 4911, 10, 2013, 0x6a5ebd58 1, 4921, 4921, 10, 2013, 0x3edd9aa4 1, 4931, 4931, 10, 2013, 0xbd66ac26 -0, 148, 148, 1, 518400, 0x6294e449 +0, 148, 148, 1, 518400, 0xcb0cf07d 1, 4941, 4941, 10, 2013, 0x313896ea 1, 4951, 4951, 10, 2013, 0x6b83a6a0 1, 4961, 4961, 10, 2013, 0x9aafb109 -0, 149, 149, 1, 518400, 0xa05721e7 +0, 149, 149, 1, 518400, 0x8374277a 1, 4976, 4976, 10, 2013, 0x5192a85a 1, 4986, 4986, 10, 2013, 0x1f919f79 1, 4996, 4996, 10, 2013, 0xc0799c40 -0, 150, 150, 1, 518400, 0x37749183 +0, 150, 150, 1, 518400, 0x72678781 1, 5006, 5006, 10, 2013, 0x2988bcd8 1, 5016, 5016, 10, 2013, 0x1482913a 1, 5026, 5026, 10, 2013, 0x74da9a94 1, 5036, 5036, 10, 2013, 0x763eb709 -0, 151, 151, 1, 518400, 0xf9d9dca0 +0, 151, 151, 1, 518400, 0x46e2d001 1, 5046, 5046, 10, 2013, 0x1285b405 1, 5062, 5062, 10, 2013, 0xb6ab9dfc -0, 152, 152, 1, 518400, 0x5f8ccf08 +0, 152, 152, 1, 518400, 0x892cac49 1, 5072, 5072, 10, 2013, 0xe4c8bf19 1, 5082, 5082, 10, 2013, 0xabbbade8 1, 5092, 5092, 10, 2013, 0xf8b69d89 1, 5102, 5102, 10, 2013, 0xce04a866 -0, 153, 153, 1, 518400, 0x7303f77b +0, 153, 153, 1, 518400, 0xd14dc25b 1, 5112, 5112, 10, 2013, 0x07528abf 1, 5122, 5122, 10, 2013, 0x74fb98bf 1, 5132, 5132, 10, 2013, 0x579fb1c9 -0, 154, 154, 1, 518400, 0x22b0513f +0, 154, 154, 1, 518400, 0x8c5e0503 1, 5147, 5147, 10, 2013, 0x7ddea2ed 1, 5157, 5157, 10, 2013, 0x296caa2c 1, 5167, 5167, 10, 2013, 0x346d9c4f -0, 155, 155, 1, 518400, 0x330485d2 +0, 155, 155, 1, 518400, 0x854e20f7 1, 5177, 5177, 10, 2013, 0x3e1fba15 1, 5187, 5187, 10, 2013, 0x48a2908f 1, 5197, 5197, 10, 2013, 0xc1938d09 -0, 156, 156, 1, 518400, 0x7f83daea +0, 156, 156, 1, 518400, 0x9a876ae7 1, 5207, 5207, 10, 2013, 0x0e96a060 1, 5217, 5217, 10, 2013, 0x7b6a9e06 1, 5232, 5232, 10, 2013, 0x5b779d28 -0, 157, 157, 1, 518400, 0xee19f2df +0, 157, 157, 1, 518400, 0xccf16959 1, 5242, 5242, 10, 2013, 0xf600aca1 1, 5252, 5252, 10, 2013, 0x3a6c9e68 1, 5262, 5262, 10, 2013, 0x0c8dc1b0 -0, 158, 158, 1, 518400, 0xb71b1c77 +0, 158, 158, 1, 518400, 0x64807453 1, 5272, 5272, 10, 2013, 0x26beb245 1, 5282, 5282, 10, 2013, 0x2bc09557 1, 5292, 5292, 10, 2013, 0x27fc8845 1, 5302, 5302, 10, 2013, 0x1025aa47 -0, 159, 159, 1, 518400, 0xbffc1856 +0, 159, 159, 1, 518400, 0xe6285a78 1, 5318, 5318, 10, 2013, 0xc2e69baa 1, 5328, 5328, 10, 2013, 0xdb249b92 1, 5338, 5338, 10, 2013, 0x6ccda29e -0, 160, 160, 1, 518400, 0xabc125aa +0, 160, 160, 1, 518400, 0xf8f353dc 1, 5348, 5348, 10, 2013, 0xeaf6a1cf 1, 5358, 5358, 10, 2013, 0x509ba397 1, 5368, 5368, 10, 2013, 0xfaf8a2df -0, 161, 161, 1, 518400, 0x5ee467f8 +0, 161, 161, 1, 518400, 0x12b87c38 1, 5378, 5378, 10, 2013, 0x41388f28 1, 5388, 5388, 10, 2013, 0xfe5eab39 1, 5403, 5403, 10, 2013, 0xd5ffa066 -0, 162, 162, 1, 518400, 0x6c2cf168 +0, 162, 162, 1, 518400, 0x845ef12d 1, 5413, 5413, 10, 2013, 0x6813a30a 1, 5423, 5423, 10, 2013, 0x9be89718 1, 5433, 5433, 10, 2013, 0xaec3a27b -0, 163, 163, 1, 518400, 0x63996b26 +0, 163, 163, 1, 518400, 0x01255194 1, 5446, 5446, 10, 2013, 0x579a983e 1, 5456, 5456, 10, 2013, 0x98cea21f 1, 5466, 5466, 10, 2013, 0xca77a58a -0, 164, 164, 1, 518400, 0xb34d789a +0, 164, 164, 1, 518400, 0x3e103dd6 1, 5476, 5476, 10, 2013, 0xcbc3b1ee 1, 5486, 5486, 10, 2013, 0xf3bb8f07 1, 5496, 5496, 10, 2013, 0x6aeebd92 -0, 165, 165, 1, 518400, 0xf49c030f +0, 165, 165, 1, 518400, 0xf47bafa0 1, 5506, 5506, 10, 2013, 0xe955a449 1, 5516, 5516, 10, 2013, 0x9436aa5b 1, 5531, 5531, 10, 2013, 0x4f0a8f9f -0, 166, 166, 1, 518400, 0x092dc41a +0, 166, 166, 1, 518400, 0x4ce46108 1, 5541, 5541, 10, 2013, 0x3551b22d 1, 5551, 5551, 10, 2013, 0x0959a3d4 1, 5561, 5561, 10, 2013, 0x2ed5a11b 1, 5571, 5571, 10, 2013, 0x8f52a5c3 -0, 167, 167, 1, 518400, 0x4134c577 +0, 167, 167, 1, 518400, 0x27ec4f32 1, 5581, 5581, 10, 2013, 0x6552978d 1, 5591, 5591, 10, 2013, 0x7dcca0c1 1, 5601, 5601, 10, 2013, 0xbcd4a3c9 -0, 168, 168, 1, 518400, 0x261de1ed +0, 168, 168, 1, 518400, 0xe0bc547a 1, 5616, 5616, 10, 2013, 0xfe41a8d8 1, 5626, 5626, 10, 2013, 0xc85aae14 1, 5636, 5636, 10, 2013, 0x1185b346 -0, 169, 169, 1, 518400, 0xcbc8566a +0, 169, 169, 1, 518400, 0xc36fc0c8 1, 5646, 5646, 10, 2013, 0xf7429a0d 1, 5656, 5656, 10, 2013, 0x48c2a160 1, 5666, 5666, 10, 2013, 0x9d85a85d -0, 170, 170, 1, 518400, 0x407a5c76 +0, 170, 170, 1, 518400, 0x34b5b437 1, 5676, 5676, 10, 2013, 0xbbe89fe9 1, 5686, 5686, 10, 2013, 0xea429fe2 1, 5702, 5702, 10, 2013, 0x221ca1d4 -0, 171, 171, 1, 518400, 0x1ed73bb2 +0, 171, 171, 1, 518400, 0x699e770b 1, 5712, 5712, 10, 2013, 0x394b925b 1, 5722, 5722, 10, 2013, 0x556dc26f 1, 5732, 5732, 10, 2013, 0xce21a5e1 -0, 172, 172, 1, 518400, 0x8467ddb5 +0, 172, 172, 1, 518400, 0xf5810fdc 1, 5742, 5742, 10, 2013, 0xbc87c0a8 1, 5752, 5752, 10, 2013, 0xbac4ac07 1, 5762, 5762, 10, 2013, 0xdeefa4aa 1, 5772, 5772, 10, 2013, 0x1f15b362 -0, 173, 173, 1, 518400, 0x0523dc73 +0, 173, 173, 1, 518400, 0xe345f768 1, 5787, 5787, 10, 2013, 0x6406b7b2 1, 5797, 5797, 10, 2013, 0x8030a03d -0, 174, 174, 1, 518400, 0x81f5e895 +0, 174, 174, 1, 518400, 0x3f4fe36a 1, 5807, 5807, 10, 2013, 0x0373a5b1 1, 5817, 5817, 10, 2013, 0x34ef93da 1, 5827, 5827, 10, 2013, 0x94c198fe 1, 5837, 5837, 10, 2013, 0xfefcabad -0, 175, 175, 1, 518400, 0xfc74608d +0, 175, 175, 1, 518400, 0x1a2d53bb 1, 5847, 5847, 10, 2013, 0x8755b3ec 1, 5857, 5857, 10, 2013, 0xe436a6fd 1, 5872, 5872, 10, 2013, 0x9cf5a11e -0, 176, 176, 1, 518400, 0xc4e0dae0 +0, 176, 176, 1, 518400, 0x7f31b326 1, 5882, 5882, 10, 2013, 0x03b8a98c 1, 5892, 5892, 10, 2013, 0x6216a138 1, 5902, 5902, 10, 2013, 0xd87b9f12 -0, 177, 177, 1, 518400, 0x98367f5b +0, 177, 177, 1, 518400, 0xc0a53fae 1, 5912, 5912, 10, 2013, 0x4ce99653 1, 5922, 5922, 10, 2013, 0x6c2ea9e2 1, 5932, 5932, 10, 2013, 0x918cae4c -0, 178, 178, 1, 518400, 0x0f1a869d +0, 178, 178, 1, 518400, 0xb58e38ee 1, 5942, 5942, 10, 2013, 0xd19fa5f2 1, 5958, 5958, 10, 2013, 0x0bdda7c6 1, 5968, 5968, 10, 2013, 0x0f9ab0ca -0, 179, 179, 1, 518400, 0x45b6ccf2 +0, 179, 179, 1, 518400, 0x920a6b68 1, 5978, 5978, 10, 2013, 0x410a92b1 1, 5988, 5988, 10, 2013, 0xcfbe9d1c 1, 5998, 5998, 10, 2013, 0x59ed9d15 -0, 180, 180, 1, 518400, 0x5f9ccb77 +0, 180, 180, 1, 518400, 0xcc2f5945 1, 6008, 6008, 10, 2013, 0x4e129e27 1, 6018, 6018, 10, 2013, 0x7bb9ac0a 1, 6028, 6028, 10, 2013, 0x826ca82b -0, 181, 181, 1, 518400, 0x5f15ea31 +0, 181, 181, 1, 518400, 0x939b6cbc 1, 6043, 6043, 10, 2013, 0x9ad5a74b 1, 6053, 6053, 10, 2013, 0x6c5f969a 1, 6063, 6063, 10, 2013, 0x8479a0e5 -0, 182, 182, 1, 518400, 0x86369f27 +0, 182, 182, 1, 518400, 0xbe7c0d58 1, 6073, 6073, 10, 2013, 0x165298ef 1, 6083, 6083, 10, 2013, 0xdcadb4a1 1, 6093, 6093, 10, 2013, 0xa90e987c 1, 6103, 6103, 10, 2013, 0x1ac5b510 -0, 183, 183, 1, 518400, 0x2e27f9fa +0, 183, 183, 1, 518400, 0x872c53f9 1, 6113, 6113, 10, 2013, 0x66728d85 1, 6128, 6128, 10, 2013, 0xe4859fc5 1, 6138, 6138, 10, 2013, 0x9901786e -0, 184, 184, 1, 518400, 0xc029a44d +0, 184, 184, 1, 518400, 0xef85f3e1 1, 6148, 6148, 10, 2013, 0x6aebb406 1, 6158, 6158, 10, 2013, 0x7d13a2cc 1, 6168, 6168, 10, 2013, 0x99b7a8cc -0, 185, 185, 1, 518400, 0xebee33b0 +0, 185, 185, 1, 518400, 0xbdbc70ff 1, 6178, 6178, 10, 2013, 0x80b8a624 1, 6188, 6188, 10, 2013, 0xbb6aa271 1, 6198, 6198, 10, 2013, 0x17af9e4a -0, 186, 186, 1, 518400, 0x19e5494f +0, 186, 186, 1, 518400, 0x16ad73b4 1, 6214, 6214, 10, 2013, 0xfaf0a8f1 1, 6224, 6224, 10, 2013, 0xd6849b93 1, 6234, 6234, 10, 2013, 0xe9829669 -0, 187, 187, 1, 518400, 0xf697bd7c +0, 187, 187, 1, 518400, 0x5356dbec 1, 6244, 6244, 10, 2013, 0x7ec98944 1, 6254, 6254, 10, 2013, 0x2b2099a4 1, 6264, 6264, 10, 2013, 0x1033a82f -0, 188, 188, 1, 518400, 0x82569002 +0, 188, 188, 1, 518400, 0xad4da4ed 1, 6274, 6274, 10, 2013, 0x5ec88990 1, 6284, 6284, 10, 2013, 0xd2a19b3d 1, 6299, 6299, 10, 2013, 0xa377b268 -0, 189, 189, 1, 518400, 0xfcb6d707 +0, 189, 189, 1, 518400, 0x06fbdb66 1, 6309, 6309, 10, 2013, 0xfa859901 1, 6319, 6319, 10, 2013, 0x1713955a 1, 6329, 6329, 10, 2013, 0x70aab0da 1, 6339, 6339, 10, 2013, 0xcdaea422 -0, 190, 190, 1, 518400, 0x82a9662b +0, 190, 190, 1, 518400, 0xdf0d6390 1, 6349, 6349, 10, 2013, 0x65c3bf80 1, 6359, 6359, 10, 2013, 0x1d75a55f 1, 6369, 6369, 10, 2013, 0xa5bea4de -0, 191, 191, 1, 518400, 0x212e16ee +0, 191, 191, 1, 518400, 0xbf4207ac 1, 6384, 6384, 10, 2013, 0x184db71c 1, 6394, 6394, 10, 2013, 0x99858ec8 1, 6404, 6404, 10, 2013, 0xb8f2aee5 -0, 192, 192, 1, 518400, 0x2ca34dca +0, 192, 192, 1, 518400, 0x119531ed 1, 6414, 6414, 10, 2013, 0x4435b2ef 1, 6424, 6424, 10, 2013, 0x8acfa6c7 1, 6434, 6434, 10, 2013, 0x42b4c01f -0, 193, 193, 1, 518400, 0xe9ebe0a5 +0, 193, 193, 1, 518400, 0xc160bca8 1, 6444, 6444, 10, 2013, 0x6e308c13 1, 6454, 6454, 10, 2013, 0x8227a0f6 1, 6470, 6470, 10, 2013, 0x6f12a7a2 -0, 194, 194, 1, 518400, 0x4e6b6917 +0, 194, 194, 1, 518400, 0xcd9b3639 1, 6480, 6480, 10, 2013, 0x785392be 1, 6490, 6490, 10, 2013, 0x81849c2b 1, 6500, 6500, 10, 2013, 0x5cf2af65 -0, 195, 195, 1, 518400, 0x7dcf20ab +0, 195, 195, 1, 518400, 0x7c4ad83a 1, 6510, 6510, 10, 2013, 0x0c6ca6b4 1, 6520, 6520, 10, 2013, 0x412fab9f 1, 6530, 6530, 10, 2013, 0x08e792b4 -0, 196, 196, 1, 518400, 0xf30fac97 +0, 196, 196, 1, 518400, 0xe3255ff4 1, 6540, 6540, 10, 2013, 0x407aace3 1, 6555, 6555, 10, 2013, 0xd26bac16 1, 6565, 6565, 10, 2013, 0xac8bb295 -0, 197, 197, 1, 518400, 0xcb9fc692 +0, 197, 197, 1, 518400, 0x257a6758 1, 6575, 6575, 10, 2013, 0xddd1949c 1, 6585, 6585, 10, 2013, 0x6b26b868 1, 6595, 6595, 10, 2013, 0x5eaba587 1, 6605, 6605, 10, 2013, 0xef0793b9 -0, 198, 198, 1, 518400, 0x5d05601e +0, 198, 198, 1, 518400, 0xcfa4f4e0 1, 6615, 6615, 10, 2013, 0xdef19bd6 1, 6625, 6625, 10, 2013, 0xca98a635 -0, 199, 199, 1, 518400, 0x456c1417 +0, 199, 199, 1, 518400, 0x5f159eff 1, 6640, 6640, 10, 2013, 0x06269a5a 1, 6650, 6650, 10, 2013, 0x32cb9952 1, 6660, 6660, 10, 2013, 0xf01fa95a 1, 6670, 6670, 10, 2013, 0xefab9e55 -0, 200, 200, 1, 518400, 0x9a0fd1ad +0, 200, 200, 1, 518400, 0x31d257bc 1, 6680, 6680, 10, 2013, 0x55a3b63a 1, 6690, 6690, 10, 2013, 0xcd36a553 1, 6700, 6700, 10, 2013, 0x2ec19877 -0, 201, 201, 1, 518400, 0x55db9716 +0, 201, 201, 1, 518400, 0x81c516b9 1, 6710, 6710, 10, 2013, 0xc18b924c 1, 6726, 6726, 10, 2013, 0xf132b04c 1, 6736, 6736, 10, 2013, 0x7975a44d -0, 202, 202, 1, 518400, 0x1f0d40d6 +0, 202, 202, 1, 518400, 0x56b7bf87 1, 6746, 6746, 10, 2013, 0x2aaf94cb 1, 6756, 6756, 10, 2013, 0x58cfa60f 1, 6766, 6766, 10, 2013, 0x9757a658 -0, 203, 203, 1, 518400, 0x73695c82 +0, 203, 203, 1, 518400, 0x4f83dd2e 1, 6776, 6776, 10, 2013, 0x67ebc0d5 1, 6786, 6786, 10, 2013, 0x3c50a70e 1, 6796, 6796, 10, 2013, 0x9c5799c6 -0, 204, 204, 1, 518400, 0xb0f10812 +0, 204, 204, 1, 518400, 0x0d4389d7 1, 6811, 6811, 10, 2013, 0x018d85b2 1, 6821, 6821, 10, 2013, 0x5367a956 -0, 205, 205, 1, 518400, 0xdec18505 -0, 208, 208, 1, 518400, 0xb147b947 -0, 240, 240, 1, 518400, 0x9d2e3977 +0, 205, 205, 1, 518400, 0x375b056f +0, 208, 208, 1, 518400, 0x9d2e3977 diff --git a/tests/ref/fate/sub-dvb b/tests/ref/fate/sub-dvb index cbd1801d64..8f33c75d70 100644 --- a/tests/ref/fate/sub-dvb +++ b/tests/ref/fate/sub-dvb @@ -1,75 +1,93 @@ #tb 0: 1/1000000 #media_type 0: subtitle #codec_id 0: dvb_subtitle -0, 15600000, 15600000, 159000, 1168, 0xd0f89d82 -0, 15759000, 15759000, 159000, 14, 0x064900eb -0, 15760000, 15760000, 239000, 1544, 0xe60f1751 -0, 15999000, 15999000, 239000, 14, 0x0729010b -0, 16000000, 16000000, 339000, 1658, 0xbe343093 -0, 16339000, 16339000, 339000, 14, 0x0809012b -0, 16340000, 16340000, 599000, 2343, 0xc68f07ef -0, 16939000, 16939000, 599000, 14, 0x08e9014b -0, 16940000, 16940000, 459000, 2568, 0x0ee657b1 -0, 17399000, 17399000, 459000, 14, 0x09c9016b -0, 17400000, 17400000, 359000, 3422, 0xba5b63ce -0, 17759000, 17759000, 359000, 14, 0x0aa9018b -0, 17760000, 17760000, 219000, 5078, 0x95b19902 -0, 17979000, 17979000, 219000, 14, 0x0b8901ab -0, 17980000, 17980000, 959000, 5808, 0xc9717b89 -0, 18939000, 18939000, 959000, 14, 0x0c6901cb -0, 18940000, 18940000, 219000, 6015, 0x0becbfac -0, 19159000, 19159000, 219000, 14, 0x064900eb -0, 19160000, 19160000, 259000, 6519, 0xfcd24d26 -0, 19419000, 19419000, 259000, 14, 0x0729010b -0, 19420000, 19420000, 99000, 7061, 0xf0320408 -0, 19519000, 19519000, 99000, 14, 0x0809012b -0, 19520000, 19520000, 219000, 4773, 0x66c93074 -0, 19739000, 19739000, 219000, 14, 0x08e9014b -0, 19740000, 19740000, 219000, 5546, 0x06052c81 -0, 19959000, 19959000, 219000, 14, 0x09c9016b -0, 19960000, 19960000, 239000, 5754, 0x904f7325 -0, 20199000, 20199000, 239000, 14, 0x0aa9018b -0, 20200000, 20200000, 139000, 6099, 0xe30cde07 -0, 20339000, 20339000, 139000, 14, 0x0b8901ab -0, 20340000, 20340000, 799000, 6839, 0x770fcb6c -0, 21139000, 21139000, 799000, 14, 0x0c6901cb -0, 21140000, 21140000, 239000, 4744, 0xa91e1b41 -0, 21379000, 21379000, 239000, 14, 0x064900eb -0, 21380000, 21380000, 339000, 5824, 0xcf6d782b -0, 21719000, 21719000, 339000, 14, 0x0729010b -0, 21720000, 21720000, 1439000, 6212, 0xabf8f7cf -0, 23159000, 23159000, 1439000, 14, 0x0809012b -0, 23160000, 23160000, 1319000, 7082, 0xd7ca10f2 -0, 24479000, 24479000, 1319000, 14, 0x08e9014b -0, 24480000, 24480000, 219000, 5345, 0x12b2cae0 -0, 24699000, 24699000, 219000, 14, 0x09c9016b -0, 24700000, 24700000, 219000, 5765, 0xc7d46192 -0, 24919000, 24919000, 219000, 14, 0x0aa9018b -0, 24920000, 24920000, 599000, 6557, 0xcb995d30 -0, 25519000, 25519000, 599000, 14, 0x0b8901ab -0, 25520000, 25520000, 219000, 7091, 0xe6ea0559 -0, 25739000, 25739000, 219000, 14, 0x0c6901cb -0, 25740000, 25740000, 239000, 7305, 0xb66c404e -0, 25979000, 25979000, 239000, 14, 0x064900eb -0, 25980000, 25980000, 359000, 7590, 0x0cc2a481 -0, 26339000, 26339000, 359000, 14, 0x0729010b -0, 26340000, 26340000, 219000, 4629, 0xe18cfea8 -0, 26559000, 26559000, 219000, 14, 0x0809012b -0, 26560000, 26560000, 719000, 4785, 0x82043fc0 -0, 27279000, 27279000, 719000, 14, 0x08e9014b -0, 27280000, 27280000, 459000, 6061, 0xbde7d245 -0, 27739000, 27739000, 459000, 14, 0x09c9016b -0, 27740000, 27740000, 239000, 6301, 0x92d01a51 -0, 27979000, 27979000, 239000, 14, 0x0aa9018b -0, 27980000, 27980000, 99000, 6736, 0xbd25a134 -0, 28079000, 28079000, 99000, 14, 0x0b8901ab -0, 28080000, 28080000, 219000, 7214, 0x7ef93c13 -0, 28299000, 28299000, 219000, 14, 0x0c6901cb -0, 28300000, 28300000, 239000, 7366, 0x5bed7fcd -0, 28539000, 28539000, 239000, 14, 0x064900eb -0, 28540000, 28540000, 599000, 4564, 0x7f4c014b -0, 29139000, 29139000, 599000, 14, 0x0729010b -0, 29140000, 29140000, 219000, 4637, 0x682626b7 -0, 29359000, 29359000, 219000, 14, 0x0809012b -0, 29360000, 29360000, 1679000, 5358, 0x29e30c48 -0, 31039000, 31039000, 1679000, 14, 0x08e9014b +0, 0, 0, 279000, 14, 0x05d900db +0, 279000, 279000, 279000, 14, 0x064900eb +0, 280000, 280000, 4999000, 14, 0x06b900fb +0, 5279000, 5279000, 4999000, 14, 0x0729010b +0, 5280000, 5280000, 5019000, 14, 0x0799011b +0, 10299000, 10299000, 5019000, 14, 0x0809012b +0, 10300000, 10300000, 3599000, 14, 0x0879013b +0, 13899000, 13899000, 3599000, 14, 0x08e9014b +0, 13900000, 13900000, 219000, 14, 0x0959015b +0, 14119000, 14119000, 219000, 14, 0x09c9016b +0, 14120000, 14120000, 1439000, 14, 0x0a39017b +0, 15559000, 15559000, 1439000, 14, 0x0aa9018b +0, 15560000, 15560000, 39000, 14, 0x0b19019b +0, 15599000, 15599000, 39000, 14, 0x0b8901ab +0, 15600000, 15600000, 159000, 1168, 0xd69da022 +0, 15759000, 15759000, 159000, 14, 0x0c6901cb +0, 15760000, 15760000, 239000, 1544, 0xc5f116f1 +0, 15999000, 15999000, 239000, 14, 0x064900eb +0, 16000000, 16000000, 339000, 1658, 0x73563033 +0, 16339000, 16339000, 339000, 14, 0x0729010b +0, 16340000, 16340000, 599000, 2343, 0x7ac2078f +0, 16939000, 16939000, 599000, 14, 0x0809012b +0, 16940000, 16940000, 459000, 2568, 0x6eaa5751 +0, 17399000, 17399000, 459000, 14, 0x08e9014b +0, 17400000, 17400000, 359000, 3422, 0xd9d0636e +0, 17759000, 17759000, 359000, 14, 0x09c9016b +0, 17760000, 17760000, 219000, 5078, 0x722c9862 +0, 17979000, 17979000, 219000, 14, 0x0aa9018b +0, 17980000, 17980000, 959000, 5808, 0x38dd7ae9 +0, 18939000, 18939000, 959000, 14, 0x0b8901ab +0, 18940000, 18940000, 219000, 6015, 0xd4d2c40c +0, 19159000, 19159000, 219000, 14, 0x0c6901cb +0, 19160000, 19160000, 259000, 6519, 0x08af4c86 +0, 19419000, 19419000, 259000, 14, 0x064900eb +0, 19420000, 19420000, 99000, 7061, 0xecf10368 +0, 19519000, 19519000, 99000, 14, 0x0729010b +0, 19520000, 19520000, 219000, 4773, 0xbee42fd4 +0, 19739000, 19739000, 219000, 14, 0x0809012b +0, 19740000, 19740000, 219000, 5546, 0xdb822be1 +0, 19959000, 19959000, 219000, 14, 0x08e9014b +0, 19960000, 19960000, 239000, 5754, 0xfdcc7285 +0, 20199000, 20199000, 239000, 14, 0x09c9016b +0, 20200000, 20200000, 139000, 6099, 0xa409dd67 +0, 20339000, 20339000, 139000, 14, 0x0aa9018b +0, 20340000, 20340000, 799000, 6839, 0xc5eecacc +0, 21139000, 21139000, 799000, 14, 0x0b8901ab +0, 21140000, 21140000, 239000, 4744, 0x4e451fa1 +0, 21379000, 21379000, 239000, 14, 0x0c6901cb +0, 21380000, 21380000, 339000, 5824, 0x5299778b +0, 21719000, 21719000, 339000, 14, 0x064900eb +0, 21720000, 21720000, 1439000, 6212, 0x6d15f72f +0, 23159000, 23159000, 1439000, 14, 0x0729010b +0, 23160000, 23160000, 1319000, 7082, 0xe5c91052 +0, 24479000, 24479000, 1319000, 14, 0x0809012b +0, 24480000, 24480000, 219000, 5345, 0x2e5eca40 +0, 24699000, 24699000, 219000, 14, 0x08e9014b +0, 24700000, 24700000, 219000, 5765, 0x118060f2 +0, 24919000, 24919000, 219000, 14, 0x09c9016b +0, 24920000, 24920000, 599000, 6557, 0x89275c90 +0, 25519000, 25519000, 599000, 14, 0x0aa9018b +0, 25520000, 25520000, 219000, 7091, 0x996904b9 +0, 25739000, 25739000, 219000, 14, 0x0b8901ab +0, 25740000, 25740000, 239000, 7305, 0xc23e44ae +0, 25979000, 25979000, 239000, 14, 0x0c6901cb +0, 25980000, 25980000, 359000, 7590, 0xc5a3a3e1 +0, 26339000, 26339000, 359000, 14, 0x064900eb +0, 26340000, 26340000, 219000, 4629, 0x7ad6fe08 +0, 26559000, 26559000, 219000, 14, 0x0729010b +0, 26560000, 26560000, 719000, 4785, 0xcd3f3f20 +0, 27279000, 27279000, 719000, 14, 0x0809012b +0, 27280000, 27280000, 459000, 6061, 0x8b04d1a5 +0, 27739000, 27739000, 459000, 14, 0x08e9014b +0, 27740000, 27740000, 239000, 6301, 0xe7de19b1 +0, 27979000, 27979000, 239000, 14, 0x09c9016b +0, 27980000, 27980000, 99000, 6736, 0x38b3a094 +0, 28079000, 28079000, 99000, 14, 0x0aa9018b +0, 28080000, 28080000, 219000, 7214, 0x0b783b73 +0, 28299000, 28299000, 219000, 14, 0x0b8901ab +0, 28300000, 28300000, 239000, 7366, 0x98bf842d +0, 28539000, 28539000, 239000, 14, 0x0c6901cb +0, 28540000, 28540000, 599000, 4564, 0x3d9600ab +0, 29139000, 29139000, 599000, 14, 0x064900eb +0, 29140000, 29140000, 219000, 4637, 0x01f02617 +0, 29359000, 29359000, 219000, 14, 0x0729010b +0, 29360000, 29360000, 1679000, 5358, 0x5b0f0ba8 +0, 31039000, 31039000, 1679000, 14, 0x0809012b +0, 31040000, 31040000, 359000, 14, 0x0879013b +0, 31399000, 31399000, 359000, 14, 0x08e9014b +0, 31400000, 31400000, 479000, 14, 0x0959015b +0, 31879000, 31879000, 479000, 14, 0x09c9016b diff --git a/tests/ref/fate/sub2video b/tests/ref/fate/sub2video index 80abe9c905..f2af5e7d1c 100644 --- a/tests/ref/fate/sub2video +++ b/tests/ref/fate/sub2video @@ -10,32 +10,31 @@ 0, 0, 0, 1, 518400, 0x83c27b82 0, 1, 1, 1, 518400, 0x4051c7f9 0, 2, 2, 1, 518400, 0xfb00e17e -1, 499000, 499000, 4960000, 1015, 0x19e092d2 -0, 3, 3, 1, 518400, 0x192abb74 -0, 4, 4, 1, 518400, 0x4669a88b -0, 5, 5, 1, 518400, 0xaababe00 -0, 6, 6, 1, 518400, 0x98a211a5 -0, 7, 7, 1, 518400, 0x440e7547 -0, 8, 8, 1, 518400, 0xca5bb496 -0, 9, 9, 1, 518400, 0xf86e0b0a -0, 10, 10, 1, 518400, 0xb80fa020 -0, 11, 11, 1, 518400, 0x41c2a54b -0, 12, 12, 1, 518400, 0x51baf353 -0, 13, 13, 1, 518400, 0x967ea7f3 -0, 14, 14, 1, 518400, 0x819e7f89 -0, 15, 15, 1, 518400, 0x192233e1 -0, 16, 16, 1, 518400, 0xc80a0eb3 -0, 17, 17, 1, 518400, 0x08260a23 -0, 18, 18, 1, 518400, 0xc92e2caf -0, 19, 19, 1, 518400, 0x3fe36eea -0, 20, 20, 1, 518400, 0x0891e8d5 -0, 21, 21, 1, 518400, 0x84655095 -0, 22, 22, 1, 518400, 0x9c7fa014 -0, 23, 23, 1, 518400, 0x9c43b656 -0, 24, 24, 1, 518400, 0x2cf46221 -0, 25, 25, 1, 518400, 0x7322e11c -0, 26, 26, 1, 518400, 0x45af1a84 -0, 27, 27, 1, 518400, 0x7b781071 +0, 3, 3, 1, 518400, 0x17f5f77c +0, 4, 4, 1, 518400, 0x427fe2ce +0, 5, 5, 1, 518400, 0x76d8fe7b +0, 6, 6, 1, 518400, 0x167c5c28 +0, 7, 7, 1, 518400, 0x1d54c98e +0, 8, 8, 1, 518400, 0x664016a4 +0, 9, 9, 1, 518400, 0x4a477278 +0, 10, 10, 1, 518400, 0xc3beffb4 +0, 11, 11, 1, 518400, 0xf052f750 +0, 12, 12, 1, 518400, 0xcd9e399a +0, 13, 13, 1, 518400, 0x1843d70b +0, 14, 14, 1, 518400, 0x4737acf9 +0, 15, 15, 1, 518400, 0x3a9961a0 +0, 16, 16, 1, 518400, 0x2dce2c74 +0, 17, 17, 1, 518400, 0x1d601ddc +0, 18, 18, 1, 518400, 0xb47429b1 +0, 19, 19, 1, 518400, 0xfab24433 +0, 20, 20, 1, 518400, 0xd69d8e91 +0, 21, 21, 1, 518400, 0xc8b8ce57 +0, 22, 22, 1, 518400, 0xf6a90e9c +0, 23, 23, 1, 518400, 0x11cc1ec4 +0, 24, 24, 1, 518400, 0xcad7cd01 +0, 25, 25, 1, 518400, 0x01596125 +0, 26, 26, 1, 518400, 0xc2b7aca0 +0, 27, 27, 1, 518400, 0x8b2d9fde 0, 28, 28, 1, 518400, 0x4f7c706c 0, 29, 29, 1, 518400, 0xb227603b 0, 30, 30, 1, 518400, 0x7b4b89c2 @@ -58,129 +57,86 @@ 0, 47, 47, 1, 518400, 0xde69683f 0, 48, 48, 1, 518400, 0x7df08fba 0, 49, 49, 1, 518400, 0xbab197ea -1, 15355000, 15355000, 4733000, 2094, 0x3c171425 -0, 77, 77, 1, 518400, 0x902285d9 +0, 77, 77, 1, 518400, 0xbab197ea 0, 100, 100, 1, 518400, 0xbab197ea -1, 48797000, 48797000, 2560000, 2480, 0x7c0edf21 -0, 244, 244, 1, 518400, 0x7a11c812 +0, 244, 244, 1, 518400, 0xbab197ea 0, 257, 257, 1, 518400, 0xbab197ea -1, 51433000, 51433000, 2366000, 3059, 0xc95b8a05 -0, 258, 258, 1, 518400, 0x34cdddee +0, 258, 258, 1, 518400, 0xbab197ea 0, 269, 269, 1, 518400, 0xbab197ea -1, 53910000, 53910000, 2696000, 2095, 0x61bb15ed -0, 270, 270, 1, 518400, 0x4db4ce51 +0, 270, 270, 1, 518400, 0xbab197ea 0, 283, 283, 1, 518400, 0xbab197ea -1, 56663000, 56663000, 1262000, 1013, 0xc9ae89b7 -0, 284, 284, 1, 518400, 0xe6bc0ea9 +0, 284, 284, 1, 518400, 0xbab197ea 0, 290, 290, 1, 518400, 0xbab197ea -1, 58014000, 58014000, 1661000, 969, 0xe01878f0 -0, 291, 291, 1, 518400, 0xa8643af7 +0, 291, 291, 1, 518400, 0xbab197ea 0, 298, 298, 1, 518400, 0xbab197ea -1, 67724000, 67724000, 1365000, 844, 0xe7db4fc1 -0, 339, 339, 1, 518400, 0xb1885c67 +0, 339, 339, 1, 518400, 0xbab197ea 0, 345, 345, 1, 518400, 0xbab197ea -1, 69175000, 69175000, 1558000, 802, 0xf48531ba -0, 346, 346, 1, 518400, 0x378e3fd0 +0, 346, 346, 1, 518400, 0xbab197ea 0, 354, 354, 1, 518400, 0xbab197ea -1, 70819000, 70819000, 1865000, 1709, 0xb4d5a1bd -0, 355, 355, 1, 518400, 0xa3782469 +0, 355, 355, 1, 518400, 0xbab197ea 0, 363, 363, 1, 518400, 0xbab197ea -1, 72762000, 72762000, 1968000, 2438, 0x99d7bc82 -0, 364, 364, 1, 518400, 0xba23a0d5 +0, 364, 364, 1, 518400, 0xbab197ea 0, 374, 374, 1, 518400, 0xbab197ea -1, 74806000, 74806000, 1831000, 2116, 0x96514097 -0, 375, 375, 1, 518400, 0x129de2f8 +0, 375, 375, 1, 518400, 0xbab197ea 0, 383, 383, 1, 518400, 0xbab197ea -1, 76716000, 76716000, 1262000, 1822, 0xefccc72e -0, 384, 384, 1, 518400, 0x19772f0f +0, 384, 384, 1, 518400, 0xbab197ea 0, 390, 390, 1, 518400, 0xbab197ea -1, 78051000, 78051000, 1524000, 987, 0x7b927a27 -0, 391, 391, 1, 518400, 0x56f54e73 +0, 391, 391, 1, 518400, 0xbab197ea 0, 398, 398, 1, 518400, 0xbab197ea -1, 79644000, 79644000, 2662000, 2956, 0x190778f7 -0, 399, 399, 1, 518400, 0x300b5247 -1, 82380000, 82380000, 2764000, 3094, 0xc021b7d3 +0, 399, 399, 1, 518400, 0xbab197ea 0, 412, 412, 1, 518400, 0xbab197ea -0, 413, 413, 1, 518400, 0x6fd028fa +0, 413, 413, 1, 518400, 0xbab197ea 0, 426, 426, 1, 518400, 0xbab197ea -1, 85225000, 85225000, 2366000, 2585, 0x74d0048f -0, 427, 427, 1, 518400, 0x01f80e9d +0, 427, 427, 1, 518400, 0xbab197ea 0, 438, 438, 1, 518400, 0xbab197ea -1, 87652000, 87652000, 1831000, 634, 0x8832fda1 -0, 439, 439, 1, 518400, 0xb48d90c0 +0, 439, 439, 1, 518400, 0xbab197ea 0, 447, 447, 1, 518400, 0xbab197ea -1, 91531000, 91531000, 2332000, 2080, 0x97a1146f -0, 458, 458, 1, 518400, 0xcb5a0173 +0, 458, 458, 1, 518400, 0xbab197ea 0, 469, 469, 1, 518400, 0xbab197ea -1, 95510000, 95510000, 3299000, 2964, 0x8b8f6684 -0, 478, 478, 1, 518400, 0xb8a323e4 +0, 478, 478, 1, 518400, 0xbab197ea 0, 494, 494, 1, 518400, 0xbab197ea -1, 98872000, 98872000, 2161000, 1875, 0x9002ef71 -0, 495, 495, 1, 518400, 0xc43518ba +0, 495, 495, 1, 518400, 0xbab197ea 0, 505, 505, 1, 518400, 0xbab197ea -1, 101124000, 101124000, 4096000, 3872, 0x20c6ed9c -0, 506, 506, 1, 518400, 0x04e38692 +0, 506, 506, 1, 518400, 0xbab197ea 0, 526, 526, 1, 518400, 0xbab197ea -1, 105303000, 105303000, 2730000, 3094, 0xf203a663 -0, 527, 527, 1, 518400, 0x856b0ee5 +0, 527, 527, 1, 518400, 0xbab197ea 0, 540, 540, 1, 518400, 0xbab197ea -1, 108106000, 108106000, 2059000, 2404, 0x41a7b429 -0, 541, 541, 1, 518400, 0x3e5beee2 +0, 541, 541, 1, 518400, 0xbab197ea 0, 551, 551, 1, 518400, 0xbab197ea -1, 141556000, 141556000, 1661000, 1088, 0xde20aa20 -0, 708, 708, 1, 518400, 0xb8bc1365 +0, 708, 708, 1, 518400, 0xbab197ea 0, 716, 716, 1, 518400, 0xbab197ea -0, 817, 817, 1, 518400, 0x83efa32d -1, 163445000, 163445000, 1331000, 339, 0x8bd186ef +0, 817, 817, 1, 518400, 0xbab197ea 0, 824, 824, 1, 518400, 0xbab197ea -0, 840, 840, 1, 518400, 0x03ea0e90 -1, 168049000, 168049000, 1900000, 1312, 0x0bf20e8d +0, 840, 840, 1, 518400, 0xbab197ea 0, 850, 850, 1, 518400, 0xbab197ea -1, 170035000, 170035000, 1524000, 1279, 0xb6c2dafe -0, 851, 851, 1, 518400, 0x8780239e +0, 851, 851, 1, 518400, 0xbab197ea 0, 858, 858, 1, 518400, 0xbab197ea -0, 861, 861, 1, 518400, 0x6eb72347 -1, 172203000, 172203000, 1695000, 1826, 0x9a1ac769 +0, 861, 861, 1, 518400, 0xbab197ea 0, 869, 869, 1, 518400, 0xbab197ea -1, 173947000, 173947000, 1934000, 1474, 0xa9b03cdc -0, 870, 870, 1, 518400, 0x9c4a3a3d +0, 870, 870, 1, 518400, 0xbab197ea 0, 879, 879, 1, 518400, 0xbab197ea -1, 175957000, 175957000, 1763000, 1019, 0x20409355 -0, 880, 880, 1, 518400, 0xc9ebfa89 +0, 880, 880, 1, 518400, 0xbab197ea 0, 889, 889, 1, 518400, 0xbab197ea -0, 946, 946, 1, 518400, 0xbaf801ef -1, 189295000, 189295000, 1968000, 1596, 0x408c726e +0, 946, 946, 1, 518400, 0xbab197ea 0, 956, 956, 1, 518400, 0xbab197ea -1, 191356000, 191356000, 1228000, 1517, 0xae8c5c2b -0, 957, 957, 1, 518400, 0x59f4e72f +0, 957, 957, 1, 518400, 0xbab197ea 0, 963, 963, 1, 518400, 0xbab197ea -1, 192640000, 192640000, 1763000, 2506, 0xa458d6d4 -0, 964, 964, 1, 518400, 0x9d5b9d69 +0, 964, 964, 1, 518400, 0xbab197ea 0, 972, 972, 1, 518400, 0xbab197ea -1, 195193000, 195193000, 1092000, 1074, 0x397ba9a8 -0, 976, 976, 1, 518400, 0x923d1ce7 +0, 976, 976, 1, 518400, 0xbab197ea 0, 981, 981, 1, 518400, 0xbab197ea -1, 196361000, 196361000, 1524000, 1715, 0x695ca41e -0, 982, 982, 1, 518400, 0x6e652cd2 +0, 982, 982, 1, 518400, 0xbab197ea 0, 989, 989, 1, 518400, 0xbab197ea -1, 197946000, 197946000, 1160000, 789, 0xc63a189e -0, 990, 990, 1, 518400, 0x25113966 +0, 990, 990, 1, 518400, 0xbab197ea 0, 996, 996, 1, 518400, 0xbab197ea -1, 199230000, 199230000, 1627000, 1846, 0xeea8c599 -0, 997, 997, 1, 518400, 0x2dc83609 +0, 997, 997, 1, 518400, 0xbab197ea 0, 1004, 1004, 1, 518400, 0xbab197ea -1, 200924000, 200924000, 1763000, 922, 0xd4a87222 -0, 1005, 1005, 1, 518400, 0x90483bc6 +0, 1005, 1005, 1, 518400, 0xbab197ea 0, 1013, 1013, 1, 518400, 0xbab197ea -0, 1053, 1053, 1, 518400, 0x3de86ab7 -1, 210600000, 210600000, 1831000, 665, 0x55580135 +0, 1053, 1053, 1, 518400, 0xbab197ea 0, 1062, 1062, 1, 518400, 0xbab197ea -1, 214771000, 214771000, 1558000, 1216, 0x50d1f6c5 -0, 1074, 1074, 1, 518400, 0x8c320e68 +0, 1074, 1074, 1, 518400, 0xbab197ea 0, 1082, 1082, 1, 518400, 0xbab197ea -0, 1128, 1128, 1, 518400, 0x81e977b2 -1, 225640000, 225640000, 2127000, 2133, 0x670c11a5 +0, 1128, 1128, 1, 518400, 0xbab197ea 0, 1139, 1139, 1, 518400, 0xbab197ea -1, 227834000, 227834000, 1262000, 1264, 0xc1d9fc57 -0, 1140, 1140, 1, 518400, 0xb046dd30 -0, 1145, 1145, 1, 518400, 0xbab197ea +0, 1140, 1140, 1, 518400, 0xbab197ea diff --git a/tests/ref/fate/sub2video_basic b/tests/ref/fate/sub2video_basic index 5f72e292c9..a6ada040b4 100644 --- a/tests/ref/fate/sub2video_basic +++ b/tests/ref/fate/sub2video_basic @@ -2,94 +2,89 @@ #media_type 0: video #codec_id 0: rawvideo #dimensions 0: 720x480 -#sar 0: 0/1 -0, 3312, 3312, 1, 1382400, 0x00000000 -0, 3312, 3312, 1, 1382400, 0x8c93c2ba +#sar 0: 1/1 +0, 3312, 3312, 1, 1382400, 0xb890339a 0, 3436, 3436, 1, 1382400, 0x00000000 -0, 3684, 3684, 1, 1382400, 0xb02e32ca +0, 3684, 3684, 1, 1382400, 0x0eca3de2 0, 3802, 3802, 1, 1382400, 0x00000000 -0, 4520, 4520, 1, 1382400, 0x83b71116 +0, 4520, 4520, 1, 1382400, 0xf0eecdec 0, 4584, 4584, 1, 1382400, 0x00000000 -0, 4586, 4586, 1, 1382400, 0x85547fd1 +0, 4586, 4586, 1, 1382400, 0x618b5b84 0, 4645, 4645, 1, 1382400, 0x00000000 -0, 4648, 4648, 1, 1382400, 0x00000000 -0, 4648, 4648, 1, 1382400, 0xb6a8f181 +0, 4648, 4648, 1, 1382400, 0xaa6720f2 0, 4715, 4715, 1, 1382400, 0x00000000 -0, 4717, 4717, 1, 1382400, 0xb64d1a2c +0, 4717, 4717, 1, 1382400, 0x060bb700 0, 4748, 4748, 1, 1382400, 0x00000000 -0, 4750, 4750, 1, 1382400, 0x7b37ecf3 +0, 4750, 4750, 1, 1382400, 0x2828be0e 0, 4792, 4792, 1, 1382400, 0x00000000 -0, 4993, 4993, 1, 1382400, 0xdc025bd1 +0, 4993, 4993, 1, 1382400, 0xf8bbc093 0, 5027, 5027, 1, 1382400, 0x00000000 -0, 5029, 5029, 1, 1382400, 0x688b294d +0, 5029, 5029, 1, 1382400, 0x7b1bf22a 0, 5068, 5068, 1, 1382400, 0x00000000 -0, 5070, 5070, 1, 1382400, 0xa2b33d1b +0, 5070, 5070, 1, 1382400, 0xdd64d2bb 0, 5117, 5117, 1, 1382400, 0x00000000 -0, 5119, 5119, 1, 1382400, 0xb3e525e3 +0, 5119, 5119, 1, 1382400, 0xd8e4cd86 0, 5168, 5168, 1, 1382400, 0x00000000 -0, 5170, 5170, 1, 1382400, 0xaa8fbdd7 +0, 5170, 5170, 1, 1382400, 0x898b5977 0, 5216, 5216, 1, 1382400, 0x00000000 -0, 5218, 5218, 1, 1382400, 0x7b7f26dd +0, 5218, 5218, 1, 1382400, 0x7159b7a6 0, 5249, 5249, 1, 1382400, 0x00000000 -0, 5251, 5251, 1, 1382400, 0x15e2f836 +0, 5251, 5251, 1, 1382400, 0x50104008 0, 5289, 5289, 1, 1382400, 0x00000000 -0, 5291, 5291, 1, 1382400, 0x0fee9b0c +0, 5291, 5291, 1, 1382400, 0x86540ec6 0, 5358, 5358, 1, 1382400, 0x00000000 -0, 5360, 5360, 1, 1382400, 0x89d62791 +0, 5360, 5360, 1, 1382400, 0x274d1a46 0, 5429, 5429, 1, 1382400, 0x00000000 -0, 5431, 5431, 1, 1382400, 0xa6a9fd74 +0, 5431, 5431, 1, 1382400, 0x5089f632 0, 5490, 5490, 1, 1382400, 0x00000000 -0, 5491, 5491, 1, 1382400, 0x7896178d +0, 5491, 5491, 1, 1382400, 0x6b89a314 0, 5537, 5537, 1, 1382400, 0x00000000 -0, 5588, 5588, 1, 1382400, 0x01751a52 +0, 5588, 5588, 1, 1382400, 0x89fafd50 0, 5647, 5647, 1, 1382400, 0x00000000 -0, 5688, 5688, 1, 1382400, 0xa3959c6f +0, 5688, 5688, 1, 1382400, 0x25f2ada2 0, 5770, 5770, 1, 1382400, 0x00000000 -0, 5772, 5772, 1, 1382400, 0x3d3ea47b +0, 5772, 5772, 1, 1382400, 0x3574406f 0, 5826, 5826, 1, 1382400, 0x00000000 -0, 5828, 5828, 1, 1382400, 0x593f8b24 +0, 5828, 5828, 1, 1382400, 0xb440b72a 0, 5931, 5931, 1, 1382400, 0x00000000 -0, 5933, 5933, 1, 1382400, 0x171f05ba +0, 5933, 5933, 1, 1382400, 0x80026884 0, 6001, 6001, 1, 1382400, 0x00000000 -0, 6003, 6003, 1, 1382400, 0xb014cdf1 +0, 6003, 6003, 1, 1382400, 0xcf9efa64 0, 6054, 6054, 1, 1382400, 0x00000000 -0, 6839, 6839, 1, 1382400, 0xd918e667 +0, 6839, 6839, 1, 1382400, 0xfeb611ad 0, 6880, 6880, 1, 1382400, 0x00000000 -0, 7386, 7386, 1, 1382400, 0xc9406331 +0, 7386, 7386, 1, 1382400, 0x5e407233 0, 7419, 7419, 1, 1382400, 0x00000000 -0, 7501, 7501, 1, 1382400, 0xaf08b10d +0, 7501, 7501, 1, 1382400, 0xa924c3bd 0, 7549, 7549, 1, 1382400, 0x00000000 -0, 7551, 7551, 1, 1382400, 0x00000000 -0, 7551, 7551, 1, 1382400, 0x853a9d93 +0, 7551, 7551, 1, 1382400, 0x5f9818b3 0, 7589, 7589, 1, 1382400, 0x00000000 -0, 7605, 7605, 1, 1382400, 0x7491a87d +0, 7605, 7605, 1, 1382400, 0x3299fcbd 0, 7647, 7647, 1, 1382400, 0x00000000 -0, 7649, 7649, 1, 1382400, 0xf7383c58 +0, 7649, 7649, 1, 1382400, 0xe113824e 0, 7697, 7697, 1, 1382400, 0x00000000 -0, 7699, 7699, 1, 1382400, 0xe66be411 +0, 7699, 7699, 1, 1382400, 0x85475a26 0, 7743, 7743, 1, 1382400, 0x00000000 -0, 8032, 8032, 1, 1382400, 0xd6850362 +0, 8032, 8032, 1, 1382400, 0xb27eef40 0, 8082, 8082, 1, 1382400, 0x00000000 -0, 8084, 8084, 1, 1382400, 0x3e1ed109 +0, 8084, 8084, 1, 1382400, 0x82590435 0, 8115, 8115, 1, 1382400, 0x00000000 -0, 8116, 8116, 1, 1382400, 0x39c1b7bd +0, 8116, 8116, 1, 1382400, 0x48179b41 0, 8160, 8160, 1, 1382400, 0x00000000 -0, 8180, 8180, 1, 1382400, 0x35b85f2e +0, 8180, 8180, 1, 1382400, 0x765378a3 0, 8207, 8207, 1, 1382400, 0x00000000 -0, 8209, 8209, 1, 1382400, 0x00000000 -0, 8209, 8209, 1, 1382400, 0x83f103e5 +0, 8209, 8209, 1, 1382400, 0x2d4124d8 0, 8247, 8247, 1, 1382400, 0x00000000 -0, 8249, 8249, 1, 1382400, 0xbc1ca9b3 +0, 8249, 8249, 1, 1382400, 0x9ca3f8c5 0, 8278, 8278, 1, 1382400, 0x00000000 -0, 8281, 8281, 1, 1382400, 0x94d4a51e +0, 8281, 8281, 1, 1382400, 0x46f3056d 0, 8321, 8321, 1, 1382400, 0x00000000 -0, 8323, 8323, 1, 1382400, 0xf88cdfde +0, 8323, 8323, 1, 1382400, 0xb8d19c43 0, 8367, 8367, 1, 1382400, 0x00000000 -0, 8565, 8565, 1, 1382400, 0xdd51423b +0, 8565, 8565, 1, 1382400, 0x06d4e160 0, 8611, 8611, 1, 1382400, 0x00000000 -0, 8669, 8669, 1, 1382400, 0x08259fa4 +0, 8669, 8669, 1, 1382400, 0xd6e06132 0, 8708, 8708, 1, 1382400, 0x00000000 -0, 8941, 8941, 1, 1382400, 0x1663fa34 +0, 8941, 8941, 1, 1382400, 0x44f7e37c 0, 8994, 8994, 1, 1382400, 0x00000000 -0, 8996, 8996, 1, 1382400, 0xda2ceb55 -0, 9027, 9027, 1, 1382400, 0x00000000 +0, 8996, 8996, 1, 1382400, 0xa1f76b2f diff --git a/tests/ref/fate/sub2video_time_limited b/tests/ref/fate/sub2video_time_limited index 9fb6fb06f9..3aebce9ff1 100644 --- a/tests/ref/fate/sub2video_time_limited +++ b/tests/ref/fate/sub2video_time_limited @@ -2,7 +2,5 @@ #media_type 0: video #codec_id 0: rawvideo #dimensions 0: 1920x1080 -#sar 0: 0/1 -0, 2, 2, 1, 8294400, 0x00000000 -0, 2, 2, 1, 8294400, 0xa87c518f -0, 10, 10, 1, 8294400, 0xa87c518f +#sar 0: 1/1 +0, 2, 2, 1, 8294400, 0x1c106cb0 From patchwork Sun Sep 12 03:21:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 30174 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp2863166iov; Sat, 11 Sep 2021 20:22:44 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyW2a2DUdIcoSRpMblIBCr25kLZTOna6SXQjuSyiRXkrpwGwWFsUojfclzUptuofsd9BOA+ X-Received: by 2002:a50:99cc:: with SMTP id n12mr6163678edb.53.1631416964515; Sat, 11 Sep 2021 20:22:44 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a10si3643290ejf.122.2021.09.11.20.22.44; Sat, 11 Sep 2021 20:22:44 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=WQWuaskS; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B50A268A477; Sun, 12 Sep 2021 06:21:57 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10olkn2062.outbound.protection.outlook.com [40.92.41.62]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B35B668A477 for ; Sun, 12 Sep 2021 06:21:54 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Vyr8L7qm1zK3/GoP81nJ9ogIsejOPhoPiBoupny1bIeV+/2EQiwnU2zs0fUcLTnw1nNtiEghxEHvG6iTHuTGYFjY02bkH0XEQktePdRpTCBKO4h3gdeb0TFmzvlS+aQqSvkJWh2Y0YZjGoJ8gx1AbvMki8/C6TW/PG2FRfhE++4Vvy6w0QzunCnYrIcl7q5Gft+HpiNbTc7ZkS/hcMP8X/f5i9ltj8talFTHvooCFsQBVbkZ46iG+i+ngv2vg2weLwh/tQ162sIjiQJz8nZ61Age1OTI9u9viiQK6+O30Bqcz6THI3CDnueV3+b6ymZuglowWAul6r+oWeE6BkcGHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=CKO00Z+wopx5s6Ftnr5+qcftpkI9g52J5OuXpmEgG7A=; b=jljzj+vmWluvQpzueRPr6eVE8CwD+VrtIaoeeAs1t7MVd/4Vq4UxiWRy3LiJbruCZFxInnM4CwwuRlLdB+4SceQjzGNvu7uzKByfm8lYyMtjhBGMPnstJ0P5GQUHHHWAljUraRTQySyNClrsmjkyFiRxRSRj3lxXKyDEirpHngfxKPgm0FbjfJDn9NiyId1nfeciDX4VQDL8y8oqMvAziIovPrklzeClYxq7+THBz6+p64kSpmQDMl1VpGoZvcnbA4raOWZQamtNDaGG/SoIa0jBbAX4h2tCbb/B30LE7w6+b7qZSpXPikorynlb+un0q4uRRI83uwd6KTotx0BrLA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CKO00Z+wopx5s6Ftnr5+qcftpkI9g52J5OuXpmEgG7A=; b=WQWuaskSCu6TTqxx1qrFxZrBtqV/0iW2/r1/d/b0R4oO1yUCtAuDsUvXSjwzQ0q5okZY/lo4d9T853tUPYPMfOnxiUhooavAdmj94gLimsr+cQcXYLMrfWY0udMJyv+jsDdCLUKahkUXxpYgRcGZTe4mK2xLa9ZGmLeLZnFa7tPR74BcXekqwQYrsO8Iz0Qa+PuWlpu2ND5oA1WXqyiuB3bnE968xOc4hxgZtX/L4mRBz96/7FEDjhNA6FbpqulHAF7FST27PBEipBqA07aZ027sJCsZ0M0KGoXfO0plp8iJ2zlMZqFqBuZfGdUXcWWl24PGd1OZ+0R3VXeAQplllg== Received: from MN2PR04MB5981.namprd04.prod.outlook.com (2603:10b6:208:da::10) by MN2PR04MB5728.namprd04.prod.outlook.com (2603:10b6:208:3d::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Sun, 12 Sep 2021 03:21:46 +0000 Received: from MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb]) by MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb%5]) with mapi id 15.20.4500.018; Sun, 12 Sep 2021 03:21:46 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v5 05/12] avfilter/subtitles: Add subtitles.c Thread-Index: AQHXp4VQqjiMYp/M4Ei1FKVxt5J5kw== Date: Sun, 12 Sep 2021 03:21:45 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [a3F6l4fjLVCrY51ztatJ8wYAjp9XD9hQCKwdzX7GBYo=] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 2ce90d2b-cf14-4a08-6bb1-08d9759c735b x-ms-traffictypediagnostic: MN2PR04MB5728: x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: xSETviGrM44Eh9B1rrd5YsLljLvsyE/Rn11DCfqRraKvVBICu2/6aSDP7yvZ1BZqM/sOIdAIn881kLQ3NoqTURbn4RLmecM+JXzU3PQNbPxSLJACIia9Z8mbjqSyv9FTjEGuVkFojIGCaaJQ2I9/QBRCtzgB1VgiNN7yuFPodG1Xo+8JuHuVZz5ZmtyV2kk72Kz98tFi30mTdzX5Hr8ldaK6QdTOzq8EQ+m/8L8I7O6fe987rxBEfdrnmpFa96i47hjJ+iXdTithCybQ5g8tbi0TzG0v2ZRY0hgyP/jdrClx203XTL7UCw71MUOxsNnLT0vKMX3KcW+TuZdbhVXRF2uwpdmjRyS1fQnx6FJNJL/kUaZ9tln9tYzGBOMtznCWDIkGvYyetBKmxont6RRhGqvB2fGECzWYfxCFlfHCnEBNr8dLbXI0lzEBjKbFoc+U x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: cidPKNVOETI1T1IWx1eh9p/3kUgGNyYTyPQ371g1keSC9O4WXqG13dztbKF0OWvGG9gwFH8FjBOtdst0bP7tDfqfIbphpdm+ouqnNEVU6N7rcX/XsIl9wrdHvLLW2IJJb3kNO45wBTWxI/ZZF7LJOQ== x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-3174-20-msonline-outlook-529c7.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR04MB5981.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 2ce90d2b-cf14-4a08-6bb1-08d9759c735b X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Sep 2021 03:21:45.9725 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR04MB5728 Subject: [FFmpeg-devel] [PATCH v5 05/12] avfilter/subtitles: Add subtitles.c X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 4fht3UoSPFNF Signed-off-by: softworkz --- libavfilter/Makefile | 1 + libavfilter/f_interleave.c | 3 ++ libavfilter/internal.h | 1 + libavfilter/subtitles.c | 61 ++++++++++++++++++++++++++++++++++++++ libavfilter/subtitles.h | 44 +++++++++++++++++++++++++++ 5 files changed, 110 insertions(+) create mode 100644 libavfilter/subtitles.c create mode 100644 libavfilter/subtitles.h diff --git a/libavfilter/Makefile b/libavfilter/Makefile index af957a5ac0..a510dd12b4 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -19,6 +19,7 @@ OBJS = allfilters.o \ framequeue.o \ graphdump.o \ graphparser.o \ + subtitles.o \ video.o \ OBJS-$(HAVE_THREADS) += pthread.o diff --git a/libavfilter/f_interleave.c b/libavfilter/f_interleave.c index b60e89762f..e8821dceba 100644 --- a/libavfilter/f_interleave.c +++ b/libavfilter/f_interleave.c @@ -32,6 +32,7 @@ #include "filters.h" #include "internal.h" #include "audio.h" +#include "subtitles.h" #include "video.h" typedef struct InterleaveContext { @@ -170,6 +171,8 @@ static av_cold int init(AVFilterContext *ctx) inpad.get_buffer.video = ff_null_get_video_buffer; break; case AVMEDIA_TYPE_AUDIO: inpad.get_buffer.audio = ff_null_get_audio_buffer; break; + case AVMEDIA_TYPE_SUBTITLE: + inpad.get_buffer.subtitle = ff_null_get_subtitles_buffer; break; default: av_assert0(0); } diff --git a/libavfilter/internal.h b/libavfilter/internal.h index 6ddf024d93..8a9951d20e 100644 --- a/libavfilter/internal.h +++ b/libavfilter/internal.h @@ -90,6 +90,7 @@ struct AVFilterPad { union { AVFrame *(*video)(AVFilterLink *link, int w, int h); AVFrame *(*audio)(AVFilterLink *link, int nb_samples); + AVFrame *(*subtitle)(AVFilterLink *link, int format); } get_buffer; /** diff --git a/libavfilter/subtitles.c b/libavfilter/subtitles.c new file mode 100644 index 0000000000..90ec479e51 --- /dev/null +++ b/libavfilter/subtitles.c @@ -0,0 +1,61 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/common.h" + +#include "subtitles.h" +#include "avfilter.h" +#include "internal.h" + + +AVFrame *ff_null_get_subtitles_buffer(AVFilterLink *link, int format) +{ + return ff_get_subtitles_buffer(link->dst->outputs[0], format); +} + +AVFrame *ff_default_get_subtitles_buffer(AVFilterLink *link, int format) +{ + AVFrame *frame = NULL; + + // TODO: + //frame = ff_frame_pool_get(link->frame_pool); + + frame = av_frame_alloc(); + if (!frame) + return NULL; + + frame->format = format; + frame->type = AVMEDIA_TYPE_SUBTITLE; + + return frame; +} + +AVFrame *ff_get_subtitles_buffer(AVFilterLink *link, int format) +{ + AVFrame *ret = NULL; + + if (link->dstpad->get_buffer.subtitle) + ret = link->dstpad->get_buffer.subtitle(link, format); + + if (!ret) + ret = ff_default_get_subtitles_buffer(link, format); + + return ret; +} diff --git a/libavfilter/subtitles.h b/libavfilter/subtitles.h new file mode 100644 index 0000000000..d3d5491652 --- /dev/null +++ b/libavfilter/subtitles.h @@ -0,0 +1,44 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVFILTER_SUBTITLES_H +#define AVFILTER_SUBTITLES_H + +#include "avfilter.h" +#include "internal.h" + +/** default handler for get_subtitles_buffer() for subtitle inputs */ +AVFrame *ff_default_get_subtitles_buffer(AVFilterLink *link, int format); + +/** get_subtitles_buffer() handler for filters which simply pass subtitles along */ +AVFrame *ff_null_get_subtitles_buffer(AVFilterLink *link, int format); + +/** + * Request a subtitles buffer with a specific set of permissions. + * + * @param link the output link to the filter from which the buffer will + * be requested + * @param format The subtitles format. + * @return A reference to the subtitles. This must be unreferenced with + * avfilter_unref_buffer when you are finished with it. +*/ +AVFrame *ff_get_subtitles_buffer(AVFilterLink *link, int format); + +#endif /* AVFILTER_SUBTITLES_H */ From patchwork Sun Sep 12 03:21:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 30179 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp2863244iov; Sat, 11 Sep 2021 20:22:54 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzL/r1sOYaDgm71CMpKoQ/z3gSQC5vEYv/LyDrqnc8BqlvqlNX5HvX4YScrlCw4fP6zAKx/ X-Received: by 2002:a17:906:c20d:: with SMTP id d13mr5645991ejz.259.1631416974541; Sat, 11 Sep 2021 20:22:54 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id bl4si4404145ejb.689.2021.09.11.20.22.54; Sat, 11 Sep 2021 20:22:54 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=MSoMJKvB; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id ABD2768AA09; Sun, 12 Sep 2021 06:21:58 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10olkn2062.outbound.protection.outlook.com [40.92.41.62]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2368D68A990 for ; Sun, 12 Sep 2021 06:21:55 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jvZ59ulFIxceyTg2Lr9wk7quFfgYNRDJ6WxdZX+pqhCCdQ1hvXgILN0afIHih8R9C/Yf4bv21U831OgdUex7dGNrM1cRNb65JOvdfZFNG41zQvYKhr9cwJC/sEiLKMP6PV+MYbsiBAAmPz4PeR62B1WE66RJBalqpSCYAvqsh798tK7Ztb4M0XUIJCWpYyFREAO2hHiWmyrPW8n2w/OWln/reCjQx7JllcHgkUklC7buzyKcL3laCPQwBa6dFpZoIRVYMEj//gQafZ2QMtPlqrrIypE9qVzMlFYRUymkiq09Lmg9qIyOpDAz2OBynf+T2+eP1EJAwmGaVUtSj3u8dg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=F7KiL0JgeBQiLvU1j+IFUBxLV16yq73XsRMrhvmcWuA=; b=neSBn0wjNm+4/22NMdzidY9dwFHUwKS+znibY9FrOW+3peGMCut3J97CthL7MFnpyV2m4cgm4u8PP9ZVS8gyUhMk2qCSwehXsoLqKUKNNiGxk8U5xqLiAKqamHLwsBdoaEH8b1LH60/leyqTGrsfQqpTAsR4/FLKpZqyFXWA2Th16CYZGPY97rEl6zBWMnD7JiXg1UefV1a/u9yGgsA4HNjFhFoYkSP5pB5efS05vX7f54VyEtt2RvTJBmHN4LcFtBZXi4926MBSP0QhcSs4h4LBHBoAH4LHzmEJ3NhBHqcNXsf64du7YGn+qVJHPsehbCHt+EtY2tIHOtCnpo+XMA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=F7KiL0JgeBQiLvU1j+IFUBxLV16yq73XsRMrhvmcWuA=; b=MSoMJKvBfc5dQduwK3CtKheLt8opXyHlfYENqToDmRKjwOhKDW4vpWOjzzgAYFAiQRXlaoKVqAVGhv4ycW64zoTE0CBe0ZeCaeeeLyCAHWn3f6LGsTzyzVZiTxdS9K3vKt/bUAemlc8dCb/HDsH0BI4j8QHpeoFvF80YODOqPLUMrsLUgsSdnZarvWfMGZZ2JkQs1em5E9LR3xva4YdDv7Sa1F0OqwlIFpGxmLAJPq1O2m6vzM3DLrgo45h/zOtpCRuRQVjDHuB33t1etaYGIQ46JYengPU0JVoVn00aM0U63N9XQgsUNlCp8fj//qab3MRJ7K7K9B9+gb5oZB+Dow== Received: from MN2PR04MB5981.namprd04.prod.outlook.com (2603:10b6:208:da::10) by MN2PR04MB5728.namprd04.prod.outlook.com (2603:10b6:208:3d::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Sun, 12 Sep 2021 03:21:48 +0000 Received: from MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb]) by MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb%5]) with mapi id 15.20.4500.018; Sun, 12 Sep 2021 03:21:48 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v5 06/12] avfilter/avfilter: Handle subtitle frames Thread-Index: AQHXp4VS2cRC4/LjO0a44pTNw1mrHQ== Date: Sun, 12 Sep 2021 03:21:48 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [Hz7viq6/avBPrxCw72IHPi47WoWfmXTGF1+ycNjqDEU=] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 9980056c-3bd6-4d81-3eb3-08d9759c74ef x-ms-traffictypediagnostic: MN2PR04MB5728: x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 7WZgrpT3OdjAUmhyxEUhHgLI89cELq0rsVnc/2RVwH3tHL4hPtoYO3c4/eetMGV0Y91aPZ3NVFnFFj3R4fDU08ITzbnpcSglaRslUoSUoZr8a84ax/ho2HTr9GbXHUvHfoGuzz1dpAKmlOgf6swHjl/uSjDN7i3dGQnF1qKCXrzzB7WA9O2CN2wEF8rcYmpDlNUCyNm2Y6aCJ2KmJL8ajqWB38tBTZQy4KY7GoEsTDtHxzh8eaSB8XexdYFB89wBt8wIgSTvqxOftSVZ9OutyI3zNEZORD/5WlKdlAb2r3fW7cO5YX+I3G99l0zRSbe+cC7j1N3qARUJbXy5v38FoXsNGFU5xVV37CqUluosyQ5PtnVgQKx4mPmgZmikA4f2lKCMN+r8wO7C8wl4WudNejmivgJyv8h2mVPPOpUO0Ko2MQHyzsCdRXklnP3xb3tX x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: K00q1bl0f8McGm+HkRjMCggz+eUs8eKIhHp6tPh+j6z9miFk2kI4+cpXomW7g1OAdq7YfcQyvUgwsdbTSxsSSqtfZZBqPM27RdVMJhMkhYQvhLUi1NuiCAXNYfr5UD4gaD1ApOJwD6VDuGHb2HfIug== x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-3174-20-msonline-outlook-529c7.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR04MB5981.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 9980056c-3bd6-4d81-3eb3-08d9759c74ef X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Sep 2021 03:21:48.5620 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR04MB5728 Subject: [FFmpeg-devel] [PATCH v5 06/12] avfilter/avfilter: Handle subtitle frames X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: IK1GzOb4sB27 Signed-off-by: softworkz --- libavfilter/avfilter.c | 8 +++++--- libavfilter/avfiltergraph.c | 5 +++++ libavfilter/formats.c | 14 ++++++++++++++ libavfilter/formats.h | 3 +++ 4 files changed, 27 insertions(+), 3 deletions(-) diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c index a04f8ed62f..961e55e7b4 100644 --- a/libavfilter/avfilter.c +++ b/libavfilter/avfilter.c @@ -56,7 +56,8 @@ void ff_tlog_ref(void *ctx, AVFrame *ref, int end) ref->linesize[0], ref->linesize[1], ref->linesize[2], ref->linesize[3], ref->pts, ref->pkt_pos); - if (ref->width) { + switch(ref->type) { + case AVMEDIA_TYPE_VIDEO: ff_tlog(ctx, " a:%d/%d s:%dx%d i:%c iskey:%d type:%c", ref->sample_aspect_ratio.num, ref->sample_aspect_ratio.den, ref->width, ref->height, @@ -64,12 +65,13 @@ void ff_tlog_ref(void *ctx, AVFrame *ref, int end) ref->top_field_first ? 'T' : 'B', /* Top / Bottom */ ref->key_frame, av_get_picture_type_char(ref->pict_type)); - } - if (ref->nb_samples) { + break; + case AVMEDIA_TYPE_AUDIO: ff_tlog(ctx, " cl:%"PRId64"d n:%d r:%d", ref->channel_layout, ref->nb_samples, ref->sample_rate); + break; } ff_tlog(ctx, "]%s", end ? "\n" : ""); diff --git a/libavfilter/avfiltergraph.c b/libavfilter/avfiltergraph.c index 45b028cd9c..7a5a4ea419 100644 --- a/libavfilter/avfiltergraph.c +++ b/libavfilter/avfiltergraph.c @@ -314,6 +314,8 @@ static int filter_link_check_formats(void *log, AVFilterLink *link, AVFilterForm return ret; break; + case AVMEDIA_TYPE_SUBTITLE: + return 0; default: av_assert0(!"reached"); } @@ -444,6 +446,9 @@ static int query_formats(AVFilterGraph *graph, void *log_ctx) if (!link) continue; + if (link->type == AVMEDIA_TYPE_SUBTITLE) + continue; + neg = ff_filter_get_negotiation(link); av_assert0(neg); for (neg_step = 1; neg_step < neg->nb_mergers; neg_step++) { diff --git a/libavfilter/formats.c b/libavfilter/formats.c index 1bf7d36195..704774d763 100644 --- a/libavfilter/formats.c +++ b/libavfilter/formats.c @@ -19,6 +19,7 @@ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ +#include "libavcodec/avcodec.h" #include "libavutil/avassert.h" #include "libavutil/channel_layout.h" #include "libavutil/common.h" @@ -430,6 +431,12 @@ int ff_add_channel_layout(AVFilterChannelLayouts **l, uint64_t channel_layout) return 0; } +int ff_add_subtitle_type(AVFilterFormats **avff, int64_t fmt) +{ + ADD_FORMAT(avff, fmt, ff_formats_unref, int, formats, nb_formats); + return 0; +} + AVFilterFormats *ff_all_formats(enum AVMediaType type) { AVFilterFormats *ret = NULL; @@ -447,6 +454,13 @@ AVFilterFormats *ff_all_formats(enum AVMediaType type) return NULL; fmt++; } + } else if (type == AVMEDIA_TYPE_SUBTITLE) { + if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_BITMAP) < 0) + return NULL; + if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_ASS) < 0) + return NULL; + if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_TEXT) < 0) + return NULL; } return ret; diff --git a/libavfilter/formats.h b/libavfilter/formats.h index 471cb42bc4..25ae511fc1 100644 --- a/libavfilter/formats.h +++ b/libavfilter/formats.h @@ -180,6 +180,9 @@ int ff_set_common_formats_from_list(AVFilterContext *ctx, const int *fmts); av_warn_unused_result int ff_add_channel_layout(AVFilterChannelLayouts **l, uint64_t channel_layout); +av_warn_unused_result +int ff_add_subtitle_type(AVFilterFormats **avff, int64_t fmt); + /** * Add *ref as a new reference to f. */ From patchwork Sun Sep 12 03:21:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 30169 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp2863299iov; Sat, 11 Sep 2021 20:23:03 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxl0Xz9ofLJqqJdB6KUw4/UdB31rg/iNjDxGdbUbdQNoXWlP06pkyeenFYBJZCxLgt8VNU2 X-Received: by 2002:a17:906:a108:: with SMTP id t8mr5698743ejy.407.1631416983284; Sat, 11 Sep 2021 20:23:03 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id k2si3326381edh.529.2021.09.11.20.23.03; Sat, 11 Sep 2021 20:23:03 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b="VgIpG/2R"; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9A7B068A9E9; Sun, 12 Sep 2021 06:21:59 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10olkn2062.outbound.protection.outlook.com [40.92.41.62]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7E84D68A930 for ; Sun, 12 Sep 2021 06:21:55 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WGuV7R0aiE0PEvJywN9ITYnpMJBkc/Go1wmff/V+L9JMhGYRO0XVsfh/+xlzZ61VdsMZf00nSs5E4eY770bZC8n6ErYUl/C5pv8XF0sybLgDSzMsJ6Hzq9SR/FvJZlqhfXLcF9xGMzxFfHIXNp9YOEcjgI13qV8Yp0W6ty+GiVzusLTIM9VWV/ssfd5qAwrMhy5IB5fJuUH8eKEA4DhFkgCZBDK3K+zJJ5SPk+Lov17yEgXUzDFWj/pGorH5Smc4AExG4AZQ4celdEnZvVYgZ1zQxNTzcBX9GVWCMSHB6pQhHdbQELI/9d/7jewfDi1/cOwP4dBUDvJu6jxkKUAfEw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=8QTwPoJvcXwg8fQk0/q5OnAXWEEi19cs1I6v8fMFbz0=; b=OqWbE/0Dr5ivem7bBKnMVHxHPs8J/NEbXJb8Fp8lsAZk99fdA9/oKueV92dR2+3XYTZhWnNLbRQkhyGYeYpL9Dk8Qjvc9CrWI3EibI7ykhbUrkzNWqDQm/dOiDoxH91rI081r6sacXLqcufg605drlOqLS5PHpw72R8iaYVPnFZahjmM7j1AKk8Uf+MG7oT9kHtYNfd4UrpZZFN/5bszh7jcOgLcJCkqBznqCSC2zu3lQaUETtQ/xw4Bx5ki9RCyE3sUa1sBIHcmuHfCEtBLWg6k8LWYKZqm9fjCkoiEnAJaLmiGcfVlJMHOcyiQJGfmgN6gu+/xkiZd8lz3X90M8A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8QTwPoJvcXwg8fQk0/q5OnAXWEEi19cs1I6v8fMFbz0=; b=VgIpG/2Rbzo2oeKcR3HZaVD4DEhnZ8HW0rq3nZfgDR/Mtge3XQWJLUPARnT6gX22jUxHpt9Pmni/jF0WUNhoJGbdeZTBlscHrW8uwutYrktljnP9uMxI8tAuJ+WU3IDMo4YREySF8UdnUuZeMVdllRzT+moGHTj3EiwAECrrUYGo1MYXCzys1N7yRAhy1DXQ4eBm7M0VE1vCQtdE9OuamWl2l8UtJew+J/uzPZxftChOvRzCZ3ZZCBH+Pr4K6/7sIWT7YmsMwHHIxwcgAnl3UpqtDI1rjVFv262dLicRWp/cvOczwokaPz12n83bS8lCWfvGp9NDDc15gwUStWcp/Q== Received: from MN2PR04MB5981.namprd04.prod.outlook.com (2603:10b6:208:da::10) by MN2PR04MB5728.namprd04.prod.outlook.com (2603:10b6:208:3d::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Sun, 12 Sep 2021 03:21:50 +0000 Received: from MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb]) by MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb%5]) with mapi id 15.20.4500.018; Sun, 12 Sep 2021 03:21:50 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v5 07/12] avfilter/sbuffer: Add sbuffersrv and sbuffersink filters Thread-Index: AQHXp4VTHq8lqVU3p0iumI+t0ZUInA== Date: Sun, 12 Sep 2021 03:21:50 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [A7oWnG1wiQdyRKuFc/pDXd8rk8nMrnuwc1a3RaX9Y/I=] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: c9536231-a18f-43a3-09d7-08d9759c7638 x-ms-traffictypediagnostic: MN2PR04MB5728: x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: SCMxg7ma+dtqsoMD5AybvLiR8JuVbkMqGqdZhdu23tLmhFRzSr4110lNg1EfaqXyNTlKbNByqNDTU7+HAoUNg+6QPqH56caipngzBDOA2LZjfO7s3WYqyk21FEsE+dB2NiSNWJstRPSzeLwemTWpW8pYroN1HvpDgg77Cwrz89yNTCGHpZuBYDLdrOj+1vD2KZsqU8U7qHYaFZWScM8+qEjiXLDxbpfVGpiWDGCg7Gs62Z/R8p5k7cavklrxh1WDb6AY55YMlBDVnHBlns7SLkZB4QrDRJxQHCyEzE8+cEUSwnNCiDhmyaWYm1EzT/STwAQa7UX8WRQqgk4xQDY8AeKG3/j9jdLUwZHksicbt0Z/UzUPK7s3Bfie+XJky0lg9cVZJmyUBPPGo93qwzNAPQhoqi30XTiUvpoxf9cvn1vv70q60fUy8j+FOQg/F8eo x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: yfDTemmC5UszNFl+62VixCiAb57XDhdXu3caxibfYdptQdR/1Nscg/DF0Qvva/yYsO3/mZBiV0wBqmrkYAhyi2GZsnVyYRVUEMphkc0zTngJx8TsvVK6GIBgyDjjiYK5anmLe8sU3DNiYJfidL2CwQ== x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-3174-20-msonline-outlook-529c7.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR04MB5981.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: c9536231-a18f-43a3-09d7-08d9759c7638 X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Sep 2021 03:21:50.7928 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR04MB5728 Subject: [FFmpeg-devel] [PATCH v5 07/12] avfilter/sbuffer: Add sbuffersrv and sbuffersink filters X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: w3tADobnkWtP Signed-off-by: softworkz --- configure | 2 +- libavfilter/allfilters.c | 10 +++--- libavfilter/buffersink.c | 63 +++++++++++++++++++++++++++++++++++ libavfilter/buffersink.h | 15 +++++++++ libavfilter/buffersrc.c | 72 ++++++++++++++++++++++++++++++++++++++++ libavfilter/buffersrc.h | 1 + libavfilter/version.h | 2 +- 7 files changed, 159 insertions(+), 6 deletions(-) diff --git a/configure b/configure index af410a9d11..2f2777df9f 100755 --- a/configure +++ b/configure @@ -7716,7 +7716,7 @@ print_enabled_components(){ fi done if [ "$name" = "filter_list" ]; then - for c in asrc_abuffer vsrc_buffer asink_abuffer vsink_buffer; do + for c in asrc_abuffer vsrc_buffer ssrc_sbuffer asink_abuffer vsink_buffer ssink_sbuffer; do printf " &ff_%s,\n" $c >> $TMPH done fi diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 0c6b2347c8..b27ef6f027 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -531,10 +531,12 @@ extern const AVFilter ff_avsrc_movie; * they are formatted to not be found by the grep * as they are manually added again (due to their 'names' * being the same while having different 'types'). */ -extern const AVFilter ff_asrc_abuffer; -extern const AVFilter ff_vsrc_buffer; -extern const AVFilter ff_asink_abuffer; -extern const AVFilter ff_vsink_buffer; +extern const AVFilter ff_asrc_abuffer; +extern const AVFilter ff_vsrc_buffer; +extern const AVFilter ff_ssrc_sbuffer; +extern const AVFilter ff_asink_abuffer; +extern const AVFilter ff_vsink_buffer; +extern const AVFilter ff_ssink_sbuffer; extern const AVFilter ff_af_afifo; extern const AVFilter ff_vf_fifo; diff --git a/libavfilter/buffersink.c b/libavfilter/buffersink.c index 8b46dcb15e..55cbaff3d5 100644 --- a/libavfilter/buffersink.c +++ b/libavfilter/buffersink.c @@ -29,6 +29,8 @@ #include "libavutil/internal.h" #include "libavutil/opt.h" +#include "libavcodec/avcodec.h" + #define FF_INTERNAL_FIELDS 1 #include "framequeue.h" @@ -57,6 +59,10 @@ typedef struct BufferSinkContext { int *sample_rates; ///< list of accepted sample rates, terminated by -1 int sample_rates_size; + /* only used for subtitles */ + enum AVSubtitleType *subtitle_types; ///< list of accepted subtitle types, must be terminated with -1 + int subtitle_types_size; + AVFrame *peeked_frame; } BufferSinkContext; @@ -168,6 +174,15 @@ AVABufferSinkParams *av_abuffersink_params_alloc(void) return NULL; return params; } + +AVSBufferSinkParams *av_sbuffersink_params_alloc(void) +{ + AVSBufferSinkParams *params = av_mallocz(sizeof(AVSBufferSinkParams)); + + if (!params) + return NULL; + return params; +} #endif static av_cold int common_init(AVFilterContext *ctx) @@ -305,6 +320,28 @@ static int asink_query_formats(AVFilterContext *ctx) return 0; } +static int ssink_query_formats(AVFilterContext *ctx) +{ + BufferSinkContext *buf = ctx->priv; + AVFilterFormats *formats = NULL; + unsigned i; + int ret; + + CHECK_LIST_SIZE(subtitle_types) + if (buf->subtitle_types_size) { + for (i = 0; i < NB_ITEMS(buf->subtitle_types); i++) + if ((ret = ff_add_subtitle_type(&formats, buf->subtitle_types[i])) < 0) + return ret; + if ((ret = ff_set_common_formats(ctx, formats)) < 0) + return ret; + } else { + if ((ret = ff_default_query_formats(ctx)) < 0) + return ret; + } + + return 0; +} + #define OFFSET(x) offsetof(BufferSinkContext, x) #define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM static const AVOption buffersink_options[] = { @@ -322,9 +359,16 @@ static const AVOption abuffersink_options[] = { { NULL }, }; #undef FLAGS +#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_SUBTITLE_PARAM +static const AVOption sbuffersink_options[] = { + { "subtitle_types", "set the supported subtitle formats", OFFSET(subtitle_types), AV_OPT_TYPE_BINARY, .flags = FLAGS }, + { NULL }, +}; +#undef FLAGS AVFILTER_DEFINE_CLASS(buffersink); AVFILTER_DEFINE_CLASS(abuffersink); +AVFILTER_DEFINE_CLASS(sbuffersink); static const AVFilterPad avfilter_vsink_buffer_inputs[] = { { @@ -363,3 +407,22 @@ const AVFilter ff_asink_abuffer = { FILTER_INPUTS(avfilter_asink_abuffer_inputs), .outputs = NULL, }; + +static const AVFilterPad avfilter_ssink_sbuffer_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + }, +}; + +AVFilter ff_ssink_sbuffer = { + .name = "sbuffersink", + .description = NULL_IF_CONFIG_SMALL("Buffer subtitle frames, and make them available to the end of the filter graph."), + .priv_class = &sbuffersink_class, + .priv_size = sizeof(BufferSinkContext), + .init = common_init, + .query_formats = ssink_query_formats, + .activate = activate, + FILTER_INPUTS(avfilter_ssink_sbuffer_inputs), + .outputs = NULL, +}; diff --git a/libavfilter/buffersink.h b/libavfilter/buffersink.h index 69ed0f29a8..b439b586c5 100644 --- a/libavfilter/buffersink.h +++ b/libavfilter/buffersink.h @@ -129,6 +129,21 @@ typedef struct AVABufferSinkParams { */ attribute_deprecated AVABufferSinkParams *av_abuffersink_params_alloc(void); + +/** + * Deprecated and unused struct to use for initializing an sbuffersink context. + */ +typedef struct AVSBufferSinkParams { + const int *subtitle_type; +} AVSBufferSinkParams; + +/** + * Create an AVSBufferSinkParams structure. + * + * Must be freed with av_free(). + */ +attribute_deprecated +AVSBufferSinkParams *av_sbuffersink_params_alloc(void); #endif /** diff --git a/libavfilter/buffersrc.c b/libavfilter/buffersrc.c index 632bfc7ad8..902a09a721 100644 --- a/libavfilter/buffersrc.c +++ b/libavfilter/buffersrc.c @@ -39,6 +39,7 @@ #include "formats.h" #include "internal.h" #include "video.h" +#include "libavcodec/avcodec.h" typedef struct BufferSourceContext { const AVClass *class; @@ -63,6 +64,9 @@ typedef struct BufferSourceContext { uint64_t channel_layout; char *channel_layout_str; + /* subtitle only */ + enum AVSubtitleType subtitle_type; + int eof; } BufferSourceContext; @@ -130,6 +134,13 @@ int av_buffersrc_parameters_set(AVFilterContext *ctx, AVBufferSrcParameters *par if (param->channel_layout) s->channel_layout = param->channel_layout; break; + case AVMEDIA_TYPE_SUBTITLE: + s->subtitle_type = param->format; + if (param->width > 0) + s->w = param->width; + if (param->height > 0) + s->h = param->height; + break; default: return AVERROR_BUG; } @@ -197,6 +208,8 @@ int attribute_align_arg av_buffersrc_add_frame_flags(AVFilterContext *ctx, AVFra CHECK_AUDIO_PARAM_CHANGE(ctx, s, frame->sample_rate, frame->channel_layout, frame->channels, frame->format, frame->pts); break; + case AVMEDIA_TYPE_SUBTITLE: + break; default: return AVERROR(EINVAL); } @@ -269,6 +282,7 @@ unsigned av_buffersrc_get_nb_failed_requests(AVFilterContext *buffer_src) #define OFFSET(x) offsetof(BufferSourceContext, x) #define A AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_AUDIO_PARAM #define V AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM +#define S AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_SUBTITLE_PARAM static const AVOption buffer_options[] = { { "width", NULL, OFFSET(w), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, V }, @@ -298,6 +312,16 @@ static const AVOption abuffer_options[] = { AVFILTER_DEFINE_CLASS(abuffer); +static const AVOption sbuffer_options[] = { + { "time_base", NULL, OFFSET(time_base), AV_OPT_TYPE_RATIONAL, { .dbl = 0 }, 0, INT_MAX, S }, + { "subtitle_type", NULL, OFFSET(subtitle_type), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, S }, + { "width", NULL, OFFSET(w), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, V }, + { "height", NULL, OFFSET(h), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, V }, + { NULL }, +}; + +AVFILTER_DEFINE_CLASS(sbuffer); + static av_cold int init_audio(AVFilterContext *ctx) { BufferSourceContext *s = ctx->priv; @@ -347,6 +371,21 @@ static av_cold int init_audio(AVFilterContext *ctx) return ret; } +static av_cold int init_subtitle(AVFilterContext *ctx) +{ + BufferSourceContext *c = ctx->priv; + + if (c->subtitle_type == AV_SUBTITLE_FMT_BITMAP) + av_log(ctx, AV_LOG_VERBOSE, "graphical subtitles - w:%d h:%d tb:%d/%d\n", + c->w, c->h, c->time_base.num, c->time_base.den); + else + av_log(ctx, AV_LOG_VERBOSE, "text subtitles - w:%d h:%d tb:%d/%d\n", + c->w, c->h, c->time_base.num, c->time_base.den); + + return 0; +} + + static av_cold void uninit(AVFilterContext *ctx) { BufferSourceContext *s = ctx->priv; @@ -381,6 +420,11 @@ static int query_formats(AVFilterContext *ctx) if ((ret = ff_set_common_channel_layouts(ctx, channel_layouts)) < 0) return ret; break; + case AVMEDIA_TYPE_SUBTITLE: + if ((ret = ff_add_format (&formats, c->subtitle_type)) < 0 || + (ret = ff_set_common_formats (ctx , formats )) < 0) + return ret; + break; default: return AVERROR(EINVAL); } @@ -408,6 +452,11 @@ static int config_props(AVFilterLink *link) if (!c->channel_layout) c->channel_layout = link->channel_layout; break; + case AVMEDIA_TYPE_SUBTITLE: + link->format = c->subtitle_type; + link->w = c->w; + link->h = c->h; + break; default: return AVERROR(EINVAL); } @@ -472,3 +521,26 @@ const AVFilter ff_asrc_abuffer = { FILTER_OUTPUTS(avfilter_asrc_abuffer_outputs), .priv_class = &abuffer_class, }; + +static const AVFilterPad avfilter_ssrc_sbuffer_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .request_frame = request_frame, + .config_props = config_props, + }, +}; + +const AVFilter ff_ssrc_sbuffer = { + .name = "sbuffer", + .description = NULL_IF_CONFIG_SMALL("Buffer subtitle frames, and make them accessible to the filterchain."), + .priv_size = sizeof(BufferSourceContext), + .query_formats = query_formats, + + .init = init_subtitle, + .uninit = uninit, + + .inputs = NULL, + FILTER_OUTPUTS(avfilter_ssrc_sbuffer_outputs), + .priv_class = &sbuffer_class, +}; diff --git a/libavfilter/buffersrc.h b/libavfilter/buffersrc.h index 08fbd18a47..929a2fa249 100644 --- a/libavfilter/buffersrc.h +++ b/libavfilter/buffersrc.h @@ -74,6 +74,7 @@ typedef struct AVBufferSrcParameters { /** * video: the pixel format, value corresponds to enum AVPixelFormat * audio: the sample format, value corresponds to enum AVSampleFormat + * subtitles: the subtitle format, value corresponds to enum AVSubtitleType */ int format; /** diff --git a/libavfilter/version.h b/libavfilter/version.h index 2110048b77..8b75364f1f 100644 --- a/libavfilter/version.h +++ b/libavfilter/version.h @@ -30,7 +30,7 @@ #include "libavutil/version.h" #define LIBAVFILTER_VERSION_MAJOR 8 -#define LIBAVFILTER_VERSION_MINOR 7 +#define LIBAVFILTER_VERSION_MINOR 8 #define LIBAVFILTER_VERSION_MICRO 101 From patchwork Sun Sep 12 03:21:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 30178 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp2863371iov; Sat, 11 Sep 2021 20:23:13 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz+4JMvjzKGEtynMsOtHd7dbuKYQ90y26KSfkG71369DnJ8o4eb7RaG2VnyHosyFhwS1Qzf X-Received: by 2002:a17:906:27c9:: with SMTP id k9mr438444ejc.124.1631416993185; Sat, 11 Sep 2021 20:23:13 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id y19si3662476edm.277.2021.09.11.20.23.12; Sat, 11 Sep 2021 20:23:13 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=PNL9DlGE; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 741F468A980; Sun, 12 Sep 2021 06:22:00 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10olkn2062.outbound.protection.outlook.com [40.92.41.62]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D720B68A9A5 for ; Sun, 12 Sep 2021 06:21:55 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=IKjqJV4JGp76CHB961U8Gn7yEYDEZMxYvr3nb3aCY7Gn6as6c0emhdY2eDzWNsDW1otixuy9PWI3OwZ9MsduEu4G6uNjVDfUlu9K9Y9i1SlzLL08lBkCPHh7PIgt90E44gmdTvF10v+VVAYRSE2V5kwaL05n/rAbIifcTV5TjLW0oZjogGvJMFzK1vBdamsX3QUX3fPFeXOp7XfvckVR5KZlm0N7dNaV4u3/xG55G0rfigOHEFROT46RrVrlWw4ZE5lL6ZBks5EJLDF1ArkaHxJo2gnWLzSUWYms/AXvcK3b7t2OymrwI1drJpbSgmxMbeRdbZzHApV6O/3mgZEz6w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=azIMLQ1EOUlLrgaibMURMDoKaSC4ILB6stYAaFrjKT8=; b=jj7UHGpK0ukBovacH2KsJVZUa9kRlrFU5/l2S1fhurIfPRMtSsgPOK9BCykozgHkJys8IBW7wG0n+GSdGkFzuQXfTAIVhHAgrdld2IOXiPnzAg/rIzW0g9Mfg9xk7daNm2UFfCgNkMwE3hVLYO8ARb8bkHw3gHjdTMaOznotp4nYFTk4nln3JrWM1RqIvVvBeETjb55hen/EaJhkXp1O29pcPhQ8ilnPyDaFFxk/W9D/prt+v5KU5SwHNVmdGvO/jBVVLVJevVsJ3ZK/BjEwBVioV39Ih/ScC2TnzPsbj57V/rM/3j48XyAgsQwv5/gFz0t03l+etvysP+B1IzQ1AA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=azIMLQ1EOUlLrgaibMURMDoKaSC4ILB6stYAaFrjKT8=; b=PNL9DlGEgn5dNetrluh86k0r3pPsV5koW/3t+MQvCp8qiGTf4Fc6dM3FkGHEa3BRRRqv5Ei4hM5/LTYYcjtdobQ5mwUiIBXUSqpWcYIvFRNsNJ5PXGlpaOQ0R0yIOlxGbztCtwhCRmNJCia9OrghUTRVRqlnkQWGjSNtu76AWNeu/NzoVpxXCnd7QIKD1AHZ4WdVPX5ZCJIDji27TjV4OLzR2eNg1l3ZWvk12DmH5aab50ZjKCTo8Q4pD4BiQNaj/KSRDCTcx8RIabWGRfYdDTR0sG3oFNBhZNAsz1CHQ66LXUrM8DHaq3UGAFdRJROeQaVK/GPEwo/1fStcJlQSNw== Received: from MN2PR04MB5981.namprd04.prod.outlook.com (2603:10b6:208:da::10) by MN2PR04MB5728.namprd04.prod.outlook.com (2603:10b6:208:3d::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Sun, 12 Sep 2021 03:21:54 +0000 Received: from MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb]) by MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb%5]) with mapi id 15.20.4500.018; Sun, 12 Sep 2021 03:21:54 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v5 08/12] avfilter/overlay_graphicsubs: Add overlay_graphicsubs and graphicsub2video filters Thread-Index: AQHXp4VVlxd/hZsYxkGtcCYuOOLHjw== Date: Sun, 12 Sep 2021 03:21:54 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [OL2q0Lw/zo4ygW8j3MD8GNSNX5mVU4y6Fgf/6LQh5Qk=] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 4b40c0cd-fb84-4aa4-d77c-08d9759c7859 x-ms-traffictypediagnostic: MN2PR04MB5728: x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: TShHHz4TAI7kJgtR7oxNeUMjeQcft852aD62k4TtjZNPGTGaoCsnWbn2vpUw85cYMLdKRsZu2W8HHemni3JXtnedE4x7kbLAPRVy/1AruJAVnWmtn55PE5dcJXIAzL5ljqwWTsJh152Rui/hGDPUzlSuBjcgW2N0Gx1MsnWdSPAP519DGh51J0OkD5qrpT5wOWk1Au+MT5jN6IRP0EOCyRbcz+Wj+5ajN+i2aIBbvr0qT66/tR8hnCxY+Pk5x2CTaCQkaR0k43nnVefuoDaHaiiwKEp/UpEPvm1GlntihMz0fh8GM80Ke2/nc/XIhUpuDRK0wNloo/BfmqRK5Zi2DVtZ/DrAEgRpdFWPAt5P+PYsd3XaeEMwB7JZdbbqhu6ZHPOJyP4PMTQwig7BbcKbm+9pgBuTx+SCMyhlGIEikZ7C8IWwGcf8Hg3YDRMEEtyUiqUszDc0okFzh4O9y3t1C4F5t/LyieYMzC7hejurEsg= x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: ngTtyclT3rpiLKNNYrDM5PfKdnZdpQ36mjbdQSrn8wvwgZd+UnrQl0nTksY/DyVE0xrNRRlBy9GrlMfDdkEoTmOa9HtV+/RYqRTSBugtEpFeuD94JHoIddml4zzMjP+EVIXvzl2fUvSaMW9w+y6MSA== x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-3174-20-msonline-outlook-529c7.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR04MB5981.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 4b40c0cd-fb84-4aa4-d77c-08d9759c7859 X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Sep 2021 03:21:54.3318 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR04MB5728 Subject: [FFmpeg-devel] [PATCH v5 08/12] avfilter/overlay_graphicsubs: Add overlay_graphicsubs and graphicsub2video filters X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 4+0qOyGma/U6 Signed-off-by: softworkz --- doc/filters.texi | 104 ++++ libavfilter/Makefile | 2 + libavfilter/allfilters.c | 2 + libavfilter/vf_overlay_graphicsubs.c | 731 +++++++++++++++++++++++++++ 4 files changed, 839 insertions(+) create mode 100644 libavfilter/vf_overlay_graphicsubs.c diff --git a/doc/filters.texi b/doc/filters.texi index 9ad6031d23..a6da018259 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -25016,6 +25016,110 @@ tools. @c man end VIDEO SINKS +@chapter Subtitle Filters +@c man begin SUBTITLE FILTERS + +When you configure your FFmpeg build, you can disable any of the +existing filters using @code{--disable-filters}. + +Below is a description of the currently available subtitle filters. + +@section graphicsub2video + +Renders graphic subtitles as video frames. + +This filter replaces the previous "sub2video" hack which did the conversion implicitly and up-front as subtitle filtering wasn't possible at that time. +To retain compatibility with earlier sub2video command lines, this filter is being auto-inserted in those cases. + +For overlaying graphicsal subtitles it is recommended to use the 'overlay_graphicsubs' filter which is more efficient and takes less processing resources. + +This filter is still useful in cases where the overlay is done with hardware acceleration (e.g. overlay_qsv, overlay_vaapi, overlay_cuda) for preparing the overlay frames. + +It accepts the following parameters: + +@table @option +@item size, s +Set the size of the output video frame. + +@end table + +@subsection Examples + +@itemize +@item +Overlay PGS subtitles +(not recommended - better use overlay_graphicsubs) +@example +ffmpeg -i "https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv" -filter_complex "[0:1]graphicsub2video[subs];[0:0][subs]overlay" output.mp4 +@end example + +@item +Overlay PGS subtitles implicitly +The graphicsub2video is inserted automatically for compatibility with legacy command lines. +@example +ffmpeg -i "https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv" -filter_complex "[0:0][0:1]overlay" output.mp4 +@end example +@end itemize + +@section overlay_graphicsubs + +Overlay graphic subtitles onto a video stream. + +This filter can blend graphical subtitles on a video stream directly, i.e. without creating full-size alpha images first. +The blending operation is limited to the area of the subtitle rectangles, which also means that no processing is done at times where no subtitles are to be displayed. + + +It accepts the following parameters: + +@table @option +@item x +@item y +Set the expression for the x and y coordinates of the overlaid video +on the main video. Default value is "0" for both expressions. In case +the expression is invalid, it is set to a huge value (meaning that the +overlay will not be displayed within the output visible area). + +@item eof_action +See @ref{framesync}. + +@item eval +Set when the expressions for @option{x}, and @option{y} are evaluated. + +It accepts the following values: +@table @samp +@item init +only evaluate expressions once during the filter initialization or +when a command is processed + +@item frame +evaluate expressions for each incoming frame +@end table + +Default value is @samp{frame}. + +@item shortest +See @ref{framesync}. + +@end table + +@subsection Examples + +@itemize +@item +Overlay PGS subtitles +@example +ffmpeg -i "https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv" -filter_complex "[0:1]graphicsub2video[subs];[0:0][subs]overlay" output.mp4 +@end example + +@item +Overlay PGS subtitles implicitly +The graphicsub2video is inserted automatically for compatibility with legacy command lines. +@example +ffmpeg -i "https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv" -filter_complex "[0:0][0:1]overlay" output.mp4 +@end example +@end itemize +@c man end SUBTITLE FILTERS + @chapter Multimedia Filters @c man begin MULTIMEDIA FILTERS diff --git a/libavfilter/Makefile b/libavfilter/Makefile index a510dd12b4..f9b33d4ead 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -289,6 +289,7 @@ OBJS-$(CONFIG_FSPP_FILTER) += vf_fspp.o qp_table.o OBJS-$(CONFIG_GBLUR_FILTER) += vf_gblur.o OBJS-$(CONFIG_GEQ_FILTER) += vf_geq.o OBJS-$(CONFIG_GRADFUN_FILTER) += vf_gradfun.o +OBJS-$(CONFIG_GRAPHICSUB2VIDEO_FILTER) += vf_overlay_graphicsubs.o framesync.o OBJS-$(CONFIG_GRAPHMONITOR_FILTER) += f_graphmonitor.o OBJS-$(CONFIG_GRAYWORLD_FILTER) += vf_grayworld.o OBJS-$(CONFIG_GREYEDGE_FILTER) += vf_colorconstancy.o @@ -362,6 +363,7 @@ OBJS-$(CONFIG_OVERLAY_CUDA_FILTER) += vf_overlay_cuda.o framesync.o vf OBJS-$(CONFIG_OVERLAY_OPENCL_FILTER) += vf_overlay_opencl.o opencl.o \ opencl/overlay.o framesync.o OBJS-$(CONFIG_OVERLAY_QSV_FILTER) += vf_overlay_qsv.o framesync.o +OBJS-$(CONFIG_OVERLAY_GRAPHICSUBS_FILTER) += vf_overlay_graphicsubs.o framesync.o OBJS-$(CONFIG_OVERLAY_VULKAN_FILTER) += vf_overlay_vulkan.o vulkan.o OBJS-$(CONFIG_OWDENOISE_FILTER) += vf_owdenoise.o OBJS-$(CONFIG_PAD_FILTER) += vf_pad.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index b27ef6f027..ca983db5b4 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -344,6 +344,7 @@ extern const AVFilter ff_vf_oscilloscope; extern const AVFilter ff_vf_overlay; extern const AVFilter ff_vf_overlay_opencl; extern const AVFilter ff_vf_overlay_qsv; +extern const AVFilter ff_vf_overlay_graphicsubs; extern const AVFilter ff_vf_overlay_vulkan; extern const AVFilter ff_vf_overlay_cuda; extern const AVFilter ff_vf_owdenoise; @@ -522,6 +523,7 @@ extern const AVFilter ff_avf_showvolume; extern const AVFilter ff_avf_showwaves; extern const AVFilter ff_avf_showwavespic; extern const AVFilter ff_vaf_spectrumsynth; +extern const AVFilter ff_svf_graphicsub2video; /* multimedia sources */ extern const AVFilter ff_avsrc_amovie; diff --git a/libavfilter/vf_overlay_graphicsubs.c b/libavfilter/vf_overlay_graphicsubs.c new file mode 100644 index 0000000000..15c07c9bc4 --- /dev/null +++ b/libavfilter/vf_overlay_graphicsubs.c @@ -0,0 +1,731 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * overlay graphical subtitles on top of a video frame + */ + +#include "avfilter.h" +#include "formats.h" +#include "libavutil/common.h" +#include "libavutil/eval.h" +#include "libavutil/avstring.h" +#include "libavutil/pixdesc.h" +#include "libavutil/imgutils.h" +#include "libavutil/opt.h" +#include "internal.h" +#include "drawutils.h" +#include "framesync.h" + +#include "libavcodec/avcodec.h" + +enum var_name { + VAR_MAIN_W, VAR_MW, + VAR_MAIN_H, VAR_MH, + VAR_OVERLAY_W, VAR_OW, + VAR_OVERLAY_H, VAR_OH, + VAR_HSUB, + VAR_VSUB, + VAR_X, + VAR_Y, + VAR_N, + VAR_POS, + VAR_T, + VAR_VARS_NB +}; + +typedef struct OverlaySubsContext { + const AVClass *class; + int x, y; ///< position of overlaid picture + int w, h; + AVFrame *outpicref; + + int main_is_packed_rgb; + uint8_t main_rgba_map[4]; + int main_has_alpha; + uint8_t overlay_rgba_map[4]; + int eval_mode; ///< EvalMode + + FFFrameSync fs; + + int main_pix_step[4]; ///< steps per pixel for each plane of the main output + int hsub, vsub; ///< chroma subsampling values + const AVPixFmtDescriptor *main_desc; ///< format descriptor for main input + + double var_values[VAR_VARS_NB]; + char *x_expr, *y_expr; + + AVExpr *x_pexpr, *y_pexpr; +} OverlaySubsContext; + +static const char *const var_names[] = { + "main_w", "W", ///< width of the main video + "main_h", "H", ///< height of the main video + "overlay_w", "w", ///< width of the overlay video + "overlay_h", "h", ///< height of the overlay video + "hsub", + "vsub", + "x", + "y", + "n", ///< number of frame + "pos", ///< position in the file + "t", ///< timestamp expressed in seconds + NULL +}; + +#define MAIN 0 +#define OVERLAY 1 + +#define R 0 +#define G 1 +#define B 2 +#define A 3 + +#define Y 0 +#define U 1 +#define V 2 + +enum EvalMode { + EVAL_MODE_INIT, + EVAL_MODE_FRAME, + EVAL_MODE_NB +}; + +static av_cold void overlay_graphicsubs_uninit(AVFilterContext *ctx) +{ + OverlaySubsContext *s = ctx->priv; + + ff_framesync_uninit(&s->fs); + av_expr_free(s->x_pexpr); s->x_pexpr = NULL; + av_expr_free(s->y_pexpr); s->y_pexpr = NULL; +} + +static inline int normalize_xy(double d, int chroma_sub) +{ + if (isnan(d)) + return INT_MAX; + return (int)d & ~((1 << chroma_sub) - 1); +} + +static void eval_expr(AVFilterContext *ctx) +{ + OverlaySubsContext *s = ctx->priv; + + s->var_values[VAR_X] = av_expr_eval(s->x_pexpr, s->var_values, NULL); + s->var_values[VAR_Y] = av_expr_eval(s->y_pexpr, s->var_values, NULL); + /* It is necessary if x is expressed from y */ + s->var_values[VAR_X] = av_expr_eval(s->x_pexpr, s->var_values, NULL); + s->x = normalize_xy(s->var_values[VAR_X], s->hsub); + s->y = normalize_xy(s->var_values[VAR_Y], s->vsub); +} + +static int set_expr(AVExpr **pexpr, const char *expr, const char *option, void *log_ctx) +{ + int ret; + AVExpr *old = NULL; + + if (*pexpr) + old = *pexpr; + ret = av_expr_parse(pexpr, expr, var_names, + NULL, NULL, NULL, NULL, 0, log_ctx); + if (ret < 0) { + av_log(log_ctx, AV_LOG_ERROR, + "Error when evaluating the expression '%s' for %s\n", + expr, option); + *pexpr = old; + return ret; + } + + av_expr_free(old); + return 0; +} + +static int overlay_graphicsubs_query_formats(AVFilterContext *ctx) +{ + AVFilterFormats *formats; + AVFilterLink *inlink0 = ctx->inputs[0]; + AVFilterLink *inlink1 = ctx->inputs[1]; + AVFilterLink *outlink = ctx->outputs[0]; + int ret; + static const enum AVSubtitleType subtitle_fmts[] = { AV_SUBTITLE_FMT_BITMAP, AV_SUBTITLE_FMT_NONE }; + static const enum AVPixelFormat supported_pix_fmts[] = { + AV_PIX_FMT_YUV420P, AV_PIX_FMT_YUV422P, AV_PIX_FMT_YUV444P, + AV_PIX_FMT_ARGB, AV_PIX_FMT_RGBA, + AV_PIX_FMT_ABGR, AV_PIX_FMT_BGRA, + AV_PIX_FMT_RGB24, AV_PIX_FMT_BGR24, + AV_PIX_FMT_NONE + }; + + /* set input0 video formats */ + formats = ff_make_format_list(supported_pix_fmts); + if ((ret = ff_formats_ref(formats, &inlink0->outcfg.formats)) < 0) + return ret; + + /* set input1 subtitle formats */ + formats = ff_make_format_list(subtitle_fmts); + if ((ret = ff_formats_ref(formats, &inlink1->outcfg.formats)) < 0) + return ret; + + /* set output0 video formats */ + formats = ff_make_format_list(supported_pix_fmts); + if ((ret = ff_formats_ref(formats, &outlink->incfg.formats)) < 0) + return ret; + + return 0; +} + +static int config_output(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + OverlaySubsContext *s = ctx->priv; + int ret; + + if ((ret = ff_framesync_init_dualinput(&s->fs, ctx)) < 0) + return ret; + + outlink->w = ctx->inputs[MAIN]->w; + outlink->h = ctx->inputs[MAIN]->h; + outlink->time_base = ctx->inputs[MAIN]->time_base; + + return ff_framesync_configure(&s->fs); +} + +// divide by 255 and round to nearest +// apply a fast variant: (X+127)/255 = ((X+127)*257+257)>>16 = ((X+128)*257)>>16 +#define FAST_DIV255(x) ((((x) + 128) * 257) >> 16) + +// calculate the non-pre-multiplied alpha, applying the general equation: +// alpha = alpha_overlay / ( (alpha_main + alpha_overlay) - (alpha_main * alpha_overlay) ) +// (((x) << 16) - ((x) << 9) + (x)) is a faster version of: 255 * 255 * x +// ((((x) + (y)) << 8) - ((x) + (y)) - (y) * (x)) is a faster version of: 255 * (x + y) +#define UNPREMULTIPLY_ALPHA(x, y) ((((x) << 16) - ((x) << 9) + (x)) / ((((x) + (y)) << 8) - ((x) + (y)) - (y) * (x))) + +/** + * Blend image in src to destination buffer dst at position (x, y). + */ +static av_always_inline void blend_packed_rgb(const AVFilterContext *ctx, + const AVFrame *dst, const AVSubtitleRect *src, + int x, int y, + int is_straight) +{ + OverlaySubsContext *s = ctx->priv; + int i, imax, j, jmax; + const int src_w = src->w; + const int src_h = src->h; + const int dst_w = dst->width; + const int dst_h = dst->height; + uint8_t alpha; ///< the amount of overlay to blend on to main + const int dr = s->main_rgba_map[R]; + const int dg = s->main_rgba_map[G]; + const int db = s->main_rgba_map[B]; + const int da = s->main_rgba_map[A]; + const int dstep = s->main_pix_step[0]; + const int sr = s->overlay_rgba_map[R]; + const int sg = s->overlay_rgba_map[G]; + const int sb = s->overlay_rgba_map[B]; + const int sa = s->overlay_rgba_map[A]; + uint32_t *pal = (uint32_t *)src->data[1]; + int slice_start, slice_end; + uint8_t *S, *sp, *d, *dp; + + i = FFMAX(-y, 0); + imax = FFMIN3(-y + dst_h, FFMIN(src_h, dst_h), y + src_h); + + slice_start = i; + slice_end = i + imax; + + sp = src->data[0] + slice_start * src->linesize[0]; + dp = dst->data[0] + (slice_start + y) * dst->linesize[0]; + + for (i = slice_start; i < slice_end; i++) { + j = FFMAX(-x, 0); + S = sp + j; + d = dp + ((x + j) * dstep); + + for (jmax = FFMIN(-x + dst_w, src_w); j < jmax; j++) { + uint32_t val = pal[*S]; + const uint8_t *sval = (uint8_t *)&val; + alpha = sval[sa]; + + // if the main channel has an alpha channel, alpha has to be calculated + // to create an un-premultiplied (straight) alpha value + if (s->main_has_alpha && alpha != 0 && alpha != 255) { + const uint8_t alpha_d = d[da]; + alpha = UNPREMULTIPLY_ALPHA(alpha, alpha_d); + } + + switch (alpha) { + case 0: + break; + case 255: + d[dr] = sval[sr]; + d[dg] = sval[sg]; + d[db] = sval[sb]; + break; + default: + // main_value = main_value * (1 - alpha) + overlay_value * alpha + // since alpha is in the range 0-255, the result must divided by 255 + d[dr] = is_straight ? FAST_DIV255(d[dr] * (255 - alpha) + sval[sr] * alpha) : + FFMIN(FAST_DIV255(d[dr] * (255 - alpha)) + sval[sr], 255); + d[dg] = is_straight ? FAST_DIV255(d[dg] * (255 - alpha) + sval[sg] * alpha) : + FFMIN(FAST_DIV255(d[dg] * (255 - alpha)) + sval[sg], 255); + d[db] = is_straight ? FAST_DIV255(d[db] * (255 - alpha) + sval[sb] * alpha) : + FFMIN(FAST_DIV255(d[db] * (255 - alpha)) + sval[sb], 255); + } + + if (s->main_has_alpha) { + switch (alpha) { + case 0: + break; + case 255: + d[da] = sval[sa]; + break; + default: + // apply alpha compositing: main_alpha += (1-main_alpha) * overlay_alpha + d[da] += FAST_DIV255((255 - d[da]) * S[sa]); + } + } + d += dstep; + S += 1; + } + dp += dst->linesize[0]; + sp += src->linesize[0]; + } +} + +static av_always_inline void blend_plane_8_8bits(const AVFilterContext *ctx, const AVFrame *dst, const AVSubtitleRect *src, + const uint32_t *yuv_pal, int src_w, int src_h, int dst_w, int dst_h, int plane, int hsub, int vsub, + int x, int y, int dst_plane, int dst_offset, int dst_step) +{ + const int src_wp = AV_CEIL_RSHIFT(src_w, hsub); + const int src_hp = AV_CEIL_RSHIFT(src_h, vsub); + const int dst_wp = AV_CEIL_RSHIFT(dst_w, hsub); + const int dst_hp = AV_CEIL_RSHIFT(dst_h, vsub); + const int yp = y >> vsub; + const int xp = x >> hsub; + uint8_t *s, *sp, *d, *dp, *dap; + int imax, i, j, jmax; + int slice_start, slice_end; + + i = FFMAX(-yp, 0); \ + imax = FFMIN3(-yp + dst_hp, FFMIN(src_hp, dst_hp), yp + src_hp); \ + + slice_start = i; + slice_end = i + imax; + + sp = src->data[0] + (slice_start << vsub) * src->linesize[0]; + dp = dst->data[dst_plane] + (yp + slice_start) * dst->linesize[dst_plane] + dst_offset; + + dap = dst->data[3] + ((yp + slice_start) << vsub) * dst->linesize[3]; + + for (i = slice_start; i < slice_end; i++) { + j = FFMAX(-xp, 0); + d = dp + (xp + j) * dst_step; + s = sp + (j << hsub); + jmax = FFMIN(-xp + dst_wp, src_wp); + + for (; j < jmax; j++) { + uint32_t val = yuv_pal[*s]; + const uint8_t *sval = (uint8_t *)&val; + const int alpha = sval[3]; + const int max = 255, mid = 128; + const int d_int = *d; + const int sval_int = sval[plane]; + + switch (alpha) { + case 0: + break; + case 255: + *d = sval[plane]; + break; + default: + if (plane > 0) + *d = av_clip(FAST_DIV255((d_int - mid) * (max - alpha) + (sval_int - mid) * alpha) , -mid, mid) + mid; + else + *d = FAST_DIV255(d_int * (max - alpha) + sval_int * alpha); + break; + } + + d += dst_step; + s += 1 << hsub; + } + dp += dst->linesize[dst_plane]; + sp += (1 << vsub) * src->linesize[0]; + dap += (1 << vsub) * dst->linesize[3]; + } +} + +#define RGB2Y(r, g, b) (uint8_t)(((66 * (r) + 129 * (g) + 25 * (b) + 128) >> 8) + 16) +#define RGB2U(r, g, b) (uint8_t)(((-38 * (r) - 74 * (g) + 112 * (b) + 128) >> 8) + 128) +#define RGB2V(r, g, b) (uint8_t)(((112 * (r) - 94 * (g) - 18 * (b) + 128) >> 8) + 128) +/* Converts R8 G8 B8 color to YUV. */ +static av_always_inline void rgb_2_yuv(uint8_t r, uint8_t g, uint8_t b, uint8_t* y, uint8_t* u, uint8_t* v) +{ + *y = RGB2Y((int)r, (int)g, (int)b); + *u = RGB2U((int)r, (int)g, (int)b); + *v = RGB2V((int)r, (int)g, (int)b); +} + + +static av_always_inline void blend_yuv_8_8bits(AVFilterContext *ctx, AVFrame *dst, const AVSubtitleRect *src, int hsub, int vsub, int x, int y) +{ + OverlaySubsContext *s = ctx->priv; + const int src_w = src->w; + const int src_h = src->h; + const int dst_w = dst->width; + const int dst_h = dst->height; + const int sr = s->overlay_rgba_map[R]; + const int sg = s->overlay_rgba_map[G]; + const int sb = s->overlay_rgba_map[B]; + const int sa = s->overlay_rgba_map[A]; + uint32_t *pal = (uint32_t *)src->data[1]; + uint32_t yuvpal[256]; + + for (int i = 0; i < 256; ++i) { + const uint8_t *rgba = (uint8_t *)&pal[i]; + uint8_t *yuva = (uint8_t *)&yuvpal[i]; + rgb_2_yuv(rgba[sr], rgba[sg], rgba[sb], &yuva[Y], &yuva[U], &yuva[V]); + yuva[3] = rgba[sa]; + } + + blend_plane_8_8bits(ctx, dst, src, yuvpal, src_w, src_h, dst_w, dst_h, Y, 0, 0, x, y, s->main_desc->comp[Y].plane, s->main_desc->comp[Y].offset, s->main_desc->comp[Y].step); + blend_plane_8_8bits(ctx, dst, src, yuvpal, src_w, src_h, dst_w, dst_h, U, hsub, vsub, x, y, s->main_desc->comp[U].plane, s->main_desc->comp[U].offset, s->main_desc->comp[U].step); + blend_plane_8_8bits(ctx, dst, src, yuvpal, src_w, src_h, dst_w, dst_h, V, hsub, vsub, x, y, s->main_desc->comp[V].plane, s->main_desc->comp[V].offset, s->main_desc->comp[V].step); +} + +static int config_input_main(AVFilterLink *inlink) +{ + int ret; + AVFilterContext *ctx = inlink->dst; + OverlaySubsContext *s = inlink->dst->priv; + const AVPixFmtDescriptor *pix_desc = av_pix_fmt_desc_get(inlink->format); + + av_image_fill_max_pixsteps(s->main_pix_step, NULL, pix_desc); + ff_fill_rgba_map(s->overlay_rgba_map, AV_PIX_FMT_RGB32); // it's actually AV_PIX_FMT_PAL8); + + s->hsub = pix_desc->log2_chroma_w; + s->vsub = pix_desc->log2_chroma_h; + + s->main_desc = pix_desc; + + s->main_is_packed_rgb = ff_fill_rgba_map(s->main_rgba_map, inlink->format) >= 0; + s->main_has_alpha = !(pix_desc->flags & AV_PIX_FMT_FLAG_ALPHA); + + /* Finish the configuration by evaluating the expressions + now when both inputs are configured. */ + s->var_values[VAR_MAIN_W ] = s->var_values[VAR_MW] = ctx->inputs[MAIN ]->w; + s->var_values[VAR_MAIN_H ] = s->var_values[VAR_MH] = ctx->inputs[MAIN ]->h; + s->var_values[VAR_OVERLAY_W] = s->var_values[VAR_OW] = ctx->inputs[OVERLAY]->w; + s->var_values[VAR_OVERLAY_H] = s->var_values[VAR_OH] = ctx->inputs[OVERLAY]->h; + s->var_values[VAR_HSUB] = 1<log2_chroma_w; + s->var_values[VAR_VSUB] = 1<log2_chroma_h; + s->var_values[VAR_X] = NAN; + s->var_values[VAR_Y] = NAN; + s->var_values[VAR_N] = 0; + s->var_values[VAR_T] = NAN; + s->var_values[VAR_POS] = NAN; + + if ((ret = set_expr(&s->x_pexpr, s->x_expr, "x", ctx)) < 0 || + (ret = set_expr(&s->y_pexpr, s->y_expr, "y", ctx)) < 0) + return ret; + + if (s->eval_mode == EVAL_MODE_INIT) { + eval_expr(ctx); + av_log(ctx, AV_LOG_VERBOSE, "x:%f xi:%d y:%f yi:%d\n", + s->var_values[VAR_X], s->x, + s->var_values[VAR_Y], s->y); + } + + av_log(ctx, AV_LOG_VERBOSE, + "main w:%d h:%d fmt:%s overlay w:%d h:%d fmt:%s\n", + ctx->inputs[MAIN]->w, ctx->inputs[MAIN]->h, + av_get_pix_fmt_name(ctx->inputs[MAIN]->format), + ctx->inputs[OVERLAY]->w, ctx->inputs[OVERLAY]->h, + av_get_pix_fmt_name(ctx->inputs[OVERLAY]->format)); + return 0; +} + +static int do_blend(FFFrameSync *fs) +{ + AVFilterContext *ctx = fs->parent; + AVFrame *mainpic, *second; + OverlaySubsContext *s = ctx->priv; + AVFilterLink *inlink = ctx->inputs[0]; + unsigned num_rects, i; + int ret; + + ret = ff_framesync_dualinput_get_writable(fs, &mainpic, &second); + if (ret < 0) + return ret; + if (!second) + return ff_filter_frame(ctx->outputs[0], mainpic); + + if (s->eval_mode == EVAL_MODE_FRAME) { + int64_t pos = mainpic->pkt_pos; + + s->var_values[VAR_N] = inlink->frame_count_out; + s->var_values[VAR_T] = mainpic->pts == AV_NOPTS_VALUE ? + NAN : mainpic->pts * av_q2d(inlink->time_base); + s->var_values[VAR_POS] = pos == -1 ? NAN : pos; + + s->var_values[VAR_OVERLAY_W] = s->var_values[VAR_OW] = second->width; + s->var_values[VAR_OVERLAY_H] = s->var_values[VAR_OH] = second->height; + s->var_values[VAR_MAIN_W ] = s->var_values[VAR_MW] = mainpic->width; + s->var_values[VAR_MAIN_H ] = s->var_values[VAR_MH] = mainpic->height; + + eval_expr(ctx); + av_log(ctx, AV_LOG_DEBUG, "n:%f t:%f pos:%f x:%f xi:%d y:%f yi:%d\n", + s->var_values[VAR_N], s->var_values[VAR_T], s->var_values[VAR_POS], + s->var_values[VAR_X], s->x, + s->var_values[VAR_Y], s->y); + } + + num_rects = second->num_subtitle_rects > 1 ? 1 : second->num_subtitle_rects; + + for (i = 0; i < num_rects; i++) { + const AVSubtitleRect *sub_rect = second->subtitle_rects[i]; + + if (sub_rect->type != AV_SUBTITLE_FMT_BITMAP) { + av_log(NULL, AV_LOG_WARNING, "sub2video: non-bitmap subtitle\n"); + return AVERROR_INVALIDDATA; + } + + switch (inlink->format) { + case AV_PIX_FMT_YUV420P: + blend_yuv_8_8bits(ctx, mainpic, sub_rect, 1, 1, sub_rect->x + s->x, sub_rect->y + s->y); + break; + case AV_PIX_FMT_YUV422P: + blend_yuv_8_8bits(ctx, mainpic, sub_rect, 1, 0, sub_rect->x + s->x, sub_rect->y + s->y); + break; + case AV_PIX_FMT_YUV444P: + blend_yuv_8_8bits(ctx, mainpic, sub_rect, 0, 0, sub_rect->x + s->x, sub_rect->y + s->y); + break; + case AV_PIX_FMT_RGB24: + case AV_PIX_FMT_BGR24: + case AV_PIX_FMT_ARGB: + case AV_PIX_FMT_RGBA: + case AV_PIX_FMT_BGRA: + case AV_PIX_FMT_ABGR: + blend_packed_rgb(ctx, mainpic, sub_rect, sub_rect->x + s->x, sub_rect->y + s->y, 1); + break; + default: + av_assert0(0); + break; + } + } + + return ff_filter_frame(ctx->outputs[0], mainpic); +} + +static av_cold int overlay_graphicsubs_init(AVFilterContext *ctx) +{ + OverlaySubsContext *s = ctx->priv; + + s->fs.on_event = do_blend; + return 0; +} + +static int overlay_graphicsubs_activate(AVFilterContext *ctx) +{ + OverlaySubsContext *s = ctx->priv; + return ff_framesync_activate(&s->fs); +} + +static int graphicsub2video_query_formats(AVFilterContext *ctx) +{ + AVFilterFormats *formats; + AVFilterLink *inlink = ctx->inputs[0]; + AVFilterLink *outlink = ctx->outputs[0]; + static const enum AVSubtitleType subtitle_fmts[] = { AV_SUBTITLE_FMT_BITMAP, AV_SUBTITLE_FMT_NONE }; + static const enum AVPixelFormat pix_fmts[] = { AV_PIX_FMT_RGB32, AV_PIX_FMT_NONE }; + int ret; + + /* set input subtitle formats */ + formats = ff_make_format_list(subtitle_fmts); + if ((ret = ff_formats_ref(formats, &inlink->outcfg.formats)) < 0) + return ret; + + /* set output video formats */ + formats = ff_make_format_list(pix_fmts); + if ((ret = ff_formats_ref(formats, &outlink->incfg.formats)) < 0) + return ret; + + return 0; +} + +static int graphicsub2video_config_input(AVFilterLink *inlink) +{ + AVFilterContext *ctx = inlink->dst; + OverlaySubsContext *s = ctx->priv; + + if (s->w <= 0 || s->h <= 0) { + s->w = inlink->w; + s->h = inlink->h; + } + return 0; +} + +static int graphicsub2video_config_output(AVFilterLink *outlink) +{ + const AVFilterContext *ctx = outlink->src; + OverlaySubsContext *s = ctx->priv; + const AVPixFmtDescriptor *pix_desc = av_pix_fmt_desc_get(outlink->format); + + outlink->w = s->w; + outlink->h = s->h; + outlink->sample_aspect_ratio = (AVRational){1,1}; + + av_image_fill_max_pixsteps(s->main_pix_step, NULL, pix_desc); + ff_fill_rgba_map(s->overlay_rgba_map, AV_PIX_FMT_RGB32); + + s->hsub = pix_desc->log2_chroma_w; + s->vsub = pix_desc->log2_chroma_h; + + s->main_desc = pix_desc; + + s->main_is_packed_rgb = ff_fill_rgba_map(s->main_rgba_map, outlink->format) >= 0; + s->main_has_alpha = !(pix_desc->flags & AV_PIX_FMT_FLAG_ALPHA); + + return 0; +} + +static int graphicsub2video_filter_frame(AVFilterLink *inlink, AVFrame *src_frame) +{ + AVFilterLink *outlink = inlink->dst->outputs[0]; + AVFrame *out; + const unsigned num_rects = src_frame->num_subtitle_rects; + unsigned int i; + + out = ff_get_video_buffer(outlink, outlink->w, outlink->h); + if (!out) + return AVERROR(ENOMEM); + + memset(out->data[0], 0, out->linesize[0] * out->height); + + out->pts = src_frame->pts; + + + for (i = 0; i < num_rects; i++) { + const AVSubtitleRect *sub_rect = src_frame->subtitle_rects[i]; + + if (sub_rect->type != AV_SUBTITLE_FMT_BITMAP) { + av_log(NULL, AV_LOG_WARNING, "sub2video: non-bitmap subtitle\n"); + return AVERROR_INVALIDDATA; + } + + blend_packed_rgb(inlink->dst, out, sub_rect, sub_rect->x, sub_rect->y, 1); + } + + av_frame_free(&src_frame); + return ff_filter_frame(outlink, out); +} + +#define OFFSET(x) offsetof(OverlaySubsContext, x) +#define FLAGS (AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_FILTERING_PARAM) + +static const AVOption overlay_graphicsubs_options[] = { + { "x", "set the x expression", OFFSET(x_expr), AV_OPT_TYPE_STRING, {.str = "0"}, 0, 0, FLAGS }, + { "y", "set the y expression", OFFSET(y_expr), AV_OPT_TYPE_STRING, {.str = "0"}, 0, 0, FLAGS }, + { "eof_action", "Action to take when encountering EOF from secondary input ", + OFFSET(fs.opt_eof_action), AV_OPT_TYPE_INT, { .i64 = EOF_ACTION_REPEAT }, + EOF_ACTION_REPEAT, EOF_ACTION_PASS, .flags = FLAGS, "eof_action" }, + { "repeat", "Repeat the previous frame.", 0, AV_OPT_TYPE_CONST, { .i64 = EOF_ACTION_REPEAT }, .flags = FLAGS, "eof_action" }, + { "endall", "End both streams.", 0, AV_OPT_TYPE_CONST, { .i64 = EOF_ACTION_ENDALL }, .flags = FLAGS, "eof_action" }, + { "pass", "Pass through the main input.", 0, AV_OPT_TYPE_CONST, { .i64 = EOF_ACTION_PASS }, .flags = FLAGS, "eof_action" }, + { "eval", "specify when to evaluate expressions", OFFSET(eval_mode), AV_OPT_TYPE_INT, {.i64 = EVAL_MODE_FRAME}, 0, EVAL_MODE_NB-1, FLAGS, "eval" }, + { "init", "eval expressions once during initialization", 0, AV_OPT_TYPE_CONST, {.i64=EVAL_MODE_INIT}, .flags = FLAGS, .unit = "eval" }, + { "frame", "eval expressions per-frame", 0, AV_OPT_TYPE_CONST, {.i64=EVAL_MODE_FRAME}, .flags = FLAGS, .unit = "eval" }, + { "shortest", "force termination when the shortest input terminates", OFFSET(fs.opt_shortest), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, FLAGS }, + { "repeatlast", "repeat overlay of the last overlay frame", OFFSET(fs.opt_repeatlast), AV_OPT_TYPE_BOOL, {.i64=1}, 0, 1, FLAGS }, + { NULL } +}; + +static const AVOption graphicsub2video_options[] = { + { "size", "set video size", OFFSET(w), AV_OPT_TYPE_IMAGE_SIZE, {.str = NULL}, 0, 0, FLAGS }, + { "s", "set video size", OFFSET(w), AV_OPT_TYPE_IMAGE_SIZE, {.str = NULL}, 0, 0, FLAGS }, + { NULL } +}; + +FRAMESYNC_DEFINE_CLASS(overlay_graphicsubs, OverlaySubsContext, fs); + +static const AVFilterPad overlay_graphicsubs_inputs[] = { + { + .name = "main", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_input_main, + .flags = AVFILTERPAD_FLAG_NEEDS_WRITABLE, + }, + { + .name = "overlay", + .type = AVMEDIA_TYPE_SUBTITLE, + }, +}; + +static const AVFilterPad overlay_graphicsubs_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_output, + }, +}; + +const AVFilter ff_vf_overlay_graphicsubs = { + .name = "overlay_graphicsubs", + .description = NULL_IF_CONFIG_SMALL("Overlay graphical subtitles on top of the input."), + .preinit = overlay_graphicsubs_framesync_preinit, + .init = overlay_graphicsubs_init, + .uninit = overlay_graphicsubs_uninit, + .priv_size = sizeof(OverlaySubsContext), + .priv_class = &overlay_graphicsubs_class, + .query_formats = overlay_graphicsubs_query_formats, + .activate = overlay_graphicsubs_activate, + FILTER_INPUTS(overlay_graphicsubs_inputs), + FILTER_OUTPUTS(overlay_graphicsubs_outputs), +}; + +AVFILTER_DEFINE_CLASS(graphicsub2video); + +static const AVFilterPad graphicsub2video_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .filter_frame = graphicsub2video_filter_frame, + .config_props = graphicsub2video_config_input, + }, +}; + +static const AVFilterPad graphicsub2video_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = graphicsub2video_config_output, + }, +}; + +AVFilter ff_svf_graphicsub2video = { + .name = "graphicsub2video", + .description = NULL_IF_CONFIG_SMALL("Convert graphical subtitles to video"), + .query_formats = graphicsub2video_query_formats, + .priv_size = sizeof(OverlaySubsContext), + .priv_class = &graphicsub2video_class, + FILTER_INPUTS(graphicsub2video_inputs), + FILTER_OUTPUTS(graphicsub2video_outputs), +}; From patchwork Sun Sep 12 03:21:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 30168 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp2863468iov; Sat, 11 Sep 2021 20:23:24 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzPH9kne0ROZTFIgLrAfW998stF3B0jXKgQVBdj7CtE5zd16Rqws+UTVFzMP8ca5So4KSpj X-Received: by 2002:a05:6402:4404:: with SMTP id y4mr6042724eda.52.1631417003776; Sat, 11 Sep 2021 20:23:23 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id qf34si4317228ejc.197.2021.09.11.20.23.23; Sat, 11 Sep 2021 20:23:23 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=iGk+QhAR; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5AB9A68AA47; Sun, 12 Sep 2021 06:22:01 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10olkn2051.outbound.protection.outlook.com [40.92.41.51]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D558268A980 for ; Sun, 12 Sep 2021 06:21:59 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=aiLqvmkxakNvEz2cSs+cJirby4oorHeOsmCtGZATcEvJ+nudIY19uXU+ur53j7YiEM18ce4C5d3G7IlUK5xPsLTDNX/QoivhQzEhWSDeVU1ohl8hKcwVek5XMnNnT6Xghp65BfBmqg7s9gz4B8Pmuwe8H+ZUYDyGP7fNMlDQJRFhNV+G2cdB83WfLLP8CyqNDss8CQYFlUUgMnVtGQJo6rGI9LzZwjNIwOHIMNlyXwEO1v4jKx1fEMU9S/xvwul6sm6AmPiyfmcuBk1D9URKjXHLWIgdjq57ySnFyG4Q2L6or7MHPVCTWUzoq26ckWzyOQQmWb8uh5DpLRmVbhgtzA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=HGFd1SvKLvLHtxbX8OTMNo0BhrrN7rfniWIsUIY+oHo=; b=CTN2xibwdncILHx8ki7YpXq4+PANFv7VzGt4RqFNfvv+4YjRcLmCox7nn5W4KEsTajATsdWEol/O80h/S6AHaU47q11309b5RQ/IvVVViMxNIGlP08257WcLrdgJ3kmvjTE7yAtG3hynaXLjmPcp6ucadpahj/nj/nL95IG3H1uag4tVQcC9KhWu4qoDv0HNVRc7tATz9KFmQlp8a+cOTEWoot9+woT3UECfdfRm2j54Lq+gvsa8hxAFRtENBYIhdbSkwV0kZOU4NOWrKSfSFNrCBilwgh61OwD0eYXfAFs/sFio6Wvk0+P0A9zyehc23jzZGhBEZeQnnUugp3B9UQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HGFd1SvKLvLHtxbX8OTMNo0BhrrN7rfniWIsUIY+oHo=; b=iGk+QhARX0TDyJahZPBCPBuZt3rxfHxy/XHtsVsjWarLnx7ZVn5CdfolXsh8b6aKmETtx7CIHoBnw/k8I3FwfwOp3Zcyxvmtlxv7lqAxOn6VkvHQCsfYXWmWt3eMeA9XkX6IuB46Jk7NRCPdmLmLGsUT+BROGUoMPAnNw/Cb5bn4tddC8BAlGh6o6XJ5qaNbRfI+L56CcTgycncqbqsgXygX4yrlpfTalzEDfDwWzXPWlV0B6LmxDs6N3q2SOQZBUAG4GDTqAT1KiuaFC4Z68ZpLNXHFjmBP30NxmsLxhmxSbeh0cNXQFihCU3S4YTMFz1g8qHDOLsg+2CV+H8zwXA== Received: from MN2PR04MB5981.namprd04.prod.outlook.com (2603:10b6:208:da::10) by MN2PR04MB5728.namprd04.prod.outlook.com (2603:10b6:208:3d::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Sun, 12 Sep 2021 03:21:57 +0000 Received: from MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb]) by MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb%5]) with mapi id 15.20.4500.018; Sun, 12 Sep 2021 03:21:57 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v5 09/12] avfilter/overlay_textsubs: Add overlay_textsubs and textsubs2video filters Thread-Index: AQHXp4VXQPfa6W73wUKk0MPyVqNENQ== Date: Sun, 12 Sep 2021 03:21:57 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [cmqaSNvkQoFpz/AD2mXBCDqH54NGAp0xexc0FSgiU6Q=] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 1fcbb49d-fc39-4c5a-8d39-08d9759c7a1d x-ms-traffictypediagnostic: MN2PR04MB5728: x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: ZyLtnFgrdDIKwsHbkH5/3iAZWIddymxOkIffV5c/IMFzFF3R/5e0H0zO7cWn7cn6NGQunSsyQmPH03cW3XbJ3XTfHO7NL5braGuiA3J96KZYW9tXSX5by9+OtTAphIP8u24I/2xhjXOm5wtIYZQDT5jU4ECNC8XpORp3crXZBjBXz5YAnFnH1qKbGm1lu+YMk80RA5zLuTNpLDaOvKqOKOktdWz4vSOzcyirKBnC3GYbtORnDbtW9enAVrXnjk+8vaYAlDCE4K8rap0+ZKHty9bqobCnfQFEg/N+vBq7iwPksQtHa0xQNzs+mCLj5bkH8Iv2a8gnBpLdmJkf6LXIEqakdMyaxVhMZKRSB8rStow4PQrpL/sEUB+yKbxpIfKAj6bql33UtjUVY5pG2H+oPktf2j3SiNB/0gFfpjxOdIGJM31uH5ELW+/fexTc1/Pbo+Iod7I1ePHSZkm4/s1kzsZlyZvjeYOxawxDXv72fN0= x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: HMcd4IqUxJjwDcWHSUcVUsUQHxxLwcBR4JpBEMUC5jA1SSKvjlIheoMb5VaF0EzHsUxT2RdWlfs1aSo7+QVDJVu3Q/CqgvgvlMxCDdbtbMgqUgqa8vjSK8o3UKGVsooCesBL+FyqqLYe72JE8aZ51A== x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-3174-20-msonline-outlook-529c7.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR04MB5981.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 1fcbb49d-fc39-4c5a-8d39-08d9759c7a1d X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Sep 2021 03:21:57.3101 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR04MB5728 Subject: [FFmpeg-devel] [PATCH v5 09/12] avfilter/overlay_textsubs: Add overlay_textsubs and textsubs2video filters X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: lSlbBvAzu7ai Signed-off-by: softworkz --- configure | 2 + doc/filters.texi | 69 ++++ libavfilter/Makefile | 2 + libavfilter/allfilters.c | 2 + libavfilter/avfilter.c | 18 +- libavfilter/vf_overlay_textsubs.c | 633 ++++++++++++++++++++++++++++++ 6 files changed, 721 insertions(+), 5 deletions(-) create mode 100644 libavfilter/vf_overlay_textsubs.c diff --git a/configure b/configure index 2f2777df9f..7840567d31 100755 --- a/configure +++ b/configure @@ -3624,6 +3624,7 @@ openclsrc_filter_deps="opencl" overlay_opencl_filter_deps="opencl" overlay_qsv_filter_deps="libmfx" overlay_qsv_filter_select="qsvvpp" +overlay_textsubs_filter_deps="avcodec libass" overlay_vulkan_filter_deps="vulkan_lib libglslang" owdenoise_filter_deps="gpl" pad_opencl_filter_deps="opencl" @@ -3669,6 +3670,7 @@ superequalizer_filter_deps="avcodec" superequalizer_filter_select="rdft" surround_filter_deps="avcodec" surround_filter_select="rdft" +textsub2video_filter_deps="avcodec libass" tinterlace_filter_deps="gpl" tinterlace_merge_test_deps="tinterlace_filter" tinterlace_pad_test_deps="tinterlace_filter" diff --git a/doc/filters.texi b/doc/filters.texi index a6da018259..1d76461ada 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -25118,6 +25118,75 @@ The graphicsub2video is inserted automatically for compatibility with legacy com ffmpeg -i "https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv" -filter_complex "[0:0][0:1]overlay" output.mp4 @end example @end itemize + +@section overlay_textsubs + +Overlay text subtitles onto a video stream. + +This filter supersedes the classic @ref{subtitles} filter opposed to which it does no longer require to open and access the source stream separately, which is often causing problems or doesn't even work for non-local or slow sources. + +Inputs: +- 0: Video [YUV420P, YUV422P, YUV444P, ARGB, RGBA, ABGR, BGRA, RGB24, BGR24] +- 1: Subtitles [text] + +Outputs: +- 0: Video (same as input) + +It accepts the following parameters: + +@table @option + +@item alpha +Process alpha channel, by default alpha channel is untouched. + +@item fonts_dir +Set a directory path containing fonts that can be used by the filter. +These fonts will be used in addition to whatever the font provider uses. + +@item force_style +Override default style or script info parameters of the subtitles. It accepts a +string containing ASS style format @code{KEY=VALUE} couples separated by ",". + +@end table + +@section textsub2video + +Converts text subtitles to video frames. + +For overlaying text subtitles onto video frames it is recommended to use the overlay_textsubs filter. +The textsub2video is useful for for creating transparent text-frames when overlay is done via hw acceleration + +Inputs: +- 0: Subtitles [text] + +Outputs: +- 0: Video [ARGB, RGBA, ABGR, BGRA] + +It accepts the following parameters: + +@table @option + +@item rate, r +Set the framerate for updating overlay frames. +Normally, overlay frames will only be updated each time when the subtitles to display are changing. +In cases where subtitles include advanced features (like animation), this parameter determines the frequency by which the overlay frames should be updated. + +@item size, s +Set the output frame size. +Allows to override the size of output video frames. + +@item alpha +Process alpha channel, by default alpha channel is untouched. + +@item fonts_dir +Set a directory path containing fonts that can be used by the filter. +These fonts will be used in addition to whatever the font provider uses. + +@item force_style +Override default style or script info parameters of the subtitles. It accepts a +string containing ASS style format @code{KEY=VALUE} couples separated by ",". + +@end table @c man end SUBTITLE FILTERS @chapter Multimedia Filters diff --git a/libavfilter/Makefile b/libavfilter/Makefile index f9b33d4ead..0e752c5bf9 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -364,6 +364,7 @@ OBJS-$(CONFIG_OVERLAY_OPENCL_FILTER) += vf_overlay_opencl.o opencl.o \ opencl/overlay.o framesync.o OBJS-$(CONFIG_OVERLAY_QSV_FILTER) += vf_overlay_qsv.o framesync.o OBJS-$(CONFIG_OVERLAY_GRAPHICSUBS_FILTER) += vf_overlay_graphicsubs.o framesync.o +OBJS-$(CONFIG_OVERLAY_TEXTSUBS_FILTER) += vf_overlay_textsubs.o OBJS-$(CONFIG_OVERLAY_VULKAN_FILTER) += vf_overlay_vulkan.o vulkan.o OBJS-$(CONFIG_OWDENOISE_FILTER) += vf_owdenoise.o OBJS-$(CONFIG_PAD_FILTER) += vf_pad.o @@ -451,6 +452,7 @@ OBJS-$(CONFIG_SWAPRECT_FILTER) += vf_swaprect.o OBJS-$(CONFIG_SWAPUV_FILTER) += vf_swapuv.o OBJS-$(CONFIG_TBLEND_FILTER) += vf_blend.o framesync.o OBJS-$(CONFIG_TELECINE_FILTER) += vf_telecine.o +OBJS-$(CONFIG_TEXTSUB2VIDEO_FILTER) += vf_overlay_textsubs.o OBJS-$(CONFIG_THISTOGRAM_FILTER) += vf_histogram.o OBJS-$(CONFIG_THRESHOLD_FILTER) += vf_threshold.o framesync.o OBJS-$(CONFIG_THUMBNAIL_FILTER) += vf_thumbnail.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index ca983db5b4..77463aa4c8 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -345,6 +345,7 @@ extern const AVFilter ff_vf_overlay; extern const AVFilter ff_vf_overlay_opencl; extern const AVFilter ff_vf_overlay_qsv; extern const AVFilter ff_vf_overlay_graphicsubs; +extern const AVFilter ff_vf_overlay_textsubs; extern const AVFilter ff_vf_overlay_vulkan; extern const AVFilter ff_vf_overlay_cuda; extern const AVFilter ff_vf_owdenoise; @@ -524,6 +525,7 @@ extern const AVFilter ff_avf_showwaves; extern const AVFilter ff_avf_showwavespic; extern const AVFilter ff_vaf_spectrumsynth; extern const AVFilter ff_svf_graphicsub2video; +extern const AVFilter ff_svf_textsub2video; /* multimedia sources */ extern const AVFilter ff_avsrc_amovie; diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c index 961e55e7b4..e28171e919 100644 --- a/libavfilter/avfilter.c +++ b/libavfilter/avfilter.c @@ -463,7 +463,7 @@ static int64_t guess_status_pts(AVFilterContext *ctx, int status, AVRational lin return AV_NOPTS_VALUE; } -static int ff_request_frame_to_filter(AVFilterLink *link) +static int ff_request_frame_to_filter(AVFilterLink *link, int input_index) { int ret = -1; @@ -472,8 +472,8 @@ static int ff_request_frame_to_filter(AVFilterLink *link) link->frame_blocked_in = 1; if (link->srcpad->request_frame) ret = link->srcpad->request_frame(link); - else if (link->src->inputs[0]) - ret = ff_request_frame(link->src->inputs[0]); + else if (link->src->inputs[input_index]) + ret = ff_request_frame(link->src->inputs[input_index]); if (ret < 0) { if (ret != AVERROR(EAGAIN) && ret != link->status_in) ff_avfilter_link_set_in_status(link, ret, guess_status_pts(link->src, ret, link->time_base)); @@ -1171,6 +1171,14 @@ static int forward_status_change(AVFilterContext *filter, AVFilterLink *in) { unsigned out = 0, progress = 0; int ret; + int input_index = 0; + + for (int i = 0; i < in->dst->nb_inputs; i++) { + if (&in->dst->input_pads[i] == in->dstpad) { + input_index = i; + break; + } + } av_assert0(!in->status_out); if (!filter->nb_outputs) { @@ -1180,7 +1188,7 @@ static int forward_status_change(AVFilterContext *filter, AVFilterLink *in) while (!in->status_out) { if (!filter->outputs[out]->status_in) { progress++; - ret = ff_request_frame_to_filter(filter->outputs[out]); + ret = ff_request_frame_to_filter(filter->outputs[out], input_index); if (ret < 0) return ret; } @@ -1217,7 +1225,7 @@ static int ff_filter_activate_default(AVFilterContext *filter) for (i = 0; i < filter->nb_outputs; i++) { if (filter->outputs[i]->frame_wanted_out && !filter->outputs[i]->frame_blocked_in) { - return ff_request_frame_to_filter(filter->outputs[i]); + return ff_request_frame_to_filter(filter->outputs[i], 0); } } return FFERROR_NOT_READY; diff --git a/libavfilter/vf_overlay_textsubs.c b/libavfilter/vf_overlay_textsubs.c new file mode 100644 index 0000000000..2e9e193348 --- /dev/null +++ b/libavfilter/vf_overlay_textsubs.c @@ -0,0 +1,633 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * overlay text subtitles on top of a video frame + */ + +#include +#include + +#include "avfilter.h" +#include "formats.h" +#include "libavutil/common.h" +#include "libavutil/avstring.h" +#include "libavutil/opt.h" +#include "internal.h" +#include "drawutils.h" +#include "filters.h" + +#include "libavcodec/avcodec.h" +#include "libavutil/pixdesc.h" + +typedef struct TextSubsContext { + const AVClass *class; + ASS_Library *library; + ASS_Renderer *renderer; + ASS_Track *track; + + AVFrame *last_sub; + + char *default_font_path; + char *fonts_dir; + char *fc_file; + double font_scale; + double font_size; + char *force_style; + char *language; + int margin; + + int alpha; + FFDrawContext draw; + + int got_header; + int out_w, out_h; + AVRational frame_rate; + AVFrame *last_frame; + int eof; +} TextSubsContext; + +/* libass supports a log level ranging from 0 to 7 */ +static const int ass_libavfilter_log_level_map[] = { + AV_LOG_QUIET, /* 0 */ + AV_LOG_PANIC, /* 1 */ + AV_LOG_FATAL, /* 2 */ + AV_LOG_ERROR, /* 3 */ + AV_LOG_WARNING, /* 4 */ + AV_LOG_INFO, /* 5 */ + AV_LOG_VERBOSE, /* 6 */ + AV_LOG_DEBUG, /* 7 */ +}; + +static void ass_log(int ass_level, const char *fmt, va_list args, void *ctx) +{ + const int ass_level_clip = av_clip(ass_level, 0, FF_ARRAY_ELEMS(ass_libavfilter_log_level_map) - 1); + const int level = ass_libavfilter_log_level_map[ass_level_clip]; + + av_vlog(ctx, level, fmt, args); + av_log(ctx, level, "\n"); +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + TextSubsContext *s = ctx->priv; + + if (s->track) + ass_free_track(s->track); + if (s->renderer) + ass_renderer_done(s->renderer); + if (s->library) + ass_library_done(s->library); + + s->track = NULL; + s->renderer = NULL; + s->library = NULL; + + if (s->last_frame) + av_frame_unref(s->last_frame); +} + +static int overlay_textsubs_query_formats(AVFilterContext *ctx) +{ + AVFilterFormats *formats; + AVFilterLink *inlink0 = ctx->inputs[0]; + AVFilterLink *inlink1 = ctx->inputs[1]; + AVFilterLink *outlink = ctx->outputs[0]; + static const enum AVSubtitleType subtitle_fmts[] = { AV_SUBTITLE_FMT_ASS, AV_SUBTITLE_FMT_NONE }; + int ret; + + /* set input0 video formats */ + formats = ff_draw_supported_pixel_formats(0); + if ((ret = ff_formats_ref(formats, &inlink0->outcfg.formats)) < 0) + return ret; + + /* set input1 subtitle formats */ + formats = ff_make_format_list(subtitle_fmts); + if ((ret = ff_formats_ref(formats, &inlink1->outcfg.formats)) < 0) + return ret; + + /* set output0 video formats */ + formats = ff_draw_supported_pixel_formats(0); + if ((ret = ff_formats_ref(formats, &outlink->incfg.formats)) < 0) + return ret; + + return 0; +} + +static int config_output(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + + outlink->w = ctx->inputs[0]->w; + outlink->h = ctx->inputs[0]->h; + outlink->time_base = ctx->inputs[0]->time_base; + + return 0; +} + +static int config_input_main(AVFilterLink *inlink) +{ + AVFilterContext *ctx = inlink->dst; + TextSubsContext *s = inlink->dst->priv; + int ret; + + ret = ff_draw_init(&s->draw, inlink->format, s->alpha ? FF_DRAW_PROCESS_ALPHA : 0); + if (ret < 0) { + av_log(ctx, AV_LOG_ERROR, "Could not initialize ff_draw.\n"); + return ret; + } + + ass_set_frame_size (s->renderer, inlink->w, inlink->h); + ass_set_pixel_aspect(s->renderer, av_q2d(inlink->sample_aspect_ratio)); + + return 0; +} + +/* libass stores an RGBA color in the format RRGGBBTT, where TT is the transparency level */ +#define AR(c) ( (c)>>24) +#define AG(c) (((c)>>16)&0xFF) +#define AB(c) (((c)>>8) &0xFF) +#define AA(c) ((0xFF-(c)) &0xFF) + +static void overlay_ass_image(TextSubsContext *s, AVFrame *picref, + const ASS_Image *image) +{ + for (; image; image = image->next) { + uint8_t rgba_color[] = {AR(image->color), AG(image->color), AB(image->color), AA(image->color)}; + FFDrawColor color; + ff_draw_color(&s->draw, &color, rgba_color); + ff_blend_mask(&s->draw, &color, + picref->data, picref->linesize, + picref->width, picref->height, + image->bitmap, image->stride, image->w, image->h, + 3, 0, image->dst_x, image->dst_y); + } +} + +static void process_header(AVFilterContext *link, AVFrame *sub) +{ + TextSubsContext *s = link->priv; + ASS_Track *track = s->track; + ASS_Style *style; + int sid = 0; + + if (!track) + return; + + if (sub && sub->subtitle_header) + ass_process_codec_private(s->track, sub->subtitle_header, strlen(sub->subtitle_header)); + else { + AVCodecContext temp_context; + ff_ass_subtitle_header_default(&temp_context); + if (!temp_context.subtitle_header) + return; + + ass_process_codec_private(s->track, (char*)temp_context.subtitle_header, temp_context.subtitle_header_size); + } + + if (s->language) + s->track->Language = strdup(s->language); + + if (!s->track->event_format) { + s->track->event_format = strdup("ReadOrder, Layer, Style, Name, MarginL, MarginR, MarginV, Effect, Text"); + } + + if (s->track->n_styles == 0) { + sid = ass_alloc_style(track); + style = &s->track->styles[sid]; + style->Name = strdup("Default"); + style->PrimaryColour = 0xffffff00; + style->SecondaryColour = 0x00ffff00; + style->OutlineColour = 0x00000000; + style->BackColour = 0x00000080; + style->Bold = 200; + style->ScaleX = 1.0; + style->ScaleY = 1.0; + style->Spacing = 0; + style->BorderStyle = 1; + style->Outline = 2; + style->Shadow = 3; + style->Alignment = 2; + } + else + style = &s->track->styles[sid]; + + style->FontSize = s->font_size; + style->MarginL = style->MarginR = style->MarginV = s->margin; + + track->default_style = sid; + + s->got_header = 1; +} + +static int filter_video_frame(AVFilterLink *inlink, AVFrame *frame) +{ + AVFilterContext *ctx = inlink->dst; + TextSubsContext *s = ctx->priv; + int detect_change = 0; + + int64_t time_ms = frame->pts * av_q2d(inlink->time_base) * 1000; + + ASS_Image *image = ass_render_frame(s->renderer, s->track, time_ms, &detect_change); + + if (detect_change) + av_log(ctx, AV_LOG_DEBUG, "Change happened at time ms:%lld\n", time_ms); + + overlay_ass_image(s, frame, image); + + return ff_filter_frame(ctx->outputs[0], frame); +} + +static int filter_subtitle_frame(AVFilterLink *inlink, AVFrame *frame) +{ + AVFilterContext *ctx = inlink->dst; + TextSubsContext *s = ctx->priv; + + if (frame->buf[0] && frame->buf[0]->data) { + AVFrame *sub = (AVFrame *)frame->buf[0]->data; + + if (sub != s->last_sub) { + unsigned i; + s->last_sub = sub; + + if (!s->got_header) + process_header(ctx, sub); + + const int64_t start_time = av_rescale_q(sub->subtitle_pts, AV_TIME_BASE_Q, av_make_q(1, 1000)); + const int64_t duration = sub->subtitle_end_time; + + for (i = 0; i < sub->num_subtitle_rects; i++) { + char *ass_line = sub->subtitle_rects[i]->ass; + if (!ass_line) + break; + ass_process_chunk(s->track, ass_line, strlen(ass_line), start_time, duration); + } + } + } + + av_frame_free(&frame); + return 0; +} + +static av_cold int init(AVFilterContext *ctx) +{ + TextSubsContext *s = ctx->priv; + + s->library = ass_library_init(); + + if (!s->library) { + av_log(ctx, AV_LOG_ERROR, "Could not initialize libass.\n"); + return AVERROR(EINVAL); + } + + ass_set_message_cb(s->library, ass_log, ctx); + + /* Initialize fonts */ + if (s->fonts_dir) + ass_set_fonts_dir(s->library, s->fonts_dir); + + ass_set_extract_fonts(s->library, 1); + + s->renderer = ass_renderer_init(s->library); + if (!s->renderer) { + av_log(ctx, AV_LOG_ERROR, "Could not initialize libass renderer.\n"); + return AVERROR(EINVAL); + } + + s->track = ass_new_track(s->library); + if (!s->track) { + av_log(ctx, AV_LOG_ERROR, "ass_new_track() failed!\n"); + return AVERROR(EINVAL); + } + + ass_set_fonts(s->renderer, s->default_font_path, NULL, 1, s->fc_file, 1); + + if (s->force_style) { + char **list = NULL; + char *temp = NULL; + char *ptr = av_strtok(s->force_style, ",", &temp); + int i = 0; + while (ptr) { + av_dynarray_add(&list, &i, ptr); + if (!list) { + return AVERROR(ENOMEM); + } + ptr = av_strtok(NULL, ",", &temp); + } + av_dynarray_add(&list, &i, NULL); + if (!list) { + return AVERROR(ENOMEM); + } + ass_set_style_overrides(s->library, list); + av_free(list); + } + + return 0; +} + +static int textsub2video_query_formats(AVFilterContext *ctx) +{ + AVFilterFormats *formats; + AVFilterLink *inlink = ctx->inputs[0]; + AVFilterLink *outlink = ctx->outputs[0]; + static const enum AVSubtitleType subtitle_fmts[] = { AV_SUBTITLE_FMT_ASS, AV_SUBTITLE_FMT_NONE }; + int ret; + + /* set input0 subtitle format */ + formats = ff_make_format_list(subtitle_fmts); + if ((ret = ff_formats_ref(formats, &inlink->outcfg.formats)) < 0) + return ret; + + /* set output0 video format */ + formats = ff_draw_supported_pixel_formats(AV_PIX_FMT_FLAG_ALPHA); + if ((ret = ff_formats_ref(formats, &outlink->incfg.formats)) < 0) + return ret; + + return 0; +} + +static int textsub2video_config_input(AVFilterLink *inlink) +{ + AVFilterContext *ctx = inlink->dst; + TextSubsContext *s = ctx->priv; + + if (s->out_w <= 0 || s->out_h <= 0) { + s->out_w = inlink->w; + s->out_h = inlink->h; + } + + return 0; +} + +static int textsub2video_config_output(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + TextSubsContext *s = ctx->priv; + int ret; + + ret = ff_draw_init(&s->draw, outlink->format, FF_DRAW_PROCESS_ALPHA); + if (ret < 0) { + av_log(ctx, AV_LOG_ERROR, "Could not initialize ff_draw.\n"); + return ret; + } + + if (s->out_w <= 0 || s->out_h <= 0) { + av_log(ctx, AV_LOG_ERROR, "No output image size set.\n"); + return AVERROR(EINVAL); + } + + ass_set_frame_size (s->renderer, s->out_w, s->out_h); + + outlink->w = s->out_w; + outlink->h = s->out_h; + outlink->sample_aspect_ratio = (AVRational){1,1}; + outlink->frame_rate = s->frame_rate; + + return 0; +} + +static int textsub2video_filter_frame(AVFilterLink *inlink, AVFrame *frame) +{ + AVFilterContext *ctx = inlink->dst; + TextSubsContext *s = ctx->priv; + + if (frame->buf[0] && frame->buf[0]->data) { + AVFrame *sub = (AVFrame *)frame->buf[0]->data; + + av_log(ctx, AV_LOG_VERBOSE, "textsub2video_filter_frame num_subtitle_rects: %d\n", sub->num_subtitle_rects); + + if (sub != s->last_sub) { + unsigned i; + s->last_sub = sub; + + if (!s->got_header) + process_header(ctx, sub); + + const int64_t start_time = av_rescale_q(sub->subtitle_pts, AV_TIME_BASE_Q, av_make_q(1, 1000)); + const int64_t duration = sub->subtitle_end_time; + + for (i = 0; i < sub->num_subtitle_rects; i++) { + char *ass_line = sub->subtitle_rects[i]->ass; + if (!ass_line) + break; + ass_process_chunk(s->track, ass_line, strlen(ass_line), start_time, duration); + } + } + } + + av_frame_free(&frame); + return 0; +} + +static int textsub2video_request_frame(AVFilterLink *outlink) +{ + TextSubsContext *s = outlink->src->priv; + const AVFilterLink *inlink = outlink->src->inputs[0]; + int64_t last_pts = outlink->current_pts; + int64_t next_pts; + int i, detect_change = 0; + AVFrame *out; + ASS_Image *image; + + if (last_pts == AV_NOPTS_VALUE) + last_pts = inlink->current_pts * av_q2d(inlink->time_base) / av_q2d(outlink->time_base); + + next_pts = last_pts + (1.0 / av_q2d(outlink->frame_rate) / av_q2d(outlink->time_base)); + + int64_t time_ms = next_pts * av_q2d(outlink->time_base) * 1000; + + image = ass_render_frame(s->renderer, s->track, time_ms, &detect_change); + + if (detect_change) + av_log(outlink->src, AV_LOG_DEBUG, "Change happened at time ms:%lld\n", time_ms); + else if (s->last_frame) { + out = av_frame_clone(s->last_frame); + if (!out) + return AVERROR(ENOMEM); + + out->pts = next_pts; + return ff_filter_frame(outlink, out); + } + + out = ff_get_video_buffer(outlink, outlink->w, outlink->h); + if (!out) + return AVERROR(ENOMEM); + + for (i = 2; i < AV_NUM_DATA_POINTERS; i++) { + if (out->buf[i] && i != 1) + memset(out->buf[i]->data, 0, out->buf[i]->size); + } + + out->pts = next_pts; + + if (image) + overlay_ass_image(s, out, image); + + if (s->last_frame) + av_frame_unref(s->last_frame); + + s->last_frame = av_frame_clone(out); + + return ff_filter_frame(outlink, out); +} + +static int activate(AVFilterContext *ctx) +{ + AVFilterLink *inlink = ctx->inputs[0]; + AVFilterLink *outlink = ctx->outputs[0]; + TextSubsContext *s = ctx->priv; + AVFrame *in; + int ret; + + FF_FILTER_FORWARD_STATUS_BACK(outlink, inlink); + + ret = ff_inlink_consume_frame(inlink, &in); + if (ret < 0) + return ret; + if (ret > 0) + return textsub2video_filter_frame(inlink, in); + + if (ff_outlink_frame_wanted(outlink)) { + if (!s->eof && ff_outlink_get_status(ctx->inputs[0])) { + s->eof = 1; + } + if (!s->eof && ff_inlink_queued_frames(ctx->inputs[0]) == 0) + ff_inlink_request_frame(ctx->inputs[0]); + if (s->eof && ff_inlink_queued_frames(ctx->inputs[0]) <= 0) { + ff_outlink_set_status(outlink, AVERROR_EOF, AV_NOPTS_VALUE); + } + else { + return textsub2video_request_frame(outlink); + } + + return 0; + } + + return FFERROR_NOT_READY; +} + +#define OFFSET(x) offsetof(TextSubsContext, x) +#define FLAGS (AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_FILTERING_PARAM) + +static const AVOption overlay_textsubs_options[] = { + {"alpha", "enable processing of alpha channel", OFFSET(alpha), AV_OPT_TYPE_BOOL, {.i64 = 0 }, 0, 1, FLAGS}, + {"font_scale", "font scale factor", OFFSET(font_scale), AV_OPT_TYPE_DOUBLE, {.dbl = 1.0 }, 0.0, 100.0, FLAGS}, + {"font_size", "default font size", OFFSET(font_size), AV_OPT_TYPE_DOUBLE, {.dbl = 18.0}, 0.0, 100.0, FLAGS}, + {"force_style", "force subtitle style", OFFSET(force_style), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, FLAGS}, + {"margin", "default margin", OFFSET(margin), AV_OPT_TYPE_INT, {.i64 = 20 }, 0, INT_MAX, FLAGS}, + {"default_font_path", "path to default font", OFFSET(default_font_path), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, FLAGS}, + {"fonts_dir", "directory to scan for fonts", OFFSET(fonts_dir), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, FLAGS}, + {"fontsdir", "directory to scan for fonts", OFFSET(fonts_dir), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, FLAGS}, + {"fontconfig_file", "fontconfig file to load", OFFSET(fc_file), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, FLAGS}, + {"language", "default language", OFFSET(language), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, FLAGS}, + { NULL } +}; + +static const AVOption textsub2video_options[] = { + {"rate", "set frame rate", OFFSET(frame_rate), AV_OPT_TYPE_VIDEO_RATE, {.str="8"}, 0, INT_MAX, FLAGS}, + {"r", "set frame rate", OFFSET(frame_rate), AV_OPT_TYPE_VIDEO_RATE, {.str="8"}, 0, INT_MAX, FLAGS}, + {"size", "set video size", OFFSET(out_w), AV_OPT_TYPE_IMAGE_SIZE, {.str = NULL}, 0, 0, FLAGS}, + {"s", "set video size", OFFSET(out_w), AV_OPT_TYPE_IMAGE_SIZE, {.str = NULL}, 0, 0, FLAGS}, + {"font_scale", "font scale factor", OFFSET(font_scale), AV_OPT_TYPE_DOUBLE, {.dbl = 1.0 }, 0.0, 100.0, FLAGS}, + {"font_size", "default font size", OFFSET(font_size), AV_OPT_TYPE_DOUBLE, {.dbl = 18.0}, 0.0, 100.0, FLAGS}, + {"force_style", "force subtitle style", OFFSET(force_style), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, FLAGS}, + {"margin", "default margin", OFFSET(margin), AV_OPT_TYPE_INT, {.i64 = 20 }, 0, INT_MAX, FLAGS}, + {"default_font_path", "path to default font", OFFSET(default_font_path), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, FLAGS}, + {"fonts_dir", "directory to scan for fonts", OFFSET(fonts_dir), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, FLAGS}, + {"fontsdir", "directory to scan for fonts", OFFSET(fonts_dir), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, FLAGS}, + {"fontconfig_file", "fontconfig file to load", OFFSET(fc_file), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, FLAGS}, + {"language", "default language", OFFSET(language), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, FLAGS}, + { NULL } +}; + +#if CONFIG_OVERLAY_TEXTSUBS_FILTER + +AVFILTER_DEFINE_CLASS(overlay_textsubs); + +static const AVFilterPad overlay_textsubs_inputs[] = { + { + .name = "main", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_input_main, + .flags = AVFILTERPAD_FLAG_NEEDS_WRITABLE, + .filter_frame = filter_video_frame, + }, + { + .name = "overlay", + .type = AVMEDIA_TYPE_SUBTITLE, + .filter_frame = filter_subtitle_frame, + }, +}; + +static const AVFilterPad overlay_textsubs_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_output, + }, +}; + +const AVFilter ff_vf_overlay_textsubs = { + .name = "overlay_textsubs", + .description = NULL_IF_CONFIG_SMALL("Overlay textual subtitles on top of the input."), + .init = init, + .uninit = uninit, + .priv_size = sizeof(TextSubsContext), + .priv_class = &overlay_textsubs_class, + .query_formats = overlay_textsubs_query_formats, + FILTER_INPUTS(overlay_textsubs_inputs), + FILTER_OUTPUTS(overlay_textsubs_outputs), +}; +#endif + +#if CONFIG_TEXTSUB2VIDEO_FILTER + +AVFILTER_DEFINE_CLASS(textsub2video); + +static const AVFilterPad textsub2video_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .config_props = textsub2video_config_input, + }, +}; + +static const AVFilterPad textsub2video_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = textsub2video_config_output, + }, +}; + +AVFilter ff_svf_textsub2video = { + .name = "textsub2video", + .description = NULL_IF_CONFIG_SMALL("Convert textual subtitles to video frames"), + .init = init, + .uninit = uninit, + .query_formats = textsub2video_query_formats, + .priv_size = sizeof(TextSubsContext), + .priv_class = &textsub2video_class, + .activate = activate, + FILTER_INPUTS(textsub2video_inputs), + FILTER_OUTPUTS(textsub2video_outputs), +}; +#endif \ No newline at end of file From patchwork Sun Sep 12 03:22:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 30177 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp2863548iov; Sat, 11 Sep 2021 20:23:35 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzj+ZZXxs0XTKDigWXE2M66d7MuBBQc/kLQnCnblzs22wkumTH3CaklOJUfVQ7UPaK9E/jx X-Received: by 2002:a17:906:6403:: with SMTP id d3mr5505539ejm.37.1631417015170; Sat, 11 Sep 2021 20:23:35 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id g21si3335570edb.488.2021.09.11.20.23.34; Sat, 11 Sep 2021 20:23:35 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=MeHH3OHU; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7FD9B68AA33; Sun, 12 Sep 2021 06:22:05 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10olkn2055.outbound.protection.outlook.com [40.92.41.55]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2F53E68AA36 for ; Sun, 12 Sep 2021 06:22:02 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=biillLFG+VoPsjbK4vTiDB0TdVc6B82EsTUntwiGgIdmNk3Od1lnYaUImrQxeTIabHHtarYIa943ypp3gNC+y1m3dzfJYTpPXxnbNmQfz6kEvlR6SbskFjs3tpMJupusLHyOkGWjuYf+GaUkLFFD+dsOjtnhRcT5T3tVhTtQmOURHRs74e7eBZOx7LS5Vl5SxbmL0Pjh4SjZzhIOaZdOCX6h7kdBeI/Ddr0KH7ZjhqKtDZ7ey0hCYwW1lr1o+aD+vgBeIooEIFOOZTm11BeWK0ZWb/g+S1LcJ16dsg0mH0ricntXCZaSPAANFlGy8WUN+81N3Xyp2KRKCPc2sDJxmA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=VPZxf1rfZFeIgWwYxXAGnRkc1Sp+wWfkgagKn8TExac=; b=FsXNrfLGWST9oreqn0Tozzy34Hkh6/7GLZEghUAi6BWMwOBi4VvuJJ2iPswnVnZqcYZX0zDIz34G4MYTeJyTvoxqTpefe74KFjMOFqHx1hTqo27U51U/U8VC61QLsWVgA37QiYwGmV6v3ntm3Cs59rj1/4oD3GWeXjrr0CDTJ+OS1FNXG5q1e0uLBK3WOhv3sD5n7sQSehFYkp6jor80NQJelQ83Od2TOlgu9fjPHKtsnd5QZvy1v/YQ1O0JhuDgN4mPHVc8UZnB2GuPtnAtKpPfbTQ96IJ3DLGKqC/sDLtZOq8t4ZB64GQYjXBk+CVfi93P40P1z0mxj6f98yzwVg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VPZxf1rfZFeIgWwYxXAGnRkc1Sp+wWfkgagKn8TExac=; b=MeHH3OHUrryPhZSG7Fz6TtrP7acW/xQqFCB3hIS6t8wq8XxluWTC8SSLr9TVKBhlIhB/AoDmDLGndUSTwetV00ZKXQJ4g9YJrwCfyYsvhcVNKk9aa/beF/QHbt5yzLTi/neNZ4oLAdF8XqNBKw/qvPpx1F1Yw12/lABrWxP5R/pStwRp2f5Nfm1PmIJSyWmjEGCZ4HvUy9j+TFr3aSlICBb2QXFS4Hxlj0+xdvxNSWUvz33qPZ7ZnyFracg7j4iFDNb46KMzUi9nruXYISL4SPsLeB0F++7TJcXh28tAb06flY1qG3AH/RKonFZkfnHPtpYXjoi5tv6jXK18D1V3sQ== Received: from MN2PR04MB5981.namprd04.prod.outlook.com (2603:10b6:208:da::10) by MN2PR04MB5728.namprd04.prod.outlook.com (2603:10b6:208:3d::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Sun, 12 Sep 2021 03:22:00 +0000 Received: from MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb]) by MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb%5]) with mapi id 15.20.4500.018; Sun, 12 Sep 2021 03:22:00 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v5 10/12] avfilter/textmod: Add textmod filter Thread-Index: AQHXp4VZP+/AoDMGh0qlZlcupmM3Xg== Date: Sun, 12 Sep 2021 03:22:00 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [TqHUk2YIZELO0PodEXUVBZGqXLIN26fNMGDksUii7OM=] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 6ea88129-f2c8-4ef0-3e96-08d9759c7c20 x-ms-traffictypediagnostic: MN2PR04MB5728: x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: Jq1MI/CjrQ+jpRv1Kfw0WkdN/1Vm7qys7983n/k1bY+r42yEdoSjruNZ7gU7Mq9RxCIKkgS2BkbOMrXRR5EA7UDnR5Rtszb4M0Our2JQ8Z7HgRASTqLx/d2BaL0Q8jgw1qL3pdTxyrDX74Ly7ZaGyslR/NB3V1J6SldxAPX9XhMtEUX1WCEWRcj8G0WuZRGtLhpYRl/0ntMYCzNNqnNqZgz1eEr7eChEgkrCr0w3QYF7guPiNOqJuuX7VMyyenrvztu53xga8KIKdT+KiwZG2G7oBTFQmg1pvM9KOcTJCyuU+T0M4BifgO+bRiA5kjsYs/4sYpzvs/oI+v4ww+VCjMU2iGkodEO0oCRKI0z/UIsQ4ZxWUye0jcStqs4QeKQs2JZatJW7aCYYT2+qZ4eWK8pZJpOUiyaukU2ZFUFTeGKeo0yP35TWowa3pOt6SBm8eM+GL9P8o6WMNR5EokP32akhWbKHNTOCiAYHKp0gZD8= x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: OGanyo/foTtDmN0xZbajC/dBsgcdwmk0pvTEEiqWgRqxl1cidEiHeipuV2v4rXrvEL7/cq14/Te2KYwzdyyFIe3KdhvHPeMNq6L35o20WlZ+XTleJL937MmHUYyMlWRUzPWgoHKeeIuxm9lXcv4w3g== x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-3174-20-msonline-outlook-529c7.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR04MB5981.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 6ea88129-f2c8-4ef0-3e96-08d9759c7c20 X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Sep 2021 03:22:00.6142 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR04MB5728 Subject: [FFmpeg-devel] [PATCH v5 10/12] avfilter/textmod: Add textmod filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: FBrrG6uJ5SOr Signed-off-by: softworkz --- doc/filters.texi | 64 +++++++ libavfilter/Makefile | 3 + libavfilter/allfilters.c | 1 + libavfilter/sf_textmod.c | 381 +++++++++++++++++++++++++++++++++++++++ 4 files changed, 449 insertions(+) create mode 100644 libavfilter/sf_textmod.c diff --git a/doc/filters.texi b/doc/filters.texi index 1d76461ada..9fd2876d63 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -25024,6 +25024,70 @@ existing filters using @code{--disable-filters}. Below is a description of the currently available subtitle filters. +@section textmod + +Modify subtitle text in a number of ways. + +It accepts the following parameters: + +@table @option +@item mode +The kind of text modification to apply + +Supported operation modes are: + +@table @var +@item 0, leet +Convert subtitle text to 'leet speak'. It's primarily useful for testing as the modification will be visible with almost all text lines. +@item 1, to_upper +Change all text to upper case. Might improve readability. +@item 2, to_lower +Change all text to lower case. +@item 3, replace_chars +Replace one or more characters. Requires the find and replace parameters to be specified. +Both need to be equal in length. +The first char in find is replaced by the first char in replace, same for all subsequent chars. +@item 4, remove_chars +Remove certain characters. Requires the find parameter to be specified. +All chars in the find parameter string will be removed from all subtitle text. +@item 5, replace_words +Replace one or more words. Requires the find and replace parameters to be specified. Multiple words must be separated by the delimiter char specified vie the separator parameter (default: ','). +The number of words in the find and replace parameters needs to be equal. +The first word in find is replaced by the first word in replace, same for all subsequent words +@item 6, remove_words +Remove certain words. Requires the find parameter to be specified. Multiple words must be separated by the delimiter char specified vie the separator parameter (default: ','). +All words in the find parameter string will be removed from all subtitle text. +@end table + +@item find +Required for replace_chars, remove_chars, replace_words and remove_words. + +@item replace +Required for replace_chars and replace_words. + +@item separator +Delimiter character for words. Used with replace_words and remove_words- Must be a single character. +The default is '.'. + +@end table + +@subsection Examples + +@itemize +@item +Change all characters to upper case while keeping all styles and animations: +@example +ffmpeg -i "https://streams.videolan.org/ffmpeg/mkv_subtitles.mkv" -filter_complex "[0:s]textmod=mode=to_upper" -map 0 -y out.mkv +@end example +@item +Mark the 100-pixel-wide region on the left edge of the frame as very +uninteresting (to be encoded at much lower quality than the rest of +the frame). +@example +addroi=0:0:100:ih:+1/5 +@end example +@end itemize + @section graphicsub2video Renders graphic subtitles as video frames. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 0e752c5bf9..5a5a4be47e 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -534,6 +534,9 @@ OBJS-$(CONFIG_YUVTESTSRC_FILTER) += vsrc_testsrc.o OBJS-$(CONFIG_NULLSINK_FILTER) += vsink_nullsink.o +# subtitle filters +OBJS-$(CONFIG_TEXTMOD_FILTER) += sf_textmod.o + # multimedia filters OBJS-$(CONFIG_ABITSCOPE_FILTER) += avf_abitscope.o OBJS-$(CONFIG_ADRAWGRAPH_FILTER) += f_drawgraph.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 77463aa4c8..6d7a535ee8 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -524,6 +524,7 @@ extern const AVFilter ff_avf_showvolume; extern const AVFilter ff_avf_showwaves; extern const AVFilter ff_avf_showwavespic; extern const AVFilter ff_vaf_spectrumsynth; +extern const AVFilter ff_sf_textmod; extern const AVFilter ff_svf_graphicsub2video; extern const AVFilter ff_svf_textsub2video; diff --git a/libavfilter/sf_textmod.c b/libavfilter/sf_textmod.c new file mode 100644 index 0000000000..7c23ded9ef --- /dev/null +++ b/libavfilter/sf_textmod.c @@ -0,0 +1,381 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * text subtitle filter which allows to modify subtitle text in several ways + */ + +#include + +#include "libavutil/avassert.h" +#include "libavutil/avstring.h" +#include "libavutil/opt.h" +#include "avfilter.h" +#include "internal.h" +#include "libavcodec/avcodec.h" +#include "libavcodec/ass_split.h" + +static const char* leet_src = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"; +static const char* leet_dst = "abcd3f6#1jklmn0pq257uvwxyzAB(D3F6#1JKLMN0PQ257UVWXYZ"; + +enum TextModOperation { + OP_LEET, + OP_TO_UPPER, + OP_TO_LOWER, + OP_REPLACE_CHARS, + OP_REMOVE_CHARS, + OP_REPLACE_WORDS, + OP_REMOVE_WORDS, + NB_OPS, +}; + +typedef struct TextModContext { + const AVClass *class; + enum AVSubtitleType format; + enum TextModOperation operation; + char *find; + char *replace; + char *separator; + char **find_list; + int nb_find_list; + char **replace_list; + int nb_replace_list; +} TextModContext; + +static char **split_string(char *source, int *nb_elems, char delim) +{ + char **list = NULL; + char *temp = NULL; + char *ptr = av_strtok(source, &delim, &temp); + + while (ptr) { + av_dynarray_add(&list, nb_elems, ptr); + if (!list) + return NULL; + + ptr = av_strtok(NULL, &delim, &temp); + } + + av_dynarray_add(&list, nb_elems, NULL); + + return list; +} + +static int init(AVFilterContext *ctx) +{ + TextModContext *s = ctx->priv; + + switch (s->operation) { + case OP_REPLACE_CHARS: + case OP_REMOVE_CHARS: + case OP_REPLACE_WORDS: + case OP_REMOVE_WORDS: + if (!s->find || !strlen(s->find)) { + av_log(ctx, AV_LOG_ERROR, "Selected mode requires the 'find' parameter to be specified"); + return AVERROR(EINVAL); + } + break; + } + + switch (s->operation) { + case OP_REPLACE_CHARS: + case OP_REPLACE_WORDS: + if (!s->replace || !strlen(s->replace)) { + av_log(ctx, AV_LOG_ERROR, "Selected mode requires the 'replace' parameter to be specified"); + return AVERROR(EINVAL); + } + break; + } + + if (s->operation == OP_REPLACE_CHARS && strlen(s->find) != strlen(s->replace)) { + av_log(ctx, AV_LOG_ERROR, "Selected mode requires the 'find' and 'replace' parameters to have the same length"); + return AVERROR(EINVAL); + } + + if (s->operation == OP_REPLACE_WORDS || s->operation == OP_REMOVE_WORDS) { + if (!s->separator || strlen(s->separator) != 1) { + av_log(ctx, AV_LOG_ERROR, "Selected mode requires a single separator char to be specified"); + return AVERROR(EINVAL); + } + + s->find_list = split_string(s->find, &s->nb_find_list, *s->separator); + if (!s->find_list) + return AVERROR(ENOMEM); + + if (s->operation == OP_REPLACE_WORDS) { + + s->replace_list = split_string(s->replace, &s->nb_replace_list, *s->separator); + if (!s->replace_list) + return AVERROR(ENOMEM); + + if (s->nb_find_list != s->nb_replace_list) { + av_log(ctx, AV_LOG_ERROR, "The number of words in 'find' and 'replace' needs to be equal"); + return AVERROR(EINVAL); + } + } + } + + return 0; +} + +static void uninit(AVFilterContext *ctx) +{ + TextModContext *s = ctx->priv; + int i; + + for (i = 0; i < s->nb_find_list; i++) { + av_free(&s->find_list[i]); + } + s->nb_find_list = 0; + av_freep(&s->find_list); + + for (i = 0; i < s->nb_replace_list; i++) { + av_free(&s->replace_list[i]); + } + s->nb_replace_list = 0; + av_freep(&s->replace_list); +} + +static int query_formats(AVFilterContext *ctx) +{ + AVFilterFormats *formats; + AVFilterLink *inlink = ctx->inputs[0]; + AVFilterLink *outlink = ctx->outputs[0]; + static const enum AVSubtitleType subtitle_fmts[] = { AV_SUBTITLE_FMT_ASS, AV_SUBTITLE_FMT_NONE }; + int ret; + + /* set input subtitle format */ + formats = ff_make_format_list(subtitle_fmts); + if ((ret = ff_formats_ref(formats, &inlink->outcfg.formats)) < 0) + return ret; + + /* set output video format */ + if ((ret = ff_formats_ref(formats, &outlink->incfg.formats)) < 0) + return ret; + + return 0; +} + +static char *process_text(TextModContext *s, char *text) +{ + const char *char_src = s->find; + const char *char_dst = s->replace; + char *result = NULL; + int escape_level = 0, k = 0; + + switch (s->operation) { + case OP_LEET: + case OP_REPLACE_CHARS: + + if (s->operation == OP_LEET) { + char_src = leet_src; + char_dst = leet_dst; + } + + result = av_strdup(text); + if (!result) + return NULL; + + for (size_t n = 0; n < strlen(result); n++) { + if (result[n] == '{') + escape_level++; + + if (!escape_level) { + for (size_t t = 0; t < FF_ARRAY_ELEMS(char_src); t++) { + if (result[n] == char_src[t]) { + result[n] = char_dst[t]; + break; + } + } + } + + if (result[n] == '}') + escape_level--; + } + + break; + case OP_TO_UPPER: + case OP_TO_LOWER: + + result = av_strdup(text); + if (!result) + return NULL; + + for (size_t n = 0; n < strlen(result); n++) { + if (result[n] == '{') + escape_level++; + if (!escape_level) + result[n] = s->operation == OP_TO_LOWER ? av_tolower(result[n]) : av_toupper(result[n]); + if (result[n] == '}') + escape_level--; + } + + break; + case OP_REMOVE_CHARS: + + result = av_strdup(text); + if (!result) + return NULL; + + for (size_t n = 0; n < strlen(result); n++) { + int skip_char = 0; + + if (result[n] == '{') + escape_level++; + + if (!escape_level) { + for (size_t t = 0; t < FF_ARRAY_ELEMS(char_src); t++) { + if (result[n] == char_src[t]) { + skip_char = 1; + break; + } + } + } + + if (!skip_char) + result[k++] = result[n]; + + if (result[n] == '}') + escape_level--; + } + + result[k] = 0; + + break; + case OP_REPLACE_WORDS: + case OP_REMOVE_WORDS: + + result = av_strdup(text); + if (!result) + return NULL; + + for (int n = 0; n < s->nb_find_list; n++) { + char *tmp = result; + const char *replace = (s->operation == OP_REPLACE_WORDS) ? s->replace_list[n] : ""; + + result = av_strireplace(result, s->find_list[n], replace); + if (!result) + return NULL; + + av_free(tmp); + } + + break; + } + + return result; +} + +static char *process_dialog(TextModContext *s, char *ass_line) +{ + ASSDialog *dialog = ff_ass_split_dialog(NULL, ass_line); + char *result, *text; + + if (!dialog) + return NULL; + + text = process_text(s, dialog->text); + if (!text) + return NULL; + + result = ff_ass_get_dialog(dialog->readorder, dialog->layer, dialog->style, dialog->name, text); + + av_free(text); + ff_ass_free_dialog(&dialog); + return result; +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *src_frame) +{ + TextModContext *s = inlink->dst->priv; + AVFilterLink *outlink = inlink->dst->outputs[0]; + int ret; + AVFrame *out; + + outlink->format = inlink->format; + + out = av_frame_clone(src_frame); + if (!out) + return AVERROR(ENOMEM); + + for (unsigned i = 0; i < out->num_subtitle_rects; i++) { + + AVSubtitleRect *rect = out->subtitle_rects[i]; + + if (rect->ass) { + char *tmp = rect->ass; + rect->ass = process_dialog(s, rect->ass); + av_free(tmp); + if (!rect->ass) + return AVERROR(ENOMEM); + } + } + + av_frame_free(&src_frame); + return ff_filter_frame(outlink, out); +} + +#define OFFSET(x) offsetof(TextModContext, x) +#define FLAGS (AV_OPT_FLAG_SUBTITLE_PARAM | AV_OPT_FLAG_FILTERING_PARAM) + +static const AVOption textmod_options[] = { + { "mode", "set operation mode", OFFSET(operation), AV_OPT_TYPE_INT, {.i64=OP_LEET}, OP_LEET, NB_OPS-1, FLAGS, "mode" }, + { "leet", "convert text to 'leet speak'", 0, AV_OPT_TYPE_CONST, {.i64=OP_LEET}, 0, 0, FLAGS, "mode" }, + { "to_upper", "change to upper case", 0, AV_OPT_TYPE_CONST, {.i64=OP_TO_UPPER}, 0, 0, FLAGS, "mode" }, + { "to_lower", "change to lower case", 0, AV_OPT_TYPE_CONST, {.i64=OP_TO_LOWER}, 0, 0, FLAGS, "mode" }, + { "replace_chars", "replace characters", 0, AV_OPT_TYPE_CONST, {.i64=OP_REPLACE_CHARS}, 0, 0, FLAGS, "mode" }, + { "remove_chars", "remove characters", 0, AV_OPT_TYPE_CONST, {.i64=OP_REMOVE_CHARS}, 0, 0, FLAGS, "mode" }, + { "replace_words", "replace words", 0, AV_OPT_TYPE_CONST, {.i64=OP_REPLACE_WORDS}, 0, 0, FLAGS, "mode" }, + { "remove_words", "remove words", 0, AV_OPT_TYPE_CONST, {.i64=OP_REMOVE_WORDS}, 0, 0, FLAGS, "mode" }, + { "find", "chars/words to find or remove", OFFSET(find), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, FLAGS, NULL }, + { "replace", "chars/words to replace", OFFSET(replace), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, FLAGS, NULL }, + { "separator", "word separator (default: ',')", OFFSET(separator), AV_OPT_TYPE_STRING, {.str = ","}, 0, 0, FLAGS, NULL }, + { NULL }, +}; + +AVFILTER_DEFINE_CLASS(textmod); + +static const AVFilterPad inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .filter_frame = filter_frame, + }, +}; + +static const AVFilterPad outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + }, +}; + +const AVFilter ff_sf_textmod = { + .name = "textmod", + .description = NULL_IF_CONFIG_SMALL("Modify subtitle text in several ways"), + .init = init, + .uninit = uninit, + .query_formats = query_formats, + .priv_size = sizeof(TextModContext), + .priv_class = &textmod_class, + FILTER_INPUTS(inputs), + FILTER_OUTPUTS(outputs), +}; From patchwork Sun Sep 12 03:22:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 30176 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp2863641iov; Sat, 11 Sep 2021 20:23:46 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxwA09L9E7UXSnaD+wasMYV3bx1j29iUgVrf1d1D4B9vSxu1/QC+BEF8YYtFzddmmH8zOSo X-Received: by 2002:a05:6402:697:: with SMTP id f23mr5862973edy.253.1631417026368; Sat, 11 Sep 2021 20:23:46 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id da3si3201211edb.255.2021.09.11.20.23.46; Sat, 11 Sep 2021 20:23:46 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=oQDqe0X+; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8E63C68AA67; Sun, 12 Sep 2021 06:22:09 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10olkn2025.outbound.protection.outlook.com [40.92.41.25]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0EC2368AA4B for ; Sun, 12 Sep 2021 06:22:07 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Aq5gaPOOpdxdvFDILh1qZGx2QDinbZdqRfMCBtLm0nEhnw++qz6NG6vP5+VHqOfTpGzfbE4CeAJo//alSEcKojhQKl0g9qBL2xFvsKBclSqyXArC0NPKo/r4gkHsIRbHR7xwUY6mFMhtBXwLGLcKJm2GDJBHc6IebTz+kJ24Rm0TlDj2sEhEB6wGYPMslo9ZWG7xlKExdnUtJX93nJBMI0lOcubn27W9wxE5omcnV6WKQyd8FDDVYiRb8tk8sW8S9T2VVvDLsTI0q548BQJsqZIBhvVJefWITpWsSwpw4hdvnhhz1BXkvT+d0Un3vt7bNHsujWlvDcZJo7DFX0RUeA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=aqv6slxUggHk3KxScMa1FvkVo4oJF/4VN9icEPRh++U=; b=kgzPeAERazrhxV4vNOL+I6f1A3FdavnV1sgD72VJ5MSfR23E6KQpsQMeFDNXp25X0w+xB+ZBwA0wUF71hKpqYqbdHBLFzpV16EBSU/Zr790ugQK/IQvvd1VRW+8h6vrmUbPn2tZBwEjQROlfxxcKhQp3zgvq+xx8d5ap+xfawDFLyna2aUfDa3fnnQLmrBWh5GlxQgHJAWxR1AZlXTFX1M8v5MUBqYsDK1E01pF8592vDR4zbKPjQkkjLLtt/LqpZAlgUmV/LSzAVjOXL9TbWNDqV4x5wTvJIh8lt1edHO3rEH0Ohd94UZSC0EnTBOdVhq81OAygOr/Tl459WFj/nQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=aqv6slxUggHk3KxScMa1FvkVo4oJF/4VN9icEPRh++U=; b=oQDqe0X+toFbyB3KhKrj8atoNycZj5Q5meu0dnGm2q2jW+OFWv2Yni0AhobcooMbCV3ZsGdy5IXvqrhYxv9BOybU71FREe9ag25OE+nO01j0aatXGeiHGwWInieD6fpGbn6rO7JcjmtN4UwV8tJjrFKtfDpy43djeVaAQRo49N2/v23OXi+Egz0cxJol7XNgIaeP+6BKjwO8OaBcy5xsRQ2W5p1rWoiME97YlL8nb0NjVQ2Ey0Tcbs3jwG2HshXelt5BbnGM7av1pUvrwMlEFIiOKRYa5vf5CyUkQd0DklPonqY1DOu9+vo/NOz+8w8alg2l2smYLKoRaw+bhN50+g== Received: from MN2PR04MB5981.namprd04.prod.outlook.com (2603:10b6:208:da::10) by MN2PR04MB5728.namprd04.prod.outlook.com (2603:10b6:208:3d::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Sun, 12 Sep 2021 03:22:05 +0000 Received: from MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb]) by MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb%5]) with mapi id 15.20.4500.018; Sun, 12 Sep 2021 03:22:05 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v5 11/12] avcodec/ass_split: Extend ass dialog parsing Thread-Index: AQHXp4VcbR5M0ggNG0qPJDRa6r5NHQ== Date: Sun, 12 Sep 2021 03:22:05 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [f0/uXTDmz54k4dQqZdGK8Oc+B8F4n7KVKc2QkohMp4I=] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: f5616128-5875-4350-2b9e-08d9759c7f31 x-ms-traffictypediagnostic: MN2PR04MB5728: x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: kwu42Rv1dL/GqeGJDwGMj2uf0keMt1y7wGK7mrxahYAk1uQsvbuWmljbeHHKuAsTG9mBif1EeeY2BdCgw22P8TjzPiqJ3BNpGuqAGz2q4h8ScMpznBlnJQ3MRQgjD+k2NiY79P1bu6D4xpLDrzeHdyVNI5Kw15mFfWkKcCB8pOXl5/oqmLaV+OlB++p2vFuFKZfjr1aT2xWAjebpYkBhbRj5TDsKkg+bG5AngwFYVaXCSBNpmhLGfiEs48UPi/WEqljLc+oOYg4DdaHcirlN89r0q/UXcZbxMis8GXEG5rQY2PlpFFinNRexDZz/1dS1MKZvxPFdkhtO2+GrXrAhqQluaHjZVLPA15Y9X6TpvxJj1Wx4OctsXmMgQzHHRSTdRnqqIs/ITXlG6DPEpX2D2SSTwPlcrex00lSRT3EzoWoWVvd4hHG/tXTCbiAJodBS x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: /WgND1qfkC6J8jgUyB7RPKgnx4vAU53AmkekxCGrX39PaMXHOowIR6sgc68q9vY4KK8svGt6euZl3LmQ0qbBwkTeqlBHsA0KlLVpkjN2JVMn3ozc5wrZp4V0zIjnFwshIMQoFaP70BL+9SEVioaxUg== x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-3174-20-msonline-outlook-529c7.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR04MB5981.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: f5616128-5875-4350-2b9e-08d9759c7f31 X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Sep 2021 03:22:05.7693 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR04MB5728 Subject: [FFmpeg-devel] [PATCH v5 11/12] avcodec/ass_split: Extend ass dialog parsing X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: DM6DXWwWkZwr Signed-off-by: softworkz --- libavcodec/ass_split.c | 12 ++++++++++-- libavcodec/ass_split.h | 2 ++ 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/libavcodec/ass_split.c b/libavcodec/ass_split.c index 05c5453e53..e58585d3da 100644 --- a/libavcodec/ass_split.c +++ b/libavcodec/ass_split.c @@ -497,8 +497,8 @@ int ff_ass_split_override_codes(const ASSCodesCallbacks *callbacks, void *priv, while (*buf == '\\') { char style[2], c[2], sep[2], c_num[2] = "0", tmp[128] = {0}; unsigned int color = 0xFFFFFFFF; - int len, size = -1, an = -1, alpha = -1; - int x1, y1, x2, y2, t1 = -1, t2 = -1; + int len, size = -1, an = -1, alpha = -1, scale = 0; + int x1, y1, x2, y2, t1 = -1, t2 = -1, accel = 1; if (sscanf(buf, "\\%1[bisu]%1[01\\}]%n", style, c, &len) > 1) { int close = c[0] == '0' ? 1 : c[0] == '1' ? 0 : -1; len += close != -1; @@ -546,6 +546,14 @@ int ff_ass_split_override_codes(const ASSCodesCallbacks *callbacks, void *priv, } else if (sscanf(buf, "\\org(%d,%d)%1[\\}]%n", &x1, &y1, sep, &len) > 2) { if (callbacks->origin) callbacks->origin(priv, x1, y1); + } else if (sscanf(buf, "\\t(%d,%d,%1[\\}]%n", &t1, &t2, sep, &len) > 2 || + sscanf(buf, "\\t(%d,%d,%d,%1[\\}]%n", &t1, &t2, &accel, sep, &len) > 3) { + if (callbacks->animate) + callbacks->animate(priv, t1, t2, accel, tmp); + } else if (sscanf(buf, "\\p%1[\\}]%n", sep, &len) > 0 || + sscanf(buf, "\\p%u%1[\\}]%n", &scale, sep, &len) > 1) { + if (callbacks->drawing_mode) + callbacks->drawing_mode(priv, scale); } else { len = strcspn(buf+1, "\\}") + 2; /* skip unknown code */ } diff --git a/libavcodec/ass_split.h b/libavcodec/ass_split.h index a45fb9b8a1..bda7bb27db 100644 --- a/libavcodec/ass_split.h +++ b/libavcodec/ass_split.h @@ -156,7 +156,9 @@ typedef struct { * @{ */ void (*move)(void *priv, int x1, int y1, int x2, int y2, int t1, int t2); + void (*animate)(void *priv, int t1, int t2, int accel, char *style); void (*origin)(void *priv, int x, int y); + void (*drawing_mode)(void *priv, int scale); /** @} */ /** From patchwork Sun Sep 12 03:22:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 30171 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp2863707iov; Sat, 11 Sep 2021 20:23:57 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzrbf/Tq+30WQfZKn/ONLCL57cIC9XZTuwFLG/h9O4ccV7d/tP1nbFTzrfK25b/m65wT+Wj X-Received: by 2002:a50:ab42:: with SMTP id t2mr6171324edc.113.1631417037479; Sat, 11 Sep 2021 20:23:57 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id v18si3184150edx.605.2021.09.11.20.23.57; Sat, 11 Sep 2021 20:23:57 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=Fizvi4rR; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id AB4EC68AA7F; Sun, 12 Sep 2021 06:22:12 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10olkn2086.outbound.protection.outlook.com [40.92.41.86]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1F68F68AA6A for ; Sun, 12 Sep 2021 06:22:10 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eBVGOeQ5WTbHGHuOTSDAfp3LhkmIIX72uSLSv1iNa3WEhQE4XIlWwJQwx2AcvfRmydR5J0GzMo6bZ4q5JXXHoocrweMu7dZpzr8NPoDk1Tr69yCKJFmKwlD/PNA6D8vbeHH22gKofXYy/q3JRQ34gL/FlFNf+qyl7Wdi/g+GVPzC4jAf3DfoPR1co7q2IP8LZYjPw7hOrmqEkoX8g3ak4Xp83AcxJ9C2mwKDr4M5bz0977r2jwiPqFddOq2n4b6pEayR+A8Lutg7I9//yHFdKWbjN5qZeNfTODL5Ur6NYCSSlNM+8sndJ5PQnS/twdGqodiip21RQLamgQLpCaJ3xA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=Gv3UCl0YQw1n3DMu4JWPrxqWc/RDXW7ZAxtW2ve7iyg=; b=lhe3M3gkffHpCgrCnfW1mw612gdvz7xtre4jjVjRymi6SvaM3d9PjEG42/xY4eieVMxOsFY52KTQhgj7EVUFxMvD3mCgoVU5Z/Oms85od0Si+eSKgbwmup1lNbsziFQYJh1EGaMdLhQZiN3f1PsUXAdWK7QgeQVnWigXf4c/2gI3WhfeAOBIxECE+MFjWY0rEKaODQf86TSvOEKTe/Z61MNE6oY0AsAsG60TanQIjld8FWJ67fCzUoH6VQ5TxhorJWJjIvRZeRTJT+my1PnErMdttrlbe2kFRxd96JgqyI/fUvh77dmMGthyfDo5rPkRI/D5vTqySuEk5RvweZutsg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Gv3UCl0YQw1n3DMu4JWPrxqWc/RDXW7ZAxtW2ve7iyg=; b=Fizvi4rRnCQr1cTQ0+zbzYC6EdiafKIIiz16CUNi+SmJIGXGiQUrQtjV+dFILFh5sZkexZZWkg/v/9GKgoOpCIol+pKB2cuc1TQtIIHljW49A3pz8MZkKnQ42zvlvSyVx2wn/ddRUBpLbG/BSy2e9L3mPSlhdV6OF4kliZHwPTS9v7coawRewqboK5WU77FbasOXvIafMxTCpiYY+angKvXqWhf0QJqR6O9tCRTlN/AT9oybXODNE5AFcq3wPja3WxNcEz7y76fED4zeGJ7O9nJ7TIklzqSJtAN10krGujlrjxDvIup8ol0Jz8wz5mxjBSwd5m2Hso11gh21amz6xg== Received: from MN2PR04MB5981.namprd04.prod.outlook.com (2603:10b6:208:da::10) by MN2PR04MB5728.namprd04.prod.outlook.com (2603:10b6:208:3d::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Sun, 12 Sep 2021 03:22:08 +0000 Received: from MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb]) by MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb%5]) with mapi id 15.20.4500.018; Sun, 12 Sep 2021 03:22:08 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v5 12/12] avfilter/stripstyles: Add stripstyles filter Thread-Index: AQHXp4VeYrniaPUvMEurCkDYPEmD9Q== Date: Sun, 12 Sep 2021 03:22:08 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [MF1C250X5nRmcDN5XJqpb9DMxgIetK8OcNLHrpiAlTk=] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 68299079-0ccb-43d3-8f62-08d9759c80ed x-ms-traffictypediagnostic: MN2PR04MB5728: x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: /ts4ruDvmOuNBLvVXAIDQrIpanma6veGJQM1YzXRWpVR4qSFS8e4IxrJ2XFI+mPaCkzsl+RRybrR/rMKVsl6VKUmMAOR/dJYFnp7VuX+/ZiNanElVIku2IZ9E/np2okleCs+Ul6ovirUXUt6ZbfbUSb8CNTJbImC0p4CU8ZIIT8qpo88OXHcKhs5f6vPd+Z3JWPrmiD/nMP6zUSsA6zAMNz2lTrcNDtNftr40CkLfc3VWUkk3hKHL9toTgBvdZQazlL8A1z/hvyU59iTl6nknXTEaraqc7ZpMs/p7aF6lDiBs1Wlbh9SLcdONsI+g0RnmO1WCUI05nPeHi/NXtQe3a2owqQeW01QUg/QozsMx4S9Rqhi4K5qf2VbljrJMTl2XbZY7QtIcUthcHacuXkRV2tYtk+5xYHdfAZtL1B1J3OVakMCCdHhwtsIr4XLH8JFxj/8U1vuxD4kNAH3WJVeB1RaDWU3oASNPYW/vo6tpxw= x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: nw4rJecd/tPCEJA0QYyOd3KyiTuOS+pt5jj/kyP4fmT2v2uivDGC3TMNHjTCbwze4te8xSrAuZW4DjU6QuZ9p778/2Iuy0CTOC5kS4cqiuAiFnmnCwGWDYaKwjVSW7ynVdWSmdsx4GZ2T7UJe83dcw== x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-3174-20-msonline-outlook-529c7.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR04MB5981.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 68299079-0ccb-43d3-8f62-08d9759c80ed X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Sep 2021 03:22:08.7246 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR04MB5728 Subject: [FFmpeg-devel] [PATCH v5 12/12] avfilter/stripstyles: Add stripstyles filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 7/zL/HjWSJj4 Signed-off-by: softworkz --- doc/filters.texi | 24 ++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/sf_stripstyles.c | 215 +++++++++++++++++++++++++++++++++++ 4 files changed, 241 insertions(+) create mode 100644 libavfilter/sf_stripstyles.c diff --git a/doc/filters.texi b/doc/filters.texi index 9fd2876d63..6c74dacee3 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -25024,6 +25024,30 @@ existing filters using @code{--disable-filters}. Below is a description of the currently available subtitle filters. +@section stripstyles + +Remove all inline styles from subtitle events. + +It accepts the following parameters: + +@table @option +@item remove_animated +Also remove text which is subject to animation (default: true) +Usually, animated text elements are used used in addition to static subtitle lines for creating effects, so in most cases it is safe to remove the animation content. +If subtitle text is missing, try setting this to false. + +@end table + +@subsection Examples + +@itemize +@item +Change all characters to upper case while keeping all styles and animations: +@example +ffmpeg -i "https://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.mkv" -filter_complex "[0:1]stripstyles" -map 0 output.mkv +@end example +@end itemize + @section textmod Modify subtitle text in a number of ways. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 5a5a4be47e..626e1bed74 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -536,6 +536,7 @@ OBJS-$(CONFIG_NULLSINK_FILTER) += vsink_nullsink.o # subtitle filters OBJS-$(CONFIG_TEXTMOD_FILTER) += sf_textmod.o +OBJS-$(CONFIG_STRIPSTYLES_FILTER) += sf_stripstyles.o # multimedia filters OBJS-$(CONFIG_ABITSCOPE_FILTER) += avf_abitscope.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 6d7a535ee8..8f7d697408 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -524,6 +524,7 @@ extern const AVFilter ff_avf_showvolume; extern const AVFilter ff_avf_showwaves; extern const AVFilter ff_avf_showwavespic; extern const AVFilter ff_vaf_spectrumsynth; +extern const AVFilter ff_sf_stripstyles; extern const AVFilter ff_sf_textmod; extern const AVFilter ff_svf_graphicsub2video; extern const AVFilter ff_svf_textsub2video; diff --git a/libavfilter/sf_stripstyles.c b/libavfilter/sf_stripstyles.c new file mode 100644 index 0000000000..20e42b47e0 --- /dev/null +++ b/libavfilter/sf_stripstyles.c @@ -0,0 +1,215 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * text subtitle filter which removes inline-styles from subtitles + */ + +#include + +#include "libavutil/avassert.h" +#include "libavutil/opt.h" +#include "avfilter.h" +#include "internal.h" +#include "libavcodec/avcodec.h" +#include "libavcodec/ass_split.h" + +typedef struct StripStylesContext { + const AVClass *class; + enum AVSubtitleType format; + int remove_animated; +} StripStylesContext; + +typedef struct DialogContext { + const AVClass *class; + const StripStylesContext* ss_ctx; + AVBPrint buffer; + int drawing_scale; + int is_animated; +} DialogContext; + +static void dialog_text_cb(void *priv, const char *text, int len) +{ + DialogContext *s = priv; + if (!s->drawing_scale && (!s->is_animated || !s->ss_ctx->remove_animated)) + av_bprint_append_data(&s->buffer, text, len); +} + +static void dialog_new_line_cb(void *priv, int forced) +{ + DialogContext *s = priv; + if (!s->drawing_scale && !s->is_animated) + av_bprint_append_data(&s->buffer, forced ? "\\N" : "\\n", 2); +} + +static void dialog_drawing_mode_cb(void *priv, int scale) +{ + DialogContext *s = priv; + s->drawing_scale = scale; +} + +static void dialog_animate_cb(void *priv, int t1, int t2, int accel, char *style) +{ + DialogContext *s = priv; + s->is_animated = 1; +} + +static void dialog_move_cb(void *priv, int x1, int y1, int x2, int y2, int t1, int t2) +{ + DialogContext *s = priv; + if (t1 >= 0 || t2 >= 0) + s->is_animated = 1; +} + +static const ASSCodesCallbacks dialog_callbacks = { + .text = dialog_text_cb, + .new_line = dialog_new_line_cb, + .drawing_mode = dialog_drawing_mode_cb, + .animate = dialog_animate_cb, + .move = dialog_move_cb, +}; + +static int query_formats(AVFilterContext *ctx) +{ + AVFilterFormats *formats; + AVFilterLink *inlink = ctx->inputs[0]; + AVFilterLink *outlink = ctx->outputs[0]; + static const enum AVSubtitleType subtitle_fmts[] = { AV_SUBTITLE_FMT_ASS, AV_SUBTITLE_FMT_NONE }; + int ret; + + /* set input subtitle format */ + formats = ff_make_format_list(subtitle_fmts); + if ((ret = ff_formats_ref(formats, &inlink->outcfg.formats)) < 0) + return ret; + + /* set output video format */ + if ((ret = ff_formats_ref(formats, &outlink->incfg.formats)) < 0) + return ret; + + return 0; +} + +static char *ass_get_line(int readorder, int layer, const char *style, + const char *speaker, const char *effect, const char *text) +{ + return av_asprintf("%d,%d,%s,%s,0,0,0,%s,%s", + readorder, layer, style ? style : "Default", + speaker ? speaker : "", effect, text); +} + +static char *process_dialog(StripStylesContext *s, const char *ass_line) +{ + ASSDialog *dialog = ff_ass_split_dialog(NULL, ass_line); + char *result = NULL, *text; + DialogContext dlg_ctx = { 0 }; + + if (!dialog) + return NULL; + + dlg_ctx.ss_ctx = s; + + av_bprint_init(&dlg_ctx.buffer, 0, AV_BPRINT_SIZE_UNLIMITED); + + ff_ass_split_override_codes(&dialog_callbacks, &dlg_ctx, dialog->text); + + av_bprint_finalize(&dlg_ctx.buffer, &text); + + if (text && strlen(text) > 0) + result = ass_get_line(dialog->readorder, dialog->layer, dialog->style, dialog->name, dialog->effect, text); + + av_free(text); + ff_ass_free_dialog(&dialog); + + return result; +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *src_frame) +{ + StripStylesContext *s = inlink->dst->priv; + AVFilterLink *outlink = inlink->dst->outputs[0]; + AVFrame *out; + + outlink->format = inlink->format; + + out = av_frame_clone(src_frame); + if (!out) + return AVERROR(ENOMEM); + + for (unsigned i = 0; i < out->num_subtitle_rects; i++) { + + AVSubtitleRect *rect = out->subtitle_rects[i]; + + if (rect->ass) { + if (rect->ass) { + char *tmp = rect->ass; + rect->ass = process_dialog(s, rect->ass); + + if (rect->ass) { + av_log(inlink->dst, AV_LOG_DEBUG, "original: %s\n", tmp); + av_log(inlink->dst, AV_LOG_DEBUG, "stripped: %s\n", rect->ass); + } + else + rect->ass = av_strdup(""); + + av_free(tmp); + } + } + } + + av_frame_free(&src_frame); + return ff_filter_frame(outlink, out); +} + +#define OFFSET(x) offsetof(StripStylesContext, x) +#define FLAGS (AV_OPT_FLAG_SUBTITLE_PARAM | AV_OPT_FLAG_FILTERING_PARAM) + +static const AVOption stripstyles_options[] = { + { "remove_animated", "remove animated text (default: yes)", OFFSET(remove_animated), AV_OPT_TYPE_BOOL, {.i64 = 1 }, 0, 1, FLAGS, 0 }, + { NULL }, +}; + +AVFILTER_DEFINE_CLASS(stripstyles); + +static const AVFilterPad inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .filter_frame = filter_frame, + }, +}; + +static const AVFilterPad outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + }, +}; + +const AVFilter ff_sf_stripstyles = { + .name = "stripstyles", + .description = NULL_IF_CONFIG_SMALL("Strip subtitle inline styles"), + .query_formats = query_formats, + .priv_size = sizeof(StripStylesContext), + .priv_class = &stripstyles_class, + FILTER_INPUTS(inputs), + FILTER_OUTPUTS(outputs), +}; +