From patchwork Thu Sep 16 20:30:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 30278 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp1581850iov; Thu, 16 Sep 2021 13:30:29 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzWY3jpWH+RnHoRynxN1tZ7B9bIpgMe6VTXbp0VuCYj2jTMD+ABbp/1pInXM0E5XOty/2Bn X-Received: by 2002:a17:907:3e20:: with SMTP id hp32mr8142929ejc.536.1631824228772; Thu, 16 Sep 2021 13:30:28 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id s12si4488570ejc.488.2021.09.16.13.30.28; Thu, 16 Sep 2021 13:30:28 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=HWVhHXyq; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9BA2C68B0A1; Thu, 16 Sep 2021 23:30:16 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11olkn2053.outbound.protection.outlook.com [40.92.18.53]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0406268B075 for ; Thu, 16 Sep 2021 23:30:10 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Wns5amWPHmH1DqGMriGUxF9EXG5d85UocnoWE2JUmMud3ZzBwDc5T2+ZFvlsjmr8Qxc83o2jhczD4BFetECdPXP6Oy/ZqEnrvGcqxOoFzaMXoeUCIS5eDa0mNZS6NfHW8kwKZcRirdKDsNgJWq92qJxPkgCKsuzBKIbWNFfHEvy5bi4ka5Kn/X/Gc5c4uJwZnqnIuf/Mh1Jxmx1kqaSiQFCcJCELUqKg0qBJbjAk6eiWJqvzKB8vLGmPFlqKI3UZsoHxKelC0NVPMh1GhkZHTLV9nJR8sFNebuD9k80EJ6Bw1UEuahyaDihLgcTnGxEr0uLO9DbRaPnJ6YJGJ7lwhA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=Fkj/mZ7KoRSJXTWn307tmMaEfOIhmEm0GgeScmkFL+Y=; b=Xx1nT+HeHdhpdBJJ9GPJDRYCfr3aa5VzKAYVwDBm/2krteRMKXahZa4uZj/XEdzUwBhmJTe3rH+cU2p7QVBLWOwQH5iUgUaYbBxkXBocJjIruQy91PVbJ4pCSRLpD9xT8DyEhP9YWPezH4PbA7/I1IBSGLRH9HW20JZP9mKOdTjt8cW+b9m51VZuJmuMRIJ9QlveoUZ816yWI9XqVeHplAxIVFq+fQXXHeKrdOodzJgjJzfZVzDW9im81h6VfRijLRFLsrRSDAZtlKC6dKfunzK2QVFE6dD3czmE88BRAxllUfPDKDcW8fnAufpfkKsLcDfBx6Sld2Li+ZFVSqB8+Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Fkj/mZ7KoRSJXTWn307tmMaEfOIhmEm0GgeScmkFL+Y=; b=HWVhHXyqApKUCgCqHmJCdwK4TvVlWDmf+NBC2tq2LSkYXPZRKh7AJXI+/aQrq2lwHpYbk+qUkS48D4Pb1qhLmE848LCtOAY5YdKBLsoU0J5Zy32IGqQohvA4QL12NPcfl5xwLGTmSmwR6vhGg6kz0jht7T+UG1jTpxiK8FjaNUOy8WmGxp9axYl6ezX5NO6wylIRFTGrUvuN7Exv2Md0xzV2TnmUPqthT2yZKSJh2mE0zLJRkhlRjn8v6VyrcHOuX7IP19irbm7n93i6/coi53OHGMlCYEBWRQ2WreBfJHO2rJuwNyBaMCS+SULdM+0E9uk7xBEYJF8alUbu1si3Ng== Received: from MN2PR04MB5981.namprd04.prod.outlook.com (2603:10b6:208:da::10) by MN2PR04MB6240.namprd04.prod.outlook.com (2603:10b6:208:e2::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Thu, 16 Sep 2021 20:30:05 +0000 Received: from MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb]) by MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb%5]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021 20:30:05 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v6 01/11] global: Prepare AVFrame for subtitle handling Thread-Index: AQHXqzmianQ2fgXk3UCWt7lTpPcN2w== Date: Thu, 16 Sep 2021 20:30:05 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [u3+DFeV5noPZSR9Cm60d+PSkvBvdMvkH] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 36e73e9b-d0c5-4699-0a06-08d97950c49b x-ms-traffictypediagnostic: MN2PR04MB6240: x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: i1L+oIO9pYGG/ITgoiCmD0a7tbPOQHyCgx42ylACuBJQg+iXqIK91CxA3J+koz30PFcUYj+8fDiZ6zG4hR30/0gddL7N7qnZfuml/uxldLNrJNj/tKoDffk5b5mgYoVF96R6a6GL1FutAB3639JItva0paDGA2baAS8qDDeGJI6OvOhQBkV9nk87mbVebkF6Am66Z7R0zfLB+j4pVSUOi6gesE6BDDlMjBXhOAUexwuIE89iLhZiNVtSMnC2OXc7nVKrSjD/k5Ect6qOyihkqPwC4sVKPl1g2BnLh6Fl1UBOv0wNK8oXoaIU5xlhskQKj1eiKpZmuPwMptljrfpbmAMBg4CthCgsVOR3L4MC80TvijU2gWjBVCKeRUA69/9DVP7FL/wXHXqAsGmd6IMRwn3y0XtXGgMXZzIDNSpk+t0+4LM0ClKKgCFAEMstYeSQ x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: ljMQx8JvLkSySd+9xrLEs1SHNLELS5tr4N7WHQhIBkiw/srn7MYP4RoIBCnbgMSkimWCbnzdRQkoYwN1phGYcdvqGdW71dTJ5D7Q7neq19R+tRw28s5Az0c0uasylpUTTZbJzp9Icx2t59aFHejB0A== x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-3174-20-msonline-outlook-529c7.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR04MB5981.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 36e73e9b-d0c5-4699-0a06-08d97950c49b X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Sep 2021 20:30:05.1196 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR04MB6240 Subject: [FFmpeg-devel] [PATCH v6 01/11] global: Prepare AVFrame for subtitle handling X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: w2B/0Ct8qVz5 Root commit for adding subtitle filtering capabilities. In detail: - Add type (AVMediaType) field to AVFrame Replaces previous way of distinction which was based on checking width and height to determine whether a frame is audio or video - Add subtitle fields to AVFrame - Add new struct AVSubtitleArea, similar to AVSubtitleRect, but different allocation logic. Cannot and must not be used interchangeably, hence the new struct - Move enum AVSubtitleType, AVSubtitle and AVSubtitleRect to avutil - Add public-named members to enum AVSubtitleType (AV_SUBTITLE_FMT_) - Add avcodec_decode_subtitle3 which takes subtitle frames, serving as compatibility shim to legacy subtitle decoding - Add additional methods for conversion between old and new API Signed-off-by: softworkz --- libavcodec/avcodec.c | 19 ---- libavcodec/avcodec.h | 105 +++++++----------- libavcodec/decode.c | 24 +++- libavcodec/pgssubdec.c | 1 + libavcodec/utils.c | 11 ++ libavfilter/vf_subtitles.c | 28 ++--- libavformat/utils.c | 1 + libavutil/Makefile | 2 + libavutil/frame.c | 184 +++++++++++++++++++++++++++--- libavutil/frame.h | 93 +++++++++++++++- libavutil/subfmt.c | 221 +++++++++++++++++++++++++++++++++++++ libavutil/subfmt.h | 186 +++++++++++++++++++++++++++++++ libavutil/version.h | 2 +- 13 files changed, 757 insertions(+), 120 deletions(-) create mode 100644 libavutil/subfmt.c create mode 100644 libavutil/subfmt.h diff --git a/libavcodec/avcodec.c b/libavcodec/avcodec.c index 2dd7dd84e0..963f52c4bd 100644 --- a/libavcodec/avcodec.c +++ b/libavcodec/avcodec.c @@ -426,25 +426,6 @@ void avcodec_flush_buffers(AVCodecContext *avctx) av_bsf_flush(avci->bsf); } -void avsubtitle_free(AVSubtitle *sub) -{ - int i; - - for (i = 0; i < sub->num_rects; i++) { - av_freep(&sub->rects[i]->data[0]); - av_freep(&sub->rects[i]->data[1]); - av_freep(&sub->rects[i]->data[2]); - av_freep(&sub->rects[i]->data[3]); - av_freep(&sub->rects[i]->text); - av_freep(&sub->rects[i]->ass); - av_freep(&sub->rects[i]); - } - - av_freep(&sub->rects); - - memset(sub, 0, sizeof(*sub)); -} - av_cold int avcodec_close(AVCodecContext *avctx) { int i; diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index ffd58c333f..de23067d29 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -35,6 +35,7 @@ #include "libavutil/frame.h" #include "libavutil/log.h" #include "libavutil/pixfmt.h" +#include "libavutil/subfmt.h" #include "libavutil/rational.h" #include "codec.h" @@ -1670,7 +1671,7 @@ typedef struct AVCodecContext { /** * Header containing style information for text subtitles. - * For SUBTITLE_ASS subtitle type, it should contain the whole ASS + * For AV_SUBTITLE_FMT_ASS subtitle type, it should contain the whole ASS * [Script Info] and [V4+ Styles] section, plus the [Events] line and * the Format line following. It shouldn't include any Dialogue line. * - encoding: Set/allocated/freed by user (before avcodec_open2()) @@ -2233,63 +2234,8 @@ typedef struct AVHWAccel { * @} */ -enum AVSubtitleType { - SUBTITLE_NONE, - - SUBTITLE_BITMAP, ///< A bitmap, pict will be set - - /** - * Plain text, the text field must be set by the decoder and is - * authoritative. ass and pict fields may contain approximations. - */ - SUBTITLE_TEXT, - - /** - * Formatted text, the ass field must be set by the decoder and is - * authoritative. pict and text fields may contain approximations. - */ - SUBTITLE_ASS, -}; - #define AV_SUBTITLE_FLAG_FORCED 0x00000001 -typedef struct AVSubtitleRect { - int x; ///< top left corner of pict, undefined when pict is not set - int y; ///< top left corner of pict, undefined when pict is not set - int w; ///< width of pict, undefined when pict is not set - int h; ///< height of pict, undefined when pict is not set - int nb_colors; ///< number of colors in pict, undefined when pict is not set - - /** - * data+linesize for the bitmap of this subtitle. - * Can be set for text/ass as well once they are rendered. - */ - uint8_t *data[4]; - int linesize[4]; - - enum AVSubtitleType type; - - char *text; ///< 0 terminated plain UTF-8 text - - /** - * 0 terminated ASS/SSA compatible event line. - * The presentation of this is unaffected by the other values in this - * struct. - */ - char *ass; - - int flags; -} AVSubtitleRect; - -typedef struct AVSubtitle { - uint16_t format; /* 0 = graphics */ - uint32_t start_display_time; /* relative to packet pts, in ms */ - uint32_t end_display_time; /* relative to packet pts, in ms */ - unsigned num_rects; - AVSubtitleRect **rects; - int64_t pts; ///< Same as packet pts, in AV_TIME_BASE -} AVSubtitle; - /** * Return the LIBAVCODEC_VERSION_INT constant. */ @@ -2425,13 +2371,6 @@ int avcodec_open2(AVCodecContext *avctx, const AVCodec *codec, AVDictionary **op */ int avcodec_close(AVCodecContext *avctx); -/** - * Free all allocated data in the given subtitle struct. - * - * @param sub AVSubtitle to free. - */ -void avsubtitle_free(AVSubtitle *sub); - /** * @} */ @@ -2522,11 +2461,43 @@ enum AVChromaLocation avcodec_chroma_pos_to_enum(int xpos, int ypos); * must be freed with avsubtitle_free if *got_sub_ptr is set. * @param[in,out] got_sub_ptr Zero if no subtitle could be decompressed, otherwise, it is nonzero. * @param[in] avpkt The input AVPacket containing the input buffer. + * + * @deprecated Use avcodec_decode_subtitle3 instead. */ int avcodec_decode_subtitle2(AVCodecContext *avctx, AVSubtitle *sub, int *got_sub_ptr, AVPacket *avpkt); +/** + * Decode a subtitle message. + * Return a negative value on error, otherwise return the number of bytes used. + * If no subtitle could be decompressed, got_sub_ptr is zero. + * Otherwise, the subtitle is stored in *sub. + * Note that AV_CODEC_CAP_DR1 is not available for subtitle codecs. This is for + * simplicity, because the performance difference is expected to be negligible + * and reusing a get_buffer written for video codecs would probably perform badly + * due to a potentially very different allocation pattern. + * + * Some decoders (those marked with AV_CODEC_CAP_DELAY) have a delay between input + * and output. This means that for some packets they will not immediately + * produce decoded output and need to be flushed at the end of decoding to get + * all the decoded data. Flushing is done by calling this function with packets + * with avpkt->data set to NULL and avpkt->size set to 0 until it stops + * returning subtitles. It is safe to flush even those decoders that are not + * marked with AV_CODEC_CAP_DELAY, then no subtitles will be returned. + * + * @note The AVCodecContext MUST have been opened with @ref avcodec_open2() + * before packets may be fed to the decoder. + * + * @param avctx the codec context + * @param[out] frame The preallocated AVFrame in which the decoded subtitle will be stored, + * must be freed with av_frame_free if *got_sub_ptr is set. + * @param[in,out] got_sub_ptr Zero if no subtitle could be decompressed, otherwise, it is nonzero. + * @param[in] avpkt The input AVPacket containing the input buffer. + */ +int avcodec_decode_subtitle3(AVCodecContext *avctx, AVFrame *frame, + int *got_sub_ptr, AVPacket *avpkt); + /** * Supply raw packet data as input to a decoder. * @@ -3120,6 +3091,14 @@ void avcodec_flush_buffers(AVCodecContext *avctx); */ int av_get_audio_frame_duration(AVCodecContext *avctx, int frame_bytes); +/** + * Return subtitle format from a codec descriptor + * + * @param codec_descriptor codec descriptor + * @return the subtitle type (e.g. bitmap, text) + */ +enum AVSubtitleType av_get_subtitle_format_from_codecdesc(const AVCodecDescriptor *codec_descriptor); + /* memory */ /** diff --git a/libavcodec/decode.c b/libavcodec/decode.c index 643f9d6a30..42aa01f5a1 100644 --- a/libavcodec/decode.c +++ b/libavcodec/decode.c @@ -841,10 +841,7 @@ int avcodec_decode_subtitle2(AVCodecContext *avctx, AVSubtitle *sub, avctx->pkt_timebase, ms); } - if (avctx->codec_descriptor->props & AV_CODEC_PROP_BITMAP_SUB) - sub->format = 0; - else if (avctx->codec_descriptor->props & AV_CODEC_PROP_TEXT_SUB) - sub->format = 1; + sub->format = av_get_subtitle_format_from_codecdesc(avctx->codec_descriptor); for (unsigned i = 0; i < sub->num_rects; i++) { if (avctx->sub_charenc_mode != FF_SUB_CHARENC_MODE_IGNORE && @@ -868,6 +865,25 @@ int avcodec_decode_subtitle2(AVCodecContext *avctx, AVSubtitle *sub, return ret; } +int avcodec_decode_subtitle3(AVCodecContext *avctx, AVFrame *frame, + int *got_sub_ptr, AVPacket *avpkt) +{ + int ret; + AVSubtitle subtitle = { 0 }; + + ret = avcodec_decode_subtitle2(avctx, &subtitle, got_sub_ptr, avpkt); + + if (ret >= 0) { + ret = av_frame_get_buffer2(frame, 0); + if (ret < 0) + return ret; + av_frame_put_subtitle(frame, &subtitle); + } + + avsubtitle_free(&subtitle); + + return ret; +} enum AVPixelFormat avcodec_default_get_format(struct AVCodecContext *avctx, const enum AVPixelFormat *fmt) { diff --git a/libavcodec/pgssubdec.c b/libavcodec/pgssubdec.c index 55eda4c2a9..efb16e4bb3 100644 --- a/libavcodec/pgssubdec.c +++ b/libavcodec/pgssubdec.c @@ -32,6 +32,7 @@ #include "libavutil/colorspace.h" #include "libavutil/imgutils.h" #include "libavutil/opt.h" +#include "libavutil/subfmt.h" #define RGBA(r,g,b,a) (((unsigned)(a) << 24) | ((r) << 16) | ((g) << 8) | (b)) #define MAX_EPOCH_PALETTES 8 // Max 8 allowed per PGS epoch diff --git a/libavcodec/utils.c b/libavcodec/utils.c index cfc07cbcb8..767dfabd1e 100644 --- a/libavcodec/utils.c +++ b/libavcodec/utils.c @@ -818,6 +818,17 @@ int av_get_audio_frame_duration(AVCodecContext *avctx, int frame_bytes) return FFMAX(0, duration); } +enum AVSubtitleType av_get_subtitle_format_from_codecdesc(const AVCodecDescriptor *codec_descriptor) +{ + if(codec_descriptor->props & AV_CODEC_PROP_BITMAP_SUB) + return AV_SUBTITLE_FMT_BITMAP; + + if(codec_descriptor->props & AV_CODEC_PROP_TEXT_SUB) + return AV_SUBTITLE_FMT_ASS; + + return AV_SUBTITLE_FMT_UNKNOWN; +} + int av_get_audio_frame_duration2(AVCodecParameters *par, int frame_bytes) { int duration = get_audio_frame_duration(par->codec_id, par->sample_rate, diff --git a/libavfilter/vf_subtitles.c b/libavfilter/vf_subtitles.c index d0bafcd3cf..ee4d7ab94a 100644 --- a/libavfilter/vf_subtitles.c +++ b/libavfilter/vf_subtitles.c @@ -35,14 +35,12 @@ # include "libavformat/avformat.h" #endif #include "libavutil/avstring.h" -#include "libavutil/imgutils.h" #include "libavutil/opt.h" #include "libavutil/parseutils.h" #include "drawutils.h" #include "avfilter.h" #include "internal.h" #include "formats.h" -#include "video.h" typedef struct AssContext { const AVClass *class; @@ -306,6 +304,7 @@ static av_cold int init_subtitles(AVFilterContext *ctx) AVStream *st; AVPacket pkt; AssContext *ass = ctx->priv; + enum AVSubtitleType subtitle_format; /* Init libass */ ret = init(ctx); @@ -386,16 +385,19 @@ static av_cold int init_subtitles(AVFilterContext *ctx) ret = AVERROR_DECODER_NOT_FOUND; goto end; } + dec_desc = avcodec_descriptor_get(st->codecpar->codec_id); - if (dec_desc && !(dec_desc->props & AV_CODEC_PROP_TEXT_SUB)) { + subtitle_format = av_get_subtitle_format_from_codecdesc(dec_desc); + + if (subtitle_format != AV_SUBTITLE_FMT_ASS) { av_log(ctx, AV_LOG_ERROR, - "Only text based subtitles are currently supported\n"); - ret = AVERROR_PATCHWELCOME; + "Only text based subtitles are supported by this filter\n"); + ret = AVERROR_INVALIDDATA; goto end; } + if (ass->charenc) av_dict_set(&codec_opts, "sub_charenc", ass->charenc, 0); - av_dict_set(&codec_opts, "sub_text_format", "ass", 0); dec_ctx = avcodec_alloc_context3(dec); if (!dec_ctx) { @@ -449,18 +451,18 @@ static av_cold int init_subtitles(AVFilterContext *ctx) dec_ctx->subtitle_header_size); while (av_read_frame(fmt, &pkt) >= 0) { int i, got_subtitle; - AVSubtitle sub = {0}; + AVFrame *sub = av_frame_alloc(); if (pkt.stream_index == sid) { - ret = avcodec_decode_subtitle2(dec_ctx, &sub, &got_subtitle, &pkt); + ret = avcodec_decode_subtitle3(dec_ctx, sub, &got_subtitle, &pkt); if (ret < 0) { av_log(ctx, AV_LOG_WARNING, "Error decoding: %s (ignored)\n", av_err2str(ret)); } else if (got_subtitle) { - const int64_t start_time = av_rescale_q(sub.pts, AV_TIME_BASE_Q, av_make_q(1, 1000)); - const int64_t duration = sub.end_display_time; - for (i = 0; i < sub.num_rects; i++) { - char *ass_line = sub.rects[i]->ass; + const int64_t start_time = av_rescale_q(sub->subtitle_pts, AV_TIME_BASE_Q, av_make_q(1, 1000)); + const int64_t duration = sub->subtitle_end_time; + for (i = 0; i < sub->num_subtitle_areas; i++) { + char *ass_line = sub->subtitle_areas[i]->ass; if (!ass_line) break; ass_process_chunk(ass->track, ass_line, strlen(ass_line), @@ -469,7 +471,7 @@ static av_cold int init_subtitles(AVFilterContext *ctx) } } av_packet_unref(&pkt); - avsubtitle_free(&sub); + av_frame_free(&sub); } end: diff --git a/libavformat/utils.c b/libavformat/utils.c index a322137fcf..bb327b29b6 100644 --- a/libavformat/utils.c +++ b/libavformat/utils.c @@ -32,6 +32,7 @@ #include "libavutil/opt.h" #include "libavutil/parseutils.h" #include "libavutil/pixfmt.h" +#include "libavutil/subfmt.h" #include "libavutil/thread.h" #include "libavutil/time.h" #include "libavutil/timestamp.h" diff --git a/libavutil/Makefile b/libavutil/Makefile index 410ac636f7..04e1101bf3 100644 --- a/libavutil/Makefile +++ b/libavutil/Makefile @@ -74,6 +74,7 @@ HEADERS = adler32.h \ sha512.h \ spherical.h \ stereo3d.h \ + subfmt.h \ threadmessage.h \ time.h \ timecode.h \ @@ -159,6 +160,7 @@ OBJS = adler32.o \ slicethread.o \ spherical.o \ stereo3d.o \ + subfmt.o \ threadmessage.o \ time.o \ timecode.o \ diff --git a/libavutil/frame.c b/libavutil/frame.c index b0ceaf7145..6d3dd3c3f4 100644 --- a/libavutil/frame.c +++ b/libavutil/frame.c @@ -26,6 +26,7 @@ #include "imgutils.h" #include "mem.h" #include "samplefmt.h" +#include "subfmt.h" #include "hwcontext.h" #define CHECK_CHANNELS_CONSISTENCY(frame) \ @@ -50,6 +51,9 @@ const char *av_get_colorspace_name(enum AVColorSpace val) return name[val]; } #endif + +static int frame_copy_subtitles(AVFrame *dst, const AVFrame *src, int copy_data); + static void get_frame_defaults(AVFrame *frame) { if (frame->extended_data != frame->data) @@ -72,7 +76,12 @@ static void get_frame_defaults(AVFrame *frame) frame->colorspace = AVCOL_SPC_UNSPECIFIED; frame->color_range = AVCOL_RANGE_UNSPECIFIED; frame->chroma_location = AVCHROMA_LOC_UNSPECIFIED; - frame->flags = 0; + frame->subtitle_start_time = 0; + frame->subtitle_end_time = 0; + frame->num_subtitle_areas = 0; + frame->subtitle_areas = NULL; + frame->subtitle_pts = 0; + frame->subtitle_header = NULL; } static void free_side_data(AVFrameSideData **ptr_sd) @@ -243,23 +252,52 @@ static int get_audio_buffer(AVFrame *frame, int align) } +static int get_subtitle_buffer(AVFrame *frame) +{ + // Buffers in AVFrame->buf[] are not used in case of subtitle frames. + // To accomodate with existing code, checking ->buf[0] to determine + // whether a frame is ref-counted or has data, we're adding a 1-byte + // buffer here, which marks the subtitle frame to contain data. + frame->buf[0] = av_buffer_alloc(1); + if (!frame->buf[0]) { + av_frame_unref(frame); + return AVERROR(ENOMEM); + } + return 0; +} + int av_frame_get_buffer(AVFrame *frame, int align) +{ + if (frame->width > 0 && frame->height > 0) + frame->type = AVMEDIA_TYPE_VIDEO; + else if (frame->nb_samples > 0 && (frame->channel_layout || frame->channels > 0)) + frame->type = AVMEDIA_TYPE_AUDIO; + + return av_frame_get_buffer2(frame, align); +} + +int av_frame_get_buffer2(AVFrame *frame, int align) { if (frame->format < 0) return AVERROR(EINVAL); - if (frame->width > 0 && frame->height > 0) + switch(frame->type) { + case AVMEDIA_TYPE_VIDEO: return get_video_buffer(frame, align); - else if (frame->nb_samples > 0 && (frame->channel_layout || frame->channels > 0)) + case AVMEDIA_TYPE_AUDIO: return get_audio_buffer(frame, align); - - return AVERROR(EINVAL); + case AVMEDIA_TYPE_SUBTITLE: + return get_subtitle_buffer(frame); + default: + return AVERROR(EINVAL); + } } static int frame_copy_props(AVFrame *dst, const AVFrame *src, int force_copy) { int ret, i; + dst->type = src->type; dst->key_frame = src->key_frame; dst->pict_type = src->pict_type; dst->sample_aspect_ratio = src->sample_aspect_ratio; @@ -290,6 +328,10 @@ static int frame_copy_props(AVFrame *dst, const AVFrame *src, int force_copy) dst->colorspace = src->colorspace; dst->color_range = src->color_range; dst->chroma_location = src->chroma_location; + dst->subtitle_start_time = src->subtitle_start_time; + dst->subtitle_end_time = src->subtitle_end_time; + dst->subtitle_pts = src->subtitle_pts; + dst->subtitle_header = src->subtitle_header; av_dict_copy(&dst->metadata, src->metadata, 0); @@ -331,6 +373,7 @@ int av_frame_ref(AVFrame *dst, const AVFrame *src) av_assert1(dst->width == 0 && dst->height == 0); av_assert1(dst->channels == 0); + dst->type = src->type; dst->format = src->format; dst->width = src->width; dst->height = src->height; @@ -344,7 +387,7 @@ int av_frame_ref(AVFrame *dst, const AVFrame *src) /* duplicate the frame data if it's not refcounted */ if (!src->buf[0]) { - ret = av_frame_get_buffer(dst, 0); + ret = av_frame_get_buffer2(dst, 0); if (ret < 0) goto fail; @@ -366,6 +409,10 @@ int av_frame_ref(AVFrame *dst, const AVFrame *src) } } + /* copy subtitle areas */ + if (src->type == AVMEDIA_TYPE_SUBTITLE) + frame_copy_subtitles(dst, src, 0); + if (src->extended_buf) { dst->extended_buf = av_mallocz_array(sizeof(*dst->extended_buf), src->nb_extended_buf); @@ -455,6 +502,21 @@ void av_frame_unref(AVFrame *frame) av_buffer_unref(&frame->opaque_ref); av_buffer_unref(&frame->private_ref); + for (i = 0; i < frame->num_subtitle_areas; i++) { + AVSubtitleArea *area = frame->subtitle_areas[i]; + + for (i = 0; i < FF_ARRAY_ELEMS(area->buf); i++) { + av_buffer_unref(&area->buf[i]); + area->data[i] = NULL; + } + + av_freep(&area->text); + av_freep(&area->ass); + av_free(area); + } + + av_freep(&frame->subtitle_areas); + get_frame_defaults(frame); } @@ -472,6 +534,7 @@ void av_frame_move_ref(AVFrame *dst, AVFrame *src) int av_frame_is_writable(AVFrame *frame) { + AVSubtitleArea *area; int i, ret = 1; /* assume non-refcounted frames are not writable */ @@ -484,6 +547,15 @@ int av_frame_is_writable(AVFrame *frame) for (i = 0; i < frame->nb_extended_buf; i++) ret &= !!av_buffer_is_writable(frame->extended_buf[i]); + for (i = 0; i < frame->num_subtitle_areas; i++) { + area = frame->subtitle_areas[i]; + if (area) { + for (i = 0; i < FF_ARRAY_ELEMS(area->buf); i++) + if (area->buf[i]) + ret &= !!av_buffer_is_writable(area->buf[i]); + } + } + return ret; } @@ -499,6 +571,7 @@ int av_frame_make_writable(AVFrame *frame) return 0; memset(&tmp, 0, sizeof(tmp)); + tmp.type = frame->type; tmp.format = frame->format; tmp.width = frame->width; tmp.height = frame->height; @@ -509,7 +582,7 @@ int av_frame_make_writable(AVFrame *frame) if (frame->hw_frames_ctx) ret = av_hwframe_get_buffer(frame->hw_frames_ctx, &tmp, 0); else - ret = av_frame_get_buffer(&tmp, 0); + ret = av_frame_get_buffer2(&tmp, 0); if (ret < 0) return ret; @@ -544,14 +617,22 @@ AVBufferRef *av_frame_get_plane_buffer(AVFrame *frame, int plane) uint8_t *data; int planes, i; - if (frame->nb_samples) { - int channels = frame->channels; - if (!channels) - return NULL; - CHECK_CHANNELS_CONSISTENCY(frame); - planes = av_sample_fmt_is_planar(frame->format) ? channels : 1; - } else + switch(frame->type) { + case AVMEDIA_TYPE_VIDEO: planes = 4; + break; + case AVMEDIA_TYPE_AUDIO: + { + int channels = frame->channels; + if (!channels) + return NULL; + CHECK_CHANNELS_CONSISTENCY(frame); + planes = av_sample_fmt_is_planar(frame->format) ? channels : 1; + break; + } + default: + return NULL; + } if (plane < 0 || plane >= planes || !frame->extended_data[plane]) return NULL; @@ -675,17 +756,39 @@ static int frame_copy_audio(AVFrame *dst, const AVFrame *src) return 0; } +static int frame_copy_subtitles(AVFrame *dst, const AVFrame *src, int copy_data) +{ + dst->format = src->format; + + dst->num_subtitle_areas = src->num_subtitle_areas; + + if (src->num_subtitle_areas) { + dst->subtitle_areas = av_malloc_array(src->num_subtitle_areas, sizeof(AVSubtitleArea *)); + + for (unsigned i = 0; i < src->num_subtitle_areas; i++) { + dst->subtitle_areas[i] = av_mallocz(sizeof(AVSubtitleArea)); + av_subtitle_area2area(dst->subtitle_areas[i], src->subtitle_areas[i], copy_data); + } + } + + return 0; +} + int av_frame_copy(AVFrame *dst, const AVFrame *src) { if (dst->format != src->format || dst->format < 0) return AVERROR(EINVAL); - if (dst->width > 0 && dst->height > 0) + switch(dst->type) { + case AVMEDIA_TYPE_VIDEO: return frame_copy_video(dst, src); - else if (dst->nb_samples > 0 && dst->channels > 0) + case AVMEDIA_TYPE_AUDIO: return frame_copy_audio(dst, src); - - return AVERROR(EINVAL); + case AVMEDIA_TYPE_SUBTITLE: + return frame_copy_subtitles(dst, src, 1); + default: + return AVERROR(EINVAL); + } } void av_frame_remove_side_data(AVFrame *frame, enum AVFrameSideDataType type) @@ -831,3 +934,48 @@ int av_frame_apply_cropping(AVFrame *frame, int flags) return 0; } + +/** + * Copies subtitle data from AVSubtitle (deprecated) to AVFrame + * + * @note This is a compatibility method for conversion to the legacy API + */ +void av_frame_put_subtitle(AVFrame *frame, AVSubtitle *sub) +{ + frame->subtitle_start_time = sub->start_display_time; + frame->subtitle_end_time = sub->end_display_time; + frame->subtitle_pts = sub->pts; + frame->num_subtitle_areas = sub->num_rects; + + if (sub->num_rects) { + frame->subtitle_areas = av_malloc_array(sub->num_rects, sizeof(AVSubtitleArea *)); + + for (unsigned i = 0; i < sub->num_rects; i++) { + frame->subtitle_areas[i] = av_mallocz(sizeof(AVSubtitleArea)); + av_subtitle_rect2area(frame->subtitle_areas[i], sub->rects[i]); + } + } +} + +/** + * Copies subtitle data from AVFrame to AVSubtitle (deprecated) + * + * @note This is a compatibility method for conversion to the legacy API + */ +void av_frame_get_subtitle(AVSubtitle *sub, AVFrame *frame) +{ + sub->start_display_time = frame->subtitle_start_time; + sub->end_display_time = frame->subtitle_end_time; + sub->pts = frame->subtitle_pts; + sub->num_rects = frame->num_subtitle_areas; + + if (frame->num_subtitle_areas) { + sub->rects = av_malloc_array(frame->num_subtitle_areas, sizeof(AVSubtitle *)); + + for (unsigned i = 0; i < frame->num_subtitle_areas; i++) { + sub->rects[i] = av_mallocz(sizeof(AVSubtitleRect)); + av_subtitle_area2rect(sub->rects[i], frame->subtitle_areas[i]); + } + } +} + diff --git a/libavutil/frame.h b/libavutil/frame.h index ff2540a20f..63e49b8574 100644 --- a/libavutil/frame.h +++ b/libavutil/frame.h @@ -34,6 +34,7 @@ #include "rational.h" #include "samplefmt.h" #include "pixfmt.h" +#include "subfmt.h" #include "version.h" @@ -271,7 +272,7 @@ typedef struct AVRegionOfInterest { } AVRegionOfInterest; /** - * This structure describes decoded (raw) audio or video data. + * This structure describes decoded (raw) audio, video or subtitle data. * * AVFrame must be allocated using av_frame_alloc(). Note that this only * allocates the AVFrame itself, the buffers for the data must be managed @@ -302,6 +303,13 @@ typedef struct AVRegionOfInterest { */ typedef struct AVFrame { #define AV_NUM_DATA_POINTERS 8 + /** + * Media type of the frame (audio, video, subtitles..) + * + * See AVMEDIA_TYPE_xxx + */ + enum AVMediaType type; + /** * pointer to the picture/channel planes. * This might be different from the first allocated byte @@ -371,7 +379,7 @@ typedef struct AVFrame { /** * format of the frame, -1 if unknown or unset * Values correspond to enum AVPixelFormat for video frames, - * enum AVSampleFormat for audio) + * enum AVSampleFormat for audio, AVSubtitleType for subtitles) */ int format; @@ -462,6 +470,39 @@ typedef struct AVFrame { */ uint64_t channel_layout; + /** + * Display start time, relative to packet pts, in ms. + */ + uint32_t subtitle_start_time; + + /** + * Display end time, relative to packet pts, in ms. + */ + uint32_t subtitle_end_time; + + /** + * Number of items in the @ref subtitle_areas array. + */ + unsigned num_subtitle_areas; + + /** + * Array of subtitle areas, may be empty. + */ + AVSubtitleArea **subtitle_areas; + + /** + * Usually the same as packet pts, in AV_TIME_BASE. + * + * @deprecated This is kept for compatibility reasons and corresponds to + * AVSubtitle->pts. Might be removed in the future. + */ + int64_t subtitle_pts; + + /** + * Header containing style information for text subtitles. + */ + char *subtitle_header; + /** * AVBuffer references backing the data for this frame. If all elements of * this array are NULL, then this frame is not reference counted. This array @@ -721,6 +762,8 @@ void av_frame_move_ref(AVFrame *dst, AVFrame *src); /** * Allocate new buffer(s) for audio or video data. * + * Note: For subtitle data, use av_frame_get_buffer2 + * * The following fields must be set on frame before calling this function: * - format (pixel format for video, sample format for audio) * - width and height for video @@ -740,9 +783,39 @@ void av_frame_move_ref(AVFrame *dst, AVFrame *src); * recommended to pass 0 here unless you know what you are doing. * * @return 0 on success, a negative AVERROR on error. + * + * @deprecated Use @ref av_frame_get_buffer2 instead and set @ref AVFrame.type + * before calling. */ +attribute_deprecated int av_frame_get_buffer(AVFrame *frame, int align); +/** + * Allocate new buffer(s) for audio, video or subtitle data. + * + * The following fields must be set on frame before calling this function: + * - format (pixel format for video, sample format for audio) + * - width and height for video + * - nb_samples and channel_layout for audio + * - type (AVMediaType) + * + * This function will fill AVFrame.data and AVFrame.buf arrays and, if + * necessary, allocate and fill AVFrame.extended_data and AVFrame.extended_buf. + * For planar formats, one buffer will be allocated for each plane. + * + * @warning: if frame already has been allocated, calling this function will + * leak memory. In addition, undefined behavior can occur in certain + * cases. + * + * @param frame frame in which to store the new buffers. + * @param align Required buffer size alignment. If equal to 0, alignment will be + * chosen automatically for the current CPU. It is highly + * recommended to pass 0 here unless you know what you are doing. + * + * @return 0 on success, a negative AVERROR on error. + */ +int av_frame_get_buffer2(AVFrame *frame, int align); + /** * Check if the frame data is writable. * @@ -844,6 +917,22 @@ AVFrameSideData *av_frame_get_side_data(const AVFrame *frame, */ void av_frame_remove_side_data(AVFrame *frame, enum AVFrameSideDataType type); +/** + * Copies subtitle data from AVSubtitle to AVFrame. + * + * @deprecated This is a compatibility method for interoperability with + * the legacy subtitle API. + */ +void av_frame_put_subtitle(AVFrame *frame, AVSubtitle *sub); + +/** + * Copies subtitle data from AVFrame to AVSubtitle. + * + * @deprecated This is a compatibility method for interoperability with + * the legacy subtitle API. + */ +void av_frame_get_subtitle(AVSubtitle *sub, AVFrame *frame); + /** * Flags for frame cropping. diff --git a/libavutil/subfmt.c b/libavutil/subfmt.c new file mode 100644 index 0000000000..83a5e2e11d --- /dev/null +++ b/libavutil/subfmt.c @@ -0,0 +1,221 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include +#include "common.h" +#include "subfmt.h" + +typedef struct SubtitleFmtInfo { + enum AVSubtitleType fmt; + char* name; +} SubtitleFmtInfo; + +static SubtitleFmtInfo sub_fmt_info[AV_SUBTITLE_FMT_NB] = { + {.fmt = AV_SUBTITLE_FMT_UNKNOWN, .name = "Unknown subtitle format", }, + {.fmt = AV_SUBTITLE_FMT_BITMAP, .name = "Graphical subtitles", }, + {.fmt = AV_SUBTITLE_FMT_TEXT, .name = "Text subtitles (plain)", }, + {.fmt = AV_SUBTITLE_FMT_ASS, .name = "Text subtitles (ass)", }, +}; + +const char *av_get_subtitle_fmt_name(enum AVSubtitleType sub_fmt) +{ + if (sub_fmt < 0 || sub_fmt >= AV_SUBTITLE_FMT_NB) + return NULL; + return sub_fmt_info[sub_fmt].name; +} + +enum AVSubtitleType av_get_subtitle_fmt(const char *name) +{ + for (int i = 0; i < AV_SUBTITLE_FMT_NB; i++) + if (!strcmp(sub_fmt_info[i].name, name)) + return i; + return AV_SUBTITLE_FMT_NONE; +} + +int av_subtitle_area2rect(AVSubtitleRect *dst, const AVSubtitleArea *src) +{ + dst->x = src->x; + dst->y = src->y; + dst->w = src->w; + dst->h = src->h; + dst->nb_colors = src->nb_colors; + dst->type = src->type; + + switch (dst->type) { + case AV_SUBTITLE_FMT_BITMAP: + + if (src->h > 0 && src->w > 0 && src->buf[0]) { + uint32_t *pal; + AVBufferRef *buf = src->buf[0]; + dst->data[0] = av_mallocz(buf->size); + memcpy(dst->data[0], buf->data, buf->size); + dst->linesize[0] = src->linesize[0]; + + dst->data[1] = av_mallocz(256 * 4); + pal = (uint32_t *)dst->data[1]; + + for (unsigned i = 0; i < 256; i++) { + pal[i] = src->pal[i]; + } + } + + break; + case AV_SUBTITLE_FMT_TEXT: + + if (src->text) + dst->text = av_strdup(src->text); + + break; + case AV_SUBTITLE_FMT_ASS: + + if (src->ass) + dst->ass = av_strdup(src->ass); + + break; + default: + + av_log(NULL, AV_LOG_ERROR, "Subtitle rect has invalid format: %d", dst->type); + return AVERROR(ENOMEM); + } + + return 0; +} + +int av_subtitle_rect2area(AVSubtitleArea *dst, const AVSubtitleRect *src) +{ + dst->x = src->x; + dst->y = src->y; + dst->w = src->w; + dst->h = src->h; + dst->nb_colors = src->nb_colors; + dst->type = src->type; + + switch (dst->type) { + case AV_SUBTITLE_FMT_BITMAP: + + if (src->h > 0 && src->w > 0 && src->data[0]) { + AVBufferRef *buf = av_buffer_allocz(src->h * src->linesize[0]); + memcpy(buf->data, src->data[0], buf->size); + + dst->data[0] = buf->data; + dst->buf[0] = buf; + dst->linesize[0] = src->linesize[0]; + } + + if (src->data[1]) { + uint32_t *pal = (uint32_t *)src->data[1]; + + for (unsigned i = 0; i < 256; i++) { + dst->pal[i] = pal[i]; + } + } + + break; + case AV_SUBTITLE_FMT_TEXT: + + if (src->text) + dst->text = av_strdup(src->text); + + break; + case AV_SUBTITLE_FMT_ASS: + + if (src->ass) + dst->ass = av_strdup(src->ass); + + break; + default: + + av_log(NULL, AV_LOG_ERROR, "Subtitle area has invalid format: %d", dst->type); + return AVERROR(ENOMEM); + } + + return 0; +} + +int av_subtitle_area2area(AVSubtitleArea *dst, const AVSubtitleArea *src, int copy_data) +{ + dst->x = src->x; + dst->y = src->y; + dst->w = src->w; + dst->h = src->h; + dst->nb_colors = src->nb_colors; + dst->type = src->type; + + switch (dst->type) { + case AV_SUBTITLE_FMT_BITMAP: + + if (src->h > 0 && src->w > 0 && src->buf[0]) { + dst->buf[0] = av_buffer_ref(src->buf[0]); + if (!dst->buf[0]) { + return AVERROR(ENOMEM); + } + + if (copy_data) { + const int ret = av_buffer_make_writable(&dst->buf[0]); + if (ret < 0) + return ret; + } + + dst->data[0] = dst->buf[0]->data; + dst->linesize[0] = src->linesize[0]; + } + + for (unsigned i = 0; i < 256; i++) + dst->pal[i] = src->pal[i]; + + break; + case AV_SUBTITLE_FMT_TEXT: + + if (src->text) + dst->text = av_strdup(src->text); + + break; + case AV_SUBTITLE_FMT_ASS: + + if (src->ass) + dst->ass = av_strdup(src->ass); + + break; + default: + + av_log(NULL, AV_LOG_ERROR, "Subtitle area has invalid format: %d", dst->type); + return AVERROR(ENOMEM); + } + + return 0; +} + +void avsubtitle_free(AVSubtitle *sub) +{ + for (unsigned i = 0; i < sub->num_rects; i++) { + av_freep(&sub->rects[i]->data[0]); + av_freep(&sub->rects[i]->data[1]); + av_freep(&sub->rects[i]->data[2]); + av_freep(&sub->rects[i]->data[3]); + av_freep(&sub->rects[i]->text); + av_freep(&sub->rects[i]->ass); + av_freep(&sub->rects[i]); + } + + av_freep(&sub->rects); + + memset(sub, 0, sizeof(*sub)); +} + diff --git a/libavutil/subfmt.h b/libavutil/subfmt.h new file mode 100644 index 0000000000..c812a53fa6 --- /dev/null +++ b/libavutil/subfmt.h @@ -0,0 +1,186 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVUTIL_SUBFMT_H +#define AVUTIL_SUBFMT_H + +#include + +#include "buffer.h" + +enum AVSubtitleType { + + /** + * Subtitle format unknown. + */ + AV_SUBTITLE_FMT_NONE = -1, + + /** + * Subtitle format unknown. + */ + AV_SUBTITLE_FMT_UNKNOWN = 0, + + /** + * Bitmap area in AVSubtitleRect.data, pixfmt AV_PIX_FMT_PAL8. + */ + AV_SUBTITLE_FMT_BITMAP = 1, + SUBTITLE_BITMAP = 1, ///< Deprecated, use AV_SUBTITLE_FMT_BITMAP instead. + + /** + * Plain text in AVSubtitleRect.text. + */ + AV_SUBTITLE_FMT_TEXT = 2, + SUBTITLE_TEXT = 2, ///< Deprecated, use AV_SUBTITLE_FMT_TEXT instead. + + /** + * Text Formatted as per ASS specification, contained AVSubtitleRect.ass. + */ + AV_SUBTITLE_FMT_ASS = 3, + SUBTITLE_ASS = 3, ///< Deprecated, use AV_SUBTITLE_FMT_ASS instead. + + AV_SUBTITLE_FMT_NB, +}; + +typedef struct AVSubtitleArea { +#define AV_NUM_BUFFER_POINTERS 1 + + enum AVSubtitleType type; + int flags; + + int x; ///< top left corner of area. + int y; ///< top left corner of area. + int w; ///< width of area. + int h; ///< height of area. + int nb_colors; ///< number of colors in bitmap palette (@ref pal). + + /** + * Buffers and line sizes for the bitmap of this subtitle. + * + * @{ + */ + AVBufferRef *buf[AV_NUM_BUFFER_POINTERS]; + uint8_t *data[AV_NUM_BUFFER_POINTERS]; + int linesize[AV_NUM_BUFFER_POINTERS]; + /** + * @} + */ + + uint32_t pal[256]; ///< RGBA palette for the bitmap. + + char *text; ///< 0-terminated plain UTF-8 text + char *ass; ///< 0-terminated ASS/SSA compatible event line. + +} AVSubtitleArea; + +typedef struct AVSubtitleRect { + unsigned nb_refs; + int x; ///< top left corner of pict, undefined when pict is not set + int y; ///< top left corner of pict, undefined when pict is not set + int w; ///< width of pict, undefined when pict is not set + int h; ///< height of pict, undefined when pict is not set + int nb_colors; ///< number of colors in pict, undefined when pict is not set + + /** + * data+linesize for the bitmap of this subtitle. + */ + uint8_t *data[4]; + int linesize[4]; + + enum AVSubtitleType type; + + char *text; ///< 0 terminated plain UTF-8 text + + /** + * 0-terminated ASS/SSA compatible event line. + */ + char *ass; + + int flags; +} AVSubtitleRect; + +typedef struct AVSubtitle { + uint16_t format; /* 0 = graphics */ + uint32_t start_display_time; /* relative to packet pts, in ms */ + uint32_t end_display_time; /* relative to packet pts, in ms */ + unsigned num_rects; + AVSubtitleRect **rects; + int64_t pts; ///< Same as packet pts, in AV_TIME_BASE +} AVSubtitle; + +/** + * Return the name of sub_fmt, or NULL if sub_fmt is not + * recognized. + */ +const char *av_get_subtitle_fmt_name(enum AVSubtitleType sub_fmt); + +/** + * Return a subtitle format corresponding to name, or AV_SUBTITLE_FMT_NONE + * on error. + * + * @param name Subtitle format name. + */ +enum AVSubtitleType av_get_subtitle_fmt(const char *name); + +/** + * Copy a subtitle area. + * + * @param dst The target area. + * @param src The source area. + * @param copy_data Determines whether to copy references or actual + * data from @ref AVSubtitleArea.buf. + * + * @return 0 on success. + */ +int av_subtitle_area2area(AVSubtitleArea *dst, const AVSubtitleArea *src, int copy_data); + +/** + * Copy data from @ref AVSubtitleArea to @ref AVSubtitleRect. + * + * @param dst The target rect (@ref AVSubtitleRect). + * @param src The source area (@ref AVSubtitleArea). + * + * @return 0 on success. + * + * @deprecated This is a compatibility method for interoperability with + * the legacy subtitle API. + */ +int av_subtitle_area2rect(AVSubtitleRect *dst, const AVSubtitleArea *src); + +/** + * Copy data from @ref AVSubtitleRect to @ref AVSubtitleArea. + * + * @param dst The source area (@ref AVSubtitleArea). + * @param src The target rect (@ref AVSubtitleRect). + * + * @return 0 on success. + * + * @deprecated This is a compatibility method for interoperability with + * the legacy subtitle API. + */ +int av_subtitle_rect2area(AVSubtitleArea *dst, const AVSubtitleRect *src); + +/** + * Free all allocated data in the given subtitle struct. + * + * @param sub AVSubtitle to free. + */ +void avsubtitle_free(AVSubtitle *sub); + +#endif /* AVUTIL_SUBFMT_H */ diff --git a/libavutil/version.h b/libavutil/version.h index f220e192c8..2b5ac540a4 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,7 +79,7 @@ */ #define LIBAVUTIL_VERSION_MAJOR 57 -#define LIBAVUTIL_VERSION_MINOR 5 +#define LIBAVUTIL_VERSION_MINOR 6 #define LIBAVUTIL_VERSION_MICRO 100 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \ From patchwork Thu Sep 16 20:30:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 30282 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp1582010iov; Thu, 16 Sep 2021 13:30:41 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyC/x0+S4kB7+bSJqCRLKGMu+SLzmp4An8whT3fB1jlWgqVF75prCbjowS8xdrWVs+oFbG1 X-Received: by 2002:a17:906:7855:: with SMTP id p21mr8311266ejm.330.1631824241292; Thu, 16 Sep 2021 13:30:41 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id z12si4808544edp.308.2021.09.16.13.30.40; Thu, 16 Sep 2021 13:30:41 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=c2WWJ6Ke; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D084668B0AD; Thu, 16 Sep 2021 23:30:21 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11olkn2053.outbound.protection.outlook.com [40.92.18.53]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A5B7768B09D for ; Thu, 16 Sep 2021 23:30:15 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=QoYzX8Vh0MPWIyX3kLxFfGwaYdOR+8mLun156oy1oBfL42esblDk6ECsRIbfjo7ixCZFoqd4YayYEH2teYj+tCQRlMNDmlcD/W0C05XBQfYYG3OW9Samrmnu4DQlrvoU/zwRyiJX34dSTzMUMGEmn9swS9QfM6HuzxdZhbqdLP18s67iNV3V/ySOhWgzJznpbMjCbTtzUec+xmHaTc+IPITgOBWVOwEnJSugke7yL9+W80q3HCSkecU1L2yNns1CiZS0ewxHCIi10nckLyBXhZq4MpC04HQdeGIACB4ZwA8GCY7ynXy57JdKMnu/5p9KNYqeiyqLLBQQoyZbvl1S3Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=Kw0n43sOF/IOlPb4RDjDtEtt845l3X0fM6mQhVt4t90=; b=AFZKjGF4fBd1DTDWnbRqRf8OzPVdy4QKrT+M/RZG27AW1DPbeis90skwo7Rp/Stch0O9B06WKDMKqQ4++Hom6BoT+kTdit/tqbEv6a48jk0tsg9vALFPDTIraoawyfWywMJt657LLpxUmv8yVRGMDAfY/tKEgwuET9UUduSV/cM9HVV8CfXPOVXT17eHgtFhZuFsEgLsaKYX2l2ZjkuK/eaYAWScNiCUYvevJhXUyclXRYJjGr/+VH9r3prG/Xmq9U7FTqdXosoYfCHxMXfKSeqgHEWJGeLCboTSfxuw7GLJdoox/gXmvs4SI1G8p7xra3FRylg1dk6Dt5bIarf6jg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Kw0n43sOF/IOlPb4RDjDtEtt845l3X0fM6mQhVt4t90=; b=c2WWJ6Kewl59vbLzxRPOxijGgfUEWSoJo8Zm7po4tWqAzRdaH/dmlI4PIO37iNYVgy5kq2NghigerhcRz07lcJ+YGGDuO2aUJOwTYeocyskojz93t8H1V/nwQjShwwLuVnUUXsSMjnnY1Ulwul/W+YGHqzV+YsyhvdMetEvMm0qy6oJQMDUuVtec4it9J+XOoPXMf1ad6IsFQgWNLuTX7mvEZ+BHZgMINi4EmQah0XS9Dwvls60rtMeb6StIkKXOHmXevTiQkcRTDBZT489Ey33gsfwf33QqQ7I5gG6I5SnMV2l3aEbBRKLjQh6z/JEyjQPuabMdNtjq2BL04o6rJg== Received: from MN2PR04MB5981.namprd04.prod.outlook.com (2603:10b6:208:da::10) by MN2PR04MB6240.namprd04.prod.outlook.com (2603:10b6:208:e2::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Thu, 16 Sep 2021 20:30:08 +0000 Received: from MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb]) by MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb%5]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021 20:30:08 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v6 02/11] fftools/play, probe: Adjust for subtitle changes Thread-Index: AQHXqzmkycYKIbA690OenS64Q7UuYQ== Date: Thu, 16 Sep 2021 20:30:08 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [/EQnC5+jZPQMavJapyZtFUsjW7IHhoy9] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: dc05a120-5022-4943-8b33-08d97950c6aa x-ms-traffictypediagnostic: MN2PR04MB6240: x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: OWzcgr95+95xCzRVDqxps6ysUkKciH8deoTDjcUa09mg7ULEjURQUQMiFgGD6hx4chyuAEgldJ5/AebekucvMrip96S+454Mg8BwBdl1dgOnNAU8a8cPN2kYakfx0MY8nL+xaIxCazRQLTA6589mYTg00euJLgDjGfz9tmksBYmlGz6Yk5h5ybqz09aGX/ZUq0Xk07sS60JB+9bqCWwZKGF1cdm915+Q6Yk1R/qDZSVZoQEeWJAsVWcWsYUdL1aSIUSTDv+eT2uUbeyU81OIKvJege+8LhA1vseh73g/3FHuNU3Gy49tRlsOyTPATkTa39bA6vCIZj7Wb3YefbZZehZ7sKpi7pCbA2g7qEyG4gxP7VtXxcFsaahzfiLL3nGXQVQ7hCs3Y5YO01wax2Q3a0s2pBKr8nFDtc3vgDhQcFaOwrFX3PH6QS1ZNmplYfQy x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: oxC/ADiL7IZzGFwqf0P6aUhbYT1KHq0AbjIqA9vVCNlykfYKxzVhdO7E4rdhyHUjEduy/CEmWk20W21fuJ5cQvsZD1RBAzP8xqIiErkpmcFIufKjkS/bbhzNLGQttzTZEfK7w4f+SkaEX94FG165Lw== x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-3174-20-msonline-outlook-529c7.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR04MB5981.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: dc05a120-5022-4943-8b33-08d97950c6aa X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Sep 2021 20:30:08.6726 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR04MB6240 Subject: [FFmpeg-devel] [PATCH v6 02/11] fftools/play, probe: Adjust for subtitle changes X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: zq6g4AlB6eVE Signed-off-by: softworkz --- fftools/ffplay.c | 50 +++++++++++++++++++++++------------------------ fftools/ffprobe.c | 49 ++++++++++++++++++++++++++++++++++------------ 2 files changed, 62 insertions(+), 37 deletions(-) diff --git a/fftools/ffplay.c b/fftools/ffplay.c index ccea0e4578..08e5d53f9c 100644 --- a/fftools/ffplay.c +++ b/fftools/ffplay.c @@ -152,7 +152,6 @@ typedef struct Clock { /* Common struct for handling all types of decoded data and allocated render buffers. */ typedef struct Frame { AVFrame *frame; - AVSubtitle sub; int serial; double pts; /* presentation timestamp for the frame */ double duration; /* estimated duration of the frame */ @@ -586,7 +585,7 @@ static int decoder_init(Decoder *d, AVCodecContext *avctx, PacketQueue *queue, S return 0; } -static int decoder_decode_frame(Decoder *d, AVFrame *frame, AVSubtitle *sub) { +static int decoder_decode_frame(Decoder *d, AVFrame *frame) { int ret = AVERROR(EAGAIN); for (;;) { @@ -654,7 +653,7 @@ static int decoder_decode_frame(Decoder *d, AVFrame *frame, AVSubtitle *sub) { if (d->avctx->codec_type == AVMEDIA_TYPE_SUBTITLE) { int got_frame = 0; - ret = avcodec_decode_subtitle2(d->avctx, sub, &got_frame, d->pkt); + ret = avcodec_decode_subtitle3(d->avctx, frame, &got_frame, d->pkt); if (ret < 0) { ret = AVERROR(EAGAIN); } else { @@ -683,7 +682,6 @@ static void decoder_destroy(Decoder *d) { static void frame_queue_unref_item(Frame *vp) { av_frame_unref(vp->frame); - avsubtitle_free(&vp->sub); } static int frame_queue_init(FrameQueue *f, PacketQueue *pktq, int max_size, int keep_last) @@ -981,7 +979,7 @@ static void video_image_display(VideoState *is) if (frame_queue_nb_remaining(&is->subpq) > 0) { sp = frame_queue_peek(&is->subpq); - if (vp->pts >= sp->pts + ((float) sp->sub.start_display_time / 1000)) { + if (vp->pts >= sp->pts + ((float) sp->frame->subtitle_start_time / 1000)) { if (!sp->uploaded) { uint8_t* pixels[4]; int pitch[4]; @@ -993,8 +991,8 @@ static void video_image_display(VideoState *is) if (realloc_texture(&is->sub_texture, SDL_PIXELFORMAT_ARGB8888, sp->width, sp->height, SDL_BLENDMODE_BLEND, 1) < 0) return; - for (i = 0; i < sp->sub.num_rects; i++) { - AVSubtitleRect *sub_rect = sp->sub.rects[i]; + for (i = 0; i < sp->frame->num_subtitle_areas; i++) { + AVSubtitleArea *sub_rect = sp->frame->subtitle_areas[i]; sub_rect->x = av_clip(sub_rect->x, 0, sp->width ); sub_rect->y = av_clip(sub_rect->y, 0, sp->height); @@ -1041,13 +1039,15 @@ static void video_image_display(VideoState *is) int i; double xratio = (double)rect.w / (double)sp->width; double yratio = (double)rect.h / (double)sp->height; - for (i = 0; i < sp->sub.num_rects; i++) { - SDL_Rect *sub_rect = (SDL_Rect*)sp->sub.rects[i]; - SDL_Rect target = {.x = rect.x + sub_rect->x * xratio, - .y = rect.y + sub_rect->y * yratio, - .w = sub_rect->w * xratio, - .h = sub_rect->h * yratio}; - SDL_RenderCopy(renderer, is->sub_texture, sub_rect, &target); + for (i = 0; i < sp->frame->num_subtitle_areas; i++) { + AVSubtitleArea *area = sp->frame->subtitle_areas[i]; + SDL_Rect sub_rect = { .x = area->x, .y = area->y, + .w = area->w, .h = area->h}; + SDL_Rect target = {.x = rect.x + sub_rect.x * xratio, + .y = rect.y + sub_rect.y * yratio, + .w = sub_rect.w * xratio, + .h = sub_rect.h * yratio}; + SDL_RenderCopy(renderer, is->sub_texture, &sub_rect, &target); } #endif } @@ -1651,13 +1651,13 @@ retry: sp2 = NULL; if (sp->serial != is->subtitleq.serial - || (is->vidclk.pts > (sp->pts + ((float) sp->sub.end_display_time / 1000))) - || (sp2 && is->vidclk.pts > (sp2->pts + ((float) sp2->sub.start_display_time / 1000)))) + || (is->vidclk.pts > (sp->pts + ((float) sp->frame->subtitle_end_time / 1000))) + || (sp2 && is->vidclk.pts > (sp2->pts + ((float) sp2->frame->subtitle_start_time / 1000)))) { if (sp->uploaded) { int i; - for (i = 0; i < sp->sub.num_rects; i++) { - AVSubtitleRect *sub_rect = sp->sub.rects[i]; + for (i = 0; i < sp->frame->num_subtitle_areas; i++) { + AVSubtitleRect *sub_rect = sp->frame->subtitle_areas[i]; uint8_t *pixels; int pitch, j; @@ -1774,7 +1774,7 @@ static int get_video_frame(VideoState *is, AVFrame *frame) { int got_picture; - if ((got_picture = decoder_decode_frame(&is->viddec, frame, NULL)) < 0) + if ((got_picture = decoder_decode_frame(&is->viddec, frame)) < 0) return -1; if (got_picture) { @@ -2048,7 +2048,7 @@ static int audio_thread(void *arg) return AVERROR(ENOMEM); do { - if ((got_frame = decoder_decode_frame(&is->auddec, frame, NULL)) < 0) + if ((got_frame = decoder_decode_frame(&is->auddec, frame)) < 0) goto the_end; if (got_frame) { @@ -2246,14 +2246,14 @@ static int subtitle_thread(void *arg) if (!(sp = frame_queue_peek_writable(&is->subpq))) return 0; - if ((got_subtitle = decoder_decode_frame(&is->subdec, NULL, &sp->sub)) < 0) + if ((got_subtitle = decoder_decode_frame(&is->subdec, sp->frame)) < 0) break; pts = 0; - if (got_subtitle && sp->sub.format == 0) { - if (sp->sub.pts != AV_NOPTS_VALUE) - pts = sp->sub.pts / (double)AV_TIME_BASE; + if (got_subtitle && sp->frame->format == AV_SUBTITLE_FMT_BITMAP) { + if (sp->frame->subtitle_pts != AV_NOPTS_VALUE) + pts = sp->frame->subtitle_pts / (double)AV_TIME_BASE; sp->pts = pts; sp->serial = is->subdec.pkt_serial; sp->width = is->subdec.avctx->width; @@ -2263,7 +2263,7 @@ static int subtitle_thread(void *arg) /* now we can update the picture count */ frame_queue_push(&is->subpq); } else if (got_subtitle) { - avsubtitle_free(&sp->sub); + av_frame_free(&sp->frame); } } return 0; diff --git a/fftools/ffprobe.c b/fftools/ffprobe.c index d8e968321e..880f05a6c2 100644 --- a/fftools/ffprobe.c +++ b/fftools/ffprobe.c @@ -2208,22 +2208,43 @@ static void show_packet(WriterContext *w, InputFile *ifile, AVPacket *pkt, int p fflush(stdout); } -static void show_subtitle(WriterContext *w, AVSubtitle *sub, AVStream *stream, +static void show_subtitle(WriterContext *w, AVFrame *sub, AVStream *stream, AVFormatContext *fmt_ctx) { AVBPrint pbuf; + const char *s; av_bprint_init(&pbuf, 1, AV_BPRINT_SIZE_UNLIMITED); writer_print_section_header(w, SECTION_ID_SUBTITLE); print_str ("media_type", "subtitle"); - print_ts ("pts", sub->pts); - print_time("pts_time", sub->pts, &AV_TIME_BASE_Q); - print_int ("format", sub->format); - print_int ("start_display_time", sub->start_display_time); - print_int ("end_display_time", sub->end_display_time); - print_int ("num_rects", sub->num_rects); + print_ts ("pts", sub->subtitle_pts); + print_time("pts_time", sub->subtitle_pts, &AV_TIME_BASE_Q); + + // Remain compatible with previous outputs + switch (sub->format) { + case AV_SUBTITLE_FMT_BITMAP: + print_int ("format", 0); + break; + case AV_SUBTITLE_FMT_TEXT: + print_int ("format", 1); + break; + case AV_SUBTITLE_FMT_ASS: + print_int ("format", 1); + break; + default: + print_int ("format", -1); + break; + } + + s = av_get_subtitle_fmt_name(sub->format); + if (s) print_str ("format_str", s); + else print_str_opt("format_str", "unknown"); + + print_int ("start_display_time", sub->subtitle_start_time); + print_int ("end_display_time", sub->subtitle_end_time); + print_int ("num_subtitle_rects", sub->num_subtitle_areas); writer_print_section_footer(w); @@ -2388,7 +2409,7 @@ static av_always_inline int process_frame(WriterContext *w, AVFormatContext *fmt_ctx = ifile->fmt_ctx; AVCodecContext *dec_ctx = ifile->streams[pkt->stream_index].dec_ctx; AVCodecParameters *par = ifile->streams[pkt->stream_index].st->codecpar; - AVSubtitle sub; + AVFrame *sub = NULL; int ret = 0, got_frame = 0; clear_log(1); @@ -2416,8 +2437,12 @@ static av_always_inline int process_frame(WriterContext *w, break; case AVMEDIA_TYPE_SUBTITLE: - if (*packet_new) - ret = avcodec_decode_subtitle2(dec_ctx, &sub, &got_frame, pkt); + if (*packet_new) { + sub = av_frame_alloc(); + if (!sub) + return AVERROR(ENOMEM); + ret = avcodec_decode_subtitle3(dec_ctx, sub, &got_frame, pkt); + } *packet_new = 0; break; default: @@ -2434,11 +2459,11 @@ static av_always_inline int process_frame(WriterContext *w, nb_streams_frames[pkt->stream_index]++; if (do_show_frames) if (is_sub) - show_subtitle(w, &sub, ifile->streams[pkt->stream_index].st, fmt_ctx); + show_subtitle(w, sub, ifile->streams[pkt->stream_index].st, fmt_ctx); else show_frame(w, frame, ifile->streams[pkt->stream_index].st, fmt_ctx); if (is_sub) - avsubtitle_free(&sub); + av_frame_free(&sub); } return got_frame || *packet_new; } From patchwork Thu Sep 16 20:30:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 30289 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp1582154iov; Thu, 16 Sep 2021 13:30:51 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwSzv9So0YwHWkaQeJ7HkRHkBwVbmiOuaKObk38htBqmvMcRtQ4cjNUS/7Qo3c7vILxcuJJ X-Received: by 2002:a17:906:7d42:: with SMTP id l2mr8335855ejp.467.1631824251694; Thu, 16 Sep 2021 13:30:51 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id p12si5939846ejy.328.2021.09.16.13.30.51; Thu, 16 Sep 2021 13:30:51 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=gqz1jbqe; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 073E768B0B9; Thu, 16 Sep 2021 23:30:28 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11olkn2053.outbound.protection.outlook.com [40.92.18.53]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 279BA68B0A8 for ; Thu, 16 Sep 2021 23:30:21 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=oEjhTUhNqMlEP2RORSo5fYGqjOXrY8tZfT4OCrRKUlqa1j4VJ4c0f+JeLaUgiJWSY4C2z0IIRoo9Y+OQXWY12bS932xd/ybBdIqvq7JD1wb3mlLRO5hSSGjPXOQERSqzD+godzqEN1OLYoEnZLQeL08dbE62jVH3KvswXAqi0R9A2oBgZsPFwIJ/VA0KF8f0gpVXF60G6LP4DYaRhnXEHNodbvB746a9fTFr8Fw0+CFM2MVwOMLRFlXSc25MoUD/2UODgjhYCPfk4kOSQoQ9QrcyZl0Drkq7W2+SZJakexND8HtOp/hnflbJIvzJFc2WPQAGyAgGkFKHWVkkFZx9xg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=orpW6RR7e7LBS6mNMdyEtL+51fnO2nPytJGzWgJ/9MQ=; b=HYvTYh5Nr49v6FsxTa+fd7YUHSFjZlGCdDTs6IrDFRfkcxHfKeIahFzhdD6SMyZaGpD2/gnC3qe1vFFKhQut766Ysc+6E2ZrYYwOi5pkn5d3gfwGRXgn9L65adGsqFkOhTf6kz59w2XF0fMhGdsvb9efZEnLLRbY/sM0W8cJVY4Ezcxk7kDDVD4Mq6RYPg/ivdJfhWJlEWoMwf7KkUsvchC7E+lSIG/QGm7NxBc5IUzXpwtshZnjE7DBn6/u9z1wTxQWI1nSF5CIcTi1naiJGiFRNQLrG3oTnib0eKjUYfk7nFKcOachGUI85ITdoBM17sr+igIkCRhlrXK/MzO7Vw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=orpW6RR7e7LBS6mNMdyEtL+51fnO2nPytJGzWgJ/9MQ=; b=gqz1jbqeh3Igjh3ghLMwU1B2duQmPWR+K2AVe9XsDeysCnCk+g90VPBQkCdGL8FF4E2O+vgpBJX+Wv/xzot0ncZ/2IpQXRYx5szdiWA4NMpdfuqIt4b8pf7E2+IXstbarVv2rGIEccWdlnjujVJ8msv2cm3iwszVkmPACy0quQuBM1i9BCLwIggrQwUExqRsjeZLvuD0hUMB48aeEn3rBEWijmR7RyYe71L+ZwKmYFayk7+hqA8Gl6pcP5X3JisY+G49WCG0597ALSfk8yJZ01SZ6DdpXkcpoBQUJl2UE0/rWLPWSi5pw45e0+rVjQENx9oa18dhPvgYsyYAVJWM/w== Received: from MN2PR04MB5981.namprd04.prod.outlook.com (2603:10b6:208:da::10) by MN2PR04MB6240.namprd04.prod.outlook.com (2603:10b6:208:e2::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Thu, 16 Sep 2021 20:30:11 +0000 Received: from MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb]) by MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb%5]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021 20:30:11 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v6 03/11] avfilter/subtitles: Add subtitles.c for subtitle frame allocation Thread-Index: AQHXqzmlEhCzS3NqtE2Gwq0b+CBQwQ== Date: Thu, 16 Sep 2021 20:30:11 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [oJfcQuOubAy8hnSO1PfHMGaqX4Q5lUIk] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: a4bacca8-bcba-4793-b273-08d97950c867 x-ms-traffictypediagnostic: MN2PR04MB6240: x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 5Xq+Vb7CWfhGsAVcshxSOdJFCkbMy5xnxgwNlUVSNYyek5KI6h56hQk6MkCuonQk5oMtyp1ZeMPY7+tNodzgBaN8CDM4fgERzL9/NUCZA0U2OT0Pft4+m64yGMdMEFOsTlpZ7T/GfCea788FRLi6BcCmVYJdLa9sJq7sA5kZpz/3pmzqO38AgRNkZhiIN/7pMqs6nGMbvkJGSLtX8G6e32GuU7OdGzzWs6Dg4qDRmyCycAabd0GclJgnga7XqgvWoRPyIhtN1DDu52/NkzM1BAJBK6n2Zq0HLf3JI2JRy9/bqgBb16o/mUaz5wCeOZ4WG7lrXuzLU1EWFa1p8DPKLw/OU8NVaPU37xVeVLwywOZc7n5BfvnsoIxf4awMRgyvNwokgOH8O/z71t8NOXTuzp3JUPi9hE/1wbntqJaTfUOIRck3RYjntBiPz32V5y7p x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: v44u9a+Oqry8ICMCUJfpFsBstDq7dSXuTfh+zhFFkhO37lfI1k0ISTfMsmA2bBqr6sj3SlBSlTzb0L0DjOsu97CprmSj4tWh5IhyGwG/7HQmwVGLMcPnXkQAlJUhTlXdrcbtencJWvMTx02zzT1XZg== x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-3174-20-msonline-outlook-529c7.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR04MB5981.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: a4bacca8-bcba-4793-b273-08d97950c867 X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Sep 2021 20:30:11.5270 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR04MB6240 Subject: [FFmpeg-devel] [PATCH v6 03/11] avfilter/subtitles: Add subtitles.c for subtitle frame allocation X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: rxwN/bB2yag5 Analog to avfilter/video.c and avfilter/audio.c Signed-off-by: softworkz --- libavfilter/Makefile | 1 + libavfilter/avfilter.c | 4 +++ libavfilter/internal.h | 1 + libavfilter/subtitles.c | 61 +++++++++++++++++++++++++++++++++++++++++ libavfilter/subtitles.h | 44 +++++++++++++++++++++++++++++ 5 files changed, 111 insertions(+) create mode 100644 libavfilter/subtitles.c create mode 100644 libavfilter/subtitles.h diff --git a/libavfilter/Makefile b/libavfilter/Makefile index f059f3fef8..041d3c5382 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -19,6 +19,7 @@ OBJS = allfilters.o \ framequeue.o \ graphdump.o \ graphparser.o \ + subtitles.o \ video.o \ OBJS-$(HAVE_THREADS) += pthread.o diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c index c614eb0740..ac46bde782 100644 --- a/libavfilter/avfilter.c +++ b/libavfilter/avfilter.c @@ -43,6 +43,7 @@ #include "formats.h" #include "framepool.h" #include "internal.h" +#include "subtitles.h" #include "libavutil/ffversion.h" const char av_filter_ffversion[] = "FFmpeg version " FFMPEG_VERSION; @@ -1474,6 +1475,9 @@ int ff_inlink_make_frame_writable(AVFilterLink *link, AVFrame **rframe) case AVMEDIA_TYPE_AUDIO: out = ff_get_audio_buffer(link, frame->nb_samples); break; + case AVMEDIA_TYPE_SUBTITLE: + out = ff_get_subtitles_buffer(link, link->format); + break; default: return AVERROR(EINVAL); } diff --git a/libavfilter/internal.h b/libavfilter/internal.h index e7c154aff0..8977dda2b3 100644 --- a/libavfilter/internal.h +++ b/libavfilter/internal.h @@ -90,6 +90,7 @@ struct AVFilterPad { union { AVFrame *(*video)(AVFilterLink *link, int w, int h); AVFrame *(*audio)(AVFilterLink *link, int nb_samples); + AVFrame *(*subtitle)(AVFilterLink *link, int format); } get_buffer; /** diff --git a/libavfilter/subtitles.c b/libavfilter/subtitles.c new file mode 100644 index 0000000000..fd2e7c3c0f --- /dev/null +++ b/libavfilter/subtitles.c @@ -0,0 +1,61 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/common.h" + +#include "subtitles.h" +#include "avfilter.h" +#include "internal.h" + + +AVFrame *ff_null_get_subtitles_buffer(AVFilterLink *link, int format) +{ + return ff_get_subtitles_buffer(link->dst->outputs[0], format); +} + +AVFrame *ff_default_get_subtitles_buffer(AVFilterLink *link, int format) +{ + AVFrame *frame; + + frame = av_frame_alloc(); + if (!frame) + return NULL; + + frame->format = format; + frame->type = AVMEDIA_TYPE_SUBTITLE; + + if (av_frame_get_buffer2(frame, 0) < 0) + return NULL; + + return frame; +} + +AVFrame *ff_get_subtitles_buffer(AVFilterLink *link, int format) +{ + AVFrame *ret = NULL; + + if (link->dstpad->get_buffer.subtitle) + ret = link->dstpad->get_buffer.subtitle(link, format); + + if (!ret) + ret = ff_default_get_subtitles_buffer(link, format); + + return ret; +} diff --git a/libavfilter/subtitles.h b/libavfilter/subtitles.h new file mode 100644 index 0000000000..d3d5491652 --- /dev/null +++ b/libavfilter/subtitles.h @@ -0,0 +1,44 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVFILTER_SUBTITLES_H +#define AVFILTER_SUBTITLES_H + +#include "avfilter.h" +#include "internal.h" + +/** default handler for get_subtitles_buffer() for subtitle inputs */ +AVFrame *ff_default_get_subtitles_buffer(AVFilterLink *link, int format); + +/** get_subtitles_buffer() handler for filters which simply pass subtitles along */ +AVFrame *ff_null_get_subtitles_buffer(AVFilterLink *link, int format); + +/** + * Request a subtitles buffer with a specific set of permissions. + * + * @param link the output link to the filter from which the buffer will + * be requested + * @param format The subtitles format. + * @return A reference to the subtitles. This must be unreferenced with + * avfilter_unref_buffer when you are finished with it. +*/ +AVFrame *ff_get_subtitles_buffer(AVFilterLink *link, int format); + +#endif /* AVFILTER_SUBTITLES_H */ From patchwork Thu Sep 16 20:30:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 30281 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp1582262iov; Thu, 16 Sep 2021 13:31:01 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw0TdKVXxRneQsGkhBUKol5aYuS5/6WkeMt9rgyL6fpalz1iN4pD1plMQd925BmMpdc+CEY X-Received: by 2002:a50:becb:: with SMTP id e11mr7754744edk.161.1631824261646; Thu, 16 Sep 2021 13:31:01 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id m4si4855244edv.496.2021.09.16.13.31.01; Thu, 16 Sep 2021 13:31:01 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=aiaQG3MB; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 235EB68B0C1; Thu, 16 Sep 2021 23:30:29 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11olkn2053.outbound.protection.outlook.com [40.92.18.53]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B0C4668B0A8 for ; Thu, 16 Sep 2021 23:30:26 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JNMtLV/VsLB5WzW1wyyvUVBfom2TV4HjseuN1Ug3j7+SncsN/WSe9TNJtoERePCQYmzO5Nmf20+kGMRoRrocf0dsSZmDM6/AgCtQ00dvFpF9VWSWqWdRq2WD9SE072TYWnrDPm5ojNlkA6ybZ/juSqVn/EdMJgPWArr+gUrP5nOOl4RQToI6r+ElY9SBlQPabCqMPw1ASDLfL3UmQ5iiE3J4wg7y4GZCrgNi4DLVAjRYRBlGN9yIiWlJZe8bFzpNNqkpA/OnBW989qnokNYiSzirW3n+IWwLjpYT2WbTx+2IVCWGqvuR/xNxc3Af54yWLhU85W7z0INT2U/xBT7JAw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=Xr8d11wZykDqFMTRb+5mfEdthuwTJ2wtrxVs94vWML4=; b=kkDqDxtElkOZVrx9xJuW+rJ3qJzyw8vf0Q9gbCCYongSNkbJOevf8RtGBsJiLZqjxqNgzr7RhNYvhGs2+aFf6xF52S9yA+lsFynxXktKgGQMhIPkBUNRTKH5VMoe8BQqrnxgeAtDnlvE4JWB/aUjHP5IlLMsIPB+Qi7fPY2bxn+HN5/bS7hXM0aBVc6W4mzv0cSzYPTYcKsG99woR5cCYhWTu5pcjzpvS4VztMuxSnLT9F8ZhNRR4BLKaEC/GaSxY9iNUi/GAcdIlu1ObPxnJIDfyBi7+CudSj/tEU/3Fly5kL4zk0P+9L5o/ciUXe5yEoy7YxzvRXyhWgX/xr48gQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Xr8d11wZykDqFMTRb+5mfEdthuwTJ2wtrxVs94vWML4=; b=aiaQG3MB3dtfwtU6IF7FMEEZIGus3n/eDClpWaGyl+R3tGWQuqywJ6NtIsZ1RwKArU3sQSGqQZTWiegpJDpfKSRrG/uU7cXknN4h4ZJnyVZ5Zi050IDV47tgybFJeNnWS3bJz0O41OFamhxttb29aXnp7/R1Y1xQHG+HLkGYkKLjKr2fGPWFL8wfFzaSLWG4EK7TngKJwnUliTV/4GXSqGmspx4J4mparP0QLV7e6Hx8hzQxpOLrxFJsuqF7E5GrPCiNeKYWiH3TPi9boHdxREh2fSrHDTlBqvAjPSMuqJpLjsu9c03A3qRZS+LM3Ktwsovx3imkVTeLSPTueTEKEw== Received: from MN2PR04MB5981.namprd04.prod.outlook.com (2603:10b6:208:da::10) by MN2PR04MB6240.namprd04.prod.outlook.com (2603:10b6:208:e2::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Thu, 16 Sep 2021 20:30:15 +0000 Received: from MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb]) by MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb%5]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021 20:30:15 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v6 04/11] avfilter/avfilter: Handle subtitle frames Thread-Index: AQHXqzmnWMGyDyrEr0+jhZqZ5bRitQ== Date: Thu, 16 Sep 2021 20:30:15 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [v7W90IW/WG0i9g0qXVz1mnnfoVqfqx+U] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 3b344511-2b28-4a9d-32f1-08d97950ca80 x-ms-traffictypediagnostic: MN2PR04MB6240: x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: KOMJtyCB59Cs1ydJuNfW58j5mWOxC9kaJZQPmqrtdFwga94SM+F1Z1T/8SCT09iSaaQtSZ8vrIw3ZYi/NEnr2g/bJFvBNyIGQx4OT0kqrklAW7Y0r5VMu5vcpkEk8FYnIMwZAC+3yq4Dj+cCQuak6O6W2PBVQZh4sv0iRV1QlmiiaJXc6m0+GMpkOquieOy0JzFMs1qB1sg6wCjR8flZsThc+hTSVMpJ9VxFpxm7XjVYoB/px20Vb5pcyD5fRxv4Jje++ohxzZ1e6WMVRwTzJNFhPuSWsbgL4iKlVT2r7W9vAAfYAufxxZzpZKhfmpvpM7IOIevYYMnjr8gkwgdJDdXrShTU1sur6v2Ge9RAbLADtyBsvqS/NLWgfg2NKP1Tu9ATUIjEs9SR2wjfHgnZJYs6fGOM0nSxdvKw9XSS2G199TmaL5Tf6fuBbx1R+2gr x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: cD6Fuxbu3iAG2KF/q2ztA8yw8FzApkU3h1CfQpkXOPyQK1T9XILrKi1UycNEgVdfz7eyb0X2J6TQQjH/O/rWIluYvZjrtUfvRKaFqpCvt3awUZEe/WyN5WvADaiKQgvMX4vYLPE85QKHzaWu18Yjbg== x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-3174-20-msonline-outlook-529c7.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR04MB5981.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 3b344511-2b28-4a9d-32f1-08d97950ca80 X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Sep 2021 20:30:15.0500 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR04MB6240 Subject: [FFmpeg-devel] [PATCH v6 04/11] avfilter/avfilter: Handle subtitle frames X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: OaokvuD6wOlN Signed-off-by: softworkz --- libavfilter/avfilter.c | 8 +++++--- libavfilter/avfiltergraph.c | 5 +++++ libavfilter/formats.c | 14 ++++++++++++++ libavfilter/formats.h | 3 +++ 4 files changed, 27 insertions(+), 3 deletions(-) diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c index ac46bde782..3f303fbafa 100644 --- a/libavfilter/avfilter.c +++ b/libavfilter/avfilter.c @@ -57,7 +57,8 @@ void ff_tlog_ref(void *ctx, AVFrame *ref, int end) ref->linesize[0], ref->linesize[1], ref->linesize[2], ref->linesize[3], ref->pts, ref->pkt_pos); - if (ref->width) { + switch(ref->type) { + case AVMEDIA_TYPE_VIDEO: ff_tlog(ctx, " a:%d/%d s:%dx%d i:%c iskey:%d type:%c", ref->sample_aspect_ratio.num, ref->sample_aspect_ratio.den, ref->width, ref->height, @@ -65,12 +66,13 @@ void ff_tlog_ref(void *ctx, AVFrame *ref, int end) ref->top_field_first ? 'T' : 'B', /* Top / Bottom */ ref->key_frame, av_get_picture_type_char(ref->pict_type)); - } - if (ref->nb_samples) { + break; + case AVMEDIA_TYPE_AUDIO: ff_tlog(ctx, " cl:%"PRId64"d n:%d r:%d", ref->channel_layout, ref->nb_samples, ref->sample_rate); + break; } ff_tlog(ctx, "]%s", end ? "\n" : ""); diff --git a/libavfilter/avfiltergraph.c b/libavfilter/avfiltergraph.c index 45b028cd9c..7a5a4ea419 100644 --- a/libavfilter/avfiltergraph.c +++ b/libavfilter/avfiltergraph.c @@ -314,6 +314,8 @@ static int filter_link_check_formats(void *log, AVFilterLink *link, AVFilterForm return ret; break; + case AVMEDIA_TYPE_SUBTITLE: + return 0; default: av_assert0(!"reached"); } @@ -444,6 +446,9 @@ static int query_formats(AVFilterGraph *graph, void *log_ctx) if (!link) continue; + if (link->type == AVMEDIA_TYPE_SUBTITLE) + continue; + neg = ff_filter_get_negotiation(link); av_assert0(neg); for (neg_step = 1; neg_step < neg->nb_mergers; neg_step++) { diff --git a/libavfilter/formats.c b/libavfilter/formats.c index 1bf7d36195..704774d763 100644 --- a/libavfilter/formats.c +++ b/libavfilter/formats.c @@ -19,6 +19,7 @@ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ +#include "libavcodec/avcodec.h" #include "libavutil/avassert.h" #include "libavutil/channel_layout.h" #include "libavutil/common.h" @@ -430,6 +431,12 @@ int ff_add_channel_layout(AVFilterChannelLayouts **l, uint64_t channel_layout) return 0; } +int ff_add_subtitle_type(AVFilterFormats **avff, int64_t fmt) +{ + ADD_FORMAT(avff, fmt, ff_formats_unref, int, formats, nb_formats); + return 0; +} + AVFilterFormats *ff_all_formats(enum AVMediaType type) { AVFilterFormats *ret = NULL; @@ -447,6 +454,13 @@ AVFilterFormats *ff_all_formats(enum AVMediaType type) return NULL; fmt++; } + } else if (type == AVMEDIA_TYPE_SUBTITLE) { + if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_BITMAP) < 0) + return NULL; + if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_ASS) < 0) + return NULL; + if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_TEXT) < 0) + return NULL; } return ret; diff --git a/libavfilter/formats.h b/libavfilter/formats.h index 471cb42bc4..25ae511fc1 100644 --- a/libavfilter/formats.h +++ b/libavfilter/formats.h @@ -180,6 +180,9 @@ int ff_set_common_formats_from_list(AVFilterContext *ctx, const int *fmts); av_warn_unused_result int ff_add_channel_layout(AVFilterChannelLayouts **l, uint64_t channel_layout); +av_warn_unused_result +int ff_add_subtitle_type(AVFilterFormats **avff, int64_t fmt); + /** * Add *ref as a new reference to f. */ From patchwork Thu Sep 16 20:30:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 30280 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp1582391iov; Thu, 16 Sep 2021 13:31:12 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz+JVDS1A8flxV5PKH5msVQpjTVwcyGUp3fLW0hFkwEErYNf9Ymrxq4Qm2FaeI75aRmyBEe X-Received: by 2002:a17:907:2717:: with SMTP id w23mr8197421ejk.283.1631824272238; Thu, 16 Sep 2021 13:31:12 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id jg17si4794554ejc.92.2021.09.16.13.31.11; Thu, 16 Sep 2021 13:31:12 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=CR5Y73TP; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 24EB368B0C6; Thu, 16 Sep 2021 23:30:30 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11olkn2036.outbound.protection.outlook.com [40.92.19.36]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5629768B0AE for ; Thu, 16 Sep 2021 23:30:22 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MtzFTA/JaD+BShaf4H2CTRMaoIwOBAD8qmPHN5Qi/yec9dtXv1CKdlFW2wvm9eBWMLnLK4RLSGKIbIo0C6b8tm6qmEp0MAvLyrAR43IHVPtzJ88bQ6W9hlZy/fWNVCXrWyUB2a/MTcmBTT+ijLpBAhxneNUhRLlwDwyXd3co4faCQ63BItuVB5LRO2g2NIOcHkVmC5wN7ZCqkOrECBVyR6Jy8jDzZy6qrBfa7+fPhGfDDuI6/rkLzuqgBXLA2sU+cAL3RQbOCDSi0vmDSA0rOPMv0nvQKMPUGPv0aKzgNHrvRY/pUTm/D1ucNinywALVMai9BeNunQjGG4On96uv/Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=i72SyqrAAjQHAPCdkW8wTwdhi8QDvY6rwVpQKprHslA=; b=iCUR1cKtErqPo5uMfYLJzymR02BsIrWQ5wnLTTmPo6iogbCWkqYxRQRO420lbfPEmwsTaZ1ZQ/k/hY96uhNSX8mxUiD+uo+XG8hEjyky8yeCcqoq60Fa+mNcaQFk32GDuVVAx+jLbUNWB/SR0qlPTI5OgxYHti9Hk+99aYsTBnoaqRwf5h4pl4xHQnjMNohkBDt27AjIkA8UG56MzvqQKo9tct7sYOMS5tXS8OxOb2HSeseZityrMadbhgvVCQ52azAHlSH3WTb0JFiBOFg9spHO1PH+ttIzjPF+4eWgm7ppdNtOFJx+vpVeuzE1j++7wV32RvZ3tT4Vg1SDIow4vw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=i72SyqrAAjQHAPCdkW8wTwdhi8QDvY6rwVpQKprHslA=; b=CR5Y73TP0S+EajKkT/cRTq0G64l3aDjPqLd7lFeIq3hOLEVlHpVyLCme85v+fsvSSM8IP4klIHm2xdykhR3eS82cZDbhnNOPyDntktRP+Uq/5pxVjnIoNaiEOZH5oZl0Tm40FTgwv2+Eou5AJ1NQ/Yz171FOk4Beq9lAJrU5n/R99WqgdVbeTfyIYIhlK3K2CdeaEk14KAG4jzT9Qgk9UICFN57t/1tIYMiFO+BAuw7Q0rxdLSnH2MkcHgw8RQj++S0ihHBJ0javbENWe23+wAxHs6ICIxJ2C0E/YAnzYKM+jBuhOqRnNP5aSXoS8XAJioGWQ/GW+kRpu4VXlnR7Ug== Received: from MN2PR04MB5981.namprd04.prod.outlook.com (2603:10b6:208:da::10) by MN2PR04MB6048.namprd04.prod.outlook.com (2603:10b6:208:e5::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Thu, 16 Sep 2021 20:30:19 +0000 Received: from MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb]) by MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb%5]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021 20:30:19 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v6 05/11] avfilter/sbuffer: Add sbuffersrv and sbuffersink filters Thread-Index: AQHXqzmqg6+8slrrEECGY/IBtHuOWQ== Date: Thu, 16 Sep 2021 20:30:19 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [ij+sOzcrO0o/RvifoUxDPFVzRPSfPZjV] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: f4eccf7e-496b-4fa3-a6a5-08d97950cd55 x-ms-traffictypediagnostic: MN2PR04MB6048: x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: JYpqYr887IUIjlA6+vg+uhyMQq+r01BHPCfVnRLrshWLkm36Pik2SPsQZ0jccSn7G3ATuYmWc5a033rVbOgwBnXvGeNuunNdQd/XUmdLlz/MP1rI2LXx7/HEiRq+4eSXOcxGj9pkFXWs8XyS+Njj4qvWdDOf/gEgyQYp/VeewtUn/zy46Sspoi/uzebqw6c59El5mKbE3dzGEkyziuUVvrzX7ojBUQxKVzlaFENip9yMulCdyO0HzaMVt0uw0w346U7/mCUraepXYARhIOnKQDVovesA9hP8BGfOCuNyRXzUIV/E0V7dZFUNOYfk98a3EyFVk4Lq3yISS4+j5AaKKEgX6wLykZ0yXOVtJU63Jb2+RYVy7d0+bAoK0Z/WfSyBViFGUiIokiUGOPES7SLPjLhCLKahPYRU1OhtCH43JXdDFHA78I9S0Npz2h8ElwNE x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: Nbv8uKRCZz3XPytfWIS3hCbafQ7lPs5kM8pylTS1+LPkBBNbmMxITm7axu0xLx2X7j6YQGEiohm3AxdJc6RyjGY2bR/lfWMp+p3XCYtlXRSQ3sDuHpncBZz1LYspns/aotbKLatDrTZtvbjw7BnzAw== x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-3174-20-msonline-outlook-529c7.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR04MB5981.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: f4eccf7e-496b-4fa3-a6a5-08d97950cd55 X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Sep 2021 20:30:19.8243 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR04MB6048 Subject: [FFmpeg-devel] [PATCH v6 05/11] avfilter/sbuffer: Add sbuffersrv and sbuffersink filters X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: FdEYybuWdKWH Signed-off-by: softworkz --- configure | 2 +- libavfilter/allfilters.c | 10 +++--- libavfilter/buffersink.c | 63 +++++++++++++++++++++++++++++++++++ libavfilter/buffersink.h | 15 +++++++++ libavfilter/buffersrc.c | 72 ++++++++++++++++++++++++++++++++++++++++ libavfilter/buffersrc.h | 1 + libavfilter/version.h | 2 +- 7 files changed, 159 insertions(+), 6 deletions(-) diff --git a/configure b/configure index 7ac23123c7..74c5a41be1 100755 --- a/configure +++ b/configure @@ -7718,7 +7718,7 @@ print_enabled_components(){ fi done if [ "$name" = "filter_list" ]; then - for c in asrc_abuffer vsrc_buffer asink_abuffer vsink_buffer; do + for c in asrc_abuffer vsrc_buffer ssrc_sbuffer asink_abuffer vsink_buffer ssink_sbuffer; do printf " &ff_%s,\n" $c >> $TMPH done fi diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index ddd6404228..154eba5bb2 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -533,10 +533,12 @@ extern const AVFilter ff_avsrc_movie; * they are formatted to not be found by the grep * as they are manually added again (due to their 'names' * being the same while having different 'types'). */ -extern const AVFilter ff_asrc_abuffer; -extern const AVFilter ff_vsrc_buffer; -extern const AVFilter ff_asink_abuffer; -extern const AVFilter ff_vsink_buffer; +extern const AVFilter ff_asrc_abuffer; +extern const AVFilter ff_vsrc_buffer; +extern const AVFilter ff_ssrc_sbuffer; +extern const AVFilter ff_asink_abuffer; +extern const AVFilter ff_vsink_buffer; +extern const AVFilter ff_ssink_sbuffer; extern const AVFilter ff_af_afifo; extern const AVFilter ff_vf_fifo; diff --git a/libavfilter/buffersink.c b/libavfilter/buffersink.c index 8b46dcb15e..55cbaff3d5 100644 --- a/libavfilter/buffersink.c +++ b/libavfilter/buffersink.c @@ -29,6 +29,8 @@ #include "libavutil/internal.h" #include "libavutil/opt.h" +#include "libavcodec/avcodec.h" + #define FF_INTERNAL_FIELDS 1 #include "framequeue.h" @@ -57,6 +59,10 @@ typedef struct BufferSinkContext { int *sample_rates; ///< list of accepted sample rates, terminated by -1 int sample_rates_size; + /* only used for subtitles */ + enum AVSubtitleType *subtitle_types; ///< list of accepted subtitle types, must be terminated with -1 + int subtitle_types_size; + AVFrame *peeked_frame; } BufferSinkContext; @@ -168,6 +174,15 @@ AVABufferSinkParams *av_abuffersink_params_alloc(void) return NULL; return params; } + +AVSBufferSinkParams *av_sbuffersink_params_alloc(void) +{ + AVSBufferSinkParams *params = av_mallocz(sizeof(AVSBufferSinkParams)); + + if (!params) + return NULL; + return params; +} #endif static av_cold int common_init(AVFilterContext *ctx) @@ -305,6 +320,28 @@ static int asink_query_formats(AVFilterContext *ctx) return 0; } +static int ssink_query_formats(AVFilterContext *ctx) +{ + BufferSinkContext *buf = ctx->priv; + AVFilterFormats *formats = NULL; + unsigned i; + int ret; + + CHECK_LIST_SIZE(subtitle_types) + if (buf->subtitle_types_size) { + for (i = 0; i < NB_ITEMS(buf->subtitle_types); i++) + if ((ret = ff_add_subtitle_type(&formats, buf->subtitle_types[i])) < 0) + return ret; + if ((ret = ff_set_common_formats(ctx, formats)) < 0) + return ret; + } else { + if ((ret = ff_default_query_formats(ctx)) < 0) + return ret; + } + + return 0; +} + #define OFFSET(x) offsetof(BufferSinkContext, x) #define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM static const AVOption buffersink_options[] = { @@ -322,9 +359,16 @@ static const AVOption abuffersink_options[] = { { NULL }, }; #undef FLAGS +#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_SUBTITLE_PARAM +static const AVOption sbuffersink_options[] = { + { "subtitle_types", "set the supported subtitle formats", OFFSET(subtitle_types), AV_OPT_TYPE_BINARY, .flags = FLAGS }, + { NULL }, +}; +#undef FLAGS AVFILTER_DEFINE_CLASS(buffersink); AVFILTER_DEFINE_CLASS(abuffersink); +AVFILTER_DEFINE_CLASS(sbuffersink); static const AVFilterPad avfilter_vsink_buffer_inputs[] = { { @@ -363,3 +407,22 @@ const AVFilter ff_asink_abuffer = { FILTER_INPUTS(avfilter_asink_abuffer_inputs), .outputs = NULL, }; + +static const AVFilterPad avfilter_ssink_sbuffer_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + }, +}; + +AVFilter ff_ssink_sbuffer = { + .name = "sbuffersink", + .description = NULL_IF_CONFIG_SMALL("Buffer subtitle frames, and make them available to the end of the filter graph."), + .priv_class = &sbuffersink_class, + .priv_size = sizeof(BufferSinkContext), + .init = common_init, + .query_formats = ssink_query_formats, + .activate = activate, + FILTER_INPUTS(avfilter_ssink_sbuffer_inputs), + .outputs = NULL, +}; diff --git a/libavfilter/buffersink.h b/libavfilter/buffersink.h index 69ed0f29a8..b439b586c5 100644 --- a/libavfilter/buffersink.h +++ b/libavfilter/buffersink.h @@ -129,6 +129,21 @@ typedef struct AVABufferSinkParams { */ attribute_deprecated AVABufferSinkParams *av_abuffersink_params_alloc(void); + +/** + * Deprecated and unused struct to use for initializing an sbuffersink context. + */ +typedef struct AVSBufferSinkParams { + const int *subtitle_type; +} AVSBufferSinkParams; + +/** + * Create an AVSBufferSinkParams structure. + * + * Must be freed with av_free(). + */ +attribute_deprecated +AVSBufferSinkParams *av_sbuffersink_params_alloc(void); #endif /** diff --git a/libavfilter/buffersrc.c b/libavfilter/buffersrc.c index 632bfc7ad8..902a09a721 100644 --- a/libavfilter/buffersrc.c +++ b/libavfilter/buffersrc.c @@ -39,6 +39,7 @@ #include "formats.h" #include "internal.h" #include "video.h" +#include "libavcodec/avcodec.h" typedef struct BufferSourceContext { const AVClass *class; @@ -63,6 +64,9 @@ typedef struct BufferSourceContext { uint64_t channel_layout; char *channel_layout_str; + /* subtitle only */ + enum AVSubtitleType subtitle_type; + int eof; } BufferSourceContext; @@ -130,6 +134,13 @@ int av_buffersrc_parameters_set(AVFilterContext *ctx, AVBufferSrcParameters *par if (param->channel_layout) s->channel_layout = param->channel_layout; break; + case AVMEDIA_TYPE_SUBTITLE: + s->subtitle_type = param->format; + if (param->width > 0) + s->w = param->width; + if (param->height > 0) + s->h = param->height; + break; default: return AVERROR_BUG; } @@ -197,6 +208,8 @@ int attribute_align_arg av_buffersrc_add_frame_flags(AVFilterContext *ctx, AVFra CHECK_AUDIO_PARAM_CHANGE(ctx, s, frame->sample_rate, frame->channel_layout, frame->channels, frame->format, frame->pts); break; + case AVMEDIA_TYPE_SUBTITLE: + break; default: return AVERROR(EINVAL); } @@ -269,6 +282,7 @@ unsigned av_buffersrc_get_nb_failed_requests(AVFilterContext *buffer_src) #define OFFSET(x) offsetof(BufferSourceContext, x) #define A AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_AUDIO_PARAM #define V AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM +#define S AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_SUBTITLE_PARAM static const AVOption buffer_options[] = { { "width", NULL, OFFSET(w), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, V }, @@ -298,6 +312,16 @@ static const AVOption abuffer_options[] = { AVFILTER_DEFINE_CLASS(abuffer); +static const AVOption sbuffer_options[] = { + { "time_base", NULL, OFFSET(time_base), AV_OPT_TYPE_RATIONAL, { .dbl = 0 }, 0, INT_MAX, S }, + { "subtitle_type", NULL, OFFSET(subtitle_type), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, S }, + { "width", NULL, OFFSET(w), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, V }, + { "height", NULL, OFFSET(h), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, V }, + { NULL }, +}; + +AVFILTER_DEFINE_CLASS(sbuffer); + static av_cold int init_audio(AVFilterContext *ctx) { BufferSourceContext *s = ctx->priv; @@ -347,6 +371,21 @@ static av_cold int init_audio(AVFilterContext *ctx) return ret; } +static av_cold int init_subtitle(AVFilterContext *ctx) +{ + BufferSourceContext *c = ctx->priv; + + if (c->subtitle_type == AV_SUBTITLE_FMT_BITMAP) + av_log(ctx, AV_LOG_VERBOSE, "graphical subtitles - w:%d h:%d tb:%d/%d\n", + c->w, c->h, c->time_base.num, c->time_base.den); + else + av_log(ctx, AV_LOG_VERBOSE, "text subtitles - w:%d h:%d tb:%d/%d\n", + c->w, c->h, c->time_base.num, c->time_base.den); + + return 0; +} + + static av_cold void uninit(AVFilterContext *ctx) { BufferSourceContext *s = ctx->priv; @@ -381,6 +420,11 @@ static int query_formats(AVFilterContext *ctx) if ((ret = ff_set_common_channel_layouts(ctx, channel_layouts)) < 0) return ret; break; + case AVMEDIA_TYPE_SUBTITLE: + if ((ret = ff_add_format (&formats, c->subtitle_type)) < 0 || + (ret = ff_set_common_formats (ctx , formats )) < 0) + return ret; + break; default: return AVERROR(EINVAL); } @@ -408,6 +452,11 @@ static int config_props(AVFilterLink *link) if (!c->channel_layout) c->channel_layout = link->channel_layout; break; + case AVMEDIA_TYPE_SUBTITLE: + link->format = c->subtitle_type; + link->w = c->w; + link->h = c->h; + break; default: return AVERROR(EINVAL); } @@ -472,3 +521,26 @@ const AVFilter ff_asrc_abuffer = { FILTER_OUTPUTS(avfilter_asrc_abuffer_outputs), .priv_class = &abuffer_class, }; + +static const AVFilterPad avfilter_ssrc_sbuffer_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .request_frame = request_frame, + .config_props = config_props, + }, +}; + +const AVFilter ff_ssrc_sbuffer = { + .name = "sbuffer", + .description = NULL_IF_CONFIG_SMALL("Buffer subtitle frames, and make them accessible to the filterchain."), + .priv_size = sizeof(BufferSourceContext), + .query_formats = query_formats, + + .init = init_subtitle, + .uninit = uninit, + + .inputs = NULL, + FILTER_OUTPUTS(avfilter_ssrc_sbuffer_outputs), + .priv_class = &sbuffer_class, +}; diff --git a/libavfilter/buffersrc.h b/libavfilter/buffersrc.h index 08fbd18a47..929a2fa249 100644 --- a/libavfilter/buffersrc.h +++ b/libavfilter/buffersrc.h @@ -74,6 +74,7 @@ typedef struct AVBufferSrcParameters { /** * video: the pixel format, value corresponds to enum AVPixelFormat * audio: the sample format, value corresponds to enum AVSampleFormat + * subtitles: the subtitle format, value corresponds to enum AVSubtitleType */ int format; /** diff --git a/libavfilter/version.h b/libavfilter/version.h index 24b59acde6..ab5eb14634 100644 --- a/libavfilter/version.h +++ b/libavfilter/version.h @@ -30,7 +30,7 @@ #include "libavutil/version.h" #define LIBAVFILTER_VERSION_MAJOR 8 -#define LIBAVFILTER_VERSION_MINOR 9 +#define LIBAVFILTER_VERSION_MINOR 10 #define LIBAVFILTER_VERSION_MICRO 100 From patchwork Thu Sep 16 20:30:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 30284 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp1582542iov; Thu, 16 Sep 2021 13:31:26 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy5vPcYRkrb8FstijT72WzKKKh8AeqUsAA6I50dZFxryIy1cuiQI5o4ew9MHLbs1C5u9yyy X-Received: by 2002:aa7:c80a:: with SMTP id a10mr8452015edt.174.1631824286041; Thu, 16 Sep 2021 13:31:26 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id cc9si6738833edb.38.2021.09.16.13.31.24; Thu, 16 Sep 2021 13:31:26 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b="Tua1mkn/"; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2FC9B68B0CD; Thu, 16 Sep 2021 23:30:31 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11olkn2036.outbound.protection.outlook.com [40.92.19.36]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E0F4E68B0AE for ; Thu, 16 Sep 2021 23:30:27 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DAdmorVrqaycCcifLrh5ClNCPPV7NO2Kp1rYNdkV2UZFTws6gAnyNtPzyHg8oBgEmZXYNPCDnKKm+7k+AmQEDUNi12Egp99H7QgtDblacRN8bNp0prPDZ1b16Lo7/ENt3+OPdVMGY1eQY1jf4ToBaHH7tl2RA1Z2rQOFmvH3kKpSutR68hpghO/Ps9GpYb3ladF0C5ehOxZFFe3Rf7aJTP2gKrf5TpWPaw0w2tOHyPpzY1NzwQVD4PYTgjXTZf+V3I5LCLGTPQ+fpqnUzmVcbRw48givCD4dd8ncZMd1BxSPjfTCf1SNvG6yAw9JkQxNOicoHbrdknzjGPwwwL0OhQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=monymAfwfAxx6MkBfKeOskS83vjX/MoR50KzlJ89fZI=; b=LZZ5Qbye2+2yWevkQVAxeiCbBCHW9Mszuqg9ZAk0afbaiVv9TTkjHmRmzZk9h+WxzVgTJ7JkK4WlGnOtN/2dbKxgX/LKggtSyzUvarPhT8Hf+L6UbhISrjkeil3oO7iX/IykvpiTxKYWNmFBxvLub3AABqUCpjjddU53F7z+nNYYrU8g9SPa1P0N8dptRg28Vz4qm1i3TNKzmSyjkeRe3fhYWCvzLJyWl65PVI74O8clVxBfIKMZBm3gr4LxvmccAnof112mJ/iHxtTMFNGO+elD557UOpLECg6RtP7nnykTzOpy2D1bwcFC0PLMqJ9kr3uYSItPQUjBqUh6j4qwvA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=monymAfwfAxx6MkBfKeOskS83vjX/MoR50KzlJ89fZI=; b=Tua1mkn/q9qncYLO5qY7d0cNyyYL342YNO7sssBqqaQCp942Cu+EZc+8gpNVPLnKZP3y1wO8RihqCLmns5hUjQnct5HLNXdeiRqde2xlabh93X7BwI9TXrYpTkfOwFpvp4IO/E0JyUcSAbgK4XqRS4Eoo95w9bANvHFR+KnrVIRAmt5Cp+1y/EM1YEK0Sj4WxnYR0P6FywfXZBp3bUELjVKVXROETHDPiI6pu+Ba602volfiE0lNL8TrbHS5H6hMigZ6xSCYIkLmPolTyUhKzt/jFAJ9+OQwvL6ctZCnNv+i3xC/IBvg3n15NXwFNCt64y2TmERvFSskfeEfg3t/LA== Received: from MN2PR04MB5981.namprd04.prod.outlook.com (2603:10b6:208:da::10) by MN2PR04MB6048.namprd04.prod.outlook.com (2603:10b6:208:e5::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Thu, 16 Sep 2021 20:30:23 +0000 Received: from MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb]) by MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb%5]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021 20:30:23 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v6 06/11] avfilter/overlay_graphicsubs: Add overlay_graphicsubs and graphicsub2video filters Thread-Index: AQHXqzmsXQcbqoElZkSuvMPFbGhImQ== Date: Thu, 16 Sep 2021 20:30:23 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [ba5bp7oHSRlRmrt69M1FqUAeYT2xn1cp] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: fe9a3724-bd81-40c0-9d79-08d97950cf5c x-ms-traffictypediagnostic: MN2PR04MB6048: x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 7xJjrqE9YEE53EEErpdNc6AYHu8X44PlWpL7ikC1LQIwqixyQBrkma5gOjzy5/f38RJJ3tO7WrVsDAfKByvFka5DYmSZnhHIWsSw0c2sLpsgEJtMwAFhZmovDXB1RuucVhuoEVInubr0qsvKtaAAnHqbW9aM+yo3gWZRMLpQgHB9GQr1Pq6nuNwCXR4Yw2s3a9FqigZ51lYBD0Kr28VH6zBImwtduMryOfH1l+BX1XHRJyy0Rx332gtv+ttrLNUcGCLtVodsIND0YIJ8hXEsg8k1AW6Od8xXXzY2M7SrG2T6hQFwaEn2CDZJGMrfySzc5AMulJhIkXG5r5GvCsQjwSWPjzT/mAU2zs4j4/A4x8WPaXccS760MzsLkwFUyHN8oxncGIaL8xb7dr4JlR+MVYlHbgKCP+uvZdn3UFqZQGX50Y1oq79F6pYzK5cSXULBNJxx/k7MO6u+97wzq04iEgQ3L700azTS1EilcUYdh4U= x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: Y16fCEaLy8Xu1myljAvozn/S+xcAXdHRI8jNMQjBqA+Nf3PPH2GFrMuFxCCb5QlquJ5PL/mVBz+LeqlmGquT/IsYc/VYx1bd8VB+rbTFvR2vXswm6Sfb8+yHx+FEqSdyejLH8/XzDW3brUSxn6f95Q== x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-3174-20-msonline-outlook-529c7.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR04MB5981.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: fe9a3724-bd81-40c0-9d79-08d97950cf5c X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Sep 2021 20:30:23.1454 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR04MB6048 Subject: [FFmpeg-devel] [PATCH v6 06/11] avfilter/overlay_graphicsubs: Add overlay_graphicsubs and graphicsub2video filters X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: r79G2q8cJP+/ - overlay_graphicsubs (VS -> V) Overlay graphic subtitles onto a video stream - graphicsub2video {S -> V) Converts graphic subtitles to video frames (with alpha) Gets auto-inserted for retaining compatibility with sub2video command lines Signed-off-by: softworkz --- doc/filters.texi | 104 ++++ libavfilter/Makefile | 2 + libavfilter/allfilters.c | 2 + libavfilter/vf_overlay_graphicsubs.c | 727 +++++++++++++++++++++++++++ 4 files changed, 835 insertions(+) create mode 100644 libavfilter/vf_overlay_graphicsubs.c diff --git a/doc/filters.texi b/doc/filters.texi index 8f20ccf8c6..e281af7155 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -25079,6 +25079,110 @@ tools. @c man end VIDEO SINKS +@chapter Subtitle Filters +@c man begin SUBTITLE FILTERS + +When you configure your FFmpeg build, you can disable any of the +existing filters using @code{--disable-filters}. + +Below is a description of the currently available subtitle filters. + +@section graphicsub2video + +Renders graphic subtitles as video frames. + +This filter replaces the previous "sub2video" hack which did the conversion implicitly and up-front as subtitle filtering wasn't possible at that time. +To retain compatibility with earlier sub2video command lines, this filter is being auto-inserted in those cases. + +For overlaying graphicsal subtitles it is recommended to use the 'overlay_graphicsubs' filter which is more efficient and takes less processing resources. + +This filter is still useful in cases where the overlay is done with hardware acceleration (e.g. overlay_qsv, overlay_vaapi, overlay_cuda) for preparing the overlay frames. + +It accepts the following parameters: + +@table @option +@item size, s +Set the size of the output video frame. + +@end table + +@subsection Examples + +@itemize +@item +Overlay PGS subtitles +(not recommended - better use overlay_graphicsubs) +@example +ffmpeg -i "https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv" -filter_complex "[0:1]graphicsub2video[subs];[0:0][subs]overlay" output.mp4 +@end example + +@item +Overlay PGS subtitles implicitly +The graphicsub2video is inserted automatically for compatibility with legacy command lines. +@example +ffmpeg -i "https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv" -filter_complex "[0:0][0:1]overlay" output.mp4 +@end example +@end itemize + +@section overlay_graphicsubs + +Overlay graphic subtitles onto a video stream. + +This filter can blend graphical subtitles on a video stream directly, i.e. without creating full-size alpha images first. +The blending operation is limited to the area of the subtitle rectangles, which also means that no processing is done at times where no subtitles are to be displayed. + + +It accepts the following parameters: + +@table @option +@item x +@item y +Set the expression for the x and y coordinates of the overlaid video +on the main video. Default value is "0" for both expressions. In case +the expression is invalid, it is set to a huge value (meaning that the +overlay will not be displayed within the output visible area). + +@item eof_action +See @ref{framesync}. + +@item eval +Set when the expressions for @option{x}, and @option{y} are evaluated. + +It accepts the following values: +@table @samp +@item init +only evaluate expressions once during the filter initialization or +when a command is processed + +@item frame +evaluate expressions for each incoming frame +@end table + +Default value is @samp{frame}. + +@item shortest +See @ref{framesync}. + +@end table + +@subsection Examples + +@itemize +@item +Overlay PGS subtitles +@example +ffmpeg -i "https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv" -filter_complex "[0:1]graphicsub2video[subs];[0:0][subs]overlay" output.mp4 +@end example + +@item +Overlay PGS subtitles implicitly +The graphicsub2video is inserted automatically for compatibility with legacy command lines. +@example +ffmpeg -i "https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv" -filter_complex "[0:0][0:1]overlay" output.mp4 +@end example +@end itemize +@c man end SUBTITLE FILTERS + @chapter Multimedia Filters @c man begin MULTIMEDIA FILTERS diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 041d3c5382..8fcc25989e 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -290,6 +290,7 @@ OBJS-$(CONFIG_FSPP_FILTER) += vf_fspp.o qp_table.o OBJS-$(CONFIG_GBLUR_FILTER) += vf_gblur.o OBJS-$(CONFIG_GEQ_FILTER) += vf_geq.o OBJS-$(CONFIG_GRADFUN_FILTER) += vf_gradfun.o +OBJS-$(CONFIG_GRAPHICSUB2VIDEO_FILTER) += vf_overlay_graphicsubs.o framesync.o OBJS-$(CONFIG_GRAPHMONITOR_FILTER) += f_graphmonitor.o OBJS-$(CONFIG_GRAYWORLD_FILTER) += vf_grayworld.o OBJS-$(CONFIG_GREYEDGE_FILTER) += vf_colorconstancy.o @@ -363,6 +364,7 @@ OBJS-$(CONFIG_OVERLAY_CUDA_FILTER) += vf_overlay_cuda.o framesync.o vf OBJS-$(CONFIG_OVERLAY_OPENCL_FILTER) += vf_overlay_opencl.o opencl.o \ opencl/overlay.o framesync.o OBJS-$(CONFIG_OVERLAY_QSV_FILTER) += vf_overlay_qsv.o framesync.o +OBJS-$(CONFIG_OVERLAY_GRAPHICSUBS_FILTER) += vf_overlay_graphicsubs.o framesync.o OBJS-$(CONFIG_OVERLAY_VULKAN_FILTER) += vf_overlay_vulkan.o vulkan.o OBJS-$(CONFIG_OWDENOISE_FILTER) += vf_owdenoise.o OBJS-$(CONFIG_PAD_FILTER) += vf_pad.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 154eba5bb2..10a310d20d 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -345,6 +345,7 @@ extern const AVFilter ff_vf_oscilloscope; extern const AVFilter ff_vf_overlay; extern const AVFilter ff_vf_overlay_opencl; extern const AVFilter ff_vf_overlay_qsv; +extern const AVFilter ff_vf_overlay_graphicsubs; extern const AVFilter ff_vf_overlay_vulkan; extern const AVFilter ff_vf_overlay_cuda; extern const AVFilter ff_vf_owdenoise; @@ -524,6 +525,7 @@ extern const AVFilter ff_avf_showvolume; extern const AVFilter ff_avf_showwaves; extern const AVFilter ff_avf_showwavespic; extern const AVFilter ff_vaf_spectrumsynth; +extern const AVFilter ff_svf_graphicsub2video; /* multimedia sources */ extern const AVFilter ff_avsrc_amovie; diff --git a/libavfilter/vf_overlay_graphicsubs.c b/libavfilter/vf_overlay_graphicsubs.c new file mode 100644 index 0000000000..5443625adc --- /dev/null +++ b/libavfilter/vf_overlay_graphicsubs.c @@ -0,0 +1,727 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * overlay graphical subtitles on top of a video frame + */ + +#include "avfilter.h" +#include "formats.h" +#include "libavutil/common.h" +#include "libavutil/eval.h" +#include "libavutil/avstring.h" +#include "libavutil/pixdesc.h" +#include "libavutil/imgutils.h" +#include "libavutil/opt.h" +#include "internal.h" +#include "drawutils.h" +#include "framesync.h" + +#include "libavcodec/avcodec.h" + +enum var_name { + VAR_MAIN_W, VAR_MW, + VAR_MAIN_H, VAR_MH, + VAR_OVERLAY_W, VAR_OW, + VAR_OVERLAY_H, VAR_OH, + VAR_HSUB, + VAR_VSUB, + VAR_X, + VAR_Y, + VAR_N, + VAR_POS, + VAR_T, + VAR_VARS_NB +}; + +typedef struct OverlaySubsContext { + const AVClass *class; + int x, y; ///< position of overlaid picture + int w, h; + AVFrame *outpicref; + + int main_is_packed_rgb; + uint8_t main_rgba_map[4]; + int main_has_alpha; + uint8_t overlay_rgba_map[4]; + int eval_mode; ///< EvalMode + + FFFrameSync fs; + + int main_pix_step[4]; ///< steps per pixel for each plane of the main output + int hsub, vsub; ///< chroma subsampling values + const AVPixFmtDescriptor *main_desc; ///< format descriptor for main input + + double var_values[VAR_VARS_NB]; + char *x_expr, *y_expr; + + AVExpr *x_pexpr, *y_pexpr; +} OverlaySubsContext; + +static const char *const var_names[] = { + "main_w", "W", ///< width of the main video + "main_h", "H", ///< height of the main video + "overlay_w", "w", ///< width of the overlay video + "overlay_h", "h", ///< height of the overlay video + "hsub", + "vsub", + "x", + "y", + "n", ///< number of frame + "pos", ///< position in the file + "t", ///< timestamp expressed in seconds + NULL +}; + +#define MAIN 0 +#define OVERLAY 1 + +#define R 0 +#define G 1 +#define B 2 +#define A 3 + +#define Y 0 +#define U 1 +#define V 2 + +enum EvalMode { + EVAL_MODE_INIT, + EVAL_MODE_FRAME, + EVAL_MODE_NB +}; + +static av_cold void overlay_graphicsubs_uninit(AVFilterContext *ctx) +{ + OverlaySubsContext *s = ctx->priv; + + ff_framesync_uninit(&s->fs); + av_expr_free(s->x_pexpr); s->x_pexpr = NULL; + av_expr_free(s->y_pexpr); s->y_pexpr = NULL; +} + +static inline int normalize_xy(double d, int chroma_sub) +{ + if (isnan(d)) + return INT_MAX; + return (int)d & ~((1 << chroma_sub) - 1); +} + +static void eval_expr(AVFilterContext *ctx) +{ + OverlaySubsContext *s = ctx->priv; + + s->var_values[VAR_X] = av_expr_eval(s->x_pexpr, s->var_values, NULL); + s->var_values[VAR_Y] = av_expr_eval(s->y_pexpr, s->var_values, NULL); + /* It is necessary if x is expressed from y */ + s->var_values[VAR_X] = av_expr_eval(s->x_pexpr, s->var_values, NULL); + s->x = normalize_xy(s->var_values[VAR_X], s->hsub); + s->y = normalize_xy(s->var_values[VAR_Y], s->vsub); +} + +static int set_expr(AVExpr **pexpr, const char *expr, const char *option, void *log_ctx) +{ + int ret; + AVExpr *old = NULL; + + if (*pexpr) + old = *pexpr; + ret = av_expr_parse(pexpr, expr, var_names, + NULL, NULL, NULL, NULL, 0, log_ctx); + if (ret < 0) { + av_log(log_ctx, AV_LOG_ERROR, + "Error when evaluating the expression '%s' for %s\n", + expr, option); + *pexpr = old; + return ret; + } + + av_expr_free(old); + return 0; +} + +static int overlay_graphicsubs_query_formats(AVFilterContext *ctx) +{ + AVFilterFormats *formats; + AVFilterLink *inlink0 = ctx->inputs[0]; + AVFilterLink *inlink1 = ctx->inputs[1]; + AVFilterLink *outlink = ctx->outputs[0]; + int ret; + static const enum AVSubtitleType subtitle_fmts[] = { AV_SUBTITLE_FMT_BITMAP, AV_SUBTITLE_FMT_NONE }; + static const enum AVPixelFormat supported_pix_fmts[] = { + AV_PIX_FMT_YUV420P, AV_PIX_FMT_YUV422P, AV_PIX_FMT_YUV444P, + AV_PIX_FMT_ARGB, AV_PIX_FMT_RGBA, + AV_PIX_FMT_ABGR, AV_PIX_FMT_BGRA, + AV_PIX_FMT_RGB24, AV_PIX_FMT_BGR24, + AV_PIX_FMT_NONE + }; + + /* set input0 video formats */ + formats = ff_make_format_list(supported_pix_fmts); + if ((ret = ff_formats_ref(formats, &inlink0->outcfg.formats)) < 0) + return ret; + + /* set input1 subtitle formats */ + formats = ff_make_format_list(subtitle_fmts); + if ((ret = ff_formats_ref(formats, &inlink1->outcfg.formats)) < 0) + return ret; + + /* set output0 video formats */ + formats = ff_make_format_list(supported_pix_fmts); + if ((ret = ff_formats_ref(formats, &outlink->incfg.formats)) < 0) + return ret; + + return 0; +} + +static int config_output(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + OverlaySubsContext *s = ctx->priv; + int ret; + + if ((ret = ff_framesync_init_dualinput(&s->fs, ctx)) < 0) + return ret; + + outlink->w = ctx->inputs[MAIN]->w; + outlink->h = ctx->inputs[MAIN]->h; + outlink->time_base = ctx->inputs[MAIN]->time_base; + + return ff_framesync_configure(&s->fs); +} + +// divide by 255 and round to nearest +// apply a fast variant: (X+127)/255 = ((X+127)*257+257)>>16 = ((X+128)*257)>>16 +#define FAST_DIV255(x) ((((x) + 128) * 257) >> 16) + +// calculate the non-pre-multiplied alpha, applying the general equation: +// alpha = alpha_overlay / ( (alpha_main + alpha_overlay) - (alpha_main * alpha_overlay) ) +// (((x) << 16) - ((x) << 9) + (x)) is a faster version of: 255 * 255 * x +// ((((x) + (y)) << 8) - ((x) + (y)) - (y) * (x)) is a faster version of: 255 * (x + y) +#define UNPREMULTIPLY_ALPHA(x, y) ((((x) << 16) - ((x) << 9) + (x)) / ((((x) + (y)) << 8) - ((x) + (y)) - (y) * (x))) + +/** + * Blend image in src to destination buffer dst at position (x, y). + */ +static av_always_inline void blend_packed_rgb(const AVFilterContext *ctx, + const AVFrame *dst, const AVSubtitleArea *src, + int x, int y, + int is_straight) +{ + OverlaySubsContext *s = ctx->priv; + int i, imax, j, jmax; + const int src_w = src->w; + const int src_h = src->h; + const int dst_w = dst->width; + const int dst_h = dst->height; + uint8_t alpha; ///< the amount of overlay to blend on to main + const int dr = s->main_rgba_map[R]; + const int dg = s->main_rgba_map[G]; + const int db = s->main_rgba_map[B]; + const int da = s->main_rgba_map[A]; + const int dstep = s->main_pix_step[0]; + const int sr = s->overlay_rgba_map[R]; + const int sg = s->overlay_rgba_map[G]; + const int sb = s->overlay_rgba_map[B]; + const int sa = s->overlay_rgba_map[A]; + int slice_start, slice_end; + uint8_t *S, *sp, *d, *dp; + + i = FFMAX(-y, 0); + imax = FFMIN3(-y + dst_h, FFMIN(src_h, dst_h), y + src_h); + + slice_start = i; + slice_end = i + imax; + + sp = src->data[0] + slice_start * src->linesize[0]; + dp = dst->data[0] + (slice_start + y) * dst->linesize[0]; + + for (i = slice_start; i < slice_end; i++) { + j = FFMAX(-x, 0); + S = sp + j; + d = dp + ((x + j) * dstep); + + for (jmax = FFMIN(-x + dst_w, src_w); j < jmax; j++) { + uint32_t val = src->pal[*S]; + const uint8_t *sval = (uint8_t *)&val; + alpha = sval[sa]; + + // if the main channel has an alpha channel, alpha has to be calculated + // to create an un-premultiplied (straight) alpha value + if (s->main_has_alpha && alpha != 0 && alpha != 255) { + const uint8_t alpha_d = d[da]; + alpha = UNPREMULTIPLY_ALPHA(alpha, alpha_d); + } + + switch (alpha) { + case 0: + break; + case 255: + d[dr] = sval[sr]; + d[dg] = sval[sg]; + d[db] = sval[sb]; + break; + default: + // main_value = main_value * (1 - alpha) + overlay_value * alpha + // since alpha is in the range 0-255, the result must divided by 255 + d[dr] = is_straight ? FAST_DIV255(d[dr] * (255 - alpha) + sval[sr] * alpha) : + FFMIN(FAST_DIV255(d[dr] * (255 - alpha)) + sval[sr], 255); + d[dg] = is_straight ? FAST_DIV255(d[dg] * (255 - alpha) + sval[sg] * alpha) : + FFMIN(FAST_DIV255(d[dg] * (255 - alpha)) + sval[sg], 255); + d[db] = is_straight ? FAST_DIV255(d[db] * (255 - alpha) + sval[sb] * alpha) : + FFMIN(FAST_DIV255(d[db] * (255 - alpha)) + sval[sb], 255); + } + + if (s->main_has_alpha) { + switch (alpha) { + case 0: + break; + case 255: + d[da] = sval[sa]; + break; + default: + // apply alpha compositing: main_alpha += (1-main_alpha) * overlay_alpha + d[da] += FAST_DIV255((255 - d[da]) * S[sa]); + } + } + d += dstep; + S += 1; + } + dp += dst->linesize[0]; + sp += src->linesize[0]; + } +} + +static av_always_inline void blend_plane_8_8bits(const AVFilterContext *ctx, const AVFrame *dst, const AVSubtitleArea *area, + const uint32_t *yuv_pal, int src_w, int src_h, int dst_w, int dst_h, int plane, int hsub, int vsub, + int x, int y, int dst_plane, int dst_offset, int dst_step) +{ + const int src_wp = AV_CEIL_RSHIFT(src_w, hsub); + const int src_hp = AV_CEIL_RSHIFT(src_h, vsub); + const int dst_wp = AV_CEIL_RSHIFT(dst_w, hsub); + const int dst_hp = AV_CEIL_RSHIFT(dst_h, vsub); + const int yp = y >> vsub; + const int xp = x >> hsub; + uint8_t *s, *sp, *d, *dp, *dap; + int imax, i, j, jmax; + int slice_start, slice_end; + + i = FFMAX(-yp, 0); \ + imax = FFMIN3(-yp + dst_hp, FFMIN(src_hp, dst_hp), yp + src_hp); \ + + slice_start = i; + slice_end = i + imax; + + sp = area->data[0] + (slice_start << vsub) * area->linesize[0]; + dp = dst->data[dst_plane] + (yp + slice_start) * dst->linesize[dst_plane] + dst_offset; + + dap = dst->data[3] + ((yp + slice_start) << vsub) * dst->linesize[3]; + + for (i = slice_start; i < slice_end; i++) { + j = FFMAX(-xp, 0); + d = dp + (xp + j) * dst_step; + s = sp + (j << hsub); + jmax = FFMIN(-xp + dst_wp, src_wp); + + for (; j < jmax; j++) { + uint32_t val = yuv_pal[*s]; + const uint8_t *sval = (uint8_t *)&val; + const int alpha = sval[3]; + const int max = 255, mid = 128; + const int d_int = *d; + const int sval_int = sval[plane]; + + switch (alpha) { + case 0: + break; + case 255: + *d = sval[plane]; + break; + default: + if (plane > 0) + *d = av_clip(FAST_DIV255((d_int - mid) * (max - alpha) + (sval_int - mid) * alpha) , -mid, mid) + mid; + else + *d = FAST_DIV255(d_int * (max - alpha) + sval_int * alpha); + break; + } + + d += dst_step; + s += 1 << hsub; + } + dp += dst->linesize[dst_plane]; + sp += (1 << vsub) * area->linesize[0]; + dap += (1 << vsub) * dst->linesize[3]; + } +} + +#define RGB2Y(r, g, b) (uint8_t)(((66 * (r) + 129 * (g) + 25 * (b) + 128) >> 8) + 16) +#define RGB2U(r, g, b) (uint8_t)(((-38 * (r) - 74 * (g) + 112 * (b) + 128) >> 8) + 128) +#define RGB2V(r, g, b) (uint8_t)(((112 * (r) - 94 * (g) - 18 * (b) + 128) >> 8) + 128) +/* Converts R8 G8 B8 color to YUV. */ +static av_always_inline void rgb_2_yuv(uint8_t r, uint8_t g, uint8_t b, uint8_t* y, uint8_t* u, uint8_t* v) +{ + *y = RGB2Y((int)r, (int)g, (int)b); + *u = RGB2U((int)r, (int)g, (int)b); + *v = RGB2V((int)r, (int)g, (int)b); +} + + +static av_always_inline void blend_yuv_8_8bits(AVFilterContext *ctx, AVFrame *dst, const AVSubtitleArea *area, int hsub, int vsub, int x, int y) +{ + OverlaySubsContext *s = ctx->priv; + const int src_w = area->w; + const int src_h = area->h; + const int dst_w = dst->width; + const int dst_h = dst->height; + const int sr = s->overlay_rgba_map[R]; + const int sg = s->overlay_rgba_map[G]; + const int sb = s->overlay_rgba_map[B]; + const int sa = s->overlay_rgba_map[A]; + uint32_t yuvpal[256]; + + for (int i = 0; i < 256; ++i) { + uint8_t *rgba = (uint8_t *)&area->pal[i]; + uint8_t *yuva = (uint8_t *)&yuvpal[i]; + rgb_2_yuv(rgba[sr], rgba[sg], rgba[sb], &yuva[Y], &yuva[U], &yuva[V]); + yuva[3] = rgba[sa]; + } + + blend_plane_8_8bits(ctx, dst, area, yuvpal, src_w, src_h, dst_w, dst_h, Y, 0, 0, x, y, s->main_desc->comp[Y].plane, s->main_desc->comp[Y].offset, s->main_desc->comp[Y].step); + blend_plane_8_8bits(ctx, dst, area, yuvpal, src_w, src_h, dst_w, dst_h, U, hsub, vsub, x, y, s->main_desc->comp[U].plane, s->main_desc->comp[U].offset, s->main_desc->comp[U].step); + blend_plane_8_8bits(ctx, dst, area, yuvpal, src_w, src_h, dst_w, dst_h, V, hsub, vsub, x, y, s->main_desc->comp[V].plane, s->main_desc->comp[V].offset, s->main_desc->comp[V].step); +} + +static int config_input_main(AVFilterLink *inlink) +{ + int ret; + AVFilterContext *ctx = inlink->dst; + OverlaySubsContext *s = inlink->dst->priv; + const AVPixFmtDescriptor *pix_desc = av_pix_fmt_desc_get(inlink->format); + + av_image_fill_max_pixsteps(s->main_pix_step, NULL, pix_desc); + ff_fill_rgba_map(s->overlay_rgba_map, AV_PIX_FMT_RGB32); // it's actually AV_PIX_FMT_PAL8); + + s->hsub = pix_desc->log2_chroma_w; + s->vsub = pix_desc->log2_chroma_h; + + s->main_desc = pix_desc; + + s->main_is_packed_rgb = ff_fill_rgba_map(s->main_rgba_map, inlink->format) >= 0; + s->main_has_alpha = !!(pix_desc->flags & AV_PIX_FMT_FLAG_ALPHA); + + /* Finish the configuration by evaluating the expressions + now when both inputs are configured. */ + s->var_values[VAR_MAIN_W ] = s->var_values[VAR_MW] = ctx->inputs[MAIN ]->w; + s->var_values[VAR_MAIN_H ] = s->var_values[VAR_MH] = ctx->inputs[MAIN ]->h; + s->var_values[VAR_OVERLAY_W] = s->var_values[VAR_OW] = ctx->inputs[OVERLAY]->w; + s->var_values[VAR_OVERLAY_H] = s->var_values[VAR_OH] = ctx->inputs[OVERLAY]->h; + s->var_values[VAR_HSUB] = 1<log2_chroma_w; + s->var_values[VAR_VSUB] = 1<log2_chroma_h; + s->var_values[VAR_X] = NAN; + s->var_values[VAR_Y] = NAN; + s->var_values[VAR_N] = 0; + s->var_values[VAR_T] = NAN; + s->var_values[VAR_POS] = NAN; + + if ((ret = set_expr(&s->x_pexpr, s->x_expr, "x", ctx)) < 0 || + (ret = set_expr(&s->y_pexpr, s->y_expr, "y", ctx)) < 0) + return ret; + + if (s->eval_mode == EVAL_MODE_INIT) { + eval_expr(ctx); + av_log(ctx, AV_LOG_VERBOSE, "x:%f xi:%d y:%f yi:%d\n", + s->var_values[VAR_X], s->x, + s->var_values[VAR_Y], s->y); + } + + av_log(ctx, AV_LOG_VERBOSE, + "main w:%d h:%d fmt:%s overlay w:%d h:%d fmt:%s\n", + ctx->inputs[MAIN]->w, ctx->inputs[MAIN]->h, + av_get_pix_fmt_name(ctx->inputs[MAIN]->format), + ctx->inputs[OVERLAY]->w, ctx->inputs[OVERLAY]->h, + av_get_pix_fmt_name(ctx->inputs[OVERLAY]->format)); + return 0; +} + +static int do_blend(FFFrameSync *fs) +{ + AVFilterContext *ctx = fs->parent; + AVFrame *mainpic, *second; + OverlaySubsContext *s = ctx->priv; + AVFilterLink *inlink = ctx->inputs[0]; + unsigned num_rects, i; + int ret; + + ret = ff_framesync_dualinput_get_writable(fs, &mainpic, &second); + if (ret < 0) + return ret; + if (!second) + return ff_filter_frame(ctx->outputs[0], mainpic); + + if (s->eval_mode == EVAL_MODE_FRAME) { + int64_t pos = mainpic->pkt_pos; + + s->var_values[VAR_N] = inlink->frame_count_out; + s->var_values[VAR_T] = mainpic->pts == AV_NOPTS_VALUE ? + NAN : mainpic->pts * av_q2d(inlink->time_base); + s->var_values[VAR_POS] = pos == -1 ? NAN : pos; + + s->var_values[VAR_OVERLAY_W] = s->var_values[VAR_OW] = second->width; + s->var_values[VAR_OVERLAY_H] = s->var_values[VAR_OH] = second->height; + s->var_values[VAR_MAIN_W ] = s->var_values[VAR_MW] = mainpic->width; + s->var_values[VAR_MAIN_H ] = s->var_values[VAR_MH] = mainpic->height; + + eval_expr(ctx); + av_log(ctx, AV_LOG_DEBUG, "n:%f t:%f pos:%f x:%f xi:%d y:%f yi:%d\n", + s->var_values[VAR_N], s->var_values[VAR_T], s->var_values[VAR_POS], + s->var_values[VAR_X], s->x, + s->var_values[VAR_Y], s->y); + } + + for (i = 0; i < second->num_subtitle_areas; i++) { + const AVSubtitleArea *sub_area = second->subtitle_areas[i]; + + if (sub_area->type != AV_SUBTITLE_FMT_BITMAP) { + av_log(NULL, AV_LOG_WARNING, "overlay_graphicsub: non-bitmap subtitle\n"); + return AVERROR_INVALIDDATA; + } + + switch (inlink->format) { + case AV_PIX_FMT_YUV420P: + blend_yuv_8_8bits(ctx, mainpic, sub_area, 1, 1, sub_area->x + s->x, sub_area->y + s->y); + break; + case AV_PIX_FMT_YUV422P: + blend_yuv_8_8bits(ctx, mainpic, sub_area, 1, 0, sub_area->x + s->x, sub_area->y + s->y); + break; + case AV_PIX_FMT_YUV444P: + blend_yuv_8_8bits(ctx, mainpic, sub_area, 0, 0, sub_area->x + s->x, sub_area->y + s->y); + break; + case AV_PIX_FMT_RGB24: + case AV_PIX_FMT_BGR24: + case AV_PIX_FMT_ARGB: + case AV_PIX_FMT_RGBA: + case AV_PIX_FMT_BGRA: + case AV_PIX_FMT_ABGR: + blend_packed_rgb(ctx, mainpic, sub_area, sub_area->x + s->x, sub_area->y + s->y, 1); + break; + default: + av_assert0(0); + break; + } + } + + return ff_filter_frame(ctx->outputs[0], mainpic); +} + +static av_cold int overlay_graphicsubs_init(AVFilterContext *ctx) +{ + OverlaySubsContext *s = ctx->priv; + + s->fs.on_event = do_blend; + return 0; +} + +static int overlay_graphicsubs_activate(AVFilterContext *ctx) +{ + OverlaySubsContext *s = ctx->priv; + return ff_framesync_activate(&s->fs); +} + +static int graphicsub2video_query_formats(AVFilterContext *ctx) +{ + AVFilterFormats *formats; + AVFilterLink *inlink = ctx->inputs[0]; + AVFilterLink *outlink = ctx->outputs[0]; + static const enum AVSubtitleType subtitle_fmts[] = { AV_SUBTITLE_FMT_BITMAP, AV_SUBTITLE_FMT_NONE }; + static const enum AVPixelFormat pix_fmts[] = { AV_PIX_FMT_RGB32, AV_PIX_FMT_NONE }; + int ret; + + /* set input subtitle formats */ + formats = ff_make_format_list(subtitle_fmts); + if ((ret = ff_formats_ref(formats, &inlink->outcfg.formats)) < 0) + return ret; + + /* set output video formats */ + formats = ff_make_format_list(pix_fmts); + if ((ret = ff_formats_ref(formats, &outlink->incfg.formats)) < 0) + return ret; + + return 0; +} + +static int graphicsub2video_config_input(AVFilterLink *inlink) +{ + AVFilterContext *ctx = inlink->dst; + OverlaySubsContext *s = ctx->priv; + + if (s->w <= 0 || s->h <= 0) { + s->w = inlink->w; + s->h = inlink->h; + } + return 0; +} + +static int graphicsub2video_config_output(AVFilterLink *outlink) +{ + const AVFilterContext *ctx = outlink->src; + OverlaySubsContext *s = ctx->priv; + const AVPixFmtDescriptor *pix_desc = av_pix_fmt_desc_get(outlink->format); + + outlink->w = s->w; + outlink->h = s->h; + outlink->sample_aspect_ratio = (AVRational){1,1}; + + av_image_fill_max_pixsteps(s->main_pix_step, NULL, pix_desc); + ff_fill_rgba_map(s->overlay_rgba_map, AV_PIX_FMT_RGB32); + + s->hsub = pix_desc->log2_chroma_w; + s->vsub = pix_desc->log2_chroma_h; + + s->main_desc = pix_desc; + + s->main_is_packed_rgb = ff_fill_rgba_map(s->main_rgba_map, outlink->format) >= 0; + s->main_has_alpha = !!(pix_desc->flags & AV_PIX_FMT_FLAG_ALPHA); + + return 0; +} + +static int graphicsub2video_filter_frame(AVFilterLink *inlink, AVFrame *src_frame) +{ + AVFilterLink *outlink = inlink->dst->outputs[0]; + AVFrame *out; + const unsigned num_rects = src_frame->num_subtitle_areas; + unsigned int i; + + out = ff_get_video_buffer(outlink, outlink->w, outlink->h); + if (!out) + return AVERROR(ENOMEM); + + memset(out->data[0], 0, out->linesize[0] * out->height); + + out->pts = src_frame->pts; + + + for (i = 0; i < num_rects; i++) { + const AVSubtitleArea *sub_rect = src_frame->subtitle_areas[i]; + + if (sub_rect->type != AV_SUBTITLE_FMT_BITMAP) { + av_log(NULL, AV_LOG_WARNING, "sub2video: non-bitmap subtitle\n"); + return AVERROR_INVALIDDATA; + } + + blend_packed_rgb(inlink->dst, out, sub_rect, sub_rect->x, sub_rect->y, 1); + } + + av_frame_free(&src_frame); + return ff_filter_frame(outlink, out); +} + +#define OFFSET(x) offsetof(OverlaySubsContext, x) +#define FLAGS (AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_FILTERING_PARAM) + +static const AVOption overlay_graphicsubs_options[] = { + { "x", "set the x expression", OFFSET(x_expr), AV_OPT_TYPE_STRING, {.str = "0"}, 0, 0, FLAGS }, + { "y", "set the y expression", OFFSET(y_expr), AV_OPT_TYPE_STRING, {.str = "0"}, 0, 0, FLAGS }, + { "eof_action", "Action to take when encountering EOF from secondary input ", + OFFSET(fs.opt_eof_action), AV_OPT_TYPE_INT, { .i64 = EOF_ACTION_REPEAT }, + EOF_ACTION_REPEAT, EOF_ACTION_PASS, .flags = FLAGS, "eof_action" }, + { "repeat", "Repeat the previous frame.", 0, AV_OPT_TYPE_CONST, { .i64 = EOF_ACTION_REPEAT }, .flags = FLAGS, "eof_action" }, + { "endall", "End both streams.", 0, AV_OPT_TYPE_CONST, { .i64 = EOF_ACTION_ENDALL }, .flags = FLAGS, "eof_action" }, + { "pass", "Pass through the main input.", 0, AV_OPT_TYPE_CONST, { .i64 = EOF_ACTION_PASS }, .flags = FLAGS, "eof_action" }, + { "eval", "specify when to evaluate expressions", OFFSET(eval_mode), AV_OPT_TYPE_INT, {.i64 = EVAL_MODE_FRAME}, 0, EVAL_MODE_NB-1, FLAGS, "eval" }, + { "init", "eval expressions once during initialization", 0, AV_OPT_TYPE_CONST, {.i64=EVAL_MODE_INIT}, .flags = FLAGS, .unit = "eval" }, + { "frame", "eval expressions per-frame", 0, AV_OPT_TYPE_CONST, {.i64=EVAL_MODE_FRAME}, .flags = FLAGS, .unit = "eval" }, + { "shortest", "force termination when the shortest input terminates", OFFSET(fs.opt_shortest), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, FLAGS }, + { "repeatlast", "repeat overlay of the last overlay frame", OFFSET(fs.opt_repeatlast), AV_OPT_TYPE_BOOL, {.i64=1}, 0, 1, FLAGS }, + { NULL } +}; + +static const AVOption graphicsub2video_options[] = { + { "size", "set video size", OFFSET(w), AV_OPT_TYPE_IMAGE_SIZE, {.str = NULL}, 0, 0, FLAGS }, + { "s", "set video size", OFFSET(w), AV_OPT_TYPE_IMAGE_SIZE, {.str = NULL}, 0, 0, FLAGS }, + { NULL } +}; + +FRAMESYNC_DEFINE_CLASS(overlay_graphicsubs, OverlaySubsContext, fs); + +static const AVFilterPad overlay_graphicsubs_inputs[] = { + { + .name = "main", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_input_main, + .flags = AVFILTERPAD_FLAG_NEEDS_WRITABLE, + }, + { + .name = "overlay", + .type = AVMEDIA_TYPE_SUBTITLE, + }, +}; + +static const AVFilterPad overlay_graphicsubs_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_output, + }, +}; + +const AVFilter ff_vf_overlay_graphicsubs = { + .name = "overlay_graphicsubs", + .description = NULL_IF_CONFIG_SMALL("Overlay graphical subtitles on top of the input."), + .preinit = overlay_graphicsubs_framesync_preinit, + .init = overlay_graphicsubs_init, + .uninit = overlay_graphicsubs_uninit, + .priv_size = sizeof(OverlaySubsContext), + .priv_class = &overlay_graphicsubs_class, + .query_formats = overlay_graphicsubs_query_formats, + .activate = overlay_graphicsubs_activate, + FILTER_INPUTS(overlay_graphicsubs_inputs), + FILTER_OUTPUTS(overlay_graphicsubs_outputs), +}; + +AVFILTER_DEFINE_CLASS(graphicsub2video); + +static const AVFilterPad graphicsub2video_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .filter_frame = graphicsub2video_filter_frame, + .config_props = graphicsub2video_config_input, + }, +}; + +static const AVFilterPad graphicsub2video_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = graphicsub2video_config_output, + }, +}; + +AVFilter ff_svf_graphicsub2video = { + .name = "graphicsub2video", + .description = NULL_IF_CONFIG_SMALL("Convert graphical subtitles to video"), + .query_formats = graphicsub2video_query_formats, + .priv_size = sizeof(OverlaySubsContext), + .priv_class = &graphicsub2video_class, + FILTER_INPUTS(graphicsub2video_inputs), + FILTER_OUTPUTS(graphicsub2video_outputs), +}; From patchwork Thu Sep 16 20:30:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 30288 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp1582827iov; Thu, 16 Sep 2021 13:31:50 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwP8aytxbMaH3fVadCsjpk5g1IZ0ycNTK3tApPmylCKyUqiwE2Ck4Yoh9SEUWq/kprPdJgQ X-Received: by 2002:a05:6402:358c:: with SMTP id y12mr8430430edc.50.1631824310329; Thu, 16 Sep 2021 13:31:50 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id c24si4991935ejj.636.2021.09.16.13.31.49; Thu, 16 Sep 2021 13:31:50 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=ULdzw1nv; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5C5F768B0DF; Thu, 16 Sep 2021 23:30:34 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12olkn2039.outbound.protection.outlook.com [40.92.22.39]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 851CF68B0D1 for ; Thu, 16 Sep 2021 23:30:31 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FIF65YvgAyCHy/mPwGvRwmqD/D34FrQ3GIiQiWCJSZUml3+qJKHNyiFgR0dHHG3Lh7jXxSYqUi5K/wrMGFx1VdfkynKQHtQ4XX7KIHUlgmvUb2XmEc2m4KHbo9/YCA9zViUyEem70iskbTrhL2PPI7S1s3oG3d8AWT8aFHfGiIpapBc1CvrHPqpAnaxqSr+paI6Mw0BuenVZaqfqx9VjrPcUIecG9w7uMNdj0SMwwKi3LD/ETmiMUphvW0pbzrumB+F6iFctZm+7fGYIlK1LfJ7VQXgrXlUe6YpmyxaQvG0UP+7EqVkMFKut9zAsXDIGkOt09NvJ1efVyC0vFPzfew== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=ONsjo93eaW5D+dGdoFQH6ugxthdOCie0xz/gCArk6B0=; b=ntS7FCqksUhRrNJVjc5AVaGwwTLXGe7HxJRbbPcG8d04LGXp9x8S653/O4A83TBrNte2/HVYy/Q3WP4j6sVQfGcQbmcHKt9ycgz6vxkKDcCe8c0Jnv/f6W8vCIb4yyH3BHPXV415idj6ltUoe/XnGJsUEcxzQZ9AraYlIlo0rDqOSAmQicuR/ja5CUrL5WP6wqtjBrgDsZ7DYUR3J3Z62gc0joQ/qVy/5id4gSc8u338RbdD4bUwPPcNxouT8wIlYcdpbUEb1lJTJ9PZ3d3p/Z0umFcOrT1kVpPE4JR5eCVC+hky5ei3rwjCeWD9ZChhbpruTm3TDl151x2B5Y40Aw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ONsjo93eaW5D+dGdoFQH6ugxthdOCie0xz/gCArk6B0=; b=ULdzw1nv6H7ok25TCaVEE5uReFGF8Sa/WfQQkQehbfGyso3k2di7Q3pynL/OaiNxu/jHUcjlE6quyb+APUjZIUizwHfvCFZek07gxVpyzihu2bnphORzkt/06me4Ps5PfwFBqLLTmAMBmWhP/k6MrLBm3AYkMIu23PbreK3Mw9fEZAZyhU9Ze8ZrpTpgMXCLX9K8FoPUI4BYgdG1HsZWT1k1IdyR6NsDjqQsRCDcuMG/cjSPzgxduRCpwmMbuoYjEYr9MtOtkFy14ZKDk8QakIL89OWSGycfvTb1Jd/GDuMXRJF50b/wqV0DQESPWBcCj+IwCGKhhQMZ1QfA7AdKZg== Received: from MN2PR04MB5981.namprd04.prod.outlook.com (2603:10b6:208:da::10) by MN2PR04MB5629.namprd04.prod.outlook.com (2603:10b6:208:a6::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Thu, 16 Sep 2021 20:30:25 +0000 Received: from MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb]) by MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb%5]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021 20:30:25 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v6 07/11] fftools/ffmpeg: Replace sub2video with subtitle frame filtering Thread-Index: AQHXqzmunxuapaz+FkK4BIrxXLJHyA== Date: Thu, 16 Sep 2021 20:30:25 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [7IIa08Z5v/1zT53pEPXRp5IjoVXIQFsS] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 58d05adb-8887-45b7-2854-08d97950d0dc x-ms-traffictypediagnostic: MN2PR04MB5629: x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 6zWpnMY16SJlPAj1i7dZQBpXfwNOOMdae3+QLdYeix3iugwfvXBQkiUzhQ+OCWq5ykKiF5EbPHaTJ35YhcGxRTmVMhaVOcIqOniDZ4CvfZKYc9Et2sMt3oa7z1tyyQSecuUvB0wbw49fcwOq7lHVRCYeZAXXOTnYpTzVxd1Zxr59159WqnT7xcV4e7EnJb/DggGFWAmPXJ3VC8OcezMxm7ibyzu/pIlJfkwxOTA6pvs7QnbfSMQtyASS43ML61bBvEYB1p6MW56qlP8y7f26/b1m0piFJbQRIfDL0043dr0+tZoFT64WcMZLKBXZxTO/Em7q1X60zKyekCfznmZNm22c4jheEIQZVONvnoKUH8ywUWRHNCxELVcsWAqPrNNlAVqzn1d8rHk3ONh70WMt9a2wXUBuiUWp2qKnnju1qWowHfJkv353VAfczTeLJYv7 x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: qT4W1Ak0xCe60SSc7SDeJkgjcBUxfR+AKpFf6d/2Wdkrl/MHy2I+28sAqR6b9IhKMhIjj8Tc37faZNaXEeoOQ2Skpy2mXkwckhm/hd4bC3nZVgPPXi0HhejPBln8H8Y7iL2ooA7kNJj6dg0mJd/eRQ== x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-3174-20-msonline-outlook-529c7.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR04MB5981.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 58d05adb-8887-45b7-2854-08d97950d0dc X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Sep 2021 20:30:25.6920 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR04MB5629 Subject: [FFmpeg-devel] [PATCH v6 07/11] fftools/ffmpeg: Replace sub2video with subtitle frame filtering X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 21W+4iWFoUdr This commit actually enables subtitle filtering in ffmpeg by sending and receiving subtitle frames to and from a filtergraph. The heartbeat functionality from the previous sub2video implementation is retained and applied to all subtitle frames (bitmap, text, ..). The other part of sub2video functionality is retained by auto-insertion of the new graphicsub2video filter. Signed-off-by: softworkz --- fftools/ffmpeg.c | 459 ++++++++++------------ fftools/ffmpeg.h | 14 +- fftools/ffmpeg_filter.c | 198 +++++++--- fftools/ffmpeg_hw.c | 2 +- fftools/ffmpeg_opt.c | 3 +- tests/ref/fate/filter-overlay-dvdsub-2397 | 181 +++++---- tests/ref/fate/sub-dvb | 162 ++++---- tests/ref/fate/sub2video | 44 --- tests/ref/fate/sub2video_basic | 93 +++-- tests/ref/fate/sub2video_time_limited | 4 +- 10 files changed, 571 insertions(+), 589 deletions(-) diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index 5365f711e5..80ed9461a3 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -169,163 +169,6 @@ static int restore_tty; static void free_input_threads(void); #endif -/* sub2video hack: - Convert subtitles to video with alpha to insert them in filter graphs. - This is a temporary solution until libavfilter gets real subtitles support. - */ - -static int sub2video_get_blank_frame(InputStream *ist) -{ - int ret; - AVFrame *frame = ist->sub2video.frame; - - av_frame_unref(frame); - ist->sub2video.frame->width = ist->dec_ctx->width ? ist->dec_ctx->width : ist->sub2video.w; - ist->sub2video.frame->height = ist->dec_ctx->height ? ist->dec_ctx->height : ist->sub2video.h; - ist->sub2video.frame->format = AV_PIX_FMT_RGB32; - if ((ret = av_frame_get_buffer(frame, 0)) < 0) - return ret; - memset(frame->data[0], 0, frame->height * frame->linesize[0]); - return 0; -} - -static void sub2video_copy_rect(uint8_t *dst, int dst_linesize, int w, int h, - AVSubtitleRect *r) -{ - uint32_t *pal, *dst2; - uint8_t *src, *src2; - int x, y; - - if (r->type != SUBTITLE_BITMAP) { - av_log(NULL, AV_LOG_WARNING, "sub2video: non-bitmap subtitle\n"); - return; - } - if (r->x < 0 || r->x + r->w > w || r->y < 0 || r->y + r->h > h) { - av_log(NULL, AV_LOG_WARNING, "sub2video: rectangle (%d %d %d %d) overflowing %d %d\n", - r->x, r->y, r->w, r->h, w, h - ); - return; - } - - dst += r->y * dst_linesize + r->x * 4; - src = r->data[0]; - pal = (uint32_t *)r->data[1]; - for (y = 0; y < r->h; y++) { - dst2 = (uint32_t *)dst; - src2 = src; - for (x = 0; x < r->w; x++) - *(dst2++) = pal[*(src2++)]; - dst += dst_linesize; - src += r->linesize[0]; - } -} - -static void sub2video_push_ref(InputStream *ist, int64_t pts) -{ - AVFrame *frame = ist->sub2video.frame; - int i; - int ret; - - av_assert1(frame->data[0]); - ist->sub2video.last_pts = frame->pts = pts; - for (i = 0; i < ist->nb_filters; i++) { - ret = av_buffersrc_add_frame_flags(ist->filters[i]->filter, frame, - AV_BUFFERSRC_FLAG_KEEP_REF | - AV_BUFFERSRC_FLAG_PUSH); - if (ret != AVERROR_EOF && ret < 0) - av_log(NULL, AV_LOG_WARNING, "Error while add the frame to buffer source(%s).\n", - av_err2str(ret)); - } -} - -void sub2video_update(InputStream *ist, int64_t heartbeat_pts, AVSubtitle *sub) -{ - AVFrame *frame = ist->sub2video.frame; - int8_t *dst; - int dst_linesize; - int num_rects, i; - int64_t pts, end_pts; - - if (!frame) - return; - if (sub) { - pts = av_rescale_q(sub->pts + sub->start_display_time * 1000LL, - AV_TIME_BASE_Q, ist->st->time_base); - end_pts = av_rescale_q(sub->pts + sub->end_display_time * 1000LL, - AV_TIME_BASE_Q, ist->st->time_base); - num_rects = sub->num_rects; - } else { - /* If we are initializing the system, utilize current heartbeat - PTS as the start time, and show until the following subpicture - is received. Otherwise, utilize the previous subpicture's end time - as the fall-back value. */ - pts = ist->sub2video.initialize ? - heartbeat_pts : ist->sub2video.end_pts; - end_pts = INT64_MAX; - num_rects = 0; - } - if (sub2video_get_blank_frame(ist) < 0) { - av_log(ist->dec_ctx, AV_LOG_ERROR, - "Impossible to get a blank canvas.\n"); - return; - } - dst = frame->data [0]; - dst_linesize = frame->linesize[0]; - for (i = 0; i < num_rects; i++) - sub2video_copy_rect(dst, dst_linesize, frame->width, frame->height, sub->rects[i]); - sub2video_push_ref(ist, pts); - ist->sub2video.end_pts = end_pts; - ist->sub2video.initialize = 0; -} - -static void sub2video_heartbeat(InputStream *ist, int64_t pts) -{ - InputFile *infile = input_files[ist->file_index]; - int i, j, nb_reqs; - int64_t pts2; - - /* When a frame is read from a file, examine all sub2video streams in - the same file and send the sub2video frame again. Otherwise, decoded - video frames could be accumulating in the filter graph while a filter - (possibly overlay) is desperately waiting for a subtitle frame. */ - for (i = 0; i < infile->nb_streams; i++) { - InputStream *ist2 = input_streams[infile->ist_index + i]; - if (!ist2->sub2video.frame) - continue; - /* subtitles seem to be usually muxed ahead of other streams; - if not, subtracting a larger time here is necessary */ - pts2 = av_rescale_q(pts, ist->st->time_base, ist2->st->time_base) - 1; - /* do not send the heartbeat frame if the subtitle is already ahead */ - if (pts2 <= ist2->sub2video.last_pts) - continue; - if (pts2 >= ist2->sub2video.end_pts || ist2->sub2video.initialize) - /* if we have hit the end of the current displayed subpicture, - or if we need to initialize the system, update the - overlayed subpicture and its start/end times */ - sub2video_update(ist2, pts2 + 1, NULL); - for (j = 0, nb_reqs = 0; j < ist2->nb_filters; j++) - nb_reqs += av_buffersrc_get_nb_failed_requests(ist2->filters[j]->filter); - if (nb_reqs) - sub2video_push_ref(ist2, pts2); - } -} - -static void sub2video_flush(InputStream *ist) -{ - int i; - int ret; - - if (ist->sub2video.end_pts < INT64_MAX) - sub2video_update(ist, INT64_MAX, NULL); - for (i = 0; i < ist->nb_filters; i++) { - ret = av_buffersrc_add_frame(ist->filters[i]->filter, NULL); - if (ret != AVERROR_EOF && ret < 0) - av_log(NULL, AV_LOG_WARNING, "Flush the frame error.\n"); - } -} - -/* end of sub2video hack */ - static void term_exit_sigsafe(void) { #if HAVE_TERMIOS_H @@ -526,7 +369,6 @@ static void ffmpeg_cleanup(int ret) avfilter_graph_free(&fg->graph); for (j = 0; j < fg->nb_inputs; j++) { InputFilter *ifilter = fg->inputs[j]; - struct InputStream *ist = ifilter->ist; while (av_fifo_size(ifilter->frame_queue)) { AVFrame *frame; @@ -536,15 +378,6 @@ static void ffmpeg_cleanup(int ret) } av_fifo_freep(&ifilter->frame_queue); av_freep(&ifilter->displaymatrix); - if (ist->sub2video.sub_queue) { - while (av_fifo_size(ist->sub2video.sub_queue)) { - AVSubtitle sub; - av_fifo_generic_read(ist->sub2video.sub_queue, - &sub, sizeof(sub), NULL); - avsubtitle_free(&sub); - } - av_fifo_freep(&ist->sub2video.sub_queue); - } av_buffer_unref(&ifilter->hw_frames_ctx); av_freep(&ifilter->name); av_freep(&fg->inputs[j]); @@ -636,12 +469,14 @@ static void ffmpeg_cleanup(int ret) av_frame_free(&ist->filter_frame); av_packet_free(&ist->pkt); av_dict_free(&ist->decoder_opts); - avsubtitle_free(&ist->prev_sub.subtitle); - av_frame_free(&ist->sub2video.frame); + av_frame_free(&ist->prev_sub.subtitle); av_freep(&ist->filters); av_freep(&ist->hwaccel_device); av_freep(&ist->dts_buffer); + av_frame_free(&ist->subtitle_heartbeat.recent_sub); + av_freep(&ist->subtitle_heartbeat.header); + avcodec_free_context(&ist->dec_ctx); av_freep(&input_streams[i]); @@ -1061,17 +896,21 @@ error: exit_program(1); } -static void do_subtitle_out(OutputFile *of, - OutputStream *ost, - AVSubtitle *sub) +static void do_subtitle_out(OutputFile *of, OutputStream *ost, AVFrame *frame) { - int subtitle_out_max_size = 1024 * 1024; + const int subtitle_out_max_size = 1024 * 1024; int subtitle_out_size, nb, i; AVCodecContext *enc; AVPacket *pkt = ost->pkt; + AVSubtitle out_sub = { 0 }; int64_t pts; - if (sub->pts == AV_NOPTS_VALUE) { + if (!frame) + return; + + av_log(NULL, AV_LOG_TRACE, "do_subtitle_out: sub->pts: %"PRId64" frame->pts: %"PRId64"\n", frame->subtitle_pts, frame->pts); + + if (frame->subtitle_pts == AV_NOPTS_VALUE) { av_log(NULL, AV_LOG_ERROR, "Subtitle packets must have a pts\n"); if (exit_on_error) exit_program(1); @@ -1097,30 +936,35 @@ static void do_subtitle_out(OutputFile *of, nb = 1; /* shift timestamp to honor -ss and make check_recording_time() work with -t */ - pts = sub->pts; + pts = frame->subtitle_pts; if (output_files[ost->file_index]->start_time != AV_NOPTS_VALUE) pts -= output_files[ost->file_index]->start_time; + + ost->sync_opts = av_rescale_q(pts, AV_TIME_BASE_Q, enc->time_base); + if (!check_recording_time(ost)) + return; + + frame->subtitle_pts = pts; + // subtitle_start_time is required to be 0 + frame->subtitle_pts += av_rescale_q(frame->subtitle_start_time, (AVRational){ 1, 1000 }, AV_TIME_BASE_Q); + frame->subtitle_end_time -= frame->subtitle_start_time; + frame->subtitle_start_time = 0; + + av_frame_get_subtitle(&out_sub, frame); + for (i = 0; i < nb; i++) { - unsigned save_num_rects = sub->num_rects; + const unsigned save_num_rects = out_sub.num_rects; - ost->sync_opts = av_rescale_q(pts, AV_TIME_BASE_Q, enc->time_base); - if (!check_recording_time(ost)) - return; + ost->frames_encoded++; - sub->pts = pts; - // start_display_time is required to be 0 - sub->pts += av_rescale_q(sub->start_display_time, (AVRational){ 1, 1000 }, AV_TIME_BASE_Q); - sub->end_display_time -= sub->start_display_time; - sub->start_display_time = 0; if (i == 1) - sub->num_rects = 0; + out_sub.num_rects = 0; - ost->frames_encoded++; + subtitle_out_size = avcodec_encode_subtitle(enc, subtitle_out, subtitle_out_max_size, &out_sub); - subtitle_out_size = avcodec_encode_subtitle(enc, subtitle_out, - subtitle_out_max_size, sub); if (i == 1) - sub->num_rects = save_num_rects; + out_sub.num_rects = save_num_rects; + if (subtitle_out_size < 0) { av_log(NULL, AV_LOG_FATAL, "Subtitle encoding failed\n"); exit_program(1); @@ -1129,19 +973,21 @@ static void do_subtitle_out(OutputFile *of, av_packet_unref(pkt); pkt->data = subtitle_out; pkt->size = subtitle_out_size; - pkt->pts = av_rescale_q(sub->pts, AV_TIME_BASE_Q, ost->mux_timebase); - pkt->duration = av_rescale_q(sub->end_display_time, (AVRational){ 1, 1000 }, ost->mux_timebase); + pkt->pts = av_rescale_q(frame->subtitle_pts, AV_TIME_BASE_Q, ost->mux_timebase); + pkt->duration = av_rescale_q(frame->subtitle_end_time, (AVRational){ 1, 1000 }, ost->mux_timebase); if (enc->codec_id == AV_CODEC_ID_DVB_SUBTITLE) { /* XXX: the pts correction is handled here. Maybe handling it in the codec would be better */ if (i == 0) - pkt->pts += av_rescale_q(sub->start_display_time, (AVRational){ 1, 1000 }, ost->mux_timebase); + pkt->pts += av_rescale_q(frame->subtitle_start_time, (AVRational){ 1, 1000 }, ost->mux_timebase); else - pkt->pts += av_rescale_q(sub->end_display_time, (AVRational){ 1, 1000 }, ost->mux_timebase); + pkt->pts += av_rescale_q(frame->subtitle_end_time, (AVRational){ 1, 1000 }, ost->mux_timebase); } pkt->dts = pkt->pts; output_packet(of, pkt, ost, 0); } + + avsubtitle_free(&out_sub); } static void do_video_out(OutputFile *of, @@ -1578,8 +1424,11 @@ static int reap_filters(int flush) } do_audio_out(of, ost, filtered_frame); break; + case AVMEDIA_TYPE_SUBTITLE: + + do_subtitle_out(of, ost, filtered_frame); + break; default: - // TODO support subtitle filters av_assert0(0); } @@ -2175,7 +2024,8 @@ static int ifilter_has_all_input_formats(FilterGraph *fg) int i; for (i = 0; i < fg->nb_inputs; i++) { if (fg->inputs[i]->format < 0 && (fg->inputs[i]->type == AVMEDIA_TYPE_AUDIO || - fg->inputs[i]->type == AVMEDIA_TYPE_VIDEO)) + fg->inputs[i]->type == AVMEDIA_TYPE_VIDEO || + fg->inputs[i]->type == AVMEDIA_TYPE_SUBTITLE)) return 0; } return 1; @@ -2279,7 +2129,7 @@ static int ifilter_send_eof(InputFilter *ifilter, int64_t pts) // the filtergraph was never configured if (ifilter->format < 0) ifilter_parameters_from_codecpar(ifilter, ifilter->ist->st->codecpar); - if (ifilter->format < 0 && (ifilter->type == AVMEDIA_TYPE_AUDIO || ifilter->type == AVMEDIA_TYPE_VIDEO)) { + if (ifilter->format < 0 && (ifilter->type == AVMEDIA_TYPE_AUDIO || ifilter->type == AVMEDIA_TYPE_VIDEO || ifilter->type == AVMEDIA_TYPE_SUBTITLE)) { av_log(NULL, AV_LOG_ERROR, "Cannot determine format of input stream %d:%d after EOF\n", ifilter->ist->file_index, ifilter->ist->st->index); return AVERROR_INVALIDDATA; } @@ -2534,83 +2384,178 @@ fail: return err < 0 ? err : ret; } -static int transcode_subtitles(InputStream *ist, AVPacket *pkt, int *got_output, +static void subtitle_resend_current(InputStream *ist, int64_t heartbeat_pts) +{ + AVFrame *frame; + int64_t pts, end_pts; + + if (ist->subtitle_heartbeat.recent_sub) { + frame = av_frame_clone(ist->subtitle_heartbeat.recent_sub); + + pts = av_rescale_q(frame->subtitle_pts + frame->subtitle_start_time * 1000LL, AV_TIME_BASE_Q, ist->st->time_base); + end_pts = av_rescale_q(frame->subtitle_pts + frame->subtitle_end_time * 1000LL, AV_TIME_BASE_Q, ist->st->time_base); + } + else { + frame = av_frame_alloc(); + if (!frame) { + av_log(ist->dec_ctx, AV_LOG_ERROR, "Unable to alloc frame (out of memory).\n"); + return; + } + + frame->type = AVMEDIA_TYPE_SUBTITLE; + frame->format = av_get_subtitle_format_from_codecdesc(ist->dec_ctx->codec_descriptor); + + frame->width = ist->subtitle_heartbeat.w; + frame->height = ist->subtitle_heartbeat.h; + + + pts = ist->subtitle_heartbeat.end_pts <= 0 ? heartbeat_pts : ist->subtitle_heartbeat.end_pts; + end_pts = INT64_MAX; + + frame->subtitle_pts = av_rescale_q(pts, ist->st->time_base, AV_TIME_BASE_Q); + frame->subtitle_end_time = 1000; + } + + frame->pts = pts; + ist->subtitle_heartbeat.last_pts = pts; + ist->subtitle_heartbeat.end_pts = end_pts; + + send_frame_to_filters(ist, frame); +} + +static void subtitle_heartbeat(InputStream *ist, int64_t pts) +{ + const InputFile *infile = input_files[ist->file_index]; + int i; + int64_t pts2; + + /* When a frame is read from a file, examine all subtitle streams in + the same file and send the subtitle frame again. Otherwise, decoded + video frames could be accumulating in the filter graph while a filter + (possibly overlay) is desperately waiting for a subtitle frame. */ + for (i = 0; i < infile->nb_streams; i++) { + InputStream *ist2 = input_streams[infile->ist_index + i]; + if (!ist2->subtitle_heartbeat.is_active) + continue; + /* subtitles seem to be usually muxed ahead of other streams; + if not, subtracting a larger time here is necessary */ + pts2 = av_rescale_q(pts, ist->st->time_base, ist2->st->time_base) - 1; + /* do not send the heartbeat frame if the subtitle is already ahead */ + if (pts2 <= ist2->subtitle_heartbeat.last_pts) + continue; + if (pts2 >= ist2->subtitle_heartbeat.end_pts) { + /* if we have hit the end of the current displayed subpicture, + or if we need to initialize the system, update the + overlayed subpicture and its start/end times */ + if (ist2->subtitle_heartbeat.recent_sub) + av_frame_free(&ist2->subtitle_heartbeat.recent_sub); + subtitle_resend_current(ist2, pts2 + 1); + } + //for (j = 0, nb_reqs = 0; j < ist2->nb_filters; j++) + // nb_reqs += av_buffersrc_get_nb_failed_requests(ist2->filters[j]->filter); + //if (nb_reqs) + // subtitle_heartbeat_resend_current(ist2, pts2); + } +} + +static int decode_subtitles(InputStream *ist, AVPacket *pkt, int *got_output, int *decode_failed) { - AVSubtitle subtitle; - int free_sub = 1; - int i, ret = avcodec_decode_subtitle2(ist->dec_ctx, - &subtitle, got_output, pkt); + AVFrame *decoded_frame; + AVCodecContext *avctx = ist->dec_ctx; + int i = 0, ret = 0, err = 0; + int64_t pts, end_pts; - check_decode_result(NULL, got_output, ret); + if (!ist->decoded_frame && !(ist->decoded_frame = av_frame_alloc())) + return AVERROR(ENOMEM); + if (!ist->filter_frame && !(ist->filter_frame = av_frame_alloc())) + return AVERROR(ENOMEM); + + decoded_frame = ist->decoded_frame; + decoded_frame->type = AVMEDIA_TYPE_SUBTITLE; + decoded_frame->format = av_get_subtitle_format_from_codecdesc(avctx->codec_descriptor); + + if (!ist->subtitle_heartbeat.header && avctx->subtitle_header) + ist->subtitle_heartbeat.header = av_strdup((char *)avctx->subtitle_header); + + ret = avcodec_decode_subtitle3(avctx, decoded_frame, got_output, pkt); + decoded_frame->subtitle_header = ist->subtitle_heartbeat.header; + + if (ret != AVERROR_EOF) + check_decode_result(NULL, got_output, ret); if (ret < 0 || !*got_output) { *decode_failed = 1; - if (!pkt->size) - sub2video_flush(ist); + if (!pkt->size) { + // Flush + for (i = 0; i < ist->nb_filters; i++) { + ret = av_buffersrc_add_frame(ist->filters[i]->filter, NULL); + if (ret != AVERROR_EOF && ret < 0) + av_log(NULL, AV_LOG_WARNING, "Flush the frame error.\n"); + } + } return ret; } if (ist->fix_sub_duration) { int end = 1; - if (ist->prev_sub.got_output) { - end = av_rescale(subtitle.pts - ist->prev_sub.subtitle.pts, + if (ist->prev_sub.got_output && ist->prev_sub.subtitle) { + end = av_rescale(decoded_frame->subtitle_pts - ist->prev_sub.subtitle->subtitle_pts, 1000, AV_TIME_BASE); - if (end < ist->prev_sub.subtitle.end_display_time) { - av_log(ist->dec_ctx, AV_LOG_DEBUG, + if (end < ist->prev_sub.subtitle->subtitle_end_time) { + av_log(avctx, AV_LOG_DEBUG, "Subtitle duration reduced from %"PRId32" to %d%s\n", - ist->prev_sub.subtitle.end_display_time, end, + ist->prev_sub.subtitle->subtitle_end_time, end, end <= 0 ? ", dropping it" : ""); - ist->prev_sub.subtitle.end_display_time = end; + ist->prev_sub.subtitle->subtitle_end_time = end; } } FFSWAP(int, *got_output, ist->prev_sub.got_output); FFSWAP(int, ret, ist->prev_sub.ret); - FFSWAP(AVSubtitle, subtitle, ist->prev_sub.subtitle); + FFSWAP(AVFrame*, decoded_frame, ist->prev_sub.subtitle); if (end <= 0) - goto out; + return end; } - if (!*got_output) + if (!*got_output || !decoded_frame) return ret; - if (ist->sub2video.frame) { - sub2video_update(ist, INT64_MIN, &subtitle); - } else if (ist->nb_filters) { - if (!ist->sub2video.sub_queue) - ist->sub2video.sub_queue = av_fifo_alloc(8 * sizeof(AVSubtitle)); - if (!ist->sub2video.sub_queue) - exit_program(1); - if (!av_fifo_space(ist->sub2video.sub_queue)) { - ret = av_fifo_realloc2(ist->sub2video.sub_queue, 2 * av_fifo_size(ist->sub2video.sub_queue)); - if (ret < 0) - exit_program(1); - } - av_fifo_generic_write(ist->sub2video.sub_queue, &subtitle, sizeof(subtitle), NULL); - free_sub = 0; - } + decoded_frame->type = AVMEDIA_TYPE_SUBTITLE; + decoded_frame->format = av_get_subtitle_format_from_codecdesc(avctx->codec_descriptor); - if (!subtitle.num_rects) - goto out; + if ((ret = av_frame_get_buffer2(decoded_frame, 0)) < 0) + return ret; - ist->frames_decoded++; + pts = av_rescale_q(decoded_frame->subtitle_pts + decoded_frame->subtitle_start_time * 1000LL, + AV_TIME_BASE_Q, ist->st->time_base); + end_pts = av_rescale_q(decoded_frame->subtitle_pts + decoded_frame->subtitle_end_time * 1000LL, + AV_TIME_BASE_Q, ist->st->time_base); - for (i = 0; i < nb_output_streams; i++) { - OutputStream *ost = output_streams[i]; + ist->subtitle_heartbeat.last_pts = decoded_frame->pts = pts; + ist->subtitle_heartbeat.end_pts = end_pts; - if (!ost->pkt && !(ost->pkt = av_packet_alloc())) - exit_program(1); - if (!check_output_constraints(ist, ost) || !ost->encoding_needed - || ost->enc->type != AVMEDIA_TYPE_SUBTITLE) - continue; + if (ist->nb_filters == 0) { + for (i = 0; i < nb_output_streams; i++) { + OutputStream *ost = output_streams[i]; - do_subtitle_out(output_files[ost->file_index], ost, &subtitle); + if (!ost->pkt && !(ost->pkt = av_packet_alloc())) + exit_program(1); + if (!check_output_constraints(ist, ost) || !ost->encoding_needed + || ost->enc->type != AVMEDIA_TYPE_SUBTITLE) + continue; + + do_subtitle_out(output_files[ost->file_index], ost, decoded_frame); + } } + else + err = send_frame_to_filters(ist, decoded_frame); -out: - if (free_sub) - avsubtitle_free(&subtitle); - return ret; + av_frame_free(&ist->subtitle_heartbeat.recent_sub); + ist->subtitle_heartbeat.recent_sub = av_frame_clone(decoded_frame); + + av_frame_unref(ist->filter_frame); + av_frame_unref(decoded_frame); + return err < 0 ? err : ret; } static int send_filter_eof(InputStream *ist) @@ -2718,7 +2663,7 @@ static int process_input_packet(InputStream *ist, const AVPacket *pkt, int no_eo case AVMEDIA_TYPE_SUBTITLE: if (repeating) break; - ret = transcode_subtitles(ist, avpkt, &got_output, &decode_failed); + ret = decode_subtitles(ist, avpkt, &got_output, &decode_failed); if (!pkt && ret >= 0) ret = AVERROR_EOF; av_packet_unref(avpkt); @@ -2991,8 +2936,6 @@ FF_ENABLE_DEPRECATION_WARNINGS av_log(NULL, AV_LOG_WARNING, "Warning using DVB subtitles for filtering and output at the same time is not fully supported, also see -compute_edt [0|1]\n"); } - av_dict_set(&ist->decoder_opts, "sub_text_format", "ass", AV_DICT_DONT_OVERWRITE); - /* Useful for subtitles retiming by lavf (FIXME), skipping samples in * audio, and video decoders such as cuvid or mediacodec */ ist->dec_ctx->pkt_timebase = ist->st->time_base; @@ -3590,19 +3533,14 @@ static int init_output_stream(OutputStream *ost, AVFrame *frame, } if (ist && ist->dec->type == AVMEDIA_TYPE_SUBTITLE && ost->enc->type == AVMEDIA_TYPE_SUBTITLE) { - int input_props = 0, output_props = 0; - AVCodecDescriptor const *input_descriptor = - avcodec_descriptor_get(dec->codec_id); - AVCodecDescriptor const *output_descriptor = - avcodec_descriptor_get(ost->enc_ctx->codec_id); - if (input_descriptor) - input_props = input_descriptor->props & (AV_CODEC_PROP_TEXT_SUB | AV_CODEC_PROP_BITMAP_SUB); - if (output_descriptor) - output_props = output_descriptor->props & (AV_CODEC_PROP_TEXT_SUB | AV_CODEC_PROP_BITMAP_SUB); - if (input_props && output_props && input_props != output_props) { - snprintf(error, error_len, - "Subtitle encoding currently only possible from text to text " - "or bitmap to bitmap"); + AVCodecDescriptor const *input_descriptor = avcodec_descriptor_get(dec->codec_id); + AVCodecDescriptor const *output_descriptor = avcodec_descriptor_get(ost->enc_ctx->codec_id); + const enum AVSubtitleType in_subtitle_format = output_descriptor ? av_get_subtitle_format_from_codecdesc(input_descriptor) : AV_SUBTITLE_FMT_UNKNOWN; + const enum AVSubtitleType out_subtitle_format = output_descriptor ? av_get_subtitle_format_from_codecdesc(output_descriptor) : AV_SUBTITLE_FMT_UNKNOWN; + + if (in_subtitle_format != AV_SUBTITLE_FMT_UNKNOWN && out_subtitle_format != AV_SUBTITLE_FMT_UNKNOWN + && in_subtitle_format != out_subtitle_format) { + snprintf(error, error_len, "Subtitle encoding is only possible from text to text or bitmap to bitmap"); return AVERROR_INVALIDDATA; } } @@ -4635,7 +4573,7 @@ static int process_input(int file_index) av_ts2timestr(input_files[ist->file_index]->ts_offset, &AV_TIME_BASE_Q)); } - sub2video_heartbeat(ist, pkt->pts); + subtitle_heartbeat(ist, pkt->pts); process_input_packet(ist, pkt, 0); @@ -4847,6 +4785,7 @@ static int transcode(void) /* at the end of stream, we must flush the decoder buffers */ for (i = 0; i < nb_input_streams; i++) { ist = input_streams[i]; + ist->subtitle_heartbeat.is_active = 0; if (!input_files[ist->file_index]->eof_reached) { process_input_packet(ist, NULL, 0); } diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index 30225e9ffe..69d75fae49 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -349,17 +349,17 @@ typedef struct InputStream { struct { /* previous decoded subtitle and related variables */ int got_output; int ret; - AVSubtitle subtitle; + AVFrame *subtitle; } prev_sub; - struct sub2video { + struct subtitle_heartbeat { + int is_active; int64_t last_pts; int64_t end_pts; - AVFifoBuffer *sub_queue; ///< queue of AVSubtitle* before filter init - AVFrame *frame; + AVFrame *recent_sub; int w, h; - unsigned int initialize; ///< marks if sub2video_update should force an initialization - } sub2video; + char *header; + } subtitle_heartbeat; /* decoded data from this stream goes into all those filters * currently video and audio only */ @@ -661,8 +661,6 @@ int filtergraph_is_simple(FilterGraph *fg); int init_simple_filtergraph(InputStream *ist, OutputStream *ost); int init_complex_filtergraph(FilterGraph *fg); -void sub2video_update(InputStream *ist, int64_t heartbeat_pts, AVSubtitle *sub); - int ifilter_parameters_from_frame(InputFilter *ifilter, const AVFrame *frame); int ffmpeg_parse_options(int argc, char **argv); diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c index da0d4faf54..57c28640dc 100644 --- a/fftools/ffmpeg_filter.c +++ b/fftools/ffmpeg_filter.c @@ -22,6 +22,9 @@ #include "ffmpeg.h" +#include +#include + #include "libavfilter/avfilter.h" #include "libavfilter/buffersink.h" #include "libavfilter/buffersrc.h" @@ -221,9 +224,8 @@ static void init_input_filter(FilterGraph *fg, AVFilterInOut *in) enum AVMediaType type = avfilter_pad_get_type(in->filter_ctx->input_pads, in->pad_idx); int i; - // TODO: support other filter types - if (type != AVMEDIA_TYPE_VIDEO && type != AVMEDIA_TYPE_AUDIO) { - av_log(NULL, AV_LOG_FATAL, "Only video and audio filters supported " + if (type != AVMEDIA_TYPE_VIDEO && type != AVMEDIA_TYPE_AUDIO && type != AVMEDIA_TYPE_SUBTITLE) { + av_log(NULL, AV_LOG_FATAL, "Only video, audio and subtitle filters supported " "currently.\n"); exit_program(1); } @@ -244,8 +246,9 @@ static void init_input_filter(FilterGraph *fg, AVFilterInOut *in) for (i = 0; i < s->nb_streams; i++) { enum AVMediaType stream_type = s->streams[i]->codecpar->codec_type; if (stream_type != type && - !(stream_type == AVMEDIA_TYPE_SUBTITLE && - type == AVMEDIA_TYPE_VIDEO /* sub2video hack */)) + // in the followng case we auto-insert the graphicsub2video conversion filter + // for retaining compatibility with the previous sub2video hack + !(stream_type == AVMEDIA_TYPE_SUBTITLE && type == AVMEDIA_TYPE_VIDEO)) continue; if (check_stream_specifier(s, s->streams[i], *p == ':' ? p + 1 : p) == 1) { st = s->streams[i]; @@ -416,6 +419,37 @@ static int insert_filter(AVFilterContext **last_filter, int *pad_idx, return 0; } +static int configure_output_subtitle_filter(FilterGraph *fg, OutputFilter *ofilter, AVFilterInOut *out) +{ + OutputStream *ost = ofilter->ost; + AVFilterContext *last_filter = out->filter_ctx; + int pad_idx = out->pad_idx; + int ret; + char name[255]; + + snprintf(name, sizeof(name), "out_%d_%d", ost->file_index, ost->index); + ret = avfilter_graph_create_filter(&ofilter->filter, + avfilter_get_by_name("sbuffersink"), + name, NULL, NULL, fg->graph); + + if (ret < 0) + return ret; + + ////snprintf(name, sizeof(name), "trim_out_%d_%d", + //// ost->file_index, ost->index); + ////ret = insert_trim(of->start_time, of->recording_time, + //// &last_filter, &pad_idx, name); + ////if (ret < 0) + //// return ret; + + ////ost->st->codecpar->codec_tag = MKTAG('a', 's', 's', 's'); + + if ((ret = avfilter_link(last_filter, pad_idx, ofilter->filter, 0)) < 0) + return ret; + + return 0; +} + static int configure_output_video_filter(FilterGraph *fg, OutputFilter *ofilter, AVFilterInOut *out) { char *pix_fmts; @@ -594,7 +628,8 @@ static int configure_output_audio_filter(FilterGraph *fg, OutputFilter *ofilter, int i; for (i=0; ictx->nb_streams; i++) - if (of->ctx->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_VIDEO) + if (of->ctx->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_VIDEO || + of->ctx->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_SUBTITLE) break; if (ictx->nb_streams) { @@ -628,6 +663,7 @@ static int configure_output_filter(FilterGraph *fg, OutputFilter *ofilter, switch (avfilter_pad_get_type(out->filter_ctx->output_pads, out->pad_idx)) { case AVMEDIA_TYPE_VIDEO: return configure_output_video_filter(fg, ofilter, out); case AVMEDIA_TYPE_AUDIO: return configure_output_audio_filter(fg, ofilter, out); + case AVMEDIA_TYPE_SUBTITLE: return configure_output_subtitle_filter(fg, ofilter, out); default: av_assert0(0); return 0; } } @@ -647,51 +683,103 @@ void check_filter_outputs(void) } } -static int sub2video_prepare(InputStream *ist, InputFilter *ifilter) +static int configure_input_subtitle_filter(FilterGraph *fg, InputFilter *ifilter, + AVFilterInOut *in) { - AVFormatContext *avf = input_files[ist->file_index]->ctx; - int i, w, h; + AVFilterContext *last_filter; + const AVFilter *buffer_filt = avfilter_get_by_name("sbuffer"); + InputStream *ist = ifilter->ist; + AVBPrint args; + char name[255]; + int ret, pad_idx = 0; + int w, h; + AVBufferSrcParameters *par = av_buffersrc_parameters_alloc(); + + if (!par) + return AVERROR(ENOMEM); + memset(par, 0, sizeof(*par)); + par->format = AV_PIX_FMT_NONE; + + if (ist->dec_ctx->codec_type == AVMEDIA_TYPE_AUDIO) { + av_log(NULL, AV_LOG_ERROR, "Cannot connect subtitle filter to audio input\n"); + ret = AVERROR(EINVAL); + goto fail; + } + + if (ist->dec_ctx->codec_type == AVMEDIA_TYPE_VIDEO) { + av_log(NULL, AV_LOG_ERROR, "Cannot connect subtitle filter to video input\n"); + ret = AVERROR(EINVAL); + goto fail; + } + + if (!ist->subtitle_heartbeat.header && ist->dec_ctx->subtitle_header) + ist->subtitle_heartbeat.header = av_strdup((char *)ist->dec_ctx->subtitle_header); + + ist->subtitle_heartbeat.is_active = 1; - /* Compute the size of the canvas for the subtitles stream. - If the subtitles codecpar has set a size, use it. Otherwise use the - maximum dimensions of the video streams in the same file. */ w = ifilter->width; h = ifilter->height; + if (!(w && h)) { - for (i = 0; i < avf->nb_streams; i++) { - if (avf->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_VIDEO) { - w = FFMAX(w, avf->streams[i]->codecpar->width); - h = FFMAX(h, avf->streams[i]->codecpar->height); - } - } - if (!(w && h)) { - w = FFMAX(w, 720); - h = FFMAX(h, 576); - } - av_log(avf, AV_LOG_INFO, "sub2video: using %dx%d canvas\n", w, h); + w = ist->dec_ctx->width; + h = ist->dec_ctx->height; } - ist->sub2video.w = ifilter->width = w; - ist->sub2video.h = ifilter->height = h; - ifilter->width = ist->dec_ctx->width ? ist->dec_ctx->width : ist->sub2video.w; - ifilter->height = ist->dec_ctx->height ? ist->dec_ctx->height : ist->sub2video.h; + if (!(w && h) && ist->dec_ctx->subtitle_header) { + ASSSplitContext *ass_ctx = ff_ass_split((char *)ist->dec_ctx->subtitle_header); + ASS *ass = (ASS *)ass_ctx; + w = ass->script_info.play_res_x; + h = ass->script_info.play_res_y; + ff_ass_split_free(ass_ctx); + } - /* rectangles are AV_PIX_FMT_PAL8, but we have no guarantee that the - palettes for all rectangles are identical or compatible */ - ifilter->format = AV_PIX_FMT_RGB32; + ist->subtitle_heartbeat.w = w; + ist->subtitle_heartbeat.h = h; + av_log(ifilter, AV_LOG_INFO, "subtitle input filter: decoding size %dx%d\n", ist->subtitle_heartbeat.w, ist->subtitle_heartbeat.h); - ist->sub2video.frame = av_frame_alloc(); - if (!ist->sub2video.frame) - return AVERROR(ENOMEM); - ist->sub2video.last_pts = INT64_MIN; - ist->sub2video.end_pts = INT64_MIN; + ifilter->width = w; + ifilter->height = h; + + ist->subtitle_heartbeat.last_pts = INT64_MIN; + + snprintf(name, sizeof(name), "graph %d subtitle input from stream %d:%d", fg->index, + ist->file_index, ist->st->index); + + + av_bprint_init(&args, 0, AV_BPRINT_SIZE_AUTOMATIC); + av_bprintf(&args, + "subtitle_type=%d:time_base=%d/%d:", + ifilter->format, + ist->st->time_base.num, ist->st->time_base.den); + if ((ret = avfilter_graph_create_filter(&ifilter->filter, buffer_filt, name, + args.str, NULL, fg->graph)) < 0) + goto fail; - /* sub2video structure has been (re-)initialized. - Mark it as such so that the system will be - initialized with the first received heartbeat. */ - ist->sub2video.initialize = 1; + par->hw_frames_ctx = ifilter->hw_frames_ctx; + par->format = ifilter->format; + par->width = ifilter->width; + par->height = ifilter->height; + + ret = av_buffersrc_parameters_set(ifilter->filter, par); + if (ret < 0) + goto fail; + av_freep(&par); + last_filter = ifilter->filter; + + if (in->filter_ctx->input_pads[in->pad_idx].type == AVMEDIA_TYPE_VIDEO) { + ret = insert_filter(&last_filter, &pad_idx, "graphicsub2video", NULL); + if (ret < 0) + return ret; + } + + if ((ret = avfilter_link(last_filter, 0, in->filter_ctx, in->pad_idx)) < 0) + return ret; return 0; +fail: + av_freep(&par); + + return ret; } static int configure_input_video_filter(FilterGraph *fg, InputFilter *ifilter, @@ -709,8 +797,15 @@ static int configure_input_video_filter(FilterGraph *fg, InputFilter *ifilter, char name[255]; int ret, pad_idx = 0; int64_t tsoffset = 0; - AVBufferSrcParameters *par = av_buffersrc_parameters_alloc(); + AVBufferSrcParameters *par; + if (ist->dec_ctx->codec_type == AVMEDIA_TYPE_SUBTITLE) { + // Automatically insert conversion filter to retain compatibility + // with sub2video command lines + return configure_input_subtitle_filter(fg, ifilter, in); + } + + par = av_buffersrc_parameters_alloc(); if (!par) return AVERROR(ENOMEM); memset(par, 0, sizeof(*par)); @@ -725,12 +820,6 @@ static int configure_input_video_filter(FilterGraph *fg, InputFilter *ifilter, if (!fr.num) fr = av_guess_frame_rate(input_files[ist->file_index]->ctx, ist->st, NULL); - if (ist->dec_ctx->codec_type == AVMEDIA_TYPE_SUBTITLE) { - ret = sub2video_prepare(ist, ifilter); - if (ret < 0) - goto fail; - } - sar = ifilter->sample_aspect_ratio; if(!sar.den) sar = (AVRational){0,1}; @@ -742,7 +831,7 @@ static int configure_input_video_filter(FilterGraph *fg, InputFilter *ifilter, tb.num, tb.den, sar.num, sar.den); if (fr.num && fr.den) av_bprintf(&args, ":frame_rate=%d/%d", fr.num, fr.den); - snprintf(name, sizeof(name), "graph %d input from stream %d:%d", fg->index, + snprintf(name, sizeof(name), "graph %d video input from stream %d:%d", fg->index, ist->file_index, ist->st->index); @@ -961,6 +1050,7 @@ static int configure_input_filter(FilterGraph *fg, InputFilter *ifilter, switch (avfilter_pad_get_type(in->filter_ctx->input_pads, in->pad_idx)) { case AVMEDIA_TYPE_VIDEO: return configure_input_video_filter(fg, ifilter, in); case AVMEDIA_TYPE_AUDIO: return configure_input_audio_filter(fg, ifilter, in); + case AVMEDIA_TYPE_SUBTITLE: return configure_input_subtitle_filter(fg, ifilter, in); default: av_assert0(0); return 0; } } @@ -1128,19 +1218,6 @@ int configure_filtergraph(FilterGraph *fg) } } - /* process queued up subtitle packets */ - for (i = 0; i < fg->nb_inputs; i++) { - InputStream *ist = fg->inputs[i]->ist; - if (ist->sub2video.sub_queue && ist->sub2video.frame) { - while (av_fifo_size(ist->sub2video.sub_queue)) { - AVSubtitle tmp; - av_fifo_generic_read(ist->sub2video.sub_queue, &tmp, sizeof(tmp), NULL); - sub2video_update(ist, INT64_MIN, &tmp); - avsubtitle_free(&tmp); - } - } - } - return 0; fail: @@ -1163,6 +1240,7 @@ int ifilter_parameters_from_frame(InputFilter *ifilter, const AVFrame *frame) ifilter->sample_rate = frame->sample_rate; ifilter->channels = frame->channels; ifilter->channel_layout = frame->channel_layout; + ifilter->type = frame->type; av_freep(&ifilter->displaymatrix); sd = av_frame_get_side_data(frame, AV_FRAME_DATA_DISPLAYMATRIX); diff --git a/fftools/ffmpeg_hw.c b/fftools/ffmpeg_hw.c index 14e702bd92..be69d54aaf 100644 --- a/fftools/ffmpeg_hw.c +++ b/fftools/ffmpeg_hw.c @@ -449,7 +449,7 @@ int hw_device_setup_for_encode(OutputStream *ost) AVBufferRef *frames_ref = NULL; int i; - if (ost->filter) { + if (ost->filter && ost->filter->filter) { frames_ref = av_buffersink_get_hw_frames_ctx(ost->filter->filter); if (frames_ref && ((AVHWFramesContext*)frames_ref->data)->format == diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c index 6ca28cf974..21b5228bb2 100644 --- a/fftools/ffmpeg_opt.c +++ b/fftools/ffmpeg_opt.c @@ -2149,8 +2149,9 @@ static void init_output_filter(OutputFilter *ofilter, OptionsContext *o, switch (ofilter->type) { case AVMEDIA_TYPE_VIDEO: ost = new_video_stream(o, oc, -1); break; case AVMEDIA_TYPE_AUDIO: ost = new_audio_stream(o, oc, -1); break; + case AVMEDIA_TYPE_SUBTITLE: ost = new_subtitle_stream(o, oc, -1); break; default: - av_log(NULL, AV_LOG_FATAL, "Only video and audio filters are supported " + av_log(NULL, AV_LOG_FATAL, "Only video, audio and subtitle filters are supported " "currently.\n"); exit_program(1); } diff --git a/tests/ref/fate/filter-overlay-dvdsub-2397 b/tests/ref/fate/filter-overlay-dvdsub-2397 index 483e5fa4e0..99081b4519 100644 --- a/tests/ref/fate/filter-overlay-dvdsub-2397 +++ b/tests/ref/fate/filter-overlay-dvdsub-2397 @@ -490,368 +490,367 @@ 1, 3877, 3877, 10, 2013, 0x95a39f9c 1, 3887, 3887, 10, 2013, 0x4f7ea123 1, 3897, 3897, 10, 2013, 0x9efb9ba1 -0, 117, 117, 1, 518400, 0xbf8523da +0, 117, 117, 1, 518400, 0xc44e1d4c 1, 3907, 3907, 10, 2013, 0xf395b2cd 1, 3917, 3917, 10, 2013, 0x261a881e 1, 3927, 3927, 10, 2013, 0x7f2d9f72 1, 3937, 3937, 10, 2013, 0x0105b38d -0, 118, 118, 1, 518400, 0x41890ed6 +0, 118, 118, 1, 518400, 0xad1a084c 1, 3952, 3952, 10, 2013, 0x0e5db67e 1, 3962, 3962, 10, 2013, 0xfc9baf97 -0, 119, 119, 1, 518400, 0x588534fc +0, 119, 119, 1, 518400, 0x52d52e73 1, 3972, 3972, 10, 2013, 0x8e02a1b1 1, 3982, 3982, 10, 2013, 0x6eecaac8 1, 3992, 3992, 10, 2013, 0xf5558f0c 1, 4002, 4002, 10, 2013, 0x512ba99b -0, 120, 120, 1, 518400, 0x2145ebc1 +0, 120, 120, 1, 518400, 0xc732e546 1, 4012, 4012, 10, 2013, 0x932b9932 1, 4022, 4022, 10, 2013, 0xc01ea987 -0, 121, 121, 1, 518400, 0x28bca595 +0, 121, 121, 1, 518400, 0xc36f9f14 1, 4038, 4038, 10, 2013, 0x10879cf7 1, 4048, 4048, 10, 2013, 0x90679338 1, 4058, 4058, 10, 2013, 0x077d8a9e 1, 4068, 4068, 10, 2013, 0x969fa57c -0, 122, 122, 1, 518400, 0x77dc951e +0, 122, 122, 1, 518400, 0x78428e8b 1, 4078, 4078, 10, 2013, 0xe049ab07 1, 4088, 4088, 10, 2013, 0xf535b3b3 1, 4098, 4098, 10, 2013, 0xfe76bd37 -0, 123, 123, 1, 518400, 0xe8924c17 +0, 123, 123, 1, 518400, 0xf0f8458d 1, 4108, 4108, 10, 2013, 0xde79ad8c 1, 4123, 4123, 10, 2013, 0xe89b9c47 1, 4133, 4133, 10, 2013, 0xc570b0f0 -0, 124, 124, 1, 518400, 0xadb4cccc +0, 124, 124, 1, 518400, 0x7083c653 1, 4143, 4143, 10, 2013, 0xee709cd9 1, 4153, 4153, 10, 2013, 0xcfe5afab 1, 4163, 4163, 10, 2013, 0x98ff8ce4 -0, 125, 125, 1, 518400, 0x1d7b56ac +0, 125, 125, 1, 518400, 0xa105502c 1, 4173, 4173, 10, 2013, 0x9d19b44c 1, 4183, 4183, 10, 2013, 0x4349917a 1, 4193, 4193, 10, 2013, 0xbf54a59a -0, 126, 126, 1, 518400, 0xad5739a4 +0, 126, 126, 1, 518400, 0xd411331a 1, 4208, 4208, 10, 2013, 0xc4a399e0 1, 4218, 4218, 10, 2013, 0x1bf58ff0 1, 4228, 4228, 10, 2013, 0x3518ac56 -0, 127, 127, 1, 518400, 0x2733d35a +0, 127, 127, 1, 518400, 0x83b0ccdb 1, 4238, 4238, 10, 2013, 0xcd38c1de 1, 4248, 4248, 10, 2013, 0xbe7d9c4d 1, 4258, 4258, 10, 2013, 0xe113a306 1, 4268, 4268, 10, 2013, 0x083197ea -0, 128, 128, 1, 518400, 0x78e76da2 +0, 128, 128, 1, 518400, 0xa9be671a 1, 4278, 4278, 10, 2013, 0x1929b1eb 1, 4294, 4294, 10, 2013, 0x5d6ea5af 1, 4304, 4304, 10, 2013, 0x05519d53 -0, 129, 129, 1, 518400, 0x6c076013 +0, 129, 129, 1, 518400, 0xaeb75983 1, 4314, 4314, 10, 2013, 0x5773b380 1, 4324, 4324, 10, 2013, 0xaa70a8f5 1, 4334, 4334, 10, 2013, 0x990db0ec -0, 130, 130, 1, 518400, 0x7854f2b1 +0, 130, 130, 1, 518400, 0x81f8ec13 1, 4344, 4344, 10, 2013, 0x91d3a623 1, 4354, 4354, 10, 2013, 0xc91f9824 1, 4364, 4364, 10, 2013, 0x1d058abf -0, 131, 131, 1, 518400, 0xd2ae1ecd +0, 131, 131, 1, 518400, 0x8aaa1839 1, 4379, 4379, 10, 2013, 0x8de1b8d5 1, 4389, 4389, 10, 2013, 0x7872b06b 1, 4399, 4399, 10, 2013, 0xa084c203 -0, 132, 132, 1, 518400, 0xf5eab38d +0, 132, 132, 1, 518400, 0xc98bacf5 1, 4409, 4409, 10, 2013, 0xff90ae8d 1, 4419, 4419, 10, 2013, 0x61dead8e 1, 4429, 4429, 10, 2013, 0xee76b284 -0, 133, 133, 1, 518400, 0x994d3e9c +0, 133, 133, 1, 518400, 0x31083804 1, 4439, 4439, 10, 2013, 0xe888af7f 1, 4449, 4449, 10, 2013, 0x5d57b115 1, 4464, 4464, 10, 2013, 0xcdbfb1d0 -0, 134, 134, 1, 518400, 0x95ab705a +0, 134, 134, 1, 518400, 0x540a69dc 1, 4474, 4474, 10, 2013, 0x2e28a952 1, 4484, 4484, 10, 2013, 0x4795a994 1, 4494, 4494, 10, 2013, 0x7e7ea304 1, 4504, 4504, 10, 2013, 0x9502c1e1 -0, 135, 135, 1, 518400, 0x3c83c5ce +0, 135, 135, 1, 518400, 0x80d3bf46 1, 4514, 4514, 10, 2013, 0xf7c78ab2 1, 4524, 4524, 10, 2013, 0x24049816 1, 4534, 4534, 10, 2013, 0x52089dcf -0, 136, 136, 1, 518400, 0xfa22c508 +0, 136, 136, 1, 518400, 0x2967be7f 1, 4550, 4550, 10, 2013, 0x2150a0b1 1, 4560, 4560, 10, 2013, 0x3c2e9b93 1, 4570, 4570, 10, 2013, 0x491f932b -0, 137, 137, 1, 518400, 0xddda1712 +0, 137, 137, 1, 518400, 0x5a3b1092 1, 4580, 4580, 10, 2013, 0x31359cf8 1, 4590, 4590, 10, 2013, 0x1b00ac3f 1, 4600, 4600, 10, 2013, 0x8d7ab3cb -0, 138, 138, 1, 518400, 0x985a3b93 +0, 138, 138, 1, 518400, 0x8741350b 1, 4610, 4610, 10, 2013, 0xb2c2a4de 1, 4620, 4620, 10, 2013, 0x80a4abf2 1, 4635, 4635, 10, 2013, 0x0701a4ee -0, 139, 139, 1, 518400, 0xea63c5e7 +0, 139, 139, 1, 518400, 0xd5a9bf60 1, 4645, 4645, 10, 2013, 0xdc1ba5bc 1, 4655, 4655, 10, 2013, 0x6083a8a4 1, 4665, 4665, 10, 2013, 0x6226ad45 -0, 140, 140, 1, 518400, 0xef64983d +0, 140, 140, 1, 518400, 0xc05f91ba 1, 4675, 4675, 10, 2013, 0x2732a205 1, 4685, 4685, 10, 2013, 0x0f62a0d3 1, 4695, 4695, 10, 2013, 0xc1799249 -0, 141, 141, 1, 518400, 0x747bb193 +0, 141, 141, 1, 518400, 0x3fdaab0b 1, 4705, 4705, 10, 2013, 0xbccfa9c8 1, 4720, 4720, 10, 2013, 0xded096e7 1, 4730, 4730, 10, 2013, 0x7f0daf43 -0, 142, 142, 1, 518400, 0xb8748862 +0, 142, 142, 1, 518400, 0xab7281d9 1, 4740, 4740, 10, 2013, 0xc47ea682 1, 4750, 4750, 10, 2013, 0x5a72b07a 1, 4760, 4760, 10, 2013, 0x386faa8c 1, 4770, 4770, 10, 2013, 0xf9919a91 -0, 143, 143, 1, 518400, 0xaab55a5f +0, 143, 143, 1, 518400, 0xc80053d6 1, 4780, 4780, 10, 2013, 0x4908897e 1, 4790, 4790, 10, 2013, 0x4882b594 -0, 144, 144, 1, 518400, 0x7b468add +0, 144, 144, 1, 518400, 0x6526845c 1, 4806, 4806, 10, 2013, 0x113e98d1 1, 4816, 4816, 10, 2013, 0x5098b30d 1, 4826, 4826, 10, 2013, 0x0ef7b857 1, 4836, 4836, 10, 2013, 0x216ea176 -0, 145, 145, 1, 518400, 0xf2078707 +0, 145, 145, 1, 518400, 0x1b788089 1, 4846, 4846, 10, 2013, 0xf906944a 1, 4856, 4856, 10, 2013, 0xee9b92fb 1, 4866, 4866, 10, 2013, 0xd6029209 -0, 146, 146, 1, 518400, 0x6a2d931e +0, 146, 146, 1, 518400, 0xfa8e8ca9 1, 4876, 4876, 10, 2013, 0x2256a12e 1, 4891, 4891, 10, 2013, 0x89de8e4a 1, 4901, 4901, 10, 2013, 0x0bf0a584 -0, 147, 147, 1, 518400, 0xbbe3c417 +0, 147, 147, 1, 518400, 0xb278bda1 1, 4911, 4911, 10, 2013, 0x6a5ebd58 1, 4921, 4921, 10, 2013, 0x3edd9aa4 1, 4931, 4931, 10, 2013, 0xbd66ac26 -0, 148, 148, 1, 518400, 0x6294e449 +0, 148, 148, 1, 518400, 0xb0c3ddca 1, 4941, 4941, 10, 2013, 0x313896ea 1, 4951, 4951, 10, 2013, 0x6b83a6a0 1, 4961, 4961, 10, 2013, 0x9aafb109 -0, 149, 149, 1, 518400, 0xa05721e7 +0, 149, 149, 1, 518400, 0x10351b53 1, 4976, 4976, 10, 2013, 0x5192a85a 1, 4986, 4986, 10, 2013, 0x1f919f79 1, 4996, 4996, 10, 2013, 0xc0799c40 -0, 150, 150, 1, 518400, 0x37749183 +0, 150, 150, 1, 518400, 0xc1408aee 1, 5006, 5006, 10, 2013, 0x2988bcd8 1, 5016, 5016, 10, 2013, 0x1482913a 1, 5026, 5026, 10, 2013, 0x74da9a94 1, 5036, 5036, 10, 2013, 0x763eb709 -0, 151, 151, 1, 518400, 0xf9d9dca0 +0, 151, 151, 1, 518400, 0xf016d615 1, 5046, 5046, 10, 2013, 0x1285b405 1, 5062, 5062, 10, 2013, 0xb6ab9dfc -0, 152, 152, 1, 518400, 0x5f8ccf08 +0, 152, 152, 1, 518400, 0xa768c892 1, 5072, 5072, 10, 2013, 0xe4c8bf19 1, 5082, 5082, 10, 2013, 0xabbbade8 1, 5092, 5092, 10, 2013, 0xf8b69d89 1, 5102, 5102, 10, 2013, 0xce04a866 -0, 153, 153, 1, 518400, 0x7303f77b +0, 153, 153, 1, 518400, 0x11c3f11e 1, 5112, 5112, 10, 2013, 0x07528abf 1, 5122, 5122, 10, 2013, 0x74fb98bf 1, 5132, 5132, 10, 2013, 0x579fb1c9 -0, 154, 154, 1, 518400, 0x22b0513f +0, 154, 154, 1, 518400, 0xcd9a4ac4 1, 5147, 5147, 10, 2013, 0x7ddea2ed 1, 5157, 5157, 10, 2013, 0x296caa2c 1, 5167, 5167, 10, 2013, 0x346d9c4f -0, 155, 155, 1, 518400, 0x330485d2 +0, 155, 155, 1, 518400, 0x4ade7f5e 1, 5177, 5177, 10, 2013, 0x3e1fba15 1, 5187, 5187, 10, 2013, 0x48a2908f 1, 5197, 5197, 10, 2013, 0xc1938d09 -0, 156, 156, 1, 518400, 0x7f83daea +0, 156, 156, 1, 518400, 0x655dd46b 1, 5207, 5207, 10, 2013, 0x0e96a060 1, 5217, 5217, 10, 2013, 0x7b6a9e06 1, 5232, 5232, 10, 2013, 0x5b779d28 -0, 157, 157, 1, 518400, 0xee19f2df +0, 157, 157, 1, 518400, 0x5ab5ec61 1, 5242, 5242, 10, 2013, 0xf600aca1 1, 5252, 5252, 10, 2013, 0x3a6c9e68 1, 5262, 5262, 10, 2013, 0x0c8dc1b0 -0, 158, 158, 1, 518400, 0xb71b1c77 +0, 158, 158, 1, 518400, 0x45dc15e6 1, 5272, 5272, 10, 2013, 0x26beb245 1, 5282, 5282, 10, 2013, 0x2bc09557 1, 5292, 5292, 10, 2013, 0x27fc8845 1, 5302, 5302, 10, 2013, 0x1025aa47 -0, 159, 159, 1, 518400, 0xbffc1856 +0, 159, 159, 1, 518400, 0x201911d3 1, 5318, 5318, 10, 2013, 0xc2e69baa 1, 5328, 5328, 10, 2013, 0xdb249b92 1, 5338, 5338, 10, 2013, 0x6ccda29e -0, 160, 160, 1, 518400, 0xabc125aa +0, 160, 160, 1, 518400, 0x0fbc1f46 1, 5348, 5348, 10, 2013, 0xeaf6a1cf 1, 5358, 5358, 10, 2013, 0x509ba397 1, 5368, 5368, 10, 2013, 0xfaf8a2df -0, 161, 161, 1, 518400, 0x5ee467f8 +0, 161, 161, 1, 518400, 0x7e316179 1, 5378, 5378, 10, 2013, 0x41388f28 1, 5388, 5388, 10, 2013, 0xfe5eab39 1, 5403, 5403, 10, 2013, 0xd5ffa066 -0, 162, 162, 1, 518400, 0x6c2cf168 +0, 162, 162, 1, 518400, 0x73bbeaed 1, 5413, 5413, 10, 2013, 0x6813a30a 1, 5423, 5423, 10, 2013, 0x9be89718 1, 5433, 5433, 10, 2013, 0xaec3a27b -0, 163, 163, 1, 518400, 0x63996b26 +0, 163, 163, 1, 518400, 0x3a7c648a 1, 5446, 5446, 10, 2013, 0x579a983e 1, 5456, 5456, 10, 2013, 0x98cea21f 1, 5466, 5466, 10, 2013, 0xca77a58a -0, 164, 164, 1, 518400, 0xb34d789a +0, 164, 164, 1, 518400, 0x9f707209 1, 5476, 5476, 10, 2013, 0xcbc3b1ee 1, 5486, 5486, 10, 2013, 0xf3bb8f07 1, 5496, 5496, 10, 2013, 0x6aeebd92 -0, 165, 165, 1, 518400, 0xf49c030f +0, 165, 165, 1, 518400, 0x9f25fc5c 1, 5506, 5506, 10, 2013, 0xe955a449 1, 5516, 5516, 10, 2013, 0x9436aa5b 1, 5531, 5531, 10, 2013, 0x4f0a8f9f -0, 166, 166, 1, 518400, 0x092dc41a +0, 166, 166, 1, 518400, 0x2ed8bd75 1, 5541, 5541, 10, 2013, 0x3551b22d 1, 5551, 5551, 10, 2013, 0x0959a3d4 1, 5561, 5561, 10, 2013, 0x2ed5a11b 1, 5571, 5571, 10, 2013, 0x8f52a5c3 -0, 167, 167, 1, 518400, 0x4134c577 +0, 167, 167, 1, 518400, 0xb493becb 1, 5581, 5581, 10, 2013, 0x6552978d 1, 5591, 5591, 10, 2013, 0x7dcca0c1 1, 5601, 5601, 10, 2013, 0xbcd4a3c9 -0, 168, 168, 1, 518400, 0x261de1ed +0, 168, 168, 1, 518400, 0x7df6db57 1, 5616, 5616, 10, 2013, 0xfe41a8d8 1, 5626, 5626, 10, 2013, 0xc85aae14 1, 5636, 5636, 10, 2013, 0x1185b346 -0, 169, 169, 1, 518400, 0xcbc8566a +0, 169, 169, 1, 518400, 0x1cb94fca 1, 5646, 5646, 10, 2013, 0xf7429a0d 1, 5656, 5656, 10, 2013, 0x48c2a160 1, 5666, 5666, 10, 2013, 0x9d85a85d -0, 170, 170, 1, 518400, 0x407a5c76 +0, 170, 170, 1, 518400, 0x70db55d8 1, 5676, 5676, 10, 2013, 0xbbe89fe9 1, 5686, 5686, 10, 2013, 0xea429fe2 1, 5702, 5702, 10, 2013, 0x221ca1d4 -0, 171, 171, 1, 518400, 0x1ed73bb2 +0, 171, 171, 1, 518400, 0xc1d9351b 1, 5712, 5712, 10, 2013, 0x394b925b 1, 5722, 5722, 10, 2013, 0x556dc26f 1, 5732, 5732, 10, 2013, 0xce21a5e1 -0, 172, 172, 1, 518400, 0x8467ddb5 +0, 172, 172, 1, 518400, 0xa4b0d717 1, 5742, 5742, 10, 2013, 0xbc87c0a8 1, 5752, 5752, 10, 2013, 0xbac4ac07 1, 5762, 5762, 10, 2013, 0xdeefa4aa 1, 5772, 5772, 10, 2013, 0x1f15b362 -0, 173, 173, 1, 518400, 0x0523dc73 +0, 173, 173, 1, 518400, 0x3730d5e9 1, 5787, 5787, 10, 2013, 0x6406b7b2 1, 5797, 5797, 10, 2013, 0x8030a03d -0, 174, 174, 1, 518400, 0x81f5e895 +0, 174, 174, 1, 518400, 0x9673e1ec 1, 5807, 5807, 10, 2013, 0x0373a5b1 1, 5817, 5817, 10, 2013, 0x34ef93da 1, 5827, 5827, 10, 2013, 0x94c198fe 1, 5837, 5837, 10, 2013, 0xfefcabad -0, 175, 175, 1, 518400, 0xfc74608d +0, 175, 175, 1, 518400, 0x877959d5 1, 5847, 5847, 10, 2013, 0x8755b3ec 1, 5857, 5857, 10, 2013, 0xe436a6fd 1, 5872, 5872, 10, 2013, 0x9cf5a11e -0, 176, 176, 1, 518400, 0xc4e0dae0 +0, 176, 176, 1, 518400, 0x04f3d421 1, 5882, 5882, 10, 2013, 0x03b8a98c 1, 5892, 5892, 10, 2013, 0x6216a138 1, 5902, 5902, 10, 2013, 0xd87b9f12 -0, 177, 177, 1, 518400, 0x98367f5b +0, 177, 177, 1, 518400, 0x4f3078bc 1, 5912, 5912, 10, 2013, 0x4ce99653 1, 5922, 5922, 10, 2013, 0x6c2ea9e2 1, 5932, 5932, 10, 2013, 0x918cae4c -0, 178, 178, 1, 518400, 0x0f1a869d +0, 178, 178, 1, 518400, 0x8a127ff8 1, 5942, 5942, 10, 2013, 0xd19fa5f2 1, 5958, 5958, 10, 2013, 0x0bdda7c6 1, 5968, 5968, 10, 2013, 0x0f9ab0ca -0, 179, 179, 1, 518400, 0x45b6ccf2 +0, 179, 179, 1, 518400, 0x5864c64f 1, 5978, 5978, 10, 2013, 0x410a92b1 1, 5988, 5988, 10, 2013, 0xcfbe9d1c 1, 5998, 5998, 10, 2013, 0x59ed9d15 -0, 180, 180, 1, 518400, 0x5f9ccb77 +0, 180, 180, 1, 518400, 0xdaccc4c0 1, 6008, 6008, 10, 2013, 0x4e129e27 1, 6018, 6018, 10, 2013, 0x7bb9ac0a 1, 6028, 6028, 10, 2013, 0x826ca82b -0, 181, 181, 1, 518400, 0x5f15ea31 +0, 181, 181, 1, 518400, 0xd999e376 1, 6043, 6043, 10, 2013, 0x9ad5a74b 1, 6053, 6053, 10, 2013, 0x6c5f969a 1, 6063, 6063, 10, 2013, 0x8479a0e5 -0, 182, 182, 1, 518400, 0x86369f27 +0, 182, 182, 1, 518400, 0x8af39876 1, 6073, 6073, 10, 2013, 0x165298ef 1, 6083, 6083, 10, 2013, 0xdcadb4a1 1, 6093, 6093, 10, 2013, 0xa90e987c 1, 6103, 6103, 10, 2013, 0x1ac5b510 -0, 183, 183, 1, 518400, 0x2e27f9fa +0, 183, 183, 1, 518400, 0x5e72f33d 1, 6113, 6113, 10, 2013, 0x66728d85 1, 6128, 6128, 10, 2013, 0xe4859fc5 1, 6138, 6138, 10, 2013, 0x9901786e -0, 184, 184, 1, 518400, 0xc029a44d +0, 184, 184, 1, 518400, 0x14af9d92 1, 6148, 6148, 10, 2013, 0x6aebb406 1, 6158, 6158, 10, 2013, 0x7d13a2cc 1, 6168, 6168, 10, 2013, 0x99b7a8cc -0, 185, 185, 1, 518400, 0xebee33b0 +0, 185, 185, 1, 518400, 0x50b82d10 1, 6178, 6178, 10, 2013, 0x80b8a624 1, 6188, 6188, 10, 2013, 0xbb6aa271 1, 6198, 6198, 10, 2013, 0x17af9e4a -0, 186, 186, 1, 518400, 0x19e5494f +0, 186, 186, 1, 518400, 0xc068429c 1, 6214, 6214, 10, 2013, 0xfaf0a8f1 1, 6224, 6224, 10, 2013, 0xd6849b93 1, 6234, 6234, 10, 2013, 0xe9829669 -0, 187, 187, 1, 518400, 0xf697bd7c +0, 187, 187, 1, 518400, 0x8934b6d1 1, 6244, 6244, 10, 2013, 0x7ec98944 1, 6254, 6254, 10, 2013, 0x2b2099a4 1, 6264, 6264, 10, 2013, 0x1033a82f -0, 188, 188, 1, 518400, 0x82569002 +0, 188, 188, 1, 518400, 0x11d08947 1, 6274, 6274, 10, 2013, 0x5ec88990 1, 6284, 6284, 10, 2013, 0xd2a19b3d 1, 6299, 6299, 10, 2013, 0xa377b268 -0, 189, 189, 1, 518400, 0xfcb6d707 +0, 189, 189, 1, 518400, 0x8a27d041 1, 6309, 6309, 10, 2013, 0xfa859901 1, 6319, 6319, 10, 2013, 0x1713955a 1, 6329, 6329, 10, 2013, 0x70aab0da 1, 6339, 6339, 10, 2013, 0xcdaea422 -0, 190, 190, 1, 518400, 0x82a9662b +0, 190, 190, 1, 518400, 0xab265f7d 1, 6349, 6349, 10, 2013, 0x65c3bf80 1, 6359, 6359, 10, 2013, 0x1d75a55f 1, 6369, 6369, 10, 2013, 0xa5bea4de -0, 191, 191, 1, 518400, 0x212e16ee +0, 191, 191, 1, 518400, 0xff491040 1, 6384, 6384, 10, 2013, 0x184db71c 1, 6394, 6394, 10, 2013, 0x99858ec8 1, 6404, 6404, 10, 2013, 0xb8f2aee5 -0, 192, 192, 1, 518400, 0x2ca34dca +0, 192, 192, 1, 518400, 0x822b4704 1, 6414, 6414, 10, 2013, 0x4435b2ef 1, 6424, 6424, 10, 2013, 0x8acfa6c7 1, 6434, 6434, 10, 2013, 0x42b4c01f -0, 193, 193, 1, 518400, 0xe9ebe0a5 +0, 193, 193, 1, 518400, 0x4523d9f4 1, 6444, 6444, 10, 2013, 0x6e308c13 1, 6454, 6454, 10, 2013, 0x8227a0f6 1, 6470, 6470, 10, 2013, 0x6f12a7a2 -0, 194, 194, 1, 518400, 0x4e6b6917 +0, 194, 194, 1, 518400, 0xfc3c626e 1, 6480, 6480, 10, 2013, 0x785392be 1, 6490, 6490, 10, 2013, 0x81849c2b 1, 6500, 6500, 10, 2013, 0x5cf2af65 -0, 195, 195, 1, 518400, 0x7dcf20ab +0, 195, 195, 1, 518400, 0x237319e5 1, 6510, 6510, 10, 2013, 0x0c6ca6b4 1, 6520, 6520, 10, 2013, 0x412fab9f 1, 6530, 6530, 10, 2013, 0x08e792b4 -0, 196, 196, 1, 518400, 0xf30fac97 +0, 196, 196, 1, 518400, 0x892ca5d8 1, 6540, 6540, 10, 2013, 0x407aace3 1, 6555, 6555, 10, 2013, 0xd26bac16 1, 6565, 6565, 10, 2013, 0xac8bb295 -0, 197, 197, 1, 518400, 0xcb9fc692 +0, 197, 197, 1, 518400, 0xc4c0bfc7 1, 6575, 6575, 10, 2013, 0xddd1949c 1, 6585, 6585, 10, 2013, 0x6b26b868 1, 6595, 6595, 10, 2013, 0x5eaba587 1, 6605, 6605, 10, 2013, 0xef0793b9 -0, 198, 198, 1, 518400, 0x5d05601e +0, 198, 198, 1, 518400, 0x57c85956 1, 6615, 6615, 10, 2013, 0xdef19bd6 1, 6625, 6625, 10, 2013, 0xca98a635 -0, 199, 199, 1, 518400, 0x456c1417 +0, 199, 199, 1, 518400, 0xd6300d46 1, 6640, 6640, 10, 2013, 0x06269a5a 1, 6650, 6650, 10, 2013, 0x32cb9952 1, 6660, 6660, 10, 2013, 0xf01fa95a 1, 6670, 6670, 10, 2013, 0xefab9e55 -0, 200, 200, 1, 518400, 0x9a0fd1ad +0, 200, 200, 1, 518400, 0xd3dacaec 1, 6680, 6680, 10, 2013, 0x55a3b63a 1, 6690, 6690, 10, 2013, 0xcd36a553 1, 6700, 6700, 10, 2013, 0x2ec19877 -0, 201, 201, 1, 518400, 0x55db9716 +0, 201, 201, 1, 518400, 0x65429052 1, 6710, 6710, 10, 2013, 0xc18b924c 1, 6726, 6726, 10, 2013, 0xf132b04c 1, 6736, 6736, 10, 2013, 0x7975a44d -0, 202, 202, 1, 518400, 0x1f0d40d6 +0, 202, 202, 1, 518400, 0xec803a15 1, 6746, 6746, 10, 2013, 0x2aaf94cb 1, 6756, 6756, 10, 2013, 0x58cfa60f 1, 6766, 6766, 10, 2013, 0x9757a658 -0, 203, 203, 1, 518400, 0x73695c82 +0, 203, 203, 1, 518400, 0x7a9a55c9 1, 6776, 6776, 10, 2013, 0x67ebc0d5 1, 6786, 6786, 10, 2013, 0x3c50a70e 1, 6796, 6796, 10, 2013, 0x9c5799c6 -0, 204, 204, 1, 518400, 0xb0f10812 +0, 204, 204, 1, 518400, 0xcac30160 1, 6811, 6811, 10, 2013, 0x018d85b2 1, 6821, 6821, 10, 2013, 0x5367a956 -0, 205, 205, 1, 518400, 0xdec18505 -0, 208, 208, 1, 518400, 0xb147b947 -0, 240, 240, 1, 518400, 0x9d2e3977 +0, 205, 205, 1, 518400, 0x7e187e4f +0, 208, 208, 1, 518400, 0x0be0b2a2 diff --git a/tests/ref/fate/sub-dvb b/tests/ref/fate/sub-dvb index cbd1801d64..8f33c75d70 100644 --- a/tests/ref/fate/sub-dvb +++ b/tests/ref/fate/sub-dvb @@ -1,75 +1,93 @@ #tb 0: 1/1000000 #media_type 0: subtitle #codec_id 0: dvb_subtitle -0, 15600000, 15600000, 159000, 1168, 0xd0f89d82 -0, 15759000, 15759000, 159000, 14, 0x064900eb -0, 15760000, 15760000, 239000, 1544, 0xe60f1751 -0, 15999000, 15999000, 239000, 14, 0x0729010b -0, 16000000, 16000000, 339000, 1658, 0xbe343093 -0, 16339000, 16339000, 339000, 14, 0x0809012b -0, 16340000, 16340000, 599000, 2343, 0xc68f07ef -0, 16939000, 16939000, 599000, 14, 0x08e9014b -0, 16940000, 16940000, 459000, 2568, 0x0ee657b1 -0, 17399000, 17399000, 459000, 14, 0x09c9016b -0, 17400000, 17400000, 359000, 3422, 0xba5b63ce -0, 17759000, 17759000, 359000, 14, 0x0aa9018b -0, 17760000, 17760000, 219000, 5078, 0x95b19902 -0, 17979000, 17979000, 219000, 14, 0x0b8901ab -0, 17980000, 17980000, 959000, 5808, 0xc9717b89 -0, 18939000, 18939000, 959000, 14, 0x0c6901cb -0, 18940000, 18940000, 219000, 6015, 0x0becbfac -0, 19159000, 19159000, 219000, 14, 0x064900eb -0, 19160000, 19160000, 259000, 6519, 0xfcd24d26 -0, 19419000, 19419000, 259000, 14, 0x0729010b -0, 19420000, 19420000, 99000, 7061, 0xf0320408 -0, 19519000, 19519000, 99000, 14, 0x0809012b -0, 19520000, 19520000, 219000, 4773, 0x66c93074 -0, 19739000, 19739000, 219000, 14, 0x08e9014b -0, 19740000, 19740000, 219000, 5546, 0x06052c81 -0, 19959000, 19959000, 219000, 14, 0x09c9016b -0, 19960000, 19960000, 239000, 5754, 0x904f7325 -0, 20199000, 20199000, 239000, 14, 0x0aa9018b -0, 20200000, 20200000, 139000, 6099, 0xe30cde07 -0, 20339000, 20339000, 139000, 14, 0x0b8901ab -0, 20340000, 20340000, 799000, 6839, 0x770fcb6c -0, 21139000, 21139000, 799000, 14, 0x0c6901cb -0, 21140000, 21140000, 239000, 4744, 0xa91e1b41 -0, 21379000, 21379000, 239000, 14, 0x064900eb -0, 21380000, 21380000, 339000, 5824, 0xcf6d782b -0, 21719000, 21719000, 339000, 14, 0x0729010b -0, 21720000, 21720000, 1439000, 6212, 0xabf8f7cf -0, 23159000, 23159000, 1439000, 14, 0x0809012b -0, 23160000, 23160000, 1319000, 7082, 0xd7ca10f2 -0, 24479000, 24479000, 1319000, 14, 0x08e9014b -0, 24480000, 24480000, 219000, 5345, 0x12b2cae0 -0, 24699000, 24699000, 219000, 14, 0x09c9016b -0, 24700000, 24700000, 219000, 5765, 0xc7d46192 -0, 24919000, 24919000, 219000, 14, 0x0aa9018b -0, 24920000, 24920000, 599000, 6557, 0xcb995d30 -0, 25519000, 25519000, 599000, 14, 0x0b8901ab -0, 25520000, 25520000, 219000, 7091, 0xe6ea0559 -0, 25739000, 25739000, 219000, 14, 0x0c6901cb -0, 25740000, 25740000, 239000, 7305, 0xb66c404e -0, 25979000, 25979000, 239000, 14, 0x064900eb -0, 25980000, 25980000, 359000, 7590, 0x0cc2a481 -0, 26339000, 26339000, 359000, 14, 0x0729010b -0, 26340000, 26340000, 219000, 4629, 0xe18cfea8 -0, 26559000, 26559000, 219000, 14, 0x0809012b -0, 26560000, 26560000, 719000, 4785, 0x82043fc0 -0, 27279000, 27279000, 719000, 14, 0x08e9014b -0, 27280000, 27280000, 459000, 6061, 0xbde7d245 -0, 27739000, 27739000, 459000, 14, 0x09c9016b -0, 27740000, 27740000, 239000, 6301, 0x92d01a51 -0, 27979000, 27979000, 239000, 14, 0x0aa9018b -0, 27980000, 27980000, 99000, 6736, 0xbd25a134 -0, 28079000, 28079000, 99000, 14, 0x0b8901ab -0, 28080000, 28080000, 219000, 7214, 0x7ef93c13 -0, 28299000, 28299000, 219000, 14, 0x0c6901cb -0, 28300000, 28300000, 239000, 7366, 0x5bed7fcd -0, 28539000, 28539000, 239000, 14, 0x064900eb -0, 28540000, 28540000, 599000, 4564, 0x7f4c014b -0, 29139000, 29139000, 599000, 14, 0x0729010b -0, 29140000, 29140000, 219000, 4637, 0x682626b7 -0, 29359000, 29359000, 219000, 14, 0x0809012b -0, 29360000, 29360000, 1679000, 5358, 0x29e30c48 -0, 31039000, 31039000, 1679000, 14, 0x08e9014b +0, 0, 0, 279000, 14, 0x05d900db +0, 279000, 279000, 279000, 14, 0x064900eb +0, 280000, 280000, 4999000, 14, 0x06b900fb +0, 5279000, 5279000, 4999000, 14, 0x0729010b +0, 5280000, 5280000, 5019000, 14, 0x0799011b +0, 10299000, 10299000, 5019000, 14, 0x0809012b +0, 10300000, 10300000, 3599000, 14, 0x0879013b +0, 13899000, 13899000, 3599000, 14, 0x08e9014b +0, 13900000, 13900000, 219000, 14, 0x0959015b +0, 14119000, 14119000, 219000, 14, 0x09c9016b +0, 14120000, 14120000, 1439000, 14, 0x0a39017b +0, 15559000, 15559000, 1439000, 14, 0x0aa9018b +0, 15560000, 15560000, 39000, 14, 0x0b19019b +0, 15599000, 15599000, 39000, 14, 0x0b8901ab +0, 15600000, 15600000, 159000, 1168, 0xd69da022 +0, 15759000, 15759000, 159000, 14, 0x0c6901cb +0, 15760000, 15760000, 239000, 1544, 0xc5f116f1 +0, 15999000, 15999000, 239000, 14, 0x064900eb +0, 16000000, 16000000, 339000, 1658, 0x73563033 +0, 16339000, 16339000, 339000, 14, 0x0729010b +0, 16340000, 16340000, 599000, 2343, 0x7ac2078f +0, 16939000, 16939000, 599000, 14, 0x0809012b +0, 16940000, 16940000, 459000, 2568, 0x6eaa5751 +0, 17399000, 17399000, 459000, 14, 0x08e9014b +0, 17400000, 17400000, 359000, 3422, 0xd9d0636e +0, 17759000, 17759000, 359000, 14, 0x09c9016b +0, 17760000, 17760000, 219000, 5078, 0x722c9862 +0, 17979000, 17979000, 219000, 14, 0x0aa9018b +0, 17980000, 17980000, 959000, 5808, 0x38dd7ae9 +0, 18939000, 18939000, 959000, 14, 0x0b8901ab +0, 18940000, 18940000, 219000, 6015, 0xd4d2c40c +0, 19159000, 19159000, 219000, 14, 0x0c6901cb +0, 19160000, 19160000, 259000, 6519, 0x08af4c86 +0, 19419000, 19419000, 259000, 14, 0x064900eb +0, 19420000, 19420000, 99000, 7061, 0xecf10368 +0, 19519000, 19519000, 99000, 14, 0x0729010b +0, 19520000, 19520000, 219000, 4773, 0xbee42fd4 +0, 19739000, 19739000, 219000, 14, 0x0809012b +0, 19740000, 19740000, 219000, 5546, 0xdb822be1 +0, 19959000, 19959000, 219000, 14, 0x08e9014b +0, 19960000, 19960000, 239000, 5754, 0xfdcc7285 +0, 20199000, 20199000, 239000, 14, 0x09c9016b +0, 20200000, 20200000, 139000, 6099, 0xa409dd67 +0, 20339000, 20339000, 139000, 14, 0x0aa9018b +0, 20340000, 20340000, 799000, 6839, 0xc5eecacc +0, 21139000, 21139000, 799000, 14, 0x0b8901ab +0, 21140000, 21140000, 239000, 4744, 0x4e451fa1 +0, 21379000, 21379000, 239000, 14, 0x0c6901cb +0, 21380000, 21380000, 339000, 5824, 0x5299778b +0, 21719000, 21719000, 339000, 14, 0x064900eb +0, 21720000, 21720000, 1439000, 6212, 0x6d15f72f +0, 23159000, 23159000, 1439000, 14, 0x0729010b +0, 23160000, 23160000, 1319000, 7082, 0xe5c91052 +0, 24479000, 24479000, 1319000, 14, 0x0809012b +0, 24480000, 24480000, 219000, 5345, 0x2e5eca40 +0, 24699000, 24699000, 219000, 14, 0x08e9014b +0, 24700000, 24700000, 219000, 5765, 0x118060f2 +0, 24919000, 24919000, 219000, 14, 0x09c9016b +0, 24920000, 24920000, 599000, 6557, 0x89275c90 +0, 25519000, 25519000, 599000, 14, 0x0aa9018b +0, 25520000, 25520000, 219000, 7091, 0x996904b9 +0, 25739000, 25739000, 219000, 14, 0x0b8901ab +0, 25740000, 25740000, 239000, 7305, 0xc23e44ae +0, 25979000, 25979000, 239000, 14, 0x0c6901cb +0, 25980000, 25980000, 359000, 7590, 0xc5a3a3e1 +0, 26339000, 26339000, 359000, 14, 0x064900eb +0, 26340000, 26340000, 219000, 4629, 0x7ad6fe08 +0, 26559000, 26559000, 219000, 14, 0x0729010b +0, 26560000, 26560000, 719000, 4785, 0xcd3f3f20 +0, 27279000, 27279000, 719000, 14, 0x0809012b +0, 27280000, 27280000, 459000, 6061, 0x8b04d1a5 +0, 27739000, 27739000, 459000, 14, 0x08e9014b +0, 27740000, 27740000, 239000, 6301, 0xe7de19b1 +0, 27979000, 27979000, 239000, 14, 0x09c9016b +0, 27980000, 27980000, 99000, 6736, 0x38b3a094 +0, 28079000, 28079000, 99000, 14, 0x0aa9018b +0, 28080000, 28080000, 219000, 7214, 0x0b783b73 +0, 28299000, 28299000, 219000, 14, 0x0b8901ab +0, 28300000, 28300000, 239000, 7366, 0x98bf842d +0, 28539000, 28539000, 239000, 14, 0x0c6901cb +0, 28540000, 28540000, 599000, 4564, 0x3d9600ab +0, 29139000, 29139000, 599000, 14, 0x064900eb +0, 29140000, 29140000, 219000, 4637, 0x01f02617 +0, 29359000, 29359000, 219000, 14, 0x0729010b +0, 29360000, 29360000, 1679000, 5358, 0x5b0f0ba8 +0, 31039000, 31039000, 1679000, 14, 0x0809012b +0, 31040000, 31040000, 359000, 14, 0x0879013b +0, 31399000, 31399000, 359000, 14, 0x08e9014b +0, 31400000, 31400000, 479000, 14, 0x0959015b +0, 31879000, 31879000, 479000, 14, 0x09c9016b diff --git a/tests/ref/fate/sub2video b/tests/ref/fate/sub2video index 80abe9c905..b53c3d95ad 100644 --- a/tests/ref/fate/sub2video +++ b/tests/ref/fate/sub2video @@ -10,7 +10,6 @@ 0, 0, 0, 1, 518400, 0x83c27b82 0, 1, 1, 1, 518400, 0x4051c7f9 0, 2, 2, 1, 518400, 0xfb00e17e -1, 499000, 499000, 4960000, 1015, 0x19e092d2 0, 3, 3, 1, 518400, 0x192abb74 0, 4, 4, 1, 518400, 0x4669a88b 0, 5, 5, 1, 518400, 0xaababe00 @@ -58,129 +57,86 @@ 0, 47, 47, 1, 518400, 0xde69683f 0, 48, 48, 1, 518400, 0x7df08fba 0, 49, 49, 1, 518400, 0xbab197ea -1, 15355000, 15355000, 4733000, 2094, 0x3c171425 0, 77, 77, 1, 518400, 0x902285d9 0, 100, 100, 1, 518400, 0xbab197ea -1, 48797000, 48797000, 2560000, 2480, 0x7c0edf21 0, 244, 244, 1, 518400, 0x7a11c812 0, 257, 257, 1, 518400, 0xbab197ea -1, 51433000, 51433000, 2366000, 3059, 0xc95b8a05 0, 258, 258, 1, 518400, 0x34cdddee 0, 269, 269, 1, 518400, 0xbab197ea -1, 53910000, 53910000, 2696000, 2095, 0x61bb15ed 0, 270, 270, 1, 518400, 0x4db4ce51 0, 283, 283, 1, 518400, 0xbab197ea -1, 56663000, 56663000, 1262000, 1013, 0xc9ae89b7 0, 284, 284, 1, 518400, 0xe6bc0ea9 0, 290, 290, 1, 518400, 0xbab197ea -1, 58014000, 58014000, 1661000, 969, 0xe01878f0 0, 291, 291, 1, 518400, 0xa8643af7 0, 298, 298, 1, 518400, 0xbab197ea -1, 67724000, 67724000, 1365000, 844, 0xe7db4fc1 0, 339, 339, 1, 518400, 0xb1885c67 0, 345, 345, 1, 518400, 0xbab197ea -1, 69175000, 69175000, 1558000, 802, 0xf48531ba 0, 346, 346, 1, 518400, 0x378e3fd0 0, 354, 354, 1, 518400, 0xbab197ea -1, 70819000, 70819000, 1865000, 1709, 0xb4d5a1bd 0, 355, 355, 1, 518400, 0xa3782469 0, 363, 363, 1, 518400, 0xbab197ea -1, 72762000, 72762000, 1968000, 2438, 0x99d7bc82 0, 364, 364, 1, 518400, 0xba23a0d5 0, 374, 374, 1, 518400, 0xbab197ea -1, 74806000, 74806000, 1831000, 2116, 0x96514097 0, 375, 375, 1, 518400, 0x129de2f8 0, 383, 383, 1, 518400, 0xbab197ea -1, 76716000, 76716000, 1262000, 1822, 0xefccc72e 0, 384, 384, 1, 518400, 0x19772f0f 0, 390, 390, 1, 518400, 0xbab197ea -1, 78051000, 78051000, 1524000, 987, 0x7b927a27 0, 391, 391, 1, 518400, 0x56f54e73 0, 398, 398, 1, 518400, 0xbab197ea -1, 79644000, 79644000, 2662000, 2956, 0x190778f7 0, 399, 399, 1, 518400, 0x300b5247 -1, 82380000, 82380000, 2764000, 3094, 0xc021b7d3 0, 412, 412, 1, 518400, 0xbab197ea 0, 413, 413, 1, 518400, 0x6fd028fa 0, 426, 426, 1, 518400, 0xbab197ea -1, 85225000, 85225000, 2366000, 2585, 0x74d0048f 0, 427, 427, 1, 518400, 0x01f80e9d 0, 438, 438, 1, 518400, 0xbab197ea -1, 87652000, 87652000, 1831000, 634, 0x8832fda1 0, 439, 439, 1, 518400, 0xb48d90c0 0, 447, 447, 1, 518400, 0xbab197ea -1, 91531000, 91531000, 2332000, 2080, 0x97a1146f 0, 458, 458, 1, 518400, 0xcb5a0173 0, 469, 469, 1, 518400, 0xbab197ea -1, 95510000, 95510000, 3299000, 2964, 0x8b8f6684 0, 478, 478, 1, 518400, 0xb8a323e4 0, 494, 494, 1, 518400, 0xbab197ea -1, 98872000, 98872000, 2161000, 1875, 0x9002ef71 0, 495, 495, 1, 518400, 0xc43518ba 0, 505, 505, 1, 518400, 0xbab197ea -1, 101124000, 101124000, 4096000, 3872, 0x20c6ed9c 0, 506, 506, 1, 518400, 0x04e38692 0, 526, 526, 1, 518400, 0xbab197ea -1, 105303000, 105303000, 2730000, 3094, 0xf203a663 0, 527, 527, 1, 518400, 0x856b0ee5 0, 540, 540, 1, 518400, 0xbab197ea -1, 108106000, 108106000, 2059000, 2404, 0x41a7b429 0, 541, 541, 1, 518400, 0x3e5beee2 0, 551, 551, 1, 518400, 0xbab197ea -1, 141556000, 141556000, 1661000, 1088, 0xde20aa20 0, 708, 708, 1, 518400, 0xb8bc1365 0, 716, 716, 1, 518400, 0xbab197ea 0, 817, 817, 1, 518400, 0x83efa32d -1, 163445000, 163445000, 1331000, 339, 0x8bd186ef 0, 824, 824, 1, 518400, 0xbab197ea 0, 840, 840, 1, 518400, 0x03ea0e90 -1, 168049000, 168049000, 1900000, 1312, 0x0bf20e8d 0, 850, 850, 1, 518400, 0xbab197ea -1, 170035000, 170035000, 1524000, 1279, 0xb6c2dafe 0, 851, 851, 1, 518400, 0x8780239e 0, 858, 858, 1, 518400, 0xbab197ea 0, 861, 861, 1, 518400, 0x6eb72347 -1, 172203000, 172203000, 1695000, 1826, 0x9a1ac769 0, 869, 869, 1, 518400, 0xbab197ea -1, 173947000, 173947000, 1934000, 1474, 0xa9b03cdc 0, 870, 870, 1, 518400, 0x9c4a3a3d 0, 879, 879, 1, 518400, 0xbab197ea -1, 175957000, 175957000, 1763000, 1019, 0x20409355 0, 880, 880, 1, 518400, 0xc9ebfa89 0, 889, 889, 1, 518400, 0xbab197ea 0, 946, 946, 1, 518400, 0xbaf801ef -1, 189295000, 189295000, 1968000, 1596, 0x408c726e 0, 956, 956, 1, 518400, 0xbab197ea -1, 191356000, 191356000, 1228000, 1517, 0xae8c5c2b 0, 957, 957, 1, 518400, 0x59f4e72f 0, 963, 963, 1, 518400, 0xbab197ea -1, 192640000, 192640000, 1763000, 2506, 0xa458d6d4 0, 964, 964, 1, 518400, 0x9d5b9d69 0, 972, 972, 1, 518400, 0xbab197ea -1, 195193000, 195193000, 1092000, 1074, 0x397ba9a8 0, 976, 976, 1, 518400, 0x923d1ce7 0, 981, 981, 1, 518400, 0xbab197ea -1, 196361000, 196361000, 1524000, 1715, 0x695ca41e 0, 982, 982, 1, 518400, 0x6e652cd2 0, 989, 989, 1, 518400, 0xbab197ea -1, 197946000, 197946000, 1160000, 789, 0xc63a189e 0, 990, 990, 1, 518400, 0x25113966 0, 996, 996, 1, 518400, 0xbab197ea -1, 199230000, 199230000, 1627000, 1846, 0xeea8c599 0, 997, 997, 1, 518400, 0x2dc83609 0, 1004, 1004, 1, 518400, 0xbab197ea -1, 200924000, 200924000, 1763000, 922, 0xd4a87222 0, 1005, 1005, 1, 518400, 0x90483bc6 0, 1013, 1013, 1, 518400, 0xbab197ea 0, 1053, 1053, 1, 518400, 0x3de86ab7 -1, 210600000, 210600000, 1831000, 665, 0x55580135 0, 1062, 1062, 1, 518400, 0xbab197ea -1, 214771000, 214771000, 1558000, 1216, 0x50d1f6c5 0, 1074, 1074, 1, 518400, 0x8c320e68 0, 1082, 1082, 1, 518400, 0xbab197ea 0, 1128, 1128, 1, 518400, 0x81e977b2 -1, 225640000, 225640000, 2127000, 2133, 0x670c11a5 0, 1139, 1139, 1, 518400, 0xbab197ea -1, 227834000, 227834000, 1262000, 1264, 0xc1d9fc57 0, 1140, 1140, 1, 518400, 0xb046dd30 -0, 1145, 1145, 1, 518400, 0xbab197ea diff --git a/tests/ref/fate/sub2video_basic b/tests/ref/fate/sub2video_basic index 5f72e292c9..31e64242a4 100644 --- a/tests/ref/fate/sub2video_basic +++ b/tests/ref/fate/sub2video_basic @@ -2,94 +2,89 @@ #media_type 0: video #codec_id 0: rawvideo #dimensions 0: 720x480 -#sar 0: 0/1 -0, 3312, 3312, 1, 1382400, 0x00000000 -0, 3312, 3312, 1, 1382400, 0x8c93c2ba +#sar 0: 1/1 +0, 3312, 3312, 1, 1382400, 0xc637b893 0, 3436, 3436, 1, 1382400, 0x00000000 -0, 3684, 3684, 1, 1382400, 0xb02e32ca +0, 3684, 3684, 1, 1382400, 0x4c2960ca 0, 3802, 3802, 1, 1382400, 0x00000000 -0, 4520, 4520, 1, 1382400, 0x83b71116 +0, 4520, 4520, 1, 1382400, 0x5fa18966 0, 4584, 4584, 1, 1382400, 0x00000000 -0, 4586, 4586, 1, 1382400, 0x85547fd1 +0, 4586, 4586, 1, 1382400, 0x55f4b7b1 0, 4645, 4645, 1, 1382400, 0x00000000 -0, 4648, 4648, 1, 1382400, 0x00000000 -0, 4648, 4648, 1, 1382400, 0xb6a8f181 +0, 4648, 4648, 1, 1382400, 0xdfa4cf32 0, 4715, 4715, 1, 1382400, 0x00000000 -0, 4717, 4717, 1, 1382400, 0xb64d1a2c +0, 4717, 4717, 1, 1382400, 0x35023df8 0, 4748, 4748, 1, 1382400, 0x00000000 -0, 4750, 4750, 1, 1382400, 0x7b37ecf3 +0, 4750, 4750, 1, 1382400, 0xed933219 0, 4792, 4792, 1, 1382400, 0x00000000 -0, 4993, 4993, 1, 1382400, 0xdc025bd1 +0, 4993, 4993, 1, 1382400, 0x1b26389a 0, 5027, 5027, 1, 1382400, 0x00000000 -0, 5029, 5029, 1, 1382400, 0x688b294d +0, 5029, 5029, 1, 1382400, 0xf0c7028b 0, 5068, 5068, 1, 1382400, 0x00000000 -0, 5070, 5070, 1, 1382400, 0xa2b33d1b +0, 5070, 5070, 1, 1382400, 0x395f521d 0, 5117, 5117, 1, 1382400, 0x00000000 -0, 5119, 5119, 1, 1382400, 0xb3e525e3 +0, 5119, 5119, 1, 1382400, 0x1ea87415 0, 5168, 5168, 1, 1382400, 0x00000000 -0, 5170, 5170, 1, 1382400, 0xaa8fbdd7 +0, 5170, 5170, 1, 1382400, 0xc6effdc1 0, 5216, 5216, 1, 1382400, 0x00000000 -0, 5218, 5218, 1, 1382400, 0x7b7f26dd +0, 5218, 5218, 1, 1382400, 0xba6846f8 0, 5249, 5249, 1, 1382400, 0x00000000 -0, 5251, 5251, 1, 1382400, 0x15e2f836 +0, 5251, 5251, 1, 1382400, 0x033c5d5b 0, 5289, 5289, 1, 1382400, 0x00000000 -0, 5291, 5291, 1, 1382400, 0x0fee9b0c +0, 5291, 5291, 1, 1382400, 0xef5abf66 0, 5358, 5358, 1, 1382400, 0x00000000 -0, 5360, 5360, 1, 1382400, 0x89d62791 +0, 5360, 5360, 1, 1382400, 0xec747954 0, 5429, 5429, 1, 1382400, 0x00000000 -0, 5431, 5431, 1, 1382400, 0xa6a9fd74 +0, 5431, 5431, 1, 1382400, 0xfa34bcaf 0, 5490, 5490, 1, 1382400, 0x00000000 -0, 5491, 5491, 1, 1382400, 0x7896178d +0, 5491, 5491, 1, 1382400, 0x8b7a709b 0, 5537, 5537, 1, 1382400, 0x00000000 -0, 5588, 5588, 1, 1382400, 0x01751a52 +0, 5588, 5588, 1, 1382400, 0xc333382f 0, 5647, 5647, 1, 1382400, 0x00000000 -0, 5688, 5688, 1, 1382400, 0xa3959c6f +0, 5688, 5688, 1, 1382400, 0xabe5dfcf 0, 5770, 5770, 1, 1382400, 0x00000000 -0, 5772, 5772, 1, 1382400, 0x3d3ea47b +0, 5772, 5772, 1, 1382400, 0x56948101 0, 5826, 5826, 1, 1382400, 0x00000000 -0, 5828, 5828, 1, 1382400, 0x593f8b24 +0, 5828, 5828, 1, 1382400, 0xb747834a 0, 5931, 5931, 1, 1382400, 0x00000000 -0, 5933, 5933, 1, 1382400, 0x171f05ba +0, 5933, 5933, 1, 1382400, 0x3448baad 0, 6001, 6001, 1, 1382400, 0x00000000 -0, 6003, 6003, 1, 1382400, 0xb014cdf1 +0, 6003, 6003, 1, 1382400, 0xaabe4f37 0, 6054, 6054, 1, 1382400, 0x00000000 -0, 6839, 6839, 1, 1382400, 0xd918e667 +0, 6839, 6839, 1, 1382400, 0x8a48cd6f 0, 6880, 6880, 1, 1382400, 0x00000000 -0, 7386, 7386, 1, 1382400, 0xc9406331 +0, 7386, 7386, 1, 1382400, 0x49518c43 0, 7419, 7419, 1, 1382400, 0x00000000 -0, 7501, 7501, 1, 1382400, 0xaf08b10d +0, 7501, 7501, 1, 1382400, 0x4a72fa21 0, 7549, 7549, 1, 1382400, 0x00000000 -0, 7551, 7551, 1, 1382400, 0x00000000 -0, 7551, 7551, 1, 1382400, 0x853a9d93 +0, 7551, 7551, 1, 1382400, 0xa82f7de8 0, 7589, 7589, 1, 1382400, 0x00000000 -0, 7605, 7605, 1, 1382400, 0x7491a87d +0, 7605, 7605, 1, 1382400, 0xeba0b5f3 0, 7647, 7647, 1, 1382400, 0x00000000 -0, 7649, 7649, 1, 1382400, 0xf7383c58 +0, 7649, 7649, 1, 1382400, 0xd6a91770 0, 7697, 7697, 1, 1382400, 0x00000000 -0, 7699, 7699, 1, 1382400, 0xe66be411 +0, 7699, 7699, 1, 1382400, 0x222f827c 0, 7743, 7743, 1, 1382400, 0x00000000 -0, 8032, 8032, 1, 1382400, 0xd6850362 +0, 8032, 8032, 1, 1382400, 0x3270f4ff 0, 8082, 8082, 1, 1382400, 0x00000000 -0, 8084, 8084, 1, 1382400, 0x3e1ed109 +0, 8084, 8084, 1, 1382400, 0x40813cb3 0, 8115, 8115, 1, 1382400, 0x00000000 -0, 8116, 8116, 1, 1382400, 0x39c1b7bd +0, 8116, 8116, 1, 1382400, 0x9d8fde41 0, 8160, 8160, 1, 1382400, 0x00000000 -0, 8180, 8180, 1, 1382400, 0x35b85f2e +0, 8180, 8180, 1, 1382400, 0xc6d7a701 0, 8207, 8207, 1, 1382400, 0x00000000 -0, 8209, 8209, 1, 1382400, 0x00000000 -0, 8209, 8209, 1, 1382400, 0x83f103e5 +0, 8209, 8209, 1, 1382400, 0x9d45f2dc 0, 8247, 8247, 1, 1382400, 0x00000000 -0, 8249, 8249, 1, 1382400, 0xbc1ca9b3 +0, 8249, 8249, 1, 1382400, 0x8525ee40 0, 8278, 8278, 1, 1382400, 0x00000000 -0, 8281, 8281, 1, 1382400, 0x94d4a51e +0, 8281, 8281, 1, 1382400, 0x5b26b98b 0, 8321, 8321, 1, 1382400, 0x00000000 -0, 8323, 8323, 1, 1382400, 0xf88cdfde +0, 8323, 8323, 1, 1382400, 0x51be311f 0, 8367, 8367, 1, 1382400, 0x00000000 -0, 8565, 8565, 1, 1382400, 0xdd51423b +0, 8565, 8565, 1, 1382400, 0x00a4f2a3 0, 8611, 8611, 1, 1382400, 0x00000000 -0, 8669, 8669, 1, 1382400, 0x08259fa4 +0, 8669, 8669, 1, 1382400, 0x40a445e8 0, 8708, 8708, 1, 1382400, 0x00000000 -0, 8941, 8941, 1, 1382400, 0x1663fa34 +0, 8941, 8941, 1, 1382400, 0x43ef5128 0, 8994, 8994, 1, 1382400, 0x00000000 -0, 8996, 8996, 1, 1382400, 0xda2ceb55 -0, 9027, 9027, 1, 1382400, 0x00000000 +0, 8996, 8996, 1, 1382400, 0x3c3e3819 diff --git a/tests/ref/fate/sub2video_time_limited b/tests/ref/fate/sub2video_time_limited index 9fb6fb06f9..715af02fee 100644 --- a/tests/ref/fate/sub2video_time_limited +++ b/tests/ref/fate/sub2video_time_limited @@ -2,7 +2,5 @@ #media_type 0: video #codec_id 0: rawvideo #dimensions 0: 1920x1080 -#sar 0: 0/1 -0, 2, 2, 1, 8294400, 0x00000000 +#sar 0: 1/1 0, 2, 2, 1, 8294400, 0xa87c518f -0, 10, 10, 1, 8294400, 0xa87c518f From patchwork Thu Sep 16 20:30:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 30287 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp1582672iov; Thu, 16 Sep 2021 13:31:37 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyZwKdHPogLhMqzNYLNwEM4m95FRFrxo6WyzbkkucgN1vs80KuRFGbaD6LbEMz+Ou84Q6Dm X-Received: by 2002:a17:906:dbc9:: with SMTP id yc9mr8537029ejb.439.1631824297313; Thu, 16 Sep 2021 13:31:37 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id yd27si2931204ejb.451.2021.09.16.13.31.37; Thu, 16 Sep 2021 13:31:37 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=Q2UMNfqn; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4040E68B0D0; Thu, 16 Sep 2021 23:30:33 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11olkn2028.outbound.protection.outlook.com [40.92.19.28]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D884268B0C2 for ; Thu, 16 Sep 2021 23:30:30 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PglxaVVoO0ePfusWicU4oHTGE7R+VBG5sshLj7Z4okbPX7sqFdypXBVyqnc3owSYNzjygkJRF1uK0lggsueEMmrGTn2kUXBKYZsLtrUE2GtXg4Jn/dTvLY2suaV62cQJeXmtieNcUvYe55KJD+1MUsxFb3l6iJpS7NrM1wuyBxWB1C0PTl3sEM50rq0ZHMTWd0GQICcem5WrT9i3Curu0qFxYj1ng04tfLXJFuthDXYtd4TIIxAPxyUwNIe4D4co3sVxAKuAMX/qgZw6dn7485K0+RflXKi9xVsfd+vn21c5aPNRc1Js+s4HyETP/3b2fY/mj927JjpzmdiYUlK+yg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=fSFpm37Rn7evQy4flG7Mr49rbXotKxYHiMeJ6SFe5t0=; b=OMU/4xCIKrLk9tP+/obEO4H0/h2KhkUncVagi4F17ShJ8AI7vc0L5sV5/wj7qHkfAN6Zq6S1NWEBeNCQ5fTUqqoyqEupY2azgCR5aftyjItmeQmxzzUioH3AU2dXituOU9juhZu1rX0vHUPuD5boLokHoYnvcnhXEhW0Nw8ciCVz/aTwvUmDcXXZOquHf6CQgiAcG1DO+zkK1bNdv+MHu+Lo0XyU0fHffgQlZ4xT0LZ19hLmnlvsbv3tqxJ3H28NDQ53if5oysJwHZk063l7yYtQto1SebxkQY/mXgpkOw5hFL0Oundl4j1O1JD6WMGzaSKMELYSREzUM0fDb84Xjw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fSFpm37Rn7evQy4flG7Mr49rbXotKxYHiMeJ6SFe5t0=; b=Q2UMNfqnSU6lLZudyKGs+BMNlQ7PC+YSh11wnHxeOSa4gjaKf56b6+4yW5qXO7cdA1zx9biJI3LAf5UZyQIPQScIUopNAARo2sFglKManji2AUjId0XyrogPcnv8GKemJgUyj/nPyO9V5qbXUwVuIEof/yFcgGaF3y85n6U1tOiEdk9chZqzdx2j8Ime7x3emkn+j/ZxzzQKhr/2KUFo88qFLcQO1dfESoUXwi+LKSoy+VUBEstkx7TiJ005NGPN7hMC/EsxJni7I1WaFpkJM3hls7sVUzenSe/TQxGLgOKOL+Xymbl/hf45Bc8eMkNfoNtsDEwXokQHi7XJGcOOsg== Received: from MN2PR04MB5981.namprd04.prod.outlook.com (2603:10b6:208:da::10) by MN2PR04MB6048.namprd04.prod.outlook.com (2603:10b6:208:e5::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Thu, 16 Sep 2021 20:30:28 +0000 Received: from MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb]) by MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb%5]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021 20:30:28 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v6 08/11] avfilter/overlay_textsubs: Add overlay_textsubs and textsubs2video filters Thread-Index: AQHXqzmvJO7gxPe+rk+8/Rd8tfrkjg== Date: Thu, 16 Sep 2021 20:30:28 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [JTPRPYUHs44jnlVgleHD6azfw8rIoe0U] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 8ed0e12c-b957-47bd-dade-08d97950d252 x-ms-traffictypediagnostic: MN2PR04MB6048: x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 5FxlxCV8zlhJsrq+zhNGTgzB7BTMQpjK3cgv0xIp4COAZCrYd1W6KMRz8MwNCRLRaGzPO5X9N+zOYjrtsLEwMkH+CZx0PmVChErR3Gr/X9RbWrT2BeuvXtb1MoTr60G0/0cdflC1ZR6iiVb0h1My4nBs+yPnij49P9p4GrOoeyk6e0BJ3vAM8Uas/cQEtSLu5VeCQHEsU2HK0VfdZ05Za8TsIGKEi1CPeEh8ubfjNOVFcvp+qBlPlUnFH06TzLyPepGRB/0DMuPVf51pa7Ig3LqeoxvJLmYsbDSA208Li9u+Qri6SjHLXeI2jUgt+wn7YgA/dEkyGnHNVM951co65LqoLcBpooAqEBUGJQNv0/WAFSIlGO/D0B09cv8M1Dh4sHvtO7IXQ2cmcykW6P5bxVzdDm60mwnqrw/ffbdF2xOjoqhmVldy0DtKXTGOyCg+TRCHfLoPVNf1cXF2ypiFEIfYhxAlhJHqFRDOwXXKy/g= x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: 69j5uLoSsutRi1VirU/u/lRRCTk8hcoX7U4zj73AG9cOhaWqaMEbuwqO9ezRuWF9IjgARV3cFHEUONFkLuCHMYmhWNxBFIDBoNEj6aJ7farm9oArv2RGxG3O4FP36US0bkfXs8cgp1A+axezl2K+YQ== x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-3174-20-msonline-outlook-529c7.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR04MB5981.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 8ed0e12c-b957-47bd-dade-08d97950d252 X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Sep 2021 20:30:28.2146 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR04MB6048 Subject: [FFmpeg-devel] [PATCH v6 08/11] avfilter/overlay_textsubs: Add overlay_textsubs and textsubs2video filters X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: +gZDrQxBRC0Y - overlay_textsubs {VS -> V) Overlay text subtitles onto a video stream. - textsubs2video {S -> V) Converts text subtitles to video frames Signed-off-by: softworkz --- configure | 2 + doc/filters.texi | 69 ++++ libavfilter/Makefile | 2 + libavfilter/allfilters.c | 2 + libavfilter/avfilter.c | 18 +- libavfilter/vf_overlay_textsubs.c | 613 ++++++++++++++++++++++++++++++ 6 files changed, 701 insertions(+), 5 deletions(-) create mode 100644 libavfilter/vf_overlay_textsubs.c diff --git a/configure b/configure index 74c5a41be1..6d6a12754d 100755 --- a/configure +++ b/configure @@ -3624,6 +3624,7 @@ openclsrc_filter_deps="opencl" overlay_opencl_filter_deps="opencl" overlay_qsv_filter_deps="libmfx" overlay_qsv_filter_select="qsvvpp" +overlay_textsubs_filter_deps="avcodec libass" overlay_vulkan_filter_deps="vulkan_lib libglslang" owdenoise_filter_deps="gpl" pad_opencl_filter_deps="opencl" @@ -3669,6 +3670,7 @@ superequalizer_filter_deps="avcodec" superequalizer_filter_select="rdft" surround_filter_deps="avcodec" surround_filter_select="rdft" +textsub2video_filter_deps="avcodec libass" tinterlace_filter_deps="gpl" tinterlace_merge_test_deps="tinterlace_filter" tinterlace_pad_test_deps="tinterlace_filter" diff --git a/doc/filters.texi b/doc/filters.texi index e281af7155..1c38ecd8ec 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -25181,6 +25181,75 @@ The graphicsub2video is inserted automatically for compatibility with legacy com ffmpeg -i "https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv" -filter_complex "[0:0][0:1]overlay" output.mp4 @end example @end itemize + +@section overlay_textsubs + +Overlay text subtitles onto a video stream. + +This filter supersedes the classic @ref{subtitles} filter opposed to which it does no longer require to open and access the source stream separately, which is often causing problems or doesn't even work for non-local or slow sources. + +Inputs: +- 0: Video [YUV420P, YUV422P, YUV444P, ARGB, RGBA, ABGR, BGRA, RGB24, BGR24] +- 1: Subtitles [text] + +Outputs: +- 0: Video (same as input) + +It accepts the following parameters: + +@table @option + +@item alpha +Process alpha channel, by default alpha channel is untouched. + +@item fonts_dir +Set a directory path containing fonts that can be used by the filter. +These fonts will be used in addition to whatever the font provider uses. + +@item force_style +Override default style or script info parameters of the subtitles. It accepts a +string containing ASS style format @code{KEY=VALUE} couples separated by ",". + +@end table + +@section textsub2video + +Converts text subtitles to video frames. + +For overlaying text subtitles onto video frames it is recommended to use the overlay_textsubs filter. +The textsub2video is useful for for creating transparent text-frames when overlay is done via hw acceleration + +Inputs: +- 0: Subtitles [text] + +Outputs: +- 0: Video [ARGB, RGBA, ABGR, BGRA] + +It accepts the following parameters: + +@table @option + +@item rate, r +Set the framerate for updating overlay frames. +Normally, overlay frames will only be updated each time when the subtitles to display are changing. +In cases where subtitles include advanced features (like animation), this parameter determines the frequency by which the overlay frames should be updated. + +@item size, s +Set the output frame size. +Allows to override the size of output video frames. + +@item alpha +Process alpha channel, by default alpha channel is untouched. + +@item fonts_dir +Set a directory path containing fonts that can be used by the filter. +These fonts will be used in addition to whatever the font provider uses. + +@item force_style +Override default style or script info parameters of the subtitles. It accepts a +string containing ASS style format @code{KEY=VALUE} couples separated by ",". + +@end table @c man end SUBTITLE FILTERS @chapter Multimedia Filters diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 8fcc25989e..c0b1cc7001 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -365,6 +365,7 @@ OBJS-$(CONFIG_OVERLAY_OPENCL_FILTER) += vf_overlay_opencl.o opencl.o \ opencl/overlay.o framesync.o OBJS-$(CONFIG_OVERLAY_QSV_FILTER) += vf_overlay_qsv.o framesync.o OBJS-$(CONFIG_OVERLAY_GRAPHICSUBS_FILTER) += vf_overlay_graphicsubs.o framesync.o +OBJS-$(CONFIG_OVERLAY_TEXTSUBS_FILTER) += vf_overlay_textsubs.o OBJS-$(CONFIG_OVERLAY_VULKAN_FILTER) += vf_overlay_vulkan.o vulkan.o OBJS-$(CONFIG_OWDENOISE_FILTER) += vf_owdenoise.o OBJS-$(CONFIG_PAD_FILTER) += vf_pad.o @@ -453,6 +454,7 @@ OBJS-$(CONFIG_SWAPRECT_FILTER) += vf_swaprect.o OBJS-$(CONFIG_SWAPUV_FILTER) += vf_swapuv.o OBJS-$(CONFIG_TBLEND_FILTER) += vf_blend.o framesync.o OBJS-$(CONFIG_TELECINE_FILTER) += vf_telecine.o +OBJS-$(CONFIG_TEXTSUB2VIDEO_FILTER) += vf_overlay_textsubs.o OBJS-$(CONFIG_THISTOGRAM_FILTER) += vf_histogram.o OBJS-$(CONFIG_THRESHOLD_FILTER) += vf_threshold.o framesync.o OBJS-$(CONFIG_THUMBNAIL_FILTER) += vf_thumbnail.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 10a310d20d..8543fa22e9 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -346,6 +346,7 @@ extern const AVFilter ff_vf_overlay; extern const AVFilter ff_vf_overlay_opencl; extern const AVFilter ff_vf_overlay_qsv; extern const AVFilter ff_vf_overlay_graphicsubs; +extern const AVFilter ff_vf_overlay_textsubs; extern const AVFilter ff_vf_overlay_vulkan; extern const AVFilter ff_vf_overlay_cuda; extern const AVFilter ff_vf_owdenoise; @@ -526,6 +527,7 @@ extern const AVFilter ff_avf_showwaves; extern const AVFilter ff_avf_showwavespic; extern const AVFilter ff_vaf_spectrumsynth; extern const AVFilter ff_svf_graphicsub2video; +extern const AVFilter ff_svf_textsub2video; /* multimedia sources */ extern const AVFilter ff_avsrc_amovie; diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c index 3f303fbafa..c6281ab0bc 100644 --- a/libavfilter/avfilter.c +++ b/libavfilter/avfilter.c @@ -464,7 +464,7 @@ static int64_t guess_status_pts(AVFilterContext *ctx, int status, AVRational lin return AV_NOPTS_VALUE; } -static int ff_request_frame_to_filter(AVFilterLink *link) +static int ff_request_frame_to_filter(AVFilterLink *link, int input_index) { int ret = -1; @@ -473,8 +473,8 @@ static int ff_request_frame_to_filter(AVFilterLink *link) link->frame_blocked_in = 1; if (link->srcpad->request_frame) ret = link->srcpad->request_frame(link); - else if (link->src->inputs[0]) - ret = ff_request_frame(link->src->inputs[0]); + else if (link->src->inputs[input_index]) + ret = ff_request_frame(link->src->inputs[input_index]); if (ret < 0) { if (ret != AVERROR(EAGAIN) && ret != link->status_in) ff_avfilter_link_set_in_status(link, ret, guess_status_pts(link->src, ret, link->time_base)); @@ -1171,6 +1171,14 @@ static int forward_status_change(AVFilterContext *filter, AVFilterLink *in) { unsigned out = 0, progress = 0; int ret; + int input_index = 0; + + for (int i = 0; i < in->dst->nb_inputs; i++) { + if (&in->dst->input_pads[i] == in->dstpad) { + input_index = i; + break; + } + } av_assert0(!in->status_out); if (!filter->nb_outputs) { @@ -1180,7 +1188,7 @@ static int forward_status_change(AVFilterContext *filter, AVFilterLink *in) while (!in->status_out) { if (!filter->outputs[out]->status_in) { progress++; - ret = ff_request_frame_to_filter(filter->outputs[out]); + ret = ff_request_frame_to_filter(filter->outputs[out], input_index); if (ret < 0) return ret; } @@ -1217,7 +1225,7 @@ static int ff_filter_activate_default(AVFilterContext *filter) for (i = 0; i < filter->nb_outputs; i++) { if (filter->outputs[i]->frame_wanted_out && !filter->outputs[i]->frame_blocked_in) { - return ff_request_frame_to_filter(filter->outputs[i]); + return ff_request_frame_to_filter(filter->outputs[i], 0); } } return FFERROR_NOT_READY; diff --git a/libavfilter/vf_overlay_textsubs.c b/libavfilter/vf_overlay_textsubs.c new file mode 100644 index 0000000000..a1a56ae21d --- /dev/null +++ b/libavfilter/vf_overlay_textsubs.c @@ -0,0 +1,613 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * overlay text subtitles on top of a video frame + */ + +#include +#include + +#include "avfilter.h" +#include "formats.h" +#include "libavutil/common.h" +#include "libavutil/avstring.h" +#include "libavutil/opt.h" +#include "internal.h" +#include "drawutils.h" +#include "filters.h" + +#include "libavcodec/avcodec.h" +#include "libavutil/pixdesc.h" + +typedef struct TextSubsContext { + const AVClass *class; + ASS_Library *library; + ASS_Renderer *renderer; + ASS_Track *track; + + char *default_font_path; + char *fonts_dir; + char *fc_file; + double font_scale; + double font_size; + char *force_style; + char *language; + int margin; + + int alpha; + FFDrawContext draw; + + int got_header; + int out_w, out_h; + AVRational frame_rate; + AVFrame *last_frame; + int eof; +} TextSubsContext; + +/* libass supports a log level ranging from 0 to 7 */ +static const int ass_libavfilter_log_level_map[] = { + AV_LOG_QUIET, /* 0 */ + AV_LOG_PANIC, /* 1 */ + AV_LOG_FATAL, /* 2 */ + AV_LOG_ERROR, /* 3 */ + AV_LOG_WARNING, /* 4 */ + AV_LOG_INFO, /* 5 */ + AV_LOG_VERBOSE, /* 6 */ + AV_LOG_DEBUG, /* 7 */ +}; + +static void ass_log(int ass_level, const char *fmt, va_list args, void *ctx) +{ + const int ass_level_clip = av_clip(ass_level, 0, FF_ARRAY_ELEMS(ass_libavfilter_log_level_map) - 1); + const int level = ass_libavfilter_log_level_map[ass_level_clip]; + + av_vlog(ctx, level, fmt, args); + av_log(ctx, level, "\n"); +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + TextSubsContext *s = ctx->priv; + + if (s->track) + ass_free_track(s->track); + if (s->renderer) + ass_renderer_done(s->renderer); + if (s->library) + ass_library_done(s->library); + + s->track = NULL; + s->renderer = NULL; + s->library = NULL; + + if (s->last_frame) + av_frame_unref(s->last_frame); +} + +static int overlay_textsubs_query_formats(AVFilterContext *ctx) +{ + AVFilterFormats *formats; + AVFilterLink *inlink0 = ctx->inputs[0]; + AVFilterLink *inlink1 = ctx->inputs[1]; + AVFilterLink *outlink = ctx->outputs[0]; + static const enum AVSubtitleType subtitle_fmts[] = { AV_SUBTITLE_FMT_ASS, AV_SUBTITLE_FMT_NONE }; + int ret; + + /* set input0 video formats */ + formats = ff_draw_supported_pixel_formats(0); + if ((ret = ff_formats_ref(formats, &inlink0->outcfg.formats)) < 0) + return ret; + + /* set input1 subtitle formats */ + formats = ff_make_format_list(subtitle_fmts); + if ((ret = ff_formats_ref(formats, &inlink1->outcfg.formats)) < 0) + return ret; + + /* set output0 video formats */ + formats = ff_draw_supported_pixel_formats(0); + if ((ret = ff_formats_ref(formats, &outlink->incfg.formats)) < 0) + return ret; + + return 0; +} + +static int config_output(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + + outlink->w = ctx->inputs[0]->w; + outlink->h = ctx->inputs[0]->h; + outlink->time_base = ctx->inputs[0]->time_base; + + return 0; +} + +static int config_input_main(AVFilterLink *inlink) +{ + AVFilterContext *ctx = inlink->dst; + TextSubsContext *s = inlink->dst->priv; + int ret; + + ret = ff_draw_init(&s->draw, inlink->format, s->alpha ? FF_DRAW_PROCESS_ALPHA : 0); + if (ret < 0) { + av_log(ctx, AV_LOG_ERROR, "Could not initialize ff_draw.\n"); + return ret; + } + + ass_set_frame_size (s->renderer, inlink->w, inlink->h); + ass_set_pixel_aspect(s->renderer, av_q2d(inlink->sample_aspect_ratio)); + + return 0; +} + +/* libass stores an RGBA color in the format RRGGBBTT, where TT is the transparency level */ +#define AR(c) ( (c)>>24) +#define AG(c) (((c)>>16)&0xFF) +#define AB(c) (((c)>>8) &0xFF) +#define AA(c) ((0xFF-(c)) &0xFF) + +static void overlay_ass_image(TextSubsContext *s, AVFrame *picref, + const ASS_Image *image) +{ + for (; image; image = image->next) { + uint8_t rgba_color[] = {AR(image->color), AG(image->color), AB(image->color), AA(image->color)}; + FFDrawColor color; + ff_draw_color(&s->draw, &color, rgba_color); + ff_blend_mask(&s->draw, &color, + picref->data, picref->linesize, + picref->width, picref->height, + image->bitmap, image->stride, image->w, image->h, + 3, 0, image->dst_x, image->dst_y); + } +} + +static void process_header(AVFilterContext *link, AVFrame *sub) +{ + TextSubsContext *s = link->priv; + ASS_Track *track = s->track; + ASS_Style *style; + int sid = 0; + + if (!track) + return; + + if (sub && sub->subtitle_header) + ass_process_codec_private(s->track, sub->subtitle_header, strlen(sub->subtitle_header)); + else { + AVCodecContext temp_context; + ff_ass_subtitle_header_default(&temp_context); + if (!temp_context.subtitle_header) + return; + + ass_process_codec_private(s->track, (char*)temp_context.subtitle_header, temp_context.subtitle_header_size); + } + + if (s->language) + s->track->Language = strdup(s->language); + + if (!s->track->event_format) { + s->track->event_format = strdup("ReadOrder, Layer, Style, Name, MarginL, MarginR, MarginV, Effect, Text"); + } + + if (s->track->n_styles == 0) { + sid = ass_alloc_style(track); + style = &s->track->styles[sid]; + style->Name = strdup("Default"); + style->PrimaryColour = 0xffffff00; + style->SecondaryColour = 0x00ffff00; + style->OutlineColour = 0x00000000; + style->BackColour = 0x00000080; + style->Bold = 200; + style->ScaleX = 1.0; + style->ScaleY = 1.0; + style->Spacing = 0; + style->BorderStyle = 1; + style->Outline = 2; + style->Shadow = 3; + style->Alignment = 2; + } + else + style = &s->track->styles[sid]; + + style->FontSize = s->font_size; + style->MarginL = style->MarginR = style->MarginV = s->margin; + + track->default_style = sid; + + s->got_header = 1; +} + +static int filter_video_frame(AVFilterLink *inlink, AVFrame *frame) +{ + AVFilterContext *ctx = inlink->dst; + TextSubsContext *s = ctx->priv; + int detect_change = 0; + + int64_t time_ms = frame->pts * av_q2d(inlink->time_base) * 1000; + + ASS_Image *image = ass_render_frame(s->renderer, s->track, time_ms, &detect_change); + + if (detect_change) + av_log(ctx, AV_LOG_DEBUG, "Change happened at time ms:%lld\n", time_ms); + + overlay_ass_image(s, frame, image); + + return ff_filter_frame(ctx->outputs[0], frame); +} + +static int filter_subtitle_frame(AVFilterLink *inlink, AVFrame *sub) +{ + AVFilterContext *ctx = inlink->dst; + TextSubsContext *s = ctx->priv; + + if (!s->got_header) + process_header(ctx, sub); + + const int64_t start_time = av_rescale_q(sub->subtitle_pts, AV_TIME_BASE_Q, av_make_q(1, 1000)); + const int64_t duration = sub->subtitle_end_time; + + for (unsigned i = 0; i < sub->num_subtitle_areas; i++) { + char *ass_line = sub->subtitle_areas[i]->ass; + if (!ass_line) + break; + ass_process_chunk(s->track, ass_line, strlen(ass_line), start_time, duration); + } + + av_frame_free(&sub); + return 0; +} + +static av_cold int init(AVFilterContext *ctx) +{ + TextSubsContext *s = ctx->priv; + + s->library = ass_library_init(); + + if (!s->library) { + av_log(ctx, AV_LOG_ERROR, "Could not initialize libass.\n"); + return AVERROR(EINVAL); + } + + ass_set_message_cb(s->library, ass_log, ctx); + + /* Initialize fonts */ + if (s->fonts_dir) + ass_set_fonts_dir(s->library, s->fonts_dir); + + ass_set_extract_fonts(s->library, 1); + + s->renderer = ass_renderer_init(s->library); + if (!s->renderer) { + av_log(ctx, AV_LOG_ERROR, "Could not initialize libass renderer.\n"); + return AVERROR(EINVAL); + } + + s->track = ass_new_track(s->library); + if (!s->track) { + av_log(ctx, AV_LOG_ERROR, "ass_new_track() failed!\n"); + return AVERROR(EINVAL); + } + + ass_set_fonts(s->renderer, s->default_font_path, NULL, 1, s->fc_file, 1); + + if (s->force_style) { + char **list = NULL; + char *temp = NULL; + char *ptr = av_strtok(s->force_style, ",", &temp); + int i = 0; + while (ptr) { + av_dynarray_add(&list, &i, ptr); + if (!list) { + return AVERROR(ENOMEM); + } + ptr = av_strtok(NULL, ",", &temp); + } + av_dynarray_add(&list, &i, NULL); + if (!list) { + return AVERROR(ENOMEM); + } + ass_set_style_overrides(s->library, list); + av_free(list); + } + + return 0; +} + +static int textsub2video_query_formats(AVFilterContext *ctx) +{ + AVFilterFormats *formats; + AVFilterLink *inlink = ctx->inputs[0]; + AVFilterLink *outlink = ctx->outputs[0]; + static const enum AVSubtitleType subtitle_fmts[] = { AV_SUBTITLE_FMT_ASS, AV_SUBTITLE_FMT_NONE }; + int ret; + + /* set input0 subtitle format */ + formats = ff_make_format_list(subtitle_fmts); + if ((ret = ff_formats_ref(formats, &inlink->outcfg.formats)) < 0) + return ret; + + /* set output0 video format */ + formats = ff_draw_supported_pixel_formats(AV_PIX_FMT_FLAG_ALPHA); + if ((ret = ff_formats_ref(formats, &outlink->incfg.formats)) < 0) + return ret; + + return 0; +} + +static int textsub2video_config_input(AVFilterLink *inlink) +{ + AVFilterContext *ctx = inlink->dst; + TextSubsContext *s = ctx->priv; + + if (s->out_w <= 0 || s->out_h <= 0) { + s->out_w = inlink->w; + s->out_h = inlink->h; + } + + return 0; +} + +static int textsub2video_config_output(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + TextSubsContext *s = ctx->priv; + int ret; + + ret = ff_draw_init(&s->draw, outlink->format, FF_DRAW_PROCESS_ALPHA); + if (ret < 0) { + av_log(ctx, AV_LOG_ERROR, "Could not initialize ff_draw.\n"); + return ret; + } + + if (s->out_w <= 0 || s->out_h <= 0) { + av_log(ctx, AV_LOG_ERROR, "No output image size set.\n"); + return AVERROR(EINVAL); + } + + ass_set_frame_size (s->renderer, s->out_w, s->out_h); + + outlink->w = s->out_w; + outlink->h = s->out_h; + outlink->sample_aspect_ratio = (AVRational){1,1}; + outlink->frame_rate = s->frame_rate; + + return 0; +} + +static int textsub2video_filter_frame(AVFilterLink *inlink, AVFrame *sub) +{ + AVFilterContext *ctx = inlink->dst; + TextSubsContext *s = ctx->priv; + + av_log(ctx, AV_LOG_VERBOSE, "textsub2video_filter_frame num_subtitle_rects: %d\n", sub->num_subtitle_areas); + + if (!s->got_header) + process_header(ctx, sub); + + const int64_t start_time = av_rescale_q(sub->subtitle_pts, AV_TIME_BASE_Q, av_make_q(1, 1000)); + const int64_t duration = sub->subtitle_end_time; + + for (unsigned i = 0; i < sub->num_subtitle_areas; i++) { + char *ass_line = sub->subtitle_areas[i]->ass; + if (!ass_line) + break; + ass_process_chunk(s->track, ass_line, strlen(ass_line), start_time, duration); + } + + av_frame_free(&sub); + return 0; +} + +static int textsub2video_request_frame(AVFilterLink *outlink) +{ + TextSubsContext *s = outlink->src->priv; + const AVFilterLink *inlink = outlink->src->inputs[0]; + int64_t last_pts = outlink->current_pts; + int64_t next_pts; + int i, detect_change = 0; + AVFrame *out; + ASS_Image *image; + + if (last_pts == AV_NOPTS_VALUE) + last_pts = inlink->current_pts * av_q2d(inlink->time_base) / av_q2d(outlink->time_base); + + next_pts = last_pts + (1.0 / av_q2d(outlink->frame_rate) / av_q2d(outlink->time_base)); + + int64_t time_ms = next_pts * av_q2d(outlink->time_base) * 1000; + + image = ass_render_frame(s->renderer, s->track, time_ms, &detect_change); + + if (detect_change) + av_log(outlink->src, AV_LOG_DEBUG, "Change happened at time ms:%lld\n", time_ms); + else if (s->last_frame) { + out = av_frame_clone(s->last_frame); + if (!out) + return AVERROR(ENOMEM); + + out->pts = next_pts; + return ff_filter_frame(outlink, out); + } + + out = ff_get_video_buffer(outlink, outlink->w, outlink->h); + if (!out) + return AVERROR(ENOMEM); + + for (i = 2; i < AV_NUM_DATA_POINTERS; i++) { + if (out->buf[i] && i != 1) + memset(out->buf[i]->data, 0, out->buf[i]->size); + } + + out->pts = next_pts; + + if (image) + overlay_ass_image(s, out, image); + + if (s->last_frame) + av_frame_unref(s->last_frame); + + s->last_frame = av_frame_clone(out); + + return ff_filter_frame(outlink, out); +} + +static int activate(AVFilterContext *ctx) +{ + AVFilterLink *inlink = ctx->inputs[0]; + AVFilterLink *outlink = ctx->outputs[0]; + TextSubsContext *s = ctx->priv; + AVFrame *in; + int ret; + + FF_FILTER_FORWARD_STATUS_BACK(outlink, inlink); + + ret = ff_inlink_consume_frame(inlink, &in); + if (ret < 0) + return ret; + if (ret > 0) + return textsub2video_filter_frame(inlink, in); + + if (ff_outlink_frame_wanted(outlink)) { + if (!s->eof && ff_outlink_get_status(ctx->inputs[0])) { + s->eof = 1; + } + if (!s->eof && ff_inlink_queued_frames(ctx->inputs[0]) == 0) + ff_inlink_request_frame(ctx->inputs[0]); + if (s->eof && ff_inlink_queued_frames(ctx->inputs[0]) <= 0) { + ff_outlink_set_status(outlink, AVERROR_EOF, AV_NOPTS_VALUE); + } + else { + return textsub2video_request_frame(outlink); + } + + return 0; + } + + return FFERROR_NOT_READY; +} + +#define OFFSET(x) offsetof(TextSubsContext, x) +#define FLAGS (AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_FILTERING_PARAM) + +static const AVOption overlay_textsubs_options[] = { + {"alpha", "enable processing of alpha channel", OFFSET(alpha), AV_OPT_TYPE_BOOL, {.i64 = 0 }, 0, 1, FLAGS}, + {"font_scale", "font scale factor", OFFSET(font_scale), AV_OPT_TYPE_DOUBLE, {.dbl = 1.0 }, 0.0, 100.0, FLAGS}, + {"font_size", "default font size", OFFSET(font_size), AV_OPT_TYPE_DOUBLE, {.dbl = 18.0}, 0.0, 100.0, FLAGS}, + {"force_style", "force subtitle style", OFFSET(force_style), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, FLAGS}, + {"margin", "default margin", OFFSET(margin), AV_OPT_TYPE_INT, {.i64 = 20 }, 0, INT_MAX, FLAGS}, + {"default_font_path", "path to default font", OFFSET(default_font_path), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, FLAGS}, + {"fonts_dir", "directory to scan for fonts", OFFSET(fonts_dir), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, FLAGS}, + {"fontsdir", "directory to scan for fonts", OFFSET(fonts_dir), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, FLAGS}, + {"fontconfig_file", "fontconfig file to load", OFFSET(fc_file), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, FLAGS}, + {"language", "default language", OFFSET(language), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, FLAGS}, + { NULL } +}; + +static const AVOption textsub2video_options[] = { + {"rate", "set frame rate", OFFSET(frame_rate), AV_OPT_TYPE_VIDEO_RATE, {.str="8"}, 0, INT_MAX, FLAGS}, + {"r", "set frame rate", OFFSET(frame_rate), AV_OPT_TYPE_VIDEO_RATE, {.str="8"}, 0, INT_MAX, FLAGS}, + {"size", "set video size", OFFSET(out_w), AV_OPT_TYPE_IMAGE_SIZE, {.str = NULL}, 0, 0, FLAGS}, + {"s", "set video size", OFFSET(out_w), AV_OPT_TYPE_IMAGE_SIZE, {.str = NULL}, 0, 0, FLAGS}, + {"font_scale", "font scale factor", OFFSET(font_scale), AV_OPT_TYPE_DOUBLE, {.dbl = 1.0 }, 0.0, 100.0, FLAGS}, + {"font_size", "default font size", OFFSET(font_size), AV_OPT_TYPE_DOUBLE, {.dbl = 18.0}, 0.0, 100.0, FLAGS}, + {"force_style", "force subtitle style", OFFSET(force_style), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, FLAGS}, + {"margin", "default margin", OFFSET(margin), AV_OPT_TYPE_INT, {.i64 = 20 }, 0, INT_MAX, FLAGS}, + {"default_font_path", "path to default font", OFFSET(default_font_path), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, FLAGS}, + {"fonts_dir", "directory to scan for fonts", OFFSET(fonts_dir), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, FLAGS}, + {"fontsdir", "directory to scan for fonts", OFFSET(fonts_dir), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, FLAGS}, + {"fontconfig_file", "fontconfig file to load", OFFSET(fc_file), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, FLAGS}, + {"language", "default language", OFFSET(language), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, FLAGS}, + { NULL } +}; + +#if CONFIG_OVERLAY_TEXTSUBS_FILTER + +AVFILTER_DEFINE_CLASS(overlay_textsubs); + +static const AVFilterPad overlay_textsubs_inputs[] = { + { + .name = "main", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_input_main, + .flags = AVFILTERPAD_FLAG_NEEDS_WRITABLE, + .filter_frame = filter_video_frame, + }, + { + .name = "overlay", + .type = AVMEDIA_TYPE_SUBTITLE, + .filter_frame = filter_subtitle_frame, + }, +}; + +static const AVFilterPad overlay_textsubs_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_output, + }, +}; + +const AVFilter ff_vf_overlay_textsubs = { + .name = "overlay_textsubs", + .description = NULL_IF_CONFIG_SMALL("Overlay textual subtitles on top of the input."), + .init = init, + .uninit = uninit, + .priv_size = sizeof(TextSubsContext), + .priv_class = &overlay_textsubs_class, + .query_formats = overlay_textsubs_query_formats, + FILTER_INPUTS(overlay_textsubs_inputs), + FILTER_OUTPUTS(overlay_textsubs_outputs), +}; +#endif + +#if CONFIG_TEXTSUB2VIDEO_FILTER + +AVFILTER_DEFINE_CLASS(textsub2video); + +static const AVFilterPad textsub2video_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .config_props = textsub2video_config_input, + }, +}; + +static const AVFilterPad textsub2video_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = textsub2video_config_output, + }, +}; + +AVFilter ff_svf_textsub2video = { + .name = "textsub2video", + .description = NULL_IF_CONFIG_SMALL("Convert textual subtitles to video frames"), + .init = init, + .uninit = uninit, + .query_formats = textsub2video_query_formats, + .priv_size = sizeof(TextSubsContext), + .priv_class = &textsub2video_class, + .activate = activate, + FILTER_INPUTS(textsub2video_inputs), + FILTER_OUTPUTS(textsub2video_outputs), +}; +#endif \ No newline at end of file From patchwork Thu Sep 16 20:30:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 30283 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp1582956iov; Thu, 16 Sep 2021 13:32:04 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxzBfEZXr3AcnxFLG/VslS8TiOB1U+H3Qt8d+SwgN4DLNgI6PTVEGUERjp/jH96QkeqLERi X-Received: by 2002:a05:6402:198:: with SMTP id r24mr8589413edv.93.1631824324001; Thu, 16 Sep 2021 13:32:04 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id l16si4772276edj.330.2021.09.16.13.32.03; Thu, 16 Sep 2021 13:32:03 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=QuGT84kp; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3435768B0D8; Thu, 16 Sep 2021 23:30:36 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11olkn2103.outbound.protection.outlook.com [40.92.19.103]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3A0C768B0DE for ; Thu, 16 Sep 2021 23:30:33 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=nuRrM9hlkuKQrxI79BkQ85Ed/0UPqCoPZVqmom2Vbx7kTM5fzcpTJCMswjYvQUqjAKfPKy69TZTcAfc+51Gt8OMpdkH1BDjwezNHVvrjZRAXStwYJSuE5nV8h35PjJVe7eNn+2BsON64ouKe9oYurR7FmVy74bzIYrwEY7BhKXUabZE+gEdIAiaZpTB1zSwlg/rG/EMrAOeQtPHKmCw3mnV6xyOMU2wx6XtwKAzWyboKTodNxGRPcZClCRuF8kKFJ0k28Z7E/Lwht0ghV0rf+xZt441XsryZ5IB9ITFoR2Zyong1sFtol0KG+/e10U+BqbQ4a4nqIfOUQo+24Anj2A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=7aJ5arrdqi4Nmzkvd9wQ/zBRN4PeDD+62gloXB82Tfk=; b=OU+u5wvaKuKvt5yxCu2x0cxBP0goUaXqx4hdhP9oFwhuiKi2raCz/MC5DLvBfWY56rcW3sp4qnxtlT+QI4itrXDaOjy3ff5CeBZRlhvk2WKnvmsoPqy9ZyrraydZ2dmPZ+k5bkeqsR0e4UxYC47BS+huPcF+AdhZ/8Duld0dEIxKOVA5dbhf3m1jbLydV9gjc9G0VuV+r9kStJN3XQ52rlrt3qINl1siYUFspYT/rIZb3kHDKlXlUJ+7ME6Ab86BaG8VDAMfEPBxcw2gJLKDvcO8p9tjr8FxkVyOtrO4ip7yhJC4B94xo0bBO/9znRchjQ6Xj1sZTMzxJaInFWwbZg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=7aJ5arrdqi4Nmzkvd9wQ/zBRN4PeDD+62gloXB82Tfk=; b=QuGT84kpepEp+gziCFbhk4CKuvMGpEI9e2UcwSP8qhNO42CEPPBq/ErSO23dK2WpSP2nMG//QfTzEW5ux20KjtaoD/Qmqwq6NTm6zRt0c2BJJTMnkMT8Wf5CoXs5MKQabaSigEj1x7Ak7oAt3fFLrVf/1gdogeTrNdBdFc07jWyNSCEz0zuBf/l/pRkQDGRrb3t6lhG5fRBOqPxBUq6s3xj1swzjf4CAy9I4bN9w7Jjr70X8zw+XzJ4m+ORArqL4Fl7PM1kjcGMC1ovIGXnuEwgG0C6jJfd80Gpb18zmZOpg2XaRc+S04FpwaW/SoPjwIhDy/BYIvSLH4tPFNvfTkQ== Received: from MN2PR04MB5981.namprd04.prod.outlook.com (2603:10b6:208:da::10) by MN2PR04MB6048.namprd04.prod.outlook.com (2603:10b6:208:e5::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Thu, 16 Sep 2021 20:30:31 +0000 Received: from MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb]) by MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb%5]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021 20:30:31 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v6 09/11] avfilter/textmod: Add textmod filter Thread-Index: AQHXqzmxAuQeDOXoMUa18WZpd3ermg== Date: Thu, 16 Sep 2021 20:30:31 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [kyGxIHbfo655CaDsiTfQDepl9/G3Vb2g] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 311aad41-926f-4056-710e-08d97950d46a x-ms-traffictypediagnostic: MN2PR04MB6048: x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 8tYu6/e8P3FHxa5HcSWz2jeBiCCrIaYq4Mh1990+9UpJRSj90HBHOlEFSxeZ1wdvU2evcELrG4GEn9tPzD+KlEnUKG9aNxF1XanUfIyT3jPJXZ6+MgLwk+0Aew8k8OVu4L5CIB67EMujyukLkvIZg7Y7xMhUAPbOZEfYLaaK1CRxtiZLMaCOSrqZ1hg7cAiGR/B8IbpbRWAjYoAmtUg6Rld50kiPMa/2JdLvEFS8/fc84doehr3YN8jkUI5gDQWDqIirOItDQa8viltQLvGufSOedilfZtGiEW82Ly2cRzpb0AUjG9LRvvDpm/WdXAz6rISlQiM93i8Thu8tPu5mXI19Jf5j9dCw/nKzaspqRusQr9XlZnsge3jbpl28MXtFJ2Gsp4P3BJmJVu5qR2UMIZyVdsCHgNaZae3/wV1VpKqOFNBgJX3bk+ybCvrAGLfejUV+OLC9qc9qjjQlSmauRkkM81wWA2fkQ19ToZMMjoA= x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: wNaZ/NcRoXis989tNkU1FlCufJ305cf+ol8PpdSsKpATjTKuCszJd8u1EAwdB0/m+WGPBiYd5Br0rr9+D9nq5dyRggaF2s7dJr9IO8fVid2ms6BRxnFRWpFl1NSzgIEuALdA4PsH9s37pPjm9RQvrA== x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-3174-20-msonline-outlook-529c7.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR04MB5981.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 311aad41-926f-4056-710e-08d97950d46a X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Sep 2021 20:30:31.6836 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR04MB6048 Subject: [FFmpeg-devel] [PATCH v6 09/11] avfilter/textmod: Add textmod filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: ubhof8v55tla - textmod {S -> S) Modify subtitle text in a number of ways Signed-off-by: softworkz --- doc/filters.texi | 64 +++++++ libavfilter/Makefile | 3 + libavfilter/allfilters.c | 1 + libavfilter/sf_textmod.c | 372 +++++++++++++++++++++++++++++++++++++++ 4 files changed, 440 insertions(+) create mode 100644 libavfilter/sf_textmod.c diff --git a/doc/filters.texi b/doc/filters.texi index 1c38ecd8ec..1832dc8847 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -25087,6 +25087,70 @@ existing filters using @code{--disable-filters}. Below is a description of the currently available subtitle filters. +@section textmod + +Modify subtitle text in a number of ways. + +It accepts the following parameters: + +@table @option +@item mode +The kind of text modification to apply + +Supported operation modes are: + +@table @var +@item 0, leet +Convert subtitle text to 'leet speak'. It's primarily useful for testing as the modification will be visible with almost all text lines. +@item 1, to_upper +Change all text to upper case. Might improve readability. +@item 2, to_lower +Change all text to lower case. +@item 3, replace_chars +Replace one or more characters. Requires the find and replace parameters to be specified. +Both need to be equal in length. +The first char in find is replaced by the first char in replace, same for all subsequent chars. +@item 4, remove_chars +Remove certain characters. Requires the find parameter to be specified. +All chars in the find parameter string will be removed from all subtitle text. +@item 5, replace_words +Replace one or more words. Requires the find and replace parameters to be specified. Multiple words must be separated by the delimiter char specified vie the separator parameter (default: ','). +The number of words in the find and replace parameters needs to be equal. +The first word in find is replaced by the first word in replace, same for all subsequent words +@item 6, remove_words +Remove certain words. Requires the find parameter to be specified. Multiple words must be separated by the delimiter char specified vie the separator parameter (default: ','). +All words in the find parameter string will be removed from all subtitle text. +@end table + +@item find +Required for replace_chars, remove_chars, replace_words and remove_words. + +@item replace +Required for replace_chars and replace_words. + +@item separator +Delimiter character for words. Used with replace_words and remove_words- Must be a single character. +The default is '.'. + +@end table + +@subsection Examples + +@itemize +@item +Change all characters to upper case while keeping all styles and animations: +@example +ffmpeg -i "https://streams.videolan.org/ffmpeg/mkv_subtitles.mkv" -filter_complex "[0:s]textmod=mode=to_upper" -map 0 -y out.mkv +@end example +@item +Mark the 100-pixel-wide region on the left edge of the frame as very +uninteresting (to be encoded at much lower quality than the rest of +the frame). +@example +addroi=0:0:100:ih:+1/5 +@end example +@end itemize + @section graphicsub2video Renders graphic subtitles as video frames. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index c0b1cc7001..630f701a5f 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -536,6 +536,9 @@ OBJS-$(CONFIG_YUVTESTSRC_FILTER) += vsrc_testsrc.o OBJS-$(CONFIG_NULLSINK_FILTER) += vsink_nullsink.o +# subtitle filters +OBJS-$(CONFIG_TEXTMOD_FILTER) += sf_textmod.o + # multimedia filters OBJS-$(CONFIG_ABITSCOPE_FILTER) += avf_abitscope.o OBJS-$(CONFIG_ADRAWGRAPH_FILTER) += f_drawgraph.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 8543fa22e9..63e41b637d 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -526,6 +526,7 @@ extern const AVFilter ff_avf_showvolume; extern const AVFilter ff_avf_showwaves; extern const AVFilter ff_avf_showwavespic; extern const AVFilter ff_vaf_spectrumsynth; +extern const AVFilter ff_sf_textmod; extern const AVFilter ff_svf_graphicsub2video; extern const AVFilter ff_svf_textsub2video; diff --git a/libavfilter/sf_textmod.c b/libavfilter/sf_textmod.c new file mode 100644 index 0000000000..6a29a2d3c0 --- /dev/null +++ b/libavfilter/sf_textmod.c @@ -0,0 +1,372 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * text subtitle filter which allows to modify subtitle text in several ways + */ + +#include + +#include "libavutil/avassert.h" +#include "libavutil/avstring.h" +#include "libavutil/opt.h" +#include "avfilter.h" +#include "internal.h" +#include "libavcodec/avcodec.h" +#include "libavcodec/ass_split.h" + +static const char* leet_src = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"; +static const char* leet_dst = "abcd3f6#1jklmn0pq257uvwxyzAB(D3F6#1JKLMN0PQ257UVWXYZ"; + +enum TextModOperation { + OP_LEET, + OP_TO_UPPER, + OP_TO_LOWER, + OP_REPLACE_CHARS, + OP_REMOVE_CHARS, + OP_REPLACE_WORDS, + OP_REMOVE_WORDS, + NB_OPS, +}; + +typedef struct TextModContext { + const AVClass *class; + enum AVSubtitleType format; + enum TextModOperation operation; + char *find; + char *replace; + char *separator; + char **find_list; + int nb_find_list; + char **replace_list; + int nb_replace_list; +} TextModContext; + +static char **split_string(char *source, int *nb_elems, char delim) +{ + char **list = NULL; + char *temp = NULL; + char *ptr = av_strtok(source, &delim, &temp); + + while (ptr) { + av_dynarray_add(&list, nb_elems, ptr); + if (!list) + return NULL; + + ptr = av_strtok(NULL, &delim, &temp); + } + + return list; +} + +static int init(AVFilterContext *ctx) +{ + TextModContext *s = ctx->priv; + + switch (s->operation) { + case OP_REPLACE_CHARS: + case OP_REMOVE_CHARS: + case OP_REPLACE_WORDS: + case OP_REMOVE_WORDS: + if (!s->find || !strlen(s->find)) { + av_log(ctx, AV_LOG_ERROR, "Selected mode requires the 'find' parameter to be specified"); + return AVERROR(EINVAL); + } + break; + } + + switch (s->operation) { + case OP_REPLACE_CHARS: + case OP_REPLACE_WORDS: + if (!s->replace || !strlen(s->replace)) { + av_log(ctx, AV_LOG_ERROR, "Selected mode requires the 'replace' parameter to be specified"); + return AVERROR(EINVAL); + } + break; + } + + if (s->operation == OP_REPLACE_CHARS && strlen(s->find) != strlen(s->replace)) { + av_log(ctx, AV_LOG_ERROR, "Selected mode requires the 'find' and 'replace' parameters to have the same length"); + return AVERROR(EINVAL); + } + + if (s->operation == OP_REPLACE_WORDS || s->operation == OP_REMOVE_WORDS) { + if (!s->separator || strlen(s->separator) != 1) { + av_log(ctx, AV_LOG_ERROR, "Selected mode requires a single separator char to be specified"); + return AVERROR(EINVAL); + } + + s->find_list = split_string(s->find, &s->nb_find_list, *s->separator); + if (!s->find_list) + return AVERROR(ENOMEM); + + if (s->operation == OP_REPLACE_WORDS) { + + s->replace_list = split_string(s->replace, &s->nb_replace_list, *s->separator); + if (!s->replace_list) + return AVERROR(ENOMEM); + + if (s->nb_find_list != s->nb_replace_list) { + av_log(ctx, AV_LOG_ERROR, "The number of words in 'find' and 'replace' needs to be equal"); + return AVERROR(EINVAL); + } + } + } + + return 0; +} + +static void uninit(AVFilterContext *ctx) +{ + TextModContext *s = ctx->priv; + int i; + + s->nb_find_list = 0; + av_freep(&s->find_list); + + s->nb_replace_list = 0; + av_freep(&s->replace_list); +} + +static int query_formats(AVFilterContext *ctx) +{ + AVFilterFormats *formats; + AVFilterLink *inlink = ctx->inputs[0]; + AVFilterLink *outlink = ctx->outputs[0]; + static const enum AVSubtitleType subtitle_fmts[] = { AV_SUBTITLE_FMT_ASS, AV_SUBTITLE_FMT_NONE }; + int ret; + + /* set input subtitle format */ + formats = ff_make_format_list(subtitle_fmts); + if ((ret = ff_formats_ref(formats, &inlink->outcfg.formats)) < 0) + return ret; + + /* set output video format */ + if ((ret = ff_formats_ref(formats, &outlink->incfg.formats)) < 0) + return ret; + + return 0; +} + +static char *process_text(TextModContext *s, char *text) +{ + const char *char_src = s->find; + const char *char_dst = s->replace; + char *result = NULL; + int escape_level = 0, k = 0; + + switch (s->operation) { + case OP_LEET: + case OP_REPLACE_CHARS: + + if (s->operation == OP_LEET) { + char_src = leet_src; + char_dst = leet_dst; + } + + result = av_strdup(text); + if (!result) + return NULL; + + for (size_t n = 0; n < strlen(result); n++) { + if (result[n] == '{') + escape_level++; + + if (!escape_level) { + for (size_t t = 0; t < FF_ARRAY_ELEMS(char_src); t++) { + if (result[n] == char_src[t]) { + result[n] = char_dst[t]; + break; + } + } + } + + if (result[n] == '}') + escape_level--; + } + + break; + case OP_TO_UPPER: + case OP_TO_LOWER: + + result = av_strdup(text); + if (!result) + return NULL; + + for (size_t n = 0; n < strlen(result); n++) { + if (result[n] == '{') + escape_level++; + if (!escape_level) + result[n] = s->operation == OP_TO_LOWER ? av_tolower(result[n]) : av_toupper(result[n]); + if (result[n] == '}') + escape_level--; + } + + break; + case OP_REMOVE_CHARS: + + result = av_strdup(text); + if (!result) + return NULL; + + for (size_t n = 0; n < strlen(result); n++) { + int skip_char = 0; + + if (result[n] == '{') + escape_level++; + + if (!escape_level) { + for (size_t t = 0; t < FF_ARRAY_ELEMS(char_src); t++) { + if (result[n] == char_src[t]) { + skip_char = 1; + break; + } + } + } + + if (!skip_char) + result[k++] = result[n]; + + if (result[n] == '}') + escape_level--; + } + + result[k] = 0; + + break; + case OP_REPLACE_WORDS: + case OP_REMOVE_WORDS: + + result = av_strdup(text); + if (!result) + return NULL; + + for (int n = 0; n < s->nb_find_list; n++) { + char *tmp = result; + const char *replace = (s->operation == OP_REPLACE_WORDS) ? s->replace_list[n] : ""; + + result = av_strireplace(result, s->find_list[n], replace); + if (!result) + return NULL; + + av_free(tmp); + } + + break; + } + + return result; +} + +static char *process_dialog(TextModContext *s, char *ass_line) +{ + ASSDialog *dialog = ff_ass_split_dialog(NULL, ass_line); + char *result, *text; + + if (!dialog) + return NULL; + + text = process_text(s, dialog->text); + if (!text) + return NULL; + + result = ff_ass_get_dialog(dialog->readorder, dialog->layer, dialog->style, dialog->name, text); + + av_free(text); + ff_ass_free_dialog(&dialog); + return result; +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *frame) +{ + TextModContext *s = inlink->dst->priv; + AVFilterLink *outlink = inlink->dst->outputs[0]; + int ret; + + outlink->format = inlink->format; + + av_frame_make_writable(frame); + + if (!frame) + return AVERROR(ENOMEM); + + for (unsigned i = 0; i < frame->num_subtitle_areas; i++) { + + AVSubtitleArea *area = frame->subtitle_areas[i]; + + if (area->ass) { + char *tmp = area->ass; + area->ass = process_dialog(s, area->ass); + av_free(tmp); + if (!area->ass) + return AVERROR(ENOMEM); + } + } + + return ff_filter_frame(outlink, frame); +} + +#define OFFSET(x) offsetof(TextModContext, x) +#define FLAGS (AV_OPT_FLAG_SUBTITLE_PARAM | AV_OPT_FLAG_FILTERING_PARAM) + +static const AVOption textmod_options[] = { + { "mode", "set operation mode", OFFSET(operation), AV_OPT_TYPE_INT, {.i64=OP_LEET}, OP_LEET, NB_OPS-1, FLAGS, "mode" }, + { "leet", "convert text to 'leet speak'", 0, AV_OPT_TYPE_CONST, {.i64=OP_LEET}, 0, 0, FLAGS, "mode" }, + { "to_upper", "change to upper case", 0, AV_OPT_TYPE_CONST, {.i64=OP_TO_UPPER}, 0, 0, FLAGS, "mode" }, + { "to_lower", "change to lower case", 0, AV_OPT_TYPE_CONST, {.i64=OP_TO_LOWER}, 0, 0, FLAGS, "mode" }, + { "replace_chars", "replace characters", 0, AV_OPT_TYPE_CONST, {.i64=OP_REPLACE_CHARS}, 0, 0, FLAGS, "mode" }, + { "remove_chars", "remove characters", 0, AV_OPT_TYPE_CONST, {.i64=OP_REMOVE_CHARS}, 0, 0, FLAGS, "mode" }, + { "replace_words", "replace words", 0, AV_OPT_TYPE_CONST, {.i64=OP_REPLACE_WORDS}, 0, 0, FLAGS, "mode" }, + { "remove_words", "remove words", 0, AV_OPT_TYPE_CONST, {.i64=OP_REMOVE_WORDS}, 0, 0, FLAGS, "mode" }, + { "find", "chars/words to find or remove", OFFSET(find), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, FLAGS, NULL }, + { "replace", "chars/words to replace", OFFSET(replace), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, FLAGS, NULL }, + { "separator", "word separator (default: ',')", OFFSET(separator), AV_OPT_TYPE_STRING, {.str = ","}, 0, 0, FLAGS, NULL }, + { NULL }, +}; + +AVFILTER_DEFINE_CLASS(textmod); + +static const AVFilterPad inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .filter_frame = filter_frame, + }, +}; + +static const AVFilterPad outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + }, +}; + +const AVFilter ff_sf_textmod = { + .name = "textmod", + .description = NULL_IF_CONFIG_SMALL("Modify subtitle text in several ways"), + .init = init, + .uninit = uninit, + .query_formats = query_formats, + .priv_size = sizeof(TextModContext), + .priv_class = &textmod_class, + FILTER_INPUTS(inputs), + FILTER_OUTPUTS(outputs), +}; From patchwork Thu Sep 16 20:30:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 30286 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp1583085iov; Thu, 16 Sep 2021 13:32:16 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxkiYITkoRNO9FmsdVRxKv9uZEpuPrYeu92pux5LQwAZFqWnRG+JROpun+3YDz9v8lLSMI3 X-Received: by 2002:a17:906:2887:: with SMTP id o7mr8128325ejd.425.1631824336092; Thu, 16 Sep 2021 13:32:16 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id k28si4507830edk.420.2021.09.16.13.32.15; Thu, 16 Sep 2021 13:32:16 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=cruU3K8W; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 287E468B09E; Thu, 16 Sep 2021 23:30:40 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11olkn2034.outbound.protection.outlook.com [40.92.19.34]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CDB5168B09E for ; Thu, 16 Sep 2021 23:30:37 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fAYQntzNebnJGYv5BBvDJvvDN/iuTY+rqgqp9KwmHcWcPZRvKkBWm+t/7SAIyewj9ePo8wfOw+LnuLgFKhcnJ48v7tA/oidLMW0CfSBkrbWq9oa7vDX2nkGgrQ+sMAjulL2lDw5272RqWgHm8+BoAsq1D2F9oTEmRHl0LMxjFvbHG5HWVFFXJbO2rstN3XpW69auRhNSK/aX4AM3B1bO/6h5ztCi+idB5pxf59VdvOw6bOC/8LM6yoEKIgfn4SFDjN+6Ymy3tp72ZjBMf73fTlzeLsoVTNrGTjanxASQd3K2YedtY7OhMZwEuGgVsO+l3V9XjeL/EsCxoRFc7RfRbQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=aqv6slxUggHk3KxScMa1FvkVo4oJF/4VN9icEPRh++U=; b=gkH39cnS8kVFY/UYql+tU5++wayb9xDzyrJWwJOiHfr7CTd31cnb2zim7FhG2VZcFGpIVnK9DSDmg/HgxWKeNwup+xhu1HHdrDUt8Wh9tuJzVHRfHy+mzdMIUecY2zWSiBn9z+nwmqUJ1ipmwRF+UoHsC1VdDU18ga1sbsHtZ/XRnU+ZuqXtvi6dyiufbdKPrqlLxLLAD8Gl1LaHpUT7qRxJ8prP2pJwI41/JUhhOmaqvAm17R9LGnKMQ+pOUBaegq12R4Rv4nunBQOeKCyblcQV3aGcwDDeucnIC3Q9Hdjhx+2SFwHE3EQdn1OOfZvq9BVlNA1Mpy0TZqWSwTatdw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=aqv6slxUggHk3KxScMa1FvkVo4oJF/4VN9icEPRh++U=; b=cruU3K8Way/l2/O7ki+Sm2vxdxIwR3+BlwTRdl/bc+N/5bhCVNabQEumEog5wystnzipoBhiu4JU4USyNwLEcUSh7UUVmnkMjgMiI83XlVtIGfDpoSs5+6zdm72uvfvibxLDCxvb06yicw9rbFzPcjAqsQIS7/NrB3fCIWJB9OwJd4g2HE3ftdE3k3KXfGQl4KwlYxRX3QdmLTIZ1gM5w9EuLIj2XryWPEkpCvirc3A5J2ufbTSlpu+9b2+BgWtdFw4cKgh6sTb44WrLaZuE61zKCrCxxAhxmUweJzcG5rmuKqN8Jp9rqEaqFBpHAlC6LxSfbmmBfLWL+keQwayNLw== Received: from MN2PR04MB5981.namprd04.prod.outlook.com (2603:10b6:208:da::10) by MN2PR04MB6048.namprd04.prod.outlook.com (2603:10b6:208:e5::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Thu, 16 Sep 2021 20:30:34 +0000 Received: from MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb]) by MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb%5]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021 20:30:34 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v6 10/11] avcodec/ass_split: Extend ass dialog parsing Thread-Index: AQHXqzmzbgPdnK4SRUqXgCumVhZ6Gg== Date: Thu, 16 Sep 2021 20:30:34 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [6E/4WBbQunRf4nrXJYgPBgAAy9VGtkFx] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: a5bb2224-895d-4c57-409f-08d97950d634 x-ms-traffictypediagnostic: MN2PR04MB6048: x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: IiQdrkgfrTh8FGWd5MO/JO1zkI+8RPAYqyC1PnaVTfkGyRVxEhyYjm4SzKgGKiMD0Cd0u0fW85bbeYxIFnLLb4O4ZIhQVfvzRtZKENYez21gmCWcy/aOgnTjlQDQfnhc2rxzKDhtJYcQfHvksbPu6u0o9YBcr3fXtmESWMAFKJsRtqSmU5H1m+2cE2o+z98af0a5ZxIauK/k/K4WnLH8VPNY0PQl7y+qug3ioMA9DAGE7cOD0Mk35+Lm4WFQ075l5oGNATf/o2cN4F3pUs1V8KL7IKj7ePpgRpVcQiuiexw6tQH0UUQ6bmUjFxkKnRj/kfIV+WbBJMeXsTKIIjhlGRGIO0nSzZkoyzDi5mxZqqDgPkP3jyiQXN+MG1OfY5bwCDohFWBnSDw2O9eogpI8Ak+45lmlu5322+yRsoDPq4g4rtUPPwMfFUOudz16rNpR x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: xdscrV374IwlAJqrIbew0E0h8f/dBVkcKlCQmt45cmYTUt0rjHy0v4bt97tIYc084cesY9uZj/djpXp5IhkAiDRSztThaqVHYMMmsL0itCZpTF+XZGy0CnjLaYek01dJmZICI/8lm4APGs3x8frK/A== x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-3174-20-msonline-outlook-529c7.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR04MB5981.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: a5bb2224-895d-4c57-409f-08d97950d634 X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Sep 2021 20:30:34.7199 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR04MB6048 Subject: [FFmpeg-devel] [PATCH v6 10/11] avcodec/ass_split: Extend ass dialog parsing X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: QcAQmW58N+P9 Signed-off-by: softworkz --- libavcodec/ass_split.c | 12 ++++++++++-- libavcodec/ass_split.h | 2 ++ 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/libavcodec/ass_split.c b/libavcodec/ass_split.c index 05c5453e53..e58585d3da 100644 --- a/libavcodec/ass_split.c +++ b/libavcodec/ass_split.c @@ -497,8 +497,8 @@ int ff_ass_split_override_codes(const ASSCodesCallbacks *callbacks, void *priv, while (*buf == '\\') { char style[2], c[2], sep[2], c_num[2] = "0", tmp[128] = {0}; unsigned int color = 0xFFFFFFFF; - int len, size = -1, an = -1, alpha = -1; - int x1, y1, x2, y2, t1 = -1, t2 = -1; + int len, size = -1, an = -1, alpha = -1, scale = 0; + int x1, y1, x2, y2, t1 = -1, t2 = -1, accel = 1; if (sscanf(buf, "\\%1[bisu]%1[01\\}]%n", style, c, &len) > 1) { int close = c[0] == '0' ? 1 : c[0] == '1' ? 0 : -1; len += close != -1; @@ -546,6 +546,14 @@ int ff_ass_split_override_codes(const ASSCodesCallbacks *callbacks, void *priv, } else if (sscanf(buf, "\\org(%d,%d)%1[\\}]%n", &x1, &y1, sep, &len) > 2) { if (callbacks->origin) callbacks->origin(priv, x1, y1); + } else if (sscanf(buf, "\\t(%d,%d,%1[\\}]%n", &t1, &t2, sep, &len) > 2 || + sscanf(buf, "\\t(%d,%d,%d,%1[\\}]%n", &t1, &t2, &accel, sep, &len) > 3) { + if (callbacks->animate) + callbacks->animate(priv, t1, t2, accel, tmp); + } else if (sscanf(buf, "\\p%1[\\}]%n", sep, &len) > 0 || + sscanf(buf, "\\p%u%1[\\}]%n", &scale, sep, &len) > 1) { + if (callbacks->drawing_mode) + callbacks->drawing_mode(priv, scale); } else { len = strcspn(buf+1, "\\}") + 2; /* skip unknown code */ } diff --git a/libavcodec/ass_split.h b/libavcodec/ass_split.h index a45fb9b8a1..bda7bb27db 100644 --- a/libavcodec/ass_split.h +++ b/libavcodec/ass_split.h @@ -156,7 +156,9 @@ typedef struct { * @{ */ void (*move)(void *priv, int x1, int y1, int x2, int y2, int t1, int t2); + void (*animate)(void *priv, int t1, int t2, int accel, char *style); void (*origin)(void *priv, int x, int y); + void (*drawing_mode)(void *priv, int scale); /** @} */ /** From patchwork Thu Sep 16 20:30:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 30285 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp1583164iov; Thu, 16 Sep 2021 13:32:25 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwKJAcGQyYnJL5uRKqIP9FeW30YPXrZa6MxoH3EuAw8DNZ2tATIRfbmTD22IqjeXRdrhlZi X-Received: by 2002:a17:906:a3d2:: with SMTP id ca18mr8309930ejb.274.1631824345549; Thu, 16 Sep 2021 13:32:25 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id c3si4679630ejc.450.2021.09.16.13.32.25; Thu, 16 Sep 2021 13:32:25 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=lVn3whmi; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 474E368B0B2; Thu, 16 Sep 2021 23:30:49 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11olkn2105.outbound.protection.outlook.com [40.92.19.105]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6EEAC68B0F0 for ; Thu, 16 Sep 2021 23:30:42 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JuLoAECJ9qMmAELIZopuK1k7Fe1q8VLMOFEzaeG0jkI2mIJkLOBrtuVJBq5ivsMDUWDCS8V1jwN0U7vgQFQtV4EJQsZYm5ppzi28f8yXQ6Rk7WaPZrAcmAR260Od6+bv5EwYl4ZVm6bv1KMaxjFPrZEvWFl+JqE/jTPwksXJ83edistoeVWlA18c6ZWo6l9hBWLWmMkfLZgi5uTsnF2S7OSxWmoT04TR2jDnfSlENacoOemkwF+hBz4YjD1kwvi7weSjsZroE0mII1cdnolVZVdDLNKciVhoM0PKFtmqqivEm1b9YYNRvQcE6qGgGugAlUWKGHTHxrNrgUnA+iNJlA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=ekkWCjtY6x80lHtruIf3oicG7ybmC79W2fUW7QkhxgM=; b=Kmj15zr1JXOK61MMVdaxaCF/vkx1YDX5dBdnN9ahAGkW5vej04wH8up7Z1ArqqcMqbDnZIY8E4EsPRENpcnUlppKCA+13fAro+VvJombJGQRgIdzinVnGyrN+pl82LRsHqpbaNJuzTZjmTmCZ0TG8iL/gdMwFau7Vl0GKNYFW21JaI8bbVECQ4+nQgeWA/I0u75n13Sf0s6htSdh0ZbINOtNbAVDMILWcqKzT6ARd+/yOkny75+pYrrvZKjpAvf6e2aHF3Wz+7ZUSfB0sPxnVR1mrPnSEzF1KH35AWEW2aIn+oCiIvULfS9NtfAPByjMo3sWkj+ZYCefMXz/jL/4PA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ekkWCjtY6x80lHtruIf3oicG7ybmC79W2fUW7QkhxgM=; b=lVn3whmiXHlyA3tphaHjMREy4oqBVVaKq0ACtwzXDQsfEVN3CEy80JSYFt/D9B+5c0Bb95U7J6zynNZjC9gGxbmvAmAsjoOV0ai+ExWB7sPHTtvfLkndUhQLfsiCDAlJnZQbaKKQsls/NyAqMt4XWarJumki1eD9DBTOe3vvcG0zQQd1kW+nvnoPcuYhPkrc4oPCG3cMD4PVsKYq0Khgj4UZOLeBHEQk24J9SnzAJK6HGdRgALGkO1n9lke/FRvFiNflc6KBs9ODowKNSc80i3LshSwd7Ca9zjGhIqVyOHK2YpHa1mzDNiz8z3GpwrTFk4/7i+rrrBLxufAuyfIwLw== Received: from MN2PR04MB5981.namprd04.prod.outlook.com (2603:10b6:208:da::10) by MN2PR04MB6048.namprd04.prod.outlook.com (2603:10b6:208:e5::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Thu, 16 Sep 2021 20:30:40 +0000 Received: from MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb]) by MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb%5]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021 20:30:40 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v6 11/11] avfilter/stripstyles: Add stripstyles filter Thread-Index: AQHXqzm2npN8qp+SM0mmUHG5M6gG6A== Date: Thu, 16 Sep 2021 20:30:40 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [te3yWS8BQ7hqJhY0NhCIRkibD7i0tTOr] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 2a6cd1db-7557-48a2-5aef-08d97950d96e x-ms-traffictypediagnostic: MN2PR04MB6048: x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 7DosyIv82iFORHn5Uoe2/oCc2nOg6amDIm09aEZQy70hj7OyBuBZKq9+vCHJlyjglXPkdt8sfnRLMrtoQM50GO7INKVORGGbe2lYweIhm01N1KBT+eYhy2hU9S5AQ37uFRjJEOVebHWdao11bqiN5G8rajxsuJTDjZ0dZF5HUgap9/pr45wzaH0uBCzxbZYmZweYbXCLVZt+lHIPxt26RsRgIFPFY3paIYwM/5hIo8JvPCtTbbmRm+Pi+c0I7LiIFslgp5RisbHD3SuZMIrN1RzmJBrY8xLqTlgBTGFwzuECfUrLB5TPh/7uIN7T9+OW2h5X78/MEsZKy+Lz+X8Ez7UgBnIfw+vOBaEXnhKfttwdIbt4CPtDWRELTDr5kJ3Q02aKuoLtan03lGFbqQ1sE/Cxuhg6UGAzQiq0izkyuHjx3sNo7BpDytuKjBcNlmpkXK/PeAhPM7fFSgskKXzjnsyOPMdHfkjvzq6NiYEAvfk= x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: JqOx9gMzSmZcQ/g4wZQZu9wYGychBHmVMjEGISrcY/tjcRp/1+haqP1vXWHcs+vW7SQyRrSeHyRJ657meBBG87EuDLjELCKsDH4uHbDg0XMb0O3ZO2VwYHkXOQ6BCVFBQ08E5WBb8v0dFT+u/OOf+g== x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-3174-20-msonline-outlook-529c7.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR04MB5981.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 2a6cd1db-7557-48a2-5aef-08d97950d96e X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Sep 2021 20:30:40.0699 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR04MB6048 Subject: [FFmpeg-devel] [PATCH v6 11/11] avfilter/stripstyles: Add stripstyles filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: VXs1m7/UHaYk - stripstyles {S -> S) Remove all inline styles from subtitle events Signed-off-by: softworkz --- doc/filters.texi | 24 ++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/sf_stripstyles.c | 211 +++++++++++++++++++++++++++++++++++ 4 files changed, 237 insertions(+) create mode 100644 libavfilter/sf_stripstyles.c diff --git a/doc/filters.texi b/doc/filters.texi index 1832dc8847..c8bf444d2c 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -25087,6 +25087,30 @@ existing filters using @code{--disable-filters}. Below is a description of the currently available subtitle filters. +@section stripstyles + +Remove all inline styles from subtitle events. + +It accepts the following parameters: + +@table @option +@item remove_animated +Also remove text which is subject to animation (default: true) +Usually, animated text elements are used used in addition to static subtitle lines for creating effects, so in most cases it is safe to remove the animation content. +If subtitle text is missing, try setting this to false. + +@end table + +@subsection Examples + +@itemize +@item +Change all characters to upper case while keeping all styles and animations: +@example +ffmpeg -i "https://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.mkv" -filter_complex "[0:1]stripstyles" -map 0 output.mkv +@end example +@end itemize + @section textmod Modify subtitle text in a number of ways. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 630f701a5f..94ec11c4ce 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -538,6 +538,7 @@ OBJS-$(CONFIG_NULLSINK_FILTER) += vsink_nullsink.o # subtitle filters OBJS-$(CONFIG_TEXTMOD_FILTER) += sf_textmod.o +OBJS-$(CONFIG_STRIPSTYLES_FILTER) += sf_stripstyles.o # multimedia filters OBJS-$(CONFIG_ABITSCOPE_FILTER) += avf_abitscope.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 63e41b637d..88dd97adf1 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -526,6 +526,7 @@ extern const AVFilter ff_avf_showvolume; extern const AVFilter ff_avf_showwaves; extern const AVFilter ff_avf_showwavespic; extern const AVFilter ff_vaf_spectrumsynth; +extern const AVFilter ff_sf_stripstyles; extern const AVFilter ff_sf_textmod; extern const AVFilter ff_svf_graphicsub2video; extern const AVFilter ff_svf_textsub2video; diff --git a/libavfilter/sf_stripstyles.c b/libavfilter/sf_stripstyles.c new file mode 100644 index 0000000000..15c00e73b1 --- /dev/null +++ b/libavfilter/sf_stripstyles.c @@ -0,0 +1,211 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * text subtitle filter which removes inline-styles from subtitles + */ + +#include + +#include "libavutil/avassert.h" +#include "libavutil/opt.h" +#include "avfilter.h" +#include "internal.h" +#include "libavcodec/avcodec.h" +#include "libavcodec/ass_split.h" + +typedef struct StripStylesContext { + const AVClass *class; + enum AVSubtitleType format; + int remove_animated; +} StripStylesContext; + +typedef struct DialogContext { + const AVClass *class; + const StripStylesContext* ss_ctx; + AVBPrint buffer; + int drawing_scale; + int is_animated; +} DialogContext; + +static void dialog_text_cb(void *priv, const char *text, int len) +{ + DialogContext *s = priv; + if (!s->drawing_scale && (!s->is_animated || !s->ss_ctx->remove_animated)) + av_bprint_append_data(&s->buffer, text, len); +} + +static void dialog_new_line_cb(void *priv, int forced) +{ + DialogContext *s = priv; + if (!s->drawing_scale && !s->is_animated) + av_bprint_append_data(&s->buffer, forced ? "\\N" : "\\n", 2); +} + +static void dialog_drawing_mode_cb(void *priv, int scale) +{ + DialogContext *s = priv; + s->drawing_scale = scale; +} + +static void dialog_animate_cb(void *priv, int t1, int t2, int accel, char *style) +{ + DialogContext *s = priv; + s->is_animated = 1; +} + +static void dialog_move_cb(void *priv, int x1, int y1, int x2, int y2, int t1, int t2) +{ + DialogContext *s = priv; + if (t1 >= 0 || t2 >= 0) + s->is_animated = 1; +} + +static const ASSCodesCallbacks dialog_callbacks = { + .text = dialog_text_cb, + .new_line = dialog_new_line_cb, + .drawing_mode = dialog_drawing_mode_cb, + .animate = dialog_animate_cb, + .move = dialog_move_cb, +}; + +static int query_formats(AVFilterContext *ctx) +{ + AVFilterFormats *formats; + AVFilterLink *inlink = ctx->inputs[0]; + AVFilterLink *outlink = ctx->outputs[0]; + static const enum AVSubtitleType subtitle_fmts[] = { AV_SUBTITLE_FMT_ASS, AV_SUBTITLE_FMT_NONE }; + int ret; + + /* set input subtitle format */ + formats = ff_make_format_list(subtitle_fmts); + if ((ret = ff_formats_ref(formats, &inlink->outcfg.formats)) < 0) + return ret; + + /* set output video format */ + if ((ret = ff_formats_ref(formats, &outlink->incfg.formats)) < 0) + return ret; + + return 0; +} + +static char *ass_get_line(int readorder, int layer, const char *style, + const char *speaker, const char *effect, const char *text) +{ + return av_asprintf("%d,%d,%s,%s,0,0,0,%s,%s", + readorder, layer, style ? style : "Default", + speaker ? speaker : "", effect, text); +} + +static char *process_dialog(StripStylesContext *s, const char *ass_line) +{ + ASSDialog *dialog = ff_ass_split_dialog(NULL, ass_line); + char *result = NULL, *text; + DialogContext dlg_ctx = { 0 }; + + if (!dialog) + return NULL; + + dlg_ctx.ss_ctx = s; + + av_bprint_init(&dlg_ctx.buffer, 0, AV_BPRINT_SIZE_UNLIMITED); + + ff_ass_split_override_codes(&dialog_callbacks, &dlg_ctx, dialog->text); + + av_bprint_finalize(&dlg_ctx.buffer, &text); + + if (text && strlen(text) > 0) + result = ass_get_line(dialog->readorder, dialog->layer, dialog->style, dialog->name, dialog->effect, text); + + av_free(text); + ff_ass_free_dialog(&dialog); + + return result; +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *frame) +{ + StripStylesContext *s = inlink->dst->priv; + AVFilterLink *outlink = inlink->dst->outputs[0]; + + outlink->format = inlink->format; + + av_frame_make_writable(frame); + if (!frame) + return AVERROR(ENOMEM); + + for (unsigned i = 0; i < frame->num_subtitle_areas; i++) { + + AVSubtitleArea *area = frame->subtitle_areas[i]; + + if (area->ass) { + char *tmp = area->ass; + area->ass = process_dialog(s, area->ass); + + if (area->ass) { + av_log(inlink->dst, AV_LOG_DEBUG, "original: %s\n", tmp); + av_log(inlink->dst, AV_LOG_DEBUG, "stripped: %s\n", area->ass); + } + else + area->ass = av_strdup(""); + + av_free(tmp); + } + } + + return ff_filter_frame(outlink, frame); +} + +#define OFFSET(x) offsetof(StripStylesContext, x) +#define FLAGS (AV_OPT_FLAG_SUBTITLE_PARAM | AV_OPT_FLAG_FILTERING_PARAM) + +static const AVOption stripstyles_options[] = { + { "remove_animated", "remove animated text (default: yes)", OFFSET(remove_animated), AV_OPT_TYPE_BOOL, {.i64 = 1 }, 0, 1, FLAGS, 0 }, + { NULL }, +}; + +AVFILTER_DEFINE_CLASS(stripstyles); + +static const AVFilterPad inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .filter_frame = filter_frame, + }, +}; + +static const AVFilterPad outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + }, +}; + +const AVFilter ff_sf_stripstyles = { + .name = "stripstyles", + .description = NULL_IF_CONFIG_SMALL("Strip subtitle inline styles"), + .query_formats = query_formats, + .priv_size = sizeof(StripStylesContext), + .priv_class = &stripstyles_class, + FILTER_INPUTS(inputs), + FILTER_OUTPUTS(outputs), +}; +