From patchwork Tue Sep 21 23:54:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 30447 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:6506:0:0:0:0:0 with SMTP id z6csp2918885iob; Tue, 21 Sep 2021 16:54:46 -0700 (PDT) X-Google-Smtp-Source: ABdhPJymbyV4IPT45Z9AgvxMgVH4w/52WsD2+FlVRmPWi1ey+I8RKmROqV2aMV/NK4oQMhwgrGAN X-Received: by 2002:a17:907:760d:: with SMTP id jx13mr35364869ejc.194.1632268486462; Tue, 21 Sep 2021 16:54:46 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id f13si185915edf.345.2021.09.21.16.54.45; Tue, 21 Sep 2021 16:54:46 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=sqBa1Adh; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8BADD68B08F; Wed, 22 Sep 2021 02:54:39 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12olkn2108.outbound.protection.outlook.com [40.92.22.108]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id F1A4168AF67 for ; Wed, 22 Sep 2021 02:54:30 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=C/JqJwT7YvCHuYwHrD5cI0t//pL2/6BPvLJFECJ6UDvYeJZ1y8ty9WP5RWwFTVHYBLvKVKbWZesnQ1aQ0idvGq/1WQMamOufl09XEtJVJGmDC4FtOJeFgpRYUrP/4LqsPshPqIUdCvNIg3GuMaNA++DdGmTiR2Wy+nyRIXE47Yq/gs/kCzAc+3dfbHetjPN99RwKIyMbCIAVgr/R1kPx7vU+ZgrqfP9GEhvDmCDKoT2WM1Y6FRe1gxbT1pDl/9oM+fMPno8+wiA0ZDxnkBp5Vj03KKc2uNkfpSQJXRKinfz5P+NhQyyMrm9lxwNy8qMJ1t0iSUPhcYgRDPh0tUDFBw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=EeVzqY+hwDa44MX2EuQ+6GuB3TvXxhtzSvYN64GCoLo=; b=oUwSGKvWjpwjxkIDSJ2sP+TT+ywd92A5qRfSRMpzKOBwUC1wpdcEVqIY/wcu62KGcWvLiCp4b0fYKhL9ybkecsjMdWFb5dpyAp+/l/2YzAfXgLIp+9BYfVyjBAaFHnFgTxSlstjFw7wluv0Bomc6VEwAjDy1csDuoO5Ni0ovwH0eSv6JPu7rg1lm9k0ial8E1kzgbI8Ld096YTSfvWJmWJeTpJokm37w00jcBy+pUeFuTTdmdN7rXx6d8DxiN9tYAYXDP0z2c5jsLABf+FnW/ZMUQkYK76//IdORYp5Du5ljTrPlMKLScXMbpDM+8JI/C+L6dCF2c+aTekwy/E/KIQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=EeVzqY+hwDa44MX2EuQ+6GuB3TvXxhtzSvYN64GCoLo=; b=sqBa1Adh5wnT7a/EO45XttYsxXkItHV2NnjHyPxeSBVIZBh6wvbbzRrWJsPsffw3PV5hgvch7NAD+1uirr30F2tY5y8MrMMNLRDWudqrjOQ/rueWMLQpSOh7z71FfdjB9i/Mwe1Oe4W/T3jxYqIaJgIqIsaaTolxCEJSESA1dRApXLSSDc9Du9UKIdEfQZXfr1WWiUASBYKH4bAPKvQM52y1w6GSk3f44UF4NqOUr82WhWmhW9vkvsDsIsNuzzKjHGk3QTlWG0GQV42c+g6R2xVvsVVcN0QaMi7YMY8wvgVvRjtuRkVHGslSZEcg1o7PldDQVvaISytTprxphOB0KQ== Received: from MN2PR04MB5981.namprd04.prod.outlook.com (2603:10b6:208:da::10) by MN2PR04MB5808.namprd04.prod.outlook.com (2603:10b6:208:a3::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Tue, 21 Sep 2021 23:54:23 +0000 Received: from MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb]) by MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb%5]) with mapi id 15.20.4523.019; Tue, 21 Sep 2021 23:54:23 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v8 01/13] global: Prepare AVFrame for subtitle handling Thread-Index: AQHXr0QABgkxrwAK6kaizxeiySzDmA== Date: Tue, 21 Sep 2021 23:54:23 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [FGhjVEV7lIetr84SLIdnF9kvHY+3LveN] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: bcead891-e93c-4f0a-f67b-08d97d5b234b x-ms-traffictypediagnostic: MN2PR04MB5808: x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: R9q+oTD1Ch0ydEKcIvdgSkXxpc99XZ71XO6Xbq8cyOzHGXcavotTX9aN15HWBSxOS2RkCFEGZJB5rPsKBLz7FGNKAesf01NwJY88jgb8hJIQoww1Pt+ZmAvmDlJoEN6i5GIYILrXzv1W2lA2/AuNnI/R2LUbbv++GAmLtu0v8GHex5EWe35Yh8xKlt8uo1fdMjYaghGQN16G2fF364m+j41ngqB+qOZwsUw/wAZ7u/lHGKWEoPWspoaM+FjYDwKuCb/VQFloeT0wTkYynD1xqOhm3h4J8zPfaKHhJY9ci3AETDgk0oE2++vZxqQurshS0uo8ZMd5RFZ/aZCerAaXiquPCVEbTUwmX0n/sFdNdLies8yrKMS4WOUOYTj4mAACyBVD4UTkYLTbv6DDwY8c3puYqR5N7OCIyormv9+T0d0hJc8j1f1KYmWKTQD4ZZoC x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: b0KAYmJyrYIxUnHDYYw67ZQCpRgMKWb62XB9qrQ+u0G156IgW/TW/9XsnUqKyyuF5dOpaH7RAeAG6NpmutAbf6DxadS+rzLbkQtnXr2JIf5HMo0qk4s10AxZjEArxaDmNPKhXpVLBz6IpkmIW7x4EA== x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-3174-20-msonline-outlook-529c7.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR04MB5981.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: bcead891-e93c-4f0a-f67b-08d97d5b234b X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Sep 2021 23:54:23.6762 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR04MB5808 Subject: [FFmpeg-devel] [PATCH v8 01/13] global: Prepare AVFrame for subtitle handling X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: v969BtoWK6PQ Root commit for adding subtitle filtering capabilities. In detail: - Add type (AVMediaType) field to AVFrame Replaces previous way of distinction which was based on checking width and height to determine whether a frame is audio or video - Add subtitle fields to AVFrame - Add new struct AVSubtitleArea, similar to AVSubtitleRect, but different allocation logic. Cannot and must not be used interchangeably, hence the new struct - Move enum AVSubtitleType, AVSubtitle and AVSubtitleRect to avutil - Add public-named members to enum AVSubtitleType (AV_SUBTITLE_FMT_) - Add avcodec_decode_subtitle3 which takes subtitle frames, serving as compatibility shim to legacy subtitle decoding - Add additional methods for conversion between old and new API Signed-off-by: softworkz --- libavcodec/avcodec.c | 19 ---- libavcodec/avcodec.h | 105 +++++++----------- libavcodec/decode.c | 24 +++- libavcodec/pgssubdec.c | 1 + libavcodec/utils.c | 11 ++ libavfilter/vf_subtitles.c | 27 +++-- libavformat/utils.c | 1 + libavutil/Makefile | 2 + libavutil/frame.c | 187 +++++++++++++++++++++++++++---- libavutil/frame.h | 93 +++++++++++++++- libavutil/subfmt.c | 219 +++++++++++++++++++++++++++++++++++++ libavutil/subfmt.h | 185 +++++++++++++++++++++++++++++++ 12 files changed, 754 insertions(+), 120 deletions(-) create mode 100644 libavutil/subfmt.c create mode 100644 libavutil/subfmt.h diff --git a/libavcodec/avcodec.c b/libavcodec/avcodec.c index 2dd7dd84e0..963f52c4bd 100644 --- a/libavcodec/avcodec.c +++ b/libavcodec/avcodec.c @@ -426,25 +426,6 @@ void avcodec_flush_buffers(AVCodecContext *avctx) av_bsf_flush(avci->bsf); } -void avsubtitle_free(AVSubtitle *sub) -{ - int i; - - for (i = 0; i < sub->num_rects; i++) { - av_freep(&sub->rects[i]->data[0]); - av_freep(&sub->rects[i]->data[1]); - av_freep(&sub->rects[i]->data[2]); - av_freep(&sub->rects[i]->data[3]); - av_freep(&sub->rects[i]->text); - av_freep(&sub->rects[i]->ass); - av_freep(&sub->rects[i]); - } - - av_freep(&sub->rects); - - memset(sub, 0, sizeof(*sub)); -} - av_cold int avcodec_close(AVCodecContext *avctx) { int i; diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index 7ee8bc2b7c..1d76352208 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -35,6 +35,7 @@ #include "libavutil/frame.h" #include "libavutil/log.h" #include "libavutil/pixfmt.h" +#include "libavutil/subfmt.h" #include "libavutil/rational.h" #include "codec.h" @@ -1674,7 +1675,7 @@ typedef struct AVCodecContext { /** * Header containing style information for text subtitles. - * For SUBTITLE_ASS subtitle type, it should contain the whole ASS + * For AV_SUBTITLE_FMT_ASS subtitle type, it should contain the whole ASS * [Script Info] and [V4+ Styles] section, plus the [Events] line and * the Format line following. It shouldn't include any Dialogue line. * - encoding: Set/allocated/freed by user (before avcodec_open2()) @@ -2238,63 +2239,8 @@ typedef struct AVHWAccel { * @} */ -enum AVSubtitleType { - SUBTITLE_NONE, - - SUBTITLE_BITMAP, ///< A bitmap, pict will be set - - /** - * Plain text, the text field must be set by the decoder and is - * authoritative. ass and pict fields may contain approximations. - */ - SUBTITLE_TEXT, - - /** - * Formatted text, the ass field must be set by the decoder and is - * authoritative. pict and text fields may contain approximations. - */ - SUBTITLE_ASS, -}; - #define AV_SUBTITLE_FLAG_FORCED 0x00000001 -typedef struct AVSubtitleRect { - int x; ///< top left corner of pict, undefined when pict is not set - int y; ///< top left corner of pict, undefined when pict is not set - int w; ///< width of pict, undefined when pict is not set - int h; ///< height of pict, undefined when pict is not set - int nb_colors; ///< number of colors in pict, undefined when pict is not set - - /** - * data+linesize for the bitmap of this subtitle. - * Can be set for text/ass as well once they are rendered. - */ - uint8_t *data[4]; - int linesize[4]; - - enum AVSubtitleType type; - - char *text; ///< 0 terminated plain UTF-8 text - - /** - * 0 terminated ASS/SSA compatible event line. - * The presentation of this is unaffected by the other values in this - * struct. - */ - char *ass; - - int flags; -} AVSubtitleRect; - -typedef struct AVSubtitle { - uint16_t format; /* 0 = graphics */ - uint32_t start_display_time; /* relative to packet pts, in ms */ - uint32_t end_display_time; /* relative to packet pts, in ms */ - unsigned num_rects; - AVSubtitleRect **rects; - int64_t pts; ///< Same as packet pts, in AV_TIME_BASE -} AVSubtitle; - /** * Return the LIBAVCODEC_VERSION_INT constant. */ @@ -2430,13 +2376,6 @@ int avcodec_open2(AVCodecContext *avctx, const AVCodec *codec, AVDictionary **op */ int avcodec_close(AVCodecContext *avctx); -/** - * Free all allocated data in the given subtitle struct. - * - * @param sub AVSubtitle to free. - */ -void avsubtitle_free(AVSubtitle *sub); - /** * @} */ @@ -2527,11 +2466,43 @@ enum AVChromaLocation avcodec_chroma_pos_to_enum(int xpos, int ypos); * must be freed with avsubtitle_free if *got_sub_ptr is set. * @param[in,out] got_sub_ptr Zero if no subtitle could be decompressed, otherwise, it is nonzero. * @param[in] avpkt The input AVPacket containing the input buffer. + * + * @deprecated Use avcodec_decode_subtitle3 instead. */ int avcodec_decode_subtitle2(AVCodecContext *avctx, AVSubtitle *sub, int *got_sub_ptr, AVPacket *avpkt); +/** + * Decode a subtitle message. + * Return a negative value on error, otherwise return the number of bytes used. + * If no subtitle could be decompressed, got_sub_ptr is zero. + * Otherwise, the subtitle is stored in *sub. + * Note that AV_CODEC_CAP_DR1 is not available for subtitle codecs. This is for + * simplicity, because the performance difference is expected to be negligible + * and reusing a get_buffer written for video codecs would probably perform badly + * due to a potentially very different allocation pattern. + * + * Some decoders (those marked with AV_CODEC_CAP_DELAY) have a delay between input + * and output. This means that for some packets they will not immediately + * produce decoded output and need to be flushed at the end of decoding to get + * all the decoded data. Flushing is done by calling this function with packets + * with avpkt->data set to NULL and avpkt->size set to 0 until it stops + * returning subtitles. It is safe to flush even those decoders that are not + * marked with AV_CODEC_CAP_DELAY, then no subtitles will be returned. + * + * @note The AVCodecContext MUST have been opened with @ref avcodec_open2() + * before packets may be fed to the decoder. + * + * @param avctx the codec context + * @param[out] frame The preallocated AVFrame in which the decoded subtitle will be stored, + * must be freed with av_frame_free if *got_sub_ptr is set. + * @param[in,out] got_sub_ptr Zero if no subtitle could be decompressed, otherwise, it is nonzero. + * @param[in] avpkt The input AVPacket containing the input buffer. + */ +int avcodec_decode_subtitle3(AVCodecContext *avctx, AVFrame *frame, + int *got_sub_ptr, AVPacket *avpkt); + /** * Supply raw packet data as input to a decoder. * @@ -3125,6 +3096,14 @@ void avcodec_flush_buffers(AVCodecContext *avctx); */ int av_get_audio_frame_duration(AVCodecContext *avctx, int frame_bytes); +/** + * Return subtitle format from a codec descriptor + * + * @param codec_descriptor codec descriptor + * @return the subtitle type (e.g. bitmap, text) + */ +enum AVSubtitleType av_get_subtitle_format_from_codecdesc(const AVCodecDescriptor *codec_descriptor); + /* memory */ /** diff --git a/libavcodec/decode.c b/libavcodec/decode.c index 294c040716..423aff1200 100644 --- a/libavcodec/decode.c +++ b/libavcodec/decode.c @@ -845,10 +845,7 @@ int avcodec_decode_subtitle2(AVCodecContext *avctx, AVSubtitle *sub, avctx->pkt_timebase, ms); } - if (avctx->codec_descriptor->props & AV_CODEC_PROP_BITMAP_SUB) - sub->format = 0; - else if (avctx->codec_descriptor->props & AV_CODEC_PROP_TEXT_SUB) - sub->format = 1; + sub->format = av_get_subtitle_format_from_codecdesc(avctx->codec_descriptor); for (unsigned i = 0; i < sub->num_rects; i++) { if (avctx->sub_charenc_mode != FF_SUB_CHARENC_MODE_IGNORE && @@ -872,6 +869,25 @@ int avcodec_decode_subtitle2(AVCodecContext *avctx, AVSubtitle *sub, return ret; } +int avcodec_decode_subtitle3(AVCodecContext *avctx, AVFrame *frame, + int *got_sub_ptr, AVPacket *avpkt) +{ + int ret; + AVSubtitle subtitle = { 0 }; + + ret = avcodec_decode_subtitle2(avctx, &subtitle, got_sub_ptr, avpkt); + + if (ret >= 0) { + ret = av_frame_get_buffer2(frame, 0); + if (ret < 0) + return ret; + av_frame_put_subtitle(frame, &subtitle); + } + + avsubtitle_free(&subtitle); + + return ret; +} enum AVPixelFormat avcodec_default_get_format(struct AVCodecContext *avctx, const enum AVPixelFormat *fmt) { diff --git a/libavcodec/pgssubdec.c b/libavcodec/pgssubdec.c index 388639a110..5709058b1c 100644 --- a/libavcodec/pgssubdec.c +++ b/libavcodec/pgssubdec.c @@ -32,6 +32,7 @@ #include "libavutil/colorspace.h" #include "libavutil/imgutils.h" #include "libavutil/opt.h" +#include "libavutil/subfmt.h" #define RGBA(r,g,b,a) (((unsigned)(a) << 24) | ((r) << 16) | ((g) << 8) | (b)) #define MAX_EPOCH_PALETTES 8 // Max 8 allowed per PGS epoch diff --git a/libavcodec/utils.c b/libavcodec/utils.c index 387d285633..96ab67c390 100644 --- a/libavcodec/utils.c +++ b/libavcodec/utils.c @@ -817,6 +817,17 @@ int av_get_audio_frame_duration(AVCodecContext *avctx, int frame_bytes) return FFMAX(0, duration); } +enum AVSubtitleType av_get_subtitle_format_from_codecdesc(const AVCodecDescriptor *codec_descriptor) +{ + if(codec_descriptor->props & AV_CODEC_PROP_BITMAP_SUB) + return AV_SUBTITLE_FMT_BITMAP; + + if(codec_descriptor->props & AV_CODEC_PROP_TEXT_SUB) + return AV_SUBTITLE_FMT_ASS; + + return AV_SUBTITLE_FMT_UNKNOWN; +} + int av_get_audio_frame_duration2(AVCodecParameters *par, int frame_bytes) { int duration = get_audio_frame_duration(par->codec_id, par->sample_rate, diff --git a/libavfilter/vf_subtitles.c b/libavfilter/vf_subtitles.c index f08a7fba9d..ee4d7ab94a 100644 --- a/libavfilter/vf_subtitles.c +++ b/libavfilter/vf_subtitles.c @@ -35,14 +35,12 @@ # include "libavformat/avformat.h" #endif #include "libavutil/avstring.h" -#include "libavutil/imgutils.h" #include "libavutil/opt.h" #include "libavutil/parseutils.h" #include "drawutils.h" #include "avfilter.h" #include "internal.h" #include "formats.h" -#include "video.h" typedef struct AssContext { const AVClass *class; @@ -306,6 +304,7 @@ static av_cold int init_subtitles(AVFilterContext *ctx) AVStream *st; AVPacket pkt; AssContext *ass = ctx->priv; + enum AVSubtitleType subtitle_format; /* Init libass */ ret = init(ctx); @@ -386,13 +385,17 @@ static av_cold int init_subtitles(AVFilterContext *ctx) ret = AVERROR_DECODER_NOT_FOUND; goto end; } + dec_desc = avcodec_descriptor_get(st->codecpar->codec_id); - if (dec_desc && !(dec_desc->props & AV_CODEC_PROP_TEXT_SUB)) { + subtitle_format = av_get_subtitle_format_from_codecdesc(dec_desc); + + if (subtitle_format != AV_SUBTITLE_FMT_ASS) { av_log(ctx, AV_LOG_ERROR, - "Only text based subtitles are currently supported\n"); - ret = AVERROR_PATCHWELCOME; + "Only text based subtitles are supported by this filter\n"); + ret = AVERROR_INVALIDDATA; goto end; } + if (ass->charenc) av_dict_set(&codec_opts, "sub_charenc", ass->charenc, 0); @@ -448,18 +451,18 @@ static av_cold int init_subtitles(AVFilterContext *ctx) dec_ctx->subtitle_header_size); while (av_read_frame(fmt, &pkt) >= 0) { int i, got_subtitle; - AVSubtitle sub = {0}; + AVFrame *sub = av_frame_alloc(); if (pkt.stream_index == sid) { - ret = avcodec_decode_subtitle2(dec_ctx, &sub, &got_subtitle, &pkt); + ret = avcodec_decode_subtitle3(dec_ctx, sub, &got_subtitle, &pkt); if (ret < 0) { av_log(ctx, AV_LOG_WARNING, "Error decoding: %s (ignored)\n", av_err2str(ret)); } else if (got_subtitle) { - const int64_t start_time = av_rescale_q(sub.pts, AV_TIME_BASE_Q, av_make_q(1, 1000)); - const int64_t duration = sub.end_display_time; - for (i = 0; i < sub.num_rects; i++) { - char *ass_line = sub.rects[i]->ass; + const int64_t start_time = av_rescale_q(sub->subtitle_pts, AV_TIME_BASE_Q, av_make_q(1, 1000)); + const int64_t duration = sub->subtitle_end_time; + for (i = 0; i < sub->num_subtitle_areas; i++) { + char *ass_line = sub->subtitle_areas[i]->ass; if (!ass_line) break; ass_process_chunk(ass->track, ass_line, strlen(ass_line), @@ -468,7 +471,7 @@ static av_cold int init_subtitles(AVFilterContext *ctx) } } av_packet_unref(&pkt); - avsubtitle_free(&sub); + av_frame_free(&sub); } end: diff --git a/libavformat/utils.c b/libavformat/utils.c index 2157f93e7b..b669d9ea84 100644 --- a/libavformat/utils.c +++ b/libavformat/utils.c @@ -31,6 +31,7 @@ #include "libavutil/opt.h" #include "libavutil/parseutils.h" #include "libavutil/pixfmt.h" +#include "libavutil/subfmt.h" #include "libavutil/thread.h" #include "libavutil/time.h" diff --git a/libavutil/Makefile b/libavutil/Makefile index 410ac636f7..04e1101bf3 100644 --- a/libavutil/Makefile +++ b/libavutil/Makefile @@ -74,6 +74,7 @@ HEADERS = adler32.h \ sha512.h \ spherical.h \ stereo3d.h \ + subfmt.h \ threadmessage.h \ time.h \ timecode.h \ @@ -159,6 +160,7 @@ OBJS = adler32.o \ slicethread.o \ spherical.o \ stereo3d.o \ + subfmt.o \ threadmessage.o \ time.o \ timecode.o \ diff --git a/libavutil/frame.c b/libavutil/frame.c index 2617cda2b5..89eac75899 100644 --- a/libavutil/frame.c +++ b/libavutil/frame.c @@ -26,6 +26,7 @@ #include "imgutils.h" #include "mem.h" #include "samplefmt.h" +#include "subfmt.h" #include "hwcontext.h" #define CHECK_CHANNELS_CONSISTENCY(frame) \ @@ -50,6 +51,9 @@ const char *av_get_colorspace_name(enum AVColorSpace val) return name[val]; } #endif + +static int frame_copy_subtitles(AVFrame *dst, const AVFrame *src, int copy_data); + static void get_frame_defaults(AVFrame *frame) { if (frame->extended_data != frame->data) @@ -72,7 +76,12 @@ static void get_frame_defaults(AVFrame *frame) frame->colorspace = AVCOL_SPC_UNSPECIFIED; frame->color_range = AVCOL_RANGE_UNSPECIFIED; frame->chroma_location = AVCHROMA_LOC_UNSPECIFIED; - frame->flags = 0; + frame->subtitle_start_time = 0; + frame->subtitle_end_time = 0; + frame->num_subtitle_areas = 0; + frame->subtitle_areas = NULL; + frame->subtitle_pts = 0; + frame->subtitle_header = NULL; } static void free_side_data(AVFrameSideData **ptr_sd) @@ -243,23 +252,52 @@ static int get_audio_buffer(AVFrame *frame, int align) } +static int get_subtitle_buffer(AVFrame *frame) +{ + // Buffers in AVFrame->buf[] are not used in case of subtitle frames. + // To accomodate with existing code, checking ->buf[0] to determine + // whether a frame is ref-counted or has data, we're adding a 1-byte + // buffer here, which marks the subtitle frame to contain data. + frame->buf[0] = av_buffer_alloc(1); + if (!frame->buf[0]) { + av_frame_unref(frame); + return AVERROR(ENOMEM); + } + return 0; +} + int av_frame_get_buffer(AVFrame *frame, int align) +{ + if (frame->width > 0 && frame->height > 0) + frame->type = AVMEDIA_TYPE_VIDEO; + else if (frame->nb_samples > 0 && (frame->channel_layout || frame->channels > 0)) + frame->type = AVMEDIA_TYPE_AUDIO; + + return av_frame_get_buffer2(frame, align); +} + +int av_frame_get_buffer2(AVFrame *frame, int align) { if (frame->format < 0) return AVERROR(EINVAL); - if (frame->width > 0 && frame->height > 0) + switch(frame->type) { + case AVMEDIA_TYPE_VIDEO: return get_video_buffer(frame, align); - else if (frame->nb_samples > 0 && (frame->channel_layout || frame->channels > 0)) + case AVMEDIA_TYPE_AUDIO: return get_audio_buffer(frame, align); - - return AVERROR(EINVAL); + case AVMEDIA_TYPE_SUBTITLE: + return get_subtitle_buffer(frame); + default: + return AVERROR(EINVAL); + } } static int frame_copy_props(AVFrame *dst, const AVFrame *src, int force_copy) { int ret, i; + dst->type = src->type; dst->key_frame = src->key_frame; dst->pict_type = src->pict_type; dst->sample_aspect_ratio = src->sample_aspect_ratio; @@ -290,6 +328,10 @@ static int frame_copy_props(AVFrame *dst, const AVFrame *src, int force_copy) dst->colorspace = src->colorspace; dst->color_range = src->color_range; dst->chroma_location = src->chroma_location; + dst->subtitle_start_time = src->subtitle_start_time; + dst->subtitle_end_time = src->subtitle_end_time; + dst->subtitle_pts = src->subtitle_pts; + dst->subtitle_header = src->subtitle_header; av_dict_copy(&dst->metadata, src->metadata, 0); @@ -331,6 +373,7 @@ int av_frame_ref(AVFrame *dst, const AVFrame *src) av_assert1(dst->width == 0 && dst->height == 0); av_assert1(dst->channels == 0); + dst->type = src->type; dst->format = src->format; dst->width = src->width; dst->height = src->height; @@ -344,7 +387,7 @@ int av_frame_ref(AVFrame *dst, const AVFrame *src) /* duplicate the frame data if it's not refcounted */ if (!src->buf[0]) { - ret = av_frame_get_buffer(dst, 0); + ret = av_frame_get_buffer2(dst, 0); if (ret < 0) goto fail; @@ -366,6 +409,10 @@ int av_frame_ref(AVFrame *dst, const AVFrame *src) } } + /* copy subtitle areas */ + if (src->type == AVMEDIA_TYPE_SUBTITLE) + frame_copy_subtitles(dst, src, 0); + if (src->extended_buf) { dst->extended_buf = av_calloc(src->nb_extended_buf, sizeof(*dst->extended_buf)); @@ -436,7 +483,7 @@ AVFrame *av_frame_clone(const AVFrame *src) void av_frame_unref(AVFrame *frame) { - int i; + unsigned i, n; if (!frame) return; @@ -455,6 +502,19 @@ void av_frame_unref(AVFrame *frame) av_buffer_unref(&frame->opaque_ref); av_buffer_unref(&frame->private_ref); + for (i = 0; i < frame->num_subtitle_areas; i++) { + AVSubtitleArea *area = frame->subtitle_areas[i]; + + for (n = 0; n < FF_ARRAY_ELEMS(area->buf); n++) + av_buffer_unref(&area->buf[n]); + + av_freep(&area->text); + av_freep(&area->ass); + av_free(area); + } + + av_freep(&frame->subtitle_areas); + get_frame_defaults(frame); } @@ -472,7 +532,8 @@ void av_frame_move_ref(AVFrame *dst, AVFrame *src) int av_frame_is_writable(AVFrame *frame) { - int i, ret = 1; + AVSubtitleArea *area; + unsigned i, n, ret = 1; /* assume non-refcounted frames are not writable */ if (!frame->buf[0]) @@ -484,6 +545,15 @@ int av_frame_is_writable(AVFrame *frame) for (i = 0; i < frame->nb_extended_buf; i++) ret &= !!av_buffer_is_writable(frame->extended_buf[i]); + for (i = 0; i < frame->num_subtitle_areas; i++) { + area = frame->subtitle_areas[i]; + if (area) { + for (n = 0; n < FF_ARRAY_ELEMS(area->buf); n++) + if (area->buf[n]) + ret &= !!av_buffer_is_writable(area->buf[n]); + } + } + return ret; } @@ -499,6 +569,7 @@ int av_frame_make_writable(AVFrame *frame) return 0; memset(&tmp, 0, sizeof(tmp)); + tmp.type = frame->type; tmp.format = frame->format; tmp.width = frame->width; tmp.height = frame->height; @@ -509,7 +580,7 @@ int av_frame_make_writable(AVFrame *frame) if (frame->hw_frames_ctx) ret = av_hwframe_get_buffer(frame->hw_frames_ctx, &tmp, 0); else - ret = av_frame_get_buffer(&tmp, 0); + ret = av_frame_get_buffer2(&tmp, 0); if (ret < 0) return ret; @@ -544,14 +615,22 @@ AVBufferRef *av_frame_get_plane_buffer(AVFrame *frame, int plane) uint8_t *data; int planes, i; - if (frame->nb_samples) { - int channels = frame->channels; - if (!channels) - return NULL; - CHECK_CHANNELS_CONSISTENCY(frame); - planes = av_sample_fmt_is_planar(frame->format) ? channels : 1; - } else + switch(frame->type) { + case AVMEDIA_TYPE_VIDEO: planes = 4; + break; + case AVMEDIA_TYPE_AUDIO: + { + int channels = frame->channels; + if (!channels) + return NULL; + CHECK_CHANNELS_CONSISTENCY(frame); + planes = av_sample_fmt_is_planar(frame->format) ? channels : 1; + break; + } + default: + return NULL; + } if (plane < 0 || plane >= planes || !frame->extended_data[plane]) return NULL; @@ -675,17 +754,39 @@ static int frame_copy_audio(AVFrame *dst, const AVFrame *src) return 0; } +static int frame_copy_subtitles(AVFrame *dst, const AVFrame *src, int copy_data) +{ + dst->format = src->format; + + dst->num_subtitle_areas = src->num_subtitle_areas; + + if (src->num_subtitle_areas) { + dst->subtitle_areas = av_malloc_array(src->num_subtitle_areas, sizeof(AVSubtitleArea *)); + + for (unsigned i = 0; i < src->num_subtitle_areas; i++) { + dst->subtitle_areas[i] = av_mallocz(sizeof(AVSubtitleArea)); + av_subtitle_area2area(dst->subtitle_areas[i], src->subtitle_areas[i], copy_data); + } + } + + return 0; +} + int av_frame_copy(AVFrame *dst, const AVFrame *src) { if (dst->format != src->format || dst->format < 0) return AVERROR(EINVAL); - if (dst->width > 0 && dst->height > 0) + switch(dst->type) { + case AVMEDIA_TYPE_VIDEO: return frame_copy_video(dst, src); - else if (dst->nb_samples > 0 && dst->channels > 0) + case AVMEDIA_TYPE_AUDIO: return frame_copy_audio(dst, src); - - return AVERROR(EINVAL); + case AVMEDIA_TYPE_SUBTITLE: + return frame_copy_subtitles(dst, src, 1); + default: + return AVERROR(EINVAL); + } } void av_frame_remove_side_data(AVFrame *frame, enum AVFrameSideDataType type) @@ -831,3 +932,49 @@ int av_frame_apply_cropping(AVFrame *frame, int flags) return 0; } + +/** + * Copies subtitle data from AVSubtitle (deprecated) to AVFrame + * + * @note This is a compatibility method for conversion to the legacy API + */ +void av_frame_put_subtitle(AVFrame *frame, AVSubtitle *sub) +{ + frame->format = sub->format; + frame->subtitle_start_time = sub->start_display_time; + frame->subtitle_end_time = sub->end_display_time; + frame->subtitle_pts = sub->pts; + frame->num_subtitle_areas = sub->num_rects; + + if (sub->num_rects) { + frame->subtitle_areas = av_malloc_array(sub->num_rects, sizeof(AVSubtitleArea *)); + + for (unsigned i = 0; i < sub->num_rects; i++) { + frame->subtitle_areas[i] = av_mallocz(sizeof(AVSubtitleArea)); + av_subtitle_rect2area(frame->subtitle_areas[i], sub->rects[i]); + } + } +} + +/** + * Copies subtitle data from AVFrame to AVSubtitle (deprecated) + * + * @note This is a compatibility method for conversion to the legacy API + */ +void av_frame_get_subtitle(AVSubtitle *sub, AVFrame *frame) +{ + sub->start_display_time = frame->subtitle_start_time; + sub->end_display_time = frame->subtitle_end_time; + sub->pts = frame->subtitle_pts; + sub->num_rects = frame->num_subtitle_areas; + + if (frame->num_subtitle_areas) { + sub->rects = av_malloc_array(frame->num_subtitle_areas, sizeof(AVSubtitle *)); + + for (unsigned i = 0; i < frame->num_subtitle_areas; i++) { + sub->rects[i] = av_mallocz(sizeof(AVSubtitleRect)); + av_subtitle_area2rect(sub->rects[i], frame->subtitle_areas[i]); + } + } +} + diff --git a/libavutil/frame.h b/libavutil/frame.h index ff2540a20f..63e49b8574 100644 --- a/libavutil/frame.h +++ b/libavutil/frame.h @@ -34,6 +34,7 @@ #include "rational.h" #include "samplefmt.h" #include "pixfmt.h" +#include "subfmt.h" #include "version.h" @@ -271,7 +272,7 @@ typedef struct AVRegionOfInterest { } AVRegionOfInterest; /** - * This structure describes decoded (raw) audio or video data. + * This structure describes decoded (raw) audio, video or subtitle data. * * AVFrame must be allocated using av_frame_alloc(). Note that this only * allocates the AVFrame itself, the buffers for the data must be managed @@ -302,6 +303,13 @@ typedef struct AVRegionOfInterest { */ typedef struct AVFrame { #define AV_NUM_DATA_POINTERS 8 + /** + * Media type of the frame (audio, video, subtitles..) + * + * See AVMEDIA_TYPE_xxx + */ + enum AVMediaType type; + /** * pointer to the picture/channel planes. * This might be different from the first allocated byte @@ -371,7 +379,7 @@ typedef struct AVFrame { /** * format of the frame, -1 if unknown or unset * Values correspond to enum AVPixelFormat for video frames, - * enum AVSampleFormat for audio) + * enum AVSampleFormat for audio, AVSubtitleType for subtitles) */ int format; @@ -462,6 +470,39 @@ typedef struct AVFrame { */ uint64_t channel_layout; + /** + * Display start time, relative to packet pts, in ms. + */ + uint32_t subtitle_start_time; + + /** + * Display end time, relative to packet pts, in ms. + */ + uint32_t subtitle_end_time; + + /** + * Number of items in the @ref subtitle_areas array. + */ + unsigned num_subtitle_areas; + + /** + * Array of subtitle areas, may be empty. + */ + AVSubtitleArea **subtitle_areas; + + /** + * Usually the same as packet pts, in AV_TIME_BASE. + * + * @deprecated This is kept for compatibility reasons and corresponds to + * AVSubtitle->pts. Might be removed in the future. + */ + int64_t subtitle_pts; + + /** + * Header containing style information for text subtitles. + */ + char *subtitle_header; + /** * AVBuffer references backing the data for this frame. If all elements of * this array are NULL, then this frame is not reference counted. This array @@ -721,6 +762,8 @@ void av_frame_move_ref(AVFrame *dst, AVFrame *src); /** * Allocate new buffer(s) for audio or video data. * + * Note: For subtitle data, use av_frame_get_buffer2 + * * The following fields must be set on frame before calling this function: * - format (pixel format for video, sample format for audio) * - width and height for video @@ -740,9 +783,39 @@ void av_frame_move_ref(AVFrame *dst, AVFrame *src); * recommended to pass 0 here unless you know what you are doing. * * @return 0 on success, a negative AVERROR on error. + * + * @deprecated Use @ref av_frame_get_buffer2 instead and set @ref AVFrame.type + * before calling. */ +attribute_deprecated int av_frame_get_buffer(AVFrame *frame, int align); +/** + * Allocate new buffer(s) for audio, video or subtitle data. + * + * The following fields must be set on frame before calling this function: + * - format (pixel format for video, sample format for audio) + * - width and height for video + * - nb_samples and channel_layout for audio + * - type (AVMediaType) + * + * This function will fill AVFrame.data and AVFrame.buf arrays and, if + * necessary, allocate and fill AVFrame.extended_data and AVFrame.extended_buf. + * For planar formats, one buffer will be allocated for each plane. + * + * @warning: if frame already has been allocated, calling this function will + * leak memory. In addition, undefined behavior can occur in certain + * cases. + * + * @param frame frame in which to store the new buffers. + * @param align Required buffer size alignment. If equal to 0, alignment will be + * chosen automatically for the current CPU. It is highly + * recommended to pass 0 here unless you know what you are doing. + * + * @return 0 on success, a negative AVERROR on error. + */ +int av_frame_get_buffer2(AVFrame *frame, int align); + /** * Check if the frame data is writable. * @@ -844,6 +917,22 @@ AVFrameSideData *av_frame_get_side_data(const AVFrame *frame, */ void av_frame_remove_side_data(AVFrame *frame, enum AVFrameSideDataType type); +/** + * Copies subtitle data from AVSubtitle to AVFrame. + * + * @deprecated This is a compatibility method for interoperability with + * the legacy subtitle API. + */ +void av_frame_put_subtitle(AVFrame *frame, AVSubtitle *sub); + +/** + * Copies subtitle data from AVFrame to AVSubtitle. + * + * @deprecated This is a compatibility method for interoperability with + * the legacy subtitle API. + */ +void av_frame_get_subtitle(AVSubtitle *sub, AVFrame *frame); + /** * Flags for frame cropping. diff --git a/libavutil/subfmt.c b/libavutil/subfmt.c new file mode 100644 index 0000000000..f68fea1536 --- /dev/null +++ b/libavutil/subfmt.c @@ -0,0 +1,219 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include +#include "common.h" +#include "subfmt.h" + +typedef struct SubtitleFmtInfo { + enum AVSubtitleType fmt; + const char* name; +} SubtitleFmtInfo; + +static SubtitleFmtInfo sub_fmt_info[AV_SUBTITLE_FMT_NB] = { + {.fmt = AV_SUBTITLE_FMT_UNKNOWN, .name = "Unknown subtitle format", }, + {.fmt = AV_SUBTITLE_FMT_BITMAP, .name = "Graphical subtitles", }, + {.fmt = AV_SUBTITLE_FMT_TEXT, .name = "Text subtitles (plain)", }, + {.fmt = AV_SUBTITLE_FMT_ASS, .name = "Text subtitles (ass)", }, +}; + +const char *av_get_subtitle_fmt_name(enum AVSubtitleType sub_fmt) +{ + if (sub_fmt < 0 || sub_fmt >= AV_SUBTITLE_FMT_NB) + return NULL; + return sub_fmt_info[sub_fmt].name; +} + +enum AVSubtitleType av_get_subtitle_fmt(const char *name) +{ + for (int i = 0; i < AV_SUBTITLE_FMT_NB; i++) + if (!strcmp(sub_fmt_info[i].name, name)) + return i; + return AV_SUBTITLE_FMT_NONE; +} + +int av_subtitle_area2rect(AVSubtitleRect *dst, const AVSubtitleArea *src) +{ + dst->x = src->x; + dst->y = src->y; + dst->w = src->w; + dst->h = src->h; + dst->nb_colors = src->nb_colors; + dst->type = src->type; + + switch (dst->type) { + case AV_SUBTITLE_FMT_BITMAP: + + if (src->h > 0 && src->w > 0 && src->buf[0]) { + uint32_t *pal; + AVBufferRef *buf = src->buf[0]; + dst->data[0] = av_mallocz(buf->size); + memcpy(dst->data[0], buf->data, buf->size); + dst->linesize[0] = src->linesize[0]; + + dst->data[1] = av_mallocz(256 * 4); + pal = (uint32_t *)dst->data[1]; + + for (unsigned i = 0; i < 256; i++) { + pal[i] = src->pal[i]; + } + } + + break; + case AV_SUBTITLE_FMT_TEXT: + + if (src->text) + dst->text = av_strdup(src->text); + + break; + case AV_SUBTITLE_FMT_ASS: + + if (src->ass) + dst->ass = av_strdup(src->ass); + + break; + default: + + av_log(NULL, AV_LOG_ERROR, "Subtitle rect has invalid format: %d", dst->type); + return AVERROR(ENOMEM); + } + + return 0; +} + +int av_subtitle_rect2area(AVSubtitleArea *dst, const AVSubtitleRect *src) +{ + dst->x = src->x; + dst->y = src->y; + dst->w = src->w; + dst->h = src->h; + dst->nb_colors = src->nb_colors; + dst->type = src->type; + + switch (dst->type) { + case AV_SUBTITLE_FMT_BITMAP: + + if (src->h > 0 && src->w > 0 && src->data[0]) { + AVBufferRef *buf = av_buffer_allocz(src->h * src->linesize[0]); + memcpy(buf->data, src->data[0], buf->size); + + dst->buf[0] = buf; + dst->linesize[0] = src->linesize[0]; + } + + if (src->data[1]) { + uint32_t *pal = (uint32_t *)src->data[1]; + + for (unsigned i = 0; i < 256; i++) { + dst->pal[i] = pal[i]; + } + } + + break; + case AV_SUBTITLE_FMT_TEXT: + + if (src->text) + dst->text = av_strdup(src->text); + + break; + case AV_SUBTITLE_FMT_ASS: + + if (src->ass) + dst->ass = av_strdup(src->ass); + + break; + default: + + av_log(NULL, AV_LOG_ERROR, "Subtitle area has invalid format: %d", dst->type); + return AVERROR(ENOMEM); + } + + return 0; +} + +int av_subtitle_area2area(AVSubtitleArea *dst, const AVSubtitleArea *src, int copy_data) +{ + dst->x = src->x; + dst->y = src->y; + dst->w = src->w; + dst->h = src->h; + dst->nb_colors = src->nb_colors; + dst->type = src->type; + + switch (dst->type) { + case AV_SUBTITLE_FMT_BITMAP: + + if (src->h > 0 && src->w > 0 && src->buf[0]) { + dst->buf[0] = av_buffer_ref(src->buf[0]); + if (!dst->buf[0]) { + return AVERROR(ENOMEM); + } + + if (copy_data) { + const int ret = av_buffer_make_writable(&dst->buf[0]); + if (ret < 0) + return ret; + } + + dst->linesize[0] = src->linesize[0]; + } + + for (unsigned i = 0; i < 256; i++) + dst->pal[i] = src->pal[i]; + + break; + case AV_SUBTITLE_FMT_TEXT: + + if (src->text) + dst->text = av_strdup(src->text); + + break; + case AV_SUBTITLE_FMT_ASS: + + if (src->ass) + dst->ass = av_strdup(src->ass); + + break; + default: + + av_log(NULL, AV_LOG_ERROR, "Subtitle area has invalid format: %d", dst->type); + return AVERROR(ENOMEM); + } + + return 0; +} + +void avsubtitle_free(AVSubtitle *sub) +{ + for (unsigned i = 0; i < sub->num_rects; i++) { + av_freep(&sub->rects[i]->data[0]); + av_freep(&sub->rects[i]->data[1]); + av_freep(&sub->rects[i]->data[2]); + av_freep(&sub->rects[i]->data[3]); + av_freep(&sub->rects[i]->text); + av_freep(&sub->rects[i]->ass); + av_freep(&sub->rects[i]); + } + + av_freep(&sub->rects); + + memset(sub, 0, sizeof(*sub)); +} + diff --git a/libavutil/subfmt.h b/libavutil/subfmt.h new file mode 100644 index 0000000000..372e3db413 --- /dev/null +++ b/libavutil/subfmt.h @@ -0,0 +1,185 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVUTIL_SUBFMT_H +#define AVUTIL_SUBFMT_H + +#include + +#include "buffer.h" + +enum AVSubtitleType { + + /** + * Subtitle format unknown. + */ + AV_SUBTITLE_FMT_NONE = -1, + + /** + * Subtitle format unknown. + */ + AV_SUBTITLE_FMT_UNKNOWN = 0, + + /** + * Bitmap area in AVSubtitleRect.data, pixfmt AV_PIX_FMT_PAL8. + */ + AV_SUBTITLE_FMT_BITMAP = 1, + SUBTITLE_BITMAP = 1, ///< Deprecated, use AV_SUBTITLE_FMT_BITMAP instead. + + /** + * Plain text in AVSubtitleRect.text. + */ + AV_SUBTITLE_FMT_TEXT = 2, + SUBTITLE_TEXT = 2, ///< Deprecated, use AV_SUBTITLE_FMT_TEXT instead. + + /** + * Text Formatted as per ASS specification, contained AVSubtitleRect.ass. + */ + AV_SUBTITLE_FMT_ASS = 3, + SUBTITLE_ASS = 3, ///< Deprecated, use AV_SUBTITLE_FMT_ASS instead. + + AV_SUBTITLE_FMT_NB, +}; + +typedef struct AVSubtitleArea { +#define AV_NUM_BUFFER_POINTERS 1 + + enum AVSubtitleType type; + int flags; + + int x; ///< top left corner of area. + int y; ///< top left corner of area. + int w; ///< width of area. + int h; ///< height of area. + int nb_colors; ///< number of colors in bitmap palette (@ref pal). + + /** + * Buffers and line sizes for the bitmap of this subtitle. + * + * @{ + */ + AVBufferRef *buf[AV_NUM_BUFFER_POINTERS]; + int linesize[AV_NUM_BUFFER_POINTERS]; + /** + * @} + */ + + uint32_t pal[256]; ///< RGBA palette for the bitmap. + + char *text; ///< 0-terminated plain UTF-8 text + char *ass; ///< 0-terminated ASS/SSA compatible event line. + +} AVSubtitleArea; + +typedef struct AVSubtitleRect { + unsigned nb_refs; + int x; ///< top left corner of pict, undefined when pict is not set + int y; ///< top left corner of pict, undefined when pict is not set + int w; ///< width of pict, undefined when pict is not set + int h; ///< height of pict, undefined when pict is not set + int nb_colors; ///< number of colors in pict, undefined when pict is not set + + /** + * data+linesize for the bitmap of this subtitle. + */ + uint8_t *data[4]; + int linesize[4]; + + enum AVSubtitleType type; + + char *text; ///< 0 terminated plain UTF-8 text + + /** + * 0-terminated ASS/SSA compatible event line. + */ + char *ass; + + int flags; +} AVSubtitleRect; + +typedef struct AVSubtitle { + uint16_t format; /* 0 = graphics */ + uint32_t start_display_time; /* relative to packet pts, in ms */ + uint32_t end_display_time; /* relative to packet pts, in ms */ + unsigned num_rects; + AVSubtitleRect **rects; + int64_t pts; ///< Same as packet pts, in AV_TIME_BASE +} AVSubtitle; + +/** + * Return the name of sub_fmt, or NULL if sub_fmt is not + * recognized. + */ +const char *av_get_subtitle_fmt_name(enum AVSubtitleType sub_fmt); + +/** + * Return a subtitle format corresponding to name, or AV_SUBTITLE_FMT_NONE + * on error. + * + * @param name Subtitle format name. + */ +enum AVSubtitleType av_get_subtitle_fmt(const char *name); + +/** + * Copy a subtitle area. + * + * @param dst The target area. + * @param src The source area. + * @param copy_data Determines whether to copy references or actual + * data from @ref AVSubtitleArea.buf. + * + * @return 0 on success. + */ +int av_subtitle_area2area(AVSubtitleArea *dst, const AVSubtitleArea *src, int copy_data); + +/** + * Copy data from @ref AVSubtitleArea to @ref AVSubtitleRect. + * + * @param dst The target rect (@ref AVSubtitleRect). + * @param src The source area (@ref AVSubtitleArea). + * + * @return 0 on success. + * + * @deprecated This is a compatibility method for interoperability with + * the legacy subtitle API. + */ +int av_subtitle_area2rect(AVSubtitleRect *dst, const AVSubtitleArea *src); + +/** + * Copy data from @ref AVSubtitleRect to @ref AVSubtitleArea. + * + * @param dst The source area (@ref AVSubtitleArea). + * @param src The target rect (@ref AVSubtitleRect). + * + * @return 0 on success. + * + * @deprecated This is a compatibility method for interoperability with + * the legacy subtitle API. + */ +int av_subtitle_rect2area(AVSubtitleArea *dst, const AVSubtitleRect *src); + +/** + * Free all allocated data in the given subtitle struct. + * + * @param sub AVSubtitle to free. + */ +void avsubtitle_free(AVSubtitle *sub); + +#endif /* AVUTIL_SUBFMT_H */