From patchwork Fri Dec 3 19:31:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 31927 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp1098028iog; Fri, 3 Dec 2021 11:33:01 -0800 (PST) X-Google-Smtp-Source: ABdhPJysU1OTXmnompNgouaNDsfc+asldS+Q97Sq/gyj5tJqiqm3N0A9uKodNj01NthEjZH/ggvG X-Received: by 2002:a50:f09b:: with SMTP id v27mr29062313edl.53.1638559981827; Fri, 03 Dec 2021 11:33:01 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id m20si5941815edd.409.2021.12.03.11.33.01; Fri, 03 Dec 2021 11:33:01 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=N29OnEO0; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 673D168AEBC; Fri, 3 Dec 2021 21:31:36 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10olkn2103.outbound.protection.outlook.com [40.92.42.103]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CD1EC68AE9C for ; Fri, 3 Dec 2021 21:31:30 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HFdyM5Eh+IQ8OzIZMHvZJLWN/mx75ZSGIBvHuDfvXvUY9d7Y/X3MyzF/w7fEGKK+K7mGeCmWNtSxOg3kkQYupNFsQr5CRJa5svhRP3qiQlezOfEvYjaMCu8PaoSWia3sfY25VpB8G4Yk9CQIK3OeOoKud6BlRK4F3BsuLj6EGKE4Iy7lsJSNWgkT1PD3Nlv0KNxw/9GsXl1GcBbEnlBnfFfRsTSLt9RCcJa4hLTqh/DDSC7aVcQD+VO1J9ImlEJTwq1fTYHEdzlM7Hs5+pCVcjwIlNXa2JRr+jThXmCgUDNkFnBlHpb+qeEmdZ8bGMb2yOEWQWF/BxDLTCe6rZfK5g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=OC3YZ7RTmGUom1xJfzyuSQSI8aphbB1PgaQ+EAUh5Ro=; b=iKeNetHCyDEWMD3pO3EjjNm1ui6VvHh1bk5TqjBc6loD8WfNJvzaFSQeSKXCG5SUtKayk1z/iVtpDSPZWmmjxoTSjzO1NQowmVKVKMmv3dh5zNh+Oty78eRakQQWcVXnB531VH5aG0rLV8pCNYmHoFnCq7ib6YYMfAzMStWPVNEkiA/gWJHAJRfcaMo/HpxwFpb9m1IYDE9ijIqFHDIKZorge3yZyJ7+wZw3UgorzGywYDp4t2WCM/v8hhlAc86ZIGpaxoYdlHR4NP+tDM0InVpYuZwGA9jB/aMOTOk2KVq55iemHpCW84xej0OfchHQQVneubtfxJqNQrfIi04c9Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OC3YZ7RTmGUom1xJfzyuSQSI8aphbB1PgaQ+EAUh5Ro=; b=N29OnEO02Ivrr2HZYg7eRauueVo1TFRp4IVuvVBYaGBf3XN+mXzRc9iYdiKg5GAUcHz1R3tYJXb9+3UgTWB8sntDWrDqqTSL/iMsR72k93F2ISV67e02cpX+ylM762CEUEW1xQHOxzENnwj3eN4BpxQtQg22JjtFzdZm0aYIHI8L/qePL9gVdgLcF81x2amiQqGilU1joWyxxt4XyceWoQSX/q8sB9PT7St2JAcrB/xUP5JB5XoqLm9zRlqGsUH6AYU/sOhJDEtEx8GyNzoIk1Qd4Bj1oo8YHEEzXSxVCY725EaHEG/US2BX44IPn7wBtvvF1arIPEIjQqaBvhag3Q== Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::20) by DM8P223MB0333.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:8::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4734.23; Fri, 3 Dec 2021 19:31:23 +0000 Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775]) by DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::9c8d:fc63:9488:9775%8]) with mapi id 15.20.4755.016; Fri, 3 Dec 2021 19:31:23 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v19 10/20] avfilter/avfilter: Handle subtitle frames Thread-Index: AQHX6HxaLgDADGiGZ0G6A3Rzfouzsg== Date: Fri, 3 Dec 2021 19:31:22 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [nkrdM2NhBxtz6OTNOio8dg8BOVIc1mHE] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: a844167c-a47c-4433-53c9-08d9b6937d69 x-ms-traffictypediagnostic: DM8P223MB0333: x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: H7phqAM5q2lQy5+AQC9QLkrTsAZqdFatAW6o+dnZPLwTagkw5/OEx6MeDBOeZSwjg8UR9RAtEZShC2YQTpxwAxx29vaQ/YL8aKCzGJl+RfFRtZ/MZ2T7A8s0v7K+4LGvHGv1YifRRN9O6WAykcQJLGej6ZfJ7S2vq4a2BErAReaBTqF3xabFxon81BDIS+lHmZpqMR4qt7ntV9s62ZS1Fdxru4KYv8QnhYOqa7+nck7B76h9LdUlt8aIOkBR3l+e123EWeQbdwP1LwROucLfNspHwv6txlyjAsc8aUo296I39S1zFA/lyA19N4VL2FyQj2s6Px1oAHbIHxFFQ2c4iN+a28uaV8dEbowb6JvDJBnIyb1ckdIg+cMPrBNlpF4cKZnv++mcEEHkqWa6SRlYBgF9lZsD8fOeap5hP7Nyg52PfuCaHrlLA6+YUWdmERG06GtImHNjLjjDiRv40xtvCynzsMvdGTk/ysQ0KKObNi/ugEHls0gW+8W1PfzmnA81st3GWbzqYZI5vzg3s/2MMMae1nmZlEzfPSYzdM9Zijf9ARCduupKut8NRbMgCb+JmYd8SXXoh7IfPrmzpinDPw== x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?q?xjbzfsJ/Ab1CyXN1A1ewZwFV97Sm?= =?utf-8?q?Wc6u8SIgh77I+qgNcLixyal9jgNjp4JAXfQbabMTDjffD82BnSA462heVoBKeRTNq?= =?utf-8?q?hs93x0kvHesIgImqaCWYZE2SbjOIdEcZfYphik5fJv9fvT7hAaR+An9l/P3xUcnVg?= =?utf-8?q?rGhbryFQscqV+jcQv3T6ZeTA1VmXkLo/xXP5uJC21fbK0u2F+SNKh4mmLKJTlXBTN?= =?utf-8?q?1n3M9OWPPtcJGWAYpY5DgIcS8iyMvJK5foQ2erJ1pJUJXKQgTHMpT4LBIk5Mg5c4k?= =?utf-8?q?OUjQ1yZf/rTrODjAkIp+/nJWaeOpj6gQJTSm1Wx5nJu08F5SngaRBEq7MvTJD5boG?= =?utf-8?q?BMCUnQqF4MemfkoZ1NfQTvcVxU4GWCOdpFPbP+mqWYlQCleNSTqqQ1HakAJ6yFJNs?= =?utf-8?q?CZvshy2XZ4QB3yFB0LIfQaaY0xb70sZjMsZ+PGogCfQOzQOl8nfEvddX4iK2ak66w?= =?utf-8?q?SAhtDd1M48WsEZaN6pW4b8ozRFEy3y8ojf/4S4xoNONTwHV8dtq16MrwMa8zrKbiX?= =?utf-8?q?w7OTjvHhI1QkvqwSGltRqKPWUOypcQ0ntyytGjuHLywrEBxeGGG5UCXoUIKqkYnZK?= =?utf-8?q?jUAI4OYyqCcbot3lTFQBRZW8QblcvoVecGxjwBS0WyAcZb5ksEgQt9W7uJukeNAkU?= =?utf-8?q?9EkdnaHpYgu/F3caVeJZn44oxFacKyvciYa+0v7YoahZRyyInto3tG7jRQN8R8fy2?= =?utf-8?q?MSw6x+hKQ821ipyzzh7MJz6DQHc37nbOmjRfb3fWmR0VciSLfAqoIAvUzhhB80Pv5?= =?utf-8?q?OEUCvtisQ5aOjfwkZUk1Asp5k1SXMiLK2Rq9DpSVtfNXpD2w13icTWv9eyguDRngU?= =?utf-8?q?CSo+SOhkgrDY4LWBtcZE3J9bD7XhRCJdPprzXuyFtReGvw24gQlAPWGbNIStIQjQy?= =?utf-8?q?N7GG+hUTmibAuEDzUL5sQ2+JOk4M6d4h1TD2nYVFYsrQ=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-1ff67.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: a844167c-a47c-4433-53c9-08d9b6937d69 X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Dec 2021 19:31:22.9849 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8P223MB0333 Subject: [FFmpeg-devel] [PATCH v19 10/20] avfilter/avfilter: Handle subtitle frames X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: UB3779foIUdL Signed-off-by: softworkz --- doc/APIchanges | 3 +++ libavfilter/avfilter.c | 8 +++++--- libavfilter/avfilter.h | 11 +++++++++++ libavfilter/avfiltergraph.c | 5 +++++ libavfilter/formats.c | 22 ++++++++++++++++++++++ libavfilter/formats.h | 3 +++ libavfilter/internal.h | 18 +++++++++++++++--- 7 files changed, 64 insertions(+), 6 deletions(-) diff --git a/doc/APIchanges b/doc/APIchanges index ef165c00b0..6865dc31c6 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -14,6 +14,9 @@ libavutil: 2021-04-27 API changes, most recent first: +2021-12-03 - xxxxxxxxxx - lavc 9.1.100 - avfilter.h + Add fields subs_list and sub_fmt to AVFilter struct + 2021-12-03 - xxxxxxxxxx - lavc 60.1.100 - avcodec.h Deprecate avcodec_encode_subtitle(), use regular encode api now diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c index df5b8f483c..75d5e86539 100644 --- a/libavfilter/avfilter.c +++ b/libavfilter/avfilter.c @@ -56,7 +56,8 @@ static void tlog_ref(void *ctx, AVFrame *ref, int end) ref->linesize[0], ref->linesize[1], ref->linesize[2], ref->linesize[3], ref->pts, ref->pkt_pos); - if (ref->width) { + switch(ref->type) { + case AVMEDIA_TYPE_VIDEO: ff_tlog(ctx, " a:%d/%d s:%dx%d i:%c iskey:%d type:%c", ref->sample_aspect_ratio.num, ref->sample_aspect_ratio.den, ref->width, ref->height, @@ -64,12 +65,13 @@ static void tlog_ref(void *ctx, AVFrame *ref, int end) ref->top_field_first ? 'T' : 'B', /* Top / Bottom */ ref->key_frame, av_get_picture_type_char(ref->pict_type)); - } - if (ref->nb_samples) { + break; + case AVMEDIA_TYPE_AUDIO: ff_tlog(ctx, " cl:%"PRId64"d n:%d r:%d", ref->channel_layout, ref->nb_samples, ref->sample_rate); + break; } ff_tlog(ctx, "]%s", end ? "\n" : ""); diff --git a/libavfilter/avfilter.h b/libavfilter/avfilter.h index f7208754a7..ee2f5b594d 100644 --- a/libavfilter/avfilter.h +++ b/libavfilter/avfilter.h @@ -45,6 +45,7 @@ #include "libavutil/log.h" #include "libavutil/samplefmt.h" #include "libavutil/pixfmt.h" +#include "libavutil/subfmt.h" #include "libavutil/rational.h" #include "libavfilter/version.h" @@ -327,6 +328,12 @@ typedef struct AVFilter { * and outputs use the same sample rate and channel count/layout. */ const enum AVSampleFormat *samples_list; + /** + * Analogous to pixels, but delimited by AV_SUBTITLE_FMT_NONE + * and restricted to filters that only have AVMEDIA_TYPE_SUBTITLE + * inputs and outputs. + */ + const enum AVSubtitleType *subs_list; /** * Equivalent to { pix_fmt, AV_PIX_FMT_NONE } as pixels_list. */ @@ -335,6 +342,10 @@ typedef struct AVFilter { * Equivalent to { sample_fmt, AV_SAMPLE_FMT_NONE } as samples_list. */ enum AVSampleFormat sample_fmt; + /** + * Equivalent to { sub_fmt, AV_SUBTITLE_FMT_NONE } as subs_list. + */ + enum AVSubtitleType sub_fmt; } formats; int priv_size; ///< size of private data to allocate for the filter diff --git a/libavfilter/avfiltergraph.c b/libavfilter/avfiltergraph.c index b8b432e98b..f4987654af 100644 --- a/libavfilter/avfiltergraph.c +++ b/libavfilter/avfiltergraph.c @@ -311,6 +311,8 @@ static int filter_link_check_formats(void *log, AVFilterLink *link, AVFilterForm return ret; break; + case AVMEDIA_TYPE_SUBTITLE: + return 0; default: av_assert0(!"reached"); } @@ -441,6 +443,9 @@ static int query_formats(AVFilterGraph *graph, void *log_ctx) if (!link) continue; + if (link->type == AVMEDIA_TYPE_SUBTITLE) + continue; + neg = ff_filter_get_negotiation(link); av_assert0(neg); for (neg_step = 1; neg_step < neg->nb_mergers; neg_step++) { diff --git a/libavfilter/formats.c b/libavfilter/formats.c index ba62f73248..46dbbd2975 100644 --- a/libavfilter/formats.c +++ b/libavfilter/formats.c @@ -19,6 +19,7 @@ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ +#include "libavcodec/avcodec.h" #include "libavutil/avassert.h" #include "libavutil/channel_layout.h" #include "libavutil/common.h" @@ -431,6 +432,12 @@ int ff_add_channel_layout(AVFilterChannelLayouts **l, uint64_t channel_layout) return 0; } +int ff_add_subtitle_type(AVFilterFormats **avff, int64_t fmt) +{ + ADD_FORMAT(avff, fmt, ff_formats_unref, int, formats, nb_formats); + return 0; +} + AVFilterFormats *ff_make_formats_list_singleton(int fmt) { int fmts[2] = { fmt, -1 }; @@ -450,6 +457,13 @@ AVFilterFormats *ff_all_formats(enum AVMediaType type) return NULL; fmt++; } + } else if (type == AVMEDIA_TYPE_SUBTITLE) { + if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_BITMAP) < 0) + return NULL; + if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_ASS) < 0) + return NULL; + if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_TEXT) < 0) + return NULL; } return ret; @@ -724,6 +738,10 @@ int ff_default_query_formats(AVFilterContext *ctx) type = AVMEDIA_TYPE_AUDIO; formats = ff_make_format_list(f->formats.samples_list); break; + case FF_FILTER_FORMATS_SUBFMTS_LIST: + type = AVMEDIA_TYPE_SUBTITLE; + formats = ff_make_format_list(f->formats.subs_list); + break; case FF_FILTER_FORMATS_SINGLE_PIXFMT: type = AVMEDIA_TYPE_VIDEO; formats = ff_make_formats_list_singleton(f->formats.pix_fmt); @@ -732,6 +750,10 @@ int ff_default_query_formats(AVFilterContext *ctx) type = AVMEDIA_TYPE_AUDIO; formats = ff_make_formats_list_singleton(f->formats.sample_fmt); break; + case FF_FILTER_FORMATS_SINGLE_SUBFMT: + type = AVMEDIA_TYPE_SUBTITLE; + formats = ff_make_formats_list_singleton(f->formats.sub_fmt); + break; default: av_assert2(!"Unreachable"); /* Intended fallthrough */ diff --git a/libavfilter/formats.h b/libavfilter/formats.h index a884d15213..94754ebd88 100644 --- a/libavfilter/formats.h +++ b/libavfilter/formats.h @@ -180,6 +180,9 @@ int ff_set_common_formats_from_list(AVFilterContext *ctx, const int *fmts); av_warn_unused_result int ff_add_channel_layout(AVFilterChannelLayouts **l, uint64_t channel_layout); +av_warn_unused_result +int ff_add_subtitle_type(AVFilterFormats **avff, int64_t fmt); + /** * Add *ref as a new reference to f. */ diff --git a/libavfilter/internal.h b/libavfilter/internal.h index fc09ef574c..192b0ae196 100644 --- a/libavfilter/internal.h +++ b/libavfilter/internal.h @@ -149,9 +149,11 @@ static av_always_inline int ff_filter_execute(AVFilterContext *ctx, avfilter_act enum FilterFormatsState { /** - * The default value meaning that this filter supports all formats - * and (for audio) sample rates and channel layouts/counts as long - * as these properties agree for all inputs and outputs. + * The default value meaning that this filter supports + * - For video: all formats + * - For audio: all sample rates and channel layouts/counts + * - For subtitles: all subtitle formats + * as long as these properties agree for all inputs and outputs. * This state is only allowed in case all inputs and outputs actually * have the same type. * The union is unused in this state. @@ -162,8 +164,10 @@ enum FilterFormatsState { FF_FILTER_FORMATS_QUERY_FUNC, ///< formats.query active. FF_FILTER_FORMATS_PIXFMT_LIST, ///< formats.pixels_list active. FF_FILTER_FORMATS_SAMPLEFMTS_LIST, ///< formats.samples_list active. + FF_FILTER_FORMATS_SUBFMTS_LIST, ///< formats.subs_list active. FF_FILTER_FORMATS_SINGLE_PIXFMT, ///< formats.pix_fmt active FF_FILTER_FORMATS_SINGLE_SAMPLEFMT, ///< formats.sample_fmt active. + FF_FILTER_FORMATS_SINGLE_SUBFMT, ///< formats.sub_fmt active. }; #define FILTER_QUERY_FUNC(func) \ @@ -175,16 +179,24 @@ enum FilterFormatsState { #define FILTER_SAMPLEFMTS_ARRAY(array) \ .formats.samples_list = array, \ .formats_state = FF_FILTER_FORMATS_SAMPLEFMTS_LIST +#define FILTER_SUBFMTS_ARRAY(array) \ + .formats.subs_list = array, \ + .formats_state = FF_FILTER_FORMATS_SUBFMTS_LIST #define FILTER_PIXFMTS(...) \ FILTER_PIXFMTS_ARRAY(((const enum AVPixelFormat []) { __VA_ARGS__, AV_PIX_FMT_NONE })) #define FILTER_SAMPLEFMTS(...) \ FILTER_SAMPLEFMTS_ARRAY(((const enum AVSampleFormat[]) { __VA_ARGS__, AV_SAMPLE_FMT_NONE })) +#define FILTER_SUBFMTS(...) \ + FILTER_SUBFMTS_ARRAY(((const enum AVSubtitleType[]) { __VA_ARGS__, AV_SUBTITLE_FMT_NONE })) #define FILTER_SINGLE_PIXFMT(pix_fmt_) \ .formats.pix_fmt = pix_fmt_, \ .formats_state = FF_FILTER_FORMATS_SINGLE_PIXFMT #define FILTER_SINGLE_SAMPLEFMT(sample_fmt_) \ .formats.sample_fmt = sample_fmt_, \ .formats_state = FF_FILTER_FORMATS_SINGLE_SAMPLEFMT +#define FILTER_SINGLE_SUBFMT(sub_fmt_) \ + .formats.sub_fmt = sub_fmt_, \ + .formats_state = FF_FILTER_FORMATS_SINGLE_SUBFMT #define FILTER_INOUTPADS(inout, array) \ .inout = array, \