From patchwork Mon Aug 30 08:16:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 29868 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp3737261iov; Mon, 30 Aug 2021 01:16:27 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwj7OfXfhtbO2HOlDLiAYlWQGzdNP9Th6Mhm/qMhY11E9tZnY7Cfnh0hEULVI0KkZvuXjYi X-Received: by 2002:a17:906:5e4b:: with SMTP id b11mr4958617eju.472.1630311387258; Mon, 30 Aug 2021 01:16:27 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id v20si14545892ejv.515.2021.08.30.01.16.26; Mon, 30 Aug 2021 01:16:27 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com header.s=selector1 header.b=BCUptc5A; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 03BA368A17B; Mon, 30 Aug 2021 11:16:24 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10olkn2083.outbound.protection.outlook.com [40.92.40.83]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 98AC9687F49 for ; Mon, 30 Aug 2021 11:16:17 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LjXVotLn/tT/eX8Gx9q0FczXqXIZnGco+zUvujT4zLDO0gA1n93ik557pemQt63LhBsLwAUuiRGolYUaOchMv8Syt8Vi+k5kq0QmPBs4Zlub5/pIe8tlljKL+urIumOru4khZH+ouKATvLmZ8TnOtML4gA0K0wnz7H6aGu+/dIgvJwqfvE/WFhiPR++giuW+9vbHVxg5GgFCh9oRml2670RsjjAv18Yo6OJLnjAGIaS0Zjqr5B8r8Bt9JwtJOvrAVvEf+5AvE7TiyHvaK2RQ3bbDimoIbapjhrcFiiEe2Bp6iHX0978elBwJgBs77HanJpcjZzVjgyNatUqtxzjczQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eePRXanIq62EyxIaiHemR82cVZuXiQRZl+tcVSyCtEk=; b=VkLgrcab9cN5+9jGh+FvTXtJz8lBvI7jkkwo4INuRLOkERfLOtMTZnzYW5jbZ/UU8J65D/dUK7ptoreSMZBOT4uJ+IpIo2k7IM8mdFKm0/JhcyuHLqkpCd9nnh8ZZja/sdf1uiYTbYhyKcDXXP1GkAvzf50BQfTz0uRUqgg6ZiQHKwe4t6h+8l+hHAElYlDEWml+HyjkGZG5H9FX1BSiM8eXVzJ9KLvehWVIlQLup1QSmnB+RZM2+BqMsHf2I7PjnYds+SA7kUYxkPABr5toCmuo00g4QykPfWtMPi2MaMoj3jaquPejOTQLQ8TCncnwbYpzfJpEcBph+GUHiXq+fw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eePRXanIq62EyxIaiHemR82cVZuXiQRZl+tcVSyCtEk=; b=BCUptc5Adq1+TnJ6ap+6X14RHTnKv5lkOggoSqTjiNGhXu2jeWkjbvHVf4nVk+Usm1QvgoySpGGO1kthk/GKZdEo/4oJ9rGBjgxzXYZxEG0IJoQi+IyVPyxZZq81DonloWZvkF9qnWD3XbPT97aCA2a9G8RQcaSfqQaYbTEYhvKxa6Crmr/47x52fKqioqF6Bmbq121OS+R8OxrmNyKGIF6wq4Jb/SbIjFU6WbP81ijhvDWdRtDGnJ6ejy1ozMspYDIBNz5DxVDN1zaY04u9sJkS6/KBjwA3oGpACMzduN61eAG3Bwd4PJJCXE8sn+QTfJmJYifsFxPGevy3afpn7Q== Received: from MN2PR04MB5981.namprd04.prod.outlook.com (2603:10b6:208:da::10) by MN2PR04MB6191.namprd04.prod.outlook.com (2603:10b6:208:de::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Mon, 30 Aug 2021 08:16:15 +0000 Received: from MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb]) by MN2PR04MB5981.namprd04.prod.outlook.com ([fe80::ecfe:2528:2012:22cb%5]) with mapi id 15.20.4457.024; Mon, 30 Aug 2021 08:16:14 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v2 1/8] lavu/frame: avframe add type property Thread-Index: AdeUhtWMaTdAZp5sQQiYaJJEL0FffQ== Date: Mon, 30 Aug 2021 08:16:14 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [WDA0SPo5HCtlcamJdSpYcAjCMcKRX/uL] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 26e6fe13-dbf0-490c-a015-08d96b8e6f7c x-ms-traffictypediagnostic: MN2PR04MB6191: x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: JwPkQa2bg6rM/21nHY8sqZVR7imTRD0W3/GQSxYpD6kQL+dcx0At1/qDOTvcYA0b+ZnRDfM1MAeDgTg+IuL+wAby1/usYKRaSRiOyNo5m8LMoCNvOY7lW+rY+4EVY25mu4TW9y/WpeK9pzUufMZD92QC+1FgtpvSHhPoc/oBTZPfE0M2FoWv2KygjOWoEALZs6xdRAqaUbdi0EFoOmOgF436EfVSs8Q2Kb1D97HLfoGTuH8Mmz+UK7MFMoiZNIYZ4+LslUSDCA0DxUdDqORKHMdOTQVz49ma7PML3mFLiUa6zdQMX9zemjf71/iLZrCENiBFJrE8uhCWU8tY0HsagQByDB3Q90PXCOsKN6KGj0l9h02DtOK5Ioc98jszdo4vyD9H2k0lyrfR3/56z5z/xFL6F3iNK+k3YaC96C2gtr56cyK3V4rIxn9CykkN83Ux x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: PdfI8y66iIB2YwLJ3d0cO+zSE7hmMtxLMgoCuE3VJXNkTipCm0VGIVX6wHTqrAW4JLxBrj5JLYz/mXhdFUFKh13WNNs0IB3ZaPOu54VvkvGoZcrsbdP74e04vI2j1OgkFzLPdN0bmHTJTY1rMYzXbA== x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-3174-20-msonline-outlook-529c7.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR04MB5981.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 26e6fe13-dbf0-490c-a015-08d96b8e6f7c X-MS-Exchange-CrossTenant-originalarrivaltime: 30 Aug 2021 08:16:14.8442 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR04MB6191 Subject: [FFmpeg-devel] [PATCH v2 1/8] lavu/frame: avframe add type property X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: QUT8D1CAcLx6 Signed-off-by: softworkz --- v2 Update: - Implemented Andreas' suggestions - overlay_subs filter: - removed duplicated code - implemented direct (no pre-conversion) blending of graphical subtitle rects - Supported input formats: - all packed RGB formats (with and without alpha) - yuv420p, yuv422p, yuv444p libavutil/frame.c | 74 ++++++++++++++++++++++++++++++++++++--------- libavutil/frame.h | 39 ++++++++++++++++++++++-- libavutil/version.h | 2 +- 3 files changed, 97 insertions(+), 18 deletions(-) diff --git a/libavutil/frame.c b/libavutil/frame.c index b0ceaf7145..7d95849cef 100644 --- a/libavutil/frame.c +++ b/libavutil/frame.c @@ -244,22 +244,39 @@ static int get_audio_buffer(AVFrame *frame, int align) } int av_frame_get_buffer(AVFrame *frame, int align) +{ + if (frame->width > 0 && frame->height > 0) + return av_frame_get_buffer2(frame, AVMEDIA_TYPE_VIDEO, align); + else if (frame->nb_samples > 0 && (frame->channel_layout || frame->channels > 0)) + return av_frame_get_buffer2(frame, AVMEDIA_TYPE_AUDIO, align); + + return AVERROR(EINVAL); +} + +int av_frame_get_buffer2(AVFrame *frame, enum AVMediaType type, int align) { if (frame->format < 0) return AVERROR(EINVAL); - if (frame->width > 0 && frame->height > 0) + frame->type = type; + + switch(frame->type) { + case AVMEDIA_TYPE_VIDEO: return get_video_buffer(frame, align); - else if (frame->nb_samples > 0 && (frame->channel_layout || frame->channels > 0)) + case AVMEDIA_TYPE_AUDIO: return get_audio_buffer(frame, align); - - return AVERROR(EINVAL); + case AVMEDIA_TYPE_SUBTITLE: + return 0; + default: + return AVERROR(EINVAL); + } } static int frame_copy_props(AVFrame *dst, const AVFrame *src, int force_copy) { int ret, i; + dst->type = src->type; dst->key_frame = src->key_frame; dst->pict_type = src->pict_type; dst->sample_aspect_ratio = src->sample_aspect_ratio; @@ -331,6 +348,7 @@ int av_frame_ref(AVFrame *dst, const AVFrame *src) av_assert1(dst->width == 0 && dst->height == 0); av_assert1(dst->channels == 0); + dst->type = src->type; dst->format = src->format; dst->width = src->width; dst->height = src->height; @@ -499,6 +517,7 @@ int av_frame_make_writable(AVFrame *frame) return 0; memset(&tmp, 0, sizeof(tmp)); + tmp.type = frame->type; tmp.format = frame->format; tmp.width = frame->width; tmp.height = frame->height; @@ -544,14 +563,22 @@ AVBufferRef *av_frame_get_plane_buffer(AVFrame *frame, int plane) uint8_t *data; int planes, i; - if (frame->nb_samples) { - int channels = frame->channels; - if (!channels) - return NULL; - CHECK_CHANNELS_CONSISTENCY(frame); - planes = av_sample_fmt_is_planar(frame->format) ? channels : 1; - } else + switch(frame->type) { + case AVMEDIA_TYPE_VIDEO: planes = 4; + break; + case AVMEDIA_TYPE_AUDIO: + { + int channels = frame->channels; + if (!channels) + return NULL; + CHECK_CHANNELS_CONSISTENCY(frame); + planes = av_sample_fmt_is_planar(frame->format) ? channels : 1; + break; + } + default: + return NULL; + } if (plane < 0 || plane >= planes || !frame->extended_data[plane]) return NULL; @@ -675,17 +702,34 @@ static int frame_copy_audio(AVFrame *dst, const AVFrame *src) return 0; } +static int frame_copy_subtitles(AVFrame *dst, const AVFrame *src) +{ + dst->type = AVMEDIA_TYPE_SUBTITLE; + dst->format = src->format; + + if (src->buf[0]) { + dst->buf[0] = av_buffer_ref(src->buf[0]); + dst->data[0] = src->data[0]; + } + + return 0; +} + int av_frame_copy(AVFrame *dst, const AVFrame *src) { if (dst->format != src->format || dst->format < 0) return AVERROR(EINVAL); - if (dst->width > 0 && dst->height > 0) + switch(dst->type) { + case AVMEDIA_TYPE_VIDEO: return frame_copy_video(dst, src); - else if (dst->nb_samples > 0 && dst->channels > 0) + case AVMEDIA_TYPE_AUDIO: return frame_copy_audio(dst, src); - - return AVERROR(EINVAL); + case AVMEDIA_TYPE_SUBTITLE: + return frame_copy_subtitles(dst, src); + default: + return AVERROR(EINVAL); + } } void av_frame_remove_side_data(AVFrame *frame, enum AVFrameSideDataType type) diff --git a/libavutil/frame.h b/libavutil/frame.h index ff2540a20f..c104815df9 100644 --- a/libavutil/frame.h +++ b/libavutil/frame.h @@ -271,7 +271,7 @@ typedef struct AVRegionOfInterest { } AVRegionOfInterest; /** - * This structure describes decoded (raw) audio or video data. + * This structure describes decoded (raw) audio, video or subtitle data. * * AVFrame must be allocated using av_frame_alloc(). Note that this only * allocates the AVFrame itself, the buffers for the data must be managed @@ -302,6 +302,13 @@ typedef struct AVRegionOfInterest { */ typedef struct AVFrame { #define AV_NUM_DATA_POINTERS 8 + /** + * Media type of the frame (audio, video, subtitles..) + * + * See AVMEDIA_TYPE_xxx + */ + enum AVMediaType type; + /** * pointer to the picture/channel planes. * This might be different from the first allocated byte @@ -371,7 +378,7 @@ typedef struct AVFrame { /** * format of the frame, -1 if unknown or unset * Values correspond to enum AVPixelFormat for video frames, - * enum AVSampleFormat for audio) + * enum AVSampleFormat for audio, AVSubtitleType for subtitles) */ int format; @@ -721,6 +728,8 @@ void av_frame_move_ref(AVFrame *dst, AVFrame *src); /** * Allocate new buffer(s) for audio or video data. * + * Note: For subtitle data, use av_frame_get_buffer2 + * * The following fields must be set on frame before calling this function: * - format (pixel format for video, sample format for audio) * - width and height for video @@ -743,6 +752,32 @@ void av_frame_move_ref(AVFrame *dst, AVFrame *src); */ int av_frame_get_buffer(AVFrame *frame, int align); +/** + * Allocate new buffer(s) for audio, video or subtitle data. + * + * The following fields must be set on frame before calling this function: + * - format (pixel format for video, sample format for audio) + * - width and height for video + * - nb_samples and channel_layout for audio + * + * This function will fill AVFrame.data and AVFrame.buf arrays and, if + * necessary, allocate and fill AVFrame.extended_data and AVFrame.extended_buf. + * For planar formats, one buffer will be allocated for each plane. + * + * @warning: if frame already has been allocated, calling this function will + * leak memory. In addition, undefined behavior can occur in certain + * cases. + * + * @param frame frame in which to store the new buffers. + * @param type media type to set for the frame. + * @param align Required buffer size alignment. If equal to 0, alignment will be + * chosen automatically for the current CPU. It is highly + * recommended to pass 0 here unless you know what you are doing. + * + * @return 0 on success, a negative AVERROR on error. + */ +int av_frame_get_buffer2(AVFrame *frame, enum AVMediaType type, int align); + /** * Check if the frame data is writable. * diff --git a/libavutil/version.h b/libavutil/version.h index 4b77387b08..1e6a80f86e 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,7 +79,7 @@ */ #define LIBAVUTIL_VERSION_MAJOR 57 -#define LIBAVUTIL_VERSION_MINOR 4 +#define LIBAVUTIL_VERSION_MINOR 5 #define LIBAVUTIL_VERSION_MICRO 101 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \