From patchwork Tue Jul 5 20:26:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 36681 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:8b27:b0:88:1bbf:7fd2 with SMTP id l39csp3727656pzh; Tue, 5 Jul 2022 13:27:31 -0700 (PDT) X-Google-Smtp-Source: AGRyM1uYJmOq92QImtpjePTVgKGeAH2PazdzJCweGWmapbmB65cyVimnjLRQWdXv/wkhCO1W8Ai+ X-Received: by 2002:a05:6402:4446:b0:43a:3f52:4172 with SMTP id o6-20020a056402444600b0043a3f524172mr19121215edb.417.1657052851108; Tue, 05 Jul 2022 13:27:31 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id cz15-20020a0564021caf00b0043a4cff7e3csi9100967edb.423.2022.07.05.13.27.30; Tue, 05 Jul 2022 13:27:31 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@outlook.com header.s=selector1 header.b=LjayF+Y+; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=outlook.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 26EAB68B9CD; Tue, 5 Jul 2022 23:27:28 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05olkn2101.outbound.protection.outlook.com [40.92.89.101]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id AA84F68B9A6 for ; Tue, 5 Jul 2022 23:27:21 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=d9ARYveReDpBTmNposWvpi5VSiGbkikinySFPYW70i5rwNyL92cVhDN9/yaKUjRefu9FGifCT+uZBLcIFqhc7vYrJJcZ8BJTU+KFYgAfc6GermM3G/9sXXXPdBG+DnNFWwSgD5rltiiQj2e6uPmYxG7BWD6ubSUuAeTUJxis3TYa5ZPYmiH6GJ1H8pzqNxv1gqapmooDi0ek3lALbTmedbkQ+nVR+Yqq1SwnVe/y8wbGKf+2FGQ0JZp0H1+aUlgtrVj9RkFwo+gsSJUYvM4S30UfPfEfHA5JmO4zG0k254tCnmY/eWBcU9IINcvd5Dd3IFDgFuVPGcoq7SRjb8xhhQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=K1/KgpRAzAkMCPl40mjc39UF/fqpJSVFpe6U3MFwXxA=; b=eWb5rhSy1kZ62yK/QhYmTttZm98RY10OSh0FoCpm2BNqcTtmFBq/77qWcY3mZHivjOrqYslliX1dZnN4AvuXD7iJKZwK0LhQuCAkQBG5+9IObHDRQPtNEIt46hqF4niI9x3z19Lb3GAEJ1Wg4AZUkMIRha6OSWRePGiF39eiNH4Rq4chyY4Ss7ghutcAuvWXEbNnwyEieYxUsjYkleaIUcUlcN5Zp5LEfkdYzp/BJiy07X+pySKadq5IuZ5r3M4LhHCCHWLXAcZNDUpRGdcwJjiKn/hxFS1DEq5lnco6CKSTt1QACZMhLkW+cEHp/RrsR8E1DG20GL7YWe427jqT3w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=K1/KgpRAzAkMCPl40mjc39UF/fqpJSVFpe6U3MFwXxA=; b=LjayF+Y+SHMau8yp6cMPig/quXKkE/cjkju4e38+Et1DU5a7EX6N1VjsuaTDwidq7EDO2JTQlHAIcyLdrWWRAcGB9E9UfpbBpfN/hyt9pzWuJhNHEmM0L8xc45oK8IrGHhccWepFUtWfk/tDz2as6vVocfnk/hrf1OWeqz+qGct6aaqZOuKfhACbV4jOF7OBSU6LSHXcdH7/0EGreKjUfoFrCvuEXPsXoSJXFkgZTfmYlCvpR3bclNHG7Pe4S5QUinglui8dnWq8B3TENXYGU61zi9/V5U4KLDW2TuUpZiVSvrI/honjCf0oRk4Nu+c1dd3IFJaZIhRueKAAv7MIvA== Received: from DB6PR0101MB2214.eurprd01.prod.exchangelabs.com (2603:10a6:4:42::27) by PA4PR01MB9452.eurprd01.prod.exchangelabs.com (2603:10a6:102:2af::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5395.21; Tue, 5 Jul 2022 20:27:20 +0000 Received: from DB6PR0101MB2214.eurprd01.prod.exchangelabs.com ([fe80::60b9:9f29:40cc:f01c]) by DB6PR0101MB2214.eurprd01.prod.exchangelabs.com ([fe80::60b9:9f29:40cc:f01c%10]) with mapi id 15.20.5395.021; Tue, 5 Jul 2022 20:27:20 +0000 From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Tue, 5 Jul 2022 22:26:45 +0200 Message-ID: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: X-TMN: [5/mi2oms1+EDD1g+033dbVX9Q4CYqQo0] X-ClientProxiedBy: AS9PR06CA0419.eurprd06.prod.outlook.com (2603:10a6:20b:461::15) To DB6PR0101MB2214.eurprd01.prod.exchangelabs.com (2603:10a6:4:42::27) X-Microsoft-Original-Message-ID: <20220705202650.797218-2-andreas.rheinhardt@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b0308307-fe8a-4506-8ca2-08da5ec4c2d0 X-MS-Exchange-SLBlob-MailProps: 6H6McBavlAgBibG/zEyozrtda0BUz69/pHpVZawF/WwT6OyZ5fAIe/h3GwVRdCw70aR71JWhVCHi7G4yUSr4JIAuh9oyMg63ZjkFSwEMEi13XQrteth13pCujImrzmjwRB/GZk5VQilMTIjdc+GoHXCQGibEN4D+h/Krg8nnxKuwbvfyaAUuG+4k/qL5P8fEOeeSYpP3lD5Zc7H8Klek1h0IB3chInpB2pnglaBKL2M69mge0xzb5QWyly3YO4udvw2VHXpTmW0MqbZcS9xxQUCQk8Ehxkd/87++4NDF0xeuXhJaSqgAIqDm5Aw5ci5hzW6h3BGvwswH1F7+V5qXctsn+bx0WfmuZxMMuCugtEm7Iyl2TroMXjwSGWViNPzbq1Y85Gsz4mIxWVe9xUmauMwTZ/7M6MW3LVCSbVlIBReV8Xladjs+ewoXpdD1yZN8Nq+RAvKp4wsLl7p/AEYIY285B0NRLSl/zcgQSXJN9GF4edPvDm3glJoPr+oImIeuCai/ivDUl7FeQHlQFe3CZ8V0hvk9PJmz26NNCCqfj5VY4g/ydglw9r9xkcwvwx7XOwcsiG7mObo414WxDKjgi/tSzB/TY5VOomR1B4VpVwybAffQmORJyMEtbOVPVuht/zGKKVMUhVjv4DpvErWxGVqUV+AaRvUSDxWLGfOfyTtAiiEH0Y0+K4dyBK3/EEPV7nGrDZuTtc+t+NZwPVJGA5QSvX7Z6uP4+enIYZfBxx7DE1+JKs3FEGjxJUmkMOE8UI91v9DZd60= X-MS-TrafficTypeDiagnostic: PA4PR01MB9452:EE_ X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: WytcETSD8GAwa0rdRLZ2k8mrbhqn9MkDKuqoshnw7X0F2Hb6L7vzdkzIHz/5Gf10BA1u+IFmBbQQnuHkejR8F5jlCUz1TaD7uYIgwcYv7L3dK9Cs2GPAHozspgj3jpllsUFty1It4JZGH8HohnAOcbTfjlBIbSipppkA8rdhOX5GY3DlneqpMJfau2dnIjglYM8kGAEDSEjwK/gWY722ANVMJ2G5kKKW0hX0X1XCDIhBZo4Zrfp/oQQ2zEFdVe4ECCeoRBWLWx/6T/KAnkTF2+i/XuTkkq4epgnaoKHBw6hwcDN9PkybN0RRoDqrXnQDxMf+zSkq8qwLyQcNJLJn9zmHLjRBEPZNb+oo1NXnE4R8DmfCNFWzDo/i3iTu5x8q6yCkaLkS/xv2UbEN5T4LeynqMkXV5emNuWsaLssgfAGOKdIQf0ESMLgGmHEi8tLjfBXC6uhxYZ2Jsu0z4efZDPmkf33aiswOQTZrceUX1msRoOy0D5a9LAAp5bu/u/la3EjTYCOxHbwANZgAJOLErw3m45cyUd/CQT2+BVEGka0AYjb3rXysqbVyfWjpw1JBZDhG7C8Pf060D6jE5jC0/O0OxE5fUYUZWT1s2QUyHMf9qvwaMLyLLFH5AUwjC235AAMOF+k6kBSU11pOY3/34ALstJHPmSxgLZTp4UJqr2w= X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: IWsZy4JmyKG2dxGNoEEHTqsTQCIlChahB4NgL+u0Re+a0RgPMQ7Py8fB7vhOdKM+2WFR+fTtU9NTlqXDj3VGzf6xMlDgvLU5phi8TknfV6uwEtUEDsjwzEgeznXU2seBGyS9gF+Ijs6ZdsJq15fatVfRBtzCprAQGGFFDY5QH5dhMXp4KC6xxE4XYO5Cc+tWHJgkN/+3RFRpDAKp2jrexroeid3rLLoux8Qv7KWHjnsbAwekXc0haEuaCYZIdr6cfVi2JFh9wPvCIdbnFjHlanhln0eRR9ww/mKCE3EN2uTbeThmzpIVhs66O3XNlTuO5OYvh/kLqTUvDjfLNlIYLxscM4bTtI5djNLKiL5/GdH/YotSkaRLz3qks8hchjTfc54k4IAArLjhArGszXT6ncNWq0Xyi6+cGzrRKvGb9OGRv83pel+KLrMeyz943Mh3mxPOZtX6/RiALbIjtgBGR2Bw4ocB/FT1GCIZyznRYA8ZIXlfvVgsbHssoXaJxnLkqUSPnk8Y4gTXQYA8WroVPJZWhXf09IE4+E+y7nedUoaXtUtRkVOP9BCxUj5GHoxGAAKoSQQBIYlFYziWWThmIH1QJK/3trpyeDi+C3FLi0GX/pIR/wGMokVYTFGnrYcHTXSIA5qykmsbDR0qA5h8YWwRiwwGEHRbZEPFPwr0pCEiGOcUwoEMJ5flR1yZR3jVnj1T7sfoqCHTuDNSFlICIIj+/DX+uy284Cmiyc6RJdD+irZCb5EMkemq2UKCBbxjmEvhKGHmrEEHQ04hL/jlagH5D7IyMme/wLpkdum2x2yLtEFfontNlxbuc+Zq9pl8dkfgMyF5kkcVIqkXDFNo902xAGLFj6ItLMl5Jaf/RGVOltx/Z/mtNdyoJhSK2u4yG8RSaV0xpRQyJfVdsoJm51w6emnWZYeiUGVNs9EVMDGbVfwONXStlN5/aEWPi3XzZqYwSdFpQB+YBkHTKM43iLA/IJ0znh/La5Jl8JTol3H29McveXU0oHPCxk3ldZ5x1SHJXKVkJDz0U54ToW8hYN6X948KGCZSC6wqnAhIogWCZ3gywT+e2Yqacjlwa20UF1N2OY4rrh0qGa11pk0QWjwBdgyPhjv0/1l4Qp8VDJWTfmKWIjQmteKO24F+qbyTLBYZqSrLfTXhq514b/hjRL7ZuG2yI3CrTDziW3WccdG6RTLr8xR8PVQ0oYOffBceq4+u/X5z15Ff4Ky47w9/pYZU8gPfFyNfiVgt/tbFMK9HzOIqOPX5kbHoAcgYS2fek49pICERjVq3AGO5tiMtv6wnfMsoc5an8vF4JZYS2TU= X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: b0308307-fe8a-4506-8ca2-08da5ec4c2d0 X-MS-Exchange-CrossTenant-AuthSource: DB6PR0101MB2214.eurprd01.prod.exchangelabs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Jul 2022 20:27:20.4008 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR01MB9452 Subject: [FFmpeg-devel] [PATCH 3/8] avutil/mem: Add av_fast_realloc_array() X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt , Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: JoQ1qJBrLTVA From: Andreas Rheinhardt This is an array-equivalent of av_fast_realloc(). Its advantages compared to using av_fast_realloc() for allocating arrays are as follows: a) It performs its own overflow checks for the multiplication that is implicit in array allocations. (And it only needs to perform these checks (as well as the multiplication itself) in case the array needs to be reallocated.) b) It allows to limit the number of elements to an upper bound given by the caller. This allows to restrict the number of allocated elements to fit into an int and therefore makes this function usable with counters of this type. It can also be used to avoid overflow checks in the caller: E.g. setting it to UINT_MAX - 1 elements makes it safe to increase the desired number of elements in steps of one. And it avoids overallocations in situations where one already has an upper bound. c) av_fast_realloc_array() will always allocate in multiples of array elements; no memory is wasted with partial elements. d) By returning an int, av_fast_realloc_array() can distinguish between ordinary allocation failures (meriting AVERROR(ENOMEM)) and failures because of allocation limits (by returning AVERROR(ERANGE)). e) It is no longer possible for the user to accidentally lose the pointer by using ptr = av_fast_realloc(ptr, ...). Because of e) there is no need to set the number of allocated elements to zero on failure. av_fast_realloc() usually allocates size + size / 16 + 32 bytes if size bytes are desired and if the already existing buffer isn't big enough. av_fast_realloc_array() instead allocates nb + (nb + 14) / 16. Rounding up is done in order not to reallocate in steps of one if the current number is < 16; adding 14 instead of 15 has the effect of only allocating one element if one element is desired. This is done with an eye towards applications where arrays might commonly only contain one element (as happens with the Matroska CueTrackPositions). Which of the two functions allocates faster depends upon the size of the elements. E.g. if the elements have a size of 32B and the desired size is incremented in steps of one, allocations happen at 1, 3, 5, 7, 9, 11, 13, 15, 17, 20, 23, 26 ... for av_fast_realloc(), 1, 2, 4, 6, 8, 10, 12, 14, 16, 18, 21, 24 ... for av_fast_realloc_array(). For element sizes of 96B, the numbers are 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 15, 17, 19, 21, 23, 25, 27, 30 ... for av_fast_realloc() whereas the pattern for av_fast_realloc_array() is unchanged. Signed-off-by: Andreas Rheinhardt --- This patch (and some of the other patches of this patchset) are mostly the same as the one in these threads: https://ffmpeg.org/pipermail/ffmpeg-devel/2019-December/254836.html https://ffmpeg.org/pipermail/ffmpeg-devel/2020-January/255182.html doc/APIchanges | 3 +++ libavutil/mem.c | 33 +++++++++++++++++++++++++++++++++ libavutil/mem.h | 30 ++++++++++++++++++++++++++++++ libavutil/version.h | 2 +- 4 files changed, 67 insertions(+), 1 deletion(-) diff --git a/doc/APIchanges b/doc/APIchanges index 20b944933a..f633ae6fee 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -14,6 +14,9 @@ libavutil: 2021-04-27 API changes, most recent first: +2022-07-05 - xxxxxxxxxx - lavu 57.28.100 - mem.h + Add av_fast_realloc_array() to simplify reallocating of arrays. + 2022-06-12 - xxxxxxxxxx - lavf 59.25.100 - avio.h Add avio_vprintf(), similar to avio_printf() but allow to use it from within a function taking a variable argument list as input. diff --git a/libavutil/mem.c b/libavutil/mem.c index 18aff5291f..6e3942ae63 100644 --- a/libavutil/mem.c +++ b/libavutil/mem.c @@ -532,6 +532,39 @@ void *av_fast_realloc(void *ptr, unsigned int *size, size_t min_size) return ptr; } +int av_fast_realloc_array(void *ptr, size_t *nb_allocated, + size_t min_nb, size_t max_nb, size_t elsize) +{ + void *array; + size_t nb, max_alloc_size_bytes; + + if (min_nb <= *nb_allocated) + return 0; + + max_alloc_size_bytes = atomic_load_explicit(&max_alloc_size, memory_order_relaxed); + max_nb = FFMIN(max_nb, max_alloc_size_bytes / elsize); + + if (min_nb > max_nb) + return AVERROR(ERANGE); + + nb = min_nb + (min_nb + 14) / 16; + + /* If min_nb is so big that the above calculation overflowed, + * just allocate as much as we are allowed to. */ + nb = nb < min_nb ? max_nb : FFMIN(nb, max_nb); + + memcpy(&array, ptr, sizeof(array)); + + array = av_realloc(array, nb * elsize); + if (!array) + return AVERROR(ENOMEM); + + memcpy(ptr, &array, sizeof(array)); + *nb_allocated = nb; + + return 0; +} + static inline void fast_malloc(void *ptr, unsigned int *size, size_t min_size, int zero_realloc) { size_t max_size; diff --git a/libavutil/mem.h b/libavutil/mem.h index d91174196c..f040de08fc 100644 --- a/libavutil/mem.h +++ b/libavutil/mem.h @@ -375,11 +375,41 @@ int av_reallocp_array(void *ptr, size_t nmemb, size_t size); * @return `ptr` if the buffer is large enough, a pointer to newly reallocated * buffer if the buffer was not large enough, or `NULL` in case of * error + * @see av_fast_realloc_array() * @see av_realloc() * @see av_fast_malloc() */ void *av_fast_realloc(void *ptr, unsigned int *size, size_t min_size); +/** + * Reallocate the given array if it is not large enough, otherwise do nothing. + * + * If `ptr` points to `NULL`, then a new uninitialized array is allocated. + * + * @param[in,out] ptr Pointer to `NULL` or pointer to an already + * allocated array. `*ptr` will be set to point + * to the new array on success. + * @param[in,out] nb_allocated Pointer to the number of elements of the array + * `*ptr`. `*nb_allocated` is updated to the new + * number of allocated elements. + * @param[in] min_nb Desired minimal number of elements in array `*ptr` + * @param[in] max_nb Maximal number of elements to allocate. + * @param[in] elsize Size of a single element of the array. + * Must not be zero. + * @return 0 on success, < 0 on failure. On failure, `*ptr` is not freed and + * `*ptr` as well as `*nb_allocated` are unchanged. + * @note `max_nb` can be used to limit allocations and make this function + * usable with counters of types other than size_t. It can also be used + * to avoid overflow checks in callers: E.g. setting it to `UINT_MAX - 1` + * means that incrementing an unsigned int in steps of one need not be + * checked for overflow. + * @see av_fast_realloc() + * @see av_realloc() + * @see av_fast_malloc() + */ +int av_fast_realloc_array(void *ptr, size_t *nb_allocated, + size_t min_nb, size_t max_nb, size_t elsize); + /** * Allocate a buffer, reusing the given one if large enough. * diff --git a/libavutil/version.h b/libavutil/version.h index 2e9e02dda8..87178e9e9a 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,7 +79,7 @@ */ #define LIBAVUTIL_VERSION_MAJOR 57 -#define LIBAVUTIL_VERSION_MINOR 27 +#define LIBAVUTIL_VERSION_MINOR 28 #define LIBAVUTIL_VERSION_MICRO 100 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \