From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 9462D43AC5 for ; Tue, 12 Jul 2022 14:12:30 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id EC57668B8AE; Tue, 12 Jul 2022 17:12:27 +0300 (EEST) Received: from EUR03-DBA-obe.outbound.protection.outlook.com (mail-dbaeur03olkn2090.outbound.protection.outlook.com [40.92.58.90]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 8012468B687 for ; Tue, 12 Jul 2022 17:12:21 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YaEJE1jPzf4S7b256wEjNMHFoqLWU9WBQTboLGIj7cASEdUq4D+RwiVewTX3JZ1CunPP2gldJeVW8bNHDZUSdGLqjlzxJpm+WqMGBPZ5PmQ1pV4Gi7xz2c5OSEHC+NweFGmNd52Yu21utTxLj7lePermRN/hqpKMMynOT5qRrqAcyTvWrsfuOEL6hAUzrV+ldrFbgRKT9NRuHdodCQPsM3RVwurj2lGv1sMu2MqQVAmzRURpkQjZFUMo/cRTScD2MPwEWtcFT9UqjlYLPLSYgfcufSA8ymqaF5OWaq7wuDBXpVII4RToZn/c+qcvD1B0pxzhe5rw5o52QVcXsQFA7g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=gjJ/zHla98/W8eIKc3qpZGGiW8wp4Vc4dXgpc7IKS0I=; b=MCJh244LAXyqA2Oyf6UZopw4Bk4S8bjAzP0bI4di0L5oI4AwNxJO3QB1H4SCqOYOoB2IHgXA8PQXb/BNxM1bs1qOzsZHA6IyYEQaM4D4+GmflBPkMT2o/wTZNHzRoDPz4aUCgEKcJ/ncTAZHkBzE98RUFCiAq+2333kpyucr6xw/vhwPr/bl/OjfnefSC9J8Jk5NNhEEfr8T8FwlZcaH7FsvRgurA3QZzimGnv5xpV5mYQFKoMwYNDEsBhFJ8lqlMhpRje7tDmynteqMHuwKIrX13W0EN/hiEluvHJxHa1OoIXOWrq7Bb9BRQ02wKhIa8KlrF8jTHIr0BSNQKSU0QA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gjJ/zHla98/W8eIKc3qpZGGiW8wp4Vc4dXgpc7IKS0I=; b=IRiXMPlAl0ExRvlIVXgIn//tpueH85A3E9s6LLEEKVPfeSx+XHZac3BX9CqqBzfjYxbjhn/2AUSwk4QrPExoVzT/z5F6MBJdngWQCWadw1BGaY6PgT0js7yhqGOCVJQKYYaAN/zIXr7GljuRaS6V9yOGNSPNlGBSp3pHV4UqVgakjrA25ZfSmjauQoCsVMNFUKjw9dvg7CfG8/urQmTkU3kFt+rzhLygWwDpbLA4VB1URY4qeoFCFXLLi71J8RpB7tHTQ1lCjo2EwUeo7xBp2Fbb9GPK9QeT1lmpP1fgfAxlVhfjXLwClaOc3djebbooe+PPxaHWrALdOPUcRK4ZkA== Received: from DB6PR0101MB2214.eurprd01.prod.exchangelabs.com (2603:10a6:4:42::27) by DB9PR01MB8221.eurprd01.prod.exchangelabs.com (2603:10a6:10:2c5::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.15; Tue, 12 Jul 2022 14:12:18 +0000 Received: from DB6PR0101MB2214.eurprd01.prod.exchangelabs.com ([fe80::210e:b627:bcc9:8c46]) by DB6PR0101MB2214.eurprd01.prod.exchangelabs.com ([fe80::210e:b627:bcc9:8c46%11]) with mapi id 15.20.5417.026; Tue, 12 Jul 2022 14:12:18 +0000 Message-ID: Date: Tue, 12 Jul 2022 16:12:16 +0200 Content-Language: en-US To: ffmpeg-devel@ffmpeg.org References: From: Andreas Rheinhardt In-Reply-To: X-TMN: [dX3QBcLaxMKcrQlLtWBSWIIm/Zsd8UBR] X-ClientProxiedBy: ZR0P278CA0183.CHEP278.PROD.OUTLOOK.COM (2603:10a6:910:44::13) To DB6PR0101MB2214.eurprd01.prod.exchangelabs.com (2603:10a6:4:42::27) X-Microsoft-Original-Message-ID: <0e769b5b-5b07-bea5-70c8-72e50b413e2e@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 5fa93780-0e4d-46dc-0f80-08da64108797 X-MS-TrafficTypeDiagnostic: DB9PR01MB8221:EE_ X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: igluP80Uabe0NjLM8wFRsQCP5gn3h1iY8vai2OJ47FgvKRFTeCZZEVC/3tRh2ZCefSeUKTmhCyJder1m34iDKuk9d2RnJGlxW2zO2RUNtEcWGpAc8eHEfcsTpOpBqrUcWnaYoxZOiGHzIzLAGuuRhkRp4U2G5TK6Y7HodkMbcTre/ZjOgLrAt3uCwB59FO2t9uImFEN129azGqqPy3XCxptDfcMGGQ1nj8P0l7/yDe/FDaSsPtsH+rsb+m3ud4q6vTtx/2sg7X+oLvOxqgWPaLnSyMgrYL7K5Yd8YSSwKipNwfPVLSdt7CcpOTve3W7d1z4P6U8lY1MkmHk6F0L0qF1b9Bk/cOdIjLs2rL+vVnzDutWcBhaCayXCS21vbe6UXCPo20qDpx1CZIRWAKFK+z60Ky6DuPSt9t/6BpBLByxsztwvSmvtksq2WN4eXkHOHzaPkjiWK8iju3Hu4xQT1cDfSAE3u3HBOCjzB6Psjlnn43ovNW6vYV7vIwjBq/Z8mF+Ti2OC43weZDjq2+XuqyCDjG0zm16bXYMc080Y2MATja4wSaiPeafSvCsOS4gIwBwWYvNV5uidw/lPJsDMX/zlWCaDnSUtsp5SVQSiXEEM1WPc/MObZAH8RhBm7D8XQ66cQkPlOO11xNZiuudj3PHj7XGJN3yZdw7q60uwj8zEd20bxpbc8g60cjO3uefLRGN8Mw1trYuGhXyV7wpb/FP28B7ij3px4TskneuJiRn+q9JRWRFcvnUwiXh7QEZXgrNpAzFWQYOUX/9juCX++A== X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?Wmt0RGptSGZtSDBMYnlENjQ2dTFjRFFpbFZJMFU5UGxuU00wOGY1SGFUQVkx?= =?utf-8?B?b0QyNkRZam1kYUp0b1ZSYU8xYnd4dUQrZjJqUXJQMlNMdFJUeWd2bE1qM0NK?= =?utf-8?B?TmlXSkNLWURpUGxBQzUxSjRYZzFHN0ViRm4rRldISUlRR3o4MGd5SHN2UWxu?= =?utf-8?B?ZEloa2lEWEJkKzllRkFHVWZYRmJlU3k1YXVIb2NrM0N5OUI3TnFmZC92c3oy?= =?utf-8?B?NFp0aGVURlJwR3FlYUJjZzVwdDh1THlpS0l0SURQaFpkNkJ3NmN6akpWK1ZC?= =?utf-8?B?dlJEdVlVcnQvY1ZLZHQyallYQWJuQTRXTGJPL2ZZeXVVRVowVDFmMTg4TFFO?= =?utf-8?B?SGF5VVZCdVBSZEQzMjZWK0xFWGlIb1ZYcDkzZTAwT3NVbE1RanoybENxWTho?= =?utf-8?B?TXpmekZwOWx3R2g5dXo3SmUreVJhOU1Ob0NMeHU3am9XazhidzdmMlYwTDRi?= =?utf-8?B?SXowQUwrOHpJNDh6MHRlUkZKQXpFeUdCV3ZPdnFTbUk0T2FQb3B1MkxDaGVn?= =?utf-8?B?L0pXT0o2MUpPM3BUYUFpUUtKN3dJVUxvYk9rdEo0UzMvNzZPS0pWU1Q4bktB?= =?utf-8?B?NDcvbkJTYTAxNlFQWVhBb0QvdkF5dEFBWk1uUEphbEFQQmNSQ3RSY2JRZGV4?= =?utf-8?B?c0crVDhDdGhlVU9sUXpOVk9IamVBOFowVHJrdXhXZDdMYTlVRmpXdndwQWhB?= =?utf-8?B?cWducStEWmJHSzJFdFZqMjVMV1pQQTJ3a25zSi9Jb0x5eTM3VjZzekovamRq?= =?utf-8?B?cG16YVdESWl0NktTeHc4ZHZPTjc2bTlQU0FDL2dRSlVkZUZmQWdPMFBZT1Ft?= =?utf-8?B?Q2ExQUE3S05UenF3b0RiMlNwNFQxcG5nbUJwQ0lmMit2SUsyYVRHTjEzaDhS?= =?utf-8?B?VGxUUC9VUXN3L2Nvd2Z6dGJ0Tm1vQk94Mk9aY0pZakttVWMrS3UrQU9FSk5V?= =?utf-8?B?MUZteWdRdlV0VlRkWHJoQzRTamx6Rnp5ZXgzWklnbHcyQ3YzMTI5eGxjTm1p?= =?utf-8?B?NTVNOGc5MkNwSUk2UldJd3NhN2NnS1BZYThxbWs5L2hHSWUxZnhXTm8vNUFQ?= =?utf-8?B?ZjNNTUQxa0RsYU9WT2lUMHRVc0ZMQWgzd0tFMkk3ZDNyRnR4ZjlwYVN4Ny9N?= =?utf-8?B?YUdsSmdidlZJWFNXR0ZkS2lRVENuRlFra3RpTnN2blVSci9SSDdQeVM2cC9Y?= =?utf-8?B?VHN2aFlwUFJqQ3EzdG9KQkNwMjJFT3JWL2wyMTlaVWJJckJUejNQTVRBTlYz?= =?utf-8?B?U1FTUVBDamVLc3NnZzR3Z0VJYVN6emg4UmQzYTlZSG92clJUWjFnTjE2dHlZ?= =?utf-8?B?aWtqekluYXQ5WnY5MjZLemxuaWFHbUhqNkh2RXhEQjJhUEFBUFdQYm5UdzZp?= =?utf-8?B?SEx5Sk14K3VLNUliZjE2ZmlPdnVoS092YzNscy8zSEcra09ERmxyNUpTcy9M?= =?utf-8?B?NTMxMm5ldkhnRGhic0hiMHNaV010eVUvYmZFYitCSjI3dTRGbmRWNnlqQlBP?= =?utf-8?B?VWRVRys2dkZWcHk2L25mT1VjZ1pjdWR2emxIRDlmckpNeWZIU29hQUxnaVFi?= =?utf-8?B?WlBrNlJpZlAzanlTK0RFdjFpa1ZpM0N0a0tydy9XMFRoZnFvdlhuSnVjNmJu?= =?utf-8?B?VmZXaDFVMERYc2lyYWhqMnRKL0MwYnBiSjN5b0c5NHI4c3o4eDNKVU5saDB2?= =?utf-8?B?RUx1d3RKUWNDRi9YMUYyZmNKb3NZd3l3ajhXa0xZTUlmdTFSK3k2QUJTVHox?= =?utf-8?Q?V/powVqARljZNBaUeE=3D?= X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5fa93780-0e4d-46dc-0f80-08da64108797 X-MS-Exchange-CrossTenant-AuthSource: DB6PR0101MB2214.eurprd01.prod.exchangelabs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Jul 2022 14:12:18.7643 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR01MB8221 Subject: Re: [FFmpeg-devel] [PATCH 3/8] avutil/mem: Add av_fast_realloc_array() X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: Andreas Rheinhardt: > From: Andreas Rheinhardt > > This is an array-equivalent of av_fast_realloc(). Its advantages > compared to using av_fast_realloc() for allocating arrays are as > follows: > > a) It performs its own overflow checks for the multiplication that is > implicit in array allocations. (And it only needs to perform these > checks (as well as the multiplication itself) in case the array needs to > be reallocated.) > b) It allows to limit the number of elements to an upper bound given > by the caller. This allows to restrict the number of allocated elements > to fit into an int and therefore makes this function usable with > counters of this type. It can also be used to avoid overflow checks in > the caller: E.g. setting it to UINT_MAX - 1 elements makes it safe to > increase the desired number of elements in steps of one. And it avoids > overallocations in situations where one already has an upper bound. > c) av_fast_realloc_array() will always allocate in multiples of array > elements; no memory is wasted with partial elements. > d) By returning an int, av_fast_realloc_array() can distinguish between > ordinary allocation failures (meriting AVERROR(ENOMEM)) and failures > because of allocation limits (by returning AVERROR(ERANGE)). > e) It is no longer possible for the user to accidentally lose the > pointer by using ptr = av_fast_realloc(ptr, ...). > > Because of e) there is no need to set the number of allocated elements > to zero on failure. > > av_fast_realloc() usually allocates size + size / 16 + 32 bytes if size > bytes are desired and if the already existing buffer isn't big enough. > av_fast_realloc_array() instead allocates nb + (nb + 14) / 16. Rounding > up is done in order not to reallocate in steps of one if the current > number is < 16; adding 14 instead of 15 has the effect of only > allocating one element if one element is desired. This is done with an > eye towards applications where arrays might commonly only contain one > element (as happens with the Matroska CueTrackPositions). > > Which of the two functions allocates faster depends upon the size of > the elements. E.g. if the elements have a size of 32B and the desired > size is incremented in steps of one, allocations happen at > 1, 3, 5, 7, 9, 11, 13, 15, 17, 20, 23, 26 ... for av_fast_realloc(), > 1, 2, 4, 6, 8, 10, 12, 14, 16, 18, 21, 24 ... for > av_fast_realloc_array(). For element sizes of 96B, the numbers are > 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 15, 17, 19, 21, 23, 25, 27, 30 ... > for av_fast_realloc() whereas the pattern for av_fast_realloc_array() is > unchanged. > > Signed-off-by: Andreas Rheinhardt > --- > This patch (and some of the other patches of this patchset) > are mostly the same as the one in these threads: > https://ffmpeg.org/pipermail/ffmpeg-devel/2019-December/254836.html > https://ffmpeg.org/pipermail/ffmpeg-devel/2020-January/255182.html > > doc/APIchanges | 3 +++ > libavutil/mem.c | 33 +++++++++++++++++++++++++++++++++ > libavutil/mem.h | 30 ++++++++++++++++++++++++++++++ > libavutil/version.h | 2 +- > 4 files changed, 67 insertions(+), 1 deletion(-) > > diff --git a/doc/APIchanges b/doc/APIchanges > index 20b944933a..f633ae6fee 100644 > --- a/doc/APIchanges > +++ b/doc/APIchanges > @@ -14,6 +14,9 @@ libavutil: 2021-04-27 > > API changes, most recent first: > > +2022-07-05 - xxxxxxxxxx - lavu 57.28.100 - mem.h > + Add av_fast_realloc_array() to simplify reallocating of arrays. > + > 2022-06-12 - xxxxxxxxxx - lavf 59.25.100 - avio.h > Add avio_vprintf(), similar to avio_printf() but allow to use it > from within a function taking a variable argument list as input. > diff --git a/libavutil/mem.c b/libavutil/mem.c > index 18aff5291f..6e3942ae63 100644 > --- a/libavutil/mem.c > +++ b/libavutil/mem.c > @@ -532,6 +532,39 @@ void *av_fast_realloc(void *ptr, unsigned int *size, size_t min_size) > return ptr; > } > > +int av_fast_realloc_array(void *ptr, size_t *nb_allocated, > + size_t min_nb, size_t max_nb, size_t elsize) > +{ > + void *array; > + size_t nb, max_alloc_size_bytes; > + > + if (min_nb <= *nb_allocated) > + return 0; > + > + max_alloc_size_bytes = atomic_load_explicit(&max_alloc_size, memory_order_relaxed); > + max_nb = FFMIN(max_nb, max_alloc_size_bytes / elsize); > + > + if (min_nb > max_nb) > + return AVERROR(ERANGE); > + > + nb = min_nb + (min_nb + 14) / 16; > + > + /* If min_nb is so big that the above calculation overflowed, > + * just allocate as much as we are allowed to. */ > + nb = nb < min_nb ? max_nb : FFMIN(nb, max_nb); > + > + memcpy(&array, ptr, sizeof(array)); > + > + array = av_realloc(array, nb * elsize); > + if (!array) > + return AVERROR(ENOMEM); > + > + memcpy(ptr, &array, sizeof(array)); > + *nb_allocated = nb; > + > + return 0; > +} > + > static inline void fast_malloc(void *ptr, unsigned int *size, size_t min_size, int zero_realloc) > { > size_t max_size; > diff --git a/libavutil/mem.h b/libavutil/mem.h > index d91174196c..f040de08fc 100644 > --- a/libavutil/mem.h > +++ b/libavutil/mem.h > @@ -375,11 +375,41 @@ int av_reallocp_array(void *ptr, size_t nmemb, size_t size); > * @return `ptr` if the buffer is large enough, a pointer to newly reallocated > * buffer if the buffer was not large enough, or `NULL` in case of > * error > + * @see av_fast_realloc_array() > * @see av_realloc() > * @see av_fast_malloc() > */ > void *av_fast_realloc(void *ptr, unsigned int *size, size_t min_size); > > +/** > + * Reallocate the given array if it is not large enough, otherwise do nothing. > + * > + * If `ptr` points to `NULL`, then a new uninitialized array is allocated. > + * > + * @param[in,out] ptr Pointer to `NULL` or pointer to an already > + * allocated array. `*ptr` will be set to point > + * to the new array on success. > + * @param[in,out] nb_allocated Pointer to the number of elements of the array > + * `*ptr`. `*nb_allocated` is updated to the new > + * number of allocated elements. > + * @param[in] min_nb Desired minimal number of elements in array `*ptr` > + * @param[in] max_nb Maximal number of elements to allocate. > + * @param[in] elsize Size of a single element of the array. > + * Must not be zero. > + * @return 0 on success, < 0 on failure. On failure, `*ptr` is not freed and > + * `*ptr` as well as `*nb_allocated` are unchanged. > + * @note `max_nb` can be used to limit allocations and make this function > + * usable with counters of types other than size_t. It can also be used > + * to avoid overflow checks in callers: E.g. setting it to `UINT_MAX - 1` > + * means that incrementing an unsigned int in steps of one need not be > + * checked for overflow. > + * @see av_fast_realloc() > + * @see av_realloc() > + * @see av_fast_malloc() > + */ > +int av_fast_realloc_array(void *ptr, size_t *nb_allocated, > + size_t min_nb, size_t max_nb, size_t elsize); > + > /** > * Allocate a buffer, reusing the given one if large enough. > * > diff --git a/libavutil/version.h b/libavutil/version.h > index 2e9e02dda8..87178e9e9a 100644 > --- a/libavutil/version.h > +++ b/libavutil/version.h > @@ -79,7 +79,7 @@ > */ > > #define LIBAVUTIL_VERSION_MAJOR 57 > -#define LIBAVUTIL_VERSION_MINOR 27 > +#define LIBAVUTIL_VERSION_MINOR 28 > #define LIBAVUTIL_VERSION_MICRO 100 > > #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \ Anton really dislikes the av_fast_* naming and instead wants this to be called av_realloc_array_reuse(). I don't care either way. Any more opinions on this (or on the patch itself)? - Andreas _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".