From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 2EF8D4360C for ; Sun, 19 Jun 2022 05:13:27 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id EF6CE68B2D8; Sun, 19 Jun 2022 08:13:23 +0300 (EEST) Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-oln040092075098.outbound.protection.outlook.com [40.92.75.98]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A17B468B294 for ; Sun, 19 Jun 2022 08:13:17 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=I+/Co2/OgVwYYnyxP12KGv19eB2h5ZmhIw/Q06MtOciaeFhe4GYWp80Ht8RDlbcl+YwvXQaQCgBSh/oHh8IgwG9bfbovq44Ajod8IiWilEGvTC5EurYNwDVb8biyzDRdVJ9RJ6QyFiazVSq1pdq0ibJ4cJGmfbA2MS+QjAPVYNmZBJ0vZaH9WlqVZ3qBVUoTKNy8bDn1WJyAOqgcQBRnItNOHTp1YZNG1merBDY3SjcDQe6DXoBuW5wCFuxxUdEsLRnmNyEe6TM0ovYbKhT2+36ras0CqqvLMkFLi6JtnZvrxB/cG7+Oiq+WTBoddreaHV+tVgUG7p2HVSBPXZ7+2Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=p7A5EbhqeIULAgZFLESnMQ1cLaNaVpfsF5R+CtfMOGE=; b=C/87ovjpFxVnB+ZcfklFlYCmJnRzD1GUz4n6KSSlyIiVTk+glCeTPT7yz37WTykAZbUgme04j2DDlaBbmQTB1IFcFAJu4fBhtXHtMsLkbduAmG1b66oXKhqU3e8AoMLdUap1Nq3tUsOcWlviC2TYHM5jQtmSeAIaoTTsK0zM0GUkwowU4fTubiOvtbQYqmoo/deGKhrUn3ptdmuNxMHkNasglQnsxIZn/DfUHcXNaUTs3QrZxhigrNLw3K/KMTbT8OhnpGxSVsr8bQI0bFAsV1TNTRXBBLTQQO6k+/nM6DGGkHRk9RpuvS2nRzXW3zemAZMklK1Lgf068vUrn1pP7g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=p7A5EbhqeIULAgZFLESnMQ1cLaNaVpfsF5R+CtfMOGE=; b=MLwpPfY6WR/0WzRrEVheNTDyAGShkwhYo6ET4nELzeUJlZ+ZCOWzdK6voPPxgotlM+oNzF3doIbsi2Krv07baqslYwBjwFASYGwqqlF/bFK4dyZEA+Uo0Tlw1lZ6Om8cyzC+SF4RXAofMl5eL3wOM2vyaurbswi/YTiJQ1rHxjnzSMiR+0PQSBIsCIzlUeWBIxPJuAHHM9E3aF8FPKd/s8GDYGq+X94f0gPPTUAYnaBWpK7piWU16QXyf9n5n37UWTcvUQXVdw4HDZZlCjf9TrAh8VdXSTwcqbu9cqNeRzSDdAHnRHf+gbKvCkClPqFCUOFyAM/aKcSH3ttiSQVFhw== Received: from DB6PR0101MB2214.eurprd01.prod.exchangelabs.com (2603:10a6:4:42::27) by AM0PR0102MB3539.eurprd01.prod.exchangelabs.com (2603:10a6:208:5::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5353.16; Sun, 19 Jun 2022 04:58:38 +0000 Received: from DB6PR0101MB2214.eurprd01.prod.exchangelabs.com ([fe80::60b9:9f29:40cc:f01c]) by DB6PR0101MB2214.eurprd01.prod.exchangelabs.com ([fe80::60b9:9f29:40cc:f01c%10]) with mapi id 15.20.5353.020; Sun, 19 Jun 2022 04:58:38 +0000 Message-ID: Date: Sun, 19 Jun 2022 06:58:34 +0200 Content-Language: en-US To: ffmpeg-devel@ffmpeg.org References: <20220617093141.9826-1-nil-admirari@mailo.com> From: Andreas Rheinhardt In-Reply-To: <20220617093141.9826-1-nil-admirari@mailo.com> X-TMN: [9TBTv71SdcgQUWkSFe15q/DY0GcZJezR] X-ClientProxiedBy: ZR0P278CA0190.CHEP278.PROD.OUTLOOK.COM (2603:10a6:910:44::17) To DB6PR0101MB2214.eurprd01.prod.exchangelabs.com (2603:10a6:4:42::27) X-Microsoft-Original-Message-ID: <87609bf3-b2cc-e02c-4319-2fbec50054e5@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f065b891-741e-4455-fc1e-08da51b05e24 X-MS-TrafficTypeDiagnostic: AM0PR0102MB3539:EE_ X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Xz5uIbYhf17hbb+1yHmXm+1QN6V0awe7bOCfJw6xiSw9L9TeIJ7nxuvwho0mG6tDtRWws8Y2VxEQ8FyfW28DTLJ7AxkI/yh0Yer1vP2qNaPKf2F8tCTI25z2+PQ72VVCqO1MzYn36h6af7dhSG16I2iuELqf8MH3d84ltJOARHf5b+cmMMcoXS1l+zyO701lnkRw+0ANX86OvpKeoUSc0blKyt5gL9ugcObAKLFZKgigg2WTbpI+Tbfd3ri3ifPutdUdZIvXxwhN05SP764VnRsUZzWgcPreugMtpzIYSOis2W01Nkwlw2HXDivNVC2BgJfABeRX5eY6GABNpftGQz5HRFLJUGjQVkG0DQFegY/L0mB3wkDI3ktGHUnbdIWSag+YdiryoQxw6UolcN5YzvNO890MipSkEu5wPvjcvcaysPWmiyojI9qidWgthbocjXN6mNMc3/ZFscXwA3yAvpjq4D7FgdG54JBXZIyWyJUHKF59uvIL4UZg9Yudn2A+MynZ4iyTELGewNHDbcLjN4DWG9ukmvH6ZaWgwAHrcrUnN37oJ5VZd0R2JStRlEX48WLkaj8YrqJCm4HaJcaUTA== X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?MWhLK08rckhsSVFzY3Mvb2x6dnZUWFkxRGdybE8yK1Y5WFRCdXFRNjZMQ2lD?= =?utf-8?B?a09tMlJaZjlZWnA1b3k5WVlrME1TQ3JGZmI2bEFsN29VWGtIcGladU13aGky?= =?utf-8?B?ZTVUalViOXNYaFpRZ2pQTkh6NXQ3M0cxVUxiVDFYcndOQ1g2SklZZ3c1Mmt2?= =?utf-8?B?K1EyaXNRSXJOa0s5dlhGSk02Lys4U0FKc3BjY09BQUU4eDh6K09CZUpudW5I?= =?utf-8?B?cXp0NVQzRkt1QXRGQUpuamE2U3NJYjJlaWxLemVVSUxucXpKdHA4bFZHelNq?= =?utf-8?B?NFRmRGR6SzBZUlZ3UUxlb3RBY0FqMXFXVXI3Y1hnUVZsYnpvNHQxSk56Ui91?= =?utf-8?B?bEFhZ05RTmMwUHZweDRhYzhqWVVjdDNDQUZVbDdpekVZdloxQmFuK1FhUlk1?= =?utf-8?B?a05Bc25LYU9TSEhhSHNOdGczYVV2Y0hLdUNGaFp0K1l5b1h5cTZMditzSkRH?= =?utf-8?B?WFd0MHJVdm9kdHhTNUg4U05DTDdRV1RTRDR3NFc4QXNtaVJJZEdUaUZMczk1?= =?utf-8?B?eW9XNzNGVEtrTFN3MkRuRFVuYTdsbk83UE4wK01YQzBuR1ZlMXVVQllzOXoy?= =?utf-8?B?ZFI2ZGxPamIrR0NzeG9PTHNaQjdaVVhXdDY0aWk5aU5iUVFZalVaL1gzRmJP?= =?utf-8?B?RmFwOWM2KzlmS0FvdndTaXlvN1BDU0J2ZC9TaVNldlVuaTRLM2oyVldnWnpJ?= =?utf-8?B?eGZjS3ZZS25SazVIOUQ4Z0FwTjZhVURBVXc2ZEgwbnRvVE9sQm5CZlVNbXBE?= =?utf-8?B?UmVodHVQa3djR2hQY2pYRGdNNjhVRXE1eG5ZT2ZJQmpGVXR2QWpvc0t2YlVL?= =?utf-8?B?cjRxd2FYSDZTSnhTR1plWWFjRHhBMnFWRGlLYS92bkFhUkNreEQzN1ZDamJ2?= =?utf-8?B?REFiL0hvUTlGbWxnOE1TYXgvRit4Y1h1dWlpYk5EdFRQaW1vbXRkMnZGL1kv?= =?utf-8?B?cmMyTnU3V2pnY0E4dHFqaGxKL0gvUlZDT0x5cVNkb0l1Y2FHUFBFSGg0QytS?= =?utf-8?B?U0IwYlppNitjbGN6NEdxNHVQR3c5QkxwQlBRWjlUN1lNamE1OFh5cXh1aWNX?= =?utf-8?B?MDhXM3BiM2l1cUN4aEUwT2JBTHF2VE1valBjR052dm1ZZ0RkNFpFNHA0RjdI?= =?utf-8?B?MERyVVFsUDFqa2NWQkxNaFRnYmszb1dyUVhqWnM3QmJzMWNXTVFnQmExMys2?= =?utf-8?B?aGsybEp3Q2JDSDFBOFcxQ3d5Syt3Wm14VFhRK2xmbXhOSlJyZG1DZWxjampK?= =?utf-8?B?b21heFBsemFIVW1YOW1ZKytPWWxnYU5HeStqWlRkbVJxeWR3ak84UWFrRXYz?= =?utf-8?B?OSt2M0o1OStYMkl1ZklNWEQ0bDVCMHZhOE1oeit3NzZTM0Ntc1dMMUhVVFBV?= =?utf-8?B?R1J2QUxrSHZVL2hEbnRXZTFmMUJGMDBac3FYY3Y2aDR0eVpqNDdLU2dKZ1dV?= =?utf-8?B?UWRZUDVaZVdtTmlWaG4rZzdBYWRDOWxJQS9yOUtqejhNTDZueHh3dDBGZVcy?= =?utf-8?B?ZEIrcUVqdTgxUExpbWwvZFEweWtSS0ZJQk5Scng1dyttdExHNVB2a09RSE1U?= =?utf-8?B?a1ZERTZJSjlwOXgyNjQ3WnFQYUwveTVPNDRjekFKRE81R3Rmb1hkQk9KTHh2?= =?utf-8?B?eHRaYU1QUjRlSVBuSUhUbEpkQmlwVXdQLytBaE5iemhwZWJWQWl6T21iSFBX?= =?utf-8?B?VmFmOEI3cHh3Y3pSUjRNQjZGSGpRVzIzMlFFM2dFekN5TmhOVjNjVUh4WDNE?= =?utf-8?B?V095ZGZRNlNCR1VyR1g2bFVncTVDeElxTm1sbWV5K2ZYbkRCYUNJQW03U0pw?= =?utf-8?B?QlVBZVZRMHc3c2VKOHEvM1Zkblh0aXR6cFk2c2lRdTZtanErRkRnMEdFQVpz?= =?utf-8?B?ZVhtMXkwTkp5S0hWOFVNUWJJSUdCS3FWYTdqWXVyY2MxYW5zYm9uaytvRU15?= =?utf-8?B?cEdQVmdWWEtlZ1VZVW5UQVN3RlQwYUZtd2t3SllBejNmYThPOFVkaHVkdGpE?= =?utf-8?B?THpHY09OWXZBPT0=?= X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: f065b891-741e-4455-fc1e-08da51b05e24 X-MS-Exchange-CrossTenant-AuthSource: DB6PR0101MB2214.eurprd01.prod.exchangelabs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jun 2022 04:58:38.1608 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0102MB3539 Subject: Re: [FFmpeg-devel] [PATCH v17 1/5] libavutil: Add wchartoutf8(), wchartoansi(), utf8toansi() and getenv_utf8() X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: Nil Admirari: > wchartoutf8() converts strings returned by WinAPI into UTF-8, > which is FFmpeg's preffered encoding. > > Some external dependencies, such as AviSynth, are still > not Unicode-enabled. utf8toansi() converts UTF-8 strings > into ANSI in two steps: UTF-8 -> wchar_t -> ANSI. > wchartoansi() is responsible for the second step of the conversion. > Conversion in just one step is not supported by WinAPI. > > Since these character converting functions allocate the buffer > of necessary size, they also facilitate the removal of MAX_PATH limit > in places where fixed-size ANSI/WCHAR strings were used > as filename buffers. > > getenv_utf8() wraps _wgetenv() converting its input from > and its output to UTF-8. Compared to plain getenv(), > getenv_utf8() requires a cleanup. > > Because of that, in places that only test the existence of > an environment variable or compare its value with a string > consisting entirely of ASCII characters, the use of plain getenv() > is still preferred. (libavutil/log.c check_color_terminal() > is an example of such a place.) > > Plain getenv() is also preffered in UNIX-only code, > such as bktr.c, fbdev_common.c, oss.c in libavdevice > or af_ladspa.c in libavfilter. > --- > configure | 1 + > libavutil/getenv_utf8.h | 71 ++++++++++++++++++++++++++++++++++++++ > libavutil/wchar_filename.h | 51 +++++++++++++++++++++++++++ > 3 files changed, 123 insertions(+) > create mode 100644 libavutil/getenv_utf8.h > > diff --git a/configure b/configure > index 3dca1c4bd3..fa37a74531 100755 > --- a/configure > +++ b/configure > @@ -2272,6 +2272,7 @@ SYSTEM_FUNCS=" > fcntl > getaddrinfo > getauxval > + getenv > gethrtime > getopt > GetModuleHandle > diff --git a/libavutil/getenv_utf8.h b/libavutil/getenv_utf8.h > new file mode 100644 > index 0000000000..161e3e6202 > --- /dev/null > +++ b/libavutil/getenv_utf8.h > @@ -0,0 +1,71 @@ > +/* > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +#ifndef AVUTIL_GETENV_UTF8_H > +#define AVUTIL_GETENV_UTF8_H > + > +#include > + > +#include "mem.h" > + > +#ifdef HAVE_GETENV > + > +#ifdef _WIN32 > + > +#include "libavutil/wchar_filename.h" > + > +static inline char *getenv_utf8(const char *varname) > +{ > + wchar_t *varname_w, *var_w; > + char *var; > + > + if (utf8towchar(varname, &varname_w)) > + return NULL; > + if (!varname_w) > + return NULL; > + > + var_w = _wgetenv(varname_w); > + av_free(varname_w); > + > + if (!var_w) > + return NULL; > + if (wchartoutf8(var_w, &var)) > + return NULL; > + > + return var; > + > + // No CP_ACP fallback compared to other *_utf8() functions: > + // non UTF-8 strings must not be returned. > +} > + > +#else > + > +static inline char *getenv_utf8(const char *varname) > +{ > + return av_strdup(getenv(varname)); This forces allocations and frees in scenarios where this is wholly unnecessary. This can be avoided by adding a custom deallocator for strings returned via getenv_utf8: Namely a define/wrapper around av_free in the _WIN32 and a no-op else. > +} > + > +#endif // _WIN32 > + > +#else > + > +#define getenv_utf8(x) NULL > + > +#endif // HAVE_GETENV > + > +#endif // AVUTIL_GETENV_UTF8_H > diff --git a/libavutil/wchar_filename.h b/libavutil/wchar_filename.h > index f36d9dfea3..a6d71e52e5 100644 > --- a/libavutil/wchar_filename.h > +++ b/libavutil/wchar_filename.h > @@ -41,6 +41,57 @@ static inline int utf8towchar(const char *filename_utf8, wchar_t **filename_w) > return 0; > } > > +av_warn_unused_result > +static inline int wchartocp(unsigned int code_page, const wchar_t *filename_w, > + char **filename) > +{ > + DWORD flags = code_page == CP_UTF8 ? WC_ERR_INVALID_CHARS : 0; > + int num_chars = WideCharToMultiByte(code_page, flags, filename_w, -1, > + NULL, 0, NULL, NULL); > + if (num_chars <= 0) { > + *filename = NULL; > + return 0; > + } > + *filename = av_malloc_array(num_chars, sizeof *filename); > + if (!*filename) { > + errno = ENOMEM; > + return -1; > + } > + WideCharToMultiByte(code_page, flags, filename_w, -1, > + *filename, num_chars, NULL, NULL); > + return 0; > +} > + > +av_warn_unused_result > +static inline int wchartoutf8(const wchar_t *filename_w, char **filename) > +{ > + return wchartocp(CP_UTF8, filename_w, filename); > +} > + > +av_warn_unused_result > +static inline int wchartoansi(const wchar_t *filename_w, char **filename) > +{ > + return wchartocp(CP_ACP, filename_w, filename); > +} > + > +av_warn_unused_result > +static inline int utf8toansi(const char *filename_utf8, char **filename) > +{ > + wchar_t *filename_w = NULL; > + int ret = -1; > + if (utf8towchar(filename_utf8, &filename_w)) > + return -1; > + > + if (!filename_w) { > + *filename = NULL; > + return 0; > + } > + > + ret = wchartoansi(filename_w, filename); > + av_free(filename_w); > + return ret; > +} > + > /** > * Checks for extended path prefixes for which normalization needs to be skipped. > * see .NET6: PathInternal.IsExtended() _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".