From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id A7C1643025 for ; Fri, 17 Jun 2022 19:17:07 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3F51E68B8B7; Fri, 17 Jun 2022 22:17:04 +0300 (EEST) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11olkn2014.outbound.protection.outlook.com [40.92.19.14]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5264B68B838 for ; Fri, 17 Jun 2022 22:16:58 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=I3zirpb01vShaV9C+1+ytLWHatq5rzJRRrIKMJExdiJp3FhESiXnPiHd/K3XonK+xF9nRAXbFOgnMwB+gbEwY5AyWfXjfeY3v2oj9/IJ0Y8yxdM1fVzmGKvideVxOFelO8EiFkZ1WU+rfp0KsBO0OKfqu0eIYXFQHmYwA+yn95CBdomdznzjqeohhOtnrsWCVVmcKORYjim5yA58K1uhjoiOgh/iMfjXnjMYGhlWjAyHGbnUzhFRX4eV8520VoDWu7iS++ZJ0Ub7gwv5osNkJHhkx1UsraxOvORQwWNLzY/TixvKb1UbR0aNzb/WPl1+bvceMgWERQSxcyxGovCA/g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=BWE13hkTme8ZWr1PMEnzFG05N/S632dxZLRfdP3ynNw=; b=mN7WMnKLRyBDfFzifOaV8l7PGrpjl8CdxYWMG/U7yuOPp4TQ21xis2OJschLPtmKKi1LxS7C0sjsnSQp/y2iKuaLCizmAH99e8/KCIatNnBSP0GEzhpf34Xa4zb7LfZ/x3/ybDELDSG7BucowodOkx4EJ9z5ke/VsYaS47hu4Djfr45LqhHz6pX1mxBya01JYlGZMi2Qi/qRkm/cNJCRPIvXG8XhkdGLh3oDmyp6xa0X09ouv4ugSbh2B1RvriBBbLnFhZe9f7dH2nKdftf/Cuy347EZ7cP5q4/h+vm2yD6OKoQKO06FsbCSmO7J6oVHHpkQdy/nc/Kd8OuhYkGZ6w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BWE13hkTme8ZWr1PMEnzFG05N/S632dxZLRfdP3ynNw=; b=Q5xkdvIsCdJ3xPD+p8vk1raqY30IHsMg7tzS+SYsKW8oHy50mxNa1y6zT28rutgmJpElwN0S6BeQi3plyAnJfMEsk606rPzmguvFwYYc/rgT0Nd2jlSCytyQt+jgYL/MMV62aAJGazBiQOOHshLu7u3MYxccvPDKG/VreKmsRvJLOpofM95S1uGdobHGh5Tcp4Z4XGcAR+4dzyQHYP+2Eh7R27AIY5Fdoa5u/WZccHvV5I0dPnfOpqkP4doldMw8rj2ivb9ct2/eGV0Kjqe8vpg4P3pt6nFM1OVdefmpVgcRDYwyaAzfL8UBxDUweW5j7LBq4wwxvbw9QbPBiABP1A== Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::20) by SJ1P223MB0481.NAMP223.PROD.OUTLOOK.COM (2603:10b6:a03:45c::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5353.16; Fri, 17 Jun 2022 19:16:54 +0000 Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::d08c:b865:df15:6a]) by DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::d08c:b865:df15:6a%6]) with mapi id 15.20.5353.018; Fri, 17 Jun 2022 19:16:54 +0000 From: Soft Works To: FFmpeg development discussions and patches Thread-Topic: [FFmpeg-devel] [PATCH v17 1/5] libavutil: Add wchartoutf8(), wchartoansi(), utf8toansi() and getenv_utf8() Thread-Index: AQHYgi0jPoSArUYG2Um2Cp3MFxsjlq1T+MCA Date: Fri, 17 Jun 2022 19:16:54 +0000 Message-ID: References: <20220617093141.9826-1-nil-admirari@mailo.com> In-Reply-To: <20220617093141.9826-1-nil-admirari@mailo.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-tmn: [rZCTwCwtrNhQV3T2o7/cADMcsFKcSHbV] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 6caa0ffa-e480-41e3-d668-08da5095f095 x-ms-traffictypediagnostic: SJ1P223MB0481:EE_ x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: iNmU4O3q80ssKo7FmTFLWLdvoB5rRfp/26+C9PkahDTG72+TW4kTWDwf9/r6hefv08bvQPDAcYuVpS+1vrKDkwMo9Qrtisxop5896EKWv19r5limZZLWgnZfKYJrN+8Qky6+hLJv5psIwaSlvgD7DohwmNIC9dx63O3CQpBMao2/AkySW/dH6WvLcZi6zWJGmxB6WsjnNE5DXSUaZu7Gy9FHqk6j8BNBHRra90Fnh0nw3b5SCePhhw0r/1lBLd1IL/Pf4DPZJ60vRzbxZaMLRXQVgwGpjbi9hIoYR7TFPo4lYAD7PvXNOVLKNmwumWZ8peddUplTCRJZ8TO3r3UypkCJZmKmTvAVEnLfAkeH+ybLPgLeWLNfGaEaUzUgtq7ZiVkjcohCXi+fHMwAsTpIJx3/yEYXvQ3OHEnIfHRPJuRA4s9lDJJ5t69Gt3MKFFfmVSs2VkiT6tRLOhK/3C+J6LaHftzPm5ak3Ey5y08ee36HDmUj2MZ5f6ko2xnHyv9eMw8zP+mwf+88kL3Aej0nnja0yqZsFycGWbWjat5/ua83CS7hrBUX9siOeq1rhPvpdovFgtqWHkYabbrvbTtvtg== x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?B?SDlkZ3U2ZCtITzVURGE2RVpjQ3JoUzJIc1EvZExFWmlaRWFNazFxaEJGT0Fn?= =?utf-8?B?T3psWlRwYTRQanlmKzl0Nm5aQmRSdnE2WUVsSkhCQVgvZDFXU09pb3ZkRE1B?= =?utf-8?B?aHNjZGdWaGU1YWU2ZWxlUUVtK1lGbUlBZHBIMGxvZWF1dlB2Z3J2NmQ0Z2Uv?= =?utf-8?B?ZzFWWTJTcWFIejB0eDVGaVNWbm9LdmlMMjV2b3VLWVlyQlpJVGpVRzNnT2E4?= =?utf-8?B?cmt6VnJiditNRHVJdnhwN3E2TTVtRHNSeTRxRmNlWGhwcVhPTFBURmV5dCt0?= =?utf-8?B?aU5tUFhWbENoMVo3OVlVUDRRYnR3WG1WWHU4dkd0YTRCV0dEYWZ6ZTQ5czZj?= =?utf-8?B?ajcrVUtsQUtYcTZuaXZsb3pjNERJTForMmRJdC92NTV6amRxVGRURURXNVVi?= =?utf-8?B?cUVKd3NaRFBMR0ZJbEk3MFlaY1FRTHlUTUtRL3RJOUVDcUw1UWgvVk1tS1Vt?= =?utf-8?B?MUxsY3pyd2k2Q0xnUVBOWEZSSzRtc3B5bXlEcldzbnBNZ2xpK1dVTnh2R0RD?= =?utf-8?B?RFlkdTcvVEdpaW9xMGV3QllJSWVxcEJaeTloZ2ZFZGJldHM5ZnRreGJzNG5O?= =?utf-8?B?UHYzU2dEd0wyQk40RTVzS0xPa1A0VXUwa05kRGZWNC8ybkpYRnRWK3R0endI?= =?utf-8?B?S1FaL2ZkbXhsT3lYVzBvOERWU3VMOWJVaFdKa0RvOEU0VnhGSjM2UVVCS1Bu?= =?utf-8?B?VE9jdzlNVzBpWWZjNUNqZ1BoVmFSc0dZZmRaSmI4Rk5PZ3UxZWFIaW1ZdVV1?= =?utf-8?B?ZjJET0twOVpjZjJzb2E0MmdLMmw3LytucUFTZGdraXcrWUFTRTY0bStwQnJL?= =?utf-8?B?aGJUbUhJMWtldXdxSDV0elA5RzFKKzRLZmEycEpCMERrMityV0ZWR1E3VjJL?= =?utf-8?B?TmFUaFY4QUxma3F0S2lydlh5TnU1REZxOWJMNXdncTZhR0p2NnlBc2dCeldj?= =?utf-8?B?UFNoWUZFN0hySlplSTl4RjlWcFBKRVNnZk8yQ2JTaHRVNHJQL003QzlDWUx1?= =?utf-8?B?Q1JFT3BrNVk4SklwdWw3MmJwcGdVM3g0UVZBMENyajNFNmcxYnMvSmNUMGVU?= =?utf-8?B?VlJKMW8zTVJPRGlJMERjdlRzYXJsT2ppWU5rM0ZzUFhnZml1V1lkQWdEcm54?= =?utf-8?B?SHlPTkFKQ2Z3RFVySnhzSTVhQVU3ZHQ1elFQQy9oRWZaWVpHVW1sdGxCUG5w?= =?utf-8?B?c0k3RGdmS0tXYTB2SzZOcXZVK0llRVViQXR2Qjh1ZWI3cEJsOTIxeUdDSXAx?= =?utf-8?B?OUp6Vy9CM0ZnWHlnc2RoRExtN2ZGS0Nud0xwTXJaVk5HV00vRnhsOHd0b09u?= =?utf-8?B?QVFwQXVKWWNad2RYSGtHZ0xmb3loTnowOVBpQy9henlWMERLYllENVg4dklW?= =?utf-8?B?L3Y3anAzb3c5Q0hlSEVHUE8vZ3Y2SHk2Y3pSeXpGU3RWVEozazUrNkFoajJ4?= =?utf-8?B?T0lnT1ZJMnAyYzBWazBoaFRTNURlTUJOWkNCUENsWnpvZ3VGTGRxVm1CaUN4?= =?utf-8?B?ME9kL3dXa3NPKy80NVY0SUZlbnhmTzU0WFhkbGdTbjluYy8vQkpVazlCY2xN?= =?utf-8?B?dkFFUnJXblM3ek1ld2hITW1HQmQxbXBMNWN5b3Fid3ZnY2lZb3JxaWk5dEo4?= =?utf-8?B?NWs3WUdSRGwxOHZVYnd6czNvQmdhOG55SFNNcC8weUlKR0psQUxuMkI3U0di?= =?utf-8?B?S1lDNXlkQlEzaGtMUUFPNUJib3BkZ1BxZEN3aU5DTU9xbnc0dzR6MEdkZXAv?= =?utf-8?B?ZTA0UXpkNXI3UlFodnZEZjRlbnBadlA1UVZaR2NVbzN4TEdZRXlla0N2aWZq?= =?utf-8?B?elhScmE4R05FYXJWRkxRWnY1Nmx6S3lhUTgzdkQxSENZUytMWEEwQWFlbWt5?= =?utf-8?B?MFRuWnkybmxENFBBeHZWc3BJQU9mR2t3VEdocjZJcnlBdDhSUjZPS0FTRkdT?= =?utf-8?Q?YSCmMmaUYfQ=3D?= MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-1ff67.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 6caa0ffa-e480-41e3-d668-08da5095f095 X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Jun 2022 19:16:54.2891 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ1P223MB0481 Subject: Re: [FFmpeg-devel] [PATCH v17 1/5] libavutil: Add wchartoutf8(), wchartoansi(), utf8toansi() and getenv_utf8() X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: > -----Original Message----- > From: ffmpeg-devel On Behalf Of Nil > Admirari > Sent: Friday, June 17, 2022 11:32 AM > To: ffmpeg-devel@ffmpeg.org > Subject: [FFmpeg-devel] [PATCH v17 1/5] libavutil: Add wchartoutf8(), > wchartoansi(), utf8toansi() and getenv_utf8() > > wchartoutf8() converts strings returned by WinAPI into UTF-8, > which is FFmpeg's preffered encoding. > > Some external dependencies, such as AviSynth, are still > not Unicode-enabled. utf8toansi() converts UTF-8 strings > into ANSI in two steps: UTF-8 -> wchar_t -> ANSI. > wchartoansi() is responsible for the second step of the conversion. > Conversion in just one step is not supported by WinAPI. > > Since these character converting functions allocate the buffer > of necessary size, they also facilitate the removal of MAX_PATH limit > in places where fixed-size ANSI/WCHAR strings were used > as filename buffers. > > getenv_utf8() wraps _wgetenv() converting its input from > and its output to UTF-8. Compared to plain getenv(), > getenv_utf8() requires a cleanup. > > Because of that, in places that only test the existence of > an environment variable or compare its value with a string > consisting entirely of ASCII characters, the use of plain getenv() > is still preferred. (libavutil/log.c check_color_terminal() > is an example of such a place.) > > Plain getenv() is also preffered in UNIX-only code, > such as bktr.c, fbdev_common.c, oss.c in libavdevice > or af_ladspa.c in libavfilter. > --- > configure | 1 + > libavutil/getenv_utf8.h | 71 > ++++++++++++++++++++++++++++++++++++++ > libavutil/wchar_filename.h | 51 +++++++++++++++++++++++++++ > 3 files changed, 123 insertions(+) > create mode 100644 libavutil/getenv_utf8.h > > diff --git a/configure b/configure > index 3dca1c4bd3..fa37a74531 100755 > --- a/configure > +++ b/configure > @@ -2272,6 +2272,7 @@ SYSTEM_FUNCS=" > fcntl > getaddrinfo > getauxval > + getenv > gethrtime > getopt > GetModuleHandle > diff --git a/libavutil/getenv_utf8.h b/libavutil/getenv_utf8.h > new file mode 100644 > index 0000000000..161e3e6202 > --- /dev/null > +++ b/libavutil/getenv_utf8.h > @@ -0,0 +1,71 @@ > +/* > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later > version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA > 02110-1301 USA > + */ > + > +#ifndef AVUTIL_GETENV_UTF8_H > +#define AVUTIL_GETENV_UTF8_H > + > +#include > + > +#include "mem.h" > + > +#ifdef HAVE_GETENV > + > +#ifdef _WIN32 > + > +#include "libavutil/wchar_filename.h" > + > +static inline char *getenv_utf8(const char *varname) > +{ > + wchar_t *varname_w, *var_w; > + char *var; > + > + if (utf8towchar(varname, &varname_w)) > + return NULL; > + if (!varname_w) > + return NULL; > + > + var_w = _wgetenv(varname_w); > + av_free(varname_w); > + > + if (!var_w) > + return NULL; > + if (wchartoutf8(var_w, &var)) > + return NULL; > + > + return var; > + > + // No CP_ACP fallback compared to other *_utf8() functions: > + // non UTF-8 strings must not be returned. > +} > + > +#else > + > +static inline char *getenv_utf8(const char *varname) > +{ > + return av_strdup(getenv(varname)); > +} > + > +#endif // _WIN32 > + > +#else > + > +#define getenv_utf8(x) NULL > + > +#endif // HAVE_GETENV > + > +#endif // AVUTIL_GETENV_UTF8_H > diff --git a/libavutil/wchar_filename.h b/libavutil/wchar_filename.h > index f36d9dfea3..a6d71e52e5 100644 > --- a/libavutil/wchar_filename.h > +++ b/libavutil/wchar_filename.h > @@ -41,6 +41,57 @@ static inline int utf8towchar(const char > *filename_utf8, wchar_t **filename_w) > return 0; > } > > +av_warn_unused_result > +static inline int wchartocp(unsigned int code_page, const wchar_t > *filename_w, > + char **filename) > +{ > + DWORD flags = code_page == CP_UTF8 ? WC_ERR_INVALID_CHARS : 0; > + int num_chars = WideCharToMultiByte(code_page, flags, > filename_w, -1, > + NULL, 0, NULL, NULL); > + if (num_chars <= 0) { > + *filename = NULL; > + return 0; > + } > + *filename = av_malloc_array(num_chars, sizeof *filename); > + if (!*filename) { > + errno = ENOMEM; > + return -1; > + } > + WideCharToMultiByte(code_page, flags, filename_w, -1, > + *filename, num_chars, NULL, NULL); > + return 0; > +} > + > +av_warn_unused_result > +static inline int wchartoutf8(const wchar_t *filename_w, char > **filename) > +{ > + return wchartocp(CP_UTF8, filename_w, filename); > +} > + > +av_warn_unused_result > +static inline int wchartoansi(const wchar_t *filename_w, char > **filename) > +{ > + return wchartocp(CP_ACP, filename_w, filename); > +} > + > +av_warn_unused_result > +static inline int utf8toansi(const char *filename_utf8, char > **filename) > +{ > + wchar_t *filename_w = NULL; > + int ret = -1; > + if (utf8towchar(filename_utf8, &filename_w)) > + return -1; > + > + if (!filename_w) { > + *filename = NULL; > + return 0; > + } > + > + ret = wchartoansi(filename_w, filename); > + av_free(filename_w); > + return ret; > +} > + > /** > * Checks for extended path prefixes for which normalization needs > to be skipped. > * see .NET6: PathInternal.IsExtended() > -- LGTM for the whole patchset. I didn't look at the getenv part, but I think Martin did (or will do). Thanks for all your effort (and patience)! Best wishes, softworkz _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".