From patchwork Wed Jun 15 20:03:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nil Admirari X-Patchwork-Id: 36249 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:1a22:b0:84:42e0:ad30 with SMTP id cj34csp557167pzb; Wed, 15 Jun 2022 13:04:26 -0700 (PDT) X-Google-Smtp-Source: AGRyM1vaCpB8dcWRAQ3f37hnECzpdy86papaojr2BpuorpPQmHBZv7MXhd8ERaguC4oBkOyFXlWC X-Received: by 2002:a17:907:3f81:b0:6ff:1a3d:9092 with SMTP id hr1-20020a1709073f8100b006ff1a3d9092mr1333351ejc.319.1655323466400; Wed, 15 Jun 2022 13:04:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1655323466; cv=none; d=google.com; s=arc-20160816; b=K5KzIk8TzqdCTZ5fmFIYWWL8E95YeayNY9IwlzaOeVZ9i8uxXQA305oi0sjO/LzeT3 6M5yl2B1AjHx1hGb+gBdhmzp43RhHlf55uqkWnNJeei3Y4GIR9PGrh328KXDWTCtn7FO B/YtMOqd/L62NcUEdG4qlIMlr3cH6azD+ujXRVb0Gly7a4bWCsGffkCqdmLtI8rqTMRr NvJM6CAQ8pWLZPYZAUXJRGcL7SmpZkxFJ8s6bD1FT2Ykub9PCRcHvlG3NAMnefN2fU23 au8MUjsKVLO52DwROj+CulL38W+9/5jd/ao9/0U5yaKJQ9CCECLu9ZqTElkJEp5Ol04q pY0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=11kip4i8Sav1iIliFot8XBqHoKVrK2inHG73IKelW5g=; b=H3toZKOTGY5tWK8RcajOZNZwI5lfhT0XYLwJRGDHcNAUqBYx11vAW4bsIpZxMatTd0 46u8Uq/Fy3Nn5krPaApvEFUarnlS6sxbJ/5kFgKhuAvDzva3D5tgdkKRBINkDQT37Gpa K3vkY9E2eRhVQW4TqUjG2hz35ihenZZsjs7AqDANYLeBs5oPOD0z3KNNZPL8NWIO6lBB 1hCAuKlFGhgTKnsqrfXOyciIrLD6zHUe4m7abifTN4mOtnFIhtQgIOjRVgM9hPfSBtmQ c12X2LEaT0KLYCRAEOLRhaULJ8VU9qlkrQNbZBFDo69qxBGVssD5cQALI1UtEE13pGrm obFw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@mailo.com header.s=mailo header.b=FOkSI4DM; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=mailo.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id gn12-20020a1709070d0c00b00711f55a31casi17049461ejc.775.2022.06.15.13.04.26; Wed, 15 Jun 2022 13:04:26 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@mailo.com header.s=mailo header.b=FOkSI4DM; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=mailo.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id AD5D768B476; Wed, 15 Jun 2022 23:04:06 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from msg-4.mailo.com (ip-15.mailobj.net [213.182.54.15]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C513068B6F9 for ; Wed, 15 Jun 2022 23:03:56 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=mailo.com; s=mailo; t=1655323430; bh=01k4QNYur3NG83ukxaih9+jx9TiqD61+2yWGYIa6HWg=; h=X-EA-Auth:From:To:Subject:Date:Message-Id:X-Mailer:MIME-Version: Content-Transfer-Encoding; b=FOkSI4DMkLTqOU0DV76BFekuP+mutr7EAQADlerQ2J1pzIVeYQpVOV4tPiKZl2ZGK 4M5o5wNZxXJagKJMo4xNxXhUXhYlM+hg0suZC6kvlGwl8UDTyPGyoWaEsxP1Urnkwo EX3+hvSfyFATiyuIUKSsX3/1OARu4fHTo1u8CUIc= Received: by b-2.in.mailobj.net [192.168.90.12] with ESMTP via ip-206.mailobj.net [213.182.55.206] Wed, 15 Jun 2022 22:03:50 +0200 (CEST) X-EA-Auth: kCaegZIR2ZVuT1/nCWwL9J7/fNF2roqk/VJoRiJgWiaZLFn+3jhXDXfuBeT3WBrMil+WKWgB/aojzHxbq7ILr3+hBuk07WZsPrYwHUJctqs= From: Nil Admirari To: ffmpeg-devel@ffmpeg.org Date: Wed, 15 Jun 2022 23:03:40 +0300 Message-Id: <20220615200344.17211-1-nil-admirari@mailo.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v16 1/5] libavutil: Add wchartoutf8(), wchartoansi(), utf8toansi() and getenv_utf8() X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 6chd13u8G5sE wchartoutf8() converts strings returned by WinAPI into UTF-8, which is FFmpeg's preffered encoding. Some external dependencies, such as AviSynth, are still not Unicode-enabled. utf8toansi() converts UTF-8 strings into ANSI in two steps: UTF-8 -> wchar_t -> ANSI. wchartoansi() is responsible for the second step of the conversion. Conversion in just one step is not supported by WinAPI. Since these character converting functions allocate the buffer of necessary size, they also facilitate the removal of MAX_PATH limit in places where fixed-size ANSI/WCHAR strings were used as filename buffers. getenv_utf8() wraps _wgetenv() converting its input from and its output to UTF-8. Compared to plain getenv(), getenv_utf8() requires a cleanup. Because of that, in places that only test the existence of an environment variable or compare its value with a string consisting entirely of ASCII characters, the use of plain getenv() is still preferred. (libavutil/log.c check_color_terminal() is an example of such a place.) Plain getenv() is also preffered in UNIX-only code, such as bktr.c, fbdev_common.c, oss.c in libavdevice or af_ladspa.c in libavfilter. --- configure | 1 + libavutil/getenv_utf8.h | 71 ++++++++++++++++++++++++++++++++++++++ libavutil/wchar_filename.h | 51 +++++++++++++++++++++++++++ 3 files changed, 123 insertions(+) create mode 100644 libavutil/getenv_utf8.h diff --git a/configure b/configure index 3dca1c4bd3..fa37a74531 100755 --- a/configure +++ b/configure @@ -2272,6 +2272,7 @@ SYSTEM_FUNCS=" fcntl getaddrinfo getauxval + getenv gethrtime getopt GetModuleHandle diff --git a/libavutil/getenv_utf8.h b/libavutil/getenv_utf8.h new file mode 100644 index 0000000000..161e3e6202 --- /dev/null +++ b/libavutil/getenv_utf8.h @@ -0,0 +1,71 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVUTIL_GETENV_UTF8_H +#define AVUTIL_GETENV_UTF8_H + +#include + +#include "mem.h" + +#ifdef HAVE_GETENV + +#ifdef _WIN32 + +#include "libavutil/wchar_filename.h" + +static inline char *getenv_utf8(const char *varname) +{ + wchar_t *varname_w, *var_w; + char *var; + + if (utf8towchar(varname, &varname_w)) + return NULL; + if (!varname_w) + return NULL; + + var_w = _wgetenv(varname_w); + av_free(varname_w); + + if (!var_w) + return NULL; + if (wchartoutf8(var_w, &var)) + return NULL; + + return var; + + // No CP_ACP fallback compared to other *_utf8() functions: + // non UTF-8 strings must not be returned. +} + +#else + +static inline char *getenv_utf8(const char *varname) +{ + return av_strdup(getenv(varname)); +} + +#endif // _WIN32 + +#else + +#define getenv_utf8(x) NULL + +#endif // HAVE_GETENV + +#endif // AVUTIL_GETENV_UTF8_H diff --git a/libavutil/wchar_filename.h b/libavutil/wchar_filename.h index f36d9dfea3..a6d71e52e5 100644 --- a/libavutil/wchar_filename.h +++ b/libavutil/wchar_filename.h @@ -41,6 +41,57 @@ static inline int utf8towchar(const char *filename_utf8, wchar_t **filename_w) return 0; } +av_warn_unused_result +static inline int wchartocp(unsigned int code_page, const wchar_t *filename_w, + char **filename) +{ + DWORD flags = code_page == CP_UTF8 ? WC_ERR_INVALID_CHARS : 0; + int num_chars = WideCharToMultiByte(code_page, flags, filename_w, -1, + NULL, 0, NULL, NULL); + if (num_chars <= 0) { + *filename = NULL; + return 0; + } + *filename = av_malloc_array(num_chars, sizeof *filename); + if (!*filename) { + errno = ENOMEM; + return -1; + } + WideCharToMultiByte(code_page, flags, filename_w, -1, + *filename, num_chars, NULL, NULL); + return 0; +} + +av_warn_unused_result +static inline int wchartoutf8(const wchar_t *filename_w, char **filename) +{ + return wchartocp(CP_UTF8, filename_w, filename); +} + +av_warn_unused_result +static inline int wchartoansi(const wchar_t *filename_w, char **filename) +{ + return wchartocp(CP_ACP, filename_w, filename); +} + +av_warn_unused_result +static inline int utf8toansi(const char *filename_utf8, char **filename) +{ + wchar_t *filename_w = NULL; + int ret = -1; + if (utf8towchar(filename_utf8, &filename_w)) + return -1; + + if (!filename_w) { + *filename = NULL; + return 0; + } + + ret = wchartoansi(filename_w, filename); + av_free(filename_w); + return ret; +} + /** * Checks for extended path prefixes for which normalization needs to be skipped. * see .NET6: PathInternal.IsExtended()