Message ID | 20220613162626.11541-1-nil-admirari@mailo.com |
---|---|
State | New |
Headers | show |
Series | [FFmpeg-devel,v14,1/5] libavutil: Add wchartoutf8(), wchartoansi(), utf8toansi() and getenv_utf8() | expand |
Context | Check | Description |
---|---|---|
andriy/make_x86 | success | Make finished |
andriy/make_fate_x86 | success | Make fate finished |
On Mon, 13 Jun 2022, Nil Admirari wrote: > wchartoutf8() converts strings returned by WinAPI into UTF-8, > which is FFmpeg's preffered encoding. > > Some external dependencies, such as AviSynth, are still > not Unicode-enabled. utf8toansi() converts UTF-8 strings > into ANSI in two steps: UTF-8 -> wchar_t -> ANSI. > wchartoansi() is responsible for the second step of the conversion. > Conversion in just one step is not supported by WinAPI. > > Since these character converting functions allocate the buffer > of necessary size, they also facilitate the removal of MAX_PATH limit > in places where fixed-size ANSI/WCHAR strings were used > as filename buffers. > > getenv_utf8() wraps _wgetenv() converting its input from > and its output to UTF-8. Compared to plain getenv(), > getenv_utf8() requires a cleanup. > > Because of that, in places that only test the existence of > an environment variable or compare its value with a string > consisting entirely of ASCII characters, the use of plain getenv() > is still preferred. (libavutil/log.c check_color_terminal() > is an example of such a place.) > > Plain getenv() is also preffered in UNIX-only code, > such as bktr.c, fbdev_common.c, oss.c in libavdevice > or af_ladspa.c in libavfilter. > --- > libavutil/getenv_utf8.h | 63 ++++++++++++++++++++++++++++++++++++++ > libavutil/wchar_filename.h | 51 ++++++++++++++++++++++++++++++ > 2 files changed, 114 insertions(+) > create mode 100644 libavutil/getenv_utf8.h > > diff --git a/libavutil/getenv_utf8.h b/libavutil/getenv_utf8.h > new file mode 100644 > index 0000000000..2c48a36355 > --- /dev/null > +++ b/libavutil/getenv_utf8.h > @@ -0,0 +1,63 @@ > +/* > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +#ifndef AVUTIL_GETENV_UTF8_H > +#define AVUTIL_GETENV_UTF8_H > + > +#include <stdlib.h> > + > +#include "mem.h" > + > +#ifdef _WIN32 > + > +#include "libavutil/wchar_filename.h" > + > +static inline char *getenv_utf8(const char *varname) > +{ > + wchar_t *varname_w, *var_w; > + char *var; > + > + if (utf8towchar(varname, &varname_w)) > + return NULL; > + if (!varname_w) > + return NULL; > + > + var_w = _wgetenv(varname_w); > + av_free(varname_w); > + > + if (!var_w) > + return NULL; > + if (wchartoutf8(var_w, &var)) > + return NULL; > + > + return var; > + > + // No CP_ACP fallback compared to other *_utf8() functions: > + // non UTF-8 strings must not be returned. > +} Note, there are Windows configurations that entirely lack getenv() (and presumably _wgetenv() too), namely the Windows Store App / Windows Phone App API subsets for Windows 8.x. I think those functions were allowed again in Windows 10 though... In configure, we have this: check_func_headers stdlib.h getenv enabled getenv || echo "#define getenv(x) NULL" >> $TMPH I guess we'd might have to add getenv to e.g. the SYSTEM_FUNCS list, so we'd get a HAVE_GETENV in config.h - then we could make getenv_utf8 a no-op if HAVE_GETENV is 0. (I'm not in a situation where I can test and investigate properly right now, I can maybe look closer into it later.) // Martin
> I guess we'd might have to add getenv to e.g. the SYSTEM_FUNCS list, so > we'd get a HAVE_GETENV in config.h - then we could make getenv_utf8 a > no-op if HAVE_GETENV is 0. Done: https://ffmpeg.org/pipermail/ffmpeg-devel/2022-June/297596.html
diff --git a/libavutil/getenv_utf8.h b/libavutil/getenv_utf8.h new file mode 100644 index 0000000000..2c48a36355 --- /dev/null +++ b/libavutil/getenv_utf8.h @@ -0,0 +1,63 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVUTIL_GETENV_UTF8_H +#define AVUTIL_GETENV_UTF8_H + +#include <stdlib.h> + +#include "mem.h" + +#ifdef _WIN32 + +#include "libavutil/wchar_filename.h" + +static inline char *getenv_utf8(const char *varname) +{ + wchar_t *varname_w, *var_w; + char *var; + + if (utf8towchar(varname, &varname_w)) + return NULL; + if (!varname_w) + return NULL; + + var_w = _wgetenv(varname_w); + av_free(varname_w); + + if (!var_w) + return NULL; + if (wchartoutf8(var_w, &var)) + return NULL; + + return var; + + // No CP_ACP fallback compared to other *_utf8() functions: + // non UTF-8 strings must not be returned. +} + +#else + +static inline char *getenv_utf8(const char *varname) +{ + return av_strdup(getenv(varname)); +} + +#endif // _WIN32 + +#endif // AVUTIL_GETENV_UTF8_H diff --git a/libavutil/wchar_filename.h b/libavutil/wchar_filename.h index f36d9dfea3..a6d71e52e5 100644 --- a/libavutil/wchar_filename.h +++ b/libavutil/wchar_filename.h @@ -41,6 +41,57 @@ static inline int utf8towchar(const char *filename_utf8, wchar_t **filename_w) return 0; } +av_warn_unused_result +static inline int wchartocp(unsigned int code_page, const wchar_t *filename_w, + char **filename) +{ + DWORD flags = code_page == CP_UTF8 ? WC_ERR_INVALID_CHARS : 0; + int num_chars = WideCharToMultiByte(code_page, flags, filename_w, -1, + NULL, 0, NULL, NULL); + if (num_chars <= 0) { + *filename = NULL; + return 0; + } + *filename = av_malloc_array(num_chars, sizeof *filename); + if (!*filename) { + errno = ENOMEM; + return -1; + } + WideCharToMultiByte(code_page, flags, filename_w, -1, + *filename, num_chars, NULL, NULL); + return 0; +} + +av_warn_unused_result +static inline int wchartoutf8(const wchar_t *filename_w, char **filename) +{ + return wchartocp(CP_UTF8, filename_w, filename); +} + +av_warn_unused_result +static inline int wchartoansi(const wchar_t *filename_w, char **filename) +{ + return wchartocp(CP_ACP, filename_w, filename); +} + +av_warn_unused_result +static inline int utf8toansi(const char *filename_utf8, char **filename) +{ + wchar_t *filename_w = NULL; + int ret = -1; + if (utf8towchar(filename_utf8, &filename_w)) + return -1; + + if (!filename_w) { + *filename = NULL; + return 0; + } + + ret = wchartoansi(filename_w, filename); + av_free(filename_w); + return ret; +} + /** * Checks for extended path prefixes for which normalization needs to be skipped. * see .NET6: PathInternal.IsExtended()