From patchwork Sun May 15 22:17:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 34711 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:a885:b0:7f:4be2:bd17 with SMTP id ca5csp1281551pzb; Sun, 15 May 2022 15:18:03 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzD1h1Cha9a4SXQfOI8qr7PYY1MptappjkqhPCTKUrEoFwoSf/oT5/zLnnfIv+3n+oZnsGJ X-Received: by 2002:a17:907:72ce:b0:6f4:5a57:320c with SMTP id du14-20020a17090772ce00b006f45a57320cmr12756801ejc.75.1652653083573; Sun, 15 May 2022 15:18:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1652653083; cv=none; d=google.com; s=arc-20160816; b=Lymp2jURlr+INhia/1JzojltYz+pzTTFwUNCPHJIojUssh8wXgxZBBZBQ8BRSj6b9t sYL9nh4XPdGCqlB1koOq51nB6Lrp1ugUzMA4Nv1bUN8Yx4dLtL5+zksiInLNtRw6IYIe OE23H/hCPTAOvRaLse3PqRO8B1edSkcsryvbc/i0jljECpHoFqu+hUDUNzCB1lYA5CZC 3u/Y+arA/uYmj+7HP5NTEK94Omtma8xY7qs2hUX/R3/xOkoM0Re46jyqIPvohWXOtKBG 7rCVF9+MTjCQgGuWS8SqfWxKpQMD6/104bUC1w+yXQOO0hCzc59gKezKc8733MPyJ2R9 FLeQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:from:references :in-reply-to:message-id:dkim-signature:delivered-to; bh=VhDZfUtyWlSp2wpgbssGyG+dzempqQNWx6r7zkf5zEs=; b=iu8vzhonQldo1WjNBjNXa4xs1QRNzZROcv4zinXPgR+wFghA6/lYeJbLCtkplieoSn w0kcnmHnaefdiZ7qxj4P/xUu2xa0FvLME/gvSe1kXAUKHMLMholKupY+Qry0uFdJeQ/R HzFjJrPewonv3RoHX+EKgmw7I04delEBgZfBvnjSdFBa+QodKnOTPYk8kg57o+sWzIa0 lCza1mxYGQWx+vqUQ9lIuGiHBuUbIwqXV+qSIHEM8ZfAuITT5VaeS/jXiktUcEgstyNQ Vab6ImoR2jFL9pTVFEate+SkmmBlXuLZOB6fwxHn4jrkzSnvzl+SdH55GOQZLcdJwBFV 7Zgg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=mWhpA2Xj; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id nc1-20020a1709071c0100b006f40ceae093si10171077ejc.19.2022.05.15.15.18.03; Sun, 15 May 2022 15:18:03 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=mWhpA2Xj; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7D3B368B4F2; Mon, 16 May 2022 01:18:00 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f44.google.com (mail-pj1-f44.google.com [209.85.216.44]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 60D5D68B4BC for ; Mon, 16 May 2022 01:17:54 +0300 (EEST) Received: by mail-pj1-f44.google.com with SMTP id gg20so2721758pjb.1 for ; Sun, 15 May 2022 15:17:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=GibITtRjQT0sMewjfHX7c4WpoDSHlRXWB+xKoD8kBG8=; b=mWhpA2XjEcMA8QHz8DOLJpmXbcHPKkJC1QaXZc7hzfQKNko3zFvDbuo7wvCVYNsI3H LpOcIUoWwoA0Kh/NZrzcD6dzuyOu3bcNMR0ESrqmwFA038bNBBn80FoWsqJNr+R0Ca4i E5MGeXC3Mr0tal6EDaIW3IUNLw1uoPnD9euOkybKpCebxZTqlvS+6dJ2wDlIPwSQY5X9 gZ6iMTXX8EEnHsdYlW4ZJq8/t0F5Rb3a7NGYQaq8BiHChXBa5gojmMVrJuF3q/uE5bzR scPTWFKBq6GY9aXBdMAuxlNLsAlAn7S/n47U92fMKxafBm1doJHTqoueSb5baoQPp86J Jinw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=GibITtRjQT0sMewjfHX7c4WpoDSHlRXWB+xKoD8kBG8=; b=jKtf5XPHJH6WTqskuh6buxdweQ0Eop+9kX1UECFBsBqBWm7QgbThET1NU3HucaFNkT IX5wkzZXmS0S3eWZiQdh1KCIRVi4DAzcenH5HHxw6bXJ1PERs8LwPrTX5xpAiVnp2OPC bFHhVbcEt2t9OUbGZ/4uffCALfTMKUGfHtL7Qd9RAlrBdV2FUGIQj7ipaj1udCuHGMRs cLCP/Q8X0SngZrw2K82CrEldrtPLHNFYGIyCT0mBTDx1IYUuVEZEx7HLmsFoOwMG4jtz LNRy08n8VNFzWNBWiVE99EqKfnpEzclXabbciLkC/+Ut83ZBYf+HqD+0CcC4TNROCqv4 gpAg== X-Gm-Message-State: AOAM5333VmJNwz9y1bT7IYXXgOROAxXj9QmUwHZcoN3pqlAdeV8llCFk OuQNcZ8n9QFct0nXM8ZBi6IefFSenGWFYw== X-Received: by 2002:a17:903:41c7:b0:15e:b1f4:3530 with SMTP id u7-20020a17090341c700b0015eb1f43530mr15366307ple.84.1652653072341; Sun, 15 May 2022 15:17:52 -0700 (PDT) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id h14-20020a170902680e00b0015e8d4eb2afsm2750857plk.249.2022.05.15.15.17.51 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 15 May 2022 15:17:51 -0700 (PDT) Message-Id: In-Reply-To: References: From: ffmpegagent Date: Sun, 15 May 2022 22:17:49 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v2 0/2] Support long file names on Windows X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: softworkz Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Lmv1HW0w0vvm This patchset adds support for long file and directory paths on Windows. The implementation follows the same logic that .NET is using internally, with the only exception that it doesn't expand short path components in 8.3 format. .NET does this as the same function is also used for other purposes, but in our case, that's not required. Short (8.3) paths are working as well with the extended path prefix, even when longer than 260. Successfully tested: * Regular paths wth drive letter * Regular UNC paths * Long paths wth drive letter * Long paths wth drive letter and forward slashes * Long UNC paths * Prefixed paths wth drive letter * Prefixed UNC paths I have kept the individual functions separate on purpose, to make it easy to compare with the .NET impl. (compilers should inlinie those anyway) v2 * wchar_filename: Improve comments and function documentation * os_support: adjust defines to use win32_stat softworkz (2): avutil/wchar_filename,file_open: Support long file names on Windows avformat/os_support: Support long file names on Windows libavformat/os_support.h | 26 +++++-- libavutil/file_open.c | 2 +- libavutil/wchar_filename.h | 156 +++++++++++++++++++++++++++++++++++++ 3 files changed, 178 insertions(+), 6 deletions(-) base-commit: d2d8b9b972ba2df6b2a2ebe29f5307cbb7a69c33 Published-As: https://github.com/ffstaging/FFmpeg/releases/tag/pr-ffstaging-28%2Fsoftworkz%2Fsubmit_long_filenames-v2 Fetch-It-Via: git fetch https://github.com/ffstaging/FFmpeg pr-ffstaging-28/softworkz/submit_long_filenames-v2 Pull-Request: https://github.com/ffstaging/FFmpeg/pull/28 Range-diff vs v1: 1: 26c579e4ee ! 1: b66dbdf40c avutil/wchar_filename,file_open: Support long file names on Windows @@ libavutil/wchar_filename.h: static inline int utf8towchar(const char *filename_u return 0; } + ++/** ++ * Checks for extended path prefixes for which normalization needs to be skipped. ++ * see .NET6: PathInternal.IsExtended() ++ */ +static inline int path_is_extended(const wchar_t *path) +{ -+ // see .NET6: PathInternal.IsExtended() + size_t len = wcslen(path); + if (len >= 4 && path[0] == L'\\' && (path[1] == L'\\' || path[1] == L'?') && path[2] == L'?' && path[3] == L'\\') + return 1; @@ libavutil/wchar_filename.h: static inline int utf8towchar(const char *filename_u + return 0; +} + ++/** ++ * Performs path normalization by calling GetFullPathNameW(). ++ * see .NET6: PathHelper.GetFullPathName() ++ */ +static inline int get_full_path_name(wchar_t **ppath_w) +{ + int num_chars; + wchar_t *temp_w; + -+ // see .NET6: PathHelper.GetFullPathName() + num_chars = GetFullPathNameW(*ppath_w, 0, NULL, NULL); + if (num_chars <= 0) { + errno = EINVAL; @@ libavutil/wchar_filename.h: static inline int utf8towchar(const char *filename_u + return 0; +} + ++/** ++ * Normalizes a Windows file or folder path. ++ * Expansion of short paths (with 8.3 path components) is currently omitted ++ * as it is not required for accessing long paths. ++ * see .NET6: PathHelper.Normalize(). ++ */ +static inline int path_normalize(wchar_t **ppath_w) +{ + int ret; + -+ // see .NET6: PathHelper.Normalize() + if ((ret = get_full_path_name(ppath_w)) < 0) + return ret; + + /* What .NET does at this point is to call PathHelper.TryExpandShortFileName() -+ in case the path contains a '~' character. -+ We don't need to do this as we don't need to normalize the file name -+ for presentation, and the extended path prefix works with 8.3 path -+ components as well */ ++ * in case the path contains a '~' character. ++ * We don't need to do this as we don't need to normalize the file name ++ * for presentation, and the extended path prefix works with 8.3 path ++ * components as well ++ */ + return 0; +} + ++/** ++ * Adds an extended path or UNC prefix to longs paths or paths ending ++ * with a space or a dot. (' ' or '.'). ++ * This function expects that the path has been normalized before by ++ * calling path_normalize(). ++ * see .NET6: PathInternal.EnsureExtendedPrefix() * ++ */ +static inline int add_extended_prefix(wchar_t **ppath_w) +{ + const wchar_t *unc_prefix = L"\\\\?\\UNC\\"; @@ libavutil/wchar_filename.h: static inline int utf8towchar(const char *filename_u + if (len < 2) + return 0; + -+ // see .NET6: PathInternal.EnsureExtendedPrefix() ++ /* We're skipping the check IsPartiallyQualified() because ++ * we know we have called GetFullPathNameW() already, also ++ * we don't check IsDevice() because device paths are not ++ * allowed to be long paths and we're calling this only ++ * for long paths. ++ */ + if (path_w[0] == L'\\' && path_w[1] == L'\\') { ++ // The length of unc_prefix is 6 plus 1 for terminating zeros + temp_w = (wchar_t *)av_calloc(len + 6 + 1, sizeof(wchar_t)); + if (!temp_w) { + errno = ENOMEM; @@ libavutil/wchar_filename.h: static inline int utf8towchar(const char *filename_u + wcscpy(temp_w, unc_prefix); + wcscat(temp_w, path_w + 2); + } else { ++ // The length of extended_path_prefix is 4 plus 1 for terminating zeros + temp_w = (wchar_t *)av_calloc(len + 4 + 1, sizeof(wchar_t)); + if (!temp_w) { + errno = ENOMEM; @@ libavutil/wchar_filename.h: static inline int utf8towchar(const char *filename_u + return 0; +} + ++/** ++ * Converts a file or folder path to wchar_t for use with Windows file ++ * APIs. Paths with extended path prefix (either '\\?\' or \??\') are ++ * left unchanged. ++ * All other paths are normalized and converted to absolute paths. ++ * Longs paths (>= 260) are prefixed with the extended path or extended ++ * UNC path prefix. ++ * see .NET6: Path.GetFullPath() and Path.GetFullPathInternal() ++ */ +static inline int get_extended_win32_path(const char *path, wchar_t **ppath_w) +{ + int ret; + size_t len; + -+ // see .NET6: Path.GetFullPath() and Path.GetFullPathInternal() + if ((ret = utf8towchar(path, ppath_w)) < 0) + return ret; + + if (path_is_extended(*ppath_w)) { -+ /* \\?\ paths are considered normalized by definition. Windows doesn't normalize \\?\ -+ paths and neither should we. Even if we wanted to, GetFullPathName does not work -+ properly with device paths. If one wants to pass a \\?\ path through normalization -+ one can chop off the prefix, pass it to GetFullPath and add it again. */ ++ /* Paths prefixed with '\\?\' or \??\' are considered normalized by definition. ++ * Windows doesn't normalize those paths and neither should we. ++ */ + return 0; + } + 2: acd81c61c3 ! 2: 8ecbafe2b7 avformat/os_support: Support long file names on Windows @@ Commit message Signed-off-by: softworkz ## libavformat/os_support.h ## +@@ + # ifdef stat + # undef stat + # endif +-# define stat _stati64 ++# define stat win32_stat ++ ++ struct win32_stat ++ { ++ _dev_t st_dev; ++ _ino_t st_ino; ++ unsigned short st_mode; ++ short st_nlink; ++ short st_uid; ++ short st_gid; ++ _dev_t st_rdev; ++ __int64 st_size; ++ __time64_t st_atime; ++ __time64_t st_mtime; ++ __time64_t st_ctime; ++ }; ++ + # ifdef fstat + # undef fstat + # endif @@ libavformat/os_support.h: static inline int win32_##name(const char *filename_utf8) \ wchar_t *filename_w; \ int ret; \