From patchwork Mon May 16 21:23:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 34712 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:a885:b0:7f:4be2:bd17 with SMTP id ca5csp1732788pzb; Mon, 16 May 2022 14:23:37 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz20sWe4KMaA8Kp9dBSmYLZ8BJ+tLWLqoq50z8YD4D+sQl6uOPiWHWQ8q9DYNfsi+pOFQmO X-Received: by 2002:a17:907:761c:b0:6d6:e553:7bd1 with SMTP id jx28-20020a170907761c00b006d6e5537bd1mr16428803ejc.5.1652736217069; Mon, 16 May 2022 14:23:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1652736217; cv=none; d=google.com; s=arc-20160816; b=VH4eJqN4U7JjftV27+2CJuEX7Dfwl7kSVKWRq2ggrmpR4A97x2Fk9tiLqHgQj4NZju 0HrKZ+OGkA4oWDW7m/g6ZTpS9y6S8Vb3bWV/DkgzD44q8FUFy4v4CVv+SFd//ZGRizc4 U4dHYGbkr3hFwdf44Ctg/C1kaGz+pA485ggCL/2n9udE0H9Eu8HEiw93Ft8oRHrFhcYp mVijcuKKeUJ9Y+Tcs+OtN+wOycb8ds/mtLUhqN9QOSBGCPR/Up0QEoywXdEaivHjKbnY VDV4ubNIKzLb/i3Nw1DpjKIlfGWwg78g58kKkmkkJLZuo5svO3L9wQVCA67FplZG4Udi O4PQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:from:references :in-reply-to:message-id:dkim-signature:delivered-to; bh=mXvbD+YfcWlrqm6nhSmggbeoi1PFGovOC7QGbKS10Cc=; b=yVzHQG1ZnWH5YcONpqfMrItrTHIWI5So6maLUes3vPIFvowKT6z/6WRP+C49ln2GUC QWxyK3oOu5bF9cRSspK9w0qdZ6HTTztjVKoTLD5Y6f7jFYsti3susOkm/DXkTwjPXXdj KLwsHE9tRaRlcwbaTN1f4tKzfuXSDuAI8a9qFoeU4xAtHSnB9Y3pK45jZxRiucDh7yoU 6HPkJk+zUFjNQ0Hj2NqEaWKD5VsqQiSdMLQj4Cnzo78+Cjnw67qBmVW7ACeXvq7TD1LM WLpLwI0LSXZe+Ma5sofRU+qtmGy7iRlpmQq7O4U7Ruu2PhI8Yr/2PYVB3b7ZaYHIlmHX efpg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=amaZEZ72; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id cw19-20020a170906479300b006f39874cf75si438074ejc.647.2022.05.16.14.23.36; Mon, 16 May 2022 14:23:37 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=amaZEZ72; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 742BF68B423; Tue, 17 May 2022 00:23:33 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id DE1BF68B18C for ; Tue, 17 May 2022 00:23:26 +0300 (EEST) Received: by mail-pl1-f176.google.com with SMTP id q18so15628538pln.12 for ; Mon, 16 May 2022 14:23:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=QMKN3iRDkFoRyjoRP7NWXUMARP0LO2nDVVsTtdGRldU=; b=amaZEZ72tzSOg+CSFBmbT1NXJHCDfYs+dr66ppdZQxynsiDyZZTAhLYfVl9VxWe/bC TmYZ/u3n2vBqJofAYg8KNOs2YI7cTr8Af/tfYkBcbqSWarjH4reaPNI+TAy39NZCypaG Q1YKlKNsTSQBdvei+zclA+KuPun3iHPYxrcv5ANxeBR0iLG9SLB1jr0nkdzUR6O0j291 aU0vVQ35MBItneF4NLmP/41B9jmxvcOzbOWKBefDKhZipMX6jgUca9/DyXwRy4+GPP62 BlTB7E7i0ooPq5Z9UEWdbBOd5e8nwiGW2GrNS2HtQQ72TPAVCevjDmHRJbxH8pUd0sMC 0pcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=QMKN3iRDkFoRyjoRP7NWXUMARP0LO2nDVVsTtdGRldU=; b=KYAXyWT87cK8LyCeIuh/0Fbd2SNT0c7ndM03987J563rl4vtAWQfg1dldnhHvPoYNZ 3Nn0xPSIE3GnFFzVsTiukLIiKE5NUNbdDw7cGr5AwoKuEznFSzK1JCICV90Tr5ovRN1B MMeJly+L9M8zVt+CfchakLGjwrg4dDxGc3wAbNdNG1OXULSbO14P2+nEHrsFUl/IJ111 Mw9jRmpWT8AdwzC5xzo3dWXXJD+hxEukyfOnj/dhDtbpDDs1zszmudmw9N3JyWcjxHaJ CHPqrGmE/6iCt+YgxOxRDkDdqEFFzq6BAlLq5BbtI/4114vmdB0YZv7sUP5i0VmavHnE jYTA== X-Gm-Message-State: AOAM530imsUv94JmMhAmYuG7/7PDYvXWY372Qy7Ft6KybVpnpSGD3eGK XEiVFYzuM6ZQb8w+LEnfaW5DmuGkjKGNAQ== X-Received: by 2002:a17:902:7296:b0:151:62b1:e2b0 with SMTP id d22-20020a170902729600b0015162b1e2b0mr19222794pll.165.1652736204902; Mon, 16 May 2022 14:23:24 -0700 (PDT) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id c11-20020a170903234b00b0015e8d4eb21csm7735150plh.102.2022.05.16.14.23.24 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 16 May 2022 14:23:24 -0700 (PDT) Message-Id: In-Reply-To: References: From: ffmpegagent Date: Mon, 16 May 2022 21:23:21 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v3 0/2] Support long file names on Windows X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: softworkz Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: yHj3RgiIDGyb This patchset adds support for long file and directory paths on Windows. The implementation follows the same logic that .NET is using internally, with the only exception that it doesn't expand short path components in 8.3 format. .NET does this as the same function is also used for other purposes, but in our case, that's not required. Short (8.3) paths are working as well with the extended path prefix, even when longer than 260. Successfully tested: * Regular paths wth drive letter * Regular UNC paths * Long paths wth drive letter * Long paths wth drive letter and forward slashes * Long UNC paths * Prefixed paths wth drive letter * Prefixed UNC paths I have kept the individual functions separate on purpose, to make it easy to compare with the .NET impl. (compilers should inlinie those anyway) v2 * wchar_filename: Improve comments and function documentation * os_support: adjust defines to use win32_stat v3 * removed length check in path_is_extended() * added path_is_device_path() check in add_extended_prefix() * add_extended_prefix(): clarified doc and add checks * clarified string allocation length calculation * replaced 260 with MAX_PATH * removed redundant checks after normalization softworkz (2): avutil/wchar_filename,file_open: Support long file names on Windows avformat/os_support: Support long file names on Windows libavformat/os_support.h | 26 ++++-- libavutil/file_open.c | 2 +- libavutil/wchar_filename.h | 166 +++++++++++++++++++++++++++++++++++++ 3 files changed, 188 insertions(+), 6 deletions(-) base-commit: e3580f60775c897c3b13b178c57ab191ecc4a031 Published-As: https://github.com/ffstaging/FFmpeg/releases/tag/pr-ffstaging-28%2Fsoftworkz%2Fsubmit_long_filenames-v3 Fetch-It-Via: git fetch https://github.com/ffstaging/FFmpeg pr-ffstaging-28/softworkz/submit_long_filenames-v3 Pull-Request: https://github.com/ffstaging/FFmpeg/pull/28 Range-diff vs v2: 1: b66dbdf40c ! 1: ce70f7b021 avutil/wchar_filename,file_open: Support long file names on Windows @@ libavutil/wchar_filename.h: static inline int utf8towchar(const char *filename_u + */ +static inline int path_is_extended(const wchar_t *path) +{ -+ size_t len = wcslen(path); -+ if (len >= 4 && path[0] == L'\\' && (path[1] == L'\\' || path[1] == L'?') && path[2] == L'?' && path[3] == L'\\') ++ if (path[0] == L'\\' && (path[1] == L'\\' || path[1] == L'?') && path[2] == L'?' && path[3] == L'\\') ++ return 1; ++ ++ return 0; ++} ++ ++/** ++ * Checks for a device path prefix. ++ * see .NET6: PathInternal.IsDevicePath() ++ */ ++static inline int path_is_device_path(const wchar_t *path) ++{ ++ if (path[0] == L'\\' && path[1] == L'\\' && path[2] == L'.' && path[3] == L'\\') + return 1; + + return 0; @@ libavutil/wchar_filename.h: static inline int utf8towchar(const char *filename_u + * Adds an extended path or UNC prefix to longs paths or paths ending + * with a space or a dot. (' ' or '.'). + * This function expects that the path has been normalized before by -+ * calling path_normalize(). ++ * calling path_normalize() and it doesn't check whether the path is ++ * actually long (> MAX_PATH). + * see .NET6: PathInternal.EnsureExtendedPrefix() * + */ +static inline int add_extended_prefix(wchar_t **ppath_w) @@ libavutil/wchar_filename.h: static inline int utf8towchar(const char *filename_u + const size_t len = wcslen(path_w); + wchar_t *temp_w; + -+ if (len < 2) ++ /* We're skipping the check IsPartiallyQualified() because ++ * we expect to have called GetFullPathNameW() already. */ ++ if (len < 2 || path_is_extended(*ppath_w) || path_is_device_path(*ppath_w)) { + return 0; ++ } + -+ /* We're skipping the check IsPartiallyQualified() because -+ * we know we have called GetFullPathNameW() already, also -+ * we don't check IsDevice() because device paths are not -+ * allowed to be long paths and we're calling this only -+ * for long paths. -+ */ + if (path_w[0] == L'\\' && path_w[1] == L'\\') { -+ // The length of unc_prefix is 6 plus 1 for terminating zeros -+ temp_w = (wchar_t *)av_calloc(len + 6 + 1, sizeof(wchar_t)); ++ /* unc_prefix length is 8 plus 1 for terminating zeros, ++ * we subtract 2 for the leading '\\' of the original path */ ++ temp_w = (wchar_t *)av_calloc(len - 2 + 8 + 1, sizeof(wchar_t)); + if (!temp_w) { + errno = ENOMEM; + return -1; @@ libavutil/wchar_filename.h: static inline int utf8towchar(const char *filename_u + * APIs. Paths with extended path prefix (either '\\?\' or \??\') are + * left unchanged. + * All other paths are normalized and converted to absolute paths. -+ * Longs paths (>= 260) are prefixed with the extended path or extended ++ * Longs paths (>= MAX_PATH) are prefixed with the extended path or extended + * UNC path prefix. + * see .NET6: Path.GetFullPath() and Path.GetFullPathInternal() + */ @@ libavutil/wchar_filename.h: static inline int utf8towchar(const char *filename_u + + // see .NET6: PathInternal.EnsureExtendedPrefixIfNeeded() + len = wcslen(*ppath_w); -+ if (len >= 260 || (*ppath_w)[len - 1] == L' ' || (*ppath_w)[len - 1] == L'.') { ++ if (len >= MAX_PATH) { + if ((ret = add_extended_prefix(ppath_w)) < 0) + return ret; + } 2: 8ecbafe2b7 = 2: a5268800a4 avformat/os_support: Support long file names on Windows