Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
 help / color / mirror / Atom feed
From: ffmpegagent <ffmpegagent@gmail.com>
To: ffmpeg-devel@ffmpeg.org
Cc: "Martin Storsjö" <martin@martin.st>,
	softworkz <softworkz@hotmail.com>,
	"Hendrik Leppkes" <h.leppkes@gmail.com>
Subject: [FFmpeg-devel] [PATCH v8 0/3] Support long file names on Windows
Date: Thu, 26 May 2022 09:28:47 +0000
Message-ID: <pull.28.v8.ffstaging.FFmpeg.1653557330.ffmpegagent@gmail.com> (raw)
In-Reply-To: <pull.28.v7.ffstaging.FFmpeg.1653430851.ffmpegagent@gmail.com>

This patchset adds support for long file and directory paths on Windows. The
implementation follows the same logic that .NET is using internally, with
the only exception that it doesn't expand short path components in 8.3
format. .NET does this as the same function is also used for other purposes,
but in our case, that's not required. Short (8.3) paths are working as well
with the extended path prefix, even when longer than 260.

Successfully tested:

 * Regular paths wth drive letter
 * Regular UNC paths
 * Long paths wth drive letter
 * Long paths wth drive letter and forward slashes
 * Long UNC paths
 * Prefixed paths wth drive letter
 * Prefixed UNC paths

I have kept the individual functions separate on purpose, to make it easy to
compare with the .NET impl. (compilers should inlinie those anyway)

v2

 * wchar_filename: Improve comments and function documentation
 * os_support: adjust defines to use win32_stat

v3

 * removed length check in path_is_extended()
 * added path_is_device_path() check in add_extended_prefix()
 * add_extended_prefix(): clarified doc and add checks
 * clarified string allocation length calculation
 * replaced 260 with MAX_PATH
 * removed redundant checks after normalization

v4

 * rebased. no changes

v5

 * resolved the ugly struct duplication
 * compatible with _USE_32BIT_TIME_T

v6

 * wchar_filename.h: added links to .NET source code
 * wchar_filename.h: free allocations on error
 * os_support.hs: use clean and safe way to redirect stat() calls

v7

 * os_support.h: remapped fstat with win32_stat structure
 * os_support.h: use int64_t for some members
 * avformat/file: remove _WIN32 condition

v8

 * os_support.h: documented win32_stat structure
 * os_support.h: renamed function parameters

softworkz (3):
  avutil/wchar_filename,file_open: Support long file names on Windows
  avformat/os_support: Support long file names on Windows
  avformat/file: remove _WIN32 condition

 libavformat/file.c         |   4 -
 libavformat/os_support.h   | 116 ++++++++++++++++++------
 libavutil/file_open.c      |   2 +-
 libavutil/wchar_filename.h | 180 +++++++++++++++++++++++++++++++++++++
 4 files changed, 272 insertions(+), 30 deletions(-)


base-commit: 6076dbcb55d0c9b6693d1acad12a63f7268301aa
Published-As: https://github.com/ffstaging/FFmpeg/releases/tag/pr-ffstaging-28%2Fsoftworkz%2Fsubmit_long_filenames-v8
Fetch-It-Via: git fetch https://github.com/ffstaging/FFmpeg pr-ffstaging-28/softworkz/submit_long_filenames-v8
Pull-Request: https://github.com/ffstaging/FFmpeg/pull/28

Range-diff vs v7:

 1:  960aa795ff = 1:  960aa795ff avutil/wchar_filename,file_open: Support long file names on Windows
 2:  7751335906 ! 2:  d0cc40a0d4 avformat/os_support: Support long file names on Windows
     @@ libavformat/os_support.h
      +
      +#  define stat win32_stat
      +
     ++    /*
     ++     * The POSIX definition for the stat() function uses a struct of the
     ++     * same name (struct stat), that why it takes this extra effort  for
     ++     * redirecting/replacing the stat() function with our own one which
     ++     * is capable to handle long path names on Windows.
     ++     * The struct below roughly follows the POSIX definition. Time values
     ++     * are 64bit, but in cases when _USE_32BIT_TIME_T is defined, they
     ++     * will be set to values no larger than INT32_MAX which corresponds
     ++     * to file times up to the year 2038.
     ++     */
      +    struct win32_stat
      +    {
      +        _dev_t         st_dev;     /* ID of device containing file */
     @@ libavformat/os_support.h: DEF_FS_FUNCTION(unlink, _wunlink, _unlink)
      -fallback:                                                 \
      -    /* filename may be be in CP_ACP */                    \
      -    return afunc(filename_utf8, par);                     \
     -+static inline int win32_access(const char *filename_utf8, int par)
     ++static inline int win32_access(const char *filename_utf8, int mode)
      +{
      +    wchar_t *filename_w;
      +    int ret;
     @@ libavformat/os_support.h: DEF_FS_FUNCTION(unlink, _wunlink, _unlink)
      +        return -1;
      +    if (!filename_w)
      +        goto fallback;
     -+    ret = _waccess(filename_w, par);
     ++    ret = _waccess(filename_w, mode);
      +    av_free(filename_w);
      +    return ret;
      +fallback:
     -+    return _access(filename_utf8, par);
     ++    return _access(filename_utf8, mode);
      +}
      +
     -+static inline void copy_stat(struct _stati64 *winstat, struct win32_stat *par)
     ++static inline void copy_stat(struct _stati64 *crtstat, struct win32_stat *buf)
      +{
     -+    par->st_dev   = winstat->st_dev;
     -+    par->st_ino   = winstat->st_ino;
     -+    par->st_mode  = winstat->st_mode;
     -+    par->st_nlink = winstat->st_nlink;
     -+    par->st_uid   = winstat->st_uid;
     -+    par->st_gid   = winstat->st_gid;
     -+    par->st_rdev  = winstat->st_rdev;
     -+    par->st_size  = winstat->st_size;
     -+    par->st_atime = winstat->st_atime;
     -+    par->st_mtime = winstat->st_mtime;
     -+    par->st_ctime = winstat->st_ctime;
     ++    buf->st_dev   = crtstat->st_dev;
     ++    buf->st_ino   = crtstat->st_ino;
     ++    buf->st_mode  = crtstat->st_mode;
     ++    buf->st_nlink = crtstat->st_nlink;
     ++    buf->st_uid   = crtstat->st_uid;
     ++    buf->st_gid   = crtstat->st_gid;
     ++    buf->st_rdev  = crtstat->st_rdev;
     ++    buf->st_size  = crtstat->st_size;
     ++    buf->st_atime = crtstat->st_atime;
     ++    buf->st_mtime = crtstat->st_mtime;
     ++    buf->st_ctime = crtstat->st_ctime;
       }
       
      -DEF_FS_FUNCTION2(access, _waccess, _access, int)
      -DEF_FS_FUNCTION2(stat, _wstati64, _stati64, struct stat*)
     -+static inline int win32_stat(const char *filename_utf8, struct win32_stat *par)
     ++static inline int win32_stat(const char *filename_utf8, struct win32_stat *buf)
      +{
     -+    struct _stati64 winstat = { 0 };
     ++    struct _stati64 crtstat = { 0 };
      +    wchar_t *filename_w;
      +    int ret;
      +
     @@ libavformat/os_support.h: DEF_FS_FUNCTION(unlink, _wunlink, _unlink)
      +        return -1;
      +
      +    if (filename_w) {
     -+        ret = _wstat64(filename_w, &winstat);
     ++        ret = _wstat64(filename_w, &crtstat);
      +        av_free(filename_w);
      +    } else
     -+        ret = _stat64(filename_utf8, &winstat);
     ++        ret = _stat64(filename_utf8, &crtstat);
      +
     -+    copy_stat(&winstat, par);
     ++    copy_stat(&crtstat, buf);
      +
      +    return ret;
      +}
      +
     -+static inline int win32_fstat(int fd, struct win32_stat *par)
     ++static inline int win32_fstat(int fd, struct win32_stat *buf)
      +{
     -+    struct _stati64 winstat = { 0 };
     ++    struct _stati64 crtstat = { 0 };
      +    int ret;
      +
     -+    ret = _fstat64(fd, &winstat);
     ++    ret = _fstat64(fd, &crtstat);
      +
     -+    copy_stat(&winstat, par);
     ++    copy_stat(&crtstat, buf);
      +
      +    return ret;
      +}
 3:  0522fc2315 = 3:  e13c6b0aaa avformat/file: remove _WIN32 condition

-- 
ffmpeg-codebot
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

  parent reply	other threads:[~2022-05-26  9:29 UTC|newest]

Thread overview: 83+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-05-13  9:53 [FFmpeg-devel] [PATCH 0/2] " ffmpegagent
2022-05-13  9:53 ` [FFmpeg-devel] [PATCH 1/2] avutil/wchar_filename, file_open: " softworkz
2022-05-15 19:02   ` nil-admirari
2022-05-15 20:24     ` Soft Works
2022-05-16  8:34       ` nil-admirari
2022-05-13  9:53 ` [FFmpeg-devel] [PATCH 2/2] avformat/os_support: " softworkz
2022-05-15 19:13   ` nil-admirari
2022-05-15 22:14     ` Soft Works
2022-05-16  8:19       ` nil-admirari
2022-05-13 10:02 ` [FFmpeg-devel] [PATCH 0/2] " Soft Works
2022-05-15 19:36 ` nil-admirari
2022-05-15 20:31   ` Soft Works
2022-05-16  8:45     ` nil-admirari
2022-05-17  0:37       ` Soft Works
2022-05-15 22:17 ` [FFmpeg-devel] [PATCH v2 " ffmpegagent
2022-05-15 22:17   ` [FFmpeg-devel] [PATCH v2 1/2] avutil/wchar_filename, file_open: " softworkz
2022-05-16  8:12     ` nil-admirari
2022-05-16 21:14       ` Soft Works
2022-05-17 15:06         ` nil-admirari
2022-05-17 15:28           ` Soft Works
2022-05-17 15:43             ` Soft Works
2022-05-20 17:51               ` nil-admirari
2022-05-20 18:03                 ` Soft Works
2022-05-21 11:08                   ` nil-admirari
2022-05-21 11:12                     ` Soft Works
2022-05-23 15:35                       ` nil-admirari
2022-05-23 15:47                         ` Soft Works
2022-05-23 17:12                           ` Hendrik Leppkes
2022-05-24  5:32                             ` Soft Works
2022-05-24  5:54                               ` Soft Works
2022-05-24  9:47                           ` Soft Works
2022-05-24 12:11                             ` nil-admirari
2022-05-15 22:17   ` [FFmpeg-devel] [PATCH v2 2/2] avformat/os_support: " softworkz
2022-05-16 21:23   ` [FFmpeg-devel] [PATCH v3 0/2] " ffmpegagent
2022-05-16 21:23     ` [FFmpeg-devel] [PATCH v3 1/2] avutil/wchar_filename, file_open: " softworkz
2022-05-16 21:23     ` [FFmpeg-devel] [PATCH v3 2/2] avformat/os_support: " softworkz
2022-05-23 11:29     ` [FFmpeg-devel] [PATCH v4 0/2] " ffmpegagent
2022-05-23 11:29       ` [FFmpeg-devel] [PATCH v4 1/2] avutil/wchar_filename, file_open: " softworkz
2022-05-23 11:29       ` [FFmpeg-devel] [PATCH v4 2/2] avformat/os_support: " softworkz
2022-05-24  8:43       ` [FFmpeg-devel] [PATCH v5 0/2] " ffmpegagent
2022-05-24  8:43         ` [FFmpeg-devel] [PATCH v5 1/2] avutil/wchar_filename, file_open: " softworkz
2022-05-24  9:09           ` Martin Storsjö
2022-05-24  8:43         ` [FFmpeg-devel] [PATCH v5 2/2] avformat/os_support: " softworkz
2022-05-24  9:23           ` Martin Storsjö
2022-05-24  9:33             ` Soft Works
2022-05-24 10:25               ` Martin Storsjö
2022-05-24 11:15                 ` Soft Works
2022-05-24 11:26                   ` Martin Storsjö
2022-05-24 12:31                     ` Soft Works
2022-05-24 12:44                       ` Martin Storsjö
2022-05-24 13:41                         ` Soft Works
2022-05-24 13:58         ` [FFmpeg-devel] [PATCH v6 0/2] " ffmpegagent
2022-05-24 13:58           ` [FFmpeg-devel] [PATCH v6 1/2] avutil/wchar_filename, file_open: " softworkz
2022-05-24 20:55             ` Martin Storsjö
2022-05-24 13:58           ` [FFmpeg-devel] [PATCH v6 2/2] avformat/os_support: " softworkz
2022-05-24 20:58             ` Martin Storsjö
2022-05-24 22:12               ` Soft Works
2022-05-25  7:09                 ` Martin Storsjö
2022-05-24 22:20           ` [FFmpeg-devel] [PATCH v7 0/3] " ffmpegagent
2022-05-24 22:20             ` [FFmpeg-devel] [PATCH v7 1/3] avutil/wchar_filename, file_open: " softworkz
2022-05-24 22:20             ` [FFmpeg-devel] [PATCH v7 2/3] avformat/os_support: " softworkz
2022-05-25 14:47               ` nil-admirari
2022-05-25 15:28                 ` Soft Works
2022-05-25 19:17                   ` nil-admirari
2022-05-26  5:09                     ` Soft Works
2022-05-25 18:50                 ` Martin Storsjö
2022-05-24 22:20             ` [FFmpeg-devel] [PATCH v7 3/3] avformat/file: remove _WIN32 condition softworkz
2022-05-25  7:34             ` [FFmpeg-devel] [PATCH v7 0/3] Support long file names on Windows Martin Storsjö
2022-05-26  9:28             ` ffmpegagent [this message]
2022-05-26  9:28               ` [FFmpeg-devel] [PATCH v8 1/3] avutil/wchar_filename, file_open: " softworkz
2022-05-26  9:28               ` [FFmpeg-devel] [PATCH v8 2/3] avformat/os_support: " softworkz
2022-05-26  9:28               ` [FFmpeg-devel] [PATCH v8 3/3] avformat/file: remove _WIN32 condition softworkz
2022-05-26 21:26               ` [FFmpeg-devel] [PATCH v8 0/3] Support long file names on Windows Martin Storsjö
2022-06-09 10:03               ` Martin Storsjö
2022-06-09 19:37                 ` nil-admirari
2022-06-09 20:15                   ` Soft Works
2022-06-09 20:22                     ` nil-admirari
2022-06-09 21:32                     ` Soft Works
2022-06-09 20:21                   ` Martin Storsjö
2022-06-09 20:57                     ` nil-admirari
2022-06-09 21:02                       ` Martin Storsjö
2022-06-13 16:42                         ` nil-admirari
2022-06-09 21:03                       ` Soft Works

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=pull.28.v8.ffstaging.FFmpeg.1653557330.ffmpegagent@gmail.com \
    --to=ffmpegagent@gmail.com \
    --cc=ffmpeg-devel@ffmpeg.org \
    --cc=h.leppkes@gmail.com \
    --cc=martin@martin.st \
    --cc=softworkz@hotmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://master.gitmailbox.com/ffmpegdev/0 ffmpegdev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 ffmpegdev ffmpegdev/ https://master.gitmailbox.com/ffmpegdev \
		ffmpegdev@gitmailbox.com
	public-inbox-index ffmpegdev

Example config snippet for mirrors.


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git